I'm working on android app, which determines which font is used on a text image. So I need to extract every character from image and don't know how to do it precisely. Furthermore, when I'm trying to process an image I have one result...but my classmate has different (for example, more or less noise). The problem with character detection is that:
1) it detects also some noise blobs on image and shows it in rectangles (I thought about detectMultiScale... but I have doubts about it, maybe there are easiest ways to detect characters)
2) it detects several contours of one character (for example inner and outer radius of letter "o")
And question for the future: I'm going to create a DB with images (for now just 3 fonts) of different letters of fonts and compare them with an image of letters from photo. Maybe someone could recommend a better way to do it.
So this is part of code with image processing(I'm still playing with values of blur, threshold and Canny... but there was no really positive result):
Imgproc.cvtColor(sImage, grayImage, Imgproc.COLOR_BGR2GRAY); //градации серого
Imgproc.GaussianBlur(grayImage,blurImage,new Size(5, 5),0); //размытие
Imgproc.adaptiveThreshold(blurImage, thresImage, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 101, 39);
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Imgproc.Canny(thresImage, binImage, 30, 10, 3, true); //контур
Imgproc.findContours(binImage, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE, new Point(0, 0));
hierarchy.release();
Imgproc.drawContours(binImage, contours, -1, new Scalar(255, 255, 255));//, 2, 8, hierarchy, 0, new Point());
MatOfPoint2f approxCurve = new MatOfPoint2f();
//For each contour found
for (int i = 0; i < contours.size(); i++) {
//Convert contours(i) from MatOfPoint to MatOfPoint2f
MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
//Processing on mMOP2f1 which is in type MatOfPoint2f
double approxDistance = Imgproc.arcLength(contour2f, true) * 0.02;
Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);
//Convert back to MatOfPoint
MatOfPoint points = new MatOfPoint(approxCurve.toArray());
// Get bounding rect of contour
Rect rect = Imgproc.boundingRect(points);
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(binImage, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 255, 255), 5);
}
And screen (not actually with processing values from code, just one with better results):
Original:
(unfortunately, I can't add more than 2 links to show more examples)
There were situations, when picture from this screen looked pretty good, but another pictures looked like with shapeless blobs.
Your code is fine, you just need to make a minor tweaks to get it work properly.
Firstly, the image size is very large, you can safely reduce it to 20% of current size without suffering a major loss in accuracy. Due to larger image size all the functions would perform slower.
You dont need to perform adaptive threshold before Canny, canny works perfectly on gray-scale images as well, You need to adjust the params as:
Canny(img, threshold1=170, threshold2=250)
which yields an image as:
[Optional] If you want to de-noise the image then you can try with morphological operations like erode and dilate.
Now you are ready to find the contours. The mistake in your code was using Imgproc.RETR_TREE flag you need to use Imgproc.RETR_EXTERNAL flag to get only the outer contours and not the nested inner contours.
At this step you may have some unwanted small contours, which can be filtered as:
// ** Below code if for reference purposes only, consult OpenCV docs for proper API methods
int character_area_lower_thresh = 10;
for (Contour c:contours) {
if (Imgproc.contourArea(c) > character_area_lower_thresh) {
// Desired contour, do what ever you want to do
Rect r = Imgproc.boundingRect(c);
}
}
Related
I'm currently trying to improve the recognition rate of GoogleCloud Vision, so I am building a preprocessing pipeline.
I currently can create a mask which overlays the characters in the image, but as you can see in the examples below, it also shows the lines. Now since those lines can cross through characters, I'd like to remove them from the mask without destroying the characters, if possible.
Current steps:
Line detection:
InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU) -> HoughLinesP
Mask generation: InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU)-> ConnectedComponents
ImageExamples:(Due to privacy protection, sharing a full Image is not possible)
The images show the original image, the mask and the lines recognized.
The following code is used to generate the mask and find the lines
Mat picture = Imgcodecs.imread(path);
Imgproc.cvtColor(picture, picture, Imgproc.COLOR_BGR2GRAY);
Imgcodecs.imwrite("/home/meik/Pictures/asdfGray.png", picture);
Mat blackhatElement = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_RECT, new Size(7, 7));
Imgproc.morphologyEx(picture, picture, Imgproc.MORPH_BLACKHAT, blackhatElement);
Imgproc.GaussianBlur(picture, picture, new Size(5, 3), 0);
Imgproc.threshold(picture, picture, 0, 255, Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU);
/**
* Line Detection with Canny and HoughLines(P)
*/
Mat lines = new Mat();
Mat linesResult = Mat.zeros(picture.rows(),picture.cols(), CvType.CV_8UC1);
Imgproc.HoughLinesP(picture, lines,1, Math.PI/180,100, 20, 0);
System.out.println("lines rows:" + lines.rows());
for (int x = 0; x < lines.rows(); x++) {
double[] l = lines.get(x, 0);
Imgproc.line(linesResult, new Point(l[0], l[1]), new Point(l[2], l[3]), new Scalar(255, 255, 255), 1, Imgproc.LINE_8, 0);
}
/**End of line detection*/
Mat kernel = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_CROSS, new Size(3,3));
Imgproc.dilate(linesResult,linesResult,kernel);
Core.bitwise_not(linesResult,linesResult);
I've found this paper talking about the problem, but am struggling to understand their methodology.
How do I proceed from here on to remove lines without destroying the characters?
I dont really think you need to refer paper to do this.
Just use color info or hough line to find out a straightline which is really long
use that info to create a mask image.
Then use the opencv inpaint to remove it.
https://docs.opencv.org/2.4/modules/photo/doc/inpainting.html
e.g. what you want is similar to the bottom image. It ask to remove the traffic light poles. and you want the writing guideline to be removed. essentially, its the same thing
How about some simple image preprocessing?
For example using a threshold to only maintain a certain color range (instead of directly converting the image to grayscale).
Something like this is integrated in GIMP, see
https://docs.gimp.org/2.8/en/gimp-tool-threshold.html
You probably want to experiment with various thresholds.
I wanna make an ellipse mask for cropping an image so only the contents inside of the ellipse will be shown.
Could you inspect my code?
public static Mat cropImage(Mat imageOrig, MatOfPoint contour){
Rect rect = Imgproc.boundingRect(contour);
MatOfPoint2f contour2f = new MatOfPoint2f(contour.toArray());
RotatedRect boundElps = Imgproc.fitEllipse(contour2f);
Mat out = imageOrig.submat(rect);
// the line function is working
Imgproc.line(out, new Point(0,0), new Point(out.width(), out.height()), new Scalar(0,0,255), 5);
// but not this one
Imgproc.ellipse(out, boundElps, new Scalar(255, 0, 0), 99);
return out;
}//cropImage
It seems like it's not working at all. Though you can see the line function I've done to test if it is working on the right image and I can see a line but there's no ellipse.
Here's a sample output of my cropImage function.
TIA
You're retrieving the ellipse coordinates in the imageOrig coordinates system, but you're showing it on the cropped out image.
If you want to show the ellipse on the crop, you need to translate the ellipse center to account for the translation introduced by the crop (top-left coordinates of rect), something like:
boundElps.center().x -= rect.x; boundElps.center().y -= rect.y;
You can try this out:
RotatedRect rRect = Imgproc.minAreaRect(contour2f);
Imgproc.ellipse(out, rRect , new Scalar(255, 0, 0), 3);
You should check for the minimum requirements for using the fitEllipse as shown in this post.
The function fitEllipse requires atleast 5 points.
Note: Although the reference I mention is for Python, I hope you can do the same for Java.
for cnt in contours:
area = cv2.contourArea(cnt)
# Probably this can help but not required
if area < 2000 or area > 4000:
continue
# This is the check I'm referring to
if len(cnt) < 5:
continue
ellipse = cv2.fitEllipse(cnt)
cv2.ellipse(roi, ellipse, (0, 255, 0), 2)
Hope it helps!
I'm trying to detect skin on images using opencv (new at it) but I've managed to get SOME of the skin to be detected, however the rest seems to cause some noise on the image. Here's the original image:
The result of my code is:
The code that prouduces this:
Mat image = Imgcodecs.imread(name);
Imgproc.pyrMeanShiftFiltering(image, image, 10, 20);
Imgproc.blur(image, image, new Size(3, 3));
Mat hsv = new Mat();
Imgproc.cvtColor(image, hsv, Imgproc.COLOR_BGR2HSV);
Mat bw = new Mat();
Core.inRange(hsv, new Scalar(0, 10, 60), new Scalar(20, 150, 255), bw);
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(bw, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE, new Point(0, 0));
int s = findBiggestContour(contours);
Mat drawing = new Mat(image.size(), CvType.CV_8UC1);
Imgproc.drawContours(drawing, contours, s, new Scalar(255), -1, 8, hierarchy, 0, new Point(0, 0));
Imgcodecs.imwrite(rename(name), drawing);
How do I fix the code to detect the remaining skin on the image and get rid of the noise?
I'm using Java with OpenCV 3.0.0.
Since you're using the findBiggestConour() I think you only draw the biggest match, not all of them. Since the biggest contour happens to be the one on the second image only that is shown.
Just to add to what JanSLO said here above, I tried your code and instead of drawing just the biggest contour, I drew all contours and got the following result.
//c++ code, not java
Mat drawing = Mat::zeros(img.size(), CV_8UC1 );
for(int i=0; i < contours.size(); ++i) {
drawContours(drawing, contours, i, Scalar(255), 3, 8, hierarchy, 0, Point(0, 0));
}
imwrite("data/skin_detect_out.jpg", drawing);
I am pleasantly surprised at the result, since this is such a simple piece of code. More advanced pixel based skin detection methods involve making a probability model of skin pixels using training data, and using that model to classify whether a given pixel is skin or not.
I want to get all the outer contours with RETR_EXTERNAL but for some weird reason openCV thinks that the image border is a contour too and therefore discards all inner contours. What exactly am I doing wrong here?
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Imgproc.findContours(imageA, contours, hierarchy, Imgproc.RETR_EXTERNAL,
Imgproc.CHAIN_APPROX_SIMPLE);
for (int i = 0; i < contours.size(); i++) {
double[] c = hierarchy.get(0, i);
Rect rect = Imgproc.boundingRect(contours.get(i));
Core.rectangle(image, new Point(rect.x, rect.y),
new Point(rect.x + rect.width, rect.y + rect.height),
new Scalar(0, 255, 0), 3);
}
Input (imageA was processed to this before contour-finding):
Output:
EDIT:
Problem partially solved
Inverting the pixels so that black is the background and white the foreground helped with the image above image. However I still get inner contours on some images. Like this one:
Input
Output
Your input image isnt good enought o extract the contours you want to have.
Your input contours are these (part of your image):
each color is a single contour (and some of the white ones)
For the red contour I've drawn the bounding rectangle which is the same method that you used to display the contours. All the other colored contours aren't inside of the red contour, but just inside of the bounding rectangle, that's why they are found even though you selected to only find the outer contours.
What you really want is something like this:
but to get that result, your input image must have that lines of the ellipse connected, too!!
For your input image it will be very hard to extract those lines, without getting lines of the ground too, but an easy approach could be to use a couple of dilation operations followed by the same number of erosion operations on your input image, before extracting contours. This won't be stable for all setting though ;)
I am developing an Android app that calculates the sum of all points of the being-seen dominoes pieces -shown in picture- using OpenCV for Android.
The problem is, I can't find a way to filtering other contours and counting only dots I see in the dominoes, I tried to use Canny edge finding then use HoughCircles, but with no result, as I don't have an absolute top view of the rocks and HoughCircles detect perfect circles only :)
Here is my code:
public Mat onCameraFrame(Mat inputFrame) {
inputFrame.copyTo(mRgba);
Mat grey = new Mat();
// Make it greyscale
Imgproc.cvtColor(mRgba, grey, Imgproc.COLOR_RGBA2GRAY);
// init contours arraylist
List<MatOfPoint> contours = new ArrayList<MatOfPoint>(200);
//blur
Imgproc.GaussianBlur(grey, grey, new Size(9,9), 10);
Imgproc.threshold(grey, grey, 80, 150, Imgproc.THRESH_BINARY);
// because findContours modifies the image I back it up
Mat greyCopy = new Mat();
grey.copyTo(greyCopy);
Imgproc.findContours(greyCopy, contours, new Mat(), Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE);
// Now I have my controus pefectly
MatOfPoint2f mMOP2f1 = new MatOfPoint2f();
//a list for only selected contours
List<MatOfPoint> SelectedContours = new ArrayList<MatOfPoint>(400);
for(int i=0;i<contours.size();i++)
{
if(here I should put a condition that distinguishes my spots, eg: if contour inside is black and is a black disk)
{
SelectedContours.add(contours.get(i));
}
}
Imgproc.drawContours(mRgba, SelectedContours, -1, new Scalar(255,0,0,255), 1);
return mRgba;
}
EDIT:
One unique feature of my contours after threshold is they're totally black from inside, is there anyway I could calculate the mean color/intensity for a given contour ?
There is a similiar problem and possible solution on SO, titled Detection of coins (and fit ellipses) on an image. Here you will find some recomendations about opencv's function fitEllipse.
You should take a look at this for more info on opencv's function fitEllipse.
Also, to detect only black elements in an image, you can use HSV color model, to find only black colors. You can find an explanation here.