At the first contact with Java OpenCV (3.3.1, windows 8 x64) I'm trying to join two different size images with ROI dynamically. Here a bit of my code:
Mat _mat = Utils.imageFileToMat(new File("angelina_jolie.jpg")); //Angelina's face
Mat grayMat = new Mat();
Imgproc.cvtColor(_mat, grayMat, Imgproc.COLOR_BGR2GRAY);
Rect rect = new Rect(new Point(168, 104), new Point(254, 190)); //Angelina's eye ROI
Mat truncated = _mat.submat(rect); //Angelina's eye mat
Mat merge = _mat.clone();
truncated.copyTo(merge);
//show _mat
//show truncated
//show merge
What I want to see is Angelina Jolie with her eye on grayscale.
What I see is assertions or the truncated image only (just the eye).
I tried with copyTo(mat, mask), setOf, and a lot of things but always get a new assertion.
Should I change the size of truncated to the size of mat to match sizes? how can I do that programmatically?
Mat::copyTo documentation:
The method copies the matrix data to another matrix. Before copying
the data, the method invokes :
m.create(this->size(),this->type());
so that the destination matrix is reallocated
if needed. While m.copyTo(m); works flawlessly, the function does not
handle the case of a partial overlap between the source and the
destination matrices.
When the operation mask is specified, if the Mat::create call shown
above reallocates the matrix, the newly allocated matrix is
initialized with all zeros before copying the data.
#param m
Destination matrix. If it does not have a proper size or type before
the operation, it is reallocated.
Since you're your src and dst images don't have the same size and channels, the destination image is reallocated and initialized with zeros. To avoid that make sure both images have same dimensions and number of channels.
Imgproc.cvtColor(grayMat, grayMat, Imgproc.COLOR_GRAY2BGR);
Now create a mask:
Mat mask = new Mat(_mat.size(), CvType.CV_8UC1, new Scalar(0));
Imgproc.rectangle(mask, new Point(168, 104), new Point(254, 190),new Scalar(255));
// copy gray to _mat based on mask
Mat merge = _mat.clone();
grayMat.copyTo(merge,mask);
Related
I'm currently trying to improve the recognition rate of GoogleCloud Vision, so I am building a preprocessing pipeline.
I currently can create a mask which overlays the characters in the image, but as you can see in the examples below, it also shows the lines. Now since those lines can cross through characters, I'd like to remove them from the mask without destroying the characters, if possible.
Current steps:
Line detection:
InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU) -> HoughLinesP
Mask generation: InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU)-> ConnectedComponents
ImageExamples:(Due to privacy protection, sharing a full Image is not possible)
The images show the original image, the mask and the lines recognized.
The following code is used to generate the mask and find the lines
Mat picture = Imgcodecs.imread(path);
Imgproc.cvtColor(picture, picture, Imgproc.COLOR_BGR2GRAY);
Imgcodecs.imwrite("/home/meik/Pictures/asdfGray.png", picture);
Mat blackhatElement = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_RECT, new Size(7, 7));
Imgproc.morphologyEx(picture, picture, Imgproc.MORPH_BLACKHAT, blackhatElement);
Imgproc.GaussianBlur(picture, picture, new Size(5, 3), 0);
Imgproc.threshold(picture, picture, 0, 255, Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU);
/**
* Line Detection with Canny and HoughLines(P)
*/
Mat lines = new Mat();
Mat linesResult = Mat.zeros(picture.rows(),picture.cols(), CvType.CV_8UC1);
Imgproc.HoughLinesP(picture, lines,1, Math.PI/180,100, 20, 0);
System.out.println("lines rows:" + lines.rows());
for (int x = 0; x < lines.rows(); x++) {
double[] l = lines.get(x, 0);
Imgproc.line(linesResult, new Point(l[0], l[1]), new Point(l[2], l[3]), new Scalar(255, 255, 255), 1, Imgproc.LINE_8, 0);
}
/**End of line detection*/
Mat kernel = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_CROSS, new Size(3,3));
Imgproc.dilate(linesResult,linesResult,kernel);
Core.bitwise_not(linesResult,linesResult);
I've found this paper talking about the problem, but am struggling to understand their methodology.
How do I proceed from here on to remove lines without destroying the characters?
I dont really think you need to refer paper to do this.
Just use color info or hough line to find out a straightline which is really long
use that info to create a mask image.
Then use the opencv inpaint to remove it.
https://docs.opencv.org/2.4/modules/photo/doc/inpainting.html
e.g. what you want is similar to the bottom image. It ask to remove the traffic light poles. and you want the writing guideline to be removed. essentially, its the same thing
How about some simple image preprocessing?
For example using a threshold to only maintain a certain color range (instead of directly converting the image to grayscale).
Something like this is integrated in GIMP, see
https://docs.gimp.org/2.8/en/gimp-tool-threshold.html
You probably want to experiment with various thresholds.
I'm working on android app, which determines which font is used on a text image. So I need to extract every character from image and don't know how to do it precisely. Furthermore, when I'm trying to process an image I have one result...but my classmate has different (for example, more or less noise). The problem with character detection is that:
1) it detects also some noise blobs on image and shows it in rectangles (I thought about detectMultiScale... but I have doubts about it, maybe there are easiest ways to detect characters)
2) it detects several contours of one character (for example inner and outer radius of letter "o")
And question for the future: I'm going to create a DB with images (for now just 3 fonts) of different letters of fonts and compare them with an image of letters from photo. Maybe someone could recommend a better way to do it.
So this is part of code with image processing(I'm still playing with values of blur, threshold and Canny... but there was no really positive result):
Imgproc.cvtColor(sImage, grayImage, Imgproc.COLOR_BGR2GRAY); //градации серого
Imgproc.GaussianBlur(grayImage,blurImage,new Size(5, 5),0); //размытие
Imgproc.adaptiveThreshold(blurImage, thresImage, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 101, 39);
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Imgproc.Canny(thresImage, binImage, 30, 10, 3, true); //контур
Imgproc.findContours(binImage, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE, new Point(0, 0));
hierarchy.release();
Imgproc.drawContours(binImage, contours, -1, new Scalar(255, 255, 255));//, 2, 8, hierarchy, 0, new Point());
MatOfPoint2f approxCurve = new MatOfPoint2f();
//For each contour found
for (int i = 0; i < contours.size(); i++) {
//Convert contours(i) from MatOfPoint to MatOfPoint2f
MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
//Processing on mMOP2f1 which is in type MatOfPoint2f
double approxDistance = Imgproc.arcLength(contour2f, true) * 0.02;
Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);
//Convert back to MatOfPoint
MatOfPoint points = new MatOfPoint(approxCurve.toArray());
// Get bounding rect of contour
Rect rect = Imgproc.boundingRect(points);
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(binImage, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 255, 255), 5);
}
And screen (not actually with processing values from code, just one with better results):
Original:
(unfortunately, I can't add more than 2 links to show more examples)
There were situations, when picture from this screen looked pretty good, but another pictures looked like with shapeless blobs.
Your code is fine, you just need to make a minor tweaks to get it work properly.
Firstly, the image size is very large, you can safely reduce it to 20% of current size without suffering a major loss in accuracy. Due to larger image size all the functions would perform slower.
You dont need to perform adaptive threshold before Canny, canny works perfectly on gray-scale images as well, You need to adjust the params as:
Canny(img, threshold1=170, threshold2=250)
which yields an image as:
[Optional] If you want to de-noise the image then you can try with morphological operations like erode and dilate.
Now you are ready to find the contours. The mistake in your code was using Imgproc.RETR_TREE flag you need to use Imgproc.RETR_EXTERNAL flag to get only the outer contours and not the nested inner contours.
At this step you may have some unwanted small contours, which can be filtered as:
// ** Below code if for reference purposes only, consult OpenCV docs for proper API methods
int character_area_lower_thresh = 10;
for (Contour c:contours) {
if (Imgproc.contourArea(c) > character_area_lower_thresh) {
// Desired contour, do what ever you want to do
Rect r = Imgproc.boundingRect(c);
}
}
I am working on a Rubik's side scanner to determine what state the cube is in. I am quite new to computer vision and using it so it has been a little bit of a challenge. What I have done so far is that I use a video capture and at certain frames capture that frame and save it for image processing. Here is what it looks like.
When the photo is taken the cube is in the same position each time so I don't have to worry about locating the stickers.
What I am having trouble doing is getting a small range of pixels in each square to determine its HSV.
I know the ranges of HSV are roughly
Red = Hue(0...9) AND Hue(151..180)
Orange = Hue(10...15)
Yellow = Hue(16..45)
Green = Hue(46..100)
Blue = Hue(101..150)
White = Saturation(0..20) AND Value(230..255)
So after I have captured the image I then load it and split the HSV values of the image but don't know how to get the certain pixel coordinates of the image. How do I do so?
BufferedImage getOneFrame() {
currFrame++;
//At the 90th frame I capture that frame and save that frame
if (currFrame == 120) {
cap.read(mat2Img.mat);
mat2Img.getImage(mat2Img.mat);
Imgcodecs.imwrite("firstImage.png", mat2Img.mat);
}
cap.read(mat2Img.mat);
return mat2Img.getImage(mat2Img.mat);
}
public void splitChannels() {
IplImage firstShot = cvLoadImage("firstImage.png");
//I split the channels so that I can determine the value of the pixel range
IplImage hsv = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), firstShot.nChannels());
IplImage hue = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
IplImage sat = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
IplImage val = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
cvSplit( hsv, hue, sat, val, null );
//How do I get a small range of pixels of my images to determine get their HSV?
}
If I understand your question right, you know the coordinates of all areas that interest you. Save the information about each area into cvRect objects.
You can traverse the rectangle area by looping. Make a double loop. In outer loop start at rect.y and stop before rect.y + rect.height. In inner loop, do a similar thing in x direction. Inside the loop, use CV_IMAGE_ELEM macro to access individual pixel values and compute whatever you need.
One advice though: There are several advantages to using Mat instead of IplImage when working with OpenCV. I recommend that you start using 'Mat', unless you have some special reasons to do so, of course. Click to see the documentation and take a look at one of constructors that takes one Mat and one Rect as parameters. This constructor is your good friend - you can create a new Mat object (without copying any data) which will only contain the area inside the rectangle.
In my android app I get an image from the gallery as a bitmap with something like this
Bitmap bitm = getMyImage("Thanks!");
and I have a Mat called mat declared like this:
Mat mat = new Mat(bitm.getHeight(), bitm.getWidth(), CVType.CV_8UC3);
I'm trying to get contour areas from the image, which I've successfully gotten then draw it back on the original image with:
Imgproc.drawContours(mat, contours, -1, new Scalar(200,200,0), 2);
displayMat(mat);
If I use it like that, it works but the contours are drawn on a blank image which isn't what I want. I want it to be drawn on the original image. If I use
Utils.bitmapToMat(bitm, mat);
before the previous snippet of code, the displayed image is just the preprossed image without the 'Drawn' contours. Why?
Asfaik Android uses images with alpha values, so CV_8UC4 is the right data type.
So
Mat mat = new Mat(bitm.getHeight(), bitm.getWidth(), CVType.CV_8UC3);
Imgproc.drawContours(mat, contours, -1, new Scalar(200,200,0), 2);
displayMat(mat);
draws the contours correctly on an empty/blank 8UC3 image (if memory empty).
But if you want to draw on the input image by first converting Utils.bitmapToMat(bitm, mat); you'll overwrite your 8UC3 memory and replace it by 8UC4 data. After that you draw Scalar(200,200,0) which will use a 4th channel, but cv::Scalar automatically adds those channels with default zero values, so you draw your contours in transparent. So use Scalar(200,200,0,255) instead and it should give your expected results.
Mat mat = new Mat(bitm.getHeight(), bitm.getWidth(), CVType.CV_8UC3);
Utils.bitmapToMat(bitm, mat);
Imgproc.drawContours(mat, contours, -1, new Scalar(200,200,0,255), 2);
displayMat(mat);
The other method would be to convert the bitmap to 8UC3, but I'm not sure how to do that.
I'm frustrated trying to find a method that would allow me to scale a Mat object to a different size. Please, could anybody help me with this?
My project is using the Java wrapper, but I'll be happy if the answer provided is for the C++ native OpenCV library.
If by resizing you mean scaling an image use the resize() function as follows:
resize(src, dst, dst.size(), 0, 0, interpolation);
Otherwise if you just need to change the number of rows of your Mat use the Mat::reshape() function. Pay attention that the reshape return a new Mat header:
cv::Mat dst = src.reshape ( 0, newRowVal );
Finally if you want to arbitrarily reshape the Mat (changing rows and cols) you probably need to define a new Mat with the destination dimensions and copy the src Mat to it:
Mat dst(newRowVal, newColVal, src.type());
src.setTo(0);
src.copyTo(dst(Rect(Point(0, 0), src.size())));
You can use resize() function
Create a new Mat result of new dimensions
resize(input // input image
result // result image
result.size() // new dimensions
0,
0,
INTER_CUBIC // interpolation method
);
to know more interpolation methods, you can check this doc: geometric_transformations.html#resize