I've just learnt how to detect a color from OpenCV Java, Getting region of interest from image.
Ultimately, I want to know how can I detect the AA battery(with and or without black tape)
I'm now trying to detect the battery in the picture, but the battery is not fully black, and in turn giving me weird result:
I covered the battery with black tape and tried again the result seems better but it is detecting the battery in two separate sections:
Code:
private Bitmap findRoiBlack(Bitmap sourceBitmap) {
Bitmap roiBitmap = null;
Scalar green = new Scalar(0, 255, 0, 255);
Mat sourceMat = new Mat(sourceBitmap.getWidth(), sourceBitmap.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(sourceBitmap, sourceMat);
Mat roiTmp = sourceMat.clone();
final Mat hsvMat = new Mat();
sourceMat.copyTo(hsvMat);
// convert mat to HSV format for Core.inRange()
Imgproc.cvtColor(hsvMat, hsvMat, Imgproc.COLOR_RGB2HSV);
Scalar lowerb = new Scalar(0, 0, 0); // lower color border for BLACK
Scalar upperb = new Scalar(180, 255, 30); // upper color border for BLACK
//Scalar lowerb = new Scalar(0, 0, 200); // lower color border for WHITE
//Scalar upperb = new Scalar(180, 255, 255); // upper color border for WHITE
Core.inRange(hsvMat, lowerb, upperb, roiTmp); // select only blue pixels
// find contours
List<MatOfPoint> contours = new ArrayList<>();
List<RotatedRect> boundingRects = new ArrayList<>();
Imgproc.findContours(roiTmp, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// find appropriate bounding rectangles
for (MatOfPoint contour : contours) {
MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
double rectangleArea = boundingRect.size.area();
// test min ROI area in pixels
if (rectangleArea > 400) {
Point rotated_rect_points[] = new Point[4];
boundingRect.points(rotated_rect_points);
Rect rect = Imgproc.boundingRect(new MatOfPoint(rotated_rect_points));
// test vertical ROI orientation
if (rect.height > rect.width) {
Imgproc.rectangle(sourceMat, rect.tl(), rect.br(), green, 3);
}
}
}
roiBitmap = Bitmap.createBitmap(sourceMat.cols(), sourceMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(sourceMat, roiBitmap);
return roiBitmap;
}
Easiest way - is add color marker to battery. Other way is set solid, well distinguishable background for vertical channel of Your installation (may be even backlight - in this case You should find just black/low brightness object on white/high brightness background). If it's not possible and You have solid background - try to "invert" approach: don't try find battery (because it has many colors) - find background (because it has one solid color) - object with "non background" colors probably battery (and You have additional hints: battery is "vertical" rectangle with 1/4 proportions (AAA battery has 10.5 mm diameter and 44.6 mm length), it is approximately on the vertical center of image and has chromium-plated high brightness elements on top and bottom, etc.).
Related
I have a grayscale image, this image is set to a rectangle with an ImagePattern. like this:
ImagePattern imagePattern = new ImagePattern(new Image("File:resources/images/image.png"));
Rectangle rectangle = new Rectangle();
rectangle.setFill(imagePattern);
The only problem is I want the user to choose the colour of the image, so I want to change the hue of the image.
I found following question on StackOverflow, the first answer https://stackoverflow.com/a/18124868/15277155
shows how a coloured image is changed to a red image.
The only problem I have is that answer is done with an ImageView instead of an Imagepattern.
Is there any way This can be done with an ImagePattern.
Or that I can place the ImageView inside a rectangle?
Based on #jewelsea's comment this is the code I have.
ImagePattern imagePattern = new ImagePattern(new
Image("File:resources/images/image.png"));
Rectangle rectangle = new Rectangle();
rectangle.setFill(imagePattern);
ColorAdjust colorAdjust = new ColorAdjust();
// define target color
Color targetColor = Color.GREEN;
double hue = map( (targetColor.getHue() + 180) % 360, 0, 360, -1, 1);
colorAdjust.setHue(hue);
// use saturation as it is enter code here
double saturation = targetColor.getSaturation();
colorAdjust.setSaturation(saturation);
double brightness = map( targetColor.getBrightness(), 0, 1, -1, 0);
colorAdjust.setBrightness(brightness);
// apply color adjustment
rectangle.setEffect(colorAdjust);
rectangle.setFill(imagePattern);
I tested this on a yellow image (PNG with transparent background, but no opacity) and it worked.
Then I tried it on an image with only colors ranging from white to black (and grey) (also a PNG with transparent background, but also with opacity in the colors) and it didn't change those colors.
The ColorAdjust effect will work with any node, not just an ImageView.
So you can apply the ColorAdjust effect to your rectangle, which is filled with the ImagePattern, and the effect will be applied to the ImagePattern.
I am developing an application to detect the lesion area, for this I am using the grabcut to detect the ROI and remove the background from the image. However in some images it is not working well. He ends up not identifying the borders of the region of interest well. The watershed can better identify the edges for this type of work, however I am having difficulties making this transition from grabcut to watershed. Before processing the grabcut, the user uses touchevent to mark a rectangle around the image of interest (wound area) to facilitate the work of the algorithm. As the image below.
However, using other wound images, segmentation is not good, showing flaws in ROI detection.
Image using grabcut in app
Image using watershed in desktop
this is the code:
private fun extractForegroundFromBackground(coordinates: Coordinates, currentPhotoPath: String): String {
// TODO: Provide complex object that has both path and extension
val width = bitmap?.getWidth()!!
val height = bitmap?.getHeight()!!
val rgba = Mat()
val gray_mat = Mat()
val threeChannel = Mat()
Utils.bitmapToMat(bitmap, gray_mat)
cvtColor(gray_mat, rgba, COLOR_RGBA2RGB)
cvtColor(rgba, threeChannel, COLOR_RGB2GRAY)
threshold(threeChannel, threeChannel, 100.0, 255.0, THRESH_OTSU)
val rect = Rect(coordinates.first, coordinates.second)
val fg = Mat(rect.size(), CvType.CV_8U)
erode(threeChannel, fg, Mat(), Point(-1.0, -1.0), 10)
val bg = Mat(rect.size(), CvType.CV_8U)
dilate(threeChannel, bg, Mat(), Point(-1.0, -1.0), 5)
threshold(bg, bg, 1.0, 128.0, THRESH_BINARY_INV)
val markers = Mat(rgba.size(), CvType.CV_8U, Scalar(0.0))
Core.add(fg, bg, markers)
val marker_tempo = Mat()
markers.convertTo(marker_tempo, CvType.CV_32S)
watershed(rgba, marker_tempo)
marker_tempo.convertTo(markers, CvType.CV_8U)
val imgBmpExit = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565)
Utils.matToBitmap(markers, imgBmpExit)
image.setImageBitmap(imgBmpExit)
// Run the grab cut algorithm with a rectangle (for subsequent iterations with touch-up strokes,
// flag should be Imgproc.GC_INIT_WITH_MASK)
//Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT)
// Create a matrix of 0s and 1s, indicating whether individual pixels are equal
// or different between "firstMask" and "source" objects
// Result is stored back to "firstMask"
//Core.compare(mark, source, mark, Core.CMP_EQ)
// Create a matrix to represent the foreground, filled with white color
val foreground = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(255.0, 255.0, 255.0))
// Copy the foreground matrix to the first mask
srcImage.copyTo(foreground, mark)
// Create a red color
val color = Scalar(255.0, 0.0, 0.0, 255.0)
// Draw a rectangle using the coordinates of the bounding box that surrounds the foreground
rectangle(srcImage, coordinates.first, coordinates.second, color)
// Create a new matrix to represent the background, filled with black color
val background = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(0.0, 0.0, 0.0))
val mask = Mat(foreground.size(), CvType.CV_8UC1, Scalar(255.0, 255.0, 255.0))
// Convert the foreground's color space from BGR to gray scale
cvtColor(foreground, mask, Imgproc.COLOR_BGR2GRAY)
// Separate out regions of the mask by comparing the pixel intensity with respect to a threshold value
threshold(mask, mask, 254.0, 255.0, Imgproc.THRESH_BINARY_INV)
// Create a matrix to hold the final image
val dst = Mat()
// copy the background matrix onto the matrix that represents the final result
background.copyTo(dst)
val vals = Mat(1, 1, CvType.CV_8UC3, Scalar(0.0))
// Replace all 0 values in the background matrix given the foreground mask
background.setTo(vals, mask)
// Add the sum of the background and foreground matrices by applying the mask
Core.add(background, foreground, dst, mask)
// Save the final image to storage
Imgcodecs.imwrite(currentPhotoPath + "_tmp.png", dst)
// Clean up used resources
firstMask.release()
source.release()
//bg.release()
//fg.release()
vals.release()
dst.release()
return currentPhotoPath
}
Exit:
How do I update the code to use watershed instead of grabcut?
A description of how to apply the watershed algorithm in OpenCV is here, although it is in Python. The documentation also contains some potentially useful examples. Since you already have a binary image, all that's left is to apply the Euclidean Distance Transform (EDT) and the watershed function. So instead of Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT), you would have:
Mat dist = new Mat();
Imgproc.distanceTransform(srcImage, dist, Imgproc.DIST_L2, Imgproc.DIST_MASK_3); // use L2 for Euclidean Distance
Mat markers = Mat.zeros(dist.size(), CvType.CV_32S);
Imgproc.watershed(dist, markers); # apply watershed to resultant image from EDT
Mat mark = Mat.zeros(markers.size(), CvType.CV_8U);
markers.convertTo(mark, CvType.CV_8UC1);
Imgproc.threshold(mark, firstMask, 0, 255, Imgproc.THRESH_BINARY + Imgproc.THRESH_OTSU); # threshold results to get binary image
The thresholding step is described here. Also, optionally, before you apply Imgproc.watershed, you may want to apply some morphological operations to the result of EDT i.e; dilation, erosion:
Imgproc.dilate(dist, dist, Mat.ones(3, 3, CvType.CV_8U));
If you're not familiar with morphological operations when it comes to processing binary images, the OpenCV documentation contains some good, quick examples.
Hope this helps!
if (DETECT_RED_OBJECTS_ONLY)
{
Imgproc.cvtColor(inputFrame, gray,
Imgproc.COLOR_BGR2RGB);
}
else
{
Imgproc.cvtColor(inputFrame, gray, Imgproc.COLOR_BGR2GRAY);
}
Imgproc.cvtColor(inputFrame, dst, Imgproc.COLOR_BGR2RGB);
// down-scale and upscale the image to filter out the noise
Imgproc.pyrDown(gray, downscaled, new Size(gray.cols() / 2, gray.rows() 2));
Imgproc.pyrUp(downscaled, upscaled, gray.size());
if (DETECT_RED_OBJECTS_ONLY) {
// convert the image from RGBA to HSV
Imgproc.cvtColor(upscaled, hsv, Imgproc.COLOR_RGB2HSV);
// threshold the image for the lower and upper HSV red range
Core.inRange(hsv, HSV_LOW_RED1, HSV_LOW_RED2, lowerRedRange);
Core.inRange(hsv, HSV_HIGH_RED1, HSV_HIGH_RED2, upperRedRange);
// put the two thresholded images together
Core.addWeighted(lowerRedRange, 1.0, upperRedRange, 1.0, 0.0, bw);
// apply canny to get edges only
System.out.println(bw);
Imgproc.Canny(bw, bw, 0, 255);
} else {
// Use Canny instead of threshold to catch squares with gradient shading
Imgproc.Canny(upscaled, bw, 0, 255);
}
// dilate canny output to remove potential
// holes between edge segments
Imgproc.dilate(bw, bw, new Mat(), new Point(-1, 1), 1);
// find contours and store them all as a list
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
contourImage = bw.clone();
Imgproc.findContours(
contourImage,
contours,
hierarchyOutputVector,
Imgproc.RETR_EXTERNAL,
Imgproc.CHAIN_APPROX_SIMPLE
);
System.out.println("contours" + contours);
// loop over all found contours
for (MatOfPoint cnt : contours) {
MatOfPoint2f curve = new MatOfPoint2f(cnt.toArray());
// approximates a polygonal curve with the specified precision
Imgproc.approxPolyDP(
curve,
approxCurve,
0.02 * Imgproc.arcLength(curve, true),
true
);
int numberVertices = (int) approxCurve.total();
When I use triangle as an input image , this one works and detect triangle[counters become 1 and numberVertices becomes 3]. But when i input a rectangle image , contours become 2 and numberVertices becomes 2. It must be 4 . Isn't it. Can anyone help me to find the bug.
If you are using this in latest visual studio, I suggest you to use :
JavaList List<MatOfPoint> contours = new JavaList<MatOfPoint>();
this might change the behavior of your contours. Please let me know if this fixed your bug. Otherwise lets try with something else
I am able to detect and draw a bounding rectangle around a region of interest like so:
Now I need to find the "x" value of the bottom horizontal line of the green bounding rectangle. My main aim is to find the "x" value of the base of the battery, and the "x" value of the blue strip of paper, so that I can calculate the distance between them.
There are only 2 values, rect.tl() and rect.br(), that I can see be used to draw the bounding rectangle Imgproc.rectangle(sourceMat, rect.tl(), rect.br(), green, 3);. I'm under the assumption that the "x" value from the bottom right point of the bounding rectangle will be the x co-ordinate of the pixel point.
How do i get the 'x' value from rect.br()
When i print out rect.br() like so Log.e("BR", rect.br().toString()); i get this:
E/BR: {793.0, 1686.0}
private Bitmap findRoi(Bitmap sourceBitmap) {
Bitmap roiBitmap = null;
Scalar green = new Scalar(0, 255, 0, 255);
Mat sourceMat = new Mat(sourceBitmap.getWidth(), sourceBitmap.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(sourceBitmap, sourceMat);
Mat roiTmp = sourceMat.clone();
final Mat hsvMat = new Mat();
sourceMat.copyTo(hsvMat);
// convert mat to HSV format for Core.inRange()
Imgproc.cvtColor(hsvMat, hsvMat, Imgproc.COLOR_RGB2HSV);
Scalar lowerb = new Scalar(85, 50, 40); // lower color border for BLUE
Scalar upperb = new Scalar(135, 255, 255); // upper color border for BLUE
Core.inRange(hsvMat, lowerb, upperb, roiTmp); // select only blue pixels
// find contours
List<MatOfPoint> contours = new ArrayList<>();
List<RotatedRect> boundingRects = new ArrayList<>();
Imgproc.findContours(roiTmp, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// find appropriate bounding rectangles
for (MatOfPoint contour : contours) {
MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
double rectangleArea = boundingRect.size.area();
// test min ROI area in pixels
if (rectangleArea > 40001) {//400000
Point rotated_rect_points[] = new Point[4];
boundingRect.points(rotated_rect_points);
Rect rect = Imgproc.boundingRect(new MatOfPoint(rotated_rect_points));
// test horizontal ROI orientation
if (rect.width > rect.height) {
Imgproc.rectangle(sourceMat, rect.tl(), rect.br(), green, 3);
}
}
}
roiBitmap = Bitmap.createBitmap(sourceMat.cols(), sourceMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(sourceMat, roiBitmap);
return roiBitmap;
}
I don't know a lot about Java but AFAIK it must be similar to c++, so whether you know about it or not,I try my code in c++. Suppose that you are working on the below rectangle:
For having the above Rect in OpenCV:
Rect r = Rect(20,20,30,20); // Rect(x,y,width,height)
And we could access the below-right x with using the br() like the below.
int x2=r.br().x ;
Below pic will tell you the whole story.
--------------read edit below---------------
I am trying to detect the edge of the pupils and iris within various images. I am altering parameters and such but I can only manage to ever get one iris/pupil outline correct, or get unnecessary outlines in the background, or none at all. Is the some specific parameters that I should try to try and get the correct outlines. Or is there a way that I can crop the image just to the eyes, so the system can focus on that part?
This is my UPDATED method:
private void findPupilIris() throws IOException {
//converts and saves image in grayscale
Mat newimg = Imgcodecs.imread("/Users/.../pic.jpg");
Mat des = new Mat(newimg.rows(), newimg.cols(), newimg.type());
Mat norm = new Mat();
Imgproc.cvtColor(newimg, des, Imgproc.COLOR_BGR2HSV);
List<Mat> hsv = new ArrayList<Mat>();
Core.split(des, hsv);
Mat v = hsv.get(2); //gets the grey scale version
Imgcodecs.imwrite("/Users/Lisa-Maria/Documents/CapturedImages/B&Wpic.jpg", v); //only writes mats
CLAHE clahe = Imgproc.createCLAHE(2.0, new Size(8,8) ); //2.0, new Size(8,8)
clahe.apply(v,v);
// Imgproc.GaussianBlur(v, v, new Size(9,9), 3); //adds left pupil boundary and random circle on 'a'
// Imgproc.GaussianBlur(v, v, new Size(9,9), 13); //adds right outer iris boundary and random circle on 'a'
Imgproc.GaussianBlur(v, v, new Size(9,9), 7); //adds left outer iris boundary and random circle on left by hair
// Imgproc.GaussianBlur(v, v, new Size(7,7), 15);
Core.addWeighted(v, 1.5, v, -0.5, 0, v);
Imgcodecs.imwrite("/Users/.../after.jpg", v); //only writes mats
if (v != null) {
Mat circles = new Mat();
Imgproc.HoughCircles( v, circles, Imgproc.CV_HOUGH_GRADIENT, 2, v.rows(), 100, 20, 20, 200 );
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
System.out.println("circles.cols() " + circles.cols());
if(circles.cols() > 0) {
System.out.println("1");
for (int x = 0; x < circles.cols(); x++) {
System.out.println("2");
double vCircle[] = circles.get(0, x);
if(vCircle == null) {
break;
}
Point pt = new Point(Math.round(vCircle[0]), Math.round(vCircle[1]));
int radius = (int) Math.round(vCircle[2]);
//draw the found circle
Imgproc.circle(v, pt, radius, new Scalar(255,0,0),2); //newimg
//Imgproc.circle(des, pt, radius/3, new Scalar(225,0,0),2); //pupil
Imgcodecs.imwrite("/Users/.../Houghpic.jpg", v); //newimg
//draw the mask: white circle on black background
// Mat mask = new Mat( new Size( des.cols(), des.rows() ), CvType.CV_8UC1 );
// Imgproc.circle(mask, pt, radius, new Scalar(255,0,0),2);
// des.copyTo(des,mask);
// Imgcodecs.imwrite("/Users/..../mask.jpg", des); //newimg
Imgproc.logPolar(des, norm, pt, radius, Imgproc.WARP_FILL_OUTLIERS);
Imgcodecs.imwrite("/Users/..../Normalised.jpg",norm);
}
}
}
}
Result: hough pic
Following discussion in comments, I am posting a general answer with some results I got on the worst case image uploaded by the OP.
Note : The code I am posting is in Python, since it is the fastest for me to write
Step 1. As you ask for a way to crop the image, so as to focus on the eyes only, you might want to look at Face Detection. Since, the image essentially requires to find eyes only, I did the following:
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
eyes = eye_cascade.detectMultiScale(v) // v is the value channel of the HSV image
// The results "eyes" gives you the dimensions of the rectangle where the eyes are detected as [x, y, w, h]
// Just for drawing
cv2.rectangle(v, (x1, y1), (x1+w1, y1+h1), (0, 255, 0), 2)
cv2.rectangle(v, (x2, y2), (x2+w2, y2+h2), (0, 255, 0), 2)
Now, once you have the bounding rectangles, you can crop the rectangles from the image like:
crop_eye1 = v[y1:y1+h1, x1:x1+w1]
crop_eye2 = v[y2:y2+h2, x2:x2+w2]
After you obtain the rectangles, I would suggest looking into different color spaces instead of RGB/BGR, HSV/Lab/Luv in particular.
Because the R, G, and B components of an object’s color in a digital image are all correlated with the amount of light hitting the object, and therefore with each other, image descriptions in terms of those components make object discrimination difficult. Descriptions in terms of hue/lightness/chroma or hue/lightness/saturation are often more relevant
Then, once, you have the eyes, its time to equalize the contrast of the image, however, I suggest using CLAHE and play with the parameters for clipLimit and tileGridSize. Here is a code which I implemented a while back in Java:
private static Mat clahe(Mat image, int ClipLimit, Size size){
CLAHE clahe = Imgproc.createCLAHE();
clahe.setClipLimit(ClipLimit);
clahe.setTilesGridSize(size);
Mat dest_image = new Mat();
clahe.apply(image, dest_image);
return dest_image;
}
Once you are satisfied, you should sharpen the image so that HoughCircle is robust. You should look at unsharpMask. Here is the code in Java for UnsharpMask I implemented in Java:
private static Mat unsharpMask(Mat input_image, Size size, double sigma){
// Make sure the {input_image} is gray.
Mat sharpend_image = new Mat(input_image.rows(), input_image.cols(), input_image.type());
Mat Blurred_image = new Mat(input_image.rows(), input_image.cols(), input_image.type());
Imgproc.GaussianBlur(input_image, Blurred_image, size, sigma);
Core.addWeighted(input_image, 2.0D, Blurred_image, -1.0D, 0.0D, sharpened_image);
return sharpened_image;
}
Alternatively, you could use bilateral filter, which is edge preserving smoothing, or read through this for defining a custom kernel for sharpening image.
Hope it helps and best of luck!