We Keep Coding

How to apply Canny edge detector effectively on image using Opencv in Android?

I am making an Android application to detect multiple objects from an image, then process those objects and compare them with reference objects to detect anomalies. I tested different image edge detectors in python and Prewitt operator gave me the best result as shown below https://i.imgur.com/4iwOx9s.png For android, I used Canny edge detector, but the result is not as good as Prewitt as shown below https://i.imgur.com/Bax1Wxw.png
The purpose of applying Canny edge detector is the detect the largest contour first then extract this contour and detect every object found in this contour (which in my case 3 objects).
Here is the java code I tried
Bitmap bmp = BitmapFactory.decodeResource(getResources(), R.drawable.coffret);
//compress bitmap
bmp = getResizedBitmap(bmp, 500);
Mat rgbMat = new Mat();
Utils.bitmapToMat(bmp, rgbMat);
Mat grayMat = new Mat();
Mat bwMat = new Mat();
Imgproc.cvtColor(rgbMat, grayMat, Imgproc.COLOR_RGB2GRAY);
Imgproc.equalizeHist(grayMat, grayMat);
//Imgproc.adaptiveThreshold(grayMat, grayMat, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 15, 40);
Imgproc.Canny(grayMat, bwMat, 50, 200, 3, false);
//find largest contour
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(bwMat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_NONE);
double maxArea = -1;
int maxAreaIdx = -1;
if (contours.size() > 0) {
MatOfPoint temp_contour = contours.get(0); //the largest is at the index 0 for starting point
MatOfPoint2f approxCurve = new MatOfPoint2f();
Mat largest_contour = contours.get(0);
List<MatOfPoint> largest_contours = new ArrayList<MatOfPoint>();
for (int idx = 0; idx < contours.size(); idx++) {
temp_contour = contours.get(idx);
double contourarea = Imgproc.contourArea(temp_contour);
//compare this contour to the previous largest contour found
if (contourarea > maxArea) {
//check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f( temp_contour.toArray() );
int contourSize = (int)temp_contour.total();
Imgproc.approxPolyDP(new_mat, approxCurve, contourSize*0.05, true);
if (approxCurve.total() == 4) {
maxArea = contourarea;
maxAreaIdx = idx;
largest_contours.add(temp_contour);
largest_contour = temp_contour;
}
}
}
if (largest_contours.size() >= 1) {
MatOfPoint temp_largest = largest_contours.get(largest_contours.size()-1);
largest_contours = new ArrayList<MatOfPoint>();
largest_contours.add(temp_largest);
Imgproc.cvtColor(bwMat, bwMat, Imgproc.COLOR_BayerBG2RGB);
Imgproc.drawContours(bwMat, largest_contours, -1, new Scalar(0, 255, 0), 1);
}
}
Utils.matToBitmap(bwMat, bmp);
Matrix matrix = new Matrix();
matrix.postRotate(180);
bmp = Bitmap.createBitmap(bmp, 0, 0, bmp.getWidth(), bmp.getHeight(), matrix, true);
imgView.setImageBitmap(bmp);
As you can notice, using the Prewitt operator the text is clear and contours are more defined.
I think I am not applying Canny correctly that's why the largest contour is not detected. What am I doing wrong?
Edit:
Here is the original image
https://i.imgur.com/BtyZOvj.jpg

It seems that you are applying Canny edge detector correct. However, sometimes, it could be trick to define proper minimum and maximum values to threshold the edges.
Maybe the interval you are using (i.e., 50 to 200) is large for your purpose. You could try higher minimum and maximum values with tight intervals, for instance, 175 to 200.
As the image gets complex you need to take special care of these parameters. Sometimes, different sections of your image will require different minimum and maximum thresholds. Take a look at this reference, it contains a decent explanation about Improvement on Canny edge detection. At the end of this page, there is a explanation of how to determine the dual-threshold value.

how to extract an object from the camera preview android opencv

I am trying to scan a MTG card using OpenCV on android. I have it to where I can detect the edges of the card and even draw an outline around it but am confused on how to extract just the card from the background and then exit from the camera preview. Here is my code so far:
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
Mat result = new Mat();
Mat mask = new Mat();
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
mRgba = inputFrame.rgba();
Imgproc.Canny(mRgba, result, 40, 120);
Imgproc.GaussianBlur(result, result, new Size(9,9), 2, 2);
Imgproc.findContours(result, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE, new Point(0,0));
Imgproc.drawContours(mask, contours, -1, new Scalar(0, 255, 0), 1);
hierarchy.release();
for ( int contourIdx=0; contourIdx < contours.size(); contourIdx++ )
{
// Minimum size allowed for consideration
MatOfPoint2f approxCurve = new MatOfPoint2f();
MatOfPoint2f contour2f = new MatOfPoint2f( contours.get(contourIdx).toArray() );
//Processing on mMOP2f1 which is in type MatOfPoint2f
double approxDistance = Imgproc.arcLength(contour2f, true)*0.02;
Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);
//Convert back to MatOfPoint
MatOfPoint points = new MatOfPoint( approxCurve.toArray() );
// Get bounding rect of contour
Rect rect = Imgproc.boundingRect(points);
Imgproc.rectangle(mRgba, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 0, 0, 255), 3);
}
Bitmap card = Bitmap.createBitmap(result.cols(), result.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(result, card);
return mRgba;
}
}
Here is an example of what it looks like when run the code. As you can see the card is outlined by a red rectangle now but how do I extract just whats in the rectangle, save it to a mat or bitmap and then exit the camera?
screenshot of outlined card

I didn't test it, but can't you just use...
return new Mat(mRgba, rect);

How to remove overlapping contours

I am working on a program where I am trying to extract the colored squares from a puzzle. I take a frame from a video capture then find the all contours. I then remove contours that aren't in the shape of a square (This works alright but looking for a better method). The main problem I am facing is that there are overlapping contours. I use RETR_TREE to get all contours but when using RETR_EXTERNAL The contours become harder to detect. Is there a way I can improve the detection of squares? Or a way that I can remove the overlapping contours in the image.
Here is an image of where there are overlapping contours:
There were 11 contours found in this image but I want only 9.(I draw the rects to see the overlapping a little easier)
. How can I remove the inner contours?
Here is my code:
public Mat captureFrame(Mat capturedFrame){
Mat newFrame = new Mat();
capturedFrame.copyTo(newFrame);
//Gray
Mat gray = new Mat();
Imgproc.cvtColor(capturedFrame, gray, Imgproc.COLOR_RGB2GRAY);
//Blur
Mat blur = new Mat();
Imgproc.blur(gray, blur, new Size(3,3));
//Canny image
Mat canny = new Mat();
Imgproc.Canny(blur, canny, 20, 40, 3, true);
//Dilate image to increase size of lines
Mat kernel = Imgproc.getStructuringElement(1, new Size(3,3));
Mat dilated = new Mat();
Imgproc.dilate(canny,dilated, kernel);
List<MatOfPoint> contours = new ArrayList<>();
List<MatOfPoint> squareContours = new ArrayList<>();
//find contours
Imgproc.findContours(dilated, contours, new Mat(), Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE);
//Remove contours that aren't close to a square shape.
//Wondering if there is a way I can improve this?
for(int i = 0; i < contours.size(); i++){
double area = Imgproc.contourArea( contours.get(i));
MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
double perimeter = Imgproc.arcLength(contour2f, true);
//Found squareness equation on wiki...
//https://en.wikipedia.org/wiki/Shape_factor_(image_analysis_and_microscopy)
double squareness = 4 * Math.PI * area / Math.pow(perimeter, 2);
//add contour to new List if it has a square shape.
if(squareness >= 0.7 && squareness <= 0.9 && area >= 3000){
squareContours.add(contours.get(i));
}
}
MatOfPoint2f approxCurve = new MatOfPoint2f();
for(int n = 0; n < squareContours.size(); n++){
//Convert contours(n) from MatOfPoint to MatOfPoint2f
MatOfPoint2f contour2f = new MatOfPoint2f( squareContours.get(n).toArray());
//Processing on mMOP2f1 which is in type MatOfPoint2f
double approxDistance = Imgproc.arcLength(contour2f, true)*0.02;
Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);
//Convert back to MatOfPoint
MatOfPoint points = new MatOfPoint( approxCurve.toArray());
// Get bounding rect of contour
Rect rect = Imgproc.boundingRect(points);
//length and width should be about the same
if(rect.height - rect.width < Math.abs(10)){
System.out.printf("%s , %s \n", rect.height, rect.width);
}
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(newFrame, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height),new Scalar (255, 0, 0, 255), 3);
}
return newFrame;
}

Fortunately, cv::findContours also provides us with hierarchy matrix, which you have ignored in your snippet, the hierarchy is very useful for all modes other than RETR_EXTERNAL, You may find the detailed doc of hierarchy matrix here.
Hierarchy matrix contains the data in the format [Next, Previous, First_Child, Parent] for each contour. Now you may filter contours, which a logic such as select only those contours where parent == -1, this will eliminate the sub contours inside the parent contour.
To make use of the hierarchy matrix you need to call cv::findContours as:
cv::Mat hierarchy;
Imgproc.findContours(dilated, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_NONE);

OpenCV + Android + Vehicle number Plate Recognition

I'm developing a Android app to detect vehicle number plate. i did image processing up to findContours level of image. Now i need to convert following C++ code to Opencv Based Android java.
This is original image
This is after Otsu thresholding image
This is my andoid+opencv code (working 100%)
ImageView imgView = (ImageView) findViewById(R.id.imageView1);
Bitmap bmp = BitmapFactory.decodeResource(getResources(),car);
//First convert Bitmap to Mat
Mat ImageMatin = new Mat ( bmp.getHeight(), bmp.getWidth(), CvType.CV_8U, new Scalar(4));
Mat ImageMatout = new Mat ( bmp.getHeight(), bmp.getWidth(), CvType.CV_8U, new Scalar(4));
Mat ImageMatBk = new Mat ( bmp.getHeight(), bmp.getWidth(), CvType.CV_8U, new Scalar(4));
Mat ImageMatTopHat = new Mat ( bmp.getHeight(), bmp.getWidth(), CvType.CV_8U, new Scalar(4));
Mat temp = new Mat ( bmp.getHeight(), bmp.getWidth(), CvType.CV_8U, new Scalar(4));
Bitmap myBitmap32 = bmp.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(myBitmap32, ImageMatin);
//Converting RGB to Gray.
Imgproc.cvtColor(ImageMatin, ImageMatBk, Imgproc.COLOR_RGB2GRAY,8);
Imgproc.dilate(ImageMatBk, temp, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(9, 9)));
Imgproc.erode(temp, ImageMatTopHat, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(9,9)));
//Core.absdiff(current, previous, difference);
Core.absdiff(ImageMatTopHat, ImageMatBk, ImageMatout);
//Sobel operator in horizontal direction.
Imgproc.Sobel(ImageMatout,ImageMatout,CvType.CV_8U,1,0,3,1,0.4,Imgproc.BORDER_DEFAULT);
//Converting GaussianBlur
Imgproc.GaussianBlur(ImageMatout, ImageMatout, new Size(5,5),2);
Imgproc.dilate(ImageMatout, ImageMatout, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(3,3)));
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(17, 3));
Imgproc.morphologyEx(ImageMatout, ImageMatout, Imgproc.MORPH_CLOSE, element);
//threshold image
Imgproc.threshold(ImageMatout, ImageMatout, 0, 255, Imgproc.THRESH_OTSU+Imgproc.THRESH_BINARY);
Now I need to extract number Plate
Please help me to convert following C++ code to java+opencv:.
std::vector rects;
std::vector<std::vector >::iterator itc = contours.begin();
while (itc != contours.end())
{
cv::RotatedRect mr = cv::minAreaRect(cv::Mat(*itc));
float area = fabs(cv::contourArea(*itc));
float bbArea=mr.size.width * mr.size.height;
float ratio = area/bbArea;
if( (ratio < 0.45) || (bbArea < 400) ){
itc= contours.erase(itc);
}else{
++itc;
rects.push_back(mr);
}
}

Looking at http://docs.opencv.org/java and the documentation for findContours in particular
instead of
std::vector<std::vector<cv::Point> > contours;
you will have
java.util.ArrayList<MatOfPoint> contours;
You can use contours.listIterator() to traverse the list. Something like the below (not compiled let alone run,likely to contain major blunders):
import java.util.*;
import org.opencv.imgproc.Imgproc;
import org.opencv.core.*;
/* ... */
ArrayList<RotatedRect> rects = new ArrayList<RotatedRect>()
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(image, contours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
ListIterator<MatOfPoint> itc = contours.listIterator();
while(itc.hasNext())
{
MatOfPoint2f mp2f = new MatOfPoint2f(itc.next().toArray());
RotatedRect mr = Imgproc.minAreaRect(mp2f);
double area = Math.abs(Imgproc.contourArea(mp2f));
double bbArea= mr.size.area();
double ratio = area / bbArea;
if( (ratio < 0.45) || (bbArea < 400) )
{
itc.remove(); // other than deliberately making the program slow,
// does erasing the contour have any purpose?
}
else
{
rects.add(mr);
}
}

java.lang.IndexOutOfBoundsException error when applying OpenCV methods in Android

The code is about finding the largest rectangle in an Android device's camera using OpenCV. The app always force close but I can't find the trouble.
The input of the method is a Mat obtained by CvCameraViewFrame.rgba().
private Mat findLargestRectangle(Mat original_image)
{
Mat imgSource = original_image;
// convert the image to black and white
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BGR2GRAY);
// convert the image to black and white does (8 bit)
Imgproc.Canny(imgSource, imgSource, 50, 50);
// apply gaussian blur to smoothen lines of dots
Imgproc.GaussianBlur(imgSource, imgSource, new Size(5, 5), 5);
// find the contours
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgSource, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
double maxArea = -1;
int maxAreaIdx = -1;
MatOfPoint temp_contour = contours.get(0); // the largest is at the
// index 0 for starting
// point
MatOfPoint2f approxCurve = new MatOfPoint2f();
Mat largest_contour = contours.get(0);
List<MatOfPoint> largest_contours = new ArrayList<MatOfPoint>();
for (int idx = 0; idx < contours.size(); idx++)
{
temp_contour = contours.get(idx);
double contourarea = Imgproc.contourArea(temp_contour);
// compare this contour to the previous largest contour found
if (contourarea > maxArea)
{
// check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f(temp_contour.toArray());
int contourSize = (int) temp_contour.total();
Imgproc.approxPolyDP(new_mat, approxCurve, contourSize * 0.05, true);
if (approxCurve.total() == 4)
{
maxArea = contourarea;
maxAreaIdx = idx;
largest_contours.add(temp_contour);
largest_contour = temp_contour;
}
}
}
MatOfPoint temp_largest = largest_contours.get(largest_contours.size() - 1);
largest_contours = new ArrayList<MatOfPoint>();
largest_contours.add(temp_largest);
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BayerBG2RGB);
Imgproc.drawContours(imgSource, largest_contours, -1, new Scalar(0, 255, 0), 1);
// create the new image here using the largest detected square
Toast.makeText(getApplicationContext(), "Largest Contour: ", Toast.LENGTH_LONG).show();
return imgSource;
}
Here are the error information in LogCat:
error opening trace file: No such file or directory (2)
Tegra Version detected: 0
FATAL EXCEPTION: Thread-14346
java.lang.IndexOutOfBoundsException: Invalid index 0, size is 0
at java.util.ArrayList.throwIndexOutOfBoundsException(ArrayList.java:251)
at java.util.ArrayList.get(ArrayList.java:304)
at org.opencv.samples.tutorial2.Tutorial2Activity.findLargestRectangle(Tutorial2Activity.java:221)
at org.opencv.samples.tutorial2.Tutorial2Activity.onCameraFrame(Tutorial2Activity.java:169)
at org.opencv.android.CameraBridgeViewBase.deliverAndDrawFrame(CameraBridgeViewBase.java:387)
at org.opencv.android.JavaCameraView$CameraWorker.run(JavaCameraView.java:328)
at java.lang.Thread.run(Thread.java:856)
And the 221st line of the Tutorial2Avctivity is:
MatOfPoint temp_contour = contours.get(0);
Please tell me what the errors are. Thanks a lot!

This simply means that the size of the ArrayList of contours is empty.
I would suggest adding a conditional block to allow your application cope with the scenario that no contours could be found in the input image.
If you're consistently getting no contours, regardless of the input image, you may want to review the documentation (or Javadoc):
image – Source, an 8-bit single-channel image. Non-zero pixels are treated as 1’s. Zero pixels remain 0’s, so the image is treated as binary . You can use compare() , inRange() , threshold() , adaptiveThreshold() , Canny() , and others to create a binary image out of a grayscale or color one.
Perhaps the threshold values in your Canny() function is not yielding any edges.
If you're still having no luck, perhaps you could test with the input image and values from this blog post.

Try with this,
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgSource, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
You have to add some value into counter before below line
MatOfPoint temp_contour = contours.get(0);