We Keep Coding

How to apply Canny edge detector effectively on image using Opencv in Android?

I am making an Android application to detect multiple objects from an image, then process those objects and compare them with reference objects to detect anomalies. I tested different image edge detectors in python and Prewitt operator gave me the best result as shown below https://i.imgur.com/4iwOx9s.png For android, I used Canny edge detector, but the result is not as good as Prewitt as shown below https://i.imgur.com/Bax1Wxw.png
The purpose of applying Canny edge detector is the detect the largest contour first then extract this contour and detect every object found in this contour (which in my case 3 objects).
Here is the java code I tried
Bitmap bmp = BitmapFactory.decodeResource(getResources(), R.drawable.coffret);
//compress bitmap
bmp = getResizedBitmap(bmp, 500);
Mat rgbMat = new Mat();
Utils.bitmapToMat(bmp, rgbMat);
Mat grayMat = new Mat();
Mat bwMat = new Mat();
Imgproc.cvtColor(rgbMat, grayMat, Imgproc.COLOR_RGB2GRAY);
Imgproc.equalizeHist(grayMat, grayMat);
//Imgproc.adaptiveThreshold(grayMat, grayMat, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 15, 40);
Imgproc.Canny(grayMat, bwMat, 50, 200, 3, false);
//find largest contour
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(bwMat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_NONE);
double maxArea = -1;
int maxAreaIdx = -1;
if (contours.size() > 0) {
MatOfPoint temp_contour = contours.get(0); //the largest is at the index 0 for starting point
MatOfPoint2f approxCurve = new MatOfPoint2f();
Mat largest_contour = contours.get(0);
List<MatOfPoint> largest_contours = new ArrayList<MatOfPoint>();
for (int idx = 0; idx < contours.size(); idx++) {
temp_contour = contours.get(idx);
double contourarea = Imgproc.contourArea(temp_contour);
//compare this contour to the previous largest contour found
if (contourarea > maxArea) {
//check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f( temp_contour.toArray() );
int contourSize = (int)temp_contour.total();
Imgproc.approxPolyDP(new_mat, approxCurve, contourSize*0.05, true);
if (approxCurve.total() == 4) {
maxArea = contourarea;
maxAreaIdx = idx;
largest_contours.add(temp_contour);
largest_contour = temp_contour;
}
}
}
if (largest_contours.size() >= 1) {
MatOfPoint temp_largest = largest_contours.get(largest_contours.size()-1);
largest_contours = new ArrayList<MatOfPoint>();
largest_contours.add(temp_largest);
Imgproc.cvtColor(bwMat, bwMat, Imgproc.COLOR_BayerBG2RGB);
Imgproc.drawContours(bwMat, largest_contours, -1, new Scalar(0, 255, 0), 1);
}
}
Utils.matToBitmap(bwMat, bmp);
Matrix matrix = new Matrix();
matrix.postRotate(180);
bmp = Bitmap.createBitmap(bmp, 0, 0, bmp.getWidth(), bmp.getHeight(), matrix, true);
imgView.setImageBitmap(bmp);
As you can notice, using the Prewitt operator the text is clear and contours are more defined.
I think I am not applying Canny correctly that's why the largest contour is not detected. What am I doing wrong?
Edit:
Here is the original image
https://i.imgur.com/BtyZOvj.jpg

It seems that you are applying Canny edge detector correct. However, sometimes, it could be trick to define proper minimum and maximum values to threshold the edges.
Maybe the interval you are using (i.e., 50 to 200) is large for your purpose. You could try higher minimum and maximum values with tight intervals, for instance, 175 to 200.
As the image gets complex you need to take special care of these parameters. Sometimes, different sections of your image will require different minimum and maximum thresholds. Take a look at this reference, it contains a decent explanation about Improvement on Canny edge detection. At the end of this page, there is a explanation of how to determine the dual-threshold value.

OpenCV Java Harris Corner Detection

I am developing an Android application and I want to make use of Harris corner detection. I want to draw the corners detected but I cannot seem to find the documentation for the Java code.
My code so far:
Mat inputImage = inputFrame.rgba();
Imgproc.cornerHarris(inputImage, inputImage, 7, 5, 0.05, Imgproc.BORDER_DEFAULT);
How can I detect and display the corners?

For Java you can try this piece of code.
private void Harris(Mat Scene, Mat Object, int thresh) {
// This function implements the Harris Corner detection. The corners at intensity > thresh
// are drawn.
Mat Harris_scene = new Mat();
Mat Harris_object = new Mat();
Mat harris_scene_norm = new Mat(), harris_object_norm = new Mat(), harris_scene_scaled = new Mat(), harris_object_scaled = new Mat();
int blockSize = 9;
int apertureSize = 5;
double k = 0.1;
Imgproc.cornerHarris(Scene, Harris_scene,blockSize, apertureSize,k);
Imgproc.cornerHarris(Object, Harris_object, blockSize,apertureSize,k);
Core.normalize(Harris_scene, harris_scene_norm, 0, 255, Core.NORM_MINMAX, CvType.CV_32FC1, new Mat());
Core.normalize(Harris_object, harris_object_norm, 0, 255, Core.NORM_MINMAX, CvType.CV_32FC1, new Mat());
Core.convertScaleAbs(harris_scene_norm, harris_scene_scaled);
Core.convertScaleAbs(harris_object_norm, harris_object_scaled);
for( int j = 0; j < harris_scene_norm.rows() ; j++){
for( int i = 0; i < harris_scene_norm.cols(); i++){
if ((int) harris_scene_norm.get(j,i)[0] > thresh){
Imgproc.circle(harris_scene_scaled, new Point(i,j), 5 , new Scalar(0), 2 ,8 , 0);
}
}
}
for( int j = 0; j < harris_object_norm.rows() ; j++){
for( int i = 0; i < harris_object_norm.cols(); i++){
if ((int) harris_object_norm.get(j,i)[0] > thresh){
Imgproc.circle(harris_object_scaled, new Point(i,j), 5 , new Scalar(0), 2 ,8 , 0);
}
}
}
}
I just wrote this following code here

I know this is not ideal, but it is also not so bad - you can look at the c++ documentation and examples, the translation to Java is usually straight forward:
One example: Harris corner detector. (you did not mention your version, this is from v2.4).

If anyone is still looking for OpenCV Java Samples, you can find it here in the following links.
Complete Java Samples
https://github.com/opencv/opencv/tree/master/samples/java/tutorial_code
Motion Tracking
https://github.com/opencv/opencv/tree/master/samples/java/tutorial_code/TrackingMotion
Harris Corner Detection
https://github.com/opencv/opencv/blob/master/samples/java/tutorial_code/TrackingMotion/harris_detector/CornerHarrisDemo.java

HoughLinesP not detecting lines OpenCV android

I am working with OpenCV 3.0 for Android. I have an image in which i want to detect angle of hands inside circular dials. for that i am working on HoughLinesP to detect hands.
Here is the code.
Mat imgSource = new Mat(), imgCirclesOut = new Mat(),imgLinesOut=new Mat();
//grey opencv
Imgproc.cvtColor(Image, imgSource, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur( imgSource, imgSource, new Size(9, 9), 2, 2 );
int threshold = 0;
int minLineSize = 0;
int lineGap = 0;
Imgproc.HoughLinesP(imgSource, imgLinesOut, 1, Math.PI/180, threshold, minLineSize, lineGap);
for( int j = 0; i < imgLinesOut.cols(); i++ )
{
double[] vec=imgLinesOut.get(0,j);
Point pt1, pt2;
pt1=new Point(vec[0],vec[1]);
pt2=new Point(vec[2],vec[3]);
Imgproc.line( Image, pt1, pt2, new Scalar(0,0,255), 3, Core.LINE_AA,0);
}
But result is
What i need is the angle of hands in these circles. Any help regarding this issue is highly appreciated. Thanks in ADvance
Edit
I have updated my code with this
Mat imgSource = new Mat(), imgCirclesOut = new Mat(),imgLinesOut=new Mat();
Imgproc.GaussianBlur( Image, imgSource, new Size(5, 5), 2, 2 );
int threshold = 20;
int minLineSize = 0;
int lineGap = 10;
Imgproc.Canny(imgSource, imgSource, 70, 100);
Imgproc.HoughLinesP(imgSource, imgLinesOut, 1, Math.PI/180, threshold, minLineSize, lineGap);
for( int j = 0; j < imgLinesOut.cols(); j++ )
{
double[] vec=imgLinesOut.get(0,j);
Point pt1, pt2;
pt1=new Point(vec[0],vec[1]);
pt2=new Point(vec[2],vec[3]);
Imgproc.line( imgSource, pt1, pt2, new Scalar(0,0,255), 3, Core.LINE_AA,0);
}
as suggested by #Micka, there is no need of Graying image(I removed cvtcolor). I also decreased value of GuassianBlur Size to 5. I have added Canny on image too for edges.
Resulting blur image is

Detecting lines can be a problem in such small images, since you have to few points to fill the Hough accumulator properly.
I propose to use a different approach:
Segment each circle (dial)
Extract the largest dark blob (hand)
Below is a simple implementation of this idea. The code is in C++, but you can easily port to Java, or at least use as a reference.
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int, char**)
{
Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);
Mat3b res;
cvtColor(img, res, COLOR_GRAY2BGR);
// Find dials
vector<Vec3f> circles;
HoughCircles(img, circles, CV_HOUGH_GRADIENT, 1, img.cols/10, 400, 40);
// For each dial
for (int i = 0; i < circles.size(); ++i)
{
// Segment the dial
Mat1b dial(img.size(), uchar(255));
Mat1b mask(img.size(), uchar(0));
circle(mask, Point(circles[i][0], circles[i][1]), circles[i][2], Scalar(255), CV_FILLED);
img.copyTo(dial, mask);
// Apply threshold and open
Mat1b bin;
threshold(dial, bin, 127, 255, THRESH_BINARY_INV);
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(5,5));
morphologyEx(bin, bin, MORPH_OPEN, kernel);
// Get min area rect
vector<Point> points;
findNonZero(bin, points);
RotatedRect r = minAreaRect(points);
// Draw min area rect
Point2f pts[4];
r.points(pts);
for (int j = 0; j < 4; ++j) {
line(res, pts[j], pts[(j + 1) % 4], Scalar(0, 255, 0), 1);
}
}
imshow("Result", res);
waitKey();
return 0;
}
Starting from this image:
I find hands here:

for( int j = 0; j < imgLinesOut.size(); j++ )
This will give the size of the vector.To iterate through that vector

java.lang.IndexOutOfBoundsException error when applying OpenCV methods in Android

The code is about finding the largest rectangle in an Android device's camera using OpenCV. The app always force close but I can't find the trouble.
The input of the method is a Mat obtained by CvCameraViewFrame.rgba().
private Mat findLargestRectangle(Mat original_image)
{
Mat imgSource = original_image;
// convert the image to black and white
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BGR2GRAY);
// convert the image to black and white does (8 bit)
Imgproc.Canny(imgSource, imgSource, 50, 50);
// apply gaussian blur to smoothen lines of dots
Imgproc.GaussianBlur(imgSource, imgSource, new Size(5, 5), 5);
// find the contours
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgSource, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
double maxArea = -1;
int maxAreaIdx = -1;
MatOfPoint temp_contour = contours.get(0); // the largest is at the
// index 0 for starting
// point
MatOfPoint2f approxCurve = new MatOfPoint2f();
Mat largest_contour = contours.get(0);
List<MatOfPoint> largest_contours = new ArrayList<MatOfPoint>();
for (int idx = 0; idx < contours.size(); idx++)
{
temp_contour = contours.get(idx);
double contourarea = Imgproc.contourArea(temp_contour);
// compare this contour to the previous largest contour found
if (contourarea > maxArea)
{
// check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f(temp_contour.toArray());
int contourSize = (int) temp_contour.total();
Imgproc.approxPolyDP(new_mat, approxCurve, contourSize * 0.05, true);
if (approxCurve.total() == 4)
{
maxArea = contourarea;
maxAreaIdx = idx;
largest_contours.add(temp_contour);
largest_contour = temp_contour;
}
}
}
MatOfPoint temp_largest = largest_contours.get(largest_contours.size() - 1);
largest_contours = new ArrayList<MatOfPoint>();
largest_contours.add(temp_largest);
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BayerBG2RGB);
Imgproc.drawContours(imgSource, largest_contours, -1, new Scalar(0, 255, 0), 1);
// create the new image here using the largest detected square
Toast.makeText(getApplicationContext(), "Largest Contour: ", Toast.LENGTH_LONG).show();
return imgSource;
}
Here are the error information in LogCat:
error opening trace file: No such file or directory (2)
Tegra Version detected: 0
FATAL EXCEPTION: Thread-14346
java.lang.IndexOutOfBoundsException: Invalid index 0, size is 0
at java.util.ArrayList.throwIndexOutOfBoundsException(ArrayList.java:251)
at java.util.ArrayList.get(ArrayList.java:304)
at org.opencv.samples.tutorial2.Tutorial2Activity.findLargestRectangle(Tutorial2Activity.java:221)
at org.opencv.samples.tutorial2.Tutorial2Activity.onCameraFrame(Tutorial2Activity.java:169)
at org.opencv.android.CameraBridgeViewBase.deliverAndDrawFrame(CameraBridgeViewBase.java:387)
at org.opencv.android.JavaCameraView$CameraWorker.run(JavaCameraView.java:328)
at java.lang.Thread.run(Thread.java:856)
And the 221st line of the Tutorial2Avctivity is:
MatOfPoint temp_contour = contours.get(0);
Please tell me what the errors are. Thanks a lot!

This simply means that the size of the ArrayList of contours is empty.
I would suggest adding a conditional block to allow your application cope with the scenario that no contours could be found in the input image.
If you're consistently getting no contours, regardless of the input image, you may want to review the documentation (or Javadoc):
image – Source, an 8-bit single-channel image. Non-zero pixels are treated as 1’s. Zero pixels remain 0’s, so the image is treated as binary . You can use compare() , inRange() , threshold() , adaptiveThreshold() , Canny() , and others to create a binary image out of a grayscale or color one.
Perhaps the threshold values in your Canny() function is not yielding any edges.
If you're still having no luck, perhaps you could test with the input image and values from this blog post.

Try with this,
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgSource, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
You have to add some value into counter before below line
MatOfPoint temp_contour = contours.get(0);

Capturing an A4 size document. Can OpenCV do this in Android?

This is my first question on Stackoverflow.
I'm a software engineer in profession(Java, C#) and I have 0 knowledge on image processing and Android related technologies. I'm writing an android application for my masters thesis to support visually impaired people in my country to read a document from their Android smartphones, in our native language.
I have selected the sample size of the document as A4, and the app should eventually automatically focus on the document once the whole A4 doc is in camera's view (audible notification should be given to user), and it should then capture that image.
Then I plan to run the document through tesseract engine to convert it into OCR. (Some other guy is doing the text-to-speech part of this application)
I googled thorough couple of applications and came up with the documentation of OpenCV. The http://docs.opencv.org/opencv_tutorials.pdf explains something about "Creating Bounding boxes and circles for contours" and looks like its gonna be my life saver.
My MSC project is a 300 hour part time project so I fear that I will get ended up with nothing after spending time to convert C++/ Python examples to Java by myself to learn OpenCV. I went through the JavaCV as well, but looks like its still in a growing stage, so most probably I will have to convert the examples by myself.
What I wanted to ask from experts is that whether the OpenCV can really do a thing like this?
Thanks in advance!
Edit. I had a look at the link on the comment and trying to port the C++ example to Java. Here is what I got so far. Still there are couple of things to do though...
int thresh = 50, N = 11;
// helper function:
// finds a cosine of angle between vectors
// from pt0->pt1 and from pt0->pt2
static double angle(Point pt1, Point pt2, Point pt0)
{
double dx1 = pt1.x - pt0.x;
double dy1 = pt1.y - pt0.y;
double dx2 = pt2.x - pt0.x;
double dy2 = pt2.y - pt0.y;
return (dx1*dx2 + dy1*dy2)/Math.sqrt((dx1*dx1 + dy1*dy1)*(dx2*dx2 + dy2*dy2) + 1e-10);
}
public void find_squares(Mat image, Vector<Vector<Point> > squares)
{
Imgproc img = new Imgproc();
// blur will enhance edge detection
org.opencv.core.Mat blurred = new org.opencv.core.Mat();
Imgproc.medianBlur(image, blurred, 9);
Mat gray0 = new Mat(blurred.size(), CvType.CV_8U);
Mat gray = new Mat();
// Vector<Vector<Point> > contours;
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
// find squares in every color plane of the image
for (int c = 0; c < 3; c++)
{
int ch[] = {c, 0};
// Core.mixChannels(blurred, 1, gray0, 1, ch, 1);
List<Mat> src = new ArrayList<Mat>();
src.add(blurred);
List<Mat> dest = new ArrayList<Mat>();
dest.add(gray0);
MatOfInt a = new MatOfInt(ch);
Core.mixChannels(src, dest, a);
// try several threshold levels
final int threshold_level = 2;
for (int l = 0; l < threshold_level; l++)
{
// Use Canny instead of zero threshold level!
// Canny helps to catch squares with gradient shading
if (l == 0)
{
Imgproc.Canny(gray0, gray, 10, 20, 3, false);
// Dilate helps to remove potential holes between edge segments
Point point = new Point(-1, -1);
Imgproc.dilate(gray, gray, new Mat(), point, 1);
}
else
{
// TODO
// gray = gray0 >= (l+1) * 255 / threshold_level;
}
// Find contours and store them in a list. //TODO
Imgproc.findContours(gray, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// Test contours
MatOfPoint2f approx = new MatOfPoint2f();
for (int i = 0; i < contours.size(); i++)
{
// approximate contour with accuracy proportional
// to the contour perimeter
double epilson = Imgproc.arcLength(new MatOfPoint2f(contours.get(i)), true);
epilson *= 0.02;
Imgproc.approxPolyDP(new MatOfPoint2f(contours.get(i)), approx, epilson, true);
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
// Mat mmm = new Mat();
// MatOfPoint ppp = new MatOfPoint();
if (/*TODO*/approx.size().area() == 4 &&
Math.abs(Imgproc.contourArea(approx)) > 1000 &&
Imgproc.isContourConvex(/*TODO*/approx))
{
double maxCosine = 0;
for (int j = 2; j < 5; j++)
{
double cosine = Math.abs(angle(approx[j % 4], approx[j - 2], approx[j - 1]));
maxCosine = /*TODO*/MAX(maxCosine, cosine);
}
if (maxCosine < 0.3)
squares./*TODO*/push_back(approx);
}
}
}
}
}
}

Just to answer the question, Yes, this can be done in OpenCv (among many other things) and I have completed the project that I explained in the question. Also voted up Abid's answer for the link he provided:)