We Keep Coding

How to detect octogonal shapes using contours

I want to detect octagonal stop sign like the following image with contours but i can't figure out how it is done
Stop Sign:
I already manage to detect triangles
Mat ROI = new Mat();
Mat bgrClone = bgr.clone();
MatOfPoint approxContour = new MatOfPoint();
MatOfPoint2f approxContour2f = new MatOfPoint2f();
List<MatOfPoint> contourDraw = new ArrayList<MatOfPoint>();
for(int i = 0; i < contourList.size(); i++) {
MatOfPoint2f contour2f = new MatOfPoint2f(contourList.get(i).toArray());
double approxDistance = Imgproc.arcLength(contour2f, true) * 0.02;//0.225
Imgproc.approxPolyDP(contour2f, approxContour2f, approxDistance, true);
approxContour2f.convertTo(approxContour, CvType.CV_32S);
if (approxContour.size().height == 3 && (Imgproc.contourArea(contour2f) > 3000) ) { //&& (Imgproc.contourArea(contour2f) > 5000)
contourDraw.add(approxContour);
Imgproc.drawContours(bgr, contourDraw, -1, new Scalar(0,255,0), 1);
Rect cord = Imgproc.boundingRect(approxContour);
Core.rectangle(bgr, new Point(cord.x, cord.y), new Point(cord.x+cord.width, cord.y+cord.height),new Scalar(0,255,0), 1);
ROI = bgrClone.submat(cord.y, cord.y+cord.height, cord.x, cord.x+cord.width);
showResult(ROI);
}
}
I am using Java but any in any language would be appreciated.

You can perform shape detection using contours with OpenCV. I implemented the following in python.
Steps:
Since you are dealing with traffic signs, most of them would be red in color. In the RGB color space the amount of red is the highest in the red channel. But since white color also comprises of red I did not use it. I rather used the blue channel where red content is minimal.
Blue channel image:
Next I applied Otsu threshold to the image above and inverted it.
Otsu threshold and Inversion:
Finally I found contours and marked those contours having roughly 8 arcs around it.
Final Result:
Code:
import numpy as np
import cv2
import os
path = r'C:\Desktop\Stack\contour\shape_detection'
filename = 'stop.png'
img = cv2.imread(os.path.join(path, filename), 1)
cv2.imshow('img1',img[:,:,0])
ret,thresh1 = cv2.threshold(img[:,:,0], 0, 255,cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
cv2.imshow('thresh1', thresh1)
_, contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for cnt in contours:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
print len(approx)
if len(approx)==8:
print "octagon"
cv2.drawContours(img, [cnt], 0, (0, 255, 0), 6)
cv2.imshow('sign', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Find Similarity Of Two Hand Sign Images OpenCv

I am Working on Sign Translator Application For Handy People. In My application User Will Give one sign image from camera or gallery given image will be compare with database images and show the result With Alphabetic Sign.
but my problem is i am not getting good similarity between two images Some Time result is accurate some time not.
Please Refer me Some Idea Or source Code.
Thanks in advance.
Scalar lowerThreshold = new Scalar(0, 48, 80); // Blue color – lower hsv values
Scalar upperThreshold = new Scalar(20, 255, 255); // Blue color – higher hsv values
FeatureDetector detector = FeatureDetector.create(FeatureDetector.PYRAMID_FAST);
DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
//orb orb bruteforce with filter method
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
//crash on surf flanbased
Mat img1 = new Mat();
Mat img2 = new Mat();
Utils.bitmapToMat(defaultImage,img1);
Utils.bitmapToMat(databaseImage,img2);
Mat descriptors1 = new Mat();
MatOfKeyPoint keypoints1 = new MatOfKeyPoint();
detector.detect(img1, keypoints1);
extractor.compute(img1, keypoints1, descriptors1);
//second image
Mat descriptors2 = new Mat();
MatOfKeyPoint keypoints2 = new MatOfKeyPoint();
detector.detect(img2, keypoints2);
extractor.compute(img2, keypoints2, descriptors2);
//matcher image descriptors
MatOfDMatch matches = new MatOfDMatch();
matcher.match(descriptors1,descriptors2,matches);
//Filter matches by distance
MatOfDMatch filtered = filterMatchesByDistance(matches);
int total = (int) matches.size().height;
int Match= (int) filtered.size().height;
Log.d("LOG", "total:" + total + " Match:"+Match);
int percent = (int)((Match * 100.0f) / total);
if(percent>max){
max=percent;
maximumPercentage.setMaximum(percent);
maximumPercentage.setImageId(id);
imageId=id;
Log.d("Maximum Percentage: ",String.valueOf(max)+"%");
Log.d("MaxId: ",String.valueOf(imageId));
}
id++;
Log.d("matchingOImages: ",String.valueOf(percent)+"%");
filter matching result method
List<DMatch> matches_original = matches.toList();
List<DMatch> matches_filtered = new ArrayList<DMatch>();
int DIST_LIMIT = 30;
// Check all the matches distance and if it passes add to list of filtered matches
Log.d("DISTFILTER", "ORG SIZE:" + matches_original.size() + "");
for (int i = 0; i < matches_original.size(); i++) {
DMatch d = matches_original.get(i);
if (Math.abs(d.distance) <= DIST_LIMIT) {
matches_filtered.add(d);
}
}
Log.d("DISTFILTER", "FIL SIZE:" + matches_filtered.size() + "");
MatOfDMatch mat = new MatOfDMatch();
mat.fromList(matches_filtered);
return mat;

Ok well i think you just entered the modern age of neural networks.
As it can be overwhelming how this stuff works, and often takes years of study, there are some shortcuts to get things done.
For the quickest result I think you might start here:
( Assuming you rather dont want to dive that deep into the innerworkings of a neural net, but rather would use existing software, or services ) https://cloud.google.com/automl/

OpenCV - Java - How to remove some pixels around a cluster

I am doing a project where I need to identify certain areas of the image. After processing the image and removing all the unnecessary things I finally get the area which I need as shown in the image (area inside the green circle).
I am unable to draw a circle around that area using OpenCV. I am currently using the Java version of OpenCV. If someone can point me to the right direction on how to implement that green circle over the image, it will be very helpful.
Things I have tried to detect that area.
blob detector - Did not achieve much.
Cluster - Same as blob detector.
HoughCircles - Draws unnecessary circles in the image.
FindContour - Did not draw anything since it is not a perfect circle, ellipse or any other well known polygon.
I appreciate your help.

Here is a solution:
Opening in order to clean the image from all the thin/elongate patterns.
Connected component labeling in order to count the remaining patterns
Size counting of each remaining pattern
The biggest pattern is the one you want to circle.
Note: is you want to perfectly preserve the pattern, you can replace the opening by an opening by reconstruction (erosion + geodesic reconstruction).

I finally found a solution for my problem. I used the feature detector from the OpenCV library and gave the right threshold to the detector. This did the trick for me. The code in Java looks like below.
public static void main(String[] args){
try{
//Validation whether a file name is passed to the function
if(args.length == 0){
System.out.println("here...");
log.error("No file was passed to the function");
throw new IOException();
}
//Read the image from the input
Mat inputMat = Highgui.imread(args[0],Highgui.CV_LOAD_IMAGE_GRAYSCALE);
//Create a feature detector. In this case we are using SURF (Speeded-Up Robust Features) detector.
MatOfKeyPoint objectKeyPoints = new MatOfKeyPoint();
FeatureDetector featureDetector = FeatureDetector.create(FeatureDetector.SURF);
//A temporary file is created to input Hessian Threshold to the SURF detector
File tempFile = File.createTempFile("config", ".yml");
String settings = "%YAML:1.0\nhessianThreshold: 7000.\noctaves: 3\noctaveLayers: 4\nupright: 0\n";
FileWriter writer = new FileWriter(tempFile, false);
writer.write(settings);
writer.close();
//Read the configuration from the temporary file to assign the threshold for the detector
featureDetector.read(tempFile.getPath());
//Detect the features in the image provided
featureDetector.detect(inputMat, objectKeyPoints);
//Iterate through the list of key points detected in the previous step and find the Key Point with the largest size
List<KeyPoint> objectKeyPointList = objectKeyPoints.toList();
KeyPoint impKeyPoint = new KeyPoint();
for(int i=0; i<objectKeyPointList.size(); i++){
if(impKeyPoint == null){
impKeyPoint = objectKeyPointList.get(i);
}
else if(impKeyPoint.size < objectKeyPointList.get(i).size){
impKeyPoint = objectKeyPointList.get(i);
}
}
//If the size of the Key Point is greater than 120 then reduce the size to 120 and if the size is less than 120 then increase to 120
if(impKeyPoint.size > 120){
KeyPoint tempKeyPoint = new KeyPoint();
tempKeyPoint = impKeyPoint;
tempKeyPoint.size = 120;
impKeyPoint = tempKeyPoint;
}
else if(impKeyPoint.size < 120){
KeyPoint tempKeyPoint = new KeyPoint();
tempKeyPoint = impKeyPoint;
tempKeyPoint.size = 120;
impKeyPoint = tempKeyPoint;
}
//Convert the Key Point to MatOfKeyPoint since drawKeyPoints accepts only MatOfKeyPoint
MatOfKeyPoint impMatOfKeyPoint = new MatOfKeyPoint(impKeyPoint);
//Mat for drawing the circle in the image
Mat outputImage = new Mat(inputMat.rows(), inputMat.cols(), Highgui.CV_LOAD_IMAGE_COLOR);
//Green color for the circle
Scalar greenCircle = new Scalar(0, 255, 0);
//Draw the circle around the optic nerve when detected
Features2d.drawKeypoints(inputMat, impMatOfKeyPoint, outputImage, greenCircle, Features2d.DRAW_RICH_KEYPOINTS);
//Write the image to a file
Highgui.imwrite("surf_keypoints.png", outputImage);
}catch(Exception e){
log.fatal(e.getMessage());
}
}
Hope this is helpful for others.

How can I draw rectangle with MatOfKeyPoint for Text Detection | Java

I am working on a real time text detection and recognition with OpenCV4Android. Recognition part is totally completed. However, I have to ask question about text detection. I' m using the MSER FeatureDetector for detection text.
This is the real time and calling the method part:
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
carrierMat = inputFrame.gray();
carrierMat = General.MSER(carrierMat);
return carrierMat;
}
And this is the basic MSER implementation:
private static FeatureDetector fd = FeatureDetector.create(FeatureDetector.MSER);
private static MatOfKeyPoint mokp = new MatOfKeyPoint();
private static Mat edges = new Mat();
public static Mat MSER(Mat mat) {
//for mask
Imgproc.Canny(mat, edges, 400, 450);
fd.detect(mat, mokp, edges);
//for drawing keypoints
Features2d.drawKeypoints(mat, mokp, mat);
return mat;
}
It works fine for finding text with edges mask.
I would like to draw a rectangles for clusters like this:
or this:
You can assume that I have the right points.
As you can see, fd.detect() method is returning a MatOfKeyPoint. Hence I' ve tried this method for drawing rectangle:
public static Mat MSER_(Mat mat) {
fd.detect(mat, mokp);
KeyPoint[] refKp = mokp.toArray();
Point[] refPts = new Point[refKp.length];
for (int i = 0; i < refKp.length; i++) {
refPts[i] = refKp[i].pt;
}
MatOfPoint2f refMatPt = new MatOfPoint2f(refPts);
MatOfPoint2f approxCurve = new MatOfPoint2f();
//Processing on mMOP2f1 which is in type MatOfPoint2f
double approxDistance = Imgproc.arcLength(refMatPt, true) * 0.02;
Imgproc.approxPolyDP(refMatPt, approxCurve, approxDistance, true);
//Convert back to MatOfPoint
MatOfPoint points = new MatOfPoint(approxCurve.toArray());
// Get bounding rect
Rect rect = Imgproc.boundingRect(points);
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(mat, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), Detect_Color_, 5);
//Features2d.drawKeypoints(mat, mokp, mat);
return mat;
}
But when I was trying to Imgproc.arcLength() method, it suddenly stopped. I gave a random approxDistance value for Imgproc.approxPolyDP() method like 0.1, it doesn' t work really efficiently.
So how can I draw rectangle for detected text?

I tested your code and had exactly the same problem.
For now I still can't find the problem within.
But I found a project using both "MSER" and "Morphological".
you can find it here .
The project have very simple structure and the author put the
text detection in "onCameraFrame" method just like you.
I implemented the method from that project and it worked,
but the result was still not very good.
If you seek better text detection tool, here's two of them.
Stroke Width Transform(SWT):
A whole new method for finding text area. It's fast and efficient. however it is only available in c++ or python. you can find some example here.
Class-specific Extremal Regions using class ERFilter:An advanced version of the MSER. Unfortunately, it is only available in OpenCV 3.0.0-dev. You can't use it in current version of OpenCV4Android. The document is here.
To be honest I am new in this area(2 months), but I hope these information can help you finish your project.
(update:2015/9/13)
I've translated a c++ method from a post.
It works far better than the first github project I mentioned.
Here is the code:
public void apply(Mat src, Mat dst) {
if (dst != src) {
src.copyTo(dst);
}
Mat img_gray,img_sobel, img_threshold, element;
img_gray=new Mat();
Imgproc.cvtColor(src, img_gray, Imgproc.COLOR_RGB2GRAY);
img_sobel=new Mat();
Imgproc.Sobel(img_gray, img_sobel, CvType.CV_8U, 1, 0, 3, 1, 0,Core.BORDER_DEFAULT);
img_threshold=new Mat();
Imgproc.threshold(img_sobel, img_threshold, 0, 255, Imgproc.THRESH_OTSU+Imgproc.THRESH_BINARY);
element=new Mat();
element = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(17, 3) );
Imgproc.morphologyEx(img_threshold, img_threshold, Imgproc.MORPH_CLOSE, element);
//Does the trick
List<MatOfPoint> contours=new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Imgproc.findContours(img_threshold, contours, hierarchy, 0, 1);
List<MatOfPoint> contours_poly=new ArrayList<MatOfPoint>(contours.size());
contours_poly.addAll(contours);
MatOfPoint2f mMOP2f1,mMOP2f2;
mMOP2f1=new MatOfPoint2f();
mMOP2f2=new MatOfPoint2f();
for( int i = 0; i < contours.size(); i++ )
if (contours.get(i).toList().size()>100)
{
contours.get(i).convertTo(mMOP2f1, CvType.CV_32FC2);
Imgproc.approxPolyDP(mMOP2f1,mMOP2f2, 3, true );
mMOP2f2.convertTo(contours_poly.get(i), CvType.CV_32S);
Rect appRect=Imgproc.boundingRect(contours_poly.get(i));
if (appRect.width>appRect.height)
{
Imgproc.rectangle(dst, new Point(appRect.x,appRect.y) ,new Point(appRect.x+appRect.width,appRect.y+appRect.height), new Scalar(255,0,0));
}
}
}

Quick & fast template matching on screen. Coordinates needed too. Java

I need a way to find an image on the screen. I've searched for ways to do this on SO but some take extremely long. I need it to be fast and efficient, does not need to be accurate. Basically i'm planning to compare or search for a small pixelated image, say 11x10 pixels for example, on the screen.
I also need a way to know the x and y coordinates of the small image on the screen.
Although I've looked through many tools out there like JavaCV and OpenCV, I just wanted to see if there are any other ways to do this.
TL;DR
I need a fast way to search for a small (11x10 example.) image on the screen and know its x,y coordinates.

I think you many find this answer relevant! But it is for Windows & in c++. But i'm sure that you can convert it very easily to any language.

This question is very old, But im trying to acheive the exact same thing here. Ive found that combining these answers would do the trick:
Convert BufferedImage TYPE_INT_RGB to OpenCV Mat Object
OpenCV Template Matching example in Android
The reason you need to do a conversion is because when u grab a screenshot with awt.Robot class its in the INT_RGB format. The matching template example expects bytes and you cannot grab byte data from this type of image directly.
Heres my implementation of these two answers, but it is incomplete. The output is all screwed up and i think it may have something to do with the IntBuffer/ByteBuffers.
-Edit-
I've added a new helper method that converts a INT_RGB to a BYTE_BGR. I can now grab the coordinates of template on the image using matchLoc.This seems to work pretty well, I was able to use this with a robot that clicks the start menu for me based on the template.
private BufferedImage FindTemplate() {
System.out.println("\nRunning Template Matching");
int match_method = Imgproc.TM_SQDIFF;
BufferedImage screenShot = null;
try {
Robot rob = new Robot();
screenShot = rob.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
} catch (AWTException ex) {
Logger.getLogger(MainGUI.class.getName()).log(Level.SEVERE, null, ex);
}
if(screenShot == null) return;
Mat img = BufferedImageToMat(convertIntRGBTo3ByteBGR(screenShot));
String templateFile = "C:\\Temp\\template1.JPG";
Mat templ = Highgui.imread(templateFile);
// / Create the result matrix
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
// / Do the Matching and Normalize
Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
Highgui.imwrite("out2.png", result);
// / Localizing the best match with minMaxLoc
MinMaxLocResult mmr = Core.minMaxLoc(result);
Point matchLoc;
if (match_method == Imgproc.TM_SQDIFF
|| match_method == Imgproc.TM_SQDIFF_NORMED) {
matchLoc = mmr.minLoc;
} else {
matchLoc = mmr.maxLoc;
}
Graphics2D graphics = screenShot.createGraphics();
graphics.setColor(Color.red);
graphics.setStroke(new BasicStroke(3));
graphics.drawRect(matchLoc.x, matchLoc.y, templ.width(), templ.height());
graphics.dispose();
return screenShot;
}
private Mat BufferedImageToMat(BufferedImage img){
int[] data = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
Mat mat = new Mat(img.getHeight(), img.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, byteBuffer.array());
return mat;
}`
private BufferedImage convertIntRGBTo3ByteBGR(BufferedImage img){
BufferedImage convertedImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = convertedImage.createGraphics();
graphics.drawImage(img, 0, 0, null);
graphics.dispose();
return convertedImage;
}
Results:
Template: