I am doing a project where I need to identify certain areas of the image. After processing the image and removing all the unnecessary things I finally get the area which I need as shown in the image (area inside the green circle).
I am unable to draw a circle around that area using OpenCV. I am currently using the Java version of OpenCV. If someone can point me to the right direction on how to implement that green circle over the image, it will be very helpful.
Things I have tried to detect that area.
blob detector - Did not achieve much.
Cluster - Same as blob detector.
HoughCircles - Draws unnecessary circles in the image.
FindContour - Did not draw anything since it is not a perfect circle, ellipse or any other well known polygon.
I appreciate your help.
Here is a solution:
Opening in order to clean the image from all the thin/elongate patterns.
Connected component labeling in order to count the remaining patterns
Size counting of each remaining pattern
The biggest pattern is the one you want to circle.
Note: is you want to perfectly preserve the pattern, you can replace the opening by an opening by reconstruction (erosion + geodesic reconstruction).
I finally found a solution for my problem. I used the feature detector from the OpenCV library and gave the right threshold to the detector. This did the trick for me. The code in Java looks like below.
public static void main(String[] args){
try{
//Validation whether a file name is passed to the function
if(args.length == 0){
System.out.println("here...");
log.error("No file was passed to the function");
throw new IOException();
}
//Read the image from the input
Mat inputMat = Highgui.imread(args[0],Highgui.CV_LOAD_IMAGE_GRAYSCALE);
//Create a feature detector. In this case we are using SURF (Speeded-Up Robust Features) detector.
MatOfKeyPoint objectKeyPoints = new MatOfKeyPoint();
FeatureDetector featureDetector = FeatureDetector.create(FeatureDetector.SURF);
//A temporary file is created to input Hessian Threshold to the SURF detector
File tempFile = File.createTempFile("config", ".yml");
String settings = "%YAML:1.0\nhessianThreshold: 7000.\noctaves: 3\noctaveLayers: 4\nupright: 0\n";
FileWriter writer = new FileWriter(tempFile, false);
writer.write(settings);
writer.close();
//Read the configuration from the temporary file to assign the threshold for the detector
featureDetector.read(tempFile.getPath());
//Detect the features in the image provided
featureDetector.detect(inputMat, objectKeyPoints);
//Iterate through the list of key points detected in the previous step and find the Key Point with the largest size
List<KeyPoint> objectKeyPointList = objectKeyPoints.toList();
KeyPoint impKeyPoint = new KeyPoint();
for(int i=0; i<objectKeyPointList.size(); i++){
if(impKeyPoint == null){
impKeyPoint = objectKeyPointList.get(i);
}
else if(impKeyPoint.size < objectKeyPointList.get(i).size){
impKeyPoint = objectKeyPointList.get(i);
}
}
//If the size of the Key Point is greater than 120 then reduce the size to 120 and if the size is less than 120 then increase to 120
if(impKeyPoint.size > 120){
KeyPoint tempKeyPoint = new KeyPoint();
tempKeyPoint = impKeyPoint;
tempKeyPoint.size = 120;
impKeyPoint = tempKeyPoint;
}
else if(impKeyPoint.size < 120){
KeyPoint tempKeyPoint = new KeyPoint();
tempKeyPoint = impKeyPoint;
tempKeyPoint.size = 120;
impKeyPoint = tempKeyPoint;
}
//Convert the Key Point to MatOfKeyPoint since drawKeyPoints accepts only MatOfKeyPoint
MatOfKeyPoint impMatOfKeyPoint = new MatOfKeyPoint(impKeyPoint);
//Mat for drawing the circle in the image
Mat outputImage = new Mat(inputMat.rows(), inputMat.cols(), Highgui.CV_LOAD_IMAGE_COLOR);
//Green color for the circle
Scalar greenCircle = new Scalar(0, 255, 0);
//Draw the circle around the optic nerve when detected
Features2d.drawKeypoints(inputMat, impMatOfKeyPoint, outputImage, greenCircle, Features2d.DRAW_RICH_KEYPOINTS);
//Write the image to a file
Highgui.imwrite("surf_keypoints.png", outputImage);
}catch(Exception e){
log.fatal(e.getMessage());
}
}
Hope this is helpful for others.
Related
I am stack with this problem for a couple of days. I want to make an android app that takes a picture and extracts HOG features of that image for future processing. The problem is that the code below always returns the HOG descriptors with rezo values.
#Override
public void onPictureTaken(byte[] data, Camera camera) {
Log.i(TAG, "Saving a bitmap to file");
// The camera preview was automatically stopped. Start it again.
mCamera.startPreview();
mCamera.setPreviewCallback(this);
this.disableView();
Bitmap bitmapPicture = BitmapFactory.decodeByteArray(data, 0, data.length);
myImage = new Mat(bitmapPicture.getWidth(), bitmapPicture.getHeight(), CvType.CV_8UC1);
Utils.bitmapToMat(bitmapPicture, myImage);
Bitmap bm = Bitmap.createBitmap(myImage.cols(), myImage.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(myImage.clone(), bm);
// find the imageview and draw it!
ImageView iv = (ImageView) getRootView().findViewById(R.id.imageView);
this.setVisibility(SurfaceView.GONE);
iv.setVisibility(ImageView.VISIBLE);
Mat forHOGim = new Mat();
org.opencv.core.Size sz = new org.opencv.core.Size(64,128);
Imgproc.resize( myImage, myImage, sz );
Imgproc.cvtColor(myImage,forHOGim,Imgproc.COLOR_RGB2GRAY);
//forHOGim = myImage.clone();
MatOfFloat descriptors = new MatOfFloat(); //an empty vector of descriptors
org.opencv.core.Size winStride = new org.opencv.core.Size(64/2,128/2); //50% overlap in the sliding window
org.opencv.core.Size padding = new org.opencv.core.Size(0,0); //no padding around the image
MatOfPoint locations = new MatOfPoint(); ////an empty vector of locations, so perform full search
//HOGDescriptor hog = new HOGDescriptor();
HOGDescriptor hog = new HOGDescriptor(sz,new org.opencv.core.Size(16,16),new org.opencv.core.Size(8,8),new org.opencv.core.Size(8,8),9);
Log.i(TAG,"Constructed");
hog.compute(forHOGim , descriptors, new org.opencv.core.Size(16,16), padding, locations);
Log.i(TAG,"Computed");
Log.i(TAG,String.valueOf(hog.getDescriptorSize())+" "+descriptors.size());
Log.i(TAG,String.valueOf(descriptors.get(12,0)[0]));
double dd=0.0;
for (int i=0;i<3780;i++){
if (descriptors.get(i,0)[0]!=dd) Log.i(TAG,"NOT ZERO");
}
Bitmap bm2 = Bitmap.createBitmap(forHOGim.cols(), forHOGim.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(forHOGim,bm2);
iv.setImageBitmap(bm2);
}
So in the logcat I never get the NOT ZERO message. The problem is that whatever changes I do to this code I always have zeros in the descriptors MatOfFloat... And the strange part is, if I uncomment the HOGDescriptor hog = new HOGDescriptor(); and use that one instead of the one I am using now, my application crashes...
The rest of the code runs fine, the picture is always taken and displayed on my image view as I expect.
Any help will be appreciated.
Thanks in advance.
The problem was inside the library. When I executed the same code with OpenCV 2.4.13 for Linux and not for Android, the code worked great as expected. So I hope they will fix any problems with the HOGDescriptor for OpenCV4Android.
I have an application to capture video of the screen and save to a file. I give the user the ability to pick between 480, 720, and "Full Screen" video sizes. A 480 will record in a small box on the screen, 720 will record in a larger box, and of course, "Full Screen" will record in an even larger box. However, this full screen box is NOT the actual screen resolution. It is the app window size, which happens to be around 1700x800. The Video Tool works perfectly for the 480 and 720 options, and will also work if "Full Screen" is overwridden to be the entire screen of 1920x1080.
My question: Are only certain sizes allowed? Does it have to fit a certain aspect ratio, or be an "acceptable" resolution? My code, below, is modified from the xuggle CaptureScreenToFile.java file (the location of the problem is noted by comments):
public void run() {
try {
String parent = "Videos";
String outFile = parent + "example" + ".mp4";
file = new File(outFile);
// This is the robot for taking a snapshot of the screen. It's part of Java AWT
final Robot robot = new Robot();
final Rectangle customResolution = where; //defined resolution (custom record size - in this case, 1696x813)
final Toolkit toolkit = Toolkit.getDefaultToolkit();
final Rectangle fullResolution = new Rectangle(toolkit.getScreenSize()); //full resolution (1920x1080)
// First, let's make a IMediaWriter to write the file.
final IMediaWriter writer = ToolFactory.makeWriter(outFile);
writer.setForceInterleave(false);
// We tell it we're going to add one video stream, with id 0,
// at position 0, and that it will have a fixed frame rate of
// FRAME_RATE.
writer.addVideoStream(0, 0, FRAME_RATE, customResolution.width, customResolution.height); //if I use fullResolution, it works just fine - but captures more of the screen than I want.
// Now, we're going to loop
long startTime = System.nanoTime();
while (recording) {
// take the screen shot
BufferedImage screen = robot.createScreenCapture(fullResolution); //tried capturing using customResolution, but did not work. Instead, this captures full screen, then tries to trim it below (also does not work).
// convert to the right image type
BufferedImage bgrScreen = convertToType(screen, BufferedImage.TYPE_3BYTE_BGR); //Do I need to convert after trimming?
BufferedImage trimmedScreen = bgrScreen.getSubimage((int)customResolution.getX(), (int)customResolution.getY(), (int)customResolution.getWidth(), (int)customResolution.getHeight());
// encode the image
try{
//~~~~Problem is this line of code!~~~~ Error noted below.
writer.encodeVideo(0, trimmedScreen, System.nanoTime() - startTime, TimeUnit.NANOSECONDS); //tried using trimmedScreen and bgrScreen
} catch (Exception e) {
e.printStackTrace();
}
// sleep for framerate milliseconds
Thread.sleep((long) (1000 / FRAME_RATE.getDouble()));
}
// Finally we tell the writer to close and write the trailer if
// needed
writer.close();
} catch (Throwable e) {
System.err.println("an error occurred: " + e.getMessage());
}
}
public static BufferedImage convertToType(BufferedImage sourceImage, int targetType) {
BufferedImage image;
// if the source image is already the target type, return the source image
if (sourceImage.getType() == targetType)
image = sourceImage;
// otherwise create a new image of the target type and draw the new image
else {
image = new BufferedImage(sourceImage.getWidth(), sourceImage.getHeight(), targetType);
image.getGraphics().drawImage(sourceImage, 0, 0, null);
}
return image;
}
Error:
java.lang.RuntimeException: could not open stream com.xuggle.xuggler.IStream#2834912[index:0;id:0;streamcoder:com.xuggle.xuggler.IStreamCoder#2992432[codec=com.xuggle.xuggler.ICodec#2930320[type=CODEC_TYPE_VIDEO;id=CODEC_ID_H264;name=libx264;];time base=1/50;frame rate=0/0;pixel type=YUV420P;width=1696;height=813;];framerate:0/0;timebase:1/90000;direction:OUTBOUND;]: Operation not permitted
Note: The file is successfully created, but has size of zero, and cannot be opened by Windows Media Player, with the following error text:
Windows Media Player cannot play the file. The Player might not support the file type or might not support the codec that was used to compress the file.
Sorry for the wordy question. I'm interested in learning WHAT and WHY, not just a solution. So if anyone can explain why it isn't working, or point me towards material to help, I'd appreciate it. Thanks!
Try to have the dimension even numbers 1696x812
I recently started working with the Java bindings for OpenCV to make a quick and dirty project to do template matching. Basically I am trying to read a set of jpg images (saved in MS Paint) into Mats and then use template matching to find their locations from a screen shot taken with Java.Robot.
When it comes time to do the template matching this error is thrown
OpenCV Error: Assertion failed ((depth == CV_8U || depth == CV_32F)
&& type == _templ.type() && _img.dims() <= 2) in cv::matchTemplate
After searching it looks like the issue is that the two Mats I am trying to use do not have the same "type". What I am not sure of is what this refers to. I assume it is the Mats CvType, if I print out the CvType of the image and template I get a type() of 4 == CvType.CV_32SC1 for my template I get a type() of 20 == CvType.CV_32SC3.
But I feel like this is not the correct type() I am trying to compare, I have feeling it refers to the data type of how the data is stored in the Mat? But I have no good links to back this up just remembrances from many SO searches.
Here is my code for loading in my jpg images into a Mat
Mat pic_ = Imgcodecs.imread("MyPath\\image.jpg");
pic_.convertTo(pic_, CvType.CV_32SC1);
Here the second line turns my type() from 20 to 16, though as per my last comment I don't think this is the proper way to alter the Mat to match the image?... Because convertTo'ing this Mat to match the type of the screen shot `(below) does not fix the error?
Here is how I am creating the image Mat
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
So I first take a screenshot with Java.Robot.createScreenCapture I then convert it to a Mat with
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_INT_RGB);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_32SC1);
int[] data = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
From what I could tell the BufferedImage created by Robot is of type BufferedImage.TYPE_3BYTE_BGR which gives me an error "DataBufferInt cannot be cast to DataBufferByte" when trying to get the pixel data. So per the linked question I redraw the BufferedImage as type BufferedImage.TYPE_INT_RGB and pull out the data as a DataBufferInt.
So in all, should I be trying to match the Mat.type() or does my problem lie elsewhere? If not elsewhere how can I alter either of the Mats so that they can be used with Imgproc.matchTemplate properly?
I feel like the easiest solution would be to convert the image loaded from file to match the screenshot Mat?
EDIT: The exact section of code that gives the error is below
// Mat imageTemplate is a function argument; the loaded jpg image
// Take a picture of the screen
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
// Create the result matrix
int result_cols = screenImage.cols() - imageTemplate.cols() + 1;
int result_rows = screenImage.rows() - imageTemplate.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32SC1);
newStatus("ScreenType: " + screenImage.type());
newStatus("TemplaType: " + imageTemplate.type());
// Choose a matching method
int matchMethod = Imgproc.TM_SQDIFF_NORMED;
// Do the Matching and Normalize
Imgproc.matchTemplate(screenImage, imageTemplate, result, matchMethod);
// Error occurs on previous line
As #Miki pointed out in the comments the answer was getting the channe type to match for the image and template. I ended up changing my bufferedImageToMat function.
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
My templates are read in as CvType.CV_8UC3, so it was just a matter of creating a Mat from the screen image with this type!
I need a way to find an image on the screen. I've searched for ways to do this on SO but some take extremely long. I need it to be fast and efficient, does not need to be accurate. Basically i'm planning to compare or search for a small pixelated image, say 11x10 pixels for example, on the screen.
I also need a way to know the x and y coordinates of the small image on the screen.
Although I've looked through many tools out there like JavaCV and OpenCV, I just wanted to see if there are any other ways to do this.
TL;DR
I need a fast way to search for a small (11x10 example.) image on the screen and know its x,y coordinates.
I think you many find this answer relevant! But it is for Windows & in c++. But i'm sure that you can convert it very easily to any language.
This question is very old, But im trying to acheive the exact same thing here. Ive found that combining these answers would do the trick:
Convert BufferedImage TYPE_INT_RGB to OpenCV Mat Object
OpenCV Template Matching example in Android
The reason you need to do a conversion is because when u grab a screenshot with awt.Robot class its in the INT_RGB format. The matching template example expects bytes and you cannot grab byte data from this type of image directly.
Heres my implementation of these two answers, but it is incomplete. The output is all screwed up and i think it may have something to do with the IntBuffer/ByteBuffers.
-Edit-
I've added a new helper method that converts a INT_RGB to a BYTE_BGR. I can now grab the coordinates of template on the image using matchLoc.This seems to work pretty well, I was able to use this with a robot that clicks the start menu for me based on the template.
private BufferedImage FindTemplate() {
System.out.println("\nRunning Template Matching");
int match_method = Imgproc.TM_SQDIFF;
BufferedImage screenShot = null;
try {
Robot rob = new Robot();
screenShot = rob.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
} catch (AWTException ex) {
Logger.getLogger(MainGUI.class.getName()).log(Level.SEVERE, null, ex);
}
if(screenShot == null) return;
Mat img = BufferedImageToMat(convertIntRGBTo3ByteBGR(screenShot));
String templateFile = "C:\\Temp\\template1.JPG";
Mat templ = Highgui.imread(templateFile);
// / Create the result matrix
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
// / Do the Matching and Normalize
Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
Highgui.imwrite("out2.png", result);
// / Localizing the best match with minMaxLoc
MinMaxLocResult mmr = Core.minMaxLoc(result);
Point matchLoc;
if (match_method == Imgproc.TM_SQDIFF
|| match_method == Imgproc.TM_SQDIFF_NORMED) {
matchLoc = mmr.minLoc;
} else {
matchLoc = mmr.maxLoc;
}
Graphics2D graphics = screenShot.createGraphics();
graphics.setColor(Color.red);
graphics.setStroke(new BasicStroke(3));
graphics.drawRect(matchLoc.x, matchLoc.y, templ.width(), templ.height());
graphics.dispose();
return screenShot;
}
private Mat BufferedImageToMat(BufferedImage img){
int[] data = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
Mat mat = new Mat(img.getHeight(), img.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, byteBuffer.array());
return mat;
}`
private BufferedImage convertIntRGBTo3ByteBGR(BufferedImage img){
BufferedImage convertedImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = convertedImage.createGraphics();
graphics.drawImage(img, 0, 0, null);
graphics.dispose();
return convertedImage;
}
Results:
Template:
I am trying to read 2D Data matrix barcode using zxing library(GenericMultipleBarcodeReader). I have multiple barcodes on a single image.
The problem is that the efficiency of the zing reader is very low, it
recognizes 1 barcode from image 1.png and no barcode from image 2.png which has 48 barcodes. Is there
any way to get 100% efficiency or any other library which results 100%
My code to read barcode is:
public static void main(String[] args) throws Exception {
BufferedImage image = ImageIO.read(new File("1.png"));
if (image != null) {
LuminanceSource source = new BufferedImageLuminanceSource(image);
BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source));
DataMatrixReader dataMatrixReader = new DataMatrixReader();
Hashtable<DecodeHintType, Object> hints = new Hashtable<DecodeHintType, Object>();
hints.put(DecodeHintType.TRY_HARDER, Boolean.TRUE);
GenericMultipleBarcodeReader reader = new GenericMultipleBarcodeReader(
dataMatrixReader);
Result[] results = reader.decodeMultiple(bitmap, hints);
for (Result result : results) {
System.out.println(result.toString());
}
}
}
And images I used are:
Please help to resolve this issue.
Thanks
It doesn't quite work this way. It will not read barcodes in a grid, as it makes an assumption that it can cut up the image in a certain way that won't be compatible with grids. You will have to write your own method to cut up the image into scannable regions.
It is also the case that the Data Matrix decoder assumes the center of the image is inside the barcode. This is another reason you need to pre-chop the image into squares around the cylinders and then scan. It ought to work fairly well then.
An alternative solution is to consider a barcode engine that can detect multiple barcodes in various orientations on one document. If you're running on Windows, the ClearImage Barcode SDK has a Java API and should be able to handle your needs without pre-processing. You can test if their engine can read your image using their Online Barcode Reader.
Some sample code:
public static void testDataMatrix () {
try {
String filename = "1.png ";
CiServer objCi = new CiServer();
Ci = objCi.getICiServer();
ICiDataMatrix reader = Ci.CreateDataMatrix(); // read DataMatrix Barcode
reader.getImage().Open(filename, 1);
int n = reader.Find(0); // find all the barcodes in the doc
for (i = 1; i <= n; i++) {
ICiBarcode Bc = reader.getBarcodes().getItem(i); // getItem is 1-based
System.out.println("Barcode " + i + " has Text: " + Bc.getText());
}
} catch (Exception ex) {System.out.println(ex.getMessage());}
}
Disclaimer: I've done some work for Inlite in the past.