I am working in face detection using the YCbCr color space. When I apply it on a human face, there are gaps that denote the nose, eye and mouth and the resultant patch looks like (a). In order to remove these gaps, I apply a morphological dilation operation and I get the resulting image shown in (b) but my requirement is to get patch like it is shown in (c). This means that I want to remove the outer contours from the processed patch.
Can anyone please suggest that how can I remove these outer contours?
I have a few suggestions for you, though it's hard to verify this without the actual raw images themselves. Try one of these and see if you get something meaningful.
Method #1 - Use imfill followed by imopen
One suggestion I have is to use imfill to fill in any of the holes in the image, followed by a call to imopen to perform morphological opening (i.e. erosion followed by dilation as alluded to by user Paul R). Opening (via imopen) removes any small isolated regions in the image subject to the desired structuring element.
Assuming your image is stored in the variable BW, something like this may work:
BW2 = imfill(BW, 'holes');
se = strel('square', 5);
BW2 = imopen(BW2, se);
BW2 is the final image.
Method #2 - Use bwareaopen followed by imdilate
I can also suggest using the function bwareaopen which removes objects whose areas fall under a certain amount. Try something small like an area of 80 pixels to remove those isolated areas, then use the dilation (imdilate) command that you alluded to in your post:
BW2 = bwareaopen(BW, 80);
%// Place your code for dilation here using BW2
Method #3 - Open your image with imopen then perform imdilate
One final thing I can suggest is to open your image first to remove the spurious small pixel areas, then perform your dilation code as you suggested:
se = strel('square', 5);
BW2 = imopen(BW, se);
%// Place your code for dilation here using BW2
You should do the following steps:
Fill holes => result. It fills all the holes into the face.
Opening (erosion + dilation) => result. It erases all the small patterns outside the shape.
Even better: you replace the step 2 by an "opening by reconstruction" which is an erosion followed by a geodesic reconstruction. This operation does not modify the main pattern. See the result.
All these operations should be available in OpenCV or ImageJ.
Related
I'm developing a program with Java and Sikuli and I want to click on a red image with a specific shape that is located on screen.
The problem is that on the screen there is another image with that same shape but different color, blue.
import org.sikuli.script.Screen;
this.screen.type("C:\\Images\\TestImage.png", "a"); // this is what I'm using.
My mouse keeps moving between the two images because it can't tell the difference in color.
There is no way Sikuli can make the right choice for you. It can only locate a matching are based on your pattern (color in this case). To work around this issue you should provide some reference points that are unique and can be used to "help" Sikuli find the right match. For example, if the pattern you are interested in is located at the left side of the screen, then you can limit the search to the left side of the screen only. Or if you have a unique visual object in the are of interested you can use it as a pivot and look only around it.
On top of that, if you have few similar items appear in some ordered fashion (one under another for example), you can let Sikuli find all of them, calculate their coordinates and select the object you need based on these coordinates.
Searching using color is possible with Brobot, a state-based, testable, automation framework for Java. Brobot wraps Sikuli methods and uses Sikuli variables such as Match in its core functionality. Its color methods depend heavily on matrix operations with OpenCV. The Brobot documentation gives more detailed information on the framework. Additionally, there is a page in the documentation specifically for color searches.
Color Selection Methods
There are 3 methods used to identify which colors to search for.
Color averaging. The average color of all pixels is determined from all image files (a single Brobot image can contain multiple image files). The range of acceptable colors around the target color can be adjusted, as well as the required size of the color region.
K-Means. Images that contain a variety of colors are not suitable to color averaging. The k-Means method from OpenCV finds the salient colors in an image, which are then used in the search. Adjustable options include the number of k-Means, acceptable color range, and required size.
Histogram. Brobot divides images into 5 regions (4 corners + the middle region) in order to preserve some spatial color characteristics (for example, blue sky on the top of an image will not match blue sea on the bottom of an image). The search can be adjusted by color range, as well as the number of histogram bins for hue, saturation, and value.
Chained Find Operations
Each of these color methods can be used independently or combined with the traditional Sikuli pattern searches by chaining find operations together. There are two main flavors of chained find operations:
Nested finds. Matches are searched for inside the matches from the previous find operation. The last find operation is responsible for providing the function’s return value.
Confirmed finds. The matches from the first find operation are filtered by the results of following find operations. The later find operations act as confirmation or rejection of the initial matches. Confirmed matches from the first find operation are returned.
Documentation for nested finds and confirmed finds
Finding the Red Image
For your specific question, it would be best to use a chain of two find operations, set up as a confirmed find. The first operation would be a pattern search (equivalent to findAll in Sikuli) and the second operation, the confirmation operation, would be a color search.
Actions in Brobot are built with:
The action configuration (an ActionOptions variable)
The objects to act on (images, regions, locations, etc.)
Brobot encourages defining states with a collection of images that belong together. The code below assumes you have a state StateWithRedImage that contains the image RedImage. The results are returned in a Matches object, which contains information about the matches and the action performed.
public class RedImageFinder {
private final Action action;
private final StateWithRedImage stateWithRedImage;
public RedImageFinder(Action action, StateWithRedImage stateWithRedImage) {
this.action = action;
this.stateWithRedImage = stateWithRedImage;
}
public Matches find() {
ActionOptions actionOptions = new ActionOptions.Builder()
.setAction(ActionOptions.Action.FIND)
.setFind(ActionOptions.Find.ALL)
.addFind(ActionOptions.Find.COLOR)
.keepLargerMatches(true)
.build();
return action.perform(actionOptions, stateWithRedImage.getRedImage());
}
}
Disclaimer: I'm the developer of Brobot. It's free and open source.
Here's something that might help. Create a Region and try to find the image in that region as in example in the link
http://seleniumqg.blogspot.com/2017/06/findfailed-exception-in-sikuili-script.html?m=1
I have some lists of images. They always only contain growing objects. For example:
image01:
image 01+n:
image 01+n+n:
image 01+n+n+n:
What do i like to get? (This is the third picture, but modified)
If two (or more) objects merge i like to get the center point of the merge (red point) and also two lines from this point to the background.
I already tried today to solve this problem. I thought it is not so hard, but unfortunately it does not seem to be that easy.
I tried this:
1) For every picture make all objects black
2) Get all objects on every picture. Get the center for each object.
3) Check on every image if on one object is more than one center of a polygon of the previous image. If this is the case two objects merged.
4) If a merge was detected take the centers of the two merged object of the previous frame. Connect them with the A* algorithm on the current frame. Search now the pixel which was white on the previous frame. This is the "merge point".
5) Try now to get the two nearest white points and use them to draw the "break lines" (blue lines on the last image).
Unfortunately this seems not to be that easy and programming this is also hard. Probably an already implemented algorithm (java) exists which could do this job much easier? I implemented everything which I described (quick and dirty), but it is slow and buggy. Something doesn't work and it is hard to debug it. But I think there should be an easier solution.
So your problem is software development and algorithm development.
Because you say buggy and slow: for slow I dont care for now, it has to work; but the buggy is an issue (software development).
Part 3 is essentially correct: you take an object in the current frame and if it contains two objects in the previous frame its a merge.
Your connecting line of centers idea is not correct because the centers can be anywhere and the line will not pass through the connecting region.
Check the change that has happened in the current frame from the previous frame: that will be a hallo around both previous objects. Check the growth (part of this hallo) that is closest to both previous objects. That is take the growth of one object and check which part of the growth is close to the second object; do vice versa. This will give you the merging area.
It's a full project.
I want to do text detection in image using java. I am using OpenIMAJ to do that using MSER algorithm (as the first stage), but it takes a lot of processing time and in most images the exception OutOfMemoryError is returned.
I tried to change the parameters, and also change the algorithm source code, but the problem still there.
When I did MSER algorithm using Matlab it was fast and no OutOfMemoryError.
This is my code:
MSERFeatureGenerator mser = new MSERFeatureGenerator(delta, maxArea, minArea, maxVariation, minDiversity, PixelsFeature.class);
List<Component> up_regions = mser.generateMSERs(flattenImg, MSERDirection.Up);
The error actually occurs when I call the following method:
List<MergeTreeBuilder> mergeTrees = mser.performWatershed(Transforms.calculateIntensityNTSC(img));
Example of image with no problem:
Example2 of image that make OutOfMemoryError:
please help.
OpenIMAJ implements the fast MSER algorithm defined by Nister and Stewinius (see http://link.springer.com/chapter/10.1007%2F978-3-540-88688-4_14). The OpenIMAJ implementation is very fast at finding the pivot pixels of each maximally stable region (you can see this by removing the reference to PixelsFeature.class from your code).
The reason you are getting OOMs and bad performance using PixelsFeature is that the underlying watershed algorithm is creating a connected component for every region for every one of 256 grey-levels (this happens before the maximally stable components are found, so is going to create a really massive tree structure with overlapping pixel sets at each level). This is not something you want to do...
I don't have any code to hand to demonstrate an alternative approach, but what you probably want to do is compute the pivot pixels and then work backwards to get the ConnectedComponents using a flood-fill like approach. If the MSER direction was Up, then starting at each pivot pixel do a flood fill in the image to find the connected component made up of all pixel values less than or equal to the pivot pixel value (note that the pivot pixels represent their grey-level as an integer; you'll need to divide by 255 to get it as a lot compatible with the input image).
This is my image:
I want to remove the the noise(little black unrelated pixels) from my image.
I tried gaussian, cvtColor and adaptiveThreshold but none did the job.
Ideas? If you have a code in java that will help.
Noise removal could be tricky depending on the application. for your image a simple morphology operation like closing/Opening could solve the problem. Opening operation helps to remove the noises in your image. Closing will help to fill the existing holes.
Since OpenCV c++ APIs are like Java you can use the following:
cv::Mat Kernel = cv::Mat(cv::Size(Maximum_Width_of_Noise,Maximum_Height_of_noise),CV_8UC1,cv::Scalar(255));
cv::morphologyEx(Input_Image,temp,cv::MORPH_OPEN,Kernel);
cv::morphologyEx(temp,Output_Image,cv::MORPH_CLOSE,Kernel);
There are many methods but there's a trade off between accuracy/speed here.
The result of above operation is:
I am writing a game on Android, and it is coming along well enough. I am trying to keep everything as efficient as possible, so I am storing as much as I can in Vertex Buffer Objects to avoid unnecessary CPU overhead. However the simple act of drawing lots of unrelated primitives, or even a varying length string of sprites efficiently (such as drawing text to the screen) is escaping me.
The purpose of these primitives is menus and buttons, as well as text.
For drawing the menus, I could just make a vertex array for each element (menu background, buttons, etc), but since they are all just quads, this feels very inefficient. I could also create a sort of drawQuad() function that lets me just transparently load a single saved vertex array with data for xy/height&width/color/texture/whatever. However, reloading each element of the array with the new coordinates and other data each time, to copy it to the Float Buffer (For C++ guys, this is a special step you have to do in Java to pass the data to GL) so I can resend it to the GPU also feels lacking in efficiency, though I don't know how else I could do it. (One boost in efficiency I could see is setting the quad coordinates to be a unit square and then using Uniforms to scale it, but this seems unscalable).
For text it is even worse since I don't know how long the text will be and don't want to have to create larger buffers for larger text (causing the GC to randomly fire later). The alternate is to draw each letter with a independent draw command, but this also seems very inefficient for even a hundred letters on the screen (Since I read that you should try to have as few draw commands as possible).
It is also possible that I am looking way too deep into the necessary optimization of openGL, but I don't want to back myself into a corner with some terrible design early on.
You should try looking into the idea of interleaving data for your glDrawArrays calls.
Granted this link is for iphone, but there is a nice graphic at the bottom of the page that details this concept. http://iphonedevelopment.blogspot.com/2009/06/opengl-es-from-ground-up-part-8.html
I'm going to assume for drawing your characters that you are specifying some vertex coords and some texture coords into some sort of font bitmap to pick the correct character.
So you could envision your FloatBuffer as looking like
[vertex 1][texcoord 1][vertex 2][texcoord 2][vertex 3][texcoord 3]
[vertex 2][texcoord 2][vertex 3][texcoord 3][vertex 4][texcoord 4]
The above would represent a single character in your sentence if you're using GL_TRIANGLES, and you could expand on this idea to have vertices 5 - 8 to represent the second character and so on and so forth. Now you could draw all of your text on screen with a single glDrawArrays call. Now you might be worried about having redundant data in your FloatBuffer, but the savings will be huge. For example, in rendering a teapot with 1200 vertices and having this redundant data in my buffer, I was able to get a very visible speed increase over calling glDrawArrays for each individual triangle maybe something like 10 times better.
I have a small demo on sourceforge where I use data interleaving to render the teapot I mentioned earlier.
Its the ShaderProgramTutorial.rar. https://sourceforge.net/projects/androidopengles/files/ShaderProgram/
Look in teapot.java in the onDrawFrame function to see it.
On a side note you might find some of the other things on that sourceforge page helpful in your future Android OpenGL ES 2.0 fun!