I'm trying to make a program that can read the information off of a nutritional label but Tesseract is having lots of issues actually being able to read anything. I've tried a number of different Image processing techniques using OpenCV but not much seems to help.
Here are some of my better looking attempts (which happen to be the simplest):
Tango bottle label uneditied
Tango bottle label edited
Output:
200k], Saturates, 09
Irn Bru bottle label unedited
Irn Bru bottle label edited
Output
This is just changing the images to grey scale, a 3x3 Gaussian blur and Otsu binarisation.
I would appreciate any help on how to make the text more readable using OpenCV or any other image processing library.
Would it be simpler to forego using Tesseract and use machine learning for this?
First of all read this StackOverflow Answer regarding OCR prepossessing.
The most important steps described above are the Image Binarization and Image Denoising
Here is an example:
Original Image
Grey Scale
Unsharp Masking
Binarization
Now ready to apply OCR
JAVA code
Imgproc.cvtColor(original, grey, Imgproc.COLOR_RGB2GRAY, 0);
Imgproc.GaussianBlur(grey, blur, new Size(0, 0), 3);
Core.addWeighted(blur, 1.5, unsharp, -0.5, 0, unsharp);
Imgproc.threshold(unsharp,binary,127,255,Imgproc.THRESH_BINARY);
MatOfInt params = new MatOfInt(Imgcodecs.CV_IMWRITE_PNG_COMPRESSION);
File ocrImage = new File("ocrImage.png");
Imgcodecs.imwrite(ocrImage,binary,params);
/*initialize OCR ...*/
lept.PIX image = pixRead(ocrImage);
api.SetImage(image);
String ocrOutput = api.GetUTF8Text();
Related
I'm trying to write an OpenCV program in Java that takes a photo of a marker with a phone and finds circular contours on the marker. I have gotten it working in the Android emulator (where the photo has perfect lighting conditions), but can't get it to work with the phone itself.
This is the marker I'm trying to capture:
After using the combination of transforming to grayscale, a Gaussian blur and a Canny edge detector, I get this output:
If I then try to find the contours on the image, the number of contours returned is really high (over 1000), but they aren't closed (because the edges seem to be too weak.
The contours drawn over the original image:
This is the code I use for the segmentation:
Mat processed = new Mat(original.height(), original.width(), original.type());
Imgproc.cvtColor(original, processed, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(processed, processed, new Size(7, 7), 0.1);
Imgproc.Canny(processed, processed, 50, 50*3, 3, false);
I have tried tweaking the different parameters (thresholds etc.), but feel like that isn't the ideal solution. I'm thinking there has to be a way to enhance the edges returned by the canny detector, but I haven't found anything myself.
Any help is appreciated!
You want something like this?
Noise is usually present - it can be reduced - you can distinguish the circles with adaptiveThreshold and then find the circles using the method I described below. The point is, the real-world image you get from the camera may contain a whole bunch of other circles - so it might be best to find all the circles. Compare them all in size. Find the 4 circles that are most similar to your marker in terms of color, size and placement.
A quick python code to distinguish the circles:
# Make a gray version
gry = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
# Thresh
ada = cv2.adaptiveThreshold(
gry, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2
)
# Remove noises
out = cv2.medianBlur(ada, 7)
# Invert colors (Not needed)
out = ~out
I tested the Python code and it works; you can find an equivalent for Java or C++. I tried to explain the Java code but I wrote it on the fly and did not test it. The Java code I wrote probably has errors, but it does get the point. With a bit change will probably work. I also wrote the code that should find the circles as the last block. Working with it is tricky and requires adjusting the parameters.
Java:
...
Imgproc.cvtColor(im, gry, Imgproc.COLOR_RGBA2GRAY);
Imgproc.adaptiveThreshold(gry, ada, 255,
Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 11, 2);
Imgproc.medianBlur(ada, out, 7);
...
and for finding circles:
Java:
...
SimpleBlobDetector_Params params = new SimpleBlobDetector_Params();
params.set_filterByCircularity(true);
params.set_minCircularity(0.01f);
params.set_maxArea(50000);//important
SimpleBlobDetector detector = SimpleBlobDetector.create(params);
// List of detected points
MatOfKeyPoint keyPoints = new MatOfKeyPoint();
detector.detect(ada, keyPoints);
// Draw circles on final image
Scalar color = new Scalar(127, 0, 255);
for (KeyPoint key: keyPoints.toList()) {
Imgproc.circle(im, key.pt, (int) (key.size / 2.0f), color, 3/*Thickness*/);
}
...
You can look into dilation operation in opencv to enhance the edges.
https://docs.opencv.org/4.x/db/df6/tutorial_erosion_dilatation.html
Also look into this example below with Canny edge detection, approxPolyDP and minEnclosingCircle. It is very close to your question.
https://docs.opencv.org/4.x/da/d0c/tutorial_bounding_rects_circles.html
https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#ga0012a5fdaea70b8a9970165d98722b4c
https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#ga8ce13c24081bbc7151e9326f412190f1
I would like to preprocess a given picture by thresholding it in order to then hand it over to Tesseract. I first did that using Gimp (2.8.16) and a fixed range of 130 – 255. When I then implemented it in OpenCV (3.1) using Java, I first forgot to call cvtColor resulting in a picture that still had some colors in it (these areas were white in Gimp). Besides that, the picture was as expected. However, when I implemented the corresponding call, I got a picture that was different to the one I would have expected. It seems that the areas that were colored previously are now black while the remaining picture is similar to the one I created with Gimp.
Is there anything that I am missing to create a more similar output?
The reason I am asking this question is that, unfortunately, Tesseract (with psm 6) creates quite different results for the two images:
for the one created in Gimp: "2011 1 L 0006"
for the second one created with OpenCV: "2011ÔÇö] L 0 0006 1"
Here is the code that I used:
Mat thres = new Mat();
Mat tmp = new Mat();
Imgproc.cvtColor(src, tmp, Imgproc.COLOR_BGR2GRAY); // tmp = src.clone(); in my first attempt
Imgproc.threshold(tmp, thres, 130, 255, Imgproc.THRESH_BINARY);
Imgcodecs.imwrite("output.jpg", thres);
Here are the pictures:
Given picture:
Picture created with Gimp:
First result using OpenCV:
Second result using OpenCV:
In the first case, You are doing thresholding on color image (tmp = src.clone() creates another copy of src which is a color image). So you are getting result like that and in the second case, you are first converting to grayscale and then thresholding which gives a better result. Thresholding is good on gray scale images.
iText 7 just came out May 2016, and while some of the tutorials have been helpful, some of the more advanced functions have been harder to figure out. This page has an example of how to use text as a watermark (about 90% of the way down the page), but I can't figure out how to use an Image as a watermark, and I really have no idea where to start with the new release. Anyone know how to use an Image as a watermark in iText 7? Any ideas where to start?
I'm not 100% positive this is the right way to do this but I'd say I'm 95% confident.
Using the tutorial for iText 7 that you linked to as a starting guide along with the iText 5 version we can use a "graphics state" to modify the current canvas.
(The code below is C# but you should be able to convert it to Java pretty easily, pretty much just lowercase the first letter of properties and methods. Also, I'm using full namespace paths just so you know where things are at.)
First, create a custom state and set its transparency:
//Create a transparent state
iText.Kernel.Pdf.Extgstate.PdfExtGState tranState = new iText.Kernel.Pdf.Extgstate.PdfExtGState();
tranState.SetFillOpacity(0.5f);
Second, get your image:
//Get your image somehow
iText.IO.Image.ImageData myImageData = ImageDataFactory.Create("D:\\14.jpg", false);
iText.Layout.Element.Image myImage = new iText.Layout.Element.Image(myImageData);
Third (and optional), change your image if needed:
//Position, rotate and scale it as needed
myImage.SetFixedPosition(100, 100);
myImage.SetRotationAngle(45);
myImage.ScaleAbsolute(200, 200);
Fourth, save the pdfCanvas (from the tutorial) state and set a new one:
pdfCanvas.SaveState().SetExtGState(tranState);
Fifth, add your image to the higher level canvas (once again, from the tutorial):
canvas.Add(myImage);
And sixth, reset the pdfCanvas state:
pdfCanvas.RestoreState();
Update by Bruno:
Adding images is explained in Chapter 3 of the "iText 7: Building Blocks" tutorial. In chapter 3 of "iText 7: Jump-Start tutorial", we work with a PdfCanvas and Canvas object. The missing information about how to create and add an image is in the "Building Blocks" tutorial.
I am doing the image processing and I am interested in normalization of the image. Somewhere I come to know about the ImageJ library.
So could anyone help me in finding example code for Image normalization using ImageJ library.
I know It isn't with imageJ. But I would help you with java.
Try to use Catalano Framework.
The next version (1.2) will have more algorithms for image normalization as:
Gray World
White Patch
Modified White Patch
Example:
FastBitmap fb = new FastBitmap(bufferedImage);
HistogramEqualization hist = new HistogramEqualization();
hist.applyInPlace(fb);
bufferedImage = fb.toBufferedImage();
I'm trying to create thumbnails for uploaded images in a JRuby/Rails app using the Image Voodoo plugin - the problem is the resized thumbnails look like... ass.
It seems that the code to generate the thumbnails is absolutely doing everything correctly to set the interpolation rendering hint to "bicubic", but it isn't honoring them on our dev environment (OS X), or on the production web server (Linux).
I've extracted out the code to generate the thumbnails, rewritten it as a straight Java app (ie kicked off from a main() method) with the interpolation rendering hint explicitly set to "bicubic", and have reproduced the (lack of) bicubic and bilinear resizing.
As expected on both OS X and Linux the thumbanils are ugly and pixelated, but on Windows, it resizes the images nicely with bicubic interpolation used.
Is there any JVM environment setting and/or additional libraries that I'm missing to make it work? I'm doing a lot of banging of head against wall for this one.
I realize this question was asked a while ago, but incase anyone else is still running into this.
The reason the thumbnails look like ass are caused by two things (primarily the first one):
Non-incremental image scaling in Java is very rough, throws a lot of pixel data out and averages the result once regardless of the rendering hint.
Processing a poorly supported BufferedImage type in Java2D (typically GIFs) can result in very poor looking/dithered results.
As it turns out the old AreaAveragingScaleFilter does a decent job of making good looking thumbnails, but it is slow and deprecated by the Java2D team -- unfortunately they didn't replace it with any nice out-of-the-box alternative and left us sort of on our own.
Chris Campbell (from the Java2D team) addressed this a few years ago with the concept of incremental scaling -- instead of going from your starting resolution to the target resolution in one operation, you do it in steps, and the result looks much better.
Given that the code for this is decently large, I wrote all the best-practices up into a library called imgscalr and released it under the Apache 2 license.
The most basic usage looks like this:
BufferedImage img = ImageIO.read(...); // load image
BufferedImage scaledImg = Scalr.resize(img, 640);
In this use-case the library uses what is called it's "automatic" scaling mode and will fit the resulting image (honoring it's proportions) within a bounding box of 640x640. So if the image is not a square and is a standard 4:3 image, it will resize it to 640x480 -- the argument is just it's largest dimension.
There are a slew of other methods on the Scalr class (all static and easy to use) that allow you to control everything.
For the best looking thumbnails possible, the command would look like this:
BufferedImage img = ImageIO.read(...); // load image
BufferedImage scaledImg = Scalr.resize(img, Method.QUALITY,
150, 100, Scalr.OP_ANTIALIAS);
The Scalr.OP_ANTIALIAS is optional, but a lot of users feel that when you scale down to a small enough thumbnail in Java, some of the transitions between pixel values are a little too discrete and make the image look "sharp", so a lot of users asked for a way to soften the thumbnail a bit.
That is done through a ConvolveOp and if you have never used them before, trying to figure out the right "kernel" to use is... a pain in the ass. That OP_ANTIALIAS constant defined on the class it the best looking anti-aliasing op I found after a week of testing with another user who had deployed imgscalr into their social network in Brazil (used to scale the profile photos). I included it to make everyone's life a bit easier.
Also, ontop of all these examples, you might have noticed when you scale GIFs and some other types of images (BMPs) that sometimes the scaled result looks TERRIBLE compared to the original... that is because of the image being in a poorly supported BufferedImage type and Java2D falling back to using it's software rendering pipeline instead of the hardware accelerated one for better supported image types.
imgscalr will take care of all of that for you and keep the image in the best supported image type possible to avoid that.
Anyway, that is a REALLY long way of saying "You can use imgscalr to do all that for you and not have to worry about anything".
maybe is this a solution for you:
public BufferedImage resizeImage(BufferedImage source, int width, int height)
{
BufferedImage result = new BufferedImage(widht, height, BufferedImage.TYPE_INT_ARGB);
Graphics g = result.getGraphics();
g.drawImage(source, 0, 0, widht, height, null);
g.dispose();
return result;
}
In the end, upgrading to the latest version of ImageVoodoo seemed to improve quality.
Looking through the source code, it looks like they're doing some funky AWT rendering, and then pulling that out. Nasty, but it seems to work.
Still not as good as ImageMagick, but better than it was.
#Riyad, the code for incremental scaling isn't "decently large", it's quite small (As you can see from a post back in 2007, http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html#creating-scaled-instances) having a library that gives other options might be useful, but making a library to use a library is nonsense.