I want to recognize numbers in the following image
I am currently using Tess4J library in eclipse java project but it only recognizes the characters in a plane color background. For this image it could not even identify that there are characters(numbers) on this image. Help me find a way to accomplish this task.
Here is my current code:
import net.sourceforge.tess4j.*;
import java.io.File;
public class Main {
public static void main(String[] args) {
File imageFile = new File("image.png");
Tesseract instance = Tesseract.getInstance();
try {
String result = instance.doOCR(imageFile);
System.out.println(result);
} catch (TesseractException e) {
System.err.println(e.getMessage());
}
}
}
and if there is way to count the squares separated by yellow lines.
Thank you
If your image is representative, then all you need as a first step is a binarization at a threshold close to the maximum value followed by discarding of small components.
f = Import["http://i.stack.imgur.com/6AXwH.jpg"]
step1 = SelectComponents[Binarize[ColorConvert[f, "Grayscale"], 0.9],
"Count", #1 > 100 &]
Now, if you know that the digits cannot be too tall or too thin (this is dependent on image dimensions), then you can filter the remaining components based on its bounding box.
SelectComponents[step1, "BoundingBox",
And[10 < #[[2, 1]] - #[[1, 1]] < 100, 50 < #[[2, 2]] - #[[1, 2]] < 100] &]
To separate each of the regions, you could consider using a colorspace where there is a channel dedicated to the yellow color. CMYK is a possibility here, and again all you need is a threshold at a high value, together with the basic morphological closing to complete the lines (since in your example the lines do not extend to the border of the image). Instead of using morphological closings here, you could detect the lines using Hough or RANSAC, for example.
rects = Closing[
Closing[Binarize[ColorSeparate[f, "CMYK"][[3]], 0.9],
ConstantArray[1, {1, 15}]], ConstantArray[1, {15, 1}]] (* left image *)
Colorize[MorphologicalComponents[ColorNegate[rects]],
ColorFunction -> "Rainbow"] (* right image *)
The tools used here are very simple, and almost any image processing library will provide them. There are also more robust approaches that could be taken, but for the given image it is not needed.
Related
I'm making a small Java program that can modify an existing PDF, and save the changes in a new PDF, using iTextPDF 7. I started with a rotating feature, using setRotation() on a PdfDocument, but my PDF output is rotating 90 degree less than the value I put as a parameter.
So setRotation(90) makes no changes,
setRotation(180) does a single clockwise change,
setRotation(270) does a double turn (180 degree rotation).
etc..
Here's my Code:
import java.io.IOException;
import com.itextpdf.kernel.pdf.PdfDocument;
import com.itextpdf.kernel.pdf.PdfReader;
import com.itextpdf.kernel.pdf.PdfWriter;
public class rotateMyPDF {
public static void main(String[] args) throws IOException {
PdfReader in_pdf = new PdfReader("in.pdf");
PdfDocument pdfDoc = new PdfDocument(in_pdf, new PdfWriter("out.pdf"));
for (int i=1; i<=pdfDoc.getNumberOfPages(); i++) {
pdfDoc.getPage(i).setRotation(270); //error here?
}
pdfDoc.close();
System.out.println("done.");
}
}
Seems like a bug? Or is there an error in my code? Or is this intended functionality?
If I put setRotation(0), the PDF pages are rotated 90 degrees counter clockwise!
You appear to misunderstand the setRotation method. It does not add to the current rotation but it sets the rotation value.
Thus, if your page already had page rotation applied, setting the same value with setRotation will change nothing etc.
Your source page appears to already be rotated by 90°. That would explain the observation:
So setRotation(90) makes no changes,
setRotation(180) does a single clockwise change,
setRotation(270) does a double turn (180 degree rotation).
etc..
Simply first read the current rotation value with getRotation, add the desired extra rotation, and then set that sum using setRotation.
I'm currently trying to improve the recognition rate of GoogleCloud Vision, so I am building a preprocessing pipeline.
I currently can create a mask which overlays the characters in the image, but as you can see in the examples below, it also shows the lines. Now since those lines can cross through characters, I'd like to remove them from the mask without destroying the characters, if possible.
Current steps:
Line detection:
InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU) -> HoughLinesP
Mask generation: InputImage -> Grayscale -> Blackhat -> GaussianBlur -> Threshhold(OTSU)-> ConnectedComponents
ImageExamples:(Due to privacy protection, sharing a full Image is not possible)
The images show the original image, the mask and the lines recognized.
The following code is used to generate the mask and find the lines
Mat picture = Imgcodecs.imread(path);
Imgproc.cvtColor(picture, picture, Imgproc.COLOR_BGR2GRAY);
Imgcodecs.imwrite("/home/meik/Pictures/asdfGray.png", picture);
Mat blackhatElement = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_RECT, new Size(7, 7));
Imgproc.morphologyEx(picture, picture, Imgproc.MORPH_BLACKHAT, blackhatElement);
Imgproc.GaussianBlur(picture, picture, new Size(5, 3), 0);
Imgproc.threshold(picture, picture, 0, 255, Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU);
/**
* Line Detection with Canny and HoughLines(P)
*/
Mat lines = new Mat();
Mat linesResult = Mat.zeros(picture.rows(),picture.cols(), CvType.CV_8UC1);
Imgproc.HoughLinesP(picture, lines,1, Math.PI/180,100, 20, 0);
System.out.println("lines rows:" + lines.rows());
for (int x = 0; x < lines.rows(); x++) {
double[] l = lines.get(x, 0);
Imgproc.line(linesResult, new Point(l[0], l[1]), new Point(l[2], l[3]), new Scalar(255, 255, 255), 1, Imgproc.LINE_8, 0);
}
/**End of line detection*/
Mat kernel = Imgproc.getStructuringElement(Imgproc.CV_SHAPE_CROSS, new Size(3,3));
Imgproc.dilate(linesResult,linesResult,kernel);
Core.bitwise_not(linesResult,linesResult);
I've found this paper talking about the problem, but am struggling to understand their methodology.
How do I proceed from here on to remove lines without destroying the characters?
I dont really think you need to refer paper to do this.
Just use color info or hough line to find out a straightline which is really long
use that info to create a mask image.
Then use the opencv inpaint to remove it.
https://docs.opencv.org/2.4/modules/photo/doc/inpainting.html
e.g. what you want is similar to the bottom image. It ask to remove the traffic light poles. and you want the writing guideline to be removed. essentially, its the same thing
How about some simple image preprocessing?
For example using a threshold to only maintain a certain color range (instead of directly converting the image to grayscale).
Something like this is integrated in GIMP, see
https://docs.gimp.org/2.8/en/gimp-tool-threshold.html
You probably want to experiment with various thresholds.
Im working with PDFClown to analyze and work with PDFDocuments. My aim is to highlight all numbers within a table. For all numbers which belong together (For example: All numbers in one column of a table) I will create one TextMarkup with a List of Quads. First of all it looks like everythink work well: All highlights on the left belong to one TextMarkup and all Highlights on the right belong to another TextMarkup.
But when analyzing the size of the TextMarkup the size is bigger than it looks at the picture. So when drawing for example a rectangle arround the left TextMarkup box the rectangle intersects the other column despite no highlight of the left TextMarkup intersects the other column. Is there a way to optimize the Box of the TextMarkup? I think there is a bulbous ending of the box so that the box is intersecting the other TextMarkup
This is the code which creates the TextMarkup:
List<Quad> highlightQuads = new ArrayList<Quad>();
for (TextMarkup textMarkup : textMarkupsForOneAnnotation) {
Rectangle2D textBox = textMarkup.getBox();
Rectangle2D.Double rectangle = new Rectangle2D.Double(textBox.getX(), textBox.getY(), textBox.getWidth(), textBox.getHeight());
highlightQuads.add(Quad.get(rectangle));
}
if (highlightQuads.size() > 0) {
TextMarkup _textMarkup = new TextMarkup(pagesOfNewFile.get(lastFoundNewFilePage).getPage(), highlightQuads,"", MarkupTypeEnum.Highlight);
_textMarkup.setColor(DeviceRGBColor.get(Color.GREEN));
_textMarkup.setVisible(true);
allTextMarkUps.add(_textMarkup);
}
Here is an example file Example
Thank You !!
Your code is not really self contained (I cannot run it as it in particular misses the input data), so I could only do a bit of PDF Clown code analysis. That code analysis, though, did indeed turn up a PDF Clown implementation detail that would explain your observation.
How does PDF Clown calculate the dimensions of the markup annotation?
The markup annotation rectangle must be big enough to include all quads plus start and end decorations (rounded left and right caps on markup rectangle).
PDF Clown calculates this rectangle as follows in TextMarkup:
public void setMarkupBoxes(
List<Quad> value
)
{
PdfArray quadPointsObject = new PdfArray();
double pageHeight = getPage().getBox().getHeight();
Rectangle2D box = null;
for(Quad markupBox : value)
{
/*
NOTE: Despite the spec prescription, Point 3 and Point 4 MUST be inverted.
*/
Point2D[] markupBoxPoints = markupBox.getPoints();
quadPointsObject.add(PdfReal.get(markupBoxPoints[0].getX())); // x1.
quadPointsObject.add(PdfReal.get(pageHeight - markupBoxPoints[0].getY())); // y1.
quadPointsObject.add(PdfReal.get(markupBoxPoints[1].getX())); // x2.
quadPointsObject.add(PdfReal.get(pageHeight - markupBoxPoints[1].getY())); // y2.
quadPointsObject.add(PdfReal.get(markupBoxPoints[3].getX())); // x4.
quadPointsObject.add(PdfReal.get(pageHeight - markupBoxPoints[3].getY())); // y4.
quadPointsObject.add(PdfReal.get(markupBoxPoints[2].getX())); // x3.
quadPointsObject.add(PdfReal.get(pageHeight - markupBoxPoints[2].getY())); // y3.
if(box == null)
{box = markupBox.getBounds2D();}
else
{box.add(markupBox.getBounds2D());}
}
getBaseDataObject().put(PdfName.QuadPoints, quadPointsObject);
/*
NOTE: Box width is expanded to make room for end decorations (e.g. rounded highlight caps).
*/
double markupBoxMargin = getMarkupBoxMargin(box.getHeight());
box.setRect(box.getX() - markupBoxMargin, box.getY(), box.getWidth() + markupBoxMargin * 2, box.getHeight());
setBox(box);
refreshAppearance();
}
private static double getMarkupBoxMargin(
double boxHeight
)
{return boxHeight * .25;}
So it takes the bounding box of all the quads and adds left and right margins each as wide as a quarter of the height of this whole bounding box.
What is the result in your case?
While this added margin width is sensible if there is only a single quad, in case of your markup annotation which includes many quads on top of one another, this results in a giant, unnecessary margin.
How to improve the code?
As the added caps depend on the individual caps and not their combined bounding box, one can improve the code by using the maximum height of the individual quads instead of the height of the bounding box of all quads, e.g. like this:
Rectangle2D box = null;
double maxQuadHeight = 0;
for(Quad markupBox : value)
{
double quadHeight = markupBox.getBounds2D().getHeight();
if (quadHeight > maxQuadHeight)
maxQuadHeight = quadHeight;
...
}
...
double markupBoxMargin = getMarkupBoxMargin(maxQuadHeight);
box.setRect(box.getX() - markupBoxMargin, box.getY(), box.getWidth() + markupBoxMargin * 2, box.getHeight());
setBox(box);
If you don't want to patch PDF Clown for this, you can also execute this code (with minor adaptations) after constructing the TextMarkup _textMarkup to correct the precalculated annotation rectangle.
Is this fixing a PDF Clown error?
It is not an error as there is no need for the text markup annotation rectangle to be minimal; PDF Clown could also always use the whole crop box for each such annotation.
I would assume, though, that the author of the code wanted to calculate a somewhat minimal rectangle but only optimized for single line and so in a way did not live up to his own expectations...
Are there other problems in this code?
Yes. The text a markup annotation marks needs not be horizontal, it may be there at an angle, it could even be vertical. In such a case some margin would also be needed at the top and the bottom of the annotation rectangle, not (only) at the left and the right.
I'm a complete beginner in Java programming and I'm interested to learn more about its concepts.
Recently, I've been given an exercise which instructs me to display two versions of a picture. The picture to be displayed is provided in the form of a data file of 40,000 digits that are arranged in rows (although there is no marker between rows) and it starts from the top of the picture. So the first digit represents the top left corner of the picture and the last is the bottom right.
Basically, what the exercise wants me to construct a program that plots a dot in one of two colours for each digit. If the digit is in the range 0 to 3 the output should be one colour and for digits in the range 4 to 9 the dot should be in the other colour.
I understand I have to use arrays and also loops to perform this. I'm familiar with the fillEllipse, drawEllipse, drawRectangle and fillRectangle but this exercise is nothing I've attempted before.
Any hints on how to make this work? Your help would be greatly appreciated.
As a hint, rather than a complete solution, I would suggest looking into creating a java.awt.image.BufferedImage, and then set the colors of the individual pixels using the setRGB() method. You would then display this image using drawImage() on your Graphics object.
All what you need is how to read the digits from the file and put it into two dimension array
check this tutorial for how to read a file
Then you have to draw each pixel on a fram or a Panel
check this Java basic graphics tutorial
I hope this could help!
use Scanner to read the data like :
Scanner sc = new Scanner(new File("path_to_your_digits_file"));
int[][] digits= new int [200][200];
String line;
while(sc.hasNext()){//this means there is still a line to go
line = sc.nextLine();
//split the line and fill the array . or read char by char ,
// i dont know how your file looks like, it's just some simple string manipulation here
}
int x;
BufferedImage img = new BufferedImage(200,200,BufferedImage.TYPR_INT_RGB);
for(int i=0;i<200;i++){
for(int j=0;i<200;j++){
if(digits[i][j]>0 && digits[i][j]<=3){
x=//some color code;
}else{
x=//some other color code;
} //not sure about this loop , again idk the structure of your file;
img.setRGB(i,j,x);
}
}
JLabel lbl = new JLabel();
lbl.setSize(200,200);
ImageIcon ico=new ImageIcon(img);
lbl.setIcone(ico);
lbl.setVisible(true);
JFrame frame = new Jframe();
frame.setSize(500,500);
frame.add(lbl);
frame.setVisible(true);
I am trying to determine differences between two images during an integration test. After doing some research on the web, I stumbled over the MarvinProject and tried to create a UnitTest with it, see below.
As far as I understood the plugin DifferentRegions, it will fill the passed ImageMask differenceMask with the regions that contain the differences. The two images I pass for the test do differ, so it should print out something.
Unfortunately it does not.
I have written other tests that compare those two images byte-wise and those succeed. For those who want to try the problem, I have created a repository on GitHub and here is the ImageCompareTest.
#Test
public void tryMarvinProject() {
// init images
String root = "src/test/resources/";
MarvinImage assertedImg = MarvinImageIO.loadImage(root + "image1.bmp");
MarvinImage actualImg = MarvinImageIO.loadImage(root + "image2.bmp");
// init diff-regions plugin
DifferentRegions regions = new DifferentRegions();
regions.load();
regions.setAttribute("comparisonImage", assertedImg);
int width = assertedImg.getWidth();
int height = assertedImg.getHeight();
int type = assertedImg.getType();
// process the images and retrieve differences from the ImageMask
MarvinImageMask differenceMask = new MarvinImageMask();
regions.process(
actualImg,
new MarvinImage(new BufferedImage(width, height, type)),
new MarvinAttributes(),
differenceMask,
false);
// should contain the differences, but does not
System.out.println(differenceMask.getMaskArray());
assertNotNull(differenceMask.getMaskArray());
}
The plug-in DifferentRegions was developed for real-time video processing. The idea is to find the regions in the scene which are changing in a sequence of video frames, as shown in this Example
For image-to-image comparison, you should try DifferenceColor plug-in. Basically, it compare the two images analysing the color intensity of pixels in the same position. If the difference of two given pixels is higher than the attribute colorRange, the two pixels are considered distinct in color. The plug-in renders the different pixels in a different color to show them in the output image. If you pass an MarvinAttributes object in the process(...) method, you can get the number of different pixels, stored in the key total.
Example:
MarvinAttributes attrOut = new MarvinAttributes();
MarvinImagePlugin diff = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.difference.differenceColor");
diff.process(imageA, imageB, attrOut);
System.out.println("Distinct pixels:"+attrOut.get("total"));
PS: In order to use MarvinPluginLoader you must configure the development environment following the instructions presented here.
However, since you are trying to use image comparison for Unit Testing, you should take a look at MarvinTestCase class. It extends JUnit TestCase and provide methods to compare MarvinImage objects.
Input:
The source code below implements two test cases, one comparing imageA to imageB, and the other comparing imageA to imageC.
import marvin.image.MarvinImage;
import marvin.io.MarvinImageIO;
import marvin.test.MarvinTestCase;
public class UnitTesting extends MarvinTestCase {
private MarvinImage imageA, imageB, imageC;
public UnitTesting(){
imageA = MarvinImageIO.loadImage("./res/penguin.jpg");
imageB = MarvinImageIO.loadImage("./res/penguin.jpg");
imageC = MarvinImageIO.loadImage("./res/penguin_dark.jpg");
}
public void testSameImage(){
assertEquals(imageA, imageB);
}
public void testDistinctImages(){
assertEquals(imageA, imageC);
}
}
Using Eclipse or another IDE, run the class above as a JUnit Test. Below the output in the JUnit tab on Eclipse.