I have read from a list of image from a folder and then do cropping and segmented the character from the image. Then the file created is save to another folder with new name. For example the first source image(image1) I have segmented 7 new character images and it goes on with the next image.
My question is how can I regroup these images according to their source image like assigning them (im01-im07 is from source image1) and so on. This is the code:
for (int sf = 0; sf < listOfSrcFFiles.length; sf++) {
File imgFile = listOfSrcFFiles[sf];
String fileName = imgFile.getName();
if (fileName.equals("Thumbs.db")) {
imgFile.delete();
} else {
try{
Mat img_grayROI = Highgui.imread(imgFile.getAbsolutePath(), Highgui.CV_LOAD_IMAGE_GRAYSCALE);
Rect roi = new Rect(300, 200, 450, 200);
Mat cropped = new Mat(img_grayROI, roi);
Imgproc.threshold(cropped, cropped, 0, 255, Imgproc.THRESH_BINARY + Imgproc.THRESH_OTSU);
Point shift = new Point(0, 150);
Mat result = cropped;
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(result, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
Imgproc.floodFill(result, new Mat(), shift, new Scalar(255, 0, 0));
Core.bitwise_not(result, result);
for (int i = 0; i < contours.size(); i++) {
if (Imgproc.contourArea(contours.get(i)) > 50) {
Rect rect = Imgproc.boundingRect(contours.get(i));
/*if (rect.height > 25 && rect.height < 50 && rect.width > 10 && rect.width < 50) {*/
if (rect.height > 25 && rect.width < 25) {
Core.rectangle(result, rect.tl(), rect.br(), new Scalar(0, 0, 0));
Mat crop = new Mat(result, rect);
Highgui.imwrite(tmpFolder + fSprator + sf + "" + i + ".bmp", crop);
}
}
}
}catch(Exception e){
}
}
}
This is so I can later save them into text file and separated them according to their source image name.
Thank you, any help much appreciated.
The cleanest solution is to create a bean to hold the results.
Eg:
public class TransformResult {
private File input;
private List<File> transformedFiles = new ArrayList<>();
// getters / setters
}
public List<TransformResult> doTransform(File listOfSrcFFiles) {
List<TransformResult> results = new ArrayList<>();
for (File srcFile : listOfSrcFFiles) {
if (...) {
TransformResult result = new TransformResult(srcFile);
for (...) {
if (...) {
File transformed = doSomeTransformation(srcFile);
result.getTransformedFiles().add(transformed);
}
}
results.add(result);
}
}
return results;
}
Related
I'd like to copy the recognized text to a clean image using createBitmap but I dont know how to get it from the boxes on the original image.
This code finds Maximally stable extremal regions and highlights them on the bitmap. Further below is what I get wit it on a sample image.
private void sgmnt(Mat mImg) {
Imgproc.cvtColor(msImg, m2, Imgproc.COLOR_RGB2GRAY);
Mat mRgba = mImg;
Mat mGray = m2;
Scalar CONTOUR_COLOR = new Scalar(1, 1, 255, 1);
//Scalar CONTOUR_COLOR = new Scalar(255);
MatOfKeyPoint keyPoint = new MatOfKeyPoint();
List<KeyPoint> listPoint = new ArrayList<>();
KeyPoint kPoint = new KeyPoint();
Mat mask = Mat.zeros(mGray.size(), CvType.CV_8UC1);
int rectanx1;
int rectany1;
int rectanx2;
int rectany2;
int imgSize = mGray.height() * mGray.width();
Scalar zeros = new Scalar(255,1,1, 1);
List<MatOfPoint> contour2 = new ArrayList<MatOfPoint>();
Mat kernel = new Mat(1, 50, CvType.CV_8UC1, Scalar.all(255));
Mat morByte = new Mat();
Mat hierarchy = new Mat();
Rect rectan3 = new Rect();
FeatureDetector detector = FeatureDetector.create(FeatureDetector.MSER);
detector.detect(mGray, keyPoint);
listPoint = keyPoint.toList();
for(int ind = 0; ind < listPoint.size(); ++ind) {
kPoint = listPoint.get(ind);
rectanx1 = (int) (kPoint.pt.x - 0.5 * kPoint.size);
rectany1 = (int) (kPoint.pt.y - 0.5 * kPoint.size);
rectanx2 = (int) (kPoint.size);
rectany2 = (int) (kPoint.size);
if (rectanx1 <= 0) {
rectanx1 = 1;
}
if (rectany1 <= 0) {
rectany1 = 1;
}
if ((rectanx1 + rectanx2) > mGray.width()) {
rectanx2 = mGray.width() - rectanx1;
}
if ((rectany1 + rectany2) > mGray.height()) {
rectany2 = mGray.height() - rectany1;
}
Rect rectant = new Rect(rectanx1, rectany1, rectanx2, rectany2);
try{
Mat roi = new Mat(mask, rectant);
roi.setTo(CONTOUR_COLOR);
}
catch (Exception ex) {
Log.d("mylog", "mat roi error " + ex.getMessage());
}
}
Imgproc.morphologyEx(mask, morByte, Imgproc.MORPH_DILATE, kernel);
Imgproc.findContours(morByte, contour2, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
for(int i = 0; i<contour2.size(); ++i){
rectan3 = Imgproc.boundingRect(contour2.get(i));
if(rectan3.area() > 0.5 * imgSize || rectan3.area()<100 || rectan3.width / rectan3.height < 2){
Mat roi = new Mat(morByte, rectan3);
roi.setTo(zeros);
}else{
Imgproc.rectangle(mRgba, rectan3.br(), rectan3.tl(), CONTOUR_COLOR);
}
}
}
Here's an example of what I get:
My problem is I want to somehow get the text inside the boxes.
i have a problem with character segmentation from water meter,
i have a image water meter
Then i have threshold this image after threshold
And i have get a register meter after segmentation
My problem is to get each character from register meter..
Here my code:
System.out.println("Character Segmentation the image at " + path + "... ");
// get the jpeg image from the internal resource folder
Mat image = MatImageFromPath;
// convert the image in gray scale
Imgproc.cvtColor(image, image, Imgproc.COLOR_BGR2GRAY);
// thresholding the image to make a binary image
Imgproc.threshold(image, image, 100, 255, Imgproc.THRESH_BINARY_INV);
// finding the contours
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Imgproc.findContours(image, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
Imgproc.drawContours(image, contours, 3, new Scalar(0, 255, 0),3);
// finding best bounding rectangle for a contour whose distance is closer to the image center that other ones
Rect rect = new Rect();
ArrayList<Rect> contourRects = new ArrayList<Rect>();
Point p1 = new Point(rect.x,rect.y);
Point p2 = new Point((rect.x+rect.width),(rect.y+rect.height));
int i=0;
for (MatOfPoint contour : contours) {
rect = Imgproc.boundingRect(contour);
contourRects.add(rect);
contour2f = new MatOfPoint2f( contours.get(i).toArray() );
i++;
}
Collections.sort(contourRects, new Comparator<Rect>(){
#Override
public int compare(Rect o1, Rect o2){
return o1.x-o2.x;
}
});
for (int j = 0; j <= contourRects.size()-1; j++) {
if ((contourRects.get(j).width >= 50 && contourRects.get(j).width <= 120) &&
(contourRects.get(j).height >= 150 && contourRects.get(j).height <= 210)) {
System.out.println(contourRects.get(j).width);
Mat result = image.submat(contourRects.get(j));
Imgcodecs.imwrite("BoundingBox/"+(j+1)+".png", result);
// write the new image on disk
path = "BoundingBox/"+(j+1)+".png";
} else {
continue;
}
}
Imgcodecs.imwrite("BoundingBox/draw.png", image);
label2.setIcon(ImageFromPath(path));
All images at : images
can anyone help my problem? Thanks in advance
I am working on a licence plate recognition software using OpenCV, Tesseract and Java but experiencing issues, I cant seem to segment my text correctly, its not always that I get all characters to be detected and bounded with a bounding box these are some of my outputs with my code...and also when I detect the characters I never know which character is in which box so when I pass them through tesseract they get jumbled, how do I format my string?
This one fails despite the clearly visible characters:
Here Z and 6 fail to be detected even when 6 is clearly visible:
Below is my code:
originalFrame = image.clone();
roiColor = image.clone();
Imgproc.cvtColor(image, image, Imgproc.COLOR_BGR2GRAY, 0);
originalFrameGrayScale = image.clone();
Mat morph = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(9, 9));
Imgproc.morphologyEx(image, image, Imgproc.MORPH_TOPHAT, morph);
Imgproc.Sobel(image, image, -1, 2, 0);
Imgproc.GaussianBlur(image, image, new Size(5,5), 3,3);
Imgproc.morphologyEx(image, image, Imgproc.MORPH_CLOSE, morph);
Imgproc.threshold(image, image, 200, 255, Imgproc.THRESH_OTSU);
Vector<Rect> rectangles = detectionContour(image);
Mat roi = originalFrameGrayScale.clone();
if(!rectangles.isEmpty()){
roi = originalFrameGrayScale.submat(rectangles.get(0));
roiBlack = roi.clone();
roiColor = roiColor.submat(rectangles.get(0));
Imgproc.rectangle(originalFrame, rectangles.get(0).br(), rectangles.get(0).tl(), new Scalar(0,0,255), 2);
}
Imgproc.medianBlur(roi, roi, 3);
Imgproc.adaptiveThreshold(roi, roi, 225, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY, 15, 3);
roiBinarize = roi.clone();
Mat erode = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(1, 1));
Mat dilate = Imgproc.getStructuringElement(Imgproc.MORPH_RECT,new Size(1, 1));
Imgproc.morphologyEx(roi, roi, Imgproc.MORPH_OPEN, dilate);
Imgproc.morphologyEx(roi, roi, Imgproc.MORPH_OPEN, erode);
Imgproc.Canny(roi, roi, 150, 150 * 3, 3, true);
Vector<Rect> letters = detectionPlateCharacterContour(roi);
doTesseractOCR(letters, roiBinarize);
private static void doTesseractOCR(Vector<Rect> letters, Mat plate){
Tesseract instance = new Tesseract(); //
instance.setLanguage(LANGUAGE);
String resultPlate = "";
for(int i= 0; i < letters.size(); i++){
BufferedImage letter = OpenCvUtils.Mat2bufferedImage(plate.submat(letters.get(i)));
try {
String result = instance.doOCR(letter);
resultPlate += result + " position "+i;
} catch (TesseractException e) {
System.err.println(e.getMessage());
}
System.out.println("Tesseract output: "+resultPlate);
}
}
private static Vector<Rect> detectionPlateCharacterContour(Mat roi) {
Mat contHierarchy = new Mat();
Mat imageMat = roi.clone();
Rect rect = null;
List<MatOfPoint> contours = new ArrayList<>();
Imgproc.findContours(imageMat, contours, contHierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
Vector<Rect> rect_array = new Vector<>();
for (int i = 0; i < contours.size(); i++) {
rect = Imgproc.boundingRect(contours.get(i));
double ratio = 0;
if(rect.height > rect.width){
ratio = rect.height/rect.width;
}else{
ratio = rect.width/rect.height;
}
Logger.printMessage("Ratio of letter: "+ratio);
double contourarea = Imgproc.contourArea(contours.get(i));
if (contourarea >= 100 && contourarea <= 1000 && ( ratio >= 1 && ratio <= 2)) {
Imgproc.rectangle(roiColor, rect.br(), rect.tl(), new Scalar(255,0,0));
rect_array.add(rect);
}
}
contHierarchy.release();
return rect_array;
}
I tried to implement the KNN findNearest function. My program shall recognize numbers in a picture, but I get an error if the program uses findNearest. Here's the code:
`private void searchingData() {
img_gray = new Mat();
img_blur = new Mat();
img_thres = new Mat();
Imgproc.cvtColor(img, img_gray, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur(img_gray, img_blur, new Size(5,5), 0);
//Imgproc.adaptiveThreshold(img_blur, img_thres, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 7, 5);
Imgproc.Canny(img_blur, img_thres, 10, 100);
Imgproc.findContours(img_thres, contours, new Mat(), Imgproc.RETR_LIST,Imgproc.CHAIN_APPROX_SIMPLE);
for(int i=0; i< contours.size();i++){
Rect rect = Imgproc.boundingRect(contours.get(i));
if (rect.height < 50 && rect.height > 20){
System.out.println(rect.x +","+rect.y+","+rect.height+","+rect.width);
Mat subImg = new Mat();
Imgproc.resize(img.submat(rect), subImg, new Size(10,10));
//Found numbers then try to recognize it
recognize(subImg);
}
}
}
public void learn() { //get the training data and train the KNN
Mat sample, training_img = new Mat(), res = new Mat();
for (int i = 1; i < 10; i++) {
String path = String.format(".../Documents/numbers/%03d.png", i);
sample = Imgcodecs.imread(path, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
Mat m = new Mat(new Size(1,1), CvType.CV_32FC1);
m.setTo(new Scalar(i));
res.push_back(m);
training_img.push_back(prepareImg(sample));
}
model = KNearest.create();
model.train(training_img, Ml.ROW_SAMPLE, res);
}
private Mat prepareImg(Mat img) { //change image in the right format
Mat sample = new Mat(), sized = new Mat();
Imgproc.resize(img, sized, new Size(10,10));
sized.reshape(1,1).convertTo(sample, CvType.CV_32FC1);
return sample;
}
public void recognize(Mat getImg) {
Mat results = new Mat();
Mat dists = new Mat();
float result = model.findNearest(prepareImg(getImg), 2, results, new Mat(), dists);
//result should be the number in getImg
System.out.println(result);
}`
OpenCV Error: Assertion failed (test_samples.type() == CV_32F &&
test_samples.cols == samples.cols) in
cv::ml::BruteForceImpl::findNearest, file
C:\builds\master_PackSlaveAddon-win64-vc12-static\opencv\modules\ml\src\knearest.cpp,
line 325 Exception in thread "main" CvException
[org.opencv.core.CvException: cv::Exception:
C:\builds\master_PackSlaveAddon-win64-vc12-static\opencv\modules\ml\src\knearest.cpp:325:
error: (-215) test_samples.type() == CV_32F && test_samples.cols ==
samples.cols in function cv::ml::BruteForceImpl::findNearest ]
Thanks a lot!
I have been having some issues getting the outline of the detected object in the correct place, its as if the coordinates are in the wrong place. I have the hessian set to 2000 and I have filtered for matches that are less than 3 times the minimum distance. Any help would be appreciated.
Results from running matching and homography:
Code sample below:
public static void findMatches()
{
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
//Load Image 1
Mat img_object = Highgui.imread("./resources/Database/box.png");
//Load Image 2
Mat img_scene = Highgui.imread("./resources/Database/box_in_scene.png");
//Check if either image is null if so exit application
if (img_object == null || img_scene == null)
{
System.exit(0);
}
//Convert Image 1 to greyscale
Mat grayImageobject = new Mat(img_object.rows(), img_object.cols(), img_object.type());
Imgproc.cvtColor(img_object, grayImageobject, Imgproc.COLOR_BGRA2GRAY);
Core.normalize(grayImageobject, grayImageobject, 0, 255, Core.NORM_MINMAX);
//Convert image 2 to greyscale
Mat grayImageScene = new Mat(img_scene.rows(), img_scene.cols(), img_scene.type());
Imgproc.cvtColor(img_scene, grayImageScene, Imgproc.COLOR_BGRA2GRAY);
Core.normalize(grayImageScene, grayImageScene, 0, 255, Core.NORM_MINMAX);
//Create a SURF feature detector
FeatureDetector detector = FeatureDetector.create(4); //4 = SURF
//Cannot input hessian value as normal so we have to write the desired value into a
//file and then read value from file into detector.read
try (Writer writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream("hessian.txt"), "utf-8"))) {
writer.write("%YAML:1.0\nhessianThreshold: 2000.\noctaves:3\noctaveLayers: 4\nupright: 0\n");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
detector.read("hessian.txt");
//Mat of keypoints for object and scene
MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
MatOfKeyPoint keypoints_scene = new MatOfKeyPoint();
//Detect keypoints in scene and object storing them in mat of keypoints
detector.detect(img_object, keypoints_object);
detector.detect(img_scene, keypoints_scene);
DescriptorExtractor extractor = DescriptorExtractor.create(2); //2 = SURF;
Mat descriptor_object = new Mat();
Mat descriptor_scene = new Mat() ;
extractor.compute(img_object, keypoints_object, descriptor_object);
extractor.compute(img_scene, keypoints_scene, descriptor_scene);
DescriptorMatcher matcher = DescriptorMatcher.create(1); // 1 = FLANNBASED
MatOfDMatch matches = new MatOfDMatch();
matcher.match(descriptor_object, descriptor_scene, matches);
List<DMatch> matchesList = matches.toList();
Double max_dist = 0.0;
Double min_dist = 100.0;
for(int i = 0; i < descriptor_object.rows(); i++){
Double dist = (double) matchesList.get(i).distance;
if(dist < min_dist) min_dist = dist;
if(dist > max_dist) max_dist = dist;
}
System.out.println("-- Max dist : " + max_dist);
System.out.println("-- Min dist : " + min_dist);
LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
MatOfDMatch gm = new MatOfDMatch();
for(int i = 0; i < descriptor_object.rows(); i++){
if(matchesList.get(i).distance < 3*min_dist){
good_matches.addLast(matchesList.get(i));
}
}
gm.fromList(good_matches);
Mat img_matches = new Mat();
Features2d.drawMatches(img_object,keypoints_object,img_scene,keypoints_scene, gm, img_matches, new Scalar(255,0,0), new Scalar(0,0,255), new MatOfByte(), 2);
if(good_matches.size() >= 10){
LinkedList<Point> objList = new LinkedList<Point>();
LinkedList<Point> sceneList = new LinkedList<Point>();
List<KeyPoint> keypoints_objectList = keypoints_object.toList();
List<KeyPoint> keypoints_sceneList = keypoints_scene.toList();
for(int i = 0; i<good_matches.size(); i++){
objList.addLast(keypoints_objectList.get(good_matches.get(i).queryIdx).pt);
sceneList.addLast(keypoints_sceneList.get(good_matches.get(i).trainIdx).pt);
}
MatOfPoint2f obj = new MatOfPoint2f();
obj.fromList(objList);
MatOfPoint2f scene = new MatOfPoint2f();
scene.fromList(sceneList);
Mat homography = Calib3d.findHomography(obj, scene);
Mat obj_corners = new Mat(4,1,CvType.CV_32FC2);
Mat scene_corners = new Mat(4,1,CvType.CV_32FC2);
obj_corners.put(0, 0, new double[] {0,0});
obj_corners.put(1, 0, new double[] {img_object.cols(),0});
obj_corners.put(2, 0, new double[] {img_object.cols(),img_object.rows()});
obj_corners.put(3, 0, new double[] {0,img_object.rows()});
//Compute the most probable perspective transformation
//out of several pairs of corresponding points.
//Imgproc.getPerspectiveTransform(obj_corners, scene_corners);
Core.perspectiveTransform(obj_corners,scene_corners, homography);
Core.line(img_matches, new Point(scene_corners.get(0,0)), new Point(scene_corners.get(1,0)), new Scalar(0, 255, 0),4);
Core.line(img_matches, new Point(scene_corners.get(1,0)), new Point(scene_corners.get(2,0)), new Scalar(0, 255, 0),4);
Core.line(img_matches, new Point(scene_corners.get(2,0)), new Point(scene_corners.get(3,0)), new Scalar(0, 255, 0),4);
Core.line(img_matches, new Point(scene_corners.get(3,0)), new Point(scene_corners.get(0,0)), new Scalar(0, 255, 0),4);
Highgui.imwrite("./resources/ImageMatching" + ".jpg", img_matches);
createWindow("Image Matching", "resources/ImageMatching.jpg");
}
else
{
System.out.println("Not enough Matches");
System.exit(0);
}
}
The coordinates are in the correct place, you're simply drawing on the wrong image.
Your coordinates are relative to the second image img_scene . So if you draw your lines only on that image they'll be correct.
If you want to draw the lines on a composed image, where img_scene is translated to the right by the width of the first image img_object, you simply need to add img_object.cols() to the points x coordinates.
For example:
Core.line(img_matches,
new Point(scene_corners.get(0,0)[0] + img_object.cols(), scene_corners.get(0,0)[1]),
new Point(scene_corners.get(1,0)[0] + img_object.cols(), scene_corners.get(1,0)[1]),
new Scalar(0, 255, 0),4);
for the first line, and the same for next 3 lines.