Related
I want to make application that specified detects with line detection. First I made my application about really basic line detection. Below code is about part of line detection.
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame){
mRgba=inputFrame.rgba();
mGray=inputFrame.gray();
mat3=inputFrame.rgba();
// Code about ROI(Region of Interest)
double m_dWscale = (double) 5/6;
double m_dHscale = (double) 1/2;
double m_dWscale1 = (double) 4/6;
int mRoiWidth = (int)(mRgba.size().width * m_dWscale);
int mRoiHeight = (int)(mRgba.size().height * m_dHscale);
int mRoiWidth1 = (int)(mRgba.size().width * m_dWscale1);
int mRoiX = (int) (mRgba.size().width - mRoiWidth) ;
int mRoiY = (int) (mRgba.size().height - mRoiHeight) ;
roi_rect = new Rect(mRoiX,mRoiY,mRoiWidth1,mRoiHeight);
m_matRoi = mRgba.submat(roi_rect);
Imgproc.cvtColor(m_matRoi, m_matRoi, Imgproc.COLOR_BGR2GRAY, 1);
m_matRoi.copyTo(mRgba.submat(roi_rect));
//Code about BGR to HSV
scalarLow=new Scalar(0,0,200);
scalarHigh=new Scalar(180,255,255);
Imgproc.cvtColor(mRgba.submat(roi_rect),mat1,Imgproc.COLOR_BGR2HSV);
Core.inRange(mat1,scalarLow,scalarHigh,mat2);
Core.bitwise_and(mRgba.submat(roi_rect),mRgba.submat(roi_rect),mat1,mat2);
mRgba=mat3;
Imgproc.dilate(mat1,mat1,new Mat(),new Point(1,2),2);
Mat edges=new Mat();
//Code about Canny Edge
Imgproc.Canny(mat1,edges,90,150);
//Code about Hough transform
Mat lines=new Mat();
Point p1=new Point();
Point p2=new Point();
double a,b;
double x0,y0;
Imgproc.HoughLinesP(edges,lines,1.0,Math.PI/180.0,50,100.0,10.0);
for(int i=0;i<lines.rows();i++) {
double[] l = lines.get(i, 0);
Imgproc.line(mRgba.submat(roi_rect), new Point(l[0], l[1]), new Point(l[2], l[3]), new Scalar(0, 0, 255.0), 3);
}
//returns output
return mRgba;
}
And I want to combine upper code with tensorflow lite detection(see the below code please)
Code about tensorflow lite
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame){
mRgba=inputFrame.rgba();
mGray=inputFrame.gray();
Mat out=new Mat();
out= objectDetectorClass.recognizeImage(mRgba); //objectDetectorClass is another class which acts about object detection
return out;
}
So line detection code returns mRgba and object detection code returns out. I have no idea how to return both lines and object detection.
Thank you!
ps1) I forgot to upload github link about tensorflow lite, object detection application code. https://github.com/bendahouwael/Vehicle-Detection-App-Android. Hope this github helps you.
I solved the issue.
The answer was to return
//return mRgba;
Mat out=new Mat();
out= objectDetectorClass.recognizeImage(mRgba);
return out;
In my app I'm trying to do face recognition on a specific image using Open CV, here first I'm training one image and then after training that image if I run face recognition on that image it successfully recognizes that trained face. However, when I turn to another picture of the same person recognition does not work. It just works on the trained image, so my question is how do I rectify it?
Update:
What i want to do is that user should select image of a person from storage and then after training that selected image i want to fetch all images from storage which matches face of my trained image
Here is my activity class:
public class MainActivity extends AppCompatActivity {
private Mat rgba,gray;
private CascadeClassifier classifier;
private MatOfRect faces;
private ArrayList<Mat> images;
private ArrayList<String> imagesLabels;
private Storage local;
ImageView mimage;
Button prev,next;
ArrayList<Integer> imgs;
private int label[] = new int[1];
private double predict[] = new double[1];
Integer pos = 0;
private String[] uniqueLabels;
FaceRecognizer recognize;
private boolean trainfaces() {
if(images.isEmpty())
return false;
List<Mat> imagesMatrix = new ArrayList<>();
for (int i = 0; i < images.size(); i++)
imagesMatrix.add(images.get(i));
Set<String> uniqueLabelsSet = new HashSet<>(imagesLabels); // Get all unique labels
uniqueLabels = uniqueLabelsSet.toArray(new String[uniqueLabelsSet.size()]); // Convert to String array, so we can read the values from the indices
int[] classesNumbers = new int[uniqueLabels.length];
for (int i = 0; i < classesNumbers.length; i++)
classesNumbers[i] = i + 1; // Create incrementing list for each unique label starting at 1
int[] classes = new int[imagesLabels.size()];
for (int i = 0; i < imagesLabels.size(); i++) {
String label = imagesLabels.get(i);
for (int j = 0; j < uniqueLabels.length; j++) {
if (label.equals(uniqueLabels[j])) {
classes[i] = classesNumbers[j]; // Insert corresponding number
break;
}
}
}
Mat vectorClasses = new Mat(classes.length, 1, CvType.CV_32SC1); // CV_32S == int
vectorClasses.put(0, 0, classes); // Copy int array into a vector
recognize = LBPHFaceRecognizer.create(3,8,8,8,200);
recognize.train(imagesMatrix, vectorClasses);
if(SaveImage())
return true;
return false;
}
public void cropedImages(Mat mat) {
Rect rect_Crop=null;
for(Rect face: faces.toArray()) {
rect_Crop = new Rect(face.x, face.y, face.width, face.height);
}
Mat croped = new Mat(mat, rect_Crop);
images.add(croped);
}
public boolean SaveImage() {
File path = new File(Environment.getExternalStorageDirectory(), "TrainedData");
path.mkdirs();
String filename = "lbph_trained_data.xml";
File file = new File(path, filename);
recognize.save(file.toString());
if(file.exists())
return true;
return false;
}
private BaseLoaderCallback callbackLoader = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch(status) {
case BaseLoaderCallback.SUCCESS:
faces = new MatOfRect();
//reset
images = new ArrayList<Mat>();
imagesLabels = new ArrayList<String>();
local.putListMat("images", images);
local.putListString("imagesLabels", imagesLabels);
images = local.getListMat("images");
imagesLabels = local.getListString("imagesLabels");
break;
default:
super.onManagerConnected(status);
break;
}
}
};
#Override
protected void onResume() {
super.onResume();
if(OpenCVLoader.initDebug()) {
Log.i("hmm", "System Library Loaded Successfully");
callbackLoader.onManagerConnected(BaseLoaderCallback.SUCCESS);
} else {
Log.i("hmm", "Unable To Load System Library");
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION, this, callbackLoader);
}
}
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
prev = findViewById(R.id.btprev);
next = findViewById(R.id.btnext);
mimage = findViewById(R.id.mimage);
local = new Storage(this);
imgs = new ArrayList();
imgs.add(R.drawable.jonc);
imgs.add(R.drawable.jonc2);
imgs.add(R.drawable.randy1);
imgs.add(R.drawable.randy2);
imgs.add(R.drawable.imgone);
imgs.add(R.drawable.imagetwo);
mimage.setBackgroundResource(imgs.get(pos));
prev.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
if(pos!=0){
pos--;
mimage.setBackgroundResource(imgs.get(pos));
}
}
});
next.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
if(pos<5){
pos++;
mimage.setBackgroundResource(imgs.get(pos));
}
}
});
Button train = (Button)findViewById(R.id.btn_train);
train.setOnClickListener(new View.OnClickListener() {
#RequiresApi(api = Build.VERSION_CODES.KITKAT)
#Override
public void onClick(View view) {
rgba = new Mat();
gray = new Mat();
Mat mGrayTmp = new Mat();
Mat mRgbaTmp = new Mat();
classifier = FileUtils.loadXMLS(MainActivity.this);
Bitmap icon = BitmapFactory.decodeResource(getResources(),
imgs.get(pos));
Bitmap bmp32 = icon.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(bmp32, mGrayTmp);
Utils.bitmapToMat(bmp32, mRgbaTmp);
Imgproc.cvtColor(mGrayTmp, mGrayTmp, Imgproc.COLOR_BGR2GRAY);
Imgproc.cvtColor(mRgbaTmp, mRgbaTmp, Imgproc.COLOR_BGRA2RGBA);
/*Core.transpose(mGrayTmp, mGrayTmp); // Rotate image
Core.flip(mGrayTmp, mGrayTmp, -1); // Flip along both*/
gray = mGrayTmp;
rgba = mRgbaTmp;
Imgproc.resize(gray, gray, new Size(200,200.0f/ ((float)gray.width()/ (float)gray.height())));
if(gray.total() == 0)
Toast.makeText(getApplicationContext(), "Can't Detect Faces", Toast.LENGTH_SHORT).show();
classifier.detectMultiScale(gray,faces,1.1,3,0|CASCADE_SCALE_IMAGE, new Size(30,30));
if(!faces.empty()) {
if(faces.toArray().length > 1)
Toast.makeText(getApplicationContext(), "Mutliple Faces Are not allowed", Toast.LENGTH_SHORT).show();
else {
if(gray.total() == 0) {
Log.i("hmm", "Empty gray image");
return;
}
cropedImages(gray);
imagesLabels.add("Baby");
Toast.makeText(getApplicationContext(), "Picture Set As Baby", Toast.LENGTH_LONG).show();
if (images != null && imagesLabels != null) {
local.putListMat("images", images);
local.putListString("imagesLabels", imagesLabels);
Log.i("hmm", "Images have been saved");
if(trainfaces()) {
images.clear();
imagesLabels.clear();
}
}
}
}else {
/* Bitmap bmp = null;
Mat tmp = new Mat(250, 250, CvType.CV_8U, new Scalar(4));
try {
//Imgproc.cvtColor(seedsImage, tmp, Imgproc.COLOR_RGB2BGRA);
Imgproc.cvtColor(gray, tmp, Imgproc.COLOR_GRAY2RGBA, 4);
bmp = Bitmap.createBitmap(tmp.cols(), tmp.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(tmp, bmp);
} catch (CvException e) {
Log.d("Exception", e.getMessage());
}*/
/* mimage.setImageBitmap(bmp);*/
Toast.makeText(getApplicationContext(), "Unknown Face", Toast.LENGTH_SHORT).show();
}
}
});
Button recognize = (Button)findViewById(R.id.btn_recognize);
recognize.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
if(loadData())
Log.i("hmm", "Trained data loaded successfully");
rgba = new Mat();
gray = new Mat();
faces = new MatOfRect();
Mat mGrayTmp = new Mat();
Mat mRgbaTmp = new Mat();
classifier = FileUtils.loadXMLS(MainActivity.this);
Bitmap icon = BitmapFactory.decodeResource(getResources(),
imgs.get(pos));
Bitmap bmp32 = icon.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(bmp32, mGrayTmp);
Utils.bitmapToMat(bmp32, mRgbaTmp);
Imgproc.cvtColor(mGrayTmp, mGrayTmp, Imgproc.COLOR_BGR2GRAY);
Imgproc.cvtColor(mRgbaTmp, mRgbaTmp, Imgproc.COLOR_BGRA2RGBA);
/*Core.transpose(mGrayTmp, mGrayTmp); // Rotate image
Core.flip(mGrayTmp, mGrayTmp, -1); // Flip along both*/
gray = mGrayTmp;
rgba = mRgbaTmp;
Imgproc.resize(gray, gray, new Size(200,200.0f/ ((float)gray.width()/ (float)gray.height())));
if(gray.total() == 0)
Toast.makeText(getApplicationContext(), "Can't Detect Faces", Toast.LENGTH_SHORT).show();
classifier.detectMultiScale(gray,faces,1.1,3,0|CASCADE_SCALE_IMAGE, new Size(30,30));
if(!faces.empty()) {
if(faces.toArray().length > 1)
Toast.makeText(getApplicationContext(), "Mutliple Faces Are not allowed", Toast.LENGTH_SHORT).show();
else {
if(gray.total() == 0) {
Log.i("hmm", "Empty gray image");
return;
}
recognizeImage(gray);
}
}else {
Toast.makeText(getApplicationContext(), "Unknown Face", Toast.LENGTH_SHORT).show();
}
}
});
}
private void recognizeImage(Mat mat) {
Rect rect_Crop=null;
for(Rect face: faces.toArray()) {
rect_Crop = new Rect(face.x, face.y, face.width, face.height);
}
Mat croped = new Mat(mat, rect_Crop);
recognize.predict(croped, label, predict);
int indice = (int)predict[0];
Log.i("hmmcheck:",String.valueOf(label[0])+" : "+String.valueOf(indice));
if(label[0] != -1 && indice < 125)
Toast.makeText(getApplicationContext(), "Welcome "+uniqueLabels[label[0]-1]+"", Toast.LENGTH_SHORT).show();
else
Toast.makeText(getApplicationContext(), "You're not the right person", Toast.LENGTH_SHORT).show();
}
private boolean loadData() {
String filename = FileUtils.loadTrained();
if(filename.isEmpty())
return false;
else
{
recognize.read(filename);
return true;
}
}
}
My File Utils Class:
public class FileUtils {
private static String TAG = FileUtils.class.getSimpleName();
private static boolean loadFile(Context context, String cascadeName) {
InputStream inp = null;
OutputStream out = null;
boolean completed = false;
try {
inp = context.getResources().getAssets().open(cascadeName);
File outFile = new File(context.getCacheDir(), cascadeName);
out = new FileOutputStream(outFile);
byte[] buffer = new byte[4096];
int bytesread;
while((bytesread = inp.read(buffer)) != -1) {
out.write(buffer, 0, bytesread);
}
completed = true;
inp.close();
out.flush();
out.close();
} catch (IOException e) {
Log.i(TAG, "Unable to load cascade file" + e);
}
return completed;
}
public static CascadeClassifier loadXMLS(Activity activity) {
InputStream is = activity.getResources().openRawResource(R.raw.lbpcascade_frontalface);
File cascadeDir = activity.getDir("cascade", Context.MODE_PRIVATE);
File mCascadeFile = new File(cascadeDir, "lbpcascade_frontalface_improved.xml");
FileOutputStream os = null;
try {
os = new FileOutputStream(mCascadeFile);
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = is.read(buffer)) != -1) {
os.write(buffer, 0, bytesRead);
}
is.close();
os.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return new CascadeClassifier(mCascadeFile.getAbsolutePath());
}
public static String loadTrained() {
File file = new File(Environment.getExternalStorageDirectory(), "TrainedData/lbph_trained_data.xml");
return file.toString();
}
}
These are the images i'm trying to compare here face of person is same still in recognition it's not matching!
Update
According to the new edit in the question, you need a way to identify new people on the fly whose photos might not have been available during the training phase of the model. These tasks are called few shot learning. This is similar to the requirements of the intelligence/police agencies to find their targets using CCTV camera footage. As usually there are not enough images of a specific target, during training, they use models such as FaceNet. I really suggest reading the paper, however, I explain a few of its highlights here:
Generally, the last layer of a classifier is a n*1 vector with n-1 of
the elements almost equal to zero, and one close to 1. The element close to 1, determines the prediction of the classifier about the input's label.
The authors figured out that if they train a
classifier network with a specific loss function on a huge dataset of faces, you can use the semi-final layer output as a representation of any face, irrespective of it being in the training set or not, the authors call this vector Face Embedding.
The previous result means that with a very well trained FaceNet model, you can summarise any face into a vector. The very interesting attribute of this approach is that the vectors of a specific person's face in different angles/positions/states have are proximate in the euclidian space (this property is enforced by the loss function that the authors chose).
In summary, you have a model that gets faces as input and returns vectors. The vectors close to each other are very likely to belong to the same person (For checking that you can use KNN or just simple euclidian distance).
One implementation of FaceNet can be found here. I suggest you try to run it on your computer to get to know what you are actually dealing with. After that, it might be best to do the following:
Transform the FaceNet model mentioned in the repository to its
tflite version (this blogpost might help)
For each photo submitted by the user, use Face API to extract the face(s)
Use the minified model in your app to get the face embeddings of the extracted face.
Process all the images in the gallery of the user, getting the vectors for the faces in the photos.
Then compare each vector found in step4 with each vector found in step3 to get the matches.
Original Answer
You came across one of the most prevalent challenges of machine learning: Overfitting. Face detection and recognition is a huge area of research on its own and almost all the reasonably accurate models are using some kind of deep learning. Note that even detecting a face accurately is not as easy as it seems, however, as you are doing it on android, you can use Face API for this task. (Other more advanced techniques such as MTCNN are too slow/difficult to deploy on a handset). It has been shown that just feeding the model with a face photo with a lot of background noise or multiple people inside does not work. So, you really cannot skip this step.
After getting a nice trimmed face of the candidate targets from the background, you need to overcome the challenge of recognising the detected faces. Again, all the competent models to the best of my knowledge, are using some sort of deep learning/convolutional neural networks. Using them on a mobile phone is a challenge, but thanks to Tensorflow Lite you can minify them and run them within your app. A project about face recognition on android phones that I had worked on is here that you can check.
Keep in mind that any good model should be trained on numerous instances of labelled data, however there are a plethora of models already trained on large datasets of faces or other image recognition tasks, to tweak them and use their existing knowledge, we can employ transfer learning, for a quick start on object detection and transfer learning that is closely related to your case check this blog post.
Overall, you have to get numerous instances of the faces that you want to detect plus numerous face pics of people that you don't care about, then you need to train a model based on the above-mentioned resources, and then you need to use TensorFlow lite to decrease its size and embed it within your app. For each frame then, you call android Face API and feed (the probably detected face) into the model and identify the person.
Depending on your level of tolerance for delay and the number of training set size and number of targets, you can get various results, however, %90+ accuracy is easily achievable if you have only a few target people.
If I understand correctly, you're training the classifier with a single image. In that case, this one specific image is everything the classifier will be able to ever recognise. You would need a noticeably bigger training set of pictures showing the same person, something like 5 or 10 different images at the very least.
1) Change threshold value while initializing LBPHrecognizer to -> LBPHFaceRecognizer(1, 8, 8, 8, 100)
2) train each face with atleast 2-3 pictures since these recognizers mainly work on comparison
3) Set accuracy threshold while recognizing. Do something like this:
//predicting result
// LoadData is a static class that contains trained recognizer
// _result is the gray frame image captured by the camera
LBPHFaceRecognizer.PredictionResult ER = LoadData.recog.Predict(_result);
int temp_result = ER.Label;
imageBox1.SizeMode = PictureBoxSizeMode.StretchImage;
imageBox1.Image = _result.Mat;
//Displaying predicted result on screen
// LBPH returns -1 if face is recognized
if ((temp_result != -1) && (ER.Distance < 55)){
//I get best accuracy at 55, you should try different values to determine best results
// Do something with detected image
}
I'm new to OpenCV and I want to run a Java program for face detection using OpenCV.
Only including one haarcascade xml file doesn't give me expected results. So I need to run two,three haarcascade files in the same program. (specially "haarcascade_frontalface_alt.xml" and "haarcascade_profileface.xml" together).
I tried to do it with the following code but it didn't work. Please mention how to proceed.
Thank you.
public class LiveFeed extends WatchDogBaseFrame {
private DaemonThread myThread = null;
int count = 0;
VideoCapture webSource = null;
Mat frame = new Mat();
MatOfByte mem = new MatOfByte();
CascadeClassifier faceDetector1 = new CascadeClassifier("/home/erandi/NetBeansProjects/WatchDog/src/ueg/watchdog/view/haarcascade_frontalface_alt.xml");
CascadeClassifier faceDetector2 = new CascadeClassifier("/home/erandi/NetBeansProjects/WatchDog/src/ueg/watchdog/view/haarcascade_eye.xml");
MatOfRect faceDetections = new MatOfRect();
public LiveFeed(WatchDogBaseFrame parentFrame) {
super(parentFrame);
initComponents();
super.setCloseOperation();
jButtonExit.setVisible(false);
}
//class of demon thread
public class DaemonThread implements Runnable {
protected volatile boolean runnable = false;
#Override
public void run() {
synchronized (this) {
while (runnable) {
if (webSource.grab()) {
try {
webSource.retrieve(frame);
Graphics graphics = jPanelVideo.getGraphics();
faceDetector1.detectMultiScale(frame, faceDetections);
faceDetector2.detectMultiScale(frame, faceDetections);
for (Rect rect : faceDetections.toArray()) {
// System.out.println("ttt");
Imgproc.rectangle(frame, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height),
new Scalar(0, 255, 0));
}
Imgcodecs.imencode(".bmp", frame, mem);
Image im = ImageIO.read(new ByteArrayInputStream(mem.toArray()));
BufferedImage buff = (BufferedImage) im;
if (graphics.drawImage(buff, 0, 0, getWidth(), getHeight() - 150, 0, 0, buff.getWidth(), buff.getHeight(), null)) {
if (runnable == false) {
System.out.println("Paused ..... ");
this.wait();
}
}
} catch (Exception ex) {
System.out.println("Error");
}
}
}
}
}
}
Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection using a Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.
OpenCV already contains many pre-trained classifiers for face, eyes, smile etc. Those XML files are stored in opencv/data/haarcascades/ folder.
You can't run many cascade files simultaneously and increase performance. But you can use them one by one as a loop and pass input images through that loop.
Example code is given in this link: OpenCv sample code
I need a way to find an image on the screen. I've searched for ways to do this on SO but some take extremely long. I need it to be fast and efficient, does not need to be accurate. Basically i'm planning to compare or search for a small pixelated image, say 11x10 pixels for example, on the screen.
I also need a way to know the x and y coordinates of the small image on the screen.
Although I've looked through many tools out there like JavaCV and OpenCV, I just wanted to see if there are any other ways to do this.
TL;DR
I need a fast way to search for a small (11x10 example.) image on the screen and know its x,y coordinates.
I think you many find this answer relevant! But it is for Windows & in c++. But i'm sure that you can convert it very easily to any language.
This question is very old, But im trying to acheive the exact same thing here. Ive found that combining these answers would do the trick:
Convert BufferedImage TYPE_INT_RGB to OpenCV Mat Object
OpenCV Template Matching example in Android
The reason you need to do a conversion is because when u grab a screenshot with awt.Robot class its in the INT_RGB format. The matching template example expects bytes and you cannot grab byte data from this type of image directly.
Heres my implementation of these two answers, but it is incomplete. The output is all screwed up and i think it may have something to do with the IntBuffer/ByteBuffers.
-Edit-
I've added a new helper method that converts a INT_RGB to a BYTE_BGR. I can now grab the coordinates of template on the image using matchLoc.This seems to work pretty well, I was able to use this with a robot that clicks the start menu for me based on the template.
private BufferedImage FindTemplate() {
System.out.println("\nRunning Template Matching");
int match_method = Imgproc.TM_SQDIFF;
BufferedImage screenShot = null;
try {
Robot rob = new Robot();
screenShot = rob.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
} catch (AWTException ex) {
Logger.getLogger(MainGUI.class.getName()).log(Level.SEVERE, null, ex);
}
if(screenShot == null) return;
Mat img = BufferedImageToMat(convertIntRGBTo3ByteBGR(screenShot));
String templateFile = "C:\\Temp\\template1.JPG";
Mat templ = Highgui.imread(templateFile);
// / Create the result matrix
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
// / Do the Matching and Normalize
Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
Highgui.imwrite("out2.png", result);
// / Localizing the best match with minMaxLoc
MinMaxLocResult mmr = Core.minMaxLoc(result);
Point matchLoc;
if (match_method == Imgproc.TM_SQDIFF
|| match_method == Imgproc.TM_SQDIFF_NORMED) {
matchLoc = mmr.minLoc;
} else {
matchLoc = mmr.maxLoc;
}
Graphics2D graphics = screenShot.createGraphics();
graphics.setColor(Color.red);
graphics.setStroke(new BasicStroke(3));
graphics.drawRect(matchLoc.x, matchLoc.y, templ.width(), templ.height());
graphics.dispose();
return screenShot;
}
private Mat BufferedImageToMat(BufferedImage img){
int[] data = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
Mat mat = new Mat(img.getHeight(), img.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, byteBuffer.array());
return mat;
}`
private BufferedImage convertIntRGBTo3ByteBGR(BufferedImage img){
BufferedImage convertedImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = convertedImage.createGraphics();
graphics.drawImage(img, 0, 0, null);
graphics.dispose();
return convertedImage;
}
Results:
Template:
I am trying to implement a simple class that will allow a user to crop an image to be used for their profile picture. This is a java web application.
I have done some searching and found that java.awt has a BufferedImage class, and this appears (at first glance) to be perfect for what I need. However, it seems that there is a bug in this (or perhaps java, as I have seen suggested) that means that the cropping does not always work correctly.
Here is the code I am using to try to crop my image:
BufferedImage profileImage = getProfileImage(form, modelMap);
if (profileImage != null) {
BufferedImage croppedImage = profileImage
.getSubimage(form.getStartX(), form.getStartY(), form.getWidth(), form.getHeight());
System.err.println(form.getStartX());
System.err.println(form.getStartY());
File finalProfileImage = new File(form.getProfileImage());
try {
String imageType = getImageType(form.getProfileImage());
ImageIO.write(croppedImage, imageType, finalProfileImage);
}
catch (Exception e) {
logger.error("Unable to write cropped image", e);
}
}
return modelAndView;
}
protected BufferedImage getProfileImage(CropImageForm form, Map<String, Object> modelMap) {
String profileImageFileName = form.getProfileImage();
if (validImage(profileImageFileName) && imageExists(profileImageFileName)) {
BufferedImage image = null;
try {
image = getCroppableImage(form, ImageIO.read(new File(profileImageFileName)), modelMap);
}
catch (IOException e) {
logger.error("Unable to crop image, could not read profile image: [" + profileImageFileName + "]");
modelMap.put("errorMessage", "Unable to crop image. Please try again");
return null;
}
return image;
}
modelMap.put("errorMessage", "Unable to crop image. Please try again.");
return null;
}
private boolean imageExists(String profileImageFileName) {
return new File(profileImageFileName).exists();
}
private BufferedImage getCroppableImage(CropImageForm form, BufferedImage image, Map<String, Object> modelMap) {
int cropHeight = form.getHeight();
int cropWidth = form.getWidth();
if (cropHeight <= image.getHeight() && cropWidth <= image.getWidth()) {
return image;
}
modelMap.put("errorMessage", "Unable to crop image. Crop size larger than image.");
return null;
}
private boolean validImage(String profileImageFileName) {
String extension = getImageType(profileImageFileName);
return (extension.equals("jpg") || extension.equals("gif") || extension.equals("png"));
}
private String getImageType(String profileImageFileName) {
int indexOfSeparator = profileImageFileName.lastIndexOf(".");
return profileImageFileName.substring(indexOfSeparator + 1);
}
The form referred to in this code snippet is a simple POJO which contains integer values of the upper left corner to start cropping (startX and startY) and the width and height to make the new image.
What I end up with, however, is a cropped image that always starts at 0,0 rather than the startX and startY position. I have inspected the code to make sure the proper values are being passed in to the getSubimage method, and they appear to be.
Are there simple alternatives to using BufferedImage for cropping an image. I have taken a brief look at JAI. I would rather add a jar to my application than update the jdk installed on all of the production boxes, as well as any development/testing servers and local workstations.
My criteria for selecting an alternative are:
1) simple to use to crop an image as this is all I will be using it for
2) if not built into java or spring, the jar should be small and easily deployable in a web-app
Any suggestions?
Note: The comment above that there is an issue with bufferedImage or Java was something I saw in this posting: Guidance on the BufferedImage.getSubimage(int x, int y, int w, int h) method?
I have used getSubimage() numerous times before without any problems. Have you added a System.out.println(form.getStartX() + " " + form.getStartY()) before that call to make sure they're not both 0?
Also, are you at least getting an image that is form.getWidth() x form.getHeight()?
Do make sure you are not modifying/disposing profileImage in any way since the returned BufferedImage shares the same data array as the parent.
The best way is to just simply draw it across if you want a completely new and independent BufferedImage:
BufferedImage croppedImage = new BufferedImage(form.getWidth(),form.getHeight(),BufferedImage.TYPE_INT_ARGB);
Graphics g = croppedImage.getGraphics();
g.drawImage(profileImage,0,0,form.getWidth(),form.getHeight(),form.getStartX(),form.getStartY(),form.getWidth(),form.getHeight(),null);
g.dispose();
You can do it in this manner as well (code is not 100% tested as I adopted for example from an existing app i did):
import javax.imageio.*;
import java.awt.image.*;
import java.awt.geom.*;
...
BufferedImage img = ImageIO.read(imageStream);
...
/*
* w = image width, h = image height, l = crop left, t = crop top
*/
ColorModel dstCM = img.getColorModel();
BufferedImage dst = new BufferedImage(dstCM, dstCM.createCompatibleWritableRaster(w, h), dstCM.isAlphaPremultiplied(), null);
Graphics2D g = dst.createGraphics();
g.drawRenderedImage(img, AffineTransform.getTranslateInstance(-l,-t));
g.dispose();
java.io.File outputfile = new java.io.File(sessionScope.get('absolutePath') + java.io.File.separator + sessionScope.get('lastUpload'));
ImageIO.write(dst, 'png', outputfile);
Thanks for all who replied. It turns out that the problem was not in the cropping code at all.
When I displayed the image to be cropped, I resized it to fit into my layout nicely, then used a javascript cropping tool to figure out the coordinates to crop.
Since I had resized my image, but didn't take the resizing into account when I was determining the cropping coordinates, I ended up with coordinates that appeared to coincide with the top left corner.
I have changed the display to no longer resize the image, and now cropping is working beautifully.