As the title suggests, I am interested in getting the HSV value of a specific pixel using java CV. This sounds easy enough, and it seems to be straight forward in C++ or Python, but I simply cant figure out how to do it in Java. I am pretty new to OpenCV, and if I decide to do more projects using this library I will definitely write them in C++ or Python.
For reference, my goal is to do a color analysis of an object that has varying levels of lighting. The end goal is to be able to take an image of something like a t-shirt and be able to say "this t shirt is x% red".
Here is some of the code I was using. Surprisingly inRange() takes much longer than just looping through every pixel and getting RGB one by one. I want to be able to do exactly this, just with the HSV color space. If you know of a better way to accomplish this goal, please let me know as this has destroyed my entire Saturday. Thanks!
Scalar min = new Scalar(22,11,3);
Scalar max = new Scalar(103,87,74);
int sum = 0;
System.loadLibrary( Core.NATIVE_LIBRARY_NAME );
File input = new File("bluesample.jpg");
BufferedImage image = ImageIO.read(input);
byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
Mat mat1 = new Mat(image.getHeight(),image.getWidth(),CvType.CV_8UC3);
mat.put(0, 0, data);
Core.inRange(mat, min, max, mat1);
System.out.println(mat1.total());
System.out.println(mat1.total());
for (int i=0;i<mat1.rows(); i++ ){
for (int j=0;i<mat1.cols();j++){
sum += mat1.get(j, i, data);
}
}
System.out.println(sum/mat1.total());
EDIT:
try { System.loadLibrary( Core.NATIVE_LIBRARY_NAME );
File input = new File("singlehsvpix.jpg");
BufferedImage image = ImageIO.read(input);
byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, data);
Mat mat1 = new Mat(image.getHeight(),image.getWidth(),CvType.CV_8UC1);
Imgproc.cvtColor(mat, mat1, Imgproc.COLOR_RGB2HSV);
System.out.println(mat1.dump());
byte[] data1 = new byte[mat1.rows() * mat1.cols() * (int)(mat1.elemSize())];
mat1.get(0, 0, data1);
//BufferedImage image1 = new BufferedImage(mat1.cols(),mat1.rows(), BufferedImage.TYPE_BYTE_GRAY);
BufferedImage image1 = new BufferedImage(mat1.cols(),mat1.rows(), 5);
image1.getRaster().setDataElements(0, 0, mat1.cols(), mat1.rows(), data1);
File output = new File("PLS!.jpg");
ImageIO.write(image1, "jpg", output);
System.out.println(mat1.get(0, 0, data1)); // RELEVANT LINE
System.out.println("Done");
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
Is printing:
[ 54, 213, 193]
3
Done
For this pic, 54, 213, 193 are the BGR values... I guess I don't understand enough about OpenCV to know why my mat1.get is printing 3
So, you want to convert rgb to hsv.
Imgproc.cvtColor(im_rgb, im_hsv, Imgproc.COLOR_RGB2HSV);
Then, process as you like
Edit: in your code, change mat to mat1
for (int i=0;i<mat1.rows(); i++ ){
for (int j=0;i<mat1.cols();j++){
sum += mat.get(j, i, data); //this line
}
}
System.out.println(sum/mat1.total());
You are adding the value in original matrix.
Related
please help me with this as I am new to image processing.
I have the following images, there is glitterings/powder on the surface of the hand. How do I go about detecting these things on the hand?
I have tried with detecting by getting 70% of the max Intensity in the image. However, only image one works the rest does not. Could anyone how to suggest any methods that I can use to perform the detection please.(any available codes to try with will be good) Thank you?
Input Image 1: The only image that works with the above code
Input Image 2
Input Image 3
Desired Outcome
//convert from bitmap to mat
Mat mat = new Mat(bitmap1.getWidth(), bitmap1.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(bitmap1, mat);
Mat grayMat = new Mat();
Imgproc.cvtColor(mat, grayMat, Imgproc.COLOR_BGR2GRAY);
//Log.d("TAGG","intensity" + mat.dump());
int rows = mat.rows();
int cols = mat.cols();
double maxIntensity = Core.minMaxLoc(grayMat).maxVal;
double minIntensity = 0.7 * maxIntensity;
Log.d("TAGG", "intensity" + maxIntensity);
Mat thresholdMat = new Mat();
Imgproc.threshold(grayMat, thresholdMat, minIntensity, maxIntensity, Imgproc.THRESH_BINARY_INV);
Bitmap outputBitmap = Bitmap.createBitmap(thresholdMat.cols(), thresholdMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(thresholdMat, outputBitmap);
as the title says, how to convert bitmap RGB back to YUV byte[] using ScriptIntrinsicColorMatrix? below is the sample code (cannot be decoded by zxing):
public byte[] getYUVBytes(Bitmap src, boolean initOutAllocOnce){
if(!initOutAllocOnce){
outYUV = null;
}
if(outYUV == null){
outYUV = Allocation.createSized(rs, Element.U8(rs), src.getByteCount());
}
byte[] yuvData = new byte[src.getByteCount()];
Allocation in;
in = Allocation.createFromBitmap(rs, src,
Allocation.MipmapControl.MIPMAP_NONE,
Allocation.USAGE_SCRIPT);
scriptColor.setRGBtoYUV();
scriptColor.forEach(in, outYUV);
outYUV.copyTo(yuvData);
return yuvData;
}
one thing i notice is from original camera YUV byte is 3110400 bytes but after using the ScriptIntrinsicColorMatrix convertion in becomes 8294400, which i think is wrong.
Reason for YUV -> BW -> YUV is I want to convert the image into black and white (not grayscale) and back to YUV for zxing to decode at the same time show the black and white in surfaceview (like a custom camera filter).
tried below code but its a bit slow (can be decoded by zxing).
int[] intArray = null;
intArray = new int[bmp.getWidth() * bmp.getHeight()];
bmp.getPixels(intArray, 0, bmp.getWidth(), 0, 0, bmp.getWidth(), bmp.getHeight());
LuminanceSource source = new RGBLuminanceSource(cameraResolution.x, cameraResolution.y, intArray);
data = source.getMatrix();
any other alternative for RGB to YUV that is fast? if ScriptIntrinsicColorMatrix class cannot be done?
please and thank you
I'm new to JavaCv. My task is to find symbols on an image and generate pictures with single symbol on each other.
Firstly, I have this picture:
Then I do thresholding and get this:
I'm trying to use cvFindContours an then draw a rectangle around each symbol, my code:
String filename = "captcha.jpg";
IplImage firstImage=cvLoadImage(filename);
Mat src = imread(filename, CV_LOAD_IMAGE_GRAYSCALE);
Mat dst = new Mat();
threshold(src, dst, 200, 255, 0);
imwrite("out.jpg", dst);
IplImage iplImage=cvLoadImage("out.jpg",CV_8UC1);
CvMemStorage memStorage=CvMemStorage.create();
CvSeq contours=new CvSeq();
cvFindContours(iplImage,memStorage,contours,Loader.sizeof(CvContour.class),CV_RETR_CCOMP,CV_CHAIN_APPROX_SIMPLE,cvPoint(0,0));
CvSeq ptr;
CvRect bb;
for (ptr=contours;ptr!=null;ptr=ptr.h_next()){
bb=cvBoundingRect(ptr);
cvRectangle(firstImage , cvPoint( bb.x(), bb.y() ),
cvPoint( bb.x() + bb.width(), bb.y() + bb.height()),
CvScalar.BLACK, 2, 0, 0 );
}
cvSaveImage("result.jpg",firstImage);
}
I want to get output like this:, but really I get this:
Please, need your help.
You're using "out.jpg" image for findContour().
When you save dst Mat into "out.jpg", JPEG format automatically quantizes your original pixel data and creates noises to your image.
Save dst to "out.png" rather than "out.jpg", or use dst Mat directly into findContour().
Source Code ADDED: C++ version
string filename = "captcha.jpg";
Mat src = imread(filename);
Mat gray;
cvtColor(src, gray, CV_BGR2GRAY);
Mat thres;
threshold(gray, thres, 200, 255, 0);
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(thres.clone(), contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
Mat firstImage = src.clone();
for(int i=0; i< contours.sizes(); i++)
{
Rect r = boundingRect(contours[i]);
rectangle(firstImage, r, CV_RGB(255, 0, 0), 2);
}
imwrite("result.png", firstImage);
I recently started working with the Java bindings for OpenCV to make a quick and dirty project to do template matching. Basically I am trying to read a set of jpg images (saved in MS Paint) into Mats and then use template matching to find their locations from a screen shot taken with Java.Robot.
When it comes time to do the template matching this error is thrown
OpenCV Error: Assertion failed ((depth == CV_8U || depth == CV_32F)
&& type == _templ.type() && _img.dims() <= 2) in cv::matchTemplate
After searching it looks like the issue is that the two Mats I am trying to use do not have the same "type". What I am not sure of is what this refers to. I assume it is the Mats CvType, if I print out the CvType of the image and template I get a type() of 4 == CvType.CV_32SC1 for my template I get a type() of 20 == CvType.CV_32SC3.
But I feel like this is not the correct type() I am trying to compare, I have feeling it refers to the data type of how the data is stored in the Mat? But I have no good links to back this up just remembrances from many SO searches.
Here is my code for loading in my jpg images into a Mat
Mat pic_ = Imgcodecs.imread("MyPath\\image.jpg");
pic_.convertTo(pic_, CvType.CV_32SC1);
Here the second line turns my type() from 20 to 16, though as per my last comment I don't think this is the proper way to alter the Mat to match the image?... Because convertTo'ing this Mat to match the type of the screen shot `(below) does not fix the error?
Here is how I am creating the image Mat
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
So I first take a screenshot with Java.Robot.createScreenCapture I then convert it to a Mat with
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_INT_RGB);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_32SC1);
int[] data = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
From what I could tell the BufferedImage created by Robot is of type BufferedImage.TYPE_3BYTE_BGR which gives me an error "DataBufferInt cannot be cast to DataBufferByte" when trying to get the pixel data. So per the linked question I redraw the BufferedImage as type BufferedImage.TYPE_INT_RGB and pull out the data as a DataBufferInt.
So in all, should I be trying to match the Mat.type() or does my problem lie elsewhere? If not elsewhere how can I alter either of the Mats so that they can be used with Imgproc.matchTemplate properly?
I feel like the easiest solution would be to convert the image loaded from file to match the screenshot Mat?
EDIT: The exact section of code that gives the error is below
// Mat imageTemplate is a function argument; the loaded jpg image
// Take a picture of the screen
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
// Create the result matrix
int result_cols = screenImage.cols() - imageTemplate.cols() + 1;
int result_rows = screenImage.rows() - imageTemplate.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32SC1);
newStatus("ScreenType: " + screenImage.type());
newStatus("TemplaType: " + imageTemplate.type());
// Choose a matching method
int matchMethod = Imgproc.TM_SQDIFF_NORMED;
// Do the Matching and Normalize
Imgproc.matchTemplate(screenImage, imageTemplate, result, matchMethod);
// Error occurs on previous line
As #Miki pointed out in the comments the answer was getting the channe type to match for the image and template. I ended up changing my bufferedImageToMat function.
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
My templates are read in as CvType.CV_8UC3, so it was just a matter of creating a Mat from the screen image with this type!
I need a way to find an image on the screen. I've searched for ways to do this on SO but some take extremely long. I need it to be fast and efficient, does not need to be accurate. Basically i'm planning to compare or search for a small pixelated image, say 11x10 pixels for example, on the screen.
I also need a way to know the x and y coordinates of the small image on the screen.
Although I've looked through many tools out there like JavaCV and OpenCV, I just wanted to see if there are any other ways to do this.
TL;DR
I need a fast way to search for a small (11x10 example.) image on the screen and know its x,y coordinates.
I think you many find this answer relevant! But it is for Windows & in c++. But i'm sure that you can convert it very easily to any language.
This question is very old, But im trying to acheive the exact same thing here. Ive found that combining these answers would do the trick:
Convert BufferedImage TYPE_INT_RGB to OpenCV Mat Object
OpenCV Template Matching example in Android
The reason you need to do a conversion is because when u grab a screenshot with awt.Robot class its in the INT_RGB format. The matching template example expects bytes and you cannot grab byte data from this type of image directly.
Heres my implementation of these two answers, but it is incomplete. The output is all screwed up and i think it may have something to do with the IntBuffer/ByteBuffers.
-Edit-
I've added a new helper method that converts a INT_RGB to a BYTE_BGR. I can now grab the coordinates of template on the image using matchLoc.This seems to work pretty well, I was able to use this with a robot that clicks the start menu for me based on the template.
private BufferedImage FindTemplate() {
System.out.println("\nRunning Template Matching");
int match_method = Imgproc.TM_SQDIFF;
BufferedImage screenShot = null;
try {
Robot rob = new Robot();
screenShot = rob.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
} catch (AWTException ex) {
Logger.getLogger(MainGUI.class.getName()).log(Level.SEVERE, null, ex);
}
if(screenShot == null) return;
Mat img = BufferedImageToMat(convertIntRGBTo3ByteBGR(screenShot));
String templateFile = "C:\\Temp\\template1.JPG";
Mat templ = Highgui.imread(templateFile);
// / Create the result matrix
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
// / Do the Matching and Normalize
Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
Highgui.imwrite("out2.png", result);
// / Localizing the best match with minMaxLoc
MinMaxLocResult mmr = Core.minMaxLoc(result);
Point matchLoc;
if (match_method == Imgproc.TM_SQDIFF
|| match_method == Imgproc.TM_SQDIFF_NORMED) {
matchLoc = mmr.minLoc;
} else {
matchLoc = mmr.maxLoc;
}
Graphics2D graphics = screenShot.createGraphics();
graphics.setColor(Color.red);
graphics.setStroke(new BasicStroke(3));
graphics.drawRect(matchLoc.x, matchLoc.y, templ.width(), templ.height());
graphics.dispose();
return screenShot;
}
private Mat BufferedImageToMat(BufferedImage img){
int[] data = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
Mat mat = new Mat(img.getHeight(), img.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, byteBuffer.array());
return mat;
}`
private BufferedImage convertIntRGBTo3ByteBGR(BufferedImage img){
BufferedImage convertedImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = convertedImage.createGraphics();
graphics.drawImage(img, 0, 0, null);
graphics.dispose();
return convertedImage;
}
Results:
Template: