I have hand detection code right now that creates a green rectangle around the hand when detected. However, I want to fill the rectangle with another jpeg image. I was thinking of using ImagePattern where the image gets embedded inside the shape however that did not seem to work in my program.
Here is my code for creating the rectangle around the hand :
Imgproc.rectangle(frame, handsArray[i].tl(), handsArray[i].br(), new Scalar(0, 255, 0), 3);
The frame is a Mat and Imgproc is a opencv, javacv function.
Can someone please help me with this problem? I am struggling
Regards
Green Rectangle Around The Hand that needs to be filled
BufferedImage bi = null;
try {
bi = ImageIO.read(new File(trollFace));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Mat mat = new Mat(bi.getHeight(), bi.getWidth(), CvType.CV_8UC3);
byte[] data = ((DataBufferByte) bi.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
You can do it in these steps:
Submat the rectangle from your source image
Mat submat = src.submat(new Rect(200, 200, 100, 100));
Now copy the sencond image into the submat
src2.copyTo(submat);
Note: Before copying, insure the size of the second image is the same as the rectangle, if not you can use the resize() function of the Imgprc class to resize the Mat. And it is that.
Update: The below is the example.
Mat src = Imgcodecs.imread("C:\\src1.jpg");
Mat src2 = Imgcodecs.imread("C:\\src2.jpg");
Imgproc.resize(src2, src2, new Size(100, 100));
Mat submat = src.submat(new Rect(200, 200, 100, 100));
src2.copyTo(submat);
// now you can do what you want with the src1
Related
I create this image with g2d:
Here is the code:
BufferedImage bufferedImage = new BufferedImage(408, 408, BufferedImage.TYPE_INT_RGB);
Graphics2D g2d = bufferedImage.createGraphics();
List<Pixel> pixels = cacheRepo.findAll();
pixels.stream().forEach(pixel -> {
g2d.setColor(getColorFromPixel(pixel));
g2d.fillRect(getPos(pixel.getPosition().x), getPos(pixel.getPosition().y), 20, 20);
});
Now I am trying to rotate it 90° anticlok so that the bleu square appear on the lower left:
So I add this:
g2d.rotate(Math.toRadians(90));
g2d.drawRenderedImage(bufferedImage, null);
But the rotation doesn't occur (I still have the same image).
Here is the complete piece of code, with the part that save image:
// Constructs a BufferedImage of one of the predefined image types.
BufferedImage bufferedImage = new BufferedImage(408, 408, BufferedImage.TYPE_INT_RGB);
// Create a graphics which can be used to draw into the buffered image
Graphics2D g2d = bufferedImage.createGraphics();
List<Pixel> pixels = cacheRepo.findAll();
pixels.stream().forEach(pixel -> {
g2d.setColor(getColorFromPixel(pixel));
g2d.fillRect(getPos(pixel.getPosition().x), getPos(pixel.getPosition().y), 20, 20);
});
g2d.rotate(Math.toRadians(90));
g2d.drawRenderedImage(bufferedImage, null);
g2d.dispose();
// Save as PNG
File file = new File("myimage.png");
try {
ImageIO.write(bufferedImage, "png", file);
} catch (IOException e) {
e.printStackTrace();
}
Transformations should be applied BEFORE any operations you want to be effected by them, transformations won't affect anything that was done before it...
BufferedImage bufferedImage = new BufferedImage(408, 408, BufferedImage.TYPE_INT_RGB);
// Create a graphics which can be used to draw into the buffered image
Graphics2D g2d = bufferedImage.createGraphics();
g2d.rotate(Math.toRadians(90));
List<Pixel> pixels = cacheRepo.findAll();
pixels.stream().forEach(pixel -> {
g2d.setColor(getColorFromPixel(pixel));
g2d.fillRect(getPos(pixel.getPosition().x), getPos(pixel.getPosition().y), 20, 20);
});
//g2d.rotate(Math.toRadians(90));
// Not sure what you're hoping to achieve here
//g2d.drawRenderedImage(bufferedImage, null);
g2d.dispose();
If you prefer, use two BufferedImages. Render the "normal" content to the first, then use the second to paint the first, but with a rotation transformation ... because transformations do my head in 😝
using your code draw a black image
You probably need to supply a anchor point around which the image can be rotated, otherwise it will be rated about the top/left corner
And you'll forgive me, but it's not like this kind of think hasn't been asked before
Java rotate image turns part of background black
Rotate a buffered image in Java
Rotating Image with AffineTransform
Rotating an Image object
I'm rotating image in java but want to save rotated image
How to save rotated buffered image in another buffered image?
BufferedImage rotated, change resulting background
by using Canvas and JS I can draw a shape like this and have the x,y of each point :
Tha area can be choosen by more than 4 points, look at this link to have an idea.
I need to save and crop the image of the selected area by using the points. I can not use BufferedImage as it is just rectangular. Which lib in java I can use?
Okay, so starting with...
I used...
BufferedImage source = ImageIO.read(new File("Example.jpg"));
GeneralPath clip = new GeneralPath();
clip.moveTo(65, 123);
clip.lineTo(241, 178);
clip.lineTo(268, 405);
clip.lineTo(145, 512);
clip.closePath();
Rectangle bounds = clip.getBounds();
BufferedImage img = new BufferedImage(bounds.width, bounds.height, BufferedImage.TYPE_INT_ARGB);
Graphics2D g2d = img.createGraphics();
clip.transform(AffineTransform.getTranslateInstance(-65, -123));
g2d.setClip(clip);
g2d.translate(-65, -123);
g2d.drawImage(source, 0, 0, null);
g2d.dispose();
ImageIO.write(img, "png", new File("Clipped.png"));
to generate...
Now, the image is rectangular, that's just the way it works
Now, setClip is quite rough and isn't effect by any RenderingHints, you could make use of "soft clipping" instead, which is more involved, but generates a nicer results. See this example and this exmaple for more details
I recently started working with the Java bindings for OpenCV to make a quick and dirty project to do template matching. Basically I am trying to read a set of jpg images (saved in MS Paint) into Mats and then use template matching to find their locations from a screen shot taken with Java.Robot.
When it comes time to do the template matching this error is thrown
OpenCV Error: Assertion failed ((depth == CV_8U || depth == CV_32F)
&& type == _templ.type() && _img.dims() <= 2) in cv::matchTemplate
After searching it looks like the issue is that the two Mats I am trying to use do not have the same "type". What I am not sure of is what this refers to. I assume it is the Mats CvType, if I print out the CvType of the image and template I get a type() of 4 == CvType.CV_32SC1 for my template I get a type() of 20 == CvType.CV_32SC3.
But I feel like this is not the correct type() I am trying to compare, I have feeling it refers to the data type of how the data is stored in the Mat? But I have no good links to back this up just remembrances from many SO searches.
Here is my code for loading in my jpg images into a Mat
Mat pic_ = Imgcodecs.imread("MyPath\\image.jpg");
pic_.convertTo(pic_, CvType.CV_32SC1);
Here the second line turns my type() from 20 to 16, though as per my last comment I don't think this is the proper way to alter the Mat to match the image?... Because convertTo'ing this Mat to match the type of the screen shot `(below) does not fix the error?
Here is how I am creating the image Mat
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
So I first take a screenshot with Java.Robot.createScreenCapture I then convert it to a Mat with
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_INT_RGB);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_32SC1);
int[] data = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
From what I could tell the BufferedImage created by Robot is of type BufferedImage.TYPE_3BYTE_BGR which gives me an error "DataBufferInt cannot be cast to DataBufferByte" when trying to get the pixel data. So per the linked question I redraw the BufferedImage as type BufferedImage.TYPE_INT_RGB and pull out the data as a DataBufferInt.
So in all, should I be trying to match the Mat.type() or does my problem lie elsewhere? If not elsewhere how can I alter either of the Mats so that they can be used with Imgproc.matchTemplate properly?
I feel like the easiest solution would be to convert the image loaded from file to match the screenshot Mat?
EDIT: The exact section of code that gives the error is below
// Mat imageTemplate is a function argument; the loaded jpg image
// Take a picture of the screen
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);
// Create the result matrix
int result_cols = screenImage.cols() - imageTemplate.cols() + 1;
int result_rows = screenImage.rows() - imageTemplate.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32SC1);
newStatus("ScreenType: " + screenImage.type());
newStatus("TemplaType: " + imageTemplate.type());
// Choose a matching method
int matchMethod = Imgproc.TM_SQDIFF_NORMED;
// Do the Matching and Normalize
Imgproc.matchTemplate(screenImage, imageTemplate, result, matchMethod);
// Error occurs on previous line
As #Miki pointed out in the comments the answer was getting the channe type to match for the image and template. I ended up changing my bufferedImageToMat function.
private Mat bufferedImageToMat(BufferedImage inBuffImg)
{
BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
Graphics2D g2d= image.createGraphics();
g2d.drawImage(inBuffImg, 0, 0, null);
g2d.dispose();
Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
mat.put(0, 0, data);
return mat;
}
My templates are read in as CvType.CV_8UC3, so it was just a matter of creating a Mat from the screen image with this type!
I need a way to find an image on the screen. I've searched for ways to do this on SO but some take extremely long. I need it to be fast and efficient, does not need to be accurate. Basically i'm planning to compare or search for a small pixelated image, say 11x10 pixels for example, on the screen.
I also need a way to know the x and y coordinates of the small image on the screen.
Although I've looked through many tools out there like JavaCV and OpenCV, I just wanted to see if there are any other ways to do this.
TL;DR
I need a fast way to search for a small (11x10 example.) image on the screen and know its x,y coordinates.
I think you many find this answer relevant! But it is for Windows & in c++. But i'm sure that you can convert it very easily to any language.
This question is very old, But im trying to acheive the exact same thing here. Ive found that combining these answers would do the trick:
Convert BufferedImage TYPE_INT_RGB to OpenCV Mat Object
OpenCV Template Matching example in Android
The reason you need to do a conversion is because when u grab a screenshot with awt.Robot class its in the INT_RGB format. The matching template example expects bytes and you cannot grab byte data from this type of image directly.
Heres my implementation of these two answers, but it is incomplete. The output is all screwed up and i think it may have something to do with the IntBuffer/ByteBuffers.
-Edit-
I've added a new helper method that converts a INT_RGB to a BYTE_BGR. I can now grab the coordinates of template on the image using matchLoc.This seems to work pretty well, I was able to use this with a robot that clicks the start menu for me based on the template.
private BufferedImage FindTemplate() {
System.out.println("\nRunning Template Matching");
int match_method = Imgproc.TM_SQDIFF;
BufferedImage screenShot = null;
try {
Robot rob = new Robot();
screenShot = rob.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
} catch (AWTException ex) {
Logger.getLogger(MainGUI.class.getName()).log(Level.SEVERE, null, ex);
}
if(screenShot == null) return;
Mat img = BufferedImageToMat(convertIntRGBTo3ByteBGR(screenShot));
String templateFile = "C:\\Temp\\template1.JPG";
Mat templ = Highgui.imread(templateFile);
// / Create the result matrix
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
// / Do the Matching and Normalize
Imgproc.matchTemplate(img, templ, result, match_method);
Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
Highgui.imwrite("out2.png", result);
// / Localizing the best match with minMaxLoc
MinMaxLocResult mmr = Core.minMaxLoc(result);
Point matchLoc;
if (match_method == Imgproc.TM_SQDIFF
|| match_method == Imgproc.TM_SQDIFF_NORMED) {
matchLoc = mmr.minLoc;
} else {
matchLoc = mmr.maxLoc;
}
Graphics2D graphics = screenShot.createGraphics();
graphics.setColor(Color.red);
graphics.setStroke(new BasicStroke(3));
graphics.drawRect(matchLoc.x, matchLoc.y, templ.width(), templ.height());
graphics.dispose();
return screenShot;
}
private Mat BufferedImageToMat(BufferedImage img){
int[] data = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
Mat mat = new Mat(img.getHeight(), img.getWidth(), CvType.CV_8UC3);
mat.put(0, 0, byteBuffer.array());
return mat;
}`
private BufferedImage convertIntRGBTo3ByteBGR(BufferedImage img){
BufferedImage convertedImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = convertedImage.createGraphics();
graphics.drawImage(img, 0, 0, null);
graphics.dispose();
return convertedImage;
}
Results:
Template:
sorry if I am asking a silly question, I am trying to learn openCV using Java build. In opencv samples i found there is method imshow in c++ or showImage in python, but i search in opencv Java API, i cannot found imshow method.
What is the equal method of imshow() and waitKey() in opencv with Java build?
public static void showResult(Mat img) {
Imgproc.resize(img, img, new Size(640, 480));
MatOfByte matOfByte = new MatOfByte();
Highgui.imencode(".jpg", img, matOfByte);
byte[] byteArray = matOfByte.toArray();
BufferedImage bufImage = null;
try {
InputStream in = new ByteArrayInputStream(byteArray);
bufImage = ImageIO.read(in);
JFrame frame = new JFrame();
frame.setDefaultCloseOperation(DISPOSE_ON_CLOSE);
frame.getContentPane().add(new JLabel(new ImageIcon(bufImage)));
frame.pack();
frame.setVisible(true);
} catch (Exception e) {
e.printStackTrace();
}
}
There is no Java highgui wrapper yet (link). You should use swing or swt to show image. See: Opencv java - Load image to GUI
This is a readymade solution for Imshow() equivalent in Java OpenCV
Its simple to use .
API will look like:
Imshow im = new Imshow("Title");
im.showImage(matimage);
Visit here
https://github.com/master-atul/ImShow-Java-OpenCV
I made something with swing and awt.
First, get BufferedImage from Mat img by imencode and ImageIo.read
MatOfByte mMatOfByte = new MatOfByte()
imencode(".png", img, mMatOfByte);
byte[] byteArray = mMatOfByte.toArray();
InputStream in = new ByteArrayInputStream(byteArray);
BufferedImage srcImage = ImageIO.read(in);
Second, change it to target size width and height (w,h) and use Rendering Hint to get clean image in expanding or reducing case.
bufImage = new BufferedImage(w, h, srcImage.getType());
Graphics2D g = bufImage.createGraphics();
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
g.drawImage(srcImage, 0,0, w, h, 0, 0, srcImage.getWidth(), srcImage.getHeight(), null);
I used the target bufImage in JPanel's paintComponent() method.
For waitKey(), I used KeyEventDispatcher.
you have to use java swings or awt to show images..have a look at running code http://projectimageprocessing.blogspot.in/2013/05/introduction.html