I have a method in Python that makes use of OpenCV to remove the background from an image. I want the same functionality to work with android's version of OpenCV but I just cant seem to wrap my head around how the arrays work and how I can process them.
This is what I have so far in Java :
private Bitmap GetForeground(Bitmap source){
source = scale(source,300,300);
Mat mask = Mat.zeros(source.getHeight(),source.getWidth(),CvType.CV_8U);
Mat bgModel = Mat.zeros(1,65,CvType.CV_64F);
Mat ftModel = Mat.zeros(1,65,CvType.CV_64F);
int x = (int)Math.round(source.getWidth()*0.1);
int y = (int)Math.round(source.getHeight()*0.1);
int width = (int)Math.round(source.getWidth()*0.8);
int height = (int)Math.round(source.getHeight()*0.8);
Rect rect = new Rect(x,y, width,height);
Mat sourceMat = new Mat();
Utils.bitmapToMat(source, sourceMat);
Imgproc.grabCut(sourceMat, mask, rect, bgModel, ftModel, 5, Imgproc.GC_INIT_WITH_RECT);
int frameSize=sourceMat.rows()*sourceMat.cols();
byte[] buffer= new byte[frameSize];
mask.get(0,0,buffer);
for (int i = 0; i < frameSize; i++) {
if (buffer[i] == 2 || buffer[i] == 0){
buffer[i] = 0;
}else{
buffer[i] = 1 ;
}
}
byte[][] sourceArray = getMultiChannelArray(sourceMat);
byte[][][] reshapedMask = ReshapeArray(buffer, sourceMat.rows(), sourceMat.cols());
return source;
}
private byte[][][] ReshapeArray(byte[] arr, int rows, int cols){
byte[][][] out = new byte[cols][rows][1];
int index=0;
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
out[i][j][0] = arr[index];
index++;
}
}
return out;
}
public static byte[][] getMultiChannelArray(Mat m) {
//first index is pixel, second index is channel
int numChannels=m.channels();//is 3 for 8UC3 (e.g. RGB)
int frameSize=m.rows()*m.cols();
byte[] byteBuffer= new byte[frameSize*numChannels];
m.get(0,0,byteBuffer);
//write to separate R,G,B arrays
byte[][] out=new byte[frameSize][numChannels];
for (int p=0,i = 0; p < frameSize; p++) {
for (int n = 0; n < numChannels; n++,i++) {
out[p][n]=byteBuffer[i];
}
}
return out;
}
The python code I want to recreate :
image = cv2.imread('Images/handheld.jpg')
image = imutils.resize(image, height = 300)
mask = np.zeros(image.shape[:2],np.uint8)
bgModel = np.zeros((1,65),np.float64)
frModel = np.zeros((1,65),np.float64)
height, width, d = np.array(image).shape
rect = (int(width*0.1),int(height*0.1),int(width*0.8),int(height*0.8))
cv2.grabCut(image, mask, rect, bgModel,frModel, 5,cv2.GC_INIT_WITH_RECT)
mask = np.where((mask==2) | (mask == 0),0,1).astype('uint8')
image = image*mask[:,:,np.newaxis]
I have no idea how to convert the last two lines of the python code. If there is a way to just run python clean on an android device within my own project that would also be awesome.
At this point, you should consider talking a look to SL4A project which would allow you run your Python code on Android through java app.
Here are interesting links :
https://github.com/damonkohler/sl4a
https://norwied.wordpress.com/2012/04/11/run-sl4a-python-script-from-within-android-app/
http://jokar-johnk.blogspot.com/2011/02/how-to-make-android-app-with-sl4a.html
Let's see both the commands and try to convert them to Java API calls. It may not be simple 2 line in code.
mask = np.where((mask==2) | (mask == 0),0,1).astype('uint8')
In the above command, we are creating a new image mask which has uint data type of pixel values. The new mask matrix would have value 0 for every position where previous mask has a value of either 2 or 0, otherwise 1. Let's demonstrate this with an example:
mask = [
[0, 1, 1, 2],
[1, 0, 1, 3],
[0, 1, 1, 2],
[2, 3, 1, 0],
]
After this operation the output would be:
mask = [
[0, 1, 1, 0],
[1, 0, 1, 1],
[0, 1, 1, 0],
[0, 1, 1, 0],
]
So this above command is simply generating a binary mask with only 0 and 1 values. This can replicated in Java using Core.compare() method as:
// Get a mask for all `1` values in matrix.
Mat mask1vals;
Core.compare(mask, new Scalar(1), mask1vals, Core.CMP_EQ);
// Get a mask for all `3` values in matrix.
Mat mask3vals;
Core.compare(mask, new Scalar(3), mask3vals, Core.CMP_EQ);
// Create a combined mask
Mat foregroundMask;
Core.max(mask1vals, mask3vals, foregroundMask)
Now you need to multiply this foreground mask with the input image, to get final grabcut image as:
// First convert the single channel mat to 3 channel mat
Imgproc.cvtColor(foregroundMask, foregroundMask, Imgproc.COLOR_GRAY2BGR);
// Now simply take min operation
Mat out;
Core.min(foregroundMask, image, out);
Related
My output of Sobel edge detection operator is weird.
Here's my code :
BufferedImage temp = img;
float kernelx[][] = {{-1, 0, 1},{-2, 0, 2},{-1, 0, 1}};
float kernely[][] = {{-1, -2, -1},{0, 0, 0},{1, 2, 1}};
float valx = 0;
float valy = 0;
float val = 0;
for(int i=1;i<width-2;i++) {
for(int j=1;j<height-2;j++) {
valx = (kernelx[0][0]*new Color(img.getRGB(i-1, j-1)).getRed()) + (kernelx[0][2]*new Color(img.getRGB(i+1, j-1)).getRed()) +
(kernelx[1][0]*new Color(img.getRGB(i-1, j)).getRed()) + (kernelx[1][2]*new Color(img.getRGB(i+1, j)).getRed()) +
(kernelx[2][0]*new Color(img.getRGB(i-1, j+1)).getRed()) + (kernelx[2][2]*new Color(img.getRGB(i+1, j+1)).getRed());
valy = (kernely[0][0]*new Color(img.getRGB(i-1, j-1)).getRed()) + (kernely[0][1]*new Color(img.getRGB(i, j-1)).getRed()) + (kernely[0][2]*new Color(img.getRGB(i+1, j-1)).getRed()) +
(kernely[2][0]*new Color(img.getRGB(i-1, j+1)).getRed()) + (kernely[2][1]*new Color(img.getRGB(i, j+1)).getRed()) + (kernely[2][2]*new Color(img.getRGB(i+1, j+1)).getRed());
val = (float)sqrt(valx*valx+valy*valy);
val = val/1443*255;
if(val <= 127) {
val = 0;
} else {
val = 255;
}
temp.setRGB(i, j, new Color((int)val,(int)val,(int)val).getRGB());
}
File outputfile = new File("src/image/edge.png");
ImageIO.write(temp, "png", outputfile);
}
Anything wrong about my code? Please help me.
And this is the picture of the result.
Original Image:
Result Image:
There are different issues in your code:
When using a kernel of dimensions 3x3, you go from [1,1] to ]width-1,height-1[, not ]width-2, height-2[.
In java, use image.getRaster().getSample(x, y, channel), instead of 'new Color(img.getRGB(i-1, j-1)).getRed())'. It's going to be much faster and easier to understand. Same thing when writing image.getRaster().setSample(x, y, channel, value)
When computing a Sobel gradient on an image encoded on [0,max], each direction (X and Y) is going to give you values on [-4*max, 4*max]. So it's aggressive to cut the exceeded values. You may want to perform a histogram stretching instead, then you will keep much more information.
The normalization val = val/1443*255; is up to you, but not necessary.
And finally the main problem into your code. The resulting (or temp in your case) image and the original image MUST be different. Else you modify the image at the same time you process it. That explain why your have all this huge white area.
In OpenCV, is there a fast way to create a Mat object where:
odd columns are '1'
even columns are '0'
For example :
1 0 1 0 1 0
1 0 1 0 1 0
1 0 1 0 1 0
The pattern is always the same.
The size of Mat can be big, and process by looping is really slow to generate this pattern.
OpenCV repeat is there exactly for this.
#include <opencv2\opencv.hpp>
#include <vector>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
int rows = 1000;
int cols = 1000;
vector<uchar> pattern = { 1, 0 }; // change with int, double, etc according to the type you want.
Mat m;
repeat(pattern, rows, cols/2, m);
return 0;
}
COMPARISON WITH OTHER METHODS
Just a small test to measure the performance of the proposed (so far) methods:
Time in milliseconds:
#Miki [repeat] : 0.442786
#RonaldoMessi [copyTo] : 7.26822
#Derman [merge] : 1.17588
The code I used for the test:
#include <opencv2\opencv.hpp>
#include <vector>
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
int rows = 1000;
int cols = 1000;
{
// #Miki
double tic = (double)getTickCount();
vector<uchar> pattern = { 1, 0 };
Mat m1;
repeat(pattern, rows, cols / 2, m1);
double toc = ((double)getTickCount() - tic) * 1000 / getTickFrequency();
cout << "#Miki [repeat] \t\t: " << toc << endl;
}
{
// #RonaldoMessi
double tic = (double)getTickCount();
Mat m2(rows, cols, CV_8UC1);
Mat vZeros = Mat::zeros(rows, 1, CV_8UC1);
Mat vOnes = Mat::ones(rows, 1, CV_8UC1);
for (int i = 0; i < cols - 1; i += 2)
{
vOnes.col(0).copyTo(m2.col(i));
vZeros.col(0).copyTo(m2.col(i + 1));
}
double toc = ((double)getTickCount() - tic) * 1000 / getTickFrequency();
cout << "#RonaldoMessi [copyTo] \t: " << toc << endl;
}
{
// #Derman
// NOTE: corrected to give correct output
double tic = (double)getTickCount();
Mat myMat[2];
myMat[0] = cv::Mat::ones(rows, cols/2, CV_8UC1);
myMat[1] = cv::Mat::zeros(rows, cols/2, CV_8UC1);
Mat m3;
merge(myMat, 2, m3);
m3 = m3.reshape(1);
double toc = ((double)getTickCount() - tic) * 1000 / getTickFrequency();
cout << "#Derman [merge] \t: " << toc << endl;
}
getchar();
return 0;
}
You can create two column vectors vZeros and vOnes, then copy these columns to the matrix M:
int cols = A.cols;
int rows = A.rows;
Mat vZeros = Mat::zeros(rows , 1, CV_64F);
Mat vOnes = Mat::ones(rows , 1, CV_64F);
for(int i=0; i<cols-1; i+=2)
{
vOnes.col( 0 ).copyTo( M.col(i) );
vZeros.col( 0 ).copyTo( M.col(i+1) );
}
If two-channel matrix won't bother you, this could be your choice:
int rows = 5;
int cols = 5;
cv::Mat myMat[2];
myMat[0] = cv::Mat::ones(rows, cols, CV_32FC1);
myMat[1] = cv::Mat::zeros(rows, cols, CV_32FC1);
cv::Mat result;
cv::merge(myMat, 2, result);
And this is your result:
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0;
1, 0, 1, 0, 1, 0, 1, 0, 1, 0;
1, 0, 1, 0, 1, 0, 1, 0, 1, 0;
1, 0, 1, 0, 1, 0, 1, 0, 1, 0;
1, 0, 1, 0, 1, 0, 1, 0, 1, 0]
I am working with OpenCV 3.0 for Android. I have an image in which i want to detect angle of hands inside circular dials. for that i am working on HoughLinesP to detect hands.
Here is the code.
Mat imgSource = new Mat(), imgCirclesOut = new Mat(),imgLinesOut=new Mat();
//grey opencv
Imgproc.cvtColor(Image, imgSource, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur( imgSource, imgSource, new Size(9, 9), 2, 2 );
int threshold = 0;
int minLineSize = 0;
int lineGap = 0;
Imgproc.HoughLinesP(imgSource, imgLinesOut, 1, Math.PI/180, threshold, minLineSize, lineGap);
for( int j = 0; i < imgLinesOut.cols(); i++ )
{
double[] vec=imgLinesOut.get(0,j);
Point pt1, pt2;
pt1=new Point(vec[0],vec[1]);
pt2=new Point(vec[2],vec[3]);
Imgproc.line( Image, pt1, pt2, new Scalar(0,0,255), 3, Core.LINE_AA,0);
}
But result is
What i need is the angle of hands in these circles. Any help regarding this issue is highly appreciated. Thanks in ADvance
Edit
I have updated my code with this
Mat imgSource = new Mat(), imgCirclesOut = new Mat(),imgLinesOut=new Mat();
Imgproc.GaussianBlur( Image, imgSource, new Size(5, 5), 2, 2 );
int threshold = 20;
int minLineSize = 0;
int lineGap = 10;
Imgproc.Canny(imgSource, imgSource, 70, 100);
Imgproc.HoughLinesP(imgSource, imgLinesOut, 1, Math.PI/180, threshold, minLineSize, lineGap);
for( int j = 0; j < imgLinesOut.cols(); j++ )
{
double[] vec=imgLinesOut.get(0,j);
Point pt1, pt2;
pt1=new Point(vec[0],vec[1]);
pt2=new Point(vec[2],vec[3]);
Imgproc.line( imgSource, pt1, pt2, new Scalar(0,0,255), 3, Core.LINE_AA,0);
}
as suggested by #Micka, there is no need of Graying image(I removed cvtcolor). I also decreased value of GuassianBlur Size to 5. I have added Canny on image too for edges.
Resulting blur image is
Detecting lines can be a problem in such small images, since you have to few points to fill the Hough accumulator properly.
I propose to use a different approach:
Segment each circle (dial)
Extract the largest dark blob (hand)
Below is a simple implementation of this idea. The code is in C++, but you can easily port to Java, or at least use as a reference.
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int, char**)
{
Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);
Mat3b res;
cvtColor(img, res, COLOR_GRAY2BGR);
// Find dials
vector<Vec3f> circles;
HoughCircles(img, circles, CV_HOUGH_GRADIENT, 1, img.cols/10, 400, 40);
// For each dial
for (int i = 0; i < circles.size(); ++i)
{
// Segment the dial
Mat1b dial(img.size(), uchar(255));
Mat1b mask(img.size(), uchar(0));
circle(mask, Point(circles[i][0], circles[i][1]), circles[i][2], Scalar(255), CV_FILLED);
img.copyTo(dial, mask);
// Apply threshold and open
Mat1b bin;
threshold(dial, bin, 127, 255, THRESH_BINARY_INV);
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(5,5));
morphologyEx(bin, bin, MORPH_OPEN, kernel);
// Get min area rect
vector<Point> points;
findNonZero(bin, points);
RotatedRect r = minAreaRect(points);
// Draw min area rect
Point2f pts[4];
r.points(pts);
for (int j = 0; j < 4; ++j) {
line(res, pts[j], pts[(j + 1) % 4], Scalar(0, 255, 0), 1);
}
}
imshow("Result", res);
waitKey();
return 0;
}
Starting from this image:
I find hands here:
for( int j = 0; j < imgLinesOut.size(); j++ )
This will give the size of the vector.To iterate through that vector
This answer suggests that it's over 10 times faster to loop pixel array instead of using BufferedImage.getRGB. Such difference is too important to by ignored in my computer vision program. For that reason, O rewritten my IntegralImage method to calculate integral image using the pixel array:
/* Generate an integral image. Every pixel on such image contains sum of colors or all the
pixels before and itself.
*/
public static double[][][] integralImage(BufferedImage image) {
//Cache width and height in variables
int w = image.getWidth();
int h = image.getHeight();
//Create the 2D array as large as the image is
//Notice that I use [Y, X] coordinates to comply with the formula
double integral_image[][][] = new double[h][w][3];
//Variables for the image pixel array looping
final int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
//final byte[] pixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
//If the image has alpha, there will be 4 elements per pixel
final boolean hasAlpha = image.getAlphaRaster() != null;
final int pixel_size = hasAlpha?4:3;
//If there's alpha it's the first of 4 values, so we skip it
final int pixel_offset = hasAlpha?1:0;
//Coordinates, will be iterated too
//It's faster than calculating them using % and multiplication
int x=0;
int y=0;
int pixel = 0;
//Tmp storage for color
int[] color = new int[3];
//Loop through pixel array
for(int i=0, l=pixels.length; i<l; i+=pixel_size) {
//Prepare all the colors in advance
color[2] = ((int) pixels[pixel + pixel_offset] & 0xff); // blue;
color[1] = ((int) pixels[pixel + pixel_offset + 1] & 0xff); // green;
color[0] = ((int) pixels[pixel + pixel_offset + 2] & 0xff); // red;
//For every color, calculate the integrals
for(int j=0; j<3; j++) {
//Calculate the integral image field
double A = (x > 0 && y > 0) ? integral_image[y-1][x-1][j] : 0;
double B = (x > 0) ? integral_image[y][x-1][j] : 0;
double C = (y > 0) ? integral_image[y-1][x][j] : 0;
integral_image[y][x][j] = - A + B + C + color[j];
}
//Iterate coordinates
x++;
if(x>=w) {
x=0;
y++;
}
}
//Return the array
return integral_image;
}
The problem is that if I use this debug output in the for loop:
if(x==0) {
System.out.println("rgb["+pixels[pixel+pixel_offset+2]+", "+pixels[pixel+pixel_offset+1]+", "+pixels[pixel+pixel_offset]+"]");
System.out.println("rgb["+color[0]+", "+color[1]+", "+color[2]+"]");
}
This is what I get:
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
...
So how should I properly retrieve pixel array for BufferedImage images?
A bug in the code above, that is easily missed, is that the for loop doesn't loop as you'd expect. The for loop updates i, while the loop body uses pixel for its array indexing. Thus, you will only ever see the values of pixel 1, 2 and 3.
Apart from that:
The "problem" with the negative pixel values, is most likely that the code assumes a BufferedImage that stores its pixels in "pixel interleaved" form, however, they are stored "pixel packed". That is, all samples (R, G, B and A) for one pixel is stored in a single sample, an int. This will be the case for all BufferedImage.TYPE_INT_* types (while the BufferedImage.TYPE_nBYTE_* types are stored interleaved).
It's completely normal to have negative values in the raster, this will happen for any pixel that is less than 50% transparent (more than or equal to 50% opaque), because of how the 4 samples are packed into the int, and because int is a signed type in Java.
In this case, use:
int[] color = new int[3];
for (int i = 0; i < pixels.length; i++) {
// Assuming TYPE_INT_RGB, TYPE_INT_ARGB or TYPE_INT_ARGB_PRE
// For TYPE_INT_BGR, you need to reverse the colors.
// You seem to ignore alpha, is that correct?
color[0] = ((pixels[i] >> 16) & 0xff); // red;
color[1] = ((pixels[i] >> 8) & 0xff); // green;
color[2] = ( pixels[i] & 0xff); // blue;
// The rest of the computations...
}
Another possibility, is that you have created a custom type image (BufferedImage.TYPE_CUSTOM) that really uses a 32 bit unsigned int per sample. This is possible, however, int is still a signed entity in Java, so you need to mask off the sign bit. To complicate this a little, in Java -1 & 0xFFFFFFFF == -1 because any computation on an int will still be an int, unless you explicitly say otherwise (doing the same on a byte or short value would have "scaled up" to int). To get a positive value, you need to use a long value like this: -1 & 0xFFFFFFFFL (which is 4294967295).
In this case, use:
long[] color = new long[3];
for(int i = 0; i < pixels.length / pixel_size; i += pixel_size) {
// Somehow assuming BGR order in input, and RGB output (color)
// Still ignoring alpha
color[0] = (pixels[i + pixel_offset + 2] & 0xFFFFFFFFL); // red;
color[1] = (pixels[i + pixel_offset + 1] & 0xFFFFFFFFL); // green;
color[2] = (pixels[i + pixel_offset ] & 0xFFFFFFFFL); // blue;
// The rest of the computations...
}
I don't know what type of image you have, so I can't say for sure which one is the problem, but it's one of those. :-)
PS: BufferedImage.getAlphaRaster() is a possibly an expensive and also inaccurate way to tell if the image has alpha. It's better to just use image.getColorModel().hasAlpha(). See also hasAlpha vs getAlphaRaster.
hey i have using j2me to read an image
i want to do some process on that image like Darkenes , lightens
i already read image as an input stream
InputStream iStrm = getClass().getResourceAsStream("/earth.PNG");
ByteArrayOutputStream bStrm = new ByteArrayOutputStream();
int ch;
while ((ch = iStrm.read()) != -1){
bStrm.write(ch);
byte imageData[] = bStrm.toByteArray();
Image im = Image.createImage(imageData, 0, imageData.length);
how can i get RGB values or how can i add some values to the array of pixles
imageData[] so it can more lightens or darkness ,
is there header data including in the input stream i had read , that cause me error when iam adding some values to it ?
I think you should be able to do the following:
int width = im.getWidth();
int height = im.getHeight();
int[] rgbData = new int[width*height]; // since we are working with rgba
im.getRGB(rgbData, 0, width, 0, 0, width, height);
// now, the data is stored in each integer as 0xAARRGGBB,
// so high-order bits are alpha channel for each integer
Now, if you want to put them into three arrays, one for each channel, you could do the following:
int red[][] = new int[width][height];
int green[][] = new int[width][height];
int blue[][] = new int[width][height];
for (int i = 0; i < width; i++)
{
for (int j = 0; j < height; j++)
{
red[i][j] = rgb[i*width + j] & 0xFF0000;
green[i][j] = rgb[i*width + j] & 0xFF00;
blue[i][j] = rgb[i+width + j] & 0xFF;
}
}