I've been trying to convert some opencv C++ code in opencv java and I can't seem to get pixel division to work properly. I take a meanshiftsegmented mat that I convert to grayscale then to 32F.
I then compare the most downsampled then upsampled image (which is comprised of the gray meanshift mat) to the original gray meanshift mat.
I've already read Using get() and put() to access pixel values in OpenCV for Java
however, it and others like it do not work. The error message I am getting is invalid mat type 5. However, even if I were able to see the saliency map I am positive it is wrong. This is because when I pass in image 001.jpg in c++ I am supposed to see the original image + red square around the objects. In java, I am only seeing the original image at the end.
NOTE :
AbstractImageProvider.deepCopy(AbstractImageProvider.matToBufferedImage(Saliency),disp);
Is an API call that works when I attempt to show the original mat, meanShift mat, and the gray meanShift mat. It fails at showing saliency.
c++
I only did a channel split because I was testing out other colorspaces, however in java I only want to use grayscale.
input = MeanShift.clone();
input.convertTo(input, CV_32F);
for(int i = 0; i < Pyramid_Size; i++){DS_Pyramid[i] = input.clone();}
for (int i = 0; i < Pyramid_Size; i++){
for (int k = 0; k <= i; k++){ // Why don't I just downsamplex3 a copy of MeanShift.clone then upsamplex3 that same one? ...
pyrDown (DS_Pyramid[i], DS_Pyramid[i], Size(DS_Pyramid[i].cols/2, DS_Pyramid[i].rows/2));
US_Pyramid[i] = DS_Pyramid[i].clone();
}
for (int j = 0; j <= i; j++){
pyrUp (US_Pyramid[i], US_Pyramid[i], Size(US_Pyramid[i].cols*2, US_Pyramid[i].rows*2));
}
}
top = US_Pyramid[Pyramid_Size - 1].clone(); // most down sampled layer, up sampled.
split(top, top_chs);
split(input.clone(), meanShift_chs); // split into channels result
split(input.clone(), sal_chs); // holder to use for compare
float top_min = 1.0;
float ms_min = 1.0;
for (int i = 0; i < top.rows; i++){ // find the smallest value in both top and meanShift
for (int k = 0; k < top.cols; k++){ // this is so you can sub out the 0 with the min value
for (int j = 0; j < top.channels(); j++){ // later on
float a = top_chs[j].at<float>(i,k);
float b = meanShift_chs[j].at<float>(i,k);
if (a < top_min && a >= 0) {top_min = a;} // make sure you don't have a top_min of zero... that'd be bad.
if (b < ms_min && b >= 0) { ms_min = b;}
}
}
}
for (int i = 0; i < top.rows; i++){
for (int k = 0; k < top.cols; k++){
for (int j = 0; j < top.channels(); j++){
float a,b,c;
a = top_chs[j].at<float>(i,k);
b = meanShift_chs[j].at<float>(i,k);
if (a <= 0){a = top_min;} // make sure you don't divide by zero
if (b <= 0){b = ms_min;} // make sure you really don't divide by zero
if (a <= b){c = 1.0 - a/b;}
else {c = 1.0 - b/a;}
// c = sqrt(c); // makes stuff more salient, but makes noise pop out too
sal_chs[j].at<float>(i,k) = c;
}
}
}
merge(sal_chs, Saliency); // combine into saliency map
imshow("saliency", Saliency);
java
MeanShift = inputImage.clone();
Imgproc.pyrMeanShiftFiltering(MeanShift, MeanShift, MeanShift_spatialRad, MeanShift_colorRad);
Imgproc.cvtColor(MeanShift, MeanShift, Imgproc.COLOR_BGR2GRAY);
MeanShift.convertTo(MeanShift, CvType.CV_32F); // 32F between 0 - 1. ************** IMPORTANT LINE
for (int i = 0; i < PyrSize; i++){
DS_Pyramid.add(new Mat());
UP_Pyramid.add(new Mat());
}
for (int i = 0; i < PyrSize; i++){
DS_Pyramid.set(i, MeanShift);
}
for (int i = 0; i < PyrSize; i++){
for(int k = 0; k <= i; k++){ // At 0 is downsampled once, second twice, third 3 times.
Imgproc.pyrDown(DS_Pyramid.get(i), DS_Pyramid.get(i)); // pyrDown by default img.width / 2 img height / 2
Mat a = new Mat(); // save the sampled down at i
a = DS_Pyramid.get(i);
UP_Pyramid.add(a);
}
for (int j = 0; j <= i; j++){
Imgproc.pyrUp(UP_Pyramid.get(i),UP_Pyramid.get(i));
}
}
top = UP_Pyramid.get(PyrSize-1);
bot = MeanShift.clone();
Saliency = MeanShift.clone();
//http://answers.opencv.org/question/5/how-to-get-and-modify-the-pixel-of-mat-in-java/
//http://www.tutorialspoint.com/java_dip/applying_weighted_average_filter.htm
for (int i = 0; i < top.rows(); i++){
for (int j = 0; j < top.cols(); j++){
int index = i * top.rows() + j;
float[] top_temp = top.get(i, j);
float[] bot_temp = bot.get(i,j);
float[] sal_temp = bot.get(i,j);
if (top_temp[0] <= bot_temp[k]){sal_temp[0] = 1.0f - (top_temp[0]/bot_temp[0]);}
else {sal_temp[0] = 1.0f - (bot_temp[0]/top_temp[0]);}
Saliency.put(i,j, sal_temp);
}
}
AbstractImageProvider.deepCopy(AbstractImageProvider.matToBufferedImage(Saliency),disp);
Found a simple and working solution after a lot of searching. This might help you get past the error- invalid mat type 5
Code:
Mat img = Highgui.imread("Input.jpg"); //Reads image from the file system and puts into matrix
int rows = img.rows(); //Calculates number of rows
int cols = img.cols(); //Calculates number of columns
int ch = img.channels(); //Calculates number of channels (Grayscale: 1, RGB: 3, etc.)
for (int i=0; i<rows; i++)
{
for (int j=0; j<cols; j++)
{
double[] data = img.get(i, j); //Stores element in an array
for (int k = 0; k < ch; k++) //Runs for the available number of channels
{
data[k] = data[k] * 2; //Pixel modification done here
}
img.put(i, j, data); //Puts element back into matrix
}
}
Highgui.imwrite("Output.jpg", img); //Writes image back to the file system using values of the modified matrix
Note: An important point that has not been mentioned anywhere online is that the method put does not write pixels onto Input.jpg. It merely updates the values of the matrix img. Therefore, the above code does not alter anything in the input image. For producing a visible output, the matrix img needs to be written onto a file i.e., Output.jpg in this case. Also, using img.get(i, j) seems to be a better way of handling the matrix elements rather than using the accepted solution above as this helps in visualizing and working with the image matrix in a better way and does not require a large contiguous memory allocation.
Related
So I've got a school project and we have to work with a couple classes our prof gave us and make our own to make an image organizer.
The first part consists of making a set of static methods to edit the images themselves as 2D arrays of Color arrays(ColorImage type).
The first first problem is making a tool to downscale an image by a factor of f(f sided square of pixels in the original becomes 1 pixel in the output), and mine works, but I think it shouldn't and I can't figure why it works, so any help is appreciated. Specifically I'm taking about the loop that averages the colours of each position in the buffer array(avgArr[][]) (line 16). I'm thinking: the value of reds blues and greens would just be overwritten for each iteration and avgColor would just get the vlaue of the last pixel it got the rgb values off of avgArr.
static ColorImage downscaleImg(ColorImage img, int f) {
ColorImage dsi = new ColorImage(img.getWidth()/f, img.getHeight()/f);
Color[][] avgArr = new Color[f][f];
int reds = 0;
int greens = 0;
int blues = 0;
for(int i = 0; i < dsi.getWidth(); i++) {
for(int j = 0; j < dsi.getHeight(); j++) {
for(int x = i*f, xc = 0; x < i*f + (f-1); x++, xc++){
for(int y = j*f, yc = 0; y < j*f + (f-1); y++, yc++) {
avgArr[xc][yc] = img.getColor(x, y);
}
}
for(int k = 0; k < f - 1; k++){
for(int w = 0; w < f - 1; w++) {
reds += avgArr[k][w].getR();
greens += avgArr[k][w].getG();
blues += avgArr[k][w].getB();
}
}
int count = f*f;
Color avgColor = new Color(reds/count, greens/count, blues/count);
dsi.setColor(i, j, avgColor);
reds = 0;
greens = 0;
blues = 0;
}
}
return dsi;
}
Thanks,
EDIT: Turns out, it was in fact just taking the colour of, the last position of avgArr that it looked at. Any suggestions to correct are welcome.
I think you can solve your problem by summing the reds/greens/blues and then dividing them by the total pixels at the end to find the average:
int reds = 0;
int greens = 0;
int blues = 0;
...
for(int k = 0; k < f - 1; k++){
for(int w = 0; w < f - 1; w++) {
reds += avgArr[k][w].getR(); // <-- note the +=
greens += avgArr[k][w].getG();
blues += avgArr[k][w].getB();
}
}
int count = (f-1)*(f-1);
Color avgColor = new Color(reds/count, greens/count, blues/count);
I want to memory-efficient this (the game of life code of shiffman in the nature of code book). how can change the below code to have only two arrays and constantly swap them, writing the next set of states into whichever one isn’t the current array?
class GOL {
int w = 8;
int columns, rows;
int[][] board;
GOL() {
// Initialize rows, columns and set-up arrays
columns = width / w;
rows = height / w;
board = new int[columns][rows];
//next = new int[columns][rows];
// Call function to fill array with random values 0 or 1
init();
}
void init() {
for (int i = 1; i < columns - 1; i++) {
for (int j = 1; j < rows - 1; j++) {
board[i][j] = (int) random(2);
}
}
}
// The process of creating the new generation
void generate() {
int[][] next = new int[columns][rows];
// Loop through every spot in our 2D array and check spots neighbors
for (int x = 1; x < columns - 1; x++) {
for (int y = 1; y < rows - 1; y++) {
// Add up all the states in a 3x3 surrounding grid
int neighbors = 0;
for (int i = -1; i <= 1; i++) {
for (int j = -1; j <= 1; j++) {
neighbors += board[x + i][y + j];
}
}
// A little trick to subtract the current cell's state since
// we added it in the above loop
neighbors -= board[x][y];
// Rules of Life
if ((board[x][y] == 1) && (neighbors < 2)) next[x][y] = 0;
else if ((board[x][y] == 1) && (neighbors > 3)) next[x][y] = 0;
else if ((board[x][y] == 0) && (neighbors == 3)) next[x][y] = 1;
else next[x][y] = board[x][y];
}
}
// Next is now our board
board = next;
}
// This is the easy part, just draw the cells, fill 255 for '1', fill 0 for '0'
void display() {
for (int i = 0; i < columns; i++) {
for (int j = 0; j < rows; j++) {
if ((board[i][j] == 1)) fill(0);
else fill(255);
stroke(0);
rect(i * w, j * w, w, w);
}
}
}
}
You might not like this, but the honest answer is: don't bother.
how can change the below code to have only two arrays and constantly swap them, writing the next set of states into whichever one isn’t the current array
This is already exactly what the code does.
The Game of Life requires two arrays. If you're coming up against real performance issues, then look for other areas of improvement. Focusing on the array is a red herring.
There's an old saying: premature optimization is the root of all evil. In other words, you shouldn't waste time trying to fix code before it's broken.
One obvious thing you might improve is: why are you using an int[] array instead of a boolean[] array? You only need to store two states: alive or dead, so using int values seems unnecessary. You'll save a little bit of memory if you switch to a boolean[] array, but again, you probably won't even notice the improvement.
public CompressImage(){
}
// compress image method
public static short[] compress(short image[][]){
// get image dimensions
int imageLength = image.length; // row length
int imageWidth = image[0].length; // column length
// convert vertical to horizontal
// store transposed Image
short[][] transposeImage = new short[imageWidth][imageLength];
// rotate by +90
for (int i = 0; i < imageWidth; i++)
{
for (int j = 0; j < imageLength; j++)
{
short temp = image[i][j];
transposeImage[i][j] = image[j][i];
transposeImage[j][i] = temp;
}
}
short temp = image[i][j];
transposeImage[i][j] = image[j][i];
transposeImage[j][i] = temp;
Why are you swapping here? That doesn't make sense - transposeImage is a new matrix, so you don't have to do inplace editing. This is guaranteed to break if imageWidth != imageLength - see if you can figure out why.
And, actually, you're not even swapping. The three lines above are equivalent to:
transposeImage[i][j] = image[j][i];
transposeImage[j][i] = image[i][j];
The body of the nested for loop should really just be:
transposeImage[i][j] = image[j][i];
A while I did an assignment creating a tictactoe program through eclipse. It works well enough, with me clicking empty boxes to place O's, and the program inputting X's afterward. However, I was using a pretty simple code for the placement of X's:
public int putX(){
for(int i=0; i<3;i++)
for(int j = 0;j<3;j++) {
if(position[i][j]==' ') {
position[i][j]='X';
return 0;
}
}
return -1; //some error occurred. This is odd. No cells were free.
}
Because of this, the X's are just placed in the row of each column, going down until the next column. Can someone show me a simple way to randomize this program?
What we want to do is generate an array of all the possible points, and pick one of those points at random. We use a for loop to iterate through all points in the 3x3 array, and add the valid ones to our temporary array, and then we choose a random index, and place an X there.
String[] list = new String[9]; // maximum 9 points
int size = 0;
for(int i = 0; i < 3; i++) {
for(int j = 0; j < 3; j++) {
if(position[i][j] == ' ') {
list[size] = "" + i + j;
size++;
}
}
}
int index = (int) (Math.random() * (size+1));
position[Integer.parseInt(list[index].charAt(0))][Integer.parseInt(list[index].charAt(1))] = 'X';
Alternatively, instead of storing the x,y coordinates of the point in a String we could store them in a java.awt.Point like so:
Point[] list = new Point[9]; // maximum 9 points
int size = 0;
for(int i = 0; i < 3; i++) {
for(int j = 0; j < 3; j++) {
if(position[i][j] == ' ') {
list[size] = new Point(i, j);
size++;
}
}
}
int index = (int) (Math.random() * (size+1));
position[list[index].getX()][list[index].getY()] = 'X';
As you can see, the code for using a Point is practically the same, but instead of parsing the coordinates out of the String, we can just access them directly from the Class.
You should also check to make sure that there are some elements left, by checking if size is still 0 after the for loop. If so, you should probably return -1 (what your existing code does). Otherwise, at the end of the whole code return 0.
I'm trying to use SplineInterpolator
and PolynomialSplineFunction to double a dataset. I think I'm fairly far along the path (I'm probably missing some exception handling):
SplineInterpolator splineInterp;
public double[] doubledArray(double[] y){
double[] yy = new double[y.length*2];
// make a double version of y w/ -1 for "null" values
for(int i = 0; i < yy.length; i++){
if(i%2 == 0)
yy[i] = y[i];
else if(i == yy.length-1)
yy[i] = yy[0];
else
yy[i] = -1;
}
// make a corresponding x array to satisfy SplineInterpolator.interpolate
double[] x = new double[y.length];
for(int i = 0; i < x.length; i++)
x[i] = i;
splineInterp = new SplineInterpolator();
PolynomialSplineFunction polySplineF = splineInterp.interpolate(x, y);
for(int i = 0; i < yy.length; i++){
if(yy[i] == -1){
yy[i] = polySplineF.value(i);
// breaks down halfway through polySplineF.value expects and array of y.length
}
}
return yy;
}
But the above is gonna crash in the last for loop at the latest. So, do I have the first part more or less right? After I have my polynomial spline function, how do I use that to create a larger dataset?
In case anybody is following along at home, here is the implementation I came up with for this:
private double[] multiplyArray(double[] y){
// An array 2 or 4 or N times bigger than the original:
double[] yy = new double[y.length*arrayMultiplier];
// An array representing the indices of the original:
double[] x = new double[y.length];
for(int i = 0; i < x.length; i++)
x[i] = i;
// Get and instance of SplineInterpolator:
SplineInterpolator splineInterp = new SplineInterpolator();
// Use that instance's interpolate() function to a PolynomialSplineFunction
// fitting your data, points y at indices x.
PolynomialSplineFunction polySplineF = splineInterp.interpolate(x, y);
// Use the PolynomialSplineFunction to fill in your larger array by supplying
// index values divided by the arrayMultiplier
for(int i = 0; i < yy.length; i++){
yy[i] = polySplineF.value((double)(i/arrayMultiplier));
}
return yy;
}
I also figured out how to do the probably more useful fill-in-blanks use if anybody needs it.