I would like to use a quantized tensorflow lite model but the current ByteBuffer I have is using floating point. I would like this to be integer representation. Right now the model wants 270000 bytes and I am trying to pass it 1080000 bytes. Is it as simple as casting the float to int?
public ByteBuffer convertBitmapToByteBuffer(Bitmap bitmap) {
// Preallocate memory for bytebuffer
ByteBuffer byteBuffer = ByteBuffer.allocate(inputSize*inputSize*pixelSize);
byteBuffer.order(ByteOrder.nativeOrder());
// Initialize pixel data array and populate from bitmap
int [] intArray = new int[inputSize*inputSize];
bitmap.getPixels(intArray, 0, bitmap.getWidth(), 0 , 0,
bitmap.getWidth(), bitmap.getHeight());
int pixel = 0; // pixel indexer
for (int i=0; i<inputSize; i++) {
for (int j=0; j<inputSize; j++) {
int input = intArray[pixel++];
byteBuffer.putfloat((((input >> 16 & 0x000000FF) - imageMean) / imageStd));
byteBuffer.putfloat((((input >> 8 & 0x000000FF) - imageMean) / imageStd));
byteBuffer.putfloat((((input & 0x000000FF) - imageMean) / imageStd));
}
}
return byteBuffer;
}
Thanks for any tips you can provide.
Casting float to int is not the correct approach. Good news is that the quantized input values expected by the model (8-bit r, g, b values in sequence) matches exactly the same as the Bitmap pixel representation except that the model doesn't expect an alpha channel, so the conversion process should be actually easier than when you're using float inputs.
Here's what you might try instead. (I'm assuming pixelSize is 3)
int pixel = 0; // pixel indexer
for (int i=0; i<inputSize; i++) {
for (int j=0; j<inputSize; j++) {
int input = intArray[pixel++]; // pixel containing ARGB.
byteBuffer
.put((byte)((input >> 16) & 0xFF)) // R
.put((byte)((input >> 8) & 0xFF)) // G
.put((byte)((input ) & 0xFF)); // B
}
}
Related
I have an image with a lot of anti-aliased lines in it and trying to remove pixels that fall below a certain alpha channel threshold (and anything above the threshold gets converted to full 255 alpha). I've got this coded up and working, its just not as fast as I would like when running it on large images. Does anyone have an alternative method they could suggest?
//This will convert all pixels with > minAlpha to 255
public static void flattenImage(BufferedImage inSrcImg, int minAlpha)
{
//loop through all the pixels in the image
for (int y = 0; y < inSrcImg.getHeight(); y++)
{
for (int x = 0; x < inSrcImg.getWidth(); x++)
{
//get the current pixel (with alpha channel)
Color c = new Color(inSrcImg.getRGB(x,y), true);
//if the alpha value is above the threshold, convert it to full 255
if(c.getAlpha() >= minAlpha)
{
inSrcImg.setRGB(x,y, new Color(c.getRed(), c.getGreen(), c.getBlue(), 255).getRGB());
}
//otherwise set it to 0
else
{
inSrcImg.setRGB(x,y, new Color(0,0,0,0).getRGB()); //white (transparent)
}
}
}
}
per #BenoitCoudour 's comments I've modified the code accordingly, but it appears to be affecting the resulting RGB values of pixels, any idea what I might be doing wrong?
public static void flattenImage(BufferedImage src, int minAlpha)
{
int w = src.getWidth();
int h = src.getHeight();
int[] rgbArray = src.getRGB(0, 0, w, h, null, 0, w);
for (int i=0; i<w*h; i++)
{
int a = (rgbArray[i] >> 24) & 0xff;
int r = (rgbArray[i] >> 16) & 0xff;
int b = (rgbArray[i] >> 8) & 0xff;
int g = rgbArray[i] & 0xff;
if(a >= minAlpha) { rgbArray[i] = (255<<24) | (r<<16) | (g<<8) | b; }
else { rgbArray[i] = (0<<24) | (r<<16) | (g<<8) | b; }
}
src.setRGB(0, 0, w, h, rgbArray, 0, w);
}
What may slow you down is the instantiation of a Color object for every pixel.
Please see this answer to iterate over pixels in a BufferedImage and access the alpha channel : https://stackoverflow.com/a/6176783/3721907
I'll just paste the code below
public Image alpha2gray(BufferedImage src) {
if (src.getType() != BufferedImage.TYPE_INT_ARGB)
throw new RuntimeException("Wrong image type.");
int w = src.getWidth();
int h = src.getHeight();
int[] srcBuffer = src.getData().getPixels(0, 0, w, h, null);
int[] dstBuffer = new int[w * h];
for (int i=0; i<w*h; i++) {
int a = (srcBuffer[i] >> 24) & 0xff;
dstBuffer[i] = a | a << 8 | a << 16;
}
return Toolkit.getDefaultToolkit().createImage(new MemoryImageSource(w, h, pix, 0, w));
}
This is very close to what you want to achieve.
You have a theoretical complexity of O(n) which you optimize by performing byte manipulation.
You can go further and use threads (you have an embarrassing parallel problem), but since most of user machines have at most 8 physical threads it will not get you too far. You could add another level of optimization on top of this by manipulating parts of the image one at the time, adapted to the memory buffers and different cache levels in your system.
Since I already mentioned you have an embarrassing parallel problem, the best solution is to perform GPU programming.
You can follow this tutorial on simple image processing with cuda and change the code of the filter to something like this
void blur(unsigned char* input_image, unsigned char* output_image, int width, int height) {
const unsigned int offset = blockIdx.x*blockDim.x + threadIdx.x;
const int currentoffset = (offset)*4;
if(offset < width*height) {
if (input_image[currentoffset+3]>= threshold )
output_red = input_image[currentoffset];
output_green = input_image[currentoffset+1];
output_blue = input_image[currentoffset+2];
output_alpha = 255;
}else{
output_red = 0;
output_green = 0;
output_blue = 0;
output_alpha = 0;
}
}
}
output_image[currentoffset*3] = output_red;
output_image[currentoffset*3+1] = output_green;
output_image[currentoffset*3+2] = output_blue;
output_image[currentoffset*3+3] = output_alpha
}
}
If you are set on using Java you have here a great answer on how to get started on using java with nvidia gpu
This answer suggests that it's over 10 times faster to loop pixel array instead of using BufferedImage.getRGB. Such difference is too important to by ignored in my computer vision program. For that reason, O rewritten my IntegralImage method to calculate integral image using the pixel array:
/* Generate an integral image. Every pixel on such image contains sum of colors or all the
pixels before and itself.
*/
public static double[][][] integralImage(BufferedImage image) {
//Cache width and height in variables
int w = image.getWidth();
int h = image.getHeight();
//Create the 2D array as large as the image is
//Notice that I use [Y, X] coordinates to comply with the formula
double integral_image[][][] = new double[h][w][3];
//Variables for the image pixel array looping
final int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
//final byte[] pixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
//If the image has alpha, there will be 4 elements per pixel
final boolean hasAlpha = image.getAlphaRaster() != null;
final int pixel_size = hasAlpha?4:3;
//If there's alpha it's the first of 4 values, so we skip it
final int pixel_offset = hasAlpha?1:0;
//Coordinates, will be iterated too
//It's faster than calculating them using % and multiplication
int x=0;
int y=0;
int pixel = 0;
//Tmp storage for color
int[] color = new int[3];
//Loop through pixel array
for(int i=0, l=pixels.length; i<l; i+=pixel_size) {
//Prepare all the colors in advance
color[2] = ((int) pixels[pixel + pixel_offset] & 0xff); // blue;
color[1] = ((int) pixels[pixel + pixel_offset + 1] & 0xff); // green;
color[0] = ((int) pixels[pixel + pixel_offset + 2] & 0xff); // red;
//For every color, calculate the integrals
for(int j=0; j<3; j++) {
//Calculate the integral image field
double A = (x > 0 && y > 0) ? integral_image[y-1][x-1][j] : 0;
double B = (x > 0) ? integral_image[y][x-1][j] : 0;
double C = (y > 0) ? integral_image[y-1][x][j] : 0;
integral_image[y][x][j] = - A + B + C + color[j];
}
//Iterate coordinates
x++;
if(x>=w) {
x=0;
y++;
}
}
//Return the array
return integral_image;
}
The problem is that if I use this debug output in the for loop:
if(x==0) {
System.out.println("rgb["+pixels[pixel+pixel_offset+2]+", "+pixels[pixel+pixel_offset+1]+", "+pixels[pixel+pixel_offset]+"]");
System.out.println("rgb["+color[0]+", "+color[1]+", "+color[2]+"]");
}
This is what I get:
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
rgb[-16777216, -16777216, -16777216]
rgb[0, 0, 0]
...
So how should I properly retrieve pixel array for BufferedImage images?
A bug in the code above, that is easily missed, is that the for loop doesn't loop as you'd expect. The for loop updates i, while the loop body uses pixel for its array indexing. Thus, you will only ever see the values of pixel 1, 2 and 3.
Apart from that:
The "problem" with the negative pixel values, is most likely that the code assumes a BufferedImage that stores its pixels in "pixel interleaved" form, however, they are stored "pixel packed". That is, all samples (R, G, B and A) for one pixel is stored in a single sample, an int. This will be the case for all BufferedImage.TYPE_INT_* types (while the BufferedImage.TYPE_nBYTE_* types are stored interleaved).
It's completely normal to have negative values in the raster, this will happen for any pixel that is less than 50% transparent (more than or equal to 50% opaque), because of how the 4 samples are packed into the int, and because int is a signed type in Java.
In this case, use:
int[] color = new int[3];
for (int i = 0; i < pixels.length; i++) {
// Assuming TYPE_INT_RGB, TYPE_INT_ARGB or TYPE_INT_ARGB_PRE
// For TYPE_INT_BGR, you need to reverse the colors.
// You seem to ignore alpha, is that correct?
color[0] = ((pixels[i] >> 16) & 0xff); // red;
color[1] = ((pixels[i] >> 8) & 0xff); // green;
color[2] = ( pixels[i] & 0xff); // blue;
// The rest of the computations...
}
Another possibility, is that you have created a custom type image (BufferedImage.TYPE_CUSTOM) that really uses a 32 bit unsigned int per sample. This is possible, however, int is still a signed entity in Java, so you need to mask off the sign bit. To complicate this a little, in Java -1 & 0xFFFFFFFF == -1 because any computation on an int will still be an int, unless you explicitly say otherwise (doing the same on a byte or short value would have "scaled up" to int). To get a positive value, you need to use a long value like this: -1 & 0xFFFFFFFFL (which is 4294967295).
In this case, use:
long[] color = new long[3];
for(int i = 0; i < pixels.length / pixel_size; i += pixel_size) {
// Somehow assuming BGR order in input, and RGB output (color)
// Still ignoring alpha
color[0] = (pixels[i + pixel_offset + 2] & 0xFFFFFFFFL); // red;
color[1] = (pixels[i + pixel_offset + 1] & 0xFFFFFFFFL); // green;
color[2] = (pixels[i + pixel_offset ] & 0xFFFFFFFFL); // blue;
// The rest of the computations...
}
I don't know what type of image you have, so I can't say for sure which one is the problem, but it's one of those. :-)
PS: BufferedImage.getAlphaRaster() is a possibly an expensive and also inaccurate way to tell if the image has alpha. It's better to just use image.getColorModel().hasAlpha(). See also hasAlpha vs getAlphaRaster.
So far I have this:
BufferedImage image = ImageIO.read(
new URL("http://upload.wikimedia.org/wikipedia/en/2/24/Lenna.png"));
int w = image.getWidth();
int h = image.getHeight();
int[] dataBuffInt = image.getRGB(0, 0, w, h, null, 0, w);
Color c = new Color(dataBuffInt[100]);
System.out.println(c.getRed()); // = (dataBuffInt[100] >> 16) & 0xFF
System.out.println(c.getGreen()); // = (dataBuffInt[100] >> 8) & 0xFF
System.out.println(c.getBlue()); // = (dataBuffInt[100] >> 0) & 0xFF
System.out.println(c.getAlpha()); // = (dataBuffInt[100] >> 24) & 0xFF
Earlier, I tried putting the getRed, getGreen, and getBlue in a for loop but it only shows the same RGB value. How do I get all the RGB values in an image? Given that I wanna store them in different arrays.
I'm not entirely clear on the question, but assuming you mean unique RGB values, just loop, and just use say java.util.Set implementation that maintains uniqueness?
Set<Color> colors = new HashSet<Color>();
for (int datum : dataBuffInt) {
colors.add(new Color(datum));
}
System.out.println(String.format("%d different colors", colors.size()));
Or if you mean separate components?
for (int datum : dataBuffInt) {
Color color = new Color(datum);
reds.add(color.getRed());
greens.add(color.getGreen());
blues.add(color.getBlue());
}
System.out.println(String.format("reds: %d greens: %d blues: %d", reds.size(), greens.size(), blues.size()));
Are you certain when you had the for loop you were using the index variable into the array and not a static value, like 100? When I run your code with a for loop I see different values:
for (int i = 0; i < dataBuffInt.length; i++) {
Color c = new Color(dataBuffInt[i]);
System.out.println("COLOR");
System.out.println(c.getRed()); // = (dataBuffInt[100] >> 16) & 0xFF
System.out.println(c.getGreen()); // = (dataBuffInt[100] >> 8) & 0xFF
System.out.println(c.getBlue()); // = (dataBuffInt[100] >> 0) & 0xFF
System.out.println(c.getAlpha()); // = (dataBuffInt[100] >> 24) & 0xFF
System.out.println();
}
If you want unique colors you could build a set one pixel at a time:
final BufferedImage image = ImageIO.read(new URL("http://upload.wikimedia.org/wikipedia/en/2/24/Lenna.png"));
final Set<Color> uniqueColors = new HashSet<Color>(image.getWidth() * image.getHeight());
for (int y = 0; y < image.getHeight(); y++) {
for (int x = 0; x < image.getWidth(); x++) {
final int rgb = image.getRGB(x, y);
uniqueColors.add(new Color(rgb));
}
}
for (final Color color : uniqueColors) {
System.out.println(format("red: {0}, green: {1}, blue: {2}, alpha: {3}",
color.getRed(),
color.getGreen(),
color.getBlue(),
color.getAlpha()));
}
Or use your existing code and dump the array into a set.
I wanted to subtract two images pixel by pixel to check how much they are similar. Images have the same size one is little darker and beside brightness they don't differ. But I get those little dots in the result. Did I subtract those two images rigth? Both are bmp files.
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
public class Main2 {
public static void main(String[] args) throws Exception {
int[][][] ch = new int[4][4][4];
BufferedImage image1 = ImageIO.read(new File("1.bmp"));
BufferedImage image2 = ImageIO.read(new File("2.bmp"));
BufferedImage image3 = new BufferedImage(image1.getWidth(), image1.getHeight(), image1.getType());
int color;
for(int x = 0; x < image1.getWidth(); x++)
for(int y = 0; y < image1.getHeight(); y++) {
color = Math.abs(image2.getRGB(x, y) - image1.getRGB(x, y));
image3.setRGB(x, y, color);
}
ImageIO.write(image3, "bmp", new File("image.bmp"));
}
}
Image 1
Image 2
Result
The problem here is that you can't subtract the colors direcly. Each pixel is represented by one int value. This int value consists of 4 bytes. These 4 bytes represent the color components ARGB, where
A = Alpha
R = Red
G = Green
B = Blue
(Alpha is the opacity of the pixel, and always 255 (that is, the maximum value) in BMP images).
Thus, one pixel may be represented by
(255, 0, 254, 0)
When you subtract another pixel from this one, like (255, 0, 255, 0), then the third byte will underflow: It would become -1. But since this is part of ONE integer, the resulting color will be something like
(255, 0, 254, 0) -
(255, 0, 255, 0) =
(255, 255, 255, 0)
and thus, be far from what you would expect in this case.
The key point is that you have to split your color into the A,R,G and B components, and perform the computation on these components. In the most general form, it may be implemented like this:
int argb0 = image0.getRGB(x, y);
int argb1 = image1.getRGB(x, y);
int a0 = (argb0 >> 24) & 0xFF;
int r0 = (argb0 >> 16) & 0xFF;
int g0 = (argb0 >> 8) & 0xFF;
int b0 = (argb0 ) & 0xFF;
int a1 = (argb1 >> 24) & 0xFF;
int r1 = (argb1 >> 16) & 0xFF;
int g1 = (argb1 >> 8) & 0xFF;
int b1 = (argb1 ) & 0xFF;
int aDiff = Math.abs(a1 - a0);
int rDiff = Math.abs(r1 - r0);
int gDiff = Math.abs(g1 - g0);
int bDiff = Math.abs(b1 - b0);
int diff =
(aDiff << 24) | (rDiff << 16) | (gDiff << 8) | bDiff;
result.setRGB(x, y, diff);
Since these are grayscale images, the computations done here are somewhat redundant: For grayscale images, the R, G and B components are always equal. And since the opacity is always 255, it does not have to be treated explicitly here. So for your particular case, it should be sufficient to simplify this to
int argb0 = image0.getRGB(x, y);
int argb1 = image1.getRGB(x, y);
// Here the 'b' stands for 'blue' as well
// as for 'brightness' :-)
int b0 = argb0 & 0xFF;
int b1 = argb1 & 0xFF;
int bDiff = Math.abs(b1 - b0);
int diff =
(255 << 24) | (bDiff << 16) | (bDiff << 8) | bDiff;
result.setRGB(x, y, diff);
You did not "subtract one pixel from the other" correctly. getRGB returns "an integer pixel in the default RGB color model (TYPE_INT_ARGB)". What you are seeing is an "overflow" from one byte into the next, and thus from one color into the next.
Suppose you have colors 804020 - 404120 -- this is 3FFF00; the difference in the G component, 1 gets output as FF.
The correct procedure is to split the return value from getRGB into separate red, green, and blue, subtract each one, make sure they fit into unsigned bytes again (I guess your Math.abs is okay) and then write out a reconstructed new RGB value.
I found this which does what you want. It does seem to do the same thing and it may be more "correct" than your code. I assume it's possible to extract the source code.
http://tutorial.simplecv.org/en/latest/examples/image-math.html
/Fredrik Wahlgren
I am trying to implement Histogram/image Equalization on a coloured image. I am not sure if I have implemented it correct because the screen just goes black every time I apply it to a bitmap image. The algorithm is called histogram equalization.
The part of my code that does the Histogram Equalization calculation:
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
A = (pixels[index] >> 24) & 0xFF;
R = (pixels[index] >> 16) & 0xFF;
G = (pixels[index] >> 8) & 0xFF;
B = pixels[index] & 0xFF;
R = Math.round(((R - cumR[minR]) / (cumR[maxR] - cumR[minR])) * 255);
G = Math.round(((G - cumG[minG]) / (cumG[maxG] - cumG[minG])) * 255);
B = Math.round(((B - cumB[minB]) / (cumB[maxB] - cumB[minB])) * 255);
returnBitmap.setPixel(x, y, Color.argb(A, R, G, B));
++index;
}
}
The image appears black once my code is applied, why doesnt it display an equalized image?
You're not calculating the histograms correctly. You shouldn't have a histogram spot for each pixel, you have one for each value[0..255]. You want to count how many pixels have that value, not the total "value" of red.
Here's a good way to get the histogram(and cumulative) for an image. It should get you started on the right path.
// generate histogram channels
// histogram arrays should be [0...255]
for (int i = 0; i < pixels.length; i++) {
R = (pixels[i] >> 16) & 0xFF;
G = (pixels[i] >> 8) & 0xFF;
B = pixels[i] & 0xFF;
histoR[R]++;
histoG[G]++;
histoB[B]++;
}
// generate cumulative histograms
cumR[0] = histoR[0];
cumG[0] = histoG[0];
cumB[0] = histoB[0];
for(int i=1;i<histoR.length;i++){
cumR[i] = histoR[i] + histoR[i-1];
cumG[i] = histoG[i] + histoG[i-1];
cumB[i] = histoB[i] + histoB[i-1];
}
After some research, I was able to find a Histogram Equalization using a LUT example for Java and it is a better option than converting it to another Color Space such as RGB to YUV.
With minimal modification, I was able to use the following code:
Histogram Equalization for Java