I've been working on an extension of a current application to stream webcam data to an Android device. I can obtain the raw image data, in the form of a RGB byte array. The color space is sRGB. I need to send that array over the network to an Android client, who constructs it into a Bitmap image to display on the screen. My problem is that the color data is skewed. The arrays have the same hashcode before and after being sent, so I'm positive this isn't a data loss problem. I've attached a sample image of how the color looks, you can see that skin tones and darker colors reconstruct okay, but lighter colors end up with a lot of yellow/red artifacts.
Server (Windows 10) code :
while(socket.isConnected()) {
byte[] bufferArray = new byte[width * height * 3];
ByteBuffer buff = cam.getImageBytes();
for(int i = 0; i < bufferArray.length; i++) {
bufferArray[i] = buff.get();
}
out.write(bufferArray);
out.flush();
}
Client (Android) code :
while(socket.isConnected()) {
int[] colors = new int[width * height];
byte[] pixels = new byte[(width * height) * 3];
int bytesRead = 0;
for(int i = 0; i < (width * height * 3); i++) {
int temp = in.read();
if(temp == -1) {
Log.d("WARNING", "Problem reading");
break;
}
else {
pixels[i] = (byte) temp;
bytesRead++;
}
}
int colorIndex = 0;
for(int i = 0; i < pixels.length; i += 3 ) {
int r = pixels[i];
int g = pixels[i + 1];
int b = pixels[i + 2];
colors[colorIndex] = Color.rgb( r, g, b);
colorIndex++;
}
Bitmap image = Bitmap.createBitmap(colors, width, height, Bitmap.Config.ARGB_8888);
publishProgress(image);
}
The cam.getImageBytes() is from an external library, but I have tested it and it works properly. Reconstructing the RAW data into a BufferedImage works perfectly, using the code :
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
image.getRaster().setPixels(0,0,width,height, pixels);
But, of course, BufferedImages are not supported on Android.
I'm about to tear my hair out with this one, I've tried everything I can think of, so any and all insight would be extremely helpful!
Related
I'm using the following to add overlay to a video; but it doesn't perform well because it requires to process every pixels (which is A LOT of work if you have full-HD at 50 frames/sec!)
private static byte[] convert(BufferedImage image, int opacity)
{
int imgWidth = image.getWidth();
int imgHeight = image.getHeight();
byte[] buffer = new byte[imgWidth * imgHeight * 4]
int imgLoc = 0;
for(int y=0; y < imgHeight; y++)
{
for(int x=0; x < imgWidth; x++)
{
int argb = image.getRGB(x, y);
imgBuffer[(imgLoc*4)+0] = (byte)((argb>>0)&0x0FF);
imgBuffer[(imgLoc*4)+1] = (byte)((argb>>8)&0x0FF);
imgBuffer[(imgLoc*4)+2] = (byte)((argb>>16)&0x0FF);
int alpha = ((argb>>24)&0x0FF);
if (opacity < 100)
alpha = (alpha*opacity)/100;
imgBuffer[(imgLoc*4)+3] = (byte)alpha;
imgLoc++;
}
}
How can I re-write this code to perform better? I've tried many different things, e.g.:
image.getRGB(0, 0, image.getWidth(), image.getHeight(), null, 0, image.getWidth());
((DataBufferByte) image.getData().getDataBuffer()).getData()
But none of these seem to work; the overlay doesn't show up, while the slower method at least shows the overlay.
PS: I know the ARGB is converted to ABGR in the function above; I'll fix that later. The overlay should still show, albeit in different colors
In my android app, I have a car from which the user can click and select different panels. The image is relatively complicated (as opposed to the one pasted here) so its difficult to overlay the buttons in the correct spots. In addition there are a lot of different images.
The solution I would like to try:
Detect which panel was selected by using a colour mask as suggested here: https://blahti.wordpress.com/2012/06/26/images-with-clickable-areas/
Depending on the panels selected (in my example the blue and green) generate a mask.
Depending on the mask, have a red overlay on the car - just a colour filter will be fine.
(First image represents the colours used to determine which panel was clicked, second image represents the mask generated and the last image the 'result').
The only problem I'm having is: How do I dynamically create the mask? I thought of using a floodfill type method to create a new canvas with the 'mask' of the selected panels. But, I worry that it might be too computationally heavy. Any simpler suggestions?
[
UPDATE: Ok so I've come pretty far. As expected, the creation of the mask too way too long (2-4 seconds for a small image). But, then I discovered RenderScripts!! I think I can still get this to work. The only little snag that I have now is: How do I pass in the colours that have been pressed?
My current code looks like this:
// create a bitmap for the mask.
ImageView img = (ImageView) findViewById (mask);
img.setDrawingCacheEnabled(true);
Bitmap bitmap = Bitmap.createBitmap(img.getDrawingCache());
// Create a tiny bitmap to store the colours of the panels that are
//'selected'
Bitmap.Config conf = Bitmap.Config.ARGB_8888; // see other conf types
Bitmap myBitmap = Bitmap.createBitmap(pickedPanels.size(), 1, conf);
int [] myInts = new int[pickedPanels.size()];
for (int i = 0; i<pickedPanels.size(); i++){
myInts[i] = pickedPanels.get(i).intValue();
}
myBitmap.setPixels(myInts, 0, myBitmap.getWidth(), 0, 0,
myBitmap.getWidth(),0);
//Run thescript and set the output
final RenderScript rs = RenderScript.create(this);
final Allocation input = Allocation.createFromBitmap(rs, bitmap,
Allocation.MipmapControl.MIPMAP_NONE,Allocation.USAGE_SCRIPT);
final Allocation output = Allocation.createTyped(rs, input.getType());
final ScriptC_singlesource script = new
ScriptC_singlesource(rs);
script.set_image(Allocation.createFromBitmap(rs, myBitmap,
Allocation.MipmapControl.MIPMAP_NONE,Allocation.USAGE_SCRIPT));
script.set_imgWidth(pickedPanels.size());
script.forEach_root(input, output);
output.copyTo(bitmap);
img.setImageBitmap(bitmap);
ImageView destim = (ImageView) findViewById (dest);
destim.setDrawingCacheEnabled(true);
destim.setImageBitmap(bitmap);
and this is the script:
#pragma version(1)
#pragma rs java_package_name(za.co.overtake)
rs_allocation image;
int imgWidth;
uchar4 RS_KERNEL root(uchar4 in, uint32_t x, uint32_t y) {
for(int col = 0; col < imgWidth; col++){
const uchar4 colour = *(const uchar4*)rsGetElementAt(image, col,0);
if (in.r == colour.r && in.g == colour.g && in.b == colour.b){
in.r = 255;
in.g = 0;
in.b = 0;
break;
} else {
in.r = 0;
in.g = 255;
in.b = 0;
rsDebug("HELLLLLP>>", colour);
}
}
return in;
}
But, when I try and read the pixel values from myBitmap (or image in the script), RGB is always 0.
(Sorry for the bad naming, etc. I've been going crazy trying to figure this out)
Ok, finally got this figured out.
In my renderscript code I have:
#pragma version(1)
#pragma rs java_package_name(za.co.overtake)
int*reds;
int*greens;
int*blues;
int imgWidth;
uchar4 RS_KERNEL root(uchar4 in, uint32_t x, uint32_t y) {
bool colourme = false;
for(int col = 0; col < imgWidth; col++){
const int red = reds[col];
const int green = greens[col];
const int blue = blues[col];
if (in.r == red && in.g == green && in.b == blue){
colourme = true;
}
}
if (colourme) {
in.r = 255;
in.g = 0;
in.b = 0;
in.a = 50;
} else {
in.r = 0;
in.g = 0;
in.b = 0;
in.a = 0;
}
return in;
}
Then in Java
public void showDamagedPanels(int dest, int mask) {
int noOfColours = pickedPanels.size();
if (noOfColours > 0) {
ImageView img = (ImageView) findViewById (mask);
img.setDrawingCacheEnabled(true);
Bitmap bitmap = Bitmap.createBitmap(img.getDrawingCache());
img.setDrawingCacheEnabled(false);
int [] reds = new int[noOfColours];
int [] greens = new int[noOfColours];
int [] blues = new int[noOfColours];
for (int i = 0; i< noOfColours; i++){
int colour = pickedPanels.get(i);
reds[i] = (colour >> 16) & 0xFF;
greens[i] = (colour >> 8) & 0xFF;
blues[i] = (colour >> 0) & 0xFF;
}
final RenderScript rs = RenderScript.create(this);
final Allocation input = Allocation.createFromBitmap(rs, bitmap, Allocation.MipmapControl.MIPMAP_NONE,
Allocation.USAGE_SCRIPT);
final Allocation output = Allocation.createTyped(rs, input.getType());
final ScriptC_singlesource script = new ScriptC_singlesource(rs);
Allocation red = Allocation.createSized(rs, Element.I32(rs), reds.length);
red.copyFrom(reds);
script.bind_reds(red);
Allocation green = Allocation.createSized(rs, Element.I32(rs), greens.length);
green.copyFrom(greens);
script.bind_greens(green);
Allocation blue = Allocation.createSized(rs, Element.I32(rs), blues.length);
blue.copyFrom(blues);
script.bind_blues(blue);
script.set_imgWidth(pickedPanels.size());
script.forEach_root(input, output);
output.copyTo(bitmap);
ImageView destim = (ImageView) findViewById (dest);
destim.setDrawingCacheEnabled(true);
destim.setImageBitmap(bitmap);
} else {
ImageView destim = (ImageView) findViewById (dest);
destim.setImageBitmap(null);
}
}
where dest is the overlay image and mask in the image acting as the mask. So basically, when a panel is clicked - place its colour in pickedPanels. Then call the showPanels method, which calls the script. The script checks the colours and sets the resulting image red or clear.
Update: For anyone trying to use this, but having some issues: It is possible to do this without the renderscript code, but it does run a bit slower - though it has been ok in my case for small images.
private Bitmap changeColor(Bitmap src, Set<Integer> pickedPanelsList) {
int fine = getResources().getColor(R.color.colorAccent);
int width = src.getWidth();
int height = src.getHeight();
int[] pixels = new int[width * height];
// get pixel array from source
src.getPixels(pixels, 0, width, 0, 0, width, height);
Bitmap bmOut = Bitmap.createBitmap(width, height, src.getConfig());
int AGood = 100, RGood = Color.red(fine), GGood = Color.green(fine), BGood = Color.blue(fine);
int ABad = 100, RBad = Color.red(Color.RED), GBad = Color.green(Color.RED), BBad = Color.blue(Color.RED);
int pixel;
// iteration through pixels
for (int y = 0; y < height; ++y) {
for (int x = 0; x < width; ++x) {
// get current index in 2D-matrix
int index = y * width + x;
pixel = pixels[index];
if(pickedPanelsList.contains(pixel)){
pixels[index] = Color.argb(ABad, RBad, GBad, BBad);
} else if (Color.alpha(pixel) > 0){
pixels[index] = Color.argb(AGood, RGood, GGood, BGood);
}
}
}
bmOut.setPixels(pixels, 0, width, 0, 0, width, height);
return bmOut;
}
Here, the picked panel set is all the colours that should be coloured red (or chosen) and the bitmap is the mask (if I remember correctly, I did this a while ago). I've also found that doing a slight blur on the result makes the image look nicer - since it will obviously be less jagged.
I want to save a video of what I am showing with openGL using JOGL. To do this, I am writing my frames to pictures as follows and then, once I have saved all frames I'll use ffmpeg. I know that this is not the best approach but I still don't have much clear how to accelerate with tex2dimage and PBOs. Any help in that direction would be very useful.
Anyway, my problem is that if I run the opengl class it works but, if I call this class from another class, then I see that the glReadPixels is trhowing me an error. It always returns more data to buffer than memory has been allocated to my buffer "pixelsRGB". Does anyone know why?
As an example: width = 1042; height=998. Allocated=3.119.748 glPixels returned=3.121.742
public void display(GLAutoDrawable drawable) {
//Draw things.....
//bla bla bla
t++; //This is a time variable for the animation (it says to me the frame).
//Save frame
int width = drawable.getSurfaceWidth();
int height = drawable.getSurfaceHeight();
ByteBuffer pixelsRGB = Buffers.newDirectByteBuffer(width * height * 3);
gl.glReadPixels(0, 0, width,height, gl.GL_RGB, gl.GL_UNSIGNED_BYTE, pixelsRGB);
BufferedImage bufferedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
int[] pixels = new int[width * height];
int firstByte = width * height * 3;
int sourceIndex;
int targetIndex = 0;
int rowBytesNumber = width * 3;
for (int row = 0; row < height; row++) {
firstByte -= rowBytesNumber;
sourceIndex = firstByte;
for (int col = 0; col < width; col++) {
int iR = pixelsRGB.get(sourceIndex++);
int iG = pixelsRGB.get(sourceIndex++);
int iB = pixelsRGB.get(sourceIndex++);
pixels[targetIndex++] = 0xFF000000
| ((iR & 0x000000FF) << 16)
| ((iG & 0x000000FF) << 8)
| (iB & 0x000000FF);
}
}
bufferedImage.setRGB(0, 0, width, height, pixels, 0, width);
File a = new File(t+".png");
ImageIO.write(bufferedImage, "PNG", a);
}
NOTE: With pleluron's answer now it works. The good code is:
public void display(GLAutoDrawable drawable) {
//Draw things.....
//bla bla bla
t++; //This is a time variable for the animation (it says to me the frame).
//Save frame
int width = drawable.getSurfaceWidth();
int height = drawable.getSurfaceHeight();
ByteBuffer pixelsRGB = Buffers.newDirectByteBuffer(width * height * 4);
gl.glReadPixels(0, 0, width,height, gl.GL_RGBA, gl.GL_UNSIGNED_BYTE, pixelsRGB);
BufferedImage bufferedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
int[] pixels = new int[width * height];
int firstByte = width * height * 4;
int sourceIndex;
int targetIndex = 0;
int rowBytesNumber = width * 4;
for (int row = 0; row < height; row++) {
firstByte -= rowBytesNumber;
sourceIndex = firstByte;
for (int col = 0; col < width; col++) {
int iR = pixelsRGB.get(sourceIndex++);
int iG = pixelsRGB.get(sourceIndex++);
int iB = pixelsRGB.get(sourceIndex++);
sourceIndex++;
pixels[targetIndex++] = 0xFF000000
| ((iR & 0x000000FF) << 16)
| ((iG & 0x000000FF) << 8)
| (iB & 0x000000FF);
}
}
bufferedImage.setRGB(0, 0, width, height, pixels, 0, width);
File a = new File(t+".png");
ImageIO.write(bufferedImage, "PNG", a);
}
The default value of GL_PACK_ALIGNMENT set with glPixelStore is 4. It means that each row of pixelsRGB should start at an address that is a multiple of 4, and the width of your buffer (1042) times the number of bytes in a pixel (3) isn't a multiple of 4. Adding a little padding so the next row starts at a multiple of 4 will make the total byte size of your buffer larger than what you expected.
To fix it, set GL_PACK_ALIGNMENT to 1. You could also read the pixels with GL_RGBA and use a larger buffer, since the data is most likely to be stored that way both on the GPU and in BufferedImage.
Edit: BufferedImage doesn't have a convenient 'setRGBA', too bad.
I'm trying to implement a DCT on an image, and so far I have only been able to successfully read an image, grayscale it, turn it into a byte array and then output the byte array as an image. This works fine. However, in order to work on the image I need to convert the byte array to an int array and then back again, and this is where the problem comes in.
This first part reads the image and converts the image to grayscale.
BufferedImage image = ImageIO.read(new File("C:\\Users\\A00226084\\Desktop\\image.jpg"));
int width = image.getWidth();
int height = image.getHeight();
for(int i=0; i<height; i++){
for(int j=0; j<width; j++){
Color c = new Color(image.getRGB(j, i));
int red = (int)(c.getRed() * 0.299);
int green = (int)(c.getGreen() * 0.587);
int blue = (int)(c.getBlue() *0.114);
Color newColor = new Color(red+green+blue,
red+green+blue,red+green+blue);
image.setRGB(j,i,newColor.getRGB());
}
}
This part converts the image to a byte array.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(image, "jpg", baos);
byte[] pixels = baos.toByteArray();
This part then converts the byte array to an int array. I have manually done the wrapping because nothing else worked.
for(int i = 0; i < pixels.length; i ++){
if(pixels[i] < 0){
byteConverted[i] = 127 + (128 - (pixels[i] * -1));
}else{
byteConverted[i] = pixels[i];
}
}
This part converts the int array back to a byte array, somewhere between the byte > int and int > byte conversions that everything goes wrong. I have output both the before and after byte array to a file and they are identical, so I don't know why I get 'image == null' instead of the image.
for(int i = 0; i < pixels.length; i ++){
if(byteConverted[i] > 127){
pixels[i] = (byte) ( -128 + (byteConverted[i] - 127));
}else{
pixels[i] = (byte) byteConverted[i];
}
}
write2("final byte array.txt", pixels);
ByteArrayInputStream bais = new ByteArrayInputStream(pixels);
try {
BufferedImage img = ImageIO.read(bais);
System.out.println("Image Out");
System.out.println(pixels.length);
ImageIO.write(img, "jpg", new File("C:\\Users\\A00226084\\Desktop\\newImage.jpg"));
} catch (IOException e) {
e.printStackTrace();
}
hey i have using j2me to read an image
i want to do some process on that image like Darkenes , lightens
i already read image as an input stream
InputStream iStrm = getClass().getResourceAsStream("/earth.PNG");
ByteArrayOutputStream bStrm = new ByteArrayOutputStream();
int ch;
while ((ch = iStrm.read()) != -1){
bStrm.write(ch);
byte imageData[] = bStrm.toByteArray();
Image im = Image.createImage(imageData, 0, imageData.length);
how can i get RGB values or how can i add some values to the array of pixles
imageData[] so it can more lightens or darkness ,
is there header data including in the input stream i had read , that cause me error when iam adding some values to it ?
I think you should be able to do the following:
int width = im.getWidth();
int height = im.getHeight();
int[] rgbData = new int[width*height]; // since we are working with rgba
im.getRGB(rgbData, 0, width, 0, 0, width, height);
// now, the data is stored in each integer as 0xAARRGGBB,
// so high-order bits are alpha channel for each integer
Now, if you want to put them into three arrays, one for each channel, you could do the following:
int red[][] = new int[width][height];
int green[][] = new int[width][height];
int blue[][] = new int[width][height];
for (int i = 0; i < width; i++)
{
for (int j = 0; j < height; j++)
{
red[i][j] = rgb[i*width + j] & 0xFF0000;
green[i][j] = rgb[i*width + j] & 0xFF00;
blue[i][j] = rgb[i+width + j] & 0xFF;
}
}