We Keep Coding

Create simple 2D image preview from panoramic HDR

Is there some really simple and basic code for making preview for HDR images (like getting 2D BufferedImage output or something)?
I am using this HDR image.
I tried this (it uses TwelveMonkeys), but no success at all (it simply stuck/frozen at ImageReader reader = readers.next();)
I edited it a bit to suit my needs like this, testing where it got broken/stuck/frozen...and it always happen after TEST 1, that is TEST 2 is never reached, tho no IllegalArgumentException is thrown - if I remove the if() section, then TEST 3 is never reached (I am using NetBeansIDE v12.4, Win7 x64):
public BufferedImage hdrToBufferedImage(File hdrFile) throws IOException {
BufferedImage bi = null;
// Create input stream
// I WROTE DOWN THE STRING FOR THIS EXAMPLE, normally it is taken from the hdrFile
// HDR image size is 23.7MB if it matters at all?
ImageInputStream input = ImageIO.createImageInputStream(new File("Z:/HDR/spiaggia_di_mondello_4k.hdr"));
try {
// Get the reader
Iterator<ImageReader> readers = ImageIO.getImageReaders(input);
System.err.println("=====>>> TEST 1");
if (!readers.hasNext()) {
throw new IllegalArgumentException("No reader for: " + hdrFile);
}
System.err.println("=====>>> TEST 2");
ImageReader reader = readers.next();
System.err.println("=====>>> TEST 3");
try {
reader.setInput(input);
// Disable default tone mapping
HDRImageReadParam param = (HDRImageReadParam) reader.getDefaultReadParam();
param.setToneMapper(new NullToneMapper());
// Read the image, using settings from param
bi = reader.read(0, param);
} finally {
// Dispose reader in finally block to avoid memory leaks
reader.dispose();
}
} finally {
// Close stream in finally block to avoid resource leaks
input.close();
}
// Get float data
float[] rgb = ((DataBufferFloat) bi.getRaster().getDataBuffer()).getData();
// Convert the image to something easily displayable
BufferedImage converted = new ColorConvertOp(null).filter(bi, new BufferedImage(bi.getWidth(), bi.getHeight(), BufferedImage.TYPE_INT_RGB));
return converted;
}

Well, if you don't mind occasional extreme halucinogenic oversaturation of some colors here and there (I was unable solving the issue - if anyone knows how to, please, feel free to update my code), you can try this (it is using JavaHDR) + I also added a bit of brightness and contrast to it as all HDR I tested looked too dark for the preview, so if you do not like that you can remove that part from the code:
public int rgbToInteger(int r, int g, int b) {
int rgb = r;
rgb = (rgb << 8) + g;
rgb = (rgb << 8) + b;
return rgb;
}
public BufferedImage hdrToBufferedImage(File hdrFile) throws IOException {
HDRImage hdr = HDREncoder.readHDR(hdrFile, true);
int width = hdr.getWidth();
int height = hdr.getHeight();
BufferedImage bi = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
int r = (int) (hdr.getPixelValue(x, y, 0) * 255);
int g = (int) (hdr.getPixelValue(x, y, 1) * 255);
int b = (int) (hdr.getPixelValue(x, y, 2) * 255);
bi.setRGB(x, y, rgbToInteger(r, g, b));
}
}
//***** YOU CAN REMOVE THIS SMALL SECTION IF YOU FEEL THE IMAGE IS TOO BRIGHT FOR YOU
float brightness = 2f;
float contrast = 20f;
RescaleOp rescaleOp = new RescaleOp(brightness, contrast, null);
rescaleOp.filter(bi, bi);
//*****
return bi;
}

I can compile and run the code you posted (changing the path obviously) without problems on my two macOS machines, testing on all the LTS Java versions (8, 11 and 17). In addition, I run code similar to this as part of the CI/CD pipeline of my project that tests on Windows and Linux as well. I think there is something wrong with the setup in your IDE or Java on your computer. I am not able to reproduce the "freeze"-situation you describe...
Here is the output of running the program (I also printed the resulting BufferedImage for verification):
=====>>> TEST 1
=====>>> TEST 2
=====>>> TEST 3
image = BufferedImage#5a42bbf4: type = 1 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=0 IntegerInterleavedRaster: width = 1024 height = 512 #Bands = 3 xOff = 0 yOff = 0 dataOffset[0] 0
Running with the code as-is (with the NullToneMapper and no post-processing), the image looks like this, due to unnormalized values:
Running with the default/built-in tone mapper, or simply reading the image with ImageIO.read(hdrFile) as suggested in the comments, the image will look like this:
Finally, playing a bit with the code using a custom global tone mapper; param.setToneMapper(new DefaultToneMapper(0.75f)), I get a result like this:

After a long discussion with #HaraldK and his code addition, I am posting the final correct code for this problem, that is in fact mix of #qraqatit code updated a bit with the #HaraldK addition that corrects wrong color tone mapping, here it is:
public int rgbToInteger(int r, int g, int b) {
int rgb = r;
rgb = (rgb << 8) + g;
rgb = (rgb << 8) + b;
return rgb;
}
public BufferedImage hdrToBufferedImage(File hdrFile) throws IOException {
HDRImage hdr = HDREncoder.readHDR(hdrFile, true);
int width = hdr.getWidth();
int height = hdr.getHeight();
BufferedImage bi = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
float colorToneCorrection = 0.75f;
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
float r = hdr.getPixelValue(x, y, 0);
int red = (int) ((r / (colorToneCorrection + r)) * 255);
float g = hdr.getPixelValue(x, y, 1);
int green = (int) ((g / (colorToneCorrection + g)) * 255);
float b = hdr.getPixelValue(x, y, 2);
int blue = (int) (int) ((b / (colorToneCorrection + b)) * 255);
bi.setRGB(x, y, rgbToInteger(red, green, blue));
}
}
//MAKE THE RESULTING IMAGE A BIT BRIGHTER
float brightness = 1.35f;
float contrast = 0f;
RescaleOp rescaleOp = new RescaleOp(brightness, contrast, null);
rescaleOp.filter(bi, bi);
return bi;
}

glReadPixels returns more data than expected

I want to save a video of what I am showing with openGL using JOGL. To do this, I am writing my frames to pictures as follows and then, once I have saved all frames I'll use ffmpeg. I know that this is not the best approach but I still don't have much clear how to accelerate with tex2dimage and PBOs. Any help in that direction would be very useful.
Anyway, my problem is that if I run the opengl class it works but, if I call this class from another class, then I see that the glReadPixels is trhowing me an error. It always returns more data to buffer than memory has been allocated to my buffer "pixelsRGB". Does anyone know why?
As an example: width = 1042; height=998. Allocated=3.119.748 glPixels returned=3.121.742
public void display(GLAutoDrawable drawable) {
//Draw things.....
//bla bla bla
t++; //This is a time variable for the animation (it says to me the frame).
//Save frame
int width = drawable.getSurfaceWidth();
int height = drawable.getSurfaceHeight();
ByteBuffer pixelsRGB = Buffers.newDirectByteBuffer(width * height * 3);
gl.glReadPixels(0, 0, width,height, gl.GL_RGB, gl.GL_UNSIGNED_BYTE, pixelsRGB);
BufferedImage bufferedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
int[] pixels = new int[width * height];
int firstByte = width * height * 3;
int sourceIndex;
int targetIndex = 0;
int rowBytesNumber = width * 3;
for (int row = 0; row < height; row++) {
firstByte -= rowBytesNumber;
sourceIndex = firstByte;
for (int col = 0; col < width; col++) {
int iR = pixelsRGB.get(sourceIndex++);
int iG = pixelsRGB.get(sourceIndex++);
int iB = pixelsRGB.get(sourceIndex++);
pixels[targetIndex++] = 0xFF000000
| ((iR & 0x000000FF) << 16)
| ((iG & 0x000000FF) << 8)
| (iB & 0x000000FF);
}
}
bufferedImage.setRGB(0, 0, width, height, pixels, 0, width);
File a = new File(t+".png");
ImageIO.write(bufferedImage, "PNG", a);
}
NOTE: With pleluron's answer now it works. The good code is:
public void display(GLAutoDrawable drawable) {
//Draw things.....
//bla bla bla
t++; //This is a time variable for the animation (it says to me the frame).
//Save frame
int width = drawable.getSurfaceWidth();
int height = drawable.getSurfaceHeight();
ByteBuffer pixelsRGB = Buffers.newDirectByteBuffer(width * height * 4);
gl.glReadPixels(0, 0, width,height, gl.GL_RGBA, gl.GL_UNSIGNED_BYTE, pixelsRGB);
BufferedImage bufferedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
int[] pixels = new int[width * height];
int firstByte = width * height * 4;
int sourceIndex;
int targetIndex = 0;
int rowBytesNumber = width * 4;
for (int row = 0; row < height; row++) {
firstByte -= rowBytesNumber;
sourceIndex = firstByte;
for (int col = 0; col < width; col++) {
int iR = pixelsRGB.get(sourceIndex++);
int iG = pixelsRGB.get(sourceIndex++);
int iB = pixelsRGB.get(sourceIndex++);
sourceIndex++;
pixels[targetIndex++] = 0xFF000000
| ((iR & 0x000000FF) << 16)
| ((iG & 0x000000FF) << 8)
| (iB & 0x000000FF);
}
}
bufferedImage.setRGB(0, 0, width, height, pixels, 0, width);
File a = new File(t+".png");
ImageIO.write(bufferedImage, "PNG", a);
}

The default value of GL_PACK_ALIGNMENT set with glPixelStore is 4. It means that each row of pixelsRGB should start at an address that is a multiple of 4, and the width of your buffer (1042) times the number of bytes in a pixel (3) isn't a multiple of 4. Adding a little padding so the next row starts at a multiple of 4 will make the total byte size of your buffer larger than what you expected.
To fix it, set GL_PACK_ALIGNMENT to 1. You could also read the pixels with GL_RGBA and use a larger buffer, since the data is most likely to be stored that way both on the GPU and in BufferedImage.
Edit: BufferedImage doesn't have a convenient 'setRGBA', too bad.

Why raster.setPixels() is returning a grayscale image

Here i'm trying to do a fastest method to save 3 matrix(R, G and B) into a BufferedImage.
I've found this method here at StackExchange, but it doesn't work for me because the image it's being saved in a grayscale color.
If I'm doing something wrong or if there's a way of doing this faster than bufferimage.setRGB(), please help me. Thanks!
public static BufferedImage array_rasterToBuffer(int[][] imgR,
int[][]imgG, int[][] imgB) {
final int width = imgR[0].length;
final int height = imgR.length;
int numBandas = 3;
int[] pixels = new int[width*height*numBandas];
int cont=0;
System.out.println("max: "+width*height*3);
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
for (int band = 0; band < numBandas; band++) {
pixels[(((i*width)+j)*numBandas +band)] =Math.abs(( (imgR[i][j] & 0xff) >> 16 | (imgG[i][j] & 0xff) >> 8 | (imgB[i][j] & 0xff)));
cont+=1;
}
}
}
BufferedImage bufferImg = new BufferedImage(width, height,BufferedImage.TYPE_INT_RGB);
WritableRaster rast = (WritableRaster) bufferImg.getData();
rast.setPixels(0, 0, width, height, pixels);
bufferImg.setData(rast);
return bufferImg;
}

I think you are getting grey because the expression
Math.abs(( (imgR[i][j] & 0xff) >> 16 | (imgG[i][j] & 0xff) >> 8 | (imgB[i][j] & 0xff)));
does not depend on band, so your rgb values are all the same.
The expression looks dodgy anyway because you normally use the left shift operator << when packing rgb values into a single int.
I don't know for sure, as I'm not familiar with the classes you are using, but I'm guessing something like this might work
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
pixels[(((i*width)+j)*numBandas)] = imgR[i][j] & 0xFF;
pixels[(((i*width)+j)*numBandas + 1)] = imgG[i][j] & 0xFF;
pixels[(((i*width)+j)*numBandas + 2)] = imgB[i][j] & 0xFF;
}
}

If you want a faster approach, you need to get the "live" WritableRaster from the BufferedImage and set pixels in the "native" format of the image, which is "pixel packed" for TYPE_INT_RGB. This will save you multiple (at least two) array copies and some data conversion. It will also save you 2/3rds of the memory used for the conversion, as we only need a single array component per pixel.
The below method should be quite a bit faster:
public static BufferedImage array_rasterToBuffer(int[][] imgR, int[][] imgG, int[][] imgB) {
final int width = imgR[0].length;
final int height = imgR.length;
// The bands are "packed" for TYPE_INT_RGB Raster,
// so we need only one array component per pixel
int[] pixels = new int[width * height];
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
// "Pack" RGB values to native TYPE_INT_RGB format
// (NOTE: Do not use Math.abs on these values, and without alpha there won't be negative values)
pixels[((y * width) + x)] = ((imgR[y][x] & 0xff) << 16 | (imgG[y][x] & 0xff) << 8 | (imgB[y][x] & 0xff));
}
}
BufferedImage bufferImg = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
// NOTE: getRaster rather than getData for "live" view
WritableRaster rast = bufferImg.getRaster();
// NOTE: setDataElements rather than setPixels to avoid conversion
// This requires pixels to be in "native" packed RGB format (as above)
rast.setDataElements(0, 0, width, height, pixels);
// No need for setData as we were already working on the live data
// thus saving at least two expensive array copies
return bufferImg;
}
// Test method, displaying red/green/blue stripes
public static void main(String[] args) {
int[][] fooR = new int[99][99];
int[][] fooG = new int[99][99];
int[][] fooB = new int[99][99];
for (int i = 0; i < 33; i++) {
Arrays.fill(fooR[i], 0xff);
Arrays.fill(fooG[i + 33], 0xff);
Arrays.fill(fooB[i + 66], 0xff);
}
BufferedImage image = array_rasterToBuffer(fooR, fooG, fooB);
showIt(image);
}
// For demonstration only
private static void showIt(final BufferedImage image) {
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
JFrame frame = new JFrame("JPEGTest");
frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
JScrollPane scroll = new JScrollPane(new JLabel(new ImageIcon(image)));
scroll.setBorder(BorderFactory.createEmptyBorder());
frame.add(scroll);
frame.pack();
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
});
}
It is possible to optimize this further, if you don't need a "managed" (possible hardware accelerated for display) image. The trick is to create the image directly "around" your pixels array, thus saving one more array allocation and array copy in setDataElements. The downside is that in some cases the image will be a little slower to draw onto the screen. This is mainly a concern for games or smooth animations though.
Replace the lines from BufferedImage bufferImg = new BufferedImage... until the return statement, with the following code:
DataBufferInt buffer = new DataBufferInt(pixels, pixels.length);
int[] bandMasks = {0xFF0000, 0xFF00, 0xFF}; // RGB (no alpha)
WritableRaster raster = Raster.createPackedRaster(buffer, width, height, width, bandMasks, null);
ColorModel cm = new DirectColorModel(32,
0x00ff0000, // Red
0x0000ff00, // Green
0x000000ff, // Blue
0x00000000 // No Alpha
);
BufferedImage bufferImg = new BufferedImage(cm, raster, cm.isAlphaPremultiplied(), null);
PS: Note that I also changed the shifts inside the x/y loop, from right to left shifts. Might have been just a minor typo. :-)

How do I flip an image horizontally flip with glReadPixels() Bufferedimage and out put with ImageIO?

How do I flip an Screenshot image? I can't find my problem anywhere else.Example code:
/*
*#param fileLoc //Location of fileoutput destination
*#param format //"png"
*#param WIDTH //Display.width();
*#param HEIGHT //Display.height();
*/
private void getScreenImage(){
int[] pixels = new int[WIDTH * HEIGHT];
int bindex;
// allocate space for RBG pixels
ByteBuffer fb = ByteBuffer.allocateDirect(WIDTH * HEIGHT * 3);//.order(ByteOrder.nativeOrder());
// grab a copy of the current frame contents as RGB
glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGB, GL_UNSIGNED_BYTE, fb);
BufferedImage image = new BufferedImage(WIDTH, HEIGHT,BufferedImage.TYPE_INT_RGB);
// convert RGB data in ByteBuffer to integer array
for (int i=0; i < pixels.length; i++) {
bindex = i * 3;
pixels[i] =
((fb.get(bindex) << 16)) +
((fb.get(bindex+1) << 8)) +
((fb.get(bindex+2) << 0));
}
try {
//Create a BufferedImage with the RGB pixels then save as PNG
image.setRGB(0, 0, WIDTH, HEIGHT, pixels, 0 , WIDTH);
ImageIO.write(image, format , fileLoc);
}
catch (Exception e) {
System.out.println("ScreenShot() exception: " +e);
}
}
Basically the code works for capturing the screen and storing at as "png" format.
But it output's the image horizontally flipped, because glReadPixels();,
read from bottom-left to top-right.
So how do I flip the image horizontally before I ImageIO.write();?
Thanks in-front,
Rose.

E.G. of flipping an image horizontally using an AffineTransform.
import java.awt.*;
import java.awt.geom.AffineTransform;
import java.awt.image.BufferedImage;
import javax.swing.*;
public class Test001 {
public static BufferedImage getFlippedImage(BufferedImage bi) {
BufferedImage flipped = new BufferedImage(
bi.getWidth(),
bi.getHeight(),
bi.getType());
AffineTransform tran = AffineTransform.getTranslateInstance(bi.getWidth(), 0);
AffineTransform flip = AffineTransform.getScaleInstance(-1d, 1d);
tran.concatenate(flip);
Graphics2D g = flipped.createGraphics();
g.setTransform(tran);
g.drawImage(bi, 0, 0, null);
g.dispose();
return flipped;
}
Test001(BufferedImage bi) {
JPanel gui = new JPanel(new GridLayout(1,2,2,2));
gui.add(new JLabel(new ImageIcon(bi)));
gui.add(new JLabel(new ImageIcon(getFlippedImage(bi))));
JOptionPane.showMessageDialog(null, gui);
}
public static void main(String[] args) throws AWTException {
final Robot robot = new Robot();
Runnable r = new Runnable() {
#Override
public void run() {
final BufferedImage bi = robot.createScreenCapture(
new Rectangle(0, 360, 200, 100));
new Test001(bi);
}
};
SwingUtilities.invokeLater(r);
}
}

It's worth noting that it might be faster to simply read the pixels out of the buffer in the order you want them, rather than read them backwards and do a costly transform operation. Additionally, since you know for sure that the BufferedImage is TYPE_INT_RGB it should be safe to write directly into its raster.
ByteBuffer fb = BufferUtils.createByteBuffer(WIDTH * HEIGHT * 3);
BufferedImage image = new BufferedImage(WIDTH, HEIGHT, BufferedImage.TYPE_INT_RGB);
glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGB, GL_UNSIGNED_BYTE, fb);
int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
for (int i = pixels.length - 1; i >= 0; i--) {
int x = i % WIDTH, y = i / WIDTH * WIDTH;
pixels[y + WIDTH - 1 - x] = (fb.get() & 0xff) << 16 | (fb.get() & 0xff) << 8 | fb.get() & 0xff;
}

Java equivalent of JavaScript's Canvas getImageData

I'm porting a HTML5's Canvas sample to Java, so far so good, until i get on this function call :
Canvas.getContext('2d').getImageData(0, 0, 100, 100).data
I googled for a while and found this page of the canvas specification
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#pixel-manipulation
After reading it, I created this function below :
public int[] getImageDataPort(BufferedImage image) {
int width = image.getWidth();
int height = image.getHeight();
int[] ret = new int[width * height * 4];
int idx = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int color = image.getRGB(x, y);
ret[idx++] = getRed(color);
ret[idx++] = getGreen(color);
ret[idx++] = getBlue(color);
ret[idx++] = getAlpha(color);
}
}
return ret;
}
public int getRed(int color) {
return (color >> 16) & 0xFF;
}
public int getGreen(int color) {
return (color >> 8) & 0xFF;
}
public int getBlue(int color) {
return (color >> 0) & 0xFF;
}
public int getAlpha(int color) {
return (color >> 24) & 0xff;
}
There is any class on Java Graphics API that has this function built-in or i should use the one that i had created?

I think the closest thing you'll find in the standard Java API is the Raster class. You can get hold of a WritableRaster (used for low-level image manipulation) through BufferedImage.getRaster. The Raster class then provides methods such as getSamples which fills an int[] with image data.

Thanks aioobe, i've looked at the WritableRaster class and found the getPixels function which does exactly what i needed, the final result is :
public int[] getImageDataPort(BufferedImage image) {
int width = image.getWidth();
int height = image.getHeight();
int[] ret = null;
ret = image.getRaster().getPixels(0, 0, width, height, ret);
return ret;
}
The only problem that may happen is when the image.getType isn't a type that supports alpha in comparison with the code of the question, resulting in a smaller int[] ret, but one can simply convert the image type with :
public BufferedImage convertType(BufferedImage image,int type){
BufferedImage ret = new BufferedImage(image.getWidth(), image.getHeight(), type);
ColorConvertOp xformOp = new ColorConvertOp(null);
xformOp.filter(image, ret);
return ret;
}

Try
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(bi, "jpg", baos);
where bi - BufferendImage