I have a method converting BufferedImages who's type is TYPE_CUSTOM to TYPE_INT_RGB. I am using the following code, however I would really like to find a faster way of doing this.
BufferedImage newImg = new BufferedImage(
src.getWidth(),
src.getHeight(),
BufferedImage.TYPE_INT_RGB);
ColorConvertOp op = new ColorConvertOp(null);
op.filter(src, newImg);
It works fine, however it's quite slow and I am wondering if there is a faster way to do this conversion.
ColorModel Before Conversion:
ColorModel: #pixelBits = 24 numComponents = 3 color space = java.awt.color.ICC_ColorSpace#1c92586f transparency = 1 has alpha = false isAlphaPre = false
ColorModel After Conversion:
DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=0
Thanks!
Update:
Turns out working with the raw pixel data was the best way. Since the TYPE_CUSTOM was actually RGB converting it manually is simple and is about 95% faster than ColorConvertOp.
public static BufferedImage makeCompatible(BufferedImage img) throws IOException {
// Allocate the new image
BufferedImage dstImage = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_INT_RGB);
// Check if the ColorSpace is RGB and the TransferType is BYTE.
// Otherwise this fast method does not work as expected
ColorModel cm = img.getColorModel();
if ( cm.getColorSpace().getType() == ColorSpace.TYPE_RGB && img.getRaster().getTransferType() == DataBuffer.TYPE_BYTE ) {
//Allocate arrays
int len = img.getWidth()*img.getHeight();
byte[] src = new byte[len*3];
int[] dst = new int[len];
// Read the src image data into the array
img.getRaster().getDataElements(0, 0, img.getWidth(), img.getHeight(), src);
// Convert to INT_RGB
int j = 0;
for ( int i=0; i<len; i++ ) {
dst[i] = (((int)src[j++] & 0xFF) << 16) |
(((int)src[j++] & 0xFF) << 8) |
(((int)src[j++] & 0xFF));
}
// Set the dst image data
dstImage.getRaster().setDataElements(0, 0, img.getWidth(), img.getHeight(), dst);
return dstImage;
}
ColorConvertOp op = new ColorConvertOp(null);
op.filter(img, dstImage);
return dstImage;
}
BufferedImages are painfully slow. I got a solution but I'm not sure you will like it. The fastest way to process and convert buffered images is to extract the raw data array from inside the BufferedImage. You do that by calling buffImg.getRaster() and converting it into the specific raster. Then call raster.getDataStorage(). Once you have access to the raw data it is possible to write fast image processing code without all the abstraction in BufferedImages slowing it down. This technique also requires an in depth understanding of image formats and some reverse engineering on your part. This is the only way I have been able to get image processing code to run fast enough for my applications.
Example:
ByteInterleavedRaster srcRaster = (ByteInterleavedRaster)src.getRaster();
byte srcData[] = srcRaster.getDataStorage();
IntegerInterleavedRaster dstRaster = (IntegerInterleavedRaster)dst.getRaster();
int dstData[] = dstRaster.getDataStorage();
dstData[0] = srcData[0] << 16 | srcData[1] << 8 | srcData[2];
or something like that. Expect compiler errors warning you not to access low level rasters like that. The only place I have had issues with this technique is inside of applets where an access violation will occur.
I've found rendering using Graphics.drawImage() instead of ColorConvertOp 50 times faster. I can only assume that drawImage() is GPU accelerated.
i.e this is really slow, like 50ms a go for 100x200 rectangles
public void BufferdImage convert(BufferedImage input) {
BufferedImage output= new BufferedImage(input.getWidht(), input.getHeight(), BufferedImage.TYPE_BYTE_BINARY, CUSTOM_PALETTE);
ColorConvertOp op = new ColorConvertOp(input.getColorModel().getColorSpace(),
output.getColorModel().getColorSpace());
op.filter(input, output);
return output;
}
i.e however this registers < 1ms for same inputs
public void BufferdImage convert(BufferedImage input) {
BufferedImage output= new BufferedImage(input.getWidht(), input.getHeight(), BufferedImage.TYPE_BYTE_BINARY, CUSTOM_PALETTE);
Graphics graphics = output.getGraphics();
graphics.drawImage(input, 0, 0, null);
graphics.dispose();
return output;
}
Have you tried supplying any RenderingHints? No guarantees, but using
ColorConvertOp op = new ColorConvertOp(new RenderingHints(
RenderingHints.KEY_COLOR_RENDERING,
RenderingHints.VALUE_COLOR_RENDER_SPEED));
rather than the null in your code snippet might speed it up somewhat.
I suspect the problem might be that ColorConvertOp() works pixel-by-pixel (guaranteed to be "slow").
Q: Is it possible for you to use gc.createCompatibleImage()?
Q: Is your original bitmap true color, or does it use a colormap?
Q: Failing all else, would you be agreeable to writing a JNI interface? Either to your own, custom C code, or to an external library such as ImageMagick?
If you have JAI installed then you might try uninstalling it, if you can, or otherwise look for some way to disable codecLib when loading JPEG. In a past life I had similar issues (http://www.java.net/node/660804) and ColorConvertOp was the fastest at the time.
As I recall the fundamental problem is that Java2D is not at all optimized for TYPE_CUSTOM images in general. When you install JAI it comes with codecLib which has a decoder that returns TYPE_CUSTOM and gets used instead of the default. The JAI list may be able to provide more help, it's been several years.
maybe try this:
Bitmap source = Bitmap.create(width, height, RGB_565);//don't remember exactly...
Canvas c = new Canvas(source);
// then
c.draw(bitmap, 0, 0);
Then the source bitmap will be modified.
Later you can do:
onDraw(Canvas canvas){
canvas.draw(source, rectSrs,rectDestination, op);
}
if you can manage always reuse the bitmap so you be able to get better performance. As well you can use other canvas functions to draw your bitmap
Related
What I want to do is very simple : I have an image which is basically a single color image with alpha.
EDIT : since my post has been light speed tagged as duplicate, here are the main differences with the other post :
The other topic's image has several colors, mine has only one
The owner accepted answer is the one I implemented... and I'm saying I have an issue with it because it's too slow
It means that all pixels are either :
White (255;255;255) and Transparent (0 alpha)
or Black (0;0;0) and Opaque (alpha between 0 and 255)
Alpha is not always the same, I have different shades of black.
My image is produced with Photoshop with the following settings :
Mode : RGB color, 8 Bits/Channel
Format : PNG
Compression : Smallest / Slow
Interlace : None
Considering it has only one color I suppose I could use other modes (such as Grayscale maybe ?) so tell me if you have suggestions about that.
What I want to do is REPLACE the Black color by another color in my java application.
Reading the other topics it seems that changing the ColorModel is the good thing to do, but I'm honestly totally lost on how to do it correctly.
Here is what I did :
public Test() {
BufferedImage image, newImage;
IndexColorModel newModel;
try {
image = ImageIO.read(new File("E:\\MyImage.png"));
}
catch (IOException e) {e.printStackTrace();}
newModel = createColorModel();
newRaster = newModel.createCompatibleWritableRaster(image.getWidth(), image.getHeight());
newImage = new BufferedImage(newModel, newRaster, false, null);
newImage.getGraphics().drawImage(image, 0, 0, null);
}
private IndexColorModel createColorModel() {
int size = 256;
byte[] r = new byte[size];
byte[] g = new byte[size];
byte[] b = new byte[size];
byte[] a = new byte[size];
for (int i = 0; i < size; i++) {
r[i] = (byte) 21;
g[i] = (byte) 0;
b[i] = (byte) 149;
a[i] = (byte) i;
}
return new IndexColorModel(16, size, r, g, b, a);
}
This produces the expected result.
newImage is the same than image with the new color (21;0;149).
But there is a flaw : the following line is too slow :
newImage.getGraphics().drawImage(image, 0, 0, null);
Doing this on a big image can take up to a few seconds, while I need this to be instantaneous.
I'm pretty sure I'm not doing this the good way.
Could you tell me how achieve this goal efficiently ?
Please consider that my image will always be single color with alpha, so suggestions about image format are welcomed.
If your input image always in palette + alpha mode, and using an IndexColorModel compatible with the one from your createColorModel(), you should be able to do:
BufferedImage image ...; // As before
IndexColorModel newModel = createColorModel();
BufferedImage newImage = new BufferedImage(newModel, image.getRaster(), newModel.isAlphaPremultiplied(), null);
This will create a new image, with the new color model, but re-using the raster from the original. No creating of large data arrays or drawing is required, thus it should be very fast. But: Note that image and newImage will now share backing buffer, so any changes drawn in one of them, will be reflected on the other (probably not a problem in your case).
PS: You might need to tweak your createColorModel() method to get the exact result you want, but as I don't have your input file, I can't very if it works or not.
I have a program that requires going through BufferedImages pixel by pixel quite often. Normally efficiency wouldn't matter enough for me to care, but I really want every millisecond I can get.
As an example, right now, the fastest way I've found of isolating the red channel in an image looks like this:
int[] rgb = image.getRGB(0, 0, img.getWidth(), img.getHeight(), null, 0, img.getWidth());
for(int i = 0; i < rgb.length; i ++)
rgb[i] = rgb[i] & 0xFFFF0000;
image.setRGB(0, 0, img.getWidth(), img.getHeight(), rgb, 0, img.getWidth());
Which means it's going through the image once to populate the array, again to apply the filter, and finally a third time to update the pixels. Also considering that any pixel with an alpha channel of 0 gets completely zeroed out, It must be going through it at least one more time as well.
I've also tried using the individual pixel versions of getRGB() and setRGB() in a more traditional nested for loop, but that's even slower (though probably takes far less RAM since it doesn't have the int[]).
This is the kind of problem that Iterable types solve, but I can't find any way of applying that principle to images. For this project, I'm okay with total hacks that aren't "best practices" if it works.
Is there any way of iterating over the raw data of a buffered image?
The fastest way is to access the DataBuffer because you have a direct access to the pixels value, but you have to handle all the cases/types:
public void Test(BufferedImage img)
{
switch ( img.getType() )
{
case BufferedImage.TYPE_BYTE_GRAY :
case BufferedImage.TYPE_3BYTE_BGR :
byte[] bufferbyte = ((DataBufferByte)img.getRaster().getDataBuffer()).getData() ;
//...
break ;
case BufferedImage.TYPE_USHORT_GRAY :
short[] buffershort = ((DataBuffer)img.getRaster().getDataBuffer()).getData() ;
//...
break ;
//Other cases
}
}
I am a newbie in OpenGL programming. I am making a java program with OpenGL. I drew many cubes inside. I now wanted to implement a screenshot function in my program but I just couldn't make it work. The situation is as follow :
I used FPSanimator to refresh my drawable in 60 fps
I drew dozens of cubes inside my Display.
I added a KeyListener to my panel, if I pressed the alt key, the program will run the following method :
public static void exportImage() {
int[] bb = new int[Constants.PanelSize.width*Constants.PanelSize.height*4];
IntBuffer ib = IntBuffer.wrap(bb);
ib.position(0);
Constants.gl.glPixelStorei(GL2.GL_UNPACK_ALIGNMENT, 1);
Constants.gl.glReadPixels(0,0,Constants.PanelSize.width,Constants.PanelSize.height,GL2.GL_RGBA,GL2.GL_UNSIGNED_BYTE,ib);
System.out.println(Constants.gl.glGetError());
ImageExport.savePixelsToPNG(bb,Constants.PanelSize.width,Constants.PanelSize.height, "imageFilename.png");
}
// Constant is a class which I store all my global variables in static type
The output in the console was 0, which means no errors. I printed the contents in the buffer and they were all zeros.
I checked the output file and it was only 1kB.
What should I do? Are there any good suggestions for me to export the screen contents to an image file using OpenGL? I heard that there are several libraries available but I don't know which one is suitable. Any help is appreciated T_T (plz forgive me if I have any grammatical mistakes ... )
You can do something like this, supposing you are drawing to the default framebuffer:
protected void saveImage(GL4 gl4, int width, int height) {
try {
GL4 gl4 = GLContext.getCurrentGL().getGL4();
BufferedImage screenshot = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
Graphics graphics = screenshot.getGraphics();
ByteBuffer buffer = GLBuffers.newDirectByteBuffer(width * height * 4);
gl4.glReadBuffer(GL_BACK);
gl4.glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
for (int h = 0; h < height; h++) {
for (int w = 0; w < width; w++) {
graphics.setColor(new Color((buffer.get() & 0xff), (buffer.get() & 0xff),
(buffer.get() & 0xff)));
buffer.get();
graphics.drawRect(w, height - h, 1, 1);
}
}
BufferUtils.destroyDirectBuffer(buffer);
File outputfile = new File("D:\\Downloads\\texture.png");
ImageIO.write(screenshot, "png", outputfile);
} catch (IOException ex) {
Logger.getLogger(EC_DepthPeeling.class.getName()).log(Level.SEVERE, null, ex);
}
}
Essentially you create a bufferedImage and a direct buffer. Then you use Graphics to render the content of the back buffer pixel by pixel to the bufferedImage.
You need an additional buffer.get(); because that represents the alpha value and you need also height - h to flip the image.
Edit: of course you need to read it when there is what you are looking for.
You have several options:
trigger a boolean variable and call it directly from the display method, at the end, when everything you wanted has been rendered
disable the automatic buffer swapping, call from the key listener the display() method, read the back buffer and enable the swapping again
call from the key listener the same code you would call in the display
You could use Robot class to take screenshot:
BufferedImage screenshot = new Robot().createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
ImageIO.write(screenshot, "png", new File("screenshot.png"));
There are two things to consider:
You take screenshot from screen, you could determine where the cordinates of you viewport are, so you can catch only the part of interest.
Something can reside a top of you viewport(another window), so the viewport could be hided by another window, it is unlikely to occur, but it can.
When you use buffers with LWJGL, they almost always need to be directly allocated. The OpenGL library doesn't really understand how to interface with Java Arrays™, and in order for the underlying memory operations to work, they need to be applied on natively-allocated (or, in this context, directly allocated) memory.
If you're using LWJGL 3.x, that's pretty simple:
//Check the math, because for an image array, given that Ints are 4 bytes, I think you can just allocate without multiplying by 4.
IntBuffer ib = org.lwjgl.BufferUtils.createIntBuffer(Constants.PanelSize.width * Constants.PanelSize.height);
And if that function isn't available, this should suffice:
//Here you actually *do* have to multiply by 4.
IntBuffer ib = java.nio.ByteBuffer.allocateDirect(Constants.PanelSize.width * Constants.PanelSize.height * 4).asIntBuffer();
And then you do your normal code:
Constants.gl.glPixelStorei(GL2.GL_UNPACK_ALIGNMENT, 1);
Constants.gl.glReadPixels(0, 0, Constants.PanelSize.width, Constants.PanelSize.height, GL2.GL_RGBA, GL2.GL_UNSIGNED_BYTE, ib);
System.out.println(Constants.gl.glGetError());
int[] bb = new int[Constants.PanelSize.width * Constants.PanelSize.height];
ib.get(bb); //Stores the contents of the buffer into the int array.
ImageExport.savePixelsToPNG(bb, Constants.PanelSize.width, Constants.PanelSize.height, "imageFilename.png");
Nearest neighbor scaling works: The entire picture stays intact when I use TYPE_NEAREST_NEIGHBOR.
Even though it is Scala code, all used libraries are standard Java libraries.
Functions:
def getBufferedImage(imageFile: java.io.File): BufferedImage = {
ImageIO.read(imageFile)
}
def scaleImage(image: BufferedImage, minSize: Double): BufferedImage = {
val before: BufferedImage = image
val w = before.getWidth()
val h = before.getHeight()
val affit = new AffineTransform()
var scale = 1.0
if(h < w) {
if(h > 0) {
scale = minSize / h
}
} else {
if(w > 0) {
scale = minSize / w
}
}
affit.scale(scale, scale)
val affitop = new AffineTransformOp(affit, AffineTransformOp.TYPE_BICUBIC)
affitop.filter(before, null)
}
def getImageJpegByteArray(image: BufferedImage): Array[Byte] = {
val baos = new java.io.ByteArrayOutputStream()
val mcios = new MemoryCacheImageOutputStream(baos)
ImageIO.write(image, "jpeg", mcios)
mcios.close()
baos.toByteArray
}
Calling code snippet:
val img = getBufferedImage(imageFile)
val scaledImg = scaleImage(img, 512)
val result = getImageJpegByteArray(scaledImg)
// result is written to SQLite database
result is written to an SQLite database. If I download it from the database and save it as JPEG file, the resulting JPEG is
as expected if I use AffineTransformOp.TYPE_NEAREST_NEIGHBOR
completely black if I use AffineTransformOp.TYPE_BILINEAR
completely black if I use AffineTransformOp.TYPE_BICUBIC
Consequently, I accuse AffineTransformOp of being buggy...
How can I solve this problem?
File magic number of result is always ff d8 ff as expected for JPEG.
Details
Java version: Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_71
Operating System: Apple, OS X 10.9.5
Test image: http://www.photos-public-domain.com/wp-content/uploads/2012/05/thundercloud-plum-blossoms.jpg
I was able to reproduce your issue on Java 1.7.0_71 on OS X 10.10.4 (I rewrote your code in Java, I can post the full code if you are interested).
In any case, the problem is not that AffineTransformOp is buggy in itself. In my test program I displayed the image using a minimal Swing JFrame and the scaled image looked all good there. This is likely why most people in the comments did not understand the problem.
Part of the issue is that the BufferedImage returned by AffineTransformOp when you don't provide a destination to the filter method (the second parameter, null in your case), it will create one for you. This image will get type BufferedImage.TYPE_INT_ARGB. Here is the relevant code from AffineTransformOp.createCompatibleDestImage() (lines 456-468, I kept the formatting, to make it easier to spot):
ColorModel cm = src.getColorModel();
if (interpolationType != TYPE_NEAREST_NEIGHBOR &&
(cm instanceof IndexColorModel ||
cm.getTransparency() == Transparency.OPAQUE)
{
image = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
}
else {
image = new BufferedImage(cm,
src.getRaster().createCompatibleWritableRaster(w,h),
cm.isAlphaPremultiplied(), null);
}
Notice the special case for TYPE_NEAREST_NEIGHBOR, which explains why you'll get different behavior when using nearest neighbor algorithm.
Normally this is all good, however (as I said, the image displays just fine in a Swing component).
The problem arises when you try to store this image as a JPEG. During the years, there's been a lot of confusion and issues related to the ImageIO JPEG plugin and whether it will allow you to write images with alpha channel (like your TYPE_INT_ARGB image). It does allow that. But, most often ARGB JPEGs will get misinterpreted as CMYK JPEGs (as they are 4 channels, and storing ARGB data in JPEG is very exotic) and will be displayed in all funky colors. In your case though, it seems to be all black...
So, there are two possible solutions:
Either write your image in a file format that supports alpha channel, like PNG or TIFF (TIFF requires an extra plugin, so it might not be the best choice). Like this:
ImageIO.write(image, "PNG", mcios);
Or, make sure your BufferedImage is in a pixel format without alpha channel before storing as JPEG. You can do this after the scaling, but the easiest (and fastest) is to just provide the AffineTransformOp with an explicit destination image, like this:
Rectangle newSize = affitop.getBounds2D(before).getBounds();
return affitop.filter(before,
new BufferedImage(newSize.width, newSize.height, BufferedImage.TYPE_3BYTE_BGR));
Here is your image, scaled by the program, using JPEG format and the TYPE_3BYTE_BGR:
I'm sure you can rewrite my Java code back to Scala. :-)
I'm trying to take a BufferedImage, apply a Fourier transform (using jtransforms), and write the data back to the BufferedImage. But I'm stuck creating a new Raster to set the results back, am I missing something here?
BufferedImage bitmap;
float [] bitfloat = null;
bitmap = ImageIO.read(new File("filename"));
FloatDCT_2D dct = new FloatDCT_2D(bitmap.getWidth(),bitmap.getHeight());
bitfloat = bitmap.getData().getPixels(0, 0, bitmap.getWidth(), bitmap.getHeight(), bitfloat);
dct.forward(bitfloat, false);
But I'm stumped trying to finish off this line, what should I give the createRaster function? The javadocs for createRaster make little sense to me:
bitmap.setData(Raster.createRaster(`arg1`, `arg2`, `arg3`));
I'm starting to wonder if a float array is even necessary, but there aren't many examples of jtransforms out there.
Don't create a new Raster. Use WritableRaster.setPixels(int,int,int,int,float[]) to write the array back to the image.
final int w = bitmap.getWidth();
final int h = bitmap.getHeight();
final WritableRaster wr = bitmap.getData();
bitfloat = wr.getPixels(0, 0, w, h, bitfloat);
// do processing here
wr.setPixels(0, 0, w, h, bitfloat);
Note also that if you're planning to display this image, you should really copy it to a screen-compatible type; ImageIO seldom returns those.
I'm doing Google searches for FloatDCT_2D to see what package/library it's in, and it looks like there are several references to various sources, such as "edu.emory.mathcs.jtransforms.dct.FloatDCT_2D". Without knowing what custom library you're using, it's really hard to give you any advice on how to perform the transform.
My guess is in general, that you should read the input data from the original raster, perform the transform on the original data, then write the output to a new raster.
However, your last statement all on it's own looks odd... Raster.createRaster() looks like you're calling a static method with no parameters on a class you've never referenced in the code you've posted. How is that generating data for your bitmap??? Even in pseudo code, you would need to take the results of your transform and build the resultant raster.