I'm opening bunch of files using JFileChooser and for each image I create BufferedImage using image = ImageIO.read(path);. Where image is declared as a static field.
Now I've got 30files 1Mb each, and after running 60times read() my memory usage (checked in OS programs manager) grows for about 70Mb.
Because my image variable is static it's not the issue that image content is stored somewhere. So my questions is, why I'm loosing so much memory?
I'm writing app that need to load tons of pictures to memory, is there somewhere a leek? Is it garbage collector task to clean unused data?
Here is my code to read this data:
public class FileListElement {
private static final long serialVersionUID = -274112974687308718L;
private static final int IMAGE_WIDTH = 1280;
// private BufferedImage thumb;
private static BufferedImage image;
private String name;
public FileListElement(File f) throws IllegalImageException {
// BufferedImage image = null;
try {
image = ImageIO.read(f);
// if (image.getWidth() != 1280 || image.getHeight() != 720) {
// throw new IllegalImageException();
// }
} catch (IOException e) {
e.printStackTrace();
}
image.flush();
//
// thumb = scale(image, 128, 72);
// image = null;
name = "aa";
}
}
What's wrong with it?
Maybe I'm doing sth wrong? I need raw pixels from tons of images or compressed images loaded to RAM. So that I could fast access to any pixel of the image.
It's odd that loading 1Mb pic takes much more than 1Mb.
You can't count on the current memory usage being the amount of memory that is needed the garbage collection does not run constantly, especially if you are far from your max memory usage. Try loading more images, you might find there is no issue.
It's odd that loading 1Mb pic takes much more than 1Mb.
Well, I would expect the format stored on disk to possibly be compressed/smaller than a BufferedImage in memory. So I don't think this is odd.
Related
As the title says I'm trying to resize a PNG image in order to reach a target file size (in terms of MegaBytes).
I searched a lot on SO and over the web, found a lot of codes but all of them are not taking in consideration the final file size.
I've arranged some code but trying to optimize performance.
Example:
Source image dimensions = 30 MB
Target file output size = 5 MB
Current flow:
1 - Load the PNG image as BufferedImage
2 - Recursively use Scalr.resize(...) in order to resize the image
2.1 - For each step use ImageIO.write to store the compressed PNG on a temporary file
2.2 - Check the size using File.length, if size on disk is > 5 MB return to step 2
3 - Save the image using ImageIO.write(...)
This method works, fine-tuning some parameters (such as scale factor) I'm able to accomplish the task.
I'm trying to understand if I can improve all by calculating/guessing the final file size without storing the image on a temporary file.
There is a byte[] obj stored into the BufferedImage obj that I can get using BufferedImage.getData().getDataBuffer() that represents the content of the image but obviously the size of this array is 2x o 3x bigger than the final size of the file because of the PNG compression algo.
I've tried some formula in order to calculate the value, something like:
w * h * bitDepth * 8 / 1024 / 1024 but I'm sure that I'm loosing a lot of data and the accounts do not add up!
At the moment I'm using mainly this code:
static void resize(BufferedImage image, String outPath, int scalingFactor) throws Exception {
image = Scalr.resize(image, image.getWidth() - scalingFactor);
// image.getData().getDataBuffer() - the byteArray containing image
File tempFile = File.createTempFile("" + System.currentTimeMillis(), ".png");
ImageIO.write(image, "png", tempFile);
System.out.println("Calculated size in bytes is: " + tempFile.length() + " - factor: " + scalingFactor);
// MAX_SIZE defined in bytes
if (tempFile.length() > MAX_SIZE) {
// recursion starts here
resize(image, outPath, chooseFactor(tempFile, 4));
} else {
// break the recursive cycle
ImageIO.write(image, "png", new File(outPath));
}
}
static int chooseFactor(File image, int scale) {
// MEGABYTE is 1024*1024
double mbSize = (double) image.length() / MEGABYTE;
return (int) ((mbSize / scale) * 100);
}
There is a way to calculate/guess the final file size starting from BufferedImage object?
Please tell me if I have made myself clear or can I give additional information.
Also recommend a more appropriate title for the question if you think it is not explanatory enough.
Thanks.
Any monotone function along image width/height can be used to perform a binary search.
This approach will work well for many changes that may be needed
(changing from PNG to JPG, adding compression, changing optimization
targets) versus an ad-hoc solution such as directly predicting the
size of a PNG (which, for example, could simply change depending on
what library is installed on your production servers or on the client
that your application uses).
The stored bytes is expected to be monotone (anyway my implementation is safe [but not optimal] under no monotonic functions).
This function perform a binary search to the lower domain (e.g. not upscale the image) using any function:
static BufferedImage downScaleSearch(BufferedImage source, Function<BufferedImage, Boolean> downScale) {
int initialSize = Math.max(source.getWidth(), source.getHeight());
int a = 1;
int b = initialSize;
BufferedImage image = source;
while(true) {
int c = (a + b) / 2 - 1;
// fix point
if(c <= a)
return image;
BufferedImage scaled = Scalr.resize(source, c);
if(downScale.apply(scaled)) {
b = c;
} else {
// the last candidate will be the not greater than limit
image = scaled;
a = c;
}
}
}
if we are interested in the final PNG file size, the search function will be the PNG size:
static final Path output = Paths.get("/tmp/downscaled.png");
static long persistAndReturnSize(BufferedImage image) {
if(ImageIO.write(image, "png", output.toFile()))
return Files.size(output);
throw new RuntimeException("Cannot write PNG file!");
}
(you could persist to memory instead filesystem).
now, we can generate images with size no more than any fixed value
public static void main(String... args) throws IOException {
BufferedImage image = ImageIO.read(Paths.get("/home/josejuan/tmp/test.png").toFile());
for(long sz: asList(10_000, 30_000, 80_000, 150_000)) {
final long MAX_SIZE = sz;
BufferedImage bestFit = downScaleSearch(image, i -> persistAndReturnSize(i) >= MAX_SIZE);
ImageIO.write(bestFit, "png", output.toFile());
System.out.println("Size: " + sz + " >= " + Files.size(output));
}
}
with output
Size: 10000 >= 9794
Size: 30000 >= 29518
Size: 80000 >= 79050
Size: 150000 >= 143277
NOTE: if you do not use compression or you admit an approximation, probably you can replace the persistAndReturnSize function by an estimator without persist the image.
NOTE: our search space is size = 1, 2, ... but you could perform a similar search using more parameters like compression level, pixel color space, ... (although, probably, your domain then will be not monotone and you should use https://en.wikipedia.org/wiki/Gradient_descent or similar).
For a project I am working on, I was tasked with creating a way of converting an image into a non-cryptographic hash so it could be easily compared with similar images, however I ran into an issue where the JVM would begin to recklessly consume memory, despite the Java Monitoring & Management Console not reporting any increase in memory consumption.
When I first ran the application, the Task Manager would report values like this:
However after only about 30 seconds, those values would have doubled or tripled.
I used the JMMC to create a dump of the process, but it only reported around 1.3MB of usage:
The strangest part to me is that the application performs an operation which lasts for about 15 seconds, then it waits for 100 seconds (debug), and it is during the 100 seconds of thread sleeping that the memory used doubles.
Here are my two classes:
ImageHashGenerator.java
package com.arkazex.srcbot;
import java.awt.Color;
import java.awt.Image;
import java.awt.image.BufferedImage;
public class ImageHashGenerator {
public static byte[] generateHash(Image image, int resolution) {
//Resize the image
Image rscaled = image.getScaledInstance(resolution, resolution, Image.SCALE_SMOOTH);
//Convert the scaled image into a buffered image
BufferedImage scaled = convert(rscaled);
//Create the hash array
byte[] hash = new byte[resolution*resolution*3];
//Variables
Color color;
int index = 0;
//Generate the hash
for(int x = 0; x < resolution; x++) {
for(int y = 0; y < resolution; y++) {
//Get the color
color = new Color(scaled.getRGB(x, y));
//Save the colors
hash[index++] = (byte) color.getRed();
hash[index++] = (byte) color.getGreen();
hash[index++] = (byte) color.getBlue();
}
}
//Return the generated hash
return hash;
}
//Convert Image to BufferedImage
private static BufferedImage convert(Image img) {
//Create a new bufferedImage
BufferedImage image = new BufferedImage(img.getWidth(null), img.getHeight(null), BufferedImage.TYPE_3BYTE_BGR);
//Get the graphics
image.getGraphics().drawImage(img, 0, 0, null);
//Return the image
return image;
}
}
Test.java
package com.arkazex.srcbot;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
public class Test {
public static void main(String[] args) throws IOException {
//Create a hash
byte[] hash = ImageHashGenerator.generateHash(ImageIO.read(new File("img1.JPG")), 8); //Memory grows to around 150MB here
System.out.println(new String(hash));
try{ Thread.sleep(100000); } catch(Exception e) {} //Memory grows to around 300MB here
}
}
EDIT: The program stopped growing to 300MB after a few seconds for no apparent reason. I had not changed anything in the code, it just stopped doing it.
I think that what you missing here is that some of the image classes use off-heap memory. This is (presumable) invisible to the JMMC because it only gets told about on-heap usage. The OS-level memory usage monitoring sees it ... because it is looking at the total resource consumption of the JVM running your application.
The problem is that the off-heap memory blocks are only reclaimed when the corresponding on-heap image objects are finalized. That only happens when they are garbage collected.
The program stopped growing to 300MB after a few seconds for no apparent reason. I had not changed anything in the code, it just stopped doing it.
I expect that the JVM decided it was time to do a full GC (or something like that) and that caused it to free up lots of space in the off-heap memory pool. That meant the JVM no longer needed to keep growing the pool.
(I am being deliberately vague because I don't actually know how off-heap memory allocation works under the covers in a modern JVM. But if you want to investigate, the JVM source code can be downloaded ...)
See the explanation in the /** comments */
public class Test {
public static void main(String[] args) throws IOException {
//Create a hash
/** Here it allocates (3 * resolution^2 )bytes of memory to a byte array */
byte[] hash = ImageHashGenerator.generateHash(ImageIO.read(new File("img1.JPG")), 8); //Memory grows to around 150MB here
/** And here it again allocates the same memory to a String
Why print a String of 150 million chars? */
System.out.println(new String(hash));
try{ Thread.sleep(100000); } catch(Exception e) {} //Memory grows to around 300MB here
}
}
We have an application which serve images, to speed up the response time, we cache the BufferedImage directly in memory.
class Provider {
#Override
public IData render(String... layers,String coordinate) {
int rwidth = 256 , rheight = 256 ;
ArrayList<BufferedImage> result = new ArrayList<BufferedImage>();
for (String layer : layers) {
String lkey = layer + "-" + coordinate;
BufferedImage imageData = cacher.get(lkey);
if (imageData == null) {
try {
imageData = generateImage(layer, coordinate,rwidth, rheight, bbox);
cacher.put(lkey, imageData);
} catch (IOException e) {
e.printStackTrace();
continue;
}
}
if (imageData != null) {
result.add(imageData);
}
}
return new Data(rheight, rheight, width, result);
}
private BufferedImage generateImage(String layer, String coordinate,int rwidth, int rheight) throws IOException {
BufferedImage image = new BufferedImage(rwidth, rheight, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = image.createGraphics();
g.setColor(Color.RED);
g.drawString(layer+"-"+coordinate, new Random().nextInt(rwidth), new Random().nextInt(rheight));
g.dispose();
return image;
}
}
class Data implements IData {
public Data(int imageWidth, int imageHeight, int originalWidth, ArrayList<BufferedImage> images) {
this.imageResult = new BufferedImage(this.imageWidth, this.imageHeight, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = imageResult.createGraphics();
for (BufferedImage imgData : images) {
g.drawImage(imgData, 0, 0, null);
imgData = null;
}
imageResult.flush();
g.dispose();
images.clear();
}
#Override
public void save(OutputStream out, String format) throws IOException {
ImageIO.write(this.imageResult, format, out);
out.flush();
this.imageResult = null;
}
}
usage:
class ImageServlet extends HttpServlet {
void doGet(req,res){
IData data= provider.render(req.getParameter("layers").split(","));
OutputStream out=res.getOutputStream();
data.save(out,"png")
out.flush();
}
}
Note:the provider filed is a single instance.
However it seems that there is a possible memory leak because I will get Out Of Memory exception when the application keep running for about 2 minutes.
Then I use visualvm to check the memory usage:
Even I Perform GC manually, the memory can not be released.
And Though there are only 300+ BufferedImage cached, and 20M+ memory are used, 1.3G+ memory are retained. In fact, through "firebug" I can make sure that a generate image is less than 1Kb. So I think the memory usage is not healthy.
Once I do not use the cache (comment the following line):
//cacher.put(lkey, imageData);
The memory usage looks good:
So it seem that the cached BufferedImage cause the memory leak.
Then I tried to transform the BufferedImage to byte[] and cache the byte[] instead of the object itself. And the memory usage is still normal. However I found the Serialization and Deserialization for the BufferedImage will cost too much time.
So I wonder if you guys have any experience of image caching?
update:
Since there are so many people said that there is no memory leak but my cacher use too many memory, I am not sure but I have tried to cache byte[] instead of BufferedImage directly, and the memory use looks good. And I can not imagine 322 image will take up 1.5G+ memory,event as #BrettOkken said, the total size should be (256 * 256 * 4byte) * 322 / 1024 / 1024 = 80M, far less than 1Gb.
And just now,I change to cache the byte and monitor the memory again, codes change like this:
BufferedImage ig = generateImage(layer,coordinate rwidth, rheight);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ImageIO.write(ig, "png", bos);
imageData = bos.toByteArray();
tileCacher.put(lkey, imageData);
And the memory usage:
Same codes, same operation.
Note from both VisualVM screenshots that 97.5% memory consumed by 4,313 instances of int[] (Which I assume is by cached buffered image) is not consumed in non-cached version.
Although you have a less than 1K PNG image (which is compressed as per PNG format), this single image is being generated out of multiple instances of buffered image (which is not compressed). Hence you cannot directly co-relate image size from browser to memory occupied on server. So issue here is not memory leak but amount of memory required to cache this uncompressed layers of buffered images.
Strategy to resolve this is to tweak your caching mechanism:
If possible use compressed version of layers cached instead of raw
images
Ensure that you will never run out of memory by limiting cache size
by instances or by amount of memory utilized. Use either LRU or LIRS
cache eviction policy
Use custom key object with coordinate and layer as two separate
variables overriding with equals/hashcode to use as key.
Observe the behavior and if you have too many cache misses then you
will need better caching strategy or cache may be unnecessary
overhead.
I believe you are caching layers as you expect combinations of layer
and coordinates and hence cannot cache final images but depending on kind of
pattern of requests you expect you may want to consider that option if possible
Not sure what caching API you are using or what are actual values in your request. However based of visualvm it looks to me that String objects are leaking. Also as you mentioned if you turn off caching, problem is resolved.
Consider extract of below snippet of your code.
String lkey = layer + "-" + coordinate;
BufferedImage imageData = cacher.get(lkey);
Now here are few things for you to consider for this code.
You possibly getting new string objects each time for lkey
Your cache has no upper limit with and no eviction policy (e.g. LRU)
Cacher instead of doing String.equals() is doing == and since this
are new string objects they never match causing new entry each time
VisualVM is a start but it doesn't give the complete picture.
You need to trigger a heap dump while the application is using a high amount of memory.
You can trigger a heap dump from VisualVM. It can also be done automatically on an OOME if you add this vmarg to the java process:
-XX:+HeapDumpOnOutOfMemoryError
Use Memory Analyzer Tool to open and inspect the heap dump.
The tool is quite capable and can help you walk the object references to discover:
What is actually using your memory.
Why the objects from #1 aren't being garbage collected.
I have a problem saving large (f.e. 12 000 x 9 000 ) images.
I'm developing a graphical editing software ( something like simple Photoshop ) and
The user obviously has to have to ability to save the image.
Lets say I would like to save the image as .png.
Does JAVA always need to use the BufferedImage for saving drawn stuff ?
I know the equation for size of the image is:
Xsize * Ysize * 4 ( red, green, blue, alpha )
So in this case we get over 400 MB.
I know I could save the image in parts ( tiles ) but the user would have to merge them somehow anyway.
Is there any other way to save such a large image without using the BufferedImage ?
Code for saving the image:
public static void SavePanel() {
BufferedImage image = null;
image = new BufferedImage(
(int) (Main.scale * sizeX ),
(int) (Main.scale * sizeY ),
BufferedImage.TYPE_INT_RGB);
g2 = image.createGraphics();
panel.paint(g2);
try {
ImageIO.write(image, "png", new File(FactoryDialog.ProjectNameTxt.getText() + ".png"));
} catch (IOException e) {
}
}
Thank you in advance !
The ImageIO.write(..) methods accept an RenderedImage, not just a BufferedImage. I successfully exploited this fact some time ago to write out really large images. Generally, the writer implementations write out the image sequentially, and ask the RenderedImage only for the pieces they currently need.
From looking at your code, I think it should be possible to hack a RenderedImage implementation which takes your panel in it's constructor and can be passed to ImageIO for writing. During the process, ImageIO will request data from your image. You can then use the panel to create the requested pieces (Raster contents) on the fly. This way, the whole image does not have to be stored in memory at any point. A starting point for this approach is
public class PanelImage implements RenderedImage {
private final Panel panel;
public PanelImage(Panel panel) {
this.panel = panel;
}
/* implement all the missing methods, don't be afraid, most are trivial */
}
Obviously, you should also check if your panel doesn't suffer from the same problem as the BufferedImage. Depending on the nature of you application, you'll have to hold the image in memory at least once anyway (modulo using tiles). But this way you can at least avoid the duplication.
Using native image resizer like image magick instead.
I'm writing a Java Swing application that will load from disk about 1500 png images that range in size from 50kb to 75kb each. I don't need to load all of them at once, but I will need to load at 50 images at a time. I've been trying to load them using the typical:
new javax.swing.ImageIcon(getClass().getResource("myimage.jpeg")
but my application freezes and I run out of memory after about the first 30 images or so.
What is the best way to load these images in such a way that I will not overload the jvm and that i will be able to monitor which have loaded successfully so far? If possible and necessary, I'd wouldn't mind if the application showed a "loading..." screen while the images loaded, but I'm not sure how to do that.
Is caching useful here? I don't quite understand it, but I saw this question about using MediaTracker and I'm not sure how that could be implemented here.
Why not create a wrapper object for each image, load them on-demand, and make use of WeakReferences or SoftReferences. That way the garbage collector can bin the image data when necessary, and you can reload as/when the weak reference is cleared.
The upside is that the memory for the images can be cleared when required for other uses. The downside is that you will have to reload the image prior to display (or whatever you're doing with the image).
If you already know how many .pngs you are going to load, you may want to create an ImageIcon Array and load them one by one from the directory/directories (which would allow you to display a loading... screen).
What I think you should do is increasing the min/max. HeapSize of the JVM when running the application. You can specify them by e.g. adding -Xmx256m as a parameter (this sets the max-heap to 256MB) (and maybe -Xms32m [this sets the min-heap to 32mb]) see http://docs.sun.com/source/817-2180-10/pt_chap5.html#wp57033
You will either add these options when launching your app (e.g. "java -jar myApp.jar -Xmx128m") or to your system's jvm-configuration-file or to your project's build properties.
This piece of code would load the entire directory; if you want only 50 images to be loaded, just fiddle with the start and stop parameters.
As already said, you will have to set the max-heap (-Xmx) to something around 300M (e.g. resolution of 1280x1024 -> 1310720px -> 4 byte/pixel -> 5242880 bytes -> 50 images -> 256MB).
File dir = new File("/path/to/directory");
File[] files = dir.listFiles();
BufferedImage[] images = new BufferedImage[files.length];
for (int i = 0; i < files.length; i++)
{
try
{
images[i] = ImageIO.read(files[i]);
} catch (IOException ex){}
}
As stated by Tedil, you should give more memory to the app by launching with something like:
java -Xmx256m -classpath yourclasspath YourMainClass
To load the images with a "please wait" loading screen and a progress bar is tricky. It's already in the realm of Advanced Swing. If you are using Java 6 I recommend reading up the SwingWorker class.
Here's a demo that shows you one approach:
package com.barbarysoftware;
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.image.BufferedImage;
import java.util.List;
public class ImageLoadingDemo {
public static void main(String[] args) {
final JFrame frame = new JFrame();
frame.setPreferredSize(new Dimension(600, 400));
frame.getContentPane().add(new JLabel("I'm the main app frame", JLabel.CENTER));
frame.pack();
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setLocationRelativeTo(null);
frame.setVisible(true);
final JDialog pleaseWaitDialog = new JDialog(frame, "Loading images", true);
final int imageCount = 50;
final JProgressBar progressBar = new JProgressBar(0, imageCount);
final BufferedImage[] images = loadImages(frame, pleaseWaitDialog, imageCount, progressBar);
System.out.println("images = " + images);
}
private static BufferedImage[] loadImages(JFrame frame, final JDialog pleaseWaitDialog, final int imageCount, final JProgressBar progressBar) {
final BufferedImage[] images = new BufferedImage[imageCount];
SwingWorker<Void, Integer> swingWorker = new SwingWorker<Void, Integer>() {
#Override
protected Void doInBackground() throws Exception {
for (int i = 0; i < imageCount; i++) {
System.out.println("i = " + i);
publish(i);
Thread.sleep(1000); // to simulate the time needed to load an image
// images[i] = ImageIO.read(new File("... path to an image file ..."));
}
return null;
}
#Override
protected void process(List<Integer> chunks) {
final Integer integer = chunks.get(chunks.size() - 1);
progressBar.setValue(integer);
}
#Override
protected void done() {
pleaseWaitDialog.setVisible(false);
}
};
JPanel panel = new JPanel();
panel.add(progressBar);
panel.add(new JButton(new AbstractAction("Cancel") {
public void actionPerformed(ActionEvent e) {
System.exit(0);
}
}));
pleaseWaitDialog.getContentPane().add(panel);
pleaseWaitDialog.pack();
pleaseWaitDialog.setLocationRelativeTo(frame);
swingWorker.execute();
pleaseWaitDialog.setVisible(true);
return images;
}
}
You should be very careful with loading files through the classloader since these resources are not freed while the classloader is active.
Instead use another approach to load the files directly from disk using a java.io.File object. THey can then be discarded without laying invisibly around.
What are you going to do with the images? Do you really need ImageIcon instances, or will an Image do as well? In either case, it may be easier for you to control the loading if you use the synchronous methods in ImageIO instead of the ImageIcon constructors.
If you run into OutOfMemoryErrors already after 30 images, I assume that the images may have a high resolution and/or colour depth, even if they are relatively small on disk. When you load an image, the image is decompressed and requires much more memory (usually 4*width*height bytes for a colour image) than the size of the compressed file. Unless the images are very small, you are probably not able to cache 1500 uncompressed images within a reasonable amount of memory, so you will have to implement some strategy to only load the images you currently need.