How to wait for garbage collector instead throwing OutOfMemoryError by JVM? Is there is some setting for JVM or othrer options (like code practices)?
I don't want to incement JVM memory settings or tuning GC - only wait for GC with no OutOfMemoryError cause I know there is no memory leaks, just garbage preventing new allocation.
I'm afraid, your question doesn't make a lot of sense.
An OutOfMemoryException is normally thrown after the GC has run and has been unsuccessful in reclaiming enough memory for you to proceed. Waiting for the GC to run (again) is unlikely to help. And if it doesn't help, the result is that your application will just freeze.
Besides, there isn't a way to do it.
You can probably tune the threshold for when the JVM will give up and throw OOM, but this is what the JVM does by design when it detects that garbage collection is not accomplishing anything. Note that the JVM will not throw an OOM because of bad timing or just because you've created a lot of objects. It will detect that it has repeatedly run GC and GC hasn't freed up any significant amount of memory.
Some possibilities:
You are using a lot of memory on a permanent basis. This isn't necessarily a memory leak, maybe you just load some huge data and don't realize how big it is in memory.
You have a memory leak or maybe you prefer "memory used in unexpected ways". Java offers lots of easy places to lose memory. I've been killed by ThreadLocal caches in a JSON library and failure to call new String(string) when appropriate.
Temporary data is drifting into PermGen because it doesn't act all that temporary.
You don't have any big problems, but you're pushing the envelope for the amount of memory you have allocated and you haven't tuned properly. Turn on concurrent mark sweep garbage collector, turn on GC logging, and see if behavior matches with your expectations of what the app is doing.
Lastly, run a profiler to see what you're using memory on. The first iteration of any program almost always has huge low hanging fruit to clean up.
Related
we have a really strange behaviour with one of our apps running on a Tomcat 7 (with Java 1.6)
The app runs really good for some days, then we see a peak in the Garbage Collector Time, the CPU Usage is more than 10 times of he normal load and the memory isn't freed anymore:
The last drop was a restart of the app so the performance gets better. as you can see in the graph the space which is freed by the GC gets lower and lower at each run and at the end it isn't going to free any memory, so the performance of the app goes really low.
how can this behaviour be improved?
This looks like a memory leak - if the GC can't free the memory any more, it is most probably due to some code retaining references to unused objects. You should try to track the objects remaining in memory (that graphic tool of yours should have some way to peek into the heap memory regions and give you information on created objects) and make sure you erase any reference to unused objects so the GC can free them.
I have such problem that jvm is not able to perform gc in time and application freezes. "Solution" for that is to connect to application using jconsole and suggest jvm to make garbage collections. I do not have to say that it is very poor behavior of application. Are there some option for jvm to suggest to it to perform gc sooner/more often? Maybe there are some other real solution to this problem?
The problem appears not to be not enough of memory but that gc is not able to do collection in time before new data is send to application. It is so because gc appears to start to collect data to late. If is is suggested early enough by System.gc() button of jconsole then problem does not occur.
Young generation is collected by 'PS Scavenge' which is parallel collector.
Old generation is collected by 'PS MarkSweep' which is parallel mark and sweep collector.
You should check for memory leaks.
I'm pretty sure you won't get OutOfMemoryException unless there's no memory to be released and no more available memory.
There is System.gc() that does exactly what you described: It suggests to the JVM that a garbage collection should take place. (There are also command-line arguments for the JVM that can serve as directives for the memory manager.)
However, if you're running out of memory during an allocation, it typically means that the JVM did attempt a garbage collection first and it failed to release the necessary memory. In that case, you probably have memory leaks (in the sense of keeping unnecessary references) and you should get a memory profiler to check that. This is important because if you have memory leaks, then more frequent garbage collections will not solve your problem - except that maybe they will postpone its manifestation, giving you a false sense of security.
From the Java specification:
OutOfMemoryError: The Java Virtual Machine implementation has run out
of either virtual or physical memory, and the automatic storage
manager was unable to reclaim enough memory to satisfy an object
creation request.
You can deploy java melody on your server and add your application on it, it will give you detailed report of your memory leaks and memory usage. With this you will be able to optimize your system and code correctly.
I guess, either your application requires more memory to run efficiently, try tuning your JVM by setting parameters like -Xms512M -Xmx1024M.
Or,
There is memory leak which is exhausting the memory.
You should check the memory consumption pattern of your application. e.g. what memory it is occupying when it is processing more vs remain idle.
If you observe a constant surge in memory peaks, it could suggest towards a possible memory leak.
One of the best thread on memory leak issue is How to find a Java Memory Leak
Another good one is http://www.ibm.com/developerworks/library/j-leaks/
Additionally,
you may receive an OOME if you're loading a lot of classes (let's say, all classes present in your rt.jar). Since loaded classes reside in PermGen rather than heap memory, you may also want to increase your PermGen size using -XX:MaxPermSize switch.
And, of course, you're free to choose a garbage collector – ParallelGC, ConcMarkSweepGC (CMS) or G1GC (G1).
Please be aware that there're APIs in Java that may cause memory leaks by themselves (w/o any programmer's error) -- e. g. java.lang.String#substring() (see here)
If your application freezes, but gets unfrozen by a forced GC, then your problem is very probably not the memory, but some other resource leak, which is alleviated by running finalizers on dead objects. Properly written code must never rely on finalizers to do the cleanup, so try to find any unclosed resources in your application.
You can start the jvm with more memory
java -Xms512M -Xmx1024M
will start the jvm with 512Mb of memory, allowing it to grow to a gigabyte.
You can use System.gc() to suggest to the VM to run the garbage collector. There is no guarantee that it will run immediately.
I doubt if that will help, but it might work. Another thing you could look at is increasing the maximum memory size of the JVM. You can do this by giving the command line argument -Xmx512m. This would give 512 megabytes of heap size instead of the default 128.
You can use JConsole to view the memory usage of your application. This can help to see how the memory usage develops which is useful in detecting memory leaks.
I have a question on my mind. Let's assume that I have two parameters passed to JVM:
-Xms256mb -Xmx1024mb
At the beginning of the program 256MB is allocated. Next, some objects are created and JVM process tries to allocate more memory. Let's say that JVM needs to allocate 800MB. Xmx attribute allows that but the memory which is currently available on the system (let's say Linux/Windows) is 600MB. Is it possible that OutOfMemoryError will be thrown? Or maybe swap mechanism will play a role?
My second question is related to the quality of GC algorithms. Let's say that I have jdk1.5u7 and jdk1.5u22. Is it possible that in the latter JVM the memory leaks vanish and OutOfMemoryError does not occur? Can the quality of GC be better in the latest version?
The quality of the GC (barring a buggy GC) does not affect memory leaks, as memory leaks are an artifact of the application -- GC can't collect what isn't actual garbage.
If a JVM needs more memory, it will take it from the system. If the system can swap, it will swap (like any other process). If the system can not swap, your JVM will fail with a system error, not an OOM exception, because the system can not satisfy the request and and this point its effectively fatal.
As a rule, you NEVER want to have an active JVM partially swapped out. GC event will crush you as the system thrashes cycling pages through the virtual memory system. It's one thing to have a idle background JVM swapped out as a whole, but if you machine as 1G of RAM and your main process wants 1.5GB, then you have a major problem.
The JVM like room to breathe. I've seen JVMs in a GC death spiral when they didn't have enough memory, even though they didn't have memory leaks. They simply didn't have enough working set. Adding another chunk of heap transformed that JVM from awful to happy sawtooth GC graphs.
Give a JVM the memory it needs, you and it will be much happier.
"Memory" and "RAM" aren't the same thing. Memory includes virtual memory (swap), so you can allocate a total of free RAM+ free swap before you get the OutOfMemoryError.
Allocation depends on the used OS.
If you allocate too much memory, maybe you could end up having loaded portions into swap, which is slow.
If the your program runs fater os slower depends on how VM handle the memory.
I would not specify a heap that's not so big to make sure it don't occupy all the memory preventing the slows from VM.
Concerning your first question:
Actually if the machine can not allocate the 1024 MB that you asked as max heap size it will not even start the JVM.
I know this because I noticed it often trying to open eclipse with large heap size and the OS could not allocate the larger heap space the JVM failed to load. You could also try it out yourself to confirm. So the rest of the details are irrelevant to you. If course if your program uses too much swap (same as in all languages) then the performance will be horrible.
Concerning your second question:
the memory leaks vanish
Not possible as they are bugs you will have to fix
and OutOfMemoryError does not occur? Can the quality of GC be better
in the latest version?
This could happen, if for example some different algorithm in GC is used and it manages to kick-in before you seeing the exception. But if you have a memory leak then it would probable mask it or you would see it intermittent.
Also various JVMs have different GCs you can configure
Update:
I have to admit (after see #Orochi note) that I noticed the behavior on max heap on Windows. I can not say for sure that this applies to linux as well. But you could try it yourself.
Update 2:
As an answer to comments of #DennisCheung
From IBM(my emphasis):
The table shows both the maximum Java heap possible and a recommended limit for the maximum Java heap size setting ......It is important to have more physical memory than is required by all of the processes on the machine combined to prevent paging or swapping. Paging reduces the performance of the system and affects the performance of the Java memory management system.
I am having a java application and spawns lot of threads..and due to out of memory error..it dies if it runs for too much time.. Is there a jvm configuration parameter, that I can set so that it will wait for memory when no memory is available, instead of throwing out of memory error.
Back up a little. If your app is creating so many threads that the JVM runs out of memory, you really need to refactor to use some sort of thread-pooling mechanism. You could catch the out-of-memory exception and see if any threads have freed up resources and then return without handling it but that's a bad code smell to me.
I do not really think that would be feasible to set as a jvm parameter. You should have -Xmx and -Xms set to the appropriate values for the JVM. After that in your code you can check current free memory on the heap using Runtime.freeMemory. If the free memory is too low you can probably make your thread sleep till you got enough memory to process.
I have a server application that, in rare occasions, can allocate large chunks of memory.
It's not a memory leak, as these chunks can be claimed back by the garbage collector by executing a full garbage collection. Normal garbage collection frees amounts of memory that are too small: it is not adequate in this context.
The garbage collector executes these full GCs when it deems appropriate, namely when the memory footprint of the application nears the allotted maximum specified with -Xmx.
That would be ok, if it wasn't for the fact that these problematic memory allocations come in bursts, and can cause OutOfMemoryErrors due to the fact that the jvm is not able to perform a GC quickly enough to free the required memory. If I manually call System.gc() beforehand, I can prevent this situation.
Anyway, I'd prefer not having to monitor my jvm's memory allocation myself (or insert memory management into my application's logic); it would be nice if there was a way to run the virtual machine with a memory threshold, over which full GCs would be executed automatically, in order to release very early the memory I'm going to need.
Long story short: I need a way (a command line option?) to configure the jvm in order to release early a good amount of memory (i.e. perform a full GC) when memory occupation reaches a certain threshold, I don't care if this slows my application down every once in a while.
All I've found till now are ways to modify the size of the generations, but that's not what I need (at least not directly).
I'd appreciate your suggestions,
Silvio
P.S. I'm working on a way to avoid large allocations, but it could require a long time and meanwhile my app needs a little stability
UPDATE: analyzing the app with jvisualvm, I can see that the problem is in the old generation
From here (this is a 1.4.2 page, but the same option should exist in all Sun JVMs):
assuming you're using the CMS garbage collector (which I believe the server turns on by default), the option you want is
-XX:CMSInitiatingOccupancyFraction=<percent>
where % is the % of memory in use that will trigger a full GC.
Insert standard disclaimers here that messing with GC parameters can give you severe performance problems, varies wildly by machine, etc.
When you allocate large objects that do not fit into the young generation, they are immediately allocated in the tenured generation space. This space is only GC'ed when a full-GC is run which you try to force.
However I am not sure this would solve your problem. You say "JVM is not able to perform a GC quickly enough". Even if your allocations come in bursts, each allocation will cause the VM to check if it has enough space available to do it. If not - and if the object is too large for the young generation - it will cause a full GC which should "stop the world", thereby preventing new allocations from taking place in the first place. Once the GC is complete, your new object will be allocated.
If shortly after that the second large allocation is requested in your burst, it will do the same thing again. Depending on whether the initial object is still needed, it will either be able to succeed in GC'ing it, thereby making room for the next allocation, or fail if the first instance is still referenced.
You say "I need a way [...] to release early a good amount of memory (i.e. perform a full GC) when memory occupation reaches a certain threshold". This by definition can only succeed, if that "good amount of memory" is not referenced by anything in your application anymore.
From what I understand here, you might have a race condition which you might sometimes avoid by interspersing manual GC requests. In general you should never have to worry about these things - from my experience an OutOfMemoryError only occurs if there are in fact too many allocations to be fit into the heap concurrently. In all other situations the "only" problem should be a performance degradation (which might become extreme, depending on the circumstances, but this is a different problem).
I suggest you do further analysis of the exact problem to rule this out. I recommend the VisualVM tool that comes with Java 6. Start it and install the VisualGC plugin. This will allow you to see the different memory generations and their sizes. Also there is a plethora of GC related logging options, depending on which VM you use. Some options have been mentioned in other answers.
The other options for choosing which GC to use and how to tweak thresholds should not matter in your case, because they all depend on enough memory being available to contain all the objects that your application needs at any given time. These options can be helpful if you have performance problems related to heavy GC activity, but I fear they will not lead to a solution in your particular case.
Once you are more confident in what is actually happening, finding a solution will become easier.
Do you know which of the garbage collection pools are growing too large?....i.e. eden vs. survivor space? (try the JVM option -Xloggc:<file> log GC status to a file with time stamps)...When you know this, you should be able to tweak the size of the effected pool with one of the options mentioned here: hotspot options for Java 1.4
I know that page is for the 1.4 JVM, I can't seem to find the same -X options on my current 1.6 install help options, unless setting those individual pool sizes is a non-standard, non-standard feature!
The JVM is only supposed to throw an OutOfMemoryError after it has attempted to release memory via garbage collection (according to both the API docs for OutOfMemoryError and the JVM specification). Therefore your attempts to force garbage collection shouldn't make any difference. So there might be something more significant going on here - either a problem with your program not properly clearing references or, less likely, a JVM bug.
There's a very detailed explanation of how GC works here and it lists parameters to control memory available to different memory pools/generations.
Try to use -server option. It will enable parallel gc and you will have some performance increase if you use multi core processor.
Have you tried playing with G1 gc? It should be available in 1.6.0u14 onwards.