I am using ParallelOldGC as garbage collector in my application.The max heap size is set as 2 GB and working on Java hot spot 6, 27 update.
My application create so many long life objects due to which old space got full. So as per the parallel GC algorithm , Full GC trigger when the old space is almost full.As the heap size is 2 GB so to clean the old space the collector take more than 100 seconds which is not acceptable.
I am thinking to set threshold on old space say 30 so that when old space completed with 30% then Full GC will be called , as this solution will increase the FullGC count but it will decrease the application pause time.
I have observed that CMS contain such facility with XX:CMSInitiatingOccupancyFraction but due to some drawback of CMS cant switch to CMS.So is there any facility to set in ParallelOldGC
Thanks in advance.
As the heap size is 2 GB so to clean the old space the collector take more than 100 seconds which is not acceptable.
Since what you're actually trying to solve is long pause times you can set a pause time goal via -XX:MaxGCPauseMillis= and the collector will try to meet that.
It may fail for various reasons to meet it, e.g. because it's not allowed to burn enough CPU time on collecting (via GCTimeRatio) or simply because that you have so many old, live objects that the goal cannot be met (use G1 or CMS in those cases).
Related
I have an application which reads XML-responses from a server.
This is working nice, until I try to read ~200.000 XML-responses. When I reach that magic number, the handling time reduces with a factor 10.
When I let it run, at some point the JVM would say that the GC is taking 90% of CPU-time. So I first tried to optimize my code - using fields instead of local variables, using intern on my strings (Since I have a lot of copies) and so on.
This helped a bit, but it still went slow after approx 100k XML-files. I then tried using Visual VM to see what was going on, and what I saw was:
Up until 18:02, everything works fine. Then suddenly the garbage collector is going bananas, and stealing CPU-time, which then in turn stabilizes memory consumption. I would understand this, if we we're hitting maximum memory of the heap, but I've set max heap size at 8 gb.
There is nothing different happening at that point, it's basically a giant loop doing the same thing over and over.
What is happening and what can I do in this situation?
Your heap size is insufficient for your workflow. You may have memory leak, or it just specific of your application.
Normal pattern for parallel GC algorithm (which you have enabled)
Young GC
Young GC
...
Full GC
Though, once old space is full (~5.6 GiB for your setup), pattern would switch to
Full GC
Full GC
Full GC
...
Full GC is order of magnitude longer, so application would stay in GC pause (with high CPU consumption) almost all time. VisialVM incorrectly charts GC CPU usage, in reality blue spikes are as high as orange line on CPU chart.
If memory usage grows due to memory leak, you should address that.
If it is application design specific, you need increase old space by
either increasing total heap size
or reducing young space (-Xmn=SIZE option) to save more memory for old space
Hello I am having a case of 150GB heap memory program using In Memory Data grid. I have some crazy requirement from the operational department to use a single machine. Now we all know what happens in if the parallel garbage collector is used over 150GB probably it will be tens of minutes of garbage collection if the FULL GC is invoked.
My hope was that with Java 9 is coming Shenandoah low pause GC. Unfortunately from what I see it is not listed for delivery in Java 9. Does anyone knows anything about that ?
Never the less, I am wondering how G1 GC will perform for this amount of Heap memory.
And one last question. Since I have non interactive batch application that is supposed to complete in 2 hours lets say. The main goal here is to ensure that the Full GC never kicks in. If I ensure that there is plenty of memory lets say if the maximum heap that can be reached is 150 and I allocate it 250GB may I say with good confidence that the Full GC will never kick in or ? Usually full GC is triggered if the new generation + the old generation touches the maximum heap. Can it be triggered in a different way ?
There is a duplicate request made I will try to explain here why this question is not a duplicate. First we are talking about 150GB Heap which adds completely different dimension to the question. Second I dont use RMI as it is in the question mentioned, third I am asking question about G1 garbage collector in between the lines.Also once we go beyond the 32GB heap barrier we are entering the 64 bit address space you can not convince me that a question in regards of <32GB Heap is the same as a question with heap >32GB Not to mentioned that things have changed a bit since Java 7 for instance PermSpace does not exist.
The rule of thumb for a compacting GC is that it should be able to process 1 GB of live objects per core per second.
Example on an Haswell i7 (4 cores/8 threads) and 20GB heap with the parallel collector:
[24.757s][info][gc,heap ] GC(109) PSYoungGen: 129280K->0K(917504K)
[24.757s][info][gc,heap ] GC(109) ParOldGen: 19471666K->7812244K(19922944K)
[24.757s][info][gc ] GC(109) Pause Full (Ergonomics) 19141M->7629M(20352M) (23.791s, 24.757s) 966.174ms
[24.757s][info][gc,cpu ] GC(109) User=6.41s Sys=0.02s Real=0.97s
The live set after compacting is 7.6GB. It takes 6.4 seconds worth of cpu-time, due to parallelism this translates to <1s pause time.
In principle the parallel collector should be able to handle a 150GB heap with full GC times < ~2 minutes on a multi-core system, even when most of the heap consists of live objects.
Of course this is just a rule of thumb. Some things that can affect it negatively:
paging
thermal CPU throttling
workloads consisting of very large, reference-heavy objects
non-local memory traffic in NUMA configurations
other processes competing for CPU time
heavy use of weak/soft references
In some cases tuning may be necessary to achieve this throughput.
If the Parallel collector does not work despite all that then CMS and G1 can be viable alternatives but only if there is enough spare heap capacity and CPU cores available to the JVM. They need significant breathing room to do their concurrent work without risking a full GC.
It is correct I said no interactive, but still I have a strict license agreements. I need to be finished with the whole processing in an hour. So I can no afford 30 minutes stop the world event.
Basically, you don't really need low pause times in the sense that CMS, G1, Shenandoah or Zing aim for (they aim for <100ms or even <10ms even on large heaps).
All you need is that STW pauses are not so catastrophically bad that they eat a significant portion of your compute time.
This should be feasible with most of the available collectors, ignoring the serial one.
In practice there are some pathological edge cases where they may fall down, but to get to that point you need setup a system with your actual workload and do some test runs. If you experience some real problems, then you can ask a question with more details.
So, the jest of it is, a version of an application at my company is having some memory issues lately, and I'm not fully sure the best way to fix it that isn't just "Allocate more memory", so I wanted to get some guidance.
For the application, It looks like the eden heap is getting full pretty quickly when it has a concurrent users, so objects that won't be alive very long end up in the old heap. After running for a while, the old heap simply gets fulls, and never seems to automatically clean up, but manually running the garbage collection in VisualVM will clear it out (So I assume this means the old heap is full of dead objects)
Is there any setting suggested I could add so garbage collection gets run on the old heap once it gets to a certain threshold? And is there any pitfalls from changing the old/edin ratio from the stock 2:1 to 1:1? For the application, the majority of objects created are what I would consider short lived (From milliseconds to a few minutes)
It looks like the eden heap is getting full pretty quickly when it has a concurrent users, so objects that won't be alive very long end up in the old heap.
This is called "premature promotion"
After running for a while, the old heap simply gets fulls,
When it fills, the GC triggers a major or even a full collection.
never seems to automatically clean up
In which case, it is either used or it is not completely full. It might appear to be almost full, but the GC will be performed when it is actually full.
but manually running the garbage collection in VisualVM will clear it out
So the old gen wasn't almost but not actually full.
I could add so garbage collection gets run on the old heap once it gets to a certain threshold?
You can run System.gc() but this means more work for you application and slow it down. You don't want to be doing this.
If you use the CMS collector you can change the threshold at which it kicks in but unless you need low latency you might be better off leaving your settings as they are.
And is there any pitfalls from changing the old/edin ratio from the stock 2:1 to 1:1?
You reduce the old gen, you you may half the number of GCs you perform and double the amount of time an object can live and not end up in the old gen.
I work in the low latency space and usually set the young space to 24 GB and the old gen to 2 GB. I also use a lot of off heap data so I don't need much old gen. This is not an average use case, but it can work depending on your requirements.
If you are using < 32 GB, just adding a few more GB may be the simplest answer. Also you can use something like -Xmn4g -Xms6g to set the young space and maximum heap not worry about ratios.
For the application, the majority of objects created are what I would consider short lived (From milliseconds to a few minutes)
In that case, ideally you want your eden space large enough so you have a minor collection every few minutes. This way most of your objects will die in the eden space, and not be copied around.
Note: in extreme cases it is possible to have an application produce less than one GB per hour of garbage and run all day with a 24 GB Eden space without even a minor collection.
Suppose I have a JVM 1.7, which runs normally with 1G max. heap size. and uses only ~0.5G of heap.
At some moment the heap usage steeply increased in few minutes (I suspect increased load not a memory leak) and GC (PS MarkSweep) started. The GC was running for about 30 min. and the server did not respond all that time.
The heap stats shows that at that moment Old Gen heap usage increased, Eden decreased, and max Survivor increased too.
The JVM uses default option -XX:+UseParallelGC
I will try to increase the heap, limit the input data size, etc. However I wonder how I can tune the GC to not "stop the world" for half an hour. I don't mind if the server slows down but I do not want it to stop responding at all.
How to configure GC to not stop the world ?
While you can't have guarantees (unless you are on VM like Zing) you can give soft goals to GC which it will try to achieve. See more here.
Oracle Technetwork. pt. Ergonomics
XX:MaxGCPauseMillis=.This is interpreted as a hint that pause times of milliseconds or less are desired; ... Note that these adjustments may cause the garbage collector to reduce the overall throughput of the application and in some cases the desired pause time goal cannot be met.
-XX:GCTimeRatio=.The throughput goal is measured in terms of the time spent doing garbage collection vs. the time spent outside of garbage collection (referred to as application time). The goal is specified by the command line option -XX:GCTimeRatio=, which sets the ratio of garbage collection time to application time to 1 / (1 + ). For example, -XX:GCTimeRatio=19 sets a goal of 1/20 or 5% of the total time in garbage collection. The default value is 99, resulting in a goal of 1% of the time in garbage collection.
PS. However, I have an opinion that if an app with 0.5Gb heap spends 30 minutes in GC there are more chances that there's something wrong with the app, not with GC itself unless you are trying to run the app on some legacy platform.
I wonder whether or not GC time will always be guaranteed to reduce if the heap size is set to be higher.
Thanks!
No, it does not.
What you really care about is the old generation delays (Stop-The-World events). Take CMS for example, increasing the heap you will delay the garbage collection in the old generation, but when it happens it will take a lot of time, since your old generation is big. It takes about 1 second per GB for the CMS this still has to be measured.
Many applications do this kind of things though (personal experience); i.e. increase the heap.
Why?
Well because we could push the old generation collection towards the night in this way, when the application was used by 0.02% of total usage/users.