I have some code that throws an OutOfMemoryError.
I set the JVM to dump on OOM and I opened the dump in Java Flight Recorder.
When inspecting the Live Objects in JFR, I see very few objects (less than 60).
How can I find out the largest object(s) being held in memory and noncollectable at the moment the OOM was triggered?
Objects are sampled, so there is no way you can be sure to see the largest object before OOM.
That said, 60 samples are usually sufficient to find a memory leak, at least if the application has been running for some time and the leak is not negligible in size.
Samples that happens in the beginning are typically singletons and static objects that you have for the whole duration of application. Samples that happen at the end are typically short-lived objects that are to be garbage collected. In JMC you can click in "the middle" of the timeline on top to find memore leak candidates. Then you can look at stack trace and path to GC root and see if you see something suspicious.
You can also use the command line tool and do:
$ jfr print --events OldObjectSample --stack-depth 64 recording.jfr
It will list the samples in chronological order. It may be easier to see each sample than looking at an aggregate. The command line approach is described in detail here
You can't do this in an automated way (like memory analyzer tools do it with heap dumps) due to the nature of data being collected.
It is totally fine that you can see only handful of objects. The reason is how low overhead sampling works - on every new TLAB allocation JFR steps in and takes a few objects from old TLAB. Therefore you don't get all objects recorded, only a representative sample of objects being allocated. This should be enough to give you a ratio of objects in heap. Also, all of the objects reported are live at the point of recording dump.
If you think that you get too little samples to come to a proper conclusion, it might be that your heap is small relative to TLAB size and you might want to reduce TLAB size. This is not advisable in production environment as improper TLAB setting can reduce application performance.
If you had "Memory Leak Detection" to "Object types + Allocation Stack Traces + Path to GC Root" set in the profiling configuration during record, you can trace where live objects go in code after being created and you can reconstruct representative dominator tree that way.
If you care about large objects meaning large themselves (and not retaining most of heap), you can find objects that are larger than TLAB by looking at "TLAB Allocations" page and look for "Total Allocations Outside TLAB" column. This data would be collected only if profiling configuration had "Memory Profiling" set to "Object Allocation and Promotion".
By profiling configuration I mean the file that you specify with settings option when you start recording with JFR. This file can be created using JMC application's "Flight Recording Template Manager".
Related
I have a repeating process that:
gets some data from the database
builds some objects in memory, adding to a Collection
writes the data from the Collection to a file
All of the objects/Collections go out of scope or are set to null after each iteration. (The Collection is reused for each iteration.)
Using Java VisualVM, I see a graph that looks like this, which seems very odd considering that it's a repeating process. Yes, the data coming back from the database is different, but it's generally the same amount.
Why does the heap size decrease at first?
Why does the used heap get so close to the heap size in the middle?
(the ~30-second blip at 1:43 was just when VisualVM froze momentarily)
I'm not as big of an expert on GC as some are, but the general idea is that when you've started the program you've given it the initial heap size, max heap size and other relevant parameters and then it's go time.
However the GC has plenty of intelligence and different algorithms that are optimized for different kinds of tasks. A naive implementation would just keep the heap size static and then collect the garbage when it's full. That's known as a "stop the world" collection, because the collector needs to stop everything so it can perform a little (or big) clean up.
Modern GC doesn't just cause long pauses into running programs because it needs to clean up, so there's always a little clean up going on as seen from the sawtooth. But when you start a program the GC has no idea what the program is going to do and how it will use memory. Therefore it has to observe what's happening, analyze memory usage and then decide what amounts of memory it needs to keep available for immediate use, whether it needs to grow the current heap size or if it can decrease the current heap size.
Depending on the behaviour of your program and the GC algorithm being used you can see a lot of different patterns. As long as you're not experiencing linear growth that ends up in an OutOfMemoryError, you should be relatively safe. Unless of course you want to optimize what's happening to increase throughput, responsiveness etc., but that's a more advanced subject and is more relevant when you've gotten your code working the way you want it.
I am getting the following error on execution of a multi-threading program
java.lang.OutOfMemoryError: Java heap space
The above error occured in one of the threads.
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
Is there any way to increase the heap space?
What changes should I made to my program so that It will grab less heap space?
If you want to increase your heap space, you can use java -Xms<initial heap size> -Xmx<maximum heap size> on the command line. By default, the values are based on the JRE version and system configuration. You can find out more about the VM options on the Java website.
However, I would recommend profiling your application to find out why your heap size is being eaten. NetBeans has a very good profiler included with it. I believe it uses the jvisualvm under the hood. With a profiler, you can try to find where many objects are being created, when objects get garbage collected, and more.
1.- Yes, but it pretty much refers to the whole memory used by your program.
2.- Yes see Java VM options
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
Ie
java -Xmx2g assign 2 gigabytes of ram as maximum to your app
But you should see if you don't have a memory leak first.
3.- It depends on the program. Try spot memory leaks. This question would be to hard to answer. Lately you can profile using JConsole to try to find out where your memory is going to
You may want to look at this site to learn more about memory in the JVM:
http://developer.streamezzo.com/content/learn/articles/optimization-heap-memory-usage
I have found it useful to use visualgc to watch how the different parts of the memory model is filling up, to determine what to change.
It is difficult to determine which part of memory was filled up, hence visualgc, as you may want to just change the part that is having a problem, rather than just say,
Fine! I will give 1G of RAM to the JVM.
Try to be more precise about what you are doing, in the long run you will probably find the program better for it.
To determine where the memory leak may be you can use unit tests for that, by testing what was the memory before the test, and after, and if there is too big a change then you may want to examine it, but, you need to do the check while your test is still running.
You can get your heap memory size through below programe.
public class GetHeapSize {
public static void main(String[] args) {
long heapsize = Runtime.getRuntime().totalMemory();
System.out.println("heapsize is :: " + heapsize);
}
}
then accordingly you can increase heap size also by using:
java -Xmx2g
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
To increase the heap size you can use the -Xmx argument when starting Java; e.g.
-Xmx256M
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
That means you are creating more objects in your application over a period of time continuously. New objects will be stored in heap memory and that's the reason for growth in heap memory.
Heap not only contains instance variables. It will store all non-primitive data types ( Objects). These objects life time may be short (method block) or long (till the object is referenced in your application)
Is there any way to increase the heap space?
Yes. Have a look at this oracle article for more details.
There are two parameters for setting the heap size:
-Xms:, which sets the initial and minimum heap size
-Xmx:, which sets the maximum heap size
What changes should I made to my program so that It will grab less heap space?
It depends on your application.
Set the maximum heap memory as per your application requirement
Don't cause memory leaks in your application
If you find memory leaks in your application, find the root cause with help of profiling tools like MAT, Visual VM , jconsole etc. Once you find the root cause, fix the leaks.
Important notes from oracle article
Cause: The detail message Java heap space indicates object could not be allocated in the Java heap. This error does not necessarily imply a memory leak.
Possible reasons:
Improper configuration ( not allocating sufficiant memory)
Application is unintentionally holding references to objects and this prevents the objects from being garbage collected
Applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. If the finalizer thread cannot keep up, with the finalization queue, then the Java heap could fill up and this type of OutOfMemoryError exception would be thrown.
On a different note, use better Garbage collection algorithms ( CMS or G1GC)
Have a look at this question for understanding G1GC
In most of the cases, the code is not optimized. Release those objects which you think shall not be needed further. Avoid creation of objects in your loop each time. Try to use caches. I don't know how your application is doing. But In programming, one rule of normal life applies as well
Prevention is better than cure. "Don't create unnecessary objects"
Local variables are located on the stack. Heap space is occupied by objects.
You can use the -Xmx option.
Basically heap space is used up everytime you allocate a new object with new and freed some time after the object is no longer referenced. So make sure that you don't keep references to objects that you no longer need.
No, I think you are thinking of stack space. Heap space is occupied by objects. The way to increase it is -Xmx256m, replacing the 256 with the amount you need on the command line.
To avoid that exception, if you are using JUnit and Spring try adding this in every test class:
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
I have tried all Solutions but nothing worked from above solutions
Solution: In My case I was using 4GB RAM and due to that RAM usage comes out 98% so the required amount if Memory wasn't available. Please do look for this also.If such issue comes upgrade RAM and it will work fine.
Hope this will save someone Time
In netbeans, Go to 'Run' toolbar, --> 'Set Project Configuration' --> 'Customise' --> 'run' of its popped up windo --> 'VM Option' --> fill in '-Xms2048m -Xmx2048m'. It could solve heap size problem.
When viewing my remote application in JVisualVM over JMX, I see a saw-tooth of memory usage while idle:
Taking a heap dump and analysing it with JVisualVM, I see a large chunk of memory is in a few big int[] arrays which have no references and by comparing heap dumps I can see that it seems to be these that are taking the memory and being reclaimed by a GC periodically.
I am curious to track these down since it piqued my interest that my own code never knowingly allocates any int[] arrays.
I do use a lot of libs like netty so the culprit could be elsewhere. I do have other servers with much the same mix of frameworks but don't see this sawtooth there.
How can I discover who is allocating them?
Take a heapdump and find out what objects are holding them. Once you know what objects are holding the arrays you should have an easy time idea figuring out what is allocating them.
It doesn't answer your question, but my question is:
Why do you care?
You've told the jvm garbage collector (GC) it can use up to 1GB of memory. Java is using less than 250M.
The GC tries to be smart about when it garbage collects and also how hard it works at garbage collection. In your graph, there is no demand for memory. The jvm isn't anywhere near that 1GB limit you set. I see no reason the GC should try very hard at all. Not sure why you would care either.
Its a good thing for the garbage collector to be lazy. The less the GC works, the more resources there are available for your application.
Have you tried triggering GC via the JVisualVM "Perform GC" button? That button should trigger a "stop the world" garbage collection operation. Try it when the graph is in the middle of one of those saw tooth ramp ups - I predict that the usage will drop to the base of the saw tooth or below. If it does, that proves that the memory saw tooth is just garbage accumulation and GC is doing the right thing.
Here is an screenshot of memory usage for a java swing application I use:
Notice the sawtooth pattern.
You said you are worried about int[]. When I start the memory profiler and have it profile everything I can see the allocations of int[]
Basically all allocations come from an ObjectOutputStream$HandleTable.growEntries method. It looks like the thread the allocations were made on was spun up to handle a network message.
I suspect its caused by jmx itself. Possibly by rmi (do you use rmi?). Or the debugger (do you have a debugger connected?).
I just thought I'd add to this question that the sawtooth pattern is very much normal and has nothing necessarily to do with your int[] arrays. It happens because new allocations happen in the Eden-gen, and an ephemeral collection only is triggered once it has filled up, leaving the old-gen be. So as long as your program does any allocations at all, the Eden gen will fill up and then empty repeatedly. Especially, then, when you have a regular amount of allocations per unit of time, you'll see a very regular sawtooth pattern.
There are tons of articles on the web detailing how Hotspot's GC works, so there's no need for me to expand on that here. If you don't know at all how ephemeral collection works, you may want to check out Wikipedia's article on the subject (see the "Generational GC" section; "generational" and "ephemeral" are synonymous in this context).
As for the int[] arrays, however, they are a bit mysterious. I'm seeing those as well, and there's another question here on SO on the subject of them without any real answer. It's not actually normal for objects with no references to show up in a heap dump, because a heap dump normally only contains live objects (because Hotspot always performs a stop-the-world collection before actually dumping the heap). My personal guess is that they are allocated as part of some kind of internal JVM data-structure (and therefore only have references from the C++ part of Hotspot rather than from the Java heap), but that's really just a pure guess.
My simple goal: monitor the memory usage of a Java application so I can be warned when the application is getting dangerously close to throwing an OutOfMemoryError.
Yes, simple to state, but coming up with a correct solution seems very complicated. Some of the complicating factors are:
There are different heap regions, each of which can throw an OutOfMemoryError:
The permgen space, which has it's own size limit (set via -XX:MaxPermSize=)
The overall heap space (set via -Xmx)
The VM may allocate almost all of the heap before bothering to garbage collect. If the application uses a lot of soft references, then in fact this will surely happen. So just a high heap allocation percentage does not imply the application is near to throwing an OutOfMemoryError.
It would be nice if System.gc() guaranteed that the VM would reclaim all possibly reclaimable object (unreferenced and/or weakly referenced object), but it doesn't. So invoking System.gc() and then Runtime.freeMemory() is not reliable.
Objects that are queued for finalization take up memory, but (usually) are freed after they are finalized. So whether the finalizer thread has gotten to them or not affects the (apparent) memory usage (does the VM run the finalizer as a last desparate act before throwing OOM? Doesn't look like it.)
Native code takes up memory as well and too much usage of it can lead to OOM (this is not a likely case in my specific application, but does add another complication to the overall picture).
So what is a good and reliable way to answer the question: Is my Java application getting to throwing an OutOfMemoryError?
Put another way, suppose application version X runs fine and has no memory leak, but version X + 1 has a slow unrecognized memory leak. I'd like to be alerted by this monitoring before version X + 1 throws an OutOfMemoryError, but I'd like the exact same monitoring to not give false positives for version X. There may be some tuning required in setting up this monitoring - that's OK.
One possible answer might be something like: what is the maximum, over the past N "full" GC runs, of the heap utilization immediately after the GC run? If this value exceeds X% of the total allocated memory, then sound the alarms.
The idea is to determine "application memory usage" in simple number like a percentage, or even something like LOW, MEDIUM, or HIGH, and then monitor this value.
The jstat command gives lots of relevant information, the problem is boiling it down to a simple answer and avoiding false positives (or negatives) caused by the complicating factors listed above.
If you watch a memory graph of a long-running application (collected with a tool like jconsole, for example) you'll see a characteristic sawtooth pattern: memory usage climbs, then is GC'd back to a baseline, and then it climbs again. For a healthy app, the peaks and valleys are in two straight horizontal lines. For a leaking app, though, the baseline climbs. That's really what you need to watch for: if each successive GC is less effective than the last, then something is rotten in Denmark.
Search the Oracle docs page for the term Detecting Low Memory and Threshold Notifications -- you may be able to devise some alert system based upon built-in MXBeans. Garbage collection appears to be a focus of at least some of the metrics collection.
after searching the web for a while I decided to ask you for help with my problem.
My program should analyze logfiles, which are really big. They are about 100mb up to 2gb. I want to read the files using NIO-classes like FileChannel.
I don't want to save the files in memory, but I want to process the lines immediately. The code works.
Now my problem: I analyzed the Memory usage with the Eclipse MAT plugin and it says about 18mb of data is saved (that fits). But TaskManager in Windows says that about 180mb are used by the JVM.
Can you tell me WHY this is?
I don't want to save the data reading with the FileChannel, i just want to process it. I am closing the Channel afterwards - I thought every data would be deleted then?
I hope you guys can help me with the difference between the used space is shown in MAT and the used space is shown in TaskManager.
MAT will only show objects that are actively references by your program. The JVM uses more memory than that:
Its own code
Non-object data (classes, compiled bytecode e.t.c.)
Heap space that is not currently in use, but has already been allocated.
The last case is probably the most major one. Depending on how much physical memory there is on a computer, the JVM will set a default maximum size for its heap. To improve performance it will keep using up to that amount of memory with minimal garbage collection activity. That means that objects that are no longer referenced will remain in memory, rather than be garbage collected immediately, thus increasing the total amount of memory used.
As a result, the JVM will generally not free any memory it has allocated as part of its heap back to the system. This will show-up as an inordinate amount of used memory in the OS monitoring utilities.
Applications with high object allocation/de-allocation rates will be worse - I have an application that uses 1.8GB of memory, while actually requiring less than 100MB. Reducing the maximum heap size to 120 MB, though, increases the execution time by almost a full order of magnitude.