We Keep Coding

Java OutOfMemoryError, but very few Live Objects seen in JFR?

I have some code that throws an OutOfMemoryError.
I set the JVM to dump on OOM and I opened the dump in Java Flight Recorder.
When inspecting the Live Objects in JFR, I see very few objects (less than 60).
How can I find out the largest object(s) being held in memory and noncollectable at the moment the OOM was triggered?

Objects are sampled, so there is no way you can be sure to see the largest object before OOM.
That said, 60 samples are usually sufficient to find a memory leak, at least if the application has been running for some time and the leak is not negligible in size.
Samples that happens in the beginning are typically singletons and static objects that you have for the whole duration of application. Samples that happen at the end are typically short-lived objects that are to be garbage collected. In JMC you can click in "the middle" of the timeline on top to find memore leak candidates. Then you can look at stack trace and path to GC root and see if you see something suspicious.
You can also use the command line tool and do:
$ jfr print --events OldObjectSample --stack-depth 64 recording.jfr
It will list the samples in chronological order. It may be easier to see each sample than looking at an aggregate. The command line approach is described in detail here

You can't do this in an automated way (like memory analyzer tools do it with heap dumps) due to the nature of data being collected.
It is totally fine that you can see only handful of objects. The reason is how low overhead sampling works - on every new TLAB allocation JFR steps in and takes a few objects from old TLAB. Therefore you don't get all objects recorded, only a representative sample of objects being allocated. This should be enough to give you a ratio of objects in heap. Also, all of the objects reported are live at the point of recording dump.
If you think that you get too little samples to come to a proper conclusion, it might be that your heap is small relative to TLAB size and you might want to reduce TLAB size. This is not advisable in production environment as improper TLAB setting can reduce application performance.
If you had "Memory Leak Detection" to "Object types + Allocation Stack Traces + Path to GC Root" set in the profiling configuration during record, you can trace where live objects go in code after being created and you can reconstruct representative dominator tree that way.
If you care about large objects meaning large themselves (and not retaining most of heap), you can find objects that are larger than TLAB by looking at "TLAB Allocations" page and look for "Total Allocations Outside TLAB" column. This data would be collected only if profiling configuration had "Memory Profiling" set to "Object Allocation and Promotion".
By profiling configuration I mean the file that you specify with settings option when you start recording with JFR. This file can be created using JMC application's "Flight Recording Template Manager".

Use Memory Effectively And Less Volume

How we can compress the amount of memory used in the application
I checked the application memory consumption on the profile 'App' and found that the application consumes about 35 megabytes of phone memory and this is considered inefficient

You could use java memory heap analyzer to identify the parts of your application that use up most of the memory. You can then either optimize your data structures, or decide release parts of the data by setting all references to it to null.
Unintended references to data that is not needed anymore are also refered as "memory leaks". Settings those references to null will cause the garbage collector to remove it from java memory heap.
Use WeakReferences this will help you
A weak reference, simply put, is a reference that isn't strong enough to force an object to remain in memory. Weak references allow you to leverage the garbage collector's ability to determine reachability for you, so you don't have to do it yourself. 

Ways to check at runtime whether an object has been garbage collected in Java? [duplicate]

is there a way to check if an object can be fetched by the garbage collector?
Somewhere in my code I've got a reference to an object:
MyObject mo = myObject;
Then, via Eclipse Debugger, I get the objects memory location. Afterwards, I set the reference null:
mo = null;
Is there any way to check if the previously referenced object is now suitable for garbage collection or if there's somewhere another reference to it?
Thanks a lot,
Stefan

You cannot do this at runtime with an arbitrary object, and in fact it's not fully possible to do this deterministically. However, there are two options that may be suitable depending on your needs:
Take a heap dump after you set the reference to null, and then load it up in a heap analyzer tool such as jhat or a profiler that supports this. These tools should let you traverse the path from the GC roots and thus check if your object is still reachable or not.
Wrap the object in a PhantomReference with a given ReferenceQueue. When the reference is enqueued, you know that the object has been garbage collected. (Unfortunately, if the reference is unqueued it could be because the object is still reachable, or it could be because the GC just hasn't inspected the object yet. As with all GC-related questions, garbage collection is not a deterministic process!)
On the whole though, I agree that the best option is to be aware of memory leak issues and design your application to avoid them. If you do have a memory leak it should be obvious enough, and you can then focus your energies on finding the problem (again by dumping and analysing the heap for objects that are incorrectly reachable).
The steps above are relatively time-consuming, and shouldn't be something that you do after every change just to reassure yourself, but rather are tools you'd use to investigate a specific problem.

No. The only thing to do is to be careful and keep in mind that memory leaks can exist in Java when writing your application.
The only you can do, is to use tools to try to find where memory leaks come from when you noticed such a problem. I would strongly recommend Memory Analyzer for this purpose.

Using Eclipse MAT Not able to find Memory leak created by Array Object

I am debugging a OutOfMemory issue using MAT( Analyzing Heap Dump ) in old java application.
MAT shows a RMI Thread has created Array(BO[150K+]) of My Business Object(BO) which has 150k+ instances, it is consuming around 358 MB ( Xmx is 512 MB). It is a memory leak situation.
One more interesting part I noticed in all the dumps ( created after server crash ) number of instances in Array Object is same.
I am not able to understand how can I find out this Array Object , in which class this array object is created. is there any such direct/indirect feature available in MAT ?
please suggest if any such option available in visualVM or some other tool. Or Some memory analyzer which i can run over codebase.

in Eclipse MAT histogram select the array object and right click and select the
"merge shortest path to GC Root" (exclude weak references) ,
this should show you the creation path all the way upto the base class object which creates this array.

If you need a profiler that can show you where instances were allocated, you can try JProfiler. The heap walker has an "Allocations" view where you can see the cumulated call stacks for any set of objects. To get allocation call stacks, you have to switch on allocation recording, possibly at startup.
Disclaimer: My company develops JProfiler

There is functionality to use OQL on your heap dump in vanilla jVisualVM. One of the functions in OQL is heap.livepaths, which takes an instance as a parameter and outputs all paths that prevent garbage collection.
If you have a concrete class or object that you know should not be there (relevant windows closed for swing and garbage collector forced several times), you can just list those paths and get several examples of reference paths.
Rinse and repeat until all suspect objects DO get garbage collected. Then it becomes more difficult since you no longer have a lead, but you likely found several bugs in your app and may have fixed it to an acceptable degree.
Note: I usually go the hard way around analyzing memory leaks, and it may not work for very complex applications.

Java - Garbage collection

I found very less understanding in web about java gc. I request you to answer or correct my questions possibly by providing some reliable sources. I hope this post will be helpful to many people like me.
Q1) Please correct or improve my understanding about garbage collection:
gc will automatically be called by jvm during the execution of an application. Calling gc explicitly may only make the gc happen little early than it may happen without explicit call. JVM may call gc at irregular intervals, probably frequently.
gc may collect an object when (i) It's not being referenced by. and (ii) It's not referencing to.
but before destroying an object it will call finalize method of that object class where it can execute some code, normally cleanup operations wrt that object. Finalize method will be called only once per one object. If object is saved by finalize method at the first time, next time it can't escape from death.
If there are memory blocks allocated but not reclaimed due to exceptions/thread abortions they are called as memory leaks which may result in scarcity of memory in the system. OS may tackle with it but not in expected response time of running applications which need memory immediately on demand in which case applications may halt/hang with insufficient memory resource error dumps.
Q2) Garbage collection will only be for heap? ( That means, only to collect unreferenced objects? ) If so, what about static variables/constants/objects which are stored in class-method area?
Q3) How can we make a java object imperishable? Meaning... No matter what happens in the application, the object should not be reclaimed by gc through out the application life. Is it possible?
Q4) I am guessing there will be some occasions in program where gc may go wrong...meaning reclaim an object which shouldn't reclaim because it may gonna be used but gc didn't see the future use. Do such mistakes possible about what programmer should suspect or take care of this auto gc provided by java?

There are many wrong facts in your understanding.
Exceptions and thread abortions don't cause memory leaks. Memory leaks only happen when you keep references to objects for too long. A memory leak is thus caused by a bug in your code (or in a library that you're using), but not by the JVM.
A GC collects objects when the object is not reachable from any chain of references from a root object (variable on a thread stack or static variable). An object which still has references to other objects is not a problem at all. An object which is still referenced by other objects, but not reachable anymore is eligible to GC as well. That's what allows building complex graphs of objects with cyclic references without any problem.
Calling gc explicitly may cause GC to execute immediately, later, or not have any effect.
If the GC was buggy, it would be useless. It's not buggy. If it reclaims an object, then this object is not used anymore by the application.
To make an object not perishable, just make sure it's always reachable through a chain of references from a root. It could b referenced from a static variable, or by a local variable from a thread that never ends.

Q1 and Q2 I will try explaining how the variables get stored in JAVA, thus u can see how GC works.
If you declare
Object a ;
you have created a pointer to an object. It gets stored in the STACK but not in the HEAP.
If you later say
a = new Object();
The object gets placed into the HEAP. The pointer in the stack points to the place in the heap where your object is.
Further more, if you later write
a = null;
you have set the pointer in the STACK to nowhere. GC will destroy the item in the HEAP. Please note that the pointer is still in the STACK. Now if that all was a local variable ( inside function ) and the function exits at some time, the pointer in the STACK will be removed too, if it points at something in the heap, it gets removed too.
However if the variable belongs to an object that is let's say instance variable, the pointer will stay until the object itself exists - the object has memory allocated in the HEAP.
I hope you see what's going on here. It's a bit like chain reaction.
Q3 - You can make an object last until the program exits by making it STATIC.
About Q4 I am not quite sure.

On the topic of 4) the only time the GC can reclaim object you don't expect is when you play with References such as WeakReference and SoftReference. These wrapper allow the GC to clean up objects either on a collection, or when memory is low respectively. While this sounds like a great idea for caching it often doesn't behave quite the way you would expect and they are not as useful as they first appear.

Q1) Please correct or improve my understanding about garbage collection
(Most of your assumptions are correct)
A big problem in programming was memory management . At the begining the developers were responsible for its allocation
and release manually, which led to many bugs and memory leaks. Today , in
all modern platforms , including Java , the memory management is done by GC algorithms.
The Garbage Collector ( GC ) is a major component of the JVM and responsible
by releasing the memory that is no longer being used. When the
application releases all references to an object, it can be collected by the GC at any time, but the time is not determined. It depends entirely on the algorithm the garbage collector . In general , the GC will not make collections for each object. It will wait to release
blocks of objects, for optimization.
JVM may call gc at irregular intervals, probably frequently. (realy depends on the algorithm)
Q2) Garbage collection will only be for heap?
NO. The PermGen is out of the HEAP. Is where Class objects, Method, and the pool of strigs are allocated.
This space is also collected by the GC (when the FullGC is executed).
Q3) How can we make a java object imperishable?
If you have an objected that is never dereferenced, then it will always exist. (For example, in a web app, you add an object
in the applicationContext and you never take it from there. It will exist for the entire application, until it is shutdown).
Q4) Do such mistakes possible about what programmer should suspect or take care of this auto gc provided by java?
You should be aware of which GC algorithm your JVM is using, and choose one that best suit your needs.
(you can choose UseParallelGC / UseSerialGC / UseParallelOldGC / UseConcMarkSweepGC)
See here :
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html

Your understanding seem to be right except that OS doesn't do anything to reclaim the memory. When jvm starts you provide with the max heap that program can use and once that is completely used you will get an Out of Memory Error if there is a leak in the system. I also don't think that if you save your object in finalize it will cleaned up next time for sure. I think if you make finalize to reference your object back then it might never be cleaned up, i am not sure though.
The class and other static information goes to perm gen space. Once the program is restarted this area gets cleared up, but its possible to keep an handle of these information leaking which might result into Out of memory in perm gen space.
Make your object global and it will stay forever
GC has a small pause time where it rescues the objects which have been reference back. So NO you don't have to worry about GC claiming objects that are not supposed to.