I am generating a large data structure and write it to hard disk. Afterwards I want to get rid of the object, to reduce the memory consumption. My problem is that after I had forced a garbage collection the amount of used memory is at least as high as it was before garbage collection. I have added a minimal working example what I am doing.
DataStructure data = new DateStructure();
data.generateStructure(pathToData);
Writer.writeData(data);
WeakReference<Object> ref = new WeakReference<Object>(data);
data = null;
while (ref.get() != null) {
System.gc();
}
The code should force a garbage collection on the data object as it is recommended in thread:
Forcing Garbage Collection in Java?
I know this garbage collection does guarantee the deletion of the data object, but in the past I was more successful by using the garbage collection as described at the link as using simply System.gc().
Maybe someone has an answer whats the best way to get rid of large objects.
It seems that this is premature optimization (or rather an illusion of it). System.gc(); is not guaranteed to force a garbage collection. What you are doing here is busy waiting for some non-guaranteed gc to happen. But if the heap does not get filled up the JVM might not start a garbage collection at all.
I think that you should start thinking about this problem when you stress test your application and you can identify this part as a bottleneck.
So in a nutshell you can't really force a gc and this is intentional. The JVM will know when and how to free up space. I think that if you clear your references and call System.gc(); you can move on without caring about whether it gets cleaned up or not. You may read the Official documentation about how to fine-tune the garbage collector. You should rather be using some GC tuning according to the documentation than asking java to GC from your code.
Just a sidenote: the JVM will expand some of the heap's generations if the need arises. As far as I know there is a configuration option where you can set some percentage when the JVM will contract a generation. Use MinHeapFreeRatio/MaxHeapFreeRatio if you don't want Java to reserve memory which it does not need.
This idiom is broken for a whole range of reasons, here are some:
System.gc() doesn't force anything; it is just a hint to the garbage collector;
there is no guarantee when a weak reference will be cleared. The spec says "Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable". When that happens, it is up to the implementation;
even after the weak reference is cleared, there is no telling when its referent's memory will actually be reclaimed. The only thing you know at that point is that the object has transitioned from "weakly reachable" to "finalizable". It may even be resurrected from the finalizer.
From my experience, just doing System.gc a fixed number of times, for example three, with delays between them (your GC could be ConcurrentMarkSweep) in the range of half-second to second, gives much stabler results than these supposedly "smart" approaches.
A final note: never use System.gc in production code. It is almost impossible to make it bring any value to your project. I use it only for microbenchmarking.
UPDATE
In the comments you provide a key piece of information which is not in your question: you are interested in reducing the total heap size (Runtime#totalMemory) after you are done with your object, and not just the heap occupancy (Runtime#totalMemory-Runtime#freeMemory). This is completely outside of programmatic control and on certain JVM implementations it never happens: once the heap has increased, the memory is never released back to the operating system.
Related
As there are so many objects being created for a particular class some of which could be referenced and some can be anonymously lying in the heap and hence elgible for garbage collection . So is the presence of even one non referenced object in the heap is enough for garbage collector to run or is there any other criteria to assure the garbage collector is executed.
Thanks
Jayendra
we don't know when GC get executed, it depend on lots of things e.g. Type of GC. Also you can recommend to VM to launch GC by doing System.gc(). But again it is just a recommendation
GC on an object is, for typical reference implementations of the JVM, completely non-deterministic. There is no way to guarantee when, or even if, the GC will be run, never mind if it will result in a particular object reference being released (and its corresponding finalizer called.) This is regardless of the state of this particular object.
As pointed out in comments, System.gc() is merely a hint or request. In a great many cases it will result in a full GC. But consider gathering a heap dump usually results in 2-3 calls to System.gc() before the dump is generated, mostly as a way to improve the chances it actually happens.
There are experimental VMs that offer various implementations of real-time guarantees where some of this is not true.
I recently modified a bing chunk of my code and the garbage collector went crazy and I can't figure out what does it deletes. So I'd like to get the names of the things that the garbage collector deletes.
I've tried Android Device Monitor but I can't figure out where or how to find it or if it is even possible.
What should I do to figure out where to modify my code?
finalize(),
this API is provided by Java which JVM calls whenever an object is about to go through GC, although no gurantee if it will be called as per JAVA DOC, you can try printing name here.
Hope this solve ur issue.
(Actually, if the GC has "gone crazy", a more likely problem is the objects it can't delete than the ones that it deletes.)
Firstly, objects don't (in general) have names.
Secondly, I don't think it is possible to track when the GC actually deletes objects.
So tracking object deletion is unlikely to work ... and probably wouldn't help anyway.
What I think you really need is a memory profiler tool (e.g. DDMS) that can tell you were the objects are allocated, and can help you analyse all of the objects that are live (reachable) and garbage (unreachable).
The two most likely things that can make the GC use a lot of time (or "go crazy") are:
Having too much reachable data. As the heap gets close to full, the percentage of time spent attempting to collect garbage increases dramatically. A common cause of having too much reachable data is a storage leak in your application.
Creating too many temporary objects. A higher allocation rate means that the collector has to run more often.
References:
https://developer.android.com/tools/debugging/debugging-memory.html
http://developer.android.com/tools/debugging/ddms.html
I found very less understanding in web about java gc. I request you to answer or correct my questions possibly by providing some reliable sources. I hope this post will be helpful to many people like me.
Q1) Please correct or improve my understanding about garbage collection:
gc will automatically be called by jvm during the execution of an application. Calling gc explicitly may only make the gc happen little early than it may happen without explicit call. JVM may call gc at irregular intervals, probably frequently.
gc may collect an object when (i) It's not being referenced by. and (ii) It's not referencing to.
but before destroying an object it will call finalize method of that object class where it can execute some code, normally cleanup operations wrt that object. Finalize method will be called only once per one object. If object is saved by finalize method at the first time, next time it can't escape from death.
If there are memory blocks allocated but not reclaimed due to exceptions/thread abortions they are called as memory leaks which may result in scarcity of memory in the system. OS may tackle with it but not in expected response time of running applications which need memory immediately on demand in which case applications may halt/hang with insufficient memory resource error dumps.
Q2) Garbage collection will only be for heap? ( That means, only to collect unreferenced objects? ) If so, what about static variables/constants/objects which are stored in class-method area?
Q3) How can we make a java object imperishable? Meaning... No matter what happens in the application, the object should not be reclaimed by gc through out the application life. Is it possible?
Q4) I am guessing there will be some occasions in program where gc may go wrong...meaning reclaim an object which shouldn't reclaim because it may gonna be used but gc didn't see the future use. Do such mistakes possible about what programmer should suspect or take care of this auto gc provided by java?
There are many wrong facts in your understanding.
Exceptions and thread abortions don't cause memory leaks. Memory leaks only happen when you keep references to objects for too long. A memory leak is thus caused by a bug in your code (or in a library that you're using), but not by the JVM.
A GC collects objects when the object is not reachable from any chain of references from a root object (variable on a thread stack or static variable). An object which still has references to other objects is not a problem at all. An object which is still referenced by other objects, but not reachable anymore is eligible to GC as well. That's what allows building complex graphs of objects with cyclic references without any problem.
Calling gc explicitly may cause GC to execute immediately, later, or not have any effect.
If the GC was buggy, it would be useless. It's not buggy. If it reclaims an object, then this object is not used anymore by the application.
To make an object not perishable, just make sure it's always reachable through a chain of references from a root. It could b referenced from a static variable, or by a local variable from a thread that never ends.
Q1 and Q2 I will try explaining how the variables get stored in JAVA, thus u can see how GC works.
If you declare
Object a ;
you have created a pointer to an object. It gets stored in the STACK but not in the HEAP.
If you later say
a = new Object();
The object gets placed into the HEAP. The pointer in the stack points to the place in the heap where your object is.
Further more, if you later write
a = null;
you have set the pointer in the STACK to nowhere. GC will destroy the item in the HEAP. Please note that the pointer is still in the STACK. Now if that all was a local variable ( inside function ) and the function exits at some time, the pointer in the STACK will be removed too, if it points at something in the heap, it gets removed too.
However if the variable belongs to an object that is let's say instance variable, the pointer will stay until the object itself exists - the object has memory allocated in the HEAP.
I hope you see what's going on here. It's a bit like chain reaction.
Q3 - You can make an object last until the program exits by making it STATIC.
About Q4 I am not quite sure.
On the topic of 4) the only time the GC can reclaim object you don't expect is when you play with References such as WeakReference and SoftReference. These wrapper allow the GC to clean up objects either on a collection, or when memory is low respectively. While this sounds like a great idea for caching it often doesn't behave quite the way you would expect and they are not as useful as they first appear.
Q1) Please correct or improve my understanding about garbage collection
(Most of your assumptions are correct)
A big problem in programming was memory management . At the begining the developers were responsible for its allocation
and release manually, which led to many bugs and memory leaks. Today , in
all modern platforms , including Java , the memory management is done by GC algorithms.
The Garbage Collector ( GC ) is a major component of the JVM and responsible
by releasing the memory that is no longer being used. When the
application releases all references to an object, it can be collected by the GC at any time, but the time is not determined. It depends entirely on the algorithm the garbage collector . In general , the GC will not make collections for each object. It will wait to release
blocks of objects, for optimization.
JVM may call gc at irregular intervals, probably frequently. (realy depends on the algorithm)
Q2) Garbage collection will only be for heap?
NO. The PermGen is out of the HEAP. Is where Class objects, Method, and the pool of strigs are allocated.
This space is also collected by the GC (when the FullGC is executed).
Q3) How can we make a java object imperishable?
If you have an objected that is never dereferenced, then it will always exist. (For example, in a web app, you add an object
in the applicationContext and you never take it from there. It will exist for the entire application, until it is shutdown).
Q4) Do such mistakes possible about what programmer should suspect or take care of this auto gc provided by java?
You should be aware of which GC algorithm your JVM is using, and choose one that best suit your needs.
(you can choose UseParallelGC / UseSerialGC / UseParallelOldGC / UseConcMarkSweepGC)
See here :
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html
Your understanding seem to be right except that OS doesn't do anything to reclaim the memory. When jvm starts you provide with the max heap that program can use and once that is completely used you will get an Out of Memory Error if there is a leak in the system. I also don't think that if you save your object in finalize it will cleaned up next time for sure. I think if you make finalize to reference your object back then it might never be cleaned up, i am not sure though.
The class and other static information goes to perm gen space. Once the program is restarted this area gets cleared up, but its possible to keep an handle of these information leaking which might result into Out of memory in perm gen space.
Make your object global and it will stay forever
GC has a small pause time where it rescues the objects which have been reference back. So NO you don't have to worry about GC claiming objects that are not supposed to.
Why the method call System.gc() doesn't guarantee that the Garbage Collector algorithm will run at that moment? Why it cannot certainly reclaim all unused object's memory whenever it is invoked ?
Forcing object destruction is sign of bad coding so Java might wanted to avoid developers from getting addicted to it.
If Java gives you freedom of forcing of object destruction, improper use of it might have adverse affect of application performance.
This restriction allows you (force you) to put more focus into business logic than memory managemnt
JVM is the best person to decide when memory management is required and how to do it.
You can (should) trust JVM and let it handle things in better way than we can.
Still do you really want to force object destruction? If yes, WHY?
In order to ensure that programs run smoothly within the JVM, the JVM itself manages garbage collection.
Garbage collection has become quite sophisticated. When you ask the system for a GC run, which algorithm are you expecting? A "full GC"? There are multiple heaps, as well; which one is the garbage you are concerned about on? You don't know and this method doesn't indicate.
Suppose calling System.gc() always triggered a full GC. An errant program could easily grind JVM performance to a halt. Defensively, the JVM would want to limit the frequency at which it responded to such calls.
If you are running in a JVM on a non-embedded system (e.g. a server or a desktop computer), there should be no reason for you to concern yourself with any aspect of memory management other than to monitor it and code efficiently.
There are several metrics that are utilized to evaluate garbage collector performance , some of them are:
Throughput—the percentage of total time not spent in garbage collection, considered over long periods of time.
Garbage collection overhead—the inverse of throughput, that is, the percentage of total time spent in
garbage collection.
Pause time—the length of time during which application execution is stopped while garbage collection is occurring.
Frequency of collection—how often collection occurs, relative to application execution.
Footprint—a measure of size, such as heap size.
Promptness—the time between when an object becomes garbage and when the memory becomes
available.
Now If JVM listens to System.gc() like good pet and guarantees to perform action on each System.gc() call, Imagine what would be the performance of application if it is called many times within program.!!??
Throughput will decrease
Garbage Collection overhead will increase.
Application will pause many times as it is busy in recollecting the memory.
If Footprint is large , The garbage Collector would have to scan all memory area for recovering the memory , no matter if there are objects eligible for garbage collection or not.
So , after looking through these points I guess it provides the sufficient reason to JVM to not respond to System.gc on the application choice , but on its own algorithm. And Garbage Collection reclaims all unused object's memory for sure , but its invocation is entirely dependent on JVM own Algorithm rather than on the user's choice.
Source: Memory Management in the Java HotSpot™ Virtual Machine - Sun Microsystems
it cannot certainly reclaim all unused object's memory whenever it is invoked
This assumption of yours is false. In most cases the Garbage Collector could reclaim all unused objects at any point in time. However, if the standard Java library provided a method that would guarantee that, it would put a completely unreasonable burden on the GC subsystem to provide a service that is most of the time useless and could be even damaging.
I have a strange doubts. I know garbage collector has its own limitation. and if allocation is
bad then it can cause a problem for application to respond in unusual way.
So my question is that is it good programming habit to call forcefully garbage collector (System.gc()) at the end of each activity?
Update
Every one is saying that calling system.gc() not beneficial at all.Then i am wondering why its present here.DVM will decide when to run garbage collector.Then what is need of that method?
Update 2
Thanks community to help me out. But honestly i got knowledge about Garbage collection real Beauvoir from this link Java Performance Optimization
it isn't good programming habit to call forcefully garbage collector (System.gc()) at the end of each activity
Because it is useless,only DVM decide when it should be call although you called it...
System.gc(), which the VM sometimes ignores at whim, is mostly useful in two cases:
you're gobbling up memory like there's no tomorrow (usually with bitmaps).
you suspect a memory leak (such as accidentally holding onto an old Context), and want to put the VM memory in a quiescent state to see if the memory usage is creeping up, for debugging.
Under nominal circumstances, one should not use it.
I really think it depends on your situation.
Because the heap is generational, the GC may not get rid of certain large objects or bitmaps on its first pass, and its heuristics may not indicate that additional garbage collection is necessary, but there are definitely scenarios where the heuristic could be wrong, and we as the developers have knowledge of a pattern, or can predict usage that the GC cannot, and therefore calling system.gc() will benefit us.
I have seen this before in specific scenarios such as dealing with map tiling or other graphic intensive behaviors, where the native GC in Android (even on 3.0+ devices), doesn't get it right, resulting in Out of Memory errors. However, by adding a few GC calls, the Out of Memory errors are prevented, and the system continues to process albeit at a slower rate (due to garbage collection). In graphic intensive operations, this usually is that state desired (a little lag) over the application crashing because it cannot load additional resources into memory.
My only explanation for why this happens in certain scenarios appears to be timing. If user operations are slow, then the native Android GC seems to do great. However, if your user is scrolling fast, or zooming quickly, this is where I have seen the Android GC lag behind, and a few well thought out System.gc() have resulted in my applications not crashing.
calling System.gc(), doesn't do any harm. but you cant be sure that it will be of some use. because you ask the DVM to do garbage collection, but can't command it... Its dependent totally on DVM. It calls when memory is running out or may be at any time..
I tried putting System.gc() on the line before the lines where I created my bitmap in my Android app. The garbage collector freed up several megabytes in some cases and put and end to my OutOfMemoryError conditions. It did not interfere with the normal garbage collection one bit but it did make my app run faster.
No; if the system needs memory, it will call GC on its own.
Any memory used by an instance, that isn't referenced anywhere else, will become eligible for GC when the instance goes away.
Memory used by the instance itself, if no longer referenced, is also eligible for GC. You can do a code review or profiling to see if you're holding on to memory unnecessarily, but that's a different issue.
Calling GC manually is a bad coding habit...
The Developer docs on RAM usage state:
...
GC_EXPLICIT
An explicit GC, such as when you call gc() (which you should avoid calling and instead trust the GC to run when needed).
...
I've highlighted the most important and relevant part here in bold.
It is possible to ask the Android JVM to run the garbage collector by calling System.gc(). As the documentation states:
Calling the gc() method suggests that the Java Virtual Machine expend effort toward recycling unused objects in order to make the memory they currently occupy available for quick reuse. When control returns from the method call, the Java Virtual Machine has made a best effort to reclaim space from all discarded objects.
Emphasis added!
Some care is needed in interpreting "best effort" in the final sentence:
The "best effort" might be to ignore the "suggestion" entirely. Some JVMs have a configuration option to totally ignore System.gc() calls.
The "best effort" may or may not amount to a full garbage collection. That is an implementation detail.
But the bottom line is that you cannot force the GC to run.
Calling System.gc() is generally a bad idea. It makes your application inefficient, and it may introduce unwanted and unnecessary GC pauses.
The inefficiency issue comes down to the way that modern garbage collectors behave. A garbage collector's work has two parts1:
Finding the objects that are reachable.
Dealing with the objects that are not reachable.
The first part involves traversing reference chains and and marking the graph of objects starting at the GC roots. This work is proportional to the number of reachable objects.
The second part can be handled in a couple of ways, but it will typically be proportional to the size of the reachable objects.
Thus the overall cost of a GC run (in CPU time) depends mostly in the amount of non-garbage. But the benefit of the work performed is the amount of space that you managed to reclaim.
To maximize efficiency, you need to run the GC when the benefit of running the GC is at its highest; i.e. when the heap is close to full. But the problem is that if you call System.gc() you may be requesting a garbage collection when there is lots of free space.
Every one is saying that calling system.gc() not beneficial at all. Then I am wondering why its present here. DVM will decide when to run garbage collector. Then what is need of that method?
It is there for largely historical reasons. The method was present in the System class in Java 1.0. Removing it now would break a lot of legacy code. As for why gc() was included in the first place, the decision was made a long, long time ago, and we were not "in the room" when it was made. My guess is that the decision makers (in ~1995):
were a bit too optimistic about how GC technology would develop,
didn't anticipate that naive programmers would try to use gc() calls to solve memory leaks and other bugs, and / or
were simply too rushed to think too hard about it.
There are also a couple of scenarios where calling System.gc() is beneficial. One such scenario is when your application is about to start a "phase" where unscheduled GC pauses are going to give a particularly bad user experience. By running System.gc() you can take the "performance hit" at a point in time where it matters less; e.g. during a user initiated pause or while switching levels in a game.
But I don't think the above scenario corresponds to your "at the end of every activity".
The final thing to note is that calling System.gc() manually does not prevent normal OOMEs. A normal OOME is typically thrown then the JVM decides there is not enough free heap space to continue. This decision is made immediately after running a (full) GC. Running System.gc() manually won't make any difference to the decision making.
Furthermore, calling System.gc() will not cure normal2 memory leaks. If your application has a memory leak, you actually have a situation where a bunch of objects are reachable when they shouldn't be. But since they are reachable, the GC won't delete them.
The cure for OOMEs is one or more of the following:
Find the memory leaks and fix them. There are tools to help you do this.
Modify the application to use memory more efficiently; e.g. don't keep so much data in memory, or represent it in a more compact form.
Increase the application's heap size.
1 - This is a simplification, but the full story is way to complicated for this posting. I recommend you buy an read an up-to-date book on Garbage Collection of you want (or need) a deeper understanding.
2 - There are cases involving non-heap memory where manually running the GC might help as a band-aid for certain kinds of OOME. But a better solution is to find a better way to reduce non-heap memory usage and/or free up non-heap resources in a more timely fashion.