Releasing java.util.LinkedList$Entry memory - java

I have an application in which the number of java.util.LinkedList$Entry objects seems to be steadily increasing. This application contains a method that contains the following code:
final List<Boolean> correctnessList = new ArrayList<Boolean>();
final List<Double> discriminationList = new ArrayList<Double>();
final List<Double> difficultyList = new ArrayList<Double>();
final List<Double> guessingList = new ArrayList<Double>();
.
.
.
for (ItemData datum : candidateItemData) {
.
.
.
correctnessList.add(datum.isCorrect);
discriminationList.add(iRTParameter.discrimination);
difficultyList.add(iRTParameter.difficulty);
guessingList.add(iRTParameter.guessing);
.
.
.
}
The method that contains this code is called many, many times. And, of course, each time the method returns, the List objects go out of scope, and, presumably, are available for garbage collection.
However, as I said, the number of java.util.LinkedList$Entry objects seems to be steadily increasing.
Have I created a memory leak here? Should I call some method on the List objects at the end of the method so that the LinkedList$Entry objects can be garbage collected?

No, you don't need to do any explicit de-initialization for the objects to be claimable.
Your best bet is to find out why the elements are not garbage collected. To do this, use your prefered memory profiler, take a snapshot and try to trace some of those elements path to the nearest GC route (personally I'd suggest VisualVM, since it's relatively simple to use and still powerful enough for many things).
Also: in your sample you use ArrayList as your List implementation. That implementation does not use Entry objects. So you need to check where in your code you use a LinkedList.

Did you check whether the garbage collector actually ran? Under normal circumstances the JVM decides to run the gc at regular intervals or when memory is scarce.
And as Joachim Sauer said, there might be some dangling reference from an active thread to your lists. So if the gc ran but did not collect at least some of those objects (it might sometimes not collect all objects that are eligible for gc, so that's not generally a problem) you should check where the references are.
We once had a problem with database connection entries that were closed but not released and thus held tons of data in some maps. Getting rid of the references to those connections helped in that case, but it was only obvious, when we imployed a memory tracing tool (JProbe in our case).

It looks like no memory leaks here. It depends on how you use the result of this. In general, garbage collector will collect all of them.
From the other hand it will be better for memoty usage and hadling when it will be only one list and the data will be wrapped into structure contains these fields. When it will be adding the 17-th item - new blok will be allocated in memory and previous items will be moved to the new block of memory. So it is better to make it only once instead 4 time.
And the last notice is that it is better to use constructor where you can provide count of items. It will allocate appropriative block of memory. It will avoid possible reallocations in when you will fill the collection.

Related

Is it better to create a new HashSet or to reuse after invoking hashSet.clear()

I want to process some new data in a HashSet, without any old data needed or the old HashSet object. The old HashSet object isn't referred to elsewhere.
Is it better to simply do hashset = new HashSet<String>() and let JVM to free the memory of the old HashSet object or should I call hashSet.clear() and reuse the same HashSet?
According to openJDK, hashSet.clear() is:
public void clear() {
map.clear();
}
and map.clear() :
public void clear() {
modCount++;
Entry[] tab = table;
for (int i = 0; i < tab.length; i++)
tab[i] = null;
size = 0;
}
Since map.clear() iterates all the entries, will it be time consuming when the hashSet is large? Which one is recommended in this case, the constructor method or the clear() method?
Simple don't reference your HashSet anymore and let the garbage collector do the work.
Clearing the HashSet before dereferencing it does not free the memory any faster.
Clearing the HashSet is unnecessary - just remove all references to the HashSet (which happens if you write hashset = new HashSet<>();, assuming the old HashSet is not referenced anywhere else) and the garbage collector will do the rest.
Also note that, assuming you need a new HashSet of the same size, calling hashset.clear() takes more time than simply creating a new one with the appropriate capacity
I think there is a misconception here. You don't free memory in java manually. Even if you call System.gc(); that does not guarantee that the GC will be run at that moment.
As for your question: I think that you should use your references in the smallest possible scope and when the code leaves the scope of your Set the reference to it simply gets dropped and the GC will collect it.
clear(); I think is recommended to use if you have some further work with the Set: for example if you processed the data in it and you are preparing to put some other data in it.
In practice I almost never see explicit mySet = null; (this is what you should use if you want explicit dereferencing because mySet = new HashSet<>() does not make any sense) statements because it is much easier and less cumbersome to just use the appropriate scope.
It is not possible to free memory in java. All you can do is let the VM know it's okay to free it if and when it wants to. That will happen normally when things go out of scope. In general, it's okay to not worry about it. If you've profiled your app and you see a memory leak, then you can try to find it.
Definitely don't make a new HashSet. clear() will remove all the items from the HashSet, but it still won't be collected until it's out of scope.
If you don't have any references to your objects garbage collector removes them automatically. You do not need to do it manually.
It depends, if you have very small sized maps then in that case clearing would be better than spawning Objects.
And if you have lots of items in your maps then it is not even worth clearing it. As if you see the clear() method's implementation it iterates over all the elements. In that case creating new object would be a bit cheaper.
But generally it is a good idea to let the Garbage collector take care of clearing up things and you go on creating new objects while the left over reference will be cleared by Garbage collector.
But as a rule of thumb one should try and limit the number of objects being created.
As I said it depends.
The memory management is done by JVM itself. So even if you make the memory free manually or not that will be garbage collected automatically anyway.
The garbage collector reduces the coding effort regarding memory management.

Java, make sure objects are being deleted when removed from arrayList

Lets say that I'm deleting a "dead" object called "Enemy".
Using something like this:
for(int i = 0; i < enemies.size(); i++)
{
Enemy en = (Enemy) enemies.get(i);
if(en.getVisible() == true)
en.update();
else
enemies.remove(i);
}
Does the object get deleted after being removed from ArrayList? Or "should" it? I've been mainly doing C++ code before and the garbage collection confuses me.
Any way I can see if the objects are being deleted or not by the garbage collector?
Thanks
I really like seeing the opposite of the whole garbage collecting fiasco with C/C++ and Java. Java has it's own garbage collector, you do not need to worry about memory management - .remove() will suffice.
If you remove an object from ArrayList, and that object doesn't have any other reference, then it would be 'eligible' for the garbage collector. After that, you need not worry about removing it from the heap: JVM would do that through automatic garbage collector.
I think it depends on how you entered the object. If you saved it elsewhere it should be still in existence, however if you directly added it to the arrayList it probably is garbage.
P.S. your code needs a correction
enemies.remove(i);
enemies.remove(i--);
Taking my comment about ConcurrentModificationException back. You won't get it, but your loop is still not correct. Take a look:
List<String> list = new ArrayList<String>(Arrays.asList("A","B","C","D"));
for (int i = 0; i < list.size(); i++) {
String s = list.get(i);
if ("B".equals(s) || "C".equals(s)) list.remove(i);
}
System.out.println(list);
Output:
[A, C, D]
C is not removed due to i always increasing and skipping elements.
For the most part, you won't need to worry about explicit memory management in Java - as long as there are no other objects referring to them after being removed from the list, the garbage collector will (eventually) remove them.
If the objects in the enemies list are holding onto some system resource or something else that needs to be explicitly disposed of (say a file handle or something), you'll want to clean this up before losing the reference:
Enemy enemy = enemies.remove();
enemy.dispose(); // this is your method to clean up the internals - name it what you want
// continue your loop here
On a note related to your sample code, you'll want to use an Iterator rather than just a for loop iterating over the indexes so you can remove properly without running into issues around the current index and list size. You may also want to consider a different List implementation (such as LinkedList, as insert/remove from the middle of an ArrayList can get expensive if it's big.
For your other question:
Any way I can see if the objects are being deleted or not by the garbage collector?
You could override the finalize method of your class - be careful when you do this though. Also note that there are no guarantees when your objects will be garbage collected - different JVMs often manage memory slightly differently, and often only do garbage collection when it needs more memory.

Delete complex objects at runtime in Java

In Java, at some point in code I want to free memory taken by a huge HashMap<Integer, ArrayList<Integer>> object. Is it enough to point it to null like below:
Map<Integer, ArrayList<Integer>> complexObject = new HashMap<Integer, ArrayList<Integer>>(1000000);
...
complexObject = null;
?
You cannot explicitly de-allocate Java objects. But you can do the following:
Remove all references to the item you no longer need. You can do this by setting your only reference to an object to null.
Call System.gc() to "suggest" to the JVM to run the garbage collector, which deallocates no-longer used objects, although it's not guaranteed that calling this method will actually run the garbage collector.
Setting its reference to null will mark it available to the garbage collector next time it decides to run if there are indeed no more references to said object laying around anywhere. When the GC decides to run however, is not set in stone.
This is a big point where Java is different from C, but don't worry. 99.9% of the time you can trust the GC has your back.
There is no guarantee of freeing memory. GC will run and pickup nulls so that's all you can do yes.
I just read this article, there is something about freeing memory in it, too.
Check it out, nulling does not always help you.

Old gen heap space overflow

I have a very weird problem with GC in Java. I am running th following piece of code:
while(some condition){
//do a lot of work...
logger.info("Generating resulting time series...");
Collection<MetricTimeSeries> allSeries = manager.getTimeSeries();
logger.info(String.format("Generated %,d time series! Storing in files now...", allSeries.size()));
//for (MetricTimeSeries series : allSeries) {
// just empty loop
//}
}
When I look into JConsole, at the restart of every loop iteration, my old gen heap space, if I manually force GC, takes a size of about 90 MB. If I uncomment the loop, like this
while(some condition){
//do a lot of work...
logger.info("Generating resulting time series...");
Collection<MetricTimeSeries> allSeries = manager.getTimeSeries();
logger.info(String.format("Generated %,d time series! Storing in files now...", allSeries.size()));
for (MetricTimeSeries series : allSeries) {
// just empty loop
}
}
Even if I force it to refresh, it won't fall below 550MB. According to yourKit profiler, the TimeSeries objects are accessible via main thread's local var (the collection), just after the GC at the restart of a new iteration... And the collection is huge (250K time series.)... Wyy is this happening and how can I "fight" this (incorrect?) behaviour?
Yup, the garbage collector can be mysterious.. but it beats managing your own memory ;)
Collections and Maps have a way of hanging onto references longer than you might like and thus preventing garbage collection when you might expect. As you noticed, setting the allSeries reference to null itself will ear mark it for garbage collection, and thus it's contents are up for grabs as well. Another way would be to call allSeries.clear(): this will unlink all it's MetricTimeSeries objects and they will be free for garbage collection.
Why does removing the loop get around this problem also? This is the more interesting question. I'm tempted to suggest the compiler is optimizing the reference to allSeries.. but you are still calling allSeries.size() so it can't completely optimize out the reference.
To muddy the waters, different compiles (and settings) behave differently and use different garbage collectors which themselves behave differently. It's tough to say exactly what's happening under the hood without more information.
Since you're building a (large) ArrayList of time series, it will occupy the heap as long as it's referenced, and will get promoted to old if it stays long enough (or if the young generation is too small to actually hold it). I'm not sure how you're associating the information you're seeing in JConsole or Yourkit to a specific point in the program, but until the empty loop is optimized by several JIT passes, your while loop will take longer and keep the collection longer, which might explain the perceived difference while there's actually not a lot.
There's nothing incorrect about that behaviour. If you don't want to consume so much memory, you need to change your Collection so it's not an eagerly-filled ArrayList, but a lazy collection, more of a stream (if you've ever done XML processing, think DOM vs SAX) which gets evaluated as it's iterated. If you don't need the whole collection to be sorted, that's doable, especially since you seem to be saying that the collection is a concatenation of sub-collections returned by underlying objects.
If you can change your return type from Collection to Iterable, you could for example use Guava's FluentIterable.transformAndConcat() to transform the collection of underlying objects to a lazily-evaluated Iterable concatenation of their time series. Of course, the size of the collection is not directly available anymore (and if you try to get it independently of the iteration, you'll evaluate the lazy collection twice).

Java - when is this object unloaded?

Here is my code:
LinkedList <Mono> list = new LinkedList ();
list.add(new Mono (2, 2));
list.add(new Mono (1, -1));
list.remove (1);
Now, when the second item in the list is removed, is the object destroyed? IE, it undergoes garbage collection?
EDIT for new question:
Yes, the object will be eligible for garbage collection when there are no strong references remaining. However the JVM will try to do clean up garbage in big batches, so it could actually get collected at any arbitrary later time (or never if the JVM terminates before GC gets around to it)
Old answer:
Class unloading is a rare event, and will generally not happen in a timely manner (if at all.)
Specifically, even after it becomes eligible for collection it won't get collected along with "normal" new objects like your Mono instances - it's often in a special different pool (PermGen in the Oracle JVM)
You should assume that once a class is loaded it will stay loaded forever. Once you get into web applications in containers this is not always true, but anyone that has worked in those environments can tell you generally how well it (doesn't) work.
Garbage Collection in Java is generally non-deterministic insofar as when it will occur and which eligible objects the GC will evict ("free") when a GC cycle does occur.
The only reliable rule is thus:
An object will remain available (will not be GC'ed/freed/evicted) as long as it is strongly-reachable. (See "The Reachability Lifecycle of Objects" in Chapter 9 of Inside the Java Virtual Machine for what this means -- very good reading in general, if a tad dated.)
In the posted code the remove will result in the second object new Mono(1, -1) being eligible for reclamation as there are no longer any strong references to it (the object is no longer strongly-reachable). The actual eviction (if it occurs) will happen "sometime later" and may not even be the next GC cycle. Using finalizers (ick) further complicates the matter.
Note that an object is never guaranteed to be GC'ed (the JVM may just terminate [ab]normally, for instance) and the exact semantics of particular GC implementation can differ and still be a conforming virtual machine -- it all comes down to reachability.
Happy coding.
Do you mean when the Object is unloaded?
An empty list is still a list, so it'll stay in memory. It is cleared when you go:
list=somethingElse;
That is assuming that you don't assign anything else to be list.
As far as the class definition itself, it should stay in memory forever. Class definitions are in the permanent generation.
As a side note. list cannot be garbage collected at that point. Because you can add things to it after if you clear it.
the second object will be elligible for gargabe collection after removing it from the list, since there will be no more references to it .. hence out of scope .
hope this helped
Lots of answers, so here's my spin. Think about "scope", a concept in computer languages that describes where and when you can access a named bit of memory.
Here's your original code, with your purported removal of the second list member added:
LinkedList <Mono> list = new LinkedList ();
list.add(new Mono (2, 2));
list.add(new Mono (1, -1));
list.remove (1);
list.remove (2);`
At the point of list.remove(2), the object "list" can still be referenced. Sure, it's empty, but then you might decide to add a new Mono to it. You can because "list" is still in scope, so "list" is not reclaimed.
Compare to this:
{
LinkedList <Mono> list = new LinkedList ();
list.add(new Mono (2, 2));
list.add(new Mono (1, -1));
list.remove (1);
list.remove (2);`
}
After the closing brace, "list" can no longer be referenced. "list" was declared inside that scope, and when the scope was exited, "list" was removed from the namespace along with the scope itself. At that point, garbage collection could happen, since no one could possibly use "list" again.
Keep an eye on the scope of your objects. If you want to influence garbage collection, keep the scope of your objects limited to the region where they are used. As it happens, that is good programming style too.
I guess people got confused with the terminology you used. I believed you are asking if your Mono object will be "deleted"/"garbage collected".
Let's take a look at the remove(1) that you are invoking...
This is the remove function that you are calling as defined in java.util.LinkedList:
public E remove(int index) {
return remove(entry(index));
}
The function above calls the following (look at my comments in the code):
private E remove(Entry<E> e) {
if (e == header)
throw new NoSuchElementException();
E result = e.element;
e.previous.next = e.next; //Preceding element refers now to the one after the element to be removed;
e.next.previous = e.previous; //Next element refers now to the one before the one to be removed;
e.next = e.previous = null; //Element to be removed doesn't refer to anything anymore;
e.element = null;
size--;
modCount++;
return result;
}
After the function you call terminates, there is no way to refer to your Mono(1, -1) anymore. That Mono Object is not accessible anymore. This means that it will become eligible for Garbage Collection. Keep in mind that "eligible" might mean that it never be garbage collected... More on GC here.
The simple answer is that it should not matter to you when a Java object is garbage collected. The only thing that you need to know is that it will get garbage collected before your program runs out of memory ... provided that the object is unreachable.
The complicate answer includes other things:
The garbage collector typically runs at a time that you can't predict.
You can call System.gc() to suggest that the JVM runs the GC run now, but:
the JVM may ignore this hint, and
it is generally a bad idea to do this. (Running the GC is expensive, and your application has insufficient information to know when it is best to do this from an efficiency standpoint.)
Any particular run of the GC is not guaranteed to reclaim all unreachable objects. The GC has a lot of "smarts" that are aimed at making GC as efficient as possible, or reducing "pause" times. One of the "smarts" is to not GC the entire heap every time.
There are no guarantees that the GC will run at all, or that it will run before the JVM is shutdown.
Because of the above, it is a bad idea to write an application so that it depends on a specific object being reclaimed / deleted at a specific time.
(The one thing that should concern you in memory management is storage leaks; e.g. when your application keeps references to unwanted objects that prevent those objects from ever becoming unreachable. But that's not what your question is about.)
The class "Mono" cannot be unloaded since there are still references to it. The type of the list refers to it and there is still one element in the list.
I suppose you did not mean to ask whether the class is unloaded, but whether the instance is "unloaded". Each instance of a class, each object, is allocated on the heap. When the object is no longer in use, the space that it occupies in the heap can be reclaimed. This does not happen immediately however. All JVM implementations that I know use a garbage collector to clean up the memory. To really simplify things here: when there is no more space free to create a new object on the heap, the garbage collector kicks in and will check which parts of the heap are still in use. The parts that are no longer in use, can be reused for new objects.
So, the memory from an object that is no longer in use, will only be reclaimed when the garbage collector kicks in. And this is something that cannot be predicted.
Do you mean is the instance of Mono eligible for garbage collection or is the instance of list eligible for garbage collection?
The instance of mono will be eligible for garbage collection when it is removed (assuming that the code has not created over references to it.
The list is NOT eligible for garbage collection just because it is emptied. An empty list cannot be garbage collected because it is a valid object that can read and written to again.
An as others have pointed out. We are talking about eligible for garbage collection. The garbage collector does not necessarily run immediately.

Categories