I have a ConcurrentSkipListMap. I need to remove elements which are lower then key.
Here is how I can perform it:
private ConcurrentNavigableMap<Double, MyObject> myObjectsMap = new ConcurrentSkipListMap<>();
//...
myObjectsMap = myObjectsMap.tailMap(10.25, false);
Looks OK, but I am confused about these facts:
1.
The returned map is backed by this map, so changes in the returned map
are reflected in this map, and vice-versa.
Does it mean that old values won't be removed by the garbage collector?
I.e. we removed the old map and now we have a new map. But this new map is backed by the old map. So, what happens with the old map? Will it be removed or will it be sitting on a memory forever?
2.
The returned map will throw an IllegalArgumentException on an attempt
to insert a key outside its range.
So, now I can't put new keys which are less than 10.25 and more than the last maximum value?
I'am confused. How then correctly I need to remove elements from the ConcurrentSkipListMap?
Does it mean that old values won't be removed by the garbage collector?
I.e. we removed the old map and now we have a new map. But this new map is backed by the old map. So, what happens with the old map? Will it be removed or will it be sitting on a memory forever?
Yes, in point of fact. The old map is still around, and it'll stay around.
If you want to remove keys < 10.25, then do
map.headMap(10.25, false).clear();
...which will create that sub-map, remove all its elements -- removing them from the original map, too -- and then discard that submap view, letting it get garbage collected and leaving you with the original map object containing only keys >= 10.25.
Mind you, while this is guaranteed to remove keys that were < 10.25 when the operation started, there are no guarantees that new keys haven't been concurrently inserted, or that new keys might get inserted later. There's nothing you can do about that, really. If you want to be very sure you're only operating over values >= 10.25, then go ahead and use map.tailMap(10.25, true), but other values less than 10.25 might still be getting inserted, and they'll still be in memory.
Related
In my project I have a few HashMaps that I will reuse frequently, I have been informed that HashMaps can cause memory leaks and that Map#clear is not very effective and that I should set my HashMap to null, however, I think putting a null check before every use of the HashMap looks ugly, so would setting the HashMap to Maps#newHashMap accomplish the same goal, or should I set it to null and perform a null check before every use?
If you want to throw away all the entries in the map at a certain point of time, calling clear will remove all the entries. This is O(n) because it has to remove all the entries from the map.
The only reason (as far as I can imagine) to set it to null is to remove the references to the contents of the map. This will cause garbage collection to collect the objects in the map if there are no other active references to them.
But if the map is being used, initializing it with an empty map is the best thing to do as it will achieve the same purpose.
I have created a HashMap where millions of key insertions and removals happens continuously(in synchronized manner). However, removals does not remove the memory consumed by the HashMap. Due to this, my application's memory is going high and eventually at one moment, the application shuts down.
Can anyone suggest what should be the approach here to restore the memory upon removal of keys in HashMap.
Insertion:
map.put(txnId, new Timestamp(new Date().getTime()));
Removal:
Set<String> removalSet = new HashSet<String>();
removalSet.add(entry.getKey());
map.keySet().removeAll(removalSet);
After the elements are removed make the reference of HashMap as null this would make this object eligible for GC. System.gc() behavior is not guaranteed and should be avoided.
The solution which worked out was:
Use Class WeakHashMap in this case. The objects will automatically be removed if the key is no longer valid refer.
Why does java.util.Map.values() allow you to delete entries from the returned Collection when it makes no sense to remove a key value pair based on the value? The code which does this would have no idea what key the value(and hence a key) being removed is mapped from. Especially when there are duplicate values, calling remove on that Collection would result in an unexpected key being removed.
it makes no sense to remove a key value pair based on the value
I don't think you're being imaginative enough. I'll admit there probably isn't wide use for it, but there will be valid cases where it would be useful.
As a sample use case, say you had a Map<Person, TelephoneNumber> called contactList. Now you want to filter your contact list by those that are local.
To accomplish this, you could make a copy of the map, localContacts = new HashMap<>(contactList) and remove all mappings where the TelephoneNumber starts with an area code other than your local area code. This would be a valid time where you want to iterate through the values collection and remove some of the values:
Map<Person, TelephoneNumber> contactList = getContactList();
Map<Person, TelephoneNumber> localContacts = new HashMap<Person, TelephoneNumber>(contactList);
for ( Iterator<TelephoneNumber> valuesIt = localContacts.values().iterator(); valuesIt.hasNext(); ){
TelephoneNumber number = valuesIt.next();
if ( !number.getAreaCode().equals(myAreaCode) ) {
valuesIt.remove();
}
}
Especially when there are duplicate values, calling remove on that Collection would result in an unexpected key being removed.
What if you wanted to remove all mappings with that value?
It has to have a remove method because that's part of Collection. Given that, it has the choice of allowing you to remove values or throwing an UnsupportedOperationException. Since there are legitimate reasons that you might want to remove values, why not choose to allow this operation?
Maybe there's a given value where you want to remove every instance
of it from the Map.
Maybe you want to trim out every third
key/value pair for some reason.
Maybe you have a map from hotel
room number to occupancy count and you want to remove everything from
the map where the occupancy count is greater than one in order to
find a room for someone to stay in.
...if you think about it more
closely, there are plenty more examples like this...
In short: there are plenty of situations where this might be useful and implementing it doesn't harm anyone who doesn't use it, so why not?
I think there is quite often a use for removing a value based on a key; other answers show examples. Given that, if you want to remove a certain value, why would you only want one particular key of it removed? Even if you did, you'd have to know which key you wanted to remove (or not, as the case may be), and then you should just remove it by key anyway.
The Collection returned is a special Collection, and its semantics are such that it knows how values in it relate back to the Map it came from. The javadoc indicates what Collection operation the returned collection supports.
This question already has answers here:
Better practice to re-instantiate a List or invoke clear()
(4 answers)
Closed 1 year ago.
I have a Map as syntax as Map<String, String> testMap = new HashMap<String, String>();.
In this map there can be 1000 data.
When my application requires to new list of data, then I must clear the Map. But when I saw the code of Map.clear() as
/**
* Removes all of the mappings from this map.
* The map will be empty after this call returns.
*/
public void clear() {
modCount++;
Entry[] tab = table;
for (int i = 0; i < tab.length; i++)
tab[i] = null;
size = 0;
}
I realize that clear method goes in loop for n times (Where n is number of data in Map). So I thought there can be a way to redefine that Map as testMap = new HashMap<String, String>();
and previously used Map will be Garbage collected.
But I am not sure this will be a good way. I am working on mobile application.
Can you please guide me?
Complicated question. Let's see what happens.
You instantiate a new instance, which is backed with new array. So, garbage collector should clear all the key and values from the previous map, and clear the reference to itself. So O(n) algorithm is executed anyway, but in the garbage collector thread. For 1000 records you won't see any difference.
BUT. The performance guide tells you that it is always better not to create new objects, if you can. So I would go with clear() method.
Anyway, try both variants and try to measure. Always measure!
When you say Map.clear() on a Map of size n... You are asking the GC to clean up 2*n (Key & Value) objects. When you say null to the same Map, you are asking the GC to clean up 2*n+1 (1 for the Map itself) objects. Then you will have to create a new Map instance yet another overhead. So go for Map.clear(). You will be wise to preset the size of the Map while instantiating it.
I thought Creating object in java more expensive in terms of memory,so it is better to you go with .clear(),so you are using same object instead of creating new one
The idea of having clear() method is to remove references to other objects from the map, so that the key/values are not held up from gcing if the "map is referenced somewhere else".
But if your map is a local map only used by your specific code( i.e. "map is 'not' referenced somewhere else") then go ahead and use a new map instead, but setting a 1000 references to null wont be a big performance hit anyway.
don't forget the repopulation of the map
if you don't specify the capacity on the new map you will get quite a bit of overhead on the newly created map because of rehashes (which each are O(n) (at the time) and happen O(log(n)) times while this might amortize to O(n) total but if they don't happen in the first place you will still be better of)
this won't happen with the cleared map because the capacity doesn't change
I think calling new HashMap() is a better idea since it will not have to do as much processing as clearing the hashmap. Also, by creating a new hashmap you are removing the chance that the hashmap may still be binded to the control that uses the data, which would cause problems when the hashmap is to be cleared.
The map.clear() that will remove all data. Note that this will only discard all entries, but keep the internal array used to store the entries at the same size (rather than shrinking to an initial capacity). If you also need to eliminate that, the easiest way would be to discard the whole HashMap and replace it with a new instance. That of course only works if you control who has a pointer to the map.
As for reclaiming the memory, you will have to let the garbage collector do its work.
Are your values also Long? In this case, you may want to look at a more (memory-) efficient implementation than the generic HashMap, such as the TLongLongHashMap found in the GNU Trove library. That should save a lot of memory.
Is there a simple, efficient Map implementation that allows a limit on the memory to be used by the map.
My use case is that I want to allocate dynamically most of the memory available at the time of its creation but I don't want OutOFMemoryError at any time in future. Basically, I want to use this map as a cache, but but I wanna avoid heavy cache implementations like EHCache. My need is simple (at most an LRU algorithm)
I should further clarify that objects in my cache are char[] or similar primitives that will not hold references to other objects.
I can put an upper limit on max size for each entry.
You can use a LinkedHashMap to limit the number of entries in the Map:
removeEldestEntry(Map.Entry<K,V> eldest): Returns true if this map should remove its eldest entry. This method is invoked by put and putAll after inserting a new entry into the map. It provides the implementor with the opportunity to remove the eldest entry each time a new one is added. This is useful if the map represents a cache: it allows the map to reduce memory consumption by deleting stale entries.
Sample use: this override will allow the map to grow up to 100 entries and then delete the eldest entry each time a new entry is added, maintaining a steady state of 100 entries.
private static final int MAX_ENTRIES = 100;
protected boolean removeEldestEntry(Map.Entry eldest) {
return size() > MAX_ENTRIES;
}
Related questions
How do I limit the number of entries in a java hashtable?
Easy, simple to use LRU cache in java
What is a data structure kind of like a hash table, but infrequently-used keys are deleted?
For caches, a SoftHashMap is much more appropriate than a WeakHashMap. A WeakhashMap is usually used when you want to maintain an association with an object for as long as that object is alive, but without preventing it from being reclaimed.
In contrast, a SoftReference is more closely involved with memory allocation. See No SoftHashMap? for details on the differences.
WeakHashMap is also not usually appropriate as it has the association around the wrong way for a cache - it uses weak keys and hard values. That is, the key and value are removed from the map when the key is cleared by the garbage collector. This is typically not what you want for a cache - where the keys are usually lightweight identifiers (e.g. strings, or some other simple value type) - caches usually operate such that the key/value is reclaimed when the value reference is cleared.
The Commons Collections has a ReferenceMap where you can plug in what types of references you wish to use for keys and values. For a memory-sensitive cache, you will probably use hard references for keys, and soft references for values.
To obtain LRU semantics for a given number of references N, maintain a list of the last N entries fetched from the cache - as an entry is retrieved from the cache it is added to the head of the list (and the tail of the list removed.) To ensure this does not hold on to too much memory, you can create a soft reference and use that as a trigger to evict a percentage of the entries from the end of the list. (And create a new soft reference for the next trigger.)
Java Platform Solutions
If all you're looking for is a Map whose keys can be cleaned up to avoid OutOfMemoryErrors, you might want to look into WeakHashMap. It uses WeakReferences in order to allow the garbage collector to reap the map entries. It won't enforce any sort of LRU semantics, though, except those present in the generational garbage collection.
There's also LinkedHashMap, which has this in the documentation:
A special constructor is provided to
create a linked hash map whose order
of iteration is the order in which its
entries were last accessed, from
least-recently accessed to
most-recently (access-order). This
kind of map is well-suited to building
LRU caches. Invoking the put or get
method results in an access to the
corresponding entry (assuming it
exists after the invocation
completes). The putAll method
generates one entry access for each
mapping in the specified map, in the
order that key-value mappings are
provided by the specified map's entry
set iterator. No other methods
generate entry accesses. In
particular, operations on
collection-views do not affect the
order of iteration of the backing map.
So if you use this constructor to make a map whose Iterator iterates in LRU, it becomes pretty easy to prune the map. The one (fairly big) caveat is that LinkedHashMap is not synchronized whatsoever, so you're on your own for concurrency. You can just wrap it in a synchronized wrapper, but that may have throughput issues.
Roll Your Own Solution
If I had to write my own data structure for this use-case, I'd probably create some sort of data structure with a map, queue, and ReadWriteLock along with a janitor thread to handle the cleanup when too many entries were in the map. It would be possible to go slightly over the desired max size, but in the steady-state you'd stay under it.
WeakHashMap won't necessarily attain your purpose since if enough strong reference to the keys are hold by your app., you WILL see OOME.
Alternatively you could look into SoftReference, which will null out the content once the heap is scarce. However, most of the comments I seen indicate that it will not null out the reference until the heap is really really low and a lot of GC starts to kick in with severe performance hit (so I don't recommend using it for your purpose).
My recommendation is to use a simple LRU map, e.g. http://commons.apache.org/collections/apidocs/org/apache/commons/collections/LRUMap.html
thanks for replies guys!
As jasonmp85 pointed out LinkedHashMap has a constructor that allows access order. I missed out that bit when I looked at API docs. The implementation also looks quite efficient(see below). Combined with max size cap for each entry, that should solve my problem.
I will also look closely at SoftReference. Just for the record, Google Collections seems to have pretty good API for SoftKeys and SoftValues and Maps in general.
Here is a snippet from Java LikedHashMap class that shows how they maintain LRU behavior.
/**
* Removes this entry from the linked list.
*/
private void remove() {
before.after = after;
after.before = before;
}
/**
* Inserts this entry before the specified existing entry in the list.
*/
private void addBefore(Entry<K,V> existingEntry) {
after = existingEntry;
before = existingEntry.before;
before.after = this;
after.before = this;
}
/**
* This method is invoked by the superclass whenever the value
* of a pre-existing entry is read by Map.get or modified by Map.set.
* If the enclosing Map is access-ordered, it moves the entry
* to the end of the list; otherwise, it does nothing.
*/
void recordAccess(HashMap<K,V> m) {
LinkedHashMap<K,V> lm = (LinkedHashMap<K,V>)m;
if (lm.accessOrder) {
lm.modCount++;
remove();
addBefore(lm.header);
}