I hope this question is not considered too basic for this forum, but we'll see. I'm wondering how to refactor some code for better performance that is getting run a bunch of times.
Say I'm creating a word frequency list, using a Map (probably a HashMap), where each key is a String with the word that's being counted and the value is an Integer that's incremented each time a token of the word is found.
In Perl, incrementing such a value would be trivially easy:
$map{$word}++;
But in Java, it's much more complicated. Here the way I'm currently doing it:
int count = map.containsKey(word) ? map.get(word) : 0;
map.put(word, count + 1);
Which of course relies on the autoboxing feature in the newer Java versions. I wonder if you can suggest a more efficient way of incrementing such a value. Are there even good performance reasons for eschewing the Collections framework and using a something else instead?
Update: I've done a test of several of the answers. See below.
Now there is a shorter way with Java 8 using Map::merge.
myMap.merge(key, 1, Integer::sum)
or
myMap.merge(key, 1L, Long::sum)
for longs respectively.
What it does:
if key do not exists, put 1 as value
otherwise sum 1 to the value linked to key
More information here.
Some test results
I've gotten a lot of good answers to this question--thanks folks--so I decided to run some tests and figure out which method is actually fastest. The five methods I tested are these:
the "ContainsKey" method that I presented in the question
the "TestForNull" method suggested by Aleksandar Dimitrov
the "AtomicLong" method suggested by Hank Gay
the "Trove" method suggested by jrudolph
the "MutableInt" method suggested by phax.myopenid.com
Method
Here's what I did...
created five classes that were identical except for the differences shown below. Each class had to perform an operation typical of the scenario I presented: opening a 10MB file and reading it in, then performing a frequency count of all the word tokens in the file. Since this took an average of only 3 seconds, I had it perform the frequency count (not the I/O) 10 times.
timed the loop of 10 iterations but not the I/O operation and recorded the total time taken (in clock seconds) essentially using Ian Darwin's method in the Java Cookbook.
performed all five tests in series, and then did this another three times.
averaged the four results for each method.
Results
I'll present the results first and the code below for those who are interested.
The ContainsKey method was, as expected, the slowest, so I'll give the speed of each method in comparison to the speed of that method.
ContainsKey: 30.654 seconds (baseline)
AtomicLong: 29.780 seconds (1.03 times as fast)
TestForNull: 28.804 seconds (1.06 times as fast)
Trove: 26.313 seconds (1.16 times as fast)
MutableInt: 25.747 seconds (1.19 times as fast)
Conclusions
It would appear that only the MutableInt method and the Trove method are significantly faster, in that only they give a performance boost of more than 10%. However, if threading is an issue, AtomicLong might be more attractive than the others (I'm not really sure). I also ran TestForNull with final variables, but the difference was negligible.
Note that I haven't profiled memory usage in the different scenarios. I'd be happy to hear from anybody who has good insights into how the MutableInt and Trove methods would be likely to affect memory usage.
Personally, I find the MutableInt method the most attractive, since it doesn't require loading any third-party classes. So unless I discover problems with it, that's the way I'm most likely to go.
The code
Here is the crucial code from each method.
ContainsKey
import java.util.HashMap;
import java.util.Map;
...
Map<String, Integer> freq = new HashMap<String, Integer>();
...
int count = freq.containsKey(word) ? freq.get(word) : 0;
freq.put(word, count + 1);
TestForNull
import java.util.HashMap;
import java.util.Map;
...
Map<String, Integer> freq = new HashMap<String, Integer>();
...
Integer count = freq.get(word);
if (count == null) {
freq.put(word, 1);
}
else {
freq.put(word, count + 1);
}
AtomicLong
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicLong;
...
final ConcurrentMap<String, AtomicLong> map =
new ConcurrentHashMap<String, AtomicLong>();
...
map.putIfAbsent(word, new AtomicLong(0));
map.get(word).incrementAndGet();
Trove
import gnu.trove.TObjectIntHashMap;
...
TObjectIntHashMap<String> freq = new TObjectIntHashMap<String>();
...
freq.adjustOrPutValue(word, 1, 1);
MutableInt
import java.util.HashMap;
import java.util.Map;
...
class MutableInt {
int value = 1; // note that we start at 1 since we're counting
public void increment () { ++value; }
public int get () { return value; }
}
...
Map<String, MutableInt> freq = new HashMap<String, MutableInt>();
...
MutableInt count = freq.get(word);
if (count == null) {
freq.put(word, new MutableInt());
}
else {
count.increment();
}
A little research in 2016: https://github.com/leventov/java-word-count, benchmark source code
Best results per method (smaller is better):
time, ms
kolobokeCompile 18.8
koloboke 19.8
trove 20.8
fastutil 22.7
mutableInt 24.3
atomicInteger 25.3
eclipse 26.9
hashMap 28.0
hppc 33.6
hppcRt 36.5
Time\space results:
Map<String, Integer> map = new HashMap<>();
String key = "a random key";
int count = map.getOrDefault(key, 0); // ensure count will be one of 0,1,2,3,...
map.put(key, count + 1);
And that's how you increment a value with simple code.
Benefit:
No need to add a new class or use another concept of mutable int
Not relying on any library
Easy to understand what's going on exactly (Not too much abstraction)
Downside:
The hash map will be searched twice for get() and put(). So it will not be the most performant code.
Theoretically, once you call get(), you already know where to put(), so you should not have to search again. But searching in hash map usually takes a very minimal time that you can kind of ignore this performance issue.
But if you are very serious about the issue, you are a perfectionist, another way is to use merge method, this is (probably) more efficient than the previous code snippet as you will be (theoretically) searching the map only once: (though this code is not obvious from first sight, it's short and performant)
map.merge(key, 1, (a,b) -> a+b);
Suggestion: you should care about code readability more than little performance gain in most of the time. If the first code snippet is easier for you to understand then use it. But if you are able to understand the 2nd one fine then you can also go for it!
As a follow-up to my own comment: Trove looks like the way to go. If, for whatever reason, you wanted to stick with the standard JDK, ConcurrentMap and AtomicLong can make the code a tiny bit nicer, though YMMV.
final ConcurrentMap<String, AtomicLong> map = new ConcurrentHashMap<String, AtomicLong>();
map.putIfAbsent("foo", new AtomicLong(0));
map.get("foo").incrementAndGet();
will leave 1 as the value in the map for foo. Realistically, increased friendliness to threading is all that this approach has to recommend it.
Google Guava is your friend...
...at least in some cases. They have this nice AtomicLongMap. Especially nice because you are dealing with long as value in your map.
E.g.
AtomicLongMap<String> map = AtomicLongMap.create();
[...]
map.getAndIncrement(word);
Also possible to add more then 1 to the value:
map.getAndAdd(word, 112L);
It's always a good idea to look at the Google Collections Library for this kind of thing. In this case a Multiset will do the trick:
Multiset bag = Multisets.newHashMultiset();
String word = "foo";
bag.add(word);
bag.add(word);
System.out.println(bag.count(word)); // Prints 2
There are Map-like methods for iterating over keys/entries, etc. Internally the implementation currently uses a HashMap<E, AtomicInteger>, so you will not incur boxing costs.
You should be aware of the fact that your original attempt
int count = map.containsKey(word) ? map.get(word) : 0;
contains two potentially expensive operations on a map, namely containsKey and get. The former performs an operation potentially pretty similar to the latter, so you're doing the same work twice!
If you look at the API for Map, get operations usually return null when the map does not contain the requested element.
Note that this will make a solution like
map.put( key, map.get(key) + 1 );
dangerous, since it might yield NullPointerExceptions. You should check for a null first.
Also note, and this is very important, that HashMaps can contain nulls by definition. So not every returned null says "there is no such element". In this respect, containsKey behaves differently from get in actually telling you whether there is such an element. Refer to the API for details.
For your case, however, you might not want to distinguish between a stored null and "noSuchElement". If you don't want to permit nulls you might prefer a Hashtable. Using a wrapper library as was already proposed in other answers might be a better solution to manual treatment, depending on the complexity of your application.
To complete the answer (and I forgot to put that in at first, thanks to the edit function!), the best way of doing it natively, is to get into a final variable, check for null and put it back in with a 1. The variable should be final because it's immutable anyway. The compiler might not need this hint, but its clearer that way.
final HashMap map = generateRandomHashMap();
final Object key = fetchSomeKey();
final Integer i = map.get(key);
if (i != null) {
map.put(i + 1);
} else {
// do something
}
If you do not want to rely on autoboxing, you should say something like map.put(new Integer(1 + i.getValue())); instead.
Another way would be creating a mutable integer:
class MutableInt {
int value = 0;
public void inc () { ++value; }
public int get () { return value; }
}
...
Map<String,MutableInt> map = new HashMap<String,MutableInt> ();
MutableInt value = map.get (key);
if (value == null) {
value = new MutableInt ();
map.put (key, value);
} else {
value.inc ();
}
of course this implies creating an additional object but the overhead in comparison to creating an Integer (even with Integer.valueOf) should not be so much.
You can make use of computeIfAbsent method in Map interface provided in Java 8.
final Map<String,AtomicLong> map = new ConcurrentHashMap<>();
map.computeIfAbsent("A", k->new AtomicLong(0)).incrementAndGet();
map.computeIfAbsent("B", k->new AtomicLong(0)).incrementAndGet();
map.computeIfAbsent("A", k->new AtomicLong(0)).incrementAndGet(); //[A=2, B=1]
The method computeIfAbsent checks if the specified key is already associated with a value or not? If no associated value then it attempts to compute its value using the given mapping function. In any case it returns the current (existing or computed) value associated with the specified key, or null if the computed value is null.
On a side note if you have a situation where multiple threads update a common sum you can have a look at LongAdder class.Under high contention, expected throughput of this class is significantly higher than AtomicLong, at the expense of higher space consumption.
Quite simple, just use the built-in function in Map.java as followed
map.put(key, map.getOrDefault(key, 0) + 1);
Memory rotation may be an issue here, since every boxing of an int larger than or equal to 128 causes an object allocation (see Integer.valueOf(int)). Although the garbage collector very efficiently deals with short-lived objects, performance will suffer to some degree.
If you know that the number of increments made will largely outnumber the number of keys (=words in this case), consider using an int holder instead. Phax already presented code for this. Here it is again, with two changes (holder class made static and initial value set to 1):
static class MutableInt {
int value = 1;
void inc() { ++value; }
int get() { return value; }
}
...
Map<String,MutableInt> map = new HashMap<String,MutableInt>();
MutableInt value = map.get(key);
if (value == null) {
value = new MutableInt();
map.put(key, value);
} else {
value.inc();
}
If you need extreme performance, look for a Map implementation which is directly tailored towards primitive value types. jrudolph mentioned GNU Trove.
By the way, a good search term for this subject is "histogram".
I suggest to use Java 8 Map::compute().
It considers the case when a key doesn't exist, too.
Map.compute(num, (k, v) -> (v == null) ? 1 : v + 1);
Instead of calling containsKey() it is faster just to call map.get and check if the returned value is null or not.
Integer count = map.get(word);
if(count == null){
count = 0;
}
map.put(word, count + 1);
Are you sure that this is a bottleneck? Have you done any performance analysis?
Try using the NetBeans profiler (its free and built into NB 6.1) to look at hotspots.
Finally, a JVM upgrade (say from 1.5->1.6) is often a cheap performance booster. Even an upgrade in build number can provide good performance boosts. If you are running on Windows and this is a server class application, use -server on the command line to use the Server Hotspot JVM. On Linux and Solaris machines this is autodetected.
There are a couple of approaches:
Use a Bag alorithm like the sets contained in Google Collections.
Create mutable container which you can use in the Map:
class My{
String word;
int count;
}
And use put("word", new My("Word") ); Then you can check if it exists and increment when adding.
Avoid rolling your own solution using lists, because if you get innerloop searching and sorting, your performance will stink. The first HashMap solution is actually quite fast, but a proper like that found in Google Collections is probably better.
Counting words using Google Collections, looks something like this:
HashMultiset s = new HashMultiset();
s.add("word");
s.add("word");
System.out.println(""+s.count("word") );
Using the HashMultiset is quite elegent, because a bag-algorithm is just what you need when counting words.
A variation on the MutableInt approach that might be even faster, if a bit of a hack, is to use a single-element int array:
Map<String,int[]> map = new HashMap<String,int[]>();
...
int[] value = map.get(key);
if (value == null)
map.put(key, new int[]{1} );
else
++value[0];
It would be interesting if you could rerun your performance tests with this variation. It might be the fastest.
Edit: The above pattern worked fine for me, but eventually I changed to use Trove's collections to reduce memory size in some very large maps I was creating -- and as a bonus it was also faster.
One really nice feature is that the TObjectIntHashMap class has a single adjustOrPutValue call that, depending on whether there is already a value at that key, will either put an initial value or increment the existing value. This is perfect for incrementing:
TObjectIntHashMap<String> map = new TObjectIntHashMap<String>();
...
map.adjustOrPutValue(key, 1, 1);
Google Collections HashMultiset :
- quite elegant to use
- but consume CPU and memory
Best would be to have a method like : Entry<K,V> getOrPut(K);
(elegant, and low cost)
Such a method will compute hash and index only once,
and then we could do what we want with the entry
(either replace or update the value).
More elegant:
- take a HashSet<Entry>
- extend it so that get(K) put a new Entry if needed
- Entry could be your own object.
--> (new MyHashSet()).get(k).increment();
"put" need "get" (to ensure no duplicate key).
So directly do a "put",
and if there was a previous value, then do an addition:
Map map = new HashMap ();
MutableInt newValue = new MutableInt (1); // default = inc
MutableInt oldValue = map.put (key, newValue);
if (oldValue != null) {
newValue.add(oldValue); // old + inc
}
If count starts at 0, then add 1: (or any others values...)
Map map = new HashMap ();
MutableInt newValue = new MutableInt (0); // default
MutableInt oldValue = map.put (key, newValue);
if (oldValue != null) {
newValue.setValue(oldValue + 1); // old + inc
}
Notice : This code is not thread safe. Use it to build then use the map, not to concurrently update it.
Optimization : In a loop, keep old value to become the new value of next loop.
Map map = new HashMap ();
final int defaut = 0;
final int inc = 1;
MutableInt oldValue = new MutableInt (default);
while(true) {
MutableInt newValue = oldValue;
oldValue = map.put (key, newValue); // insert or...
if (oldValue != null) {
newValue.setValue(oldValue + inc); // ...update
oldValue.setValue(default); // reuse
} else
oldValue = new MutableInt (default); // renew
}
}
The various primitive wrappers, e.g., Integer are immutable so there's really not a more concise way to do what you're asking unless you can do it with something like AtomicLong. I can give that a go in a minute and update. BTW, Hashtable is a part of the Collections Framework.
I'd use Apache Collections Lazy Map (to initialize values to 0) and use MutableIntegers from Apache Lang as values in that map.
Biggest cost is having to serach the map twice in your method. In mine you have to do it just once. Just get the value (it will get initialized if absent) and increment it.
The Functional Java library's TreeMap datastructure has an update method in the latest trunk head:
public TreeMap<K, V> update(final K k, final F<V, V> f)
Example usage:
import static fj.data.TreeMap.empty;
import static fj.function.Integers.add;
import static fj.pre.Ord.stringOrd;
import fj.data.TreeMap;
public class TreeMap_Update
{public static void main(String[] a)
{TreeMap<String, Integer> map = empty(stringOrd);
map = map.set("foo", 1);
map = map.update("foo", add.f(1));
System.out.println(map.get("foo").some());}}
This program prints "2".
I don't know how efficient it is but the below code works as well.You need to define a BiFunction at the beginning. Plus, you can make more than just increment with this method.
public static Map<String, Integer> strInt = new HashMap<String, Integer>();
public static void main(String[] args) {
BiFunction<Integer, Integer, Integer> bi = (x,y) -> {
if(x == null)
return y;
return x+y;
};
strInt.put("abc", 0);
strInt.merge("abc", 1, bi);
strInt.merge("abc", 1, bi);
strInt.merge("abc", 1, bi);
strInt.merge("abcd", 1, bi);
System.out.println(strInt.get("abc"));
System.out.println(strInt.get("abcd"));
}
output is
3
1
If you're using Eclipse Collections, you can use a HashBag. It will be the most efficient approach in terms of memory usage and it will also perform well in terms of execution speed.
HashBag is backed by a MutableObjectIntMap which stores primitive ints instead of Counter objects. This reduces memory overhead and improves execution speed.
HashBag provides the API you'd need since it's a Collection that also allows you to query for the number of occurrences of an item.
Here's an example from the Eclipse Collections Kata.
MutableBag<String> bag =
HashBag.newBagWith("one", "two", "two", "three", "three", "three");
Assert.assertEquals(3, bag.occurrencesOf("three"));
bag.add("one");
Assert.assertEquals(2, bag.occurrencesOf("one"));
bag.addOccurrences("one", 4);
Assert.assertEquals(6, bag.occurrencesOf("one"));
Note: I am a committer for Eclipse Collections.
Counting using streams and getOrDefault:
String s = "abcdeff";
s.chars().mapToObj(c -> (char) c)
.forEach(c -> {
int count = countMap.getOrDefault(c, 0) + 1;
countMap.put(c, count);
});
Since a lot of people search Java topics for Groovy answers, here's how you can do it in Groovy:
dev map = new HashMap<String, Integer>()
map.put("key1", 3)
map.merge("key1", 1) {a, b -> a + b}
map.merge("key2", 1) {a, b -> a + b}
Hope I'm understanding your question correctly, I'm coming to Java from Python so I can empathize with your struggle.
if you have
map.put(key, 1)
you would do
map.put(key, map.get(key) + 1)
Hope this helps!
The simple and easy way in java 8 is the following:
final ConcurrentMap<String, AtomicLong> map = new ConcurrentHashMap<String, AtomicLong>();
map.computeIfAbsent("foo", key -> new AtomicLong(0)).incrementAndGet();
Related
Suppose we have a HashMap<String, Integer> in Java.
How do I update (increment) the integer-value of the string-key for each existence of the string I find?
One could remove and reenter the pair, but overhead would be a concern.
Another way would be to just put the new pair and the old one would be replaced.
In the latter case, what happens if there is a hashcode collision with a new key I am trying to insert? The correct behavior for a hashtable would be to assign a different place for it, or make a list out of it in the current bucket.
map.put(key, map.get(key) + 1);
should be fine. It will update the value for the existing mapping. Note that this uses auto-boxing. With the help of map.get(key) we get the value of corresponding key, then you can update with your requirement. Here I am updating to increment value by 1.
Java 8 way:
You can use computeIfPresent method and supply it a mapping function, which will be called to compute a new value based on existing one.
For example,
Map<String, Integer> words = new HashMap<>();
words.put("hello", 3);
words.put("world", 4);
words.computeIfPresent("hello", (k, v) -> v + 1);
System.out.println(words.get("hello"));
Alternatevely, you could use merge method, where 1 is the default value and function increments existing value by 1:
words.merge("hello", 1, Integer::sum);
In addition, there is a bunch of other useful methods, such as putIfAbsent, getOrDefault, forEach, etc.
The simplified Java 8 way:
map.put(key, map.getOrDefault(key, 0) + 1);
This uses the method of HashMap that retrieves the value for a key, but if the key can't be retrieved it returns the specified default value (in this case a '0').
This is supported within core Java: HashMap<K,V> getOrDefault(Object key, V defaultValue)
hashmap.put(key, hashmap.get(key) + 1);
The method put will replace the value of an existing key and will create it if doesn't exist.
Replace Integer by AtomicInteger and call one of the incrementAndGet/getAndIncrement methods on it.
An alternative is to wrap an int in your own MutableInteger class which has an increment() method, you only have a threadsafety concern to solve yet.
One line solution:
map.put(key, map.containsKey(key) ? map.get(key) + 1 : 1);
#Matthew's solution is the simplest and will perform well enough in most cases.
If you need high performance, AtomicInteger is a better solution ala #BalusC.
However, a faster solution (provided thread safety is not an issue) is to use TObjectIntHashMap which provides a increment(key) method and uses primitives and less objects than creating AtomicIntegers. e.g.
TObjectIntHashMap<String> map = new TObjectIntHashMap<String>()
map.increment("aaa");
You can increment like below but you need to check for existence so that a NullPointerException is not thrown
if(!map.containsKey(key)) {
p.put(key,1);
}
else {
p.put(key, map.getKey()+1);
}
Does the hash exist (with 0 as the value) or is it "put" to the map on the first increment? If it is "put" on the first increment, the code should look like:
if (hashmap.containsKey(key)) {
hashmap.put(key, hashmap.get(key)+1);
} else {
hashmap.put(key,1);
}
It may be little late but here are my two cents.
If you are using Java 8 then you can make use of computeIfPresent method. If the value for the specified key is present and non-null then it attempts to compute a new mapping given the key and its current mapped value.
final Map<String,Integer> map1 = new HashMap<>();
map1.put("A",0);
map1.put("B",0);
map1.computeIfPresent("B",(k,v)->v+1); //[A=0, B=1]
We can also make use of another method putIfAbsent to put a key. If the specified key is not already associated with a value (or is mapped to null) then this method associates it with the given value and returns null, else returns the current value.
In case the map is shared across threads then we can make use of ConcurrentHashMap and AtomicInteger. From the doc:
An AtomicInteger is an int value that may be updated atomically. An
AtomicInteger is used in applications such as atomically incremented
counters, and cannot be used as a replacement for an Integer. However,
this class does extend Number to allow uniform access by tools and
utilities that deal with numerically-based classes.
We can use them as shown:
final Map<String,AtomicInteger> map2 = new ConcurrentHashMap<>();
map2.putIfAbsent("A",new AtomicInteger(0));
map2.putIfAbsent("B",new AtomicInteger(0)); //[A=0, B=0]
map2.get("B").incrementAndGet(); //[A=0, B=1]
One point to observe is we are invoking get to get the value for key B and then invoking incrementAndGet() on its value which is of course AtomicInteger. We can optimize it as the method putIfAbsent returns the value for the key if already present:
map2.putIfAbsent("B",new AtomicInteger(0)).incrementAndGet();//[A=0, B=2]
On a side note if we plan to use AtomicLong then as per documentation under high contention expected throughput of LongAdder is significantly higher, at the expense of higher space consumption. Also check this question.
The cleaner solution without NullPointerException is:
map.replace(key, map.get(key) + 1);
Since I can't comment to a few answers due to less reputation, I will post a solution which I applied.
for(String key : someArray)
{
if(hashMap.containsKey(key)//will check if a particular key exist or not
{
hashMap.put(hashMap.get(key),value+1);// increment the value by 1 to an already existing key
}
else
{
hashMap.put(key,value);// make a new entry into the hashmap
}
}
Integer i = map.get(key);
if(i == null)
i = (aValue)
map.put(key, i + 1);
or
Integer i = map.get(key);
map.put(key, i == null ? newValue : i + 1);
Integer is Primitive data types http://cs.fit.edu/~ryan/java/language/java-data.html, so you need to take it out, make some process, then put it back. if you have a value which is not Primitive data types, you only need to take it out, process it, no need to put it back into the hashmap.
Use a for loop to increment the index:
for (int i =0; i<5; i++){
HashMap<String, Integer> map = new HashMap<String, Integer>();
map.put("beer", 100);
int beer = map.get("beer")+i;
System.out.println("beer " + beer);
System.out ....
}
There are misleading answers to this question here that imply Hashtable put method will replace the existing value if the key exists, this is not true for Hashtable but rather for HashMap. See Javadoc for HashMap http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html#put%28K,%20V%29
Use Java8 built in fuction 'computeIfPresent'
Example:
public class ExampleToUpdateMapValue {
public static void main(String[] args) {
Map<String,String> bookAuthors = new TreeMap<>();
bookAuthors.put("Genesis","Moses");
bookAuthors.put("Joshua","Joshua");
bookAuthors.put("Judges","Samuel");
System.out.println("---------------------Before----------------------");
bookAuthors.entrySet().stream().forEach(System.out::println);
// To update the existing value using Java 8
bookAuthors.computeIfPresent("Judges", (k,v) -> v = "Samuel/Nathan/Gad");
System.out.println("---------------------After----------------------");
bookAuthors.entrySet().stream().forEach(System.out::println);
}
}
Try:
HashMap hm=new HashMap<String ,Double >();
NOTE:
String->give the new value; //THIS IS THE KEY
else
Double->pass new value; //THIS IS THE VALUE
You can change either the key or the value in your hashmap, but you can't change both at the same time.
I am trying reset all values in a HashMap to some default value if a condition fails.
Currently i am doing this by iterating over all the keys and individually resetting the values.Is there any possible way to set a same value to all the keys without iterating?
Something like:
hm.putAll("some val") //hm is hashmap object
You can't avoid iterating but if you're using java-8, you could use the replaceAll method which will do that for you.
Apply the specified function to each entry in this map, replacing each
entry's value with the result of calling the function's Function#map
method with the current entry's key and value.
m.replaceAll((k,v) -> yourDefaultValue);
Basically it iterates through each node of the table the map holds and affect the return value of the function for each value.
#Override
public void replaceAll(BiFunction<? super K, ? super V, ? extends V> function) {
Node<K,V>[] tab;
if (function == null)
throw new NullPointerException();
if (size > 0 && (tab = table) != null) {
int mc = modCount;
for (int i = 0; i < tab.length; ++i) {
for (Node<K,V> e = tab[i]; e != null; e = e.next) {
e.value = function.apply(e.key, e.value); //<-- here
}
}
if (modCount != mc)
throw new ConcurrentModificationException();
}
}
Example:
public static void main (String[] args){
Map<String, Integer> m = new HashMap<>();
m.put("1",1);
m.put("2",2);
System.out.println(m);
m.replaceAll((k,v) -> null);
System.out.println(m);
}
Output:
{1=1, 2=2}
{1=null, 2=null}
You can't avoid iterating in some fashion.
You could get the values via Map.values() and iterate over those. You'll bypass the lookup by key and it's probably the most efficient solution (although I suspect generally that would save you relatively little, and perhaps it's not the most obvious to a casual reader of your code)
IMHO You must create your own Data Structure that extends from Map. Then you can write your method resetAll() and give the default value. A Map is a quick balanced tree that allows you to walk quick in the structure and set the value. No worries about the speed, because the tree will have the same structure before and after the reset.
Only, be carefull with concurrent threads. Maybe you should use ConcurrentHashMap.
public class MyMap<K,V> extends ConcurrentHashMap<K, V>{
public void resetAll(V value){
Iterator<Entry<K, V>> it = this.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pairs = (Map.Entry)it.next();
pairs.setValue( value );
}
}
}
Regards
If you're willing to make a copy of it ( a hasmap with default values )
You can first clear your hashmap and then move over the default values
hm.keySet().removeAll();
hm.putAll(defaultMap);
It is not possible to apply an operation to all values in a collection in less than O(n) time, however if your objection is truly with iteration itself, there are some possible alternatives, notably functional programming.
This is made most easy by the Guava library (or natively in Java 8), and their functional programming utilities. Their Maps.transformValues() provides a view of the map, with the provided function applied. This means that the function returns in O(1) time, unlike your iteration, but that the computation is done on the fly whenever you .get() from the returned map. This is obviously a tradeoff - if you only need to .get() certain elements from the transformed map, you save time by avoiding computing unnecessary values. On the other hand, if you know you'll later hit every element at least once, using this behavior means you'll actually waste time. In essence, this approach is O(k) where k is the number of lookups you plan to do. If k is always less than n, then using the transformation approach is optimal.
Read carefully however the caveat at the top of the page; iteration is a simple, easy, and generally ideally efficient way to work with the members of a map. You should only try to optimize past that when absolutely necessary.
Assuming that your problem is not with doing the iteration yourself, but with the fact that O(n) is going on at some point, I would suggest a couple of alternative approaches. Bear in mind I have no idea what you are using this for, so it might not make any sense to you.
Case A: If your set of keys is known and fixed beforehand, keep a copy (not a reference, an actual clone) somewhere with the values reset to the one you want. Then on that condition you mention, simply switch the references to use the default one.
Case B: If they keys change over time, use the idea from case A but add new entries with the default value for every new key added (or remove accordingly). Your updates should hardly notice but you can still switch back to the default in O(1).
Specifically I need a collection which uses one field A for accessing and a different one (field S) for sorting but a sorted collection which accepts duplicate would be sufficient.
I often come to this point where I need exactly this collection and TreeMap is not an option as it does not allow duplicates. So now it is time to ask here. There are several workarounds as pointed out on stackoverflow here and here - namely there are:
PriorityQueue: slow update (remove(Object) + add(Object)), and boxing of primitive keys
Fibonacci heap: memory waste (?)
TreeMap<Field_S, List<Value>>: problem for me is the memory overhead of the list, and boxing of primitive keys
sorted list or array: problem is the slow insert and remove -> should I implement one segmented sorted list?
TreeMultimap from guava (docs): external dependency and probably memory inefficient (?)
Anyone with better suggestions? Or should I role my own sorted datastructure (which one?)? Also other sources (in Java, open source, with unit tests and small deps) would be nice.
Update
More details on my use case at the moment (although I'm having similar demand in the last time). I have a collection (with millions) of references where I want to be able
to poll or get the smallest element regarding field S
and update field S with the help of field A
identical values of field S can happen. field A is actually a integer pointing into another array
the only dependency I want is trove4j. I could use a different like the mahout collections if that would be required. But not guava as although a nice lib the collections are not tuned to be memory efficient (boxing/unboxing).
So all cries for a fibonacci heap but I fear it has too many overhead per element -> that was the reason I thought about a more memory efficient "sorted+segmented array" solution.
When you need a sorted collection, you should analyze your needs carefully.
If the majority of operations is inserting and only a few are to search then using a sorted collection i.e. keep the elements sorted in the collection constantly, would not be a good option (due to the overhead of keeping the elements sorted on insert which would be the most common operation).
In this case it would be best to keep an unsorted collection and do the sorting only when needed. I.e. before the search. You could even use a simple List and sort it (using Collections.sort i.e. mergesort) when needed. But I recommend this with caution, as for this to be efficient the assumption is that you work on large data. In really small data even linear search is good enough.
If the majority of operations is searching then you could use a sorted collection which from my of point of view there are data structures to choose from (some you already mention) and you could benchmark to see which one fits your needs.
What about guava TreeMultiset? What you asked for: a sorted collection which accepts duplicates. Don't know anything about its performance though.
I decided to roll my own but not the optimal solution just a TreeMap variant. I'll keep this updated if I'll fine tune this collection regarding memory. Speed is already a lot better then the previous PriorityQueue attempt as I needed the collection.remove(Object) method (for updating an entry):
package com.graphhopper.coll;
import gnu.trove.iterator.TIntIterator;
import gnu.trove.set.hash.TIntHashSet;
import java.util.Map.Entry;
import java.util.TreeMap;
/**
* A priority queue implemented by a treemap to allow fast key update. Or should we use a standard
* b-tree?
*/
public class MySortedCollection {
private int size;
private int slidingMeanValue = 20;
private TreeMap<Integer, TIntHashSet> map;
public MySortedCollection(int size) {
map = new TreeMap<Integer, TIntHashSet>();
}
void remove(int key, int value) {
TIntHashSet set = map.get(value);
if (set == null || !set.remove(key))
throw new IllegalStateException("cannot remove key " + key + " with value " + value
+ " - did you insert " + key + "," + value + " before?");
size--;
if (set.isEmpty())
map.remove(value);
}
public void update(int key, int oldValue, int value) {
remove(key, oldValue);
insert(key, value);
}
public void insert(int key, int value) {
TIntHashSet set = map.get(value);
if (set == null)
map.put(value, set = new TIntHashSet(slidingMeanValue));
// else
// slidingMeanValue = Math.max(5, (slidingMeanValue + set.size()) / 2);
if (!set.add(key))
throw new IllegalStateException("use update if you want to update " + key);
size++;
}
public int peekValue() {
if (size == 0)
throw new IllegalStateException("collection is already empty!?");
Entry<Integer, TIntHashSet> e = map.firstEntry();
if (e.getValue().isEmpty())
throw new IllegalStateException("internal set is already empty!?");
return map.firstEntry().getKey();
}
public int peekKey() {
if (size == 0)
throw new IllegalStateException("collection is already empty!?");
TIntHashSet set = map.firstEntry().getValue();
if (set.isEmpty())
throw new IllegalStateException("internal set is already empty!?");
return set.iterator().next();
}
public int pollKey() {
size--;
if (size < 0)
throw new IllegalStateException("collection is already empty!?");
Entry<Integer, TIntHashSet> e = map.firstEntry();
TIntHashSet set = e.getValue();
TIntIterator iter = set.iterator();
if (set.isEmpty())
throw new IllegalStateException("internal set is already empty!?");
int val = iter.next();
iter.remove();
if (set.isEmpty())
map.remove(e.getKey());
return val;
}
public int size() {
return size;
}
public boolean isEmpty() {
return size == 0;
}
public int getSlidingMeanValue() {
return slidingMeanValue;
}
#Override
public String toString() {
return "size " + size + " min=(" + peekKey() + "=>" + peekValue() + ")";
}
}
You need to decide if you want external dependencies or not. I wouldn't roll my own implementation for something like this.
That said, you've told us almost nothing about what you're using this for, and what you plan to do with it. Without enough data, there's only so much we can tell you -- do you actually need to access the elements in random order? How large do you expect this collection to be? We really don't have enough data to pick out the one right data structure for your needs.
That said, here are some options I would consider.
ArrayList or PriorityQueue, depending on whether or not you actually need to support remove(Object). Do you? Are you sure? (Even if you do need to support remove(Object), I would choose this option if the collection is likely to stay small.)
Not the TreeList you linked to, but instead the Apache Commons Collections TreeList. Despite the name, it doesn't actually maintain sorted order, but what it does is support O(log n) add, remove, and get from anywhere in the list. Using binary search, you could potentially achieve O((log n)^2) time for add, remove, or lookup according to the sorted part of your values.
The TreeList you linked to, or -- if you're like me, and care about the List contract -- a custom Guava ListMultimap, obtained with Multimaps.newListMultimap(new TreeMap<K, Collection<V>>, new Supplier<List<V>>() { public List<V> get() { return new ArrayList<V>(); }}).
If you also care about primitive boxing, or can't tolerate third-party dependencies, you're going to have no choice but to write up your own data structure. I'd just adapt one of the implementations above to your primitive type, but this is going to be a royal pain.
Finally: I'd really like to hear your use case. Guava doesn't have any support for things like this because we haven't had enough demand, or seen a use case for which a more sophisticated data structure really appropriate.
I would go with skiplist - more memory efficient than a tree, allows duplicates, provides O(logn) for inserts and deletes. You can even implement an indexed skiplist, it will allow you to have indexed access, something that's hard to get with a tree.
I have good experience with TreeMultimap https://guava.dev/releases/19.0/api/docs/com/google/common/collect/TreeMultimap.html
Say I have a LinkedHashMap containing 216 entries, how would I get the first 100 values (here, of type Object) from a LinkedHashMap<Integer, Object>.
Well to start with, doing this for HashMap as per your title, doesn't make much sense - HashMap has no particular order, and the order may change between calls. It makes more sense for LinkedHashMap though.
There, I'd use Guava's Iterables.limit method:
Iterable<Object> first100Values = Iterables.limit(map.values(), 100);
or
// Or whatever type you're interested in...
Iterable<Map.Entry<Integer, Object>> firstEntries =
Iterables.limit(map.entrySet(), 100);
You can then create a list from that, or iterate over it, or whatever you want to do.
Ugly One-Liner
This ugly one-liner would do (and return a ArrayList<Object> in the question's case):
Collections.list(Collections.enumeration(lhMap.values())).subList(0, 100)
This would work for a HashMap as well, however HashMap being backed by a HashSet there's not guarantee that you will get the first 100 values that were entered; it would work on other types, with similar limitations.
Notes:
relatively unefficient (read the Javadoc to know why - though there's worse!),
careful when using views (read the Javadoc to know more),
I did mention it was ugly.
Step-By-Step Usage Example
(as per the OP's comment)
Map<Integer, Pair<Double, SelectedRoad>> hashmap3 =
new LinkedHashMap<Integer, Pair<Double, SelectedRoad>>();
// [...] add 216 elements to hasmap3 here somehow
ArrayList<Pair<Double,SelectedRoad>> firstPairs =
Collections.list(Collections.enumeration(hashmap3.values())).subList(0, 100)
// you can then view your Pairs' SelectedRow values with them with:
// (assuming that:
// - your Pair class comes from Apache Commons Lang 3.0
// - your SelectedRoad class implements a decent toString() )
for (final Pair<Double, SelectedRoad> p : firstPairs) {
System.out.println("double: " + p.left);
System.out.println("road : " + p.right);
}
You can do:
Map<Integer, Object> records;
List<Entry<Integer, Object>> firstHundredRecords
= new ArrayList<Entry<Integer, Object>>(records.entrySet()).subList(0, 100);
Although note that this will copy all the entries from the map.
To copy only the records you need with using a library.
Map<Integer, Object> records;
List<Entry<Integer, Object>> firstHundredRecords = new ArrayList<>();
for(Entry<Integer, Object> entry : records.entrySet()) {
firstHundredRecords.add(entry);
if (firstHundredRecords.size()>=100) break;
}
You can use counter. Your foreach loop will exit when your counter reached 100.
Write a loop which uses a Iterator.next() 100 times, and then stops.
I was going to say something about NavigableMap and SortedMap - but their interfaces are defined in terms of keys, not indexes. But they may be useful nevertheless, depending on what your actual underlying problem is.
Suppose we have a HashMap<String, Integer> in Java.
How do I update (increment) the integer-value of the string-key for each existence of the string I find?
One could remove and reenter the pair, but overhead would be a concern.
Another way would be to just put the new pair and the old one would be replaced.
In the latter case, what happens if there is a hashcode collision with a new key I am trying to insert? The correct behavior for a hashtable would be to assign a different place for it, or make a list out of it in the current bucket.
map.put(key, map.get(key) + 1);
should be fine. It will update the value for the existing mapping. Note that this uses auto-boxing. With the help of map.get(key) we get the value of corresponding key, then you can update with your requirement. Here I am updating to increment value by 1.
Java 8 way:
You can use computeIfPresent method and supply it a mapping function, which will be called to compute a new value based on existing one.
For example,
Map<String, Integer> words = new HashMap<>();
words.put("hello", 3);
words.put("world", 4);
words.computeIfPresent("hello", (k, v) -> v + 1);
System.out.println(words.get("hello"));
Alternatevely, you could use merge method, where 1 is the default value and function increments existing value by 1:
words.merge("hello", 1, Integer::sum);
In addition, there is a bunch of other useful methods, such as putIfAbsent, getOrDefault, forEach, etc.
The simplified Java 8 way:
map.put(key, map.getOrDefault(key, 0) + 1);
This uses the method of HashMap that retrieves the value for a key, but if the key can't be retrieved it returns the specified default value (in this case a '0').
This is supported within core Java: HashMap<K,V> getOrDefault(Object key, V defaultValue)
hashmap.put(key, hashmap.get(key) + 1);
The method put will replace the value of an existing key and will create it if doesn't exist.
Replace Integer by AtomicInteger and call one of the incrementAndGet/getAndIncrement methods on it.
An alternative is to wrap an int in your own MutableInteger class which has an increment() method, you only have a threadsafety concern to solve yet.
One line solution:
map.put(key, map.containsKey(key) ? map.get(key) + 1 : 1);
#Matthew's solution is the simplest and will perform well enough in most cases.
If you need high performance, AtomicInteger is a better solution ala #BalusC.
However, a faster solution (provided thread safety is not an issue) is to use TObjectIntHashMap which provides a increment(key) method and uses primitives and less objects than creating AtomicIntegers. e.g.
TObjectIntHashMap<String> map = new TObjectIntHashMap<String>()
map.increment("aaa");
You can increment like below but you need to check for existence so that a NullPointerException is not thrown
if(!map.containsKey(key)) {
p.put(key,1);
}
else {
p.put(key, map.getKey()+1);
}
Does the hash exist (with 0 as the value) or is it "put" to the map on the first increment? If it is "put" on the first increment, the code should look like:
if (hashmap.containsKey(key)) {
hashmap.put(key, hashmap.get(key)+1);
} else {
hashmap.put(key,1);
}
It may be little late but here are my two cents.
If you are using Java 8 then you can make use of computeIfPresent method. If the value for the specified key is present and non-null then it attempts to compute a new mapping given the key and its current mapped value.
final Map<String,Integer> map1 = new HashMap<>();
map1.put("A",0);
map1.put("B",0);
map1.computeIfPresent("B",(k,v)->v+1); //[A=0, B=1]
We can also make use of another method putIfAbsent to put a key. If the specified key is not already associated with a value (or is mapped to null) then this method associates it with the given value and returns null, else returns the current value.
In case the map is shared across threads then we can make use of ConcurrentHashMap and AtomicInteger. From the doc:
An AtomicInteger is an int value that may be updated atomically. An
AtomicInteger is used in applications such as atomically incremented
counters, and cannot be used as a replacement for an Integer. However,
this class does extend Number to allow uniform access by tools and
utilities that deal with numerically-based classes.
We can use them as shown:
final Map<String,AtomicInteger> map2 = new ConcurrentHashMap<>();
map2.putIfAbsent("A",new AtomicInteger(0));
map2.putIfAbsent("B",new AtomicInteger(0)); //[A=0, B=0]
map2.get("B").incrementAndGet(); //[A=0, B=1]
One point to observe is we are invoking get to get the value for key B and then invoking incrementAndGet() on its value which is of course AtomicInteger. We can optimize it as the method putIfAbsent returns the value for the key if already present:
map2.putIfAbsent("B",new AtomicInteger(0)).incrementAndGet();//[A=0, B=2]
On a side note if we plan to use AtomicLong then as per documentation under high contention expected throughput of LongAdder is significantly higher, at the expense of higher space consumption. Also check this question.
The cleaner solution without NullPointerException is:
map.replace(key, map.get(key) + 1);
Since I can't comment to a few answers due to less reputation, I will post a solution which I applied.
for(String key : someArray)
{
if(hashMap.containsKey(key)//will check if a particular key exist or not
{
hashMap.put(hashMap.get(key),value+1);// increment the value by 1 to an already existing key
}
else
{
hashMap.put(key,value);// make a new entry into the hashmap
}
}
Integer i = map.get(key);
if(i == null)
i = (aValue)
map.put(key, i + 1);
or
Integer i = map.get(key);
map.put(key, i == null ? newValue : i + 1);
Integer is Primitive data types http://cs.fit.edu/~ryan/java/language/java-data.html, so you need to take it out, make some process, then put it back. if you have a value which is not Primitive data types, you only need to take it out, process it, no need to put it back into the hashmap.
Use a for loop to increment the index:
for (int i =0; i<5; i++){
HashMap<String, Integer> map = new HashMap<String, Integer>();
map.put("beer", 100);
int beer = map.get("beer")+i;
System.out.println("beer " + beer);
System.out ....
}
There are misleading answers to this question here that imply Hashtable put method will replace the existing value if the key exists, this is not true for Hashtable but rather for HashMap. See Javadoc for HashMap http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html#put%28K,%20V%29
Use Java8 built in fuction 'computeIfPresent'
Example:
public class ExampleToUpdateMapValue {
public static void main(String[] args) {
Map<String,String> bookAuthors = new TreeMap<>();
bookAuthors.put("Genesis","Moses");
bookAuthors.put("Joshua","Joshua");
bookAuthors.put("Judges","Samuel");
System.out.println("---------------------Before----------------------");
bookAuthors.entrySet().stream().forEach(System.out::println);
// To update the existing value using Java 8
bookAuthors.computeIfPresent("Judges", (k,v) -> v = "Samuel/Nathan/Gad");
System.out.println("---------------------After----------------------");
bookAuthors.entrySet().stream().forEach(System.out::println);
}
}
Try:
HashMap hm=new HashMap<String ,Double >();
NOTE:
String->give the new value; //THIS IS THE KEY
else
Double->pass new value; //THIS IS THE VALUE
You can change either the key or the value in your hashmap, but you can't change both at the same time.