We Keep Coding

Iterating HashMap in order

I have a HashMap.
It has 100s of millions of observations.
What's the best way to iterate over the elements of the HashMap, in numerical order of the keys?
I considered changing to TreeMap, but did not do that since it may actually increase the load in creating the Map (as TreeMap is O(n), HashMap is O(1)).

With Java 8 you could use something similar to the following:
import static java.util.Comparator.comparing;
map.entrySet().stream()
.sorted(comparing(Entry::getKey))
.forEach(e -> doSomethingWithTheEntry(e));
That will obviously involve sorting the unsorted keys, which will come at a cost. So you need to decide whether you want to pay the cost upfront with a TreeMap or when required and keep using a HashMap.

You can't iterate over a HashMap in order. You'll have to use TreeMap for that. If you use a LinkedHashMap, you can iterate in the order the keys were inserted to the Map, but it's still not what you want (unless you insert the keys in numerical order).

If your insertion order is the same order as your keys, then you could use a LinkedHashMap.
Hash table and linked list implementation of the Map interface, with predictable iteration order. This implementation differs from HashMap in that it maintains a doubly-linked list running through all of its entries. This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order). Note that insertion order is not affected if a key is re-inserted into the map. (A key k is reinserted into a map m if m.put(k, v) is invoked when m.containsKey(k) would return true immediately prior to the invocation.)

When to use linkedhashmap over hashmap in java?

What are the practical scenario for choosing among the linkedhashmap and hashmap? I have gone through working of each and come to the conclusion that linkedhashmap maintains the order of insertion i.e elements will be retrieved in the same order as that of insertion order while hashmap won't maintain order.
So can someone tell in what practical scenarios selection of one of the collection framework and why?

LinkedHashMap will iterate in the order in which the entries were put into the map.
null Values are allowed in LinkedHashMap.
The implementation is not synchronized and uses double linked buckets.
LinkedHashMap is very similar to HashMap, but it adds awareness to the order at which items are added or accessed, so the iteration order is the same as insertion order depending on construction parameters.
LinkedHashMap also provides a great starting point for creating a Cache object by overriding the removeEldestEntry() method. This lets you create a Cache object that can expire data using some criteria that you define.
Based on linked list and hashing data structures with linked list (think of indexed-SkipList) capability to store data in the way it gets inserted in the tree. Best suited to implement LRU ( least recently used ).
LinkedHashMap extends HashMap.
It maintains a linked list of the entries in the map, in the order in which they were inserted. This allows insertion-order iteration over the map. That is,when iterating through a collection-view of a LinkedHashMap, the elements will be returned in the order in which they were inserted. Also if one inserts the key again into the LinkedHashMap, the original order is retained. This allows insertion-order iteration over the map. That is, when iterating a LinkedHashMap, the elements will be returned in the order in which they were inserted. You can also create a LinkedHashMap that returns its elements in the order in which they were last accessed.
LinkedHashMap constructors
LinkedHashMap( )
This constructor constructs an empty insertion-ordered LinkedHashMap instance with the default initial capacity (16) and load factor (0.75).
LinkedHashMap(int capacity)
This constructor constructs an empty LinkedHashMap with the specified initial capacity.
LinkedHashMap(int capacity, float fillRatio)
This constructor constructs an empty LinkedHashMap with the specified initial capacity and load factor.
LinkedHashMap(Map m)
This constructor constructs a insertion-ordered Linked HashMap with the same mappings as the specified Map.
LinkedHashMap(int capacity, float fillRatio, boolean Order)
This constructor construct an empty LinkedHashMap instance with the specified initial capacity, load factor and ordering mode.
Important methods supported by LinkedHashMap
Class clear( )
Removes all mappings from the map.
containsValue(object value )>
Returns true if this map maps one or more keys to the specified value.
get(Object key)
Returns the value to which the specified key is mapped, or null if this map contains no mapping for the key.
removeEldestEntry(Map.Entry eldest)
Below is an example of how you can use LinkedHashMap:
Map<Integer, String> myLinkedHashMapObject = new LinkedHashMap<Integer, String>();
myLinkedHashMapObject.put(3, "car");
myLinkedHashMapObject.put(5, "bus");
myLinkedHashMapObject.put(7, "nano");
myLinkedHashMapObject.put(9, "innova");
System.out.println("Modification Before" + myLinkedHashMapObject);
System.out.println("Vehicle exists: " +myLinkedHashMapObject.containsKey(3));
System.out.println("vehicle innova Exists: "+myLinkedHashMapObject.containsValue("innova"));
System.out.println("Total number of vehicles: "+ myLinkedHashMapObject.size());
System.out.println("Removing vehicle 9: " + myLinkedHashMapObject.remove(9));
System.out.println("Removing vehicle 25 (does not exist): " + myLinkedHashMapObject.remove(25));
System.out.println("LinkedHashMap After modification" + myLinkedHashMapObject);

Shopping Cart is a real life example, where we see cart number against Item we have chosen in order we selected the item. So map could be LinkedHashMap<Cart Number Vs Item Chosen>

HashMap makes absolutely no guarantees about the iteration order. It can (and will) even change completely when new elements are added.
LinkedHashMap will iterate in the order in which the entries were put into the map
LinkedHashMap also requires more memory than HashMap because of this ordering feature. As I said before LinkedHashMap uses doubly LinkedList to keep order of elements.

In most cases when using a Map you don't care whether the order of insertion is maintained. Use a HashMap if you don't care, and a LinkedHashMap is you care.
However, if you look when and where maps are used, in many cases it contains only a few entries, not enough for the performance difference of the different implementations to make a difference.

LinkedHashMap maintain insertion order of keys, i.e the order in which keys are inserted into LinkedHashMap. On the other hand HashMap doesn't maintain any order or keys or values. In terms of Performance there is not much difference between HashMap and LinkedHashMap but yes LinkedHashMap has more memory foot print than HashMap to maintain doubly linked list which it uses to keep track of insertion order of keys.
A HashMap has a better performance than a LinkedHashMap because a LinkedHashMap needs the expense of maintaining the linked list. The LinkedHashMap implements a normal hashtable, but with the added benefit of the keys of the hashtable being stored as a doubly-linked list.
Both of their methods are not synchronized.
Let's take a look their API documentation:
The HashMap is a hash table with buckets in each hash slot.
API documentation:
This implementation provides constant-time performance for the basic
operations (get and put), assuming the hash function disperses the
elements properly among the buckets. Iteration over collection views
requires time proportional to the "capacity" of the HashMap instance
(the number of buckets) plus its size (the number of key-value
mappings). Thus, it's very important not to set the initial capacity
too high (or the load factor too low) if iteration performance is
important.
LinkedHashMap is a linked list implementing the map interface. As
said in the API documentation:
Hash table and linked list implementation of the Map interface, with
predictable iteration order. This implementation differs from HashMap
in that it maintains a doubly-linked list running through all of its
entries. This linked list defines the iteration ordering, which is
normally the order in which keys were inserted into the map
(insertion-order).

One way that I have used these at work are for cached backend REST queries. These also have the added benefit of returning the data in the some order for the client. You can read more about it in the oracle docs:
https://docs.oracle.com/javase/8/docs/api/java/util/LinkedHashMap.html
This technique is particularly useful if a module takes a map on input, copies it, and later returns results whose order is determined by that of the copy. (Clients generally appreciate having things returned in the same order they were presented.)
A special constructor is provided to create a linked hash map whose order of iteration is the order in which its entries were last accessed, from least-recently accessed to most-recently (access-order). This kind of map is well-suited to building LRU caches. Invoking the put, putIfAbsent, get, getOrDefault, compute, computeIfAbsent, computeIfPresent, or merge methods results in an access to the corresponding entry (assuming it exists after the invocation completes). The replace methods only result in an access of the entry if the value is replaced. The putAll method generates one entry access for each mapping in the specified map, in the order that key-value mappings are provided by the specified map's entry set iterator. No other methods generate entry accesses. In particular, operations on collection-views do not affect the order of iteration of the backing map.

How is the internal implementation of LinkedHashMap different from HashMap implementation?

I read that HashMap has the following implementation:
main array
↓
[Entry] → Entry → Entry ← linked-list implementation
[Entry]
[Entry] → Entry
[Entry]
[null ]
So, it has an array of Entry objects.
Questions:
I was wondering how can an index of this array store multiple Entry objects in case of same hashCode but different objects.
How is this different from LinkedHashMap implementation? Its doubly linked list implementation of map but does it maintain an array like the above and how does it store pointers to the next and previous element?

HashMap does not maintain insertion order, hence it does not maintain any doubly linked list.
Most salient feature of LinkedHashMap is that it maintains insertion order of key-value pairs. LinkedHashMap uses doubly Linked List for doing so.
Entry of LinkedHashMap looks like this-
static class Entry<K, V> {
K key;
V value;
Entry<K,V> next;
Entry<K,V> before, after; //For maintaining insertion order
public Entry(K key, V value, Entry<K,V> next){
this.key = key;
this.value = value;
this.next = next;
}
}
By using before and after - we keep track of newly added entry in LinkedHashMap, which helps us in maintaining insertion order.
Before refers to previous entry and
after refers to next entry in LinkedHashMap.
For diagrams and step by step explanation please refer http://www.javamadesoeasy.com/2015/02/linkedhashmap-custom-implementation.html
Thanks..!!

So, it has an array of Entry objects.
Not exactly. It has an array of Entry object chains. A HashMap.Entry object has a next field allowing the Entry objects to be chained as a linked list.
I was wondering how can an index of this array store multiple Entry objects in case of same hashCode but different objects.
Because (as the picture in your question shows) the Entry objects are chained.
How is this different from LinkedHashMap implementation? Its doubly linked list implementation of map but does it maintain an array like the above and how does it store pointers to the next and previous element?
In the LinkedHashMap implementation, the LinkedHashMap.Entry class extends the HashMap.Entry class, by adding before and after fields. These fields are used to assemble the LinkedHashMap.Entry objects into an independent doubly-linked list that records the insertion order. So, in the LinkedHashMap class, each entry object is in two distinct chains:
There are a number of singly linked hash chains that is accessed via the main hash array. This is used for (regular) hashmap lookups.
There is a separate doubly linked list that contains all of the entry objects. It is kept in entry insertion order, and is used when you iterate the entries, keys or values in the hashmap.

Take a look for yourself. For future reference, you can just google:
java LinkedHashMap source
HashMap uses a LinkedList to handle collissions, but the difference between HashMap and LinkedHashMap is that LinkedHashMap has a predicable iteration order, which is achieved through an additional doubly-linked list, which usually maintains the insertion order of the keys. The exception is when a key is reinserted, in which case it goes back to the original position in the list.
For reference, iterating through a LinkedHashMap is more efficient than iterating through a HashMap, but LinkedHashMap is less memory efficient.
In case it wasn't clear from my above explanation, the hashing process is the same, so you get the benefits of a normal hash, but you also get the iteration benefits as stated above, since you're using a doubly linked list to maintain the ordering of your Entry objects, which is independent of the linked-list used during hashing for collisions, in case that was ambiguous..
EDIT: (in response to OP's comment):
A HashMap is backed by an array, in which some slots contain chains of Entry objects to handle the collisions. To iterate through all of the (key,value) pairs, you would need to go through all of the slots in the array and then go through the LinkedLists; hence, your overall time would be proportional to the capacity.
When using a LinkedHashMap, all you need to do is traverse through the doubly-linked list, so the overall time is proportional to the size.

Since none of the other answers actually explain how something like this could be implemented I'll give it a shot.
One way would be to have some extra information in the value (of the key->value pair) not visible to the user, that had a reference to the previous and next element inserted into the hash map. The benefits are that you can still delete elements in constant time removing from a hashmap is constant time and removing from a linked list is in this case because you have a reference to the entry. You can still insert in constant time because hash map insert is constant, linked list isn't normally but in this case you have constant time access to a spot in the linked list so you can insert in constant time, and lastly retrieval is constant time because you only have to deal with the hash map part of the structure for it.
Keep in mind that a data structure like this does not come without costs. The size of the hash map will rise significantly because of all the extra references. Each of the main methods will be slightly slower (could matter if they are called repeatedly). And the indirection of the data structure (not sure if that's a real term :P) is increased, though this might not be as big a deal because the references are guaranteed to be pointing to stuff inside the hash map.
Since the only advantage of this type of structure is that it preserves order be careful when you use it. Also when reading the answer keep in mind I don't know that this is the way it's implemented but it is how I would do it if given the task.
On the oracle docs there is a quote confirming some of my guesses.
This implementation differs from HashMap in that it maintains a doubly-linked list running through all of its entries.
Another relevant quote from the same website.
This class provides all of the optional Map operations, and permits null elements. Like HashMap, it provides constant-time performance for the basic operations (add, contains and remove), assuming the hash function disperses elements properly among the buckets. Performance is likely to be just slightly below that of HashMap, due to the added expense of maintaining the linked list, with one exception: Iteration over the collection-views of a LinkedHashMap requires time proportional to the size of the map, regardless of its capacity. Iteration over a HashMap is likely to be more expensive, requiring time proportional to its capacity.

hashCode will be mapped to any bucket by the hash function. If there is a collision in hashCode than HashMap resolve this collision by chaining i.e. it will add the value to the linked list. Below is the code which does this:
for (Entry<K,V> e = table[i]; e != null; e = e.next) {
392 Object k;
393 if (e.hash == hash && ((k = e.key) == key || key.equals(k))) {
394 `enter code here` V oldValue = e.value;
395 e.value = value;
396 e.recordAccess(this);
397 return oldValue;
398 }
399 }
You can clearly see that it traverse the linked list and if it finds the key than it replaces the old value with new else append to the linked list.
But the difference between LinkedHashMap and HashMap is LinkedHashMap maintains the insertion order. From docs:
This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order). Note that insertion order is not affected if a key is re-inserted into the map. (A key k is reinserted into a map m if m.put(k, v) is invoked when m.containsKey(k) would return true immediately prior to the invocation).

Will hashmap.keyset() return keys in the order they were added to the hashmap?

I know that the .keySet() returns a set, which is un-ordered.
As far as I can tell, that means I need to keep an array list of keys in order to track the order in which the keys were added to the hashmap, correct?

[...] that means I need to keep an array list of keys in order to track the order in which the keys were added to the hashmap, correct?
Yes, that's correct. Or, you could use a LinkedHashMap which does this for you.
From the documentation:
[...] This implementation differs from HashMap in that it maintains a doubly-linked list running through all of its entries. This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order). [...]

Does entrySet() in a LinkedHashMap also guarantee order?

I am using a linkedHashMap to guarantee order when someone tries to access it. However, when it comes time to iterate over it, does using entrySet() to return key/value pairs guarantee order as well? No changes will be made while iterating.
EDIT: Also, are there any adverse effects from iterating through the map by iterating through its keys and calling get?

According to the Javadocs, yes.
This implementation differs from HashMap in that it maintains a doubly-linked list running through all of its entries. This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order).
As for the edit, no, it should work just fine. But the entry set is somewhat faster since it avoids the overhead of looking up every key in the map during iteration.

If you're sure no changes will be made during the iteration, then proper ordering with entrySet() is guaranteed, as stated in the API.

This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order). Note that insertion order is not affected if a key is re-inserted into the map. (A key k is reinserted into a map m if m.put(k, v) is invoked when m.containsKey(k) would return true immediately prior to the invocation.)