We Keep Coding

ArrayList$Itr.checkForComodification throws java.util.ConcurrentModificationException on iterated NOT modified array list [duplicate]

I am using a Collection (a HashMap used indirectly by the JPA, it so happens), but apparently randomly the code throws a ConcurrentModificationException. What is causing it and how do I fix this problem? By using some synchronization, perhaps?
Here is the full stack-trace:
Exception in thread "pool-1-thread-1" java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(Unknown Source)
at java.util.HashMap$ValueIterator.next(Unknown Source)
at org.hibernate.collection.AbstractPersistentCollection$IteratorProxy.next(AbstractPersistentCollection.java:555)
at org.hibernate.engine.Cascade.cascadeCollectionElements(Cascade.java:296)
at org.hibernate.engine.Cascade.cascadeCollection(Cascade.java:242)
at org.hibernate.engine.Cascade.cascadeAssociation(Cascade.java:219)
at org.hibernate.engine.Cascade.cascadeProperty(Cascade.java:169)
at org.hibernate.engine.Cascade.cascade(Cascade.java:130)

This is not a synchronization problem. This will occur if the underlying collection that is being iterated over is modified by anything other than the Iterator itself.
Iterator it = map.entrySet().iterator();
while (it.hasNext()) {
Entry item = it.next();
map.remove(item.getKey());
}
This will throw a ConcurrentModificationException when the it.hasNext() is called the second time.
The correct approach would be
Iterator it = map.entrySet().iterator();
while (it.hasNext()) {
Entry item = it.next();
it.remove();
}
Assuming this iterator supports the remove() operation.

Try using a ConcurrentHashMap instead of a plain HashMap

Modification of a Collection while iterating through that Collection using an Iterator is not permitted by most of the Collection classes. The Java library calls an attempt to modify a Collection while iterating through it a "concurrent modification". That unfortunately suggests the only possible cause is simultaneous modification by multiple threads, but that is not so. Using only one thread it is possible to create an iterator for the Collection (using Collection.iterator(), or an enhanced for loop), start iterating (using Iterator.next(), or equivalently entering the body of the enhanced for loop), modify the Collection, then continue iterating.
To help programmers, some implementations of those Collection classes attempt to detect erroneous concurrent modification, and throw a ConcurrentModificationException if they detect it. However, it is in general not possible and practical to guarantee detection of all concurrent modifications. So erroneous use of the Collection does not always result in a thrown ConcurrentModificationException.
The documentation of ConcurrentModificationException says:
This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible...
Note that this exception does not always indicate that an object has been concurrently modified by a different thread. If a single thread issues a sequence of method invocations that violates the contract of an object, the object may throw this exception...
Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast operations throw ConcurrentModificationException on a best-effort basis.
Note that
the exception may be throw, not must be thrown
different threads are not required
throwing the exception cannot be guaranteed
throwing the exception is on a best-effort basis
throwing the exception happens when the concurrent modification is detected, not when it is caused
The documentation of the HashSet, HashMap, TreeSet and ArrayList classes says this:
The iterators returned [directly or indirectly from this class] are fail-fast: if the [collection] is modified at any time after the iterator is created, in any way except through the iterator's own remove method, the Iterator throws a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.
Note again that the behaviour "cannot be guaranteed" and is only "on a best-effort basis".
The documentation of several methods of the Map interface say this:
Non-concurrent implementations should override this method and, on a best-effort basis, throw a ConcurrentModificationException if it is detected that the mapping function modifies this map during computation. Concurrent implementations should override this method and, on a best-effort basis, throw an IllegalStateException if it is detected that the mapping function modifies this map during computation and as a result computation would never complete.
Note again that only a "best-effort basis" is required for detection, and a ConcurrentModificationException is explicitly suggested only for the non concurrent (non thread-safe) classes.
Debugging ConcurrentModificationException
So, when you see a stack-trace due to a ConcurrentModificationException, you can not immediately assume that the cause is unsafe multi-threaded access to a Collection. You must examine the stack-trace to determine which class of Collection threw the exception (a method of the class will have directly or indirectly thrown it), and for which Collection object. Then you must examine from where that object can be modified.
The most common cause is modification of the Collection within an enhanced for loop over the Collection. Just because you do not see an Iterator object in your source code does not mean there is no Iterator there! Fortunately, one of the statements of the faulty for loop will usually be in the stack-trace, so tracking down the error is usually easy.
A trickier case is when your code passes around references to the Collection object. Note that unmodifiable views of collections (such as produced by Collections.unmodifiableList()) retain a reference to the modifiable collection, so iteration over an "unmodifiable" collection can throw the exception (the modification has been done elsewhere). Other views of your Collection, such as sub lists, Map entry sets and Map key sets also retain references to the original (modifiable) Collection. This can be a problem even for a thread-safe Collection, such as CopyOnWriteList; do not assume that thread-safe (concurrent) collections can never throw the exception.
Which operations can modify a Collection can be unexpected in some cases. For example, LinkedHashMap.get() modifies its collection.
The hardest cases are when the exception is due to concurrent modification by multiple threads.
Programming to prevent concurrent modification errors
When possible, confine all references to a Collection object, so its is easier to prevent concurrent modifications. Make the Collection a private object or a local variable, and do not return references to the Collection or its iterators from methods. It is then much easier to examine all the places where the Collection can be modified. If the Collection is to be used by multiple threads, it is then practical to ensure that the threads access the Collection only with appropriate synchonization and locking.

In Java 8, you can use lambda expression:
map.keySet().removeIf(key -> key condition);
removeIf is a convenient default method in Collection which uses Iterator internally to iterate over the elements of the calling collection.
The extraction of the removal condition is expressed by allowing the caller to provide a Predicate<? super E>.
"I'll perform the iteration for you and test your Predicate on each one of the elements in the collection. If an element causes the test method of the Predicate to return true, I'll remove it."

It sounds less like a Java synchronization issue and more like a database locking problem.
I don't know if adding a version to all your persistent classes will sort it out, but that's one way that Hibernate can provide exclusive access to rows in a table.
Could be that isolation level needs to be higher. If you allow "dirty reads", maybe you need to bump up to serializable.

Note that the selected answer cannot be applied to your context directly before some modification, if you are trying to remove some entries from the map while iterating the map just like me.
I just give my working example here for newbies to save their time:
HashMap<Character,Integer> map=new HashMap();
//adding some entries to the map
...
int threshold;
//initialize the threshold
...
Iterator it=map.entrySet().iterator();
while(it.hasNext()){
Map.Entry<Character,Integer> item=(Map.Entry<Character,Integer>)it.next();
//it.remove() will delete the item from the map
if((Integer)item.getValue()<threshold){
it.remove();
}

Try either CopyOnWriteArrayList or CopyOnWriteArraySet depending on what you are trying to do.

I ran into this exception when try to remove x last items from list.
myList.subList(lastIndex, myList.size()).clear(); was the only solution that worked for me.

When will be concurrent modification exception will be thrown and how iterator remove method will work?

According to the javadocs,conncurrent modification exception will be thrown when we will try to structurally modify the collection while iterating over it.Only iterator remove method will not throw concurrent modification exception.
I have analyzed following cases where concurrent modification exception will not be thrown:
List<Integer> var=new ArrayList<Integer>();
var.add(3);
var.add(2);
var.add(5);
1st case:
for(int i = 0; i<var.size(); i++){
System.out.println(var.get(i));
var.remove(i);
}
2nd Case:
for(Integer i:var){
System.out.println(i);
if(i==2){
var.remove("5");
}
}
1)Can anyone please explain how these cases are not throwing Conncurrent Modification Exception?
2)Can anyone please tell how iterator.remove internally works?

The first implementation isn't using an iterator, so of course it doesn't throw. It's only Iterators that throw (though even the indexed for-loop will be corrupted, e.g. if you remove an element and then the indexes of all the other elements shift out from under your iteration). Java just won't be able to tell you.
The second implementation isn't changing the contents. var.remove("5") attempts to remove the string "5", which isn't present (the list only contains integers), so no change happens. Additionally, i == 2 is dangerous when i is a boxed capital-I Integer; consider doing for (int i : var) instead.
Finally, no implementation guarantees that a ConcurrentModificationException will be thrown; it only tries to do so to warn you your iterator has been corrupted. Whether or not an exception is thrown, the iterator is corrupted by modifications performed while an iteration is in progress, Java's just trying to warn you.

According to the Javadoc of ArrayList:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
So, the conditions under which a CME will be thrown are:
There must be an iterator (or a list iterator)
You must perform a structural modification of the list not via the iterator (i.e. add or remove an element)
And, perhaps not quite so obviously:
After the structural modification, you must call a method on an iterator created before the structural modification.
i.e. the iterator doesn't attempt to detect the CME until you call next() or remove() on it.

Deque remove is not throwing ConcurrentModificationException

The Deque class' Javadoc says:
The iterators returned by this class's iterator method are fail-fast:
If the deque is modified at any time after the iterator is created, in
any way except through the iterator's own remove method, the iterator
will generally throw a ConcurrentModificationException. Thus, in the
face of concurrent modification, the iterator fails quickly and
cleanly, rather than risking arbitrary, non-deterministic behavior at
an undetermined time in the future.
However, the below program behaves differently:
[EDIT]: I am getting error while pasting the whole code "An error occurred submitting the edit". Whew! C'mon SO.
// create an empty array deque with an initial capacity
Deque deque = new ArrayDeque(8);
// use add() method to add elements in the deque
deque.add(15);
deque.add(22);
deque.add(25);
deque.add(20);
System.out.println("printing elements using iterator:");
for(Iterator itr = deque.iterator();itr.hasNext();) {
System.out.println(itr.next());
deque.remove(); //deque is modifed after the iterator is created
}
I expected it to throw ConcurrentModificationException, but it simply prints the following output:
printing elements using iterator:
15
22
25
20
Any idea why?

It looks like it's because you consume the first element of the iterator before removing it. If you change your code to
for(Iterator itr = deque.iterator();itr.hasNext();) {
deque.remove();
System.out.println(itr.next());
}
then you'll see the exception. Your original implementation does indeed contradict the documentation.
However, looking in the implementation of ArrayDeque's iterator implementation, the next() method has this code:
E result = (E) elements[cursor];
// This check doesn't catch all possible comodifications,
// but does catch the ones that corrupt traversal
if (tail != fence || result == null)
throw new ConcurrentModificationException();
Note the following paragraph in the Deque's Javadoc:
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.

fail-fast iterator

I get this definition : As name suggest fail-fast Iterators fail as soon as they realized that structure of Collection has been changed since iteration has begun.
what it mean by since iteration has begun? is that mean after Iterator it=set.iterator() this line of code?
public static void customize(BufferedReader br) throws IOException{
Set<String> set=new HashSet<String>(); // Actual type parameter added
**Iterator it=set.iterator();**

First of all, they are fail-fast, not fail-safe.
The contract is that structural modifications (i.e. insertions/deletions) of certain types of collections invalidate existing iterators into the collection. Fail-fast iterators attempt to detect that they are not supposed to be valid and throw a ConcurrentModificationException. This is done as a service to you, the programmer, to help discover this type of bugs quicker.
In your example:
Iterator it = set.iterator();
it.next();
set.add("unique-entry"); // invalidates the iterator
it.next();
If you're lucky, the second it.next() will detect the invalid usage and throw an exception. Note that this is done on a best-effort basis and is not guaranteed.

is that mean after Iterator it=set.iterator() this line of code?
Yes. If you look at the code for HashSet.iterator() you'll see it's just this:
return map.keySet().iterator();
... which delegate's to HashMap.KeySet.iterator(). There are a few more links in the chain, but eventually you get to HashMap.HashIterator, which contains this in the constructor:
private abstract class HashIterator<E> implements Iterator<E> {
int expectedModCount; // For fast-fail
...
HashIterator() {
expectedModCount = modCount;
...
}
}
... where modCount is a field in the enclosing instance of HashMap which keeps track of the number of modifications.

The iterator being fail-fast means the following piece of code is expected to fail:
Set<String> set = new HashSet<String>();
Iterator<String> it = set.iterator();
set.add("");
it.next(); // the set has changed now, and the iterator will throw an exception
because the following series of events occur: The iterator is created, then its underlying collection changes, then the iterator is accessed.

yes, don't change the collection after using .iterator() if you are planning to iterate over it , you can use the .remove() if you want to remove the latest element though

Before Fail Fast Iterator is starting to work it’s getting count of collection and after any iteration it is checking if count is changed or not, and in case of changed count JVM will throw ConcurrentModificationException. Fail fast iterators are any iterator of collection which is inside java.util package(e.g. ArrayList, LinkedList etc) and Fail Safe iterators are iterators which are inside of java.concurrent package(e.g. CopyOnWriteArrayList, CopyOnWriteSet etc.). Fail Fast iterators will throw exception in case of concurrent modification but Fail Safe iterator is basically working with copy of collection which is not throwing exception in case of concurrent modification.

Concurrent Modification Exception thrown by .next from Iterator

Not sure exactly what's wrong here:
while(itr.hasNext())
{
Stock temp =itr.next();
}
This code is throwing a ConcurrentModificationException at itr.next();
Initialization for the iterator is
private Iterator<Stock> itr=stockList.iterator();
Any ideas?
[The basic code was copied directly from professor's slides]

This could be happening because of two reasons.
Another thread is updating stockList either directly or through its iterator
In the same thread, maybe inside this loop itself, the stockList is modified (see below for an example)
The below codes could cause ConcurrentModificationException
Iterator<Stock> itr = stockList.iterator();
while(itr.hasNext())
{
Stock temp = itr.next();
stockList.add(new Stock()); // Causes ConcurrentModificationException
stockList.remove(0) //Causes ConcurrentModificationException
}

Some other thread is modifying the underlying collection? I suspect that there is code above what you are showing us that causes the problem: a mod to the collection between the call to iterator() and the loop.

Most plausible reason is that some code has modified the underlying collection after you obtained your iterator.
Form javadoc:
The iterators returned by this class's
iterator and listIterator methods are
fail-fast: if the list is structurally
modified at any time after the
iterator is created, in any way except
through the iterator's own remove or
add methods, the iterator will throw a
ConcurrentModificationException. Thus,
in the face of concurrent
modification, the iterator fails
quickly and cleanly, rather than
risking arbitrary, non-deterministic
behavior at an undetermined time in
the future.