We Keep Coding

Thread safety of iterator.next()

If I have an Iterator shared among multiple threads, with each thread calling:
// Inside a thread
int myValue = iterator.next();
what is a possible outcome?
(Ignoring the fact that next() may throw a NoSuchElementException) If the iterator is an iterator over an ArrayList, is it possible that multiple threads may end up with the same value inside the myValue variable?
Is the code below one way to resolve this? (apart from using Java 8 streams as described here Passing a List Iterator to multiple Threads in Java).
// Inside a thread
int myValue;
synchronized(iterator)
{
myValue = iterator.next();
}

TLDR; Never share Iterators between Threads!
Considering the most common use of an iterator for looping over content, you will likely encounter the following snippet:
while(iterator.hasNext()) {
Object nextItem = iterator.next();
}
Now consider the possibility that another Thread performs the exact same operations. Since you cannot control Thread scheduling, the following may happen on an Iterator with a single element:
Thread 1: hasNext? true
Thread 2: hasNext? true
Thread 1: next() //but if this was the last element...
Thread 2: next() //...this will throw NoSuchElementException
Iterators may also support Iterator.remove(), which can lead to ConcurrentModificationException when you operate on shared Collections.
Can we end up with the same value in different Threads?
In a similar fashion to the above, consider this very simple Iterator implementation (simplified code):
class SimpleIterator implements Iterator {
ArrayList source;
int currentIndex;
hasNext() {
return currentIndex<source.size();
}
next() {
Object o = source.get(currentIndex);
currentIndex++;
return o;
}
}
here we may end up with:
Thread 1: get(currentIndex) //Object A
Thread 2: get(currentIndex) //also Object A
Thread 1: currentIndex++
Thread 2: currentIndex++ //you have skipped an element
The answer here is yes, but it's important to note that it very much depends on the implementation. It's much safer not to go there at all.
Repeat: In general you should never share Iterators between Threads.

Is it that multiple threads may end up with the same value inside the
myValue variable?
It's not guaranteed.
Since iterator is not thread-safe, you should synchronize on the collection's object e.g.
Iterator<String> iterator = obj.iterator();
synchronized (obj) {
while (iterator.hasNext()) {
int myValue = iterator.next();
//...
}
}

The behaviour of List#iterator() is inconsistent across List implementations.
ArrayList, LinkedList, will throw ConcurrentModificationException if modified during iteration. To avoid this use a synchronizedList() and lock the List during iteration.
Vectoris synchronized by deafult but the Iterator is not thread safe.
CopyOnWriteArrayList, we can iterate the List safely, even if concurrent modification is happening while iteration.

How Concurrent modification exception is handled internally by CopyOnWriteArrayList/ConcurrentHashMap?

I want to understand internally how concurrent modification exception is handled in concurrent collections like ConcurrentHashMap and CopyOnWriteArrayList.
There are so many blogs available in internet which suggest to use these two data structures to avoid concurrent modification exception. But nothing explains , how this exception is internally handled by concurrent collection.
Can someone give more insights on this? I need some detailed explanation.

The literal answer to your question is not very interesting. ConcurrentHashMap and CopyOnWriteArrayList don't throw ConcurrentModificationException because they don't include code to throw it.
It's not like ConcurrentModificationException is some low-level intrinsic thing. ArrayList and HashMap, among other collection classes, throw ConcurrentModificationException to help you. They have to include extra code to try to detect concurrent modifications, and extra code to throw an exception. ConcurrentModificationException is thrown when one of those classes detect that there is a bug somewhere that is causing an unsafe modification to your collection.
Classes that support safe concurrent modification don't throw ConcurrentModificationException because they don't need to.
If you're trying to debug a ConcurrentModificationException, there are plenty of other questions that help answer that:
Why is a ConcurrentModificationException thrown and how to debug it
Why doesn't this code throw a ConcurrentModificationException?

Here is the add() method definition of ArrayList and CopyOnWriteArrayList.
ArrayList:
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
CopyOnWriteArrayList:
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
Object[] elements = getArray();
int len = elements.length;
Object[] newElements = Arrays.copyOf(elements, len + 1);
newElements[len] = e;
setArray(newElements);
return true;
} finally {
lock.unlock();
}
}
From the above code, it is clear that CopyOnWriteArrayList takes lock before modifying the map. Here I have just posted the code of the add method. If you look on the code of remove() / addAll() or any method which modifies the List structurally you can see that it takes lock before modifying the collection. Also ArrayList's iterator's method such as next()/remove() check for modification but for CopyOnWriteArrayList's iterator's method does not check for the modification. For example :
ArrayList iterator next() method:
#SuppressWarnings("unchecked")
public E next() {
checkForComodification();
int i = cursor;
if (i >= size)
throw new NoSuchElementException();
Object[] elementData = ArrayList.this.elementData;
if (i >= elementData.length)
throw new ConcurrentModificationException();
cursor = i + 1;
return (E) elementData[lastRet = i];
}
CopyOnWriteArrayList iterator next() method:
#SuppressWarnings("unchecked")
public E next() {
if (! hasNext())
throw new NoSuchElementException();
return (E) snapshot[cursor++];
}

This will, right now, answer how CopyOnWriteArrayList avoids the need for a ConcurrentModificationException.
When you modify the collection the CopyOnWriteArrayList does two things
It prevents other threads from modifying the collection via locking
Copies all the elements in the current CopyOnWriteArrayList into a new array and then assigns that new array to the class's array instance
So how does that prevent a CME? A CME in standard collections will only be thrown as a result of iterating. The exception gets thrown if, while iterating over the collection, an add or remove is executed on the same collection instance.
The CopyOnWriteArrayList's iterator assigns the current array as a final field snapshot of the collection and uses that for iteration. If another thread (or even the same thread) tries to add to the CopyOnWriteArrayList then updates will be applied to a new copy and not the snapshot one we are currently iterating.
For instance, we know the add method looks like
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
Object[] elements = getArray();
int len = elements.length;
Object[] newElements = Arrays.copyOf(elements, len + 1);
newElements[len] = e;
setArray(newElements);
return true;
} finally {
lock.unlock();
}
}
Notice the thread local newElements assignment being made, when that is completed it will set to the class instance volatile array.
Then comes the iterator, it's defined as
static final class COWIterator<E> implements ListIterator<E> {
/** Snapshot of the array */
private final Object[] snapshot;
/** Index of element to be returned by subsequent call to next. */
private int cursor;
So when iterating, we are reading whatever was the array prior to any modifications, and since no other thread can modify the snapshot we are looking at a ConcurrentModificationException cannot happen.

Java Concurrent Exception when using copy of lists [duplicate]

This question already has answers here:
Why am I not getting a java.util.ConcurrentModificationException in this example?
(10 answers)
Closed 7 years ago.
I have the following code in my class:
private static LinkedList<MyObject> myList =
new LinkedList<MyObject>();
public static void doEventStuff(String user, String event){
LinkedList<MyObject> copy;
synchronized (myList) {
copy = new LinkedList<>(myList);
}
for (MyObject o : copy) {
... do something with objects o
}
}
public static void removeObject(MyObject o) {
synchronized (myList) {
myList.remove(o);
}
o.doCleanup();
}
public static void terminate() {
synchronized (myList) {
for (MyObject o : myList) {
o.doCleanup();
}
myList.clear();
}
}
public static List<MyObject> getMyObjectsCopy() {
synchronized (myList) {
return new LinkedList<>(myList);
}
}
My problem is a ConcurrentModificationException when calling terminate() , specifically when iterating "for (MyObject o : myList) ".
The list myList is not passed around and can only be accessed through the static methods.
Also: the method MyObject.doCleanup() ca trigger events where the method "removeObject(MyObject)" can be called, when doing the iteration inside terminate() mthod , but since all the methods synchronize
on "myList", I didn't believe a concurrency exception can happen.
Can anyone help me with this issue?

This is not multi-threading issue per se, if you remove an object from the list in a foreach loop you will get ConcurrentModificationException.
And by the way, you can use CopyOnWriteArrayList instead

ConcurrentModificationException also happens if the list was modified while iterating over it using a 'foreach' loop. synchronize will help avoid other threads from accessing your list, but your issue is not due to thread-concurrency. If you want to delete (from the same thread) while iterating over the list, you must use an iterator and call iterator.remove().

In this code:
for (MyObject o : myList) {
o.doCleanup(o);
}
You call code, which internally calls removeObject() method. In this call we make myList.remove(o), which will change a list, as a result, it works like:
for (MyObject o : myList) {
myList.remove();
}
So, it's not a concurrency issue, it's just modification of collection in forEach loop over this collection. I think the best solution for this situation is to avoid removing from myList in doCleanup() code, it looks like lack of design.
Other possible solution - another doCleanup() method version which doesn't throw an event which cause removal from collection - you already do myList.clear().
Or rewrite removeObject() method like:
public static void removeObject(MyObject o) {
synchronized (myList) {
for (Iterator<MyObject> it = myList.iterator(); it.hasNext(); ) {
MyObject o1 = it.next();
if (o1.equals(o)) {
it.remove();
}
}
}
o.doCleanup();
}
like #geert3 recommends in his answer as far as I understand, but motivation in this answer is not fully clear for me.
But I don't like last solution - it looks like a hack for design problem because in this global collection maintaining code we call doCleanup() on deleted object which should call one more removeObject() inside event handler - I think it will be better to remove this "recursion".

Getting java.util.LinkedList$ListItr.checkForComodification exception while using iterators?

I am using the listIterator() for accessing and removing items from a LinkedList in a class that implementsRunnable I am also modifying values of this list in some other part of the program concurrently.
Where I am using listIterator() in this part of the code I am getting ConcurrentModificationException from this function call:
java.util.LinkedList$ListItr.checkForComodification
Why do I get this and how do I prevent it?
#Override
public void run()
{
while(true)
{
itr = nodeAttributes.listIterator();
while (itr.hasNext())
{
System.out.println("enterred");
nodeAttribute nA = (nodeAttribute) itr.next();
//System.out.println("for");
if(!nA.isMoving && !nA.moveQueue.isEmpty())
{
if(animator != null)
animator.stop();
animator = (Animator) nA.moveQueue.poll();
//itr.remove();
animator.start();
nA.isMoving = true;
System.out.print( "animator");
}
}
System.out.println( "looping");
}
}

Your post doesn't have a question, just a statement. However what you describe is the expected behaviour. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the Iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. 
So to prevent this you need to prevent modification by a writer at the same time as a reader is iterating. Use the Collections.synchronizedList method. All access (readers and writers) should follow this pattern:
// store a single synchronized list reference for all access to use
nodeAttributes = Collections.synchronizedList(theLinkedList);
and then all readers and writers should use a synchronized (list) block.
// Readers might do:
synchronized (list) {
itr = nodeAttributes.listIterator();
while (i.hasNext())
... do stuff ...
}
Those threads that operate without iterating can just use the "atomic" methods on the return object from Collections.synchronizedList, such as add. These methods use a sync block under the covers, so they are just a shorthand and they will still block the thread while another is in a sync block.
There are many, many ways to deal with concurrent reader and writers.
One is the above, but it may lock out other threads for a long time while each iterator does it's stuff.
Another is to copy the list to an array (inside a synchronized section) and then read the array outside the lock.
Yet another would be to use a ReadWriteLock.
and there are more options, all dependent on your exact use case.

Concurrent modification exception from a static method that being called asynctask

I have a static method which I am calling from an Asynctask in doInBackGround()
In the method there is this part of code:
ArrayList<Message> messagesList = new ArrayList<Message>();
if (!clearList) {
messagesList.addAll(messages.getMessagesList());
for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
messagesList.remove(msg);
}
}
}
It is sometimes throwing 'Concurrent modification exception', I have tried declaring the method as 'synchronized' but it still didn't help, and I cannot declare the block synchronized, since it is a static method and there is no 'this' reference.
I have also tried to stop a running asynctask if I need to start another one, but it didn't help as well.
Help appreciated.

This has nothing to do with synchronization. You're using an iterator to loop over messagesList, but then using remove to modify it during the iteration. You can't do that, because ArrayList's iterators fail when the list if modified during iteration. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
Your enhanced for loop is just syntactic sugar around using an Iterator, so you can just make that explicit and then use the iterator's remove method:
Iterator<Message> it = messagesList.iterator();
while (it.hasNext()) {
if (it.next().getId().length == 0) {
it.remove();
}
}
Alternately, you can just use a simple for loop running backward and indexing into the ArrayList (since get(int) is a cheap and constant-time operation on an ArrayList, which isn't true of all Lists):
int index;
for (index = messagesList.length - 1; index >= 0; --index) {
if (messagesList.get(index).getId().length == 0) {
messagesList.remove(index);
}
}

for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
messagesList.remove(msg);
}
}
In this code you using messagesList at a time you are also remove data from messagesList thats why you facing error Concurrent modification exception..
Here better way for solved your issue. Copy All data in one arraylist for remove & remove all that data from Main List.
Message removeMsg = new ArrayList<Message>();
for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
removeMsg.add(msg);
}
}
messagesList.removeAll(removeMsg);

Iterators returned by ArrayList is fail-fast in nature.
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
You can call iterator.remove(); and change loop based on iterator explicitly rather than implicitly.
ArrayList<Message> messagesList = new ArrayList<Message>();
if (!clearList) {
messagesList.addAll(messages.getMessagesList());
for (ListIterator<Message> iterator = messagesList.listIterator();iterator.hasNext();) {
Message message = iterator.next();
if (message.getId().length() == 0) {
iterator.remove();
}
}
}
References:
The For-Each Loop
ArrayList Java docs

The for loop is potentially modifying the list over which it is iterating. This is the cause of the exception. The fact that the modification is based on a condition is the reason it does not occur all of the time as the list is not necessarily modified.
Using an Iterator is a possible solution, which provides a remove() method.

You should use Synchronize keyword for this class because static method doesn't belong to any instance

- Your problem is not associated with synchronization, but the problem of ConcurrentModification you are facing is used to protect collection from taking in object of wrong type.
Eg:
Preventing a Cat object enter into a Collection of Dog type.
- You can solve this problem by using Iterator
ArrayList<Message> messagesList = new ArrayList<Message>();
Iterator<Message> itr = messagesList.iterator();
while(itr.hasNext()){
Message m = itr.next();
itr.remove(); // Its remove() method of Iterator NOT ArrayList's
}