If I have an Iterator shared among multiple threads, with each thread calling:
// Inside a thread
int myValue = iterator.next();
what is a possible outcome?
(Ignoring the fact that next() may throw a NoSuchElementException) If the iterator is an iterator over an ArrayList, is it possible that multiple threads may end up with the same value inside the myValue variable?
Is the code below one way to resolve this? (apart from using Java 8 streams as described here Passing a List Iterator to multiple Threads in Java).
// Inside a thread
int myValue;
synchronized(iterator)
{
myValue = iterator.next();
}
TLDR; Never share Iterators between Threads!
Considering the most common use of an iterator for looping over content, you will likely encounter the following snippet:
while(iterator.hasNext()) {
Object nextItem = iterator.next();
}
Now consider the possibility that another Thread performs the exact same operations. Since you cannot control Thread scheduling, the following may happen on an Iterator with a single element:
Thread 1: hasNext? true
Thread 2: hasNext? true
Thread 1: next() //but if this was the last element...
Thread 2: next() //...this will throw NoSuchElementException
Iterators may also support Iterator.remove(), which can lead to ConcurrentModificationException when you operate on shared Collections.
Can we end up with the same value in different Threads?
In a similar fashion to the above, consider this very simple Iterator implementation (simplified code):
class SimpleIterator implements Iterator {
ArrayList source;
int currentIndex;
hasNext() {
return currentIndex<source.size();
}
next() {
Object o = source.get(currentIndex);
currentIndex++;
return o;
}
}
here we may end up with:
Thread 1: get(currentIndex) //Object A
Thread 2: get(currentIndex) //also Object A
Thread 1: currentIndex++
Thread 2: currentIndex++ //you have skipped an element
The answer here is yes, but it's important to note that it very much depends on the implementation. It's much safer not to go there at all.
Repeat: In general you should never share Iterators between Threads.
Is it that multiple threads may end up with the same value inside the
myValue variable?
It's not guaranteed.
Since iterator is not thread-safe, you should synchronize on the collection's object e.g.
Iterator<String> iterator = obj.iterator();
synchronized (obj) {
while (iterator.hasNext()) {
int myValue = iterator.next();
//...
}
}
The behaviour of List#iterator() is inconsistent across List implementations.
ArrayList, LinkedList, will throw ConcurrentModificationException if modified during iteration. To avoid this use a synchronizedList() and lock the List during iteration.
Vectoris synchronized by deafult but the Iterator is not thread safe.
CopyOnWriteArrayList, we can iterate the List safely, even if concurrent modification is happening while iteration.
Related
I have 2 inner thread classes of class Main. Sometimes, It causes ConcurrentModificationException when one add new element while another is removed. I think I don't know how to synchronize them.
Class Main{
HashSet<MyObject> set;
Thread A{
run(running){
...
set.add(obj);
...
}
}
Thread B{
run(){
while (running) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
MyObject obj= i.next();
if (!obj.isSmt()) {
i.remove();
...
}
}
}
}
}
}
The simplest solution is to isolate the reading code from the writing code. You would do that by surrounding the modifications with synchronized(set) blocks. For the first call, we must synchronize around the add call:
run(running){
...
synchronized(set) {
set.add(obj);
}
...
}
For the second call, we need to synchronize around the entire iteration to avoid concurrent modification. i.remove() is correct in a single threaded case, but as you've discovered, it doesn't work across multiple threads.
synchronized(set) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
MyObject obj= i.next();
if (!obj.isSmt()) {
i.remove();
...
}
}
}
synchronized(set) is a lock on the object set. Only one thread will be able to enter either of the synchronized blocks at a given time, preventing items from being added to the set while a thread is iterating over it.
The ConcurrentModificationException is caused by set.add(obj) in ThreadA while the iteration is in progress in ThreadB (and not by the set.remove() during the loop).
The threads need to be synchronized in order to avoid this.
Threads are synchronized using intrinsic locks over some object. You declare this using the 'synchronized' keyword:
// entire method synchronized on 'this'
synchronized SomeValue foo();
// block synchronized on obj:
synchronized( obj ) {
// stuff.
}
Details vary a lot depending on what you need synchronized. In the case of collections, it is generally safe to isolate specific operations like add() or remove(), but if you need to iterate over the elements in the collection, you need to synchronize the entire block that will carry the iteration if you use regular collection implementations:
synchronized( set ) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
...
}
}
However, it is generally possible to implement more efficient synchronizations depending on the nature of the collection, and it is very easy to make errors when synchronizing by hand. For most cases, it is generally preferable to just use one of the collection implementations found in java.util.concurrent, which implement iterators that are thread-safe and would not throw a ConcurrentModificationException from operations from a different thread.
For some reason, there's no ConcurrentHashSet implementation of Set, but it is possible to obtain an instance of one by using newSetFromMap:
HashSet<MyObject> set = Collections.newSetFromMap( new ConcurrentHashMap<MyObject,Object>() );
Use a copy of collection to remove item, you cannot remove item while iterating over collection.
To synchronize use the Lock() over collection.
Lock myLock= new Lock();
myLock.lock();
set.add(item);
myLock.unlock();
myLock.lock();
...while loop and modification..
myLock.unlock();
I have next code in background thread
private List<IStartAction> mActions = Collections.synchronizedList(new ArrayList<IStartAction>());
protected void removeNonApplicableActions() {
Iterator<IStartAction> iterator = mActions.iterator();
while (iterator.hasNext()) {
IStartAction action = iterator.next();
if (!action.isApplicable()) {
iterator.remove();
}
}
}
When i run this in main thread got ConcurrentModificationException into iterator.next().
Why is this happening? I use thread-safe collection and remove items through iterator. Collection used in only this thread.
Thread safety for a synchronized collection only applies to one method call. Between method calls the lock is released and another thread can lock the collection. If you perform two operations, anything could happen in the meantime, unless you lock it your self. e.g.
// to add two elements in a row, you must hold the lock.
synchronized(mAction) {
mAction.add(x);
// without holding the lock, anything could happen in between
mAction.add(y);
}
Similarly to iterate over a synchronized collection, you have to hold the lock, otherwise anything could happen between method calls to the iterator.
synchronized (mAction) {
for(Iterator<IStartAction> iter = mActions.iterator(); iter.hashNext();) {
IStartAction action = iter.next();
if (!action.isApplicable()) {
iter.remove();
}
}
}
I am using the listIterator() for accessing and removing items from a LinkedList in a class that implementsRunnable I am also modifying values of this list in some other part of the program concurrently.
Where I am using listIterator() in this part of the code I am getting ConcurrentModificationException from this function call:
java.util.LinkedList$ListItr.checkForComodification
Why do I get this and how do I prevent it?
#Override
public void run()
{
while(true)
{
itr = nodeAttributes.listIterator();
while (itr.hasNext())
{
System.out.println("enterred");
nodeAttribute nA = (nodeAttribute) itr.next();
//System.out.println("for");
if(!nA.isMoving && !nA.moveQueue.isEmpty())
{
if(animator != null)
animator.stop();
animator = (Animator) nA.moveQueue.poll();
//itr.remove();
animator.start();
nA.isMoving = true;
System.out.print( "animator");
}
}
System.out.println( "looping");
}
}
Your post doesn't have a question, just a statement. However what you describe is the expected behaviour. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the Iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
So to prevent this you need to prevent modification by a writer at the same time as a reader is iterating. Use the Collections.synchronizedList method. All access (readers and writers) should follow this pattern:
// store a single synchronized list reference for all access to use
nodeAttributes = Collections.synchronizedList(theLinkedList);
and then all readers and writers should use a synchronized (list) block.
// Readers might do:
synchronized (list) {
itr = nodeAttributes.listIterator();
while (i.hasNext())
... do stuff ...
}
Those threads that operate without iterating can just use the "atomic" methods on the return object from Collections.synchronizedList, such as add. These methods use a sync block under the covers, so they are just a shorthand and they will still block the thread while another is in a sync block.
There are many, many ways to deal with concurrent reader and writers.
One is the above, but it may lock out other threads for a long time while each iterator does it's stuff.
Another is to copy the list to an array (inside a synchronized section) and then read the array outside the lock.
Yet another would be to use a ReadWriteLock.
and there are more options, all dependent on your exact use case.
this is a follow-up post to Do I need a concurrent collection for adding elements to a list by many threads?
everybody there has focused on expaning of the list. I understand how that can be a problem, but .. what about adding elements ? is that thread-safe?
example code
static final Collection<String> FILES = new ArrayList<String>(1000000);
and I execute in many threads (I add less than 1000000 elements)
FILES.add(string)
is that thread safe ? what are possible problems with doing it that way ?
ArrayList<String> is not synchronized by itself. Use Collections.synchronizedList(new ArrayList<String>()) instead.
use Collections.syncronizedList(new ArrayList<String>(1000000))
Even if the list doesn't expand, it maintains its internal state, such as the current size. In a wider sense, ArrayList is specified a mutable object which is not thread-safe, therefore it is illegal to concurrently call any mutator methods on it.
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
This is the source cod of the ArrayList in 7u40-b43.
The method ensureCapacityInternal() is not thread safe,and size++ is not either.
The size++ is done by three steps,1)get the size,2) size +1,3) write the new value back.
I have a static method which I am calling from an Asynctask in doInBackGround()
In the method there is this part of code:
ArrayList<Message> messagesList = new ArrayList<Message>();
if (!clearList) {
messagesList.addAll(messages.getMessagesList());
for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
messagesList.remove(msg);
}
}
}
It is sometimes throwing 'Concurrent modification exception', I have tried declaring the method as 'synchronized' but it still didn't help, and I cannot declare the block synchronized, since it is a static method and there is no 'this' reference.
I have also tried to stop a running asynctask if I need to start another one, but it didn't help as well.
Help appreciated.
This has nothing to do with synchronization. You're using an iterator to loop over messagesList, but then using remove to modify it during the iteration. You can't do that, because ArrayList's iterators fail when the list if modified during iteration. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
Your enhanced for loop is just syntactic sugar around using an Iterator, so you can just make that explicit and then use the iterator's remove method:
Iterator<Message> it = messagesList.iterator();
while (it.hasNext()) {
if (it.next().getId().length == 0) {
it.remove();
}
}
Alternately, you can just use a simple for loop running backward and indexing into the ArrayList (since get(int) is a cheap and constant-time operation on an ArrayList, which isn't true of all Lists):
int index;
for (index = messagesList.length - 1; index >= 0; --index) {
if (messagesList.get(index).getId().length == 0) {
messagesList.remove(index);
}
}
for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
messagesList.remove(msg);
}
}
In this code you using messagesList at a time you are also remove data from messagesList thats why you facing error Concurrent modification exception..
Here better way for solved your issue. Copy All data in one arraylist for remove & remove all that data from Main List.
Message removeMsg = new ArrayList<Message>();
for (Message msg : messagesList) {
if (msg.getId().length() == 0) {
removeMsg.add(msg);
}
}
messagesList.removeAll(removeMsg);
Iterators returned by ArrayList is fail-fast in nature.
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
You can call iterator.remove(); and change loop based on iterator explicitly rather than implicitly.
ArrayList<Message> messagesList = new ArrayList<Message>();
if (!clearList) {
messagesList.addAll(messages.getMessagesList());
for (ListIterator<Message> iterator = messagesList.listIterator();iterator.hasNext();) {
Message message = iterator.next();
if (message.getId().length() == 0) {
iterator.remove();
}
}
}
References:
The For-Each Loop
ArrayList Java docs
The for loop is potentially modifying the list over which it is iterating. This is the cause of the exception. The fact that the modification is based on a condition is the reason it does not occur all of the time as the list is not necessarily modified.
Using an Iterator is a possible solution, which provides a remove() method.
You should use Synchronize keyword for this class because static method doesn't belong to any instance
- Your problem is not associated with synchronization, but the problem of ConcurrentModification you are facing is used to protect collection from taking in object of wrong type.
Eg:
Preventing a Cat object enter into a Collection of Dog type.
- You can solve this problem by using Iterator
ArrayList<Message> messagesList = new ArrayList<Message>();
Iterator<Message> itr = messagesList.iterator();
while(itr.hasNext()){
Message m = itr.next();
itr.remove(); // Its remove() method of Iterator NOT ArrayList's
}