I have next code in background thread
private List<IStartAction> mActions = Collections.synchronizedList(new ArrayList<IStartAction>());
protected void removeNonApplicableActions() {
Iterator<IStartAction> iterator = mActions.iterator();
while (iterator.hasNext()) {
IStartAction action = iterator.next();
if (!action.isApplicable()) {
iterator.remove();
}
}
}
When i run this in main thread got ConcurrentModificationException into iterator.next().
Why is this happening? I use thread-safe collection and remove items through iterator. Collection used in only this thread.
Thread safety for a synchronized collection only applies to one method call. Between method calls the lock is released and another thread can lock the collection. If you perform two operations, anything could happen in the meantime, unless you lock it your self. e.g.
// to add two elements in a row, you must hold the lock.
synchronized(mAction) {
mAction.add(x);
// without holding the lock, anything could happen in between
mAction.add(y);
}
Similarly to iterate over a synchronized collection, you have to hold the lock, otherwise anything could happen between method calls to the iterator.
synchronized (mAction) {
for(Iterator<IStartAction> iter = mActions.iterator(); iter.hashNext();) {
IStartAction action = iter.next();
if (!action.isApplicable()) {
iter.remove();
}
}
}
Related
If I have an Iterator shared among multiple threads, with each thread calling:
// Inside a thread
int myValue = iterator.next();
what is a possible outcome?
(Ignoring the fact that next() may throw a NoSuchElementException) If the iterator is an iterator over an ArrayList, is it possible that multiple threads may end up with the same value inside the myValue variable?
Is the code below one way to resolve this? (apart from using Java 8 streams as described here Passing a List Iterator to multiple Threads in Java).
// Inside a thread
int myValue;
synchronized(iterator)
{
myValue = iterator.next();
}
TLDR; Never share Iterators between Threads!
Considering the most common use of an iterator for looping over content, you will likely encounter the following snippet:
while(iterator.hasNext()) {
Object nextItem = iterator.next();
}
Now consider the possibility that another Thread performs the exact same operations. Since you cannot control Thread scheduling, the following may happen on an Iterator with a single element:
Thread 1: hasNext? true
Thread 2: hasNext? true
Thread 1: next() //but if this was the last element...
Thread 2: next() //...this will throw NoSuchElementException
Iterators may also support Iterator.remove(), which can lead to ConcurrentModificationException when you operate on shared Collections.
Can we end up with the same value in different Threads?
In a similar fashion to the above, consider this very simple Iterator implementation (simplified code):
class SimpleIterator implements Iterator {
ArrayList source;
int currentIndex;
hasNext() {
return currentIndex<source.size();
}
next() {
Object o = source.get(currentIndex);
currentIndex++;
return o;
}
}
here we may end up with:
Thread 1: get(currentIndex) //Object A
Thread 2: get(currentIndex) //also Object A
Thread 1: currentIndex++
Thread 2: currentIndex++ //you have skipped an element
The answer here is yes, but it's important to note that it very much depends on the implementation. It's much safer not to go there at all.
Repeat: In general you should never share Iterators between Threads.
Is it that multiple threads may end up with the same value inside the
myValue variable?
It's not guaranteed.
Since iterator is not thread-safe, you should synchronize on the collection's object e.g.
Iterator<String> iterator = obj.iterator();
synchronized (obj) {
while (iterator.hasNext()) {
int myValue = iterator.next();
//...
}
}
The behaviour of List#iterator() is inconsistent across List implementations.
ArrayList, LinkedList, will throw ConcurrentModificationException if modified during iteration. To avoid this use a synchronizedList() and lock the List during iteration.
Vectoris synchronized by deafult but the Iterator is not thread safe.
CopyOnWriteArrayList, we can iterate the List safely, even if concurrent modification is happening while iteration.
I have 2 inner thread classes of class Main. Sometimes, It causes ConcurrentModificationException when one add new element while another is removed. I think I don't know how to synchronize them.
Class Main{
HashSet<MyObject> set;
Thread A{
run(running){
...
set.add(obj);
...
}
}
Thread B{
run(){
while (running) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
MyObject obj= i.next();
if (!obj.isSmt()) {
i.remove();
...
}
}
}
}
}
}
The simplest solution is to isolate the reading code from the writing code. You would do that by surrounding the modifications with synchronized(set) blocks. For the first call, we must synchronize around the add call:
run(running){
...
synchronized(set) {
set.add(obj);
}
...
}
For the second call, we need to synchronize around the entire iteration to avoid concurrent modification. i.remove() is correct in a single threaded case, but as you've discovered, it doesn't work across multiple threads.
synchronized(set) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
MyObject obj= i.next();
if (!obj.isSmt()) {
i.remove();
...
}
}
}
synchronized(set) is a lock on the object set. Only one thread will be able to enter either of the synchronized blocks at a given time, preventing items from being added to the set while a thread is iterating over it.
The ConcurrentModificationException is caused by set.add(obj) in ThreadA while the iteration is in progress in ThreadB (and not by the set.remove() during the loop).
The threads need to be synchronized in order to avoid this.
Threads are synchronized using intrinsic locks over some object. You declare this using the 'synchronized' keyword:
// entire method synchronized on 'this'
synchronized SomeValue foo();
// block synchronized on obj:
synchronized( obj ) {
// stuff.
}
Details vary a lot depending on what you need synchronized. In the case of collections, it is generally safe to isolate specific operations like add() or remove(), but if you need to iterate over the elements in the collection, you need to synchronize the entire block that will carry the iteration if you use regular collection implementations:
synchronized( set ) {
for (Iterator<MyObject> i = set.iterator(); i.hasNext();) {
...
}
}
However, it is generally possible to implement more efficient synchronizations depending on the nature of the collection, and it is very easy to make errors when synchronizing by hand. For most cases, it is generally preferable to just use one of the collection implementations found in java.util.concurrent, which implement iterators that are thread-safe and would not throw a ConcurrentModificationException from operations from a different thread.
For some reason, there's no ConcurrentHashSet implementation of Set, but it is possible to obtain an instance of one by using newSetFromMap:
HashSet<MyObject> set = Collections.newSetFromMap( new ConcurrentHashMap<MyObject,Object>() );
Use a copy of collection to remove item, you cannot remove item while iterating over collection.
To synchronize use the Lock() over collection.
Lock myLock= new Lock();
myLock.lock();
set.add(item);
myLock.unlock();
myLock.lock();
...while loop and modification..
myLock.unlock();
I am using the listIterator() for accessing and removing items from a LinkedList in a class that implementsRunnable I am also modifying values of this list in some other part of the program concurrently.
Where I am using listIterator() in this part of the code I am getting ConcurrentModificationException from this function call:
java.util.LinkedList$ListItr.checkForComodification
Why do I get this and how do I prevent it?
#Override
public void run()
{
while(true)
{
itr = nodeAttributes.listIterator();
while (itr.hasNext())
{
System.out.println("enterred");
nodeAttribute nA = (nodeAttribute) itr.next();
//System.out.println("for");
if(!nA.isMoving && !nA.moveQueue.isEmpty())
{
if(animator != null)
animator.stop();
animator = (Animator) nA.moveQueue.poll();
//itr.remove();
animator.start();
nA.isMoving = true;
System.out.print( "animator");
}
}
System.out.println( "looping");
}
}
Your post doesn't have a question, just a statement. However what you describe is the expected behaviour. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the Iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
So to prevent this you need to prevent modification by a writer at the same time as a reader is iterating. Use the Collections.synchronizedList method. All access (readers and writers) should follow this pattern:
// store a single synchronized list reference for all access to use
nodeAttributes = Collections.synchronizedList(theLinkedList);
and then all readers and writers should use a synchronized (list) block.
// Readers might do:
synchronized (list) {
itr = nodeAttributes.listIterator();
while (i.hasNext())
... do stuff ...
}
Those threads that operate without iterating can just use the "atomic" methods on the return object from Collections.synchronizedList, such as add. These methods use a sync block under the covers, so they are just a shorthand and they will still block the thread while another is in a sync block.
There are many, many ways to deal with concurrent reader and writers.
One is the above, but it may lock out other threads for a long time while each iterator does it's stuff.
Another is to copy the list to an array (inside a synchronized section) and then read the array outside the lock.
Yet another would be to use a ReadWriteLock.
and there are more options, all dependent on your exact use case.
I was trying to use the iterator methods on a BlockingQueue and discovered that hasNext() is non-blocking - i.e. it will not wait until more elements are added and will instead return false when there are no elements.
So here are the questions :
Is this bad design, or wrong
expectation?
Is there a way to use the blocking
methods of the BLockingQueue with
its parent Collection class methods
(e.g. if some method were expecting
a collection, can I pass a blocking
queue and hope that its processing
will wait until the Queue has more
elements)
Here is a sample code block
public class SomeContainer{
public static void main(String[] args){
BlockingQueue bq = new LinkedBlockingQueue();
SomeContainer h = new SomeContainer();
Producer p = new Producer(bq);
Consumer c = new Consumer(bq);
p.produce();
c.consume();
}
static class Producer{
BlockingQueue q;
public Producer(BlockingQueue q) {
this.q = q;
}
void produce(){
new Thread(){
public void run() {
for(int i=0; i<10; i++){
for(int j=0;j<10; j++){
q.add(i+" - "+j);
}
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
}.start();
}
}
static class Consumer{
BlockingQueue q;
public Consumer(BlockingQueue q) {
this.q = q;
}
void consume() {
new Thread() {
public void run() {
Iterator itr = q.iterator();
while (itr.hasNext())
System.out.println(itr.next());
}
}.start();
}
}
}
This Code only prints the iteration once at the most.
Just don't use iterators with Queues. Use peek() or poll() instead or take() if it's a BlockingQueue:
void consume() {
new Thread() {
#Override
public void run() {
Object value;
// actually, when using a BlockingQueue,
// take() would be better than poll()
while ((value=q.poll())!=null)
System.out.println(value);
}
}.start();
}
A Queue is an Iterable because it is a Collection and hence needs to provide an iterator() method, but that shouldn't ever be used, or you shouldn't be using a Queue in the first place.
1) Is this bad design, or wrong expectation?
Wrong expectations since it would otherwise violate the contract of Iterator which on Iterator.next() says: Throws: NoSuchElementException - iteration has no more elements.
If next() would block the exception would never be thrown.
2) Is there a way to use the blocking methods
Yes, for instance by extending the class and overriding the next and hasNext methods to use blocking routines instead. Note that hasNext would need to always return true in this case - which again violates the contract.
if an iterator blocked on hasNext then the iteration would never finish unless you explicitly broke out of it, this would be quite a strange design.
In any case the LinkedBlockingQueue javadoc has this to say
Returns an iterator over the elements in this queue in proper sequence.
The returned <tt>Iterator</tt> is a "weakly consistent" iterator that will
never throw {#link ConcurrentModificationException}, and guarantees to
traverse elements as they existed upon construction of the iterator, and
may (but is not guaranteed to) reflect any modifications subsequent to
construction.
I think that it may be reasonable under certain circumstances to have an Iterable whose iterator() will block, although having a seperate BlockingIteratorwould be foolish. The reason for this is because that lests you use an enhanced for loop, which can,in some cases, make your code cleaner. (If it would not accomplish that in your particular circumstance, do not do this at all.)
for(Request request:requests) process(request);
However, the iterator is still not free from a termination condition! The iterator should terminate once the queue has been closed to new items, and runs out of elements.
The issue still remains, though, that if the loop was already blocking on the iterator's next() method, the only way to exit if the queue is closed is to throw an exception, which the surrounding code would need to handle correctly, If you choose to do this, make sure you explain very clearly and precisely, how your implementation works in the javadoc comments.
The Iterator for LinkedBlockingQueue has this as its hasNext implementation:
private Node<E> current;
public boolean hasNext() {
return current != null;
}
so this will only work per call. You can wrap the method in a while(true) loop if you want to wait for elements and use the standard java Iterator idiom:
while (true) {
if(itr.hasNext()) {
System.out.println(itr.next());
}
}
i have created synchronized arrayList like this
import java.text.SimpleDateFormat;
import java.util.*;
class HelloThread
{
int i=1;
List arrayList;
public void go()
{
arrayList=Collections.synchronizedList(new ArrayList());
Thread thread1=new Thread(new Runnable() {
public void run() {
while(i<=10)
{
arrayList.add(i);
i++;
}
}
});
thread1.start();
Thread thred2=new Thread(new Runnable() {
public void run() {
while(true)
{
Iterator it=arrayList.iterator();
while(it.hasNext())
{
System.out.println(it.next());
}
}
}
});
thred2.start();
}
}
public class test
{
public static void main(String[] args)
{
HelloThread hello=new HelloThread();
hello.go();
}
}
but getting exception like this
Exception in thread "Thread-1" java.util.ConcurrentModificationException
anything wrong in my approach ?
Iterator of synchronizedList is not (and can't be) synchronized, you need to synchronize on the list manually while iterating (see javadoc):
synchronized(arrayList) {
Iterator it=arrayList.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
}
Another approach is to use a CopyOnWriteArrayList instead of Collections.synchronizedList(). It implements a copy-on-write semantic and therefore doesn't require synchronization.
Consider using a CopyOnWriteArrayList which is thread-safe. Every time you add an item, a fresh copy of the underlying array is created. However, the iterator will not reflect additions to the list since the iterator was created, but is guaranteed not to throw ConcurrentModificationException.
arrayList=new CopyOnWriteArrayList();
Other answers have identified the problem:
The iterators for synchronized collections are not synchronized. In fact, they are simply the iterators returned by the collection objects inside the wrapper classes.
Many collection classes (including ArrayList) use a fail-fast mechanism to detect concurrent modifications during iteration. This behavior is clearly documented in the javadocs for the respective classes. This is what you are seeing.
Not all collection classes do this. For example, many of the java.util.Concurrent... collection classes allow concurrent modification during iteration, but relax the semantics of the iteration sequence so that the results of the modifications may or may not be apparent in the objects returned by the iterator.
The javadoc for the Collections.synchronizedList() explains how to synchronize the iterator. Basically you do this:
List list = Collections.synchronizedList(new ArrayList());
...
synchronized (list) {
Iterator i = list.iterator(); // Must be in synchronized block
while (i.hasNext())
foo(i.next());
}
(Aside: normally it is not safe to assume that doing something like this would work. In theory, the synchronized list could use a private lock object, and the synchronized statement would not lock out concurrent modifications. However the javadocs say that this is what to do in this case ... so it is safe.)
The problem with doing that is that locking the collection creates a potential concurrency bottleneck. The alternative to is to use a copy-on-write data structure that internally makes a copy of the relevant parts of the collection. This approach means that an iterator sees sees a snapshot of the collection. Modifications may be made to the collection concurrent with an iteration, but the iterator does not see them. The problem with copy-on-write is that modifications are potentially a lot more expensive.
Ultimately, you need to balance the characteristics and costs of the different collection types wrt concurrent modification versus your actual requirements. Can you get away with the iterator not seeing all concurrent modifications?
The java.util.ConcurrentModificationException occurs when you manipulate (add,remove) a collection while iterating over the same collection.
You probably want to consume the create entries in your second thread while after they have been created by your first thread. So you could use ArrayLists get( index ) and size() for control
As Spike said, you can't modify a collection while iterating it. However, I think the solution is to lock the list while iterating.
class HelloThread
{
int i=1;
List arrayList;
public void go()
{
arrayList=Collections.synchronizedList(new ArrayList());
Thread thread1=new Thread(new Runnable() {
public void run() {
while(i<=10)
{
synchronized(someLock) {
arrayList.add(i);
}
i++;
}
}
});
thread1.start();
Thread thred2=new Thread(new Runnable() {
public void run() {
while(true)
{
synchronized(someLock) {
Iterator it=arrayList.iterator();
while(it.hasNext())
{
System.out.println(it.next());
}
}
}
}
});
thred2.start();
}
}
public class test
{
public static void main(String[] args)
{
HelloThread hello=new HelloThread();
hello.go();
}
}
I'm not sure what you're trying to do, so I hope this doesn't break the functionality of your code.
You may not modify a Collection that you are iterating. You can work around this by accessing the array entries by index, not through an Iterator. I can provide more advice if you tell me the problem that you are trying to solve with this code.
Let's take a normal list (implemented by the ArrayList class) and make it synchronized. This is shown in the SynchronizedArrayList class.
We pass the Collections.synchronizedList method a new ArrayList of Strings. The method returns a synchronized List of Strings.
//Here is SynchronizedArrayList class
package com.mnas.technology.automation.utility;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import org.apache.log4j.Logger;
/**
*
* #author manoj.kumar
* #email kumarmanoj.mtech#gmail.com
*
*/
public class SynchronizedArrayList {
static Logger log = Logger.getLogger(SynchronizedArrayList.class.getName());
public static void main(String[] args) {
List<String> synchronizedList = Collections.synchronizedList(new ArrayList<String>());
synchronizedList.add("Aditya");
synchronizedList.add("Siddharth");
synchronizedList.add("Manoj");
// when iterating over a synchronized list, we need to synchronize access to the synchronized list
synchronized (synchronizedList) {
Iterator<String> iterator = synchronizedList.iterator();
while (iterator.hasNext()) {
log.info("Synchronized Array List Items: " + iterator.next());
}
}
}
}
Notice that when iterating over the list, this access is still done using a synchronized block that locks on the synchronizedList object.
In general, iterating over a synchronized collection should be done in a synchronized block
ArrayList is non-synchronized collection and should not be used in concurrent environment without explicit synchronization. To synchronize ArrayList, we can use two methods provided by JDK
Collections.synchronizedList() method – It returns synchronized list backed by the specified list. It is recommended that we should manually synchronize on the returned list when traversing it via Iterator, Spliterator or Stream. Else it may result in non-deterministic behavior. No explicit synchronization is needed to add, remove elements from synchronized arraylist
CopyOnWriteArrayList class – It is a thread-safe variant of ArrayList.
Since you are using iterator without explicit synchronization you are getting the error