I have to ensure while iterating the Vector; there is no update on that Vector to avoid ConcurrentModificationException. I can use concurrent collection. But i just want to give a try on Vector. Below is the code I have written.
public class TestConcurrentModification1 {
Vector a = new Vector();
public static void main(String[] args) {
final TestConcurrentModification1 obj = new TestConcurrentModification1();
new Thread(){
public void run(){
for(int i = 0; i < 5; i++){
try {
Thread.sleep(1);
} catch (InterruptedException e) {}
obj.a.add(""+i);
}
System.out.println(obj.a);
}
}.start();
new Thread(){
public void run(){
try {
Thread.sleep(10);
} catch (InterruptedException e) {
}
synchronized (obj.a) {
Iterator itr = obj.a.iterator();
while(itr.hasNext()) {
obj.a.add("TEST");//java.lang.OutOfMemoryError: Java heap space
//itr.remove(); //java.lang.IllegalStateException
}
}
}
}.start();
}
}
But the above code is throwing 1) OutOfMemoryError OR 2) IllegalStateException. Could you please explain what is causing these two exception. And how to achieve my goal of avoiding ConcurrentModificationException on a Vector?
I have to solve this for Java 1.4.2 or prior.
One part of your Problem:
Iterator itr = obj.a.iterator();
while(itr.hasNext()) {
obj.a.add("TEST");// <- if itr.hasNext() would have returned false in next iteration, now it won't
}
This is an infinite loop that increases memory usage in each iteration. So you'll run into an OutOfMemory sooner or later.
I suggest using a good old for-loop for inserting values. Use an iterator if you actually want to iterate something :)
More: You are synchronizing against a non-final member.
More: Iterator.remove throws ...
IllegalStateException - if the next method has not yet been called, or the remove method has already been called after the last call to the next method.
And last but not least: The race condition already mentioned by Sotirios (+1 for him).
Whenever you synchronize, make sure you synchronize every call on the critical resource.
You have a good old race condition on your hands.
Your first Thread, except for adding the first element to your Vector, serves absolutely no purpose. You can replace it with
obj.a.add("first");
The beef, as others have noted, is here
Iterator itr = obj.a.iterator();
while (itr.hasNext()) {
obj.a.add("TEST");// java.lang.OutOfMemoryError: Java
// heap space
// itr.remove(); //java.lang.IllegalStateException
}
itr.hasNext() is implemented as
public boolean hasNext() {
return cursor != elementCount;
}
Where cursor starts at 0 and elementCount is the size of your Vector. This call will never return false. Your while loop with loop, adding elements, until the program runs out of memory. The cursor never moves forward because you never call next(). If you do call next() while adding elements directly to the Vector, you will get a ConcurrentModificationException.
Related
I want to understand internally how concurrent modification exception is handled in concurrent collections like ConcurrentHashMap and CopyOnWriteArrayList.
There are so many blogs available in internet which suggest to use these two data structures to avoid concurrent modification exception. But nothing explains , how this exception is internally handled by concurrent collection.
Can someone give more insights on this? I need some detailed explanation.
The literal answer to your question is not very interesting. ConcurrentHashMap and CopyOnWriteArrayList don't throw ConcurrentModificationException because they don't include code to throw it.
It's not like ConcurrentModificationException is some low-level intrinsic thing. ArrayList and HashMap, among other collection classes, throw ConcurrentModificationException to help you. They have to include extra code to try to detect concurrent modifications, and extra code to throw an exception. ConcurrentModificationException is thrown when one of those classes detect that there is a bug somewhere that is causing an unsafe modification to your collection.
Classes that support safe concurrent modification don't throw ConcurrentModificationException because they don't need to.
If you're trying to debug a ConcurrentModificationException, there are plenty of other questions that help answer that:
Why is a ConcurrentModificationException thrown and how to debug it
Why doesn't this code throw a ConcurrentModificationException?
Here is the add() method definition of ArrayList and CopyOnWriteArrayList.
ArrayList:
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
CopyOnWriteArrayList:
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
Object[] elements = getArray();
int len = elements.length;
Object[] newElements = Arrays.copyOf(elements, len + 1);
newElements[len] = e;
setArray(newElements);
return true;
} finally {
lock.unlock();
}
}
From the above code, it is clear that CopyOnWriteArrayList takes lock before modifying the map. Here I have just posted the code of the add method. If you look on the code of remove() / addAll() or any method which modifies the List structurally you can see that it takes lock before modifying the collection. Also ArrayList's iterator's method such as next()/remove() check for modification but for CopyOnWriteArrayList's iterator's method does not check for the modification. For example :
ArrayList iterator next() method:
#SuppressWarnings("unchecked")
public E next() {
checkForComodification();
int i = cursor;
if (i >= size)
throw new NoSuchElementException();
Object[] elementData = ArrayList.this.elementData;
if (i >= elementData.length)
throw new ConcurrentModificationException();
cursor = i + 1;
return (E) elementData[lastRet = i];
}
CopyOnWriteArrayList iterator next() method:
#SuppressWarnings("unchecked")
public E next() {
if (! hasNext())
throw new NoSuchElementException();
return (E) snapshot[cursor++];
}
This will, right now, answer how CopyOnWriteArrayList avoids the need for a ConcurrentModificationException.
When you modify the collection the CopyOnWriteArrayList does two things
It prevents other threads from modifying the collection via locking
Copies all the elements in the current CopyOnWriteArrayList into a new array and then assigns that new array to the class's array instance
So how does that prevent a CME? A CME in standard collections will only be thrown as a result of iterating. The exception gets thrown if, while iterating over the collection, an add or remove is executed on the same collection instance.
The CopyOnWriteArrayList's iterator assigns the current array as a final field snapshot of the collection and uses that for iteration. If another thread (or even the same thread) tries to add to the CopyOnWriteArrayList then updates will be applied to a new copy and not the snapshot one we are currently iterating.
For instance, we know the add method looks like
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
Object[] elements = getArray();
int len = elements.length;
Object[] newElements = Arrays.copyOf(elements, len + 1);
newElements[len] = e;
setArray(newElements);
return true;
} finally {
lock.unlock();
}
}
Notice the thread local newElements assignment being made, when that is completed it will set to the class instance volatile array.
Then comes the iterator, it's defined as
static final class COWIterator<E> implements ListIterator<E> {
/** Snapshot of the array */
private final Object[] snapshot;
/** Index of element to be returned by subsequent call to next. */
private int cursor;
So when iterating, we are reading whatever was the array prior to any modifications, and since no other thread can modify the snapshot we are looking at a ConcurrentModificationException cannot happen.
I have a below method which is called by multiple threads concurrently to get the live socket. It takes LinkedBlockingQueue as the parameter and then I iterate and see if there is any liveSocket available and if it is available then I remove and return that socket.
private Optional<Holder> getSocket(final LinkedBlockingQueue<Holder> endPoints) {
Optional<Holder> liveSocket = Optional.absent();
if (!endPoints.isEmpty()) {
for (Holder state : endPoints) {
// check if socket is live? if yes then remove and return that.
if (state.isLive()) {
liveSocket = Optional.of(state);
endPoints.remove(state);
return liveSocket;
}
}
}
return Optional.absent();
}
Wanted to check if my above code is thread safe or not? Here Holder is an immutable class.
The queue manipulation operations are thread safe, so the remove() will not throw ConcurrentModificationException. However, you have thread-safety problems around the state of the objects contained in the queue.
There's a race condition between when you check the "live" state of the Holder object and when you remove it from the queue. Another thread could be running in the same code at the same time, with the likely result that both threads would take the same object. Whichever thread got to the remove() call last would get a false return, but you don't examine the result so you'd never know. Both threads would then attempt to use the same object.
You need to synchronize around the search/remove operation.
For curiosity, here's the code I used to show that ConcurrentModificationException does not occur with LinkedBlockingQueue:
public static void main(String[] args) throws Exception
{
String[] data = { "a", "b", "c", "d", "e", "f","g" };
LinkedBlockingQueue<String> lb = new LinkedBlockingQueue<>(Arrays.asList(data));
new Thread(() ->
{
try
{
Thread.sleep(2000);
lb.add("x");
System.out.println("added");
Thread.sleep(1000);
lb.remove("e");
System.out.println("removed");
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}).start();
for (String s : lb)
{
System.out.println(s);
Thread.sleep(1000);
}
}
If you substitute LinkedList for LinkedBlockingQueue you get the ConcurrentModificationException as expected.
Output:
a
b
added
c
removed
d
f
g
x
It's not only not thread-safe, it's wrong even within a single thread. You will get a ConcurrentModificationException on the remove(). You need to use an explicit Iterator and to do the removal via the Iterator.
And for correctness via multiple threads you need synchronization or a semaphore around the loop.
NB The isEmpty() test is pointless. The iteration already has to check for that. Don't keep a dog and bark yourself.
Sorry if this is a dumb question. But could someone explain me what could happens in a scenario like this?
List<Integer> scores = new Arraylist<>() ;
scores =
Collections.synchronizedList(scores)
public void add(int element) {
...
scores.add(element)
...
}
public String retrieve(int element) {
...
For (Integer e : scores)....
....
Return something
}
Let's assume that this class is a singelton and that scores is global. Multiple thread can add and retrieve the scores at the same time
In this scenario when starting the for loop and at the same time a thread is adding (or removing an element from the list) will it throw a concurrent modification exeption ?
Thank you
Bad things will happen, given the way you've written your example.
Your retrieve() method doesn't have its loop in a synchronized block, and both of your methods are accessing scores directly, instead of using the List returned by the Collections.synchronizedList() method.
If you take a look at the API for Collections.synchronizedList(), you'll notice that it says
In order to guarantee serial access, it is critical that all access to the backing list is accomplished through the returned list.
It is imperative that the user manually synchronize on the returned list when iterating over it:
Failure to follow this advice may result in non-deterministic behavior.
So you might get a ConcurrentModificationException, or something else weird might happen.
Edit
Even if all your access is via the synchronized List, you can still end up getting a ConcurrentModificationException thrown at you if you modify the List while iterating over it in another thread. That's why the Collections.synchronizedList() documentation insists that you manually wrap your iteration inside a block that is synchronized on the List it returns.
The API for ConcurrentModificationException says
For example, it is not generally permissible for one thread to modify a Collection while another thread is iterating over it. In general, the results of the iteration are undefined under these circumstances. Some Iterator implementations (including those of all the general purpose collection implementations provided by the JRE) may choose to throw this exception if this behavior is detected. Iterators that do this are known as fail-fast iterators, as they fail quickly and cleanly, rather that risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Your add method won't need to be changed, but your retrieve() method should look something like:
public String retrieve(int element) {
// stuff
synchronized (scores) { // prevent scores from being modified while iterating
for (Integer e : scores) {
// looping stuff
}
}
// more stuff
return something;
}
Sample Program
Here's a small sample program which demonstrates the behavior of safe vs unsafe access:
public class Scratch {
private List<Integer> scores = Collections.synchronizedList(new ArrayList<Integer>());
public static void main(String[] args) throws Exception {
final Scratch s = new Scratch();
s.scores.add(1);
s.scores.add(2);
s.scores.add(3);
// keep adding things to the list forever
new Thread(new Runnable() {
#Override
public void run() {
try {
int i=100;
while (true) {
Thread.sleep(100);
s.scores.add(i++);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
System.out.println("This will run fine");
s.safeLoop();
System.out.println("This will cause a ConcurrentModificationException");
s.unsafeLoop();
}
public void safeLoop() throws InterruptedException {
synchronized (scores) {
for (int i : scores) {
System.out.println("i="+i);
Thread.sleep(100);
}
}
}
public void unsafeLoop() throws InterruptedException {
for (int i : scores) {
System.out.println("i="+i);
Thread.sleep(100);
}
}
}
I'm creating a Swing application to make a Game with. It creates images in random locations off the screen and when they leave the screen I would like to remove them. Please take a look at the code snippet:
public void checkTrolls(){ //CAUSES EXCEPTION ERROR WHEN SPRITE EXIT SCREEN
for(AutomatedSprite a : trolls){
if(a.getX() < 0 - a.getImage().getWidth())
trolls.remove(a);
if(a.getY() < 0 - a.getImage().getWidth())
trolls.remove(a);
if(a.getX() > 800)
trolls.remove(a);
if(a.getY() > 600)
trolls.remove(a);
}
}
#Override
public void run() {
long beforeTime, timeDiff, sleep;
beforeTime = System.currentTimeMillis();
while(true){
dodger.update(); //update sprite
if(trolls.size() != 6){
trolls.add(new AutomatedSprite("images/troll_face.png"));
}
for(Sprite troll : trolls){
troll.update(); //UPDATES MY SPRITES
}
checkTrolls(); //CHECKS TROLLS EXITING THE SCREEN
repaint();
for(Sprite troll : trolls){
System.out.println("X: " + troll.getX());
System.out.println("Y: " + troll.getY());
}
timeDiff = System.currentTimeMillis() - beforeTime;
sleep = timeDiff - DELAY;
if(sleep < 0)
sleep = 5;
try {
Thread.sleep(sleep);
} catch (InterruptedException e) { e.printStackTrace(); }
beforeTime = System.currentTimeMillis();
}
}
trolls is a Vector of AutomatedSprites, when they leave the screen I get a ConcurrentModificationException, apparently I can't remove the instances from my vector.
So it seems that I can't remove anything from the vector while the thread is updating all my sprites, is there a way to pause my thread so I can remove the sprite?
P.S: here is the entire class in case I missed something: Pastebin
You cannot remove from a collection while iterating over it, this is true for a single threaded environment as well as a multithreaded one. syncrhonized will still cause the issue, Thread.sleep too. Use and Iterator and remove that way.
public void checkTrolls(){
for(Iterator<AutomatedSprite> itr = trolls.iterator(); itr.hasNext();){
AutomatedSpring nextElemnt = itr.next();
if(youShouldRemoveTheSprite){
itr.remove();
}
}
}
So here you use the Iterator supplied by your trolls collection. And you are asking the Iterator to safely remove the object from the collection.
Now if you are executed checkTrolls with multiple threads then you will need to synchronize. You can do that like this
public synchronized void checkTrolls(){ ...
Edit based on your recent comment/link.
It isn't so much you assigning the Iterator.next() to a variable, its that you are invoking iterator.next() many times. Each time you invoke next() you are moving the iterator to the List's next element. So at the end of one loop iteration you move the iterator to the 6'th element in the list. If you were indexing it instead it would look like:
for(Iterator<AutomatedSprite> itr = trolls.iterator(); itr.hasNext(); ){
if(trolls.get(0).getX() < 0 - trolls.get(1).getImage().getWidth() ||
trolls.get(2).getY() < 0 - trolls.get(3).getImage().getWidth() ||
trolls.get(4).getX() > 800 ||
trolls.get(5).getY() > 600){
trolls.remove(trolls.get(5));
}
Note for this example: indexing at 0,1,2,3,4... is only for demonstration, in practice it would be i = 0; start for loop trolls.get(i++).getX(), trolls.get(i++).getY() and so forth. If your list was 10,000 you would eventually get a NoSuchElementException
So for example, if you only have 3 trolls, once you get to the 4th itr.next() you'll get a NoSuchElementException. For that reason you'll want to store the next() element in a variable and work on that variable so the itr.hasNext(); returns correctly and the itr.remove() too works correctly.
In checkTrolls() you are removing elements from your Vector as you iterate over it. This will cause a ConcurrentModificationException
From Vector's javadoc
The iterators returned by this class's iterator and listIterator
methods are fail-fast: if the vector is structurally modified at any
time after the iterator is created, in any way except through the
iterator's own remove or add methods, the iterator will throw a
ConcurrentModificationException.
A couple alternatives:
Create a list of elements to remove and remove them after iterating fully.
Use a thread safe List implementation such as CopyOnWriteArrayList
Have your code inside run method in synchronized block. synchronized will make sure only one thread executing your code at a time.
run(){
synchronized(this){
....your code.
}}
You may make run() also synchronized.
A simple and somewhat naive solution would be too simply synchronize on the trolls collection. If you do this, make sure to yield in the Animator thread to give your checkTrolls function a chance to do processing and cleanup.
I think the problem might be that in 'checkTrolls' you're trying to modify a vector over which you're iterating.
I was trying to use the iterator methods on a BlockingQueue and discovered that hasNext() is non-blocking - i.e. it will not wait until more elements are added and will instead return false when there are no elements.
So here are the questions :
Is this bad design, or wrong
expectation?
Is there a way to use the blocking
methods of the BLockingQueue with
its parent Collection class methods
(e.g. if some method were expecting
a collection, can I pass a blocking
queue and hope that its processing
will wait until the Queue has more
elements)
Here is a sample code block
public class SomeContainer{
public static void main(String[] args){
BlockingQueue bq = new LinkedBlockingQueue();
SomeContainer h = new SomeContainer();
Producer p = new Producer(bq);
Consumer c = new Consumer(bq);
p.produce();
c.consume();
}
static class Producer{
BlockingQueue q;
public Producer(BlockingQueue q) {
this.q = q;
}
void produce(){
new Thread(){
public void run() {
for(int i=0; i<10; i++){
for(int j=0;j<10; j++){
q.add(i+" - "+j);
}
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
}.start();
}
}
static class Consumer{
BlockingQueue q;
public Consumer(BlockingQueue q) {
this.q = q;
}
void consume() {
new Thread() {
public void run() {
Iterator itr = q.iterator();
while (itr.hasNext())
System.out.println(itr.next());
}
}.start();
}
}
}
This Code only prints the iteration once at the most.
Just don't use iterators with Queues. Use peek() or poll() instead or take() if it's a BlockingQueue:
void consume() {
new Thread() {
#Override
public void run() {
Object value;
// actually, when using a BlockingQueue,
// take() would be better than poll()
while ((value=q.poll())!=null)
System.out.println(value);
}
}.start();
}
A Queue is an Iterable because it is a Collection and hence needs to provide an iterator() method, but that shouldn't ever be used, or you shouldn't be using a Queue in the first place.
1) Is this bad design, or wrong expectation?
Wrong expectations since it would otherwise violate the contract of Iterator which on Iterator.next() says: Throws: NoSuchElementException - iteration has no more elements.
If next() would block the exception would never be thrown.
2) Is there a way to use the blocking methods
Yes, for instance by extending the class and overriding the next and hasNext methods to use blocking routines instead. Note that hasNext would need to always return true in this case - which again violates the contract.
if an iterator blocked on hasNext then the iteration would never finish unless you explicitly broke out of it, this would be quite a strange design.
In any case the LinkedBlockingQueue javadoc has this to say
Returns an iterator over the elements in this queue in proper sequence.
The returned <tt>Iterator</tt> is a "weakly consistent" iterator that will
never throw {#link ConcurrentModificationException}, and guarantees to
traverse elements as they existed upon construction of the iterator, and
may (but is not guaranteed to) reflect any modifications subsequent to
construction.
I think that it may be reasonable under certain circumstances to have an Iterable whose iterator() will block, although having a seperate BlockingIteratorwould be foolish. The reason for this is because that lests you use an enhanced for loop, which can,in some cases, make your code cleaner. (If it would not accomplish that in your particular circumstance, do not do this at all.)
for(Request request:requests) process(request);
However, the iterator is still not free from a termination condition! The iterator should terminate once the queue has been closed to new items, and runs out of elements.
The issue still remains, though, that if the loop was already blocking on the iterator's next() method, the only way to exit if the queue is closed is to throw an exception, which the surrounding code would need to handle correctly, If you choose to do this, make sure you explain very clearly and precisely, how your implementation works in the javadoc comments.
The Iterator for LinkedBlockingQueue has this as its hasNext implementation:
private Node<E> current;
public boolean hasNext() {
return current != null;
}
so this will only work per call. You can wrap the method in a while(true) loop if you want to wait for elements and use the standard java Iterator idiom:
while (true) {
if(itr.hasNext()) {
System.out.println(itr.next());
}
}