We Keep Coding

List throwing ConcurrentModificationException passed to CompletableFutures

Scenario 1:::
So I have list of params, which is passed to 2 methods which calls web service and gets the data. These to methods just do stream.filter.collect on the list of params to get the needed parameter for rest call.
Now I have made the 2 calls parallel using CompletableFutures.
Can this throw ConcurrentModifcation exception?
Scenario 2:::
Similar setup as above , just that now one method changes the list of params and adds some objects to it. I know this is throwing Concurrent Modification exp. Should I just make list as copyonWriteArraylist or create new list with deep copy to avoid any further problems.

Scenario #1: Probably not, but your description is too vague to be sure.
Scenario #2: Most absolutely.
The only thing you need for CoModEx to occur is that the list is changed in any way. Be it add, addAll, clear, remove, retainAll, or any other method on List that has the effect of changing the list itself. Even fetching a sublist and changing THAT (as changes to sublist are visible from the 'outer' list that the sublist was created from).
CoModEx, despite the use of the word 'concurrent', has zip squat to do with threads. In fact, messing with a list from two threads simultaneously is one of the few ways you can break things (methods no longer do what their javadoc says they should) without causing a ConcurrentModificationException (will depend on how the race condition goes).
Here is a trivial way to get a CoModEx:
var list = new ArrayList<String>();
list.add("Hello");
list.add("World");
for (String item : list) if (item.equals("Hello")) list.remove(item);
That will throw it. Every time. CoModEx is thrown by iterators (and the for (x:y) constructor will implicitly create iterators, as does x.stream()..., which creates a spliterator, which also does this) when the underlying data structure was changed in any way that is not directly done by the (spl)iterator itself. For example, this is the one way you get to remove things from your own list using an iterator that does not result in CoModEx:
var it = list.iterator();
while (it.hasNext()) {
if (it.next().startsWith("Hello")) it.remove();
}
Note I'm calling iterator's remove, not list's remove, which would have caused CoModEx: That would change the underlying list (and not via the iterator directly), therefore any operation on an iterator created before the modification will throw CoModEx.
So, this is the flow:
You create an iterator from list, by entering for (String item : list).
That iterator's hasNext() is invoked to check if the for loop should be entered. It returns true
That iterator's next() is invoked for the first loop; Hello is returned.
Due to the code inside the for loop, list.remove("Hello") is invoked. This 'invalidates' all iterators that were created by this list so far.
the for loop loops, and invokes hasNext() to check if it should loop again.
hasNext realizes that it is invalid, and throws CoModEx.
ArrayList does this by having a counter which is incremented every time anything changes, and all iterators remember the value of the counter when created, and check that the list's counter value is equal to their own. If not, they throw CoModEx. Other list impls can use different mechanisms if they desire. Some go out of their way to actually allow this (such as CopyOnWriteArrayList, which explicitly details how it DOES let you modify itself during iteration).
If multiple threads are involved, all bets are off - those counter writes are not synchronized and therefore may or may not be visible by the threads involved. Don't access the same list from different threads unless you really know what you are doing.

what will happen if modify a collection while iterating without throwing the ConcurrentModificationException

As all known We can't modify a non-thread-safe collection while iterating it since it will throw a ConcurrentModificationException
But what I want to know is that what will happen if it would not throw a Exception and let the iteration and modification happened concurrently.
For example, remove an element from a HashMap while iterating it.
Remove. Since remove operation would not change the length of the underlying table in the HashMap, I think that's not an issue to the iteration.
Put. Maybe problem only occurs when the Put triggers resize() since the underlying table will be shuffled.
Is my analysis correct?

Short answer: no, your analysis is not correct.
If you remove something from a collection while you're iterating over it (without using the iterator), the iterator doesn't have a good way to keep track of where it is. Use a simpler example: List. Say the iterator is at index 10, and you remove index 5. That removal shifts all of the indices. Now you call next() on the iterator, and you.. what? Go to index 11? Stay at index 10? The iterator has no way to know.
Similarly, if you add something to a collection while you're iterating over it (without using the iterator), the iterator doesn't know if that was added before or after the current index, so the next() function is broken.
This doesn't even get into data structures where the iterator order depends on what's in the collection, but the issues are similar to the ones I listed above.

But what I want to know is that what will happen if it would not throw a Exception and let the iteration and modification happened concurrently.
This is hypothetical because the respective (non-concurrent) collections don't work that way. If we hypothesize that they did allow "concurrent" modification, then we still cannot answer without making assumptions about how iteration would then be implemented. Finally, assuming that we just removed the fast-fail tests, then the behaviour will be collection specific.
Looking at your analysis for the HashMap case, you have to consider the internal state of the iterator object. I haven't looked at any specific implementation code, but a typical HashMap iterator will have an index for a hash chain in the main hash array, and a pointer to a node within the hash chain:
A Map.remove won't change the hashmap size, so the chain index won't be invalidated. However, if the wrong entry was removed, we could find that the iterator's node pointer could refer to a node that is no longer in the chain. This could cause the iteration to return deleted map entries.
You are correct that a Map.put that triggered a resize could cause the entries to be redistributed. This could cause some entries to be skipped, and others to be returned twice.

Usually the traditional collection classes in java.util package uses an int variable (modCount) to keep track of modifications (additions and deletions).
When we ask for an Iterator from these collection classes then a object of Iterator which is returned is provided with the existing modification count variable as its expected modification count.
Upon invoking the next() method the Iterator object checks the current modification count variable value against its expected modification count value.
In case of a mismatch it fails fast by throwing ConcurrentModificationException present in java.util package, its a RuntimeException.
Do not get confused between the size of the collection object (as in your question Map) and the total buckets available. Moreover its not about the size, one addition increases the value of the modification count flag and also a deletion increases its value.

ConcurrentModificationException and HashSet.iterator()

I have a for loop like
for (int neighbour : neighbours) {
Where I may modify neighbours within the loop. Found that thats the cause of ConcurrentModificationException. And read from https://stackoverflow.com/a/8189527/292291
Hence if you want to modify the list (or any collection in general),
use iterator, because then it is aware of the modifications and hence
those will be handled properly.
So I tried:
neighboursItr = neighbours.iterator();
while (neighboursItr.hasNext()) {
// try disconnecting vertices
neighbour = neighboursItr.next();
But that doesnt fix the problem. Why?

Are you calling neightbours.remove(neighbour)? In that case, that is the problem. You need to call neightboursItr.remove() instead.

Have you considered creating a new HashSet with desired state? I mean you can iterate through the neighbours and add to the newNeighbours whatever you want.

You may only modify the collection using methods of the iterator while iterating on the collection. So you may call neighboursItr.remove(), but you may not add an element to the collection using neighbours.add(), for example.

You cannot modify collection while iterating. The only exception is using iterator.remove() method (if it is supported by target collection).
The reason is that this is how iterator works. It has to know how to jump to the next element of the collection. If collection is being changed after iterator creation it cannot do this and throws exception.
There are several solutions for this problem. For example if you want to add elements to existing collection during iteration you can create yet another collection where you store new elements and then add all these elements after your iteration is finished.

ConcurrentModificationException when adding inside a foreach loop in ArrayList [duplicate]

This question already has answers here:
Concurrent Modification Exception : adding to an ArrayList
(10 answers)
Closed 7 years ago.
I'm trying to make use of the foreach loop with the arraylist, but when I use it, it gives me error, but when I use normal for loop, it works perfectly, what could be the problem?
The code is here:
for (Pair p2 : R) {
if ((p2.getFirstElm() == p.getSecondElm()) && (p2.getFirstElm() != p2.getSecondElm()))
R.add(new Pair (p.getFirstElm(), p2.getSecondElm()));
else if ((p2.getSecondElm() == p.getFirstElm()) && (p2.getFirstElm() != p2.getSecondElm()))
R.add(new Pair (p2.getFirstElm(), p.getSecondElm()));
// else
// There are no transitive pairs in R.
}
this is the loop that doesnt work, and here is the one that works:
for (int i = 0; i < R.size(); i++) {
if ((R.get(i).getFirstElm() == p.getSecondElm()) && (R.get(i).getFirstElm() != R.get(i).getSecondElm()))
R.add(new Pair (p.getFirstElm(), R.get(i).getSecondElm()));
else if ((R.get(i).getSecondElm() == p.getFirstElm()) && (R.get(i).getFirstElm() != R.get(i).getSecondElm()))
R.add(new Pair (R.get(i).getFirstElm(), p.getSecondElm()));
//else
// There are no transitive pairs in R.
}
the error I'm getting when using the foreach loop is:
Exception in thread "main" java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
at java.util.AbstractList$Itr.next(Unknown Source)
at set.problem.fourth.PoSet.makeTransitive(PoSet.java:145)
at set.problem.fourth.PoSet.addToR(PoSet.java:87)
at set.problem.fourth.PoSetDriver.typicalTesting(PoSetDriver.java:35)
at set.problem.fourth.PoSetDriver.main(PoSetDriver.java:13)

Java Collection classes are fail-fast which means that if the
Collection will be changed while some thread is traversing over it
using iterator, the iterator.next() will throw a ConcurrentModificationException.
This situation can come in case of multithreaded as well as single
threaded environment. - www.javacodegeeks.com
You can't modify a List in a for/each loop, which is syntactic sugar around the Iterator as an implementation detail. You can only safely call .remove() when using the Iterator directly.
Note that Iterator.remove is the only safe way to modify a collection
during iteration; the behavior is unspecified if the underlying
collection is modified in any other way while the iteration is in
progress. - Java Collections
Tutorial
Calling .add() inside the for/each loop modifies the contents, and the Iterator that is used behind the scenes sees this and throws this exception.
A more subtle concern is the that the second way you list, the .size() is increasing every time you .add() so you will end up processing all the things you .add(), this could possibly cause an endless loop depending on what the input data is. I am not sure if this is what you desire.
Solution
I would create another ArrayList and .add() to it all the new things, and then after the loop, use .addAll() on the original ArrayList to combine the two lists together. This will make things explicit in what you are trying to do, that is unless your intention is process all the newly added things as you add them.
2014 Solution:
Always use Immutable collections classes and build new Immutable collection classes instead of trying to modify a single shared one. This is basically what my 2012 answer says but I wanted to make it more explicit.
Guava supports this very well, use ImmutableList.copyOf() to pass around data.
Use Iterables.filter() to filter out stuff into a new ImmutableList, no shared mutable state, means no concurrency problems!

Under the hood, the for-each loop in Java uses an Iterator for traversing the collection (see this article for a detailed explanation.) And the Iterator will throw a ConcurrentModificationException if you modify the collection while iterating over it, see this post.

The problem is that you're doing the R.add() in the first line of the loop.
In the first situation you have an iterator open to the arraylist. When you do an add and then try to iterate again the iterator notices that the data structure has changed underneath you.
In the case of the for look you're just getting a new element each time and don't have the concurrent modification issue, though the size is changing as you add more elements.
To fix the problem you probably want to add to a temporary location and add that after the loop or make a copy of the initial data and add to the original.

Java: Can not change list of iterators

I have a list of ListIterator<PointF> as a class field. I fill it in method grow(). When i try to use iterators from this list i get ConcurrentModificationException.
ListIterator<ListIterator<PointF>> i = mPoints.listIterator();
while (i.hasNext()) {
ListIterator<PointF> j = i.next();
if (j.hasNext())
PointF tmp = j.next(); // Exception here
}
I have no idea why does this code causes exeption in any method besides grow()

If the underlying list changes, the iterator that was obtained before that throws ConcurrentModificationException. So don't store iterators in instance fields.

What we can say for sure is that a ConcurrentModificationException means that the underlying iterable has been modified at some point after your call to get the iterator.
This does not always mean concurrent as in multi-threaded; one can easily trigger this exception by iterating through a list and deleting elements during the loop. So, if there are no other threads potentially modifying this, then we can say that the current thread has modified an iterator's underlying data structure at some point.
There's not enough code here to be sure, but your practice of storing iterators is a little suspicious. When did you add the (inner) iterators to mPoints? If the collection they refer to changes at any time after the iterator was created, it will throw this exception when invoked. Hence as soon as you add an iterator to the mPoints collection, the iterator's data structure is effectively locked for changes, and yet this won't be very clear in the code at all.
So I suspect this is the root cause of your problem. Unless it's for a very short term (and usually within a single lexical scope, e.g. a single method invocation) it's probably a bad idea to store iterators for the reason you're seeing. It might be better to store a reference to the underlying collections themselves, and then create the iterators during the code block above, something like:
ListIterator<Iterable<PointF>> i = mPoints.listIterator();
while (i.hasNext()) {
Iterator<PointF> j = i.next().iterator();
if (j.hasNext())
PointF tmp = j.next();
}
Then again the exact solution depends on the general architecture of your method. The main thing to bear in mind is don't store iterators long-term, because it's almost impossible to make this work reliably. Even if it does work right now, it creates a kind of invisible dependency between different parts of your code that will almost invariably be broken by someone implementing what should be a trivial change.