ConcurrentModificationException after subList() - java

I'm running into this problem in a unit test.
After executing:
List<Card> cleanCards = cards.subList(0, cards.size() - difference);
the following assert gives me a ConcurrentModificationException:
assertEquals(limit, cleanCards.size());
Error description
java.util.ConcurrentModificationException
at java.util.ArrayList$SubList.size(ArrayList.java:1057)
To my knowledge, the 'size()' method does not make structural changes to the list. Am I missing something here?

Most likely the original list is being modified between the sub-list creation and its use. subList does not create an independent new list, but rather a view of a section of the original list, and as the specs say
The semantics of the list returned by this method become undefined if
the backing list (i.e., this list) is structurally modified in any way
other than via the returned list. (Structural modifications are those
that change the size of this list, or otherwise perturb it in such a
fashion that iterations in progress may yield incorrect results.)
and in your case, the "undefined" behaviour seems to result in an exception being thrown, also known as fail-fast behaviour. I reckon an easy solution would be to change the first line above to
List<Card> cleanCards = new ArrayList<>(cards.subList(0, cards.size() - difference));
which copies the sub-list to a completely new list independent of the original.

Related

List throwing ConcurrentModificationException passed to CompletableFutures

Scenario 1:::
So I have list of params, which is passed to 2 methods which calls web service and gets the data. These to methods just do stream.filter.collect on the list of params to get the needed parameter for rest call.
Now I have made the 2 calls parallel using CompletableFutures.
Can this throw ConcurrentModifcation exception?
Scenario 2:::
Similar setup as above , just that now one method changes the list of params and adds some objects to it. I know this is throwing Concurrent Modification exp. Should I just make list as copyonWriteArraylist or create new list with deep copy to avoid any further problems.
Scenario #1: Probably not, but your description is too vague to be sure.
Scenario #2: Most absolutely.
The only thing you need for CoModEx to occur is that the list is changed in any way. Be it add, addAll, clear, remove, retainAll, or any other method on List that has the effect of changing the list itself. Even fetching a sublist and changing THAT (as changes to sublist are visible from the 'outer' list that the sublist was created from).
CoModEx, despite the use of the word 'concurrent', has zip squat to do with threads. In fact, messing with a list from two threads simultaneously is one of the few ways you can break things (methods no longer do what their javadoc says they should) without causing a ConcurrentModificationException (will depend on how the race condition goes).
Here is a trivial way to get a CoModEx:
var list = new ArrayList<String>();
list.add("Hello");
list.add("World");
for (String item : list) if (item.equals("Hello")) list.remove(item);
That will throw it. Every time. CoModEx is thrown by iterators (and the for (x:y) constructor will implicitly create iterators, as does x.stream()..., which creates a spliterator, which also does this) when the underlying data structure was changed in any way that is not directly done by the (spl)iterator itself. For example, this is the one way you get to remove things from your own list using an iterator that does not result in CoModEx:
var it = list.iterator();
while (it.hasNext()) {
if (it.next().startsWith("Hello")) it.remove();
}
Note I'm calling iterator's remove, not list's remove, which would have caused CoModEx: That would change the underlying list (and not via the iterator directly), therefore any operation on an iterator created before the modification will throw CoModEx.
So, this is the flow:
You create an iterator from list, by entering for (String item : list).
That iterator's hasNext() is invoked to check if the for loop should be entered. It returns true
That iterator's next() is invoked for the first loop; Hello is returned.
Due to the code inside the for loop, list.remove("Hello") is invoked. This 'invalidates' all iterators that were created by this list so far.
the for loop loops, and invokes hasNext() to check if it should loop again.
hasNext realizes that it is invalid, and throws CoModEx.
ArrayList does this by having a counter which is incremented every time anything changes, and all iterators remember the value of the counter when created, and check that the list's counter value is equal to their own. If not, they throw CoModEx. Other list impls can use different mechanisms if they desire. Some go out of their way to actually allow this (such as CopyOnWriteArrayList, which explicitly details how it DOES let you modify itself during iteration).
If multiple threads are involved, all bets are off - those counter writes are not synchronized and therefore may or may not be visible by the threads involved. Don't access the same list from different threads unless you really know what you are doing.

Reason Behind the java.util.ConcurrentModificationException

Stack<String> ST = new Stack<String>();
ST.add("one");
ST.add("two");
ST.add("three");
ST.add("four");
ST.addAll(ST.subList(0,2));
System.out.println(ST);
Following Simple Code give java.util.ConcurrentModificationException.I not able to figure it out what is the reason behind for this exception?
List.subList returns a view into the container, not a copy.
From the documentation
The semantics of the list returned by this method become undefined if
the backing list (i.e., this list) is structurally modified in any way
other than via the returned list. (Structural modifications are those
that change the size of this list, or otherwise perturb it in such a
fashion that iterations in progress may yield incorrect results.)
So when you call addAll, it modifies the underlying container and changes its size, invalidating the sublist. But it's still trying to iterate over the sublist to continue adding things.
The documentation for the subList method says:
The semantics of the List returned by this method become undefined if the backing list (i.e., this List) is structurally modified in any way other than via the returned List. (Structural modifications are those that change the size of the List, or otherwise perturb it in such a fashion that iterations in progress may yield incorrect results.)
In other words, you're not allowed to change the real list while you're accessing the sublist.
When you call
ST.addAll(ST.subList(0,2));
you're causing addAll to traverse the sublist and modify the real list (ST) at the same time. addAll will take the first element from the sublist and append it to ST, which is a structural modification that invalidates any sublists based on ST. Then addAll will try to take the second element from the sublist, but the sublist is now invalid because of the change that was just made to ST, so it throws the exception.
The problem happens in here:
ST.addAll(ST.subList(0,2));
The problem is that under the hood the following things are happening:
The ST.subList(0,2) call creates a wrapper for the original list.
The ST.addAll call creates an Iterator for the sublist which is actually a wrapper for a ListIterator for the original list.
Then, while iterating the sublist (and hence the list), it pushes the elements it finds back onto the original list.
The last is a concurrent modification.
Note that this is inherent, not just an implementation detail. The subList method returns a view of the original List. Therefore your statement is inherently iterating and modifying the same (underlying) list concurrently.

what will happen if modify a collection while iterating without throwing the ConcurrentModificationException

As all known We can't modify a non-thread-safe collection while iterating it since it will throw a ConcurrentModificationException
But what I want to know is that what will happen if it would not throw a Exception and let the iteration and modification happened concurrently.
For example, remove an element from a HashMap while iterating it.
Remove. Since remove operation would not change the length of the underlying table in the HashMap, I think that's not an issue to the iteration.
Put. Maybe problem only occurs when the Put triggers resize() since the underlying table will be shuffled.
Is my analysis correct?
Short answer: no, your analysis is not correct.
If you remove something from a collection while you're iterating over it (without using the iterator), the iterator doesn't have a good way to keep track of where it is. Use a simpler example: List. Say the iterator is at index 10, and you remove index 5. That removal shifts all of the indices. Now you call next() on the iterator, and you.. what? Go to index 11? Stay at index 10? The iterator has no way to know.
Similarly, if you add something to a collection while you're iterating over it (without using the iterator), the iterator doesn't know if that was added before or after the current index, so the next() function is broken.
This doesn't even get into data structures where the iterator order depends on what's in the collection, but the issues are similar to the ones I listed above.
But what I want to know is that what will happen if it would not throw a Exception and let the iteration and modification happened concurrently.
This is hypothetical because the respective (non-concurrent) collections don't work that way. If we hypothesize that they did allow "concurrent" modification, then we still cannot answer without making assumptions about how iteration would then be implemented. Finally, assuming that we just removed the fast-fail tests, then the behaviour will be collection specific.
Looking at your analysis for the HashMap case, you have to consider the internal state of the iterator object. I haven't looked at any specific implementation code, but a typical HashMap iterator will have an index for a hash chain in the main hash array, and a pointer to a node within the hash chain:
A Map.remove won't change the hashmap size, so the chain index won't be invalidated. However, if the wrong entry was removed, we could find that the iterator's node pointer could refer to a node that is no longer in the chain. This could cause the iteration to return deleted map entries.
You are correct that a Map.put that triggered a resize could cause the entries to be redistributed. This could cause some entries to be skipped, and others to be returned twice.
Usually the traditional collection classes in java.util package uses an int variable (modCount) to keep track of modifications (additions and deletions).
When we ask for an Iterator from these collection classes then a object of Iterator which is returned is provided with the existing modification count variable as its expected modification count.
Upon invoking the next() method the Iterator object checks the current modification count variable value against its expected modification count value.
In case of a mismatch it fails fast by throwing ConcurrentModificationException present in java.util package, its a RuntimeException.
Do not get confused between the size of the collection object (as in your question Map) and the total buckets available. Moreover its not about the size, one addition increases the value of the modification count flag and also a deletion increases its value.

Deciding between different approaches for instantiating a List<Foo> foo from fooService.getFoos()

If fooService.getFoos() returns List<Foo>.
then you can write this:
List<Foo> fooList = fooService.getFoos();
or this:
List<Foo> fooList = new ArrayList(fooService.getFoos());
Is there any significant difference in the resulting fooList between these two approaches?
Yes - you are creating a completely new List, containing the elements of the original one. You are duplicating the collection in memory, and iterating it from start to end. You are also not using instance provided by the service, and you can't modify the original. And finally, you've omitted the generics declaration in the 2nd snippet.
So use the first option.
Update: you indicated you are not allowed to modify the original list. This is actually a problem of fooService, not of its clients. If the service is also in your control, return Collections.unmodifiableList(originalList) - thus clients won't be able to perform modification operations (on attempt an exception will be thrown)
The second isn't really a good idea because you omit the generic part.
But the main problem is the unnecessary code which will be called. You can look at the ArrayList code source, and you'll see all the operations used in the constructor. If you only need a List, and fooService.getFoos() returns a valid List, you should stick with it.
The result of those two statement will be more or less the same unless:
later you check if your list is an instance of ArrayList and cast it, but let's face it, you would have declared ArrayList<Foo> if it was the case.
the list returned by fooService.getFoos() shouldn't be modified (for any reason) but you still want modify elements in the List on your side (without affecting the original list).
Resources :
grepcode - ArrayList
I'd stick with the first one just because it reads lots easier and makes much more sense than the second one.
In the second statement it returns only of List type. If you are sure of method returning of same type den you can use firs type.

Java: Can not change list of iterators

I have a list of ListIterator<PointF> as a class field. I fill it in method grow(). When i try to use iterators from this list i get ConcurrentModificationException.
ListIterator<ListIterator<PointF>> i = mPoints.listIterator();
while (i.hasNext()) {
ListIterator<PointF> j = i.next();
if (j.hasNext())
PointF tmp = j.next(); // Exception here
}
I have no idea why does this code causes exeption in any method besides grow()
If the underlying list changes, the iterator that was obtained before that throws ConcurrentModificationException. So don't store iterators in instance fields.
What we can say for sure is that a ConcurrentModificationException means that the underlying iterable has been modified at some point after your call to get the iterator.
This does not always mean concurrent as in multi-threaded; one can easily trigger this exception by iterating through a list and deleting elements during the loop. So, if there are no other threads potentially modifying this, then we can say that the current thread has modified an iterator's underlying data structure at some point.
There's not enough code here to be sure, but your practice of storing iterators is a little suspicious. When did you add the (inner) iterators to mPoints? If the collection they refer to changes at any time after the iterator was created, it will throw this exception when invoked. Hence as soon as you add an iterator to the mPoints collection, the iterator's data structure is effectively locked for changes, and yet this won't be very clear in the code at all.
So I suspect this is the root cause of your problem. Unless it's for a very short term (and usually within a single lexical scope, e.g. a single method invocation) it's probably a bad idea to store iterators for the reason you're seeing. It might be better to store a reference to the underlying collections themselves, and then create the iterators during the code block above, something like:
ListIterator<Iterable<PointF>> i = mPoints.listIterator();
while (i.hasNext()) {
Iterator<PointF> j = i.next().iterator();
if (j.hasNext())
PointF tmp = j.next();
}
Then again the exact solution depends on the general architecture of your method. The main thing to bear in mind is don't store iterators long-term, because it's almost impossible to make this work reliably. Even if it does work right now, it creates a kind of invisible dependency between different parts of your code that will almost invariably be broken by someone implementing what should be a trivial change.

Categories