I'm working on some code and during testing it started throwing a ConcurrentModificationException. After some research, what confuses me is that I'm not adding or removing from the list while iterating. I do iterate over the list, but it's after I've added to the list. To add clarity, below is a commented version of the code:
File f = fc.getSelectedFile();
ConfigurationManager manager = new ConfigurationManager(f);
//The next line uses the same thread to get all Host objects from the file
ArrayList<Host> configHosts = manager.getAllHosts();
//Add the found hosts to a (previously established) ArrayList
hosts.addAll(configHosts);
for(Host h : hosts) { //Exception thrown points to this line.
//This reads values from the Host object and puts them in a JTable.
//No modification is done to the object, it is just read from.
this.updateTable(h);
}
I'm not sure how this code could possibly throw the ConcurrentModificationException, unless addAll() operates in a different thread (and then updateTable() would be called as hosts were still being added). But saving that, I'm not sure what I'm doing that's inappropriate.
I found the bug in my code. The commentators were right; updateTable() was (indirectly) responsible. There's a special case where certain hosts will be removed from the global field, and my test data just had to hit this special case. I'm not deleting the question because I believe there's value in my specific error. In short, if you get a ConcurrentModificationException, check the following:
Are you trying to remove/add an item from an ArrayList while iterating over the list?
Are you sure code that is reading the ArrayList side effect free (i.e. it won't modify the list under any case) (this was my problem)?
Are you working in a multithreaded environment where this ArrayList could be modified during your iteration (if so, you might want to synchronize the loop)?
Related
When creating large lists I ran into something odd. I created sub lists, because the entire list was too large. But when checking the resulting sizes I found:
new ArrayList<>(rSet).size() != rSet.size();
Where rSet is a HashSet
When I stop eclipse and investigate, I see that rSet has 1000 items, whilst responding to .size() as having less (the number of less items fluctuates; sometimes the rSet.size() is higher than the values it actually contains). I cannot reproduce this in a separate test case; the code has too many layer to provide. But is filled from separate threads, which are ended by the time size is called.
I said I filled it from threads. I provide the Set<> rSet as parameter to all threads, and use the following method to add new items to the set:
public static void addSynchronized(final Set<?> c, final List<?> items) {
c.addAll(items);
}
I must be doing something the code disagrees with... But what?
is filled from separate threads
I think there's your problem. HashSet is not thread-safe. When writing to it from multiple threads at the same time, anything could happen.
To make it synchronized (from the docs):
Set s = Collections.synchronizedSet(new HashSet(...));
Your addSynchronized method has a misleading name, because it's not synchronized. (Having an argument named list that's actually a Set is a bit confusing as well.)
I am essentially doing the following:
Creating an object (for example a weapon object), that automatically adds that object to a list off all of those types of objects (ArrayList<Weapons>).
JPanel paints every 10 seconds with an updater thread that iterates through the ArrayList<Weapons>. I am also sending 'questions' to a server on another machine, i.e. asking if that weapon is allowed. If it is not, the weapon object is modified by the client computer. However, whenever I modify it, I receive a ConcurrentModificationException. Instead of crashing, which I actually hope it would do at this point, since the method that changes the weapon object is on a different thread, the whole program just locks up.
I have more than 1000 lines of code in this program, and more than three threads running that access the list, so if you need any code please ask but I'd rather not post right now because in my mind this seems like a trivial question for an expert at threads.
Thanks!
(Object is made >> added to list of objects >> JPanel's "Updater" thread is constantly painting all objects every 10 ticks...
Server says that object isn't allowed >> A thread on the client computer removes that object (or toggles a boolean that says it is not visible) >> ConcurrentModificationException).
Quoting the Javadoc of ArrayList
Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally.
You describe multiple threads accessing the list, and at least one of them modifying it. So, all accesses to the list must be done in mutually synchronized blocks.
e.g. to iterate the list:
synchronized (list) {
for (Weapons weapons : list) {
// ...
}
}
e.g. to remove an item from the list:
synchronized (list) {
list.remove(0);
}
I think you can use a synchronised version of collection or list, i.e. java.util.Collections.synchronizedCollection(Collection<>), java.util.Collections.synchronizedList(List<>). Also, if you iterate over the list and remove items, ensure that use an implementation that enable remove item from the iterator. A good candidate is java.util.ArrayList.
Other technique is use "monitors", you can declare an attribute like: private static Object monitor = new Object(); then when the code tries to access to the list, protect the code inside a synchronized(monitor) block. Using this technique you ensure that no other thread will be able to modify your list until no protected code is running.
Please, excuse my English :).
I have a ConcurrentSKipListSet, and I'm iterating over values in this set with a for-each loop.
Another thread at some point is going to remove an element from this set.
I think I'm running into a situation where one thread removes an element that I'm yet to iterate over (or maybe I've just started to iterate over it) and so a call being made from within the loop fails.
Some code for clarity:
for(Foo foo : fooSet) {
//do stuff
//At this point in time, another thread removes this element from the set
//do some more stuff
callService(foo.getId()); // Fails
}
Reading the docs I can't work out if this is possible or not:
Iterators are weakly consistent, returning elements reflecting the state of the set at some point at or since the creation of the iterator. They do not throw ConcurrentModificationException, and may proceed concurrently with other operations.
So is this possible, and if so, what's a good way of handling this?
Thanks
Will
I think I'm running into a situation where one thread removes an element that I'm yet to iterate over (or maybe I've just started to iterate over it) and so a call being made from within the loop fails.
I don't think that's what the javadocs are saying:
Iterators are weakly consistent, returning elements reflecting the state of the set at some point at or since the creation of the iterator. They do not throw ConcurrentModificationException, and may proceed concurrently with other operations.
This is saying that you don't have to worry about someone removing from the ConcurrentSkipListSet at the same time that you are iterating across the list. There certainly is going to be a race condition as you are moving across the iterator however. Either foo gets removed right after your iterator gets it or it was removed right before and the iterator doesn't see it.
callService(foo.getId()); // this shouldn't "fail"
If foo gets returned by the iterator, your service call won't "fail" unless it is assuming that the foo is still in the list and somehow checking it. The worst case is that you might do some operations on foo and call the service with it even though it was just removed from the list by the other thread.
I've hit this problem as well with queues that are written to and read by different threads. One approach is to mark instead of remove elements that are no longer needed. You can run a cleanup iterator after you go through the whole list. You need a global lock just for removing elements from the list, and the rest of the time your code can run in parallel. Schematically it works like this:
writer:
while() {
set.add(something);
something.markForDelete();
}
reader:
while() {
// process async
iterator iter = set.getIterator();
for(iter.hasNext()) {
... work, check isMarkedForDelete() ...
}
iter = set.getIterator();
// delete, sync
globalLock.Lock();
for(iter.hasNext()) {
if(something.isMarkedForDelete()) {
set.remove(something);
}
globalLock.Unlock();
}
}
I am using a shared library in Java that returns ArrayList; as I iterate over it, a ConcurrentModificationException could be thrown and I am looking for 100% (?) guarantee to be safe. I was thinking on something like below and I'd appreciate any input.
The data_list is the ArrayList<> returned from the MT library.
boolean pass = true;
ArrayList<Something> local = new ArrayList<Something>(256);
for (int spin=0; spin<10; ++spin)
{
try {
local.addAll(data_list);
}
catch (java.util.ConcurrentModificationException ce) {
pass = false;
}
finally {
if (pass) break;
pass = true;
}
}
Assuming variable pass is true, how should I operate on local?
There is no safe way to do this. You should not catch ConcurrentModificationException.
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.
Some collections, like HashMap, even can enter an infinite loop when used this way. Here's an explanation of how it happens.
You should not do this. There is no correct way to do this.
Either you misunderstand how the library works, or you need to switch out your library with one written by a competent developer.
What library are you using?
You don't define exactly what you mean by safe, and don't specify what kind of modifications are being performed to the list, but in many cases it may be acceptable to iterate over it manually by index, i.e.
for (int index = 0; index < data_list.size(); index ++)
local.add(data_list.get(index));
The way I see it, there are four possible kinds of modification, with varying degrees of acceptability:
New items could be appended. This solution should work appropriately for this case, as long as the list does not grow enough to trigger a backing list expansion (and as this should happen with exponentially-reducing frequency, retrying if it occurs should be guaranteed to succeed eventually).
Existing items may be modified. This solution may not present a consistent view of the contents of the list at any given time, but it would be guaranteed to provide a usable list that is representative of items that have been in the list, which may be acceptable depending on your definition of "safe".
Items may be removed. There is a small chance this solution would fail with an IndexOutOfBoundsException, and the same caveat as for items being modified would apply with regards to consistency.
Items may be inserted into the middle of the list. The same caveat as items being modified would apply, and there would also be a danger of getting duplicated values. The problems with backing array expansion from the appending case would also apply.
You've got a bad situation here, but I think your solution is as sound as possible. The new ArrayList should go in the loop so you start fresh after each failure. Actually, the best thing might be to make your "try" line look like:
local = new ArrayList<Something>( data_list );
You don't want your ArrayList to have to expand itself because that will take time when you're trying to grab the data before the list changes. This should set the size, create it, and fill it with the least wasted effort.
You might need to catch things other than ConcurrentModification. You'll probably learn what the hard way. Or just catch Throwable.
If you want to go to extremes, run the code inside the for loop in it's own thread so if it does hang you can kill it and restart it. That's going to take some work.
I think this will work, if you let "spin" get large enough.
I don't have any fundamental changes, but I think that code could be simplified a bit:
ArrayList<Something> local = new ArrayList<Something>(256);
for (int spin=0; spin<10; ++spin)
{
try {
local.addAll(data_list);
break;
}
catch (java.util.ConcurrentModificationException ce) {}
}
I am new to Java and especially to iterating through lists and modifying it's elements. I went through dozens of similar questions, but after many tries nothing worked.
In the following code sample an exceptions is thrown. It is not related to concurrent thread as I only have one thread. Netbeans output sais that the exception occurs in line 5(CustomerData customer = i.next();) :
CustomerData tempCustomer = new CustomerData("",tagID,0);
tempCustomer.setName(name);
tempCustomer.setSize(size);
for(ListIterator<CustomerData> i = customers.listIterator(); i.hasNext(); )
{
CustomerData customer = i.next();
if(customer.getTagID().contains(tagID))
{
Object[] options = {"overwrite", "do not overwrite"};
int n = JOptionPane.showOptionDialog(rootPane,
"TagID already exists. overwrite?",
"?",
JOptionPane.YES_NO_OPTION,
JOptionPane.QUESTION_MESSAGE,
null,
options,
rootPane);
if ( n == JOptionPane.YES_OPTION ){
i.set(tempCustomer);
}
}
else{
addCustomer();
}
}
The whole code is only supposed to check if the element has matching tagID and if it does, replace two values (name and size) in it. Initially I tried to use setName() and setSize() inside the for loop on the element, but after it didn't work and after reading other related questions, I assigned the values to temporary object before the loop and the used the iterator's set method to replace current element. But still no success and it even seems it never gets that far because the exception comes after line 5...
The problem is presumably that this line:
addCustomer();
is trying to modify customers while you're in the middle of iterating over it. This isn't allowed.
And anyway, it seems like a logic error: presumably you want to call addCustomer only once, after the loop, if no customer had the right tagID, but your current code tries to call addCustomer for each customer with with the wrong tagID. So you need to use a boolean variable during the loop to keep track of whether any customer matched, and then, after the loop, call addCustomer if appropriate.
ConcurentModificationException does not always indicate that an object has been concurrently modified by a different thread.
In your case the problem might be in following (guessing by addCustomer method): you are getting iterator of collection and then modifiying this collection. Because collection is modified, iterator becomes invalid and exception is thrown.
My suggestion will be create temproray list of customers which will you will append to your original customer list, when your iteration finished.
As mentioned in javadocs:
if a thread modifies a collection directly while it is iterating over
the collection with a fail-fast iterator, the iterator will throw this
exception.
I guess it is occurring after a call to addCustomer() which modifies the customers object. You can either hold the newly added objects in a different collection until end of the loop or break out of the loop and restart after adding.