I need to execute/display a series of events from a Arraylist to a JTextArea, however, each Event gets execute with different time. Following is the code, it fails while at second Event in the loop:
Thread worker = new Thread(new Runnable()
{
public void run()
{
while (eventList.size() > 0)
for (Event ev : eventList)
if(ev.ready())
{
/*try
{
Thread.sleep(1000);
} catch (InterruptedException e1)
{
e1.printStackTrace();
}*/
jTextArea.append(ev.toString() + "\n");
eventList.remove(ev);
}
}
});
worker.start();
I guess you got a ConcurrentModificationException. Try using an iterator, something like this:
Iterator<Event> it = eventList.iterator();
while(it.hasNext())
{
Event ev = it.next();
if (ev.ready())
it.remove();
}
Edit
Why did it throw a ConcurrentModificationException?
If you loop over a collection, using an Iterator directly or by using for(E : list), and you modify the collection, by calling add, remove or similar, you will get this exception. This tries to indicate there is a problem in the code. The problem is, one piece of code wants to loop over all the objects, while another piece of code adds or removes objects. The first piece of code gets in to trouble, how can it loop over everything if the collection keeps changing? So 'they' decided, you are not allowed to change a collection, when you loop over it. (Unless you change it with the iterator you use to loop, as this code does. it.remove(), it is the looping iterator and thus does not fail.) Hope that makes sense.
I may repeat what Lshtar said but I will add something of my own. Actually I took it from the book "J2EE Interview Companion".
The java.util Collection classes are fail-fast, which means that if one thread changes a collection while another thread is traversing it through with an iterator the iterator.hasNext() or iterator.next() call will throw ConcurrentModificationException. Even the synchronized collection wrapper classes SynchronizedMap and SynchronizedList are only conditionally thread-safe, which means all individual operations are thread-safe but compound
operations where flow of control depends on the results of previous operations may be subject to threading issues.
Solution:
Use ConcurrentHashMap or CopyOnWriteArrayList (java.util.concurrent package). Their iterators provide better scalability.
Related
I have a method similar to the one below:
public void addSubjectsToCategory() {
final List<Subject> subjectsList = new ArrayList<>(getSubjectList());
for (final Iterator<Subject> subjectIterator =
subjectsList.iterator(); subjectIterator.hasNext();) {
addToCategory(subjectIterator.next().getId());
}
}
When this runs concurrently for the same user (another instance), sometimes it throws NoSuchElementException. As per my understanding, sometimes subjectIterator.next() get executed when there are no elements in the list. This occurs when being accessed only. Will method synchronization solve this issue?
The stack trace is:
java.util.NoSuchElementException: null
at java.util.ArrayList$Itr.next(Unknown Source)
at org.cmos.student.subject.category.CategoryManager.addSubjectsToCategory(CategoryManager.java:221)
This stack trace fails at the addToCategory(subjectIterator.next().getId()); line.
The basic rule of iterators is that underlying collection must not be modified while the iterator is being used.
If you have a single thread, there seems to be nothing wrong with this code as long as getSubjectsList() does not return null OR addToCategory() or getId() have some strange side-effects that would modify the subjectsList. Note, however, that you could rewrite the for-loop somewhat nicer (for(Subject subject: subjectsList) ...).
Judging by your code, my best guess is that you have another thread which is modifying subjectsList somewhere else. If this is the case, using a SynchronizedList will probably not solve your problem. As far as I know, synchronization only applies to List methods such as add(), remove() etc., and does not lock a collection during iteration.
In this case, adding synchronized to the method will not help either, because the other thread is doing its nasty stuff elsewhere. If these assumptions are true, your easiest and safest way is to make a separate synchronization object (i.e. Object lock = new Object()) and then put synchronized (lock) { ... } around this for loop as well as any other place in your program that modifies the collection. This will prevent the other thread from doing any modifications while this thread is iterating, and vice versa.
subjectIterator.hasNext();) {
--- Imagine a thread switch occurs here, at this point, between the call to hasNext() and next() methods.
addToCategory(subjectIterator.next().getId());
What could happen is the following, assuming you are at the last element in the list:
thread A calls hasNext(), the result is true;
thread switch occurs to thread B;
thread B calls hasNext(), the result is also true;
thread B calls next() and gets the next element from the list; now the list is empty because it was the last one;
thread switch occurs back to thread A;
thread A is already inside the body of the for loop, because this is where it was interrupted, it already called hasNext earlier, which
was true;
so thread A calls next(), which fails now with an exception, because there are no more elements in the list.
So what you have to do in such situations, is to make the operations hasNext and next behave in an atomic way, without thread switches occurring in between.
A simple synchronization on the list solves, indeed, the problem:
public void addSubjectsToCategory() {
final ArrayBlockingQueue<Subject> subjectsList = new ArrayBlockingQueue(getSubjectList());
synchronized (subjectsList) {
for (final Iterator<Subject> subjectIterator =
subjectsList.iterator(); subjectIterator.hasNext();) {
addToCategory(subjectIterator.next().getId());
}
}
}
Note, however, that there may be performance implications with this approach. No other thread will be able to read or write from/to the same list until the iteration is over (but this is what you want). To solve this, you may want to move the synchronization inside the loop, just around hasNext and next. Or you may want to use more sophisticated synchronization mechanisms, such as read-write locks.
It sounds like another thread is calling the method and grabbing the last element while another thread is about to get the next. So when the other thread finishes and comes back to the paused thread there is nothing left. I suggest using an ArrayBlockingQueue instead of a list. This will block threads when one is already iterating.
public void addSubjectsToCategory() {
final ArrayBlockingQueue<Subject> subjectsList = new ArrayBlockingQueue(getSubjectList());
for (final Iterator<Subject> subjectIterator =
subjectsList.iterator(); subjectIterator.hasNext();) {
addToCategory(subjectIterator.next().getId());
}
}
There is a bit of a wrinkle that you may have to sort out. The ArrayBlockingQueue will block if it is empty or full and wait for a thread to either insert something or take something out, respectively, before it will unblock and allow other threads to access.
You can use Collections.synchronizedList(list) if all you need is a simple invocation Sycnchronization. But do note that the iterator that you use must be inside the Synchronized block.
As I get you are adding elements to a list which might be under reading process.
Imagine the list is empty and your other thread is reading it. These kinds of problems might lead into your problem. You could never be sure that an element is written to your list which you are trying to read , in this approach.
I was surprised not to see an answer involving the use of a CopyOnWriteArrayList or Guava's ImmutableList so I thought that I would add such an answer here.
Firstly, if your use case is such that you only have a few additions relative to many reads, consider using the CopyOnWriteArrayList to solve the concurrent list traversal problem. Method synchronization could solve your issue, but CopyOnWriteArrayList will likely have better performance if the number of concurrent accesses "vastly" exceeds the number of writes, as per that class's Javadoc.
Secondly, if your use case is such that you can add everything to your list upfront in a single-threaded manner and only then do you need iterate across it concurrently, then consider Guava's ImmutableList class. You accomplish this by first using a standard ArrayList or a LinkedList or a builder for your ImmutableList. Once your single-threaded data entry is complete, then you instantiate your ImmutableList using either ImmutableList.copyOf() or ImmutableList.build(). If your use case will allow for this write/read pattern, this will probably be your most performant option.
Hope that helps.
I would like to make a suggestion that would probably solve your problem, considering that this is a concurrency issue.
If making the method addSubjectsToCategory() synchronized solves your problem, then you have located where your concurrency issue is. It is important to locate where the problem occurs, otherwise the information you provided is useless to us, we can't help you.
IF using synchronized in your method solves your problem, then consider this answer as educational or as a more elegant solution. Otherwise, share the code where you implement your threading environment, so we can have a look.
public synchronized void addSubjectsToCategory(List subjectsList){
Iterator iterator = subjectsList.iterator();
while(iterator.hasNext())
addToCategory(iterator.next().getId());
}
or
//This semaphore should be used by all threads. Be careful not to create a
//different semaphore each time.
public static Semaphore mutex = new Semaphore(1);
public void addSubjectsToCategory(List subjectsList){
Iterator<Subject> iterator = subjectsList.iterator();
mutex.acquire();
while(iterator.hasNext())
addToCategory(iterator.next().getId());
mutex.release();
}
Synchronized is clean, tidy and elegant. You have a really small method and creating locks, imho is unnecessary.
Synchronized means that only 1 thread will be able to enter the method at a time. Which means, you should use it only if you want 1 thread active each time.
If you actually need parallel execution, then your problem is not thread-related, but has something to do with the rest of your code, which we can not see.
I have N threads that add values and one removing thread. I am thinking of the best way how to sync adding to existing list of values and removing of the list.
I guess following case is possible:
thread 1 checked condition containsKey, and entered in else block
thread 2 removed the value
thread 1 try to add value to existing list, and get returns null
I think the only approach that I can use is syncing by map value, in our case is List when we adding and when we deleting
private ConcurrentSkipListMap<LocalDateTime, List<Task>> tasks = new ConcurrentSkipListMap<>();
//Thread1,3...N
public void add(LocalDateTime time, Task task) {
if (!tasks.containsKey(time)) {
tasks.computeIfAbsent(time, k -> createValue(task));
} else {
//potentially should be synced
tasks.get(time).add(task);
}
}
private List<Task> createValue(Task val) {
return new ArrayList<>(Arrays.asList(val));
}
//thread 2
public void remove()
while(true){
Map.Entry<LocalDateTime, List<Task>> keyVal = tasks.firstEntry();
if (isSomeCondition(keyVal)) {
tasks.remove(keyVal.getKey());
for (Task t : keyVal.getValue()) {
//do task processing
}
}
}
}
About the add part you would be really inclined to use merge, but the documentation is pretty clear about it - saying that it is not guaranteed to happen atomically.
I would replace your add with merge, but under a lock.
SomeLock lock ...
public void add(LocalDateTime time, Task task) {
lock.lock();
tasks.merge...
lock.unlock();
}
And same for the remove method. But then, if you are doing things under a lock there is no need for ConcurrentSkipListMap in the first place.
On the other hand if you are OK changing to ConcurrentHashMap - it has merge that is atomic for example.
It’s not entirely clear what your remove() method is supposed to do. In its current form, it’s an infinite loop, first, it will iterate over the head elements and remove them, until the condition is not met for the head element, then, it will repeatedly poll for that head element and re-evaluate the condition. Unless, it manages to remove all elements, in which case it will bail out with an exception.
If you want to process all elements currently in the map, you may simply loop over it, the weakly consistent iterators allow you to proceed while modifying it; you may notice ongoing concurrent updates or not.
If you want to process the matching head elements only, you have to insert a condition to either, return to the caller or put the thread into sleep (or better add a notification mechanism), to avoid burning the CPU with a repeated failing test (or even throw when the map is empty).
Besides that, you can implement the operations using ConcurrentSkipListMap when you ensure that there is no interference between the functions. Assuming remove is supposed to process all current elements once, the implementation may look like
public void add(LocalDateTime time, Task task) {
tasks.merge(time, Collections.singletonList(task),
(l1,l2) -> Stream.concat(l1.stream(),l2.stream()).collect(Collectors.toList()));
}
public void remove() {
for(Map.Entry<LocalDateTime, List<Task>> keyVal : tasks.entrySet()) {
final List<Task> values = keyVal.getValue();
if(isSomeCondition(keyVal) && tasks.remove(keyVal.getKey(), values)) {
for (Task t : values) {
//do task processing
}
}
}
}
The key point is that the lists contained in the map are never modified. The merge(time, Collections.singletonList(task), … operation will even store an immutable list of a single task if there was no previous mapping. In case there are previous tasks, the merge function (l1,l2) -> Stream.concat(l1.stream(),l2.stream()).collect(Collectors.toList()) will create a new list rather than modifying the existing ones. This may have a performance impact when the lists become much larger, especially when the operation has to be repeated in the case of contention, but that’s the price for not needing lock nor additional synchronization.
The remove operation uses the remove(key, value) method which only succeeds if the map’s value still matches the expected one. This relies on the fact that neither of our methods ever modifies the lists contained in the map, but replaces them with new list instances when merging. If remove(key, value) succeeds, the list can be processed; at this time, it is not contained in the map anymore. Note that during the evaluation of isSomeCondition(keyVal), the list is still contained in the map, therefore, isSomeCondition(keyVal) must not modify it, though, I assume that this should be the case for a testing method like isSomeCondition anyway. Of course, evaluating the list within isSomeCondition also relies on the other methods never modifying the list.
I have a ConcurrentSKipListSet, and I'm iterating over values in this set with a for-each loop.
Another thread at some point is going to remove an element from this set.
I think I'm running into a situation where one thread removes an element that I'm yet to iterate over (or maybe I've just started to iterate over it) and so a call being made from within the loop fails.
Some code for clarity:
for(Foo foo : fooSet) {
//do stuff
//At this point in time, another thread removes this element from the set
//do some more stuff
callService(foo.getId()); // Fails
}
Reading the docs I can't work out if this is possible or not:
Iterators are weakly consistent, returning elements reflecting the state of the set at some point at or since the creation of the iterator. They do not throw ConcurrentModificationException, and may proceed concurrently with other operations.
So is this possible, and if so, what's a good way of handling this?
Thanks
Will
I think I'm running into a situation where one thread removes an element that I'm yet to iterate over (or maybe I've just started to iterate over it) and so a call being made from within the loop fails.
I don't think that's what the javadocs are saying:
Iterators are weakly consistent, returning elements reflecting the state of the set at some point at or since the creation of the iterator. They do not throw ConcurrentModificationException, and may proceed concurrently with other operations.
This is saying that you don't have to worry about someone removing from the ConcurrentSkipListSet at the same time that you are iterating across the list. There certainly is going to be a race condition as you are moving across the iterator however. Either foo gets removed right after your iterator gets it or it was removed right before and the iterator doesn't see it.
callService(foo.getId()); // this shouldn't "fail"
If foo gets returned by the iterator, your service call won't "fail" unless it is assuming that the foo is still in the list and somehow checking it. The worst case is that you might do some operations on foo and call the service with it even though it was just removed from the list by the other thread.
I've hit this problem as well with queues that are written to and read by different threads. One approach is to mark instead of remove elements that are no longer needed. You can run a cleanup iterator after you go through the whole list. You need a global lock just for removing elements from the list, and the rest of the time your code can run in parallel. Schematically it works like this:
writer:
while() {
set.add(something);
something.markForDelete();
}
reader:
while() {
// process async
iterator iter = set.getIterator();
for(iter.hasNext()) {
... work, check isMarkedForDelete() ...
}
iter = set.getIterator();
// delete, sync
globalLock.Lock();
for(iter.hasNext()) {
if(something.isMarkedForDelete()) {
set.remove(something);
}
globalLock.Unlock();
}
}
I am getting the a ConcurrentModificationException from my cooldown timer. I use a thread to reduce the values every second like this:
public class CoolDownTimer implements Runnable {
#Override
public void run() {
for (String s : playerCooldowns.keySet()) {
playerCooldowns.put(s, playerCooldowns.get(s) - 20);
if (playerCooldowns.get(s) <= 0) {
playerCooldowns.remove(s);
}
}
}
}
So every second it should reduce every players cooldown by 20, but the problem is that I a getting the CME every couple hours while running the program, especially when lots of people are online. How do I make it so that if it is still modifying the list, it will wait until the current operation is done and create a sort of modification queue? Thanks! Here is the stack trace:
2012-06-18 20:59:05 [WARNING] Task of 'SurvivorsClasses' generated an exception
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:839)
at java.util.HashMap$KeyIterator.next(HashMap.java:874)
at me.zachoooo.survivorclasses.CoolDownManager$CoolDownTimer.run(CoolDownManager.java:13)
at org.bukkit.craftbukkit.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:126)
at net.minecraft.server.MinecraftServer.w(MinecraftServer.java:533)
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:459)
Line 13 is the start of the for loop...
You can't modify collections when using a foreach loop.
You can however iterate over the Map.entrySet() and do everything you need:
public void run() {
for (Iterator<Map.Entry<String,Integer>> i = playerCooldowns.entrySet().iterator(); i.hasNext();) {
Map.Entry<String,Integer> entry = i.next();
entry.setValue(entry.getValue() - 20); // update via the Map.Entry
if (entry.getValue() <= 0) {
i.remove(); // remove via the iterator
}
}
}
A ConcurrentModificationException is thrown while you try to modify the contents of your Collection, at the same time while Iterating through it.
Read this and this for more discussion on it.
The reason why sometimes it might work out for you is clearly mentioned in the documentation.
The iterators returned by all of this class's "collection view methods" are fail-fast: if
the map is structurally modified at any time after the iterator is created, in any way
except through the iterator's own remove method, the iterator will throw a
ConcurrentModificationException. Thus, in the face of concurrent modification, the
iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic
behavior at an undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally
speaking, impossible to make any hard guarantees in the presence of unsynchronized
concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a
best-effort basis.
Unlike Array, Collections are Checked only during the Compilation time, NOT during the run time, thats the reason you cannot modify the Collection like put() or remove() in the loop.
Documentation on synchronizedList states that,
It is imperative that the user manually synchronize on the returned list when iterating over it:
List list = Collections.synchronizedList(new ArrayList());
...
synchronized(list) {
Iterator i = list.iterator(); // Must be in synchronized block
while (i.hasNext())
foo(i.next());
}
Failure to follow this advice may result in non-deterministic behavior.
This seems pretty clear, but I just wanted to confirm that a for each loop is prohibited. For example, I cannot do something like as follows right?
List<MyType> list = Collections.synchronizedList(new ArrayList(<MyType>));
...
synchronized(list){
for(MyType m : list){
foo(m);
m.doSomething();
}
}
Yes, you can - your enhanced for loop is basically the same as your code which explicitly uses the iterator. It boils down to the same code - it's just calling iterator() and then alternating between next() and hasNext() calls.
You can do that. The foreach loop compiles to (nearly) the same bytecode as the while loop. The keys are:
You synchronize the block around the loop because the list may change while you are iterating over it.
You use the list as the object that you are synchronizing on, since the implementation of this class locks on itself (through synchronized methods).
If possible, you might want to consider using immutability rather than synchonization.
http://docs.guava-libraries.googlecode.com/git-history/release09/javadoc/com/google/common/collect/ImmutableList.html
Of course you can, the only problem I see here is a performance issue, if your method dosomething() or foo(m) are costly to execute, you will have a performance cost. The size of your collection is also important to take in account while looping in a synchronized block, due to the fact that, when a thread acquire the lock, while in the synchronized block, looping in a huge collection will push other threads to wait.