We Keep Coding

Java ConcurrentModificationException for writing to array list from multiple thread

In java I have the following Item class
public class Item {
private final List<Integer> list = new ArrayList<>();
public synchronized List<Integer> getList() {
return list;
}
}
and I'm accessing list element of this class from multiple threads which is throwing concurrent exception error.
public class Main {
public static void main(String[] args) {
Item instance1 = new Item();
Collections.addAll(instance1.getList(),-1,-2,-3,-4,-5,-6,-7,-8,-9,-10,-11,-12,-13,-14,-15);
new Thread(new Runnable() {
#Override
public void run() {
for(int i = 1;i<=25;i++){
instance1.getList().add(i);
}
for(int i : instance1.getList()){
System.out.println(i+ " ");
}
thirdPartyLib(instance1.getList());//Third party library which is internally iterating over list
}
}).start();
new Thread(new Runnable() {
#Override
public void run() {
for(int i = 26;i<=50;i++){
instance1.getList().add(i);
}
for(int i : instance1.getList()){
System.out.println("from2 "+i);
}
thirdPartyLib(instance1.getList());//Third party library which is internally iterating over list
}
}).start();
}
}
It is perfectly working with CopyOnWriteArrayList but in real scenario list is being edited many times from multiple places therefore CopyOnWriteArrayList is not the good choice for me. Also I can't use Iterator because this error is thrown by third party library which is working with list. is there any way by which I can solve this error?

There exists Collections.synchronizedList, however you would still need to synchronize on the list when using an Iterator, so the third party lib is problematic.
new Thread(() -> {
List<Integer> list = instance1.getList();
for(int i = 1;i<=25;i++){
list.add(i);
}
synchronized (list) {
for(int i : list){
System.out.println(i+ " ");
}
}
synchronized (list) {
thirdPartyLib(list);//Third party library which is internally iterating over list
}
}).start();
new Thread(() -> {
List<Integer> list = instance1.getList();
for(int i = 26;i<=50;i++){
instance1.list.add(i);
}
synchronized (list) {
for(int i : list){
System.out.println("from2 "+i);
}
}
synchronized (list) {
thirdPartyLib(list);//Third party library which is internally iterating over list
}
}).start();
}
public class Item {
private final List<Integer> list = Collections.synchronizedList(new ArrayList<>());
public List<Integer> getList() {
return list;
}
}
This is quite unsatisfactory.
You could combine that CopyOnWriteArrayList and provide independent IntStreams for iterations, best with no List access, just an add method in Item.

Lock objects for all threads?

I have this small sample of code, while modifying the list, i lock it with synchronized, but while reading the list it comes to ConcurrentModificationException, because without "synchronized" the lock has no effect. Is it possible to lock an object for all threads which use the object, even the un-synchronized, too?
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
public class Test {
public static void main(String[] args) {
final Random r = new Random(System.currentTimeMillis());
List<Integer> list = new ArrayList<>();
new Thread(new Runnable() {
public void run() {
while (true) {
synchronized (list) {
list.add(r.nextInt());
}
}
}
}).start();
new Thread(new Runnable() {
public void run() {
while (true) {
for (Integer i : list) {
System.out.println(i);
}
}
}
}).start();
}
}
the backgorund is that i dont want to change all pieces in my code which read the object

You might consider using a concurrent implementation of List, instead of ArrayList. Perhaps a CopyOnWriteArrayList.
final List<Integer> list = new CopyOnWriteArrayList<Integer>();

Is it possible to lock an object for all threads which use the object.
In a word, No. When one thread enters a synchronized(foo) {...} block, that does not prevent other threads from accessing or modifying foo. The only thing it prevents is, it prevents other threads from synchronizing on the same object at the same time.
What you can do, is you can create your own class that encapsulates both the lock and the data that the lock protects.
class MyLockedList {
private final Object lock = new Object();
private final List<Integer> theList = new ArrayList<>();
void add(int i) {
synchronized(lock) {
theList.add(i);
}
}
void printAll() {
synchronized(lock) {
for (Integer i : theList) {
System.out.println(... i ...);
}
}
}
...
}

If you can modify the function which concurrently uses the object, just add synchronized in every critical section:
while (true) {
synchronized(list){
for (Integer i : list) {
System.out.println(i);
}
}
}
if you can't , create a specified lock that is responsible for locking the threads:
Lock lock = new Lock();
new Thread(new Runnable(){
//...
synchronized(lock){
do unsynchonized function on list
}
//...
}).start();
new Thread(new Runnable(){
//...
synchronized(lock){
do unsynchonized function on list
}
//...
}).start();
the latter may slow down the process if one of the functions already doing some locking, but in this way you can ensure you always synchronize the access to concurrent objects.

Java- Why this program not throwing concurrent Modification exception

I am trying to induce a concurrent modification exception by accessing a HashMap instance variable reference, but this program is not throwing the error. Appreciate if you could help me to understand.
package Threads;
import java.util.HashMap;
public class ProducerConsumer {
private HashMap<String, String> sharedMap = new HashMap<String,String>();
public void putMethod(){
for (int count=0; count<100; count++)
{
System.out.println("Adding into sharedMap:"+count);
sharedMap.put(Integer.toString(count),Integer.toString(count));
}
}
public static void main(String [] args) throws InterruptedException{
final ProducerConsumer pc1=new ProducerConsumer();
Thread t1= new Thread( new Runnable(){
#Override
public void run() {
pc1.putMethod();
}
});
Thread t2= new Thread( new Runnable(){
#Override
public void run() {
pc1.putMethod();
}
});
t1.start();
t2.start();
t1.join();
t2.join();
}
}

The add() method does not detect concurrent modifications and therefore will not throw a ConcurrentModificationException (that's what anonymous already said).
However, concurrent access to a HashMap can be dangerous, though. Read more about this in another post.
You can enforce a ConcurrentModificationException if you read from the HashMap in parallel:
...
public void putMethod() { ... }
public void iterateMethod() {
sharedMap.keySet().stream().forEach((s) -> {
System.out.println("Read key " + s);
}
}
public static void main(String[] args) throws InterruptedException {
...
t1.start();
Thread.sleep(20); // sleep time depends on your computer's speed ;-)
t2.start();
...
}
...

The exception needs to be thrown by the implementing method that is being invoked on the class. From the Javadoc, it looks like the HashMap Iterators are fast-fail Iterators; meaning it will throw it if you are Iteratoring while adding. The add method will add the item to the map if the key doesn't exist or replace it if it does, I don't think that would throw the exception you're trying to produce.

Android: How to correctly synchronize iteration over Set when another thread can remove elements from the Set?

Why does a call to removeListener() in the following code throw a ConcurrentModificationException when another thread is using the iterator in fireEvent()?
public class MyClass {
private Set<Object> synchronizedListeners;
public MyClass() {
synchronizedListeners = Collections.synchronizedSet(
new LinkedHashSet<Object>());
}
public void addListener(Object listener) {
synchronizedListeners.add(listener);
}
public synchronized void removeListener(Object listener) {
synchronizedListeners.remove(listener);
}
public void fireEvent() {
synchronized (synchronizedListeners) {
for (Object listener : synchronizedListeners) {
// do something with listener
}
}
}
}
From my understanding, since I'm using synchronized (synchronizedListeners) in fireEvent(), this should block any other thread that calls removeListener(), till the iteration in fireEvent() is complete at which point it should be safe to remove an element from this Set. But this doesn't seem to be the case. What am I doing wrong?
Possibly related: Java synchronized block vs. Collections.synchronizedMap
Edit: It was pointed out that I was unnecessarily synchronizing the removeListener() method. So I tried this version:
public void removeListener(Object listener) {
synchronizedListeners.remove(listener);
}
But still got the same error.
Edit 2: As assylias pointed out, the problem isn't visible in the above code. I was calling removeListener() from inside the for loop in the synchronized (synchronizedListeners) block which was causing the error. The fix I ended up using in this case is to remove the listener from another thread:
public void removeListener(final Object listener) {
new Thread() {
#Override
public void run() {
synchronizedListeners.remove(listener);
}
}.start();
}

You are synchronizing on two different objects.
The removeListener method is synchronized on the MyClass instance, whereas the loop inside the fireEvent is synchronized on synchronizedListeners set.
What you need to do is to synchronize every method that uses the synchronizedListeners on the set itself.

I can't reproduce what you describe - the code at the bottom gives the output shown below - which shows that remove is called in the middle of the iteration, but does not complete until after the iteration because you use a synchronized collection. That's the behaviour one would expect and no ConcurrentModificationException is thrown. Note that I have removed the synchronized keyword from the removeListener method as it is useless here.
fire 100000
remove 100000
done fire 100000
done remove 99999
Conclusion: the problem is somewhere else. For example, if you have a subclass that overrides the fireEvent method, or if you construct the synchronized set not exactly as in the code you posted.
public static void main(String[] args) {
final MyClass mc = new MyClass();
final Object o = new Object();
mc.addListener(o);
for (int i = 0; i < 99999; i++) {
Object o1 = new Object();
mc.addListener(o1);
}
Runnable remove = new Runnable() {
#Override
public void run() {
mc.removeListener(o);
}
};
new Thread(remove).start();
mc.fireEvent();
}
public static class MyClass {
protected Set<Object> synchronizedListeners = Collections.synchronizedSet(new LinkedHashSet<Object>());
public void addListener(Object listener) {
synchronizedListeners.add(listener);
}
public void removeListener(Object listener) {
System.out.println("remove " + synchronizedListeners.size());
synchronizedListeners.remove(listener);
System.out.println("done remove " + synchronizedListeners.size());
}
public void fireEvent() {
System.out.println("fire " + synchronizedListeners.size());
synchronized (synchronizedListeners) {
for (Object listener : synchronizedListeners) {
// do something with listener
}
}
System.out.println("done fire " + synchronizedListeners.size());
}
}

Two-way communication with a Java thread

In my application I'm performing somewhat heavy lookup operations. These operations must be done within a single thread (persistence framework limitation).
I want to cache the results. Thus, I have a class UMRCache, with an inner class Worker:
public class UMRCache {
private Worker worker;
private List<String> requests = Collections.synchronizedList<new ArrayList<String>>());
private Map<String, Object> cache = Collections.synchronizedMap(new HashMap<String, Object>());
public UMRCache(Repository repository) {
this.worker = new Worker(repository);
this.worker.start();
}
public Object get(String key) {
if (this.cache.containsKey(key)) {
// If the element is already cached, get value from cache
return this.cache.get(key);
}
synchronized (this.requests) {
// Add request to queue
this.requests.add(key);
// Notify the Worker thread that there's work to do
this.requests.notifyAll();
}
synchronized (this.cache) {
// Wait until Worker has updated the cache
this.cache.wait();
// Now, cache should contain a value for key
return this.cache.get(key);
}
}
private class Worker extends Thread {
public void run() {
boolean doRun = true;
while (doRun) {
synchronized (requests) {
while (requests.isEmpty() && doRun) {
requests.wait(); // Wait until there's work to do
}
synchronized (cache) {
Set<String> processed = new HashSet<String>();
for (String key : requests) {
// Do the lookup
Object result = respository.lookup(key);
// Save to cache
cache.put(key, result);
processed.add(key);
}
// Remove processed requests from queue
requests.removeAll(processed);
// Notify all threads waiting for their requests to be served
cache.notifyAll();
}
}
}
}
}
}
I have a testcase for this:
public class UMRCacheTest extends TestCase {
private UMRCache umrCache;
public void setUp() throws Exception {
super.setUp();
umrCache = new UMRCache(repository);
}
public void testGet() throws Exception {
for (int i = 0; i < 10000; i++) {
final List fetched = Collections.synchronizedList(new ArrayList());
final String[] keys = new String[]{"key1", "key2"};
final String[] expected = new String[]{"result1", "result2"}
final Random random = new Random();
Runnable run1 = new Runnable() {
public void run() {
for (int i = 0; i < keys.length; i++) {
final String key = keys[i];
final Object result = umrCache.get(key);
assertEquals(key, results[i]);
fetched.add(um);
try {
Thread.sleep(random.nextInt(3));
} catch (InterruptedException ignore) {
}
}
}
};
Runnable run2 = new Runnable() {
public void run() {
for (int i = keys.length - 1; i >= 0; i--) {
final String key = keys[i];
final String result = umrCache.get(key);
assertEquals(key, results[i]);
fetched.add(um);
try {
Thread.sleep(random.nextInt(3));
} catch (InterruptedException ignore) {
}
}
}
};
final Thread thread1 = new Thread(run1);
thread1.start();
final Thread thread2 = new Thread(run2);
thread2.start();
final Thread thread3 = new Thread(run1);
thread3.start();
thread1.join();
thread2.join();
thread3.join();
umrCache.dispose();
assertEquals(6, fetched.size());
}
}
}
The test fails randomly, at about 1 out of 10 runs. It will fail at the last assertion: assertEquals(6, fetched.size()), at assertEquals(key, results[i]), or sometimes the test runner will never finish.
So there's something buggy about my thread logic. Any tips?
EDIT:
I might have cracked it now, thanks to all who have helped.
The solution seems to be:
public Object get(String key) {
if (this.cache.containsKey(key)) {
// If the element is already cached, get value from cache
return this.cache.get(key);
}
synchronized (this.requests) {
// Add request to queue
this.requests.add(key);
// Notify the Worker thread that there's work to do
this.requests.notifyAll();
}
synchronized (this.cache) {
// Wait until Worker has updated the cache
while (!this.cache.containsKey(key)) {
this.cache.wait();
}
// Now, cache should contain a value for key
return this.cache.get(key);
}
}

get() method logic can miss result and get stuck
synchronized (this.requests) {
// Add request to queue
this.requests.add(key);
// Notify the Worker thread that there's work to do
this.requests.notifyAll();
}
// ----- MOMENT1. If at this moment Worker puts result into cache it
// will be missed since notification will be lost
synchronized (this.cache) {
// Wait until Worker has updated the cache
this.cache.wait();
// ----- MOMENT2. May be too late, since cache notifiation happened before at MOMENT1
// Now, cache should contain a value for key
return this.cache.get(key);
}

The variable fetched in your test is an ArrayList and is accessed and updated from your two anonymous Runnable instances.
ArrayList is not thread safe, from the documentation:
Note that this implementation is not
synchronized. If multiple threads
access an ArrayList instance
concurrently, and at least one of the
threads modifies the list
structurally, it must be synchronized
externally. (A structural modification
is any operation that adds or deletes
one or more elements, or explicitly
resizes the backing array; merely
setting the value of an element is not
a structural modification.) This is
typically accomplished by
synchronizing on some object that
naturally encapsulates the list. If no
such object exists, the list should be
"wrapped" using the
Collections.synchronizedList method.
This is best done at creation time, to
prevent accidental unsynchronized
access to the list:
Hence I think your test needs a little adjusting.

I noticed your lookup in cache isn't atomic operation:
if (this.cache.containsKey(key)) {
// If the element is already cached, get value from cache
return this.cache.get(key);
}
Since you never delete from cache in your code, you always will get some value by this code. But if, in future, you plan to clean cache, lack of atomicity here will become a problem.