I'm playing with two threads operating on an ArrayList. I know you can't modify the list while iterating, but Java is letting me reassign (replace) the entire list in a loop with no issue. Is this threadsafe?
public class Main {
static List<String> list = Arrays.asList("hi", "there", "friend");
public static void main(String... args) throws InterruptedException {
boolean isReassigned = false;
for (String s: list) {
System.out.println(s);
if (!isReassigned) {
new Thread(() -> reassignList()).start();
isReassigned = true;
}
Thread.sleep(1000);
}
}
public static void reassignList() {
System.out.println("Reassigning list....");
list = Arrays.asList("goodbye", "old", "list");
}
}
I expected concurrency issues after "hi", but I got "hi there friend".
I assume this works because Java does copy-by-value on references, but I'm worried my production app that reassigns the array every 1 minute might blow up someday. Thanks everyone!
You are not seeing the updated list contents because the enhanced for loop uses an iterator behind the scenes. And hence, you are using the iterator (to the old list) to iterate over the data (even after the assignment is done). This means that both the lists are in-scope (and not Garbage collected). Once, isReassigned is true, if you get an iterator to the list you can see the new contents.
I've changed your code a bit (added an else block) to understand this
for (String s: list) {
System.out.println(s);
if (!isReassigned) {
new Thread(() -> reassignList()).start();
isReassigned = true;
} else {
System.out.println("Latest data " + list);
}
Thread.sleep(1000);
}
This produces
hi
Reassigning list....
there
Latest data [goodbye, old, list]
friend
Latest data [goodbye, old, list]
A few other points:
You should not be setting isReassigned from the main thread. It has to be set after you assign the list.
As #Kartik points out in the comments, there can be visibility issues and the other threads may not see the made changes. Marking the list and the isReassigned volatile will solve this.
Related
I have a below method which is called by multiple threads concurrently to get the live socket. It takes LinkedBlockingQueue as the parameter and then I iterate and see if there is any liveSocket available and if it is available then I remove and return that socket.
private Optional<Holder> getSocket(final LinkedBlockingQueue<Holder> endPoints) {
Optional<Holder> liveSocket = Optional.absent();
if (!endPoints.isEmpty()) {
for (Holder state : endPoints) {
// check if socket is live? if yes then remove and return that.
if (state.isLive()) {
liveSocket = Optional.of(state);
endPoints.remove(state);
return liveSocket;
}
}
}
return Optional.absent();
}
Wanted to check if my above code is thread safe or not? Here Holder is an immutable class.
The queue manipulation operations are thread safe, so the remove() will not throw ConcurrentModificationException. However, you have thread-safety problems around the state of the objects contained in the queue.
There's a race condition between when you check the "live" state of the Holder object and when you remove it from the queue. Another thread could be running in the same code at the same time, with the likely result that both threads would take the same object. Whichever thread got to the remove() call last would get a false return, but you don't examine the result so you'd never know. Both threads would then attempt to use the same object.
You need to synchronize around the search/remove operation.
For curiosity, here's the code I used to show that ConcurrentModificationException does not occur with LinkedBlockingQueue:
public static void main(String[] args) throws Exception
{
String[] data = { "a", "b", "c", "d", "e", "f","g" };
LinkedBlockingQueue<String> lb = new LinkedBlockingQueue<>(Arrays.asList(data));
new Thread(() ->
{
try
{
Thread.sleep(2000);
lb.add("x");
System.out.println("added");
Thread.sleep(1000);
lb.remove("e");
System.out.println("removed");
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}).start();
for (String s : lb)
{
System.out.println(s);
Thread.sleep(1000);
}
}
If you substitute LinkedList for LinkedBlockingQueue you get the ConcurrentModificationException as expected.
Output:
a
b
added
c
removed
d
f
g
x
It's not only not thread-safe, it's wrong even within a single thread. You will get a ConcurrentModificationException on the remove(). You need to use an explicit Iterator and to do the removal via the Iterator.
And for correctness via multiple threads you need synchronization or a semaphore around the loop.
NB The isEmpty() test is pointless. The iteration already has to check for that. Don't keep a dog and bark yourself.
I have a list
testList= new ArrayList<String>(); // gets initialized once at the very beginning init(). This list never changes. It's static and final.
I have a static method that checks if an input value is in this List :
public static boolean isInList (String var1)
throws InterruptedException {
boolean in = false;
for (String s : testList) {
if (s.equals(var1))
{
in = true;
break;
}
}
return in;
}
I have a lot of threads that use this method concurrently and check if a certain value is in this list. It seems to work fine. However, I'm not sure if this is safe. Am I doing this correctly? Is it thread safe?
It is thread-safe as long as no thread is modifying the list while other threads are reading it.
If you are using iterators over the list, they will "fail-fast" (as in throw ConcurrentModificationException) if the list is modified under them. Other methods of accessing (i.e. get(n)) won't throw an exception but may return unexpected results.
This is all covered in great detail in the Javadoc for List and ArrayList, which you should study carefully.
ArrayList is not a thread safe object. It may works for you now, but in general, when working with threads, you should make sure you're using thread-safe objects that will work with your threads as you expect.
You can use Collections.synchronizedList()
testList = Collections.synchronizedList(new ArrayList<String>());
As long as you can guarantee that no one is writing to the list, it's safe.
Note that even if the list is static and final, the code itself doesn't guarantee that the list is never modified. I recommend using Collections.unmodifiableList() instead, because it guarantees that no element is ever added to or removed from the list.
By the way, you can rewrite your code to this:
public static boolean isInList(String var1) {
for (String s : testList) {
if (Objects.equals(s, var1)) {
return true;
}
}
return false;
}
or just
testList.contains(var1);
Here is the sample code
1) ArrayList is a single object which is passed to every thread of ThreadPool.
2) At end of execution list size should be 50, if you check the sample outputs its may not 50. Sometime it may be 41 or 47 like that, why it is behaving like that.
public class Test {
ArrayList list=new ArrayList();
public static void main(String[] args) {
ExecutorService executorService3 = Executors.newScheduledThreadPool(10);
Test test=new Test();
for(int i=0;i<5;i++)
{
Mythread t1=new Mythread(test.list);
executorService3.execute(t1);
}
executorService3.shutdown();
while(executorService3.isShutdown())
{
//---This is not giveging proper output as expected is 50.--
System.out.println("List size="+test.list.size());
break;
}
}
}
class Mythread implements Runnable {
List list=null;
Mythread(List list) {
this.list=list;
}
#Override
public void run() {
for(int i=0;i<10;i++) {
this.list.add(i);
}
}
}
Your code isn't waiting for the threads to finish execution. By the time your code calls the following line
System.out.println("List size="+test.list.size());
there's no guarantee that they have finished, and so no guarantee that the list contains the expected 50 items. Use the awaitTermination method (https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#awaitTermination(long,%20java.util.concurrent.TimeUnit)), e.g.:
executorService3.shutdown();
executorService3.awaitTermination(1, TimeUnit.SECONDS);
System.out.println("List size="+test.list.size());
(Exception handling omitted for brevity)
As it says in the Javadoc for ArrayList:
Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally.
So it's "not behaving properly" because you're not using it as described in the documentation.
As is suggested in the Javadoc, you can wrap your list in a synchronized list:
List<Integer> list = Collections.synchronizedList(new ArrayList<>());
It's a concurrency problem.
The way i see it, you have :
5 threads that will execute the run method on the same object. Multiple threads can insert a variable, on the same position in the array list, since it is not synchronized.
Could you print the content of your list ?
This question already has answers here:
Why am I not getting a java.util.ConcurrentModificationException in this example?
(10 answers)
Closed 7 years ago.
I have the following code in my class:
private static LinkedList<MyObject> myList =
new LinkedList<MyObject>();
public static void doEventStuff(String user, String event){
LinkedList<MyObject> copy;
synchronized (myList) {
copy = new LinkedList<>(myList);
}
for (MyObject o : copy) {
... do something with objects o
}
}
public static void removeObject(MyObject o) {
synchronized (myList) {
myList.remove(o);
}
o.doCleanup();
}
public static void terminate() {
synchronized (myList) {
for (MyObject o : myList) {
o.doCleanup();
}
myList.clear();
}
}
public static List<MyObject> getMyObjectsCopy() {
synchronized (myList) {
return new LinkedList<>(myList);
}
}
My problem is a ConcurrentModificationException when calling terminate() , specifically when iterating "for (MyObject o : myList) ".
The list myList is not passed around and can only be accessed through the static methods.
Also: the method MyObject.doCleanup() ca trigger events where the method "removeObject(MyObject)" can be called, when doing the iteration inside terminate() mthod , but since all the methods synchronize
on "myList", I didn't believe a concurrency exception can happen.
Can anyone help me with this issue?
This is not multi-threading issue per se, if you remove an object from the list in a foreach loop you will get ConcurrentModificationException.
And by the way, you can use CopyOnWriteArrayList instead
ConcurrentModificationException also happens if the list was modified while iterating over it using a 'foreach' loop. synchronize will help avoid other threads from accessing your list, but your issue is not due to thread-concurrency. If you want to delete (from the same thread) while iterating over the list, you must use an iterator and call iterator.remove().
In this code:
for (MyObject o : myList) {
o.doCleanup(o);
}
You call code, which internally calls removeObject() method. In this call we make myList.remove(o), which will change a list, as a result, it works like:
for (MyObject o : myList) {
myList.remove();
}
So, it's not a concurrency issue, it's just modification of collection in forEach loop over this collection. I think the best solution for this situation is to avoid removing from myList in doCleanup() code, it looks like lack of design.
Other possible solution - another doCleanup() method version which doesn't throw an event which cause removal from collection - you already do myList.clear().
Or rewrite removeObject() method like:
public static void removeObject(MyObject o) {
synchronized (myList) {
for (Iterator<MyObject> it = myList.iterator(); it.hasNext(); ) {
MyObject o1 = it.next();
if (o1.equals(o)) {
it.remove();
}
}
}
o.doCleanup();
}
like #geert3 recommends in his answer as far as I understand, but motivation in this answer is not fully clear for me.
But I don't like last solution - it looks like a hack for design problem because in this global collection maintaining code we call doCleanup() on deleted object which should call one more removeObject() inside event handler - I think it will be better to remove this "recursion".
static final Collection<String> FILES = new ArrayList<String>(1);
for (final String s : list) {
new Thread(new Runnable() {
public void run() {
List<String> file2List = getFileAsList(s);
FILES.addAll(file2List);
}
}).start();
}
This collections gets very big, but the code works perfect. I thought I will get a concurrent modifcation exception, because the FILES list has to extend its size, but it has never happened.
is this code 100% threadsafe ?
The code takes a 12 seconds to load up and a few threads are adding elements at the same time.
I tried to first create thread and later run them, but I got same results (both time and correctness)
No, the code is not thread-safe. It may or may not throw a ConcurrentModificationException, but you may end up with elements missing or elements being added twice. Changing the list to be a
Collection<String> FILES = Collections.synchronizedList(new ArrayList<String>());
might already be a solution, assuming that the most time-consuming part is the getFilesAsList method (and not adding the resulting elements to the FILES list).
BTW, an aside: When getFileAsList is accessing the hard-drive, you should perform detailed performance tests. Multi-threaded hard-drive accesses may be slower than a single-threaded one, because the hard drive head might have to jump around the drive and not be able to read data as a contiguous block.
EDIT: In response to the comment: This program will "very likely" produce ArrayIndexOutOfBoundsExceptions from time to time:
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
public class ConcurrentListTest
{
public static void main(String[] args) throws InterruptedException
{
for (int i=0; i<1000; i++)
{
runTest();
}
}
private static void runTest() throws InterruptedException
{
final Collection<String> FILES = new ArrayList<String>(1);
// With this, it will always work:
// final Collection<String> FILES = Collections.synchronizedList(new ArrayList<String>(1));
List<String> list = Arrays.asList("A", "B", "C", "D");
List<Thread> threads = new ArrayList<Thread>();
for (final String s : list)
{
Thread thread = new Thread(new Runnable()
{
#Override
public void run()
{
List<String> file2List = getFileAsList(s);
FILES.addAll(file2List);
}
});
threads.add(thread);
thread.start();
}
for (Thread thread : threads)
{
thread.join();
}
System.out.println(FILES.size());
}
private static List<String> getFileAsList(String s)
{
List<String> list = Collections.nCopies(10000, s);
return list;
}
}
But of course, there is no strict guarantee that it will. If it does not create such an exception for you, you should consider playing the lottery, because you must be remarkably lucky.
It is not thread safety at all even if you only add elements. In case that you only increase your FILES there is also some multi access problem if your collection is not thread safe.
When collection exceeds their size it has to be copied to new space and in that moment you can have problems with concurrent access, because in the moment that one thread will do the copy stuff... another can be trying add at the same time element to that collection, resizing is done by internal arraylist implementation but it is not thread-safe at all.
Check that code and lets assume that more than one thread execute it when collection capacity is full.
private int size; //it is not thread safe value!
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e; //size is not volatile value it might be cached by thread
return true;
}
public void add(int index, E element) {
rangeCheckForAdd(index);
ensureCapacityInternal(size + 1); // Increments modCount!!
System.arraycopy(elementData, index, elementData, index + 1,
size - index);
elementData[index] = element;
size++;
}
public void ensureCapacity(int minCapacity) {
if (minCapacity > 0)
ensureCapacityInternal(minCapacity);
}
private void ensureCapacityInternal(int minCapacity) {
modCount++;
// overflow-conscious code
if (minCapacity - elementData.length > 0)
grow(minCapacity);
}
In the moment that collection need to exceed its capacity and copy existed element to new internal array you can have real problems with multi-access, also size is not thread because it is not volatile and it can be cached by some thread and you can have overrides :) and this is answer why it might be not thread safe even if you use only use add operation on non synchronized collection.
You should consider using FILES=new CopyOnWriteArrayList();, orFILES= Collections.synchronizedList(new ArrayList()); where add operation is thread-safe.
Yes, you need a concurrent list to prevent a ConcurrentModificationException.
Here are some ways to initialize a concurrent list in Java:
Collections.newSetFromMap(new ConcurrentHashMap<>());
Collections.synchronizedList(new ArrayList<Object>());
new CopyOnWriteArrayList<>();
is this code 100% threadsafe ?
This code is 0% threadsafe, even by the weakest standard of interleaved operation. You are mutating shared state under a data race.
You most definitely need some kind of concurrent control; it is not obvious whether a concurrent collection is the right choice, though. A simple synchronizedList might fit the bill even better because you have a lot of processing and then a quick transfer to the accumulator list. The lock will not be contended much.