I'm reading B. Goetz Java Concurrency In Practice and now I'm at the section about thread-safe collections. He described the so-called "hidden iterators" which may throw ConcurrentModificationException. Here is the example he dispensed:
public class HiddenIterator{
#GuardedBy("this")
private final Set<Integer> set = new HashSet<Integer>();
public synchronized void add(Integer i){ set.add(i); }
public synchronized void remove(Integer i){ set.remove(i); }
public void addTenThings(){
Random r = new Random();
for(int i = 0; i < 10; i++)
add(r.nextInt());
System.out.println("DEBUG: added ten elements to set " + set)
}
}
Now, it's obviously that addTenThings() may throw ConcurrentModificationException as that printing set's content involves iterating it. But he provide the following suggestion for dealing with it:
If HiddenIterator wrapped the HashSet with a synchronizedSet,
encapsulating the synchronization, this sort of error would not occur.
I don't quite understand it. Even if we wrapped set into a synchronized-wrapper, the class would still remain NotThreadSafe. What did he mean?
This is because Collections.synchronizedSet synchronizes every method, including toString. Indeed, if you tried to iterate over a wrapped set manually, you could get ConcurrentModificationException, so you have to synchronize manual iteration yourself. But methods that do hidden iterations already do it, so you don't have to worry about that at least. Here is the corresponding piece of code from the JDK sources:
public String toString() {
synchronized (mutex) {return c.toString();}
}
Here, mutex is initialized to this in the constructor of the wrapper class, so it's basically synchronized (this).
Related
Can someone explain the output of the following program:
public class DataRace extends Thread {
static ArrayList<Integer> arr = new ArrayList<>();
public void run() {
Random random = new Random();
int local = random.nextInt(10) + 1;
arr.add(local);
}
public static void main(String[] args) {
DataRace t1 = new DataRace();
DataRace t2 = new DataRace();
DataRace t3 = new DataRace();
DataRace t4 = new DataRace();
t1.start();
t2.start();
t3.start();
t4.start();
try {
t1.join();
t2.join();
t3.join();
t4.join();
} catch (InterruptedException e) {
System.out.println("interrupted");
}
System.out.println(DataRace.arr);
}
}
Output:
[8, 5]
[9, 2, 2, 8]
[2]
I am having trouble understanding the varying number of values in my output. I would expect the main thread to either wait until all threads have finished execution as I am joining them in the try-catch block and then output four values, one from each thread, or print to the console in case of an interruption. Neither of which is really happening here.
How does it come into play here if this is due to data race in multithreading?
The main problem is that multiple threads are adding to the same shared ArrayList concurrently. ArrayList is not thread-safe. From source one can read:
Note that this implementation is not synchronized.
If multiple threads
access an ArrayList instance concurrently, and at least one of the
threads modifies the list structurally, it must be synchronized
externally. (A structural modification is any operation that adds or
deletes one or more elements, or explicitly resizes the backing array;
merely setting the value of an element is not a structural
modification.) This is typically accomplished by synchronizing on some
object that naturally encapsulates the list. If no such object exists,
the list should be "wrapped" using the Collections.synchronizedList
method. This is best done at creation time, to prevent accidental
unsynchronized access to the list:
In your code every time you call
arr.add(local);
inside the add method implementation, among others, a variable that keeps track of the size of the array will be updated. Below is shown the relevant part of the add method of the ArrayList:
private void add(E e, Object[] elementData, int s) {
if (s == elementData.length)
elementData = grow();
elementData[s] = e;
size = s + 1; // <--
}
where the variable field size is:
/**
* The size of the ArrayList (the number of elements it contains).
*
* #serial
*/
private int size;
Notice that neither is the add method synchronized nor the variable size is marked with the volatile clause. Hence, suitable to race-conditions.
Therefore, because you did not ensure mutual exclusion on the accesses to that ArrayList (e.g., surrounding the calls to the ArrayList with the synchronized clause), and because the ArrayList does not ensure that the size variable is updated atomically, each thread might see (or not) the last updated value of that variable. Hence, threads might see outdated values of the size variable, and add elements into positions that already other threads have added before. In the extreme, all threads might end-up adding an element into the same position (e.g., as one of your outputs [2]).
The aforementioned race-condition leads to undefined behavior, hence the reason why:
System.out.println(DataRace.arr);
outputs different number of elements in different execution of your code.
To make the ArrayList thread-safe or for alternatives have a look at the following SO thread: How do I make my ArrayList Thread-Safe?, where it showcases the use of Collections.synchronizedList()., CopyOnWriteArrayList among others.
An example of ensuring mutual exclusion of the accesses to the arr structure:
public void run() {
Random random = new Random();
int local = random.nextInt(10) + 1;
synchronized (arr) {
arr.add(local);
}
}
or :
static final List<Integer> arr = Collections.synchronizedList(new ArrayList<Integer>());
public void run() {
Random random = new Random();
int local = random.nextInt(10) + 1;
arr.add(local);
}
TL;DR
ArrayList is not Thread-Safe. Therefore it's behaviour in a race-condition is undefined. Use synchronized or CopyOnWriteArrayList instead.
Longer answer
ArrayList.add ultimately calls this private method:
private void add(E e, Object[] elementData, int s) {
if (s == elementData.length)
elementData = grow();
elementData[s] = e;
size = s + 1;
}
When two Threads reach this same point at the "same" time, they would have the same size (s), and both will try add an element on the same position and update the size to s + 1, thus likely keeping the result of the second.
If the size limit of the ArrayList is reached, and it has to grow(), a new bigger array is created and the contents copied, likely causing any other changes made concurrently to be lost (is possible that multiple threads will be trying to grow).
Alternatives here are to use monitors - a.k.a. synchronized, or to use Thread-Safe alternatives like CopyOnWriteArrayList.
I think there is a lot of similar or closely related questions. For example see this.
Basically the reason of this "unexpected" behabiour is because ArrayList is not thread-safe. You can try List<Integer> arr = new CopyOnWriteArrayList<>() and it will work as expected. This data structure is recommended when we want to perform read operation frequently and the number of write operations is relatively rare. For good explanation see What is CopyOnWriteArrayList in Java - Example Tutorial.
Another option is to use List<Integer> arr = Collections.synchronizedList(new ArrayList<>()).
You can also use Vector but it is not recommended (see here).
This article also will be useful - Vector vs ArrayList in Java.
I was thinking about how to solve race condition between two threads which tries to write to the same variable using immutable objects and without helping any keywords such as synchronize(lock)/volatile in java.
But I couldn't figure it out, is it possible to solve this problem with such solution at all?
public class Test {
private static IAmSoImmutable iAmSoImmutable;
private static final Runnable increment1000Times = () -> {
for (int i = 0; i < 1000; i++) {
iAmSoImmutable.increment();
}
};
public static void main(String... args) throws Exception {
for (int i = 0; i < 10; i++) {
iAmSoImmutable = new IAmSoImmutable(0);
Thread t1 = new Thread(increment1000Times);
Thread t2 = new Thread(increment1000Times);
t1.start();
t2.start();
t1.join();
t2.join();
// Prints a different result every time -- why? :
System.out.println(iAmSoImmutable.value);
}
}
public static class IAmSoImmutable {
private int value;
public IAmSoImmutable(int value) {
this.value = value;
}
public IAmSoImmutable increment() {
return new IAmSoImmutable(++value);
}
}
If you run this code you'll get different answers every time, which mean a race condition is happening.
You can not solve race condition without using any of existence synchronisation (or volatile) techniques. That what they were designed for. If it would be possible there would be no need of them.
More particularly your code seems to be broken. This method:
public IAmSoImmutable increment() {
return new IAmSoImmutable(++value);
}
is nonsense for two reasons:
1) It makes broken immutability of class, because it changes object's variable value.
2) Its result - new instance of class IAmSoImmutable - is never used.
The fundamental problem here is that you've misunderstood what "immutability" means.
"Immutability" means — no writes. Values are created, but are never modified.
Immutability ensures that there are no race conditions, because race conditions are always caused by writes: either two threads performing writes that aren't consistent with each other, or one thread performing writes and another thread performing reads that give inconsistent results, or similar.
(Caveat: even an immutable object is effectively mutable during construction — Java creates the object, then populates its fields — so in addition to being immutable in general, you need to use the final keyword appropriately and take care with what you do in the constructor. But, those are minor details.)
With that understanding, we can go back to your initial sentence:
I was thinking about how to solve race condition between two threads which tries to write to the same variable using immutable objects and without helping any keywords such as synchronize(lock)/volatile in java.
The problem here is that you actually aren't using immutable objects: your entire goal is to perform writes, and the entire concept of immutability is that no writes happen. These are not compatible.
That said, immutability certainly has its place. You can have immutable IAmSoImmutable objects, with the only writes being that you swap these objects out for each other. That helps simplify the problem, by reducing the scope of writes that you have to worry about: there's only one kind of write. But even that one kind of write will require synchronization.
The best approach here is probably to use an AtomicReference<IAmSoImmutable>. This provides a non-blocking way to swap out your IAmSoImmutable-s, while guaranteeing that no write gets silently dropped.
(In fact, in the special case that your value is just an integer, the JDK provides AtomicInteger that handles the necessary compare-and-swap loops and so on for threadsafe incrementation.)
Even if the problems are resolved by :
Avoiding the change of IAmSoImmutable.value
Reassigning the new object created within increment() back into the iAmSoImmutable reference.
There still are pieces of your code that are not atomic and that needs a sort of synchronization.
A solution would be to use a synchronized method of course
public synchronized static void increment() {
iAmSoImmutable = iAmSoImmutable.increment();
}
Thread t1 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
increment();
}
});
Thread t2 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
increment();
}
});
I am trying to run 2 concurrent threads, where one keeps adding objects to a list and the other updates these objects and may remove some of these objects from the list as well.
I have a whole project where I've used ArrayList for my methods and classes so it's difficult to change it now.
I've looked around and I found a few ways of doing this, but as I said it is difficult to change from ArrayList. I tried using synchronized and notify() for the method adding the objects to the list and wait() for the method changing these objects and potentially removing them if they meet certain criteria.
Now, I've figured out how to do this using a CopyOnWriteArrayList, but I would like to know if there's a possibility of using ArrayList itself to simulate this. so that I don't have to edit my entire code.
So, basically, I would like to do something like this, but with ArrayList:
import java.util.Iterator;
import java.util.concurrent.CopyOnWriteArrayList;
public class ListExample{
CopyOnWriteArrayList<MyObject> syncList;
public ListExample(){
syncList = new CopyOnWriteArrayList<MyObject>();
Thread thread1 = new Thread(){
public void run(){
synchronized (syncList){
for(int i = 0; i < 10; i++){
syncList.add(new MyObject(i));
}
}
}
};
Thread thread2 = new Thread(){
public void run(){
synchronized (syncList){
Iterator<MyObject> iterator = syncList.iterator();
while(iterator.hasNext()){
MyObject temp = iterator.next();
//this is just a sample list manipulation
if (temp.getID() > 3)
syncList.remove(temp);
System.out.println("Object ID: " + temp.getID() + " AND list size: " + syncList.size());
}
}
}
};
thread1.start();
thread2.start();
}
public static void main(String[] args){
new ListExample();
}
}
class MyObject{
private int ID;
public MyObject(int ID){
this.ID = ID;
}
public int getID(){
return ID;
}
public void setID(int ID){
this.ID = ID;
}
}
I've also read about Collections.synchronizedList(new ArrayList()) but again, I believe this would require me to change my code as I have a substantial number of methods that take ArrayList as a parameter.
Any guidance would be appreciated, because I am out of ideas. Thank you.
You may be interested on the collections provided by the java.util.concurrent package. They are very useful for producer/consumer scenarios, where one or more threads add things to a queue, and other threads take them. There are different methods depending on whether you want to block, or fail when the queue is full/empty.
About refactoring your methods, you should have used interfaces (e.g. List) instead of concrete implementation classes (such as ArrayList). That is the purpose of interfaces, and the Java API has a good suply of them.
As a quick solution u may extend ArrayList and make modifying methods (add/remove) synchronized. and re-factor code to replace ArrayList to your custom-ArrayList
Use Vector instead of ArrayList. Remember to store it in a List reference as Vector contains deprecated methods. Vector, unlike ArrayList, synchronizes its internal operations, and unlike CopyOnWriteArrayList, does not copy the internal array each time a modification is made.
Of course you should be using java.util.concurrent pakage. But let's look at what is happening/could happen with only ArrayList and synchronization.
In your code, if you have just ArrayList in place of CopyOnWriteArrayList, it should work as you have provided full synchronization synchronized (syncList) on whatever you are doing/manipulating in threads. You do not require any wait() notify() if whole thing is synchronized (But that's not recommended, will come to that).
But this code will give ConcurrentModificationException because once you are using iterator syncList.iterator() you should not remove element from that list, otherwise it may give undesirable results while iterating that's why it's designed to fail fast and give exception. To avoid this you can use like:
Iterator<MyObject> iterator = syncList.iterator();
ArrayList<MyObject> toBeRemoved = new ArrayList<MyObject>();
while(iterator.hasNext()){
MyObject temp = iterator.next();
//this is just a sample list manipulation
if (temp.getID() > 3)
{
//syncList.remove(temp);
toBeRemoved.add(temp);
}
System.out.println("Object ID: " + temp.getID() + " AND list size: " + syncList.size());
}
syncList.removeAll(toBeRemoved);
Now regarding synchronization, you should strive to minimize its scope otherwise there'll be unnecessary waiting between threads, thats why java.util.concurrent package is given to have high performance in multithreading (using even non blocking algorithms). Or you can also use Collections.synchronizedList(new ArrayList()) but they are not as good as concurrent classes.
If you want to use conditional synchronization like in producer/consumer problem, then you can use wait() notify() mechanism on same object (lock). But again there're already some classes to help like using java.util.concurrent.LinkedBlockingQueue.
In the "Effective Java", the author mentioned that
while (!done) i++;
can be optimized by HotSpot into
if (!done) {
while (true) i++;
}
I am very confused about it. The variable done is usually not a const, why can compiler optimize that way?
The author assumes there that the variable done is a local variable, which does not have any requirements in the Java Memory Model to expose its value to other threads without synchronization primitives. Or said another way: the value of done won't be changed or viewed by any code other than what's shown here.
In that case, since the loop doesn't change the value of done, its value can be effectively ignored, and the compiler can hoist the evaluation of that variable outside the loop, preventing it from being evaluated in the "hot" part of the loop. This makes the loop run faster because it has to do less work.
This works in more complicated expressions too, such as the length of an array:
int[] array = new int[10000];
for (int i = 0; i < array.length; ++i) {
array[i] = Random.nextInt();
}
In this case, the naive implementation would evaluate the length of the array 10,000 times, but since the variable array is never assigned and the length of the array will never change, the evaluation can change to:
int[] array = new int[10000];
for (int i = 0, $l = array.length; i < $l; ++i) {
array[i] = Random.nextInt();
}
Other optimizations also apply here unrelated to hoisting.
Hope that helps.
Joshua Bloch's "Effective Java" explains why you must be careful when sharing variables between threads. If there doesn't exist any explicit happens before relation between threads, the HotSpot compiler is allowed to optimize the code for speed reasons as shown by dmide.
Most nowadays microprocessors offer different kinds of out-of-order strategies. This leads to a weak consistency model which is also the base for Java's Platform Memory Model. The idea behind is, as long as the programmer does not explicitly express the need for an inter-thread coordination, the processor and the compiler can do different optimizations.
The two keywords volatile (atomicity & visibility) and synchronized (atomicity & visibility & mutual exclusion) are used for expressing the visibility of changes (for other threads). However, in addition you must know the happens before rules (see Goetz et al “Java Concurrency in Practice” p. 341f (JCP) and Java Language Specification §17).
So, what happens when System.out.println() is called? See above.
First of all, you need two System.out.println() calls. One in the main method (after changing done) and one in the started thread (in the while loop). Now, we must consider the program order rule and the monitor lock rule from JLS §17. Here the short version: You have a common lock object M. Everything that happens in a thread A before A unlocks M is visible to another thread B in that moment when B locks M (see JCP).
In our case the two threads share a common PrintStream object in System.out. When we take a look inside println() you see a call of synchronized(this).
Conclusion: Both threads share a common lock M which is locked and unlocked. System.out.println() “flushes” the state change of variable done.
public class StopThread {
private static boolean stopRequested;
private static synchronized void requestStop() {
stopRequested = true;
}
private static synchronized boolean stopRequested() {
return stopRequested;
}
public static void main(String[] args)
throws InterruptedException {
Thread backgroundThread = new Thread(new Runnable() {
public void run() {
int i = 0;
while (!stopRequested())
i++;
}
});
backgroundThread.start();
TimeUnit.SECONDS.sleep(1);
requestStop();
}
}
the above code is right in effective code,it is equivalent that use volatile to decorate the stopRequested.
private static boolean stopRequested() {
return stopRequested;
}
If this method omit the synchronized keyword, this program isn't working well.
I think that this change cause the hoisting when the method omit the synchronized keyword.
If you add System.out.println("i = " + i); in the while loop. The hoisting won't work, meaning the program stops as expected. The println method is thread safe so that the jvm can not optimize the code segment?
The another question is about synchronized. I have also a run() in class Note,because i want to output each element in notes every 5 minutes. But i get always exception:java.util.ConcurrentModificationException,if i try to make more meetings in main. so i applay synchronized to the list notes which may be added a new meeting when i iterate over notes.My run method like this:
Is it correct way hier on list notes to synchronized to prevent ConcurrentModificationException ?(In my program it works.I get never this exception now)
A Meeting class and Note class may likes this:
public class Meeting{
public Meeting(Note note_1,Note note_2){
note_1.addElement(this);
note_2.addElement(this);}
//another method hier
}
public class Note implements Runnable{
public final List<Meeting> notes = new ArrayList<Meeting>();
public void addElement(Meeting n){
entries.add(n);
}
#Override
public void run(){
while(true) {
for(Meeting n : notes){
System.out.println(n.toString);}
}
try{ Thread.sleep(10);}
}
}
}
I get always exception error about exception:java.util.ConcurrentModificationException if i try to make more Meeting in main ,So i changes a littel in class Note,likes this :
private static final List<Entry> entries = Collections.synchronizedList(new ArrayList<Entry>());
and in run()
#Override
public void run() {
while(true){
synchronized(notes){
for(Entry n : entries){
//do something
}
}
try {
Thread.sleep(10);
} catch (InterruptedException e ) {
}
}
}
}
From the javadoc
Note that this exception does not always indicate that an object has been concurrently modified by a different thread. If a single thread issues a sequence of method invocations that violates the contract of an object, the object may throw this exception. For example, if a thread modifies a collection directly while it is iterating over the collection with a fail-fast iterator, the iterator will thow this exception.
THis means do not change your collection in a loop and iterate over it at the same time even in the same thread.
Read to what #Navi had written.
In a nutshell - NEVER remove/add elements of a collection in for each loop.
I once had that kind of problem and I decided to use http://code.google.com/p/google-collections/
There are some map/filter functions there (if I recall that methods were in Collections2 library).
If you are not willing to make the things right, you can always use the old-school iterator technique.
I've used a CopyOnWriteArrayList before when I encountered this sort of problem.
This makes a complete copy of the underlying array for each write, so it's not very efficient, but I've found it very useful for specific circumstances (e.g. a class which manages specialised event notification to some other classes)
This array never changes during the lifetime of the iterator, so interference is impossible and the iterator is guaranteed not to throw ConcurrentModificationException