I'm getting ConcurrentModificationException, so i decided to try something new.
I have an ArrayList of points, which is used in Recycler View for download all graphs in parallel threads.
Is it possible to lock this downloads, when i'm writing to ArrayList and not having a chain download later?
Because now i'm using this in my Fragment:
public static final Object lock = new Object();
synchronized (lock) { /*download and write to ArrayList*/ }
And in every Cell i want to use
synchronized (lock) { /*download graphs*/ }
And i think when one cell is loading graph, all will wait, then second will load and so on.
ConcurrentModificationException usually indicates that a collection was modified while an iterator was in progress. It's not necessarily in a different thread :
List<Object> someList = new ArrayList<>();
someList.add("an item");
for (Object item : someList) {
// throws ConcurrentModificationException
someList.add("a new item");
}
It's difficult to know what really happen in your case. What happen if you replace ArrayLists with CopyOnWriteArrayList ? They are much less efficient than ArrayLists but it probably doesn't matter, at least compared to network operations.
Related
I am using the listIterator() for accessing and removing items from a LinkedList in a class that implementsRunnable I am also modifying values of this list in some other part of the program concurrently.
Where I am using listIterator() in this part of the code I am getting ConcurrentModificationException from this function call:
java.util.LinkedList$ListItr.checkForComodification
Why do I get this and how do I prevent it?
#Override
public void run()
{
while(true)
{
itr = nodeAttributes.listIterator();
while (itr.hasNext())
{
System.out.println("enterred");
nodeAttribute nA = (nodeAttribute) itr.next();
//System.out.println("for");
if(!nA.isMoving && !nA.moveQueue.isEmpty())
{
if(animator != null)
animator.stop();
animator = (Animator) nA.moveQueue.poll();
//itr.remove();
animator.start();
nA.isMoving = true;
System.out.print( "animator");
}
}
System.out.println( "looping");
}
}
Your post doesn't have a question, just a statement. However what you describe is the expected behaviour. From the docs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the Iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
So to prevent this you need to prevent modification by a writer at the same time as a reader is iterating. Use the Collections.synchronizedList method. All access (readers and writers) should follow this pattern:
// store a single synchronized list reference for all access to use
nodeAttributes = Collections.synchronizedList(theLinkedList);
and then all readers and writers should use a synchronized (list) block.
// Readers might do:
synchronized (list) {
itr = nodeAttributes.listIterator();
while (i.hasNext())
... do stuff ...
}
Those threads that operate without iterating can just use the "atomic" methods on the return object from Collections.synchronizedList, such as add. These methods use a sync block under the covers, so they are just a shorthand and they will still block the thread while another is in a sync block.
There are many, many ways to deal with concurrent reader and writers.
One is the above, but it may lock out other threads for a long time while each iterator does it's stuff.
Another is to copy the list to an array (inside a synchronized section) and then read the array outside the lock.
Yet another would be to use a ReadWriteLock.
and there are more options, all dependent on your exact use case.
this is a follow-up post to Do I need a concurrent collection for adding elements to a list by many threads?
everybody there has focused on expaning of the list. I understand how that can be a problem, but .. what about adding elements ? is that thread-safe?
example code
static final Collection<String> FILES = new ArrayList<String>(1000000);
and I execute in many threads (I add less than 1000000 elements)
FILES.add(string)
is that thread safe ? what are possible problems with doing it that way ?
ArrayList<String> is not synchronized by itself. Use Collections.synchronizedList(new ArrayList<String>()) instead.
use Collections.syncronizedList(new ArrayList<String>(1000000))
Even if the list doesn't expand, it maintains its internal state, such as the current size. In a wider sense, ArrayList is specified a mutable object which is not thread-safe, therefore it is illegal to concurrently call any mutator methods on it.
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
This is the source cod of the ArrayList in 7u40-b43.
The method ensureCapacityInternal() is not thread safe,and size++ is not either.
The size++ is done by three steps,1)get the size,2) size +1,3) write the new value back.
I am trying to run 2 concurrent threads, where one keeps adding objects to a list and the other updates these objects and may remove some of these objects from the list as well.
I have a whole project where I've used ArrayList for my methods and classes so it's difficult to change it now.
I've looked around and I found a few ways of doing this, but as I said it is difficult to change from ArrayList. I tried using synchronized and notify() for the method adding the objects to the list and wait() for the method changing these objects and potentially removing them if they meet certain criteria.
Now, I've figured out how to do this using a CopyOnWriteArrayList, but I would like to know if there's a possibility of using ArrayList itself to simulate this. so that I don't have to edit my entire code.
So, basically, I would like to do something like this, but with ArrayList:
import java.util.Iterator;
import java.util.concurrent.CopyOnWriteArrayList;
public class ListExample{
CopyOnWriteArrayList<MyObject> syncList;
public ListExample(){
syncList = new CopyOnWriteArrayList<MyObject>();
Thread thread1 = new Thread(){
public void run(){
synchronized (syncList){
for(int i = 0; i < 10; i++){
syncList.add(new MyObject(i));
}
}
}
};
Thread thread2 = new Thread(){
public void run(){
synchronized (syncList){
Iterator<MyObject> iterator = syncList.iterator();
while(iterator.hasNext()){
MyObject temp = iterator.next();
//this is just a sample list manipulation
if (temp.getID() > 3)
syncList.remove(temp);
System.out.println("Object ID: " + temp.getID() + " AND list size: " + syncList.size());
}
}
}
};
thread1.start();
thread2.start();
}
public static void main(String[] args){
new ListExample();
}
}
class MyObject{
private int ID;
public MyObject(int ID){
this.ID = ID;
}
public int getID(){
return ID;
}
public void setID(int ID){
this.ID = ID;
}
}
I've also read about Collections.synchronizedList(new ArrayList()) but again, I believe this would require me to change my code as I have a substantial number of methods that take ArrayList as a parameter.
Any guidance would be appreciated, because I am out of ideas. Thank you.
You may be interested on the collections provided by the java.util.concurrent package. They are very useful for producer/consumer scenarios, where one or more threads add things to a queue, and other threads take them. There are different methods depending on whether you want to block, or fail when the queue is full/empty.
About refactoring your methods, you should have used interfaces (e.g. List) instead of concrete implementation classes (such as ArrayList). That is the purpose of interfaces, and the Java API has a good suply of them.
As a quick solution u may extend ArrayList and make modifying methods (add/remove) synchronized. and re-factor code to replace ArrayList to your custom-ArrayList
Use Vector instead of ArrayList. Remember to store it in a List reference as Vector contains deprecated methods. Vector, unlike ArrayList, synchronizes its internal operations, and unlike CopyOnWriteArrayList, does not copy the internal array each time a modification is made.
Of course you should be using java.util.concurrent pakage. But let's look at what is happening/could happen with only ArrayList and synchronization.
In your code, if you have just ArrayList in place of CopyOnWriteArrayList, it should work as you have provided full synchronization synchronized (syncList) on whatever you are doing/manipulating in threads. You do not require any wait() notify() if whole thing is synchronized (But that's not recommended, will come to that).
But this code will give ConcurrentModificationException because once you are using iterator syncList.iterator() you should not remove element from that list, otherwise it may give undesirable results while iterating that's why it's designed to fail fast and give exception. To avoid this you can use like:
Iterator<MyObject> iterator = syncList.iterator();
ArrayList<MyObject> toBeRemoved = new ArrayList<MyObject>();
while(iterator.hasNext()){
MyObject temp = iterator.next();
//this is just a sample list manipulation
if (temp.getID() > 3)
{
//syncList.remove(temp);
toBeRemoved.add(temp);
}
System.out.println("Object ID: " + temp.getID() + " AND list size: " + syncList.size());
}
syncList.removeAll(toBeRemoved);
Now regarding synchronization, you should strive to minimize its scope otherwise there'll be unnecessary waiting between threads, thats why java.util.concurrent package is given to have high performance in multithreading (using even non blocking algorithms). Or you can also use Collections.synchronizedList(new ArrayList()) but they are not as good as concurrent classes.
If you want to use conditional synchronization like in producer/consumer problem, then you can use wait() notify() mechanism on same object (lock). But again there're already some classes to help like using java.util.concurrent.LinkedBlockingQueue.
I have two threads modifying the same objects. The objects are custom, non-synchronized objects in an ArrayList (not a vector). I want to make these two threads work nicely together, since they are called at the same time.
Here is the only important method in thread 1.
public void doThread1Action() {
//something...
for(myObject x : MyArrayList){
modify(x);
}
}
Here is a method in thread 2:
public void doThread2Action() {
//something...
for(myObject x : MyArrayList){
modifyAgain(x);
}
}
At the moment, when testing, I occasionally get `ConcurrentModificationExceptions``. (I think it depends on how fast thread 1 finishes its iterations, before thread 2 tries to modify the objects.)
Am I right in thinking that by simply appending synchronized to the beginning of these two methods, the threads will work together in a synchronized way and not try to access the ArrayList? Or should I change the ArrayList to a Vector?
A ConcurrentModificationException does not stem from modifying objects in a collection but from adding / removing from a collection while an iterator is active.
The shared resources is the collection and there must be a third method using and add/remove. To get concurrency right you must synchronize access to the collection resource in all methods that access it.
To avoid overly long synchronized blocks a common pattern may be to copy the collection in a synchronized block and then iterate over it. If you do it this way, be aware the problem you are talking about in first place (concurrent modification of your object) is again in place - but this time you can lock on another resource.
You do not need to synchronize access to the list as long as you don't modify it structurally, i.e. as long as you don't add or remove objects from the list. You also shouldn't see ConcurrentModificationExceptions, because these are only thrown when you structurally modify the list.
So, assuming that you only modify the objects contained in the list, but you do not add or remove or reorder objects on the list, it is possible to synchronize on the contained objects whenever you modify them, like so:
void modifyAgain(MyObject x) {
synchronized(x) {
// do the modification
}
}
I would not use the synchronized modifier on the modifyAgain() method, as that would not allow two distinct objects in the list to be modified concurrently.
The modify() method in the other thread must of course be implemented in the same way as modifyAgain().
You need to sychronsize access to the collection on the same lock, so just using synchronized keyword on the methods (assuming they are in different classes) would be locking on two different objects.
so here is an example of what you might need to do:
Object lock = new Object();
public void doThread1Action(){
//something...
synchronized(lock){
for(myObject x : MyArrayList){
modify(x);
}
}
public void doThread2Action(){
//something...
synchronized(lock){
for(myObject x : MyArrayList){
modifyAgain(x);
}
}
Also you could consider using a CopyOnWriteArrayList instead of Vector
I guess your problem is related to ConcurrentModificationException. This class in its Java docs says:
/**
* This exception may be thrown by methods that have detected
concurrent
* modification of an object when such modification is not
permissible.
*/
In your case, problem is iterator in a list and may modified. I guess by following implementation your problem will sole:
public void doThread1Action()
{
synchronized(x //for sample)
{
//something...
for(myObject x : MyArrayList)
{
modify(x);
}
}
}
and then:
public void doThread2Action()
{
synchronized(x //for sample)
{
//something...
for(myObject x : MyArrayList)
{
modifyAgain(x);
}
}
}
For take better result I want anyone correct my solution.
I have a multithreaded application, where a shared list has write-often, read-occasionally behaviour.
Specifically, many threads will dump data into the list, and then - later - another worker will grab a snapshot to persist to a datastore.
This is similar to the discussion over on this question.
There, the following solution is provided:
class CopyOnReadList<T> {
private final List<T> items = new ArrayList<T>();
public void add(T item) {
synchronized (items) {
// Add item while holding the lock.
items.add(item);
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>();
synchronized (items) {
// Make a copy while holding the lock.
for (T t : items) copy.add(t);
}
return copy;
}
}
However, in this scenario, (and, as I've learned from my question here), only one thread can write to the backing list at any given time.
Is there a way to allow high-concurrency writes to the backing list, which are locked only during the makeSnapshot() call?
synchronized (~20 ns) is pretty fast and even though other operations can allow concurrency, they can be slower.
private final Lock lock = new ReentrantLock();
private List<T> items = new ArrayList<T>();
public void add(T item) {
lock.lock();
// trivial lock time.
try {
// Add item while holding the lock.
items.add(item);
} finally {
lock.unlock();
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>(), ret;
lock.lock();
// trivial lock time.
try {
ret = items;
items = copy;
} finally {
lock.unlock();
}
return ret;
}
public static void main(String... args) {
long start = System.nanoTime();
Main<Integer> ints = new Main<>();
for (int j = 0; j < 100 * 1000; j++) {
for (int i = 0; i < 1000; i++)
ints.add(i);
ints.makeSnapshot();
}
long time = System.nanoTime() - start;
System.out.printf("The average time to add was %,d ns%n", time / 100 / 1000 / 1000);
}
prints
The average time to add was 28 ns
This means if you are creating 30 million entries per second, you will have one thread accessing the list on average. If you are creating 60 million per second, you will have concurrency issues, however you are likely to be having many more resourcing issue at this point.
Using Lock.lock() and Lock.unlock() can be faster when there is a high contention ratio. However, I suspect your threads will be spending most of the time building the objects to be created rather than waiting to add the objects.
You could use a ConcurrentDoublyLinkedList. There is an excellent implementation here ConcurrentDoublyLinkedList.
So long as you iterate forward through the list when you make your snapshot all should be well. This implementation preserves the forward chain at all times. The backward chain is sometimes inaccurate.
First of all, you should investigate if this really is too slow. Adds to ArrayLists are O(1) in the happy case, so if the list has an appropriate initial size, CopyOnReadList.add is basically just a bounds check and an assignment to an array slot, which is pretty fast. (And please, do remember that CopyOnReadList was written to be understandable, not performant.)
If you need a non-locking operation, you can have something like this:
class ConcurrentStack<T> {
private final AtomicReference<Node<T>> stack = new AtomicReference<>();
public void add(T value){
Node<T> tail, head;
do {
tail = stack.get();
head = new Node<>(value, tail);
} while (!stack.compareAndSet(tail, head));
}
public Node<T> drain(){
// Get all elements from the stack and reset it
return stack.getAndSet(null);
}
}
class Node<T> {
// getters, setters, constructors omitted
private final T value;
private final Node<T> tail;
}
Note that while adds to this structure should deal pretty well with high contention, it comes with several drawbacks. The output from drain is quite slow to iterate over, it uses quite a lot of memory (like all linked lists), and you also get things in the opposite insertion order. (Also, it's not really tested or verified, and may actually suck in your application. But that's always the risk with using code from some random dude on the intertubes.)
Yes, there is a way. It is similar to the way ConcurrentHashMap made, if you know.
You should make your own data structure not from one list for all writing threads, but use several independent lists. Each of such lists should be guarded by it's own lock. .add() method should choose list for append current item based on Thread.currentThread.id (for example, just id % listsCount). This will gives you good concurrency properties for .add() -- at best, listsCount threads will be able to write without contention.
On makeSnapshot() you should just iterate over all lists, and for each list you grab it's lock and copy content.
This is just an idea -- there are many places to improve it.
You can use a ReadWriteLock to allow multiple threads to perform add operations on the backing list in parallel, but only one thread to make the snapshot. While the snapshot is being prepared all other add and snapshot request are put on hold.
A ReadWriteLock maintains a pair of associated locks, one for
read-only operations and one for writing. The read lock may be held
simultaneously by multiple reader threads, so long as there are no
writers. The write lock is exclusive.
class CopyOnReadList<T> {
// free to use any concurrent data structure, ConcurrentLinkedQueue used as an example
private final ConcurrentLinkedQueue<T> items = new ConcurrentLinkedQueue<T>();
private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
private final Lock shared = rwLock.readLock();
private final Lock exclusive = rwLock.writeLock();
public void add(T item) {
shared.lock(); // multiple threads can attain the read lock
// try-finally is overkill if items.add() never throws exceptions
try {
// Add item while holding the lock.
items.add(item);
} finally {
shared.unlock();
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>(); // probably better idea to use a LinkedList or the ArrayList constructor with initial size
exclusive.lock(); // only one thread can attain write lock, all read locks are also blocked
// try-finally is overkill if for loop never throws exceptions
try {
// Make a copy while holding the lock.
for (T t : items) {
copy.add(t);
}
} finally {
exclusive.unlock();
}
return copy;
}
}
Edit:
The read-write lock is so named because it is based on the readers-writers problem not on how it is used. Using the read-write lock we can have multiple threads achieve read locks but only one thread achieve the write lock exclusively. In this case the problem is reversed - we want multiple threads to write (add) and only thread to read (make the snapshot). So, we want multiple threads to use the read lock even though they are actually mutating. Only thread is exclusively making the snapshot using the write lock even though snapshot only reads. Exclusive means that during making the snapshot no other add or snapshot requests can be serviced by other threads at the same time.
As #PeterLawrey pointed out, the Concurrent queue will serialize the writes aqlthough the locks will be used for as minimal a duration as possible. We are free to use any other concurrent data structure, e.g. ConcurrentDoublyLinkedList. The queue is used only as an example. The main idea is the use of read-write locks.