Currently, I have a problem when I frequently iterating through a HashMap (1-time per second)
I add new element to the map on the main thread and iterate the map on the other thread
I return an ImmutableMap.copyOf() to iterate through it, and sometimes I add a new element to the map. But it throws me a ConcurrentModificationException
java.util.ConcurrentModificationException: null
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911) ~[?:1.8.0_261]
at java.util.ArrayList$Itr.next(ArrayList.java:861) ~[?:1.8.0_261]
My pseudo-code:
class Foo {
private final Map<String, List<User>> userMap = new HashMap<>();
public static void addUser(String houseId, User user) {
if(!userMap.containsKey(houseId)) {
userMap.put(houseId, new ArrayList<>());
}
List<User> list = userMap.get(houseId);
list.add(user);
}
public static Map<String, List<User>> getAll() {
return ImmutableMap.copyOf(userMap);
}
}
// iterating task
// this task run one per second
class Task extends TimerTask {
public void run() {
for (List<User> listUser : Foo.getAll().values()) {
for (User user : listUser) {
user.sayHello(); // print hello
}
}
}
}
I thought that use an ImmutableMap.copyOf() will prevent me from this problem. Because I read that you should use an immutable copy of your list/map when iterating through it from other thread.
I think I "solve" this by using CopyOnWriteArrayList. But I'm really curious why it throws me an error.
Thank you!!!
As you can see in the stack trace, the error occurs in the ArrayList, not the Map.
In method addUser, an element is added to the list via
list.add(user);
This operation might happen, while another thread loops over the elements with for (User user : listUser) causing the error.
When you exchange the ArrayList with a CopyOnWriteArrayList, the list.add(user) internally makes a copy and adds the new user into the copy. Therefore, a potentially running iteration is not affected since it loops over the old version of the array. So the running iteration will ignore the new value. Any new iterator will include the new value.
Related
I'm using Spring with Redis and I am working with a list inside a hash. Everything works great on a single thread, problems come when I have to update list value with more than one instances.
Here is my code to put and get value from Hash:
public void put(String hashName, int key , List<myObj> myList) {
redis.opsForHash().put(hashName, String.valueOf(key), myList);
}
public List<myObj> get(String hashName, string key) {
Object map = redis.opsForHash().get(hashName,key);
if (map==null) {
log.info("no keys found");
return new ArrayList<myObj>();
}
List<myObj> myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
return myList;
}
What I do to perform update is:
List<myObj> myList= hash.get(hashName,key);
myList.add(obj);
hash.put(hashName, key, myList);
When there is more than one instance I occur in race condition. Is there a way to update list values in an atomic way?
Your current implementation is not good, because in the put() you update the whole list. In case many threads want to add a single element to the list they first obtain the current list, then add an element, then put new list. Each thread will override the result of the previous one, the last one wins. Usage of synchronized doesn't matter here.
Solution
Don't replace the whole list. Instead, add a single element to the list. Remove the method put() and add a new one like following:
public synchronized void add(String hashName, int key, myObj element) {
List<myObj> myList;
Object map = redis.opsForHash().get(hashName,key);
if (map != null) {
myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
} else {
myList = new ArrayList<myObj>();
}
myList.add(element);
redis.opsForHash().put(hashName, String.valueOf(key), myList);
}
Besides make sure there are no attempts to modify the list directly and that the only way to add elements is to use your method add(). Use Collections.unmodifiableList():
public List<myObj> get(String hashName, string key) {
Object map = redis.opsForHash().get(hashName,key);
if (map==null) {
log.info("no keys found");
return new ArrayList<myObj>();
}
List<myObj> myList= mapper.convertValue(map, new TypeReference<List<myObj>(){});
return Collections.unmodifiableList(myList);
}
Note: I am aware of the Iterator#remove() method.
In the following code sample, I don't understand why the List.remove in main method throws ConcurrentModificationException, but not in the remove method.
public class RemoveListElementDemo {
private static final List<Integer> integerList;
static {
integerList = new ArrayList<Integer>();
integerList.add(1);
integerList.add(2);
integerList.add(3);
}
public static void remove(Integer toRemove) {
for(Integer integer : integerList) {
if(integer.equals(toRemove)) {
integerList.remove(integer);
}
}
}
public static void main(String... args) {
remove(Integer.valueOf(2));
Integer toRemove = Integer.valueOf(3);
for(Integer integer : integerList) {
if(integer.equals(toRemove)) {
integerList.remove(integer);
}
}
}
}
Here's why:
As it is says in the Javadoc:
The iterators returned by this class's iterator and listIterator
methods are fail-fast: if the list is structurally modified at any
time after the iterator is created, in any way except through the
iterator's own remove or add methods, the iterator will throw a
ConcurrentModificationException.
This check is done in the next() method of the iterator (as you can see by the stacktrace). But we will reach the next() method only if hasNext() delivered true, which is what is called by the for each to check if the boundary is met. In your remove method, when hasNext() checks if it needs to return another element, it will see that it returned two elements, and now after one element was removed the list only contains two elements. So all is peachy and we are done with iterating. The check for concurrent modifications does not occur, as this is done in the next() method which is never called.
Next we get to the second loop. After we remove the second number the hasNext method will check again if can return more values. It has returned two values already, but the list now only contains one. But the code here is:
public boolean hasNext() {
return cursor != size();
}
1 != 2, so we continue to the next() method, which now realizes that someone has been messing with the list and fires the exception.
Hope that clears your question up.
Summary
List.remove() will not throw ConcurrentModificationException when it removes the second last element from the list.
One way to handle it it to remove something from a copy of a Collection (not Collection itself), if applicable. Clone the original collection it to make a copy via a Constructor.
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For your specific case, first off, i don't think final is a way to go considering you intend to modify the list past declaration
private static final List<Integer> integerList;
Also consider modifying a copy instead of the original list.
List<Integer> copy = new ArrayList<Integer>(integerList);
for(Integer integer : integerList) {
if(integer.equals(remove)) {
copy.remove(integer);
}
}
The forward/iterator method does not work when removing items. You can remove the element without error, but you will get a runtime error when you try to access removed items. You can't use the iterator because as pushy shows it will cause a ConcurrentModificationException, so use a regular for loop instead, but step backwards through it.
List<Integer> integerList;
integerList = new ArrayList<Integer>();
integerList.add(1);
integerList.add(2);
integerList.add(3);
int size= integerList.size();
//Item to remove
Integer remove = Integer.valueOf(3);
A solution:
Traverse the array in reverse order if you are going to remove a list element. Simply by going backwards through the list you avoid visiting an item that has been removed, which removes the exception.
//To remove items from the list, start from the end and go backwards through the arrayList
//This way if we remove one from the beginning as we go through, then we will avoid getting a runtime error
//for java.lang.IndexOutOfBoundsException or java.util.ConcurrentModificationException as when we used the iterator
for (int i=size-1; i> -1; i--) {
if (integerList.get(i).equals(remove) ) {
integerList.remove(i);
}
}
This snippet will always throw a ConcurrentModificationException.
The rule is "You may not modify (add or remove elements from the list) while iterating over it using an Iterator (which happens when you use a for-each loop)".
JavaDocs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException.
Hence if you want to modify the list (or any collection in general), use iterator, because then it is aware of the modifications and hence those will be handled properly.
Hope this helps.
I had that same problem but in case that I was adding en element into iterated list.
I made it this way
public static void remove(Integer remove) {
for(int i=0; i<integerList.size(); i++) {
//here is maybe fine to deal with integerList.get(i)==null
if(integerList.get(i).equals(remove)) {
integerList.remove(i);
}
}
}
Now everything goes fine because you don't create any iterator over your list, you iterate over it "manually". And condition i < integerList.size() will never fool you because when you remove/add something into List size of the List decrement/increment..
Hope it helps, for me that was solution.
If you use copy-on-write collections it will work; however when you use list.iterator(), the returned Iterator will always reference the collection of elements as it was when ( as below )
list.iterator() was called, even if another thread modifies the collection. Any
mutating methods called on a copy-on-write–based Iterator or ListIterator
(such as add, set, or remove) will throw an UnsupportedOperationException.
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
public class RemoveListElementDemo {
private static final List<Integer> integerList;
static {
integerList = new CopyOnWriteArrayList<>();
integerList.add(1);
integerList.add(2);
integerList.add(3);
}
public static void remove(Integer remove) {
for(Integer integer : integerList) {
if(integer.equals(remove)) {
integerList.remove(integer);
}
}
}
public static void main(String... args) {
remove(Integer.valueOf(2));
Integer remove = Integer.valueOf(3);
for(Integer integer : integerList) {
if(integer.equals(remove)) {
integerList.remove(integer);
}
}
}
}
This runs fine on Java 1.6
~ % javac RemoveListElementDemo.java
~ % java RemoveListElementDemo
~ % cat RemoveListElementDemo.java
import java.util.*;
public class RemoveListElementDemo {
private static final List<Integer> integerList;
static {
integerList = new ArrayList<Integer>();
integerList.add(1);
integerList.add(2);
integerList.add(3);
}
public static void remove(Integer remove) {
for(Integer integer : integerList) {
if(integer.equals(remove)) {
integerList.remove(integer);
}
}
}
public static void main(String... args) {
remove(Integer.valueOf(2));
Integer remove = Integer.valueOf(3);
for(Integer integer : integerList) {
if(integer.equals(remove)) {
integerList.remove(integer);
}
}
}
}
~ %
In my case I did it like this:
int cursor = 0;
do {
if (integer.equals(remove))
integerList.remove(cursor);
else cursor++;
} while (cursor != integerList.size());
Change Iterator for each into for loop to solve.
And the Reason is:
The iterators returned by this class's iterator and listIterator
methods are fail-fast: if the list is structurally modified at any
time after the iterator is created, in any way except through the
iterator's own remove or add methods, the iterator will throw a
ConcurrentModificationException.
--Referred Java Docs.
Check your code man....
In the main method you are trying to remove the 4th element which is not there and hence the error.
In the remove() method you are trying to remove the 3rd element which is there and hence no error.
I am using LinkedList data structure serverList to store the elements in it. As of now, it can also insert null in the LinkedList serverList which is not what I want. Is there any other data structure which I can use which will not add null element in the serverList list but maintain the insert ordering?
public List<String> getServerNames(ProcessData dataHolder) {
// some code
String localIP = getLocalIP(localPath, clientId);
String localAddress = getLocalAddress(localPath, clientId);
// some code
List<String> serverList = new LinkedList<String>();
serverList.add(localIP);
if (ppFlag) {
serverList.add(localAddress);
}
if (etrFlag) {
for (String remotePath : holderPath) {
String remoteIP = getRemoteIP(remotePath, clientId);
String remoteAddress = getRemoteAddress(remotePath, clientId);
serverList.add(remoteIP);
if (ppFlag) {
serverList.add(remoteAddress);
}
}
}
return serverList;
}
This method will return a List which I am iterating it in a for loop in normal way. I can have empty serverList if everything is null, instead of having four null values in my list. In my above code, getLocalIP, getLocalAddress, getRemoteIP and getRemoteAddress can return null and then it will add null element in the linked list. I know I can add a if check but then I need to add if check four time just before adding to Linked List. Is there any better data structure which I can use here?
One constraint I have is - This library is use under very heavy load so this code has to be fast since it will be called multiple times.
I am using LinkedList data structure serverList to store the elements in it.
That's most probably wrong, given that you're aiming at speed. An ArrayList is much faster unless you're using it as a Queue or alike.
I know I can add a if check but then I need to add if check four time just before adding to Linked List. Is there any better data structure which I can use here?
A collection silently ignoring nulls would be a bad idea. It may be useful sometimes and very surprising at other times. Moreover, it'd violate the List.add contract. So you won't find it in any serious library and you shouldn't implement it.
Just write a method
void <E> addIfNotNullTo(Collection<E> collection, E e) {
if (e != null) {
collection.add(e);
}
}
and use it. It won't make your code really shorter, but it'll make it clearer.
One constraint I have is - This library is use under very heavy load so this code has to be fast since it will be called multiple times.
Note that any IO is many orders of magnitude slower than simple list operations.
Use Apache Commons Collection:
ListUtils.predicatedList(new ArrayList(), PredicateUtils.notNullPredicate());
Adding null to this list throws IllegalArgumentException. Furthermore you can back it by any List implementation you like and if necessary you can add more Predicates to be checked.
Same exists for Collections in general.
There are data structures that do not allow null elements, such as ArrayDeque, but these will throw an exception rather than silently ignore a null element, so you'd have to check for null before insertion anyway.
If you're dead set against adding null checks before insertion, you could instead iterate over the list and remove null elements before you return it.
The simplest way would be to just override LinkedList#add() in your getServerNames() method.
List<String> serverList = new LinkedList<String>() {
public boolean add(String item) {
if (item != null) {
super.add(item);
return true;
} else
return false;
}
};
serverList.add(null);
serverList.add("NotNULL");
System.out.println(serverList.size()); // prints 1
If you then see yourself using this at several places, you can probably turn it into a class.
You can use a plain Java HashSet to store your paths. The null value may be added multiple times, but it will only ever appears once in the Set. You can remove null from the Set and then convert to an ArrayList before returning.
Set<String> serverSet = new HashSet<String>();
serverSet.add(localIP);
if (ppFlag) {
serverSet.add(localAddress);
}
if (etrFlag) {
for (String remotePath : holderPath) {
String remoteIP = getRemoteIP(remotePath, clientId);
String remoteAddress = getRemoteAddress(remotePath, clientId);
serverSet.add(remoteIP);
if (ppFlag) {
serverSet.add(remoteAddress);
}
}
}
serverSet.remove(null); // remove null from your set - no exception if null not present
List<String> serverList = new ArrayList<String>(serverSet);
return serverList;
Since you use Guava (it's tagged), I have this alternative if you have the luxury of being able to return a Collection instead of a List.
Why Collection ? Because List forces you to either return true or throw an exception. Collection allows you to return false if you didn't add anything to it.
class MyVeryOwnList<T> extends ForwardingCollection<T> { // Note: not ForwardingList
private final List<T> delegate = new LinkedList<>(); // Keep a linked list
#Override protected Collection<T> delegate() { return delegate; }
#Override public boolean add(T element) {
if (element == null) {
return false;
} else {
return delegate.add(element);
}
}
#Override public boolean addAll(Collection<? extends T> elements) {
return standardAddAll(elements);
}
}
I am trying to designing a software that convert a flowchart into java or any other code. However I repeatedly getting the ConcurrentModificationException..
But I can't use a boolean to prevent concurrentModification, because access to the linked list happens in various places.
So as a solution I created the below adapter class. However it also throws the same exception from next method. Are there any other solution or if can, plz let me know how to modify my codes...
thank you very much...
import java.util.Iterator;
import java.util.LinkedList;
public class LinkedListAdapter<T> extends LinkedList<T>{
#Override
public boolean add(T t){
boolean b;
synchronized(this){
b = super.add(t);
}
return b;
}
#Override
public T remove(){
T t;
synchronized(this){
t = super.remove();
}
return t;
}
#Override
public Iterator<T> iterator(){
final LinkedListAdapter<T> adap = this;
return
new Iterator<T>(){
private Iterator<T> iter;
{
synchronized(adap){
iter = LinkedListAdapter.this.getIterator();
}
}
#Override
public boolean hasNext() {
boolean b;
synchronized(adap){
b = iter.hasNext();
}
return b;
}
#Override
public T next() {
T t;
synchronized(adap){
t = iter.next();
}
return t;
}
#Override
public void remove() {
throw new UnsupportedOperationException();
}
};
}
protected Iterator<T> getIterator() {
Iterator<T> iter;
synchronized(this){
iter = super.iterator();
}
return iter;
}
}
The ConcurrentModificationException is usually thrown when iterating through the list and in the same time usually another thread or even the same loop tries to modify (add / remove) the contents of the list.
Using a synchronizedList or a synchronized list still has to be synchronised externally when iterating over it.
If you use ConcurrentLinkedQueue you don't have these issues.
Queue<Task> tasks = new ConcurrentLinkedQueue<Task>();
tasks.add(task); // thread safe
tasks.remove(task2); // thread safe
for(Task t: tasks) // can iterate without a CME.
Note: if you are using a queue with another thread I suggest you use an ExecutorService as this combines a Queue with a ThreadPool and make working with "background" thread much easier.
why not use LinkedBlockingQueue? http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/LinkedBlockingQueue.html
BTW, it's not neceserally have to do with synchronization. a code like this:
for(Value v : valuesList){
valueslist.add(new Value());
}
would cause this exception as well. check your code for possible modifications of the list when it's being iterated over.
This happens when you iterate over the list and add elements to it in the body of the loop. You can remove elements safely when you use the remove() method of the iterator but not by calling any of the remove() methods of the list itself.
The solution is to copy the list before you iterate over it:
List<T> copy = new ArrayList<T>( list );
for( T e : copy ) {
... you can now modify "list" safely ...
}
Java collections are fail-fast, that means that all existing Iterators become invalid the moment the underlying collection is modified - synchronizing the modification does not stop the list from invalidating all iterators.
As a workaround you can create a copy of the list to iterate over or postpone modifications until the iteration is finished. To remove entries you can also use the iterator.remove() method which keeps the iterator itself valid.
List<X> myList = ....
List<X> myThreadSafeList = synchronizedList(myList);
synchronizedList(myList)
Notice the following statement in the JavaDoc:
It is imperative that the user manually synchronize on the returned list when iterating over it:
List list = Collections.synchronizedList(new ArrayList());
...
synchronized(list) {
Iterator i = list.iterator(); // Must be in synchronized block
while (i.hasNext())
foo(i.next());
}
The answer here: Why am I getting java.util.ConcurrentModificationException? helped me a lot.
I will copy and paste it here in case anyone is looking to fix this error:
When you iterate through a list, you can't remove items from it. Doing so causes the exception.
Do:
int size = list.size();
for (int i = 0 ; i< size ; i++) {
list.add(0,"art");
list.remove(6);
System.out.println(list);
}
I'm trying to add new object to my ArrayList if it satisfy the condition.
But it got me this ConcurrentModificationExeption when I tried to run it. Hope you could help me:
public void addTaskCollection(Task t){
ListIterator<Task> iterator = this.taskCollection.listIterator();
if(this.taskCollection.isEmpty())
this.taskCollection.add(t);
while (iterator.hasNext()){
if(t.isOverlapped(iterator.next()))
this.taskCollection.add(t);
}
}
And here is the exeption error
Exception in thread "main" java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
at java.util.ArrayList$Itr.next(ArrayList.java:791)
at Diary.addTaskCollection(Diary.java:36)
at Test.main(Test.java:50)
Java Result: 1
Replace your code with:
ListIterator<Task> iterator = this.taskCollection.listIterator();
boolean marker = false;
if(taskCollection.isEmpty())
this.taskCollection.add(t);
else {
while (iterator.hasNext()) {
if(iterator.next().isOverlapped(t) == false)
marker = true;
}
}
if (marker == true)
taskCollection.add(t);
to avoid ConcurrentModificationException.
copy the array and change the original.
It seems you encounter a race condition. Multiple threads are accessing / modifying the same collection. Use a thread-safe List implementation.
Also, you must not modifying the collection (adding / removing) while iterating on it with an Iterator.
EDIT
ConcurrentModificationExeption sounds like taskCollection is accessed and modified by multiple threads at the same time (we can not say regarding the piece of code you provide if your program is single or multi threaded). If you share taskCollection between several threads, use a thread-safe list implementation.
But, the error here is actually clearly due to the fact that you add an element to the collection between the moment you get an iterator on it and the moment you use this iterator. To fix that copy the new elements in temporary list and add them all in once at the end of the iteration.
Re-formatted Truong's answer from comments:
ListIterator<Task> iterator = this.taskCollection.listIterator();
boolean marker = false;
if(taskCollection.isEmpty())
this.taskCollection.add(t);
else {
while (iterator.hasNext()) {
if(iterator.next().isOverlapped(t) == false)
marker = true;
}
if (marker == true)
taskCollection.add(t);
}
Maintain two iterators.
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class Example_v3 {
public static void main(String[] args) {
List<String> list = new ArrayList<String>();
// Insert some sample values.
list.add("Value1");
list.add("Value2");
list.add("Value3");
// Get two iterators.
Iterator<String> ite = list.iterator();
Iterator<String> ite2 = list.iterator();
// Point to the first object of the list and then, remove it.
ite.next();
ite.remove();
/* The second iterator tries to remove the first object as well. The object does
* not exist and thus, a ConcurrentModificationException is thrown. */
ite2.next();
ite2.remove();
}
}