We Keep Coding

Iterator inside Iterator ConcurrentModificationException

I am having the following problem:
Given an ArrayList (let's call it list), how can I "double-iterate" through it without getting ConcurrentModificationException?
Here's what I've tried:
iterator out = list.iterator();
iterator in;
while(out.hasNext()){
...
in = list.iterator();
while(in.hasNext()){
...
if(something)
in.remove();
}

You can't do that. A potential solution might be to mark objects to be removed, for example:
final List<Foo> toRemove = new ArrayList<>();
for (Foo a : list)
{
for (Foo b : list)
{
if (something)
{
toRemove.add(b);
}
}
}
list.removeAll(toRemove);
You may need some additional checks to see that the object isn't already marked for removal. It's impossible to say given how vague your example is.

You are trying to modify an iterator. It will give you concurrentModification exception.
In java 8 you can easily remove it using
list.removeIf(someCondition)
Try this link java8 collections

The Iterator instance provided through a call to List#iterator method preserves a count scalar allowing to detect external changes to the Collection container.
When an element is removed from the collection by any other mean than going through the same Iterator#remove(T) call, the count is not updated behind the scenes.
Therefore when you request for the #next() element through the iterator instance, the count is checked against an expected value and if both values does not match (since an element has been removed through another iterator) a ConcurrentModificationException is thrown (even though you may be working in a single threaded environment).
The solution whould be, as #Michael stated, to keep track of the container elements that should be removed then perform a bulk delete:
Collection<Object> temp = new ArrayList<>();
iterator out = list.iterator();
iterator in;
while (out.hasNext()) {
// ...
in = list.iterator();
while (in.hasNext()) {
// ...
if(something)
// just mark the element for deletion
temp.add(in.next());
}
}
// delete all the obsolete elements
list.removeAll(temp);

In collection once iterator creator If you try to modify the content not through same iterator it will throw concurrent exception.If you required some special kind of iterator then you can go ahead and implement your own.

Parallel iteration on multiple collections

I'm reading J. Bloch's "Effective Java" and now I'm at the section about for-each vs for-loop. He mentioned three cases where we couldn't use the for-each loop and one of them is the following:L
Parallel iteration— If you need to traverse multiple collections in
parallel, then you need explicit control over the iterator or index
variable, so that all iterators or index variables can be advanced in
lockstep (as demonstrated unintentionally in the buggy card and dice
examples above).
The case is not quite clear to me, I can't imagine an example.
The first thought that popped into my head was that it was just about iterating the same collection in multiple threads but it's probably not what he meant. I don't see any restrictions preventing us from doing so (read-only). Actually:
public class MyRunnable implements Runnable{
private Collection<String> col;
//CTOR ommmited
public void run(){
for(String s : col){
//print o, not modify
}
}
Then we just start some threads with the same instance. So, we're not afraid of getting ConcurrentModificationException (JavaDocs) because we perform read-only access, even by multiple threads simultaneously.
What's wrong?

I don't think he meant "in parallel" as in concurrently.
It is much simpler. Suppose you have two Collections and you want the same loop (not a nested loop) to iterate over both of them, taking the i'th element of each one in each iteration. You can't do that with the enhanced for loop, since it hides the indices and the iterator.
You must use the standard for loop (for ordered Collections) :
private List<String> one;
private List<String> two;
public void run(){
for(int i = 0; i<one.size() && i<two.size();i++){
// do something with one.get(i) and two.get(i)
}
}
Or explicit Iterators (for un-ordered Collections) :
private Set<String> one;
private Set<String> two;
public void run(){
for(Iterator<String> iterOne=one.iterator(),Iterator<String> iterTwo=two.iterator(); iterOne.hasNext()&&iterTwo.hasNext();){
// do something with iterOne.next() and iterTwo.next()
}
}

Parallel iteration— If you need to traverse multiple collections in
parallel, then you need explicit control over the iterator or index
variable, so that all iterators or index variables can be advanced in
lockstep (as demonstrated unintentionally in the buggy card and dice
examples above).
In plain English lockstep means at same time. It means you cannot iterate over more than one collection at same time using for-each. You will have to use separate iterators (
or for loop as shown by Eran) like below:
Iterator iterator1 = list1.iterator();
Iterator iterator2 = list2.iterator();
Iterator iterator3 = list3.iterator();
while (iterator1 .hasNext() && iterator2 .hasNext() && iterator3.hasNext()){
Item i1 = iterator1 .next();
Item i2 = iterator2 .next();
Item i3 = iterator3.next();
// rest of your code.
}

Java synchronization in web service

I have a java restful webservice program thats hosted on tomcat. In one of my web service methods, I load a big arraylist of objects (about 25,000 entries) from redis. This arraylist is updated once every 30 mins. There are multiple threads reading from this arraylist all the time. When, I update the arraylist I want to cause minimum disruption/delays since there could be other threads reading from it.
I was wondering what is the best way to do this? One way is to use synchronized keyword to the method that updates the list. But, the synchronized method has an overhead, since no threads can read while the update is going on. The update method itself could take few hundred millisecs since it involves reading from redis + deserialization.
class WebService {
ArrayList<Entry> list = new ArrayList<Entry>();
//need to call this every 30 mins.
void syncrhonized updateArrayList(){
//read from redis & add elements to list
}
void readFromList(){
for(Entry e: list) {
//do some processing
}
}
}
Updated the final solution:
I ended up using no explicit synchronization primitives.

Does it have to be the same List instance getting updated? Can you build a new list every 30 minutes and replace a volatile reference?
Something along these lines:
class WebService {
private volatile List<Entry> theList;
void updateList() {
List<Entry> newList = getEntriesFromRedis();
theList = Collections.unmodifiableList(newList);
}
public List<Entry> getList() {
return theList;
}
}
The advantage of this approach is that you don't have to do any other synchronization anywhere else.

A reader-writer lock (or ReadWriteLock in Java) is what you need.
A reader-writer lock will allow concurrent access for read operations, but mutually exclusive access for write.
It will look something like
class WebService {
final ReentrantReadWriteLock listRwLock = new ReentrantReadWriteLock();
ArrayList<Entry> list = new ArrayList<Entry>();
//need to call this every 30 mins.
void updateArrayList(){
listRwLock.writeLock().lock();
try {
//read from redis & add elements to list
} finally {
listRwLock.writeLock().unlock()
}
}
void readFromList(){
listRwLock.readLock().lock();
try {
for(Entry e: list) {
//do some processing
}
} finally {
listRwLock.readLock().unlock()
}
}
}

Here is the solution I finally ended up with,
class WebService {
// key = timeWindow (for ex:10:00 or 10:30 or 11:00), value = <List of entries for that timewindow>
ConcurrentHashMap<String, List<Entry>> map= new ConcurrentHashMap<String, List<Entry>>();
//have setup a timer to call this every 10 mins.
void updateArrayList(){
// populate the map for the next time window with the corresponding entries. So that its ready before we start using it. Also, clean up the expired entries for older time windows.
}
void readFromList(){
list = map.get(currentTimeWindow)
for(Entry e: list) {
//do some processing
}
}
}

ArrayList is not thread safe.. You must use vector List to make it thread safe.
You can also use Thread safe Array list by using Collections Api but I would recommend vector list since it already provides you what you want.
//Use Collecions.synzhonizedList method
List list = Collections.synchronizedList(new ArrayList());
...
//If you wanna use iterator on the synchronized list, use it
//like this. It should be in synchronized block.
synchronized (list) {
Iterator iterator = list.iterator();
while (iterator.hasNext())
...
iterator.next();
...
}
I would recommend you to through this:
http://beginnersbook.com/2013/12/difference-between-arraylist-and-vector-in-java/

ConcurrentModificationException in ArrayList<Object>

I understand that this exception is occurring because I'm trying to modify a list while I'm iterating through a list, but I don't understand how to get around it.
for(Villager e : tasked)
{
if(!e.hasTask())
{
tasked.remove(e);
}
}
The code is pretty self-explanatory. If the villager "e" does not have a task, it should be removed from this list.

Use a ListIterator, which allows you to make changes to the list through which you are iterating.
The code you use is almost indentical in use to an iterator, but removing elements has to be done explicitly using the remove() method of the iterator.
e.g.
Iterator itr = tasked.iterator();
while(itr.hasNext()) {
Villager e = itr.next();
if(!e.hasTask()) {
itr.remove();
}
}

The ConcurrentModificationException is a RuntimeException that may be thrown by methods that have detected concurrent modification of an object, when such modification is not permissible. An example of not permissible behavior is when a thread tries to modify the internal structure of a Collection, while another thread is iterating over it.
Use Iterator's remove method.
Iterator<Village> villageItr = tasked.iterator();
while(villageItr.hasNext()){
Village e=villageItr.next();
if(!e.hasTask()){
//remove that element from collection
villageItr.remove();
}
}

Create separate list (e.g. itemsToRemove), that contains all items you want to remove and then use
tasked.removeAll(itemsToRemoveList)
e.g.
List<Villager> toRemove = new ArrayList();
for(Villager e : tasked)
{
if(!e.hasTask())
{
toRemove.add(e);
}
}
tasked.removeAll(toRemove);
The Only issue with this approach, if the 'toRemove' list size is very large, you can do the opposite 'ls.retainAll(c)', by only identify what you want to keep.

If you are using java8 you can filter the list using stream API instead of iterating over it by hand.
List<Villager> villagersWithTask = tasked.stream().filter(e -> e.hasTask()).collect(Collectors.toList());

What is the difference in behavior between these two usages of synchronized on a list

List<String> list = new ArrayList<String>();
list.add("a");
...
list.add("z");
synchronized(list) {
Iterator<String> i = list.iterator();
while(i.hasNext()) {
...
}
}
and
List<String> list = new ArrayList<String>();
list.add("a");
...
list.add("z");
List<String> synchronizedList = Collections.synchronizedList(list);
synchronized(synchronizedList) {
Iterator<String> i = synchronizedList.iterator();
while(i.hasNext()) {
...
}
}
Specifically, I'm not clear as to why synchronized is required in the second instance when a synchronized list provides thread-safe access to the list.

If you don't lock around the iteration, you will get a ConcurrentModificationException if another thread modifies it during the loop.
Synchronizing all of the methods doesn't prevent that in the slightest.
This (and many other things) is why Collections.synchronized* is completely useless.
You should use the classes in java.util.concurrent. (and you should think carefully about how you will guarantee you will be safe)
As a general rule of thumb:
Slapping locks around every method is not enough to make something thread-safe.
For much more information, see my blog

synchronizedList only makes each call atomic. In your case, the loop make multiple calls so between each call/iteration another thread can modify the list. If you use one of the concurrent collections, you don't have this problem.
To see how this collection differs from ArrayList.
List<String> list = new CopyOnWriteArrayList<String>();
list.addAll(Arrays.asList("a,b,c,d,e,f,g,h,z".split(",")));
for(String s: list) {
System.out.print(s+" ");
// would trigger a ConcurrentModifcationException with ArrayList
list.clear();
}
Even though the list is cleared repeatedly, it prints the following because that wa the contents when the iterator was created.
a b c d e f g h z

The second code needs to be synchronized because of the way synchronized lists are implemented. This is explained in the javadoc:
It is imperative that the user manually synchronize on the returned list when iterating over it
The main difference between the two code snippets is the effect of the add operations:
with the synchronized list, you have a visibility guarantee: other threads will see the newly added items if they call synchronizedList.get(..) for example.
with the ArrayList, other threads might not see the newly added items immediately - they might actually not ever see them.