We Keep Coding

Iterating over a composite

In the Head First Design Patterns book, the authors describe using an iterator to traverse over composite data structures. They provide some sample code which, when executed, prints out a series of menu items stored within the composite. However, if you try to call the iterator more than once, it no longer works as expected and won't produce any results. The following code appears to be causing the problem:
public Iterator<MenuComponent> createIterator() {
if (iterator == null) {
iterator = new CompositeIterator(menuComponents.iterator());
}
return iterator;
}
In essence, they are creating a singleton iterator that cannot be reset for future iterations. Unfortunately, simply replacing this logic to return a new instance of the CompositeIterator also breaks the algorithm. An issue was raised on GitHub several years ago, although is yet to be resolved. Does anyone have any suggestions on how to overcome this issue?

As the linked issue says in the comments:
return iterator; // the `iterator' never resets to null once it's set.
We need to reset the iterator we are done with it, but not when the iterator still has elements left, because CompositeIterator depends on that.
One way to do this is to add another condition on which iterator is reset - when the iterator has no more elements:
public Iterator<MenuComponent> createIterator() {
if (iterator == null || !iterator.hasNext()) {
iterator = new CompositeIterator(menuComponents.iterator());
}
return iterator;
}

thread-safe CopyOnWriteArrayList reverse iteration

Consider the following code snippet:
private List<Listener<E>> listenerList = new CopyOnWriteArrayList<Listener<E>>();
public void addListener(Listener<E> listener) {
if (listener != null) {
listenerList.add(listener);
}
}
public void removeListener(Listener<E> listener) {
if (listener != null) {
listenerList.remove(listener);
}
}
protected final void fireChangedForward(Event<E> event) {
for (Listener<E> listener : listenerList) {
listener.changed(event);
}
}
protected final void fireChangedReversed(Event<E> event) {
final ListIterator<Listener<E>> li = listenerList.listIterator(listenerList.size());
while (li.hasPrevious()) {
li.previous().changed(event);
}
}
There is a listener list that can be modified and iterated.
I think the forward iteration (see method #fireChangedForward)
should be safe.
The question is: is the reverse iteration (see method #fireChangedReversed) also safe in a multi-threaded environment?
I doubt that, because there are two calls involved: #size and #listIterator.
If it's not thread-safe, what is the most efficient way to implement #fireChangedReversed under the following circumstances:
optimize for traversal
avoid usage of locking if possible
avoid usage of javax.swing.event.EventListenerList
prefer solution without usage of third-party lib, e.g. implementation in own code possible

Indeed, listenerList.listIterator(listenerList.size()) is not thread-safe, for exactly the reason you suggested: the list could change size between the calls to size() and listIterator(), resulting in either the omission of an element from the iteration, or IndexOutOfBoundsException being thrown.
The best way to deal with this is to clone the CopyOnWriteArrayList before getting the iterator:
CopyOnWriteArrayList<Listener<E>> listenerList = ... ;
#SuppressWarnings("unchecked")
List<Listener<E>> copy = (List<Listener<E>>)listenerList.clone();
ListIterator<Listener<E>> li = copy.listIterator(copy.size());
The clone makes a shallow copy of the list. In particular, the clone shares the internal array with the original. This isn't entirely obvious from the specification, which says merely
Returns a shallow copy of this list. (The elements themselves are not copied.)
(When I read this, I thought "Of course the elements aren't copied; this is a shallow copy!" What this really means is that neither the elements nor the array that contains them are copied.)
This is fairly inconvenient, including the lack of a covariant override of clone(), requiring an unchecked cast.
Some potential enhancements are discussed in JDK-6821196 and JDK-8149509. The former bug also links to a discussion of this issue on the concurrency-interest mailing list.

One simple way to do that is to call #toArray method and iterate over the array in reverse order.

You could always just get a ListIterator and "fast-forward" to the end of the list as such:
final ListIterator<Listener<E>> li = listenerList.listIterator();
if (li.hasNext()) {
do{
li.next();
} while (li.hasNext());
}
while (li.hasPrevious()) {
li.previous().changed(event);
}
EDIT I switched the quirky exception-handling of my previous answer for a do/while loop that places the cursor of the ListIterator after the last element, in order to be ready for the next previous call.
RE-EDIT As pointed out by #MikeFHay, a do/while loop on an iterator will throw a NoSuchElementException on an empty list. To prevent this from happening, I wrapped the do/while loop with if (li.hasNext()).

How does modifying an iterator actually change the list it's iterating over? [duplicate]

We all know that the safest "and probably only safe" way of removing an object from a collection while iterating it, is by first retrieving the Iterator, perform a loop and remove when needed;
Iterator iter=Collection.iterator();
while(iter.hasNext()){
Object o=iter.next()
if(o.equals(what i'm looking for)){
iter.remove();
}
}
What I would like to understand, and unfortunately haven't found a deep technical explanation about, is how this removal is performed,
If:
for(Object o:myCollection().getObjects()){
if(o.equals(what i'm looking for)){
myCollection.remove(o);
}
}
Will throw a ConcurrentModificationException, what does "in technical terms" Iterator.remove() do? Does it removes the object, breaks the loop and restart the loop?
I see in the official documentation:
"Removes the current element. Throws IllegalStateException if an
attempt is made to call remove() that is not preceded by a call to
next( )."
The part "removes the current element", makes me think of the exact same situation happening in a "regular" loop => (perform equality test and remove if needed), but why is the Iterator loop ConcurrentModification-safe?

The reason why you cannot modify a list while iterating over it is because the iterator has to know what to return for hasNext() and next().
How this is done is implementation specific, but you could have a look at the source code of ArrayList/AbstractList/LinkedList etc.
Also note that in some situations you can use some code like this as an alternative:
List<Foo> copyList = new ArrayList<>(origList);
for (Foo foo : copyList){
if (condition){
origList.remove(foo);
}
}
But this code will probably run slightly slower because the collection has to be copied (shallow copy only) and the element to remove has to be searched.
Also note that if you're using the iterator directly it's recommended to use a for loop instead of while loop as this limits the scope of the variable:
for (Iterator<Foo> iterator = myCollection.iterator(); iterator.hasNext();){
...
}

How exactly Iterator removes elements depends on its implementation, which may be different for different Collections. Definitely it doesn't break the loop you're in. I've just looked how ArrayList iterator is implemented and here's the code:
public void remove() {
if (lastRet < 0)
throw new IllegalStateException();
checkForComodification();
try {
ArrayList.this.remove(lastRet);
cursor = lastRet;
lastRet = -1;
expectedModCount = modCount;
} catch (IndexOutOfBoundsException ex) {
throw new ConcurrentModificationException();
}
}
So it checks for concurrent modifications, removes element using public ArrayList remove method, and increments counter of list modifications so ConcurrentModificationException won't be thrown at next iteration.

java generics runtime error.java.util.ConcurrentModificationException

I have the following example java generics code which I modified as per suggestion of people on StackOverflow.Now the compilation is going through.
import java.util.*;
public class GenericBox<T>
{
private List<T> tList;
private Iterator<T> itor;
public GenericBox()
{
tList = new ArrayList<T>();
itor = tList.listIterator();
}
public void insert(T element)
{
tList.add(element);
}
public T retrieve()
{
if(itor.hasNext())
{
return itor.next();
}
return null;
}
public static void main (String [] args)
{
GenericBox <String> strbox = new GenericBox<String>();
GenericBox <String> intbox = new GenericBox<String>();
strbox.insert(new String("karthik"));
strbox.insert(new String("kanchana"));
strbox.insert(new String("aditya"));
String s = strbox.retrieve();
System.out.println(s);
s = strbox.retrieve();
System.out.println(s);
s = strbox.retrieve();
System.out.println(s);
}
}
I am getting the following runtime error.
Exception in thread "main" java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
at java.util.ArrayList$Itr.next(ArrayList.java:791)
at GenericBox.retrieve(GenericBox.java:24)
at GenericBox.main(GenericBox.java:40)

This has nothing to do with generics, and everything to do with the fact that you're modifying a collection directly (in insert) and then trying to use an iterator which was created before the insertion. Don't do it.
From the docs of ArrayList:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
If you must keep the iterator around, use that to add a new value instead:
private ListIterator<T> itor; // Not just Iterator<T>; you need the add method
public void insert(T element)
{
itor.add(element);
}
It would be better not to keep the iterator around though - it's very rarely a good idea to use an iterator other than for a direct loop.

You get this exception because the list has been modified between the creation of the iterator and its usage. You should only call iterator() when you actually want to iterate over the list. Not before. And you shouldn't store the iterator in an instance field. Only in a local variable.

Updating a PriorityQueue when iterating it

I need to update some fixed-priority elements in a PriorityQueue based on their ID. I think it's quite a common scenario, here's an example snippet (Android 2.2):
for (Entry e : mEntries) {
if (e.getId().equals(someId)) {
e.setData(newData);
}
}
I've then made Entry "immutable" (no setter methods) so that a new Entry instance is created and returned by setData(). I modified my method into this:
for (Entry e : mEntries) {
if (e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
mEntries.remove(e);
mEntries.add(newEntry);
}
}
The code seems to work fine, but someone pointed out that modifying a queue while iterating over it is a bad idea: it may throw a ConcurrentModificationException and I'd need to add the elements I want to remove to an ArrayList and remove it later. He didn't explain why, and it looks quite an overhead to me, but I couldn't find any specific explanation on internet.
(This post is similar, but there priorities can change, which is not my case)
Can anyone clarify what's wrong with my code, how should I change it and - most of all - why?
Thanks,
Rippel
PS: Some implementation details...
PriorityQueue<Entry> mEntries = new PriorityQueue<Entry>(1, Entry.EntryComparator());
with:
public static class EntryComparator implements Comparator<Entry> {
public int compare(Entry my, Entry their) {
if (my.mPriority < their.mPriority) {
return 1;
}
else if (my.mPriority > their.mPriority) {
return -1;
}
return 0;
}
}

This code is in the Java 6 implementation of PriorityQueue:
private class Itr implements Iterator<E> {
/**
* The modCount value that the iterator believes that the backing
* Queue should have. If this expectation is violated, the iterator
* has detected concurrent modification.
*/
private int expectedModCount = modCount;
public E next() {
if(expectedModCount != modCount) {
throw new ConcurrentModificationException();
}
}
}
Now, why is this code here? If you look at the Javadoc for ConcurrentModificationException you will find that the behaviour of an iterator is undefined if modification occurs to the underlying collection before iteration completes. As such, many of the collections implement this modCount mechanism.
To fix your code
You need to ensure that you don't modify the code mid-loop. If your code is single threaded (as it appears to be) then you can simply do as your coworker suggested and copy it into a list for later inclusion. Also, the use of the Iterator.remove() method is documented to prevent ConcurrentModificationExceptions. An example:
List<Entry> toAdd = new ArrayList<Entry>();
Iterator it = mEntries.iterator();
while(it.hasNext()) {
Entry e = it.next();
if(e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
it.remove();
toAdd.add(newEntry);
}
}
mEntries.addAll(toAdd);

The Javadoc for PriorityQueue says explicitly:
"Note that this implementation is not synchronized. Multiple threads should not access a PriorityQueue instance concurrently if any of the threads modifies the list structurally. Instead, use the thread-safe PriorityBlockingQueue class."
This seems to be your case.

What's wrong in your code was already explained -- implementing iterator, which can consistently iterate through collection with intersected modification is rather hard task to do. You need to specify how to deal with removed items (will it be seen through iterator?), added items, modified items... Even if you can do it consistently it will be rather complex and unefficient implementation -- and, mostly, not very usefull, since use case "iterate without modifications" is much more common. So, java architects choose to deny modification while iterate, and most collections from Java collection API follow this, and throw ConcurrentModificationException if such modification detected.
As for your code -- for me, your just should not make items immutable. Immutability is great thing, but it should not be overused. If Entry object you use here is some kind of domain object, and you really want them to be immutable -- you can just create some kind of temporary data holder (MutableEntry) object, use it inside your algorithm, and copy data to Entry before return. From my point of view it will be best solution.

a slightly better implementation is
List<Entry> toAdd = new ArrayList<Entry>();
for (Iterator<Entry> it= mEntries.iterator();it.hasNext();) {
Entry e = it.next();
if (e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
it.remove();
toAdd.add(newEntry);
}
}
mEntries.addAll(toAdd);
this uses the remove of the iterator and a bulk add afterwards