We Keep Coding

creating custom iterator with unit tests

I am learning programming and new to this domain as i am have a mechanical background.
Yesterday I received a problem statement from prof. where he provided us an custom Iterator which is designed to iterate over given elements alternatively.
Alternate Iterator code is as following.
import java.util.Iterator;
import java.util.LinkedList;
import java.util.Queue;
public class AlternatingIterator<E> implements Iterator{
private final Queue<E> queue = new LinkedList<>();
public AlternatingIterator(Iterator<E> ... iterators) {
for(Iterator<E> iterator : iterators) {
while(iterator.hasNext())
queue.add(iterator.next());
}
}
#Override
public boolean hasNext() {
return queue.isEmpty() ? false : true;
}
#Override
public Object next() {
return queue.poll();
}
}
Now, The AlternatingIterator should alternate in order between the iterators it receives in its constructor. For example if constructed with three iterators [a,b,c], [1,2] and [x,y,z], the iterator should produce the elements in this order ‘a, 1, x, b, 2, y, c, z’
Also i have to Write unit tests for the ‘hasNext’ and ‘next’ methods.
Can we implement any other data structure than queue?
I am completely blown off and tired to understand how to solve this challenge but very confused here. If you guys can help me then i can learn important concept very quickly.
Thank you in advance and any help is appreciated.

Of course, we can use anything we can imagine. The following implementation alternates dynamically. Instead of using a queue, I store all the received iterators in an array:
import java.util.Iterator;
/**Alternates on the given iterators.*/
public class AlternatingIterator<E> implements Iterator {
/**Stores the iterators which are to be alternated on.*/
private Iterator<E>[] iterators;
/**The index of iterator, which has the next element.*/
private int nextIterator = 0;
/**Initializes a new AlternatingIterator object.
* Stores the iterators in the iterators field.
* Finds the first iterator with an available element.*/
public AlternatingIterator(Iterator<E> ... iterators) {
this.iterators = iterators;
if (!iterators[0].hasNext())
findNextIterator();
}
#Override
public boolean hasNext() {
return iterators[nextIterator].hasNext();
}
#Override
public Object next() {
E element = iterators[nextIterator].next();
findNextIterator();
return element;
}
/**Steps on iterators, until one has next element.
* It does not step on them infinitely, stops when
* the lastly used iterator is reached.*/
private void findNextIterator() {
int currentIterator = nextIterator;
// Finding iterator with element remaining.
do {
stepNextIterator();
} while (!iterators[nextIterator].hasNext() && nextIterator != currentIterator);
// If it gets around to the same iterator, then there is no iterator with element.
}
/**Increases the nextIterator value without indexing out of bounds.*/
private void stepNextIterator() {
nextIterator = (nextIterator + 1) % iterators.length;
}
}
But the same could be made statically using a queue, enqueuing all the elements from the iterators, having only that one queue as in your (your prof's) code.
#Andy Turner: Collecting the elements from the iterator results in not relying on the source collection the entire time until the last element is not obtained. Sure, from Java8 we use Streams, which won't gift us concurrent exceptions, but before Java8, buffering an/more iterator into a collection could have been safer in my opinion.
EDIT: you wrote then we can use the iterator of that given collection. Yeah, I totally forgot that, implementing an iterator for a simple queue is for sure pointless :)

A queue is helpful here, but not in the way you are using it.
There is no real point in copying all the elements from the iterators provided to the constructor into a queue, and then implementing a custom iterator from this queue: if you are going to put the elements into a collection which already implements Iterable, you may as well just use that Iterable's iterator.
But this is also probably not the point of the exercise: you can do this lazily with respect to consuming the input iterators. (Besides, what if one of the iterators is infinite...)
The idea I would suggest is to make a queue of iterators, not elements. Here is a description of how you could do it; I don't want to give you code to spoil your learning experience:
In your constructor, put the iterators from the parameter into a queue.
To implement hasNext(), pop iterators off the head of the queue for which hasNext() is false; stop when the iterator at the head of the queue has a next element (in which case return true), or the queue is empty (in which case return false).
To implement next(), pop the head iterator out of the queue, and get its next element: this is what you will return. But, before you do, if the iterator has more elements, push it onto the tail of the queue (doing this means that you will look at the next iterator on the next iteration).

Exception creating TreeSet from concurrently-modified ConcurrentSkipListSet

Generally, concurrent collections are safe to iterate; according to Javadoc: 'Iterators are weakly consistent, returning elements reflecting the state of the set at some point at or since the creation of the iterator. They do not throw ConcurrentModificationException, and may proceed concurrently with other operations.'
However, consider this:
import java.util.Random;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentSkipListSet;
public class ConcurrencyProblem {
private static volatile boolean modifierIsAlive = true;
public static void main(String[] args) {
final ConcurrentSkipListSet<Integer> concurrentSet = new ConcurrentSkipListSet<>();
Thread modifier = new Thread() {
private final Random randomGenerator = new Random();
public void run() {
while (modifierIsAlive) {
concurrentSet.add(randomGenerator.nextInt(1000));
concurrentSet.remove(randomGenerator.nextInt(1000));
}
};
};
modifier.start();
int sum = 0;
while (modifierIsAlive) {
try {
TreeSet<Integer> sortedCopy = new TreeSet<>(concurrentSet);
// make sure the copy operation is not eliminated by the compiler
sum += sortedCopy.size();
} catch (RuntimeException rte) {
modifierIsAlive = false;
rte.printStackTrace();
}
}
System.out.println("Dummy output: " + sum);
}
}
The output is
java.util.NoSuchElementException
at java.util.concurrent.ConcurrentSkipListMap$Iter.advance(ConcurrentSkipListMap.java:2299)
at java.util.concurrent.ConcurrentSkipListMap$KeyIterator.next(ConcurrentSkipListMap.java:2334)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2559)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2547)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2579)
at java.util.TreeMap.buildFromSorted(TreeMap.java:2504)
at java.util.TreeMap.addAllForTreeSet(TreeMap.java:2462)
at java.util.TreeSet.addAll(TreeSet.java:308)
at java.util.TreeSet.<init>(TreeSet.java:172)
at mtbug.ConcurrencyProblem.main(ConcurrencyProblem.java:27)
Dummy output: 44910
I'm wondering if this is a bug or a feature; we did not get a ConcurrentModificationException, but still, having to care about iteration (falling back to synchronized blocks or otherwise) kind of defeats the purpose of ConcurrentSkipListSet/Map. I've been able to reproduce this both with Java 7 and 8 (currently, 8u72 on my Linux box).

As far as I can understand from browsing the sources, the problem with TreeSet is that it calls size() before iterating and then uses it instead of calling hasNext(). This may be a bug, but I think it's just a consequence of red-black trees being complicated structures requiring careful balancing, and therefore knowing size in advance is needed to properly balance it in linear time during creation.
You may circumvent this by iterating manually and adding elements to the TreeSet, but this will lead to n log n complexity, which could be the reason why TreeSet's constructor doesn't do it that way (its API spec guarantees linear time). Of course it could still call hasNext() as it builds the tree, but then some additional actions may be required after the construction is finished to rebalance the tree, which could lead to amortized linear complexity. But red-black trees are a mess as they are, and that kind of hack would make the implementation even messier.
Still, I think it's very confusing and should probably be documented somewhere in the API docs, but I'm not sure where exactly. Probably in the part where they explain what weakly consistent iterators are. Specifically, it should be mentioned that some library classes rely on the returned size and therefore may throw NoSuchElementException. Mentioning specific classes would also help.

I'm actually starting to lean towards this being a bug in TreeSet/TreeMap (update, it is). The issue, as Sergey alludes, is that TreeMap caches the result of ConcurrentSkipListSet.size() before reading its elements.
TreeSet.addAll() calls
TreeMap.addAllForTreeSet() and passes the collection's current size and potentially concurrent Iterator to
TreeMap.buildFromSorted() which ultimately calls Iterator.next() size-times.
In other words, it assumes the Collection it is passed will not be modified during construction, which is an erroneous assumption.
Note that even if buildFromSorted() did call Iterator.hasNext() its only option at that point would be to fail, since the backing data structure was modified mid-construction.
Looking at other collections that could potentially have some issue copying concurrent structures, including ArrayList, LinkedList, and CopyOnWriteArrayList (most other collections I looked at simply for-each over the elements), explicitly copy the provided collection to an array before doing any actual work in order to avoid this exact issue. I think TreeSet and TreeMap should be doing the same thing.
We actually don't have to accept O(n log n) performance due to this bug, but it's going to be a hack. We can't simply copy the values into an array or other data structure, because then inserting into the TreeSet won't be linear time. But we can lie to TreeSet by claiming the copy is a SortedSet.
public static class IterateOnlySortedSet<E>
extends AbstractSet<E> implements SortedSet<E> {
private final ArrayList<E> elements;
private final Comparator<? super E> comparator;
public IterateOnlySortedSet(SortedSet<E> source) {
elements = new ArrayList<>(source);
comparator = source.comparator();
}
#Override
public Iterator<E> iterator() {
return elements.iterator();
}
#Override
public int size() {
return elements.size();
}
#Override
public Comparator<? super E> comparator() {
return comparator;
}
// remaining methods simply throw UnsupportedOperationException
}
Changing your TreeSet construction line to:
TreeSet<Integer> sortedCopy = new TreeSet<>(new IterateOnlySortedSet<>(concurrentSet));
Now succeeds.
Nice find :)

How does modifying an iterator actually change the list it's iterating over? [duplicate]

We all know that the safest "and probably only safe" way of removing an object from a collection while iterating it, is by first retrieving the Iterator, perform a loop and remove when needed;
Iterator iter=Collection.iterator();
while(iter.hasNext()){
Object o=iter.next()
if(o.equals(what i'm looking for)){
iter.remove();
}
}
What I would like to understand, and unfortunately haven't found a deep technical explanation about, is how this removal is performed,
If:
for(Object o:myCollection().getObjects()){
if(o.equals(what i'm looking for)){
myCollection.remove(o);
}
}
Will throw a ConcurrentModificationException, what does "in technical terms" Iterator.remove() do? Does it removes the object, breaks the loop and restart the loop?
I see in the official documentation:
"Removes the current element. Throws IllegalStateException if an
attempt is made to call remove() that is not preceded by a call to
next( )."
The part "removes the current element", makes me think of the exact same situation happening in a "regular" loop => (perform equality test and remove if needed), but why is the Iterator loop ConcurrentModification-safe?

The reason why you cannot modify a list while iterating over it is because the iterator has to know what to return for hasNext() and next().
How this is done is implementation specific, but you could have a look at the source code of ArrayList/AbstractList/LinkedList etc.
Also note that in some situations you can use some code like this as an alternative:
List<Foo> copyList = new ArrayList<>(origList);
for (Foo foo : copyList){
if (condition){
origList.remove(foo);
}
}
But this code will probably run slightly slower because the collection has to be copied (shallow copy only) and the element to remove has to be searched.
Also note that if you're using the iterator directly it's recommended to use a for loop instead of while loop as this limits the scope of the variable:
for (Iterator<Foo> iterator = myCollection.iterator(); iterator.hasNext();){
...
}

How exactly Iterator removes elements depends on its implementation, which may be different for different Collections. Definitely it doesn't break the loop you're in. I've just looked how ArrayList iterator is implemented and here's the code:
public void remove() {
if (lastRet < 0)
throw new IllegalStateException();
checkForComodification();
try {
ArrayList.this.remove(lastRet);
cursor = lastRet;
lastRet = -1;
expectedModCount = modCount;
} catch (IndexOutOfBoundsException ex) {
throw new ConcurrentModificationException();
}
}
So it checks for concurrent modifications, removes element using public ArrayList remove method, and increments counter of list modifications so ConcurrentModificationException won't be thrown at next iteration.

Updating a PriorityQueue when iterating it

I need to update some fixed-priority elements in a PriorityQueue based on their ID. I think it's quite a common scenario, here's an example snippet (Android 2.2):
for (Entry e : mEntries) {
if (e.getId().equals(someId)) {
e.setData(newData);
}
}
I've then made Entry "immutable" (no setter methods) so that a new Entry instance is created and returned by setData(). I modified my method into this:
for (Entry e : mEntries) {
if (e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
mEntries.remove(e);
mEntries.add(newEntry);
}
}
The code seems to work fine, but someone pointed out that modifying a queue while iterating over it is a bad idea: it may throw a ConcurrentModificationException and I'd need to add the elements I want to remove to an ArrayList and remove it later. He didn't explain why, and it looks quite an overhead to me, but I couldn't find any specific explanation on internet.
(This post is similar, but there priorities can change, which is not my case)
Can anyone clarify what's wrong with my code, how should I change it and - most of all - why?
Thanks,
Rippel
PS: Some implementation details...
PriorityQueue<Entry> mEntries = new PriorityQueue<Entry>(1, Entry.EntryComparator());
with:
public static class EntryComparator implements Comparator<Entry> {
public int compare(Entry my, Entry their) {
if (my.mPriority < their.mPriority) {
return 1;
}
else if (my.mPriority > their.mPriority) {
return -1;
}
return 0;
}
}

This code is in the Java 6 implementation of PriorityQueue:
private class Itr implements Iterator<E> {
/**
* The modCount value that the iterator believes that the backing
* Queue should have. If this expectation is violated, the iterator
* has detected concurrent modification.
*/
private int expectedModCount = modCount;
public E next() {
if(expectedModCount != modCount) {
throw new ConcurrentModificationException();
}
}
}
Now, why is this code here? If you look at the Javadoc for ConcurrentModificationException you will find that the behaviour of an iterator is undefined if modification occurs to the underlying collection before iteration completes. As such, many of the collections implement this modCount mechanism.
To fix your code
You need to ensure that you don't modify the code mid-loop. If your code is single threaded (as it appears to be) then you can simply do as your coworker suggested and copy it into a list for later inclusion. Also, the use of the Iterator.remove() method is documented to prevent ConcurrentModificationExceptions. An example:
List<Entry> toAdd = new ArrayList<Entry>();
Iterator it = mEntries.iterator();
while(it.hasNext()) {
Entry e = it.next();
if(e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
it.remove();
toAdd.add(newEntry);
}
}
mEntries.addAll(toAdd);

The Javadoc for PriorityQueue says explicitly:
"Note that this implementation is not synchronized. Multiple threads should not access a PriorityQueue instance concurrently if any of the threads modifies the list structurally. Instead, use the thread-safe PriorityBlockingQueue class."
This seems to be your case.

What's wrong in your code was already explained -- implementing iterator, which can consistently iterate through collection with intersected modification is rather hard task to do. You need to specify how to deal with removed items (will it be seen through iterator?), added items, modified items... Even if you can do it consistently it will be rather complex and unefficient implementation -- and, mostly, not very usefull, since use case "iterate without modifications" is much more common. So, java architects choose to deny modification while iterate, and most collections from Java collection API follow this, and throw ConcurrentModificationException if such modification detected.
As for your code -- for me, your just should not make items immutable. Immutability is great thing, but it should not be overused. If Entry object you use here is some kind of domain object, and you really want them to be immutable -- you can just create some kind of temporary data holder (MutableEntry) object, use it inside your algorithm, and copy data to Entry before return. From my point of view it will be best solution.

a slightly better implementation is
List<Entry> toAdd = new ArrayList<Entry>();
for (Iterator<Entry> it= mEntries.iterator();it.hasNext();) {
Entry e = it.next();
if (e.getId().equals(someId)) {
Entry newEntry = e.setData(newData);
it.remove();
toAdd.add(newEntry);
}
}
mEntries.addAll(toAdd);
this uses the remove of the iterator and a bulk add afterwards

How to get the next element of a SortedSet?

I have a SortedSet holding my ordered data.
I use the .first() method to return the first record, and pass it to another window.
When the other window finishes I get an event called, and I want to pass the next from the SortedSet to the window, so how to move to the next element?
launchWindow(this.set.first());
Then I have this:
onActivityResult(...) {
if (this.set.hasNext()) launchWindow(this.set.next());//hasNext/next doesn't exists in the current context for SortedSet
}
What options I have?

Instead of the Set you should pass the Iterator, then next consumer would just call next()

Don't you want to use an Iterator on the SortedSet?

The iterator solution:
You should probably have something like this:
class WindowLauncherClass {
SortedSet set = null;
Iterator setIterator = null;
public WindowLauncherClass(SortedSet set) {
this.set = set; // or you can copy it if that's what you need.
}
protected void launchWindow(Object item) {
// impl
}
public void onActivityResult() {
if ( setIterator != null && setIterator.hasNext() )
{
launchWindow(setIterator.next());
}
}
public void start() {
setIterator = set.iterator();
onActivityResult();
}
}
In the comments appeared the question about updates to the set. Will the iterator see it ?.
The normal answer is depends on the application requirements. In this case i don't have all the information and i'll try to guess.
until jdk 1.5 there was only one SortedSet implementstion ( TreeSet ). this had a fail fast iterator.
in jdk 6 appeared a new implementation: ConcurrentSkipListSet. The iterator for this sorted set is not a fail fast one.
If you are adding an element into the set that is "smaller" than the currently displayed element then you will not be able to see it anyway by a "good" (not fail fast) iterator. If you are adding an element "bigger" that the currently displayed element you will see it by a proper iterator.
The final solution is to actually reset the set and the iterator when a proper change is created. By using a ConcurrentSkipListSet initially you will see only the "bigger" changes and by using a TreeSet you will fail at every update.
If you afford to miss updates "smaller" than the current one then go for the jdk 6.0 and ConcurrentSkipListSet. If not than you'll have to keep track of what you displayed and rebuild a proper set with new items and undisplayed items.

Unless you're using some SortedSet from a third-party library, your set is also a NavigableSet (every SortedSet in java.util also implements NavigableSet). If you can make the event pass back the element it just finished working on, NavigableSet has a method higher which will get the next element higher than the one you pass in:
public void onActivityResult(Event event) {
Element element = event.processedElement;
Element next = set.higher(element);
if(next != null)
launchWindow(next);
}