We Keep Coding

List iterator illegal state exception [duplicate]

This question already has an answer here:
Why iterator.forEachRemaining doesnt remove element in the Consumer lambda?
(1 answer)
Closed 6 years ago.
There is this class
public class IteratorStuff {
private static final String EMPTY = "";
public static void main(String[] args) {
System.out.println("success:");
success(newCollection());
System.out.println("fail:");
fail(newCollection());
}
private static void fail(Collection<String> myCollection) {
Iterator<String> iterator = myCollection.iterator();
iterator.forEachRemaining(new Consumer<String>() {
public void accept(String s) {
if (s != EMPTY)
System.out.println("string = " + s);
else
iterator.remove();
}
});
}
private static Collection<String> newCollection() {
Collection<String> myList = new LinkedList<String>();
myList.add(EMPTY);
myList.add("1");
myList.add("2");
myList.add("3");
return myList;
}
private static void success(Collection<String> myCollection) {
Iterator<String> iterator = myCollection.iterator();
while (iterator.hasNext()) {
String s = iterator.next();
if (s != EMPTY)
System.out.println("string = " + s);
else
iterator.remove();
}
}
}
It iterates over a collections of Strings and removes a particular EMPTY String and prints the others. The success(Collection) implementation works fine.
The fail one breaks with an IllegalStateException. However, it is able to get the EMPTY String from the iterator. That suggests that next() must have been called. Also, in the default forEachRemaining implementation
default void forEachRemaining(Consumer<? super E> action) {
Objects.requireNonNull(action);
while (hasNext())
action.accept(next());
}
next() is called and what ever element is passed to action.accept(). On a side not I also cannot seem to find the implementation of the Iterator returned by LinkedList.
Is this a bug? How can the first element be returned and still cause an IllegalStateException?
Also, this only happens if the first element is the EMPTY String.

For future readers: THIS ANSWER IS INCORRECT!
Even though the asker has accepted this answer as solution to their problem, it may not work [for others or in general]. Please see this answer by Andreas for a more thorough analysis of the problem.
If you look at the code for LinkedList$ListItr (the ListIterator implementation returned by LinkedList#iterator()) in GrepCode you'll see that it does not update the iterator itself, it starts from the current element and does the iteration using local variables.
This means that the iterator itself, on which you never called next() is invalid. Even if you did call next() prior to entering the loop, it would remove the wrong element(s), and also probably cause ConcurrentModificationException as its position is not updated by forEachRemaining() and item removal would interfere with the iterator.
<soapbox>
For any question about the Java libraries not resolvable from the Javadoc, GrepCode is the go-to resource. Use it.
</soapbox>

The problem is that you're using a LinkedList, and it has it's own flawed implementation of forEachRemaining().
Source:
public void forEachRemaining(Consumer<? super E> action) {
Objects.requireNonNull(action);
while (modCount == expectedModCount && nextIndex < size) {
action.accept(next.item);
lastReturned = next;
next = next.next;
nextIndex++;
}
checkForComodification();
}
With the default implementation, the accept() method wouldn't be called until after next() returns.
public E next() {
checkForComodification();
if (!hasNext())
throw new NoSuchElementException();
lastReturned = next;
next = next.next;
nextIndex++;
return lastReturned.item;
}
Since remove() checks the value of lastReturned, that value needs to be set before called accept().
public void remove() {
checkForComodification();
if (lastReturned == null)
throw new IllegalStateException();
Node<E> lastNext = lastReturned.next;
unlink(lastReturned);
if (next == lastReturned)
next = lastNext;
else
nextIndex--;
lastReturned = null;
expectedModCount++;
}
As already mentioned, the forEachRemaining() implementation is bugged. It should be:
public void forEachRemaining(Consumer<? super E> action) {
Objects.requireNonNull(action);
while (modCount == expectedModCount && nextIndex < size) {
lastReturned = next;
next = next.next;
nextIndex++;
action.accept(lastReturned.item);
}
checkForComodification();
}
File a bug!
Update
ArrayList$Itr.forEachRemaining() has a similar problem, were cursor and lastRet is not set during iteration, so although javadoc of forEachRemaining() doesn't explicitly say you cannot use Iterator.remove() or ListIterator.add(), the current implementations obviously didn't expect that you would.
They don't even fail or guard in a consistent manner, so they are not consistent with normal fail-fast policy.
So perhaps filing a bug for documentation and/or fail fast logic would be more appropriate.

The problem might be -
you are working on iterator and concurrently modifying it.
private static void fail(Collection<String> myCollection) {
Iterator<String> iterator = myCollection.iterator();
iterator.forEachRemaining(new Consumer<String>() {
public void accept(String s) {
if (s != EMPTY)
System.out.println("string = " + s);
else
iterator.remove();
}
});
}
you are calling forEachRemaining method using iterator object and inside that you are also removing the object from same iterator.

Implementing an Iterator for a Set, tracking the current element

How do you code an Iterator for a Set? Given that the iterator does not have access to the underlying data storage mechanism, and can only use the Set methods, is it possible to do this?
Every implementation I've managed to find creates the Iterator as an anonymous class; however, I am trying to figure out if there is a clever way to iterate over a Set while only accessing the methods provided by Set.
So far, the best I've managed to come up with looks like this:
import java.util.*;
public class SetIterator<E> implements Iterator
{
protected E[] arrayData;
protected Set<E> set;
protected int index;
protected boolean canRemove;
public SetIterator(Set<E> set)
{
this.set = set;
this.arrayData = (E[]) set.toArray();
this.index = -1;
this.canRemove = false;
}
public E next()
{
if(this.hasNext())
{
this.canRemove = true;
return this.arrayData[++this.index];
}
else
{
throw new NoSuchElementException("There is no next element");
}
}
public boolean hasNext()
{
return this.index + 1 < this.arrayData.length;
}
public void remove()
{
if(this.canRemove)
{
this.set.remove(this.arrayData[index--]);
this.arrayData = (E[]) this.set.toArray();
this.canRemove = false;
}
else
{
throw new IllegalStateException("Cannot remove element before calling next");
}
}
}
But that feels quite kludgy.... Is there a better way?

I think your title doesn't leave much space for answers, but if I use the following as your actual question:
How do you build an Iterator for a Set?
(and understand build as in get an instance of)
I think as PM 77-1 pointed out in the comments:
call the iterator() method on it, which it has since at least Java 1.5.
Keep in mind that it depends on the actual implementation of Set, wether the elements will always be iterated over in the same order.

if we look in AbstractCollection we will see that toArray actually calls the iterator() (abstract method) to produce the array which you will use, so your method still depends on the specific iterator, so you are essentially decorating the iterator.
public Object[] toArray() {
// Estimate size of array; be prepared to see more or fewer elements
Object[] r = new Object[size()];
Iterator<E> it = iterator();
for (int i = 0; i < r.length; i++) {
if (! it.hasNext()) // fewer elements than expected
return Arrays.copyOf(r, i);
r[i] = it.next();
}
return it.hasNext() ? finishToArray(r, it) : r;
}
Still not sure what you are trying to accomplish, the underlying datastructure of the set will have different (and specific) ways to efficently iterate the data, any generic solution would sacrafice performance, using the iterable interface should be generic enough.

Homemade Stack Equals method

For my data structures class, we have to create our own Stack data type and the implementation for it as a project. The problem I'm running into is when the professor asked us to implement an equals(Object object) method. Heres what I have so far...
package stack;
import list.*;
public class Stack <E>
implements StackADT<E>//the interface
{
List <E> values;
public Stack()
{
values = new ArrayList<E>();
}
public E push(E value)
{
values.add(value);
return value;
}
public E pop()
{
return values.remove(values.size()-1);
}
public E peek()
{
return values.get(values.size()-1);
}
/** #return true only if this Stack is empty */
public boolean isEmpty()
{
return (values.size()==0);
}
/** Clear this stack, to make it an empty stack */
public void clear()
{
for (int i = 0; i < values.size()-1; i++)
{
pop();
}
}
public String toString()
{
String result = "[";
for (int i = 0; i<values.size(); i++)
{
if (i == values.size()-1)
{
result = result + values.get(i);
}
else
{
result = result + values.get(i) +",";
}
}
result = result + "]";
return result;
}
public boolean equals (Object object)
{
if (!(object instanceof StackADT))
{
return false;
}
StackADT <E> otherStack = new Stack<E>();
for(Object o: object)//heres where i run into trouble
{
otherStack.push(o);
}
for (int i=0;i<values.size()-1;i++)
{
if (!(values.get(i).equals(otherStack.pop())))
{
return false;
}
}
return true;
}
}
Our Stack is pretty much an ArrayList which we also built in our class. the problem is, I cant add the Object object into a stack because its not something thats iteratable(?able to be iterated over). Is there a better way to do this? I would think a get() would work, since the Stack I create is an ArrayList, but whenever I use get() on otherStack, it can't find the method. I had a temporary solution when I tried casting object as a stack(I hope im using the right terminology). It looked something like this
Stack otherStack = (Stack) object;
for (int i=0;i<values.size()-1;i++)
{
if (!(values.get(i).equals(otherStack.pop())))
{
return false;
}
}
return true;
}
this seemed to work, but when pop() was called on otherStack, the values in the original list(the one that becomes otherStack) that was passed into the equals() method we're also popped from the original list, leading to an incorrect result. Is there a better way to do this without adding in any other methods? I'm trying to stick as close to the formula set up by my professor as possible, so I dont want to add any extra fields or methods.
any and all help is appreciated

An equals method is not supposed to create anything, not even a temporary object. Rather than creating a new otherStack, cast the object that you have checked to be StackADT, like this:
// This should be the first line of any equals() implementation:
if (object == this) {
return true;
}
// You've got this part right: you need to check the other object's type
if (!(object instanceof StackADT)) {
return false;
}
// Now that you know the type, cast the other object to StackADT<E>
StackADT<E> otherStack = (StackADT<E>)object;
// The next step is to check the sizes:
if (values.size() != otherStack.values.size()) {
return false;
}
// Finally, go through the individual elements in a loop
In the loop that follows, do not pop the other stack. Do not do anything that can modify it. Simply go through the underlying storage (i.e. values), and check elements one by one.
Don't forget to override hashCode as well: you need to do it every time when you override equals for the object to fulfill the contract specified by java.lang.Object.

Deadlock in acquiring multiple locks

I have a following code snippet (The code is in Java, but I have tried to reduce as much clutter as possible):
class State {
public synchronized read() {
}
public synchronized write(ResourceManager rm) {
rm.request();
}
public synchronized returnResource() {
}
}
State st1 = new State();
State st2 = new State();
State st3 = new State();
class ResourceManager {
public syncronized request() {
st2 = findIdleState();
return st2.returnResource();
}
}
ResourceManager globalRM = new ResourceManager();
Thread1()
{
st1.write(globalRM);
}
Thread2()
{
st2.write(globalRM);
}
Thread3()
{
st1.read();
}
This code snippet has the possibility of entering a deadlock with the following sequence of calls:
Thread1: st1.write()
Thread1: st1.write() invokes globalRM.request()
Thread2: st2.write()
Thread1: globalRM.request() tries to invoke st2.returnResource(), but gets blocked because Thread2 is holding a lock on st2.
Thread2: st2.write() tries to invoke globalRM.request(), but gets blocked because globalRM's lock is with Thread1
Thread3: st2.read(), gets blocked.
How do I solve such a deadlock? I thought about it for a while to see there is some sort of ordered locks approach I can use to acquire the locks, but I cannot think of such a solution. The problem is that, the resource manager is global, while states are specific to each job (each job has an ID which is sequential which can be used for ordering if there is some way to use order for lock acquisition).

There are some options to avoid this scenario, each has its advantages and drawbacks:
1.) Use a single lock object for all instances. This approach is simple to implement, but limits you to one thread to aquire the lock. This can be reasonable if the synchronized blocks are short and scalability is not a big issue (e.g. desktop application aka non-server). The main selling point of this is the simplicity in implementation.
2.) Use ordered locking - this means whenever you have to aquire two or more locks, ensure that the order in which they are aquired is the same. Thats much easier said then done and can require heavy changes to the code base.
3.) Get rid of the locks completely. With the java.util.concurrent(.atomic) classes you can implement multithreaded data structures without blocking (usually using compareAndSet-flavor methods). This certainly requires changes to the code base and requires some rethinking of the structures. Usually reqiures a rewrite of critical portions of the code base.
4.) Many problems just disappear when you consequently use immutable types and objects. Combines well with the atomic (3.) approach to implement mutable super-structures (often implemented as copy-on-change).
To give any recommendation one would need to know a lot more details about what is protected by your locks.
--- EDIT ---
I needed a lock-free Set implementation, this code sample illustrates it strengths and weaknesses. I did implement iterator() as a snapshot, implementing it to throw ConcurrentModificationException and support remove() would be a little more complicated and I had no need for it. Some of the referenced utility classes I did not post (I think its completely obvious what the missing referenced pieces do).
I hope its at least a little useful as a starting point how to work with AtomicReferences.
/**
* Helper class that implements a set-like data structure
* with atomic add/remove capability.
*
* Iteration occurs always on a current snapshot, thus
* the iterator will not support remove, but also never
* throw ConcurrentModificationException.
*
* Iteration and reading the set is cheap, altering the set
* is expensive.
*/
public final class AtomicArraySet<T> extends AbstractSet<T> {
protected final AtomicReference<Object[]> reference =
new AtomicReference<Object[]>(Primitives.EMPTY_OBJECT_ARRAY);
public AtomicArraySet() {
}
/**
* Checks if the set contains the element.
*/
#Override
public boolean contains(final Object object) {
final Object[] array = reference.get();
for (final Object element : array) {
if (element.equals(object))
return true;
}
return false;
}
/**
* Adds an element to the set. Returns true if the element was added.
*
* If element is NULL or already in the set, no change is made to the
* set and false is returned.
*/
#Override
public boolean add(final T element) {
if (element == null)
return false;
while (true) {
final Object[] expect = reference.get();
final int length = expect.length;
// determine if element is already in set
for (int i=length-1; i>=0; --i) {
if (expect[i].equals(element))
return false;
}
final Object[] update = new Object[length + 1];
System.arraycopy(expect, 0, update, 0, length);
update[length] = element;
if (reference.compareAndSet(expect, update))
return true;
}
}
/**
* Adds all the given elements to the set.
* Semantically this is the same a calling add() repeatedly,
* but the whole operation is made atomic.
*/
#Override
public boolean addAll(final Collection<? extends T> collection) {
if (collection == null || collection.isEmpty())
return false;
while (true) {
boolean modified = false;
final Object[] expect = reference.get();
int length = expect.length;
Object[] temp = new Object[collection.size() + length];
System.arraycopy(expect, 0, temp, 0, length);
ELoop: for (final Object element : collection) {
if (element == null)
continue;
for (int i=0; i<length; ++i) {
if (element.equals(temp[i])) {
modified |= temp[i] != element;
temp[i] = element;
continue ELoop;
}
}
temp[length++] = element;
modified = true;
}
// check if content did not change
if (!modified)
return false;
final Object[] update;
if (temp.length == length) {
update = temp;
} else {
update = new Object[length];
System.arraycopy(temp, 0, update, 0, length);
}
if (reference.compareAndSet(expect, update))
return true;
}
}
/**
* Removes an element from the set. Returns true if the element was removed.
*
* If element is NULL not in the set, no change is made to the set and
* false is returned.
*/
#Override
public boolean remove(final Object element) {
if (element == null)
return false;
while (true) {
final Object[] expect = reference.get();
final int length = expect.length;
int i = length;
while (--i >= 0) {
if (expect[i].equals(element))
break;
}
if (i < 0)
return false;
final Object[] update;
if (length == 1) {
update = Primitives.EMPTY_OBJECT_ARRAY;
} else {
update = new Object[length - 1];
System.arraycopy(expect, 0, update, 0, i);
System.arraycopy(expect, i+1, update, i, length - i - 1);
}
if (reference.compareAndSet(expect, update))
return true;
}
}
/**
* Removes all entries from the set.
*/
#Override
public void clear() {
reference.set(Primitives.EMPTY_OBJECT_ARRAY);
}
/**
* Gets an estimation how many elements are in the set.
* (its an estimation as it only returns the current size
* and that may change at any time).
*/
#Override
public int size() {
return reference.get().length;
}
#Override
public boolean isEmpty() {
return reference.get().length <= 0;
}
#SuppressWarnings("unchecked")
#Override
public Iterator<T> iterator() {
final Object[] array = reference.get();
return (Iterator<T>) ArrayIterator.get(array);
}
#Override
public Object[] toArray() {
final Object[] array = reference.get();
return Primitives.cloneArray(array);
}
#SuppressWarnings("unchecked")
#Override
public <U extends Object> U[] toArray(final U[] array) {
final Object[] content = reference.get();
final int length = content.length;
if (array.length < length) {
// Make a new array of a's runtime type, but my contents:
return (U[]) Arrays.copyOf(content, length, array.getClass());
}
System.arraycopy(content, 0, array, 0, length);
if (array.length > length)
array[length] = null;
return array;
}
}

The answer to any deadlock is to acquire the same locks in the same order. You'll just have to figure out a way to do that.

Using ListIterator to move back and forth over a LinkedList in Java

I have a LinkedList over which I need to iterate back and forth multiple times. I am using it to keep track of a series of pages in a workflow that will be created dynamically. This does not behave as I would expect. Given this example:
LinkedList<String> navigationCases;
navigationCases.add("page1");
navigationCases.add("page2");
navigationCases.add("page3");
navigationCases.add("page4");
ListIterator navigationItr = navigationCases.listIterator();
navigationItr.next(); // Returns page1
navigationItr.next(); // Returns page2
navigationItr.previous(); //Returns page2 again
navigationItr.next(); //Returns page2 again
I thought perhaps I was building my list incorrectly, or using the Iterator wrong, but after reading the documentation, this seems to be by design:
A ListIterator has no current element; its cursor position always lies between the element that would be returned by a call to previous() and the element that would be returned by a call to next().
And:
(Next) Returns the next element in the list. This method may be called repeatedly to iterate through the list, or intermixed with calls to previous to go back and forth. (Note that alternating calls to next and previous will return the same element repeatedly.)
So after reading this, it is clear why my code is behaving the way it does. I just don't understand why it should work this way. Even remove seems to be bending over backwards to accommodate this implementation:
Note that the remove() and set(Object) methods are not defined in terms of the cursor position; they are defined to operate on the last element returned by a call to next() or previous().
Conceptually, a LinkedList seemed to model my workflow cases pretty well, but I can't use an Iterator that behaves this way. Am I missing something here, or should I just write my own class maintain a list of cases and navigate through them?

This should do your job:
public class Main {
public static void main(String[] args) {
final LinkedList<String> list = new LinkedList<String> ();
list.add ("1"); list.add ("2"); list.add ("3"); list.add ("4");
final MyIterator<String> it = new MyIterator (list.listIterator());
System.out.println(it.next());
System.out.println(it.next ());
System.out.println(it.next ());
System.out.println(it.previous ());
System.out.println(it.previous ());
System.out.println(it.next ());
}
public static class MyIterator<T> {
private final ListIterator<T> listIterator;
private boolean nextWasCalled = false;
private boolean previousWasCalled = false;
public MyIterator(ListIterator<T> listIterator) {
this.listIterator = listIterator;
}
public T next() {
nextWasCalled = true;
if (previousWasCalled) {
previousWasCalled = false;
listIterator.next ();
}
return listIterator.next ();
}
public T previous() {
if (nextWasCalled) {
listIterator.previous();
nextWasCalled = false;
}
previousWasCalled = true;
return listIterator.previous();
}
}
}
And a fiddle for it.

ListIterator was designed to behave this way. See the conversation beneath ShyJ's answer for the rationale.
I find this behavior to be beyond idiotic, and have instead written a very simple alternative. Here's the Kotlin code with a extension function for ArrayLists:
class ListIterator<E>(var list: ArrayList<E>) : Iterator<E> {
private var cursor: Int = 0
fun replace(newList: ArrayList<E>) {
list = newList
cursor = 0
}
override fun hasNext(): Boolean {
return cursor + 1 < list.size
}
override fun next(): E {
cursor++
return current()
}
fun hasPrevious(): Boolean {
return 0 <= cursor - 1
}
fun previous(): E {
cursor--
return current()
}
fun current(): E {
return list[cursor]
}
}
fun <E> ArrayList<E>.listFlippingIterator() = ListIterator(this)
If you wish to include removal functionality, I highly recommend writing the API to explicitly instruct the iterator if it should remove left or right, e.g. by defining those methods as removeNext() and removePrevious().

Do something like this (pseudocode) --
class SkipIterator extends ListIterator {
public E previous(){
E n = super.previous();
return super.previous();
}
...
}
then:
LinkedList<String> navigationCases;
navigationCases.add("page1");
navigationCases.add("page2");
navigationCases.add("page3");
navigationCases.add("page4");
SkipIterator navigationItr = (SkipIterator)navigationCases.listIterator();
navigationItr.next(); // Returns page1
navigationItr.next(); // Returns page2
navigationItr.previous(); // Returns page1
Cheers