We Keep Coding

How can I randomize the iteration sequence of a Set?

I need to use the Set collection.
Each time I start a jvm to run the program, I want to iterate through the items in the Set in a randomly decided sequence.
The iteration sequence has nothing to do with the sequence in which I placed them in the Set, right?
So, what to do? How can I randomize the iteration sequence in a Set?
Here is my method, and it does not randomize.
public static <T> void shuffle(Set<T> set) {
List<T> shuffleMe = new ArrayList<T>(set);
Collections.shuffle(shuffleMe);
set.clear();
set.addAll(shuffleMe);
}

What you need is a RandomizingIterator
Set is unordered, so randomizing an unordered Collection doesn't make any logical sense.
An ordered Set is ordered using a Comparator which means it has a fixed order, you can't shuffle it, that has no meaning as the order is determined by the Comparator or the compare() method.
Set -> List will allow you to shuffle the contents of the List and then use a custom RandomizingIterator to iterate across the Set.
Example Implementation :
Link to Gist on GitHub - TestRandomizingIterator.java
import org.junit.Test;
import javax.annotation.Nonnull;
import java.util.*;
public class TestRandomzingIterator
{
#Test
public void testRandomIteration()
{
final Set<String> set = new HashSet<String>()
{
/** Every call to iterator() will give a possibly unique iteration order, or not */
#Nonnull
#Override
public Iterator<String> iterator()
{
return new RandomizingIterator<String>(super.iterator());
}
class RandomizingIterator<T> implements Iterator<T>
{
final Iterator<T> iterator;
private RandomizingIterator(#Nonnull final Iterator<T> iterator)
{
List<T> list = new ArrayList<T>();
while(iterator.hasNext())
{
list.add(iterator.next());
}
Collections.shuffle(list);
this.iterator = list.iterator();
}
#Override
public boolean hasNext()
{
return this.iterator.hasNext();
}
#Override
public T next()
{
return this.iterator.next();
}
/**
* Modifying this makes no logical sense, so for simplicity sake, this implementation is Immutable.
* It could be done, but with added complexity.
*/
#Override
public void remove()
{
throw new UnsupportedOperationException("TestRandomzingIterator.RandomizingIterator.remove");
}
}
};
set.addAll(Arrays.asList("A", "B", "C"));
final Iterator<String> iterator = set.iterator();
while (iterator.hasNext())
{
System.out.println(iterator.next());
}
}
}
Notes:
This is a straw man example, but the intention is clear, use a custom Iterator to get custom iteration.
You can't get the normal iteration behavior back, but that doesn't seem to be a problem with your use case.
Passing the the super.iterator() to the facade is important, it will StackOverflowError otherwise, because it becomes a recursive call if you pass this to .addAll() or the List() constructor.
HashSet may appear to be ordered but it isn't guaranteed to stay ordered, the order depends on the hashCode of the objects and adding a single object may reorder the how the contents are order, the contract of the Set interface is that the order is undefined and in particular the HashSet is nothing more than a Facade over a backing Map.keySet().
There are other more supposedly light weight, but much more complex solutions that use the original Iterator and try and keep track of what has already been seen, those solutions aren't improvements over this technique unless the size of the data is excessively large, and the you are probably looking at on disk structures at that point.

You could copy the contents of the Set into a List, shuffle the List, then return a new LinkedHashSet populated from the shuffled list. Nice thing about LinkedHashSet is that its iterators return elements in the order they were inserted.
public static <T> Set<T> newShuffledSet(Collection<T> collection) {
List<T> shuffleMe = new ArrayList<T>(collection);
Collections.shuffle(shuffleMe);
return new LinkedHashSet<T>(shuffleMe);
}

According to the docs for java.util.Set:
The elements are returned in no particular order (unless this set is an instance of some class that provides a guarantee).
When you insert the elements there is no guarantee about the order they will be returned to you. If you want that behavior you will need to use a data structure which supports stable iteration order, e.g. List.

Internally HashSet sorts all its elements, AFAIR according to their hash() value. So you should use other classes like SortedSet with a custom comparator. But remember the whole idea of Set is to find elements quickly, that's why it sorts elements internally. So you have to keep "stability" of the comparison. Maybe you don't need a set after shuffling?

T[] toArray(T[] a) implementation

I am creating a SortedList class that implements List.
If I understand correctly, the method toArray(T[] a) takes an array of objects as a parameter and returns a sorted array of these objects.
In the java documentation we can read that if the Collection length is greater than the sortedList, a new array is created with the good size, and if the collection length is smaller than the sortedList, the object following the last object of the collection is set to null.
The project I am working on does not let me use null values in the sorted list, so I am implementing the method differently, using a new sortedList and the toArray() method:
public <T> T[] toArray(T[] a)
{
SortedList sort = new SortedList();
for(Object o : a)
{
sort.add(o);
}
return (T[])sort.toArray();
}
Would this be a good way to implement this method or should I expect errors using it like that?
Thank you for your time.

First a recommendation:
If you want SortedList to implement the List interface, it's a good idea to extend AbstractList instead of implementing List directly. AbstractList has already defined many of the necessary methods, including the one you're having problems with. Most List-implementations in the Java platform libraries also extend AbstractList.
If you still want to implement List directly, here is what the method is supposed to do:
Let a be the specified array.
If a is large enough, fill it with the elements from your SortedList (in the correct order) without caring about what was previously in a.
If there's room to spare in a after filling it, set a[size()] = null. Then the user will know where the list ends, unless the list contains null-elements.
If the list doesn't fit in a, create a new array of type T with the same size as the list, and fill the new one instead.
Return the array you filled. If you filled a, return a. If you made a new array, return the new array.
There are two reasons why this method is useful:
The array will not necessarily be of type Object, but of a type T decided by the user (as long as the type is valid).
The user may want to save memory and re-use an array instead of allocating more mamory to make a new one.
Here is how the Java Docs describe the method.

If you are implementing a "SortedList" class, it's probably in your best interest to maintain a sorted list internally, rather than relying on the toArray() method to sort them on the way out. In other words, users of the class may not use the toArray() method, but may instead use listIterator() to return an Iterator that is supposed to iterate over the elements of the list in the proper order.

Are you sure you need to implement List. It is often sufficient just to implement Iterable and Iterator.
public class SortedList<S extends Comparable<S>> implements Iterable<S>, Iterator<S> {
private final Iterator<S> i;
// Iterator version.
public SortedList(Iterator<S> iter, Comparator<S> compare) {
// Roll the whole lot into a TreeSet to sort it.
Set<S> sorted = new TreeSet<S>(compare);
while (iter.hasNext()) {
sorted.add(iter.next());
}
// Use the TreeSet iterator.
i = sorted.iterator();
}
// Provide a default simple comparator.
public SortedList(Iterator<S> iter) {
this(iter, new Comparator<S>() {
public int compare(S p1, S p2) {
return p1.compareTo(p2);
}
});
}
// Also available from an Iterable.
public SortedList(Iterable<S> iter, Comparator<S> compare) {
this(iter.iterator(), compare);
}
// Also available from an Iterable.
public SortedList(Iterable<S> iter) {
this(iter.iterator());
}
// Give them the iterator directly.
public Iterator<S> iterator() {
return i;
}
// Proxy.
public boolean hasNext() {
return i.hasNext();
}
// Proxy.
public S next() {
return i.next();
}
// Proxy.
public void remove() {
i.remove();
}
}
You can then do stuff like:
for ( String s : new SortedList<String>(list) )
which is usually all that you want because TreeSet provides your sortedness for you.

Is this an approach to create a read-only List in Java?

Thinking in java says:
To create a read-only List from AbstractList, you must implements get() and size().
It confused me, the code is below:
public class CountingIntegerList
extends AbstractList<Integer> {
private int size;
public CountingIntegerList(int size) {
this.size = size < 0 ? 0 : size;
}
public Integer get(int index) {
return Integer.valueOf(index);
}
public int size() { return size; }
public static void main(String[] args) {
List list = new CountingIntegerList(30);
}
}
Is list a read-only List? Why?
Ok, the answer is yes since I extend AbstractList and it throw UnsupportedOperationException if set or and is called. If I want to get a unmodifiableList, Collections.unmodifiableList() is a good choice. But remember, both of them are not deeply immutable:
List<StringBuilder> list = new ArrayList<StringBuilder>();
StringBuilder sb = new StringBuilder();
sb.append("hello");
list.add(sb);
System.out.println(list);
list = Collections.unmodifiableList(list);
sb.append("world");
System.out.println(list);
There is a flyweight pattern in CountingIntegerList. because everytime get() is called,it get caches from Integer, the source code of valueOf():
public static Integer valueOf(int i) {
final int offset = 128;
if (i >= -128 && i <= 127) { // must cache
return IntegerCache.cache[i + offset];
}
return new Integer(i);
}
Is is right?

It's read-only (even immutable) because add will throw an UnsupportedOperationException as will remove.
AbstractList handles all the work of creating iterators,
computing hashcodes and equality for you. It's very helpful.
It's completely unnecessary to wrap in unmodifiableList.
Later you ask whether AbstractList is used mainly to create unmodifiable lists. Actually it is used to create any kind of random-access list. In my course in data structures, we use abstract classes such as this to save most of the work of implementing a list class. Even the Collection interface has 13 methods, all but two of which are implements by AbstractCollection.
There is the related class AbstractSequentialList that helps create lists that are not random access (such as linked lists).

because set throws an UnsupportedOperationException, if not implemented. See Api

you could wrap your List in an UnmodifiableList
List readOnlyList = Collections.unmodifiableList(yourList)

Pass the ArrayList into Collections.unmodifiableList()
Returns an unmodifiable view of the specified list. This method allows
modules to provide users with "read-only" access to internal lists.
Query operations on the returned list "read through" to the specified
list, and attempts to modify the returned list, whether direct or via
its iterator, result in an UnsupportedOperationException. The returned
list will be serializable if the specified list is serializable.
Similarly, the returned list will implement RandomAccess if the
specified list does.

This is tricky a bit. The list can not change and/or the elements also? ;)
StringBuilder builder = new StringBuilder();
builder.append("Hello");
List<StringBuilder> list = new ArrayList<StringBuilder>();
list.add(builder);
Collection<StringBuilder> canNotChange = Collections.unmodifiableCollection(list);
builder.append(" World");
System.out.println(canNotChange.iterator().next());

Define a fixed-size list in Java

Is it possible to define a list with a fixed size that's 100? If not why isn't this available in Java?

This should do it if memory serves:
List<MyType> fixed = Arrays.asList(new MyType[100]);

A Java list is a collection of objects ... the elements of a list. The size of the list is the number of elements in that list. If you want that size to be fixed, that means that you cannot either add or remove elements, because adding or removing elements would violate your "fixed size" constraint.
The simplest way to implement a "fixed sized" list (if that is really what you want!) is to put the elements into an array and then Arrays.asList(array) to create the list wrapper. The wrapper will allow you to do operations like get and set, but the add and remove operations will throw exceptions.
And if you want to create a fixed-sized wrapper for an existing list, then you could use the Apache commons FixedSizeList class. But note that this wrapper can't stop something else changing the size of the original list, and if that happens the wrapped list will presumably reflect those changes.
On the other hand, if you really want a list type with a fixed limit (or limits) on its size, then you'll need to create your own List class to implement this. For example, you could create a wrapper class that implements the relevant checks in the various add / addAll and remove / removeAll / retainAll operations. (And in the iterator remove methods if they are supported.)
So why doesn't the Java Collections framework implement these? Here's why I think so:
Use-cases that need this are rare.
The use-cases where this is needed, there are different requirements on what to do when an operation tries to break the limits; e.g. throw exception, ignore operation, discard some other element to make space.
A list implementation with limits could be problematic for helper methods; e.g. Collections.sort.

FixedSizeList
Yes,
The Apache Commons library provides the FixedSizeList class which does not support the add, remove and clear methods (but the set method is allowed because it does not modify the List's size). Ditto for FixedSizeList in Eclipse Collections. If you try to call one of these methods, your list remains the same size.
To create your fixed size list, just call
List<YourType> fixed = FixedSizeList.decorate(Arrays.asList(new YourType[100]));
You can use unmodifiableList if you want an unmodifiable view of the specified list, or read-only access to internal lists.
List<YourType> unmodifiable = java.util.Collections.unmodifiableList(internalList);

Yes. You can pass a java array to Arrays.asList(Object[]).
List<String> fixedSizeList = Arrays.asList(new String[100]);
You cannot insert new Strings to the fixedSizeList (it already has 100 elements). You can only set its values like this:
fixedSizeList.set(7, "new value");
That way you have a fixed size list. The thing functions like an array and I can't think of a good reason to use it. I'd love to hear why you want your fixed size collection to be a list instead of just using an array.

Typically an alternative for fixed size Lists are Java arrays. Lists by default are allowed to grow/shrink in Java. However, that does not mean you cannot have a List of a fixed size. You'll need to do some work and create a custom implementation.
You can extend an ArrayList with custom implementations of the clear, add and remove methods.
e.g.
import java.util.ArrayList;
public class FixedSizeList<T> extends ArrayList<T> {
public FixedSizeList(int capacity) {
super(capacity);
for (int i = 0; i < capacity; i++) {
super.add(null);
}
}
public FixedSizeList(T[] initialElements) {
super(initialElements.length);
for (T loopElement : initialElements) {
super.add(loopElement);
}
}
#Override
public void clear() {
throw new UnsupportedOperationException("Elements may not be cleared from a fixed size List.");
}
#Override
public boolean add(T o) {
throw new UnsupportedOperationException("Elements may not be added to a fixed size List, use set() instead.");
}
#Override
public void add(int index, T element) {
throw new UnsupportedOperationException("Elements may not be added to a fixed size List, use set() instead.");
}
#Override
public T remove(int index) {
throw new UnsupportedOperationException("Elements may not be removed from a fixed size List.");
}
#Override
public boolean remove(Object o) {
throw new UnsupportedOperationException("Elements may not be removed from a fixed size List.");
}
#Override
protected void removeRange(int fromIndex, int toIndex) {
throw new UnsupportedOperationException("Elements may not be removed from a fixed size List.");
}
}

Create an array of size 100. If you need the List interface, then call Arrays.asList on it. It'll return a fixed-size list backed by the array.

If you want some flexibility, create a class that watches the size of the list.
Here's a simple example. You would need to override all the methods that change the state of the list.
public class LimitedArrayList<T> extends ArrayList<T>{
private int limit;
public LimitedArrayList(int limit){
this.limit = limit;
}
#Override
public void add(T item){
if (this.size() > limit)
throw new ListTooLargeException();
super.add(item);
}
// ... similarly for other methods that may add new elements ...

You can define a generic function like this:
#SuppressWarnings("unchecked")
public static <T> List<T> newFixedSizeList(int size) {
return (List<T>)Arrays.asList(new Object[size]);
}
And
List<String> s = newFixedSizeList(3); // All elements are initialized to null
s.set(0, "zero");
s.add("three"); // throws java.lang.UnsupportedOperationException

This should work pretty nicely. It will never grow beyond the initial size. The toList method will give you the entries in the correct chronological order. This was done in groovy - but converting it to java proper should be pretty easy.
static class FixedSizeCircularReference<T> {
T[] entries
FixedSizeCircularReference(int size) {
this.entries = new Object[size] as T[]
this.size = size
}
int cur = 0
int size
void add(T entry) {
entries[cur++] = entry
if (cur >= size) {
cur = 0
}
}
List<T> asList() {
List<T> list = new ArrayList<>()
int oldest = (cur == size - 1) ? 0 : cur
for (int i = 0; i < this.entries.length; i++) {
def e = this.entries[oldest + i < size ? oldest + i : oldest + i - size]
if (e) list.add(e)
}
return list
}
}
FixedSizeCircularReference<String> latestEntries = new FixedSizeCircularReference(100)
latestEntries.add('message 1')
// .....
latestEntries.add('message 1000')
latestEntries.asList() //Returns list of '100' messages

If you want to use ArrayList or LinkedList, it seems that the answer is no. Although there are some classes in java that you can set them fixed size, like PriorityQueue, ArrayList and LinkedList can't, because there is no constructor for these two to specify capacity.
If you want to stick to ArrayList/LinkedList, one easy solution is to check the size manually each time.
public void fixedAdd(List<Integer> list, int val, int size) {
list.add(val);
if(list.size() > size) list.remove(0);
}
LinkedList is better than ArrayList in this situation. Suppose there are many values to be added but the list size is quite samll, there will be many remove operations. The reason is that the cost of removing from ArrayList is O(N), but only O(1) for LinkedList.

The public java.util.List subclasses of the JDK don't provide a fixed size feature that doesn't make part of the List specification.
You could find it only in Queue subclasses (for example ArrayBlockingQueue, a bounded blocking queue backed by an array for example) that handle very specific requirements.
In Java, with a List type, you could implement it according to two scenarios :
1) The fixed list size is always both the actual and the maximum size.
It sounds as an array definition. So Arrays.asList() that returns a fixed-size list backed by the specified array is what you are looking for. And as with an array you can neither increase nor decrease its size but only changing its content. So adding and removing operation are not supported.
For example :
Foo[] foosInput= ...;
List<Foo> foos = Arrays.asList(foosInput);
foos.add(new Foo()); // throws an Exception
foos.remove(new Foo()); // throws an Exception
It works also with a collection as input while first we convert it into an array :
Collection<Foo> foosInput= ...;
List<Foo> foos = Arrays.asList(foosInput.toArray(Foo[]::new)); // Java 11 way
// Or
List<Foo> foos = Arrays.asList(foosInput.stream().toArray(Foo[]::new)); // Java 8 way
2) The list content is not known as soon as its creation. So you mean by fixed size list its maximum size.
You could use inheritance (extends ArrayList) but you should favor composition over that since it allows you to not couple your class with the implementation details of this implementation and provides also flexibility about the implementation of the decorated/composed.
With Guava Forwarding classes you could do :
import com.google.common.collect.ForwardingList;
public class FixedSizeList<T> extends ForwardingList<T> {
private final List<T> delegate;
private final int maxSize;
public FixedSizeList(List<T> delegate, int maxSize) {
this.delegate = delegate;
this.maxSize = maxSize;
}
#Override protected List<T> delegate() {
return delegate;
}
#Override public boolean add(T element) {
assertMaxSizeNotReached(1);
return super.add(element);
}
#Override public void add(int index, T element) {
assertMaxSizeNotReached(1);
super.add(index, element);
}
#Override public boolean addAll(Collection<? extends T> collection) {
assertMaxSizeNotReached(collection.size());
return super.addAll(collection);
}
#Override public boolean addAll(int index, Collection<? extends T> elements) {
assertMaxSizeNotReached(elements.size());
return super.addAll(index, elements);
}
private void assertMaxSizeNotReached(int size) {
if (delegate.size() + size >= maxSize) {
throw new RuntimeException("size max reached");
}
}
}
And use it :
List<String> fixedSizeList = new FixedSizeList<>(new ArrayList<>(), 3);
fixedSizeList.addAll(Arrays.asList("1", "2", "3"));
fixedSizeList.add("4"); // throws an Exception
Note that with composition, you could use it with any List implementation :
List<String> fixedSizeList = new FixedSizeList<>(new LinkedList<>(), 3);
//...
Which is not possible with inheritance.

You need either of the following depending on the type of the container of T elements you pass to the builder (Collection<T> or T[]):
In case of an existing Collection<T> YOUR_COLLECTION:
Collections.unmodifiableList(new ArrayList<>(YOUR_COLLECTION));
In case of an existing T[] YOUR_ARRAY:
Arrays.asList(YOUR_ARRAY);
Simple as that

To get a fixed-size list, you can simply use the Stream API. This will result in a fixed-size list :
List<Integer> list = Arrays.stream(new int[100])
.boxed()
.collect(Collectors.toList());
Or the old-fashioned way, This will result in a fixed-size list that is backed by the specified array:
List<Integer> list = Arrays.asList(new Integer[100]);

Yes is posible:
List<Integer> myArrayList = new ArrayList<>(100);
now, the initial capacity of myArrayList will be 100

Java: Why does calling `remove()` on a List throw UnsupportedOperation exception?

For some reason, I'm getting an UnsupportedOpeationException with the following code. Examining it in the debugger, it looks like the object I'm calling remove() on is a list.
// to optimize, remove totalSize. After taking an item from lowest, if lowest is empty, remove it from `lists`
// lists are sorted to begin with
public static <T extends Comparable<? super T>> List<T> merge(Set<List<T>> lists) {
List<T> result = new ArrayList<T>();
HashMap<List<T>, Integer> location = new HashMap<List<T>, Integer>();
int totalSize = 0; // every element in the set
for (List<T> l : lists) {
location.put(l, 0);
totalSize += l.size();
}
boolean first;
List<T> lowest = lists.iterator().next(); // the list with the lowest item to add
int index;
while (result.size() < totalSize) { // while we still have something to add
first = true;
for (List<T> l : lists) {
if (! l.isEmpty()) {
if (first) {
lowest = l;
}
else if (l.get(location.get(l)).compareTo(lowest.get(location.get(lowest))) <= 0) {
lowest = l;
}
}
}
index = location.get(lowest);
result.add(lowest.get(index));
lowest.remove(index); //problem here
}
return result;
}
The exception:
Exception in thread "main" java.lang.UnsupportedOperationException
at java.util.AbstractList.remove(Unknown Source)
at interview.questions.MergeLists.merge(MergeLists.java:72)
at interview.questions.MergeLists.main(MergeLists.java:32)
Why is this happening?

It's quite possible the underlying implementation of List you received is fixed-length, such as one created by Arrays#asList.

If you look at the API docs for the List interface you will see that a number of them are "optional operations". That means that a concrete class is permitted to throw the UnsupportedOperationException.
If, for example, the List was converted to an unmodifiable list it could not allow the remove operation to actually remove something (or the list would be modified).
So for the Set< List<>> part of the code one or mnre of the lists does not allow you to remove from it.

If you are going to be removing items from a List, then rather than use a for-each loop to iterate through the list, you should be using a ListIterator, which supports remove() in a safe manner (i.e. without leaving holes in the list or an index pointing to nowhere).

It is optional for a class implementing the Collection interface to allow objects to be removed (see Collection#remove() which is an optional operation). As stated in the javadoc, it throws
UnsupportedOperationException - if the remove operation is not supported by this collection
You are likely in that case (e.g. if your set contains a list returned by Arrays.asList as pointed out by Jeffrey).

Could it be that the Lists you pass in the set are derived from AbstractList and do not implement (support) the remove() method?
Also, it seems that location always maps to 0 for all list object that are mapped in the location HashMap?

The remove() method in the Collection interface is explicitly specified as an optional operation:
remove(Object o)
Removes a single instance of the specified element from this collection, if it is present (optional operation).
Lists do not have to support in. In fact, there is no clear semantics for it. Lists are not meant for that sort of random-access. Rather than provide some default implementation that may be inefficient or inaccurate, you get the exception.
You can write your own utility method with a for-each loop to do this if it is critical.