We Keep Coding

Why does Stream#toList's default implementation seem overcomplicated / suboptimal?

Looking at the implementation for Stream#toList, I just noticed how overcomplicated and suboptimal it seemed.
Like mentioned in the javadoc just above, this default implementation is not used by most Stream implementation, however, it could have been otherwise in my opinion.
The sources
/**
* Accumulates the elements of this stream into a {#code List}. The elements in
* the list will be in this stream's encounter order, if one exists. The returned List
* is unmodifiable; calls to any mutator method will always cause
* {#code UnsupportedOperationException} to be thrown. There are no
* guarantees on the implementation type or serializability of the returned List.
*
* <p>The returned instance may be value-based.
* Callers should make no assumptions about the identity of the returned instances.
* Identity-sensitive operations on these instances (reference equality ({#code ==}),
* identity hash code, and synchronization) are unreliable and should be avoided.
*
* <p>This is a terminal operation.
*
* #apiNote If more control over the returned object is required, use
* {#link Collectors#toCollection(Supplier)}.
*
* #implSpec The implementation in this interface returns a List produced as if by the following:
* <pre>{#code
* Collections.unmodifiableList(new ArrayList<>(Arrays.asList(this.toArray())))
* }</pre>
*
* #implNote Most instances of Stream will override this method and provide an implementation
* that is highly optimized compared to the implementation in this interface.
*
* #return a List containing the stream elements
*
* #since 16
*/
#SuppressWarnings("unchecked")
default List<T> toList() {
return (List<T>) Collections.unmodifiableList(new ArrayList<>(Arrays.asList(this.toArray())));
}
My idea of what would be better
return (List<T>) Collections.unmodifiableList(Arrays.asList(this.toArray()));
Or even
return Arrays.asList(this.toArray()));
IntelliJ's proposal
return (List<T>) List.of(this.toArray());
Is there any good reason for the implementation in the JDK sources?

The toArray method might be implemented to return an array that is then mutated afterwards, which would effectively make the returned list not immutable. That's why an explicit copy by creating a new ArrayList is done.
It's essentially a defensive copy.
This was also discussed during the review of this API, where Stuart Marks writes:
As written it's true that the default implementation does perform apparently redundant copies, but we can't be assured that toArray() actually returns a freshly created array. Thus, we wrap it using Arrays.asList and then copy it using the ArrayList constructor. This is unfortunate but necessary to avoid situations where someone could hold a reference to the internal array of a List, allowing modification of a List that's supposed to be unmodifiable.

the Collection extends Iterable, but why Collection declare Iterable Function (same as Iterable) [duplicate]

This question already has an answer here:
Method iterator() declared in java.util.Collection and in java.lang.Iterable, its superinterface?
(1 answer)
Closed 3 years ago.
my java version is "1.8.0_192"
/**
* Returns an iterator over the elements in this collection. There are no
* guarantees concerning the order in which the elements are returned
* (unless this collection is an instance of some class that provides a
* guarantee).
*
* #return an <tt>Iterator</tt> over the elements in this collection
*/
Iterator<E> iterator();
this is Collection interface.
/**
* Returns an iterator over elements of type {#code T}.
*
* #return an Iterator.
*/
Iterator<T> iterator();
this is Iterable interface.

There is really no reason to have both. If you look at the since section for Collection, you will see it was added in 1.2 while Iterable was added in 1.5, so it seems to be "historical".
We would have to ask the actual authors why the redundant declaration was not removed. With that said, there is no negative impact with having it in both places.
And no, there is no special meaning in having the iterator method on both.

why do I need Iterator interface and why should I use it?

I am new to Java so maybe to some of you my question will seem silly.
As I understand from some tutorial if I need to make on my custom object foreach the object must implement Iterable interface.
My question is why do I need Iterator interface and why should I use it?

As you mentioned, Iterable is used in foreach loops.
Not everything can be used in a foreach loop, right? What do you think this will do?
for (int a : 10)
The designers of Java wanted to make the compiler able to spot this nonsense and report it to you as a compiler error. So they thought, "what kind of stuff can be used in a foreach loop?" "Well", they thought, "objects must be able to return an iterator". And this interface is born:
public interface Iterable<T> {
/**
* Returns an iterator over elements of type {#code T}.
*
* #return an Iterator.
*/
Iterator<T> iterator();
}
The compiler can just check whether the object in the foreach loop implements Iterable or not. If it does not, spit out an error. You can think of this as a kind of "marker" to the compiler that says "Yes I can be iterated over!"
"What is an iterator then?", they thought again, "Well, an iterator should be able to return the next element and to return whether it has a next element. Some iterators should also be able to remove elements". So this interface is born:
public interface Iterator<E> {
/**
* Returns {#code true} if the iteration has more elements.
* (In other words, returns {#code true} if {#link #next} would
* return an element rather than throwing an exception.)
*
* #return {#code true} if the iteration has more elements
*/
boolean hasNext();
/**
* Returns the next element in the iteration.
*
* #return the next element in the iteration
* #throws NoSuchElementException if the iteration has no more elements
*/
E next();
/**
* Removes from the underlying collection the last element returned
* by this iterator (optional operation). This method can be called
* only once per call to {#link #next}. The behavior of an iterator
* is unspecified if the underlying collection is modified while the
* iteration is in progress in any way other than by calling this
* method.
*
* #implSpec
* The default implementation throws an instance of
* {#link UnsupportedOperationException} and performs no other action.
*
* #throws UnsupportedOperationException if the {#code remove}
* operation is not supported by this iterator
*
* #throws IllegalStateException if the {#code next} method has not
* yet been called, or the {#code remove} method has already
* been called after the last call to the {#code next}
* method
*/
default void remove() {
throw new UnsupportedOperationException("remove");
}
}

The Iterator is design pattern, it allows to go through collection of same object in certain way, this also allow to hide implementation of store element and iteration mechanism from user. As you can see in javadoc many classes implements Itarable interface, not only collections. In example it allows you to iterate through two List implementations in same performance, when ArrayList give indexes in same time but LinkedList for give certain index need to go all elements previously to this number and this much slower. But when you get Iterator from this implementation you get same performance in both cases because iterate algorithm optimised in both list in different way. ResultSet is also iterator but it does not implement interface from java.util it allow to iterate in all result of query in db in same way and hide back structures responsibles for elements store and db participation. In example when you need some optimization you may make new ResultSet implementation query db per next result invoke or what ever you want, because it also decouple client code from elements storage realization and iteration algorithms.

HashSet or HashMap without defining a hashCode() method in a new class

What happens if you design a new class and try to insert objects of that class into a HashSet or HashMap without defining a hashCode() method?
Please keep make the explanation easy. I'm studying for an exam and I'm still rusty with hashes in Java. Thank you.

A HashMap stores data into multiple singly linked lists of entries (also called buckets or bins). All the lists are registered in an array of Entry (Entry[] array)
The following picture shows the inner storage of a HashMap instance with an array of nullable entries. Each Entry can link to another Entry to form a linked list.
When a user calls put(K key, V value) or get(Object key), the function computes the index of the bucket in which the Entry should be.
This index of the bucket (linked list) is generated using hashcode of the key.
So, if you have overridden the hashCode method, it will use overridden method to compute index of the bucket
otherwise default hash code is used which is the memory address for your object. So in that case even your objects are you will have a new entry in your map. So even if you try to store logically equal objects. They wil be reataed as different by hash Map.
As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)
For example:
MyObject a = new MyObject("a", 123,"something");
MyObject b = new MyObject("a", 123,"something");
a and b will have different hashcodes.

Nothing will happen :-)
Every object has own hashCode() method that inherited from Object class. So, your every new object will be unique. By herself, they will be identified as unique by HashSet or HashMap.
Here are official comments:
/**
* Returns a hash code value for the object. This method is
* supported for the benefit of hash tables such as those provided by
* {#link java.util.HashMap}.
* <p>
* The general contract of {#code hashCode} is:
* <ul>
* <li>Whenever it is invoked on the same object more than once during
* an execution of a Java application, the {#code hashCode} method
* must consistently return the same integer, provided no information
* used in {#code equals} comparisons on the object is modified.
* This integer need not remain consistent from one execution of an
* application to another execution of the same application.
* <li>If two objects are equal according to the {#code equals(Object)}
* method, then calling the {#code hashCode} method on each of
* the two objects must produce the same integer result.
* <li>It is <em>not</em> required that if two objects are unequal
* according to the {#link java.lang.Object#equals(java.lang.Object)}
* method, then calling the {#code hashCode} method on each of the
* two objects must produce distinct integer results. However, the
* programmer should be aware that producing distinct integer results
* for unequal objects may improve the performance of hash tables.
* </ul>
* <p>
* As much as is reasonably practical, the hashCode method defined by
* class {#code Object} does return distinct integers for distinct
* objects. (This is typically implemented by converting the internal
* address of the object into an integer, but this implementation
* technique is not required by the
* Java™ programming language.)
*
* #return a hash code value for this object.
* #see java.lang.Object#equals(java.lang.Object)
* #see java.lang.System#identityHashCode
*/
public native int hashCode();

what's the usage of the code in the implementation of AbstractCollection's toArray Method

public Object[] toArray() {
// Estimate size of array; be prepared to see more or fewer elements
Object[] r = new Object[size()];
Iterator<E> it = iterator();
for (int i = 0; i < r.length; i++) {
if (! it.hasNext()) // fewer elements than expected
return Arrays.copyOf(r, i);
r[i] = it.next();
}
return it.hasNext() ? finishToArray(r, it) : r;
}
here's the code of implementation of AbstractCollection.toArray method.
if (! it.hasNext()) // fewer elements than expected
return Arrays.copyOf(r, i);
I don't understand the usage of the code above. I suspect the code is used to avoid the size changing while the the method is invoked. So I have two questions:
What I suspect is right or wrong? if it's wrong, what's the usage of this code?
If it's true, what situation can make the size changing while the method has been invoked?

Well, the method's javadoc sais it all:
/**
* {#inheritDoc}
*
* <p>This implementation returns an array containing all the elements
* returned by this collection's iterator, in the same order, stored in
* consecutive elements of the array, starting with index {#code 0}.
* The length of the returned array is equal to the number of elements
* returned by the iterator, even if the size of this collection changes
* during iteration, as might happen if the collection permits
* concurrent modification during iteration. The {#code size} method is
* called only as an optimization hint; the correct result is returned
* even if the iterator returns a different number of elements.
*
* <p>This method is equivalent to:
*
* <pre> {#code
* List<E> list = new ArrayList<E>(size());
* for (E e : this)
* list.add(e);
* return list.toArray();
* }</pre>
*/
I find two interesting things to mention here:
Yes, you're right, as the javadoc sais, this method is prepared to return correctlly even if the Collection has been modified in the mean time. That's why the initial size is just a hint. The usage of the iterator also ensures avoidance from the "concurrent modification" exception.
It's very easy to imagine a multi-threaded situation where one thread adds/removes elements from a Collection while a different thread calls the "toArray" method on it. In such a situation, if the Collection is not thread safe (like obtained via Collections.synchronizedCollection(...) method, or by manually creating synchronized access code towards it) you'll get into a situation where it's modified and toArray-ed at the same time.

I just want to mention that according to the javadoc, the method size() can return maximum Integer.MAX_VALUE. But if your collection has more elements you can't get a proper size.

you are right, the array is initialized with size() so if any element is removed while the array is being populated, you would benefit from this check.
Collections are by default not thread safe, so another thread could call remove() while the iteration is in progress :-)

While it's generally guaranteed (e.g. for all java.util.* collection classes) that a collection won't change while it is iterated (otherwise throws a ConcurrentModificationException) this is not guaranteed for all collections. It's therefore possible for another thread to add or remove elements while one thread is calling toArray(), thus changing the size of collection and therefore the resulting array. Alternatively, some implementation might only return an approximate size.
Therefore, to answer the question:
These two lines check if the end of the collection was reached before the expected size (result of size() call which defines r.length) was reached. If this is the case, a copy of the array r with the appropriate size will be made. Remember that it's not possible to resize an array.
As said, different possibilities since the contract for Collection is pretty loose. Multi-threading, approximate results by size() and others.

Andrei is the main answer. corsair raises an excellent point about Integer.MAX_VALUE.
For completeness, I will add the toArray method is supposed to work on any Collection including:
arrays with a buggy size method;
dynamic arrays - the contents of the Collection could change depending on other threads (concurrency), the time, or random numbers. An example in pseudocode
Collection < Food > thingsACatholicCanEat ; // if it is Friday should not include meat