We Keep Coding

Bad return type in lambda expression when using Java's Optional.or() with subclasses

I am trying to use Optional.or to get an object of subclass A or, if empty, an object of subclass B:
interface Node {}
class InnerNode implements Node {}
class LeafNode implements Node {}
Optional<InnerNode> createInnerNode() {}
Optional<LeafNode> createLeafNode() {}
Optional<Node> node = createInnerNode().or(() -> createLeafNode());
When doing this, I get the following compiler error:
Bad return type in lambda expression: Optional<LeafNode> cannot be converted to Optional<? extends InnerNode>
If I instead use wildcards to explicitly tell the compiler that the Optionals contain an object extending from Node:
Optional<? extends Node> optionalInner = createInnerNode();
Optional<? extends Node> optionalLeaf = createLeafNode();
Optional<Node> node = optionalInner.or(() -> optionalLeaf);
I get the following compiler error:
Bad return type in lambda expression: Optional<capture of ? extends Node> cannot be converted to Optional<? extends capture of ? extends Node>
The only way I found to make this work, is using an Optional.empty in front:
Optional<Node> node = Optional.<Node>empty() // requires generic argument
.or(() -> createInnerNode())
.or(() -> createLeafNode());
But for me, this is confusing to read. Is it somehow possible to instruct the compiler to allow the statement optionalInner.or(() -> optionalLeaf)? Or are there other alternatives to make this work?

The workaround you provided is a pretty simple answer, but you can convert an Optional<InnerNode> to an Optional<Node> trivially with map if you want to avoid a placeholder Optional.
createInnerNode.<Node>map(Function.identity()).or(() -> createLeafNode());
In general, this is also how you conceptually turn a List<Dog> to a List<Animal> or any generic collection to a different generic collection in java as well. It's somewhat annoying, but wildcards don't cut it because you want both read/write on the transformed collection.
The signature of or should tell you why your workaround works
Optional<T> or(Supplier<? extends Optional<? extends T>> supplier)
notice how it's already using the ? extends T in the supplier.
Since the first empty Optional's T is Node, the or function can accept a supplier that returns
Optional<? extends Node>
and since both the innernode and leafnode are an Optional<? extends Node>, this works.
So the problem is that you need an Optional<Node> at some point.
I don't think you can avoid doing something since an Optional<InnerNode> never type matches Optional<Node> for reasons listed in other answers (and other questions on stack overflow).
Reading the other answer and trying
Optional<? extends Node> node = createInnerNode();
means that the T is a
? extends Node
which makes or seem to want doubly nested captures? I can't stop getting
Bad return type in lambda expression: Optional<capture of ? extends Node> cannot be converted to Optional<? extends capture of ? extends Node>
when trying basically any cast with the following code
Optional<? extends Node> node = Node.createInnerNode();
node.or(() -> (Optional<? extends Node>) null);

This must be a confusing error. Let me first explain the underlying problem so you actually understand why javac is complaining, and then I'll get to how to fix it.
The underlying problem is lack of use-site variance.
The error you get just doesn't make sense for Optional, but it makes perfect sense when we use a type that produces and consumes typed data (unlike Optional which, as an instance, only produces data, e.g. when you call .get() - there is no .set() method on Optional; it's immutable).
So let's look at lists. Imagine you could do this:
Integer i = 5;
Number n = i; // This is legal java.
List<Integer> ints = new ArrayList<Integer>();
List<Number> numbers = ints; // So this should be too... right?
That 4th line is not legal. Because generics are by default invariant (Neither supertypes nor subtypes will do; in an invariant typing system, if a Number is required, then only a Number will do) - and that is correct. After all, if the above code WAS legal, then everything breaks:
Double d = 5.0;
numbers.add(d); // Fine - must be. doubles are numbers!
Integer i = ints.get(0); // um.. wait. uhoh.
Because numbers and ints are both just referencing the exact same list, if I add a double to numbers, that means I also added one to ints, and - boom. It's broken.
Now you know why generics are invariant.
This also explains a few further compiler errors:
List<Integer> ints = new ArrayList<Integer>();
List<Number> numbers = ints; // nope, won't compile.
List<? extends Number> numbers = ints; // but this will!
numbers.add(anything-except-literally-null); // this wont!
With ? extends you are telling java that you want covariance (covariance = a subtype of X is just as good as X itself. Java's non-generics typing is covariant). However, covariance in generics automatically means that ALL ways to 'send' typed data to that thing are disabled. You CANNOT add anything (except literally .add(null) as null is all types) to a List<?> or a List<? extends>. This makes sense if you think about it:
A List<? extends Number> could be a List<Integer> or a List<Double>. So how can you add anything to this list? There is nothing that is both a Double and an Integer (well, except the literal null), so nothing you could possibly pass to add is neccessarily 'safe'. Hence why the compiler always tells you its a typing violation.
This also means trivially that this won't work either:
List<? extends Number> numbers = ...;
List<Number> numbers2 = numbers; // nope, this won't compile
Back to your code
Now, for optional, none of this seems relevant. Optional can be completely covariant in its type argument because Optional does not 'consume' data - none of its methods take in an 'E'. But the compiler isn't some sort of megabrain that susses this out and lets you 'break' the variance rules of typeargs. Not so - Optional needs to stick to the same rules list does. So, if an instance of List<? extends Number> cannot be assigned to a variable of type List<Number>, then the same applies to Optional:
An instance of type Optional<? extends Node> cannot be assigned to a variable of type Optional<Node>
And that explains your errors (for both of the attempts to solve the problem stated in your question).
Even though for optional specifically these errors don't make much sense. Now you know why you are getting them.
The best fix is to not use optional here; this isn't really what its meant for and it's not very java-like (see below). But the distant second best option is to fix this stuff. The root fix is that all places that take in an Optional should ALWAYS be stated as ? extends. So, you're trying to assign it to a field of type Optional<Node>, but that field is 'wrong'. That field's type should be Optional<? extends NOde>. Then that fixes all problems:
Optional<? extends Node> node = createInnerNode().or(() -> createLeafNode());
Works fine (that's your first snippet, but with the 'type' of the target fixed).
Ah. But, I can't change the API
Well, then the API is just wrong. It happens - sometimes you need to interact with broken code. But make no mistake, that is broken code. You do the same thing you do with any other broken code: Encapsulate and work around.
You don't want to interact with broken API. So make a wrapper type or helper methods that 'paper over' the error. And in that code (the 'bridging' code that works around the buggy code you cannot change), you accept that you need to write wonky code that generates warnings.
You can 'fix things' as follows:
public Optional<Node> fixIt(Optional<? extends Node> node) {
Optional raw = node;
return raw;
}
This works because generics in the end is just a figment of javac's imagination - the JVM doesn't know what generics are at all. So we just need to 'fake out' javac to stop refusing to compile the code. The above WILL generate warnings and any errors in such hackery result in getting ClassCastExceptions on lines with zero casts on them - quite confusing. But, works fine if you know what you're doing. You can eliminate the warnings with a #SuppressWarnings annotation.
Or.. just don't use optional
Java has null baked into many many places. Not just in its core model (fields that aren't initialized begin with null values. Arrays of non-primitives are initialized with null values), but also in a great many of its most crucial libraries. The most obvious and simple example is java.util.Map: It has a get() method that returns a V. Not an Optional<V>. And in a world where Optional is the right answer, get() should of course return Optional<V>. But it does not, and it never will - java doesn't break backwards compatibility that drastically.
Thus, the hypothetical world of 'optional everywhere!' sucks - 30 years of built up code and history needs to be tossed in the bin first and the community is rather unlikely to do that. If you dislike null handling in java (and there are plenty of fine reasons to do so!), then a solution is required that lets existing code backwards-compatibly adjust, and Optional cannot deliver. (nullity annotations probably can, though! Much better; or just write better APIs. Map has .getOrDefault which can be used to ensure your map query calls don't ever return null in the first place. Once you start writing your code to never produce null values except in cases where you truly WANT any attempt to deref to throw (i.e. very rarely), then null ceases to be a problem. These more modern addons to the various java core library types make it much easier to write such code. And adding methods to interfaces is backwards compatible, so, as java.util.Map shows, existing stuff can backwards-compatibly add these).
The best thing to do with Optional is to use it solely for why it was introduced in the first place: As a return value for stream API terminals. Something like intList.stream().max() returns an OptionalInt because what is the maximum of an empty list? Its fine enough for that. If you write your own collectors or stream terminating operations, use Optional. If that's not what you are doing, don't use it.

Usuage of ? extends anyobject in Java Generics

List<? extends Number> l = new ArrayList<Integer>();
I have the above line of code.
The object 'l' can refer any objects which can be Number any of its sub types.
I know that we can not add anyting to the above list.
Then what is the use of "? extends any_object" since we can not add anything to it ?
In what kind of context it is used ?
Can anyone please explain.
Thanks!!

It’s most useful for method arguments. Consider this:
public void printFormatted(Collection<? extends Number> values) {
NumberFormat format = new DecimalFormat("0.000");
for (Number value : values) {
System.out.println(format.format(value));
}
}
If I declare the method with Collection<Number>, callers must pass a Collection (or List or Set) whose generic type is known to be Number. They cannot pass Collection<Integer>, because Collection<Integer> is not a subclass of Collection<Number> and in fact is not polymorphically compatible with Collection<Number>.
By declaring the method with Collection<? extends Number>, I am saying that callers can pass Collection<Number>, or Collection<Integer>, or Collection<Double>, etc. List<Integer> or Set<Integer> is also acceptable, since the class is polymorphic, even though the generic type is not.

One of possible examples:
When you pass such method as argument, you can iterate over elements of such such list and use methods appropriate to Number class. You need to be aware that you cannot add any elements to such list. You are able to use list.get() method and that way, you're sure that you will get an instanceof Number class.
Such list is then called a producer. If you would need to add elements to list, instead of retrieving them, you would use <? super Number>. That rule is called PECS (Producer Extends Consumer Super).

Covariance in Java - Collection cannot be added to

I was reading an interesting dzone article on covariance in java which is pretty easy to follow but there is one thing bugging me which doesnt make sense, the article is here https://dzone.com/articles/covariance-and-contravariance
I am quoting examples from the article here where it is explaining why a collection cannot be added to:
With covariance we can read items from a structure, but we cannot write anything into it. All these are valid covariant declarations.
List<? extends Number> myNums = new ArrayList<Integer>();
Because we can be sure that whatever the actual list contains, it can be upcasted to a Number (after all anything that extends Number is a Number, right?)
However, we are not allowed to put anything into a covariant structure.
myNumst.add(45); //compiler error
This would not be allowed because the compiler cannot determine what is the actual type of the object in the generic structure. It can be anything that extends Number (like Integer, Double, Long), but the compiler cannot be sure what
The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
So why does it enforce this as it seems to me like it can determine the type?

When the compiler comes across:
List<? extendsNumber> myNums = new ArrayList<Integer>();
it checks that the types on left and right hand side fit together.
But the compiler does not "remember" the specific type used for the instantiation on the right hand side.
One could say that the compiler remembers this about myNums:
it has been initialized
it is of type List<? extendsNumber>
Yes compilers can do constant folding; and data flow analysis; and it might be possible for a compiler to track that "instantiation type" information as well - but alas: the java compiler doesn't do that.
It only knows that myNums is an initialized List<? extends Numbers> - nothing more.

You are missing two points:
You are thinking in terms of local variables only:
public void myMethod() {
List<? extends Number> list = new ArrayList<Integer>();
list.add(25);
}
A compiler could easily detect the actual value of ?, here but I know of nobody who would write such a code; if you are going to use an integer list you just declare the variable as List<Integer>.
Covariance and contravariance are most useful when dealing with parameters and/or results; this example is more realistic:
public List<Integer> convert(List<? extends Number> source) {
List<Integer> target = new ArrayList<>();
for (Number number : source) {
target.add(number.intValue());
}
return target;
}
How is a compiler expected to know which is the type used to parametrize the list? Even if at compile time all the calls only pass instances of ArrayList<Integer>, at a later time code can use the method with a diferent parameter without the class being recompiled.
The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
No, what the compiler knows is that the list contains something that extends Number (including Number). It cannot tell if it is a List<Number> (in which case you may insert any instance of Number) or a List<Integer> (in which case you may only insert Integer instances). But it does know that everything you retrieve using get will be an instance of Number (even if it is not sure about the specific class).

The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
Consider a slightly different but related example:
Object obj = "";
By the argument above, the compiler should also be able to know that obj is actually a String, and so you'd be able to invoke String-specific methods on it:
obj.substring(0);
which anybody with even a little Java experience knows you can't.
You (and only you) (or, at least, the person who wrote the code) has the type information, and a wilful decision has been made to discard it: there is no reason to have declared the variable type to be Object, so why should the compiler have to put in work to try to recover that information? (*)
By the same token, if you want the compiler to know that the value of the variable
List<? extends Number> myNums = new ArrayList<Integer>();
is an ArrayList<Integer>, declare the variable to be of that type. Otherwise, it's just much simpler for the compiler to assume "this can be absolutely anything within the type bounds".
(*) I read this argument somewhere in another answer somewhere on SO. I can't remember where it was, or I'd give appropriate attribution.

Lower-Bounded Wildcards Java - Access to Methods

I understand that one reason Lower-bounded wildcards exist is so that a collection is not immutable when adding new elements.
E.g.
List<? extends Number> obj = new ArrayList<>();//Now this list is immutable
obj.add(new Integer(5));//Does not compile
List<? super Number> objTwo = new ArrayList<>();//This list is mutable
objTwo.add(new Integer(5));//Compiles
The following does not compile because I tried to get the long value of numbers.
Q1: What methods would I be able to use? Only Objects methods?:
public void testLowerBounds(List<? super Number> numbers){
if (!numbers.isEmpty()){
System.out.println(numbers.get(0).longValue());//Does not compile
}
}
How my question came about:
I am currently learning about streams and the book specifies the following stream method:
Optional<T> min(Comparator<? super T> comparator)
And implements it as follows:
Stream<String> s = Stream.of("monkey", "ape", "bonobo");
Optional<String> min = s.min((s1, s2) -> s1.length()—s2.length());
Q2: How is the comparator allowed to use string methods when is used?
If I had to answer Q2: I would say that optional is specifying "You have to pass me an implementation of Comparator that has a generic type "String" or something that implements "String". Would I be correct in saying this?
Looking forward to your response.

First of all, you should not confuse wildcard type parameters with mutability. Having a wildcard in a List’s element type does not prevent modifications, it only imposes a few practical restrictions to what you can do with the list.
Having a list declared like List<? extends Number> implies that the referenced list has an actual element type of Number or a subclass of Number, e.g. it could be a List<Integer> or List<Double>. So you can’t add an arbitrary Number instance as you can’t know whether it is compatible to the actual element type.
But you can still add null, as the null reference is known to be compatible with all reference types. Further, you can always remove elements from a list, e.g. call remove or clear without problems. You can also call methods like Collections.swap(list, index1, index2), which is interesting as it wouldn’t be legal to call list.set(index1, list.get(index2)) due to formal rules regarding wildcard types, but passing the list to another method that might use a non-wildcard type variable to represent the list’s element type works. It’s obviously correct, as it only sets elements stemming from the same list, which must be compatible.
Likewise, if you have a Comparator<Number>, you can call Collections.sort(list, comparator), as a comparator which can handle arbitrary numbers, will be able to handle whatever numbers are actually stored in the list.
To sum it up, having ? extends in a collection’s element type does not prevent modifications.
As said, you can’t insert arbitrary new elements into a list whose actual element type might be an unknown subclass of the bound, like with List<? extends Number>. But you are guaranteed to get a Number instance when retrieving an element, as every instance of subtype of Number is also an instance of Number. When you declare a List<? super Number>, its actual element type might be Number or a super type of Number, e.g. Object or Serializable. You can insert arbitrary Number instances, as you know it will be compatible to whatever actual element type the list has, as it is a super type of number. When you retrieve an instance, you only know that it is an instance of Object, as that’s the super type of all instances. To compare with the ? extends case, having a ? super declaration does not prevent reading, it only imposes some practical limitations. And likewise, you can still pass it to Collections.swap, because, regardless of how little we known about the actual type, inserting what we just retrieved from the same list, works.
In your second question, you are confusing the sides. You are now not looking at the implementation of min, but at the caller. The declaration of min(Comparator<? super T> c) allows the caller to pass any comparator being parameterized with T or a super type of T. So when you have a Stream<String>, it is valid to pass a Comparator<String> to the min method, which is exactly, what you are implementing via the (s1, s2) -> s1.length()—s2.length() lambda expression (though, I’d prefer Comparator.comparingInt(String::length)).
Within the implementation of min, there is indeed no knowledge about what either, T or the actual type argument of the Comparator, is. But it’s sufficient to know that any stream element that is of type T can be passed to the comparator’s compare method, which might expect T or a super type of T.

Why generic type is not applicable for argument extends super class for both?

Here is the problem that I have been being tried to find the solution.
We have two class definitions. One of two extends other one.
class T{}
class TT extends T{}
The requirement is that there should be a list keeps object extends T
List<? extends T> list = new ArrayList<>();
But the problem occures when I try to put a TT object ( barely seems it is a subclass of T )
into the list.
list.add(new TT());
Compilation Error Message
The method add(capture#2-of ? extends Cell) in the type List is not applicable for the arguments (Cell)

You can create a List<T> list = new ArrayList<T>(); directly, this can allow all subtypes of T into the list. This is actually little difficult to understand. when you declare it as
List<? extends T> list = ...
It means that it can allow any unknown subtypes of T into the list. But, from that declaration we cannot ensure which is the exact sub-type of T. so, we can only add null into it

List<? extends T> indicates that anything can comes out of it can be cast to T, so the true list could be any of the following:
List<T>
List<T2>
List<TT>
etc
You can see that even a new T cannot safely be added to such a collection because it could be a List<T2> which T cannot be put into. As such, such List cannot have non null entries added to them.
In this case you may simply want List<T>
So why would you ever use this?!
This contravariance can be useful for method parameters or returns, in which a collection will be read, rather than added to. A use for this could be to create a method that accepts any collection that holds items that are T, or extend T.
public static void processList(Collection<? extends Vector3d> list){
for(Vector3d vector:list){
//do something
}
}
This method could accept any collection of objects that extends Vector3d, so ArrayList<MyExtendedVector3d> would be acceptable.
Equally a method could return such a collection. An example of a use case is described in Returning a Collection<ChildType> from a method that specifies that it returns Collection<ParentType>.

The requirement is that there should be a list keeps object extends T
If you just want a List where you can store objects of any class that extend from T, then just create a List like this:
List<T> list = new ArrayList<T>();
The way you've created a list currently, will not allow you to add anything except null to it.

There are boundary rules defined for Java Generics when using WildCards
**extends Wildcard Boundary**
List means a List of objects that are instances of the class T, or subclasses of T (e.g. TT). This means a Read is fine , but insertion would fail as you dont know whether the class is Typed to T
**super Wildcard Boundary**
When you know that the list is typed to either T, or a superclass of T, it is safe to insert instances of T or subclasses of T (e.g.TT ) into the list.
In your example , you should use "super"

An addition to the other answers posted here, I would simply add that I only use wild cards for method parameters and return types. They're intended for method signatures, not implementations. When I put a wildcard into a variable declaration, I always get into trouble.