I was reading an interesting dzone article on covariance in java which is pretty easy to follow but there is one thing bugging me which doesnt make sense, the article is here https://dzone.com/articles/covariance-and-contravariance
I am quoting examples from the article here where it is explaining why a collection cannot be added to:
With covariance we can read items from a structure, but we cannot write anything into it. All these are valid covariant declarations.
List<? extends Number> myNums = new ArrayList<Integer>();
Because we can be sure that whatever the actual list contains, it can be upcasted to a Number (after all anything that extends Number is a Number, right?)
However, we are not allowed to put anything into a covariant structure.
myNumst.add(45); //compiler error
This would not be allowed because the compiler cannot determine what is the actual type of the object in the generic structure. It can be anything that extends Number (like Integer, Double, Long), but the compiler cannot be sure what
The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
So why does it enforce this as it seems to me like it can determine the type?
When the compiler comes across:
List<? extendsNumber> myNums = new ArrayList<Integer>();
it checks that the types on left and right hand side fit together.
But the compiler does not "remember" the specific type used for the instantiation on the right hand side.
One could say that the compiler remembers this about myNums:
it has been initialized
it is of type List<? extendsNumber>
Yes compilers can do constant folding; and data flow analysis; and it might be possible for a compiler to track that "instantiation type" information as well - but alas: the java compiler doesn't do that.
It only knows that myNums is an initialized List<? extends Numbers> - nothing more.
You are missing two points:
You are thinking in terms of local variables only:
public void myMethod() {
List<? extends Number> list = new ArrayList<Integer>();
list.add(25);
}
A compiler could easily detect the actual value of ?, here but I know of nobody who would write such a code; if you are going to use an integer list you just declare the variable as List<Integer>.
Covariance and contravariance are most useful when dealing with parameters and/or results; this example is more realistic:
public List<Integer> convert(List<? extends Number> source) {
List<Integer> target = new ArrayList<>();
for (Number number : source) {
target.add(number.intValue());
}
return target;
}
How is a compiler expected to know which is the type used to parametrize the list? Even if at compile time all the calls only pass instances of ArrayList<Integer>, at a later time code can use the method with a diferent parameter without the class being recompiled.
The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
No, what the compiler knows is that the list contains something that extends Number (including Number). It cannot tell if it is a List<Number> (in which case you may insert any instance of Number) or a List<Integer> (in which case you may only insert Integer instances). But it does know that everything you retrieve using get will be an instance of Number (even if it is not sure about the specific class).
The paragraph above is what doesn't make sense to me, the compiler knows that the list contains Number or anything that extends it, and the ArrayList is typed to Integer. And the compiler knows about the literal int that is inserted.
Consider a slightly different but related example:
Object obj = "";
By the argument above, the compiler should also be able to know that obj is actually a String, and so you'd be able to invoke String-specific methods on it:
obj.substring(0);
which anybody with even a little Java experience knows you can't.
You (and only you) (or, at least, the person who wrote the code) has the type information, and a wilful decision has been made to discard it: there is no reason to have declared the variable type to be Object, so why should the compiler have to put in work to try to recover that information? (*)
By the same token, if you want the compiler to know that the value of the variable
List<? extends Number> myNums = new ArrayList<Integer>();
is an ArrayList<Integer>, declare the variable to be of that type. Otherwise, it's just much simpler for the compiler to assume "this can be absolutely anything within the type bounds".
(*) I read this argument somewhere in another answer somewhere on SO. I can't remember where it was, or I'd give appropriate attribution.
Related
This question already has answers here:
Generics : List<? extends Animal> is same as List<Animal>?
(3 answers)
Closed 5 months ago.
I am going through the Java generics. I came across the below line.
public <T extends Building> List<T> fromArrayToList(T[] t) {
...
}
At compile time this will get converted like below,
public List<Building> fromArrayToList(Building[] t) {
...
}
With this understanding, I could rather use like below, same as compiled version in my code.
public List<Building> fromArrayToList(Building[] t) {
...
}
What is the reason behind using List<? extends Building> instead of List<Building> ?
Generics are almost entirely compiler-checked documentation. In the sense that the runtime (java.exe) has no idea what they are - most of the generics doesn't survive compilation in the first place ('erasure').
So, that it gets 'converted' to something else at compile time is irrelevant. Generics are entirely about having the compiler check your work. It does nothing at runtime.
So what does it do? It links things. It tells the compiler: There is some type, we don't know what it is, but in these places (more than one place, it's not useful to link one thing to nothing else), it's the same type, whatever it is. Any 'usage' of this situation gets to redefine whatever type it is, so long as they maintain the 'linkage'.
For example, trivially, imagine this method:
public Object print(Object o) {
System.out.println(o);
return o;
}
A simple-ish method that you can use to on-the-fly print things while you use them. For example, you may want to lowercase incoming usernames to check them against a (case insensitive) table of usernames like so: users.get(username.toLowerCase()), but perhaps you want to see the original, as-entered, username on the fly, so you could instead do:
users.get(print(username).toLowerCase());
but that would not work! The return type of the print method is Object, and Object does not have a .toLowerCase() method.
Yes, you have eyes. So have I: We can clearly see the object returned by print is clearly a string so it should work, but it does not. This is not a javac oversight - this is intentional behaviour: You are free to change the body of a method later. There is a difference between what you declare to be the truth about your API and what so happens to occur this particular version. public Object print(Object o) declares that you will today and forever (or at least, until you say your API is no longer backwards compatible), take 1 argument, that can be any object, and this method returns something. What? Dunno, but, it's at least an Object (not particularly useful, that).
Today it returns what you passed to it. But tomorrow? Who knows.
So let's fix this and make it work again. Add generics. We are going to link the type of the parameter with the return type:
public <T> T print(T object) {
System.out.println(object);
return object;
}
And now maps.get(print(username).toLowerCase()) does work! Great!
The reason it works is simply because you linked the types: You've told the compiler: Hey, this print method? Each time it is invoked, there is some type. No restrictions on it (well, it has to be Object or some subtype thereof, so not a primitive, other than that - anything goes). Both the return type and the param type are that type. Whatever type is most convenient at the time. So, given that rule - the compiler picks String to be that type and now the code works, great.
This is entirely the compiler doing it. If you decompile the bytecode, you'll notice the print method is just public Object print(Object object), and instead it is the caller that changed a bit - decompiling the users.get(print(username).toLowerCase()) line you notice that the compiler has injected a cast of all things, even though the java line doesn't contain a cast. That's because the compiler realised it was safe to do this, because of the generics.
The same applies to your code. You write it like the first snippet (with the T), so that this works:
class Highrise extends Building {}
Highrise[] highrises = ....;
List<Highrise> asList = fromArrayToList(highrises);
Had you gone with the 3rd snippet in your code, that would not work. In fact, the third snippet is broken. After all, I could call this:
List<Building> highrises = ...;
highrises.add(new LogCabin());
After all, a LogCabin is a building. Given that I can trivially write:
Building b = new LogCabin();
It would be bizarre indeed if highrises.add(new LogCabin()) would fail if highrises is a List<Building>. But.. you just added a logcabin to a list of highrises, you broke it.
A second thing to realize with generics is variance. Given a list of X, that cannot just be treated as a list of Y, where Y is a supertype. This makes total sense:
Building b = new LogCabin(); // fine
but this does not work:
List<Building> b = new ArrayList<LogCabin>(); // fails at compile time
why? Well, because you can invoke b.add(new Highrise()) and now there's a highrise in your list of logcabins:
List<LogCabin> logCabins = new ArrayList<LogCabin>();
List<Building> buildings = logCabins;
buildings.add(new Highrise());
LogCabin lc = logCabins.get(0);
The above code proves why invariance is neccessary. ? extends lets you opt into covariance, but, to counter the above problem, you can't call .add on a List<? extends> anything. Even just List<?> (which is short for List<? extends Object>. (Well, to get pedantic, you can call .add(null) - a literal null literal, because a null literal is all reference types at once, but that is quite useless, generally).
By disabling 'add', it now no longer matters. The problem with letting you assign a list of logcabins to a list of buildings is that this means one could add non-logcabin building to the list, but if add doesn't work at all, it no longer matters. Hence:
List<LogCabin> logCabins = new ArrayList<LogCabin>(); // fine
List<? extends Building> buildings = logCabin; // compiles fine
buildings.add(new Highrise()); // compiler error
Now you know why extends is useful.
Generics is best understood by keeping two things in mind:
Type variables are named that way for a good reason: There is a TYPE, but you do not know what it is. Just like int x; says "there is an integer, no idea if it's 0, 1, 5, 1238123, who knows? It has a specific file for any specific invocation, but it could be different every next invocation" - <T> says there is some type, who knows what it is.
They LINK things. If they don't link things, it's useless or a language hack. A type var must be used in at least 2 places.
Example of 2 place usage:
class ArrayList<T> { // first use
void add(T elem) { ... } // also here
T get(int idx) { ... } // and here
void addAll(List<? extends T> elems) {} // and here
}
at least 4 places where T is used, that's fine. Linkage achieved.
I am trying to use Optional.or to get an object of subclass A or, if empty, an object of subclass B:
interface Node {}
class InnerNode implements Node {}
class LeafNode implements Node {}
Optional<InnerNode> createInnerNode() {}
Optional<LeafNode> createLeafNode() {}
Optional<Node> node = createInnerNode().or(() -> createLeafNode());
When doing this, I get the following compiler error:
Bad return type in lambda expression: Optional<LeafNode> cannot be converted to Optional<? extends InnerNode>
If I instead use wildcards to explicitly tell the compiler that the Optionals contain an object extending from Node:
Optional<? extends Node> optionalInner = createInnerNode();
Optional<? extends Node> optionalLeaf = createLeafNode();
Optional<Node> node = optionalInner.or(() -> optionalLeaf);
I get the following compiler error:
Bad return type in lambda expression: Optional<capture of ? extends Node> cannot be converted to Optional<? extends capture of ? extends Node>
The only way I found to make this work, is using an Optional.empty in front:
Optional<Node> node = Optional.<Node>empty() // requires generic argument
.or(() -> createInnerNode())
.or(() -> createLeafNode());
But for me, this is confusing to read. Is it somehow possible to instruct the compiler to allow the statement optionalInner.or(() -> optionalLeaf)? Or are there other alternatives to make this work?
The workaround you provided is a pretty simple answer, but you can convert an Optional<InnerNode> to an Optional<Node> trivially with map if you want to avoid a placeholder Optional.
createInnerNode.<Node>map(Function.identity()).or(() -> createLeafNode());
In general, this is also how you conceptually turn a List<Dog> to a List<Animal> or any generic collection to a different generic collection in java as well. It's somewhat annoying, but wildcards don't cut it because you want both read/write on the transformed collection.
The signature of or should tell you why your workaround works
Optional<T> or(Supplier<? extends Optional<? extends T>> supplier)
notice how it's already using the ? extends T in the supplier.
Since the first empty Optional's T is Node, the or function can accept a supplier that returns
Optional<? extends Node>
and since both the innernode and leafnode are an Optional<? extends Node>, this works.
So the problem is that you need an Optional<Node> at some point.
I don't think you can avoid doing something since an Optional<InnerNode> never type matches Optional<Node> for reasons listed in other answers (and other questions on stack overflow).
Reading the other answer and trying
Optional<? extends Node> node = createInnerNode();
means that the T is a
? extends Node
which makes or seem to want doubly nested captures? I can't stop getting
Bad return type in lambda expression: Optional<capture of ? extends Node> cannot be converted to Optional<? extends capture of ? extends Node>
when trying basically any cast with the following code
Optional<? extends Node> node = Node.createInnerNode();
node.or(() -> (Optional<? extends Node>) null);
This must be a confusing error. Let me first explain the underlying problem so you actually understand why javac is complaining, and then I'll get to how to fix it.
The underlying problem is lack of use-site variance.
The error you get just doesn't make sense for Optional, but it makes perfect sense when we use a type that produces and consumes typed data (unlike Optional which, as an instance, only produces data, e.g. when you call .get() - there is no .set() method on Optional; it's immutable).
So let's look at lists. Imagine you could do this:
Integer i = 5;
Number n = i; // This is legal java.
List<Integer> ints = new ArrayList<Integer>();
List<Number> numbers = ints; // So this should be too... right?
That 4th line is not legal. Because generics are by default invariant (Neither supertypes nor subtypes will do; in an invariant typing system, if a Number is required, then only a Number will do) - and that is correct. After all, if the above code WAS legal, then everything breaks:
Double d = 5.0;
numbers.add(d); // Fine - must be. doubles are numbers!
Integer i = ints.get(0); // um.. wait. uhoh.
Because numbers and ints are both just referencing the exact same list, if I add a double to numbers, that means I also added one to ints, and - boom. It's broken.
Now you know why generics are invariant.
This also explains a few further compiler errors:
List<Integer> ints = new ArrayList<Integer>();
List<Number> numbers = ints; // nope, won't compile.
List<? extends Number> numbers = ints; // but this will!
numbers.add(anything-except-literally-null); // this wont!
With ? extends you are telling java that you want covariance (covariance = a subtype of X is just as good as X itself. Java's non-generics typing is covariant). However, covariance in generics automatically means that ALL ways to 'send' typed data to that thing are disabled. You CANNOT add anything (except literally .add(null) as null is all types) to a List<?> or a List<? extends>. This makes sense if you think about it:
A List<? extends Number> could be a List<Integer> or a List<Double>. So how can you add anything to this list? There is nothing that is both a Double and an Integer (well, except the literal null), so nothing you could possibly pass to add is neccessarily 'safe'. Hence why the compiler always tells you its a typing violation.
This also means trivially that this won't work either:
List<? extends Number> numbers = ...;
List<Number> numbers2 = numbers; // nope, this won't compile
Back to your code
Now, for optional, none of this seems relevant. Optional can be completely covariant in its type argument because Optional does not 'consume' data - none of its methods take in an 'E'. But the compiler isn't some sort of megabrain that susses this out and lets you 'break' the variance rules of typeargs. Not so - Optional needs to stick to the same rules list does. So, if an instance of List<? extends Number> cannot be assigned to a variable of type List<Number>, then the same applies to Optional:
An instance of type Optional<? extends Node> cannot be assigned to a variable of type Optional<Node>
And that explains your errors (for both of the attempts to solve the problem stated in your question).
Even though for optional specifically these errors don't make much sense. Now you know why you are getting them.
The best fix is to not use optional here; this isn't really what its meant for and it's not very java-like (see below). But the distant second best option is to fix this stuff. The root fix is that all places that take in an Optional should ALWAYS be stated as ? extends. So, you're trying to assign it to a field of type Optional<Node>, but that field is 'wrong'. That field's type should be Optional<? extends NOde>. Then that fixes all problems:
Optional<? extends Node> node = createInnerNode().or(() -> createLeafNode());
Works fine (that's your first snippet, but with the 'type' of the target fixed).
Ah. But, I can't change the API
Well, then the API is just wrong. It happens - sometimes you need to interact with broken code. But make no mistake, that is broken code. You do the same thing you do with any other broken code: Encapsulate and work around.
You don't want to interact with broken API. So make a wrapper type or helper methods that 'paper over' the error. And in that code (the 'bridging' code that works around the buggy code you cannot change), you accept that you need to write wonky code that generates warnings.
You can 'fix things' as follows:
public Optional<Node> fixIt(Optional<? extends Node> node) {
Optional raw = node;
return raw;
}
This works because generics in the end is just a figment of javac's imagination - the JVM doesn't know what generics are at all. So we just need to 'fake out' javac to stop refusing to compile the code. The above WILL generate warnings and any errors in such hackery result in getting ClassCastExceptions on lines with zero casts on them - quite confusing. But, works fine if you know what you're doing. You can eliminate the warnings with a #SuppressWarnings annotation.
Or.. just don't use optional
Java has null baked into many many places. Not just in its core model (fields that aren't initialized begin with null values. Arrays of non-primitives are initialized with null values), but also in a great many of its most crucial libraries. The most obvious and simple example is java.util.Map: It has a get() method that returns a V. Not an Optional<V>. And in a world where Optional is the right answer, get() should of course return Optional<V>. But it does not, and it never will - java doesn't break backwards compatibility that drastically.
Thus, the hypothetical world of 'optional everywhere!' sucks - 30 years of built up code and history needs to be tossed in the bin first and the community is rather unlikely to do that. If you dislike null handling in java (and there are plenty of fine reasons to do so!), then a solution is required that lets existing code backwards-compatibly adjust, and Optional cannot deliver. (nullity annotations probably can, though! Much better; or just write better APIs. Map has .getOrDefault which can be used to ensure your map query calls don't ever return null in the first place. Once you start writing your code to never produce null values except in cases where you truly WANT any attempt to deref to throw (i.e. very rarely), then null ceases to be a problem. These more modern addons to the various java core library types make it much easier to write such code. And adding methods to interfaces is backwards compatible, so, as java.util.Map shows, existing stuff can backwards-compatibly add these).
The best thing to do with Optional is to use it solely for why it was introduced in the first place: As a return value for stream API terminals. Something like intList.stream().max() returns an OptionalInt because what is the maximum of an empty list? Its fine enough for that. If you write your own collectors or stream terminating operations, use Optional. If that's not what you are doing, don't use it.
In the following code Java, I have created a list nums. I can assign the another list during the declaration. But new items cannot be added except the null. So, does it mean the nums is readonly? Why? Is it possible to add new items in that list?
List<Integer> ints = new ArrayList<Integer>();
ints.add(1);
ints.add(2);
List<? extends Number> nums = ints;
nums.add(3.14); //Generates error
nums.addAll(ints); //Generates error
nums.add(null); //works
System.out.println(nums.get(0)); //works
I have gone through this link. I can't get exact reason.
Is it possible to add new items in that list?
Nope... because that code doesn't know what it's "actually" a list of. Imagine if you could:
List<String> strings = new ArrayList<>();
List<? extends Object> objects = strings; // This is fine
objects.add(new Object()); // Fortunately this *isn't* valid...
System.out.println(strings.get(0).length()); // Or what would this do?
Basically, when you use a wildcard like ? extends T you can only get values out via the API... and when you use a wildcard like ? super T, you can only put the values in via the API - because that's what's safe.
No, it's not read-only... even though that is typically the intention.
Given a List<? extends Number> object, the compiler converts its type to List<X> where X is an unknown subtype of Number. Therefore, the object does have an add(X) method. We can call the method with an X argument... for example, null.
And since get() returns X, we could also call add() with a value from get() .... Directly invoking list.add(list.get(i)) won't work, even though it makes sense. We will need a little helper.
The classic example is Collections.reverse(List<? extends Object> list). This method will modify the list, despite the wildcard.
You can also call mutating methods like clear(), of course, on any list.
That being said, wildcard is indeed mainly for use-site variance, and most often, it conveys the intention from the API designer of whether a type-parameter is intended for in or out. For example, by declaring List<? super/extends Foo>, the API expresses that it intends to inject T in to, or, get T out of, the list.
It is a misconception that wildcard makes read/write-only. But this misconception works in most use cases. And the more people having this misconception, the more it becomes a convention...
see my article on wildcard - http://bayou.io/draft/Capturing_Wildcards.html
It's helps when you think of List<? extends Number> nums as a List of some type of thing that extends Number, but you can't be sure what. As such, everything you do with nums needs to be able to done to such a thing.
Adding null works because null can be cast into absolutely anything. Everything else you try to add will fail, because the compiler can't be 100% certain that the thing you're adding extends the thing the list is made of.
Instead of List<? extends Number> nums do List<Number> nums, because you can still put in anything that extends Number.
? doesn't mean "anything", it is closer to meaning "some specific but unknown".
Generics is compile time only.
So Compiler will decide what is actual type we are going to use.
List<? extends Number>
It means we are not sure what actual type of the object.
So Compiler not make sure what is the actual type that list have.
I'm having difficulty understanding wildcards in Java generics. Specifically I have the following questions:
If we have a LinkedList<?>, why can we not add an Object to it? I understand that it doesn't know the type of the list, but wouldn't adding an Object to the list have us covered in any situation?
Similar to the question above, if we have LinkedList<? extends Number>, why can we not add a Number to it?
Finally, if we have LinkedList<? super Number>, why can we add an Integer to the list, shouldn't we only be able to add things that are a superclass of Number?
I guess I'm trying to understand how wildcards work in general, I've read the Oracle tutorials on them, and a few other things, but I don't understand why they work I suppose.
You're confusing objects and types.
Unlike simple generic parameters, wildcards describe the type of the generic parameter.
A List<? super Number> isn't a list of superclasses of Number; it's a list of some unknown type, where that type is a superclass of number.
A LinkedList<?> might be a LinkedList<Car>.
Since an Object is not a Car, you can't add an Object to it.
In fact, since you don't know anything about what type the list contains, you can't add anything to it. (except null)
Similarly, a LinkedList<? extends Number> might be a List<Long>, so you can't add an Integer to it. (since an Integer is not a Long)
On the other hand, a List<? super Number> is definitely allowed to contain Number or any derived class, since it can only be a list of one of Number's superclasses (eg, List<Object>)
A book I am reading on Java tells me that the following two pieces of code are equivalent:
public <T extends Animal> void takeThing(ArrayList<T> list)
public void takeThing(ArrayList<? extends Animal> list);
On the opposite page, I am informed that the latter piece of code uses the '?' as a wildcard, meaning that nothing can be added to the list.
Does this mean that if I ever have a list (or other collection types?) that I can't make them simultaneously accept polymorphic arguments AND be re-sizable? Or have I simply misunderstood something?
All help/comments appreciated, even if they go slightly off topic. Thanks.
Does this mean that if I ever have a list (or other collection types?) that I can't make them simultaneously accept polymorphic arguments AND be re-sizable?
No.
The two pieces of code are not completely equivalent. In the first line, the method takeThing has a type parameter T. In the second line, you're using a wildcard.
When you would use the first version, you would specify what concrete type would be used for T. Because the concrete type is then known, there's no problem to add to the list.
In the second version, you're just saying "list is an ArrayList that contains objects of some unknown type that extends Animal". What exactly that type is, isn't known. You can't add objects to such a list because the compiler doesn't have enough information (it doesn't know what the actual type is) to check if what you're adding to the list should be allowed.
Usually, if adding to a list is involved inside a method that accepts just the list without the thing to add, you'll have somewhere else something that is an Animal and you'll want to add it to the list. In this case your method must be declared so that all the list types it accepts allow adding an Animal into them. This will have to be a List<Animal> or a list of some supertype of Animal. It can't possibly be a List<Dog>— the element you are adding could be any Animal.
This is where the concept of the lower bound, and the keyword super, come in. The type declaration List<? super Animal> matches all the acceptable types as described above. On the other hand, you won't be able to get elements out of such a list in a typesafe way because they can in general be of any type at all. If a method wants to both add and get elements of declared type Animal, the only valid type it can accept is a List<Animal>.