Related
Having been using Java 8 now for 6+ months or so, I'm pretty happy with the new API changes. One area I'm still not confident in is when to use Optional. I seem to swing between wanting to use it everywhere something may be null, and nowhere at all.
There seem to be a lot of situations when I could use it, and I'm never sure if it adds benefits (readability / null safety) or just causes additional overhead.
So, I have a few examples, and I'd be interested in the community's thoughts on whether Optional is beneficial.
1 - As a public method return type when the method could return null:
public Optional<Foo> findFoo(String id);
2 - As a method parameter when the param may be null:
public Foo doSomething(String id, Optional<Bar> barOptional);
3 - As an optional member of a bean:
public class Book {
private List<Pages> pages;
private Optional<Index> index;
}
4 - In Collections:
In general I don't think:
List<Optional<Foo>>
adds anything - especially since one can use filter() to remove null values etc, but are there any good uses for Optional in collections?
Any cases I've missed?
The main design goal of Optional is to provide a means for a function returning a value to indicate the absence of a return value. See this discussion. This allows the caller to continue a chain of fluent method calls.
This most closely matches use case #1 in the OP's question. Although, absence of a value is a more precise formulation than null since something like IntStream.findFirst could never return null.
For use case #2, passing an optional argument to a method, this could be made to work, but it's rather clumsy. Suppose you have a method that takes a string followed by an optional second string. Accepting an Optional as the second arg would result in code like this:
foo("bar", Optional.of("baz"));
foo("bar", Optional.empty());
Even accepting null is nicer:
foo("bar", "baz");
foo("bar", null);
Probably the best is to have an overloaded method that accepts a single string argument and provides a default for the second:
foo("bar", "baz");
foo("bar");
This does have limitations, but it's much nicer than either of the above.
Use cases #3 and #4, having an Optional in a class field or in a data structure, is considered a misuse of the API. First, it goes against the main design goal of Optional as stated at the top. Second, it doesn't add any value.
There are three ways to deal with the absence of a value in an Optional: to provide a substitute value, to call a function to provide a substitute value, or to throw an exception. If you're storing into a field, you'd do this at initialization or assignment time. If you're adding values into a list, as the OP mentioned, you have the additional choice of simply not adding the value, thereby "flattening" out absent values.
I'm sure somebody could come up with some contrived cases where they really want to store an Optional in a field or a collection, but in general, it is best to avoid doing this.
I'm late to the game but for what it's worth, I want to add my 2 Cents. They go against the design goal of Optional, which is well summarized by Stuart Marks's answer, but I'm still convinced of their validity (obviously).
Use Optional Everywhere
In General
I wrote an entire blog post about using Optional but it basically comes down to this:
design your classes to avoid optionality wherever feasibly possible
in all remaining cases, the default should be to use Optional instead of null
possibly make exceptions for:
local variables
return values and arguments to private methods
performance critical code blocks (no guesses, use a profiler)
The first two exceptions can reduce the perceived overhead of wrapping and unwrapping references in Optional. They are chosen such that a null can never legally pass a boundary from one instance into another.
Note that this will almost never allow Optionals in collections which is almost as bad as nulls. Just don't do it. ;)
Regarding your questions
Yes.
If overloading is no option, yes.
If other approaches (subclassing, decorating, ...) are no option, yes.
Please no!
Advantages
Doing this reduces the presence of nulls in your code base, although it does not eradicate them. But that is not even the main point. There are other important advantages:
Clarifies Intent
Using Optional clearly expresses that the variable is, well, optional. Any reader of your code or consumer of your API will be beaten over the head with the fact that there might be nothing there and that a check is necessary before accessing the value.
Removes Uncertainty
Without Optional the meaning of a null occurrence is unclear. It could be a legal representation of a state (see Map.get) or an implementation error like a missing or failed initialization.
This changes dramatically with the persistent use of Optional. Here, already the occurrence of null signifies the presence of a bug. (Because if the value were allowed to be missing, an Optional would have been used.) This makes debugging a null pointer exception much easier as the question of the meaning of this null is already answered.
More Null Checks
Now that nothing can be null anymore, this can be enforced everywhere. Whether with annotations, assertions or plain checks, you never have to think about whether this argument or that return type can be null. It can't!
Disadvantages
Of course, there is no silver bullet...
Performance
Wrapping values (especially primitives) into an extra instance can degrade performance. In tight loops this might become noticeable or even worse.
Note that the compiler might be able to circumvent the extra reference for short lived lifetimes of Optionals. In Java 10 value types might further reduce or remove the penalty.
Serialization
Optional is not serializable but a workaround is not overly complicated.
Invariance
Due to the invariance of generic types in Java, certain operations become cumbersome when the actual value type is pushed into a generic type argument. An example is given here (see "Parametric polymorphism").
Personally, I prefer to use IntelliJ's Code Inspection Tool to use #NotNull and #Nullable checks as these are largely compile time (can have some runtime checks) This has lower overhead in terms of code readability and runtime performance. It is not as rigorous as using Optional, however this lack of rigour should be backed by decent unit tests.
public #Nullable Foo findFoo(#NotNull String id);
public #NotNull Foo doSomething(#NotNull String id, #Nullable Bar barOptional);
public class Book {
private List<Pages> pages;
private #Nullable Index index;
}
List<#Nullable Foo> list = ..
This works with Java 5 and no need to wrap and unwrap values. (or create wrapper objects)
I think the Guava Optional and their wiki page puts it quite well:
Besides the increase in readability that comes from giving null a name, the biggest advantage of Optional is its idiot-proof-ness. It forces you to actively think about the absent case if you want your program to compile at all, since you have to actively unwrap the Optional and address that case. Null makes it disturbingly easy to simply forget things, and though FindBugs helps, we don't think it addresses the issue nearly as well.
This is especially relevant when you're returning values that may or may not be "present." You (and others) are far more likely to forget that other.method(a, b) could return a null value than you're likely to forget that a could be null when you're implementing other.method. Returning Optional makes it impossible for callers to forget that case, since they have to unwrap the object themselves for their code to compile.
-- (Source: Guava Wiki - Using and Avoiding null - What's the point?)
Optional adds some overhead, but I think its clear advantage is to make it explicit
that an object might be absent and it enforces that programmers handle the situation. It prevents that someone forgets the beloved != null check.
Taking the example of 2, I think it is far more explicit code to write:
if(soundcard.isPresent()){
System.out.println(soundcard.get());
}
than
if(soundcard != null){
System.out.println(soundcard);
}
For me, the Optional better captures the fact that there is no soundcard present.
My 2¢ about your points:
public Optional<Foo> findFoo(String id); - I am not sure about this. Maybe I would return a Result<Foo> which might be empty or contain a Foo. It is a similar concept, but not really an Optional.
public Foo doSomething(String id, Optional<Bar> barOptional); - I would prefer #Nullable and a findbugs check, as in Peter Lawrey's answer - see also this discussion.
Your book example - I am not sure if I would use the Optional internally, that might depend on the complexity. For the "API" of a book, I would use an Optional<Index> getIndex() to explicitly indicate that the book might not have an index.
I would not use it in collections, rather not allowing null values in collections
In general, I would try to minimize passing around nulls. (Once burnt...)
I think it is worth to find the appropriate abstractions and indicate to the fellow programmers what a certain return value actually represents.
From Oracle tutorial:
The purpose of Optional is not to replace every single null reference in your codebase but rather to help design better APIs in which—just by reading the signature of a method—users can tell whether to expect an optional value. In addition, Optional forces you to actively unwrap an Optional to deal with the absence of a value; as a result, you protect your code against unintended null pointer exceptions.
In java, just don't use them unless you are addicted to functional programming.
They have no place as method arguments (I promess someone one day will pass you a null optional, not just an optional that is empty).
They make sense for return values but they invite the client class to keep on stretching the behavior-building chain.
FP and chains have little place in an imperative language like java because it makes it very hard to debug, not just to read. When you step to the line, you can't know the state nor intent of the program; you have to step into to figure it out (into code that often isn't yours and many stack frames deep despite step filters) and you have to add lots of breakpoints down to make sure it can stop in the code/lambda you added, instead of simply walking the if/else/call trivial lines.
If you want functional programming, pick something else than java and hope you have the tools for debugging that.
1 - As a public method return type when the method could return null:
Here is a good article that shows usefulness of usecase #1. There this code
...
if (user != null) {
Address address = user.getAddress();
if (address != null) {
Country country = address.getCountry();
if (country != null) {
String isocode = country.getIsocode();
isocode = isocode.toUpperCase();
}
}
}
...
is transformed to this
String result = Optional.ofNullable(user)
.flatMap(User::getAddress)
.flatMap(Address::getCountry)
.map(Country::getIsocode)
.orElse("default");
by using Optional as a return value of respective getter methods.
Here is an interesting usage (I believe) for... Tests.
I intend to heavily test one of my projects and I therefore build assertions; only there are things I have to verify and others I don't.
I therefore build things to assert and use an assert to verify them, like this:
public final class NodeDescriptor<V>
{
private final Optional<String> label;
private final List<NodeDescriptor<V>> children;
private NodeDescriptor(final Builder<V> builder)
{
label = Optional.fromNullable(builder.label);
final ImmutableList.Builder<NodeDescriptor<V>> listBuilder
= ImmutableList.builder();
for (final Builder<V> element: builder.children)
listBuilder.add(element.build());
children = listBuilder.build();
}
public static <E> Builder<E> newBuilder()
{
return new Builder<E>();
}
public void verify(#Nonnull final Node<V> node)
{
final NodeAssert<V> nodeAssert = new NodeAssert<V>(node);
nodeAssert.hasLabel(label);
}
public static final class Builder<V>
{
private String label;
private final List<Builder<V>> children = Lists.newArrayList();
private Builder()
{
}
public Builder<V> withLabel(#Nonnull final String label)
{
this.label = Preconditions.checkNotNull(label);
return this;
}
public Builder<V> withChildNode(#Nonnull final Builder<V> child)
{
Preconditions.checkNotNull(child);
children.add(child);
return this;
}
public NodeDescriptor<V> build()
{
return new NodeDescriptor<V>(this);
}
}
}
In the NodeAssert class, I do this:
public final class NodeAssert<V>
extends AbstractAssert<NodeAssert<V>, Node<V>>
{
NodeAssert(final Node<V> actual)
{
super(Preconditions.checkNotNull(actual), NodeAssert.class);
}
private NodeAssert<V> hasLabel(final String label)
{
final String thisLabel = actual.getLabel();
assertThat(thisLabel).overridingErrorMessage(
"node's label is null! I didn't expect it to be"
).isNotNull();
assertThat(thisLabel).overridingErrorMessage(
"node's label is not what was expected!\n"
+ "Expected: '%s'\nActual : '%s'\n", label, thisLabel
).isEqualTo(label);
return this;
}
NodeAssert<V> hasLabel(#Nonnull final Optional<String> label)
{
return label.isPresent() ? hasLabel(label.get()) : this;
}
}
Which means the assert really only triggers if I want to check the label!
Optional class lets you avoid to use null and provide a better alternative:
This encourages the developer to make checks for presence in order to avoid uncaught NullPointerException's.
API becomes better documented because it's possible to see, where to expect the values which can be absent.
Optional provides convenient API for further work with the object:
isPresent(); get(); orElse(); orElseGet(); orElseThrow(); map(); filter(); flatmap().
In addition, many frameworks actively use this data type and return it from their API.
An Optional has similar semantics to an unmodifiable instance of the Iterator design pattern:
it might or might not refer to an object (as given by isPresent())
it can be dereferenced (using get()) if it does refer to an object
but it can not be advanced to the next position in the sequence (it has no next() method).
Therefore consider returning or passing an Optional in contexts where you might previously have considered using a Java Iterator.
Here are some of the methods that you can perform on an instance of Optional<T>:
map
flatMap
orElse
orElseThrow
ifPresentOrElse
get
Here are all the methods that you can perform on null:
(there are none)
This is really an apples to oranges comparison: Optional<T> is an actual instance of an object (unless it is null… but that would probably be a bug) while null is an aborted object. All you can do with null is check whether it is in fact null, or not. So if you like to use methods on objects, Optional<T> is for you; if you like to branch on special literals, null is for you.
null does not compose. You simply can’t compose a value which you can only branch on. But Optional<T> does compose.
You can, for instance, make arbitrary long chains of “apply this function if non-empty” by using map. Or you can effectively make an imperative block of code which consumes the optional if it is non-empty by using ifPresent. Or you can make an “if/else” by using ifPresentOrElse, which consumes the non-empty optional if it is non-empty or else executes some other code.
…And it is at this point that we run into the true limitations of the language in my opinion: for very imperative code you have to wrap them in lambdas and pass them to methods:
opt.ifPresentOrElse(
string -> { // if present...
// ...
}, () -> { // or else...
// ...
}
);
That might not be good enough for some people, style-wise.
It would be more seamless if Optional<T> was an algebraic data type that we could pattern match on (this is obviously pseudo-code:
match (opt) {
Present(str) => {
// ...
}
Empty =>{
// ...
}
}
But anyway, in summary: Optional<T> is a pretty robust empty-or-present object. null is just a sentinel value.
Subjectively disregarded reasons
There seems to be a few people who effectively argue that efficiency should determine whether one should use Optional<T> or branch on the null sentinel value. That seems a bit like making hard and fast rules on when to make objects rather than primitives in the general case. I think it’s a bit ridiculous to use that as the starting point for this discussion when you’re already working in a language where it’s idiomatic to make objects left-and-right, top to bottom, all the time (in my opinion).
I do not think that Optional is a general substitute for methods that potentially return null values.
The basic idea is: The absence of a value does not mean that it potentially is available in the future. It's a difference between findById(-1) and findById(67).
The main information of Optionals for the caller is that he may not count on the value given but it may be available at some time. Maybe it will disappear again and comes back later one more time. It's like an on/off switch. You have the "option" to switch the light on or off. But you have no option if you do not have a light to switch on.
So I find it too messy to introduce Optionals everywhere where previously null was potentially returned. I will still use null, but only in restricted areas like the root of a tree, lazy initialization and explicit find-methods.
Seems Optional is only useful if the type T in Optional is a primitive type like int, long, char, etc. For "real" classes, it does not make sense to me as you can use a null value anyway.
I think it was taken from here (or from another similar language concept).
Nullable<T>
In C# this Nullable<T> was introduced long ago to wrap value types.
1 - As a public method return type when the method could return null:
This is the intended use case for Optional, as seen in the JDK API docs:
Optional is primarily intended for use as a method return type where
there is a clear need to represent "no result," and where using null
is likely to cause errors.
Optional represents one of two states:
it has a value (isPresent returns true)
it doesn't have a value (isEmpty returns true)
So if you have a method that returns either something or nothing, this is the ideal use case for Optional.
Here's an example:
Optional<Guitarist> findByLastName(String lastName);
This method takes a parameter used to search for an entity in the database. It's possible that no such entity exists, so using an Optional return type is a good idea since it forces whoever is calling the method to consider the empty scenario. This reduces chances of a NullPointerException.
2 - As a method parameter when the param may be null:
Although technically possible, this is not the intended use case of Optional.
Let's consider your proposed method signature:
public Foo doSomething(String id, Optional<Bar> barOptional);
The main problem is that we could call doSomething where barOptional has one of 3 states:
an Optional with a value e.g. doSomething("123", Optional.of(new Bar())
an empty Optional e.g. doSomething("123", Optional.empty())
null e.g. doSomething("123", null)
These 3 states would need to be handled in the method implementation appropriately.
A better solution is to implement an overloaded method.
public Foo doSomething(String id);
public Foo doSomething(String id, Bar bar);
This makes it very clear to the consumer of the API which method to call, and null does not need to be passed.
3 - As an optional member of a bean:
Given your example Book class:
public class Book {
private List<Pages> pages;
private Optional<Index> index;
}
The Optional class variable suffers from the same issue as the Optional method parameter discussed above. It can have one of 3 states: present, empty, or null.
Other possible issues include:
serialization: if you implement Serializable and try to serialize an object of this class, you will encounter a java.io.NotSerializableException since Optional was not designed for this use case
transforming to JSON: when serializing to JSON an Optional field may get mapped in an undesirable way e.g. {"empty":false,"present":true}.
Although if you use the popular Jackson library, it does provide a solution to this problem.
Despite these issues, Oracle themselves published this blog post at the time of the Java 8 Optional release in 2014. It contains code examples using Optional for class variables.
public class Computer {
private Optional<Soundcard> soundcard;
public Optional<Soundcard> getSoundcard() { ... }
...
}
In the following years though, developers have found better alternatives such as implementing a getter method to create the Optional object.
public class Book {
private List<Pages> pages;
private Index index;
public Optional<Index> getIndex() {
return Optional.ofNullable(index);
}
}
Here we use the ofNullable method to return an Optional with a value if index is non-null, or otherwise an empty Optional.
4 - In Collections:
I agree that creating a List of Optional (e.g. List<Optional<Foo>>) doesn't add anything.
Instead, just don't include the item in the List if it's not present.
I've been reading up a lot about the cases where Optional should be used.
A lot of the pages I've read say that Optional shouldn't be used for private instance variables and instead should be returned by getters.
I would have thought that having private instance variables as optional would still be useful. If anybody looks at my code they can see that a value can be empty, instead of having to check the documentation to see if null could be returned.
In Scala null is never used and is only really there for interoperability with Java. It is recommended to always use an optional if a value can be null. This approach makes a lot more sense to me.
Here's a page mentioning it:
https://blog.joda.org/2015/08/java-se-8-optional-pragmatic-approach.html
Here is the example code.
private final String addressLine; // never null
private final String city; // never null
private final String postcode; // optional, thus may be null
// normal getters
public String getAddressLine() { return addressLine; }
public String getCity() { return city; }
// special getter for optional field
public Optional<String> getPostcode() {
return Optional.ofNullable(postcode);
}
The only advantage I can see is that if you want to serialize the object it is now possible as it isn't storing optional in a variable.
The disadvantage is that you don't know postcode could be null until you check the return type of the getter. If you were new to the code you might miss this add extend the class, causing a null pointer exception.
Here is a question asked about Scala's Option.
When to use Option
Why is there a difference between Java and Scala with how optional should be used?
Not all Java developers even agree with the approach you described. Please check this post by the creator of Lombok.
I guess that the reason for a different approach of using Optional in Java is that Java's community lived without it up till Java 8, so most people were used to null. In one hand a lot of new APIs (like findAny from Stream) return Optional, but still a lot of standard library methods just return null, so you always have to remember either to wrap your function calls with Optional.ofNullable or check if the value is not null.
Optional was added to Java 8 but it is discouraged to use it as class fields because Optional is not implementing Serializable (and Java's serialization is used as the default serialization engine by many frameworks or systems like Akka, Spark, Kafka, etc.).
On the other hand Option is very tightly bound with the Scala standard library.
As far as I know, no Scala's standard library APIs return null, but Option, and it is discouraged to use null at all in your Scala code. You can even configure your project that it will fail to compile if null is used.
Option is also Serializable and its normal practice to use is as class fields for values that can be empty.
If you want to use a similar approach in your Java code please check Option from Vavr. It's serializable, so it can be safely used as fields and it also has two subclasses None and Some (similarly as Scala's Option), so it can be used in Vavr's pattern matching:
Match(option).of(
Case($Some($()), "defined"),
Case($None(), "empty")
);
In Scala, Option is tightly integrated into the language's API.
Represents optional values. Instances of Option are either an instance of scala.Some or the object None.
The most idiomatic way to use an scala.Option instance is to treat it as a collection or monad and use map,flatMap, filter, or foreach.
As you can see from the quote above, there is no null explanation there, because it should be used as a monad or as a collection.
In Java, Optional is used to save us from NullPointerException cases, by using it as:
A container object which may or may not contain a non-null value. If a value is present, isPresent() will return true and get() will return the value.
A programming language that shows the user very clearly if the variable can be null or not is Kotlin, by using ? safe call, and shows you compilation errors:
var a: String = "abc"
a = null // compilation error
var b: String? = "abc"
b = null // ok
print(b)
There is the position, that Optional, like Stream, is an iterating «event» class, not deserving to be used as field in a "real" object.
Then there is the short-coming of not being Serializable (which one may work around).
However my opinion is that some variables may be optional, of cardinality 0 and 1.
Just as a list is valid too. With List fields the trend is (IMHO) that it is not let to be null initially, but to always have an (empty) list.
In the same coding style, an Optional ensures that only safe access is used, and especially one may map in a chaining style:
Optional<TextField> t = ...
String s = t.map(t::getText).orElse("N/A");
Optional<String> u = t.map(t::getText);
t.ifPresent(s -> { ... });
Especially the ifPresent ensures not working with null unintendedly.
Optional is a valuable specification.
One reason not to use Optional for instance variables is that it's too easy for the Optional reference itself to be null. This violates an implied contract around Optional, namely that you should never create or encounter a null reference to an Optional.
class Foo {
Optional<Bar> bar;
}
Foo f = new Foo();
if (f.bar.isPresent()) { // NullPointerException
...
}
This is the simplest way to create a class with an Optional field in Java. In a sense the language encourages you to write code like this.
On the other hand, in Scala, the simplest way to define Foo is
case class Foo(bar: Option[Bar])
and it's so easy to do
case class Foo(bar: Option[Bar] = None)
both of which avoid the NPE.
Assume that I have this code in my project:
public List<T> getAll(Integer parameter) {
if(parameter != null && parameter > -1) {
// do something here
}
}
My question is: how can I do the checking with Optional instead of using if, or if there is another thing I could use?
Optional's design was inspired by monads known from the world of Functional Programming. In such case, it could look like this:
public List<T> getAll(Integer parameter) {
return Optional.ofNullable(parameter)
.filter(p -> p > -1)
.map(p -> *some modification*) //this line is optional
.orElseGet(() -> *some default value, probably an empty list*)
}
In such case, condition checking is performed in the filter method. If a given Predicate is not matched, the underlying Optional will be empty.
If you want to perform an operation on the underlying value, use map method by providing it with a Function that should be applied to the element wrapped by the Optional.
At the end, you can simply perform some action using ifPresent(), thrown an exception orElseThrow or simply return a default value with orElse or orElseGet. Keep in mind that orElseGet accepts a Supplier instance which makes it evaluate the default value lazily.
If you want to dig deeper, check out #stuart-marks talk from Devoxx: https://www.youtube.com/watch?v=Ej0sss6cq14&t=2767s
In your case, there might be no need for an Optional at all. Just two methods. One with the parameter and the other without.
You should not use Optional for such a case
As Stuart Marks states in Optional - The Mother of All Bikesheds:
Rule #4: It's generally a bad idea to create an Optional for the specific purpose of chaining methods from it to get a value.
This creates a useless overhead without any improvement for code readability and maintainability. You should thus not wrap your parameter into an optional.
and
Rule #6: Avoid using Optional in fields, method parameters and collections.
Avoid using Optional in method parameters
it doesn't really work for making parameters optional
forces call sites to create Optionals for everything:
myMethod(Optional.of("some value"));
myMethod(Optional.empty());
See also Why should Java 8's Optional not be used in arguments.
Use Optional<Integer>
public List<T> getAll(Integer paramter) {
return Optional.ofNullable(parameter)
.filter(p -> p > -1);
}
The Optional<> is just an implementation of Optional pattern. The useful methods is also Optional<T>#orElse(T default) which sets the value to default in case there was a null,
If you already got interested but still can't answer your doubts, please ask for clarification in comments. I'll respond asap.
You need to be clear about what you expect of your method. Possibilities:
You want it to keep working as before, and handle the case where the client supplies a null
You want to change the method to accept Optional<Integer> as its parameter. The "no nulls" way, is to do this, and simply not accommodate callers who supply null -- if they get a NullPointerException, it's their own fault because they broke the rules.
You don't have to work with streams or lambdas to work with Optional (although of course, you can).
You could simply write:
public List<T> getAll(Optional<Integer> maybeParameter) {
if(maybeParameter.isPresent() && maybeParameter.get() > -1) {
// do something here
}
}
Look at Optional's other methods in JavaDoc, to see how else it can be used.
If you didn't want to change the signature of your method, you could use Optional.ofNullable() to turn an Integer into an Optional<Integer>:
public List<T> getAll(Integer parameter) {
Optional<Integer> maybeParameter = Optional.ofNullable(parameter);
// work with the Optional here as before...
}
However I question the value of doing this here. If your API accepts nulls, then handling it with Optional "behind the scenes" doesn't provide an advantage.
I feel the best use of Optional is to expose it as part of your API. If a parameter is optional, declare it as Optional rather than handling null (or better, use method overloading, so they don't have to pass anything at all). If a return value might not be set, make the return type Optional, rather than ever returning null.
import java.util.Optional;
public class ExampleNullCheck{
public static void main(String args[])
{
Integer a = null;
Optional<Integer> checkInt = Optional.ofNullable(a);
System.out.println(checkInt.isPresent());
}//main
}//class
Having been using Java 8 now for 6+ months or so, I'm pretty happy with the new API changes. One area I'm still not confident in is when to use Optional. I seem to swing between wanting to use it everywhere something may be null, and nowhere at all.
There seem to be a lot of situations when I could use it, and I'm never sure if it adds benefits (readability / null safety) or just causes additional overhead.
So, I have a few examples, and I'd be interested in the community's thoughts on whether Optional is beneficial.
1 - As a public method return type when the method could return null:
public Optional<Foo> findFoo(String id);
2 - As a method parameter when the param may be null:
public Foo doSomething(String id, Optional<Bar> barOptional);
3 - As an optional member of a bean:
public class Book {
private List<Pages> pages;
private Optional<Index> index;
}
4 - In Collections:
In general I don't think:
List<Optional<Foo>>
adds anything - especially since one can use filter() to remove null values etc, but are there any good uses for Optional in collections?
Any cases I've missed?
The main design goal of Optional is to provide a means for a function returning a value to indicate the absence of a return value. See this discussion. This allows the caller to continue a chain of fluent method calls.
This most closely matches use case #1 in the OP's question. Although, absence of a value is a more precise formulation than null since something like IntStream.findFirst could never return null.
For use case #2, passing an optional argument to a method, this could be made to work, but it's rather clumsy. Suppose you have a method that takes a string followed by an optional second string. Accepting an Optional as the second arg would result in code like this:
foo("bar", Optional.of("baz"));
foo("bar", Optional.empty());
Even accepting null is nicer:
foo("bar", "baz");
foo("bar", null);
Probably the best is to have an overloaded method that accepts a single string argument and provides a default for the second:
foo("bar", "baz");
foo("bar");
This does have limitations, but it's much nicer than either of the above.
Use cases #3 and #4, having an Optional in a class field or in a data structure, is considered a misuse of the API. First, it goes against the main design goal of Optional as stated at the top. Second, it doesn't add any value.
There are three ways to deal with the absence of a value in an Optional: to provide a substitute value, to call a function to provide a substitute value, or to throw an exception. If you're storing into a field, you'd do this at initialization or assignment time. If you're adding values into a list, as the OP mentioned, you have the additional choice of simply not adding the value, thereby "flattening" out absent values.
I'm sure somebody could come up with some contrived cases where they really want to store an Optional in a field or a collection, but in general, it is best to avoid doing this.
I'm late to the game but for what it's worth, I want to add my 2 Cents. They go against the design goal of Optional, which is well summarized by Stuart Marks's answer, but I'm still convinced of their validity (obviously).
Use Optional Everywhere
In General
I wrote an entire blog post about using Optional but it basically comes down to this:
design your classes to avoid optionality wherever feasibly possible
in all remaining cases, the default should be to use Optional instead of null
possibly make exceptions for:
local variables
return values and arguments to private methods
performance critical code blocks (no guesses, use a profiler)
The first two exceptions can reduce the perceived overhead of wrapping and unwrapping references in Optional. They are chosen such that a null can never legally pass a boundary from one instance into another.
Note that this will almost never allow Optionals in collections which is almost as bad as nulls. Just don't do it. ;)
Regarding your questions
Yes.
If overloading is no option, yes.
If other approaches (subclassing, decorating, ...) are no option, yes.
Please no!
Advantages
Doing this reduces the presence of nulls in your code base, although it does not eradicate them. But that is not even the main point. There are other important advantages:
Clarifies Intent
Using Optional clearly expresses that the variable is, well, optional. Any reader of your code or consumer of your API will be beaten over the head with the fact that there might be nothing there and that a check is necessary before accessing the value.
Removes Uncertainty
Without Optional the meaning of a null occurrence is unclear. It could be a legal representation of a state (see Map.get) or an implementation error like a missing or failed initialization.
This changes dramatically with the persistent use of Optional. Here, already the occurrence of null signifies the presence of a bug. (Because if the value were allowed to be missing, an Optional would have been used.) This makes debugging a null pointer exception much easier as the question of the meaning of this null is already answered.
More Null Checks
Now that nothing can be null anymore, this can be enforced everywhere. Whether with annotations, assertions or plain checks, you never have to think about whether this argument or that return type can be null. It can't!
Disadvantages
Of course, there is no silver bullet...
Performance
Wrapping values (especially primitives) into an extra instance can degrade performance. In tight loops this might become noticeable or even worse.
Note that the compiler might be able to circumvent the extra reference for short lived lifetimes of Optionals. In Java 10 value types might further reduce or remove the penalty.
Serialization
Optional is not serializable but a workaround is not overly complicated.
Invariance
Due to the invariance of generic types in Java, certain operations become cumbersome when the actual value type is pushed into a generic type argument. An example is given here (see "Parametric polymorphism").
Personally, I prefer to use IntelliJ's Code Inspection Tool to use #NotNull and #Nullable checks as these are largely compile time (can have some runtime checks) This has lower overhead in terms of code readability and runtime performance. It is not as rigorous as using Optional, however this lack of rigour should be backed by decent unit tests.
public #Nullable Foo findFoo(#NotNull String id);
public #NotNull Foo doSomething(#NotNull String id, #Nullable Bar barOptional);
public class Book {
private List<Pages> pages;
private #Nullable Index index;
}
List<#Nullable Foo> list = ..
This works with Java 5 and no need to wrap and unwrap values. (or create wrapper objects)
I think the Guava Optional and their wiki page puts it quite well:
Besides the increase in readability that comes from giving null a name, the biggest advantage of Optional is its idiot-proof-ness. It forces you to actively think about the absent case if you want your program to compile at all, since you have to actively unwrap the Optional and address that case. Null makes it disturbingly easy to simply forget things, and though FindBugs helps, we don't think it addresses the issue nearly as well.
This is especially relevant when you're returning values that may or may not be "present." You (and others) are far more likely to forget that other.method(a, b) could return a null value than you're likely to forget that a could be null when you're implementing other.method. Returning Optional makes it impossible for callers to forget that case, since they have to unwrap the object themselves for their code to compile.
-- (Source: Guava Wiki - Using and Avoiding null - What's the point?)
Optional adds some overhead, but I think its clear advantage is to make it explicit
that an object might be absent and it enforces that programmers handle the situation. It prevents that someone forgets the beloved != null check.
Taking the example of 2, I think it is far more explicit code to write:
if(soundcard.isPresent()){
System.out.println(soundcard.get());
}
than
if(soundcard != null){
System.out.println(soundcard);
}
For me, the Optional better captures the fact that there is no soundcard present.
My 2¢ about your points:
public Optional<Foo> findFoo(String id); - I am not sure about this. Maybe I would return a Result<Foo> which might be empty or contain a Foo. It is a similar concept, but not really an Optional.
public Foo doSomething(String id, Optional<Bar> barOptional); - I would prefer #Nullable and a findbugs check, as in Peter Lawrey's answer - see also this discussion.
Your book example - I am not sure if I would use the Optional internally, that might depend on the complexity. For the "API" of a book, I would use an Optional<Index> getIndex() to explicitly indicate that the book might not have an index.
I would not use it in collections, rather not allowing null values in collections
In general, I would try to minimize passing around nulls. (Once burnt...)
I think it is worth to find the appropriate abstractions and indicate to the fellow programmers what a certain return value actually represents.
From Oracle tutorial:
The purpose of Optional is not to replace every single null reference in your codebase but rather to help design better APIs in which—just by reading the signature of a method—users can tell whether to expect an optional value. In addition, Optional forces you to actively unwrap an Optional to deal with the absence of a value; as a result, you protect your code against unintended null pointer exceptions.
In java, just don't use them unless you are addicted to functional programming.
They have no place as method arguments (I promess someone one day will pass you a null optional, not just an optional that is empty).
They make sense for return values but they invite the client class to keep on stretching the behavior-building chain.
FP and chains have little place in an imperative language like java because it makes it very hard to debug, not just to read. When you step to the line, you can't know the state nor intent of the program; you have to step into to figure it out (into code that often isn't yours and many stack frames deep despite step filters) and you have to add lots of breakpoints down to make sure it can stop in the code/lambda you added, instead of simply walking the if/else/call trivial lines.
If you want functional programming, pick something else than java and hope you have the tools for debugging that.
1 - As a public method return type when the method could return null:
Here is a good article that shows usefulness of usecase #1. There this code
...
if (user != null) {
Address address = user.getAddress();
if (address != null) {
Country country = address.getCountry();
if (country != null) {
String isocode = country.getIsocode();
isocode = isocode.toUpperCase();
}
}
}
...
is transformed to this
String result = Optional.ofNullable(user)
.flatMap(User::getAddress)
.flatMap(Address::getCountry)
.map(Country::getIsocode)
.orElse("default");
by using Optional as a return value of respective getter methods.
Here is an interesting usage (I believe) for... Tests.
I intend to heavily test one of my projects and I therefore build assertions; only there are things I have to verify and others I don't.
I therefore build things to assert and use an assert to verify them, like this:
public final class NodeDescriptor<V>
{
private final Optional<String> label;
private final List<NodeDescriptor<V>> children;
private NodeDescriptor(final Builder<V> builder)
{
label = Optional.fromNullable(builder.label);
final ImmutableList.Builder<NodeDescriptor<V>> listBuilder
= ImmutableList.builder();
for (final Builder<V> element: builder.children)
listBuilder.add(element.build());
children = listBuilder.build();
}
public static <E> Builder<E> newBuilder()
{
return new Builder<E>();
}
public void verify(#Nonnull final Node<V> node)
{
final NodeAssert<V> nodeAssert = new NodeAssert<V>(node);
nodeAssert.hasLabel(label);
}
public static final class Builder<V>
{
private String label;
private final List<Builder<V>> children = Lists.newArrayList();
private Builder()
{
}
public Builder<V> withLabel(#Nonnull final String label)
{
this.label = Preconditions.checkNotNull(label);
return this;
}
public Builder<V> withChildNode(#Nonnull final Builder<V> child)
{
Preconditions.checkNotNull(child);
children.add(child);
return this;
}
public NodeDescriptor<V> build()
{
return new NodeDescriptor<V>(this);
}
}
}
In the NodeAssert class, I do this:
public final class NodeAssert<V>
extends AbstractAssert<NodeAssert<V>, Node<V>>
{
NodeAssert(final Node<V> actual)
{
super(Preconditions.checkNotNull(actual), NodeAssert.class);
}
private NodeAssert<V> hasLabel(final String label)
{
final String thisLabel = actual.getLabel();
assertThat(thisLabel).overridingErrorMessage(
"node's label is null! I didn't expect it to be"
).isNotNull();
assertThat(thisLabel).overridingErrorMessage(
"node's label is not what was expected!\n"
+ "Expected: '%s'\nActual : '%s'\n", label, thisLabel
).isEqualTo(label);
return this;
}
NodeAssert<V> hasLabel(#Nonnull final Optional<String> label)
{
return label.isPresent() ? hasLabel(label.get()) : this;
}
}
Which means the assert really only triggers if I want to check the label!
Optional class lets you avoid to use null and provide a better alternative:
This encourages the developer to make checks for presence in order to avoid uncaught NullPointerException's.
API becomes better documented because it's possible to see, where to expect the values which can be absent.
Optional provides convenient API for further work with the object:
isPresent(); get(); orElse(); orElseGet(); orElseThrow(); map(); filter(); flatmap().
In addition, many frameworks actively use this data type and return it from their API.
An Optional has similar semantics to an unmodifiable instance of the Iterator design pattern:
it might or might not refer to an object (as given by isPresent())
it can be dereferenced (using get()) if it does refer to an object
but it can not be advanced to the next position in the sequence (it has no next() method).
Therefore consider returning or passing an Optional in contexts where you might previously have considered using a Java Iterator.
Here are some of the methods that you can perform on an instance of Optional<T>:
map
flatMap
orElse
orElseThrow
ifPresentOrElse
get
Here are all the methods that you can perform on null:
(there are none)
This is really an apples to oranges comparison: Optional<T> is an actual instance of an object (unless it is null… but that would probably be a bug) while null is an aborted object. All you can do with null is check whether it is in fact null, or not. So if you like to use methods on objects, Optional<T> is for you; if you like to branch on special literals, null is for you.
null does not compose. You simply can’t compose a value which you can only branch on. But Optional<T> does compose.
You can, for instance, make arbitrary long chains of “apply this function if non-empty” by using map. Or you can effectively make an imperative block of code which consumes the optional if it is non-empty by using ifPresent. Or you can make an “if/else” by using ifPresentOrElse, which consumes the non-empty optional if it is non-empty or else executes some other code.
…And it is at this point that we run into the true limitations of the language in my opinion: for very imperative code you have to wrap them in lambdas and pass them to methods:
opt.ifPresentOrElse(
string -> { // if present...
// ...
}, () -> { // or else...
// ...
}
);
That might not be good enough for some people, style-wise.
It would be more seamless if Optional<T> was an algebraic data type that we could pattern match on (this is obviously pseudo-code:
match (opt) {
Present(str) => {
// ...
}
Empty =>{
// ...
}
}
But anyway, in summary: Optional<T> is a pretty robust empty-or-present object. null is just a sentinel value.
Subjectively disregarded reasons
There seems to be a few people who effectively argue that efficiency should determine whether one should use Optional<T> or branch on the null sentinel value. That seems a bit like making hard and fast rules on when to make objects rather than primitives in the general case. I think it’s a bit ridiculous to use that as the starting point for this discussion when you’re already working in a language where it’s idiomatic to make objects left-and-right, top to bottom, all the time (in my opinion).
I do not think that Optional is a general substitute for methods that potentially return null values.
The basic idea is: The absence of a value does not mean that it potentially is available in the future. It's a difference between findById(-1) and findById(67).
The main information of Optionals for the caller is that he may not count on the value given but it may be available at some time. Maybe it will disappear again and comes back later one more time. It's like an on/off switch. You have the "option" to switch the light on or off. But you have no option if you do not have a light to switch on.
So I find it too messy to introduce Optionals everywhere where previously null was potentially returned. I will still use null, but only in restricted areas like the root of a tree, lazy initialization and explicit find-methods.
Seems Optional is only useful if the type T in Optional is a primitive type like int, long, char, etc. For "real" classes, it does not make sense to me as you can use a null value anyway.
I think it was taken from here (or from another similar language concept).
Nullable<T>
In C# this Nullable<T> was introduced long ago to wrap value types.
1 - As a public method return type when the method could return null:
This is the intended use case for Optional, as seen in the JDK API docs:
Optional is primarily intended for use as a method return type where
there is a clear need to represent "no result," and where using null
is likely to cause errors.
Optional represents one of two states:
it has a value (isPresent returns true)
it doesn't have a value (isEmpty returns true)
So if you have a method that returns either something or nothing, this is the ideal use case for Optional.
Here's an example:
Optional<Guitarist> findByLastName(String lastName);
This method takes a parameter used to search for an entity in the database. It's possible that no such entity exists, so using an Optional return type is a good idea since it forces whoever is calling the method to consider the empty scenario. This reduces chances of a NullPointerException.
2 - As a method parameter when the param may be null:
Although technically possible, this is not the intended use case of Optional.
Let's consider your proposed method signature:
public Foo doSomething(String id, Optional<Bar> barOptional);
The main problem is that we could call doSomething where barOptional has one of 3 states:
an Optional with a value e.g. doSomething("123", Optional.of(new Bar())
an empty Optional e.g. doSomething("123", Optional.empty())
null e.g. doSomething("123", null)
These 3 states would need to be handled in the method implementation appropriately.
A better solution is to implement an overloaded method.
public Foo doSomething(String id);
public Foo doSomething(String id, Bar bar);
This makes it very clear to the consumer of the API which method to call, and null does not need to be passed.
3 - As an optional member of a bean:
Given your example Book class:
public class Book {
private List<Pages> pages;
private Optional<Index> index;
}
The Optional class variable suffers from the same issue as the Optional method parameter discussed above. It can have one of 3 states: present, empty, or null.
Other possible issues include:
serialization: if you implement Serializable and try to serialize an object of this class, you will encounter a java.io.NotSerializableException since Optional was not designed for this use case
transforming to JSON: when serializing to JSON an Optional field may get mapped in an undesirable way e.g. {"empty":false,"present":true}.
Although if you use the popular Jackson library, it does provide a solution to this problem.
Despite these issues, Oracle themselves published this blog post at the time of the Java 8 Optional release in 2014. It contains code examples using Optional for class variables.
public class Computer {
private Optional<Soundcard> soundcard;
public Optional<Soundcard> getSoundcard() { ... }
...
}
In the following years though, developers have found better alternatives such as implementing a getter method to create the Optional object.
public class Book {
private List<Pages> pages;
private Index index;
public Optional<Index> getIndex() {
return Optional.ofNullable(index);
}
}
Here we use the ofNullable method to return an Optional with a value if index is non-null, or otherwise an empty Optional.
4 - In Collections:
I agree that creating a List of Optional (e.g. List<Optional<Foo>>) doesn't add anything.
Instead, just don't include the item in the List if it's not present.
Guava's Optional pattern is great, as it helps remove the ambiguity with null. The transform method is very helpful for creating null-safe method chains when the first part of the chain may be absent, but isn't useful when other parts of the chain are absent.
This question is related to Guava Optional type, when transformation returns another Optional, which asks essentially the same question but for a different use case which I think may not be the intended use of Optional (handling errors).
Consider a method Optional<Book> findBook(String id). findBook(id).transform(Book.getName) works as expected. If there is no book found we get an Absent<String>, if there is a book found we get Present<String>.
In the common case where intermediate methods may return null/absent(), there does not seem to be an elegant way to chain the calls. For example, assume that Book has a method Optional<Publisher> getPublisher(), and we would like to get all the books published by the publisher of a book. The natural syntax would seem to be findBook(id).transform(Book.getPublisher).transform(Publisher.getPublishedBooks), however this will fail because the transform(Publisher.getPublishedBooks) call will actually return an Optional<Optional<Publisher>>.
It seems fairly reasonable to have a transform()-like method on Optional that would accept a function which returns an Optional. It would act exactly like the current implementation except that it simply would not wrap the result of the function in an Optional. The implementation (for Present) might read:
public abstract <V> Optional<V> optionalTransform(Function<? super T, Optional<V>> function) {
return function.apply(reference);
}
The implementation for Absent is unchanged from transform:
public abstract <V> Optional<V> optionalTransform(Function<? super T, Optional<V>> function) {
checkNotNull(function);
return Optional.absent();
}
It would also be nice if there were a way to handle methods that return null as opposed to Optional for working with legacy objects. Such a method would be like transform but simply call Optional.fromNullable on the result of the function.
I'm curious if anyone else has run into this annoyance and found nice workarounds (which don't involve writing your own Optional class). I'd also love to hear from the Guava team or be pointed to discussions related to the issue (I didn't find any in my searching).
You are looking for some Monad, but Guava's Optional (as opposite to for example Scala's Option) is just a Functor.
What the hell is a Functor?!
Functor and Monad are a kind of box, a context that wraps some value.
Functor containing some value of type A knows how to apply function A => B and put the result back into Functor. For example: get something out of Optional, transform, and wrap back into Optional.
In functional programming languages such method is often named 'map'.
Mona.. what?
Monad is almost the same thing as Functor, except that it consumes function returning value wrapped in Monad (A => Monad, for example Int => Optional).
This magic Monad's method is often called 'flatMap'.
Here you can find really awesome explanations for fundamental FP terms: http://adit.io/posts/2013-04-17-functors,_applicatives,_and_monads_in_pictures.html
Functors & Monads are coming!
Optional from Java 8 can be classified as both Functor (http://docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map-java.util.function.Function-) and Monad (http://docs.oracle.com/javase/8/docs/api/java/util/Optional.html#flatMap-java.util.function.Function-).
Nice mon(ad)olog, Marcin, but how can I solve my particular problem?
I'm currently working on a project that uses Java 6 and yesterday I write some helper class, called 'Optionals', which saved me a lot of time.
It provides some helper method, that allows me to turn Optional into Monads (flatMap).
Here is the code: https://gist.github.com/mkubala/046ae20946411f80ac52
Because my project's codebase still uses nulls as a return value, I introduced Optionals.lift(Function), which can be used to wrapping results into the Optional.
Why lifting result into Optional?
To avoid situation when function passed into transform might return null and whole expression would return "present of null" (which by the way is not possible with Guava's Optional, because of this postcondition -> see line #71 of https://code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/base/Present.java?r=0823847e96b1d082e94f06327cf218e418fe2228#71).
Couple of examples
Let's assume that findEntity() returns an Optional and Entity.getDecimalField(..) may return BigDecimal or null:
Optional<BigDecimal> maybeDecimalValue = Optionals.flatMap(
findEntity(),
new Function<Entity, Optional<BigDecimal>> () {
#Override
public Optional<BigDecimal> apply(Entity input) {
return Optional.fromNullable(input.getDecimalField(..));
}
}
);
Yet another example, assuming that I already have some Function, which extracts decimal values from Entities, and may return nulls:
Function<Entity, Decimal> extractDecimal = .. // extracts decimal value or null
Optional<BigDecimal> maybeDecimalValue = Optionals.flatMap(
findEntity(),
Optionals.lift(extractDecimal)
);
And last, but not least - your use case as an example:
Optional<Publisher> maybePublisher = Optionals.flatMap(findBook(id), Optionals.lift(Book.getPublisher));
// Assuming that getPublishedBooks may return null..
Optional<List<Book>> maybePublishedBooks = Optionals.flatMap(maybePublisher, Optionals.lift(Publisher.getPublishedBooks));
// ..or simpler, in case when getPublishedBooks never returns null
Optional<List<Book>> maybePublishedBooks2 = maybePublisher.transform(Publisher.getPublishedBooks);
// as a one-liner:
Optionals.flatMap(maybePublisher, Optionals.lift(Publisher.getPublishedBooks)).transform(Publisher.getPublishedBooks);
You probably figured that out, but you could add .or(Optional.absent) after every transformation that returns Optional (in your case after .transform(Book.getPublisher) reducing Optional<Optional<T>> to Optional<T>:
Optional<List<Book>> publishedBooks = findBook(id).transform(Book.getPublisher).
or(Optional.absent()).transform(Publisher.getPublishedBooks);
Unfortunately, the type of Optional.absent cannot be inferred here, so the code actually becomes:
Optional<List<Book>> publishedBooks = book.transform(Book.getPublisher).
or(Optional.<Publisher> absent()).transform(Publisher.getPublishedBoooks);
Not too convenient but there doesn't seem to be any other way.