Why java.util.Optional is broken

As of Java 8, there is a new java.util.Optional<A> class that can be used to represent the optional presence of a value in the type.

Traditionally, Java programs have returned null as a way to represent that a value isn't present. For instance, Map.get(K key) returns null if the map does not contain a key – but, null can also sometimes be a valid value stored against the key, causing ambiguities such as those documented in the Map.get javadoc.

If we have a method that returns type A, we must inspect the documentation to find out whether or not it may be null. By instead returning Optional<A>, there is a clear indication that the method may not return any value at all. Furthermore, Optionals force us to deal with the case where a value isn't returned*. Option types are relatively common these days, and their usage has helped significantly reduce a common class of problems caused by incorrectly using nulls for optionality.

When the Java 8 library team was designing Optional there was some opposition to the idea that it should contain some useful methods (essentially Optional.map and Optional.flatMap) on the somewhat spurious grounds that they didn't want their Optional to be a Monad. When I pointed out that it is anyway whether they liked it or not, they relented and put them back.

This is great, what do you mean about it being broken?

Unfortunately, when they put back map they did so in a way that is fundamentally broken.

To understand why it is broken, we need to step back a bit and look at what these two methods map and flatMap do. If I have a thing, when I map a function across it, I should get back a thing with the same structure as the thing I started with. So for instance, if I map a function across a List, I should get back a List of the same size as I started with, and with all the elements in the corresponding order. This means I can guarantee some things, like for instance if I reverse the order of the list and then map the function, I will get the same result as mapping the function and then reversing the list. I can refactor without fear.

As a consequence of this property of not being able to change the structure, we can derive other properties of things that can be mapped (or things that have a map method). The most interesting and obvious is that we can say that mapping two functions one after the other should be the same as mapping one function that is composed from the two. Let's look at some code:

Function<String, String> identity = (s -> s); // return the argument

List<String> ls = Arrays.asList("Alice", "Bob", "Christine");
List<String> l2 = Lists.map(ls, identity); // [Alice, Bob, Christine] <- the same thing!

assertThat(ls, is(equalTo(l2))); // yes!

Function<String, Integer> g = (s -> s.length());
Function<Integer, Integer> f = (i -> i * i); // square, because contrived

List<Integer> ints = Lists.map(ls, g); // [5, 3, 9]
List<Integer> squares = Lists.map(ints, f); // [25, 9, 81]

// f.compose(g) is a -> f.apply(g.apply(a))
List<Integer> squares2 = Lists.map(ls, f.compose(g)); // [25, 9, 81]

assertThat(squares, is(equalTo(squares2))); // yes!

This is a very important and useful principle, and it turns out that there is a sound theoretical basis for this result. Things that can be mapped over are known as Functors (or more precisely as covariant functors, but just functor is commonly used). The principles are known as the Functor Identity and Composition laws.

Unfortunately, java.util.Optional does not observe this principle.

So, what's wrong? And why?

The first hint is in the opening paragraph of the class javadoc:

A container object which may or may not contain a non-null value.

If we look back at Map.get we see that it is legitimate for a Map to contain null as a value for a key. This is distinct from there not being a mapping at all. Java (as part of its design) allows null to inhabit any reference type – but Optional specifically prohibits null being contained in it!

At face value this seems like an entirely legitimate thing to do. We are trying to replace usage of null as a signifier of optionality after all, right? Well, yes, but null is still a valid inhabiter of our type and may have other meaning apart from optionality, regardless of our strong opinions on this being an exceptionally bad idea or not.

It has a specific consequence for the map method and the composition law.

If we compose two functions and the first function returns a null, we will pass this null into the second function. As mentioned before, regardless of our opinion on the utility of nulls, this is what happens.

If we however map that first function on an Optional, it must become empty/none, as a non-empty Optional cannot contain a null. The second function, which may have special logic to handle the null input, does not get called at all!

We specifically have different behaviour depending which way we go, and we can no longer refactor without fear. We must know whether our functions will return null or not, or we risk introducing bugs simply by mechanical restructuring of our code.

Just flatMap that shit!

The answer is of course that we shouldn't be using map, we should be using flatMap instead. That is the method specifically designed for changing the shape of the box, and it is trivial to turn a null-as-optional returning function into a function that actually returns an Optional – although there isn't a helper for that in the standard library (sad).

The final irony is that by attempting to discourage nulls, the authors of this class have actually encouraged its use. I'm sure there are a few who will be tempted to simply return null from their functions in order to "avoid creating an unnecessary and expensive Optional reference", rather than using the correct types and combinators.

So, please use it with care, or use our more correct alternative, the next version of which (due out soon) has full integration with Java 8 and removes all other dependencies.

* Well, many Option types have some dangerous methods (like Optional.get) that do things like throw exceptions if there isn't a value.  

You might also enjoy our ebook, "Hello World! A new grad's guide to coding as a team" – a collection of essays designed to help new programmers succeed in a team setting. Grab it for yourself, your team, or the new computer science graduate in your life. Even seasoned coders might learn a thing or two.

Read it online now

Click here to download for your Kindle