ImmutableSet implements the Set interface. The functions that don't make sense to an ImmutableSet are now called "Optional Operations" for Set. I assume for situations like this. So ImmutableSet now throws an UnsupportedOperationException for many Optional Operations.
This seems backwards to me. I was taught that an Interface was a contract so that you could use impose functionality across different implementations. The approach of Optional Operations seem to fundamentally change(contradict?) what Interfaces are meant to do. Implementing this today I would have the Set Interface broken into two interfaces: one for one for immutable operations and a second extending those operations for mutators. (Very quick, off the cuff solution)
I understand that technology changes. I'm not saying It should be done one way or another. My question is, does this change reflect a change in some underlying philosophy for Java? Is it just more of a bandaid to make things backwards compatible? Did I have an incomplete understanding of Interfaces?
The Java Collections API Design FAQ answers this question in detail:
Q: Why don't you support immutability directly in the core collection interfaces so that you can do away with optional operations (and UnsupportedOperationException)?
A: This is the most controversial design decision in the whole API. Clearly, static (compile time) type checking is highly desirable, and is the norm in Java. We would have supported it if we believed it were feasible. Unfortunately, attempts to achieve this goal cause an explosion in the size of the interface hierarchy, and do not succeed in eliminating the need for runtime exceptions (though they reduce it substantially).
Doug Lea, who wrote a popular Java collections package that did reflect mutability distinctions in its interface hierarchy, no longer believes it is a viable approach, based on user experience with his collections package. In his words (from personal correspondence) "Much as it pains me to say it, strong static typing does not work for collection interfaces in Java."
To illustrate the problem in gory detail, suppose you want to add the notion of modifiability to the Hierarchy. You need four new interfaces: ModifiableCollection, ModifiableSet, ModifiableList, and ModifiableMap. What was previously a simple hierarchy is now a messy heterarchy. Also, you need a new Iterator interface for use with unmodifiable Collections, that does not contain the remove operation. Now can you do away with UnsupportedOperationException? Unfortunately not.
Consider arrays. They implement most of the List operations, but not remove and add. They are "fixed-size" Lists. If you want to capture this notion in the hierarchy, you have to add two new interfaces: VariableSizeList and VariableSizeMap. You don't have to add VariableSizeCollection and VariableSizeSet, because they'd be identical to ModifiableCollection and ModifiableSet, but you might choose to add them anyway for consistency's sake. Also, you need a new variety of ListIterator that doesn't support the add and remove operations, to go along with unmodifiable List. Now we're up to ten or twelve interfaces, plus two new Iterator interfaces, instead of our original four. Are we done? No.
Consider logs (such as error logs, audit logs and journals for recoverable data objects). They are natural append-only sequences, that support all of the List operations except for remove and set (replace). They require a new core interface, and a new iterator.
And what about immutable Collections, as opposed to unmodifiable ones? (i.e., Collections that cannot be changed by the client AND will never change for any other reason). Many argue that this is the most important distinction of all, because it allows multiple threads to access a collection concurrently without the need for synchronization. Adding this support to the type hierarchy requires four more interfaces.
Now we're up to twenty or so interfaces and five iterators, and it's almost certain that there are still collections arising in practice that don't fit cleanly into any of the interfaces. For example, the collection-views returned by Map are natural delete-only collections. Also, there are collections that will reject certain elements on the basis of their value, so we still haven't done away with runtime exceptions.
When all was said and done, we felt that it was a sound engineering compromise to sidestep the whole issue by providing a very small set of core interfaces that can throw a runtime exception.
In short, having interfaces like Set with optional operations was done to prevent an exponential explosion in the number of different interfaces needed. It is not as simple as just "immutable" and "mutable". Guava's ImmutableSet then had to implement Set to be interoperable with all other code which uses Sets. It's not ideal but there is really no better way to do it.
Related
Is there any reasons/arguments not to implement a Java collection that restricts its members based on a predicate/constraint?
Given that such functionality should be necessary often, I was expecting it to be implemented already on collections frameworks like apache-commons or Guava. But while apache indeed had it, Guava deprecated its version of it and recommend not using similar approaches.
The Collection interface contract states that a collection may place any restrictions on its elements as long as it is properly documented, so I'm unable to see why a guarded collection would be discouraged. What other option is there to, say, ensure a Integer collection never contains negative values without hiding the whole collection?
It is just a matter of preference -look at thread about checking before vs checking after - I think that is what it boils down to. Also checking only on add() i good enough only for immutable objects.
There can hardly be one ("acceptable") answer, so I'll just add some thoughts:
As mentioned in the comments, the Collection#add(E) already allows for throwing an IllegalArgumentException, with the reason
if some property of the element prevents it from being added to this collection
So one could say that this case was explicitly considered in the design of the collection interface, and there is no obvious, profound, purely technical (interface-contract related) reason to not allow creating such a collection.
However, when thinking about possible application patterns, one quickly finds cases where the observed behavior of such a collection could be ... counterintuitive, to say the least.
One was already mentioned by dcsohl in the comments, and referred to cases where such a collection would only be a view on another collection:
List<Integer> listWithIntegers = new ArrayList<Integer>();
List<Integer> listWithPositiveIntegers =
createView(listWithIntegers, e -> e > 0);
//listWithPositiveIntegers.add(-1); // Would throw IllegalArgumentException
listWithIntegers.add(-1); // Fine
// This would be true:
assert(listWithPositiveIntegers.contains(-1));
However, one could argue that
Such a collection would not necessarily have to be only a view. Instead, one could enforce that only new collections with such constraints may be created
The behavior is similar to that of Collections.unmodifiableCollection(Collection), which is widely anticipated as it is. (Although it serves a far broader and omnipresent use-case, namely avoiding the internal state of a class to be exposed by returning a modifiable version of a collection via an accessor method)
But in this case, the potential for "inconsistencies" is much higher.
For example, consider a call to Collection#addAll(Collection). It also allows throwing an IllegalArgumentException "if some property of an element of the specified collection prevents it from being added to this collection". But there are no guarantees about things like atomicity. To phrase it that way: It is not specified what the state of the collection will be when such an exception was thrown. Imagine a case like this:
List<Integer> listWithPositiveIntegers = createList(e -> e > 0);
listWithPositiveIntegers.add(1); // Fine
listWithPositiveIntegers.add(2); // Fine
listWithPositiveIntegers.add(Arrays.asList(3,-4,5)); // Throws
assert(listWithPositiveIntegers.contains(3)); // True or false?
assert(listWithPositiveIntegers.contains(5)); // True or false?
(It may be subtle, but it may be an issue).
All this might become even trickier when the condition changes after the collection has been created (regardless of whether it is only a view or not). For example, one could imagine a sequence of calls like this:
List<Integer> listWithPredicate = create(predicate);
listWithPredicate.add(-1); // Fine
someMethod();
listWithPredicate.add(-1); // Throws
Where in someMethod(), there is an innocent line like
predicate.setForbiddingNegatives(true);
One of the comments already mentioned possible performance issues. This is certainly true, but I think that this is not really a strong technical argument: There are no formal complexity guarantees for the runtime of any method of the Collection interface, anyhow. You don't know how long a collection.add(e) call takes. For a LinkedList it is O(1), but for a TreeSet it may be O(n log n) (and who knows what n is at this point in time).
Maybe the performance issue and the possible inconsistencies can be considered as special cases of a more general statement:
Such a collection would allow to basically execute arbitrary code during many operations - depending on the implementation of the predicate.
This may literally have arbitrary implications, and makes reasoning about algorithms, performance and the exact behavior (in terms of consistency) impossible.
The bottom line is: There are many possible reasons to not use such a collection. But I can't think of a strong and general technical reason. So there may be application cases for such a collection, but the caveats should be kept in mind, considering how exactly such a collection is intended to be used.
I would say that such a collection would have too many responsibilities and violate SRP.
The main issue I see here is the readability and maintainability of the code that uses the collection. Suppose you have a collection to which you allow adding only positive integers (Collection<Integer>) and you use it throughout the code. Then the requirements change and you are only allowed to add odd positive integers to it. Because there are no compile time checks, it would be much harder for you to find all the occurrences in the code where you add elements to that collection than it would be if you had a separate wrapper class which encapsulates the collection.
Although of course not even close to such an extreme, it bears some resemblance to using Object reference for all objects in the application.
The better approach is to utilize compile time checks and follow the well-established OOP principles like type safety and encapsulation. That means creating a separate wrapper class or creating a separate type for collection elements.
For example, if you really want to make quite sure that you only work with positive integers in a context, you could create a separate type PositiveInteger extends Number and then add them to a Collection<PositiveInteger>. This way you get compile time safety and converting PositiveInteger to OddPositiveInteger requires much less effort.
Enums are an excellent example of preferring dedicated types vs runtime-constrained values (constant strings or integers).
It is usually admitted that extending implementations of an interface through inheritance is not best practice, and that composition (eg. implementing the interface again from scratch) is more maintenable.
This works because the interface contract forces the user to implement all the desired functionality. However in java 8, default methods provide some default behavior which can be "manually" overriden. Consider the following example : I want to design a user database, which must have the functionalities of a List. I choose, for efficiency purposes, to back it by an ArrayList.
public class UserDatabase extends ArrayList<User>{}
This would not usually be considered great practice, and one would prefer, if actually desiring the full capabilities of a List and following the usual "composition over inheritance" motto :
public class UserDatabase implements List<User>{
//implementation here, using an ArrayList type field, or decorator pattern, etc.
}
However, if not paying attention, some methods, such as spliterator() will not be required to be overridden, as they are default methods of the List interface. The catch is, that the spliterator() method of List performs far worse than the spliterator() method of ArrayList, which has been optimised for the particular structure of an ArrayList.
This forces the developer to
be aware that ArrayList has its own, more efficient implementation of spliterator(), and manually override the spliterator() method of his own implementation of List or
lose a huge deal of performance by using the default method.
So the question is : is it still "as true" that one should prefer composition over inheritance in such situations ?
Before start thinking about performance, we always should think about correctness, i.e. in your question we should consider what using inheritance instead of delegation implies. This is already illustrated by this EclipseLink/ JPA issue. Due to the inheritance, sorting (same applies to stream operation) don’t work if the lazily populated list hasn’t populated yet.
So we have to trade off between the possibility that the specializations, overriding the new default methods, break completely in the inheritance case and the possibility that the default methods don’t work with the maximum performance in the delegation case. I think, the answer should be obvious.
Since your question is about whether the new default methods change the situation, it should be emphasized that you are talking about a performance degradation compared to something which did not even exist before. Let’s stay at the sort example. If you use delegation and don’t override the default sorting method, the default method might have lesser performance than the optimized ArrayList.sort method, but before Java 8 the latter did not exist and an algorithm not optimized for ArrayList was the standard behavior.
So you are not loosing performance with the delegation under Java 8, you are simply not gaining more, when you don’t override the default method. Due to other improvements, I suppose, that the performance will still be better than under Java 7 (without default methods).
The Stream API is not easily comparable as the API didn’t exist before Java 8. However, it’s clear that similar operations, e.g. if you implement a reduction by hand, had no other choice than going through the Iterator of your delegation list which had to be guarded against remove() attempts, hence wrap the ArrayList Iterator, or to use size() and get(int) which delegate to the backing List. So there is no scenario where a pre- default method API could exhibit better performance than the default methods of the Java 8 API, as there was no ArrayList-specific optimization in the past anyway.
That said, your API design could be improved by using composition in a different way: by not letting UserDatabase implement List<User> at all. Just offer the List via an accessor method. Then, other code won’t try to stream over the UserDatabase instance but over the list returned by the accessor method. The returned list may be a read only wrapper which provides optimal performance as it is provided by the JRE itself and takes care to override the default methods where feasible.
I don't really understand the big issue here. You can still back your UserDatabase with an ArrayList even if not extending it, and get the performance by delegation. You do not need to extend it to get the performance.
public class UserDatabase implements List<User>{
private ArrayList<User> list = new ArrayList<User>();
// implementation ...
// delegate
public Spliterator() spliterator() { return list.spliterator(); }
}
Your two points are not changing this. If you know "ArrayList has its own, more efficient implementation of spliterator()", then you can delegate it to your backing instance, and if you do not know, then the default method takes care of it.
I am still unsure whether it really makes any sense to implement the List interface, unless you are explicitly making a reusable Collection library. Better create your own API for such one-offs that does not come with future problems through the inheritance (or interface) chain.
I cannot provide an advice for every situation, but for this particular case I'd suggest not to implement the List at all. What would be the purpose of UserDatabase.set(int, User)? Do you really want to replace the i-th entry in the backing database with the completely new user? What about add(int, User)? It seems for me that you should either implement it as read-only list (throwing UnsupportedOperationException on every modification request) or support only some modification methods (like add(User) is supported, but add(int, User) is not). But the latter case would be confusing for the users. It's better to provide your own modification API which is more suitable for your task. As for read requests, probably it would be better to return a stream of users:
I'd suggest to create a method which returns the Stream:
public class UserDatabase {
List<User> list = new ArrayList<>();
public Stream<User> users() {
return list.stream();
}
}
Note that in this case you are completely free to change the implementation in future. For example, replace ArrayList with TreeSet or ConcurrentLinkedDeque or whatever.
The selection is simple based on your requirement.
Note - The below is just a use case . to illustrate the difference.
If you want a list that is not going to keep duplicates and going to do a whole bunch of things very much different from ArrayList then there is no use of extending ArrayList because you are writing everything from scratch.
In the above you should Implement List. But if you are just optimizing an implementation of ArrayList then you should copy paste the whole implementation of ArrayList and follow optimization instead of extending ArrayList. Why because multiple level of implementation makes it difficult for someone tries to sort out things.
Eg: A computer with 4GB Ram as parent and Child is having 8 GB ram. It is bad if parent has 4 GB and new Child has 4 GB to make an 8 GB. Instead of a child with 8 GB RAM implementation.
I would suggest composition in this case. But it will change based on the scenario.
It is usually admitted that extending implementations of an interface through inheritance is not best practice, and that composition (e.g. implementing the interface again from scratch) is more maintainable.
I don't think that this is accurate at all. For sure there are lots of situations where composition is preferred over inheritance, but there are lots of situations where inheritance is preferred over composition!
Its especially important to realise that the inheritance structure of your implementation classes need not look anything like the inheritance structure of your API.
Does anyone really believe, for example, that when writing a graphical library like Java swing every implementation class should reimplement the paintComponent() method? In fact a whole principal of the design is that when writing paint methods for new classes you can call super.paint() and that insures that all elements in the hierarchy are drawn, as well as handling the complications involving interfacing with the native interface further up the tree.
What is generally accepted is that extending classes not within your control that were not designed to support inheritance is dangerous and potentially a source of irritating bugs when the implementation changes. (So mark your classes as final if you reserve the right to change your implementation!). I doubt Oracle would introduce breaking changes into ArrayList implementation though! Provided you respect its documentation you should be fine....
Thats the elegance of the design. If they decide that there is a problem with the ArrayList, they will write a new implementation class, similar to when they replaced Vector back in the day, and there will be no need to introduce breaking changes.
===============
In your current example, the operative question is: why does this class exist at all?
If you are writing a class which extends the interface of list, which other methods does it implement? If it implements no new methods, what is wrong with using ArrayList?
When you know the answer that you will know what to do. If the answer "I want an object which is basically a list, but has some extra convenience methods to operate on that list", then I should use composition.
If the answer is "I want to fundamentally change the functionality of a list" then you should use inheritance, or implement from scratch. An example might be implementing an unmodifiable list by overriding ArrayList's add method to throw an exception. If you are uncomfortable with this approach you might consider implementing from scratch by extending AbstractList, which exists precisely to be inherited from to minimise the effort of reimplementation.
I know its commonly accepted to cast all List implementations down to List. Whether it is a variable, method return, or a method parameter using an ArrayList, CopyOnWriteArrayList, etc.
List<Market> mkts = new ArrayList<>();
When I'm using a Guava ImmutableList, I have the sense it can arguably be an exception to this rule (especially if I'm building in-house, complicated business applications and not a public API). Because if I cast it down to list, the deprecated mutator methods will no longer be flagged as deprecated. Also, it no longer is identified as an immutable object which is a very important part of its functionality and identity.
List<Market> mkts = ImmutableList.of(mkt1,mkt2,mkt3);
Therefore it makes sense to pass it around as an ImmutableList right? I could even argue that its a good policy for an internal API to only accept ImmutableList, so mutability and multithreading on the client side won't wreck anything inside the libary.
ImmutableList<Market> mkts = ImmutableList.of(mkt1,mkt2,mkt3);
I know there is a risk of ImmutableList itself becoming deprecated, and the day Oracle decides to create its own ImmutableList will require a lot of refactoring. But is it arguable the pros of maintaining an ImmutableList cast can outweigh the cons?
I agree with your rationale. If you are using the Guava collection library and your lists are immutable then passing them as ImmutableList is a good idea.
However:
I know there is a risk of ImmutableList itself becoming deprecated, and the day Oracle decides to create its own ImmutableList will require a lot of refactoring.
The first scenario seems unlikely, but it is a risk you take whenever you use any 3rd-party library. But the flipside is that you could chose to not upgrade your application's Guava version if they (Google) gratuitously deprecated a class or method that you relied on.
UPDATE
Louis Wasserman (who works for Google) said in a comment:
"Guava provides pretty strong compatibility guarantees for non-#Beta APIs."
So we can discount the possibility of gratuitous API changes.
The second scenario is even more unlikely (IMO). And you can be sure that if Oracle did add an immutable list class or interface, that would not require you to refactor. Oracle tries really hard to avoid breaking existing code when they enhance the standard APIs.
But having said that, it is really up to you to weigh up the pros and cons ... and how you would deal with the cons should the eventuate.
Unfortunately, there's no corresponding interface in Java (and most probably never will be). So my take is to pretend that ImmutableList is an interface. :D But seriously, it add important information which shouldn't get lost.
The ancient rule it all comes from actually states something like "program against interfaces". IIRC at the time the rules was created, there was no Java around and "interface" means programming interface, i.e., the contract, not java interface.
A method like
void strange(ArrayList list) {...}
is obviously strange, as there's no reason not to use List. A signature containing ImmutableList has a good reason.
I know there is a risk of ImmutableList itself becoming deprecated, and the day Oracle decides to create its own ImmutableList will require a lot of refactoring.
You mean Java 18? Let's see, but Guava's ImmutableList is pretty good and there's not much point in designing such a class differently. So you can hope that most changes will be in your imports only. And by 2050 there'll be worse problems than this.
Keep using List rather than ImmutableList! There is no problem with that and no reason for your API to start using ImmutableLists explicitly for several reasons:
ImmutableList is Guava only and unlikely to become standard Java at any point. Don't tie your code and coding habits to a third party library (even if it is a cool one like Guava).
Using immutable objects is good practice in Java and of particular importance when developing an API (see Effective Java Item 15 - minimize mutability). It is a general concept that can be taken for granted and does not need to be conveyed in the name of interfaces. Equally, you would not consider calling a User class that is designed for inheritance UserThatCanBeSubclassed.
In the name of stability your API should NEVER start modifying a List that was passed into it and ALWAYS make a defensive copy when passing a List to a client. Introducing ImmutableList here would lure you and the clients of your API into a false sense of security and entice them to violate that rule.
I understand your dilemma.
Personnaly, I would advise to keep using List as the reference type (to be future-proof and benefit from polymorphism), and use an #Immutable annotation to convey the information that it is immutable.
Annotations are more visible than plain javadoc comments, and you can even use the one from JSR-305 (ex-JCIP).
Some static analysis tools can even detect it and verify that your object is not mutated.
I would rather stay with just List for method parameter. There is no much benefit to enforce the caller to pass ImmutableList - it's your own method and you won't mutate list anyway, but you'd have method more reusable and generic.
As a return type, I would go with ImmutableList to let method users know that this list cannot be modified.
This is in the context of creating an interface/API.
Best practices suggest using general rather than specific types in interfaces - e.g. Map rather than HashMap.
Best practices also suggest preferring immutable types over mutable ones.
So considering both of these suggestions (and leaving aside concerns about performance/memory-footprint, 3rd-party-libraries/dependencies and convenience/functionality) should a method in a public interface look like this
public List<SomeClass> someMethod(...)
or rather this
public ImmutableList<SomeClass> someMethod(...)
When this has been discussed among the Guava folks, the following has been said:
The basic advice for types exchanged by APIs is this: choose the most general type that still conveys the relevant semantic information.
I consider the trio of semantic guarantees made by ImmutableCollection to be extremely relevant for return values in almost any circumstance (those three being immutability, lack of null elements and guaranteed iteration order). So I would virtually always return ImmutableSet, not Set.
We would really like people to view ImmutableSet etc. as being interfaces in every important sense of the word. There are only two reasons they are not: reliability of the immutability guarantee, and the fact that Java won't allow static methods on interfaces until JDK 8, and we wanted them there for convenience.
Most people think ImmutableList is an implementation for this reason, but there are actually several to dozens of different implementations of some of these types; you just don't see them.
If the method's contract guarantees that the List is immutable then the return type should be the ImmutableList rather than the List. This is much more explicit than simply mentioning that the List is immutable in the method's JavaDoc.
However, if the immutability of the list is an implementation detail, rather than a contract then the return type should be the List.
Writing APIs is about having contract with the users.
Best practices are mostly for the context where you need to write APIs for third parties and need to define the interfaces.
We can have different view in different contexts. If you are writing library which is going to be used by third party, you need to consider that they should or should not change the object state.
If API is going to be consumed internally (with in same code base) & purpose is to achieve loose coupling then you need to think about ease of writing, extensibility and maintainability.
Immutable APIs would avoid the data inconsistency specially when you made lot of assumptions on state of the object. On the other hand, mutable object would allow to save development efforts.
For all the methods in Collections that take a List as their first argument, why aren't those methods simply part of the List interface?
My intuition is: given a List object, that object itself should "know" how to perform on itself operations such as rotate(), shuffle(), or reverse(). But instead, as a Java programmer, I have to review both the methods in the List interface, as well as the static methods "over there" in the Collections class, to ensure I'm using a canonical solution.
Why were some methods placed as static standalone methods in the Collections class, instead of being added to the List interface (and presumably thus implemented by some existing or would-be base class)?
I'm trying to better understand the design decisions behind the Java collections framework.
Is there some compelling OO design principle here that I'm overlooking? Or was this distinction done simply for some practical, performance reason?
The point is that given suitable primitive operations (remove, set etc) a bunch of more high level operations (sort, shuffle, binary search) can be implemented once rather than being implemented by every single list implementation.
Effectively, java.util.Collections is like .NET's Enumerable class - full of general purpose methods which can work on any collection, so that they can share a single implementation and avoid duplication.
Rational Behind the List Interface's Methods
The List interface is a very core part of the Java runtime and is already a little onerous to fully implement all of the members when rolling out your own List implementations. So, adding extra methods that aren't directly related to the definition of a list is a bit extraneous. If you need those methods on a List implementation, why not subclass the interface and then require them?
If you where going to come along say in version 1.3 and add functionality to the List interface by adding new utility methods, you will break all past implementors of the interface.
From a Domain-Driven Design perspective, the utility methods in Collections are not part of the normal domain of a list.
Regarding OO design principals, I think it would be important to make the distinction between application OO design and language runtime OO design.
The authors of Java may do things very differently now that they have hindsight and perspective of many years of usage of the API. That said the C# IList interface is quite similar to Java's and C#'s authors did have the perspective.
It's certainly a judgement call at some level. I think the main trade-off to consider is this: When you add a method to an interface, every implementer of that interface must write code to implement it.
If the semantics of that method are such that different implementations of the interface will best implement those semantics in very different ways, then it's better to put it in the interface. (Of course, if the semantics simply can't be defined in terms of other methods in the interface, then it must be its own method in the interface.)
On the other hand, if the semantics are such that they can be defined in terms of other methods in the interface, and implementers of the interface will just tend to write the same code over and over again, then it's better to make a utility method that takes an instance of the interface as an argument.
They are utility methods and not core List functionality. The List interface would just get bloated if you added every possible operation you could do on a List. And the operations in Collections do not need to know about the internals of a List, they operate on the public interface so can happily live in an external class.
There are two explanations here:
Historical: Collections class was created after List interface. Designers chose to preserve backward compatibility of already existing interface. Otherwise a lot of developers would have to change their code.
Logical: The methods you are talking about do not require internal knowledge on List implementation and can be implemented over ANY collection implementing it.