Can I duplicate a Stream in Java 8? - java
Sometimes I want to perform a set of operations on a stream, and then process the resulting stream two different ways with other operations.
Can I do this without having to specify the common initial operations twice?
For example, I am hoping a dup() method such as the following exists:
Stream [] desired_streams = IntStream.range(1, 100).filter(n -> n % 2 == 0).dup();
Stream stream14 = desired_streams[0].filter(n -> n % 7 == 0); // multiples of 14
Stream stream10 = desired_streams[1].filter(n -> n % 5 == 0); // multiples of 10
It is not possible to duplicate a stream in this way. However, you can avoid the code duplication by moving the common part into a method or lambda expression.
Supplier<IntStream> supplier = () ->
IntStream.range(1, 100).filter(n -> n % 2 == 0);
supplier.get().filter(...);
supplier.get().filter(...);
It is not possible in general.
If you want to duplicate an input stream, or input iterator, you have two options:
A. Keep everything in a collection, say a List<>
Suppose you duplicate a stream into two streams s1 and s2. If you have advanced n1 elements in s1 and n2 elements with s2, you must keep |n2 - n1| elements in memory, just to keep pace. If your stream is infinite, there may be no upper bound for the storage required.
Take a look at Python's tee() to see what it takes:
This itertool may require significant auxiliary storage (depending on how much temporary data needs to be stored). In general, if one iterator uses most or all of the data before another iterator starts, it is faster to use list() instead of tee().
B. When possible: Copy the state of the generator that creates the elements
For this option to work, you'll probably need access to the inner workings of the stream. In other words, the generator - the part that creates the elements - should support copying in the first place. [OP: See this great answer, as an example of how this can be done for the example in the question]
It will not work on input from the user, since you'll have to copy the state of the entire "outside world". Java's Stream do not support copying, since it is designed to be as general as possible; for example, to work with files, network, keyboard, sensors, randomness etc. [OP: Another example is a stream that reads a temperature sensor on demand. It cannot be duplicated without storing a copy of the readings]
This is not only the case in Java; this is a general rule. You can see that std::istream in C++ only supports move semantics, not copy semantics ("copy constructor (deleted)"), for this reason (and others).
It's possible if you're buffering elements that you've consumed in one duplicate, but not in the other yet.
We've implemented a duplicate() method for streams in jOOλ, an Open Source library that we created to improve integration testing for jOOQ. Essentially, you can just write:
Tuple2<Seq<Integer>, Seq<Integer>> desired_streams = Seq.seq(
IntStream.range(1, 100).filter(n -> n % 2 == 0).boxed()
).duplicate();
(note: we currently need to box the stream, as we haven't implemented an IntSeq yet)
Internally, there is a LinkedList buffer storing all values that have been consumed from one stream but not from the other. That's probably as efficient as it gets if your two streams are consumed about at the same rate.
Here's how the algorithm works:
static <T> Tuple2<Seq<T>, Seq<T>> duplicate(Stream<T> stream) {
final LinkedList<T> gap = new LinkedList<>();
final Iterator<T> it = stream.iterator();
#SuppressWarnings("unchecked")
final Iterator<T>[] ahead = new Iterator[] { null };
class Duplicate implements Iterator<T> {
#Override
public boolean hasNext() {
if (ahead[0] == null || ahead[0] == this)
return it.hasNext();
return !gap.isEmpty();
}
#Override
public T next() {
if (ahead[0] == null)
ahead[0] = this;
if (ahead[0] == this) {
T value = it.next();
gap.offer(value);
return value;
}
return gap.poll();
}
}
return tuple(seq(new Duplicate()), seq(new Duplicate()));
}
More source code here
In fact, using jOOλ, you'll be able to write a complete one-liner like so:
Tuple2<Seq<Integer>, Seq<Integer>> desired_streams = Seq.seq(
IntStream.range(1, 100).filter(n -> n % 2 == 0).boxed()
).duplicate()
.map1(s -> s.filter(n -> n % 7 == 0))
.map2(s -> s.filter(n -> n % 5 == 0));
// This will yield 14, 28, 42, 56...
desired_streams.v1.forEach(System.out::println)
// This will yield 10, 20, 30, 40...
desired_streams.v2.forEach(System.out::println);
Starting with Java 12 we have Collectors::teeing that allows us to pass elements of the main stream pipeline to 2 or more downstream collectors.
Based on your example we can do the following:
#Test
void shouldProcessStreamElementsInTwoSeparateDownstreams() {
class Result {
List<Integer> multiplesOf7;
List<Integer> multiplesOf5;
Result(List<Integer> multiplesOf7, List<Integer> multiplesOf5) {
this.multiplesOf7 = multiplesOf7;
this.multiplesOf5 = multiplesOf5;
}
}
var result = IntStream.range(1, 100)
.filter(n -> n % 2 == 0)
.boxed()
.collect(Collectors.teeing(
Collectors.filtering(n -> n % 7 == 0, Collectors.toList()),
Collectors.filtering(n -> n % 5 == 0, Collectors.toList()),
Result::new
));
assertTrue(result.multiplesOf7.stream().allMatch(n -> n % 7 == 0));
assertTrue(result.multiplesOf5.stream().allMatch( n -> n % 5 == 0));
}
There are many other collectors that allows to do other things e.g. by using Collectors::mapping in downstream you can obtain two different objects/types from the same source as shown in this article.
You can also move the stream generation into separate method/function that returns this stream and call it twice.
Either,
Move the initialisation into a method, and simply call the method again
This has the advantage of being explicit about what you are doing, and also works for infinite streams.
Collect the stream and then re-stream it
In your example:
final int[] arr = IntStream.range(1, 100).filter(n -> n % 2 == 0).toArray();
Then
final IntStream s = IntStream.of(arr);
Update: This doesn't work. See explanation below, after the text of the original answer.
How silly of me. All that I need to do is:
Stream desired_stream = IntStream.range(1, 100).filter(n -> n % 2 == 0);
Stream stream14 = desired_stream.filter(n -> n % 7 == 0); // multiples of 14
Stream stream10 = desired_stream.filter(n -> n % 5 == 0); // multiples of 10
Explanation why this does not work:
If you code it up and try to collect both streams, the first one will collect fine, but trying to stream the second one will throw the exception: java.lang.IllegalStateException: stream has already been operated upon or closed.
To elaborate, streams are stateful objects (which by the way cannot be reset or rewound). You can think of them as iterators, which in turn are like pointers. So stream14 and stream10 can be thought of as references to the same pointer. Consuming the first stream all the way will cause the pointer to go "past the end." Trying to consume the second stream is like trying to access a pointer that is already "past the end," Which naturally is an illegal operation.
As the accepted answer shows, the code to create the stream must be executed twice but it can be compartmentalized into a Supplier lambda or a similar construct.
Full test code: save into Foo.java, then javac Foo.java, then java Foo
import java.util.stream.IntStream;
public class Foo {
public static void main (String [] args) {
IntStream s = IntStream.range(0, 100).filter(n -> n % 2 == 0);
IntStream s1 = s.filter(n -> n % 5 == 0);
s1.forEach(n -> System.out.println(n));
IntStream s2 = s.filter(n -> n % 7 == 0);
s2.forEach(n -> System.out.println(n));
}
}
Output:
$ javac Foo.java
$ java Foo
0
10
20
30
40
50
60
70
80
90
Exception in thread "main" java.lang.IllegalStateException: stream has already been operated upon or closed
at java.util.stream.AbstractPipeline.<init>(AbstractPipeline.java:203)
at java.util.stream.IntPipeline.<init>(IntPipeline.java:91)
at java.util.stream.IntPipeline$StatelessOp.<init>(IntPipeline.java:592)
at java.util.stream.IntPipeline$9.<init>(IntPipeline.java:332)
at java.util.stream.IntPipeline.filter(IntPipeline.java:331)
at Foo.main(Foo.java:8)
For non-infinite streams, if you have access to the source, its straight forward:
#Test
public void testName() throws Exception {
List<Integer> integers = Arrays.asList(1, 2, 4, 5, 6, 7, 8, 9, 10);
Stream<Integer> stream1 = integers.stream();
Stream<Integer> stream2 = integers.stream();
stream1.forEach(System.out::println);
stream2.forEach(System.out::println);
}
prints
1
2
4
5
6
7
8
9
10
1
2
4
5
6
7
8
9
10
For your case:
Stream originalStream = IntStream.range(1, 100).filter(n -> n % 2 == 0)
List<Integer> listOf = originalStream.collect(Collectors.toList())
Stream stream14 = listOf.stream().filter(n -> n % 7 == 0);
Stream stream10 = listOf.stream().filter(n -> n % 5 == 0);
For performance etc, read someone else's answer ;)
I used this great answer to write following class:
public class SplitStream<T> implements Stream<T> {
private final Supplier<Stream<T>> streamSupplier;
public SplitStream(Supplier<Stream<T>> t) {
this.streamSupplier = t;
}
#Override
public Stream<T> filter(Predicate<? super T> predicate) {
return streamSupplier.get().filter(predicate);
}
#Override
public <R> Stream<R> map(Function<? super T, ? extends R> mapper) {
return streamSupplier.get().map(mapper);
}
#Override
public IntStream mapToInt(ToIntFunction<? super T> mapper) {
return streamSupplier.get().mapToInt(mapper);
}
#Override
public LongStream mapToLong(ToLongFunction<? super T> mapper) {
return streamSupplier.get().mapToLong(mapper);
}
#Override
public DoubleStream mapToDouble(ToDoubleFunction<? super T> mapper) {
return streamSupplier.get().mapToDouble(mapper);
}
#Override
public <R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper) {
return streamSupplier.get().flatMap(mapper);
}
#Override
public IntStream flatMapToInt(Function<? super T, ? extends IntStream> mapper) {
return streamSupplier.get().flatMapToInt(mapper);
}
#Override
public LongStream flatMapToLong(Function<? super T, ? extends LongStream> mapper) {
return streamSupplier.get().flatMapToLong(mapper);
}
#Override
public DoubleStream flatMapToDouble(Function<? super T, ? extends DoubleStream> mapper) {
return streamSupplier.get().flatMapToDouble(mapper);
}
#Override
public Stream<T> distinct() {
return streamSupplier.get().distinct();
}
#Override
public Stream<T> sorted() {
return streamSupplier.get().sorted();
}
#Override
public Stream<T> sorted(Comparator<? super T> comparator) {
return streamSupplier.get().sorted(comparator);
}
#Override
public Stream<T> peek(Consumer<? super T> action) {
return streamSupplier.get().peek(action);
}
#Override
public Stream<T> limit(long maxSize) {
return streamSupplier.get().limit(maxSize);
}
#Override
public Stream<T> skip(long n) {
return streamSupplier.get().skip(n);
}
#Override
public void forEach(Consumer<? super T> action) {
streamSupplier.get().forEach(action);
}
#Override
public void forEachOrdered(Consumer<? super T> action) {
streamSupplier.get().forEachOrdered(action);
}
#Override
public Object[] toArray() {
return streamSupplier.get().toArray();
}
#Override
public <A> A[] toArray(IntFunction<A[]> generator) {
return streamSupplier.get().toArray(generator);
}
#Override
public T reduce(T identity, BinaryOperator<T> accumulator) {
return streamSupplier.get().reduce(identity, accumulator);
}
#Override
public Optional<T> reduce(BinaryOperator<T> accumulator) {
return streamSupplier.get().reduce(accumulator);
}
#Override
public <U> U reduce(U identity, BiFunction<U, ? super T, U> accumulator, BinaryOperator<U> combiner) {
return streamSupplier.get().reduce(identity, accumulator, combiner);
}
#Override
public <R> R collect(Supplier<R> supplier, BiConsumer<R, ? super T> accumulator, BiConsumer<R, R> combiner) {
return streamSupplier.get().collect(supplier, accumulator, combiner);
}
#Override
public <R, A> R collect(Collector<? super T, A, R> collector) {
return streamSupplier.get().collect(collector);
}
#Override
public Optional<T> min(Comparator<? super T> comparator) {
return streamSupplier.get().min(comparator);
}
#Override
public Optional<T> max(Comparator<? super T> comparator) {
return streamSupplier.get().max(comparator);
}
#Override
public long count() {
return streamSupplier.get().count();
}
#Override
public boolean anyMatch(Predicate<? super T> predicate) {
return streamSupplier.get().anyMatch(predicate);
}
#Override
public boolean allMatch(Predicate<? super T> predicate) {
return streamSupplier.get().allMatch(predicate);
}
#Override
public boolean noneMatch(Predicate<? super T> predicate) {
return streamSupplier.get().noneMatch(predicate);
}
#Override
public Optional<T> findFirst() {
return streamSupplier.get().findFirst();
}
#Override
public Optional<T> findAny() {
return streamSupplier.get().findAny();
}
#Override
public Iterator<T> iterator() {
return streamSupplier.get().iterator();
}
#Override
public Spliterator<T> spliterator() {
return streamSupplier.get().spliterator();
}
#Override
public boolean isParallel() {
return streamSupplier.get().isParallel();
}
#Override
public Stream<T> sequential() {
return streamSupplier.get().sequential();
}
#Override
public Stream<T> parallel() {
return streamSupplier.get().parallel();
}
#Override
public Stream<T> unordered() {
return streamSupplier.get().unordered();
}
#Override
public Stream<T> onClose(Runnable closeHandler) {
return streamSupplier.get().onClose(closeHandler);
}
#Override
public void close() {
streamSupplier.get().close();
}
}
When you call any method of it's class, it delegates call to
streamSupplier.get()
So, instead of:
Supplier<IntStream> supplier = () ->
IntStream.range(1, 100).filter(n -> n % 2 == 0);
supplier.get().filter(...);
supplier.get().filter(...);
You can do:
SplitStream<Integer> stream =
new SplitStream<>(() -> IntStream.range(1, 100).filter(n -> n % 2 == 0).boxed());
stream.filter(...);
stream.filter(...);
You can expand it to work with IntStream, DoubleStream, etc...
I think that the use of Concat with an empty stream could attend your need.
Try something like this:
Stream<Integer> concat = Stream.concat(Stream.of(1, 2), Stream.empty());
Straight answer is: yes
There's no specific support for this but one can implement it. The possible approaches that I see are these:
a. copy the entire stream data and then create the stream copies based on it -> the RAM consumption might be an impediment
b. read the stream and relay each of its elements to the copies -> I'll detail this approach below
The Concept
Let's imagine b. solution:
<T> List<Stream<T>> copyStream(int copiesCount, Stream<T> originalStream)
allows one to create copiesCount copies of the originalStream.
To understand the solution one has to understand the difference between a stream and the data-elements that might flow through it: for example an apple, a carrot and a potato would be data-elements while a pipe through which they move to reach some destination would be the stream. Copying a Stream it's as if creating more pipes: one has then to connect the original pipe (i.e. originalStream) to the additional ones (aka streamCopies); while in real world one can't pass an apple-object from one pipe to more pipes (i.e. streamCopies) in programming this is possible: just pass the variable containing the apple-object reference to the stream copies.
Implementation Details
The Java implementation of the Stream has a great impact on the solution's shape. First impact is related to what happens when data-elements flow through a stream (aka pipe): to actually read (& process) the elements in a Stream a terminal method has to be used, e.g. forEach. In our case originalStream.forEach must be called so that each element is read and passed to the streamCopies (aka downstream pipes); this must happen before copyStream() method returns, which is bad because forEach would block till all originalStream elements are consumed. To solve this copyStream() implementation will spawn a thread in which to call originalStream.forEach. Consuming originalStream elements means passing them to the downstream pipes (i.e. streamCopies); because there's no cache one has to ensure that each originalStream element is transferred to each streamCopies before getting to the next one. This means that all streamCopies must consume the same time: if some streamCopies is not consuming it will block all other streamCopies because originalStream will stop transferring to downstream pipes till everyone consumed current element (aka it will cache nothing for the late streamCopies consumers). But to consume a Stream in Java implies calling a terminal operation on it (e.g. forEach) which blocks the execution till the entire stream is consumed; because we need all streamCopies to be consumed in parallel this must happen on a distinct thread for each! Well, as a miscellaneous fact, one of the streamCopies could in fact be consumed on the current (main) thread. Summarizing, the solution usage would look like below:
List<Stream<?>> streamCopies = copyStream(copiesCount, originalStream);`
// start a thread for each `streamCopies` into which consume the corresponding
// stream copy (one of them could be consumed on the current thread though)
// optionally join the consuming threads
// continue your whatever business logic you have
Final Considerations
Some of the limitations apparent above can be circumvented:
the copying process is destructive, i.e. originalStream will be unusable after calling copyStream() because it'll be in a pending-consumption. If one really wants to consume it he can create an additional copy which to maybe consume on the current (main) thread (but only after starting the consumption of all other copies)
streamCopies must consume all received originalStream elements, otherwise, if one stops, the others block too (read the "Implementation Details" part again to understand why). This means each streamCopies element consumption must occur in a try...catch to ensure the lack of failures (aka processing stop). A production implementation would in fact circumvent this by wrapping each Stream copy with something overwriting close() method such that to remove the failed stream copy from the originalStream-to-streamCopies transfer logic (aka discard the underlying blockingQueue used for the communication between originalStream thread and originalStream thread -> see the implementation below). This implies that the clients would be forced to close the Stream copies but that’s not so uncommon, e.g. see Spring’s JDBCTemplate.queryForStream() outcome having same requirement.
as pointed before, each streamCopies terminal operation must be executed in a distinct thread - there's no workaround for this
The Code
Below is the code implementing the b. solution and a test checking its correctness.
#Test
void streamCopyTest() throws ExecutionException, InterruptedException {
// streamCopies are valid/normal Stream
// instances (e.g. it is allowed to be infinite)
List<Stream<String>> streamCopies = copyStream(3, Stream.of("a", "b", "c", "d"));
// The 3 copies relay on the original stream which can’t be
// consumed more than once! Consuming the copies one by one
// in the same thread isn’t possible because 1st consumed
// copy would leave nothing to consume for the others,
// so they must be consumed in parallel.
ExecutorService executorService = Executors.newCachedThreadPool();
CompletableFuture<?>[] futures =
streamCopies.stream().map(stream -> CompletableFuture.runAsync(() -> {
// the same consumption logic for all streamCopies is
// used here because this is just an example; the
// actual consumption logic could be distinct (and anything)
String outcome = stream.collect(Collectors.joining(", "));
// check the thread name in the message to differentiate the outcome
log.info("\n{}", outcome);
}, executorService)).toArray(CompletableFuture[]::new);
CompletableFuture.allOf(futures).get();
executorService.shutdown();
}
#RequiredArgsConstructor
#Slf4j
public class StreamCopiesFactory {
/**
* The amount of elements to be stored in the blockingQueue used
* to transfer elements from the original stream to its copies.
* This is very different to the cache use for the a. solution:
* here is about the transfer between original stream and its
* copies instead of the entire original stream data-copy.
* Change or make this configurable.
*/
private static final int cacheSize = 1;
/**
* Each of these stream copies must execute (their terminal operation)
* on a distinct thread! One of them could actually execute on the
* main thread, but only after all the others were started on their
* distinct thread.
*/
public static <T> List<Stream<T>> copyStream(int copies, Stream<T> stream) {
List<BlockingQueue<Object>> blockingQueues = new ArrayList<>(copies);
// creating the queues used to relay the stream's elements to the stream's copies
for (int i = 0; i < copies; i++) {
blockingQueues.add(new LinkedBlockingQueue<>(cacheSize));
}
// consume the stream copies in a distinct thread, otherwise
// bq.put (transferring for the next stream copy) would block
// because the 2nd stream copy isn't yet consuming
Executors.newSingleThreadExecutor().execute(() -> {
stream.forEach(streamElement -> blockingQueues.forEach(bq -> {
try {
bq.put(streamElement);
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
// nothing to do here other than maybe simple optimization related to the
// failed bq.put (e.g. sending END_SIGNAL into bq then skipping its next put calls)
}
}));
blockingQueues.forEach(bq -> {
try {
bq.put(END_SIGNAL);
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
// nothing to do here
}
});
});
// creating the copies
// A production implementation would wrap each Stream copy with
// something overwriting close() which to remove from blockingQueues
// the blockingQueue corresponding to the closed Stream.
return blockingQueues.stream().map(bq -> new SpliteratorCopy<T>(bq))
.map(spliterator -> StreamSupport.stream(spliterator, false))
.collect(Collectors.toList());
}
}
#RequiredArgsConstructor
#Slf4j
public class SpliteratorCopy<T> implements Spliterator<T> {
public static final Object END_SIGNAL = new Object();
private final BlockingQueue<?> blockingQueue;
#Override
public boolean tryAdvance(final Consumer<? super T> action) {
Object nextElement;
try {
nextElement = blockingQueue.take();
} catch (InterruptedException e) {
log.error(e.getMessage(), e);
throw new RuntimeException(e);
}
if (nextElement == END_SIGNAL) {
return false;
}
action.accept((T) nextElement);
return true;
}
#Override
public Spliterator<T> trySplit() {
return null;
}
#Override
public long estimateSize() {
return Long.MAX_VALUE;
}
#Override
public int characteristics() {
return Spliterator.ORDERED;
}
}
Related
Transforming a while loop to a stream in Java 8
As an exercise I'm converting some old code to functional streams. I don't know much about streams. It seems like it should be simple to convert this code, but I haven't had much luck. The method starts at a given integer, passes it to isPrime, which returns true if it's prime. and then hands off the new(next) prime number to be printed. If isPrime is false i is incremented and we check the next integer. private static int nextPrime(final int number) { int i = number + 1; while (!isPrime(i)) { i++; } return i; }
I see no reason to use a Stream for this other than to take advantage of parallelism (if the primes happen to be very far apart, but this won't be true within int bounds, so there's essentially no benefit). You can iterate over an IntStream of ascending integers (starting from number + 1) and filter only the prime numbers. When one is inevitably found, you can return the first. private static int nextPrime(final int number) { return IntStream.iterate(number + 1, i -> i + 1) .filter(Test::isPrime) .findFirst() .getAsInt(); } Note: The class that I used to test this is called Test, as seen by the method reference. You should change that to your class name.
Based on this answer, for an object like an Enumeration<T> for example, where you can only call .hasMoreElements() & .nextElement(), you can use this kind of code : public static <T> Stream<T> enumerationAsStream(Enumeration<T> e) { return StreamSupport.stream( new Spliterators.AbstractSpliterator<T>(Long.MAX_VALUE, Spliterator.ORDERED) { public boolean tryAdvance(Consumer<? super T> action) { if(e.hasMoreElements()) { action.accept(e.nextElement()); return true; } return false; } public void forEachRemaining(Consumer<? super T> action) { while(e.hasMoreElements()) action.accept(e.nextElement()); } }, false); }
Is it possible to define an optional flow or exception-like behaviour in Java 8 streams API?
Let us have a stream of objects, resulting from a sequence of operations (e.g. mapping, filtering, flatmapping, etc.). Now I want to do a certain operation on them, but only if a given predicate is true. Otherwise I want to immediately return something else. A simple example. I have a stream of different food objects. If all of them are edible I want to perform a cook operation on them and return the list of cooked food. But if any of them turns out to not be edible I want to immediately return an empty list. Few solutions come to my mind, but I am not satisfied with any of them. I could first perform an allMatch operation with isEdible predicate on the stream, but it will result in terminating it and I would need to repeat preliminary operations once more. I could persist the collection that is the result of preliminary operations before checking the edibility, but therefore I need to perform them for all elements. Which is suboptimal, because it may turn out, that the first of them is not edible and allMatch would return much, much earlier. Or I could design a hacky reduce routine, but it would also be unable to stop processing elements when predicate fails. What I hope for is something like the code below. Is it possible with current API? source.stream() // some operations .ifAny(food -> !food.isEdible(), new LinkedList<Food>()) // other operations if previous step not failed .peek(food -> food.prepare()) .collect(Collectors.toList());
Not exactly what you wanted, but using the ternary operator will make your two step solution look cleaner, it also should have optimal performance: return source.stream() .allMatch(this::isEdible) ? source.stream() .map(this::prepare()) // do stuff .collect(Collectors.toList()) : Collections.emptyList();
Here's a method I just cooked up. It's a class that wraps Collector and checks if each element passes a predicate along the way. At the end, if any element failed, it returns the default return instead of the collected return. On the downside you still have to iterate through every element even if the first one fails the predicate, but on the upside: Only have to iterate once through each element Simple in-line syntax Easily integrates with existing stream and collector syntax and use cases. So based on those I would imagine it would be worth it. public class SatisfyableCollector<T,A,R> implements Collector<T,A,R> { private Predicate<T> predicate; //The predicate used to test each element private boolean elmHasFailed; //True once an element has failed the predicate private Collector<T,A,R> collector; //The collector this wraps private R defaultResult; //The result to return at the end if an element failed public static <T,A,R> SatisfyableCollector<T,A,R> of(Collector<T,A,R> collector, Predicate<T> predicate, R defaultResult) { return new SatisfyableCollector<>(collector, predicate, defaultResult); } private SatisfyableCollector(Collector<T,A,R> collector, Predicate<T> predicate, R defaultResult) { this.predicate = predicate; this.collector = collector; this.defaultResult = defaultResult; elmHasFailed = false; } #Override public Supplier<A> supplier() { return collector.supplier(); } #Override /** Before accumulating the new element t, check it against the predicate */ public BiConsumer<A, T> accumulator() { return (a, t) -> { if (! predicate.test(t)) { elmHasFailed = true; } collector.accumulator().accept(a,t); }; } #Override public BinaryOperator<A> combiner() { return collector.combiner(); } #Override /** At the end, check if something failed. If so, return the default. * Otherwise the wrapped collector's finisher. */ public Function<A, R> finisher() { return (a) -> elmHasFailed ? defaultResult : collector.finisher().apply(a); } #Override /** Make sure IDENTITY_FINISH isn't present, or finisher() won't be called */ public Set<Characteristics> characteristics() { Set<Characteristics> originalSet = collector.characteristics(); if (! originalSet.contains(Characteristics.IDENTITY_FINISH)) return originalSet; else { HashSet<Characteristics> set = new HashSet<>(collector.characteristics()); //Make new set so we can modify it. set.remove(Characteristics.IDENTITY_FINISH); //Make sure finisher() is called return Collections.unmodifiableSet(set); } } } And can be used like this: public static void main(String[] args) { Stream<Integer> stream = Arrays.asList(1,2,3,4,5,6,7,8).stream(); //Create a list out of the stream *if* every element is even. Return empty otherwise List<Integer> listOrEmpty = stream.collect(SatisfyableCollector.of(Collectors.toList(), (x) -> x%2 == 0, new ArrayList<>()))); System.out.println(listOrEmpty); } This currently doesn't handle exception throwing gracefully, but that shouldn't be too hard to add.
Does Java 8 have cached support for suppliers?
The guava library has it's own Supplier which does not extend Java 8 Supplier. Also guava provides a cache for suppliers - Suppliers#memoize. Is there something similar, but for Java 8 Suppliers?
There's no built-in Java function for memoization, though it's not very hard to implement it, for example, like this: public static <T> Supplier<T> memoize(Supplier<T> delegate) { AtomicReference<T> value = new AtomicReference<>(); return () -> { T val = value.get(); if (val == null) { val = value.updateAndGet(cur -> cur == null ? Objects.requireNonNull(delegate.get()) : cur); } return val; }; } Note that different implementation approaches exist. The above implementation may call the delegate several times if the memoized supplier requested simultaneously several times from the different threads. Sometimes such implementation is preferred over the explicit synchronization with lock. If lock is preferred, then DCL could be used: public static <T> Supplier<T> memoizeLock(Supplier<T> delegate) { AtomicReference<T> value = new AtomicReference<>(); return () -> { T val = value.get(); if (val == null) { synchronized(value) { val = value.get(); if (val == null) { val = Objects.requireNonNull(delegate.get()); value.set(val); } } } return val; }; } Also note, as #LouisWasserman correctly mentioned in comments, you can easily transform JDK supplier into Guava supplier and vice versa using method reference: java.util.function.Supplier<String> jdkSupplier = () -> "test"; com.google.common.base.Supplier<String> guavaSupplier = jdkSupplier::get; java.util.function.Supplier<String> jdkSupplierBack = guavaSupplier::get; So it's not a big problem to switch between Guava and JDK functions.
The simplest solution would be public static <T> Supplier<T> memoize(Supplier<T> original) { ConcurrentHashMap<Object, T> store=new ConcurrentHashMap<>(); return ()->store.computeIfAbsent("dummy", key->original.get()); } However, the simplest is not always the most efficient. If you want a clean and efficient solution, resorting to an anonymous inner class to hold the mutable state will pay off: public static <T> Supplier<T> memoize1(Supplier<T> original) { return new Supplier<T>() { Supplier<T> delegate = this::firstTime; boolean initialized; public T get() { return delegate.get(); } private synchronized T firstTime() { if(!initialized) { T value=original.get(); delegate=() -> value; initialized=true; } return delegate.get(); } }; } This uses a delegate supplier which will do the first time operation and afterwards, replace itself with a supplier that unconditionally returns the captured result of the first evaluation. Since it has final fields semantics, it can be unconditionally returned without any additional synchronization. Inside the synchronized method firstTime(), there is still an initialized flag needed because in case of concurrent access during initialization, multiple threads may wait at the method’s entry before the delegate has been replaced. Hence, these threads need to detect that the initialization has been done already. All subsequent accesses will read the new delegate supplier and get the value quickly.
A simple wrapper for Guava 20 on Java 8: static <T> java.util.function.Supplier<T> memoize(java.util.function.Supplier<? extends T> supplier) { return com.google.common.base.Suppliers.memoize(supplier::get)::get; }
How to short-circuit reduce on Stream?
Suppose I have a stream of boolean values and the reduce operation that I am writing is || (OR). Can I write it in a way such that the evaluation of at least some of the elements is abandoned if a true value is encountered? I am looking for some amount of optimization (perhaps if it is a parallel stream), not necessarily full optimization although the latter would be awesome.
I suspect you want this type of construct. // stop when any element evaluates to true boolean any = stream.anyMatch(t -> t); You can check this with peek Stream.of(1, 2, 3, 4).peek(System.out::println).anyMatch(i -> i == 2); prints 1 2 For a parallel example AtomicInteger count = new AtomicInteger(); IntStream.range(0, 1000).parallel().peek(t -> count.incrementAndGet()).anyMatch(i -> i == 2); System.out.println("count: " + count); prints a number like count: 223 The exact number varies. For a referencePipeline, the anyMatch calls #Override public final boolean anyMatch(Predicate<? super P_OUT> predicate) { return evaluate(MatchOps.makeRef(predicate, MatchOps.MatchKind.ANY)); } which calls this public static <T> TerminalOp<T, Boolean> makeRef(Predicate<? super T> predicate, MatchKind matchKind) { Objects.requireNonNull(predicate); Objects.requireNonNull(matchKind); class MatchSink extends BooleanTerminalSink<T> { MatchSink() { super(matchKind); } #Override public void accept(T t) { if (!stop && predicate.test(t) == matchKind.stopOnPredicateMatches) { stop = true; value = matchKind.shortCircuitResult; } } } return new MatchOp<>(StreamShape.REFERENCE, matchKind, MatchSink::new); } where you can start to see the short circuiting code.
Functional style of Java 8's Optional.ifPresent and if-not-Present?
In Java 8, I want to do something to an Optional object if it is present, and do another thing if it is not present. if (opt.isPresent()) { System.out.println("found"); } else { System.out.println("Not found"); } This is not a 'functional style', though. Optional has an ifPresent() method, but I am unable to chain an orElse() method. Thus, I cannot write: opt.ifPresent( x -> System.out.println("found " + x)) .orElse( System.out.println("NOT FOUND")); In reply to #assylias, I don't think Optional.map() works for the following case: opt.map( o -> { System.out.println("while opt is present..."); o.setProperty(xxx); dao.update(o); return null; }).orElseGet( () -> { System.out.println("create new obj"); dao.save(new obj); return null; }); In this case, when opt is present, I update its property and save to the database. When it is not available, I create a new obj and save to the database. Note in the two lambdas I have to return null. But when opt is present, both lambdas will be executed. obj will be updated, and a new object will be saved to the database . This is because of the return null in the first lambda. And orElseGet() will continue to execute.
If you are using Java 9+, you can use ifPresentOrElse() method: opt.ifPresentOrElse( value -> System.out.println("Found: " + value), () -> System.out.println("Not found") );
For me the answer of #Dane White is OK, first I did not like using Runnable but I could not find any alternatives. Here another implementation I preferred more: public class OptionalConsumer<T> { private Optional<T> optional; private OptionalConsumer(Optional<T> optional) { this.optional = optional; } public static <T> OptionalConsumer<T> of(Optional<T> optional) { return new OptionalConsumer<>(optional); } public OptionalConsumer<T> ifPresent(Consumer<T> c) { optional.ifPresent(c); return this; } public OptionalConsumer<T> ifNotPresent(Runnable r) { if (!optional.isPresent()) { r.run(); } return this; } } Then: Optional<Any> o = Optional.of(...); OptionalConsumer.of(o).ifPresent(s -> System.out.println("isPresent " + s)) .ifNotPresent(() -> System.out.println("! isPresent")); Update 1: the above solution for the traditional way of development when you have the value and want to process it but what if I want to define the functionality and the execution will be then, check below enhancement; public class OptionalConsumer<T> implements Consumer<Optional<T>> { private final Consumer<T> c; private final Runnable r; public OptionalConsumer(Consumer<T> c, Runnable r) { super(); this.c = c; this.r = r; } public static <T> OptionalConsumer<T> of(Consumer<T> c, Runnable r) { return new OptionalConsumer(c, r); } #Override public void accept(Optional<T> t) { if (t.isPresent()) { c.accept(t.get()); } else { r.run(); } } Then could be used as: Consumer<Optional<Integer>> c = OptionalConsumer.of( System.out::println, () -> System.out.println("Not fit") ); IntStream.range(0, 100) .boxed() .map(i -> Optional.of(i) .filter(j -> j % 2 == 0)) .forEach(c); In this new code you have 3 things: can define the functionality before the existing of an object easy. not creating object reference for each Optional, only one, you have so less memory than less GC. it is implementing consumer for better usage with other components. By the way, now its name is more descriptive it is actually Consumer<Optional<?>>
Java 9 introduces ifPresentOrElse if a value is present, performs the given action with the value, otherwise performs the given empty-based action. See excellent Optional in Java 8 cheat sheet. It provides all answers for most use cases. Short summary below ifPresent() - do something when Optional is set opt.ifPresent(x -> print(x)); opt.ifPresent(this::print); filter() - reject (filter out) certain Optional values. opt.filter(x -> x.contains("ab")).ifPresent(this::print); map() - transform value if present opt.map(String::trim).filter(t -> t.length() > 1).ifPresent(this::print); orElse()/orElseGet() - turning empty Optional to default T int len = opt.map(String::length).orElse(-1); int len = opt. map(String::length). orElseGet(() -> slowDefault()); //orElseGet(this::slowDefault) orElseThrow() - lazily throw exceptions on empty Optional opt. filter(s -> !s.isEmpty()). map(s -> s.charAt(0)). orElseThrow(IllegalArgumentException::new);
An alternative is: System.out.println(opt.map(o -> "Found") .orElse("Not found")); I don't think it improves readability though. Or as Marko suggested, use a ternary operator: System.out.println(opt.isPresent() ? "Found" : "Not found");
Another solution would be to use higher-order functions as follows opt.<Runnable>map(value -> () -> System.out.println("Found " + value)) .orElse(() -> System.out.println("Not Found")) .run();
There isn't a great way to do it out of the box. If you want to be using your cleaner syntax on a regular basis, then you can create a utility class to help out: public class OptionalEx { private boolean isPresent; private OptionalEx(boolean isPresent) { this.isPresent = isPresent; } public void orElse(Runnable runner) { if (!isPresent) { runner.run(); } } public static <T> OptionalEx ifPresent(Optional<T> opt, Consumer<? super T> consumer) { if (opt.isPresent()) { consumer.accept(opt.get()); return new OptionalEx(true); } return new OptionalEx(false); } } Then you can use a static import elsewhere to get syntax that is close to what you're after: import static com.example.OptionalEx.ifPresent; ifPresent(opt, x -> System.out.println("found " + x)) .orElse(() -> System.out.println("NOT FOUND"));
If you can use only Java 8 or lower: 1) if you don't have spring-data the best way so far is: opt.<Runnable>map(param -> () -> System.out.println(param)) .orElse(() -> System.out.println("no-param-specified")) .run(); Now I know it's not so readable and even hard to understand for someone, but looks fine for me personally and I don't see another nice fluent way for this case. 2) if you're lucky enough and you can use spring-data the best way is Optionals#ifPresentOrElse: Optionals.ifPresentOrElse(opt, System.out::println, () -> System.out.println("no-param-specified")); If you can use Java 9, you should definitely go with: opt.ifPresentOrElse(System.out::println, () -> System.out.println("no-param-specified"));
You cannot call orElse after ifPresent, the reason is, orElse is called on an optiional but ifPresent returns void. So the best approach to achieve is ifPresentOrElse. It could be like this: op.ifPresentOrElse( (value) -> { System.out.println( "Value is present, its: " + value); }, () -> { System.out.println( "Value is empty"); });
The described behavior can be achieved by using Vavr (formerly known as Javaslang), an object-functional library for Java 8+, that implements most of Scala constructs (being Scala a more expressive language with a way richer type system built on JVM). It is a very good library to add to your Java projects to write pure functional code. Vavr provides the Option monad that provides functions to work with the Option type such as: fold: to map the value of the option on both cases (defined/empty) onEmpty: allows to execute a Runnable when option is empty peek: allows to consume the value of the option (when defined). and it is also Serializable on the contrary of Optional which means you can safely use it as method argument and instance member. Option follows the monad laws at difference to the Java's Optional "pseudo-monad" and provides a richer API. And of course you can make it from a Java's Optional (and the other way around): Option.ofOptional(javaOptional) –Vavr is focused on interoperability. Going to the example: // AWESOME Vavr functional collections (immutable for the gread good :) // fully convertible to Java's counterparts. final Map<String, String> map = Map("key1", "value1", "key2", "value2"); final Option<String> opt = map.get("nonExistentKey"); // you're safe of null refs! final String result = opt.fold( () -> "Not found!!!", // Option is None val -> "Found the value: " + val // Option is Some(val) ); Moreover, all Vavr types are convertible to its Java counterparts, for the sake of the example: Optional javaOptional = opt.toJava(), very easy :) Of course the conversion also exists in the other way: Option option = Option.ofOptional(javaOptional). N.B. Vavr offers a io.vavr.API class with a lot of convenient static methods =) Further reading Null reference, the billion dollar mistake N.B. This is only a very little example of what Vavr offers (pattern matching, streams a.k.a. lazy evaluated lists, monadic types, immutable collections,...).
The problem here: optional .map(object -> { System.out.println("If present."); return null; }) .orElseGet( () -> { System.out.println("If empty."); return null; }); Is that map() converts the null returned by the first function to empty(); it then returns empty(). As it returns empty(), it prompts the invocation of the second function. Note that orElseGet() does not convert the null returned by the second function to empty(), so it will return null. See the implementation of map(): public<U> Optional<U> map(Function<? super T, ? extends U> mapper) { Objects.requireNonNull(mapper); if (!isPresent()) return empty(); else { return Optional.ofNullable(mapper.apply(value)); } } And the implementation of orElseGet(): public T orElseGet(Supplier<? extends T> other) { return value != null ? value : other.get(); } Thus when executed: if optional.isPresent(), the system will print If present., then If empty., and the expression will evaluate to null. if !optional.isPresent(), the system will print If empty., and the expression will evaluate to null. If the function provided to map() returned any other value - any other value - the code would work as you expect, with the function provided to map() being executed if isPresent() and the function provided to orElseGet() if !isPresent(): For example, this: optional .map(data -> { System.out.println("If present."); return 0; }) .orElseGet( () -> { System.out.println("If empty."); return 0; }); When executed: if optional.isPresent(), the system will print If present., and the expression will evaluate to 0. if !optional.isPresent(), the system will print If empty., and the expression will evaluate to 0. If your specific case, I suggest that your insert and update methods return, say, the persisted object, or the id of the persisted object, or something similarly useful; then you can use code similar to this: final Object persist = optional .map(object -> { System.out.println("If present."); return update(object); }) .orElseGet( () -> { System.out.println("If empty."); return insert(new Object()); });
Another solution could be following: This is how you use it: final Opt<String> opt = Opt.of("I'm a cool text"); opt.ifPresent() .apply(s -> System.out.printf("Text is: %s\n", s)) .elseApply(() -> System.out.println("no text available")); Or in case you in case of the opposite use case is true: final Opt<String> opt = Opt.of("This is the text"); opt.ifNotPresent() .apply(() -> System.out.println("Not present")) .elseApply(t -> /*do something here*/); This are the ingredients: Little modified Function interface, just for the "elseApply" method Optional enhancement A little bit of curring :-) The "cosmetically" enhanced Function interface. #FunctionalInterface public interface Fkt<T, R> extends Function<T, R> { default R elseApply(final T t) { return this.apply(t); } } And the Optional wrapper class for enhancement: public class Opt<T> { private final Optional<T> optional; private Opt(final Optional<T> theOptional) { this.optional = theOptional; } public static <T> Opt<T> of(final T value) { return new Opt<>(Optional.of(value)); } public static <T> Opt<T> of(final Optional<T> optional) { return new Opt<>(optional); } public static <T> Opt<T> ofNullable(final T value) { return new Opt<>(Optional.ofNullable(value)); } public static <T> Opt<T> empty() { return new Opt<>(Optional.empty()); } private final BiFunction<Consumer<T>, Runnable, Void> ifPresent = (present, notPresent) -> { if (this.optional.isPresent()) { present.accept(this.optional.get()); } else { notPresent.run(); } return null; }; private final BiFunction<Runnable, Consumer<T>, Void> ifNotPresent = (notPresent, present) -> { if (!this.optional.isPresent()) { notPresent.run(); } else { present.accept(this.optional.get()); } return null; }; public Fkt<Consumer<T>, Fkt<Runnable, Void>> ifPresent() { return Opt.curry(this.ifPresent); } public Fkt<Runnable, Fkt<Consumer<T>, Void>> ifNotPresent() { return Opt.curry(this.ifNotPresent); } private static <X, Y, Z> Fkt<X, Fkt<Y, Z>> curry(final BiFunction<X, Y, Z> function) { return (final X x) -> (final Y y) -> function.apply(x, y); } } This should do the trick and could serve as a basic template how to deal with such requirements. The basic idea here is following. In a non functional style programming world you would probably implement a method taking two parameter where the first is a kind of runnable code which should be executed in case the value is available and the other parameter is the runnable code which should be run in case the value is not available. For the sake of better readability, you can use curring to split the function of two parameter in two functions of one parameter each. This is what I basically did here. Hint: Opt also provides the other use case where you want to execute a piece of code just in case the value is not available. This could be done also via Optional.filter.stuff but I found this much more readable. Hope that helps! Additional Info: There is another way to have say "if then else" using currying: public static <X, Y> Function<Predicate<X>, Function<Function<X, Y>, Function<Function<X, Y>, Y>>> ifThenElse(X input) { return (final Predicate<X> pred) -> (final Function<X, Y> ifPresent) -> (final Function<X, Y> ifNotPresent) -> pred.test(input) ? ifPresent.apply(input) : ifNotPresent.apply(input); } This way it is possible to say: final String result = ifThenElse("fancy") .apply(input -> input.contains("fancy")) /* test */ .apply(input -> input.toUpperCase()) /* if-case */ .apply(input -> input.toLowerCase()); /* else-case */
In case you want store the value: Pair.of<List<>, List<>> output = opt.map(details -> Pair.of(details.a, details.b))).orElseGet(() -> Pair.of(Collections.emptyList(), Collections.emptyList()));
Supposing that you have a list and avoiding the isPresent() issue (related with optionals) you could use .iterator().hasNext() to check if not present.