We Keep Coding

Reactor's Flux Fan out and Zip it?

I am trying to do following operations on Flux/Publisher which can only be read once ( think database results which can be read once). But, this question is generic enough that it can be answered in functional programming context without reactor knowledge.
count unique item
check if an element exists
Don't call the publisher/flux generator multiple times.
distinctAndHasElement(4, Flux.just(1,2,3,3,4,4,5));
Mono<Pair<Long, Boolean>> distinctAndHasElement(int toCheck, Flux<Integer> intsFlux) {
// Code that doesn't work, Due to use of non final local variable
boolean found = false;
return intsFlux.map(x -> {
if (toCheck == x) {
found = true;
}
return x;
})
.distinct()
.count()
.map(x -> Pair.of(x, found));
}
We just need ability to fan out into 2 functions that operate on the same type/domain, and zip the final result.
Following doesn't work due to constrain#3
Flux<Integer> distinct = intsFlux.distinct();
Mono<Boolean> found = distinct.hasElement(toCheck);
Mono<Long> count = distinct.count();
return Mono.zip(count, found);

What you're attempting to do is a reduction of your dataset. It means that you attempt to create a single result by merging your initial elements.
Note that count can be considered as a kind of reduction, and in your case, you want an advanced kind of count operation, that also check if at least one of the input elements is equal to a given value.
With reactor (and many other stream framework), you can use the reduce operator.
Let's try your first example with it :
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.util.function.Tuple2;
import reactor.util.function.Tuples;
public class CountAndCheck {
static Mono<Tuple2<Long, Boolean>> distinctAndHasElement(int toCheck, Flux<Integer> intsFlux) {
return intsFlux
.distinct()
.reduce(Tuples.of(0L, false), (intermediateResult, nextElement) -> {
return Tuples.of(intermediateResult.getT1() + 1L, intermediateResult.getT2() || toCheck == nextElement);
});
}
public static void main(String[] args) {
System.out.println(distinctAndHasElement(2, Flux.just(1, 2, 2, 3, 4, 4)).block());
}
}
The above program prints: [4,true]
Note: You can use the scan operator instead of reduction, to get a flux of every intermediate step in the reduction operation. It can be useful to understand how reduction is performed.

You can broadcast your Flux as described in the documentation.
Flux<Integer> distinct = intsFlux.distinct().publish().autoConnect(2);
Mono<Boolean> found = distinct.hasElement(toCheck);
Mono<Long> count = distinct.count();
return Mono.zip(count, found);

Java - Basic streams usage of forEach

I have a class called Data which has only one method:
public boolean isValid()
I have a Listof Dataand I want to loop through them via a Java 8 stream. I need to count how many valid Data objects there are in this List and print out only the valid entries.
Below is how far I've gotten but I don't understand how.
List<Data> ar = new ArrayList<>();
...
// ar is now full of Data objects.
...
int count = ar.stream()
.filter(Data::isValid)
.forEach(System.out::println)
.count(); // Compiler error, forEach() does not return type stream.
My second attempt: (horrible code)
List<Data> ar = new ArrayList<>();
...
// Must be final or compiler error will happen via inner class.
final AtomicInteger counter = new AtomicInteger();
ar.stream()
.filter(Data:isValid)
.forEach(d ->
{
System.out.println(d);
counter.incrementAndGet();
};
System.out.printf("There are %d/%d valid Data objects.%n", counter.get(), ar.size());

If you don’t need the original ArrayList, containing a mixture of valid and invalid objects, later-on, you might simply perform a Collection operation instead of the Stream operation:
ar.removeIf(d -> !d.isValid());
ar.forEach(System.out::println);
int count = ar.size();
Otherwise, you can implement it like
List<Data> valid = ar.stream().filter(Data::isValid).collect(Collectors.toList());
valid.forEach(System.out::println);
int count = valid.size();
Having a storage for something you need multiple times is not so bad. If the list is really large, you can reduce the storage memory by (typically) factor 32, using
BitSet valid = IntStream.range(0, ar.size())
.filter(index -> ar.get(index).isValid())
.collect(BitSet::new, BitSet::set, BitSet::or);
valid.stream().mapToObj(ar::get).forEach(System.out::println);
int count = valid.cardinality();
Though, of course, you can also use
int count = 0;
for(Data d: ar) {
if(d.isValid()) {
System.out.println(d);
count++;
}
}

Peek is similar to foreach, except that it lets you continue the stream.
ar.stream().filter(Data::isValid)
.peek(System.out::println)
.count();

Sum up fields based on another field in a stream [duplicate]

This question already has answers here:
Is there an aggregateBy method in the stream Java 8 api?
(3 answers)
Closed 6 years ago.
I have a List of objects that look like this:
{
value=500
category="GROCERY"
},
{
value=300
category="GROCERY"
},
{
value=100
category="FUEL"
},
{
value=300
category="SMALL APPLIANCE REPAIR"
},
{
value=200
category="FUEL"
}
I would like to transform that into a List of objects that looks like this:
{
value=800
category="GROCERY"
},
{
value=300
category="FUEL"
},
{
value=300
category="SMALL APPLIANCE REPAIR"
}
Basically add up all the values with the same category.
Should I be using flatMap? Reduce? I don't understand the nuances of these to figure it out.
Help?
EDIT:
There are close duplicates of this question:
Is there an aggregateBy method in the stream Java 8 api?
and
Sum attribute of object with Stream API
But in both cases, the end result is a map, not a list
The final solution I used, based on answers by #AndrewTobilko and #JBNizet was:
List<MyClass> myClassList = list.stream()
.collect(Collectors.groupingBy(YourClass::getCategory,
Collectors.summingInt(YourClass::getValue)))
.entrySet().stream().map(e -> new MyClass(e.getKey(), e.getValue()).collect(toList());

The Collectors class provides a 'groupingBy' that allows you to perform a 'group by' operation on a stream (similar behavior like GROUP BY in databases). Under the assumption that your list of objects is of type 'Objects', the following code should work:
Map<String, Integer> valueByCategory = myObjects.stream().collect(Collectors.groupingBy(MyObjects::getCategory, Collectors.summingInt(MyObjects::getValue)));
The code basically groups your stream by each category and runs a Collector on each group that sums up the return value of getValue() of every stream element.
See https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html

With static import of the Collectors class:
list.stream().collect(groupingBy(Class::getCategory, summingInt(Class::getValue)));
You will get a map Map<String, Integer>. Class has to have getValue and getCategory methods to write method references, something like
public class Class {
private String category;
private int value;
public String getCategory() { return category; }
public int getValue() { return value; }
}

Reduce-based method:
List<Obj> values = list.stream().collect(
Collectors.groupingBy(Obj::getCategory, Collectors.reducing((a, b) -> new Obj(a.getValue() + b.getValue(), a.getCategory())))
).values().stream().map(Optional::get).collect(Collectors.toList());
Bad thing is secondary stream() call to remap result from Optional<Obj> and intermediate Map<String, Optional<Obj>> object.
I can suggest alternative variant (less readable) using sorting:
List<Obj> values2 = list.stream()
.sorted((o1, o2) -> o1.getCategory().compareTo(o2.getCategory()))
.collect(
LinkedList<Obj>::new,
(ll, obj) -> {
Obj last = null;
if(!ll.isEmpty()) {
last = ll.getLast();
}
if (last == null || !last.getCategory().equals(obj.getCategory())) {
ll.add(new Obj(obj.getValue(), obj.getCategory())); //deep copy here
} else {
last.setValue(last.getValue() + obj.getValue());
}
},
(list1, list2) -> {
//for parallel execution do a simple merge join here
throw new RuntimeException("parallel evaluation not supported");
}
);
Here we sort list of Objs by category and then processing it sequentially, squashing consecutive objects from same category.
Unfortunately, there is no method in Java to do it without manually keeping last element or elements list (see also Collect successive pairs from a stream)
Working example with both snippets can be checked here: https://ideone.com/p3bKV8

Limit a stream and find out if there are pending elements

I have the following code that I want to translate to Java 8 streams:
public ReleaseResult releaseReources() {
List<String> releasedNames = new ArrayList<>();
Stream<SomeResource> stream = this.someResources();
Iterator<SomeResource> it = stream.iterator();
while (it.hasNext() && releasedNames.size() < MAX_TO_RELEASE) {
SomeResource resource = it.next();
if (!resource.isTaken()) {
resource.release();
releasedNames.add(resource.getName());
}
}
return new ReleaseResult(releasedNames, it.hasNext(), MAX_TO_RELEASE);
}
Method someResources() returns a Stream<SomeResource> and ReleaseResult class is as follows:
public class ReleaseResult {
private int releasedCount;
private List<String> releasedNames;
private boolean hasMoreItems;
private int releaseLimit;
public ReleaseResult(List<String> releasedNames,
boolean hasMoreItems, int releaseLimit) {
this.releasedNames = releasedNames;
this.releasedCount = releasedNames.size();
this.hasMoreItems = hasMoreItems;
this.releaseLimit = releaseLimit;
}
// getters & setters
}
My attempt so far:
public ReleaseResult releaseReources() {
List<String> releasedNames = this.someResources()
.filter(resource -> !resource.isTaken())
.limit(MAX_TO_RELEASE)
.peek(SomeResource::release)
.map(SomeResource::getName)
.collect(Collectors.toList());
return new ReleasedResult(releasedNames, ???, MAX_TO_RELEASE);
}
The problem is that I can't find a way to know if there are pending resources to process. I've thought of using releasedNames.size() == MAX_TO_RELEASE, but this doesn't take into account the case where the stream of resources has exactly MAX_TO_RELEASE elements.
Is there a way to do the same with Java 8 streams?
Note: I'm not looking for answers like "you don't have to do everything with streams" or "using loops and iterators is fine". I'm OK if using an iterator and a loop is the only way or just the best way. It's just that I'd like to know if there's a non-murky way to do the same.

Since you don’t wanna hear that you don’t need streams for everything and loops and iterators are fine, let’s demonstrate it by showing a clean solution, not relying on peek:
public ReleaseResult releaseReources() {
return this.someResources()
.filter(resource -> !resource.isTaken())
.limit(MAX_TO_RELEASE+1)
.collect(
() -> new ReleaseResult(new ArrayList<>(), false, MAX_TO_RELEASE),
(result, resource) -> {
List<String> names = result.getReleasedNames();
if(names.size() == MAX_TO_RELEASE) result.setHasMoreItems(true);
else {
resource.release();
names.add(resource.getName());
}
},
(r1, r2) -> {
List<String> names = r1.getReleasedNames();
names.addAll(r2.getReleasedNames());
if(names.size() > MAX_TO_RELEASE) {
r1.setHasMoreItems(true);
names.remove(MAX_TO_RELEASE);
}
}
);
}
This assumes that // getters & setters includes getters and setters for all non-final fields of your ReleaseResult. And that getReleasedNames() returns the list by reference. Otherwise you would have to rewrite it to provide a specialized Collector having special non-public access to ReleaseResult (implementing another builder type or temporary storage would be an unnecessary complication, it looks like ReleaseResult is already designed exactly for that use case).
We could conclude that for any nontrivial loop code that doesn’t fit into the stream’s intrinsic operations, you can find a collector solution that basically does the same as the loop in its accumulator function, but suffers from the requirement of always having to provide a combiner function. Ok, in this case we can prepend a filter(…).limit(…) so it’s not that bad…
I just noticed, if you ever dare to use that with a parallel stream, you need a way to reverse the effect of releasing the last element in the combiner in case the combined size exceeds MAX_TO_RELEASE. Generally, limits and parallel processing never play well.

I don't think there's a nice way to do this. I've found a hack that does it lazily. What you can do is convert the Stream to an Iterator, convert the Iterator back to another Stream, do the Stream operations, then finally test the Iterator for a next element!
Iterator<SomeResource> it = this.someResource().iterator();
List<String> list = StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, Spliterator.ORDERED), false)
.filter(resource -> !resource.isTaken())
.limit(MAX_TO_RELEASE)
.peek(SomeResource::release)
.map(SomeResource::getName)
.collect(Collectors.toList());
return new ReleaseResult(list, it.hasNext(), MAX_TO_RELEASE);

The only thing I can think of is
List<SomeResource> list = someResources(); // A List, rather than a Stream, is required
List<Integer> indices = IntStream.range(0, list.size())
.filter(i -> !list.get(i).isTaken())
.limit(MAX_TO_RELEASE)
.collect(Collectors.toList());
List<String> names = indices.stream()
.map(list::get)
.peek(SomeResource::release)
.map(SomeResource::getName)
.collect(Collectors.toList());
Then (I think) there are unprocessed elements if
names.size() == MAX_TO_RELEASE
&& (indices.isEmpty() || indices.get(indices.size() - 1) < list.size() - 1)

How should I check whether a Stream<T> is sorted?

With an Iterable<T>, it's easy:
T last = null;
for (T t : iterable) {
if (last != null && last.compareTo(t) > 0) {
return false;
}
last = t;
}
return true;
But I can't think of a clean way to do the same thing for a Stream<T> that avoids consuming all the elements when it doesn't have to.

There are several methods to iterate over the successive pairs of the stream. For example, you can check this question. Of course my favourite method is to use the library I wrote:
boolean unsorted = StreamEx.of(sourceStream)
.pairMap((a, b) -> a.compareTo(b) > 0)
.has(true);
It's short-circuit operation: it will finish as soon as it find the misorder. Also it works fine with parallel streams.

This is a sequential, state holding solution:
IntStream stream = IntStream.of(3, 3, 5, 6, 6, 9, 10);
final AtomicInteger max = new AtomicInteger(Integer.MIN_VALUE);
boolean sorted = stream.allMatch(n -> n >= max.getAndSet(n));
Parallelizing would need to introduce ranges. The state, max might be dealt with otherwise, but the above seems most simple.

You can grab the Stream's underlying spliterator and check it it has the SORTED characteristic. Since it's a terminal operation, you can't use the Stream after (but you can create another one from this spliterator, see also Convert Iterable to Stream using Java 8 JDK).
For example:
Stream<Integer> st = Stream.of(1, 2, 3);
//false
boolean isSorted = st.spliterator().hasCharacteristics(Spliterator.SORTED);
Stream<Integer> st = Stream.of(1, 2, 3).sorted();
//true
boolean isSorted = st.spliterator().hasCharacteristics(Spliterator.SORTED);
My example shows that the SORTED characteristic appears only if you get the Stream from a source's that reports the SORTED characteristic or you call sorted() at a point on the pipeline.
One could argue that Stream.iterate(0, x -> x + 1); creates a SORTED stream, but there is no knowledge about the semantic of the function applied iteratively. The same applies for Stream.of(...).
If the pipeline is infinite then it's the only way to know. If not, and that the spliterator does not report this characteristic, you'd need to go through the elements and see if it does not satisfy the sorted characteristic you are looking for.
This is what you already done with your iterator approach but then you need to consume some elements of the Stream (in the worst case, all elements). You can make the task parallelizable with some extra code, then it's up to you to see if it's worth it or not...

You could hijack a reduction operation to save the last value and compare it to the current value and throw an exception if it isn't sorted:
.stream().reduce((last, curr) -> {
if (((Comparable)curr).compareTo(last) < 0) {
throw new Exception();
}
return curr;
});
EDIT: I forked another answer's example and replaced it with my code to show it only does the requisite number of checks.
http://ideone.com/ZMGnVW

You could use allMatch with a multi-line lambda, checking the current value against the previous one. You'll have to wrap the last value into an array, though, so the lambda can modify it.
// infinite stream with one pair of unsorted numbers
IntStream s = IntStream.iterate(0, x -> x != 1000 ? x + 2 : x - 1);
// terminates as soon as the first unsorted pair is found
int[] last = {Integer.MIN_VALUE};
boolean sorted = s.allMatch(x -> {
boolean b = x >= last[0]; last[0] = x; return b;
});
Alternatively, just get the iterator from the stream and use a simple loop.

A naive solution uses the stream's Iterator:
public static <T extends Comparable<T>> boolean isSorted(Stream<T> stream) {
Iterator<T> i = stream.iterator();
if(!i.hasNext()) return true;
T current = i.next();
while(i.hasNext()) {
T next = i.next();
if(current == null || current.compareTo(next) > 0) return false;
current = next;
}
return true;
}
Edit: It would also be possible to use a spliterator to parallelize the task, but the gains would be questionable and the increase in complexity is probably not worth it.

I don't know how good it is , but i have just got an idea:
Make a list out of your Stream , Integer or Strings or anything.
i have written this for a List<String> listOfStream:
long countSorted = IntStream.range(1, listOfStream.size())
.map(
index -> {
if (listOfStream.get(index).compareTo(listOfStream.get(index-1)) > 0) {
return 0;
}
return index;
})
.sum();