We Keep Coding

Java stream collect check if result would contain element

As I couldn't find anything related to this, I am wondering if streams even allow this.
In my answer to another question, I have following code to add elements to a result list, only if the result list doesn't already contain it:
List<Entry<List<Integer>, Integer>> list = new ArrayList<>(diffMap.entrySet());
list.sort(Entry.comparingByValue());
List<List<Integer>> resultList = new ArrayList<>();
for (Entry<List<Integer>, Integer> entry2 : list) {
if (!checkResultContainsElement(resultList, entry2.getKey()))
resultList.add(entry2.getKey());
}
checkResultContainsElement method:
private static boolean checkResultContainsElement(List<List<Integer>> resultList, List<Integer> key) {
List<Integer> vals = resultList.stream().flatMap(e -> e.stream().map(e2 -> e2))
.collect(Collectors.toList());
return key.stream().map(e -> e).anyMatch(e -> vals.contains(e));
}
Now I am wondering, if this for-loop:
for (Entry<List<Integer>, Integer> entry2 : list) {
if (!checkResultContainsElement(resultList, entry2.getKey()))
resultList.add(entry2.getKey());
}
can be realized using streams. I don't think that .filter() method would work, as it would remove data from List<Entry<List<Integer>, Integer>> list while I don't even know if an element should be considered. I guess that a custom collector could work, but I also wouldn't know how to implement one, as the result is constantly changing with each newly added element.
I am looking for something like this (can be different if something else is better):
list.stream().sorted(Entry.comparingByValue()).collect(???);
where ??? would filter the data and return it as a list.
The values of one result list may not be contained in another one. So these lists are valid:
[1, 2, 3, 4]
[5, 6, 7, 8]
[12, 12, 12, 12]
but of these, only the first is valid:
[1, 2, 3, 4] <-- valid
[5, 3, 7, 8] <-- invalid: 3 already exists
[12, 12, 2, 12] <-- invalid: 2 already exists

If we put aside for a moment the details on whether implementation will be stream-based or not, the existing implementation of how uniqueness of the values of incoming lists is being checked can be improved.
We can gain a significant performance improvement by maintaining a Set of previously encountered values.
I.e. values from each list that was added to the resulting list would be stored in a set. And in order to ensure uniqueness of every incoming list, its values would be checked against the set.
Since operations of a stream pipeline should be stateless, as well as collector shouldn't hold a state (i.e. changes should happen only inside its mutable container). We can approach this problem by defining a container that will encompass a resulting list of lists of Foo and a set of foo-values.
I've implemented this container as a Java 16 record:
public record FooContainer(Set<Integer> fooValues, List<List<Foo>> foosList) {
public void tryAdd(List<Foo> foos) {
if (!hasValue(foos)) {
foos.forEach(foo -> fooValues.add(foo.getValue()));
foosList.add(foos);
}
}
public boolean hasValue(List<Foo> foos) {
return foos.stream().map(Foo::getValue).anyMatch(fooValues::contains);
}
}
The record shown above would is used as a mutable container of a custom collector created with Colloctors.of(). Collector's accumulator make's use of tryAdd() method defined by the container. And the finisher extracts the resulting list from the container.
Note that this operation is not parallelizable, hence combiner of the collector throws an AssertionError.
public static void main(String[] args) {
Map<List<Foo>, Integer> diffMap =
Map.of(List.of(new Foo(1), new Foo(2), new Foo(3)), 1,
List.of(new Foo(1), new Foo(4), new Foo(5)), 2,
List.of(new Foo(7), new Foo(8), new Foo(9)), 3);
List<List<Foo>> result = diffMap.entrySet().stream()
.sorted(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.collect(Collector.of(
() -> new FooContainer(new HashSet<>(), new ArrayList<>()),
FooContainer::tryAdd,
(left, right) -> {throw new AssertionError("The operation isn't parallelizable");},
FooContainer::foosList
));
System.out.println(result);
}
Output:
[[Foo{1}, Foo{2}, Foo{3}], [Foo{7}, Foo{8}, Foo{9}]]

May be something like this:-
list.stream().
sorted(Entry.comparingByValue()).
collect(ArrayList<List<Foo>>::new,(x,y)->!checkResultContainsElement(x, y.getKey()),(x,y)->x.add(y.getKey()));

Explaining Java Stream map function using only reduce and lambdas

I am currently working through a Java 8 Lambdas book (quite a popular one) and I am confused about some syntax in the ANSWER for one of the advanced questions.
The question asks the following:
Write an implementation of the Stream function 'map' using only reduce and lambda expressions. You can return a List instead of a Stream.
I'd like to highlight that I am not interested in the "best" answer to this question, I am interested in understanding the syntax of the answer to this question given in the book. The answer is as follows:
public static <I, O> List<O> map(Stream<I> stream, Function<I, O> mapper) {
return stream.reduce(new ArrayList<O>(), (acc, x) -> {
// We are copying data from acc to new list instance. It is very inefficient,
// but contract of Stream.reduce method requires that accumulator function does
// not mutate its arguments.
// Stream.collect method could be used to implement more efficient mutable reduction,
// but this exercise asks to use reduce method.
List<O> newAcc = new ArrayList<>(acc);
newAcc.add(mapper.apply(x));
return newAcc;
}, (List<O> left, List<O> right) -> {
// We are copying left to new list to avoid mutating it.
List<O> newLeft = new ArrayList<>(left);
newLeft.addAll(right);
return newLeft;
});
}
I understand what a reduce function does, and thus I understand the instantiation of the initial ArrayList, and the part which follows - creating a new ArrayList, adding the new function to the list to accumulate, and then returning the new ArrayList as the result.
The bit I do not understand is the next part:
, (List<O> left, List<O> right) -> {
// We are copying left to new list to avoid mutating it.
List<O> newLeft = new ArrayList<>(left);
newLeft.addAll(right);
return newLeft;
});
What is this doing? I understand the contents of the lambda i.e. the behaviour. But I don't understand what this lambda is doing in the entire context of the reduce function? Why wasn't the first section enough? Why do we have this extra lambda here and how is it contributing to this map function that we are creating?
So far Java 8 lambdas have been pretty straightforward, I feel as though I understood all the theory in the book so far, but maybe I misunderstood something? I wonder what I am missing here?

This last part is called a combiner and is useful if your Stream is parallel.
It will create multiple intermediate results that will need to be put together in the end.
This is exactly what this lambda is doing.
You can this by executing the following piece of code first, which will run a sequential Stream through your function. Notice how I added a System.out.println("Combining...") inside of the combiner.
public static void main(String[] args) {
Stream<Integer> boxed = IntStream.rangeClosed(1, 10).limit(25).boxed();
List<String> map = map(boxed, String::valueOf);
System.out.println(map);
}
public static <I, O> List<O> map(Stream<I> stream, Function<I, O> mapper) {
return stream.reduce(new ArrayList<O>(), (acc, x) -> {
// We are copying data from acc to new list instance. It is very inefficient,
// but contract of Stream.reduce method requires that accumulator function does
// not mutate its arguments.
// Stream.collect method could be used to implement more efficient mutable reduction,
// but this exercise asks to use reduce method.
List<O> newAcc = new ArrayList<>(acc);
newAcc.add(mapper.apply(x));
return newAcc;
}, (List<O> left, List<O> right) -> {
System.out.println("Combining...");
// We are copying left to new list to avoid mutating it.
List<O> newLeft = new ArrayList<>(left);
newLeft.addAll(right);
return newLeft;
});
}
Prints
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Now run the following part, where I made the Stream parallel
public static void main(String[] args) {
Stream<Integer> boxed = IntStream.rangeClosed(1, 10).parallel().limit(25).boxed();
List<String> map = map(boxed, String::valueOf);
System.out.println(map);
}
It prints
Combining...
Combining...
Combining...
Combining...
Combining...
Combining...
Combining...
Combining...
Combining...
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
More info about combiners

Java - How to write iterator that iterates through values of a HashSet within a Map

I've been given a problem where I have to write an iterator for an AbstractMap where the values are represented in a HashSet. This is the class that I am working with:
public class SetMap<K, V> extends AbstractMap<K, HashSet<V>> implements Iterable<V>
The instructions I have been given are as follows:
"Implement the iterator such that only values V are traversed. Values are traversed first in descending order of the size of HashSet objects associated with keys, and then in the iterator order for the HashSet."
I am basically unsure of how to approach writing my custom hasNext() and Next(), considering I am only supposed to traverse the HashSet within the Map.
I am new to Java, so traversing multidimensional maps, especially with sets as some of values, is very confusing to me.
An example of what my main function could do-
SetMap<String, Integer> map = new SetMap<>();
map.addValue("B", 4);
map.addValue("A", 0);
map.addValue("A", 1);
map.addValue("B", 3);
map.addValue("A", 2);
for (Integer value : map) {
System.out.println(value);
}
}
-is to produce something like this:
0
1
2
3
4
Any assistance, or relevant resources, would be much appreciated.

Use HashSet's get(String s)
SetMap<String, Integer> map = new SetMap<>();
map.addValue("B", 4);
map.addValue("A", 0);
map.addValue("A", 1);
map.addValue("B", 3);
map.addValue("A", 2);
for (String s : map) {
System.out.println(map.get(value));
}
}

Counting each distinct array occurrence in a list of arrays with duplicates

PROBLEM
I have a list of arrays and I want to count the occurrences of duplicates.
For example, if I have this :
{{1,2,3},
{1,0,3},
{1,2,3},
{5,2,6},
{5,2,6},
{5,2,6}}
I want a map (or any relevant collection) like this :
{ {1,2,3} -> 2,
{1,0,3} -> 1,
{5,2,6} -> 3 }
I can even lose the arrays values, I'm only interested in cardinals (e.g. 2, 1 and 3 here).
MY SOLUTION
I use the following algorithm :
First hash the arrays, and check if each hash is in an HashMap<Integer, ArrayList<int[]>>, let's name it distinctHash, where the key is the hash and the value is an ArrayList, let's name it rowList, containing the different arrays for this hash (to avoid collisions).
If the hash is not in distinctHash, put it with the value 1 in another HashMap<int[], Long> that counts each occurrence, let's call it distinctElements.
Then if the hash is in distinctHash, check if the corresponding array is contained in rowList. If it is, increment the value in distinctElements associated to the identical array found in rowList. (If you use the new array as a key you will create another key since their reference are different).
Here is the code, the boolean returned tells if a new distinct array was found, I apply this function sequentially on all of my arrays :
HashMap<int[], Long> distinctElements;
HashMap<Integer, ArrayList<int[]>> distinctHash;
private boolean addRow(int[] row) {
if (distinctHash.containsKey(hash)) {
int[] indexRow = distinctHash.get(hash).get(0);
for (int[] previousRow: distinctHash.get(hash)) {
if (Arrays.equals(previousRow, row)) {
distinctElements.put(
indexRow,
distinctElements.get(indexRow) + 1
);
return false;
}
}
distinctElements.put(row, 1L);
ArrayList<int[]> rowList = distinctHash.get(hash);
rowList.add(row);
distinctHash.put(hash, rowList);
return true;
} else {
distinctElements.put(row, 1L);
ArrayList<int[]> newValue = new ArrayList<>();
newValue.add(row);
distinctHash.put(hash, newValue);
return true;
}
}
QUESTION
The problem is that my algorithm is too slow for my needs (40s for 5,000,000 arrays, and 2h-3h for 20,000,000 arrays). Profiling with NetBeans told me that the hashing takes 70% of runtime (using Google Guava murmur3_128 hash function).
Is there another algorithm that could be faster? As I said I'm not interested in arrays values, only in the number of their occurrences. I am ready to sacrifice precision for speed so a probabilistic algorithm is fine.

Wrap the int[] in a class that implements equals and hashCode, then build Map of the wrapper class to instance count.
class IntArray {
private int[] array;
public IntArray(int[] array) {
this.array = array;
}
#Override
public int hashCode() {
return Arrays.hashCode(this.array);
}
#Override
public boolean equals(Object obj) {
return (obj instanceof IntArray && Arrays.equals(this.array, ((IntArray) obj).array));
}
#Override
public String toString() {
return Arrays.toString(this.array);
}
}
Test
int[][] input = {{1,2,3},
{1,0,3},
{1,2,3},
{5,2,6},
{5,2,6},
{5,2,6}};
Map<IntArray, Long> map = Arrays.stream(input).map(IntArray::new)
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
map.entrySet().forEach(System.out::println);
Output
[1, 2, 3]=2
[1, 0, 3]=1
[5, 2, 6]=3
Note: The above solution is faster and uses less memory than solution by Ravindra Ranwala, but it does require the creation of an extra class, so it is debatable which is better.
For smaller arrays, use the simpler solution below by Ravindra Ranwala.
For larger arrays, the above solution is likely better.
Map<List<Integer>, Long> map = Stream.of(input)
.map(a -> Arrays.stream(a).boxed().collect(Collectors.toList()))
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));

You may do it like so,
Map<List<Integer>, Long> result = Stream.of(source)
.map(a -> Arrays.stream(a).boxed().collect(Collectors.toList()))
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
And here's the output,
{[1, 2, 3]=2, [1, 0, 3]=1, [5, 2, 6]=3}

If the sequence of elements for all duplication of that array is like each other and the length of each array is not much, you can map each array to an int number and using from last part of your method. Although this method decrease the time of hashing, there are some assumptions here which might not be true for your case.

How to force max to return ALL maximum values in a Java Stream?

I've tested a bit the max function on Java 8 lambdas and streams, and it seems that in case max is executed, even if more than one object compares to 0, it returns an arbitrary element within the tied candidates without further consideration.
Is there an evident trick or function for such a max expected behavior, so that all max values are returned? I don't see anything in the API but I am sure it must exist something better than comparing manually.
For instance:
// myComparator is an IntegerComparator
Stream.of(1, 3, 5, 3, 2, 3, 5)
.max(myComparator)
.forEach(System.out::println);
// Would print 5, 5 in any order.

I believe the OP is using a Comparator to partition the input into equivalence classes, and the desired result is a list of members of the equivalence class that is the maximum according to that Comparator.
Unfortunately, using int values as a sample problem is a terrible example. All equal int values are fungible, so there is no notion of preserving the ordering of equivalent values. Perhaps a better example is using string lengths, where the desired result is to return a list of strings from an input that all have the longest length within that input.
I don't know of any way to do this without storing at least partial results in a collection.
Given an input collection, say
List<String> list = ... ;
...it's simple enough to do this in two passes, the first to get the longest length, and the second to filter the strings that have that length:
int longest = list.stream()
.mapToInt(String::length)
.max()
.orElse(-1);
List<String> result = list.stream()
.filter(s -> s.length() == longest)
.collect(toList());
If the input is a stream, which cannot be traversed more than once, it is possible to compute the result in only a single pass using a collector. Writing such a collector isn't difficult, but it is a bit tedious as there are several cases to be handled. A helper function that generates such a collector, given a Comparator, is as follows:
static <T> Collector<T,?,List<T>> maxList(Comparator<? super T> comp) {
return Collector.of(
ArrayList::new,
(list, t) -> {
int c;
if (list.isEmpty() || (c = comp.compare(t, list.get(0))) == 0) {
list.add(t);
} else if (c > 0) {
list.clear();
list.add(t);
}
},
(list1, list2) -> {
if (list1.isEmpty()) {
return list2;
}
if (list2.isEmpty()) {
return list1;
}
int r = comp.compare(list1.get(0), list2.get(0));
if (r < 0) {
return list2;
} else if (r > 0) {
return list1;
} else {
list1.addAll(list2);
return list1;
}
});
}
This stores intermediate results in an ArrayList. The invariant is that all elements within any such list are equivalent in terms of the Comparator. When adding an element, if it's less than the elements in the list, it's ignored; if it's equal, it's added; and if it's greater, the list is emptied and the new element is added. Merging isn't too difficult either: the list with the greater elements is returned, but if their elements are equal the lists are appended.
Given an input stream, this is pretty easy to use:
Stream<String> input = ... ;
List<String> result = input.collect(maxList(comparing(String::length)));

I would group by value and store the values into a TreeMap in order to have my values sorted, then I would get the max value by getting the last entry as next:
Stream.of(1, 3, 5, 3, 2, 3, 5)
.collect(groupingBy(Function.identity(), TreeMap::new, toList()))
.lastEntry()
.getValue()
.forEach(System.out::println);
Output:
5
5

I implemented more generic collector solution with custom downstream collector. Probably some readers might find it useful:
public static <T, A, D> Collector<T, ?, D> maxAll(Comparator<? super T> comparator,
Collector<? super T, A, D> downstream) {
Supplier<A> downstreamSupplier = downstream.supplier();
BiConsumer<A, ? super T> downstreamAccumulator = downstream.accumulator();
BinaryOperator<A> downstreamCombiner = downstream.combiner();
class Container {
A acc;
T obj;
boolean hasAny;
Container(A acc) {
this.acc = acc;
}
}
Supplier<Container> supplier = () -> new Container(downstreamSupplier.get());
BiConsumer<Container, T> accumulator = (acc, t) -> {
if(!acc.hasAny) {
downstreamAccumulator.accept(acc.acc, t);
acc.obj = t;
acc.hasAny = true;
} else {
int cmp = comparator.compare(t, acc.obj);
if (cmp > 0) {
acc.acc = downstreamSupplier.get();
acc.obj = t;
}
if (cmp >= 0)
downstreamAccumulator.accept(acc.acc, t);
}
};
BinaryOperator<Container> combiner = (acc1, acc2) -> {
if (!acc2.hasAny) {
return acc1;
}
if (!acc1.hasAny) {
return acc2;
}
int cmp = comparator.compare(acc1.obj, acc2.obj);
if (cmp > 0) {
return acc1;
}
if (cmp < 0) {
return acc2;
}
acc1.acc = downstreamCombiner.apply(acc1.acc, acc2.acc);
return acc1;
};
Function<Container, D> finisher = acc -> downstream.finisher().apply(acc.acc);
return Collector.of(supplier, accumulator, combiner, finisher);
}
So by default it can be collected to a list using:
public static <T> Collector<T, ?, List<T>> maxAll(Comparator<? super T> comparator) {
return maxAll(comparator, Collectors.toList());
}
But you can use other downstream collectors as well:
public static String joinLongestStrings(Collection<String> input) {
return input.stream().collect(
maxAll(Comparator.comparingInt(String::length), Collectors.joining(","))));
}

If I understood well, you want the frequency of the max value in the Stream.
One way to achieve that would be to store the results in a TreeMap<Integer, List<Integer> when you collect elements from the Stream. Then you grab the last key (or first depending on the comparator you give) to get the value which will contains the list of max values.
List<Integer> maxValues = st.collect(toMap(i -> i,
Arrays::asList,
(l1, l2) -> Stream.concat(l1.stream(), l2.stream()).collect(toList()),
TreeMap::new))
.lastEntry()
.getValue();
Collecting it from the Stream(4, 5, -2, 5, 5) will give you a List [5, 5, 5].
Another approach in the same spirit would be to use a group by operation combined with the counting() collector:
Entry<Integer, Long> maxValues = st.collect(groupingBy(i -> i,
TreeMap::new,
counting())).lastEntry(); //5=3 -> 5 appears 3 times
Basically you firstly get a Map<Integer, List<Integer>>. Then the downstream counting() collector will return the number of elements in each list mapped by its key resulting in a Map. From there you grab the max entry.
The first approaches require to store all the elements from the stream. The second one is better (see Holger's comment) as the intermediate List is not built. In both approached, the result is computed in a single pass.
If you get the source from a collection, you may want to use Collections.max one time to find the maximum value followed by Collections.frequency to find how many times this value appears.
It requires two passes but uses less memory as you don't have to build the data-structure.
The stream equivalent would be coll.stream().max(...).get(...) followed by coll.stream().filter(...).count().

I'm not really sure whether you are trying to
(a) find the number of occurrences of the maximum item, or
(b) Find all the maximum values in the case of a Comparator that is not consistent with equals.
An example of (a) would be [1, 5, 4, 5, 1, 1] -> [5, 5].
An example of (b) would be:
Stream.of("Bar", "FOO", "foo", "BAR", "Foo")
.max((s, t) -> s.toLowerCase().compareTo(t.toLowerCase()));
which you want to give [Foo, foo, Foo], rather than just FOO or Optional[FOO].
In both cases, there are clever ways to do it in just one pass. But these approaches are of dubious value because you would need to keep track of unnecessary information along the way. For example, if you start with [2, 0, 2, 2, 1, 6, 2], it would only be when you reach 6 that you would realise it was not necessary to track all the 2s.
I think the best approach is the obvious one; use max, and then iterate the items again putting all the ties into a collection of your choice. This will work for both (a) and (b).

If you'd rather rely on a library than the other answers here, StreamEx has a collector to do this.
Stream.of(1, 3, 5, 3, 2, 3, 5)
.collect(MoreCollectors.maxAll())
.forEach(System.out::println);
There's a version which takes a Comparator too for streams of items which don't have a natural ordering (i.e. don't implement Comparable).

System.out.println(
Stream.of(1, 3, 5, 3, 2, 3, 5)
.map(a->new Integer[]{a})
.reduce((a,b)->
a[0]==b[0]?
Stream.concat(Stream.of(a),Stream.of(b)).toArray() :
a[0]>b[0]? a:b
).get()
)

I was searching for a good answer on this question, but a tad more complex and couldn't find anything until I figured it out myself, which is why I'm posting if this helps anybody.
I have a list of Kittens.
Kitten is an object which has a name, age and gender. I had to return a list of all the youngest kittens.
For example:
So kitten list would contain kitten objects (k1, k2, k3, k4) and their ages would be (1, 2, 3, 1) accordingly. We want to return [k1, k4], because they are both the youngest. If only one youngest exists, the function should return [k1(youngest)].
Find the min value of the list (if it exists):
Optional<Kitten> minKitten = kittens.stream().min(Comparator.comparingInt(Kitten::getAge));
filter the list by the min value
return minKitten.map(value -> kittens.stream().filter(kitten -> kitten.getAge() == value.getAge())
.collect(Collectors.toList())).orElse(Collections.emptyList());

The following two lines will do it without implementing a separate comparator:
List<Integer> list = List.of(1, 3, 5, 3, 2, 3, 5);
list.stream().filter(i -> i == (list.stream().max(Comparator.comparingInt(i2 -> i2))).get()).forEach(System.out::println);