We Keep Coding

java stream for grouping by multiple keys [duplicate]

I have a list of class say ProductDto
public class ProductDto {
private String Id;
private String status;
private Booker booker;
private String category;
private String type;
}
I want to have a Map as below:-
Map<String,Map<String,Map<String,Booker>>
The properties are to be mapped as below:
Map<status,Map<category,Map<type,Booker>
I know one level of grouping could be done easily without any hassles using Collectors.groupingBy.
I tried to use this for nested level but it failed for me when same values started coming for fields that are keys.
My code is something like below:-
list.stream()
.collect(Collectors.groupingBy(
(FenergoProductDto productDto) ->
productDto.getStatus()
,
Collectors.toMap(k -> k.getProductCategory(), fProductDto -> {
Map<String, Booker> productTypeMap = new ProductTypes();
productTypeMap.put(fProductDto.getProductTypeName(),
createBooker(fProductDto.getBookingEntityName()));
return productTypeMap;
})
));
If anyone knows a good approach to do this by using streams, please share!

Abstract / Brief discussion
Having a map of maps of maps is questionable when seen from an object-oriented prespective, as it might seem that you're lacking some abstraction (i.e. you could create a class Result that encapsulates the results of the nested grouping). However, it's perfectly reasonable when considered exclusively from a pure data-oriented approach.
So here I present two approaches: the first one is purely data-oriented (with nested groupingBy calls, hence nested maps), while the second one is more OO-friendly and makes a better job at abstracting the grouping criteria. Just pick the one which better represents your intentions and coding standards/traditions and, more importantly, the one you most like.
Data-oriented approach
For the first approach, you can just nest the groupingBy calls:
Map<String, Map<String, Map<String, List<Booker>>>> result = list.stream()
.collect(Collectors.groupingBy(ProductDto::getStatus,
Collectors.groupingBy(ProductDto::getCategory,
Collectors.groupingBy(ProductDto::getType,
Collectors.mapping(
ProductDto::getBooker,
Collectors.toList())))));
As you see, the result is a Map<String, Map<String, Map<String, List<Booker>>>>. This is because there might be more than one ProductDto instance with the same (status, category, type) combination.
Also, as you need Booker instances instead of ProductDto instances, I'm adapting the last groupingBy collector so that it returns Bookers instead of productDtos.
About reduction
If you need to have only one Booker instance instead of a List<Booker> as the value of the innermost map, you would need a way to reduce Booker instances, i.e. convert many instances into one by means of an associative operation (accumulating the sum of some attribute being the most common one).
Object-oriented friendly approach
For the second approach, having a Map<String, Map<String, Map<String, List<Booker>>>> might be seen as bad practice or even as pure evil. So, instead of having a map of maps of maps of lists, you could have only one map of lists whose keys represent the combination of the 3 properties you want to group by.
The easiest way to do this is to use a List as the key, as lists already provide hashCode and equals implementations:
Map<List<String>, List<Booker>> result = list.stream()
.collect(Collectors.groupingBy(
dto -> Arrays.asList(dto.getStatus(), dto.getCategory(), dto.getType()),
Collectors.mapping(
ProductDto::getBooker,
Collectors.toList())))));
If you are on Java 9+, you can use List.of instead of Arrays.asList, as List.of returns a fully immutable and highly optimized list.

nested groupingBy questions and solutions:
q. print all male and female dept-wise(nested groupingBy):
ans:
employeeList.stream().collect(Collectors.groupingBy(Employee::getDepartment,Collectors.groupingBy(Employee::getGender)))
.entrySet().stream().forEach(System.out::println)
q. print the employees more than 25 and not - male and female - dept-wise
ans:
employeeList.stream().collect(
Collectors.groupingBy(Employee::getDepartment, Collectors.groupingBy(Employee::getGender, Collectors.partitioningBy(emp -> emp.getAge() > 25))))
.entrySet().stream().forEach(System.out::println);
q. eldest male and female from each department
ans:
employeeList.stream().collect(Collectors.groupingBy(Employee::getDepartment,Collectors.groupingBy(Employee::getGender,Collectors.maxBy(Comparator.comparing(Employee::getAge)))))
.entrySet().stream().forEach(System.out::println);
some more helpful questions #:
[1]: https://www.youtube.com/watch?v=AFmyV43UBgc

Merge two collections using streams, but only unique values, and using predicate instead of equals?

I'm trying to merge two collections, but do it conditionally where I only want to add unique values. And what constitutes uniqueness should be decided by a predicate (or similar), not the equals function.
For example, let's assume we have the following two collections of Person objects:
List<Employee> list1 = Arrays.asList(new Employee(1, "Adam", "Smith", Type.CEO), new Employee(2, "Bob", "Jones", Type.OfficeManager), new Employee(3, "Carl", "Lewis", Type.SalesPerson));
List<Employee> list2 = Arrays.asList(new Employee(4, "Xerxes", "Brown", Type.OfficeManager), new Employee(5, "Yuri", "Gagarin", Type.Janitor), new Employee(6, "Zain", "Wilson", Type.SalesPerson));
...and lets assume that I want to merge these lists into a new list by adding elements from both list1 and list2, but excluding elements that have an corresponding "identical" person object already added to the new list, and where the uniqueness is determined by the Type enum (Type.CEO, Type.OfficeManager etc).
Then the expected result, after the merge, is a new list that contains the following persons:
Employee(1, "Adam", "Smith", Type.CEO)
Employee(2, "Bob", "Jones", Type.OfficeManager)
Employee(3, "Carl", "Lewis", Type.SalesPerson)
Employee(5, "Yuri", "Gagarin", Type.Janitor)
What would be the "best" way to achieve this, in a general Java 8/9 way? Ie I don't want to write something that is specific to Person objects or the Type enum, and I don't want to write something that uses the equals method of the objects. Instead I would like to use a BiPredicate or something similar.
But also, I would like not having to perform any looping myself. Streams seems like a good choice, but I can't figure out how to achieve this. How can I write a BiPredicate where one value comes from one stream and the other value comes from another stream, without performing the looping myself?
The reason I want to use an BiPredicate (or similar) is that I want to be able to use this function with advanced logic, where it is not possible to simply extract some property of all the elements, and then group the values based on the uniqueness of this property.
Any suggestions?
/Jimi
Update: To clarify why I talk about a predicate, here is a more complex example:
Lets assume that we have the two collections of Employee objects as before. But this time the uniqueness logic can't be expressed using a mapping function to a specific property of the Employee object. Instead it uses some data in the EmployeeRegistry, like this: if two employees belong to the same tax bracket or if they are of the same "type" then they are considered equal. Because of this OR-logic it is not possible to reduce this to a unique key to use in grouping the data or something like that.
Update2: For the sake of simplicity, below is a less complex example, but that still is complex enough to not be a simple mapping to a field. It is a bit contrived, but that is for the sake of simplicity.
Lets assume that we have the two collections of Strings. And uniqueness is calculated like this:
If two strings are of equal length, they are considered equal
Otherwise, if two strings start with the same character, they are considered equal
Using the method Collectors.toMap, as suggested by Federico Peralta Schaffner, seems to work, although I'm not sure how I can write an hashCode() implementation that follows the standard and at the same time is efficient. The only functional implementation I can think of is one that returns a constant value (ie the same value regardless of the string).
Update 3: Considering that the OR-logic of my "equalness" algorithm breaks the equals contract, and makes it difficult (impossible?) to write an effective hashCode implementation, I am now back where I started. Ie needing something like a predicate of some sort. Here is an updated, "real world" example:
Lets assume that we have the two collections of Employee objects as before, and we want to merge these collections into one. But this time we want to avoid including people that don't get along. To determin if two people get along, we have an HumanRelationsDepartment object, with the method isOkToWorkWithEachother(Person, Person). When two people that don't get along is detected, then only of of them is to be added to the new collection. Which one can be determined by a mapping function, and the default logic could be that the first person is selected.
It is rather trivial to write old school code that solves this problem. What I'm looking for is a loop-free stream based solution. Does such a solution exist? Performance is not an issue.

For a simple merging of the two streams, you can use concat (just update reducer's logic):
Collection<Employee> merged = Stream.concat(list1.stream(), list2.stream())
.collect(Collectors.groupingBy(emp -> emp.getType(),
Collectors.reducing(null, (e1, e2) -> e1 ) ))
.values();
For element-wise merging of the 2 collections (assuming same length), you can use an index-based integer stream, to simulate a zipping of the two lists, then use a reducer that merges the two into one.
1 - Ensure lists are sorted by type, as that's what determines uniqueness:
List<Employee> list1Sorted = list1.stream()
.sorted(Comparator.comparing(Employee::getType))
.collect(Collectors.toList());
List<Employee> list2Sorted = list2.stream()
.sorted(Comparator.comparing(Employee::getType))
.collect(Collectors.toList());
2 - Declare a "reducer" that will merge 2 objects at the same index:
//This is returning an arbitrary value. You may want to add your own logic:
BiFunction<Employee, Employee, Employee> reducer = (e1, e2) -> e1;
3 - Now assume lists have the same length and simulate a zip operation:
List<Employee> mergedList = IntStream.range(0, list1.size())
.mapToObj(i -> new Employee[] {list1Sorted.get(i), list2Sorted.get(i)})
.map(e -> reducer.apply(e[0], e[1]))
.collect(Collectors.toList());
To simplify it: make a generic zip method:
public static <T> List<T> zipStreams(List<T> list1, List<T> list2, BiFunction<T, T, T> employeeMerger, Comparator<T> sortComparator) {
if(list1.size() != list2.size()) {
throw new IllegalArgumentException("Lists must be of the same length");
}
List<T> list1Sorted = sortComparator == null ? list1: list1.stream()
.sorted(sortComparator)
.collect(Collectors.toList()),
list2Sorted = sortComparator == null ? list2: list2.stream()
.sorted(sortComparator)
.collect(Collectors.toList());
return IntStream.range(0, list1Sorted.size())
.mapToObj(i -> Arrays.<T>asList(list1Sorted.get(i), list2Sorted.get(i)))
.map(list -> employeeMerger.apply(list.get(0), list.get(1)))
.collect(Collectors.toList());
}
Obviously, this is very specific to merging employee lists element-wise.
Now we can call that with:
zipStreams(list1, list2, (e1, e2) -> e1, Comparator.comparing(Employee::getType));

You can achieve what you want by means of Collectors.toMap:
Collection<Employee> merged = Stream.of(list1, list2)
.flatMap(Collection::stream)
.collect(Collectors.toMap(e -> calculateGroup(e), e -> e, (e1, e2) -> e1)))
.values();
So this creates a Map<SomeGroupType, Employee>, according to some calculateGroup method that receives an Employee instance and returns something that represents the group which the Employee belongs to. This could be some property of the Employee, i.e. type, or something more complicated that could get data from somewhere else to determine the group, i.e. tax bracket, as per the employee's annual income. This is for the key of the map, which will determine the uniqueness according to your specific needs. The only requirement of this approach is that whatever class you use for the keys, it must implement equals and hashCode consistently.
The values of the map will be just the Employee instances of the concatenated streams. For the merge function (Collectors.toMap 3rd argument), I've used (e1, e2) -> e1, meaning that we'll keep the values already present in the map when there are equal keys. If you want to overwrite values instead, change it to (e1, e2) -> e2.

// Concatenate the streams
Stream.concat(list1.stream(), list2.stream())
.collect(
// Collect similar employees together
groupingBy(
// Where "similar" is determined by a function, e.g. Employee::getType
keyFn,
// Take the first employee found
(a, b) -> a)
// Discard the keys.
.values();

Map them to a map with the unique value as the key, and then map the entries to a list.

Creating a map of maps from a map Java 8

I have a Map<String, List<SomeClassA>> that I'm trying to convert into a Map<String, Map<String, List<SomeWrapperOfClassA>>> and I'm just having so much trouble wrapping my head around how to do this.
Really, all the information needed to create the map should be in the objects of type SomeClassA - say
SomeClassA:
String attributeA;
String attributeB;
SomeClassB someOtherInfo;
SomeClassB:
String attribute C;
And I want to say it's a map based on this:
Map<attributeA's values,Map<attribute C vals, List SomeWrapperOfClassA>>
where the list is only of SomeWrapperClassA that has those values of attributeA and attributeB. I was thinking it might have to do with groupingBy, but I'm not too familiar with how to do it in such a way that its nested like this.
(or for the sake of simplicity, any help just getting the original list of SomeClassA into a Map<String, Map<String, List<SomeClassA>>> would already be a huge help.
I haven't quite gotten the hang of Java 8 and the more complex streaming concepts yet, so some help would be greatly appreciated. I'm only familiar with the basics.

I am not sure what you meant by:
Map<String, Map<String, List<SomeClassA>>>, what is the key here to group by? in general if you want to group List<SomeClassA> by say attributeA from SomeClassA you can do this:
List<SomeClassA>.stream().collect(Collectors.groupingBy(someClsA-> someClsA.getAttributeA()));

If I understand your questions correctly I would express it as follows:
Given:
class ClassA {
public String getA();
public ClassB getB();
}
class ClassB {
public String getC();
}
Map<String, List<ClassA>> input;
How would I create a Map<String, Map<String, List<ClassB>> where the key of the inner map is the result of getA?
If that's correct then you aren't looking to change the keys of the outer map at all. That makes it a good candidate for Map.replaceAll. For clarity I've split the collector into a separate method so it's clear what's happening:
input.replaceAll(listA.stream().collect(mapCollector()));
private Collector<A, ?, Map<String, List<B>> mapCollector() {
return Collectors.groupingBy(ClassA::getA,
Collectors.mapping(ClassA::getB,
Collectors.toList()));
}
Explaining the method it says group As by getA as key, then collect those As by mapping using getB then putting in a list.
If you particularly want a new map (rather than changing the original one) then:
Map<String, Map<String, List<ClassB>>> output = input.entrySet().stream()
.collect(Collectors.toMap(Map.Entry::getKey, e -> e.getValue().stream().collect(mapCollector()));

It’s not entirely clear what you want to achieve. Apparently, the keys of your source map are irrelevant and you only want to process the SomeClassA instances contained in the value lists. So, the first step is to stream over the map’s values, i.e. sourceMap.values().stream(), then flatten this Stream<List<SomeClassA>> to a Stream<SomeClassA> via flatMap(List::stream).
Grouping these SomeClassA instances by one of their properties works indeed straight-forwardly via groupingBy, which needs another nested groupingBy to group each group further by “Attribute C”:
Map<String, Map<String, List<SomeClassA>>> resultMap
= sourceMap.values().stream().flatMap(List::stream)
.collect(Collectors.groupingBy(SomeClassA::getAttributeA,
Collectors.groupingBy(a->a.getSomeOtherInfo().getAttributeC())));
To convert the SomeClassA instances to SomeWrapperClassA instances, we need another nested collector for the innermost groupingBy; the mapping collector allows to map the values before transferring to another collector, which has to be the toList() collector, which was formerly implicit.
Now, it might be better to start using import static java.util.stream.Collectors.*;
Map<String, Map<String, List<SomeWrapperClassA>>> resultMap
= sourceMap.values().stream().flatMap(List::stream)
.collect(groupingBy(SomeClassA::getAttributeA, groupingBy(
a -> a.getSomeOtherInfo().getAttributeC(),
mapping(a->new SomeWrapperClassA(a.getAttributeA(),a.getAttributeB()), toList()))));

How to create a map with Java stream API using a value outside the stream?

I want to init a Map<String, BigDecimal> and want to always put the same BigDecimal value from outside of the stream.
BigDecimal samePrice;
Set<String> set;
set.stream().collect(Collectors.toMap(Function.identity(), samePrice));
However Java complains as follows:
The method toMap(Function, Function) in the type Collectors is not applicable for the arguments
(Function, BigDecimal)
Why can't I use the BigDecimal from outside? If I write:
set.stream().collect(Collectors.toMap(Function.identity(), new BigDecimal()));
it would work, but that's of course not what I want.

The second argument (like the first one) of toMap(keyMapper, valueMapper) is a function that takes the stream element and returns the value of the map.
In this case, you want to ignore it so you can have:
set.stream().collect(Collectors.toMap(Function.identity(), e -> samePrice));
Note that your second attempt wouldn't work for the same reason.

Collectors#toMap expects two Functions
set.stream().collect(Collectors.toMap(Function.identity(), x -> samePrice));
You can find nearly the same example within the JavaDoc
Map<Student, Double> studentToGPA
students.stream().collect(toMap(Functions.identity(),
student -> computeGPA(student)));

As already said in the other answers, you need to specify a function which maps each element to the fixed value like element -> samePrice.
As an addition, if you want to specifically fill a ConcurrentHashMap, there is a neat feature that doesn’t need a stream operation at all:
ConcurrentHashMap<String,BigDecimal> map = new ConcurrentHashMap<>();
map.keySet(samePrice).addAll(set);
Unfortunately, there is no such operation for arbitrary Maps.

java 8 change list to map using instance of list

I'm try to convert a list to a map using the Collectors.toMap call. The list consists of ActivityReconcile objects. I want to pass an instance for every entry in the list into the toMap call.
The code is below and where I need the instances is denoted by ??.
final List<ActivityReconcile> activePostedList = loader.loadActivePosted(accessToken);
Map<AccountTransactionKey, ActivityReconcile> postedActiveMap =
activePostedList.stream().collect(
Collectors.toMap(
AccountTransactionKey.createNewAccountTransactionKeyFromActivityReconcileRecord(??),??));

If I understood you correctly, you will need something like
Map<AccountTransactionKey, ActivityReconcile> result = choices
.stream()
.collect(Collectors.toMap(
AccountTransactionKey::generate,
Function.identity()));
And the method (in AccountTransactionKey class) will look like
public static AccountTransactionKey generate(ActivityReconcile reconcile) {...}
I've replaced createNewAccountTransactionKeyFromActivityReconcileRec by generate for making the answer more readable and understandable.

To "fix" your code with the least changes, add a lambda parameter:
activePostedList.stream().collect(Collectors.toMap(
ar -> AccountTransactionKey.createNewAccountTransactionKeyFromActivityReconcileRecord(ar)),
o -> o));
or use a method reference:
activePostedList.stream().collect(Collectors.toMap(
AccountTransactionKey::createNewAccountTransactionKeyFromActivityReconcileRecord, o -> o));
btw, I can't recall ever seeing a method name as long as createNewAccountTransactionKeyFromActivityReconcileRecord - for readability, consider reducing it to just create(), since the return type and parameter type are enough to distinguish it from other factory methods you may have.