We Keep Coding

Have Java Streams GroupingBy result Map include a key for each value of an enum, even if value is an empty List

This question is about Java Streams' groupingBy capability.
Suppose I have a class, WorldCup:
public class WorldCup {
int year;
Country champion;
// all-arg constructor, getter/setters, etc
}
and an enum, Country:
public enum Country {
Brazil, France, USA
}
and the following code snippet:
WorldCup wc94 = new WorldCup(1994, Country.Brazil);
WorldCup wc98 = new WorldCup(1998, Country.France);
List<WorldCup> wcList = new ArrayList<WorldCup>();
wcList.add(wc94);
wcList.add(wc98);
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(Collectors.groupingBy(WorldCup::getCountry, Collectors.mapping(WorldCup::getYear));
After running this code, championsMap will contain:
Brazil: [1994]
France: [1998]
Is there a succinct way to have this list include an entry for all of the values of the enum? What I'm looking for is:
Brazil: [1994]
France: [1998]
USA: []

There are several approaches you can take.
The map which would be used for accumulating the stream data can be prepopulated with entries corresponding to every enum-member. To access all existing enum-members you can use values() method or EnumSet.allOf().
It can be achieved using three-args version of collect() or through a custom collector created via Collector.of().
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(
() -> EnumSet.allOf(Country.class).stream() // supplier
.collect(Collectors.toMap(
Function.identity(),
c -> new ArrayList<>()
)),
(Map<Country, List<Integer>> map, WorldCup next) -> // accumulator
map.get(next.getCountry()).add(next.getYear()),
(left, right) -> // combiner
right.forEach((k, v) -> left.get(k).addAll(v))
);
Another option is to add missing entries to the map after reduction of the stream has been finished.
For that we can use built-in collector collectingAndThen().
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(Collectors.collectingAndThen(
Collectors.groupingBy(WorldCup::getCountry,
Collectors.mapping(WorldCup::getYear,
Collectors.toList())),
map -> {
EnumSet.allOf(Country.class)
.forEach(country -> map.computeIfAbsent(country, k -> new ArrayList<>())); // if you're not going to mutate these lists - use Collections.emptyList()
return map;
}
));

How to group objects from a list which can belong to two or more groups?

I have a list of Items where each Item can belong to one or more category. For a limited set of categories(string) I want to create a map with category as key and list of Items as value.
Assume my Item class is defined as shown below:
public static class Item{
long id;
List<String> belongsToCategories;
public List<String> getBelongsToCategories() {
return belongsToCategories;
}
public void setBelongsToCategories(List<String> belongsToCategories) {
this.belongsToCategories = belongsToCategories;
}
public Item(long id,List<String> belongsToCategories) {
this.id = id;
this.belongsToCategories = belongsToCategories;
}
#Override
public String toString() {
return "Item{" + "id=" + id + '}';
}
}
and a list of Items:
public static void main(String[] args) {
List<Item> myItemList = new ArrayList<>();
myItemList.add(new Item(1,Arrays.asList("A","B")));
myItemList.add(new Item(2,Arrays.asList("A","C")));
myItemList.add(new Item(3,Arrays.asList("B","C")));
myItemList.add(new Item(4,Arrays.asList("D")));
myItemList.add(new Item(5,Arrays.asList("D","E")));
myItemList.add(new Item(6,Arrays.asList("A","F")));
Map<String,List<Item>> myMap= new HashMap<>();
How can i fill myMap from myList?
I thought Stream API could help, but I don't know which classifier to put in the groupingBy method when an Item can belong to one or more categories
myItemList.stream().collect(Collectors.groupingBy(classifier));
This
myItemList.stream().collect(Collectors.groupingBy(Item::getBelongsToCategories));
produces
[D, E]=[Item{id=5}]
[B, C]=[Item{id=3}]
[A, B]=[Item{id=1}]
[D]=[Item{id=4}]
[A, C]=[Item{id=2}]
[A, F]=[Item{id=6}]
expected is tough something like:
A=[Item{id=1}, Item{id=2}, Item{id=6}]
B=[Item{id=1}, Item{id=3}]
C=[Item{id=2}, Item{id=3}]
D=[Item{id=4}, Item{id=5}]
E=[Item{id=5}]
F=[Item{id=6}]

You can use groupByEach from Eclipse Collections.
Multimap<String, Item> itemsByCategory =
ListIterate.groupByEach(myItemList, Item::getBelongsToCategories);
System.out.println(itemsByCategory);
Output:
{D=[Item{id=4}, Item{id=5}],
E=[Item{id=5}],
F=[Item{id=6}],
A=[Item{id=1}, Item{id=2}, Item{id=6}],
B=[Item{id=1}, Item{id=3}],
C=[Item{id=2}, Item{id=3}]}
You can also use the Collectors2 utility class and groupByEach Collector with a Java Stream.
Multimap<String, Item> itemsByCategory = myItemList.stream().collect(
Collectors2.groupByEach(
Item::getBelongsToCategories,
Multimaps.mutable.list::empty));
Note: I am a committer for Eclipse Collections.

You can use flatMap to map to SimpleEntry and then groupingBy as :
return items.stream()
.flatMap(p -> p.getBelongsToCategories()
.stream()
.map(l -> new AbstractMap.SimpleEntry<>(l, p)))
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue,
Collectors.toList())));

Here is the solution by ListMultimap, either from Google Guava or my library abacus-common
ListMultimap<String, Item> result = Multimaps.newListMultimap(new HashMap<>(), () -> new ArrayList<>()); // by Google Guava.
// Or result = N.newListMultimap(); // By Abacus-Util
items.forEach(item -> item.getBelongsToCategories().forEach(c -> result.put(c, item)));
I think it will be a little more efficient and concise without creating the unnecessary temporary Entry Objects by flatMap.

Adding two lists of own type

I have a simple User class with a String and an int property.
I would like to add two Lists of users this way:
if the String equals then the numbers should be added and that would be its new value.
The new list should include all users with proper values.
Like this:
List1: { [a:2], [b:3] }
List2: { [b:4], [c:5] }
ResultList: {[a:2], [b:7], [c:5]}
User definition:
public class User {
private String name;
private int comments;
}
My method:
public List<User> addTwoList(List<User> first, List<User> sec) {
List<User> result = new ArrayList<>();
for (int i=0; i<first.size(); i++) {
Boolean bsin = false;
Boolean isin = false;
for (int j=0; j<sec.size(); j++) {
isin = false;
if (first.get(i).getName().equals(sec.get(j).getName())) {
int value= first.get(i).getComments() + sec.get(j).getComments();
result.add(new User(first.get(i).getName(), value));
isin = true;
bsin = true;
}
if (!isin) {result.add(sec.get(j));}
}
if (!bsin) {result.add(first.get(i));}
}
return result;
}
But it adds a whole lot of things to the list.

This is better done via the toMap collector:
Collection<User> result = Stream
.concat(first.stream(), second.stream())
.collect(Collectors.toMap(
User::getName,
u -> new User(u.getName(), u.getComments()),
(l, r) -> {
l.setComments(l.getComments() + r.getComments());
return l;
}))
.values();
First, concatenate both the lists into a single Stream<User> via Stream.concat.
Second, we use the toMap collector to merge users that happen to have the same Name and get back a result of Collection<User>.
if you strictly want a List<User> then pass the result into the ArrayList constructor i.e. List<User> resultSet = new ArrayList<>(result);
Kudos to #davidxxx, you could collect to a list directly from the pipeline and avoid an intermediate variable creation with:
List<User> result = Stream
.concat(first.stream(), second.stream())
.collect(Collectors.toMap(
User::getName,
u -> new User(u.getName(), u.getComments()),
(l, r) -> {
l.setComments(l.getComments() + r.getComments());
return l;
}))
.values()
.stream()
.collect(Collectors.toList());

You have to use an intermediate map to merge users from both lists by summing their ages.
One way is with streams, as shown in Aomine's answer. Here's another way, without streams:
Map<String, Integer> map = new LinkedHashMap<>();
list1.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
list2.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
Now, you can create a list of users, as follows:
List<User> result = new ArrayList<>();
map.forEach((name, comments) -> result.add(new User(name, comments)));
This assumes User has a constructor that accepts name and comments.
EDIT: As suggested by #davidxxx, we could improve the code by factoring out the first part:
BiConsumer<List<User>, Map<String, Integer>> action = (list, map) ->
list.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
Map<String, Integer> map = new LinkedHashMap<>();
action.accept(list1, map);
action.accept(list2, map);
This refactor would avoid DRY.

There is a pretty direct way using Collectors.groupingBy and Collectors.reducing which doesnt require setters, which is the biggest advantage since you can keep the User immutable:
Collection<Optional<User>> d = Stream
.of(first, second) // start with Stream<List<User>>
.flatMap(List::stream) // flatting to the Stream<User>
.collect(Collectors.groupingBy( // Collecting to Map<String, List<User>>
User::getName, // by name (the key)
// and reducing the list into a single User
Collectors.reducing((l, r) -> new User(l.getName(), l.getComments() + r.getComments()))))
.values(); // return values from Map<String, List<User>>
Unfortunately, the result is Collection<Optional<User>> since the reducing pipeline returns Optional since the result might not be present after all. You can stream the values and use the map() to get rid of the Optional or use Collectors.collectAndThen*:
Collection<User> d = Stream
.of(first, second) // start with Stream<List<User>>
.flatMap(List::stream) // flatting to the Stream<User>
.collect(Collectors.groupingBy( // Collecting to Map<String, List<User>>
User::getName, // by name (the key)
Collectors.collectingAndThen( // reduce the list into a single User
Collectors.reducing((l, r) -> new User(l.getName(), l.getComments() + r.getComments())),
Optional::get))) // and extract from the Optional
.values();
* Thanks to #Aomine

As alternative fairly straight and efficient :
stream the elements
collect them into a Map<String, Integer> to associate each name to the sum of comments (int)
stream the entries of the collected map to create the List of User.
Alternatively for the third step you could apply a finishing transformation to the Map collector with collectingAndThen(groupingBy()..., m -> ...
but I don't find it always very readable and here we could do without.
It would give :
List<User> users =
Stream.concat(first.stream(), second.stream())
.collect(groupingBy(User::getName, summingInt(User::getComments)))
.entrySet()
.stream()
.map(e -> new User(e.getKey(), e.getValue()))
.collect(toList());

Multiple aggregate functions in Java 8 Stream API

I have a class defined like
public class TimePeriodCalc {
private double occupancy;
private double efficiency;
private String atDate;
}
I would like to perform the following SQL statement using Java 8 Stream API.
SELECT atDate, AVG(occupancy), AVG(efficiency)
FROM TimePeriodCalc
GROUP BY atDate
I tried :
Collection<TimePeriodCalc> collector = result.stream().collect(groupingBy(p -> p.getAtDate(), ....
What can be put into the code to select multiple attributes ? I'm thinking of using multiple Collectors but really don't know how to do so.

To do it without a custom Collector (not streaming again on the result), you could do it like this. It's a bit dirty, since it is first collecting to Map<String, List<TimePeriodCalc>> and then streaming that list and get the average double.
Since you need two averages, they are collected to a Holder or a Pair, in this case I'm using AbstractMap.SimpleEntry
Map<String, SimpleEntry<Double, Double>> map = Stream.of(new TimePeriodCalc(12d, 10d, "A"), new TimePeriodCalc(2d, 16d, "A"))
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(Collectors.toList(), list -> {
double occupancy = list.stream().collect(
Collectors.averagingDouble(TimePeriodCalc::getOccupancy));
double efficiency = list.stream().collect(
Collectors.averagingDouble(TimePeriodCalc::getEfficiency));
return new AbstractMap.SimpleEntry<>(occupancy, efficiency);
})));
System.out.println(map);

Here's a way with a custom collector. It only needs one pass, but it's not very easy, especially because of generics...
If you have this method:
#SuppressWarnings("unchecked")
#SafeVarargs
static <T, A, C extends Collector<T, A, Double>> Collector<T, ?, List<Double>>
averagingManyDoubles(ToDoubleFunction<? super T>... extractors) {
List<C> collectors = Arrays.stream(extractors)
.map(extractor -> (C) Collectors.averagingDouble(extractor))
.collect(Collectors.toList());
class Acc {
List<A> averages = collectors.stream()
.map(c -> c.supplier().get())
.collect(Collectors.toList());
void add(T elem) {
IntStream.range(0, extractors.length).forEach(i ->
collectors.get(i).accumulator().accept(averages.get(i), elem));
}
Acc merge(Acc another) {
IntStream.range(0, extractors.length).forEach(i ->
averages.set(i, collectors.get(i).combiner()
.apply(averages.get(i), another.averages.get(i))));
return this;
}
List<Double> finish() {
return IntStream.range(0, extractors.length)
.mapToObj(i -> collectors.get(i).finisher().apply(averages.get(i)))
.collect(Collectors.toList());
}
}
return Collector.of(Acc::new, Acc::add, Acc::merge, Acc::finish);
}
This receives an array of functions that will extract double values from each element of the stream. These extractors are converted to Collectors.averagingDouble collectors and then the local Acc class is created with the mutable structures that are used to accumulate the averages for each collector. Then, the accumulator function forwards to each accumulator, and so with the combiner and finisher functions.
Usage is as follows:
Map<String, List<Double>> averages = list.stream()
.collect(Collectors.groupingBy(
TimePeriodCalc::getAtDate,
averagingManyDoubles(
TimePeriodCalc::getOccupancy,
TimePeriodCalc::getEfficiency)));

Assuming that your TimePeriodCalc class has all the necessary getters, this should get you the list you want:
List<TimePeriodCalc> result = new ArrayList<>(
list.stream()
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(Collectors.toList(), TimePeriodCalc::avgTimePeriodCalc)))
.values()
);
Where TimePeriodCalc.avgTimePeriodCalc is this method in the TimePeriodCalc class:
public static TimePeriodCalc avgTimePeriodCalc(List<TimePeriodCalc> list){
return new TimePeriodCalc(
list.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getOccupancy)),
list.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getEfficiency)),
list.get(0).getAtDate()
);
}
The above can be combined into this monstrosity:
List<TimePeriodCalc> result = new ArrayList<>(
list.stream()
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(
Collectors.toList(), a -> {
return new TimePeriodCalc(
a.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getOccupancy)),
a.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getEfficiency)),
a.get(0).getAtDate()
);
}
)))
.values());
With input:
List<TimePeriodCalc> list = new ArrayList<>();
list.add(new TimePeriodCalc(10,10,"a"));
list.add(new TimePeriodCalc(10,10,"b"));
list.add(new TimePeriodCalc(10,10,"c"));
list.add(new TimePeriodCalc(5,5,"a"));
list.add(new TimePeriodCalc(0,0,"b"));
This would give:
TimePeriodCalc [occupancy=7.5, efficiency=7.5, atDate=a]
TimePeriodCalc [occupancy=5.0, efficiency=5.0, atDate=b]
TimePeriodCalc [occupancy=10.0, efficiency=10.0, atDate=c]

You can chain multiple attributes like this:
Collection<TimePeriodCalc> collector = result.stream().collect(Collectors.groupingBy(p -> p.getAtDate(), Collectors.averagingInt(p -> p.getOccupancy())));
If you want more, you get the idea.

How to map to multiple elements with Java 8 streams?

I have a class like this:
class MultiDataPoint {
private DateTime timestamp;
private Map<String, Number> keyToData;
}
and i want to produce , for each MultiDataPoint
class DataSet {
public String key;
List<DataPoint> dataPoints;
}
class DataPoint{
DateTime timeStamp;
Number data;
}
of course a 'key' can be the same across multiple MultiDataPoints.
So given a List<MultiDataPoint>, how do I use Java 8 streams to convert to List<DataSet>?
This is how I am currently doing the conversion without streams:
Collection<DataSet> convertMultiDataPointToDataSet(List<MultiDataPoint> multiDataPoints)
{
Map<String, DataSet> setMap = new HashMap<>();
multiDataPoints.forEach(pt -> {
Map<String, Number> data = pt.getData();
data.entrySet().forEach(e -> {
String seriesKey = e.getKey();
DataSet dataSet = setMap.get(seriesKey);
if (dataSet == null)
{
dataSet = new DataSet(seriesKey);
setMap.put(seriesKey, dataSet);
}
dataSet.dataPoints.add(new DataPoint(pt.getTimestamp(), e.getValue()));
});
});
return setMap.values();
}

It's an interesting question, because it shows that there are a lot of different approaches to achieve the same result. Below I show three different implementations.
Default methods in Collection Framework: Java 8 added some methods to the collections classes, that are not directly related to the Stream API. Using these methods, you can significantly simplify the implementation of the non-stream implementation:
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
Map<String, DataSet> result = new HashMap<>();
multiDataPoints.forEach(pt ->
pt.keyToData.forEach((key, value) ->
result.computeIfAbsent(
key, k -> new DataSet(k, new ArrayList<>()))
.dataPoints.add(new DataPoint(pt.timestamp, value))));
return result.values();
}
Stream API with flatten and intermediate data structure: The following implementation is almost identical to the solution provided by Stuart Marks. In contrast to his solution, the following implementation uses an anonymous inner class as intermediate data structure.
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.flatMap(mdp -> mdp.keyToData.entrySet().stream().map(e ->
new Object() {
String key = e.getKey();
DataPoint dataPoint = new DataPoint(mdp.timestamp, e.getValue());
}))
.collect(
collectingAndThen(
groupingBy(t -> t.key, mapping(t -> t.dataPoint, toList())),
m -> m.entrySet().stream().map(e -> new DataSet(e.getKey(), e.getValue())).collect(toList())));
}
Stream API with map merging: Instead of flattening the original data structures, you can also create a Map for each MultiDataPoint, and then merge all maps into a single map with a reduce operation. The code is a bit simpler than the above solution:
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.map(mdp -> mdp.keyToData.entrySet().stream()
.collect(toMap(e -> e.getKey(), e -> asList(new DataPoint(mdp.timestamp, e.getValue())))))
.reduce(new HashMap<>(), mapMerger())
.entrySet().stream()
.map(e -> new DataSet(e.getKey(), e.getValue()))
.collect(toList());
}
You can find an implementation of the map merger within the Collectors class. Unfortunately, it is a bit tricky to access it from the outside. Following is an alternative implementation of the map merger:
<K, V> BinaryOperator<Map<K, List<V>>> mapMerger() {
return (lhs, rhs) -> {
Map<K, List<V>> result = new HashMap<>();
lhs.forEach((key, value) -> result.computeIfAbsent(key, k -> new ArrayList<>()).addAll(value));
rhs.forEach((key, value) -> result.computeIfAbsent(key, k -> new ArrayList<>()).addAll(value));
return result;
};
}

To do this, I had to come up with an intermediate data structure:
class KeyDataPoint {
String key;
DateTime timestamp;
Number data;
// obvious constructor and getters
}
With this in place, the approach is to "flatten" each MultiDataPoint into a list of (timestamp, key, data) triples and stream together all such triples from the list of MultiDataPoint.
Then, we apply a groupingBy operation on the string key in order to gather the data for each key together. Note that a simple groupingBy would result in a map from each string key to a list of the corresponding KeyDataPoint triples. We don't want the triples; we want DataPoint instances, which are (timestamp, data) pairs. To do this we apply a "downstream" collector of the groupingBy which is a mapping operation that constructs a new DataPoint by getting the right values from the KeyDataPoint triple. The downstream collector of the mapping operation is simply toList which collects the DataPoint objects of the same group into a list.
Now we have a Map<String, List<DataPoint>> and we want to convert it to a collection of DataSet objects. We simply stream out the map entries and construct DataSet objects, collect them into a list, and return it.
The code ends up looking like this:
Collection<DataSet> convertMultiDataPointToDataSet(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.flatMap(mdp -> mdp.getData().entrySet().stream()
.map(e -> new KeyDataPoint(e.getKey(), mdp.getTimestamp(), e.getValue())))
.collect(groupingBy(KeyDataPoint::getKey,
mapping(kdp -> new DataPoint(kdp.getTimestamp(), kdp.getData()), toList())))
.entrySet().stream()
.map(e -> new DataSet(e.getKey(), e.getValue()))
.collect(toList());
}
I took some liberties with constructors and getters, but I think they should be obvious.