Related
This question is about Java Streams' groupingBy capability.
Suppose I have a class, WorldCup:
public class WorldCup {
int year;
Country champion;
// all-arg constructor, getter/setters, etc
}
and an enum, Country:
public enum Country {
Brazil, France, USA
}
and the following code snippet:
WorldCup wc94 = new WorldCup(1994, Country.Brazil);
WorldCup wc98 = new WorldCup(1998, Country.France);
List<WorldCup> wcList = new ArrayList<WorldCup>();
wcList.add(wc94);
wcList.add(wc98);
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(Collectors.groupingBy(WorldCup::getCountry, Collectors.mapping(WorldCup::getYear));
After running this code, championsMap will contain:
Brazil: [1994]
France: [1998]
Is there a succinct way to have this list include an entry for all of the values of the enum? What I'm looking for is:
Brazil: [1994]
France: [1998]
USA: []
There are several approaches you can take.
The map which would be used for accumulating the stream data can be prepopulated with entries corresponding to every enum-member. To access all existing enum-members you can use values() method or EnumSet.allOf().
It can be achieved using three-args version of collect() or through a custom collector created via Collector.of().
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(
() -> EnumSet.allOf(Country.class).stream() // supplier
.collect(Collectors.toMap(
Function.identity(),
c -> new ArrayList<>()
)),
(Map<Country, List<Integer>> map, WorldCup next) -> // accumulator
map.get(next.getCountry()).add(next.getYear()),
(left, right) -> // combiner
right.forEach((k, v) -> left.get(k).addAll(v))
);
Another option is to add missing entries to the map after reduction of the stream has been finished.
For that we can use built-in collector collectingAndThen().
Map<Country, List<Integer>> championsMap = wcList.stream()
.collect(Collectors.collectingAndThen(
Collectors.groupingBy(WorldCup::getCountry,
Collectors.mapping(WorldCup::getYear,
Collectors.toList())),
map -> {
EnumSet.allOf(Country.class)
.forEach(country -> map.computeIfAbsent(country, k -> new ArrayList<>())); // if you're not going to mutate these lists - use Collections.emptyList()
return map;
}
));
My current approach exploiting Streams API in conjunction with forEach loop:
public Map<String, Client> clientsWithMostPurchasesInEachCategory(Map<Client, Map<Product,Integer>> shopping) {
Map<String, Client> result = new HashMap<>();
Map<Client, Map<String, BigDecimal>> temp =
shopping.entrySet()
.stream()
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.flatMapping(e -> e.getValue().entrySet().stream(),
Collectors.groupingBy(e -> e.getKey().getCategory(),
Collectors.mapping(ee -> ee.getKey().getPrice().multiply(BigDecimal.valueOf(ee.getValue())),
Collectors.reducing(BigDecimal.ZERO, BigDecimal::add))))));
/*curious, how could I refactor that piece of code, so the method uses only one stream chain? */
temp.forEach((client, value)
-> value.forEach((category, value1)
-> {
if (!result.containsKey(category) ||
temp.get(result.get(category)).get(category).compareTo(value1) < 0)
result.put(category, client);
}));
return result;
}
As the method's name sugggests, I want to find a map Map <String, Client>, containing Client with most purchases (as value) in specified category (as key) in each product's category
shopping is basically a map: Map<Client, Map<Product,Integer>>,
The outer Key represents the Client
The inner Key represents the Product. Product class members are name, category, price (BigDecimal)
the inner maps value (Integer) reprents the number of specified product which belong to a specific client
Not sure, if that's even possible? Collectors.collectingAndThen maybe could be useful?
You can use StreamEx library, and do smth like this
public static Map<String, Client> clientsWithMostPurchasesInEachCategory(Map<Client, Map<Product, Integer>> shopping) {
return EntryStream.of(shopping)
.flatMapKeyValue(((client, productQuantityMap) ->
EntryStream.of(productQuantityMap)
.mapToValue((p, q) -> p.getPrice().multiply(BigDecimal.valueOf(q)))
.mapKeys(Product::getCategory)
.map(e -> new ClientCategorySpend(client, e.getKey(), e.getValue())))
)
.groupingBy(
ClientCategorySpend::getCategory,
Collectors.collectingAndThen(Collectors.maxBy(Comparator.comparing(ClientCategorySpend::getSpend)),
t -> t.get().getClient())
);
}
You were pretty much doomed the moment you grouped by client. The
top level Collectors.groupingBy must use the category as the grouping by key.
To do that you would flatMap before collecting, so you get a flat stream of
client + category + spend elements.
Here's one way to do it. I'll first define a POJO for the elements of the flattened stream:
static class ClientCategorySpend
{
private final Client client;
private final String category;
private final BigDecimal spend;
public ClientCategorySpend(Client client, String category, BigDecimal spend)
{
this.client = client;
this.category = category;
this.spend = spend;
}
public String getCategory()
{
return category;
}
public Client getClient()
{
return client;
}
public BigDecimal getSpend()
{
return spend;
}
}
And now the function:
public static Map<String, Client> clientsWithMostPurchasesInEachCategory(Map<Client, Map<Product, Integer>> shopping)
{
// <1>
Collector<? super ClientCategorySpend, ?, BigDecimal> sumOfSpendByClient = Collectors.mapping(ClientCategorySpend::getSpend,
Collectors.reducing(BigDecimal.ZERO, BigDecimal::add));
// <2>
Collector<? super ClientCategorySpend, ?, Map<Client, BigDecimal>> clientSpendByCategory = Collectors.groupingBy(
ClientCategorySpend::getClient,
sumOfSpendByClient
);
// <3>
Collector<? super ClientCategorySpend, ?, Client> maxSpendingClientByCategory = Collectors.collectingAndThen(
clientSpendByCategory,
map -> map.entrySet().stream()
.max(Comparator.comparing(Map.Entry::getValue))
.map(Map.Entry::getKey).get()
);
return shopping.entrySet().stream()
// <4>
.flatMap(
entry -> entry.getValue().entrySet().stream().map(
entry2 -> new ClientCategorySpend(entry.getKey(),
entry2.getKey().category,
entry2.getKey().price.multiply(BigDecimal.valueOf(entry2.getValue())))
)
).collect(Collectors.groupingBy(ClientCategorySpend::getCategory, maxSpendingClientByCategory));
}
Once I have a stream of ClientCategorySpend (4), I group it by category. I use
the clientSpendByCategory collector (2) to create a map between the client and the total spend in the category. This in turn depends on sumToSpendByClient (1) which is basically a reducer that sums up the spends. You then get to use collectingAndThen as you suggested,
reducing each Map<Client, BigDecimal> to a single client using max.
This should do ;)
public Map<String, Client> clientsWithMostPurchasesInEachCategory(Map<Client, Map<Product, Integer>> shopping) {
return shopping
.entrySet()
.stream()
.map(entry -> Pair.of(
entry.getKey(),
entry.getValue()
.entrySet()
.stream()
.map(e -> Pair.of(
e.getKey().getCategory(),
e.getKey().getPrice().multiply(
BigDecimal.valueOf(e.getValue()))))
.collect(Collectors.toMap(
Pair::getKey,
Pair::getValue,
BigDecimal::add))))
// Here we have: Stream<Pair<Client, Map<String, BigDecimal>>>
// e.g.: per each Client we have a map { category -> purchase value }
.flatMap(item -> item.getValue()
.entrySet()
.stream()
.map(e -> Pair.of(
e.getKey(), Pair.of(item.getKey(), e.getValue()))))
// Here: Stream<Pair<String, Pair<Client, BigDecimal>>>
// e.g.: entries stream { category, { client, purchase value } }
// where there are category duplicates, so we must select those
// with highest purchase value for each category.
.collect(Collectors.toMap(
Pair::getKey,
Pair::getValue,
(o1, o2) -> o2.getValue().compareTo(o1.getValue()) > 0 ?
o2 : o1))
// Now we have: Map<String, Pair<Client, BigDecimal>>,
// e.g.: { category -> { client, purchase value } }
// so just get rid of unnecessary purchase value...
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey,
e -> e.getValue().getKey()));
}
Pair is org.apache.commons.lang3.tuple.Pair. If you do not want to use Appache Commons library you may use java.util.AbstractMap.SimpleEntry instead.
I have a simple User class with a String and an int property.
I would like to add two Lists of users this way:
if the String equals then the numbers should be added and that would be its new value.
The new list should include all users with proper values.
Like this:
List1: { [a:2], [b:3] }
List2: { [b:4], [c:5] }
ResultList: {[a:2], [b:7], [c:5]}
User definition:
public class User {
private String name;
private int comments;
}
My method:
public List<User> addTwoList(List<User> first, List<User> sec) {
List<User> result = new ArrayList<>();
for (int i=0; i<first.size(); i++) {
Boolean bsin = false;
Boolean isin = false;
for (int j=0; j<sec.size(); j++) {
isin = false;
if (first.get(i).getName().equals(sec.get(j).getName())) {
int value= first.get(i).getComments() + sec.get(j).getComments();
result.add(new User(first.get(i).getName(), value));
isin = true;
bsin = true;
}
if (!isin) {result.add(sec.get(j));}
}
if (!bsin) {result.add(first.get(i));}
}
return result;
}
But it adds a whole lot of things to the list.
This is better done via the toMap collector:
Collection<User> result = Stream
.concat(first.stream(), second.stream())
.collect(Collectors.toMap(
User::getName,
u -> new User(u.getName(), u.getComments()),
(l, r) -> {
l.setComments(l.getComments() + r.getComments());
return l;
}))
.values();
First, concatenate both the lists into a single Stream<User> via Stream.concat.
Second, we use the toMap collector to merge users that happen to have the same Name and get back a result of Collection<User>.
if you strictly want a List<User> then pass the result into the ArrayList constructor i.e. List<User> resultSet = new ArrayList<>(result);
Kudos to #davidxxx, you could collect to a list directly from the pipeline and avoid an intermediate variable creation with:
List<User> result = Stream
.concat(first.stream(), second.stream())
.collect(Collectors.toMap(
User::getName,
u -> new User(u.getName(), u.getComments()),
(l, r) -> {
l.setComments(l.getComments() + r.getComments());
return l;
}))
.values()
.stream()
.collect(Collectors.toList());
You have to use an intermediate map to merge users from both lists by summing their ages.
One way is with streams, as shown in Aomine's answer. Here's another way, without streams:
Map<String, Integer> map = new LinkedHashMap<>();
list1.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
list2.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
Now, you can create a list of users, as follows:
List<User> result = new ArrayList<>();
map.forEach((name, comments) -> result.add(new User(name, comments)));
This assumes User has a constructor that accepts name and comments.
EDIT: As suggested by #davidxxx, we could improve the code by factoring out the first part:
BiConsumer<List<User>, Map<String, Integer>> action = (list, map) ->
list.forEach(u -> map.merge(u.getName(), u.getComments(), Integer::sum));
Map<String, Integer> map = new LinkedHashMap<>();
action.accept(list1, map);
action.accept(list2, map);
This refactor would avoid DRY.
There is a pretty direct way using Collectors.groupingBy and Collectors.reducing which doesnt require setters, which is the biggest advantage since you can keep the User immutable:
Collection<Optional<User>> d = Stream
.of(first, second) // start with Stream<List<User>>
.flatMap(List::stream) // flatting to the Stream<User>
.collect(Collectors.groupingBy( // Collecting to Map<String, List<User>>
User::getName, // by name (the key)
// and reducing the list into a single User
Collectors.reducing((l, r) -> new User(l.getName(), l.getComments() + r.getComments()))))
.values(); // return values from Map<String, List<User>>
Unfortunately, the result is Collection<Optional<User>> since the reducing pipeline returns Optional since the result might not be present after all. You can stream the values and use the map() to get rid of the Optional or use Collectors.collectAndThen*:
Collection<User> d = Stream
.of(first, second) // start with Stream<List<User>>
.flatMap(List::stream) // flatting to the Stream<User>
.collect(Collectors.groupingBy( // Collecting to Map<String, List<User>>
User::getName, // by name (the key)
Collectors.collectingAndThen( // reduce the list into a single User
Collectors.reducing((l, r) -> new User(l.getName(), l.getComments() + r.getComments())),
Optional::get))) // and extract from the Optional
.values();
* Thanks to #Aomine
As alternative fairly straight and efficient :
stream the elements
collect them into a Map<String, Integer> to associate each name to the sum of comments (int)
stream the entries of the collected map to create the List of User.
Alternatively for the third step you could apply a finishing transformation to the Map collector with collectingAndThen(groupingBy()..., m -> ...
but I don't find it always very readable and here we could do without.
It would give :
List<User> users =
Stream.concat(first.stream(), second.stream())
.collect(groupingBy(User::getName, summingInt(User::getComments)))
.entrySet()
.stream()
.map(e -> new User(e.getKey(), e.getValue()))
.collect(toList());
I have a class defined like
public class TimePeriodCalc {
private double occupancy;
private double efficiency;
private String atDate;
}
I would like to perform the following SQL statement using Java 8 Stream API.
SELECT atDate, AVG(occupancy), AVG(efficiency)
FROM TimePeriodCalc
GROUP BY atDate
I tried :
Collection<TimePeriodCalc> collector = result.stream().collect(groupingBy(p -> p.getAtDate(), ....
What can be put into the code to select multiple attributes ? I'm thinking of using multiple Collectors but really don't know how to do so.
To do it without a custom Collector (not streaming again on the result), you could do it like this. It's a bit dirty, since it is first collecting to Map<String, List<TimePeriodCalc>> and then streaming that list and get the average double.
Since you need two averages, they are collected to a Holder or a Pair, in this case I'm using AbstractMap.SimpleEntry
Map<String, SimpleEntry<Double, Double>> map = Stream.of(new TimePeriodCalc(12d, 10d, "A"), new TimePeriodCalc(2d, 16d, "A"))
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(Collectors.toList(), list -> {
double occupancy = list.stream().collect(
Collectors.averagingDouble(TimePeriodCalc::getOccupancy));
double efficiency = list.stream().collect(
Collectors.averagingDouble(TimePeriodCalc::getEfficiency));
return new AbstractMap.SimpleEntry<>(occupancy, efficiency);
})));
System.out.println(map);
Here's a way with a custom collector. It only needs one pass, but it's not very easy, especially because of generics...
If you have this method:
#SuppressWarnings("unchecked")
#SafeVarargs
static <T, A, C extends Collector<T, A, Double>> Collector<T, ?, List<Double>>
averagingManyDoubles(ToDoubleFunction<? super T>... extractors) {
List<C> collectors = Arrays.stream(extractors)
.map(extractor -> (C) Collectors.averagingDouble(extractor))
.collect(Collectors.toList());
class Acc {
List<A> averages = collectors.stream()
.map(c -> c.supplier().get())
.collect(Collectors.toList());
void add(T elem) {
IntStream.range(0, extractors.length).forEach(i ->
collectors.get(i).accumulator().accept(averages.get(i), elem));
}
Acc merge(Acc another) {
IntStream.range(0, extractors.length).forEach(i ->
averages.set(i, collectors.get(i).combiner()
.apply(averages.get(i), another.averages.get(i))));
return this;
}
List<Double> finish() {
return IntStream.range(0, extractors.length)
.mapToObj(i -> collectors.get(i).finisher().apply(averages.get(i)))
.collect(Collectors.toList());
}
}
return Collector.of(Acc::new, Acc::add, Acc::merge, Acc::finish);
}
This receives an array of functions that will extract double values from each element of the stream. These extractors are converted to Collectors.averagingDouble collectors and then the local Acc class is created with the mutable structures that are used to accumulate the averages for each collector. Then, the accumulator function forwards to each accumulator, and so with the combiner and finisher functions.
Usage is as follows:
Map<String, List<Double>> averages = list.stream()
.collect(Collectors.groupingBy(
TimePeriodCalc::getAtDate,
averagingManyDoubles(
TimePeriodCalc::getOccupancy,
TimePeriodCalc::getEfficiency)));
Assuming that your TimePeriodCalc class has all the necessary getters, this should get you the list you want:
List<TimePeriodCalc> result = new ArrayList<>(
list.stream()
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(Collectors.toList(), TimePeriodCalc::avgTimePeriodCalc)))
.values()
);
Where TimePeriodCalc.avgTimePeriodCalc is this method in the TimePeriodCalc class:
public static TimePeriodCalc avgTimePeriodCalc(List<TimePeriodCalc> list){
return new TimePeriodCalc(
list.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getOccupancy)),
list.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getEfficiency)),
list.get(0).getAtDate()
);
}
The above can be combined into this monstrosity:
List<TimePeriodCalc> result = new ArrayList<>(
list.stream()
.collect(Collectors.groupingBy(TimePeriodCalc::getAtDate,
Collectors.collectingAndThen(
Collectors.toList(), a -> {
return new TimePeriodCalc(
a.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getOccupancy)),
a.stream().collect(Collectors.averagingDouble(TimePeriodCalc::getEfficiency)),
a.get(0).getAtDate()
);
}
)))
.values());
With input:
List<TimePeriodCalc> list = new ArrayList<>();
list.add(new TimePeriodCalc(10,10,"a"));
list.add(new TimePeriodCalc(10,10,"b"));
list.add(new TimePeriodCalc(10,10,"c"));
list.add(new TimePeriodCalc(5,5,"a"));
list.add(new TimePeriodCalc(0,0,"b"));
This would give:
TimePeriodCalc [occupancy=7.5, efficiency=7.5, atDate=a]
TimePeriodCalc [occupancy=5.0, efficiency=5.0, atDate=b]
TimePeriodCalc [occupancy=10.0, efficiency=10.0, atDate=c]
You can chain multiple attributes like this:
Collection<TimePeriodCalc> collector = result.stream().collect(Collectors.groupingBy(p -> p.getAtDate(), Collectors.averagingInt(p -> p.getOccupancy())));
If you want more, you get the idea.
I have a class like this:
class MultiDataPoint {
private DateTime timestamp;
private Map<String, Number> keyToData;
}
and i want to produce , for each MultiDataPoint
class DataSet {
public String key;
List<DataPoint> dataPoints;
}
class DataPoint{
DateTime timeStamp;
Number data;
}
of course a 'key' can be the same across multiple MultiDataPoints.
So given a List<MultiDataPoint>, how do I use Java 8 streams to convert to List<DataSet>?
This is how I am currently doing the conversion without streams:
Collection<DataSet> convertMultiDataPointToDataSet(List<MultiDataPoint> multiDataPoints)
{
Map<String, DataSet> setMap = new HashMap<>();
multiDataPoints.forEach(pt -> {
Map<String, Number> data = pt.getData();
data.entrySet().forEach(e -> {
String seriesKey = e.getKey();
DataSet dataSet = setMap.get(seriesKey);
if (dataSet == null)
{
dataSet = new DataSet(seriesKey);
setMap.put(seriesKey, dataSet);
}
dataSet.dataPoints.add(new DataPoint(pt.getTimestamp(), e.getValue()));
});
});
return setMap.values();
}
It's an interesting question, because it shows that there are a lot of different approaches to achieve the same result. Below I show three different implementations.
Default methods in Collection Framework: Java 8 added some methods to the collections classes, that are not directly related to the Stream API. Using these methods, you can significantly simplify the implementation of the non-stream implementation:
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
Map<String, DataSet> result = new HashMap<>();
multiDataPoints.forEach(pt ->
pt.keyToData.forEach((key, value) ->
result.computeIfAbsent(
key, k -> new DataSet(k, new ArrayList<>()))
.dataPoints.add(new DataPoint(pt.timestamp, value))));
return result.values();
}
Stream API with flatten and intermediate data structure: The following implementation is almost identical to the solution provided by Stuart Marks. In contrast to his solution, the following implementation uses an anonymous inner class as intermediate data structure.
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.flatMap(mdp -> mdp.keyToData.entrySet().stream().map(e ->
new Object() {
String key = e.getKey();
DataPoint dataPoint = new DataPoint(mdp.timestamp, e.getValue());
}))
.collect(
collectingAndThen(
groupingBy(t -> t.key, mapping(t -> t.dataPoint, toList())),
m -> m.entrySet().stream().map(e -> new DataSet(e.getKey(), e.getValue())).collect(toList())));
}
Stream API with map merging: Instead of flattening the original data structures, you can also create a Map for each MultiDataPoint, and then merge all maps into a single map with a reduce operation. The code is a bit simpler than the above solution:
Collection<DataSet> convert(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.map(mdp -> mdp.keyToData.entrySet().stream()
.collect(toMap(e -> e.getKey(), e -> asList(new DataPoint(mdp.timestamp, e.getValue())))))
.reduce(new HashMap<>(), mapMerger())
.entrySet().stream()
.map(e -> new DataSet(e.getKey(), e.getValue()))
.collect(toList());
}
You can find an implementation of the map merger within the Collectors class. Unfortunately, it is a bit tricky to access it from the outside. Following is an alternative implementation of the map merger:
<K, V> BinaryOperator<Map<K, List<V>>> mapMerger() {
return (lhs, rhs) -> {
Map<K, List<V>> result = new HashMap<>();
lhs.forEach((key, value) -> result.computeIfAbsent(key, k -> new ArrayList<>()).addAll(value));
rhs.forEach((key, value) -> result.computeIfAbsent(key, k -> new ArrayList<>()).addAll(value));
return result;
};
}
To do this, I had to come up with an intermediate data structure:
class KeyDataPoint {
String key;
DateTime timestamp;
Number data;
// obvious constructor and getters
}
With this in place, the approach is to "flatten" each MultiDataPoint into a list of (timestamp, key, data) triples and stream together all such triples from the list of MultiDataPoint.
Then, we apply a groupingBy operation on the string key in order to gather the data for each key together. Note that a simple groupingBy would result in a map from each string key to a list of the corresponding KeyDataPoint triples. We don't want the triples; we want DataPoint instances, which are (timestamp, data) pairs. To do this we apply a "downstream" collector of the groupingBy which is a mapping operation that constructs a new DataPoint by getting the right values from the KeyDataPoint triple. The downstream collector of the mapping operation is simply toList which collects the DataPoint objects of the same group into a list.
Now we have a Map<String, List<DataPoint>> and we want to convert it to a collection of DataSet objects. We simply stream out the map entries and construct DataSet objects, collect them into a list, and return it.
The code ends up looking like this:
Collection<DataSet> convertMultiDataPointToDataSet(List<MultiDataPoint> multiDataPoints) {
return multiDataPoints.stream()
.flatMap(mdp -> mdp.getData().entrySet().stream()
.map(e -> new KeyDataPoint(e.getKey(), mdp.getTimestamp(), e.getValue())))
.collect(groupingBy(KeyDataPoint::getKey,
mapping(kdp -> new DataPoint(kdp.getTimestamp(), kdp.getData()), toList())))
.entrySet().stream()
.map(e -> new DataSet(e.getKey(), e.getValue()))
.collect(toList());
}
I took some liberties with constructors and getters, but I think they should be obvious.