This question already has an answer here:
Sum attribute of object with Stream API
(1 answer)
Closed 2 years ago.
I want to summarize the attributes of my objects.
The objects have a structure like this:
class Foo{
String name;
int = price;
int = fees;
//etc
}
I have collected them into a List and want to get (with streams) a list of all elements that only contain the name and the sum of the price and fees. For example:
{name=Apple, price=2; fees=1}
{name=Potato, price=7, fees=10}
{name=strawberry, price=5, fees=4}
should lead to this result:
{name=Apple, sum=3}
{name=Potato, sum=17}
{name=strawberry, sum=9}
Map<String, Integer> result =
foos.stream()
.collect(
Collectors.groupingBy(
Foo::getName,
Collectors.summingInt(x -> x.getPrice() + x.getFees())
)
);
collect to a Map with a Collectors::groupingBy and a downstream collector : Collectors::summingInt.
Or you can use toMap:
foos.stream()
.collect(
Collectors.toMap(
Foo::getName,
x -> x.getPrice() + x.getFees(),
Integer::sum
)
);
This is rather easy to achieve if you just read the documentation a bit.
Implement class Bar:
class Bar {
String name;
int sum;
// mapping constructor
public Bar(Foo foo) {
this.name = foo.name;
this.sum = foo.price + foo.fees;
}
}
and remap Foo using reference to the constructor:
List<Bar> sums = listFoos.stream().map(Bar::new).collect(Collectors.toList());
Instead of constructor a method with the same functionality may be implemented.
If multiple instances of Foo need to be summarized into single Bar, a method to merge a Bar may be added to Bar:
class Bar {
// ...
public Bar merge(Bar bar) {
this.sum += bar.sum;
return this;
}
}
Then the collection of Bar can be retrieved as values of the following map:
Collection<Bar> sumBars = listFoos
.stream()
.collect(Collectors.toMap(Foo::getName, Bar::new, Bar::merge))
.values();
Related
I have two list on two Class where id and month is common
public class NamePropeties{
private String id;
private Integer name;
private Integer months;
}
public class NameEntries {
private String id;
private Integer retailId;
private Integer months;
}
List NamePropetiesList = new ArrayList<>();
List NameEntries = new ArrayList<>();
Now i want to JOIN two list (like Sql does, JOIN ON month and id coming from two results) and return the data in new list where month and id is same in the given two list.
if i will start iterating only one and check in another list then there can be a size iteration issue.
i have tried to do it in many ways but is there is any stream way?
The general idea has been sketched in the comments: iterate one list, create a map whose keys are the attributes you want to join by, then iterate the other list and check if there's an entry in the map. If there is, get the value from the map and create a new object from the value of the map and the actual element of the list.
It's better to create a map from the list with the higher number of joined elements. Why? Because searching a map is O(1), no matter the size of the map. So, if you create a map from the list with the higher number of joined elements, then, when you iterate the second list (which is smaller), you'll be iterating among less elements.
Putting all this in code:
public static <B, S, J, R> List<R> join(
List<B> bigger,
List<S> smaller,
Function<B, J> biggerKeyExtractor,
Function<S, J> smallerKeyExtractor,
BiFunction<B, S, R> joiner) {
Map<J, List<B>> map = new LinkedHashMap<>();
bigger.forEach(b ->
map.computeIfAbsent(
biggerKeyExtractor.apply(b),
k -> new ArrayList<>())
.add(b));
List<R> result = new ArrayList<>();
smaller.forEach(s -> {
J key = smallerKeyExtractor.apply(s);
List<B> bs = map.get(key);
if (bs != null) {
bs.forEach(b -> {
R r = joiner.apply(b, s);
result.add(r);
}
}
});
return result;
}
This is a generic method that joins bigger List<B> and smaller List<S> by J join keys (in your case, as the join key is a composite of String and Integer types, J will be List<Object>). It takes care of duplicates and returns a result List<R>. The method receives both lists, functions that will extract the join keys from each list and a joiner function that will create new result R elements from joined B and S elements.
Note that the map is actually a multimap. This is because there might be duplicates as per the biggerKeyExtractor join function. We use Map.computeIfAbsent to create this multimap.
You should create a class like this to store joined results:
public class JoinedResult {
private final NameProperties properties;
private final NameEntries entries;
public JoinedResult(NameProperties properties, NameEntries entries) {
this.properties = properties;
this.entries = entries;
}
// TODO getters
}
Or, if you are in Java 14+, you might just use a record:
public record JoinedResult(NameProperties properties, NameEntries entries) { }
Or actually, any Pair class from out there will do, or you could even use Map.Entry.
With the result class (or record) in place, you should call the join method this way:
long propertiesSize = namePropertiesList.stream()
.map(p -> Arrays.asList(p.getMonths(), p.getId()))
.distinct()
.count();
long entriesSize = nameEntriesList.steram()
.map(e -> Arrays.asList(e.getMonths(), e.getId()))
.distinct()
.count();
List<JoinedResult> result = propertiesSize > entriesSize ?
join(namePropertiesList,
nameEntriesList,
p -> Arrays.asList(p.getMonths(), p.getId()),
e -> Arrays.asList(e.getMonths(), e.getId()),
JoinedResult::new) :
join(nameEntriesList,
namePropertiesList,
e -> Arrays.asList(e.getMonths(), e.getId()),
p -> Arrays.asList(p.getMonths(), p.getId()),
(e, p) -> new JoinedResult(p, e));
The key is to use generics and call the join method with the right arguments (they are flipped, as per the join keys size comparison).
Note 1: we can use List<Object> as the key of the map, because all Java lists implement equals and hashCode consistently (thus they can safely be used as map keys)
Note 2: if you are on Java9+, you should use List.of instead of Arrays.asList
Note 3: I haven't checked for neither null nor invalid arguments
Note 4: there is room for improvements, i.e. key extractor functions could be memoized, join keys could be reused instead of calculated more than once and multimap could have Object values for single elements and lists for duplicates, etc
If performance and nesting (as discussed) is not too much of a concern you could employ something along the lines of a crossjoin with filtering:
Result holder class
public class Tuple<A, B> {
public final A a;
public final B b;
public Tuple(A a, B b) {
this.a = a;
this.b = b;
}
}
Join with a predicate:
public static <A, B> List<Tuple<A, B>> joinOn(
List<A> l1,
List<B> l2,
Predicate<Tuple<A, B>> predicate) {
return l1.stream()
.flatMap(a -> l2.stream().map(b -> new Tuple<>(a, b)))
.filter(predicate)
.collect(Collectors.toList());
}
Call it like this:
List<Tuple<NamePropeties, NameEntries>> joined = joinOn(
properties,
names,
t -> Objects.equals(t.a.id, t.b.id) && Objects.equals(t.a.months, t.b.months)
);
I've a List of Foo, where on each Foo I apply a processor method to get ValidItem.
If there is an error in processing, then I returned ErrorItem.
Now How to process this by Java 8 streams to get the result in form of 2 different lists
List<Foo> FooList = someList....;
class ValidItem extend Item{......}
class ErrorItem extend Item{......}
Item processItem(Foo foo){
return either an object of ValidItem or ErrorItem;
}
I believe I can do this
Map<Class,List<Item>> itemsMap =
FooList
.stream()
.map(processItem)
.collect(Collectors.groupingBy(Object::getClass));
But as List<Parent> IS NOT a List<Child> so I can't typecast the map result into List<ValidItem>
In reality ErrorItem and ValidItem are two completely different class not related at all, just for the sake of this steam processing and processItem method I kept them in same hierarchy by extending a marker Item class,.
and in many other Places in code, I cant/shouldn't refer ValidItem as Item , as It give an idea that it can be an ErroItem too.
Is there a proper way of doing it with streams, where at the end I get 2 lists. and ErrorItem and
ValidItem are not extending same Item class ?
############## Update ##############
As I said ValidItem and ErrorItem shouldn't be same, so I changed the signature of process method and passed it a list.
I know this is not how Stream shold be used. Let me know if you have better way
List<Foo> FooList = someList....;
class ValidItem {......}
class InvalidFoo{......}
ValidItem processFoo(Foo foo, List<InvalidFoo> foolist){
Do some processing on foo.
either return new ValidItem ();
OR
fooList.add(new InvalidFoo()) , and then return null;
}
List<InvalidFoo> invalidFooList = new ArrayList();
List<ValidItem> validItem =
fooList
.stream()
.map(e->processItem(e,invalidFooList))
.filter(Objects::notNull)
.collect(Collectors.toList());
now I have both invalid and valid list, but this doesn't look like a clean stream code.
With recent Java versions, you can use
for the Item processItem(Foo foo) method returning either ValidItem or ErrorItem:
Map.Entry<List<ValidItem>, List<ErrorItem>> collected = fooList.stream()
.map(this::processItem)
.collect(teeing(
flatMapping(x -> x instanceof ValidItem? Stream.of((ValidItem)x):null, toList()),
flatMapping(x -> x instanceof ErrorItem? Stream.of((ErrorItem)x):null, toList()),
AbstractMap.SimpleImmutableEntry::new
));
List<ValidItem> valid = collected.getKey();
List<ErrorItem> invalid = collected.getValue();
for the ValidItem processFoo(Foo foo, List<InvalidFoo> foolist):
Map.Entry<List<ValidItem>, List<InvalidFoo>> collected = fooList.stream()
.map(foo -> {
List<InvalidFoo> invalid = new ArrayList<>(1);
ValidItem vi = processFoo(foo, invalid);
return new AbstractMap.SimpleImmutableEntry<>(
vi == null? Collections.<ValidItem>emptyList():
Collections.singletonList(vi),
invalid);
})
.collect(teeing(
flatMapping(e -> e.getKey().stream(), toList()),
flatMapping(e -> e.getValue().stream(), toList()),
AbstractMap.SimpleImmutableEntry::new
));
List<ValidItem> valid = collected.getKey();
List<InvalidFoo> invalid = collected.getValue();
The flatMapping collector has been introduced in Java 9.
In this specific case, instead of
flatMapping(x -> x instanceof ValidItem? Stream.of((ValidItem)x): null, toList())
you can also use
filtering(x -> x instanceof ValidItem, mapping(x -> (ValidItem)x, toList()))
but each variant requires Java 9, as filtering also does not exist in Java 8. The teeing collector even requires Java 12.
However, these collectors are not hard to implement.
This answer contains a Java 8 compatible version of the flatMapping collector at the end. If you want to use the alternative with filtering and mapping, you can find a Java 8 compatible version of filtering in this answer. Finally, this answer contains a Java 8 compatible variant of the teeing collector.
When you add these collectors to your codebase, the solutions at the beginning of this answer work under Java 8 and will being easily adaptable to future versions. Assuming an import static java.util.stream.Collectors.*; in your source file, you only have to remove the backports of these methods, to switch to the standard JDK versions.
It would be better if processItem returned an Either or Pair type instead of the two variants addressed above. If you don’t want to use 3rd party libraries, you can use a Map.Entry instance as a “poor man’s pair type”.
Having a method signature like
/** Returns an entry with either, key or value, being {#code null} */
Map.Entry<ValidItem,InvalidFoo> processItem(Foo foo){
which could be implemented by returning an instance of AbstractMap.SimpleImmutableEntry,
a JDK 9+ solution could look like
Map.Entry<List<ValidItem>, List<InvalidFoo>> collected = fooList.stream()
.map(this::processItem)
.collect(teeing(
flatMapping(e -> Stream.ofNullable(e.getKey()), toList()),
flatMapping(e -> Stream.ofNullable(e.getValue()), toList()),
AbstractMap.SimpleImmutableEntry::new
));
and Java 8 compatible (when using the linked collector backports) versions:
Map.Entry<List<ValidItem>, List<InvalidFoo>> collected = fooList.stream()
.map(this::processItem)
.collect(teeing(
flatMapping(e -> Stream.of(e.getKey()).filter(Objects::nonNull), toList()),
flatMapping(e -> Stream.of(e.getValue()).filter(Objects::nonNull), toList()),
AbstractMap.SimpleImmutableEntry::new
));
or
Map.Entry<List<ValidItem>, List<InvalidFoo>> collected = fooList.stream()
.map(this::processItem)
.collect(teeing(
mapping(Map.Entry::getKey, filtering(Objects::nonNull, toList())),
mapping(Map.Entry::getValue, filtering(Objects::nonNull, toList())),
AbstractMap.SimpleImmutableEntry::new
));
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.stream.Collectors;
public class Main {
public static void main(String[] args) {
List<Foo> FooList = new ArrayList<>();
for(int i = 0 ; i < 100; i++){
FooList.add(new Foo(i+""));
}
Map<Class,List<Item>> itemsMap =
FooList
.stream()
.map(Main::processItem)
.collect(Collectors.groupingBy(Object::getClass));
List<ValidItem> validItems = itemsMap.get(ValidItem.class).stream().map((o -> (ValidItem)o)).collect(Collectors.toList());
}
public static Item processItem(Foo foo){
Random random = new Random(System.currentTimeMillis());
if(Integer.parseInt(foo.name) % 2== 0){
return new ValidItem(foo.name);
}else{
return new ErrorItem(foo.name);
}
}
static class ValidItem extends Item{
public ValidItem(String name) {
super("valid: " + name);
}
}
static class ErrorItem extends Item{
public ErrorItem(String name) {
super("error: "+name);
}
}
public static class Item {
private String name;
public Item(String name) {
this.name = name;
}
}
}
I suggest this solution.
You can use Vavr library.
final List<String> list = Arrays.asList("1", ",", "1", "0");
final List<Either<ErrorItem, ValidItem>> eithers = list.stream()
.map(MainClass::processData)
.collect(Collectors.toList());
final List<ValidItem> validItems = eithers.stream()
.filter(Either::isRight)
.map(Either::get)
.collect(Collectors.toList());
final List<ErrorItem> errorItems = eithers.stream()
.filter(Either::isLeft)
.map(Either::getLeft)
.collect(Collectors.toList());
...
private static Either<ErrorItem,ValidItem> processData(String data){
if(data.equals("1")){
return Either.right(new ValidItem());
}
return Either.left(new ErrorItem());
}
I have a list of Transactions whom I wanted to :
First Group by year
Then Group by type for every transaction in that year
Then convert the Transactions to Result object having sum of all transaction's value in sub groups.
My Code snippets looks like :
Map<Integer, Map<String, Result> res = transactions.stream().collect(Collectors
.groupingBy(Transaction::getYear,
groupingBy(Transaction::getType),
reducing((a,b)-> new Result("YEAR_TYPE", a.getAmount() + b.getAmount()))
));
Transaction Class :
class Transaction {
private int year;
private String type;
private int value;
}
Result Class :
class Result {
private String group;
private int amount;
}
it seems to be not working, what should I do to fix this making sure it works on parallel streams too?
In the context, Collectors.reducing would help you reduce two Transaction objects into a final object of the same type. In your existing code what you could have done to map to Result type was to use Collectors.mapping and then trying to reduce it.
But reducing without an identity provides and Optional wrapped value for a possible absence. Hence your code would have looked like ;
Map<Integer, Map<String, Optional<Result>>> res = transactions.stream()
.collect(Collectors.groupingBy(Transaction::getYear,
Collectors.groupingBy(Transaction::getType,
Collectors.mapping(t -> new Result("YEAR_TYPE", t.getValue()),
Collectors.reducing((a, b) ->
new Result(a.getGroup(), a.getAmount() + b.getAmount()))))));
to thanks to Holger, one can simplify this further
…and instead of Collectors.mapping(func, Collectors.reducing(op)) you
can use Collectors.reducing(id, func, op)
Instead of using this and a combination of Collectors.grouping and Collectors.reducing, transform the logic to use Collectors.toMap as:
Map<Integer, Map<String, Result>> result = transactions.stream()
.collect(Collectors.groupingBy(Transaction::getYear,
Collectors.toMap(Transaction::getType,
t -> new Result("YEAR_TYPE", t.getValue()),
(a, b) -> new Result(a.getGroup(), a.getAmount() + b.getAmount()))));
The answer would stand complete with a follow-up read over Java Streams: Replacing groupingBy and reducing by toMap.
I would use a custom collector:
Collector<Transaction, Result, Result> resultCollector =
Collector.of(Result::new, // what is the result of this collector
(a, b) -> { a.setAmount( a.getAmount() + b.getValue());
a.setGroup("YEAR_TYPE"); }, // how to accumulate a result from a transaction
(l, r) -> { l.setAmount(l.getAmount() + r.getAmount()); return l; }); // how to combine two
// result instances
// (used in parallel streams)
then you can use the collector to get the map:
Map<Integer, Map<String, Result>> collect = transactions.parallelStream().collect(
groupingBy(Transaction::getYear,
groupingBy(Transaction::getType, resultCollector) ) );
I would like to use Streams API to process a call log and calculate the total billing amount for the same phone number. Here's the code that achieves it with a hybrid approach but I would like to use fully functional approach:
List<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toList());
for (int i = 0; i< callLogs.size() -1 ;i++) {
if (callLogs.get(i).phoneNumber == callLogs.get(i+1).phoneNumber) {
callLogs.get(i).billing += callLogs.get(i+1).billing;
callLogs.remove(i+1);
}
}
You can use Collectors.groupingBy to group CallLog object by phoneNumber with Collectors.summingInt to sum the billing of grouped elements
Map<Integer, Integer> likesPerType = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.groupingBy(CallLog::getPhoneNumber, Collectors.summingInt(CallLog::getBilling)));
Map<Integer, Integer> result = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toMap(
c -> c.phoneNumber(),
c -> c.billing(),
(a, b) -> a+b
));
And if you want to have a 'List callLogs' as a result:
List<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.toMap(
c -> c.phoneNumber(),
c -> c.billing(),
(a, b) -> a+b
))
.entrySet()
.stream()
.map(entry -> toCallLog(entry.getKey(), entry.getValue()))
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toList())
You can save yourself the sorting -> collection to list -> iterating the list for values next to each other if you instead do the following
Create all CallLog objects.
Merge them by the phoneNumber field
combine the billing fields every time
Return the already merged items
This can be done using Collectors.toMap(Function, Function, BinaryOperator) where the third parameter is the merge function that defines how items with identical keys would be combined:
Collection<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.toMap( //a collector that will produce a map
CallLog::phoneNumber, //using phoneNumber as the key to group
x -> x, //the item itself as the value
(a, b) -> { //and a merge function that returns an object with combined billing
a.billing += b.billing;
return a;
}))
.values(); //just return the values from that map
In the end, you would have CallLog items with unique phoneNumber fields whose billing field is equal to the combination of all billings of the previously duplicate phoneNumbers.
What you are trying to do is to remove duplicate phone numbers, while adding their billing. The one thing streams are incompatible with are remove operations. So how can we do what you need without remove?
Well instead of sorting I would go with groupingBy phone numbers then I would map the list of groups of callLogs into callLogs with the billing already accumulated.
You could group the billing amount by phoneNumber, like VLAZ said. An example implementation could look something like this:
import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;
public class Demo {
public static void main(String[] args) {
final String s = "555123456;12.00\n"
+ "555123456;3.00\n"
+ "555123457;1.00\n"
+ "555123457;2.00\n"
+ "555123457;5.00";
final Map<Integer, Double> map = Arrays.stream(s.split("\n"))
.map(CallLog::new)
.collect(Collectors.groupingBy(CallLog::getPhoneNumber, Collectors.summingDouble(CallLog::getBilling)));
map.forEach((key, value) -> System.out.printf("%d: %.2f\n", key, value));
}
private static class CallLog {
private final int phoneNumber;
private final double billing;
public CallLog(int phoneNumber, double billing) {
this.phoneNumber = phoneNumber;
this.billing = billing;
}
public CallLog(String s) {
final String[] strings = s.split(";");
this.phoneNumber = Integer.parseInt(strings[0]);
this.billing = Double.parseDouble(strings[1]);
}
public int getPhoneNumber() {
return phoneNumber;
}
public double getBilling() {
return billing;
}
}
}
which produces the following output:
555123456: 15.00
555123457: 8.00
I have a structure like this:
Map<Long, List<Foo>>
where class Foo exposes method:
Class Foo {
public List<Bar> getBars();
public void setBars(List<Bar> bars);
}
Now I want to convert this map to List parametrized with Foo class where each item in this list is Foo instance with aggregated bars list for given long value. For example with map:
{1: [Foo1, Foo2],
2: [Foo3]}
where
Foo1.bars = [Bar1, Bar2]
Foo2.bars = [Bar3]
Foo3.bars = [Bar4, Bar5]
I want to get as a result:
[FooA, FooB]
where
FooA.bars = [Bar1, Bar2, Bar3]
FooB.bars = [Bar4, Bar5]
What would be the most elegant solution for this in Java 8?
Some of the Foo instances from map can be reused if necessary as they are not used anymore after this operation.
Assuming you have a Foo(List<Bar> bars) constructor it's quite easy:
import static java.util.stream.Collectors.*;
List<Foo> result = map.values()
.stream()
.map(
foos -> new Foo(foos.stream()
.flatMap(foo -> foo.getBars().stream())
.collect(toList())))
.collect(toList());
We take the stream of the original map values (we don't care about keys), which are lists of Foo. Each such list we flatten to get the stream of Bar, collect them to list and pass this list to the Foo(List<Bar>) constructor, so we get new Foo objects. Finally we collect them to the List.
If you don't have the Foo(List<Bar>), only setter, you should first create an empty Foo, then use the setter and return the created Foo:
List<Foo> result = map.values()
.stream()
.map(foos -> {
Foo f = new Foo();
f.setBars(foos.stream().flatMap(
foo -> foo.getBars().stream()).collect(toList()));
return f;
})
.collect(toList());
If you don't want to create new Foo objects (for example, there are additional properties you want to keep), it's better to introduce not the setBars, but addBars method (which adds new bars to the existing ones) like this:
public class Foo {
...
public Foo addBars(List<Bar> bars) {
this.bars.addAll(bars);
return this;
}
}
Now you can use the reduce terminal operation to combine the foos:
List<Foo> result = map.values()
.stream()
.map(foos -> foos.stream()
.reduce((foo1, foo2) -> foo1.addBars(foo2.getBars())).get())
.collect(toList());