We Keep Coding

Calculate sum of list within a Map of Map

I have a list of Trades that is a map of map grouped by Region and Status. Trade class has an attribute amount field.
Map<Region, Map<Status, List<Trade>>> groupedTrades
class Trade {
double amount;
}
I want to group the amounts across Trades within the list and return it as below
Map<Region, Map<Status, Double>> sumOfGroupedTradeAmounts
Double is a sum of all the amount fields within the list of trades.
How can I do this is in java8?

You can do it like this. Basically, you are:
Doing nested streaming of the entrySets.
Stream the outer map entries to get the inner map values and the outer map key.
in the inner entry set, stream the value, which is the List<Doubles> and sum them.
these are then returned in the specified map using the outer map key, inner map key and a sum of the Trade amounts resulting in a double value.
Note that I added a getter to the Trade class for amount retrieval.
class Trade {
double amount;
public double getAmount() {
return amount;
}
}
public static void main(String[] args) {
Map<Region, Map<Status, Double>> result = groupedTrades
.entrySet().stream()
// outer map starts here, keying on Region
.collect(Collectors.toMap(Entry::getKey, e -> e
.getValue().entrySet().stream()
// inner map starts here, keying on Status
.collect(Collectors.toMap(Entry::getKey,
// stream the list and sum the amounts.
ee -> ee.getValue().stream()
.mapToDouble(Trade::getAmount)
.sum()))));
}
Given the following structure where Region and Status are numbers and letters respectively,
Map<Region, Map<Status, List<Trade>>> groupedTrades = Map.of(
new Region(1),
Map.of(new Status("A"),
List.of(new Trade(10), new Trade(20),
new Trade(30)),
new Status("B"),
List.of(new Trade(1), new Trade(2))),
new Region(2),
Map.of(new Status("A"),
List.of(new Trade(2), new Trade(4),
new Trade(6)),
new Status("B"), List.of(new Trade(3),
new Trade(6), new Trade(9))));
result.forEach((k, v) -> {
System.out.println(k);
v.forEach((kk, vv) -> System.out
.println(" " + kk + " -> " + vv));
});
Here is the sample output.
2
B -> 18.0
A -> 12.0
1
B -> 3.0
A -> 60.0
If you're interested in suming all of the values. you can do it as follows by using the just created nested map of doubles.
double allSums =
// Stream the inner maps
result.values().stream()
// put all the Doubles in a single stream
.flatMap(m->m.values().stream())
// unbox the Doubles to primitive doubles
.mapToDouble(Double::doubleValue)
// and sum them
.sum();

Using only streams it might look like as follows:
Map<Region, Map<Status, Double>> result =
groupedTrades
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey, // Leave key as is
entry -> mapToDouble(entry.getValue(), MyClass::sumOfTrades))); // applies sumOfTrades (See below)
And we define little helper function, in order to keep our stream readable:
// Takes map which contains lists of trades and and applies function mapper, which returns T (might be double) on those lists
private static <T> Map<Status, T> mapToDouble(Map<Status, List<Trade>> trades, Function<List<Trade>, T> mapper) {
return trades.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey, // Leave key as is
entry -> mapper.apply(entry.getValue())));
}
// Takes a list of trades and returns its sum
private static double sumOfTrades(List<Trade> trades) {
return trades.stream().mapToDouble(Trade::getAmount).sum();
}
Bonus - sum of all trades:
double sum =
groupedTrades
// get collection of "inner" maps
.values()
.stream()
// get only their values as collection of lists
.map(Map::values)
// map the collection into stream of lists
.flatMap(Collection::stream)
// map the collection stream of lists into stream of Trade values
.flatMap(Collection::stream)
// map Trade into double value - "amount"
.mapToDouble(Trade::getAmount)
// get the sum
.sum();

Sort list of objects with respect to list of given input values

I've a value object whose structure is like:
class MyValueObject {
String travellerName;
List<String> countryTags;
// Getters and setters
}
In this I'm maintaining a list of traveller and which countries they've visited. e.g.
Richard -> India, Poland, Australia
John -> US, Australia, Maldives
Emma -> US
Now I'm adding a filter feature, which will give list of all travellers who've visited selected countries.
List of countries will be provided as an input.
List<String> countryFilter;
This filter has a multiselect option.
The filtered result should contain both AND & OR results.
That is, if input is US & Australia, result will be:
John -> US, Australia, Maldives
Richard -> India, Poland, Australia
Emma -> US
The result should be sorted in manner that AND matches should be shown above OR matches.
Question:
How should I sort the matches?
Shall I write a comparator? If yes, some example will be very helpful.
Please suggest.
Thanks.

Try this:
Building the objects.
List<Traveler> travelers = new ArrayList<>();
Traveler t1 = new Traveler();
t1.traveler = "Richard";
t1.countries = List.of("India", "Poland", "Australia");
travelers.add(t1);
t1 = new Traveler();
t1.traveler = "John";
t1.countries = List.of("US", "Australia", "Maldives");
travelers.add(t1);
t1 = new Traveler();
t1.traveler = "Emma";
t1.countries = List.of("US");
travelers.add(t1);
The filter
List<String> countryFilter = List.of("US", "Maldives");
The test predicates. These are the heart of the matter.
they both stream countryFilter and check to see if those countries are in the list of traveled countries.
or returns true if the travelers countries contain at least one country
and returns true if the traveler's countries contain all the countries.
Predicate<Traveler> or = t -> countryFilter.stream()
.anyMatch(cc -> t.countries.contains(cc));
Predicate<Traveler> and = t -> countryFilter.stream()
.allMatch(cc -> t.countries.contains(cc));
Now just stream just stream the traveler objects and apply the filters. Then prints the results.
System.out.println("Filtering on: " + countryFilter);
System.out.println("\nVisited all of the countries");
travelers.stream().filter(and)
.forEach(t -> System.out.println(t.traveler));
System.out.println("\nVisited some of the countries");
travelers.stream().filter(or)
.forEach(t -> System.out.println(t.traveler));
Prints
Filtering on: [US, Maldives]
Visited all of the countries
John
Visited some of the countries
John
Emma
The class supporting class
class Traveler {
String traveler;
List<String> countries;
public String toString() {
return traveler + " => " + countries.toString();
}
}

Instead of sorting, iterate twice, once for the AND matches and once for the OR matches:
// Say you have a list of MyValueObject type called travelers
ArrayList<MyValueObject> copyTravelers = new ArrayList<>(travelers);
List<MyValueObject> filtered = new ArrayList<>();
// AND
for (MyValueObject t : copyTravelers) {
MyValueObject traveler = t;
for (String country : countryFilter)
if (!traveler.countryTags.contains(country)) {
traveler = null;
break;
}
if (traveler != null) {
filtered.add(traveler);
copyTravelers.remove(traveler);
}
}
// OR
for (MyValueObject t : copyTravelers) {
for (String country : countryFilter)
if (traveler.countryTags.contains(country)) {
filtered.add(t);
copyTravelers.remove(t);
break;
}
}

i tried an approach where you get an ordered Map of country matches:
public Map<Long, String> sortMatches(List<MyValueObject> travellers, List<String> countries){
TreeMap<Long, List> collect = travellers.stream()
.flatMap(t -> t.countries.stream()
.filter(c -> countries.contains(c))
.map(c -> t.getTravellerName())
.collect(groupingBy(Function.identity(), counting()))
.entrySet()
.stream()
).collect(groupingBy(
e -> e.getValue(),
TreeMap::new,
listCollector
));
return collect;
}
first the matches are counted and written to a temporary Map<String, Long> where the String is the travellerName and as value there is the match count of this traveller.
in a second step a new map is created which stores as key the amount of matches and as values the travellerNames of the travellers who have visited so many countries. therefore, the listCollector is used:
Collector<Map.Entry, List, List> listCollector =
Collector.of(ArrayList::new,
(l, e) -> l.add(e.getKey()),
(l1, l2) -> { l1.addAll(l2); return l1;});

Java functional approach to join identical elements into a single one

I would like to use Streams API to process a call log and calculate the total billing amount for the same phone number. Here's the code that achieves it with a hybrid approach but I would like to use fully functional approach:
List<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toList());
for (int i = 0; i< callLogs.size() -1 ;i++) {
if (callLogs.get(i).phoneNumber == callLogs.get(i+1).phoneNumber) {
callLogs.get(i).billing += callLogs.get(i+1).billing;
callLogs.remove(i+1);
}
}

You can use Collectors.groupingBy to group CallLog object by phoneNumber with Collectors.summingInt to sum the billing of grouped elements
Map<Integer, Integer> likesPerType = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.groupingBy(CallLog::getPhoneNumber, Collectors.summingInt(CallLog::getBilling)));

Map<Integer, Integer> result = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toMap(
c -> c.phoneNumber(),
c -> c.billing(),
(a, b) -> a+b
));
And if you want to have a 'List callLogs' as a result:
List<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.toMap(
c -> c.phoneNumber(),
c -> c.billing(),
(a, b) -> a+b
))
.entrySet()
.stream()
.map(entry -> toCallLog(entry.getKey(), entry.getValue()))
.sorted(Comparator.comparingInt(callLog -> callLog.phoneNumber))
.collect(Collectors.toList())

You can save yourself the sorting -> collection to list -> iterating the list for values next to each other if you instead do the following
Create all CallLog objects.
Merge them by the phoneNumber field
combine the billing fields every time
Return the already merged items
This can be done using Collectors.toMap(Function, Function, BinaryOperator) where the third parameter is the merge function that defines how items with identical keys would be combined:
Collection<CallLog> callLogs = Arrays.stream(S.split("\n"))
.map(CallLog::new)
.collect(Collectors.toMap( //a collector that will produce a map
CallLog::phoneNumber, //using phoneNumber as the key to group
x -> x, //the item itself as the value
(a, b) -> { //and a merge function that returns an object with combined billing
a.billing += b.billing;
return a;
}))
.values(); //just return the values from that map
In the end, you would have CallLog items with unique phoneNumber fields whose billing field is equal to the combination of all billings of the previously duplicate phoneNumbers.

What you are trying to do is to remove duplicate phone numbers, while adding their billing. The one thing streams are incompatible with are remove operations. So how can we do what you need without remove?
Well instead of sorting I would go with groupingBy phone numbers then I would map the list of groups of callLogs into callLogs with the billing already accumulated.

You could group the billing amount by phoneNumber, like VLAZ said. An example implementation could look something like this:
import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;
public class Demo {
public static void main(String[] args) {
final String s = "555123456;12.00\n"
+ "555123456;3.00\n"
+ "555123457;1.00\n"
+ "555123457;2.00\n"
+ "555123457;5.00";
final Map<Integer, Double> map = Arrays.stream(s.split("\n"))
.map(CallLog::new)
.collect(Collectors.groupingBy(CallLog::getPhoneNumber, Collectors.summingDouble(CallLog::getBilling)));
map.forEach((key, value) -> System.out.printf("%d: %.2f\n", key, value));
}
private static class CallLog {
private final int phoneNumber;
private final double billing;
public CallLog(int phoneNumber, double billing) {
this.phoneNumber = phoneNumber;
this.billing = billing;
}
public CallLog(String s) {
final String[] strings = s.split(";");
this.phoneNumber = Integer.parseInt(strings[0]);
this.billing = Double.parseDouble(strings[1]);
}
public int getPhoneNumber() {
return phoneNumber;
}
public double getBilling() {
return billing;
}
}
}
which produces the following output:
555123456: 15.00
555123457: 8.00

Converting List<MyObject> to Map<String, List<String>> in Java 8 when we have duplicate elements and custom filter criteria

I have an instances of Student class.
class Student {
String name;
String addr;
String type;
public Student(String name, String addr, String type) {
super();
this.name = name;
this.addr = addr;
this.type = type;
}
#Override
public String toString() {
return "Student [name=" + name + ", addr=" + addr + "]";
}
public String getName() {
return name;
}
public String getAddr() {
return addr;
}
}
And I have a code to create a map , where it store the student name as the key and some processed addr values (a List since we have multiple addr values for the same student) as the value.
public class FilterId {
public static String getNum(String s) {
// should do some complex stuff, just for testing
return s.split(" ")[1];
}
public static void main(String[] args) {
List<Student> list = new ArrayList<Student>();
list.add(new Student("a", "test 1", "type 1"));
list.add(new Student("a", "test 1", "type 2"));
list.add(new Student("b", "test 1", "type 1"));
list.add(new Student("c", "test 1", "type 1"));
list.add(new Student("b", "test 1", "type 1"));
list.add(new Student("a", "test 1", "type 1"));
list.add(new Student("c", "test 3", "type 2"));
list.add(new Student("a", "test 2", "type 1"));
list.add(new Student("b", "test 2", "type 1"));
list.add(new Student("a", "test 3", "type 1"));
Map<String, List<String>> map = new HashMap<>();
// This will create a Map with Student names (distinct) and the test numbers (distinct List of tests numbers) associated with them.
for (Student student : list) {
if (map.containsKey(student.getName())) {
List<String> numList = map.get(student.getName());
String value = getNum(student.getAddr());
if (!numList.contains(value)) {
numList.add(value);
map.put(student.getName(), numList);
}
} else {
map.put(student.getName(), new ArrayList<>(Arrays.asList(getNum(student.getAddr()))));
}
}
System.out.println(map.toString());
}
}
Output would be :
{a=[1, 2, 3], b=[1, 2], c=[1, 3]}
How can I just do the same in java8 in a much more elegant way, may be using the streams ?
Found this Collectors.toMap in java 8 but could't find a way to actually do the same with this.
I was trying to map the elements as CSVs but that it didn't work since I couldn't figure out a way to remove the duplicates easily and the output is not what I need at the moment.
Map<String, String> map2 = new HashMap<>();
map2 = list.stream().collect(Collectors.toMap(Student::getName, Student::getAddr, (a, b) -> a + " , " + b));
System.out.println(map2.toString());
// {a=test 1 , test 1 , test 1 , test 2 , test 3, b=test 1 , test 1 , test 2, c=test 1 , test 3}

With streams, you could use Collectors.groupingBy along with Collectors.mapping:
Map<String, Set<String>> map = list.stream()
.collect(Collectors.groupingBy(
Student::getName,
Collectors.mapping(student -> getNum(student.getAddr()),
Collectors.toSet())));
I've chosen to create a map of sets instead of a map of lists, as it seems that you don't want duplicates in the lists.
If you do need lists instead of sets, it's more efficient to first collect to sets and then convert the sets to lists:
Map<String, List<String>> map = list.stream()
.collect(Collectors.groupingBy(
Student::getName,
Collectors.mapping(s -> getNum(s.getAddr()),
Collectors.collectingAndThen(Collectors.toSet(), ArrayList::new))));
This uses Collectors.collectingAndThen, which first collects and then transforms the result.
Another more compact way, without streams:
Map<String, Set<String>> map = new HashMap<>(); // or LinkedHashMap
list.forEach(s ->
map.computeIfAbsent(s.getName(), k -> new HashSet<>()) // or LinkedHashSet
.add(getNum(s.getAddr())));
This variant uses Iterable.forEach to iterate the list and Map.computeIfAbsent to group transformed addresses by student name.

First of all, the current solution is not really elegant, regardless of any streaming solution.
The pattern of
if (map.containsKey(k)) {
Value value = map.get(k);
...
} else {
map.put(k, new Value());
}
can often be simplified with Map#computeIfAbsent. In your example, this would be
// This will create a Map with Student names (distinct) and the test
// numbers (distinct List of tests numbers) associated with them.
for (Student student : list)
{
List<String> numList = map.computeIfAbsent(
student.getName(), s -> new ArrayList<String>());
String value = getNum(student.getAddr());
if (!numList.contains(value))
{
numList.add(value);
}
}
(This is a Java 8 function, but it is still unrelated to streams).
Next, the data structure that you want to build there does not seem to be the most appropriate one. In general, the pattern of
if (!list.contains(someValue)) {
list.add(someValue);
}
is a strong sign that you should not use a List, but a Set. The set will contain each element only once, and you will avoid the contains calls on the list, which are O(n) and thus may be expensive for larger lists.
Even if you really need a List in the end, it is often more elegant and efficient to first collect the elements in a Set, and afterwards convert this Set into a List in one dedicated step.
So the first part could be solved like this:
// This will create a Map with Student names (distinct) and the test
// numbers (distinct List of tests numbers) associated with them.
Map<String, Collection<String>> map = new HashMap<>();
for (Student student : list)
{
String value = getNum(student.getAddr());
map.computeIfAbsent(student.getName(), s -> new LinkedHashSet<String>())
.add(value);
}
It will create a Map<String, Collection<String>>. This can then be converted into a Map<String, List<String>> :
// Convert the 'Collection' values of the map into 'List' values
Map<String, List<String>> result =
map.entrySet().stream().collect(Collectors.toMap(
Entry::getKey, e -> new ArrayList<String>(e.getValue())));
Or, more generically, using a utility method for this:
private static <K, V> Map<K, List<V>> convertValuesToLists(
Map<K, ? extends Collection<? extends V>> map)
{
return map.entrySet().stream().collect(Collectors.toMap(
Entry::getKey, e -> new ArrayList<V>(e.getValue())));
}
I do not recommend this, but you also could convert the for loop into a stream operation:
Map<String, Set<String>> map =
list.stream().collect(Collectors.groupingBy(
Student::getName, LinkedHashMap::new,
Collectors.mapping(
s -> getNum(s.getAddr()), Collectors.toSet())));
Alternatively, you could do the "grouping by" and the conversion from Set to List in one step:
Map<String, List<String>> result =
list.stream().collect(Collectors.groupingBy(
Student::getName, LinkedHashMap::new,
Collectors.mapping(
s -> getNum(s.getAddr()),
Collectors.collectingAndThen(
Collectors.toSet(), ArrayList<String>::new))));
Or you could introduce an own collector, that does the List#contains call, but all this tends to be far less readable than the other solutions...

I think you are looking for something like below
Map<String,Set<String>> map = list.stream().
collect(Collectors.groupingBy(
Student::getName,
Collectors.mapping(e->getNum(e.getAddr()), Collectors.toSet())
));
System.out.println("Map : "+map);

Here is a version that collects everything in sets, and converts the final result to array lists:
/*
import java.util.*;
import java.util.stream.*;
import static java.util.stream.Collectors.*;
import java.util.function.*;
*/
Map<String, List<String>> map2 = list.stream().collect(groupingBy(
Student::getName, // we will group the students by name
Collector.of(
HashSet::new, // for each student name, we will collect result in a hash set
(arr, student) -> arr.add(getNum(student.getAddr())), // which we fill with processed addresses
(left, right) -> { left.addAll(right); return left; }, // we merge subresults like this
(Function<HashSet<String>, List<String>>) ArrayList::new // finish by converting to List
)
));
System.out.println(map2);
// Output:
// {a=[1, 2, 3], b=[1, 2], c=[1, 3]}
EDIT: made the finisher shorter using Marco13's hint.

Java 8 streams: sum values on a distinct key

I have a file with rows with the following column headers:
CITY_NAME COUNTY_NAME POPULATION
Atascocita Harris 65844
Austin Travis 931820
Baytown Harris 76335
...
I am using streams to attempt to generate an output similar to:
COUNTY_NAME CITIES_IN_COUNTY POPULATION_OF_COUNTY
Harris 2 142179
Travis 1 931820
...
So far I have been able to use streams to get a list of distinct county names (as these are repetitive), but now I am having issues with getting the count of cities in a distinct county and consequently the sum of the populations of cities in these counties. I have read the file into an ArrayList of type texasCitiesClass, and my code thus far looks like:
public static void main(String[] args) throws FileNotFoundException, IOException {
PrintStream output = new PrintStream(new File("output.txt"));
ArrayList<texasCitiesClass> txcArray = new ArrayList<texasCitiesClass>();
initTheArray(txcArray); // this method will read the input file and populate an arraylist
System.setOut(output);
List<String> counties;
counties = txcArray.stream()
.filter(distinctByKey(txc -> txc.getCounty())) // grab distinct county names
.distinct() // redundant?
.sorted((txc1, txc2) -> txc1.getCounty().compareTo(txc2.getCounty())); // sort alphabetically
}
public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
Map<Object, String> seen = new ConcurrentHashMap<>();
return t -> seen.put(keyExtractor.apply(t), "") == null;
}
At this point, I have a stream containing the names of unique counties. Since the sorted() operator will return a new stream, how can I obtain (and thus sum) the population values for the counties?

Given the classes (ctor, getter, setter omitted)
class Foo {
String name;
String countyName;
int pop;
}
class Aggregate {
String name;
int count;
int pop;
}
You could aggregate your values by mapping them to Aggregate Objects using Collectors.toMap and merging them using its mergeFunction. Using the TreeMap, its entries are ordered by its key.
TreeMap<String, Aggregate> collect = foos.stream()
.collect(Collectors.toMap(
Foo::getCountyName,
foo -> new Aggregate(foo.countyName,1,foo.pop),
(a, b) -> new Aggregate(b.name, a.count + 1, a.pop + b.pop),
TreeMap::new)
);
Using
List<Foo> foos = List.of(
new Foo("A", "Harris", 44),
new Foo("C", "Travis ", 99),
new Foo("B", "Harris", 66)
);
the map is
{Harris=Aggregate{name='Harris', count=2, pop=110}, Travis =Aggregate{name='Travis ', count=1, pop=99}}