Do multi level grouping and summing using Java Stream API - java

I have a class
public class Person {
private String name;
private String country;
private String city;
private String pet;
private int totalCountryToCityCount;
private int petCount;
public Person(String name, String country, String city, String pet, int total, int petCount) {
this.name = name;
this.country = country;
this.city = city;
this.pet = pet;
this.totalCountryToCityCount = total;
this.petCount = petCount;
}
public String getName() {
return name;
}
public String getCountry() {
return country;
}
public String getCity() {
return city;
}
public String getPet() {
return pet;
}
public int getPetCount() {
return petCount;
}
public int getTotalCountryToCityCount() {
return totalCountryToCityCount;
}
}
and Given a list of Person class, I have do aggregations based upon the different properties of the class.
For eg -
Person person1 = new Person("John", "USA", "NYC", "Max", 1, 2);
Person person2 = new Person("Steve", "UK", "London", "Lucy", 2, 8);
Person person3 = new Person("Anna", "USA", "NYC", "Max", 4, 32);
Person person4 = new Person("Mike", "USA", "Chicago", "Duke", 5, 1);
Person person5 = new Person("Test", "INDIA", "HYD", "Tommy", 4, 32);
Person person6 = new Person("Test1", "INDIA", "HYD", "Tommy", 4, 65);
Person person7 = new Person("Tim", "USA", "Chicago", "Duke", 5, 111);
Person person8 = new Person("Tim", "USA", "Chicago", "Puke", 5, 111);
Person person9 = new Person("Test1", "INDIA", "DELHI", "Tommy", 4, 65);
List<Person> persons = Arrays
.asList(person1, person2, person3, person4, person5, person6, person7, person8,
person9);
Now I need to get a result such that I should get the total "totalCountryToCityCount" based upon the combinations of country and city and I should get total "petCount" based upon combinations of country,city and pet. I am able to get them separately using groupingBy and summingint
private Map<String, Map<String, Integer>> getTotalCountForCountry(List<Person> persons) {
return persons.stream().collect(groupingBy(Person::getCountry, getCityCount()));
}
public Collector<Person, ?, Map<String, Integer>> getCityCount() {
return groupingBy(Person::getCity, summingInt(Person::getTotal));
}
public Map<String, Map<String, Map<String, Integer>>> threeLevelGrouping(List<Person> persons) {
return persons
.stream().collect(
groupingBy(Person::getCountry, groupByCityAndPetName()
)
);
}
private Collector<Person, ?, Map<String, Map<String, Integer>>> groupByCityAndPetName() {
return groupingBy(Person::getCity, groupByPetName());
}
private Collector<Person, ?, Map<String, Integer>> groupByPetName() {
return groupingBy(Person::getPet, summingInt(Person::getPetCount));
}
which gives the result
{USA={Chicago={Puke=111, Duke=112}, NYC={Max=34}}, UK={London={Lucy=8}}, INDIA={DELHI={Tommy=65}, HYD={Tommy=97}}}
{USA={Chicago=15, NYC=5}, UK={London=2}, INDIA={DELHI=4, HYD=8}}
but the actual result which I want is :-
{USA={Chicago={15,{Puke=111, Duke=112}}, NYC={5,{Max=34} }, UK={London={2, {Lucy=8}}, INDIA={DELHI={4, {Tommy=65}}, , HYD={8,{Tommy=97}}}}
is there a way to achieve the same using Java stream API
I also tried using the code -
personList.stream().collect(groupingBy(person -> person.getCountry(), collectingAndThen(reducing(
(a, b) -> new Person(a.getName(), a.getCountry(), a.getCity(), a.getPet(),
a.getTotal() + b.getTotal(), a.getPetCount() + b.getPetCount())),
Optional::get)))
.forEach((country, person) -> System.out.println(country + person));
But was getting the result -
USAPerson{name='John', country='USA', city='NYC'}
UKPerson{name='Steve', country='UK', city='London'}
INDIAPerson{name='Test', country='INDIA', city='HYD'}
with the counts surprisingly removed

What you are looking for really is Collectors::teeing, but only available in java-12:
System.out.println(
persons.stream()
.collect(Collectors.groupingBy(
Person::getCountry,
Collectors.groupingBy(
Person::getCity,
Collectors.teeing(
Collectors.summingInt(Person::getTotalCountryToCityCount),
Collectors.groupingBy(
Person::getPet,
Collectors.summingInt(Person::getPetCount)
),
SimpleEntry::new
)
))));
A back-port for java-8 it is available here.

Related

How to map a Map into another Map using Java Stream?

I'm learning Java with Stream and I have this Map<String, List<Customer>> map1 and I want to map it into this Map<String, List<CustomerObj>> map2. How can I do this especially using Java Stream? Any feedback will be apreciated!
You can stream the entries of map1 and collect with Collectors.toMap, using key mapper Map.Entry::getKey. For the value mapper you can get the entry value list, then stream and map from Customer to CustomerObj, and finally collect to List.
Here's and example using Integer instead of Customer and String instead of CustomerObj:
Map<String, List<Integer>> map1 = Map.of(
"A", List.of(1, 2),
"B", List.of(3, 4),
"C", List.of(5, 6));
Map<String, List<String>> map2 = map1.entrySet().stream().collect(
Collectors.toMap(Map.Entry::getKey,
entry -> entry.getValue().stream()
.map(String::valueOf).toList()));
You may do it like so:
Customer c1 = new Customer("İsmail", "Y.");
Customer c2 = new Customer("Elvis", "?.");
Map<String, List<Customer>> customerList = new HashMap<>();
customerList.put("U1", Arrays.asList(c1));
customerList.put("U2", Arrays.asList(c1, c2));
Map<String, List<CustomerObj>> customerObjList = customerList
.entrySet().stream().collect(
Collectors.toMap(Map.Entry::getKey,
entry -> entry.getValue().stream()
.map(CustomerObj::new)
// or .map(c -> new CustomerObj(c))
.collect(Collectors.toList())
)
);
}
private static class Customer {
private String firstName;
private String lastName;
public Customer(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
public String getFirstName() {
return firstName;
}
public String getLastName() {
return lastName;
}
}
private static class CustomerObj {
private String fullName;
public CustomerObj(Customer customer) {
this.fullName = String.join(" ",
customer.getFirstName(), customer.getLastName()
);
}
public String getFullName() {
return fullName;
}
}
#Oboe's answer works in Java 16+ and this answer is similar to your models and easier to understand.

List of objects - get object fields distinct count

For a list of objects I have to check for (some) fields:
all objects having same value for that field
all objects having a different value for that field
class Person {
final String name;
final int age;
final int group;
public Person( final String name, final int age, final int group ) {
this.name = name;
this.age = age;
this.group = group;
}
public String getName() {
return this.name;
}
public int getAge() {
return this.age;
}
public int getGroup() {
return this.group;
}
}
public static <T> long distinctByField( final List<Person> personList, final Function<Person, T> field ) {
return personList.stream()
.map( field )
.distinct().count();
}
public static void main( final String[] args ) {
final List<Person> personList = Arrays.asList(
new Person( "Fred", 25, 1 ),
new Person( "Bill", 22, 1 ),
new Person( "Fred", 27, 1 ),
new Person( "Lisa", 25, 1 )
);
System.out.println( distinctByField( personList, Person::getName ) );
System.out.println( distinctByField( personList, Person::getAge ) );
System.out.println( distinctByField( personList, Person::getGroup ) );
}
With result of stream/distinct/count I can compare with current list size:
if count == 1 : all objects having same value for that field
if count == list.size : all objects having different value for that field
Drawback is, i have to stream for every interested field.
Is it possible to do this with one query (for a list of interested fields) ?
It's possible using reflection:
public class ReflectionTest {
class Person {
final String name;
final int age;
final int group;
public Person(final String name, final int age, final int group) {
this.name = name;
this.age = age;
this.group = group;
}
public String getName() {
return this.name;
}
public int getAge() {
return this.age;
}
public int getGroup() {
return this.group;
}
}
#DisplayName("should determine distinct fields")
#Test
public void distinct() {
final List<Person> personList = Arrays.asList(new Person("Fred", 25, 1),
new Person("Bill", 22, 1),
new Person("Fred", 27, 1),
new Person("Lisa", 25, 1));
Map<String,Long> fieldCountMap = Stream.of("name", "age", "group")
.map(fieldName -> ReflectionUtils.findField(Person.class, fieldName))
.filter(Objects::nonNull)
.collect(Collectors.toMap(Field::getName, field -> personList.stream().map(person -> getField(field, person)).distinct().count()));
assertEquals(3,fieldCountMap.get("name"));
assertEquals(1,fieldCountMap.get("group"));
assertEquals(3,fieldCountMap.get("age"));
}
//extracted into a method because field.get throws a checked exception
Object getField(Field field, Person object) {
try {
return field.get(object);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
}
Let me first mention: This is a more or less huge downside in code quality (searching manually through all fields, using an extra class to store the results). And I doubt, this would be more efficient in terms of computation time or memory. By logic you will have to touch every field of every person and store values that already occured in order to find the distinct count for every field. Which is exactly what your solution with 3 streams does. I'd advice you to stay with it.
But here is a solution. I built a collector, that collects in one run all the different values for each field into a custom class.
static class PersonStatistic {
Set<String> names = new HashSet<>();
Set<Integer> ages = new HashSet<>();
Set<Integer> groups = new HashSet<>();
}
public static void main(final String[] args) {
final List<Person> personList = Arrays.asList(
new Person("Fred", 25, 1),
new Person("Bill", 22, 1),
new Person("Fred", 27, 1),
new Person("Lisa", 25, 1));
PersonStatistic personStatistic = personList.stream().collect(
// Create new Statistic
PersonStatistic::new,
// Merge A Person into statistic
(statistic, person) -> {
statistic.names.add(person.name);
statistic.ages.add(person.age);
statistic.groups.add(person.group);
},
// Merge second statistic into first
(stat1, stat2)-> {
stat1.names.addAll(stat2.names);
stat1.ages.addAll(stat2.ages);
stat1.groups.addAll(stat2.groups);
});
System.out.println(personStatistic.names.size());
System.out.println(personStatistic.ages.size());
System.out.println(personStatistic.groups.size());
}

Implementing pivot table in Java

I need to implement a pivot table in Java and I know how to do with Java 8 Streams features. There are a lot of good solution over the web but I need something more and I don't understand how to do that: I need to create a more dynamic table where ideally you don't know for what columns you have to aggregate.
For example, if I have the columns ("Nation", "Company", "Industry","Number of employes") I have to give as input:
A custom aggregation function (ex. sum) for the measure
A variable order of aggregation: ex., I want first aggregate for Nation and I gave as argument "Nation" or for Nation and Company and I gave as argument something like "Nation->Company".
In other words, I don't know which are the fields for my aggregation and basically I need a way to implement a generic GROUP BY SQL clause, so something like:
// Given an the Arraylist ("Nation", "Company", "Industry","Number of employes") called data with some rows
Map<String, List<Object[]>> map = data.stream().collect(
Collectors.groupingBy(row -> row[0].toString() + "-" + row[1].toString()));
for (Map.Entry<String, List<Object[]>> entry : map.entrySet()) {
final double average = entry.getValue().stream()
.mapToInt(row -> (int) row[3]).average().getAsDouble();
It's not what I need because it is too explicit.
I need to:
Split the input Arraylist in sublist by value given by the header name which I extract from my data (or more, it depends for how many column I have to group by)
Aggregate each sublist
Union the sublist
Could someone help or boost me? Thanks
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
class Input {
private String nation, company, industry;
private int employees;
public Input(String nation, String company, String industry, int employees) {
super();
this.nation = nation;
this.company = company;
this.industry = industry;
this.employees = employees;
}
public String getNation() {
return nation;
}
public void setNation(String nation) {
this.nation = nation;
}
public String getCompany() {
return company;
}
public void setCompany(String company) {
this.company = company;
}
public String getIndustry() {
return industry;
}
public void setIndustry(String industry) {
this.industry = industry;
}
public int getEmployees() {
return employees;
}
public void setEmployees(int employees) {
this.employees = employees;
}
#Override
public String toString() {
return String.format(
"Nation : %s, Company : %s, Industry : %s, Employees : %s",
nation, company, industry, employees);
}
}
public class CustomGroupBy {
// Generic GroupBy
static Map<String, List<Input>> groupBy(List<Input> input,
Function<Input, String> classifier) {
return input.stream().collect(Collectors.groupingBy(classifier));
}
public static void main(String[] args) {
List<Input> input = Arrays.asList(new Input("India", "A", "IT", 12),
new Input("USA", "B", "ELECTRICAL", 90), new Input("India",
"B", "MECHANICAL", 122), new Input("India", "B", "IT",
12), new Input("India", "C", "IT", 200));
// You need to pass this in parameter
Function<Input, String> groupByFun = i -> i.getNation() + "-"
+ i.getCompany();
// Example-1
Map<String, List<Input>> groupBy = groupBy(input, Input::getCompany);
// Example-2
Map<String, List<Input>> groupBy2 = groupBy(input, groupByFun);
System.out.println(groupBy2);
List<Double> averages = groupBy
.entrySet()
.stream()
.map(entry -> entry.getValue().stream()
.mapToInt(row -> row.getEmployees()).average()
.getAsDouble()).collect(Collectors.toList());
System.out.println(averages);
}
}
You can make it generic by passing the functional interface. It is just for your reference.
I see two ways to make it generic. The first one is to use reflection to discover the method to call from the string representation of the field. The second one, is to create a generic get method that take a String in argument and return the value of the proper field. The second one is safer so I'll focus on that one. I'll start from the answer of #Anant Goswami which had done most of the work already.
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
class Scratch {
// Input class from Anant Goswami in previous reply
static class Input {
private String nation, company, industry;
private int employees;
public Input(String nation, String company, String industry, int employees) {
super();
this.nation = nation;
this.company = company;
this.industry = industry;
this.employees = employees;
}
public String getNation() {
return nation;
}
public void setNation(String nation) {
this.nation = nation;
}
public String getCompany() {
return company;
}
public void setCompany(String company) {
this.company = company;
}
public String getIndustry() {
return industry;
}
public void setIndustry(String industry) {
this.industry = industry;
}
public int getEmployees() {
return employees;
}
public void setEmployees(int employees) {
this.employees = employees;
}
#Override
public String toString() {
return String.format(
"Nation : %s, Company : %s, Industry : %s, Employees : %s",
nation, company, industry, employees);
}
public Object get(String field){
switch (field.toLowerCase()){
case "nation": return getNation();
case "company": return getCompany();
case "industry": return getIndustry();
case "employees": return getEmployees();
default: throw new UnsupportedOperationException();
}
}
}
private static Map<String, List<Input>> group(List<Input> inputs, String... fields){
Function<Input, String> groupBy = i -> Arrays.stream(fields).map(f -> i.get(f).toString()).collect(Collectors.joining("-"));
Map<String, List<Input>> result = inputs.stream().collect(Collectors.groupingBy(groupBy));
System.out.println(result);
return result;
}
public static void main(String[] args) {
List<Input> input = Arrays.asList(new Input("India", "A", "IT", 12),
new Input("USA", "B", "ELECTRICAL", 90), new Input("India",
"B", "MECHANICAL", 122), new Input("India", "B", "IT",
12), new Input("India", "C", "IT", 200));
group(input, "company");
group(input, "nation", "Company");
}
}
Which give as output
{A=[Nation : India, Company : A, Industry : IT, Employees : 12], B=[Nation : USA, Company : B, Industry : ELECTRICAL, Employees : 90, Nation : India, Company : B, Industry : MECHANICAL, Employees : 122, Nation : India, Company : B, Industry : IT, Employees : 12], C=[Nation : India, Company : C, Industry : IT, Employees : 200]}
{India-B=[Nation : India, Company : B, Industry : MECHANICAL, Employees : 122, Nation : India, Company : B, Industry : IT, Employees : 12], India-C=[Nation : India, Company : C, Industry : IT, Employees : 200], India-A=[Nation : India, Company : A, Industry : IT, Employees : 12], USA-B=[Nation : USA, Company : B, Industry : ELECTRICAL, Employees : 90]}

Java: Convert List of Object to Set of Map

Say I have a list of of Objects (say List<User>), something like
[
{
"name": "myName",
"age": 1,
"someField": "foo"
},
{
"name": "otherName",
"age": 2,
"someField": "bar"
},
]
I want to convert this to Set<Map<String, Integer>> such that I get a set of name => age pair. So final result should be [{"myName": 1},{"otherName": 2}]
How can I use the stream and collector to do this?
You can use below piece of code for parsing JSON String and then doing some manipulation with that :
public static void main(String[] args) {
String str = "[ { \"name\": \"myName\", \"age\": 1, \"someField\": \"foo\" }, { \"name\": \"otherName\", \"age\": 2, \"someField\": \"bar\" }, ]";
JSONArray jsonobj = new JSONArray(str);
Map<String,Integer> map = new HashMap<>();
for(int i = 0;i<jsonobj.length();i++){
JSONObject obj = jsonobj.getJSONObject(i);
map.put(obj.getString("name"), obj.getInt("age"));
}
Set<Map<String,Integer>> set = new HashSet<>();
set.add(map);
System.out.println(set);
}
Library used : org.json
I ended up doing this:
final List<User> users = new ArrayList<>();
users.add(user1);
users.add(user2);
users.add(user3);
Set<Map<String, Integer>> usersSet =
users
.stream()
.map(u -> Collections.singletonMap(u.getName(), u.getAge()))
.collect(Collectors.toSet());
Here is a possible conversion to Map<String, Integer> with duplicate handling:
final List<User> users = new ArrayList<>();
users.add(new User("name 1", 25));
users.add(new User("name 1", 49));
users.add(new User("name 2", 67));
final Map<String, Integer> nameToAge = users.stream().collect(
Collectors.toMap(
user -> user.name,
user -> user.age,
(user1, user2) -> {
System.out.println("Resolving duplicates...");
return user1;
})
);
In this case the definition for type User is:
public class User {
public final String name;
public final int age;
public User(final String name, final int age) {
this.name = name;
this.age = age;
}
}

Adding Multiple Set<String> in Java

I have two sets as: set1 and set2 that I want to combine.
set1 contains personID and place as: [1-NY, 2-CA, 3-MD, 1-TX, 3-VA]
set2 contains personName and place as: [John-NY, Bill-CA, Ron-CA, Rick-MD, John-TX, Rick-VA]
I want to combine both the set such that I will get the output of personID, personName and place as: [1-John-NY, 2-Bill-CA, 2-Ron-CA, 3-Rick-MD, 1-John-TX, 3-Rick-VA].
Basically the thing is: I want to use "place" as the anchor to combine.
Set<String> set1 = new LinkedHashSet<String>();
Set<String> set2 = new LinkedHashSet<String>();
Set<String> combination = new LinkedHashSet<String>();
combination.addAll(set1);
combination.addAll(set2);
But, I am not able to get the output in my expected way. Any suggestion please.
Thanks!
You should rethink your approach a bit. In order to combine these two sets you should create some kind of look-up table. I would use simple HashMap for this.
The code is really self-explanatory, but fell free to ask questions)
Using Java 8:
Set<String> personIds = new LinkedHashSet<>(Arrays.asList("1-NY", "2-CA", "3-MD", "1-TX", "3-VA"));
Set<String> personNames = new LinkedHashSet<>(Arrays.asList("John-NY", "Bill-CA", "Ron-CA", "Rick-MD", "John-TX", "Rick-VA"));
Map<String, String> personIdMap = personIds.stream().map(v -> v.split("-"))
.collect(Collectors.toMap(v -> v[1], v -> v[0]));
Set<String> combination = new LinkedHashSet<>();
personNames.forEach(name -> {
final String[] split = name.split("-");
final String personId = personIdMap.get(split[1]);
combination.add(personId + '-' + name);
});
Using Java 7:
Set<String> personIds = new LinkedHashSet<>(Arrays.asList("1-NY", "2-CA", "3-MD", "1-TX", "3-VA"));
Set<String> personNames = new LinkedHashSet<>(Arrays.asList("John-NY", "Bill-CA", "Ron-CA", "Rick-MD", "John-TX", "Rick-VA"));
Map<String, String> personIdMap = new HashMap<>();
for (String id : personIds) {
final String[] split = id.split("-");
personIdMap.put(split[1], split[0]);
}
Set<String> combination = new LinkedHashSet<>();
for (String name : personNames) {
final String[] split = name.split("-");
final String personId = personIdMap.get(split[1]);
combination.add(personId + '-' + name);
}
As user chrylis suggests, you could use class for this propose. First, create a class Person.class to store the required values: person ID / person name / place name. For simplifying the process, a constructor with 3 parameters is used here to construct the object, but it's not the only choice. By the way, I strongly suggest you to use a unique value for each person.
public Person(String id, String name, String place) {
this.id = id;
this.name = name;
this.place = place;
}
Then create a method to combine the different information stored in the person class.
public String getCombination() {
return String.format("%s-%s-%s", id, name, place);
}
Now you can put the data into the set combinations:
Set<Person> people = new LinkedHashSet<>();
people.add(new Person("1", "John", "NY"));
people.add(new Person("2", "Bill", "CA"));
people.add(new Person("2", "Ron", "CA"));
people.add(new Person("3", "Rick", "MD"));
people.add(new Person("1", "John", "TX"));
people.add(new Person("3", "Rick", "VA"));
Set<String> combinations = new LinkedHashSet<>();
for (Person p : people) {
combinations.add(p.getCombination());
}
Here's the full implementation of class Person.
public class Person {
private String id; // maybe place id ?
private String name;
private String place;
public Person(String id, String name, String place) {
this.id = id;
this.name = name;
this.place = place;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getPlace(String place) {
return place;
}
public void setPlace(String place) {
this.place = place;
}
public String getCombination() {
return String.format("%s-%s-%s", id, name, place);
}
}

Categories