We Keep Coding

Group by object property in java flux

Given the following data structure Data and Flux<Data> what is idiomatic way to achieve grouping into series of lists based on some property:
import org.reactivestreams.Publisher;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Function;
import java.util.function.Predicate;
class Scratch {
private static class Data {
private Integer key;
private String value;
public Data(Integer key, String value) {
this.key = key;
this.value = value;
}
public Integer getKey() {
return key;
}
public String getValue() {
return value;
}
public static Data of(Integer key, String value) {
return new Data(key, value);
}
#Override
public String toString() {
return value;
}
}
public static void main(String[] args) {
Flux<Data> test = Flux.just(
Data.of(1, "Hello"),
Data.of(1, "world"),
Data.of(2, "How"),
Data.of(2, "are"),
Data.of(2, "you"),
Data.of(3, "Bye"));
test.bufferUntil(new Predicate<Data>() {
Integer prev = null;
#Override
public boolean test(Data next) {
boolean collect = prev != null && !Objects.equals(prev, next.getKey());
prev = next.getKey();
return collect;
}
}, true).subscribe(e -> System.out.println(e.toString()));
}
}
Output:
[Hello, world]
[How, are, you]
[Bye]
I am aware of groupBy function on Flux, but this gives me again a Flux, not a list. Current solution I have described above works, but it does not feel 100% idiomatic because I had to use anonymous class instead of lambda. I could have use lambda and AtomicReference outside from lambda, but that too does not feel 100% right. Any suggestions?

You can also use collectMultimap which allows you to have Map<K, Collection<T>. In this case collectMultimap will return: Mono<Map<Integer,Collection<Data>>>:
test.collectMultimap( Data::getKey )
.subscribe( dataByKey -> System.out.println( dataByKey.toString() ) );
Output:
{1=[Hello, world], 2=[How, are, you], 3=[Bye]}

Here is a solution using groupBy operator. I have grouped the data by the common key. The groupBy operator gives me a Flux of GroupedFlux. GroupedFlux is a subclass of Flux, so I apply flatMap and convert an individual groupedFlux to a List<Data> using the collectList operator. Like this, I get a Flux<List<Data>>, which I then subscribe to and print, as asked by you.
test.groupBy(Data::getKey)
.flatMap(Flux::collectList)
.subscribe(listOfStringsHavingDataWithSameKey -> System.out.println(listOfStringsHavingDataWithSameKey.toString()));
Do checkout the documentations for Flux and GroupedFlux.

Is it possible to group elements without closing the stream?

Is it possible to group elements in a Stream, but then continue streaming instead of having to create a new stream from the EntrySet of the returned map?
For example, I can do this:
public static void main(String[] args) {
// map of access date to list of users
// Person is a POJO with first name, last name, etc.
Map<Date, List<Person>> dateMap = new HashMap<>();
// ...
// output, sorted by access date, then person last name
dateMap.entrySet().stream().sorted(Map.Entry.comparingByKey()).forEach(e -> {
Date date = e.getKey();
// group persons by last name and sort
// this part seems clunky
e.getValue().stream().collect(Collectors.groupingBy(Person::getLastName, Collectors.toSet()))
.entrySet().stream().sorted(Map.Entry.comparingByKey()).forEach(e2 -> {
// pool agent id is the key
String lastName = e2.getKey();
Set<Person> personSet = e2.getValue();
float avgAge = calculateAverageAge(personSet);
int numPersons = personSet.size();
// write out row with date, lastName, avgAge, numPersons
});
});
}
Which works just fine, but seems a little clunky, especially the streaming into a map, and then immediately streaming on the entry set of that map.
Is there a way to group objects in a stream, but continue streaming?

You can shorten your code by using Map.forEach, downstream collectors, TreeMap, and IntSummaryStatistics.
By grouping into a TreeMap (instead of leaving it up to the groupingBy collector), you get the names sorted automatically. Instead of immediately getting the grouped map, you add a summarizingInt collector that turns the list of persons with the same name into IntSummaryStatistics of their ages.
public static void main(String[] args) {
Map<Date, List<Person>> dateMap = new HashMap<>();
dateMap.entrySet().stream().sorted(Map.Entry.comparingByKey()).forEach(e -> {
Date date = e.getKey();
e.getValue().stream()
.collect(Collectors.groupingBy(Person::getLastName,
TreeMap::new,
Collectors.summarizingInt(Person::getAge)))
.forEach((name, stats) -> System.out.println(date +" "+
lastName +" "+
stats.getAverage() +" "+
stats.getCount()));
});
}
If you have control over the type of the initial map, you could use TreeMap there as well, and shorten it further:
public static void main(String[] args) {
Map<Date, List<Person>> dateMap = new TreeMap<>();
dateMap.forEach((date, persons -> { ...

There are several different ways to interpret the question, but if we restate the question as, "Is it possible to group elements within a Stream without using a terminal operation and apply stream operations to the resulting groups within the same stream pipeline," then the answer is "Yes." In this restatement of the question, terminal operation is defined in the way that the Java 8 streams API defines it.
Here is an example that demonstrates this.
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Random;
import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
class StreamGrouper {
public static class GroupableObj<K extends Comparable<? super K>, T>
implements Comparable<GroupableObj<K, T>> {
private K key;
private T obj;
private Set<T> setOfObj;
public GroupableObj(K key, T obj) {
if (key == null) {
throw new NullPointerException("Key may not be null");
}
this.key = key;
this.obj = obj;
}
#Override
public int compareTo(GroupableObj<K, T> otherGroupable) {
return key.compareTo(otherGroupable.key);
}
#Override
public boolean equals(Object otherObj) {
if (otherObj == null) {
return false;
}
if (otherObj instanceof GroupableObj) {
GroupableObj<?, ?> otherGroupable =
(GroupableObj<?, ?>)otherObj;
return setOfObj == otherGroupable.setOfObj &&
key.equals(otherGroupable.key);
}
return false;
}
public Set<T> getGroup() {
return setOfObj;
}
public K getKey() {
return key;
}
public T getObject() {
return obj;
}
#Override
public int hashCode() {
return key.hashCode();
}
public void setGroup(Set<T> setOfObj) {
this.setOfObj = setOfObj;
}
}
public static class PeekGrouper<K extends Comparable<? super K>, T>
implements Consumer<GroupableObj<K, T>> {
private Map<K, Set<T>> groupMap;
public PeekGrouper() {
groupMap = new HashMap<>();
}
#Override
public void accept(GroupableObj<K, T> groupable) {
K key = groupable.getKey();
Set<T> group = groupMap.computeIfAbsent(key,
(k) -> new HashSet<T>());
groupable.setGroup(group);
group.add(groupable.getObject());
}
}
public static void main(String[] args) {
Function<Double, Long> myKeyExtractor =
(dblObj) -> Long.valueOf(
(long)(Math.floor(dblObj.doubleValue()*10.0)));
PeekGrouper<Long, Double> myGrouper = new PeekGrouper<>();
Random simpleRand = new Random(20190527L);
simpleRand.doubles(100).boxed().map((dblObj) ->
new GroupableObj<Long, Double>(
myKeyExtractor.apply(dblObj), dblObj)).peek(myGrouper).
distinct().sorted().
map(GroupableObj<Long, Double>::getGroup).
forEachOrdered((grp) -> System.out.println(grp));
}
}
In order to make a program that can be compiled and executed on its own, this example moves away from using the Person objects that are referenced in the question, but the grouping concept is the same, and the code from the question could turn into something like the following.
PeekGrouper<String, Person> myGrouper = new PeekGrouper<>();
e.getValue().stream().map((p) -> new GroupableObj<String, Person>(
p.getLastName(), p)).peek(myGrouper).distinct().sorted().
forEachOrdered(e2 -> {
String lastName = e2.getKey();
Set<Person> personSet = e2.getGroup();
float avgAge = calculateAverageAge(personSet);
int numPersons = personSet.size();
// write out row with date, lastName, avgAge, numPersons
});
Please note that in order for this example to work, it is required that the stream call both the distinct function (which reduces the stream to only a single instance of each group) and the sorted function (which ensures that the entire stream has been processed and the groups have been fully "collected" before processing continues). Also note that as implemented here GroupableObj is not safe to use with parallel streams. If the terminal operation of the stream does not require that the groups be fully "collected" when it processes the objects -- for example, if the terminal operation were something like Collectors.toList() -- then a call to sorted would not be required. The critical point is that any portion of the stream that sees the groups prior to a call to sorted and prior to the end of a terminal operation (including processing during a terminal operation) may see a group that is incomplete.
For the specific example in the question, it may be somewhat less time-efficient to sort the objects before grouping them if many of them are in the same group, but if you are willing to sort the objects before grouping them, you can achieve the same functionality without performing any streaming after doing the grouping. The following is a rewrite of the first example from this answer that demonstrates this.
import java.util.Comparator;
import java.util.HashSet;
import java.util.Random;
import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collector;
class PreSortOrderedGrouper {
public static void main(String[] args) {
Function<Double, Long> myKeyExtractor =
(dblObj) -> Long.valueOf(
(long)(Math.floor(dblObj.doubleValue()*10.0)));
Random simpleRand = new Random(20190527L);
Consumer<Set<Double>> groupProcessor =
(grp) -> System.out.println(grp);
simpleRand.doubles(100).boxed().sorted(
Comparator.comparing(myKeyExtractor)).
collect(Collector.of(HashSet<Double>::new,
(set, dblObj) -> {
if (set.isEmpty() || myKeyExtractor.apply(set.iterator().
next()) == myKeyExtractor.apply(dblObj)) {
set.add(dblObj);
} else {
groupProcessor.accept(set);
set.clear();
set.add(dblObj);
}
},
(setOne, setTwo) -> {
throw new UnsupportedOperationException();
},
(finalSet) -> {
groupProcessor.accept(finalSet);
return Integer.valueOf(0);
}));
}
}
I can't be sure that either of these examples will feel less "clunky" to you, but if the example in your question is a pattern you use frequently, you could probably adapt one or both of these examples in ways that will suit your purposes and, aside from a few utility classes, result in no more code than you are currently using.

method returning either a collection or a single value

I have a class with various properties and I would like to write a wrapper method around them in order to loop around them more easily.
Some properties return a collection of values, some a single value. And I'm looking for the best approach for this.
My first approach is to let the wrapper method return whatever the property getters return.
public class Test {
public Object getValue(String propName) {
if ("attr1".equals(propName)) return getAttribute1();
else if ("attr2".equals(propName)) return getAttribute2();
else return null;
}
public List<String> getAttribute1() {
return Arrays.asList("Hello","World");
}
public String getAttribute2() {
return "Goodbye";
}
public static void main(String[] args) {
final Test test=new Test();
Stream.of("attr1","attr2")
.forEach(p-> {
Object o=test.getValue(p);
if (o instanceof Collection) {
((Collection) o).forEach(v->System.out.println(v));
}
else {
System.out.println(o);
}
});
}
}
The bad point with this approach is that the caller has to test himself whether the result is a collection or not.
Other approach, seamless for the caller, is to always return a collection, ie. the wrapper function wraps the single values into a Collection. Here an HashSet, but we can imagine an adhoc, minimum 1 element list.
public class TestAlt {
public Collection getValue(String propName) {
if ("attr1".equals(propName))
return getAttribute1();
else if ("attr2".equals(propName)) {
Set s = new HashSet();
s.add(getAttribute2());
return s;
}
else
return null;
}
public List<String> getAttribute1() {
return Arrays.asList("Hello", "World");
}
public String getAttribute2() {
return "Goodbye";
}
public static void main(String[] args) {
final TestAlt test = new TestAlt();
Stream.of("attr1", "attr2")
.forEach(p -> {
test.getValue(p).forEach(v -> System.out.println(v));
});
}
Performance-wise, design-wise, ... what's your opinion on these approaches ? Do you have better ideas ?

Well, you could pass the action to be performed on each attribute to the object and let the object decide on how to handle it. E.g.:
in Class Test:
public void forEachAttribute(String propName, Handler h) {
if ("attr1".equals(propName))
h.handle(getAttribute1());
else if ("attr2".equals(propName)) {
getAttribute2().forEach(o -> h.handle(o))
}
}
And a class Handler with the function handle(String s), that does, what you want to do.
If you cannot edit Test, you can also move the function outside Test
public void forEachTestAttribute(Test t, String propName, Handler h)...
Performance-wise: This removes an if-clause
Design-wise: This removes a cast, but creates more classes.
*Edit: It also maintains type-security, and if there are multiple kinds of attributes (String, int, etc.) you could add more handle-functions, to still maintain type-security.

Regarding the design I would rewrite your code into this:
TestAlt.java
import java.util.*;
import java.util.stream.Stream;
public class TestAlt {
private Map<String, AttributeProcessor> map = AttributeMapFactory.createMap();
public Collection getValue(String propName) {
return Optional
.ofNullable(map.get(propName))
.map(AttributeProcessor::getAttribute)
.orElse(Arrays.asList("default")); //to avoid unexpected NPE's
}
public static void main(String[] args) {
final TestAlt test = new TestAlt();
Stream.of("attr1", "attr2")
.forEach(p -> test.getValue(p).forEach(v -> System.out.println(v)));
}
}
AttributeMapFactory.java
import java.util.HashMap;
import java.util.Map;
public class AttributeMapFactory {
public static Map<String, AttributeProcessor> createMap() {
Map<String, AttributeProcessor> map = new HashMap<>();
map.put("attr1", new HiAttributeProcessor());
map.put("attr2", new ByeAttributeProcessor());
return map;
}
}
AttributeProcessor.java
import java.util.Collection;
public interface AttributeProcessor {
Collection<String> getAttribute();
}
HiAttributeProcessor.java
import java.util.Arrays;
import java.util.Collection;
public class HiAttributeProcessor implements AttributeProcessor{
#Override
public Collection<String> getAttribute() {
return Arrays.asList("Hello", "World");
}
}
ByeAttributeProcessor.java
import java.util.Arrays;
import java.util.Collection;
public class ByeAttributeProcessor implements AttributeProcessor{
#Override
public Collection<String> getAttribute() {
return Arrays.asList("Goodbye");
}
}
The main point is that you get rid of if-else statements using map and dynamic dispatch.
The main advantage of this approach is that your code becomes more flexible to further changes. In case of this small programm it does not really matter and is an overkill. But if we are talking about large enterprise application, then yes, it becomes crucial.

How can we pass a variable field /method name in Comparator.comparing

I have a Report {String name, Date date, int score } class.
I want to be able to sort a list of reports for any member variable using the new java 8 syntax
So java 8 provides this new
list.sort(Comparator.comparing(report -> report.name))
to sort the list on name.
Lets say instead of name I want to provide a variable field name to this method eg. something like
list.sort(Comparator.comparing(report -> report.anyField))
where anyField can be name or date or score. How do I achieve this behavior.

Just create a comparator for each property.
static Map<String,Comparator<Report>> ORDER;
static {
HashMap<String,Comparator<Report>> m=new HashMap<>();
m.put("name", Comparator.comparing(r -> r.name));
m.put("date", Comparator.comparing(r -> r.date));
m.put("score", Comparator.comparingInt(r -> r.score));
ORDER=Collections.unmodifiableMap(m);
}
public static void sort(List<Report> list, String order) {
Comparator<Report> c=ORDER.get(order);
if(c==null) throw new IllegalArgumentException(order);
list.sort(c);
}
You may consider using an enum as alternative to String, which eliminates the possibility of providing a non-existent property name:
enum ReportOrder {
NAME(Comparator.comparing(r -> r.name)),
DATE(Comparator.comparing(r -> r.date)),
SCORE(Comparator.comparingInt(r -> r.score));
private Comparator<Report> cmp;
private ReportOrder(Comparator<Report> c) { cmp=c; }
public void sort(List<Report> list) {
list.sort(cmp);
}
}
Now you can just say, e.g. ReportOrder.NAME.sort(list);. Of course, the other delegation style works as well:
public static void sort(List<Report> list, ReportOrder o) {
list.sort(o.cmp);
}
 
sort(list, ReportOrder.DATE);

One very generic solution is to use Java's Reflection and some casting:
String sortBy = "name";
list.sort(Comparator.comparing(report -> {
try {
return (Comparable) report.getClass().getDeclaredField(sortBy).get(report);
} catch (Exception e) {
throw new RuntimeException("Ooops", e);
}
}));
If you use an additional library like https://github.com/jOOQ/jOOR the code becomes even simpler:
String sortBy = "score";
list.sort(Comparator.comparing(report -> Reflect.on(report).field(sortBy).get()));
Please be aware that this solution only works with fields that implement Comparable and that it has some runtime overhead.

public class Report {
//properties and getters
public static void sort(List<Report> list, Function<Report, Comparable> sortKey) {
list.sort(Comparator.comparing(sortKey));
}
}
Report.sort(reports, Report::getName);
Report.sort(reports, Report::getDate);
Report.sort(reports, Report::getScore);
Could make this into a generic util class where you can pass in a List of any class:
public class MyUtil<T> {
void sort(List<T> t, Function<T, Comparable> sortKey) {
t.sort(Comparator.comparing(sortKey));
}
}
MyUtil<Report> util = new MyUtil();
util.sort(ppl, Report::getName);

If your Report has getter method of various field you can do like this
list.sort(Comparator.comparing(Report::getFieldName));
Or with lambda expression
list.sort((ob1, ob2) -> ob1.getFieldName().compareTo(ob2.getFieldName()));

At some place, you'll have to use a bit of reflection to pull this off. Here's an example using the bean introspection mechanism of java:
public static class MyBean {
private String name;
private Date birthday;
private Integer score;
...
... (constructor, getters, setters - the usual bean stuff)
}
PropertyDescriptor[] pdesc = Introspector.getBeanInfo(MyBean.class).getPropertyDescriptors();
for(PropertyDescriptor pd : pdesc) {
if(Comparable.class.isAssignableFrom(pd.getPropertyType())) {
System.out.println("Property " + pd.getName() + " could be used for comparison");
Function<MyBean, Comparable> extractor = b -> {
try {
return (Comparable) pd.getReadMethod().invoke(b);
}
catch(Exception e) {
throw new RuntimeException(e);
}
};
Comparator<MyBean> comp = Comparator.comparing(extractor);
// do something useful with the comparator :-)
}
}
Additionally, you could go a similar way for primitive types (e.g. int, which does not implement Comparable as opposed to Integer.)

If the set of properties is fixed (name, date, score) then I think a clean way would be to pass a Comparator:
private void someMethod(Comparator<Report> comparator){
...
list.sort(comparator);
...
}
...
someMethod(Comparator.comparing(report::getName));
someMethod(Comparator.comparing(report::getDate));
someMethod(Comparator.comparingInt(report::getScore));
someMethod(Comparator.comparing(report::getName).thenComparingInt(report::getScore));

There is a Comparator class called NamedMethodComparator that will work as a Comparator for any zero arg method that returns a Comparable posted here: How to use Comparator in Java to sort

Java sorting List

I got a List in java. I get values from a SQL query.
public void ReloadPages() throws Exception {
try (Connection conn = Framework.GetDatabaseManager().GetBone().getConnection()) {
try (ResultSet set = conn.createStatement().executeQuery("SELECT * FROM habbo_shop_pages")) {
while (set.next()) {
int Id = set.getInt(1);
Pages.put(Id, new CatalogPage(set));
}
}
}
System.out.println("Loaded " + Pages.size() + " Catalog Page(s).");
}
Then I store it all. In another function, I want to retrieve certain pages from a parentid.
public LinkedList<CatalogPage> getSubPages(int parentId) {
LinkedList<CatalogPage> pages = new LinkedList<>();
for (CatalogPage page : this.Pages.values()) {
if (page.getParentId() != parentId) {
continue;
}
pages.add(page);
}
return pages;
}
How do I order the list? Now id 4 is above in the shop and 1 at the bottom, but I want it ordered by id. ORDER BY in query doesn't work.

Have your class implement Comparable and provide said sort ordering in compareTo method. And to sort, simply use Collections#sort, although be aware that it's an inline sort.

You want to sort the list in reverse order then try this:
Collections.sort(list,Collections.reverseOrder());

Like mre writes, you can let the elements in the list implement the Comparable interface. Alternatively the Collections#sort(List, Comparator) method, which allows you to sort on different keys, can be used. This can for example be used handle sorting on multiple fields of the elements in the list, e.g. implement a Comparator to sort on date and a different Comparator for sorting on id.
For more info look at the javadoc for the Comparator interface.
Example
Code:
package com.stackoverflow.questions;
import java.util.Collections;
import java.util.Comparator;
import java.util.LinkedList;
public class Question15586200 {
public static void main(String[] args) {
// Add test data
LinkedList<CatalogPage> catalogs = new LinkedList<CatalogPage>();
catalogs.add(new CatalogPage("foo 4", 4));
catalogs.add(new CatalogPage("bar 1", 1));
catalogs.add(new CatalogPage("foobar 2", 2));
catalogs.add(new CatalogPage("barfoo 3", 3));
// Sort by id
Collections.sort(catalogs, new Comparator<CatalogPage>() {
#Override
public int compare(CatalogPage o1, CatalogPage o2) {
return Integer.compare(o1.getId(), o2.getId());
}
});
// Print result
for (CatalogPage page : catalogs) {
System.err.println(page);
}
}
public static class CatalogPage {
private String name;
private int id;
public CatalogPage(String name, int id) {
this.name = name;
this.id = id;
}
public int getId() {
return id;
}
public String getName() {
return name;
}
#Override
public String toString() {
return "CatalogPage [name=" + name + ", id=" + id + "]";
}
}
}
Ouput:
CatalogPage [name=bar 1, id=1]
CatalogPage [name=foobar 2, id=2]
CatalogPage [name=barfoo 3, id=3]
CatalogPage [name=foo 4, id=4]

You can do it using Lambda expressions, added in Java version 8.
Comparator<ProductColor> byProductColorName = (color1, color2) -> Integer.compare(
color1.getId(), color2.getId());
myProductColorList.stream().sorted(byProductColorName)
.forEach(e -> System.out.println(e));
Also i found this post very usefull.

List maintains insertion order. If you want to order in by ID, then use TreeSet and write a external Comparator, by implementing a compare() method of Comparator interface and pass it to the TreeSet constructor. Values will be sorted and ordered by ID.