We Keep Coding

How to merge two groupingBy in java streams?

I have a input object
#Getter
class Txn {
private String hash;
private String withdrawId;
private String depositId;
private Integer amount;
private String date;
}
and the output object is
#Builder
#Getter
class UserTxn {
private String hash;
private String walletId;
private String txnType;
private Integer amount;
}
In the Txn object transfers the amount from the withdrawId -> depositId.
what I am doing is I am adding all the transactions (Txn objects) in a single amount grouped by hash.
but for that I have to make two streams for groupingby withdrawId and second or for depositId and then the third stream for merging them
grouping by withdrawId
var withdrawStream = txnList.stream().collect(Collectors.groupingBy(Txn::getHash, LinkedHashMap::new,
Collectors.groupingBy(Txn::getWithdrawId, LinkedHashMap::new, Collectors.toList())))
.entrySet().stream().flatMap(hashEntrySet -> hashEntrySet.getValue().entrySet().stream()
.map(withdrawEntrySet ->
UserTxn.builder()
.hash(hashEntrySet.getKey())
.walletId(withdrawEntrySet.getKey())
.txnType("WITHDRAW")
.amount(withdrawEntrySet.getValue().stream().map(Txn::getAmount).reduce(0, Integer::sum))
.build()
));
grouping by depositId
var depositStream = txnList.stream().collect(Collectors.groupingBy(Txn::getHash, LinkedHashMap::new,
Collectors.groupingBy(Txn::getDepositId, LinkedHashMap::new, Collectors.toList())))
.entrySet().stream().flatMap(hashEntrySet -> hashEntrySet.getValue().entrySet().stream()
.map(withdrawEntrySet ->
UserTxn.builder()
.hash(hashEntrySet.getKey())
.walletId(withdrawEntrySet.getKey())
.txnType("DEPOSIT")
.amount(withdrawEntrySet.getValue().stream().map(Txn::getAmount).reduce(0, Integer::sum))
.build()
));
then merging them again, using deposites - withdraws
var res = Stream.concat(withdrawStream, depositStream).collect(Collectors.groupingBy(UserTxn::getHash, LinkedHashMap::new,
Collectors.groupingBy(UserTxn::getWalletId, LinkedHashMap::new, Collectors.toList())))
.entrySet().stream().flatMap(hashEntrySet -> hashEntrySet.getValue().entrySet().stream()
.map(withdrawEntrySet -> {
var depositAmount = withdrawEntrySet.getValue().stream().filter(userTxn -> userTxn.txnType.equals("DEPOSIT")).map(UserTxn::getAmount).reduce(0, Integer::sum);
var withdrawAmount = withdrawEntrySet.getValue().stream().filter(userTxn -> userTxn.txnType.equals("WITHDRAW")).map(UserTxn::getAmount).reduce(0, Integer::sum);
var totalAmount = depositAmount-withdrawAmount;
return UserTxn.builder()
.hash(hashEntrySet.getKey())
.walletId(withdrawEntrySet.getKey())
.txnType(totalAmount > 0 ? "DEPOSIT": "WITHDRAW")
.amount(totalAmount)
.build();
}
));
My question is, How can I do this in one stream.
Like by somehow groupingBy withdrawId and depositId is one grouping.
something like
res = txnList.stream()
.collect(Collectors.groupingBy(Txn::getHash,
LinkedHashMap::new,
Collectors.groupingBy(Txn::getWithdrawId && Txn::getDepositId,
LinkedHashMap::new, Collectors.toList())))
.entrySet().stream().flatMap(hashEntrySet -> hashEntrySet.getValue().entrySet().stream()
.map(walletEntrySet ->
{
var totalAmount = walletEntrySet.getValue().stream().map(
txn -> Objects.equals(txn.getDepositId(), walletEntrySet.getKey())
? txn.getAmount() : (-txn.getAmount())).reduce(0, Integer::sum);
return UserTxn.builder()
.hash(hashEntrySet.getKey())
.walletId(walletEntrySet.getKey())
.txnType("WITHDRAW")
.amount(totalAmount)
.build();
}
));

TL;DR
For those who didn't understand the question, OP wants to generate from each Txn instance (Txn probably stands for transaction) two peaces of data: hash and withdrawId + aggregated amount, and hash and depositId + aggregated amount.
And then they want to merge the two parts together (for that reason they were creating the two streams, and then concatenating them).
Note: it seems like there's a logical flow in the original code: the same amount gets associated with withdrawId and depositId. Which doesn't reflect that this amount has been taken from one account and transferred to another. Hence, it would make sense if for depositId amount would be used as is, and for withdrawId - negated (i.e. -1 * amount).
Collectors.teeing()
You can make use of the Java 12 Collector teeing() and internally group stream elements into two distinct Maps:
the first one by grouping the stream data by withdrawId and hash.
and another one by grouping the data depositId and hash.
Teeing expects three arguments: 2 downstream Collectors and a Function combining the results produced by collectors.
As the downstream of teeing() we can use a combination of Collectors groupingBy() and summingInt(), the second one is needed to accumulate integer amount of the transaction.
Note that there's no need in using nested Collector groupingBy() instead we can create a custom type that would hold id and hash (and its equals/hashCode should be implemented based on the wrapped id and hash). Java 16 record fits into this role perfectly well:
public record HashWalletId(String hash, String walletId) {}
Instances of HashWalletId would be used as Keys in both intermediate Maps.
The finisher function of teeing() would merge the results of the two Maps together.
The only thing left is to generate instances of UserTxn out of map entries.
List<Txn> txnList = // initializing the list
List<UserTxn> result = txnList.stream()
.collect(Collectors.teeing(
Collectors.groupingBy(
txn -> new HashWalletId(txn.getHash(), txn.getWithdrawId()),
Collectors.summingInt(txn -> -1 * txn.getAmount())), // because amount has been withdrawn
Collectors.groupingBy(
txn -> new HashWalletId(txn.getHash(), txn.getDepositId()),
Collectors.summingInt(Txn::getAmount)),
(map1, map2) -> {
map2.forEach((k, v) -> map1.merge(k, v, Integer::sum));
return map1;
}
))
.entrySet().stream()
.map(entry -> UserTxn.builder()
.hash(entry.getKey().hash())
.walletId(entry.getKey().walletId())
.txnType(entry.getValue() > 0 ? "DEPOSIT" : "WITHDRAW")
.amount(entry.getValue())
.build()
)
.toList(); // remove the terminal operation if your goal is to produce a Stream

I wouldn’t use this in my code because I think it’s not readable and will be very hard to change and manage in the future(SOLID).
But in case you still want this-
If I got your design right hash is unique per user and transaction will only have deposit or withdrawal, if so, this will work-
You could triple groupBy via collectors chaining like you did in your example.
You can create the Txn type via simple map function just check which id is null.
Map<String, Map<String, Map<String, List<Txn>>>> groupBy =
txnList.stream()
.collect(Collectors.groupingBy(Txn::getHash, LinkedHashMap::new,
Collectors.groupingBy(Txn::getDepositId, LinkedHashMap::new,
Collectors.groupingBy(Txn::getWithdrawId, LinkedHashMap::new, Collectors.toList()))));
then use the logic from your example on this stream.

Converting array iteration to lambda function using Java8

I am trying to convert to Lambda function
So far I am able to convert the above code to lambda function like as shown below
Stream.of(acceptedDetails, rejectedDetails)
.filter(list -> !isNull(list) && list.length > 0)
.forEach(new Consumer<Object>() {
public void accept(Object acceptedOrRejected) {
String id;
if(acceptedOrRejected instanceof EmployeeValidationAccepted) {
id = ((EmployeeValidationAccepted) acceptedOrRejected).getId();
} else {
id = ((EmployeeValidationRejected) acceptedOrRejected).getAd().getId();
}
if(acceptedOrRejected instanceof EmployeeValidationAccepted) {
dates1.add(new Integer(id.split("something")[1]));
Integer empId = Integer.valueOf(id.split("something")[2]);
empIds1.add(empId);
} else {
dates2.add(new Integer(id.split("something")[1]));
Integer empId = Integer.valueOf(id.split("something")[2]);
empIds2.add(empId);
}
}
});
But still my goal was to avoid repeating the same logic and also to convert to Lambda function, still in my converted lambda function I feel its not clean and efficient.
This is just for my learning aspect I am doing this stuff by taking one existing code snippet.
Can anyone please tell me how can I improvise the converted Lambda function

Generally, when you try to refactor code, you should only focus on the necessary changes.
Just because you’re going to use the Stream API, there is no reason to clutter the code with checks for null or empty arrays which weren’t in the loop based code. Neither should you change BigInteger to Integer.
Then, you have two different inputs and want to get distinct results from each of them, in other words, you have two entirely different operations. While it is reasonable to consider sharing common code between them, once you identified identical code, there is no sense in trying to express two entirely different operations as a single operation.
First, let’s see how we would do this for a traditional loop:
static void addToLists(String id, List<Integer> empIdList, List<BigInteger> dateList) {
String[] array = id.split("-");
dateList.add(new BigInteger(array[1]));
empIdList.add(Integer.valueOf(array[2]));
}
List<Integer> empIdAccepted = new ArrayList<>();
List<BigInteger> dateAccepted = new ArrayList<>();
for(EmployeeValidationAccepted acceptedDetail : acceptedDetails) {
addToLists(acceptedDetail.getId(), empIdAccepted, dateAccepted);
}
List<Integer> empIdRejected = new ArrayList<>();
List<BigInteger> dateRejected = new ArrayList<>();
for(EmployeeValidationRejected rejectedDetail : rejectedDetails) {
addToLists(rejectedDetail.getAd().getId(), empIdRejected, dateRejected);
}
If we want to express the same as Stream operations, there’s the obstacle of having two results per operation. It truly took until JDK 12 to get a built-in solution:
static Collector<String,?,Map.Entry<List<Integer>,List<BigInteger>>> idAndDate() {
return Collectors.mapping(s -> s.split("-"),
Collectors.teeing(
Collectors.mapping(a -> Integer.valueOf(a[2]), Collectors.toList()),
Collectors.mapping(a -> new BigInteger(a[1]), Collectors.toList()),
Map::entry));
}
Map.Entry<List<Integer>, List<BigInteger>> e;
e = Arrays.stream(acceptedDetails)
.map(EmployeeValidationAccepted::getId)
.collect(idAndDate());
List<Integer> empIdAccepted = e.getKey();
List<BigInteger> dateAccepted = e.getValue();
e = Arrays.stream(rejectedDetails)
.map(r -> r.getAd().getId())
.collect(idAndDate());
List<Integer> empIdRejected = e.getKey();
List<BigInteger> dateRejected = e.getValue();
Since a method can’t return two values, this uses a Map.Entry to hold them.
To use this solution with Java versions before JDK 12, you can use the implementation posted at the end of this answer. You’d also have to replace Map::entry with AbstractMap.SimpleImmutableEntry::new then.
Or you use a custom collector written for this specific operation:
static Collector<String,?,Map.Entry<List<Integer>,List<BigInteger>>> idAndDate() {
return Collector.of(
() -> new AbstractMap.SimpleImmutableEntry<>(new ArrayList<>(), new ArrayList<>()),
(e,id) -> {
String[] array = id.split("-");
e.getValue().add(new BigInteger(array[1]));
e.getKey().add(Integer.valueOf(array[2]));
},
(e1, e2) -> {
e1.getKey().addAll(e2.getKey());
e1.getValue().addAll(e2.getValue());
return e1;
});
}
In other words, using the Stream API does not always make the code simpler.
As a final note, we don’t need to use the Stream API to utilize lambda expressions. We can also use them to move the loop into the common code.
static <T> void addToLists(T[] elements, Function<T,String> tToId,
List<Integer> empIdList, List<BigInteger> dateList) {
for(T t: elements) {
String[] array = tToId.apply(t).split("-");
dateList.add(new BigInteger(array[1]));
empIdList.add(Integer.valueOf(array[2]));
}
}
List<Integer> empIdAccepted = new ArrayList<>();
List<BigInteger> dateAccepted = new ArrayList<>();
addToLists(acceptedDetails, EmployeeValidationAccepted::getId, empIdAccepted, dateAccepted);
List<Integer> empIdRejected = new ArrayList<>();
List<BigInteger> dateRejected = new ArrayList<>();
addToLists(rejectedDetails, r -> r.getAd().getId(), empIdRejected, dateRejected);

A similar approach as #roookeee already posted with but maybe slightly more concise would be to store the mappings using mapping functions declared as :
Function<String, Integer> extractEmployeeId = empId -> Integer.valueOf(empId.split("-")[2]);
Function<String, BigInteger> extractDate = empId -> new BigInteger(empId.split("-")[1]);
then proceed with mapping as:
Map<Integer, BigInteger> acceptedDetailMapping = Arrays.stream(acceptedDetails)
.collect(Collectors.toMap(a -> extractEmployeeId.apply(a.getId()),
a -> extractDate.apply(a.getId())));
Map<Integer, BigInteger> rejectedDetailMapping = Arrays.stream(rejectedDetails)
.collect(Collectors.toMap(a -> extractEmployeeId.apply(a.getAd().getId()),
a -> extractDate.apply(a.getAd().getId())));
Hereafter you can also access the date of acceptance or rejection corresponding to the employeeId of the employee as well.

How about this:
class EmployeeValidationResult {
//constructor + getters omitted for brevity
private final BigInteger date;
private final Integer employeeId;
}
List<EmployeeValidationResult> accepted = Stream.of(acceptedDetails)
.filter(Objects:nonNull)
.map(this::extractValidationResult)
.collect(Collectors.toList());
List<EmployeeValidationResult> rejected = Stream.of(rejectedDetails)
.filter(Objects:nonNull)
.map(this::extractValidationResult)
.collect(Collectors.toList());
EmployeeValidationResult extractValidationResult(EmployeeValidationAccepted accepted) {
return extractValidationResult(accepted.getId());
}
EmployeeValidationResult extractValidationResult(EmployeeValidationRejected rejected) {
return extractValidationResult(rejected.getAd().getId());
}
EmployeeValidationResult extractValidationResult(String id) {
String[] empIdList = id.split("-");
BigInteger date = extractDate(empIdList[1])
Integer empId = extractId(empIdList[2]);
return new EmployeeValidationResult(date, employeeId);
}
Repeating the filter or map operations is good style and explicit about what is happening. Merging the two lists of objects into one and using instanceof clutters the implementation and makes it less readable / maintainable.

How to keep transformation result for all subsequent stages in reactor

Let's say I have a Reactor stream that consists of 4 stages:
Mono.just(event)
.map(this::map1)
.map(this::map2)
.map(this::map3)
.map(this::map4)
I want the result of this::map1 be accessible by this::map2, this::map3 and this::map4 stages.
Is there any simple way to do this with Reactor?

I think the simplest way of solving this is considering that you need to introduce some sort of "boundary" around map1. This can be achieved by a flatMap:
Mono.just(event)
.map(this::map1)
.flatMap(v1 -> Mono.just(v1)
.map(v2 -> map2(v2, v1))
.map(v3 -> map3(v3, v1))
.map(v4 -> map4(v4, v1))
);
NB: I assumed you couldn't merge the different map functions together for some reason, like simplification of the snippet

I would merge your map1, map2, map3 and map4, in a single map function, since the 4 functions depend on one another
But if you insist on using 4 seperate functions, you could pass the context along the reactive stream using a tuple, for example :
private Tuple2<String, HashMap> map3(Tuple2<String, HashMap> inputTuple) {
String input = inputTuple.getT1();
HashMap context = inputTuple.getT2();
// mapping example
String result = input + context.get("result1") + "mappingExample";
context.put("result3", result);
return Tuples.of(result, context);
}
Or just a simple map that holds all your results, for example :
private HashMap<String, String> map3(HashMap<String, String> input) {
String result3 = input.get("result2") + input.get("result1");
input.put("result3", result3);
return input;
}

What is the best way to implement the python count function in java?

I am learning how to use streams in java and I would like to know the most efficient way to copy the python count functionality into java.
For those unfamiliar with python count, see here.
I've already done a naive implementation but I doubt this would ever get added to a production level environment:
private List<String> countMessages(List<String> messages) {
Map<String, Integer> messageOccurrences = new HashMap<>();
List<String> stackedMessages = new LinkedList<String>();
this.messages.stream().filter((message) -> (messageOccurrences.containsKey(message))).forEachOrdered((message) -> {
int new_occ = messageOccurrences.get(message) + 1;
messageOccurrences.put(message, new_occ);
});
messageOccurrences.keySet().forEach((key) -> {
stackedMessages.add(key + "(" + messageOccurrences.get(key) + "times)" );
});
return stackedMessages;
}
Any improvements or pointers would be appreciated.

To answer the question "what is the best way to implement the python count function in java?".
Java already has Collections.frequency which will do exactly that.
However, if you want to do it with the streams API then I believe a generic solution would be:
public static <T> long count(Collection<T> source, T element) {
return source.stream().filter(e -> Objects.equals(e, element)).count();
}
then the use case would be:
long countHellp = count(myStringList, "hello");
long countJohn = count(peopleList, new Person("John"));
long count101 = count(integerList, 101);
...
...
or you can even pass a predicate if you wanted:
public static <T> long count(Collection<T> source, Predicate<? super T> predicate) {
return source.stream().filter(predicate).count();
}
Then the use case would be for example:
long stringsGreaterThanTen = count(myStringList, s -> s.length() > 10);
long malesCount = count(peopleList, Person::isMale);
long evens = count(integerList, i -> i % 2 == 0);
...
...
Given your comment on the post, it seems like you want to "group" then and get the count of each group.
public Map<String, Long> countMessages(List<String> messages) {
return messages.stream()
.collect(groupingBy(Function.identity(), counting()));
}
This creates a stream from the messages list and then groups them, passing a counting() as the downstream collector meaning we will retrieve a Map<String, Long> where the keys are the elements and the values are the occurrences of that specific string.
Ensure you have the import:
import static java.util.stream.Collectors.*;
for the latter solution.

How to obtain new list after groupby and sum over one attribute

I have a list of Settlement class which has the following attributes:
public class Settlement {
private String contractNo;
private String smartNo;
private String dealTrackNo;
private String buySellFlag;
private String cashFlowType;
private String location;
private String leaseNo;
private String leaseName;
private double volume;
private double price;
private double settleAmount;
// getters and setters
}
Now I would like to group the list of Settlement by SmartNo (String) and get the sum over settleAmount which becomes the new settleAmount for each SmartNo.
Since I am using Java 8, stream should be the way to go.
Groupby should be quite straight forward using the following code:
Map<String, List<Settlement>> map = list.stream()
.collect(Collectors.groupingBy(Settlement::getSmartNo));
System.out.println(map.getValues());
What if I want to get a new list after grouping by SmartNo and summing over settlementAmount? Most of the examples out there only shows how to print out the sums. What I am interested is how to get the aggregated list?

If I understand the question correctly, you need a toMap collector with custom merger like this:
list.stream().collect(Collectors.toMap(
Settlement::getSmartNo,
Function.identity(),
(s1, s2) -> s1.addAmount(s2.getSettleAmount())));
With a helper method inside Settlement class:
Settlement addAmount(double addend) {
this.settleAmount += addend;
return this;
}

I think the not-too-complex way through is a new stream on each member of the values() of your map and then a map() and reduce(). I am mapping to a new class AggregatedSettlement with just the three fields smartNo, volume and settleAmount (the last will be the sum). And then reducing by summing the settleAmounts.
List<AggregatedSettlement> aggregatedList = list.stream()
.collect(Collectors.groupingBy(Settlement::getSmartNo))
.values()
.stream()
.map(innerList -> innerList.stream()
.map(settlm -> new AggregatedSettlement(settlm.getSmartNo(),
settlm.getVolume(), settlm.getSettleAmount()))
.reduce((as1, as2) -> {
if (as1.getVolume() != as2.getVolume()) {
throw new IllegalStateException("Different volumes " + as1.getVolume()
+ " and " + as2.getVolume() + " for smartNo " + as1.getSmartNo());
}
return new AggregatedSettlement(as1.getSmartNo(), as1.getVolume(),
as1.getSettleAmount() + as2.getSettleAmount());
})
.get()
)
.collect(Collectors.toList());
I am not too happy about the call to get() on the Optional<AggregatedSettlement> that I get from reduce(); usually you should avoid get(). In this case I know that the original grouping only produced lists of at least one element, so the the reduce() cannot give an empty optional, hence the call to get() will work. A possible refinement would be orElseThrow() and a more explanatory exception.
I am sure there’s room for optimization. I am really producing quite many more AggregatedSettlement objects than we need in the end. As always, don’t optimize until you know you need to.
Edit: If only for the exercise here’s the version that doesn’t construct superfluous AggregatedSettlement objects. Instead it creates two streams on each list from your map, and it’s 5 lines longer:
List<AggregatedSettlement> aggregatedList = list.stream()
.collect(Collectors.groupingBy(Settlement::getSmartNo))
.entrySet()
.stream()
.map(entry -> {
double volume = entry.getValue()
.stream()
.mapToDouble(Settlement::getVolume)
.reduce((vol1, vol2) -> {
if (vol1 != vol2) {
throw new IllegalStateException("Different volumes " + vol1
+ " and " + vol2 + " for smartNo " + entry.getKey());
}
return vol1;
})
.getAsDouble();
double settleAmountSum = entry.getValue()
.stream()
.mapToDouble(Settlement::getSettleAmount)
.sum();
return new AggregatedSettlement(entry.getKey(), volume, settleAmountSum);
})
.collect(Collectors.toList());
Pick the one you find easier to read.
Edit 2: It seems from this answer that in Java 9 I will be able to avoid the call to Optional.get() if instead of map() I use flatMap() and instead of get() I use stream(). It will be 6 chars longer, I may still prefer it. I haven’t tried Java 9 yet, though (now I know what I’m going to do today :-) The advantage of get() is of course that it would catch a programming error where the inner list comes out empty.