java groupBy and summarize on multiple field - java

I'm trying to use the Java summarize which I have just discovered and they are just perfect for my use case.
The only issue is that I can't make it working when I need to summarize on multiple field:
final Map<PackageType, LongSummaryStatistics> map2 = artifactory.getStorageInfo()
.getRepositorySummaries()
.stream()
.map(o -> RepositorySummaryValue.from(o))
.collect(groupingBy(
k -> k.getPackageType(),
summarizingLong(k -> k.filesCount)
));
this is my RepositorySummaryValue class:
#lombok.Value
#Builder(builderClassName = "Builder")
private static class RepositorySummaryValue {
long filesCount;
#NonNull
PackageType packageType;
#NonNull
String key;
#NonNull
RepositoryType type;
long usedSpaceBytes;
#SneakyThrows
static RepositorySummaryValue from(RepositorySummary source) {
return builder()
.filesCount(source.getFilesCount())
.packageType(source.getPackageType())
.key(source.getKey())
.type(source.getType())
.usedSpaceBytes(source.getUsedSpaceBytes())
.build();
}
}
What I want is to get summarise also for summarizingLong(k -> k.usedSpaceBytes)
Any way for doing it?
=========EDIT============
I'm using Java 8

Here's an idea that with a little setup would allow any number of fields of some type T to be summarized. It may give you some ideas. It makes use of the compute* features of a map introduced in Java 8.
I used a record in lieu of a class to hold the data.
first, set up a list of method references to get the values you want.
then, initialize another list with the titles of names of those values (here I used your names but the data is just for demo). Note: There are many ways to do this. A single list with a record holding the name and method reference would be another way.
create a map of maps to house the results. The key to the outer map is the packageType, the key to the inner map is the name of the field you are summarizing for each packageType.
Now simply iterate over the list of data, and build the map. The details of how the compute* methods work are explained in the Map Interface JavaDoc but here is a quick summary.
computeIfAbsent will evaluate its second argument if the supplied key is not there. That second argument (here it's the inner map) is returned for access. In this case, another computeIfAbsent is used to see if that map has a key for the field. If not it adds it and creates a LongSummaryStatistic instance wit the key for the field name. That instance is also made available for access. The data item is then accepted and the statistics updated.
public class Summarizing {
record Data(String getPackageType, long getFilesCount,
long getUsedSpaceBytes) {
}
static List<Function<Data, Long>> summaryFields = List
.of(Data::getFilesCount, Data::getUsedSpaceBytes);
static List<String> names = List.of("filesCount", "usedSpaceBytes");
public static void main(String[] args) {
List<Data> list = List.of(new Data("foo", 10, 100),
new Data("bar", 20, 200),
new Data("foo", 30, 300),
new Data("bar", 40, 400));
Map<String, Map<String, LongSummaryStatistics>> result = new HashMap<>();
for (Data d : list) {
Map<String, LongSummaryStatistics> innerMap = result
.computeIfAbsent(d.getPackageType(),
v -> new HashMap<>());
for (int i = 0; i < summaryFields.size(); i++) {
innerMap.computeIfAbsent(names.get(i),
v -> new LongSummaryStatistics())
.accept(summaryFields.get(i)
.apply(d));
}
}
result.entrySet().forEach(e-> {
System.out.println(e.getKey());
for (Entry<?,?> ee : e.getValue().entrySet()) {
System.out.println(" " + ee);
}
});
}
}
prints
bar
filesCount=LongSummaryStatistics{count=2, sum=60, min=20, average=30.000000, max=40}
usedSpaceBytes=LongSummaryStatistics{count=2, sum=600, min=200, average=300.000000, max=400}
foo
filesCount=LongSummaryStatistics{count=2, sum=40, min=10, average=20.000000, max=30}
usedSpaceBytes=LongSummaryStatistics{count=2, sum=400, min=100, average=200.000000, max=300}

Related

Assemble values from Map using Java streams

Sorry maybe for dumb question. I am looking for elegant way to go over elements of my map and filter properties.
Let's say I have map with two elements.
Map<String, MyElement> myMap;
This is how looks my element
class MyElement {
Map <String, Property1> properties1;
Map <String, Property2> properties2;
}
MyElement[0] includes properties1 map filled with some properties, and properties2 is null.
MyElement[1] includes properties2 map filled with some properties, and properties1 is null.
It might be vise versa, I have no idea for which MyElelmet Internal Maps are null and for which are not.
I would like to go over each MyElement in map and assemble properties1 or properties2 from each element in case if it is not empty.
Result should be two separate maps (new collections)
Map <String, Property1> assembledProperties1;
Map <String, Property2> assembledProperties2;
You can think about it as a collecting results to multiple outputs (assembledProperties1, assembledProperties2).
Is there any elegant way to do it with Java streams, without ugly if statements?
Since don't want to utilize MyElement as a mutable container, you can define a special type of object that will carry references to the maps of properties.
In order to be able to perform mutable reduction on a stream of type MyElement with this object we need to define a method that will expect MyElement as a parameter to update maps based on the next element of the stream, and another method that is needed to merge partial results of execution in parallel (i.e. to combine the two objects).
public class PropertyWrapper {
private Map<String, Property1> properties1 = new HashMap<>();
private Map<String, Property2> properties2 = new HashMap<>();
public PropertyWrapper merge(MyElement element) {
if (element.getProperties1() != null) properties1.putAll(element.getProperties1());
if (element.getProperties2() != null) properties2.putAll(element.getProperties2());
return this;
}
public PropertyWrapper merge(PropertyWrapper other) {
this.properties1.putAll(other.getProperties1());
this.properties2.putAll(other.getProperties2());
return this;
}
// getters and toString()
}
With that, the actual code might look like that:
public static void main(String[] args) {
Map<String, MyElement> sourceMap =
Map.of("key1", new MyElement(Map.of("a", new Property1("a"), "b", new Property1("b")), null),
"key2", new MyElement(null, Map.of("c", new Property2("c"), "d", new Property2("d"))));
PropertyWrapper result = sourceMap.values().stream()
.collect(
PropertyWrapper::new,
PropertyWrapper::merge,
PropertyWrapper::merge);
System.out.println(result.getProperties1());
System.out.println(result.getProperties2());
}
Output
{a=Property1{a}, b=Property1{b}}
{d=Property2{d}, c=Property2{c}}
Also note that it's a good practice to avoid keeping nullable references to collections. If these fields will always be initialized with empty collection, the need of null-check will be eliminated.

Best way to maintain key-value mapping pairs in Springboot Java

I want to know the best way to handle key-value mapping in a Springboot API.
The request of the API is as:
{
"a": "1",
"b": "b"
}
This API then calls a downstream API. But before it does that I need to convert the values of the fields a and b. The conversion would be as per:
For field a:
1 -> One,
2 -> Two,
3 -> TATA
for fields b:
b -> beta,
a -> alpha,
c -> c
Both of the above mappings are constant(wont change throughout) and hence need to be initialized and maintained in our API. Now once the mapping takes place the request to the downstream API will look like.
{
"a" : "One",
"b" : "beta"
}
i.e. 1 is mapped to One for field a, and b is mapped to beta for field b; based on the key value mapping mentioned above.
Need to know the best practice in Java and Spring boot to achieve this. How do I maintain the mapping table, initialize it just once and convert incoming API requests to corresponding values at runtime.
Can't use enum because some keys are numerical. If Hash Map and Hast table are my best bets, then how do I initialize them and maintain them throughout, since the mapping is constant don't want to create them again and again for each API call? Where do I read the key-value pairs from to initialize the hashmap/table?
You could keep your translations in code in a private static final field, and initialize it in a static initialization block:
private static final Map<String, Map<String, String>> TRANSLATIONS_PER_KEY = new HashMap<>();
static {
Map<String, String> translateForA = new HashMap<>();
translateForA.put("1", "One");
...
TRANSLATIONS_PER_KEY.put("a", translateForA);
...
}
Then you could do the translation using Guava Maps.transformEntries():
Map<String, String> translated =
Maps.transformEntries(input, (key, value) ->
TRANSLATIONS_PER_KEY.get(key)
.getOrDefault(value));
If you want to avoid NullPointers, and return the original value if there is no translation, you could do it like this:
Map<String, String> translated =
Maps.transformEntries(input, (key, value) ->
TRANSLATIONS_PER_KEY.getOrDefault(key, emptyMap())
.getOrDefault(value, value));
You may create enum for all mappings with pares
#AllArgsConstructor
#Getter
public enum AType {
ONE("One", "1"),
TWO("Two", "2"),
TATA("TATA", "3");
private String translation;
private String mapping;
public String of(String v) {
for (AType aType : values()) {
if (aType.getMapping().equals(v)) {
return translation;
}
}
throw new IllegalArgumentException("try again");
}
}
and new enum for b

Simple using Map with Lists as value [duplicate]

I very much want to use Map.computeIfAbsent but it has been too long since lambdas in undergrad.
Almost directly from the docs: it gives an example of the old way to do things:
Map<String, Boolean> whoLetDogsOut = new ConcurrentHashMap<>();
String key = "snoop";
if (whoLetDogsOut.get(key) == null) {
Boolean isLetOut = tryToLetOut(key);
if (isLetOut != null)
map.putIfAbsent(key, isLetOut);
}
And the new way:
map.computeIfAbsent(key, k -> new Value(f(k)));
But in their example, I think I'm not quite "getting it." How would I transform the code to use the new lambda way of expressing this?
Recently I was playing with this method too. I wrote a memoized algorithm to calcualte Fibonacci numbers which could serve as another illustration on how to use the method.
We can start by defining a map and putting the values in it for the base cases, namely, fibonnaci(0) and fibonacci(1):
private static Map<Integer,Long> memo = new HashMap<>();
static {
memo.put(0,0L); //fibonacci(0)
memo.put(1,1L); //fibonacci(1)
}
And for the inductive step all we have to do is redefine our Fibonacci function as follows:
public static long fibonacci(int x) {
return memo.computeIfAbsent(x, n -> fibonacci(n-2) + fibonacci(n-1));
}
As you can see, the method computeIfAbsent will use the provided lambda expression to calculate the Fibonacci number when the number is not present in the map. This represents a significant improvement over the traditional, tree recursive algorithm.
Suppose you have the following code:
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class Test {
public static void main(String[] s) {
Map<String, Boolean> whoLetDogsOut = new ConcurrentHashMap<>();
whoLetDogsOut.computeIfAbsent("snoop", k -> f(k));
whoLetDogsOut.computeIfAbsent("snoop", k -> f(k));
}
static boolean f(String s) {
System.out.println("creating a value for \""+s+'"');
return s.isEmpty();
}
}
Then you will see the message creating a value for "snoop" exactly once as on the second invocation of computeIfAbsent there is already a value for that key. The k in the lambda expression k -> f(k) is just a placeolder (parameter) for the key which the map will pass to your lambda for computing the value. So in the example the key is passed to the function invocation.
Alternatively you could write: whoLetDogsOut.computeIfAbsent("snoop", k -> k.isEmpty()); to achieve the same result without a helper method (but you won’t see the debugging output then). And even simpler, as it is a simple delegation to an existing method you could write: whoLetDogsOut.computeIfAbsent("snoop", String::isEmpty); This delegation does not need any parameters to be written.
To be closer to the example in your question, you could write it as whoLetDogsOut.computeIfAbsent("snoop", key -> tryToLetOut(key)); (it doesn’t matter whether you name the parameter k or key). Or write it as whoLetDogsOut.computeIfAbsent("snoop", MyClass::tryToLetOut); if tryToLetOut is static or whoLetDogsOut.computeIfAbsent("snoop", this::tryToLetOut); if tryToLetOut is an instance method.
Another example. When building a complex map of maps, the computeIfAbsent() method is a replacement for map's get() method. Through chaining of computeIfAbsent() calls together, missing containers are constructed on-the-fly by provided lambda expressions:
// Stores regional movie ratings
Map<String, Map<Integer, Set<String>>> regionalMovieRatings = new TreeMap<>();
// This will throw NullPointerException!
regionalMovieRatings.get("New York").get(5).add("Boyhood");
// This will work
regionalMovieRatings
.computeIfAbsent("New York", region -> new TreeMap<>())
.computeIfAbsent(5, rating -> new TreeSet<>())
.add("Boyhood");
multi-map
This is really helpful if you want to create a multimap without resorting to the Google Guava library for its implementation of MultiMap.
For example, suppose you want to store a list of students who enrolled for a particular subject.
The normal solution for this using JDK library is:
Map<String,List<String>> studentListSubjectWise = new TreeMap<>();
List<String>lis = studentListSubjectWise.get("a");
if(lis == null) {
lis = new ArrayList<>();
}
lis.add("John");
//continue....
Since it have some boilerplate code, people tend to use Guava Mutltimap.
Using Map.computeIfAbsent, we can write in a single line without guava Multimap as follows.
studentListSubjectWise.computeIfAbsent("a", (x -> new ArrayList<>())).add("John");
Stuart Marks & Brian Goetz did a good talk about this
https://www.youtube.com/watch?v=9uTVXxJjuco
Came up with this comparison example (old vs new) which demonstrates both the approaches;
static Map<String, Set<String>> playerSkills = new HashMap<>();
public static void main(String[] args) {
//desired output
//player1, cricket, baseball
//player2, swimming
//old way
add("Player1","cricket");
add("Player2","swimming");
add("Player1","baseball");
System.out.println(playerSkills);
//clear
playerSkills.clear();
//new
addNew("Player1","cricket");
addNew("Player2","swimming");
addNew("Player1","baseball");
System.out.println(playerSkills);
}
private static void add(String name, String skill) {
Set<String> skills = playerSkills.get(name);
if(skills==null) {
skills= new HashSet<>();
playerSkills.put(name, skills);
}
skills.add(skill);
}
private static void addNew(String name, String skill) {
playerSkills
.computeIfAbsent(name, set -> new HashSet<>())
.add(skill);
}

Java 8 Map KeySet Stream not working as desired for use in Collector

I have been trying to learn Java 8's new functional interface features, and I am having some difficulty refactoring code that I have previously written.
As part of a test case, I want to store a list of read names in a Map structure in order to check to see if those reads have been "fixed" in a subsequent section of code. I am converting from an existing Map> data structure. The reason why I am flattening this datastructure is because the outer "String" key of the original Map is not needed in the subsequent analysis (I used it to segregate data from different sources before merging them in the intermediate data). Here is my original program logic:
public class MyClass {
private Map<String, Map<String, Short>> anchorLookup;
...
public void CheckMissingAnchors(...){
Map<String, Boolean> anchorfound = new HashMap<>();
// My old logic used the foreach syntax to populate the "anchorfound" map
for(String rg : anchorLookup.keySet()){
for(String clone : anchorLookup.get(rg).keySet()){
anchorfound.put(clone, false);
}
}
...
// Does work to identify the read name in the file. If found, the boolean in the map
// is set to "true." Afterwards, the program prints the "true" and "false" counts in
// the map
}
}
I attempted to refactor the code to use functional interfaces; however, I getting errors from my IDE (Netbeans 8.0 Patch 2 running Java 1.8.0_05):
public class MyClass {
private Map<String, Map<String, Short>> anchorLookup;
...
public void CheckMissingAnchors(...){
Map<String, Boolean> anchorfound = anchorLookup.keySet()
.stream()
.map((s) -> anchorlookup.get(s).keySet()) // at this point I am expecting a
// Stream<Set<String>> which I thought could be "streamed" for the collector method
// ; however, my IDE does not allow me to select the "stream()" method
.sequential() // this still gives me a Stream<Set<String>>
.collect(Collectors.toMap((s) -> s, (s) -> false);
// I receive an error for the preceding method call, as Stream<Set<String>> cannot be
// converted to type String
...
}
}
Is there a better way to create the "anchorfound" map using the Collection methods or is the vanilla Java "foreach" structure the best way to generate this data structure?
I apologize for any obvious errors in my code. My formal training was not in computer science but I would like to learn more about Java's implementation of functional programming concepts.
I believe what you need is a flatMap.
This way you convert each key of the outer map to a stream of the keys of the corresponding inner map, and then flatten them to a single stream of String.
public class MyClass {
private Map<String, Map<String, Short>> anchorLookup;
...
public void CheckMissingAnchors(...){
Map<String, Boolean> anchorfound = anchorLookup.keySet()
.stream()
.flatMap(s -> anchorlookup.get(s).keySet().stream())
.collect(Collectors.toMap((s) -> s, (s) -> false);
...
}
}
Eran's suggestion of flatMap is a good one, +1.
This can be simplified somewhat by using Map.values() instead of Map.keySet(), since the map's keys aren't used for any other purpose than to retrieve the values. Streaming the result of Map.values() gives a Stream<Map<String,Short>>. Here we don't care about the inner map's values, so we can use keySet() to extract the keys, giving a Stream<Set<String>>. Now we just flatMap these sets into Stream<String>. Finally we send the results into the collector as before.
The resulting code looks like this:
public class MyClass {
private Map<String, Map<String, Short>> anchorLookup;
public void checkMissingAnchors() {
Map<String, Boolean> anchorfound = anchorLookup.values().stream()
.map(Map::keySet)
.flatMap(Set::stream)
.collect(Collectors.toMap(s -> s, s -> false));
}
}

Creating complex HashMap in Java

What is the easiest way to create a HashMap like this :
( student1 => Map( name => Tim,
Scores => Map( math => 10,
physics => 20,
Computers => 30),
place => Miami,
ranking => Array(2,8,1,13),
),
student2 => Map (
...............
...............
),
............................
............................
);
I tried this :
HashMap record = new HashMap();
record.put("student1", new HashMap());
record.get("student1").put("name","Tim");
record.get("student1").put("Scores", new HashMap());
But I get error. I do it that way because, record.get("student1") is a HashMap object, so I assume a put on that should work, and so on.
If it doesnt work, what is the best way to do it ?
You get that exception because get() returns a type Object. you need to cast that to a Map.
((Map)record.get("student1")).put("name","Tim");
You can do it by type casting the Object to Map or HashMap.
HashMap record = new HashMap();
record.put("student1", new HashMap());
((HashMap)record.get("student1")).put("name","Tim");
((HashMap)record.get("student1")).put("Scores", new HashMap());
Still, as I've commented, are you sure you want this design?
Java is a statically and nominally typed language. As such the style of coding you are following is not preferable. You should be creating classes and objects instead.
That said, Guava provides several utility classes such as Lists, Iterables, Maps etc that let you construct the relevant collection objects from varargs.
While I agree with the other commenters about preferring to create proper classes for your domain objects, we can simplify the map construction syntax with some helper methods:
Map<String, Object> map = map(
entry("student1", map(
entry("name", "Tim"),
entry("scores", map(
entry("math", 10),
entry("physics", 20),
entry("Computers", 30)
)),
entry("place", "Miami"),
entry("ranking", array(2, 8, 1, 13))
)),
entry("student2", map(
// ...
))
);
// map.toString():
// {student2={}, student1={scores={math=10, physics=20, Computers=30}, name=Tim, place=Miami, ranking=[2, 8, 1, 13]}}
You just need to define a helper class like the one below and use a static import:
import static com.example.MapHelper.*;
The helper class:
package com.example;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class MapHelper {
public static class Entry {
private String key;
private Object value;
public Entry(String key, Object value) {
this.key = key;
this.value = value;
}
public String getKey() {
return key;
}
public Object getValue() {
return value;
}
}
public static Map<String, Object> map(Entry... entries) {
Map<String, Object> map = new HashMap<String, Object>();
for (Entry e : entries) {
map.put(e.getKey(), e.getValue());
}
return map;
}
public static Entry entry(String k, Object v) {
return new Entry(k, v);
}
public static List<Object> array(Object... items) {
List<Object> list = new ArrayList<Object>(items.length);
for (int i = 0; i < items.length; i++) {
list.add(i, items[i]);
}
return list;
}
}
Well the "Java" way to do this is to break up those entities into classes. From the example you gave, it looks like you can make a Student class that contains attributes like name, place, location, etc.
This is much cleaner than forcing everything into a map like this.
Why are you creating a complex set of HashMaps, you can create a Java Object instead, and then use it in for a HashMap.
So in your case, you can create a class called ScoreInfo which will have HashMap score, place and rank and then use that as a value of HashMap.
Replace
HashMap record = new HashMap();
with
Map<String,Map<String, Object>> record = new HashMap<String,Map<String, Object>>();
But, this doesn't make much sense to put different object types as values. If the following line is by mistake,
record.get("student1").put("Scores", new HashMap());
then you can simplify the definition also.
Map<String,Map> record = new HashMap<String,Map>();
Assumption: You are using JDK 1.5+

Categories