I'm trying sort List<Map<String,Object>> object based on max date using Hazelcast Jet.
Here is my java 8 code that works:
public static List<Map<String, Object>> extractDate1(List<Map<String, Object>> data) {
return data.stream().map(value -> new Object() {
Map<String, Object> theMap = value;
LocalDate date = extractDate(value);
}).sorted(Comparator.comparing(obj -> obj.date)).map(obj -> obj.theMap).collect(Collectors.toList());
}
public static LocalDate extractDate(Map<String, Object> value) {
DateTimeFormatter formatter1 = DateTimeFormatter.ofPattern("dd-MM-yyyy");
DateTimeFormatter formatter2 = DateTimeFormatter.ofPattern("yyyy-MM-dd");
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("d-MM-yyyy");
return LocalDate.parse(LocalDate.parse(value.get("effectiveDate").toString(), formatter2).format(formatter1),
formatter);
}
The above java 8 code sorts the map objects from low to high:
Below is the Jet code that I'm trying to extract also giving proper output. But I just want to make use of hazelcast jet aggregate/rolling functions
// fetching jsonb type data from db
BatchStage<Object> jobJson = dbValue
// this model holds the string json value
// converting json data to Map object
.map(model -> JsonUtil.mapFrom(model.getJosnValue())
.filter(map -> map.size() != 0)
.map(map -> {
// each json/map object will be having an array and again an array will I have multiple json objects in the
// I'm filtering json objects based on max date
List<Map<String, Object>> extractedDateValue;
if (map.containsKey("records")) {
//Here I'm calling external function (above java 8 code)
extractedDateValue = extractMapBasedOnMax(
(List<Map<String, Object>>) map.get("records"));
}
return extractedDateValue.get(extractedDateValue.size() - 1);
});
JSON data example:
{
"id": "01",
"records": [{
"location": "xyz1",
"effectiveDate": "02-03-2021"
}, {
"location": "xyz2",
"effectiveDate": "02-04-2021"
}]
}
Expeceted Output:
{
"location": "xyz2",
"effectiveDate": "02-04-2021"
}
Is it possible to achieve this through Hazelcast Jet rolling aggregations? Or any suggestions would be helpful.. Thanks
Consider flatMapping the pipeline and finding the maximum using topN. flatMap would convert each JSON structure to series of [id, location, effectiveDate] records. See the documentation of flatMap for code sample.
It's not clear whether you want to find max element in the whole collection or max element per id. Adding a groupingKey would find maximum per id.
The pipeline shape in a "metacode":
source // stream of JSON structures
.flatMap // stream [id, location, effectiveDate]
.groupingKey // for maximum per id, remove for global max
.aggregate(AggregateOpperations.topN) // finds max
.sink;
Related
I have a String input in the following format:
Input String: [{ "id":"1", "name":"A", "address":"St 1"},{ "id":"2", "name":"B", "address":"St 2"}, ...]
And I want to be able to convert this to a Map<String, Map<String, String>> format.
So, something like:
Required Output Format: {1: {id:1, name:"A", address: "St 1"} 2: {id:2, name:"B", address: "St 2"}}
I created a class to help in parsing the input:
public class Student{
private String id;
private String name;
private String address;
}
Which I am trying to do through Jackson's ObjectMapper to get the data in the format: List<Student> format and then use Collectors.toMap() to convert it to a Map of Maps format.
All the examples I have seen so far suggest an approach like:
List<Student> student = objectMapper.readValue(inputString, new TypeReference<List<Student>>(){});
Map<String, Student> studentTempMap = student.stream()
.collect(Collectors.toMap(
Student::getId,
Function.identity()
));
Which makes the studentTempMap something like:
{ 1: object of Student("id":"1", "name":"A", "address":"St 1"),
2: object of Student("id":"2", "name":"B", "address":"St 2"),
... }
A brute force approach from here:
create a new Map studentMap
Iterate over keys (which are "id") in studentTempMap.
Then create another Map, temp.
Add keys "id", "name", and "address" and values using something like studentTempMap.get(2).get("id"), and something similar for all the other keys (name and address). Where 2 would be the current iterator key over the Map studentTempMap.
Finally add a key say as 2 (current iterator) and value temp in the studentMap.
I do not want to use this brute force approach as I have a large number of Student objects.
Is there a way through ObjectMapper to get the output directly in the form of Map<String, Map<String, String>> format?
Or is there a way through Collectors.toMap to parse it to the above format?
I want to be able to convert this to a Map<String, Map<String, String>>
If you want to obtain a nested map of strings Map<String,Map<String,String>> as a result, you don't need to convert JSON into a list of POJO.
Instead, you can parse JSON into a list of maps List<Map<String,String>> and then generate a nested map.
String inputString = """
[{ "id":"1", "name":"A", "address":"St 1"},
{ "id":"2", "name":"B", "address":"St 2"}]""";
ObjectMapper objectMapper = new ObjectMapper();
List<Map<String, String>> students = objectMapper.readValue(
inputString, new TypeReference<>() {}
);
Map<String, Map<String, String>> studentMapById = students.stream()
.collect(Collectors.toMap(
map -> map.get("id"), // key
Function.identity(), // value
(left, right) -> left // resolving duplicates
));
studentMapById.forEach((k, v) -> System.out.println(k + " : " + v));
Output:
1 : {id=1, name=A, address=St 1}
2 : {id=2, name=B, address=St 2}
I have an object structure
data class File(val travelBatch: List<TravelBatch>) {
data class TravelBatch(
val currency: String,
val transactions: List<Transaction>
)
}
I want to have a map of currency to transactions. The below code I tried gives
Map<String, List<List<Transaction>> I want Map<String, List<Transaction>
file.travelBatch.groupBy({it.currency}, {it.transactions})
Need help to flatten the values in the map in kotlin?
You can use mapValues
val result = file.travelBatch
.groupBy({ it.currency }, { it.transactions })
.mapValues { it.value.flatten() }
I have this retrofit response body as LinkedHashMap< String, Json>
"-M74DWOrW0w07BpfmBVo": {
"noteContent": "Note 1 content",
"noteCurrentTime": 1589225588206,
"noteTitle": "Note 1"
},
"-M74Dc2dDAZgVk6q86Rs": {
"noteContent": "Note 2 content",
"noteCurrentTime": 1589225990674,
"noteTitle": "Note 2"
},
"-M74DmbSNQnjEU0Hw4yQ": {
"noteContent": "Note 3 content",
"noteCurrentTime": 1589225658614,
"noteTitle": "Note 3"
}
}
I need to sort by 'noteCurrentTime' value. So far, this is how get the array of sorted values.
private fun sortJsonArray(valuesArray: JSONArray): JSONArray? {
val sortedValues: MutableList<JSONObject> = ArrayList()
for (i in 0 until valuesArray.length()) {
sortedValues.add(valuesArray.getJSONObject(i))
}
sortedValues.sortWith(Comparator { lhs, rhs ->
val lid: Long = lhs.getLong("noteCurrentTime")
val rid: Long = rhs.getLong("noteCurrentTime")
lid.compareTo(rid)
})
return JSONArray(sortedValues)
}
However, this only returns sorted values, without keys, which are now in a wrong order. Is there a way to sort values of LinkedHashMap and keep the correct order of keys? Any and all help would be appreciated.
You can convert the map to a list (of key-value pairs), sort that, and then convert it back.
I don't really know what type is Json in your example Map value type. But you would convert it in the sortedBy lambda however is necessary to get your Long date.
val response: LinkedHashMap<String, Json> = //...
val sorted: Map<String, Json> = response.toList().sortBy { (_, jsonValue) ->
jsonValue.getLong("noteCurrentTime")
}.toMap()
I want to convert a List of Java POJO into a Map in a non-static method with Java 8 Stream API.
My line chart needs a list of date String values for axis x and a list of numeric values for axis y. It's a typical Map format. But my database return a List of POJOs for me. I feel dislike looping without the help of Java 8 Stream API.
I've tried the method in this [ask}(Java 8 List of Objects to Map<String, List> of values). However, I am faced with two problems.First, My POJO MoreCustomDTO contains Integer besides String. Second, When I try to use method reference IDEA complains about non-static method cannot be referenced from a static context.
POJO:
#Data
MoreCustomDTO {
private String date;
private Integer money;
}
DAO query method:
public List<MoreCustomDTO > getMoreCustomCount(#Param("format") String format, #Param("startTime") String startTime, #Param("endTime") String endTime);
Solution before Java 8:
List<MoreCustomCountDTO> customList = customDao.getMoreCustomCount(SqlUtils.DATE_TYPE_FORMAT[Integer.valueOf(type)],startTime,endTime);
Map<String, List> map = new HashMap();
List<String> dateList = new ArrayList<>();
List<Integer> moneyList = new ArrayList<>();
for (MoreCustomCountDTO data : customList) {
dates.add(data.getDate());
dailyAmounts.add(data.getMoney());
}
map.put("date", dateList);
map.put("money", moneyList);
Failure code segment:
Map<String,List> map =
customList.stream()
.flatMap(element -> {
Map<String,String> um = new HashMap<>();
um.put("date",element.getDate());
um.put("money",element.getMoney());
return um.entrySet().stream();
})
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue,
Collectors.toList())));
I get a List from my database. And it is a Object array in JSON foramt.
response:
{
"rows": [
{
"date": "2019-09-01",
"money": 0.00
},
{
"date": "2019-09-02",
"money": 0.00
}
]
}
But I want a one(key)-to-many(values) Map format.
response:
{
"map": {
"date": [
"2019-09-01",
"2019-09-02"
],
"money": [
0.00,
0.00
]
}
}
Honestly, I think your initial solution was ok. Sometimes forcing a solution to be implemented using the most fancy features of the language ends up with less clear code, which is always a bad thing.
Having said that, I think what you intended to do is something like:
Map<Object, Object> map = Stream.of(
new SimpleEntry<>(
"date",
customList.stream().map(MoreCustomDTO::getDate).collect(Collectors.toList())
),
new SimpleEntry<>(
"money",
customList.stream().map(MoreCustomDTO::getMoney).collect(Collectors.toList())
)
).collect(Collectors.toMap(SimpleEntry::getKey, SimpleEntry::getValue));
use SimpleEntry instead like this:
Map<String, List<Object>> map= customList.stream().
flatMap(element ->
Stream.of(new AbstractMap.SimpleEntry<String, Object>("date", element.getDate()),
new AbstractMap.SimpleEntry<String, Object>("money", element.getMoney()))).
collect(Collectors.groupingBy(AbstractMap.SimpleEntry::getKey, Collectors.mapping(AbstractMap.SimpleEntry::getValue, Collectors.toList())));
When converting json to Map using gson, we are having instance of LinkedTreeMap with all values String or Boolean... even numbers are converted to String...
Gson gson = new GsonBuilder().create();
Map<String, Object> result = gson.fromJson(EXAMPLE, new TypeToken<Map<String,Object>>() {}.getType());
How do we convert json to simplest HashMap with corresponding primitive wrappers? Also performance is very important in this case... I want to create as less garbage as possible and resuse parsers...
Is there a way to do that using gson? or any other lib?
Please dont suggest to create special java types for each json... I'll rather navigate trough map...
Here is an example
{ // Hashmap (as no ordering or sorting is essential)
"bool": true, // Boolean
"string": "string", // String
"int" : 123, // Long for all non floats are ok but if possible i'd like this to be Integer if it fits, otherwise Long
"long" : 100000000000000, // Long if Integer cant contain the number...
"double" : 123.123435, // all floating point nubmers will just map to Double
"object" : { // another HashMap
...
}
"array" : [ // Array or some collection like ArrayList or LinkedList
...
]
}
The goal is to convert any json as fast as possible to java Map (or Array if root of json is array) and then use some accessor methods to access data... and not invent java type for every single possible json structure...
Works fine with Jackson Databind library:
ObjectMapper mapper = new ObjectMapper();
Map<String, Object> map = mapper.readValue(jsonString, Map.class);
The values in the map will be of their corresponding type.
This test passes:
#Test
public void test() throws Exception {
String jsonString = "{\"bool\": true,"
+ "\"str\":\"strv\","
+ "\"long\": 100000000000000}";
ObjectMapper mapper = new ObjectMapper();
Map<String, Object> map = mapper.readValue(jsonString , Map.class);
assertEquals(Long.class, map.get("long").getClass());
assertEquals(Boolean.class, map.get("bool").getClass());
}