We Keep Coding

How to map bulk request response in java

We are going to consume third party REST API service which supports bulkGETOperations , before we send the request we need to assign a unique id(bid) to each element in array and the same will be returned in response if it SUCCESS(200 OK).
Could some one please help me with the way / most efficient way to map the response based on the unique bid which was passed in the request and after mapping I need to store individual fields in database based on certain condition .Below is the sample request JSON , response will be same in below format but will contain additional fields per school
"testList": [
{
"schoolIdentifier": {
"schoolId": "abc",
"schoolName": {
"name": "ABC"
}
},
"bid": 1
},
{
"schoolIdentifier": {
"schoolId": "bbb",
"schoolName": {
"name": "BCD"
}
},
"bid": 2
}
]
Note: I am aware that this can be done with Map but looking for efficient solution using java 8

You can use merge to lookup and add the response fields. Assuming you've School class with all the request and response fields and are mapped by bid id.
static Map<Integer, School> merge(Map<Integer, School> request, Map<Integer, School> response) {
Map<Integer, School> combined = new HashMap<>(request);
response.forEach(
(key, value) -> combined.merge(key, value, (req, resp) -> /** map additional fields from response **/ ));
return combined;
}

'response will be same in below format but will contain additional fields per school'
I suppose the response you receive already contains all you need in a School object and and you just need to convert the list response into a Map efficiently (with key as bid)
You are getting a list testList which contains both bid and the school Object. You can simply convert a list to the Map like :
Map<Integer, SchoolDto> map = testList.parallelStream().collect(
Collectors.toMap(bidSchoolDto -> bidSchoolDto.getBid()
, bidSchoolDto -> bidSchoolDto.getSchool()));

How to have dynamic data type variable in Java data model class?

An API response will send different data type value in different situation. I am using Gson parser to parse json response string.
eg :
1. { "value" : 1 }
2. { "value" : "Hello" }
3. { "value" : { "name" : "name", "email" : "" } }

You can use Map<String, Object> and then manually check where value is digit, String or nested Map

parsing a json which has attributes which might come as string or array of strings

I have a nested json where in the innermost array there are some keys for which the values could either be a string array or an array of array of strings. The json format is not consistent. How do I parse such a json using gson.
I have tried to write a custom de-serializer (see Gson - parsing json with field that is array or string) but that is throwing exception even before I could detect the attribute as string or array and then update the attribute accordingly.
my json is like this
{
"hits" : {
"total" : 100,
"max_score" : 1,
"hits": [
{"_index": "s1",
"_source":{
"activeOrExpired":[
["active"]
]
}
},
{"_index": "s1",
"_source":{
"activeOrExpired":[
"expired"
]
}
}
]
}
}
My java classes are
public class OuterJson {
#SerliazedName("hits")
public Hits hitsOuter;
public static class Hits {
public List<InnerHits> innerHits;
}
}
public InnerHits {
public String _index;
public Source _source;
public static class Source {
public List<List<String>> activeOrExpired;//I declare this field as
//list of list of strings
public Source() {
activeOrExpired = new ArrayList<>();
}
}
}
public class CustomDeserializer implements JsonDeserializer<OuterJson> {
#Override
public OuterJson deserialize(JsonElement elem, Type type, JsonDeserializationContext context) throws JsonParseException {
JsonObject outerObj = elem.getAsJsonObject();
JsonElement innerHits = outerObj.get("hits").getAsJsonObject().get("hits");
//I want to then detect the type of "activeOrExpired" and convert it
//to list of list of strings if it is present just as a string
//I am getting exception in the below line
InnerHits[] innerHitsArray = new Gson().fromJson(innerHits, InnerHits[].class);
//omitting below code for brevity since my code is failing above itself.
}
}
The exception is
java.lang.IllegalStateException: Expected BEGIN_ARRAY but was String at path $[0]._source.activeOrExpired[0]
Here the innermost "hits" array has the "_source" array which has a field "activeOrExpired" this field is coming either as an array of Strings or array of array of strings.
How should I design the custom deserializer to handle such case?
I am new to gson and was following the method mentioned in the above link. My code is described above, could anyone please give me some hint on progressing. Thanks!

You can use DSM stream parsing library for such a complex JSON or XML. By using DSM you don't need to create java stub file to deserialize. you can directly deserialize to your own class.
It uses YAML based mapping file.
Here is the solution to your question. I am not sure about your object structure. I only deserialize some part of it.
Mapping File:
result:
type: object # result is map.
path: /hits
fields:
hits:
path: hits
type: array
fields:
index:
path: _index
source:
path: _source/activeOrExpired
filter: $value!=null
type: array # source is also array.
Use DSM to filter JSON and deserialize.
// you can pass your class to deserialize directly to your class instead of getting map or list as a result.
//DSM dsm=new DSMBuilder(new File("path/to/maping.yaml")).create(YourClass.class);
DSM dsm=new DSMBuilder(new File("path/to/maping.yaml")).create();
Map<String,Object> hits= (Map<String,Object>)dsm.toObject(new File("path/to/data.json");
json representation of hits variable
{
"innerHits" : [ {
"index" : "s1",
"source" : [ "active" ]
}, {
"index" : "s1",
"source" : [ "expired" ]
} ]
}

Print JSON with ordered properties

I have JSON with objects in specific order:
{
"Aaa": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Bbb": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Ddd": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
]
}
},
}
And I would like to add new object Ccc between Bbb and Ddd. I tried to configure object mapper like this:
ObjectMapper mapper = new ObjectMapper()
.enable(SerializationFeature.INDENT_OUTPUT)
.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
and then print with this code, but Ccc ends at the end of file.
DefaultPrettyPrinter prettyPrinter = new DefaultPrettyPrinter();
prettyPrinter.indentArraysWith(DefaultIndenter.SYSTEM_LINEFEED_INSTANCE);
//Write whole JSON in FILE
String finalJson = mapper.writer(prettyPrinter).writeValueAsString(rootFlores);
finalJson = finalJson.replaceAll("\\[ ]", "[" + System.lineSeparator() + " ]");
finalJson = finalJson.replaceAll("/", "\\\\/");
Files.write(Paths.get("DictionaryFlores_new.json"), Collections.singleton(finalJson));
Is here a way how to print JSON ordered?

Jackson deserialization/serialization does not sort properties
According to this answer, the Jackson SORT_PROPERTIES_ALPHABETICALLY only applies to POJO properties, not Maps. In JSON there is no difference between a Map and an Object, so you need to set the order in the Map first by using a LinkedHashMap or TreeMap

By definition, the keys of an object are unordered. I guess some libraries could offer an option to control the order of the keys when stringifying, but I wouldn't count on it.
When you need a certain order in json, you need to use an array. Of course, then you'd have to move the keys to a property in the child objects, and then the resulting array could only be indexed by number (not by the key). So then you might have to do additional processing to covert the data structure in the JSON to the data structure you really want to process.

Since you seems ready to use regex to update a JSON, I would suggest a "safer" approach. Don't try to create a pattern that would unsure that you don't update a value somewhere.
Iterate you values, on object at the time. Stringify the object and append the String yourself. That way, you are in charge of the object order. Example :
StringBuilder sb = new StringBuilder("{");
List<JsonPOJO> list = new ArrayList<>();
//populate the list
for(JsonPOJO pojo : list){
sb.append(pojo.stringify()).append(",");
}
sb.setLength(sb.length() - 1); //remove the last commma
sb.append("}");
Here, you are only managing the comma between each JSON object, not create the "complex" part about the JSON. And you are in full control of the order of the value in the String representation, it will only depend on the way you populate the List.
Note: sorry for the "draft" code, I don't really have access to my system here so just write this snippet to give you a basic idea on how to "manage" a JSON without having to recreating an API completely.
Note2: I would note suggest this unless this looks really necessary. As you mention in a comment, you are have only the problem with one key where you already have a JSON with 80000 keys, so I guess this is a "bad luck" scenario asking for last resort solution ;)

Flushing a Hashmap of custom Objects in JSON Format

I have a this map:
HashMap<LatLng,ArrayList<String>> dictionary = new HashMap<LatLng,ArrayList<String>>() ;
and I'm trying to flush it to disk in JSON format using jackson like this:
ObjectMapper mapper = new ObjectMapper();
mapper.writeValue(new File(MyApplication.getOfflineData()+"/Dictionary.json"), dictionary);
this is how is written the json file:
"lat/lng: (39.151783,20.97455)": [
"/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic1.nomedia",
"/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic2.nomedia",
"/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic3.nomedia",
"/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic4.nomedia"
]
I think this is not correct. I think the "lat/lng": shouldn't exist, how can I correct this?
Thank you for your time.

When you are using complex POJO class as a key in a Map, Jackson have to generate property name using this class. Simplest implementation is using toString method. In JSON we are not able to create complex property name. Object can not be a property. To solve your problem I propose to create new POJO class which contains LatLng and List<String> properties. See example:
class Root {
private LatLng latLng;
private List<String> values;
//getters,setters
}
Now, your JSON will be looking like this:
{
"latLng" : {
"lat" : "39.151783",
"lng" : "20.97455"
},
"values" : [ "/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic1.nomedia", "/mnt/sdcard/Android/data/example.perifereia_hpeirou/files/lala/photos/pic2.nomedia" ]
}

Just flush out the values of the HashMap.
mapper.writeValue(new File(MyApplication.getOfflineData()+"/Dictionary.json"), dictionary.values())
Or override the toString() of LatLng if you want to customise representation of LatLng.