We Keep Coding

Unable to deserialize complex/dynamic JSON to Java classes using Gson

I've been trying to deserialize a JSON to Java classes using Gson, but the JSON structure is too complex for me to handle. The JSON looks like this (I've trimmed some of it because of repetitions):
{
"results":[
{
"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>":{
"type":"DV_TEXT",
"name":{
"en":"Encounter channel"
},
"attrs":[
"value"
]
},
"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>":{
"type":"DV_TEXT",
"name":{
"en":"Monitoring reason"
},
"attrs":[
"value"
]
}
},
{
"163eee06-83a4-4fd8-bf65-5d6a3ef35ac5":{
"d5760d01-84dd-42b2-8001-a69ebaa4c2df":{
"date":"2020-08-06 09:45:31",
"cols":[
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.encounter_channel.v0](0)/items[at0001](0)/value",
"value":"null"
}
]
},
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.monitoring_reason.v0](1)/items[at0001](0)/value",
"value":"null"
}
]
}
]
},
"fb366b72-d567-4d23-9f5f-356fc09aff6f":{
"date":"2020-08-06 10:02:26",
"cols":[
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.encounter_channel.v0](0)/items[at0001](0)/value",
"value":"Consulta presencial"
}
]
},
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.monitoring_reason.v0](1)/items[at0001](0)/value",
"value":"Consulta"
}
]
}
]
}
}
}
],
"pagination":{
"max":20,
"offset":0,
"nextOffset":20,
"prevOffset":0
},
"timing":"475 ms"
}
The main JSON object has three fields: results, pagination and timing. I can deserialize the pagination and timing just fine, as they always have the same structure. I cannot properly deserialize the results though.
results is always a list of two different objects. The second object, in particular, is the most complex one, as its field names are not static. The UUID name references always change on each API response. For instance, the field named "163eee06-83a4-4fd8-bf65-5d6a3ef35ac5" might have another id in the next JSON response. Therefore, I cannot give it a proper field name in the corresponding Java class. The same goes for "d5760d01-84dd-42b2-8001-a69ebaa4c2df" and "fb366b72-d567-4d23-9f5f-356fc09aff6f" in this case.
Any ideas on how to properly deserialize this kind of JSON using Gson? I've tried a couple of different approaches, but nothing has truly worked so far.
In most recent attempt I tried to use the JsonDeserializer approach in order to differentiate the type of objects in the results list. My current implementation looks like this (getters and setters were hidden because of space):
QueryResponse.java
public class QueryResponse {
private List<Map<String, ResultInterface>> results;
private Pagination pagination;
private String timing;
}
Pagination.java
public class Pagination {
private Integer max;
private Integer offset;
private Integer nextOffset;
private Integer previousOffset;
}
ResultInterface.java
public interface ResultInterface {
}
ElementDefinition.java
public class ElementDefinition implements ResultInterface {
private String type;
private Name name;
private List<String> attrs;
}
Name.java
public class Name {
private String en;
private String es;
}
Compositions.java
public class Compositions implements ResultInterface {
private Map<String, Composition> compositions;
}
Composition.java
public class Composition {
private String date;
private List<Col> cols;
}
Col.java
public class Col {
private String type;
private String path;
private List<Value> values;
}
Value.java
public class Value {
private String instanceTemplatePath;
private String value;
private String magnitude;
private String units;
private String code;
private String terminology_id;
}
ResultInterfaceDeserializer.java
public class ResultInterfaceDeserializer implements JsonDeserializer<ResultInterface> {
#Override
public ResultInterface deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
JsonObject jObject = (JsonObject) json;
JsonElement typeObj = jObject.get("type");
if (typeObj != null) {
return context.deserialize(json, ElementDefinition.class);
} else {
return context.deserialize(json, Compositions.class);
}
}
}
I'm calling Gson like this:
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(ResultInterface.class, new ResultInterfaceDeserializer());
Gson gson = builder.create();
QueryResponse queryResponse = gson.fromJson(externalJsonResponse, QueryResponse.class);
The problem with this implementation is that there is nothing named compositions in the JSON structure, thus the Compositions.java class is not correctly identified. I know I have to use Java structures like Map<String, SomeObject>, but the problem is that there are too many dynamically named Json fields here, and I cannot "grab" them if they have no fixed name identifier.
UPDATE
I managed to find a solution. I'd say it's actually a workaround and probably not the most clean or elegant solution.
The problem with my current implementation was that I was trying to "grab" a JSON field called compositions when in fact it didn't exist. So, I decided to manipulate the JSON and add that field myself (in the code).
I changed the deserializer class to:
public class ResultInterfaceDeserializer implements JsonDeserializer<ResultInterface> {
public String encloseJsonWithCompositionsField(JsonElement json) {
return "{\"compositions\":" + json.toString() + "}";
}
#Override
public ResultInterface deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
JsonObject jObject = (JsonObject) json;
if (jObject.get("type") != null) {
return context.deserialize(json, ElementDefinition.class);
} else {
JsonElement jsonWithCompositionsField = new JsonParser().parse(encloseJsonWithCompositionsField(json));
return context.deserialize(jsonWithCompositionsField, Compositions.class);
}
}
}
With this change, I can now "grab" the compositions field and get the data in Java POJOs.

You could probably solve this by registering an additional JsonDeserializer for Compositions:
public class CompositionsDeserializer implements JsonDeserializer<Compositions> {
public static final CompositionsDeserializer INSTANCE = new CompositionsDeserializer();
private CompositionsDeserializer() { }
#Override
public Compositions deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
Compositions compositions = new Compositions();
Map<String, Composition> compositionsMap = new HashMap<>();
compositions.compositions = compositionsMap;
JsonObject compositionsJson = json.getAsJsonObject();
for (Map.Entry<String, JsonElement> compositionEntry : compositionsJson.entrySet()) {
Composition composition = context.deserialize(compositionEntry.getValue(), Composition.class);
compositionsMap.put(compositionEntry.getKey(), composition);
}
return compositions;
}
}
And then register that deserializer on the GsonBuilder as well:
Gson gson = new GsonBuilder()
.registerTypeAdapter(ResultInterface.class, new ResultInterfaceDeserializer())
.registerTypeAdapter(Compositions.class, CompositionsDeserializer.INSTANCE)
.create();

DynamoDBMapper unconvert attribute error

I am getting json from dynamoDb that looks like this -
{
"id": "1234",
"payment": {
"payment_id": "2345",
"user_defined": {
"some_id": "3456"
}
}
}
My aim is to get the user_defined field in a Java HashMap<String, Object> as user_defined field can contain any user defined fields, which would be unknown until the data arrives. Everything works fine except my DynamoDBMapper cannot convert the user_defined field to a Java HashMap. It is throwing this error -
Exception occured Response[payment]; could not unconvert attribute
This is how the classes looks like -
#DynamoDBTable(tableName = "PaymentDetails")
public class Response {
private String id;
public Response() {
}
private Payment payment = new Payment();
#DynamoDBHashKey(attributeName="id")
public String getId() { return id; }
public void setId(String id) { this.id = id; }
public Payment getPayment() {
return payment;
}
public void setPayment(Payment payment) {
this.payment = payment;
}
}
The payment field mapper -
#DynamoDBDocument
public class Payment {
private String payment_id:
private HashMap<String, Object> user_defined;
public Payment() {}
public getPayment_id() {
return payment_id;
}
public setPayment_id(String payment_id) {
this.payment_id = payment_id;
}
#DynamoDBTypeConverted(converter = HashMapMarshaller.class)
public HashMap<String, Object> getUser_defined() {
return user_defined;
}
public void setUser_defined(HashMap<String, Object> user_defined) {
this.user_defined = user_defined;
}
}
The HashMapMarshaller(Just to check if Hashmap marshaller wasn't working with gson, I just defined a Hashmap, put in a value and return it, but seems to still not working) -
public class HashMapMarshaller implements DynamoDBTypeConverter<String, HashMap<String, Object>> {
#Override
public String convert(HashMap<String, Object> hashMap) {
return new Gson().toJson(hashMap);
}
#Override
public HashMap<String, Object> unconvert(String jsonString) {
System.out.println("jsonString received for unconverting is " + jsonString);
System.out.println("Unconverting attribute");
HashMap<String, Object> hashMap = new HashMap<>();
hashMap.put("key", "value");
return hashMap;
//return new Gson().fromJson(jsonString, new TypeToken<HashMap<String, Object>>(){}.getType());
}
}
Marshaller approach is till now not working for me. It is also not printing any of the printlns I've put in there. I've also tried using #DynamoDBTyped(DynamoDBMapperFieldModel.DynamoDBAttributeType.M) and using Map instead of HashMap above my user_defined getter to no avail.
I want to find out how to convert the user_defined field to Java HashMap or Map. Any help is appreciated. Thank you!

Make Map<String, Object> to Map<String, String>. It should work without any custom converters. Otherwise be specific about Map's value type. For example, Map<String, SimplePojo> should work. Don't forget to annotate SimplePojo class with #DynamoDBDocument.
With Object as a type of Map's value, DynamoDB will not able to decide which object it has to create while reading entry from DynamoDB. It should know about specific type like String, Integer, SimplePojo etc.

Jackson JSON key as value in Java

I'm using Jackson in Spring MVC application. I want to use a String value as key name for Java POJO --> JSON
"record": {
"<Dynamic record name String>": {
"value": {
....
}
}
}
So the dynamic record name String could be "abcd","xyz" or any other string value. How can I define my "record" POJO to have a key like that ?

Unfortunately, you cannot have dynamic fields in Java classes (unlike some other languages), so you have two choices:
Using Maps
Using JSON objects (i.e. JsonNode in case of Jackson)
Suppose, you have a data like this:
{
"record": {
"jon-skeet": {
"name": "Jon Skeet",
"rep": 982706
},
"darin-dimitrov": {
"name": "Darin Dimitrov",
"rep": 762173
},
"novice-user": {
"name": "Novice User",
"rep": 766
}
}
}
Create two classes to capture it, one for user and another for the object itself:
User.java:
public class User {
private String name;
private Long rep;
public String getName() { return name; }
public void setName(String name) { this.name = name; }
public Long getRep() { return rep; }
public void setRep(Long rep) { this.rep = rep; }
#Override
public String toString() {
return "User{" +
"name='" + name + '\'' +
", rep=" + rep +
'}';
}
}
Data.java:
public class Data {
private Map<String, User> record;
public Map<String, User> getRecord() { return record; }
public void setRecord(Map<String, User> record) { this.record = record; }
#Override
public String toString() {
return "Data{" +
"record=" + record +
'}';
}
}
Now, parse the JSON (I assume there is a data.json file in the root of your classpath):
public class App {
public static void main(String[] args) throws Exception {
final ObjectMapper objectMapper = new ObjectMapper();
System.out.println(objectMapper.readValue(App.class.getResourceAsStream("/data.json"), Data.class));
System.out.println(objectMapper.readTree(App.class.getResourceAsStream("/data.json")));
}
}
This will output:
Data{record={jon-skeet=User{name='Jon Skeet', rep=982706}, darin-dimitrov=User{name='Darin Dimitrov', rep=762173}, novice-user=User{name='Novice User', rep=766}}}
{"record":{"jon-skeet":{"name":"Jon Skeet","rep":982706},"darin-dimitrov":{"name":"Darin Dimitrov","rep":762173},"novice-user":{"name":"Novice User","rep":766}}}
In case of a Map you can use some static classes, like User in this case, or go completely dynamic by using Maps of Maps (Map<String, Map<String, ...>>. However, if you find yourself using too much maps, consider switching to JsonNodes. Basically, they are the same as Map and "invented" specifically for highly dynamic data. Though, you'll have some hard time working with them later...
Take a look at a complete example, I've prepared for you here.

This is in Kotlin but I have found a solution to the same problem using Jackson.
You don't need the root node "record", so you will need to get rid of it or start one node deeper(you're on your own there) but to turn the list of records that are children of their id into a list of records with id in the object follows:
val node = ObjectMapper().reader().readTree(json)
val recordList = mutableListOf<Record>()
node.fields().iterator().forEach {
val record = record(
it.key,
it.value.get("name").asText(),
it.value.get("rep").asText()
)
recordList.add(event)
}
node.fields() returns a map of children(also maps)
iterating through the parent map you will get the id from the key and then the nested data is in the value (which is another map)
each child of fields is key : value where
key = record id
value = nested data (map)
This solution, you don't need multiple classes to deserialize a list of classes.

I have my data in this format:
{
"0" : {"a": {}}, {"b": {}}, ...
"1" : {"c": {}}, {"d": {}}, ...
.
.
.
}
I am able to capture it into a map using the dynamic capture feature of jackson by using #JsonAnySetter annotation.
public class Destination{
Map<String, Object> destination = new LinkedHashMap<>();
#JsonAnySetter
void setDestination(String key, Object value) {
destination.put(key, value);
}
}

converting json string to object through GSON

I am trying to parse a json string to java object but i am not sure on the object hierarchy.
below is the json string
{
"TABLE_Detail":{
"1":{
"TABLE":"table1",
"RUN_DATE":"20170313",
"SEQ_NUM":"123",
"START_TIME":"20170313133144",
"END_TIME":"20170313133655"
},
"2":{
"TABLE":"table2",
"RUN_DATE":"20170313",
"SEQ_NUM":"123",
"START_TIME":"20170313133142",
"END_TIME":"20170313133723"
}
}
}
Here the number 1, 2 are dynamic and can go up to any number, I tried to create a outer object and have a Map of type key String and value as object TableData. The map having variable name TABLE_Detail. but the TableData object is always null. TableData object has all the variables.
Please help me on how to convert this json string to object.

Change 1 to table1 and 2 to table2:
public class TableDetails {
private TableData table1;
private TableData table2;
public TableDetails(){
}
// getter and setter
}
And if modify json format to "Koen Van Looveren" mentioned:
public class TableDetails {
List<TableData> tables;
public TableDetails() {
}
// getter and setter
}
The table class:
Table.java:
public class TableData {
private String table;
private String run_date;
private String seq_num;
private String start_time;
private String end_time;
public TableData() {
}
// getter and setter
}

you have two choice for such painfully json structure when using Gson.
using Gson parsing json as Map and write some class access returned Map.this mode works fine for access data only!
//usage
TableDetails details=new TableDetails(gson.fromJson(json, Map.class));
//classes
class TableDetails {
private Map<String, Map> context;
public TableDetails(Map root) {
this.context = (Map<String, Map>) root.get("TABLE_Detail");
}
public int size() {
return context.size();
}
public Table get(String key) {
return new Table(context.get(key));
}
}
class Table {
private Map context;
public Table(Map context) {
this.context = context;
}
public String getName() {
return get("TABLE");
}
private <T> T get(String name) {
return (T) context.get(name);
}
...
}
write your own Gson TypeAdapter,but this way may be more complex.if you interesting on write custom TypeAdapter there is a demo that I written for extract json root.gson-enclosing-plugin

You can try deserialize it into a Map<String, Map<String, TableData>>. The reason why Map<String, TableData> doesn't work it that the pesudo-array is wrapped in another object.
The following example converts a response into a List<TableData>:
public List<TableData> deserialize(String json) {
return Gson().<Map<String, Map<String, TableData>>>fromJson(json, new TypeToken<Map<String, Map<String, TableData>>>(){}.getType())
.values().iterator().next()
.entrySet().stream()
.sorted(Comparator.comparingInt(e -> Integer.parseInt(e.getKey())))
.map(Map.Entry::getValue)
.collect(Collectors.toList());
}

I was in a search for the solution, and i came across one of the site where the solution worked. i wanted to credit the below site. Thanks for all the support.
I am able to map the dynamic value 1, 2 as map keys and values are mapped correspondingly to the TableData object properties using #SerializedName gson annootation.
http://findnerd.com/list/view/Parse-Json-Object-with-dynamic-keys-using-Gson-/24094/

When using an array in json you need to use [ for opening and ] for closing
{
"TABLE_Detail": [
{
"TABLE": "table1",
"RUN_DATE": "20170313",
"SEQ_NUM": "123",
"START_TIME": "20170313133144",
"END_TIME": "20170313133655"
},
{
"TABLE": "table2",
"RUN_DATE": "20170313",
"SEQ_NUM": "123",
"START_TIME": "20170313133142",
"END_TIME": "20170313133723"
}
]
}

Jackson list deserialization. nested Lists

I'm working on creating an API that has nested lists. Jackson seems like a great tool to create objects, but I can't quite figure out how to nest a list, and I'm wondering if its possible.
My object looks like this.
public class Order {
public String name;
public List<Item> items;
}
I'm hoping there is a way to map it to json that looks something like:
{
name : "A name"
items : {
elements : [{
price : 30
}]
}
}
We want to be able to do this so we can add properties to lists.

You can write custom deserializer for List<Item> items. See below example:
class ItemsJsonDeserializer extends JsonDeserializer<List<Item>> {
#Override
public List<Item> deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException {
InnerItems innerItems = jp.readValueAs(InnerItems.class);
return innerItems.elements;
}
private static class InnerItems {
public List<Item> elements;
}
}
Now, you have to inform Jackson to use it for your property. You can do this in this way:
public class Order {
public String name;
#JsonDeserialize(using = ItemsJsonDeserializer.class)
public List<Item> items;
}

In general it is best to map JSON structure exactly to Java. In your case you could use something like:
public class Order {
public String name;
public ItemList items;
}
public class ItemList {
public List<Item> elements;
// and any properties you might want...
}
alternatively, you could probably also use (relatively) new #JsonFormat annotation:
public class Order {
public String name;
public ItemList items;
}
// annotation can be used on propery or class
#JsonFormat(shape=Shape.OBJECT) // instead of Shape.ARRAY
public class ItemList extends ArrayList<Item>
{
public Iterator<Item> getElements() { return this.iterator(); }
public String getSomeAttribute() { ... }
}
where you are forcing List or Collection to be serialized as if it was POJO, instead of normal special handling. There may be some side-effects, since introspection is used to find possible accessors, but the general approach should work

Your JSON translates to: "the object named items is of a type that has a property named elements which is a list of some sort".
So your Item class just needs an elements property:
class Item {
List<Something> getElements();
}
Note that your Java code doesn't map to your JSON. Your Java classes would map to something like:
{
"name" : "foo",
"items" : [
{ /* encoded version of Item */ }
]
}

For scala, one can try:
class ItemsJsonDeserializer extends JsonDeserializer[List[Item]] {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
override def deserialize(jp: JsonParser, dc: DeserializationContext): List[Item] = {
val oc = jp.getCodec
val nodes = oc.readTree[ObjectNode](jp).get("elements").asScala.toList
val res = nodes.map { node =>
mapper.readValue[Item](node.toString)
}
res
}
}
case class Item(price: Int)
case class Order {
name: String,
#JsonDeserialize(using = classOf[ItemsJsonDeserializer])
items: List<Item>
}