Unable to deserialize complex/dynamic JSON to Java classes using Gson - java

I've been trying to deserialize a JSON to Java classes using Gson, but the JSON structure is too complex for me to handle. The JSON looks like this (I've trimmed some of it because of repetitions):
{
"results":[
{
"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>":{
"type":"DV_TEXT",
"name":{
"en":"Encounter channel"
},
"attrs":[
"value"
]
},
"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>":{
"type":"DV_TEXT",
"name":{
"en":"Monitoring reason"
},
"attrs":[
"value"
]
}
},
{
"163eee06-83a4-4fd8-bf65-5d6a3ef35ac5":{
"d5760d01-84dd-42b2-8001-a69ebaa4c2df":{
"date":"2020-08-06 09:45:31",
"cols":[
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.encounter_channel.v0](0)/items[at0001](0)/value",
"value":"null"
}
]
},
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.monitoring_reason.v0](1)/items[at0001](0)/value",
"value":"null"
}
]
}
]
},
"fb366b72-d567-4d23-9f5f-356fc09aff6f":{
"date":"2020-08-06 10:02:26",
"cols":[
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.encounter_channel.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.encounter_channel.v0](0)/items[at0001](0)/value",
"value":"Consulta presencial"
}
]
},
{
"type":"DV_TEXT",
"path":"openEHR-EHR-CLUSTER.monitoring_reason.v0/items[at0001]/value<DV_TEXT>",
"values":[
{
"instanceTemplatePath":"prova_de_conceito.en.v1/context/other_context[at0001]/items[archetype_id=openEHR-EHR-CLUSTER.monitoring_reason.v0](1)/items[at0001](0)/value",
"value":"Consulta"
}
]
}
]
}
}
}
],
"pagination":{
"max":20,
"offset":0,
"nextOffset":20,
"prevOffset":0
},
"timing":"475 ms"
}
The main JSON object has three fields: results, pagination and timing. I can deserialize the pagination and timing just fine, as they always have the same structure. I cannot properly deserialize the results though.
results is always a list of two different objects. The second object, in particular, is the most complex one, as its field names are not static. The UUID name references always change on each API response. For instance, the field named "163eee06-83a4-4fd8-bf65-5d6a3ef35ac5" might have another id in the next JSON response. Therefore, I cannot give it a proper field name in the corresponding Java class. The same goes for "d5760d01-84dd-42b2-8001-a69ebaa4c2df" and "fb366b72-d567-4d23-9f5f-356fc09aff6f" in this case.
Any ideas on how to properly deserialize this kind of JSON using Gson? I've tried a couple of different approaches, but nothing has truly worked so far.
In most recent attempt I tried to use the JsonDeserializer approach in order to differentiate the type of objects in the results list. My current implementation looks like this (getters and setters were hidden because of space):
QueryResponse.java
public class QueryResponse {
private List<Map<String, ResultInterface>> results;
private Pagination pagination;
private String timing;
}
Pagination.java
public class Pagination {
private Integer max;
private Integer offset;
private Integer nextOffset;
private Integer previousOffset;
}
ResultInterface.java
public interface ResultInterface {
}
ElementDefinition.java
public class ElementDefinition implements ResultInterface {
private String type;
private Name name;
private List<String> attrs;
}
Name.java
public class Name {
private String en;
private String es;
}
Compositions.java
public class Compositions implements ResultInterface {
private Map<String, Composition> compositions;
}
Composition.java
public class Composition {
private String date;
private List<Col> cols;
}
Col.java
public class Col {
private String type;
private String path;
private List<Value> values;
}
Value.java
public class Value {
private String instanceTemplatePath;
private String value;
private String magnitude;
private String units;
private String code;
private String terminology_id;
}
ResultInterfaceDeserializer.java
public class ResultInterfaceDeserializer implements JsonDeserializer<ResultInterface> {
#Override
public ResultInterface deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
JsonObject jObject = (JsonObject) json;
JsonElement typeObj = jObject.get("type");
if (typeObj != null) {
return context.deserialize(json, ElementDefinition.class);
} else {
return context.deserialize(json, Compositions.class);
}
}
}
I'm calling Gson like this:
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(ResultInterface.class, new ResultInterfaceDeserializer());
Gson gson = builder.create();
QueryResponse queryResponse = gson.fromJson(externalJsonResponse, QueryResponse.class);
The problem with this implementation is that there is nothing named compositions in the JSON structure, thus the Compositions.java class is not correctly identified. I know I have to use Java structures like Map<String, SomeObject>, but the problem is that there are too many dynamically named Json fields here, and I cannot "grab" them if they have no fixed name identifier.
UPDATE
I managed to find a solution. I'd say it's actually a workaround and probably not the most clean or elegant solution.
The problem with my current implementation was that I was trying to "grab" a JSON field called compositions when in fact it didn't exist. So, I decided to manipulate the JSON and add that field myself (in the code).
I changed the deserializer class to:
public class ResultInterfaceDeserializer implements JsonDeserializer<ResultInterface> {
public String encloseJsonWithCompositionsField(JsonElement json) {
return "{\"compositions\":" + json.toString() + "}";
}
#Override
public ResultInterface deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
JsonObject jObject = (JsonObject) json;
if (jObject.get("type") != null) {
return context.deserialize(json, ElementDefinition.class);
} else {
JsonElement jsonWithCompositionsField = new JsonParser().parse(encloseJsonWithCompositionsField(json));
return context.deserialize(jsonWithCompositionsField, Compositions.class);
}
}
}
With this change, I can now "grab" the compositions field and get the data in Java POJOs.

You could probably solve this by registering an additional JsonDeserializer for Compositions:
public class CompositionsDeserializer implements JsonDeserializer<Compositions> {
public static final CompositionsDeserializer INSTANCE = new CompositionsDeserializer();
private CompositionsDeserializer() { }
#Override
public Compositions deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
Compositions compositions = new Compositions();
Map<String, Composition> compositionsMap = new HashMap<>();
compositions.compositions = compositionsMap;
JsonObject compositionsJson = json.getAsJsonObject();
for (Map.Entry<String, JsonElement> compositionEntry : compositionsJson.entrySet()) {
Composition composition = context.deserialize(compositionEntry.getValue(), Composition.class);
compositionsMap.put(compositionEntry.getKey(), composition);
}
return compositions;
}
}
And then register that deserializer on the GsonBuilder as well:
Gson gson = new GsonBuilder()
.registerTypeAdapter(ResultInterface.class, new ResultInterfaceDeserializer())
.registerTypeAdapter(Compositions.class, CompositionsDeserializer.INSTANCE)
.create();

Related

Gson: Derserialize a dynamic field

I'm using a very strange api, it's data field type is dynamic.
If error occured, the data field will be a string like this:
{
"code": 2001,
"data": "Error!"
}
And if success, the data field will be a object:
{
"code": 2000,
"data": {
"id": 1,
"name": "example"
}
}
I'm using the following Kotlin code to de-serialize it:
return Gson().fromJson(data, SimpleModel::class.java)
The model definition is down below:
import com.google.gson.annotations.SerializedName
class SimpleModel {
#SerializedName("code")
val code = 0
#SerializedName("data")
val data: SimpleData = SimpleData()
}
class SimpleData {
#SerializedName("id")
val id = ""
#SerializedName("name")
val name = ""
}
When no error occred, the code above works just fine. However when error occured, exception was thrown:
java.lang.IllegalStateException: Excepted BEGIN_OBJECT but was STRING at line 1 column x $path.data
Is there a way to de-serialize data field to an object or just anything and determine it's type by code manually?
You would need to write a custom deserializer, which decides what type to deserialize data into, depending on the runtime type of the node, and register that deserializer with your Gson instance. Unfortunately i am not familiar with kotlin syntax, so i can only give you pseudo code.
Field data in SimpleModel should be either Object, or make the class generic - SimpleModel<T>, and the field should be of type T as well.
Parse the input to gson's node type - JsonElement.
Get data field
JsonElement root = parseResponse();
root.getAsJsonObject().get("data").getAsString();
Use getAs...() methods to check type.
Get as string. If success, it's a string and set the string value in SimpleModel.
If you get exception getting as string, get it as object - getAsJsonObject(), parse the object to SimpleData and set this new object in SimpleModel.
You could use my my answer here as inspiration. Although it's about object mapper, it does the same thing - decides object type depending on the node type, and follows roughly the same algorithm i described above.
Also this guide has info about how to write yor own deserialzer and registering it.
Like the previous answer, I am not familiarized with Kotlin, and the following solution is in Java, but as I know it is easy to convert Java to Kotlin using IntelliJ built-in tools.
The success/error objects pair is a classic problem, and you can create your own way to solve it, but let's consider the following classes represent the success and error objects respectively (Java 17, pattern matching on switch enabled then):
abstract sealed class SimpleModel<T>
permits SimpleModelSuccess, SimpleModelError {
#SerializedName("code")
final int code;
SimpleModel(final int code) {
this.code = code;
}
}
final class SimpleModelSuccess<T>
extends SimpleModel<T> {
#SerializedName("data")
final T data;
private SimpleModelSuccess(final int code, final T data) {
super(code);
this.data = data;
}
}
final class SimpleModelError<T>
extends SimpleModel<T> {
#SerializedName("data") // the annotation is helping here!
final String message;
private SimpleModelError(final int code, final String message) {
super(code);
this.message = message;
}
}
The code above can explain itself. Now the core part that required more work than I thought before by providing you my first comment that appeared incomplete.
#RequiredArgsConstructor(access = AccessLevel.PRIVATE)
final class SimpleModelTypeAdapterFactory
implements TypeAdapterFactory {
#Getter
private static final TypeAdapterFactory instance = new SimpleModelTypeAdapterFactory();
#Override
#Nullable
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
if ( !SimpleModel.class.isAssignableFrom(typeToken.getRawType()) ) {
return null;
}
// let's figure out what the model is parameterized with
final Type type = typeToken.getType();
final Type typeParameter;
if ( type instanceof ParameterizedType parameterizedType ) {
typeParameter = parameterizedType.getActualTypeArguments()[0];
} else {
throw new UnsupportedOperationException("Cannot infer type parameter from " + type);
}
// then borrow their respective type adapters for both success and error cases
#SuppressWarnings("unchecked")
final TypeAdapter<T> successDelegate = (TypeAdapter<T>) gson.getDelegateAdapter(this, TypeToken.getParameterized(SimpleModelSuccess.class, typeParameter));
#SuppressWarnings("unchecked")
final TypeAdapter<T> errorDelegate = (TypeAdapter<T>) gson.getDelegateAdapter(this, TypeToken.getParameterized(SimpleModelError.class, typeParameter));
return new TypeAdapter<>() {
#Override
public void write(final JsonWriter out, final T value) {
throw new UnsupportedOperationException();
}
#Override
public T read(final JsonReader in) {
// buffer the JSON tree first
// note that this solution may be very inefficient under some circumstances
final JsonObject buffer = Streams.parse(in).getAsJsonObject();
final JsonElement dataElement = buffer.get("data");
// is it's data is {...}, the consider it is success (by the way, what is code about?)
if ( dataElement.isJsonObject() ) {
return successDelegate.fromJsonTree(buffer);
}
// if it's a primitive, consider it's an error
if ( dataElement.isJsonPrimitive() ) {
return errorDelegate.fromJsonTree(buffer);
}
// well we've done our best...
throw new JsonParseException(String.format("Cannot deduce the model for %s", buffer.getClass()));
}
};
}
}
public final class SimpleModelTypeAdapterFactoryTest {
private static final class SomeJsonProvider
implements ArgumentsProvider {
#Override
public Stream<? extends Arguments> provideArguments(final ExtensionContext context) {
return Stream.of(
Arguments.of(
"""
{
"code": 2000,
"data": {
"id": 1,
"name": "example"
}
}
"""
),
Arguments.of(
"""
{
"code": 2001,
"data": "Error!"
}
"""
)
);
}
}
#AllArgsConstructor(access = AccessLevel.PRIVATE)
#ToString
private static final class SimpleData {
private final String id;
private final String name;
}
private static final Type simpleModelOfSimpleDataType = TypeToken.getParameterized(SimpleModel.class, SimpleData.class)
.getType();
#ParameterizedTest
#ArgumentsSource(SomeJsonProvider.class)
public void test(final String json) {
final Gson gson = new GsonBuilder()
.disableHtmlEscaping()
.disableInnerClassSerialization()
.registerTypeAdapterFactory(SimpleModelTypeAdapterFactory.getInstance())
.create();
final SimpleModel<SimpleData> model = gson.fromJson(json, simpleModelOfSimpleDataType);
switch ( model ) {
case SimpleModelSuccess<SimpleData> success -> System.out.println(success.data);
case SimpleModelError<SimpleData> error -> System.out.println(error.message);
}
}
}
Here is what it prints to stdout:
SimpleModelTypeAdapterFactoryTest.SimpleData(id=1, name=example)
Error!
Well, yeah, this is a "bit" more tricky than it was suggested by my first comment.

How to work around Gson serialization giving different results when using generic wildcards?

Consider this example:
static class BaseBean { String baseField = "base"; }
static class ChildBean extends BaseBean { String childField = "child"; }
static class BaseBeanHolder {
List <? extends BaseBean> beans;
public BaseBeanHolder(List<? extends BaseBean> beans) { this.beans = beans; }
}
static class ChildBeanHolder {
List <ChildBean> beans;
public ChildBeanHolder(List<ChildBean> beans) { this.beans = beans; }
}
#Test
public void mcve() {
BaseBeanHolder baseHolder = new BaseBeanHolder(singletonList(new ChildBean()));
System.out.println(new Gson().toJson(baseHolder));
ChildBeanHolder childHolder = new ChildBeanHolder(singletonList(new ChildBean()));
System.out.println(new Gson().toJson(childHolder));
}
It prints:
{"beans":[{"baseField":"base"}]}
{"beans":[{"childField":"child","baseField":"base"}]}
So, although both lists hold child objects, only the second holder results in the child fields being serialized to JSON.
I have seen other questions, like here but I wondering whether there are reasonable workarounds to achieve what I want.
In other words: is there a way to have such one "holder" class that accepts either BaseBeans or ChildBeans (the <? extends BaseBean> does that), and that also gives me the correct results when serialising instances with Gson into JSON strings?
( note: I can't use specific type adapters, as I have no control where that actual Gson instance is coming from and how it is configured in our environment )
Generally collection implementations "takes" type from collection field declaration - not from given item on the List/Set/etc. We need to write custom serialiser which for each item find serialiser and use it. Simple implementation:
class TypeAwareListJsonSeserializer implements JsonSerializer<List<?>> {
#Override
public JsonElement serialize(List<?> src, Type typeOfSrc, JsonSerializationContext context) {
if (src == null) {
return JsonNull.INSTANCE;
}
JsonArray array = new JsonArray();
for (Object item : src) {
JsonElement jsonElement = context.serialize(item, item.getClass());
array.add(jsonElement);
}
return array;
}
}
And this is how we can use it:
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonNull;
import com.google.gson.JsonSerializationContext;
import com.google.gson.JsonSerializer;
import com.google.gson.annotations.JsonAdapter;
import java.lang.reflect.Type;
import java.util.Arrays;
import java.util.List;
public class GsonApp {
public static void main(String[] args) throws Exception {
List<BaseBean> children = Arrays.asList(new BaseBean(), new ChildBean(), new ChildBean2());
BaseBeanHolder baseHolder = new BaseBeanHolder(children);
Gson gson = new GsonBuilder()
.setPrettyPrinting()
.create();
System.out.println(gson.toJson(baseHolder));
}
}
class BaseBean {
String baseField = "base";
}
class ChildBean extends BaseBean {
String childField = "child";
}
class ChildBean2 extends BaseBean {
int bean2Int = 356;
}
class BaseBeanHolder {
#JsonAdapter(TypeAwareListJsonSeserializer.class)
private List<? extends BaseBean> beans;
// getters, setters, toString
}
Above code prints:
{
"beans": [
{
"baseField": "base"
},
{
"childField": "child",
"baseField": "base"
},
{
"bean2Int": 356,
"baseField": "base"
}
]
}
EDIT
During serialisation we lose information about type which will be needed during deserialisation process. I developed simple type information which will be stored during serialisation and used in deserialisation. It could look like below:
class TypeAwareListJsonAdapter implements JsonSerializer<List<?>>, JsonDeserializer<List<?>> {
private final String typeProperty = "#type";
#Override
public JsonElement serialize(List<?> src, Type typeOfSrc, JsonSerializationContext context) {
if (src == null) {
return JsonNull.INSTANCE;
}
JsonArray array = new JsonArray();
for (Object item : src) {
JsonObject jsonElement = (JsonObject) context.serialize(item, item.getClass());
jsonElement.addProperty(typeProperty, item.getClass().getSimpleName());
array.add(jsonElement);
}
return array;
}
#Override
public List<?> deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
final Type elementType = $Gson$Types.getCollectionElementType(typeOfT, List.class);
if (json instanceof JsonArray) {
final JsonArray array = (JsonArray) json;
final int size = array.size();
if (size == 0) {
return Collections.emptyList();
}
final List<?> suites = new ArrayList<>(size);
for (int i = 0; i < size; i++) {
JsonObject jsonElement = (JsonObject) array.get(i);
String simpleName = jsonElement.get(typeProperty).getAsString();
suites.add(context.deserialize(jsonElement, getClass(simpleName, elementType)));
}
return suites;
}
return Collections.emptyList();
}
private Type getClass(String simpleName, Type defaultType) {
try {
// you can use mapping or something else...
return Class.forName("com.model." + simpleName);
} catch (ClassNotFoundException e) {
return defaultType;
}
}
}
The biggest problem is to how to map classes to JSON values. We can use class simple name or provide Map<String, Class> and use it. Now, we can use it as above. Example app prints now:
{
"beans": [
{
"baseField": "base",
"#type": "BaseBean"
},
{
"childField": "child",
"baseField": "base",
"#type": "ChildBean"
},
{
"bean2Int": 356,
"baseField": "base",
"#type": "ChildBean2"
}
]
}
BaseBean{baseField='base'}
ChildBean{baseField='base', childField='child'}
ChildBean2{baseField='base', bean2Int=356}
Gson is built in consideration of "I am going to be used to serialize" and "I am going to be used to deserialize".
There is no way to determine from raw JSON what the exact runtime type of a descendant of BaseBean is.
You can use RuntimeTypeAdapterFactory as described here - unfortunately it's not published with the base Gson module nor is it in Maven Central as described here. This will publish enough information with the JSON that'll allow Gson to deserialize it.
More of an addendum: I just figured that at least serialization works fine with arrays, so a simple workaround was to rework the holder:
static class BaseBeanHolder {
BaseBean[] beans;
public BaseBeanHolder(BaseBean... beans) { this.beans = beans; }
}

Parse JSONArray which contains primitive and objects

I have JSON response which looks like that:
{
"response":[
"Some number (for example 8091)",
{
"Bunch of primitives inside the first JSONObject"
},
{
"Bunch of primitives inside the second JSONObject"
},
{
"Bunch of primitives inside the third JSONObject"
},
... (and so on)
]
}
So it's an array with first integer element and other elements are JSONObject.
I don't need integer element to be parsed. So how do I handle it using GSON?
I would solve this problem by creating a custom JsonDeserializer and registering it to your Gson instance before parsing. This custom deserializer would be set up to handle both ints and real objects.
First you need to build up a series of model objects to represent the data. Here's a template for what that might look like:
private static class TopLevel {
#SerializedName("response")
private final List<ResponseElement> elements;
private TopLevel() {
this.elements = null;
}
}
private static class ResponseInteger implements ResponseElement {
private final int value;
public ResponseInteger(int value) {
this.value = value;
}
}
private static class ResponseObject implements ResponseElement {
#SerializedName("id")
private final String id;
#SerializedName("text")
private final String text;
private ResponseObject() {
this.id = null;
this.text = null;
}
}
private interface ResponseElement {
// marker interface
}
TopLevel and ResponseObject have private constructors because they are going to let Gson set their fields using reflection, while ResponseInteger has a public constructor because we're going to manually invoke it from our custom deserializer.
Obviously you will have to fill out ResponseObject with the rest of its fields.
The deserializer is relatively simple. The json you posted contains only two kinds of elements, and we'll leverage this. Each time the deserializer is invoked, it checks whether the element is a primitive, and returns a ResponseInteger if so (or a ResponseObject if not).
private static class ResponseElementDeserializer implements JsonDeserializer<ResponseElement> {
#Override
public ResponseElement deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
if (json.isJsonPrimitive()) {
return new ResponseInteger(json.getAsInt());
}
else {
return context.deserialize(json, ResponseObject.class);
}
}
}
To use this deserializer, you'll have to register it with Gson using the GsonBuilder object.
private static Gson getGson() {
return new GsonBuilder()
.registerTypeAdapter(ResponseElement.class, new ResponseElementDeserializer())
.create();
}
And that's it. Now you can use this Gson object to easily parse TopLevel objects!
public void parseJson() {
TopLevel t = getGson().fromJson(json, TopLevel.class);
for (ResponseElement element : t.elements) {
System.out.println(element);
}
}
8061
[450602: Поздравляем!]
[451700: С реакцией чата и рассуждениями Папани после рипа..]
[451578: Помним...Любим...Скорбим...<br>2107 забирает лучших]
[451371: Земля тебе пухом братишка]
[451332: Доигрался, минус 900 экзов<br><br>R I P]
[451269: ]
[451242: https://www.twitch.tv/arthas подрубка<br><br>evilpapech.ru - скидка 30% на футболки!]
[451217: ]
[451181: или так це жерстко?]
[451108: ]
I used these toString() methods, which I omitted above for brevity:
#Override
public String toString() {
return Integer.toString(value);
}
#Override
public String toString() {
return "[" + id + ": " + text + "]";
}
Try this
Gson gson = new Gson();
// Reading from a file.
Example example = gson.fromJson(new FileReader("D:\\content.json"), Example.class);
POJO
package com.example;
public class Example {
private List<Integer> response = null;
public List<Integer> getResponse() {
return response;
}
public void setResponse(List<Integer> response) {
this.response = response;
}
}
Basically this structure is the wrong format for JSON data.
You need to remove the number, or put this number as a field in the same object like the one below (call ObjectA) and consider this is an array of ObjectA.
Then everything should work well. Try the code below:
public class Response {
#SerializedName("response")
#Expose
public List<ObjectA> objectA = null;
}
public class ObjectA {
#SerializedName("value")
#Expose
public Integer value;
#SerializedName("description")
#Expose
public String description;
}
Response response = new Gson().fromJson(responseString, Response.class);
Please use below ValueObject format which doesn't parse first integer element
public class ResponseVO {
public List<Response> response = new ArrayList();
public class Response {
public final long id;
public final long from_id;
...
}
}

Json response parser for Array or Object

I am writing a library to consume a Json API and I am facing a design problem when using Gson as the parsing library.
One of the endpoints returns an array of objects if everything goes well like so:
[
{
"name": "John",
"age" : 21
},
{
"name": "Sarah",
"age" : 32
},
]
However, the error schema for all the endpoints in the API is an json object instead of an array.
{
"errors": [
{
"code": 1001,
"message": "Something blew up"
}
]
}
The problem arises when modeling this in POJOs. Because the error schema is common for all the API endpoints, I decided to have an abstract ApiResponse class which will only map the errors attribute
public abstract class ApiResponse{
#SerializedName("errors")
List<ApiResponseError> errors;
}
public class ApiResponseError {
#SerializedName("code")
public Integer code;
#SerializedName("message")
public String message;
}
Now I would like to inherit from ApiResponse to have the error mapping "for free" and a POJO per API endpoint response. However, the top level json object for this response is an array (if the server succeeds to execute the request), so I can not create a new class to map it like I would like it.
I decided to still create a class extending ApiResponse:
public class ApiResponsePerson extends ApiResponse {
List<Person> persons;
}
And implemented a custom deserializer to correctly parse the json depending on the type of the top level object, and setting it to the correct field on the following class:
public class DeserializerApiResponsePerson implements JsonDeserializer<ApiResponsePerson> {
#Override
public ApiResponse deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
ApiResponsePerson response = new ApiResponsePerson();
if (json.isJsonArray()) {
Type personType = new TypeToken<List<Person>>() {}.getType();
response.persons = context.deserialize(json, personType);
return response;
}
if (json.isJsonObject()) {
JsonElement errorJson = json.getAsJsonObject().get("errors");
Type errorsType = new TypeToken<List<ApiResponseError>>() {}.getType();
response.errors = context.deserialize(errorJson, errorsType);
return response;
}
throw new JsonParseException("Unexpected Json for 'ApiResponse'");
}
}
Which I will then add to the Gson
Gson gson = new GsonBuilder()
.registerTypeAdapter(ApiResponsePerson.class, new DeserializerApiResponsePerson())
.create();
Is there any way to model this POJOs and have Gson recognize this structure without having to manually handle this scenario?
Is there any better way to accomplish this?
Am I missing any scenario where the deserializer might fail or not work as expected?
Thanks
Sometimes API responses do not fit statically typed languages like Java is very well. I would say that if you're facing a problem to align with a not very convenient response format, you have to write more code if you want it to be convenient for you. And in most cases Gson can help in such cases, but not for free.
Is there any way to model this POJOs and have Gson recognize this structure without having to manually handle this scenario?
No. Gson does not mix objects of different structure, so you still have to tell it your intentions.
Is there any better way to accomplish this?
I guess yes, for both modelling the response and implementing the way how such responses are parsed.
Am I missing any scenario where the deserializer might fail or not work as expected?
It's response format sensitive like all deserializers are, so in general it's good enough, but can be improved.
First off, let's consider you can have two cases only: a regular response and an error. This is a classic case, and it can be modelled like that:
abstract class ApiResponse<T> {
// A bunch of protected methods, no interface needed as we're considering it's a value type and we don't want to expose any of them
protected abstract boolean isSuccessful();
protected abstract T getData()
throws UnsupportedOperationException;
protected abstract List<ApiResponseError> getErrors()
throws UnsupportedOperationException;
// Since we can cover all two cases ourselves, let them all be here in this class
private ApiResponse() {
}
static <T> ApiResponse<T> success(final T data) {
return new SucceededApiResponse<>(data);
}
static <T> ApiResponse<T> failure(final List<ApiResponseError> errors) {
#SuppressWarnings("unchecked")
final ApiResponse<T> castApiResponse = (ApiResponse<T>) new FailedApiResponse(errors);
return castApiResponse;
}
// Despite those three protected methods can be technically public, let's encapsulate the state
final void accept(final IApiResponseConsumer<? super T> consumer) {
if ( isSuccessful() ) {
consumer.acceptSuccess(getData());
} else {
consumer.acceptFailure(getErrors());
}
}
// And make a couple of return-friendly accept methods
final T acceptOrNull() {
if ( !isSuccessful() ) {
return null;
}
return getData();
}
final T acceptOrNull(final Consumer<? super List<ApiResponseError>> errorsConsumer) {
if ( !isSuccessful() ) {
errorsConsumer.accept(getErrors());
return null;
}
return getData();
}
private static final class SucceededApiResponse<T>
extends ApiResponse<T> {
private final T data;
private SucceededApiResponse(final T data) {
this.data = data;
}
#Override
protected boolean isSuccessful() {
return true;
}
#Override
protected T getData() {
return data;
}
#Override
protected List<ApiResponseError> getErrors()
throws UnsupportedOperationException {
throw new UnsupportedOperationException();
}
}
private static final class FailedApiResponse
extends ApiResponse<Void> {
private final List<ApiResponseError> errors;
private FailedApiResponse(final List<ApiResponseError> errors) {
this.errors = errors;
}
#Override
protected boolean isSuccessful() {
return false;
}
#Override
protected List<ApiResponseError> getErrors() {
return errors;
}
#Override
protected Void getData()
throws UnsupportedOperationException {
throw new UnsupportedOperationException();
}
}
}
interface IApiResponseConsumer<T> {
void acceptSuccess(T data);
void acceptFailure(List<ApiResponseError> errors);
}
A trivial mapping for errors:
final class ApiResponseError {
// Since incoming DTO are read-only data bags in most-most cases, even getters may be noise here
// Gson can strip off the final modifier easily
// However, primitive values are inlined by javac, so we're cheating javac with Integer.valueOf
final int code = Integer.valueOf(0);
final String message = null;
}
And some values too:
final class Person {
final String name = null;
final int age = Integer.valueOf(0);
}
The second component is a special type adapter to tell Gson how the API responses must be deserialized. Note that type adapter, unlike JsonSerializer and JsonDeserializer work in streaming fashion not requiring the whole JSON model (JsonElement) to be stored in memory, thus you can save memory and improve the performance for large JSON documents.
final class ApiResponseTypeAdapterFactory
implements TypeAdapterFactory {
// No state, so it can be instantiated once
private static final TypeAdapterFactory apiResponseTypeAdapterFactory = new ApiResponseTypeAdapterFactory();
// Type tokens are effective value types and can be instantiated once per parameterization
private static final TypeToken<List<ApiResponseError>> apiResponseErrorsType = new TypeToken<List<ApiResponseError>>() {
};
private ApiResponseTypeAdapterFactory() {
}
static TypeAdapterFactory getApiResponseTypeAdapterFactory() {
return apiResponseTypeAdapterFactory;
}
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// Is it ApiResponse, a class we can handle?
if ( ApiResponse.class.isAssignableFrom(typeToken.getRawType()) ) {
// Trying to resolve its parameterization
final Type typeParameter = getTypeParameter0(typeToken.getType());
// And asking Gson for the success and failure type adapters to use downstream parsers
final TypeAdapter<?> successTypeAdapter = gson.getDelegateAdapter(this, TypeToken.get(typeParameter));
final TypeAdapter<List<ApiResponseError>> failureTypeAdapter = gson.getDelegateAdapter(this, apiResponseErrorsType);
#SuppressWarnings("unchecked")
final TypeAdapter<T> castTypeAdapter = (TypeAdapter<T>) new ApiResponseTypeAdapter<>(successTypeAdapter, failureTypeAdapter);
return castTypeAdapter;
}
return null;
}
private static Type getTypeParameter0(final Type type) {
// Is this type parameterized?
if ( !(type instanceof ParameterizedType) ) {
// No, it's raw
return Object.class;
}
final ParameterizedType parameterizedType = (ParameterizedType) type;
return parameterizedType.getActualTypeArguments()[0];
}
private static final class ApiResponseTypeAdapter<T>
extends TypeAdapter<ApiResponse<T>> {
private final TypeAdapter<T> successTypeAdapter;
private final TypeAdapter<List<ApiResponseError>> failureTypeAdapter;
private ApiResponseTypeAdapter(final TypeAdapter<T> successTypeAdapter, final TypeAdapter<List<ApiResponseError>> failureTypeAdapter) {
this.successTypeAdapter = successTypeAdapter;
this.failureTypeAdapter = failureTypeAdapter;
}
#Override
public void write(final JsonWriter out, final ApiResponse<T> value)
throws UnsupportedOperationException {
throw new UnsupportedOperationException();
}
#Override
public ApiResponse<T> read(final JsonReader in)
throws IOException {
final JsonToken token = in.peek();
switch ( token ) {
case BEGIN_ARRAY:
// Is it array? Assuming that the responses come as arrays only
// Otherwise a more complex parsing is required probably replaced with JsonDeserializer for some cases
// So reading the next value (entire array) and wrapping it up in an API response with the success-on state
return success(successTypeAdapter.read(in));
case BEGIN_OBJECT:
// Otherwise it's probably an error object?
in.beginObject();
final String name = in.nextName();
if ( !name.equals("errors") ) {
// Let it fail fast, what if a successful response would be here?
throw new MalformedJsonException("Expected errors` but was " + name);
}
// Constructing a failed response object and terminating the error object
final ApiResponse<T> failure = failure(failureTypeAdapter.read(in));
in.endObject();
return failure;
// A matter of style, but just to show the intention explicitly and make IntelliJ IDEA "switch on enums with missing case" to not report warnings here
case END_ARRAY:
case END_OBJECT:
case NAME:
case STRING:
case NUMBER:
case BOOLEAN:
case NULL:
case END_DOCUMENT:
throw new MalformedJsonException("Unexpected token: " + token);
default:
throw new AssertionError(token);
}
}
}
}
Now, how it all can be put together. Note that the responses do not expose their internals explicitly but rather requiring consumers to accept making its privates really encapsulated.
public final class Q43113283 {
private Q43113283() {
}
private static final String SUCCESS_JSON = "[{\"name\":\"John\",\"age\":21},{\"name\":\"Sarah\",\"age\":32}]";
private static final String FAILURE_JSON = "{\"errors\":[{\"code\":1001,\"message\":\"Something blew up\"}]}";
private static final Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(getApiResponseTypeAdapterFactory())
.create();
// Assuming that the Type instance is immutable under the hood so it might be cached
private static final Type personsApiResponseType = new TypeToken<ApiResponse<List<Person>>>() {
}.getType();
#SuppressWarnings("unchecked")
public static void main(final String... args) {
final ApiResponse<Iterable<Person>> successfulResponse = gson.fromJson(SUCCESS_JSON, personsApiResponseType);
final ApiResponse<Iterable<Person>> failedResponse = gson.fromJson(FAILURE_JSON, personsApiResponseType);
useFullyCallbackApproach(successfulResponse, failedResponse);
useSemiCallbackApproach(successfulResponse, failedResponse);
useNoCallbackApproach(successfulResponse, failedResponse);
}
private static void useFullyCallbackApproach(final ApiResponse<Iterable<Person>>... responses) {
System.out.println("<FULL CALLBACKS>");
final IApiResponseConsumer<Iterable<Person>> handler = new IApiResponseConsumer<Iterable<Person>>() {
#Override
public void acceptSuccess(final Iterable<Person> people) {
dumpPeople(people);
}
#Override
public void acceptFailure(final List<ApiResponseError> errors) {
dumpErrors(errors);
}
};
Stream.of(responses)
.forEach(response -> response.accept(handler));
}
private static void useSemiCallbackApproach(final ApiResponse<Iterable<Person>>... responses) {
System.out.println("<SEMI CALLBACKS>");
Stream.of(responses)
.forEach(response -> {
final Iterable<Person> people = response.acceptOrNull(Q43113283::dumpErrors);
if ( people != null ) {
dumpPeople(people);
}
});
}
private static void useNoCallbackApproach(final ApiResponse<Iterable<Person>>... responses) {
System.out.println("<NO CALLBACKS>");
Stream.of(responses)
.forEach(response -> {
final Iterable<Person> people = response.acceptOrNull();
if ( people != null ) {
dumpPeople(people);
}
});
}
private static void dumpPeople(final Iterable<Person> people) {
for ( final Person person : people ) {
System.out.println(person.name + " (" + person.age + ")");
}
}
private static void dumpErrors(final Iterable<ApiResponseError> errors) {
for ( final ApiResponseError error : errors ) {
System.err.println("ERROR: " + error.code + " " + error.message);
}
}
}
The code above will produce:
<FULL CALLBACKS>
John (21)
Sarah (32)
ERROR: 1001 Something blew up
<SEMI CALLBACKS>
John (21)
Sarah (32)
ERROR: 1001 Something blew up
<NO CALLBACKS>
John (21)
Sarah (32)
In your error-free case, since the top-level element is an array rather than an object, you have to use custom deserializers. You cannot escape from that. (I assume you cannot change the response formats.)
The best attempt to make the code cleaner, as far as I can see, is to create an abstract top-level deserializer class and check for error here. If there is no error, delegate parsing fields to some abstract method which will be implemented in custom serializers that you have written for each class.
This solution is almost very good for this scenario. But I would like to define the response more general, is there should be a status to identify success or failure for the request? So I prefer the json format to be like this:
for success:
{
"status": "success",
"results": [
{
"name": "John",
"age" : 21
}
]
}
for failure:
{
"status": "failure",
"errors": [
{
"code": 1001,
"message": "Something blew up"
}
]
}

What's the best way to handle JSON API responses using GSON?

I'm working on a project that communicates with an API using JSON. This is my first attempt at JSON and I've been away from java for a few/several years, so please bear with me.
Here is an idea of what the data looks like:
String 1:
[{
"apicall1":
[{
"thisField":"thisFieldData",
"thatField":"thatFieldData",
"anotherField":"anotherFieldData"
}]
}]
String 2:
[{
"apicall2":
[{
"thatField":"thatFieldData",
"someFieldsAreTheSame":"someFieldsAreTheSameData",
"otherFieldsAreNotTheSame":"otherFieldsAreNotTheSame"
}]
}]
As you can see from my data example, the API returns a JSON string that contains the api used. The array inside contains the data. The API's have a lot of data fields in common but they are unrelated beyond that.
EDIT: There are dozens of these API's types that will need to be handled.
What I am trying to do is create a response class that accepts all of the JSON strings and returns an object containing the appropriate data.
For Example:
Gson gson = new Gson(); //Custom TypeAdapter goes here if needed.
Response apicall2 = gson.fromJson(apicall2String, Response.class);
System.out.println(apicall2.thatField); //Prints thatFieldData
System.out.println(apicall2.someFieldsAreTheSame); //Prints someFieldsAreTheSameData
System.out.println(apicall2.otherFieldsAreNotTheSame); //Prints otherFieldsAreNotTheSameData
This is where I am lost. Here is what I have so far. I think I need to use a TypeAdapter here but haven't been able to figure how to apply that to my case.
public class Response { //Change to TypeAdapter possibly?
}
public class apicall1 {
String thisField;
String thatField;
String anotherField;
}
public class apicall2 {
String thatField;
String someFieldsAreTheSame;
String otherFieldsAreNotTheSame;
}
You can use Gson's TypeToken class to deserialize json into object. Below is an example:
JSON:
[{ "apicall1":
[{
"thisField":"thisFieldData",
"thatField":"thatFieldData",
"anotherField":"anotherFieldData"
}]
}]
Model:
class Response{
private List<Result> apicall1;
class Result{
private String thisField;
private String thatField;
private String anotherField;
public String getThisField() {
return thisField;
}
public void setThisField(String thisField) {
this.thisField = thisField;
}
public String getThatField() {
return thatField;
}
public void setThatField(String thatField) {
this.thatField = thatField;
}
public String getAnotherField() {
return anotherField;
}
public void setAnotherField(String anotherField) {
this.anotherField = anotherField;
}
}
public List<Result> getApicall1() {
return apicall1;
}
public void setApicall1(List<Result> apicall1) {
this.apicall1 = apicall1;
}
}
Converter:
public static void main(String[] args) {
String response = "[{ \"apicall1\": [{ \"thisField\":\"thisFieldData\", \"thatField\":\"thatFieldData\", \"anotherField\":\"anotherFieldData\" }]}]";
Gson gson = new Gson();
List<Response> responses = gson.fromJson(response, new TypeToken<List<Response>>(){}.getType());
System.out.println(responses.get(0).getApicall1().get(0).getThisField());
}
I don't know if you want both adapters in one class. Might not be the best OOP design.
To achieve it you would need to do something like so:
public class DoublyTypeAdapter implements JsonDeserializer<ApiCallTypeParent>
{
Gson gson = new Gson();
#Override
public ApiCallTypeParent deserialize(JsonElement jsonElement, Type type, JsonDeserializationContext jsonDeserializationContext)
throws JsonParseException {
JsonObject json = jsonElement.getAsJsonObject();
ApiCallTypeParent desrializeIntoMe;
// Detect which type to implement
if(apiTypeOne(type) {
desrializeIntoMe = new TypeOne();
} else {
desrializeIntoMe = new TypeTwo();
}
for (Map.Entry<String, JsonElement> entry : json.entrySet())
{
switch(entry.getKey()){
case "thisField":
desrializeIntoMe.setThisField(entry.getValue().getAsString());
break;
......
default: // We don't care
break;
}
}
return desrializeIntoMe ;
}
}

Categories