We Keep Coding

Retrofit Gson custom json generic converter

I've been working with Retrofit on a couple of my projects before but now I want to do something slightly different. I'm calling an api that wraps my response in a structure similar to this:
{ // only for demo purposes. Probably errors and data will never be populated together
"body": {
"errors": {
"username": [
"Username is too short",
"Username already exists"
]
},
"data": {
"message": "User created."
}
}
}
I'm trying to convert all that to a generic class which will wrap that response for me. What I have in mind is something like
public class ApiResponse<T> {
private T data;
private Map<String, List<String>> errors;
public ApiResponse(T data, Map<String, List<String>> errors) {
this.data = data;
this.errors = errors;
}
}
Where T can be any class.
I tried implementing a JsonDeserializer<ApiResponse<T>> based on some examples I found around the internet but I can't wrap my head around how to make it work as much automatically as possible and let Retrofit and Gson do the heavy lifting
My Converter class is as follows:
public class ApiResponseDeserializer<T> implements JsonDeserializer<ApiResponse<T>> {
private Class clazz;
public ApiResponseDeserializer(Class clazz) {
this.clazz = clazz;
}
#Override
public ApiResponse deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
final JsonObject jsonObject = json.getAsJsonObject();
final JsonObject body = jsonObject.getAsJsonObject("body");
final JsonObject errors = body.getAsJsonObject("errors");
final JsonObject data = body.getAsJsonObject("data");
Map<String, List<String>> parsedErrors = new HashMap<>();
for(String key : errors.keySet()) {
List<String> errorsList = new ArrayList<>();
JsonArray value = errors.getAsJsonArray(key);
Iterator<JsonElement> valuesIterator = value.iterator();
while(valuesIterator.hasNext()) {
String error = valuesIterator.next().getAsString();
errorsList.add(error);
}
parsedErrors.put(key, errorsList);
}
T parsedData = context.deserialize(data, clazz);
return new ApiResponse<T>(parsedData, parsedErrors);
}
}
and then when building my retrofit client
public static Retrofit getClient() {
if (okHttpClient == null) {
initOkHttp();
}
Gson gson = new GsonBuilder()
.registerTypeAdapter(ApiResponse.class, new ApiResponseDeserializer<>(......) // PROBLEM
.create();
if (retrofit == null) {
retrofit = new Retrofit.Builder()
.baseUrl(Const.API_BASE_URL)
.client(okHttpClient)
.addCallAdapterFactory(RxJava2CallAdapterFactory.create())
.addConverterFactory(GsonConverterFactory.create(gson))
.build();
}
return retrofit;
}
But I feel like it's not generic enough to be able to convert my classes automatically. And also I have no idea how should I hint Gson what type my data.
My endpoints are defined as follows:
#POST("users/signup")
Single<ApiResponse<RegisterResponseData>> register(#Body RegisterRequest request);
But how do I make a generic Retrofit instance with a generic Gson type adapter that knows how to convert my response to a ApiResponse<RegisterResponseData>? And knows that the data property from the response should be converted to an object of type RegisterResponseData...

When you specify return type in Retrofit's client it's passed to Retrofit's converter as Type and then Gson receives that type which will be your ApiResponse<RegisterResponseData>. From that point Gson will understand that data is of type RegisterResponseData and will produce your model object.
Just try it without your ApiResponseDeserializer and you'll see it's working.
Edit:
Answering your additional question in comments:
If you want to skip your "body" object in json you can write your wrapper object like this:
public class ApiResponse<T> {
#SerializedName("body")
private ApiResponseBody<T> body;
public ApiResponse() {
}
public ApiResponse(ApiData<T> body) {
this.body = body;
}
}
public class ApiResponseBody<T> {
#SerializedName("data")
private T data;
#SerializedName("errors")
private Map<String, List<String>> errors;
public ApiResponseBody() {
}
public ApiResponseBody(T data, Map<String, List<String>> errors) {
this.data = data;
this.errors = errors;
}
}
And use it in usual way
#POST("users/signup")
Single<ApiResponse<RegisterResponseData>> register(#Body RegisterRequest request);

How to have a single GSON custom serializer to apply to all subclasses?

I'm using GSON to apply a universal serializer to all subclasses of an abstract Base class. However, GSON will not call my serializer when given actual subclasses of the Base class unless explicitly told to use Base.class as a cast. Here's a simple instance of what I'm talking about.
public interface Base<T>{
String getName();
public List<Object> getChildren();
}
public class Derived1 implements Base<Integer>{
private Integer x = 5;
String getName(){
return "Name: " + x;
}
List<Object> getChildren(){
return Lists.newArrayList(new Derived2(), "Some string");
}
}
public class Derived2 implements Base<Double>{
private Double x = 6.3;
String getName(){
return "Name: " + x;
}
List<Object> getChildren(){
return new List<>();
}
}
I'm creating a serializer as follows:
JsonSerializer customAdapter = new JsonSerializer<Base>(){
#Override
JsonElement serialize(Base base, Type sourceType, JsonSerializationContext context){
JsonObject jsonObject = new JsonObject();
jsonObject.addProperty("name", base.getName());
JsonArray jsonArray = new JsonArray();
for (Object child : base.getChildren()){
jsonArray.add(context.serialize(child));
}
if (jsonArray.size() != 0){
jsonObject.add("children", jsonArray);
}
}
};
Gson customSerializer = new GsonBuilder()
.registerTypeAdapter(Base.class, customAdapter)
.create();
However, applying my custom serializer to a List of subclasses does not have the desired effect.
customSerializer.toJson(Lists.newArrayList(new Derived1(), new Derived2()));
This applies the default GSON serialization to my subclasses. Is there any easy way to get my custom serializer to use my custom adapter on all subclasses of the parent class? I suspect that one solution is to use reflection to iterate over all subclasses of Base and register the custom adapter, but I'd like to avoid something like that if possible.
Note: I don't care about deserialization right now.

Maybe you should not use JsonSerializer. Namely, this is possible if you use TypeAdapter doing the same magic by registering TypeAdapterFactory that tells Gson how to serialize any class.
See below TypeAdapterFactory and TypeAdapter in it:
public class CustomAdapterFactory implements TypeAdapterFactory {
#SuppressWarnings("unchecked")
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> typeToken) {
// If the class that type token represents is a subclass of Base
// then return your special adapter
if(Base.class.isAssignableFrom(typeToken.getRawType())) {
return (TypeAdapter<T>) customTypeAdapter;
}
return null;
}
private TypeAdapter<Base<?>> customTypeAdapter = new TypeAdapter<Base<?>>() {
#Override
public void write(JsonWriter out, Base<?> value) throws IOException {
out.beginObject();
out.value(value.getName());
out.endObject();
}
#Override
public Base<?> read(JsonReader in) throws IOException {
// Deserializing to subclasses not interesting yet.
// Actually it is impossible if the JSON does not contain
// information about the subclass to which to deserialize
return null;
}
};
}
If you do something like this:
#Slf4j
public class SubClassTest {
#Test
public void testIt() {
Gson gson = new GsonBuilder()
.setPrettyPrinting()
.registerTypeAdapterFactory(new CustomAdapterFactory())
.create();
log.info("\n{}", gson.toJson(new Derived1()));
log.info("\n{}", gson.toJson(new Derived2()));
}
}
the output will be like this:
2018-10-12 23:13:17.037 INFO
org.example.gson.subclass.SubClassTest:19 - { "name": "Name: 5" }
2018-10-12 23:13:17.043 INFO
org.example.gson.subclass.SubClassTest:20 - { "name": "Name: 6.3"
}
If it is not exactly what you want just fix the write(..) method in the customTypeAdapter.

Retrofit and Gson: parsing array/element-polymorphic objects

I am getting response in a sequence:
"parameters": {
"parameter": {
"Data":"value"
}
},
"parameters":{
"parameter": [
{
"Data":"value"
},
{
"Data":"value"
},
]
},
Getting the error if I call List<Class> parameter:
Expected BEGIN_OBJECT but getting BEGIN_ARRAY
I need to parse parameter to get values
public class ApiClient {
public static final String BASE_URL ="http://.........";
private static Retrofit retrofit = null;
public static Retrofit getClient() {
OkHttpClient client = new OkHttpClient.Builder()
.connectTimeout(1, TimeUnit.MINUTES)
.writeTimeout(1, TimeUnit.MINUTES)
.readTimeout(1, TimeUnit.MINUTES)
.addInterceptor(new ServiceGenerator("Content-Type","application/json")).build();
Gson gson = new GsonBuilder()
.setLenient()
.create();
if (retrofit==null) {
retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create(gson))
.client(client)
.build();
}
return retrofit;
}
}
public class ServiceGenerator implements Interceptor{
private String httpUsername;
private String httpPassword;
public ServiceGenerator(String httpUsername, String httpPassword) {
this.httpUsername = httpUsername;
this.httpPassword = httpPassword;
}
#Override
public Response intercept(Chain chain) throws IOException {
Request newRequest = chain.request().newBuilder()
.addHeader("Authorization", getAuthorizationValue())
.build();
return chain.proceed(newRequest);
}
private String getAuthorizationValue() {
final String userAndPassword = httpUsername + ":" + httpPassword;
return "Basic " + Base64.encodeToString(userAndPassword.getBytes(), Base64.NO_WRAP);
}
}
#POST("OneWay.json")
Call<ApiResponse> sendOneWay(#Body Query data);
#SerializedName("FlightDetails")
public ApiResponse FlightDetails;
Now I called a Class ApiResponse
But How to call both
public ApiResponse FlightDetails; & public List FlightDetails;

This is just a very trivial issue that occurs often with APIs that have weird design choices. You just have to "align" both formats to a unified form: lists can cover both cases. So, all you have to implement is a type adapter that would check if such an alignment is necessary and use either the original type adapter if the value is a list, or wrap it up in a single element list.
For simplicity, consider the following JSON documents:
single.json
{
"virtual": {
"key-1": "value-1"
}
}
multiple.json
{
"virtual": [
{
"key-1": "value-1"
},
{
"key-2": "value-2"
}
]
}
Now define a mapping with the aligned field:
final class Response {
#JsonAdapter(AlwaysListTypeAdapterFactory.class)
final List<Map<String, String>> virtual = null;
}
Note the JsonAnnotaion annotation: this is a way to tell Gson how the field must be read or written. The AlwaysListTypeAdapterFactory implementation might be as follows:
final class AlwaysListTypeAdapterFactory
implements TypeAdapterFactory {
// Always consider making constructors private
// + Gson can instantiate this factory itself
private AlwaysListTypeAdapterFactory() {
}
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// Not a list?
if ( !List.class.isAssignableFrom(typeToken.getRawType()) ) {
// Not something we can to deal with
return null;
}
// Now just return a special type adapter that could detect how to deal with objects
#SuppressWarnings("unchecked")
final TypeAdapter<T> castTypeAdapter = (TypeAdapter<T>) new AlwaysListTypeAdapter<>(
(TypeAdapter<Object>) gson.getAdapter(TypeToken.get(getTypeParameter0(typeToken.getType()))),
(TypeAdapter<List<Object>>) gson.getAdapter(typeToken)
);
return castTypeAdapter;
}
// This is used to detect the list parameterization
private static Type getTypeParameter0(final Type type) {
if ( !(type instanceof ParameterizedType) ) {
// Is it a wildcard or raw type? Then we cannot determine the real parameterization
return Object.class;
}
// Or just resolve the actual E in List<E>
final ParameterizedType parameterizedType = (ParameterizedType) type;
return parameterizedType.getActualTypeArguments()[0];
}
private static final class AlwaysListTypeAdapter<E>
extends TypeAdapter<List<E>> {
private final TypeAdapter<E> elementTypeAdapter;
private final TypeAdapter<List<E>> listTypeAdapter;
private AlwaysListTypeAdapter(final TypeAdapter<E> elementTypeAdapter, final TypeAdapter<List<E>> listTypeAdapter) {
this.elementTypeAdapter = elementTypeAdapter;
this.listTypeAdapter = listTypeAdapter;
}
#Override
public void write(final JsonWriter out, final List<E> value)
throws IOException {
listTypeAdapter.write(out, value);
}
#Override
public List<E> read(final JsonReader in)
throws IOException {
final JsonToken token = in.peek();
switch ( token ) {
case BEGIN_ARRAY:
// If the next token is [, assume is a normal list, and just delegate the job to Gson internals
return listTypeAdapter.read(in);
case BEGIN_OBJECT:
case STRING:
case NUMBER:
case BOOLEAN:
case NULL:
// Any other value? Wrap it up ourselves, but use the element type adapter
// Despite Collections.singletonList() might be used, Gson returns mutable ArrayList instances, so we do...
final List<E> list = new ArrayList<>();
list.add(elementTypeAdapter.read(in));
return list;
case END_ARRAY:
case END_OBJECT:
case NAME:
case END_DOCUMENT:
// Something terrible here...
throw new MalformedJsonException("Unexpected token: " + token + " at " + in);
default:
// If someday Gson adds a new token
throw new AssertionError(token);
}
}
}
}
The test:
public static void main(final String... args)
throws IOException {
for ( final String resource : ImmutableList.of("single.json", "multiple.json") ) {
try ( final Reader reader = getPackageResourceReader(Q43634110.class, resource) ) {
final Response response = gson.fromJson(reader, Response.class);
System.out.println(resource);
System.out.println("\t" + response.virtual);
}
}
}
Output:
single.json
[{key-1=value-1}]
multiple.json
[{key-1=value-1}, {key-2=value-2}]

You could use this website to generate the java object for you
http://www.jsonschema2pojo.org/ just put the json response and choose Json for Source type and Gson for Annotation style.
and copy generated java class to your application and use it for the retrofit response .

The problem which you have here is that for the same json field you have different types. So the first time you are getting a JSON object and the second time a JSON array and this obviously will crash as you strictly defined to be parsed as an array (List).
You need to handle this case dynamically by your side or ask by the API guys to fix the bad data structure which seems you are getting back (except if it's on purpose like that).
To understand better the JSON types read this http://www.json.org/

Gson custom serialization

I wish to have a custom GSON deserializer such that whenever it is deserializing a JSON object (i.e. anything within curly brackets { ... }), it will look for a $type node and deserialize using its inbuilt deserializing capability to that type. If no $type object is found, it just does what it normal does.
So for example, I would want this to work:
{
"$type": "my.package.CustomMessage"
"payload" : {
"$type": "my.package.PayloadMessage",
"key": "value"
}
}
public class CustomMessage {
public Object payload;
}
public class PayloadMessage implements Payload {
public String key;
}
Calling: Object customMessage = gson.fromJson(jsonString, Object.class).
So currently if I change the payload type to the Payload interface:
public class CustomMessage {
public Payload payload;
}
Then the following TypeAdapaterFactory will do what I want:
final TypeAdapter<T> delegate = gson.getDelegateAdapter(this, type);
final TypeAdapter<JsonElement> elementAdapter = gson.getAdapter(JsonElement.class);
final PojoTypeAdapter thisAdapter = this;
public T read(JsonReader reader) throws IOException {
JsonElement jsonElement = (JsonElement)elementAdapter.read(reader);
if (!jsonElement.isJsonObject()) {
return delegate.fromJsonTree(jsonElement);
}
JsonObject jsonObject = jsonElement.getAsJsonObject();
JsonElement typeElement = jsonObject.get("$type");
if (typeElement == null) {
return delegate.fromJsonTree(jsonElement);
}
try {
return (T) gson.getDelegateAdapter(
thisAdapter,
TypeToken.get(Class.forName(typeElement.getAsString()))).fromJsonTree(jsonElement);
} catch (ClassNotFoundException ex) {
throw new IOException(ex.getMessage());
}
}
However, I would like it to work when payload is of type Object or any type for that matter, and throw some sort of type match exception if it can't assign the variable.

Looking at the source for Gson, I have found what I think is the issue:
// built-in type adapters that cannot be overridden
factories.add(TypeAdapters.JSON_ELEMENT_FACTORY);
factories.add(ObjectTypeAdapter.FACTORY);
// user's type adapters
factories.addAll(typeAdapterFactories);
As you can see the ObjectTypeAdapter will take precedence over my factory.
The only solution as far as I can see is to use reflection to remove the ObjectTypeAdapter from the list or insert my factory before it. I have done this and it works.

I don't know how you can achieve it with Gson but you have such a feature in Genson by default.
To enable it just do:
Genson genson = new Genson.Builder().setWithClassMetadata(true).create();
You can also register aliases for your class names:
Genson genson = new Genson.Builder().addAlias("myClass", my.package.SomeClass.class).create();
This has however some limitations:
at the moment you can't change the key used to identify the type, it is #class
it must be present in your json before the other properties - but looks fine as it is the case in your examples
Works only with json objects and not arrays or litterals

This code skeleton works on your example but should be improved and tested with different scenarios.
public class PojoTypeAdapaterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> type) {
// check types we support
if (type.getRawType().isAssignableFrom(CustomMessage.class) || type.getRawType().isAssignableFrom(PayloadMessage.class)) {
return new PojoTypeAdapter<T>(gson, type);
}
else return null;
}
private class PojoTypeAdapter<T> extends TypeAdapter<T> {
private Gson gson;
private TypeToken<T> type;
private PojoTypeAdapter(final Gson gson, final TypeToken<T> type) {
this.gson = gson;
this.type = type;
}
public T read(JsonReader reader) throws IOException {
final TypeAdapter<T> delegate = gson.getDelegateAdapter(PojoTypeAdapaterFactory.this, this.type);
final TypeAdapter<JsonElement> elementAdapter = this.gson.getAdapter(JsonElement.class);
JsonElement jsonElement = elementAdapter.read(reader);
if (!jsonElement.isJsonObject()) {
return (T) this.gson.getAdapter(JsonElement.class).fromJsonTree(jsonElement);
}
JsonObject jsonObject = jsonElement.getAsJsonObject();
JsonElement typeElement = jsonObject.get("$type");
if (typeElement == null) {
return delegate.fromJsonTree(jsonElement);
}
try {
final Class myClass = Class.forName(typeElement.getAsString());
final Object myInstance = myClass.newInstance();
final JsonObject jsonValue = jsonElement.getAsJsonObject().get("value").getAsJsonObject();
for (Map.Entry<String, JsonElement> jsonEntry : jsonValue.entrySet()) {
final Field myField = myClass.getDeclaredField(jsonEntry.getKey());
myField.setAccessible(true);
Object value = null;
if (jsonEntry.getValue().isJsonArray()) {
//value = ...;
}
else if (jsonEntry.getValue().isJsonPrimitive()) {
final TypeAdapter fieldAdapter = this.gson.getAdapter(myField.getType());
value = fieldAdapter.fromJsonTree(jsonEntry.getValue());
}
else if (jsonEntry.getValue().isJsonObject()) {
value = this.fromJsonTree(jsonEntry.getValue());
}
myField.set(myInstance, value);
}
return (T) myInstance;
}
catch (ClassNotFoundException | IllegalAccessException | IllegalArgumentException | InstantiationException | NoSuchFieldException | SecurityException e) {
throw new IOException(e);
}
}
#Override
public void write(final JsonWriter out, final T value) throws IOException {
out.beginObject();
out.name("$type");
out.value(value.getClass().getName());
out.name("value");
final TypeAdapter<T> delegateAdapter = (TypeAdapter<T>) this.gson.getDelegateAdapter(PojoTypeAdapaterFactory.this, TypeToken.<T>get(value.getClass()));
delegateAdapter.write(out, value);
out.endObject();
}
}
}
The generated JSON is not exactly the same though, as it contains an additional value entry:
{
"$type": "my.package.CustomMessage",
"value": {
"payload": {
"$type": "my.package.PayloadMessage",
"value": {
"key": "hello"
}
}
}
}

Parsing single json entry to multiple objects with Gson

I have a json response that looks like this:
{
"id":"001",
"name":"Name",
"param_distance":"10",
"param_sampling":"2"
}
And I have two classes: Teste and Parameters
public class Test {
private int id;
private String name;
private Parameters params;
}
public class Parameters {
private double distance;
private int sampling;
}
My question is: is there a way to make Gson understand that some of the json attributes should go to the Parameters class, or the only way is to "manually" parse this ?
EDIT
Well, just to make my comment in #MikO's answer more readable:
I'll add a list of an object to the json output, so json response should look like this:
{
"id":"001",
"name":"Name",
"param_distance":"10",
"param_sampling":"2",
"events":[
{
"id":"01",
"value":"22.5"
},
{
"id":"02",
"value":"31.0"
}
]
}
And the Deserializer class would look like this:
public class TestDeserializer implements JsonDeserializer<Test> {
#Override
public Test deserialize(JsonElement json, Type type,
JsonDeserializationContext context) throws JsonParseException {
JsonObject obj = json.getAsJsonObject();
Test test = new Test();
test.setId(obj.get("id").getAsInt());
test.setName(obj.get("name").getAsString());
Parameters params = new Parameters();
params.setDistance(obj.get("param_distance").getAsDouble());
params.setSampling(obj.get("param_sampling").getAsInt());
test.setParameters(params);
Gson eventGson = new Gson();
Type eventsType = new TypeToken<List<Event>>(){}.getType();
List<Event> eventList = eventGson.fromJson(obj.get("events"), eventsType);
test.setEvents(eventList);
return test;
}
}
And doing:
GsonBuilder gBuilder = new GsonBuilder();
gBuilder.registerTypeAdapter(Test.class, new TestDeserializer());
Gson gson = gBuilder.create();
Test test = gson.fromJson(reader, Test.class);
Gives me the test object the way I wanted.

The way to make Gson understand it is to write a custom deserializer by creating a TypeAdapter for your Test class. You can find information in Gson's User Guide. It is not exactly a manual parsing, but it is not that different, since you have to tell Gson how to deal with each JSON value...
It should be something like this:
private class TestDeserializer implements JsonDeserializer<Test> {
public Test deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
throws JsonParseException {
JsonObject obj = json.getAsJsonObject();
int id = obj.get("id").getAsInt();
String name = obj.get("name").getAsString();
double distance = obj.get("param_distance").getAsDouble();
int sampling = obj.get("param_sampling").getAsInt();
//assuming you have suitable constructors...
Test test = new Test(id, name, new Parameters(distance, sampling));
return test;
}
}
Then you have to register the TypeAdapter with:
GsonBuilder gson = new GsonBuilder();
gson.registerTypeAdapter(Test.class, new TestDeserializer());
And finally you just have to parse your JSON as usual, with:
gson.fromJson(yourJsonString, Test.class);
Gson will automatically use your custom deserializer to parse your JSON into your Test class.