We Keep Coding

GSON TypeAdapter: DeSerialize Polymorphic Objects Based on "Type" Field

I'm using Retrofit with the default Gson parser for JSON processing. Oftentimes, I have a series of 4~5 related but slightly different objects, which are all subtypes of a common base (let's call it "BaseType"). I know we can deserialize the different JSONs to their respective child models by checking the "type" field. The most commonly prescribed way is to extend a JsonDeserializer and register it as a type adapter in the Gson instance:
class BaseTypeDeserializer implements JsonDeserializer<BaseType> {
private static final String TYPE_FIELD = "type";
#Override
public BaseType deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
if (json.isJsonObject() && json.getAsJsonObject().has(TYPE_FIELD)) {
JsonObject jsonObject = json.getAsJsonObject();
final String type = jsonObject.get(TYPE_FIELD).getAsString();
if ("type_a".equals(type)) {
return context.deserialize(json, AType.class);
} else if ("type_b".equals(type)) {
return context.deserialize(json, BType.class);
} ...
// If you need to deserialize as BaseType,
// deserialize without the current context
// or you will infinite loop
return new Gson().fromJson(json, typeOfT);
} else {
// Return a blank object on error
return new BaseType();
}
}
}
However, in my experience this is really slow, and seemingly because we have to load up the entire JSON document into a JsonElement and then traverse it to find the type field. I also don't like it that this deserializer has to be run on every one of our REST calls, even though the data isn't always necessarily being mapped to a BaseType (or its children).
This foursquare blog post mentioned using TypeAdapters as an alternative but it didn't really go further with an example.
Anybody here know how to use TypeAdapterFactory to deserialize based on a 'type' field without having to read up the entire json stream into a JsonElement object tree?

The custom deserializer should only be run when you have a BaseType or a sub-classes in the deserialization data, not every request. You register it based on the type, and it is only called when gson need to serialize that type.
Do you deserialize BaseType as well as the sub-classes? If so, this line is going to kill your performance --
return new Gson().fromJson(json, typeOfT);
creation of new Gson objects is not cheap. You are creating one each time you deserialize a base class object. Moving this call to a constructor of BaseTypeDeserializer and stashing it in a member variable will improve performance (assuming you do deserialize the base class).
The issue with creating a TypeAdapter or TypeAdapterFactory for selecting type based on the field is that you need to know the type before you start consuming the stream. If the type field is part of the object, you cannot know the type at that point. The post you linked to mentions as much --
Deserializers written using TypeAdapters may be less flexible than
those written with JsonDeserializers. Imagine you want a type field to
determine what an object field deserializes to. With the streaming
API, you need to guarantee that type comes down in the response before
object.
If you can get the type before the object in the JSON stream, you can do it, otherwise your TypeAdapter implementation is probably going to mirror your current implementation, except that the first thing you do is convert to Json tree yourself so you can find the type field. That is not going to save you much over your current implementation.
If your subclasses are similar and you don't have any field conflicts between them (fields with the same name but different types), you can use a data transfer object that has all the fields. Use gson to deserialize that, and then use it create your objects.
public class MyDTO {
String type;
// Fields from BaseType
String fromBase;
// Fields from TypeA
String fromA;
// Fields from TypeB
// ...
}
public class BaseType {
String type;
String fromBase;
public BaseType(MyDTO dto) {
type = dto.type;
fromBase = dto.fromBase;
}
}
public class TypeA extends BaseType {
String fromA;
public TypeA(MyDTO dto) {
super(dto);
fromA = dto.fromA;
}
}
you can then create a TypeAdapterFactory that handles the conversion from DTO to your object --
public class BaseTypeAdapterFactory implements TypeAdapterFactory {
public <T> TypeAdapter<T> create(Gson gson, final TypeToken<T> type) {
if(BaseType.class.isAssignableFrom(type.getRawType())) {
TypeAdapter<T> delegate = gson.getDelegateAdapter(this, type);
return newItemAdapter((TypeAdapter<BaseType>) delegate,
gson.getAdapter(new TypeToken<MyDTO>(){}));
} else {
return null;
}
}
private TypeAdapter newItemAdapter(
final TypeAdapter<BaseType> delagateAdapter,
final TypeAdapter<MyDTO> dtoAdapter) {
return new TypeAdapter<BaseType>() {
#Override
public void write(JsonWriter out, BaseType value) throws IOException {
delagateAdapter.write(out, value);
}
#Override
public BaseType read(JsonReader in) throws IOException {
MyDTO dto = dtoAdapter.read(in);
if("base".equals(dto.type)) {
return new BaseType(dto);
} else if ("type_a".equals(dto.type)) {
return new TypeA(dto);
} else {
return null;
}
}
};
}
}
and use like this --
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new BaseTypeAdapterFactory())
.create();
BaseType base = gson.fromJson(baseString, BaseType.class);

Serialization Of Polymorphic Array Failing

I have enabled Polymorphic serialization support by adding annotations on the base class. I am able to seriazlize an individual object successfully and it is writing the type information as part of serialized data. However, the same is not happening if I store the objects in a list and serialize it.
It seems this issue was fixed in 1.6.3 (http://jira.codehaus.org/browse/JACKSON-362)
I am using Jackson 2.3.2 and still facing the issue.
Does somebody know how to fix this?
Code:
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY,property = "type")
#JsonSubTypes({#Type(value = Derived.class, name = "derived")})
public abstract class Base {
}
public class Derived extends Base {
private String field;
public String getField() {
return field;
}
public void setField(String field) {
this.field = field;
}
}
public class Test {
public static void main(String[] args) throws JsonProcessingException {
ObjectMapper mapper = new ObjectMapper();
Derived d = new Derived();
d.setField("Name");
Base b = d;
System.out.println(mapper.writeValueAsString(b));
List<Base> list = new ArrayList<Base>();
list.add(d);
System.out.println(mapper.writeValueAsString(list));
}
}
Output:
{"type":"derived","field":"Name"}
[{"field":"Name"}]
Thanks,
Praveen

The answer is at https://github.com/FasterXML/jackson-databind/issues/699
This is due to Java type erasure: when serializing a List, all Jackson see as a type is List (roughly equivalent to List). And since type Object does not have polymorphic type information (annotation), none will be written.
So this is not a bug in Jackson, but an unfortunate feature of Java Type Erasure.
It does not apply to arrays, since they retain element type information (arrays are not generic; arrays of different types are different classes, whereas generic typing is mostly compile-time syntactic sugar).
There are three main ways to deal with this:
pass full generic type using TypeReference (ObjectMapper has method like mapper.writerFor(new TypeReference<List<Base>>() { }).writeValue(....)
Sub-class List to something like public class BaseList extends ArrayList<Base>() { }, and pass that: this type WILL retain type information
Avoid using root-level List and Maps
I personally recommend doing (3), since this avoids all related problems with type erasure.
In my opinion JSON root value should always be a JSON Object, usually serialized to/from POJO.
Approach (2) will however work, and this is what most users do. It does require use of an additional helper class.
Approach (1) may or may not work; problem being that forcing type information does also affect actual value serialization. So while it will add type id, it may result in some properties not being serialized.

This problem can be solved by using arrays, instead of list (since list does type erasure):
for example, your above test-case could be written as:
public class Test {
public static void main(String[] args) throws JsonProcessingException {
ObjectMapper mapper = new ObjectMapper();
Derived d = new Derived();
d.setField("Name");
Base b = d;
System.out.println(mapper.writeValueAsString(b));
List<Base> list = new ArrayList<Base>();
list.add(d);
System.out.println(mapper.writeValueAsString(
list.toArray(new Base[list.size]) // <--This Part
));
}
}

I had the same issue with object array. Object[] doesn't carry type information but individual objects do. It's a shame that jackson doesn't handle that automatically.
Two possible solutions:
1. Typed array serialization works just fine:
Base[] myArray = Base[]{d};
mapper.writeValueAsString(myArray)
this will actually produce expected result as Base[] has type information.
I solved that my issue with custom serializer.
Serializer:
public class ObjectArraySerializer extends StdSerializer<Object[]> {
public ObjectArraySerializer(final Class<Object[]> vc) {
super(vc);
}
#Override
public void serialize(
final Object[] data,
final JsonGenerator gen,
final SerializerProvider provider) throws IOException {
gen.writeStartArray();
for (Object obj : data) {
gen.writeObject(obj);
}
gen.writeEndArray();
}
}
ObjectMapper configuration:
ObjectMapper objectMapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(
Object[].class,
new ObjectArraySerializer(Object[].class));
objectMapper.registerModule(module);

Gson - deserialization to specific object type based on field value

I want to deserialize json objects to specific types of objects (using Gson library) based on type field value, eg.:
[
{
"type": "type1",
"id": "131481204101",
"url": "http://something.com",
"name": "BLAH BLAH",
"icon": "SOME_STRING",
"price": "FREE",
"backgroundUrl": "SOME_STRING"
},
{
....
}
]
So type field will have different (but known) values. Based on that value I need to deserialize that json object to appropriate model object, eg.: Type1Model, Type2Model etc.
I know I can easily do that before deserialization by converting it to JSONArray, iterate through it and resolve which type it should be deserialized to. But I think it's ugly approach and I'm looking for better way. Any suggestions?

You may implement a JsonDeserializer and use it while parsing your Json value to a Java instance. I'll try to show it with a code which is going to give you the idea:
1) Define your custom JsonDeserializer class which creates different instance of classes by incoming json value's id property:
class MyTypeModelDeserializer implements JsonDeserializer<MyBaseTypeModel> {
#Override
public MyBaseTypeModel deserialize(final JsonElement json, final Type typeOfT, final JsonDeserializationContext context)
throws JsonParseException {
JsonObject jsonObject = json.getAsJsonObject();
JsonElement jsonType = jsonObject.get("type");
String type = jsonType.getAsString();
MyBaseTypeModel typeModel = null;
if("type1".equals(type)) {
typeModel = new Type1Model();
} else if("type2".equals(type)) {
typeModel = new Type2Model();
}
// TODO : set properties of type model
return typeModel;
}
}
2) Define a base class for your different instance of java objects:
class MyBaseTypeModel {
private String type;
// TODO : add other shared fields here
}
3) Define your different instance of java objects' classes which extend your base class:
class Type1Model extends MyBaseTypeModel {
// TODO: add specific fields for this class
}
class Type2Model extends MyBaseTypeModel {
// TODO: add specific fields for this class
}
4) Use these classes while parsing your json value to a bean:
GsonBuilder gsonBuilder = new GsonBuilder();
gsonBuilder.registerTypeAdapter(MyBaseTypeModel.class, new MyTypeModelDeserializer());
Gson gson = gsonBuilder.create();
MyBaseTypeModel myTypeModel = gson.fromJson(myJsonString, MyBaseTypeModel.class);
I can not test it right now but I hope you get the idea. Also this link would be very helpful.

#stephane-k 's answer works, but it is a bit confusing and could be improved upon (see comments to his answer)
Copy https://github.com/google/gson/blob/master/extras/src/main/java/com/google/gson/typeadapters/RuntimeTypeAdapterFactory.java into your project. (It's ok; these classes are designed to be copy/pasted https://github.com/google/gson/issues/845#issuecomment-217231315)
Setup model inheritance:
// abstract is optional
abstract class BaseClass {
}
class Type1Model extends BaseClass {
}
class Type2Model extends BaseClass {
}
Setup GSON or update existing GSON:
RuntimeTypeAdapterFactory<BaseClass> typeAdapterFactory = RuntimeTypeAdapterFactory
.of(BaseClass.class, "type")
.registerSubtype(Type1Model.class, "type1")
.registerSubtype(Type2Model.class, "type2");
Gson gson = new GsonBuilder().registerTypeAdapterFactory(typeAdapterFactory)
.create();
Deserialize your JSON into base class:
String jsonString = ...
BaseClass baseInstance = gson.fromJson(jsonString, BaseClass.class);
baseInstance will be instanceof either Type1Model or Type2Model.
From here you can either code to an interface or check instanceof and cast.

use https://github.com/google/gson/blob/master/extras/src/main/java/com/google/gson/typeadapters/RuntimeTypeAdapterFactory.java
then configure it with
public static final class JsonAdapterFactory extends
RuntimeTypeAdapterFactory<MediumSummaryInfo> {
public JsonAdapterFactory() {
super(MyBaseType.class, "type");
registerSubtype(MySubtype1.class, "type1");
registerSubtype(MySubtype2.class, "type2");
}
}
and add the annotation:
#JsonAdapter(MyBaseType.JsonAdapterFactory.class)
to MyBaseType
Much better.

If you have a lot of sub types and you do not want to or cannot maintain a list of them, you can also use an annotation based approach.
Here is the required code and also some usage examples:
https://gist.github.com/LostMekka/d90ade1fe051732d6b4ac60deea4f9c2
(it is Kotlin, but can easily be ported to Java)
For me, this approach is especially appealing, since I write a small library that does not know all possible sub types at compile time.

Group list of objects in a list attribute during deserialization using Jackson

I have a JSON Structure looking like this:
[
{
"id": 0,
"name": "Foo"
},
{
"id": 1,
"name": "Bar"
}
]
and a corresponding Java Object for Data binding:
public class Thing {
public int id;
public String name;
}
I know how I could deserialize the JSON list into a list of Thing.
Now here comes the tricky part: What I want to do is deserializing the JSON into a class looking like the following snippet by only doing changes to this class:
public class Things {
private List<Thing> things;
public void setThings(List<Thing> things) {
this.things = things;
}
public List<Thing> getThings() {
return this.things;
}
}
This is because the JSON deserialization is build in deep in our application by using an ObjectMapper like this:
private static <T> T parseJson(Object source, Class<T> t) {
TypeReference<T> ref = new TypeReference<T>() {
};
TypeFactory tf = TypeFactory.defaultInstance();
//[...]
obj = mapper.readValue((String) source, tf.constructType(ref));
//[...]
return obj;
}
Are there any annotations with which I can achieve what I want or do I have to make changes to the mapper-code?
Much thanks in advance, McFarlane

The whole point of TypeReference, as described in this link, is to use the generic type argument to retrieve type information.
Internally it does the following
Type superClass = getClass().getGenericSuperclass();
...
_type = ((ParameterizedType) superClass).getActualTypeArguments()[0];
where getActualTypeArguments()[0] will give you the actual type argument. In this case, that will be the type variable T, regardless of what you pass in for the Class<T> t parameter of your method.
The proper usage is
TypeReference<List<Thing>> ref = new TypeReference<List<Thing>>() {};
...
List<Thing> thingsList = ...;
Things things = new Things();
things.setThings(thingsList);
In other words, no, you'll need to change your mapper code to achieve what you want.
As far as I know, you won't be able to map a root JSON array as a property of a class. The alternatives is the TypeReference example above or some other ones found here.

How to create object that can be added to a List<T> array?

I am trying to create a JSONMessage class that can parse a json string and add the objects contained in the message to the List<T> t.
The object contained in the message implements the interface JSONSerialisation and implements the two methods toJSON() and fromJSON().
The code below does not work because I cannot instantiate the Type T and I get an error on the row t2.fromJSON... (as t2 has not been initialized).
I am not sure if my approach here is correct and if what I try to is achievable (create a generic JSONMessage) that I can use to encode/parse different type of objects. If that approach is not possible, I would appreciate hints of how I could achieve a similar result.
Regards
Interface
public interface JSONSerialisation {
public JSONObject toJSON();
public void fromJSON(JSONObject jsonObject);
}
Class
public class JSONMessage<T extends JSONSerialisation> {
private List<T> t;
public JSONMessage(String json) {
parseJSONMessage(json);
}
public void parseJSONMessage(String json) {
try {
this.t = new ArrayList<T>();
JSONObject jsonObject;
JSONArray lineItems;
jsonObject = new JSONObject(json);
this.messageHeader = new MessageHeader(jsonObject.getJSONObject("Header"));
lineItems = jsonObject.getJSONArray("Data");
int size = lineItems.length();
for (int i = 0; i < size; i++) {
T t2;
t2.fromJSON(lineItems.getJSONObject(i));
t.add(t2);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}

Looks like you are trying to replace the functionality available from most JSON deserialization libraries. You will notice that most of them require a parametized class, as a solution to the very problem you are facing. It's not the most elegant solution, but it will work. I would drop the generic parameter from the deserialization class, and the stateful variables as well:
public class JSONMessage {
public static <T extends JSONSerialisation> Collection<T> parseJSONMessage(Class<T> clazz, String json) {
try {
final JSONObject jsonObject = new JSONObject(json);
final JSONArray lineItems = jsonObject.getJSONArray("Data");
Collection<T> results = new ArrayList<T>(lineItems.size());
for (final JSONElement elem: lineItems) {
T result = clazz.newInstance();
result.fromJSON(elem);
results.add(result);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
public JSONMessage() {
super();
}
}
The code is obviously missing some error handling, and I made up a bogus JSONElement type in the enhanced for loop, but you get the general idea. Good luck.

Try this:
this.t = new ArrayList<JSONSerialisation>();
If you eliminate all references to this inside that method you can make it static. Just create a List in scope and return it.
I don't see what generics are buying you here. How likely is it that you're going to have different implementations of that JSON interface?

Have you considered including the object class in your JSON message?
What about a factory class that implements fromJSON(JSONObject) so you can create an instance of the object without first having an instance of the object? The factory could use information about the context within which the JSON was generated to determine the appropriate Java classes. For example, if you've just invoked web-service X you might know the response will either be an piece of data of type Y or an error message of type Z.
In your code above, what if T has a sub-class? How would you distinguish between them?
Ignoring JSON, I thought I'd also answer the part of your question on generics: if you know you have a List, what is T? Java generics work by type-erasure, which means during compilation a lot of the generic type information is erased. However, if you are serializing a POJO into JSON there is good news. The generic type information is retained for classes, fields and methods, so if you had a class like this:
class Example extends ArrayList<String> {
private Set<Integer> someIDs;
public Map<String,Long> getLongMap() {
...
}
}
Then it is possible to use reflection to discover that the class implements List where T is String, that the someIDs field is an instance of Set where T is Integer and the getLongMap method returns an instance of Map where K is String and V is Long. However working your way through the reflection API to get to this information can get quite involved. My code to just identify Collection, Iterator and Enumeration types runs to over 100 lines. I would not recommend trying to do it if you have alternatives.