We Keep Coding

Serialize class with generic type using gson?

I have the following class
private static class ClassWithGenericType<T> {
Set<T> values;
}
If I initialize now the class with a Set of Enum-values, serialize and deserialize the object by using gson, the Set of the deserialized object does not contain the Enum-values, but the values as String.
I think this is because the generic type is thrown away through the serialization. I saw, that I could use new TypeToken<...>(){}.getType();, but the problem is, that the class above is part of a bigger object, so I cannot call gson.fromJson(classWithGenericType, typeToken) directly.
Is there a smart way of solving this problem? I thought of a TypeAdapter, which does not serialize only the values of the Set, but also it's type.

I found now a solution and created a TypeAdapter.
public class SetTypeAdapterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(Gson gson, #NonNull TypeToken<T> type) {
if (!Set.class.isAssignableFrom(type.getRawType())) {
return null;
}
return (TypeAdapter<T>) new SetTypeAdapter(gson);
}
}
public class SetTypeAdapter extends TypeAdapter<Set<?>> {
public static final String TYPE = "#type";
public static final String DATA = "#data";
private final Gson gson;
public SetTypeAdapter(#NonNull Gson gson) {
this.gson = gson;
}
#Override
public void write(final JsonWriter out, final Set<?> set
) throws IOException {
out.beginArray();
for (Object item : set) {
out.beginObject();
out.name(TYPE).value(item.getClass().getName());
out.name(DATA).jsonValue(gson.toJson(item));
out.endObject();
}
out.endArray();
}
#Override
public Set<?> read(final JsonReader in) throws IOException {
final Set<Object> set = Sets.newHashSet();
in.beginArray();
while (in.hasNext()) {
in.beginObject();
set.add(readNextObject(in));
in.endObject();
}
in.endArray();
return set;
}
private Object readNextObject(JsonReader in) throws IOException {
try {
checkNextName(in, TYPE);
Class<?> cls = Class.forName(in.nextString());
checkNextName(in, DATA);
return gson.fromJson(in, cls);
} catch (ClassNotFoundException exception) {
throw new IOException(exception);
}
}
private void checkNextName(JsonReader in, String name) throws IOException {
if (!in.nextName().equals(name)) {
throw new IOException("Name was not: " + name);
}
}
}
We can add the factory to the GsonBuilder and afterwards we are capable of serializing a Set with generic types.
var gsonBuilder = new GsonBuilder();
gsonBuilder.registerTypeAdapterFactory(new SetTypeAdapterFactory());
var gson = gsonBuilder.create();
The serialized Set has then the following structure:
[
{
"#type":<class_name_first_element>,
"#data":<first_element_as_json>
},
...
]

Gson custom deserializer for base class

I need to consume a REST API and I'm using Gson, which would be great if some dozens of my model classes wouldn't require a custom Gson deserializer.
I think that I should use a custom TypeAdapterFactory but the documentation is poor and I'm having an hard time.
The classes I'm interested follow more or less this pattern:
public class APIResource {
#SerializedName("id")
private Integer id;
//Constructor and getter
}
public class B extends APIResource {
#SerializedName("field")
String field;
#SerializedName("resources")
List<APIResource> resourceList;
//Constructor and getter
}
public class C extends B {
#SerializedName("other_fields")
List<Object> otherFieldList;
#SerializedName("resource")
APIResource resource;
#SerializedName("b_list")
List<B> bList;
//Constructor and getter
}
Some times the id is contained in the JSON as a string named "url" that I have to parse.
The JSONs are quite complex, containing several objects and arrays and their structure is almost aleatory.
The "url" name could be anywhere in the JSON and I can't get it to work using beginObject() and beginArray()
I think my custom TypeAdapterFactory should be something like this
public class ResourceTypeAdapterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(Gson gson, #NonNull TypeToken<T> type) {
if (!APIResource.class.isAssignableFrom(type.getRawType())) {
return null;
}
TypeAdapter<T> defaultTypeAdapter = gson.getDelegateAdapter(this, type);
return new TypeAdapter<T>() {
#Override
public void write(JsonWriter out, T value) throws IOException {
defaultTypeAdapter.write(out, value);
}
#Override
public T read(JsonReader in) throws IOException {
//if the name is "url" use the urlToId method, else
return defaultTypeAdapter.read(in);
}
}.nullSafe();
}
Integer urlToId(String url) {
Matcher matcher = Pattern
.compile("/-?[0-9]+/$")
.matcher(url);
return matcher.find() ?
Integer.valueOf(matcher.group().replace("/","")):
null;
}
}

I solved it, if someone encounted the same problem this is my solution
public class ResourceTypeAdapterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(Gson gson, #NonNull TypeToken<T> type) {
if (!APIResource.class.isAssignableFrom(type.getRawType())) {
return null;
}
final TypeAdapter<T> delegateAdapter = gson.getDelegateAdapter(this, type);
final TypeAdapter<JsonElement> elementAdapter = gson.getAdapter(JsonElement.class);
return new TypeAdapter<T>() {
#Override
public void write(JsonWriter out, T value) throws IOException {
delegateAdapter.write(out, value);
}
#Override
public T read(JsonReader in) throws IOException {
JsonElement tree = elementAdapter.read(in);
afterRead(tree);
return delegateAdapter.fromJsonTree(tree);
}
protected void afterRead(#NonNull JsonElement jsonElement) {
if(jsonElement instanceof JsonObject) {
JsonObject jsonObject = ((JsonObject)jsonElement);
for(Map.Entry<String,JsonElement> entry : jsonObject.entrySet()){
if(entry.getValue() instanceof JsonPrimitive) {
if(entry.getKey().equalsIgnoreCase("url")) {
String val = jsonObject.get(entry.getKey()).toString();
jsonObject.addProperty("id", urlToId(val));
}
} else {
afterRead(entry.getValue());
}
}
}
}
}.nullSafe();
}
Integer urlToId(#NonNull String url) {
Matcher matcher = Pattern
.compile("/-?[0-9]+/$")
.matcher(url.replace("\"", ""));
return matcher.find() ?
Integer.valueOf(matcher.group().replace("/","")):
null;
}
}

Gson treat failed field parsing as null

Is there a way to configure Gson so that it treats any failed field parse as null instead of throwing a parse exception? Ideally we could catch and log the exception -- but we want the option to keep going with the program even if some fields (or subfields) do not parse as expected.
Example:
Malformed JSON:
{
"dog": []
}
With classes:
class Farm {
public Dog dog;
}
class Dog {
public String name;
}
Gson gson = new Gson();
Farm oldMcdonald = gson.fromJson(json, Farm.class); // should not throw exception
assertNull(oldMcdonald.dog); // should pass

In Gson, it can be implemented pretty easy.
Despite the following solution, I guess, seems not to work in any case (for example, primitives), it can be enhanced if necessary.
final class JsonFailSafeTypeAdapterFactory
implements TypeAdapterFactory {
private static final TypeAdapterFactory instance = new JsonFailSafeTypeAdapterFactory();
private JsonFailSafeTypeAdapterFactory() {
}
static TypeAdapterFactory get() {
return instance;
}
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// We can support non-primitive types only
if ( typeToken.getRawType().isPrimitive() ) {
return null;
}
final TypeAdapter<T> delegateTypeAdapter = gson.getAdapter(typeToken);
return new JsonFailSafeTypeAdapter<>(delegateTypeAdapter);
}
private static final class JsonFailSafeTypeAdapter<T>
extends TypeAdapter<T> {
private final TypeAdapter<T> delegateTypeAdapter;
private JsonFailSafeTypeAdapter(final TypeAdapter<T> delegateTypeAdapter) {
this.delegateTypeAdapter = delegateTypeAdapter;
}
#Override
public void write(final JsonWriter out, final T value)
throws IOException {
delegateTypeAdapter.write(out, value);
}
#Override
public T read(final JsonReader in)
throws IOException {
try {
return delegateTypeAdapter.read(in);
} catch ( final MalformedJsonException | RuntimeException ignored ) {
// Once we get into unexpected JSON token, let's *always* consider a fallback to the default value
// Well, the default is always `null` anyway, but we'll do more work
return fallback(in);
}
}
private static <T> T fallback(final JsonReader in)
throws IOException {
final JsonToken jsonToken = in.peek();
switch ( jsonToken ) {
case BEGIN_ARRAY:
case BEGIN_OBJECT:
case NAME:
case STRING:
case NUMBER:
case BOOLEAN:
case NULL:
// Assume we're at the beginning of a complex JSON value or a JSON primitive
in.skipValue();
break;
case END_ARRAY:
// Not sure if we skipValue() can fast-forward this one
in.endArray();
break;
case END_OBJECT:
// The same
in.endObject();
break;
case END_DOCUMENT:
// do nothing
break;
default:
throw new AssertionError(jsonToken);
}
// Just return null (at least at the moment)
return null;
}
}
}
Now just register the above type factory to handle all types (except java.lang.Object if I'm not mistaken).
private static final Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(JsonFailSafeTypeAdapterFactory.get())
.create();
public static void main(final String... args)
throws IOException {
try ( final JsonReader jsonReader = Resources.getPackageResourceJsonReader(Q50002961.class, "farm.json") ) {
final Farm oldMcdonald = gson.fromJson(jsonReader, Farm.class);
if ( oldMcdonald.dog != null ) {
throw new AssertionError();
}
System.out.println(oldMcdonald);
}
}
Example output:
q50002961.Farm#626b2d4a
Another option is also specifying target fields if there is no need to register the factory globally. For instance:
final class Farm {
#JsonAdapter(JsonFailSafeTypeAdapterFactory.class)
final Dog dog = null;
}

I will post a solution for your problem but it would still require you to change the code on your side. For example if you have configured a property as an object and you receive an array - there is no way to map that properly. So I would suggest to change everything in your code to List and write a custom mapper that creates a list with one element when an object is received. This way you will be flexible to what you receive but you will also need to add some logic to handle problems when you have more than one objects to the array. For your example what would you do if you get 2 dogs? What is the correct behavior?
So I would do it like that:
public class MainClass {
public static <T> void main(String[] args) throws IOException {
Gson gson = new GsonBuilder().registerTypeAdapterFactory(new ArrayAdapterFactory()).create();
// Here I do the opposite - add one dog but expect a collection
String json = "{ \"dog\": {name=\"Snoopy\"} }";
Farm oldMcdonald = gson.fromJson(json, Farm.class); // should not throw exception
System.out.println("Dog:"+oldMcdonald.dog.get(0).name); //Works properly
}
}
class Farm {
#Expose
public List<Dog> dog; //All such properties become a list. You handle the situation when there are more than one values
}
class Dog {
#Expose
public String name;
}
class ArrayAdapter<T> extends TypeAdapter<List<T>> {
private Class<T> adapterclass;
public ArrayAdapter(Class<T> adapterclass) {
this.adapterclass = adapterclass;
}
public List<T> read(JsonReader reader) throws IOException {
List<T> list = new ArrayList<T>();
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new ArrayAdapterFactory())
.create();
if (reader.peek() == JsonToken.BEGIN_OBJECT) {
T inning = gson.fromJson(reader, adapterclass);
list.add(inning);
// return null; here if you want to return null instead of list with one element
} else if (reader.peek() == JsonToken.BEGIN_ARRAY) {
reader.beginArray();
while (reader.hasNext()) {
T inning = gson.fromJson(reader, adapterclass);
list.add(inning);
}
reader.endArray();
}
return list;
}
public void write(JsonWriter writer, List<T> value) throws IOException {
}
}
class ArrayAdapterFactory implements TypeAdapterFactory {
#SuppressWarnings({ "unchecked", "rawtypes" })
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> type) {
TypeAdapter<T> typeAdapter = null;
try {
if (type.getRawType() == List.class)
typeAdapter = new ArrayAdapter(
(Class) ((ParameterizedType) type.getType())
.getActualTypeArguments()[0]);
} catch (Exception e) {
e.printStackTrace();
}
return typeAdapter;
}
}
Thanks to http://sachinpatil.com/blog/2012/07/03/gson/ for the idea

Gson: parsing a non-standard JSON format

Does Gson have a way to read in non-standard JSON files?
Instead of a typical file like:
[{obj1},{objN}]
I have a file like this:
{obj1}
{objN}
Where there are no square brackets or commas and each object is separated by a newline character.

Yes, it has. Gson supports lenient reading. For example, the following JSON document (non-standard.json):
{
"foo": 1
}
{
"bar": 1
}
you can use the following reading way:
private static final Gson gson = new Gson();
private static final TypeAdapter<JsonElement> jsonElementTypeAdapter = gson.getAdapter(JsonElement.class);
public static void main(final String... args)
throws IOException {
try ( final Reader reader = getPackageResourceReader(Q43528208.class, "non-standard.json") ) {
final JsonReader jsonReader = new JsonReader(reader);
jsonReader.setLenient(true); // this makes it work
while ( jsonReader.peek() != END_DOCUMENT ) {
final JsonElement jsonElement = jsonElementTypeAdapter.read(jsonReader);
System.out.println(jsonElement);
}
}
}
Output:
{"foo":1}
{"bar":1}
I'm not sure if you can write a robust deserializer this way though.
Update
In order to simplify the Gson support, we can implement a few convenient reading methods:
// A shortcut method for the below implementation: aggregates the whole result into a single list
private static <T> List<T> parseToListLenient(final JsonReader jsonReader, final IMapper<? super JsonReader, ? extends T> mapper)
throws IOException {
final List<T> list = new ArrayList<>();
parseLenient(jsonReader, in -> list.add(mapper.map(in)));
return list;
}
// A convenient strategy-accepting method to configure a JsonReader instance to make it lenient and do read
// The consumer defines the strategy what to do with the current JsonReader token
private static void parseLenient(final JsonReader jsonReader, final IConsumer<? super JsonReader> consumer)
throws IOException {
final boolean isLenient = jsonReader.isLenient();
try {
jsonReader.setLenient(true);
while ( jsonReader.peek() != END_DOCUMENT ) {
consumer.accept(jsonReader);
}
} finally {
jsonReader.setLenient(isLenient);
}
}
// Since Java 8 Consumer inteface does not allow checked exceptions to be rethrown
private interface IConsumer<T> {
void accept(T value)
throws IOException;
}
private interface IMapper<T, R> {
R map(T value)
throws IOException;
}
Then simple reading is really simple, and we can just use the methods above:
final Gson gson = new Gson();
final TypeToken<Map<String, Integer>> typeToken = new TypeToken<Map<String, Integer>>() {
};
final TypeAdapter<Map<String, Integer>> typeAdapter = gson.getAdapter(typeToken);
try ( final JsonReader jsonReader = getPackageResourceJsonReader(Q43528208.class, "non-standard.json") ) {
final List<Map<String, Integer>> maps = parseToListLenient(jsonReader, typeAdapter::read);
System.out.println(maps);
}
Deserialization via Gson directly would require more complicated implementation:
// This is just a marker not meant to be instantiated but to create a sort of "gateway" to dispatch types in Gson
#SuppressWarnings("unused")
private static final class LenientListMarker<T> {
private LenientListMarker() {
throw new AssertionError("must not be instantiated");
}
}
private static void doDeserialize()
throws IOException {
final Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new TypeAdapterFactory() {
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// Check if the given type is the lenient list marker class
if ( !LenientListMarker.class.isAssignableFrom(typeToken.getRawType()) ) {
// Not the case? Just delegate the job to Gson
return null;
}
final Type listElementType = getTypeParameter0(typeToken.getType());
final TypeAdapter<?> listElementAdapter = gson.getAdapter(TypeToken.get(listElementType));
#SuppressWarnings("unchecked")
final TypeToken<List<?>> listTypeToken = (TypeToken<List<?>>) TypeToken.getParameterized(List.class, listElementType);
final TypeAdapter<List<?>> listAdapter = gson.getAdapter(listTypeToken);
final TypeAdapter<List<?>> typeAdapter = new TypeAdapter<List<?>>() {
#Override
public void write(final JsonWriter out, final List<?> value)
throws IOException {
// Always write a well-formed list
listAdapter.write(out, value);
}
#Override
public List<?> read(final JsonReader in)
throws IOException {
// Delegate the job to the reading method - we only have to tell how to obtain the list values
return parseToListLenient(in, listElementAdapter::read);
}
};
#SuppressWarnings("unchecked")
final TypeAdapter<T> castTypeAdapter = (TypeAdapter<T>) typeAdapter;
return castTypeAdapter;
}
// A simple method to resolve actual type parameter
private Type getTypeParameter0(final Type type) {
if ( !(type instanceof ParameterizedType) ) {
// List or List<?>
return Object.class;
}
return ((ParameterizedType) type).getActualTypeArguments()[0];
}
})
.create();
// This type declares a marker specialization to be used during deserialization
final Type type = new TypeToken<LenientListMarker<Map<String, Integer>>>() {
}.getType();
try ( final JsonReader jsonReader = getPackageResourceJsonReader(Q43528208.class, "non-standard.json") ) {
// This is where we're a sort of cheating:
// We tell Gson to deserialize LenientListMarker<Map<String, Integer>> but the type adapter above will return a list
final List<Map<String, Integer>> maps = gson.fromJson(jsonReader, type);
System.out.println(maps);
}
}
The output is now for Map<String, Integer>s, not JsonElements:
[{foo=1}, {bar=1}]
Update 2
TypeToken.getParameterized workaround:
#SuppressWarnings("unchecked")
final TypeToken<List<?>> listTypeToken = (TypeToken<List<?>>) TypeToken.get(new ParameterizedType() {
#Override
public Type getRawType() {
return List.class;
}
#Override
public Type[] getActualTypeArguments() {
return new Type[]{ listElementType };
}
#Override
public Type getOwnerType() {
return null;
}
});

We can have one more program to introduce comma(,) and construct a well formed JSON

With spark 2, we can add multiline as read option.
spark.df.option("multiline","true").json("data.json")

Gson custom serialization

I wish to have a custom GSON deserializer such that whenever it is deserializing a JSON object (i.e. anything within curly brackets { ... }), it will look for a $type node and deserialize using its inbuilt deserializing capability to that type. If no $type object is found, it just does what it normal does.
So for example, I would want this to work:
{
"$type": "my.package.CustomMessage"
"payload" : {
"$type": "my.package.PayloadMessage",
"key": "value"
}
}
public class CustomMessage {
public Object payload;
}
public class PayloadMessage implements Payload {
public String key;
}
Calling: Object customMessage = gson.fromJson(jsonString, Object.class).
So currently if I change the payload type to the Payload interface:
public class CustomMessage {
public Payload payload;
}
Then the following TypeAdapaterFactory will do what I want:
final TypeAdapter<T> delegate = gson.getDelegateAdapter(this, type);
final TypeAdapter<JsonElement> elementAdapter = gson.getAdapter(JsonElement.class);
final PojoTypeAdapter thisAdapter = this;
public T read(JsonReader reader) throws IOException {
JsonElement jsonElement = (JsonElement)elementAdapter.read(reader);
if (!jsonElement.isJsonObject()) {
return delegate.fromJsonTree(jsonElement);
}
JsonObject jsonObject = jsonElement.getAsJsonObject();
JsonElement typeElement = jsonObject.get("$type");
if (typeElement == null) {
return delegate.fromJsonTree(jsonElement);
}
try {
return (T) gson.getDelegateAdapter(
thisAdapter,
TypeToken.get(Class.forName(typeElement.getAsString()))).fromJsonTree(jsonElement);
} catch (ClassNotFoundException ex) {
throw new IOException(ex.getMessage());
}
}
However, I would like it to work when payload is of type Object or any type for that matter, and throw some sort of type match exception if it can't assign the variable.

Looking at the source for Gson, I have found what I think is the issue:
// built-in type adapters that cannot be overridden
factories.add(TypeAdapters.JSON_ELEMENT_FACTORY);
factories.add(ObjectTypeAdapter.FACTORY);
// user's type adapters
factories.addAll(typeAdapterFactories);
As you can see the ObjectTypeAdapter will take precedence over my factory.
The only solution as far as I can see is to use reflection to remove the ObjectTypeAdapter from the list or insert my factory before it. I have done this and it works.

I don't know how you can achieve it with Gson but you have such a feature in Genson by default.
To enable it just do:
Genson genson = new Genson.Builder().setWithClassMetadata(true).create();
You can also register aliases for your class names:
Genson genson = new Genson.Builder().addAlias("myClass", my.package.SomeClass.class).create();
This has however some limitations:
at the moment you can't change the key used to identify the type, it is #class
it must be present in your json before the other properties - but looks fine as it is the case in your examples
Works only with json objects and not arrays or litterals

This code skeleton works on your example but should be improved and tested with different scenarios.
public class PojoTypeAdapaterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> type) {
// check types we support
if (type.getRawType().isAssignableFrom(CustomMessage.class) || type.getRawType().isAssignableFrom(PayloadMessage.class)) {
return new PojoTypeAdapter<T>(gson, type);
}
else return null;
}
private class PojoTypeAdapter<T> extends TypeAdapter<T> {
private Gson gson;
private TypeToken<T> type;
private PojoTypeAdapter(final Gson gson, final TypeToken<T> type) {
this.gson = gson;
this.type = type;
}
public T read(JsonReader reader) throws IOException {
final TypeAdapter<T> delegate = gson.getDelegateAdapter(PojoTypeAdapaterFactory.this, this.type);
final TypeAdapter<JsonElement> elementAdapter = this.gson.getAdapter(JsonElement.class);
JsonElement jsonElement = elementAdapter.read(reader);
if (!jsonElement.isJsonObject()) {
return (T) this.gson.getAdapter(JsonElement.class).fromJsonTree(jsonElement);
}
JsonObject jsonObject = jsonElement.getAsJsonObject();
JsonElement typeElement = jsonObject.get("$type");
if (typeElement == null) {
return delegate.fromJsonTree(jsonElement);
}
try {
final Class myClass = Class.forName(typeElement.getAsString());
final Object myInstance = myClass.newInstance();
final JsonObject jsonValue = jsonElement.getAsJsonObject().get("value").getAsJsonObject();
for (Map.Entry<String, JsonElement> jsonEntry : jsonValue.entrySet()) {
final Field myField = myClass.getDeclaredField(jsonEntry.getKey());
myField.setAccessible(true);
Object value = null;
if (jsonEntry.getValue().isJsonArray()) {
//value = ...;
}
else if (jsonEntry.getValue().isJsonPrimitive()) {
final TypeAdapter fieldAdapter = this.gson.getAdapter(myField.getType());
value = fieldAdapter.fromJsonTree(jsonEntry.getValue());
}
else if (jsonEntry.getValue().isJsonObject()) {
value = this.fromJsonTree(jsonEntry.getValue());
}
myField.set(myInstance, value);
}
return (T) myInstance;
}
catch (ClassNotFoundException | IllegalAccessException | IllegalArgumentException | InstantiationException | NoSuchFieldException | SecurityException e) {
throw new IOException(e);
}
}
#Override
public void write(final JsonWriter out, final T value) throws IOException {
out.beginObject();
out.name("$type");
out.value(value.getClass().getName());
out.name("value");
final TypeAdapter<T> delegateAdapter = (TypeAdapter<T>) this.gson.getDelegateAdapter(PojoTypeAdapaterFactory.this, TypeToken.<T>get(value.getClass()));
delegateAdapter.write(out, value);
out.endObject();
}
}
}
The generated JSON is not exactly the same though, as it contains an additional value entry:
{
"$type": "my.package.CustomMessage",
"value": {
"payload": {
"$type": "my.package.PayloadMessage",
"value": {
"key": "hello"
}
}
}
}