We Keep Coding

Fill empty values based on parent values of nested JSON Object with Gson

I'm deserializing a JSON data structure like the following example with GSON:
{
"id": 1,
"data": [
{
"value": 2
},
{
"value": 2
}
]
}
This JSON is mapped to the following classes:
class Element {
Integer id;
List<SubElement> data;
...
}
class SubElement {
Integer id;
Integer value;
...
}
I would like to be able to fill the field id of the SubElement object based on parent id field using GSON at the time of deserialization.
The point of doing such a thing is that I receive this kind of JSON in several parts of my application and for instance I would like to ommit this kind of code:
child.setId(parent.getId());
I tried to implement a custom deserializer JsonDeserializer<SubElement> but I don't have access to the root element, so no access to id.
Is there any way to do this type of custom deserialization? Or should I go ahead implementing some sort of solution using Reflection like traversing the whole object looking for the field of interest and update it accordingly?
I'm looking for a general solution that could be valid for other nested objects lacking id.

Is there any way to do this type of custom deserialization?
Yes, but it does not necessarily needs to be a part of the Gson deserialization unless you want that thing to be centralized. You can always adjust the object after invoking its fromJson method (hence not that reliable). So if you think that Gson should be responsible for that, you can implement it as a post-processor:
public interface IPostProcessorResolver<T> {
#Nullable
Consumer<T> resolvePostProcessor(#Nonnull TypeToken<?> typeToken);
}
public final class PostProcessingTypeAdapterFactory
implements TypeAdapterFactory {
private final IPostProcessorResolver<?> resolvePostProcessor;
private PostProcessingTypeAdapterFactory(final IPostProcessorResolver<?> resolvePostProcessor) {
this.resolvePostProcessor = resolvePostProcessor;
}
public static TypeAdapterFactory create(final IPostProcessorResolver<?> resolvePostProcessor) {
return new PostProcessingTypeAdapterFactory(resolvePostProcessor);
}
#Override
#Nullable
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
#Nullable
#SuppressWarnings("unchecked")
final Consumer<T> postProcessor = (Consumer<T>) resolvePostProcessor.resolvePostProcessor(typeToken);
if ( postProcessor == null ) {
return null;
}
final TypeAdapter<T> delegateTypeAdapter = gson.getDelegateAdapter(this, typeToken);
return new TypeAdapter<T>() {
#Override
public void write(final JsonWriter out, final T value)
throws IOException {
delegateTypeAdapter.write(out, value);
}
#Override
public T read(final JsonReader in)
throws IOException {
// First, read the object
final T object = delegateTypeAdapter.read(in);
// Second, tell it to adjust itself
postProcessor.accept(object);
return object;
}
};
}
}
Then, a test:
final Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(PostProcessingTypeAdapterFactory.create((IPostProcessorResolver<Element>) typeToken -> {
if ( typeToken.getRawType() != Element.class ) {
return null;
}
return element -> {
if ( element.data == null ) {
return;
}
for ( #Nullable final SubElement datum : element.data ) {
if ( datum != null ) {
datum.id = element.id;
}
}
};
}))
.create();
...
final Element element = gson.fromJson(jsonReader, Element.class);
Assertions.assertNotNull(element.data);
Assertions.assertEquals(1, (int) element.id);
Assertions.assertEquals(1, (int) element.data.get(0).id);
Assertions.assertEquals(1, (int) element.data.get(1).id);
Note that you can implement more complex strategies by merely implementing the IPostProcessorResolver: multi types support with type-resolving; reflection use; etc.

Gson can't deserialize inherited class?

I have a simple Json structure like:
{"MessageType":"TimeData","TimeData":{"hh":12,"mm":13,"ms":15,"ss":14}}
and I devised the following classes to deserialize it:
public class JsonMessage
{
public enum MessageTypes{
WhoAreYou,
TimeData
}
JsonMessage(){
}
public MessageTypes MessageType;
}
class TimeData extends JsonMessage{
int hh;
int mm;
int ss;
int ms;
TimeData() {
}
}
I need to split deserialization into tow phases:
1- deserialize to read the MessageType.
2- proceed with the rest of deserialization based on the MessageType
The code is straightforward:
public void dispatch(Object message, IoSession session)
{
Gson gson = new Gson();
JsonMessage result = gson.fromJson(message.toString(), JsonMessage.class);
System.out.println(result.MessageType.toString());
switch (result.MessageType)
{
case WhoAreYou:{
//.....
break;
}
case TimeUpdate:
TimeData res = new Gson().fromJson(message.toString(), TimeData.class);
System.out.println(res.hh);
break;
default:break;
}
}
My Program can enter the correct switch-case(which is TimeUpdate) but it doesn't parse it correctly (The println prints 0 instead of 12)
where do you think I have done something wrong?
thank you

The issue is that your JSON represents an Object that contains another object you're interested in while your Java is just a single object.
You can actually just write deserializers for each type and use them once you determine the MessageType:
public static void main(String[] args)
{
Gson gson = new GsonBuilder().registerTypeAdapter(TimeData.class, new TimeDataDeserializer()).create();
String json = "{\"MessageType\":\"TimeData\",\"TimeData\":{\"hh\":12,\"mm\":13,\"ms\":15,\"ss\":14}}";
JsonMessage message = gson.fromJson(json, JsonMessage.class);
switch(message.MessageType)
{
case TimeData:
TimeData td = new GsonBuilder()
.registerTypeAdapter(TimeData.class, new TimeDataDeserializer())
.create()
.fromJson(json, TimeData.class);
td.MessageType = message.MessageType
System.out.println(td.hh);
break;
default:
break;
}
}
class TimeDataDeserializer implements JsonDeserializer<TimeData>
{
#Override
public TimeData deserialize(JsonElement je, Type type, JsonDeserializationContext jdc)
throws JsonParseException
{
JsonObject jo = je.getAsJsonObject().getAsJsonObject("TimeData");
Gson g = new Gson();
return g.fromJson(jo, TimeData.class);
}
}

I managed to solve this similar problem by implementing a custom JsonDeserializer in the following way.
First you attach to your enum the subclasses based on the type and a method to retrieve the correct Class<?> according to the enum name:
enum MessageType {
WHO_ARE_YOU(WhoAreYou.class),
TIME_UPDATE(TimeUpdate.class);
public final Class<?> clazz;
MessageType(Class<?> clazz) { this.clazz = clazz; }
public static MessageType forName(String name) {
for (MessageType t : values())
if (name.equals(t.name()))
return t;
return NULL;
}
}
Then in the deserialize method I did the following:
public JsonMessage deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException
{
JsonObject object = json.getAsJsonObject();
String kind = object.get("messageType").getAsString();
Class<?> clazz = MessageType.forName(kind).clazz;
JsonMessage result = null;
try {
result = (JsonMessage)clazz.newInstance();
Field[] fs = clazz.getFields();
for (Field f : fs) {
Object value = context.deserialize(object.get(f.getName()), f.getType());
if (value != null)
f.set(result, value);
}
}
catch (Exception e) {
e.printStackTrace();
}
}
Everything is managed by reflection so that a correct object is created and then all fields are deserialized accordingly.
I had a complex hierarchy of objects so I preferred to go this way to let the gson deserializer manage everything. Of course you will need to register the serializer with the gson parser instance.
A NOTE: Your naming of things is quite incorrect according to Java standards. enum constants should be ALL_CAPITALIZED, enum class names should be singular (eg. MessageType). Instance variables should be camelcased (eg. messageType not MessageType)

Trim all strings elements in a complex object

Can I write a generic method to trim all strings within an complex object (object containing other objects)? Should java reflection api be used to achieve this?Thanks.
I have provided a sample below. However in reality there could be multiple objects within objects. Each object might contain a collection of String or collection of other objects which may contain String. Is there a way to trim the Strings - ones directly with the objects and ones within collection.
public class School{
private List<Course> courses;
private List<Student> students;
// Getters and Setters
}
public class Course{
private String name;
private String xxx;
private String yyy;
private List<String> zzzList;
}
public class Student{
private Map<String,String> xxx;
private List<Course> courseList;
}

Yes, reflection is the way. Basically, you need to:
get the class of the top level object (with [object].getClass())
get all the fields of the object (with clazz.getFields() - beware, it works only with public fields)
check if the field is String (either get field.getType() and check it's a string, or do a field.get(the object) and a instanceof String)
if it's the case, replace the string in the object with the trimmed one, using field.set([your object],[trimmed string])
if the field is an object but not a string, call your method recursively
That will do the trick.
---- just seen your update
Trimming strings in collection will be more tricky, since the strings are not exposed as public fields of the collection (List for example).
You will need something more clever, that will check if an object is an instance of List, or Map, or etc... (or a derived class!).
Main problem is also that java generics are done with erasing type at compile type. So you cannot know that your field is List[String] or List[Integer] or whatever. Every List[?] becomes List.
Still you can try to do it like that:
if field type is List
iterate through the list values
if a value is instanceof String, you have to remove it from the list and insert in place the trimmed version
if a value is an object, there you go again recursively with your method.
Not very interesting in real life samples, but more on a library side maybe.
Long way to go though!

Yes, you can do that with reflection, quite easily. Just check if the field is instanceof String.
The exact way to do it depends on your object structure.

/*********************************************************************************************
* Trim first level children of string type in this object
* #param obj which all string properties to be trimmed
*********************************************************************************************/
public static void trimAll(final Object obj)
throws LocalException
{
if (obj==null) return;
final Class c = obj.getClass();
final Method[] methods = c.getMethods();
final Class[] SETTER_ARGS = new Class[]{String.class};
final Object[] SETTER_VAL = new Object[1];
final String SET = "set";
final String GET = "get";
final String SPACE = "\u0020";
final String TAB = "\t";
for (final Method m:methods)
{
try
{
final String name=m.getName();
if (
name.length()>GET.length()
&& name.indexOf(GET)==0
&& m.getReturnType().equals(String.class)
&& m.getParameterTypes().length==0)
{
final String v = (String)m.invoke(obj);
if (v!=null && (v.contains(SPACE) || v.contains(TAB)) )
{
final Method setter=c.getMethod(SET+name.substring(3),SETTER_ARGS);
if (setter!=null)
{
SETTER_VAL[0]=v.trim();
setter.invoke(obj,SETTER_VAL);
}
}
}
}
catch (final Throwable e)
{
throw new LocalException(LocalException.EC_GENERAL_EXCEPTION,e);
}
}
}

We can also use Jackson to serialize and then deserialize the object. While deserializing we can use custom deserializer to trim all the String values.
Create a deserializer like this:
public class TrimStringToNullDeserializer extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser jsonParser, DeserializationContext deserializationContext)
throws IOException {
String value = jsonParser.getValueAsString();
if (isNull(value)) {
return null;
}
value = value.trim();
if (value.length() == 0) {
value = null;
}
return value;
}
And then we can use Jackson to trim all values:
public class TrimStringToNullConfiguration {
private ObjectMapper objectMapper;
public Client trimToNull(Client inputClient) throws JsonProcessingException {
return getObjectMapper().readValue(getObjectMapper().writeValueAsString(inputClient), Client.class);
}
private ObjectMapper getObjectMapper() {
if (isNull(objectMapper)) {
objectMapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addDeserializer(String.class, new TrimStringToNullDeserializer());
objectMapper.registerModule(module);
}
return objectMapper;
}
I have placed a working example over here.

private <T> T toTrim(T t) {
Field[] fields = t.getClass().getFields();
for (Field field : fields) {
try {
if (field.get(t) instanceof String) {
Object o = field.get(t);
String s = (String) o;
field.set(t, s.trim().toUpperCase());
}
} catch (IllegalAccessException e) {
log.info("Error converting field "+ field.getName() );
}
}
return t;
}

if (yourObject instanceof String){
yourObject = yourObject.trim();
}
Hope it helps :)

java/scala: faster type-aware serialization of only basic types?

in scala, i have a need to serialize objects that are limited to a small set of basic types: array, list, map, set, int, boolean, etc. i want to be able to serialize and deserialize those in a way that preserves the type information in the serialized format. specifically, if i have serialized an Array[Any], i want to be able to deserialize it and only specify that the resulting object is Array[Any]. that is, i don't want to specify a structure definition for every single thing i'm going to serialize. at the same time it needs to be able to distinguish between int and long, tuple and array, etc.
for example:
val obj = Array[Any](...) // can have any basic types in here
val ser = serialize(obj)
val newObj = deserialize[Array[Any]](ser) // recovers the exact types from the original obj
json is not appropriate for this case because it has a many-to-one mapping of scala types to json types. i'm currently using java serialization but it's extremely slow. since i don't need to serialize any arbitrary object type, is there a faster alternative for my narrower use case?

I don't about speed or indeed availability of library support, but have you looked at ASN.1?

I'd use a simple interface like this:
public interface Serializer{
public <T> T deserialize(String serializedData);
public String serialize(Object data);
}
And an enum to implement it:
public enum StandardSerializer implements Serializer{
INTEGER("I", Integer.class, int.class){
#Override
protected Integer doDeserialize(final String stripped){
return Integer.valueOf(stripped);
}
},
STRING("I", String.class){
#Override
protected Object doDeserialize(final String stripped){
return stripped;
}
},
LIST("L", List.class){
#Override
protected String doSerialize(final Object data){
final Iterator<?> it = ((List<?>) ((List<?>) data)).iterator();
final StringBuilder sb = new StringBuilder();
if(it.hasNext()){
Object next = it.next();
sb.append(StandardSerializer
.forType(next.getClass())
.serialize(next));
while(it.hasNext()){
sb.append(',');
next = it.next();
sb.append(StandardSerializer
.forType(next.getClass())
.serialize(next));
}
}
return sb.toString();
}
#Override
protected Object doDeserialize(final String stripped){
final List<Object> list = new ArrayList<Object>();
for(final String item : stripped.split(",")){
list.add(StandardSerializer.forData(item).deserialize(item));
}
return list;
}
}
/* feel free to implement more enum entries */
;
private static final String DELIMITER = ":";
public static StandardSerializer forType(final Class<?> type){
for(final StandardSerializer candidate : values()){
for(final Class<?> supportedType : candidate.supportedClasses){
if(supportedType.isAssignableFrom(type)) return candidate;
}
}
throw new IllegalArgumentException("Unmapped type: " + type);
}
private final String prefix;
private final Class<?>[] supportedClasses;
private StandardSerializer(final String prefix,
final Class<?>... supportedClasses){
this.prefix = prefix;
this.supportedClasses = supportedClasses;
}
private String base64decode(final String removePrefix){
// TODO call one of the many base64 libraries here
return null;
}
private String base64encode(final String data){
// TODO call one of the many base64 libraries here
return null;
}
#SuppressWarnings("unchecked")
#Override
public final <T> T deserialize(final String serializedData){
return (T) doDeserialize(base64decode(removePrefix(serializedData)));
}
public static StandardSerializer forData(final String serializedData){
final String prefix =
serializedData.substring(0, serializedData.indexOf(DELIMITER));
for(final StandardSerializer candidate : values()){
if(candidate.prefix.equals(prefix)) return candidate;
}
throw new IllegalArgumentException("Unknown prefix: " + prefix);
}
protected abstract Object doDeserialize(String strippedData);
private String removePrefix(final String serializedData){
return serializedData.substring(prefix.length() + DELIMITER.length());
}
// default implementation calles toString()
protected String doSerialize(final Object data){
return data.toString();
}
#Override
public String serialize(final Object data){
return new StringBuilder()
.append(prefix)
.append(DELIMITER)
.append(base64encode(doSerialize(data)))
.toString();
}
}
Now here's how you can code against that:
List<?> list = Arrays.asList("abc",123);
String serialized = StandardSerializer.forType(list.getClass()).serialize(list);
List<?> unserialized = StandardSerializer.forData(serialized)
.deserialize(serialized);
(While you might choose a different format for serialization, using an enum strategy pattern is probably still a good idea)

How to keep fields sequence in Gson serialization

Seems like Gson.toJson(Object object) generates JSON code with randomly spread fields of the object. Is there way to fix fields order somehow?
public class Foo {
public String bar;
public String baz;
public Foo( String bar, String baz ) {
this.bar = bar;
this.baz = baz;
}
}
Gson gson = new Gson();
String jsonRequest = gson.toJson(new Foo("bar","baz"));
The string jsonRequest can be:
{ "bar":"bar", "baz":"baz" } (correct)
{ "baz":"baz", "bar":"bar" } (wrong sequence)

You'd need to create a custom JSON serializer.
E.g.
public class FooJsonSerializer implements JsonSerializer<Foo> {
#Override
public JsonElement serialize(Foo foo, Type type, JsonSerializationContext context) {
JsonObject object = new JsonObject();
object.add("bar", context.serialize(foo.getBar());
object.add("baz", context.serialize(foo.getBaz());
// ...
return object;
}
}
and use it as follows:
Gson gson = new GsonBuilder().registerTypeAdapter(Foo.class, new FooJsonSerializer()).create();
String json = gson.toJson(foo);
// ...
This maintains the order as you've specified in the serializer.
See also:
Gson User Guide - Custom serializers and deserializers

If GSON doesn't support definition of field order, there are other libraries that do. Jackson allows definining this with #JsonPropertyOrder, for example. Having to specify one's own custom serializer seems like awful lot of work to me.
And yes, I agree in that as per JSON specification, application should not expect specific ordering of fields.

Actually Gson.toJson(Object object) doesn't generate fields in random order. The order of resulted json depends on literal sequence of the fields' names.
I had the same problem and it was solved by literal order of properties' names in the class.
The example in the question will always return the following jsonRequest:
{ "bar":"bar", "baz":"baz" }
In order to have a specific order you should modify fields' names, ex: if you want baz to be first in order then comes bar:
public class Foo {
public String f1_baz;
public String f2_bar;
public Foo ( String f1_baz, String f2_bar ) {
this.f1_baz = f1_baz;
this.f2_bar = f2_bar;
}
}
jsonRequest will be { "f1_baz ":"baz", "f2_bar":"bar" }

Here's my solution for looping over json text files in a given directory and writing over the top of them with sorted versions:
private void standardizeFormat(File dir) throws IOException {
File[] directoryListing = dir.listFiles();
if (directoryListing != null) {
for (File child : directoryListing) {
String path = child.getPath();
JsonReader jsonReader = new JsonReader(new FileReader(path));
Gson gson = new GsonBuilder().setPrettyPrinting().registerTypeAdapter(LinkedTreeMap.class, new SortedJsonSerializer()).create();
Object data = gson.fromJson(jsonReader, Object.class);
JsonWriter jsonWriter = new JsonWriter(new FileWriter(path));
jsonWriter.setIndent(" ");
gson.toJson(data, Object.class, jsonWriter);
jsonWriter.close();
}
}
}
private class SortedJsonSerializer implements JsonSerializer<LinkedTreeMap> {
#Override
public JsonElement serialize(LinkedTreeMap foo, Type type, JsonSerializationContext context) {
JsonObject object = new JsonObject();
TreeSet sorted = Sets.newTreeSet(foo.keySet());
for (Object key : sorted) {
object.add((String) key, context.serialize(foo.get(key)));
}
return object;
}
}
It's pretty hacky because it depends on the fact that Gson uses LinkedTreeMap when the Type is simply Object. This is an implementation details that is probably not guaranteed. Anyway, it's good enough for my short-lived purposes...