We Keep Coding

Parse JSON object containing multiple nested objects without making class for each nested object

I'm using GSON in Android to parse a JSON object, part of which contains multiple nested objects holding all of the same fields. For example, the JSON structure looks similar to this:
{
"name": "nestedJSONExample",
"divisions": {
"division1": {
"id": string
"name": string,
"alsoKnownAs": [
string
],
}
"division2": {
"id": string
"name": string,
"alsoKnownAs": [
string
],
}
...
"division99" {
"id": string
"name": string,
"alsoKnownAs": [
string
],
}
}
}
In this example all of the "division##" nested objects contain all of the same fields, is there a way to parse this JSON into a Java class without creating model classes for each "division##" object?
i.e. can I create a Java structure like:
divisions.division##.id
without having to make classes for each individual division?

You seem to have a little confusion: you don't need a mapping class for each division## node since you can reuse one class multiple times regardless the property names. You might need from zero to two custom mapping classes regarding the way you prefer:
0 custom mapping classes if traversing a parsed JSON object on your own;
1 custom mapping class if applying advanced parsing techniques and combining the mapping with type adapters or JSON objects;
2 custom mapping classes for exact mapping.
The examples below are written with Java 8 language features and Java 8 Stream API but can be re-written with Java 6 easily. The JSON constant below is just a String with the following JSON document:
{
"name": "nestedJSONExample",
"divisions": {
"division1": {"id": "id1", "name": "name1", "alsoKnownAs": ["alsoKnownAs1A"]},
"division2": {"id": "id2", "name": "name2", "alsoKnownAs": ["alsoKnownAs2A"]},
"division3": {"id": "id3", "name": "name3", "alsoKnownAs": ["alsoKnownAs3A"]},
"division4": {"id": "id4", "name": "name4", "alsoKnownAs": ["alsoKnownAs4A"]},
"division5": {"id": "id5", "name": "name5", "alsoKnownAs": ["alsoKnownAs5A"]},
"division6": {"id": "id6", "name": "name6", "alsoKnownAs": ["alsoKnownAs6A"]}
}
}
No mappings
JsonElement is a built-in Gson class representing any JSON element. Combining JsonElement class and its child classes elements, Gson can build a JSON tree that reflects a given JSON document structure. So just traversing from the root is enough.
final Gson gson = new Gson();
final List<String> ids = gson.fromJson(JSON, JsonElement.class)
.getAsJsonObject()
.get("divisions") // get the divisions property
.getAsJsonObject()
.entrySet() // and traverse its key/value pairs
.stream()
.map(Entry::getValue) // discarding the keys
.map(JsonElement::getAsJsonObject)
.map(jo -> jo.get("id")) // take the id property from the every `division` object
.map(JsonElement::getAsJsonPrimitive)
.map(JsonPrimitive::getAsString)
.collect(toList());
System.out.println(ids);
Exact mappings
Here you could need just two mapping classes to describe the relations between JSON objects. The divisions node can be just a Map holding arbitrary keys and Division values.
final class OuterWithMap {
//String name;
Map<String, Division> divisions;
}
final class Division {
String id;
//String name;
//List<String> alsoKnownAs;
}
final Gson gson = new Gson();
final List<String> ids = gson.fromJson(JSON, OuterWithMap.class)
.divisions
.values() // use map values only ignoring the keys
.stream()
.map(d -> d.id)
.collect(toList());
System.out.println(ids);
Not exact mappings
This is the most complicated one and shows advanced techniques in parsing JSON with Gson and mapping given JSON documents to mapping classes may not reflect the real structure therefore making transformations on-fly.
final class OuterWithList {
//String name;
#JsonAdapter(NoKeysTypeAdapterFactory.class)
List<Division> divisions;
}
final class NoKeysTypeAdapterFactory
implements TypeAdapterFactory {
// No accessible constructor needed - Gson can instantiate it itself
private NoKeysTypeAdapterFactory() {
}
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// Is it a list?
if ( List.class.isAssignableFrom(typeToken.getRawType()) ) {
// Try to determine the list element type
final Type elementType = getElementType(typeToken.getType());
// And create a custom type adapter instance bound to the specific list type
#SuppressWarnings("unchecked")
final TypeAdapter<T> typeAdapter = (TypeAdapter<T>) getNoKeysTypeAdapter(gson, elementType);
return typeAdapter;
}
// Otherwise just tell Gson try to find another appropriate parser
return null;
}
private static Type getElementType(final Type type) {
// Is it a generic type with type parameters?
if ( type instanceof ParameterizedType ) {
final ParameterizedType parameterizedType = (ParameterizedType) type;
// If yes, then just take the first type argument since java.util.List can only one type
return parameterizedType.getActualTypeArguments()[0];
}
// Otherwise java.lang.Object due to either Java generics type erasure or raw types usage
return Object.class;
}
}
final class NoKeysTypeAdapter<E>
extends TypeAdapter<List<E>> {
private final Gson gson;
private final Type elementType;
private NoKeysTypeAdapter(final Gson gson, final Type elementType) {
this.gson = gson;
this.elementType = elementType;
}
static <E> TypeAdapter<List<E>> getNoKeysTypeAdapter(final Gson gson, final Type elementType) {
return new NoKeysTypeAdapter<>(gson, elementType);
}
#Override
public void write(final JsonWriter out, final List<E> value) {
throw new UnsupportedOperationException();
}
#Override
public List<E> read(final JsonReader in)
throws IOException {
final List<E> list = new ArrayList<>();
// Make sure that the next JSON stream token is `{`
in.beginObject();
// Read until the object ends
while ( in.peek() != END_OBJECT ) {
// Ignore the found JSON object property name
in.nextName();
// And delegate the property value parsing to a downstream parser
final E element = gson.fromJson(in, elementType);
list.add(element);
}
// Make sure that the JSON stream is finished with the `}` token
in.endObject();
return list;
}
}
Using a special querying library
There are some libraries like JsonPath that can make querying JSON documents somewhat easier. JsonPath can work without Gson, however, as far as I understand, it uses another JSON parsing library, and does not parse JSON itself (but I don't know how it actually is). Example of use:
final JsonPath jsonPath = JsonPath.compile("$.divisions.*.id");
final List<String> ids = jsonPath.<JSONArray>read(JSON)
.stream()
.map(o -> (String) o)
.collect(toList());
System.out.println(ids);
All four examples above have the following output:
[id1, id2, id3, id4, id5, id6]

Using GSON your best bet is to write a custom Deserializer (example) or a TypeAdapter (example), this will allow you to do whatever you want with the structure then return a single (top level) object

GSON - Optional and required fields with naming policy

I need a function, that reads a json file and control the structur of the json file. Required fields should be defined. For that
I found a question that resolve a part of my problem Gson optional and required fields. But in this case the naming convention has not power any more. In my case I used following GsonBuilder:
this.gsonUpperCamelCase = new GsonBuilder()
.registerTypeAdapter(TestClass.class, new AnnotatedDeserializer<TestClass>())
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
Every key-value from JSON, that is in this case the deserialized java object need to be lowercase. Otherwise it will throw JsonParseException.
For example I have this class:
class TestClass {
#JsonRequired
private String testName;
//getter & setter
Then this JSON-file can not be deserialized:
{
"TestName":"name"
}
But I want to get sure that UPPER_CAMEL_CASE is used in this case. Thx.

SerializedName is the annotation that can help you on this. Modifying the TestClass as below, you should be able to deserialize a JSON with TestName, tn, tn2 and when serializing, it always uses testName.
static class TestClass {
#JsonRequired
#SerializedName(value="testName", alternate = {"TestName", "tn", "tn2"})
private String testName;
}
public static void main(String[] args) {
Gson gsonUpperCamelCase = new GsonBuilder()
.registerTypeAdapter(TestClass.class,
new AnnotatedDeserializer<TestClass>())
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
TestClass tc = gsonUpperCamelCase.fromJson("{\r\n" +
" \"TestName\":\"name\"\r\n" +
"}", TestClass.class);
System.out.println(tc.testName);
System.out.println(gsonUpperCamelCase.toJson(tc));
}
Output
name
{"testName":"name"}

How to use dynamic json value on my POJO class with Gson?

{
"localeCode": "",
"map": {
"DynamicName1": [],
"DynamicName2": [
{
"date": "2016-05-15T00:00:00",
"seqId": 1,
"status": 10
},
{
"date": "2016-05-16T00:00:00",
"seqId": 83,
"status": 10
}
],
"DynamicName3": [],
"DynamicName4": []
},
"respCode": 100,
"respMsg": "success",
"status": 1
}
How to correctly map this kind of json. If you can see that, Dynamic is a dynamic name. So far I have done this :
public class MapModel {
public MapObject map;
public static class MapObject{
public java.util.Map<String, Student> queryStudent;
public static class Student{
public String date;
public String seqId;
public String status;
}
}
}
But when run the app. I'm getting NullPointerException. Can somebody help me?

You're getting the NullPointerException accessing queryStudent of your MapObject inside your MapModel since it's not correctly filled when you're trying to deserialize your Json.
So to solve your problem look at Gson documentation where you can see:
You can serialize the collection with Gson without doing anything
specific: toJson(collection) would write out the desired output.
However, deserialization with fromJson(json, Collection.class) will
not work since Gson has no way of knowing how to map the input to the
types. Gson requires that you provide a genericised version of
collection type in fromJson(). So, you have three options:
Use Gson's parser API (low-level streaming parser or the DOM parser
JsonParser) to parse the array elements and then use Gson.fromJson()
on each of the array elements.This is the preferred approach. Here is
an example that demonstrates how to do this.
Register a type adapter for Collection.class that looks at each of the
array members and maps them to appropriate objects. The disadvantage
of this approach is that it will screw up deserialization of other
collection types in Gson.
Register a type adapter for MyCollectionMemberType and use fromJson()
with Collection.
Since your MapObject containts a java.util.Map but your class itself it's not generic, I think that a good approach for your case is create a Deserializer.
Before this try to clean up your class definition, to provide constructors to make the deserializer easy to build. Your POJO classes could be:
Student class
public class Student{
public String date;
public String seqId;
public String status;
public Student(String date, String seqId, String status){
this.date = date;
this.seqId = seqId;
this.status = status;
}
}
MapObject class
Note: I change you Map definition, since in your Json seems that could be multiple students for each DynamicName (look at DynamicName2 from your question), so I use Map<String,List<Student>> instead of Map<String,Student>:
public class MapObject{
public Map<String,List<Student>> queryStudent;
public MapObject(Map<String,List<Student>> value){
this.queryStudent = value;
}
}
MapModel class
public class MapModel {
public MapObject map;
}
Now create a Deserializer for your MapObject:
public class MapObjectDeserializer implements JsonDeserializer<MapObject> {
public MapObject deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
throws JsonParseException {
Map<String,List<Student>> queryStudents = new HashMap<String,List<Student>>();
// for each DynamicElement...
for (Map.Entry<String,JsonElement> entry : json.getAsJsonObject().entrySet()) {
List<Student> students = new ArrayList<Student>();
// each dynamicElement has an Array so convert and add an student
// for each array entry
for(JsonElement elem : entry.getValue().getAsJsonArray()){
students.add(new Gson().fromJson(elem,Student.class));
}
// put the dinamic name and student on the map
queryStudents.put(entry.getKey(),students);
}
// finally create the mapObject
return new MapObject(queryStudents);
}
}
Finally register the Deserializer and parse your Json:
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(MapObject.class, new MapObjectDeserializer());
Gson gson = builder.create();
MapModel object = gson.fromJson(YourJson,MapModel.class);
DISCLAIMER: For fast prototyping I test this using groovy, I try to keep the Java syntax but I can forget something, anyway I think that this can put you on the right direction.
Hope it helps,

Using Gson to elegantly handle nested json objects?

I'm using Gson to parse responses from a server on Android. Each response has some useless (to me) data on it that complicates my Gson models. Here is the general hierarchy of json returned:
response: {
date: 1406253006807,
otherUselessData1: "This is some useless data",
otherUselessData2: "This is some useless data",
usefulJsonObject: { <---- This is really the object that I care about
}
}
Everything above or at the same level as usefulJsonObject I could really do without. The useless data is returned for every request, and the actual response is embedded beneath as the usefulJsonObject. This wouldn't be a big problem but it's really cluttering up my gson model objects.
For example:
Let's say I have 3 requests I can make: A, B, and C. For each response it seems I need to make a minimum of 3 custom classes.
public class ResponseA {
#SerializedName("response") ResponseObjectA responseObject;
public static class ResponseObjectA {
#SerializedName("usefulJsonObject") UsefulObjectA usefulObject;
}
public static class UsefulObjectA {
}
}
I've tried a few solutions, but I haven't found anything elegant that wouldn't add an extra step to my process. I'm using retrofit to do my http requests and it's really nice that it just returns the fully parsed gson object to me. I've thought of other solutions like having the useful object just be a JsonElement and then doing a 2nd gson call after the first comes back. Again, not ideal.
I just wanted to know if I was missing something. Surely I'm not the only one who's encountered something like this, and so I thought I'd ask how other people would handle something like this.

It is initialization Instance value, not NULL value. Check my example.
Address.java
public class Address {
public Address(){
}
}
Person.java
public class Person {
private String name;
private String nrc;
private Address address;
public Person(String name, String nrc, Address address) {
this.name = name;
this.nrc = nrc;
this.address = address;
}
}
The following Json string is equalvent to
Person person = new Person("Zaw Than Oo", "11111", null);
{
"name": "Zaw Than Oo",
"nrc": "11111"
}
The following Json string is equalvent to
Person person = new Person("Zaw Than Oo", "11111", new Address());
{
"name": "Zaw Than Oo",
"nrc": "11111",
"address": {} <-- here use less object for you.
}
Even if you don't create new Instance, Other lib/api(you used) may be create that instance by Reflection.
Short to the Point
{
...
"xxx": {} --> new instance without data/value
...
}
{
...
--> null value
...
}

I never found an elegant way dealing with just Gson. I tried several options with Generics, all of which didn't work or left something to be desired.
Since I'm using Retrofit, I decided to override the GsonConverter, and just filter out the unnecessary information from all my requests. It ends up not being as flexible, as in I can't use the same Retrofit network interface for calls to other servers, but I'm not really doing that, and it also has the down side of having 2 rounds of json parsing calls (meh). You could probably do this more efficiently, but this is working for me for now.
public class CustomGsonConverter extends GsonConverter {
private Gson mGson;
public CustomGsonConverter(Gson gson) {
super(gson);
this.mGson = gson;
}
public CustomGsonConverter(Gson gson, String encoding) {
super(gson, encoding);
this.mGson = gson;
}
#Override public Object fromBody(TypedInput body, Type type) throws ConversionException {
try {
CustomResponse customResponse = mGson.fromJson(new InputStreamReader(body.in()), CustomResponse.class);
return mGson.fromJson(customResponse.responseObject.data, type);
} catch (IOException e) {
throw new ConversionException(e);
}
}
public static class CustomResponse {
#SerializedName("rsp") ResponseObject responseObject;
public static class ResponseObject {
// #SerializedName("date") long date;
#SerializedName("data") JsonElement data;
}
}
}
Maybe there is a better way that I'm just not realizing.

Can I use Gson to serialize method-local classes and anonymous classes?

Example:
import com.google.gson.Gson;
class GsonDemo {
private static class Static {String key = "static";}
private class NotStatic {String key = "not static";}
void testGson() {
Gson gson = new Gson();
System.out.println(gson.toJson(new Static()));
// expected = actual: {"key":"static"}
System.out.println(gson.toJson(new NotStatic()));
// expected = actual: {"key":"not static"}
class MethodLocal {String key = "method local";}
System.out.println(gson.toJson(new MethodLocal()));
// expected: {"key":"method local"}
// actual: null (be aware: the String "null")
Object extendsObject = new Object() {String key = "extends Object";};
System.out.println(gson.toJson(extendsObject));
// expected: {"key":"extends Object"}
// actual: null (be aware: the String "null")
}
public static void main(String... arguments) {
new GsonDemo().testGson();
}
}
I would like these serializations especially in unit tests. Is there a way to do so?
I found Serializing anonymous classes with Gson, but the argumentation is only valid for de-serialization.

FWIW, Jackson will serialize anonymous and local classes just fine.
public static void main(String[] args) throws Exception
{
ObjectMapper mapper = new ObjectMapper();
class MethodLocal {public String key = "method local";}
System.out.println(mapper.writeValueAsString(new MethodLocal()));
// {"key":"method local"}
Object extendsObject = new Object() {public String key = "extends Object";};
System.out.println(mapper.writeValueAsString(extendsObject));
// {"key":"extends Object"}
}
Note that Jackson by default won't access non-public fields through reflection, as Gson does, but it could be configured to do so. The Jackson way is to use regular Java properties (through get/set methods), instead. (Configuring it to use private fields does slow down the runtime performance, a bit, but it's still way faster than Gson.)