We Keep Coding

Jackson - Working with arrays of objects, appending and removing

I'm working with the Jackson API in Java for dealing with JSON. I've been working with it a bit here and there, but nothing too in-depth.
Currently, I'm looking for a good way to take an array of JSON objects (either via a stream or String) that was created from a list of POJOs and append or remove a POJO. In the case of appending, duplicate checking isn't really necessary. As a simple example, let's say I have this array built from a list of Java objects with a single variable named "field":
[{"field":"value"},{"field":"value2"}]
And I'd like to append an object of the same type with "field" set to "value3". I could simply deserialize the whole array into a List of Java Objects, add the new object, then serialize it back into JSON, but that feels like overkill. It would be better if I could use Jackson to simply serialize the new object and append it to the end of the JSON array. The same would apply to removing an existing object from the array.

I've found a way, but strangely, it's over twice as slow as the direct deserialize-add-reserialze method with a list of 500 POJOs that have three fields each, and it only gets worse with more objects.
ObjectMapper mapper = new ObjectMapper();
JsonParser parser = mapper.getJsonFactory().createJsonParser(input);
JsonGenerator gen = mapper.getJsonFactory().createJsonGenerator(output, JsonEncoding.UTF8);
gen.writeStartArray();
parser.nextToken();
while (parser.nextToken() == JsonToken.START_OBJECT) {
//gen.writeTree(parser.readValueAsTree());
//parser.skipChildren();
//EDIT: This is much faster as the only method in the loop:
gen.copyCurrentStructure(parser);
}
gen.writeTree(mapper.valueToTree(/*new Object to add*/);
gen.writeEndArray();
gen.close();
parser.close();
Even if I don't get each object as a tree and instead move them iteratively as fields/values, it's a bit faster, but still considerably slower than the alternative. Is this to be expected or is there a better way to handle it as streaming data rather than the bulk JSON-to-Java-to-JSON method?
EDIT: AHA! Found that the JsonGenerator can directly copy the current structure from a JsonParser withcopyCurrentStructure(JsonParser). Using this in the while loop is faster and now outruns the bruteforce method by a considerable amount.

safe json to java serialization/deserialization

I would like to know how you would do this :
I have a java data object. This object contains object attributes that contains object attributes...
I serialize my objects using json-io. But when deserializing, I want to protect the data if an object fails to deserialize. So I got the idea to json-serialize each attribute before serializing any object. This way, if an attribute fails to deserialize, the object itself is safe.
this means that if I have an object {a:v1,b:{ba:v2,bb:{bba:v3,bbb:v4}}}, I will serialize it like this instead : "{\"a\":\"v1\",\"b\":\"{\\\"ba\\\":\\\"v2\\\",\\\"bb\\\":\\\"{\\\\\\\"bba\\\\\\\":\\\\\\\"v3\\\\\\\",\\\\\\\"bbb\\\\\\\":\\\\\\\"v4\\\\\\\"}\\\"}\"}"
You see that on a very simple object it becomes very uneasy to read. Imagine with a more complex one !
I would like to do something about it, but I found no good ideas. Maybe the best would be when I read/write the json to call a function able to remove/re-add the "\" where they are needed, but I don't succeed to find the correct algorithm... Any hint ? Or maybe another method ?
Thanks a lot !

Ok. My approach was wrong. I created a Data object like a Map that contains Data or String. Then, I can safely deserialize Data elements. It means that this time I have a tree, where all the trunk is a safe Data object, and only the leafs are json-serialized generic objects.
public class Data implements Serializable {
private final Map<String, Data> node = new HashMap<>();//safe tree object
private final Map<String, String> leafs = new HashMap<>();//unsafe serialized object
}
In my previous example, the result would be :
//initial object
{a:v1,b:{ba:v2,bb:{bba:v3,bbb:v4}}}
//becomes
{"a":"{v1}","b":{"ba":"{v2}","bb":{"bba":"{v3}","bbb":"{v4}"}}}
And the \ would appear only if one of the unsafe values contains a String.
Thanks a lot Ryan, Talking to you helped me a lot to better define the problems, which is always the first step to the solution.

Jackson handling Wrapped elements

I'm parsing the response from last.fm API. But it seems that they used some wrapper for some of the responses, which is causing a bit of a pain. To put an example:
{
"artists":{
"artist":[
{
"name":"Coldplay",
"playcount":"816763",
"listeners":"120815",
"mbid":"cc197bad-dc9c-440d-a5b5-d52ba2e14234",
"url":"http:\/\/www.last.fm\/music\/Coldplay",
"streamable":"1"
},
{
"name":"Radiohead",
"playcount":"846668",
"listeners":"99135",
"mbid":"a74b1b7f-71a5-4011-9441-d0b5e4122711",
"url":"http:\/\/www.last.fm\/music\/Radiohead",
"streamable":"1"
}
],
"#attr":{
"page":"1",
"perPage":"2",
"totalPages":"500",
"total":"1000"
}
}
}
Not only the response is wrapped in the artists object, but the array of object has also an object wrapper.
So a wrapper class like:
public class LastFMArtistWrapper {
public List<Artist> artists;
}
Would not work. I worked around this, creating two wrapper classes, but this looks really ugly. Is there any way we can use something like the #XMLElementWrapper in Jackson?

The JSON response you are getting back from the provider is a serialized representation of a hierarchy of different objects, but from your description, it sounds like you really only need to use and work with a specific subset of this representation, the collection of artists.
One solution of mirroring this representation involves creating the same hierarchy of Java classes, which creates extra overhead in the form of unneeded classes. From what I understand, this is what you wish to avoid.
The org.json project created a generic JSONObject class, which represents a single, generic key/value pair in a larger JSON representation. A JSONObject can contain other JSONObjects and JSONArrays, mirroring the representation without the extra overhead of maintaining and writing extra classes.
Thus, these two objects can be reused throughout multiple layers of hierarchy in a JSON representation, without requiring you to replicate the structure. Here is an example of how you could proceed:
// jsonText is the string representation of your JSON
JSONObject jsonObjectWrapper = new JSONObject(jsonText);
// get the "artists" object
JSONObject jsonArtists = jsonObjectWrapper.get("artists");
// get the array and pass it to Jackson's ObjectMapper, using TypeReference
// to deserialize the JSON ArrayList to a Java ArrayList.
List<Artist> artists = objectMapper.readValue(
jsonObjectWrapper.getString("artist"),
new TypeReference<ArrayList<Artist>>() { });
Using the above method, you cut out the extra overhead of having to write extra layers of POJO objects that do nothing but add unnecessary clutter.
TestCollectionDeserialization contains some examples of the readValue method when working with collections and may be helpful.

Jackson multiple objects and huge json files

I get the feeling that the answer might be a duplicate of this: Jackson - Json to POJO With Multiple Entries but I think that potentially the question is different enough. Also I'm using raw data binding rather than full data binding.
So like the asker of that question, I have multiple objects in a file and I'm trying to turn them into POJOs and stuff them into a database of my design so I can access the data quickly rather than slowly.
The files here are in the order of tens of GB, with up to millions of objects in each file. Anyway here is what I have so far:
ObjectMapper mapper = new ObjectMapper();
Map<String,Object> data = mapper.readValue(new File("foo.json"), Map.class);
System.out.println(data.get("bar"));
And this works great for printing the bar element of the first object in foo, but I need a way to iterate through every element in a way that won't eat up all my memory.
Thanks.

You don't have to choose between Streaming (JsonParser) and ObjectMapper, do both!
Traverse a bit with parser, but then call JsonParser.readValueAs(MyType.class) to bind individual JSON Object.
Or, call ObjectMapper's readValue() method passing JsonParser at appropriate points. Or use ObjectMapper.reader(Type.class).readValues() and iterate that way.

Use this code sample to see the basic idea.
final InputStream in = new FileInputStream("json.json");
try {
for (Iterator it = new ObjectMapper().readValues(
new JsonFactory().createJsonParser(in), Map.class); it.hasNext();)
System.out.println(it.next());
}
finally { in.close();} }

Assuming you have an array wrapping your objects, create a JsonParser and then call readValuesAs with the appropriate type. It gives you back an Iterator with all your objects that reads through the file as you consume the objects.

JSON handling Array and object with same Java code

I have a simple question for android/java implementing JSON. Is there any method possible to handle an array or object with the same code. In my Json response there are many placed where JSON returns a single argument and some times many arguments in an array. I will have to write a long code to handle whether object contains an array or an object otherwise.
Also if somebody help me, how to check whether a element contains JSONArray or JSONObject?

Simply test (instanceof) the type of the object you get back from the JSON parser to see whether it's an array, dictionary, or single object.
If you want to avoid duplication where the array is an array of the instances you'd get back in the simpler case, first test for non-array, and if that's what you have, create an array and stick the single object into it, before passing it to your generalized array-handling logic.