I'm working with the Jackson API in Java for dealing with JSON. I've been working with it a bit here and there, but nothing too in-depth.
Currently, I'm looking for a good way to take an array of JSON objects (either via a stream or String) that was created from a list of POJOs and append or remove a POJO. In the case of appending, duplicate checking isn't really necessary. As a simple example, let's say I have this array built from a list of Java objects with a single variable named "field":
[{"field":"value"},{"field":"value2"}]
And I'd like to append an object of the same type with "field" set to "value3". I could simply deserialize the whole array into a List of Java Objects, add the new object, then serialize it back into JSON, but that feels like overkill. It would be better if I could use Jackson to simply serialize the new object and append it to the end of the JSON array. The same would apply to removing an existing object from the array.
I've found a way, but strangely, it's over twice as slow as the direct deserialize-add-reserialze method with a list of 500 POJOs that have three fields each, and it only gets worse with more objects.
ObjectMapper mapper = new ObjectMapper();
JsonParser parser = mapper.getJsonFactory().createJsonParser(input);
JsonGenerator gen = mapper.getJsonFactory().createJsonGenerator(output, JsonEncoding.UTF8);
gen.writeStartArray();
parser.nextToken();
while (parser.nextToken() == JsonToken.START_OBJECT) {
//gen.writeTree(parser.readValueAsTree());
//parser.skipChildren();
//EDIT: This is much faster as the only method in the loop:
gen.copyCurrentStructure(parser);
}
gen.writeTree(mapper.valueToTree(/*new Object to add*/);
gen.writeEndArray();
gen.close();
parser.close();
Even if I don't get each object as a tree and instead move them iteratively as fields/values, it's a bit faster, but still considerably slower than the alternative. Is this to be expected or is there a better way to handle it as streaming data rather than the bulk JSON-to-Java-to-JSON method?
EDIT: AHA! Found that the JsonGenerator can directly copy the current structure from a JsonParser withcopyCurrentStructure(JsonParser). Using this in the while loop is faster and now outruns the bruteforce method by a considerable amount.
Related
I'm building a complex JSON object with Jackson and I want to structure the code to display the JSON structure clearly, so I'm trying to use the JsonNode API with chained with, put and set methods. At one place inside all this chained stuff I need to add two arrays one after the other. Here's a simplified excerpt:
com.fasterxml.jackson.databind.ObjectNode json = new ObjectNode(JsonNodeFactory.instance);
json.with("data")
.set("array1", arrayNode1)
.set("array2", arrayNode1);
I want it to create this JSON:
{
"data": {
"array1": [...],
"array2": [...]
}
}
First set is fine because it's called on ObjectNode. The problem is it returns JsonNode, which doesn't have a set method, and so the second set call results in compilation error.
How can I chain setting two arrays?
It doesn't really improve readability that much, but if you have Guava on the classpath, you can do the following:
ObjectNode json = new ObjectNode(JsonNodeFactory.instance);
json.with("data")
.setAll(ImmutableMap.of(
"array1", arraynode1,
"array2", arraynode2));
setAll returns the object node.
I would like to know how you would do this :
I have a java data object. This object contains object attributes that contains object attributes...
I serialize my objects using json-io. But when deserializing, I want to protect the data if an object fails to deserialize. So I got the idea to json-serialize each attribute before serializing any object. This way, if an attribute fails to deserialize, the object itself is safe.
this means that if I have an object {a:v1,b:{ba:v2,bb:{bba:v3,bbb:v4}}}, I will serialize it like this instead : "{\"a\":\"v1\",\"b\":\"{\\\"ba\\\":\\\"v2\\\",\\\"bb\\\":\\\"{\\\\\\\"bba\\\\\\\":\\\\\\\"v3\\\\\\\",\\\\\\\"bbb\\\\\\\":\\\\\\\"v4\\\\\\\"}\\\"}\"}"
You see that on a very simple object it becomes very uneasy to read. Imagine with a more complex one !
I would like to do something about it, but I found no good ideas. Maybe the best would be when I read/write the json to call a function able to remove/re-add the "\" where they are needed, but I don't succeed to find the correct algorithm... Any hint ? Or maybe another method ?
Thanks a lot !
Ok. My approach was wrong. I created a Data object like a Map that contains Data or String. Then, I can safely deserialize Data elements. It means that this time I have a tree, where all the trunk is a safe Data object, and only the leafs are json-serialized generic objects.
public class Data implements Serializable {
private final Map<String, Data> node = new HashMap<>();//safe tree object
private final Map<String, String> leafs = new HashMap<>();//unsafe serialized object
}
In my previous example, the result would be :
//initial object
{a:v1,b:{ba:v2,bb:{bba:v3,bbb:v4}}}
//becomes
{"a":"{v1}","b":{"ba":"{v2}","bb":{"bba":"{v3}","bbb":"{v4}"}}}
And the \ would appear only if one of the unsafe values contains a String.
Thanks a lot Ryan, Talking to you helped me a lot to better define the problems, which is always the first step to the solution.
I'm parsing the response from last.fm API. But it seems that they used some wrapper for some of the responses, which is causing a bit of a pain. To put an example:
{
"artists":{
"artist":[
{
"name":"Coldplay",
"playcount":"816763",
"listeners":"120815",
"mbid":"cc197bad-dc9c-440d-a5b5-d52ba2e14234",
"url":"http:\/\/www.last.fm\/music\/Coldplay",
"streamable":"1"
},
{
"name":"Radiohead",
"playcount":"846668",
"listeners":"99135",
"mbid":"a74b1b7f-71a5-4011-9441-d0b5e4122711",
"url":"http:\/\/www.last.fm\/music\/Radiohead",
"streamable":"1"
}
],
"#attr":{
"page":"1",
"perPage":"2",
"totalPages":"500",
"total":"1000"
}
}
}
Not only the response is wrapped in the artists object, but the array of object has also an object wrapper.
So a wrapper class like:
public class LastFMArtistWrapper {
public List<Artist> artists;
}
Would not work. I worked around this, creating two wrapper classes, but this looks really ugly. Is there any way we can use something like the #XMLElementWrapper in Jackson?
The JSON response you are getting back from the provider is a serialized representation of a hierarchy of different objects, but from your description, it sounds like you really only need to use and work with a specific subset of this representation, the collection of artists.
One solution of mirroring this representation involves creating the same hierarchy of Java classes, which creates extra overhead in the form of unneeded classes. From what I understand, this is what you wish to avoid.
The org.json project created a generic JSONObject class, which represents a single, generic key/value pair in a larger JSON representation. A JSONObject can contain other JSONObjects and JSONArrays, mirroring the representation without the extra overhead of maintaining and writing extra classes.
Thus, these two objects can be reused throughout multiple layers of hierarchy in a JSON representation, without requiring you to replicate the structure. Here is an example of how you could proceed:
// jsonText is the string representation of your JSON
JSONObject jsonObjectWrapper = new JSONObject(jsonText);
// get the "artists" object
JSONObject jsonArtists = jsonObjectWrapper.get("artists");
// get the array and pass it to Jackson's ObjectMapper, using TypeReference
// to deserialize the JSON ArrayList to a Java ArrayList.
List<Artist> artists = objectMapper.readValue(
jsonObjectWrapper.getString("artist"),
new TypeReference<ArrayList<Artist>>() { });
Using the above method, you cut out the extra overhead of having to write extra layers of POJO objects that do nothing but add unnecessary clutter.
TestCollectionDeserialization contains some examples of the readValue method when working with collections and may be helpful.
I get the feeling that the answer might be a duplicate of this: Jackson - Json to POJO With Multiple Entries but I think that potentially the question is different enough. Also I'm using raw data binding rather than full data binding.
So like the asker of that question, I have multiple objects in a file and I'm trying to turn them into POJOs and stuff them into a database of my design so I can access the data quickly rather than slowly.
The files here are in the order of tens of GB, with up to millions of objects in each file. Anyway here is what I have so far:
ObjectMapper mapper = new ObjectMapper();
Map<String,Object> data = mapper.readValue(new File("foo.json"), Map.class);
System.out.println(data.get("bar"));
And this works great for printing the bar element of the first object in foo, but I need a way to iterate through every element in a way that won't eat up all my memory.
Thanks.
You don't have to choose between Streaming (JsonParser) and ObjectMapper, do both!
Traverse a bit with parser, but then call JsonParser.readValueAs(MyType.class) to bind individual JSON Object.
Or, call ObjectMapper's readValue() method passing JsonParser at appropriate points. Or use ObjectMapper.reader(Type.class).readValues() and iterate that way.
Use this code sample to see the basic idea.
final InputStream in = new FileInputStream("json.json");
try {
for (Iterator it = new ObjectMapper().readValues(
new JsonFactory().createJsonParser(in), Map.class); it.hasNext();)
System.out.println(it.next());
}
finally { in.close();} }
Assuming you have an array wrapping your objects, create a JsonParser and then call readValuesAs with the appropriate type. It gives you back an Iterator with all your objects that reads through the file as you consume the objects.
Let's say I have an iterator of Strings. I want to create an Iterator of Java objects, and efficiently convert from one to the other. I'm not sure what the best way to do this is...the docs I've seen seem to create a new parser per String, but I'm not sure if there is an easier way?
Thanks!
Usually I would recommend just creating new JsonParser (and it does work), but if JSON Strings are very short, alternate method would be to create equivalent of StringReader that works on List or array of Strings -- sort of like java.io.SequenceInputStream, but one that works on Strings.
This should have bit lower overhead, as long as you take care NOT to concatenate Strings, but just represent Reader over equivalent of concatenated sequence.
Jackson can then read a sequence of JSON values using such Reader -- either explicitly one by one, or more conveniently, using ObjectMapper.readValues(...) (or methods from ObjectReader, instance of which you can create using various factory methods ObjectMapper has): something like
ObjectMapper mapper = new ObjectMapper();
MyReader reader = new MyReader(listOfStrings);
MappingIterator<BeanType> it = mapper.reader(BeanType.class).readValues(reader);
while (it.hasNext()) {
BeanType bean = it.nextValue();
}