We Keep Coding

Jackson: deserialize JSON extract deep attribute into parent class

I have some trouble wording my title, so if my question should be re-worded, I'd be happy to repost this question for clarification. :)
Problem: I have this JSON structure
{
"name": "Bob",
"attributes": {
"evaluation": {
"stats": [
{
"testDate": "2020-02-04",
"score": 50
},
{
"testDate": "2020-04-01",
"score": 90
},
{
"testDate": "2020-05-10",
"score": 85
}
],
"survey": {...}
},
"interests": {...},
"personality": [...],
"someRandomUnknownField": {...}
}
}
attributes is any random number of fields except for evaluation.stats that we want to extract out. I want to be able to deserialize into the following classes:
public class Person {
String name;
Map<String, Object> attributes;
List<Stat> stats;
}
public class Stat {
LocalDate date;
int score;
}
When I serialize it back to JSON, I should expect something like this:
{
"name": "Bob",
"attributes" : {
"evaluation": {
"survey": {...}
},
"interests" : {...},
"personality": {...},
"someRandomUnknownField": {...}
},
"stats": [
{
"testDate": "2020-02-04",
"score": 50
},
{
"testDate": "2020-04-01",
"score": 90
},
{
"testDate": "2020-05-10",
"score": 85
}
]
}
I could technically map the whole Person class to its own custom deserializer, but I want to leverage the built-in Jackson deserializers and annotations as much as possible. It's also imperative that stats is extracted (i.e., stats shouldn't also exist under attributes). I'm having trouble finding a simple and maintainable serialization/deserialization scheme. Any help would be appreciate!

I'm not sure if this meets your criterion for a simple and maintainable serialization/deserialization scheme, but you can manipulate the JSON tree to transform your starting JSON into the structure you need:
Assuming I start with a string containing your initial JSON:
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
JsonNode root = mapper.readTree(inputJson);
// copy the "stats" node to the root of the JSON:
ArrayNode statsNode = (ArrayNode) root.path("attributes").path("evaluation").path("stats");
((ObjectNode) root).set("stats", statsNode);
// delete the original "stats" node:
ObjectNode evalNode = (ObjectNode) root.path("attributes").path("evaluation");
evalNode.remove("stats");
This now gives you the JSON you need to deserialize to your Person class:
Person person = mapper.treeToValue(root, Person.class);
When you serialize the Person object you get the following JSON output:
{
"name" : "Bob",
"attributes" : {
"evaluation" : {
"survey" : { }
},
"interests" : { },
"personality" : [ ],
"someRandomUnknownField" : { }
},
"stats" : [ {
"score" : 50,
"testDate" : "2020-02-04"
}, {
"score" : 90,
"testDate" : "2020-04-01"
}, {
"score" : 85,
"testDate" : "2020-05-10"
} ]
}
Just to note, to get this to work, you need the java.time module:
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>2.11.3</version>
</dependency>
And you saw how this was registered in the above code:
mapper.registerModule(new JavaTimeModule());
I also annotated the LocalDate field in the Stat class, as follows:
#JsonProperty("testDate")
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
private LocalDate date;
Very minor note: In your starting JSON (in the question) you showed this:
"personality": [...],
But in your expected final JSON you had this:
"personality": {...},
I assumed this was probably a typo, and it should be an array, not an object, in both cases.

How to return a list of multiple JSON items in java

I am trying to figure out how to return multiple JSON items. Right now I am able, to return a single JSON like so:
{
"result": {
"userId": "abcde123",
"telephoneNumber": "1-555-5555555"
},
"error": null
}
But I would like to return multiple JSON items, like so:
{
"result": {{
"userId": "abcde123",
"telephoneNumber": "1-555-5555555"
}
{
"userId": "fghi456",
"telephoneNumber": "1-333-3333333"
}
},
"error": null
}
I can view the multiple JSON items as string, like below, but I would like to return it as multiple JSON items, but I don't know how:
[LDAPModel(userId=abcde123, telephoneNumber=1-555-5555555), LDAPModel(userId=fghi456, telephoneNumber=1-333-3333333]
I am a complete beginner in Java, and I don't know the syntax or much in Java. But I was given these codes (including the one below) from SpringBoot; I really don't understand what it is doing, and so I have no idea how create an output of list.
Currently, this is what I was given:
public Optional<LDAPModel> getDirectReports(String cdsID) {
LdapQuery ldapQuery = LdapQueryBuilder.query()
.searchScope(SearchScope.SUBTREE)
.where("objectclass").is("person")
.and("managerID").like(cdsID);
List<LDAPModel> ldapModelList = ldapTemplate.search(ldapQuery, (Attributes attrs) ->
LDAPModel.builder()
.userId(getValue(attrs, "userid"))
.telephoneNumber(getValue(attrs, "phoneNumber"))
.build());
// for (int ii = 0; ii < ldapModelList.size(); ii++) {
// Optional.of(ldapModelList.get(ii));
// ldapModelList.isEmpty() ? Optional.empty() : Optional.of(ldapModelList.get(ii));
// }
return ldapModelList.isEmpty() ? Optional.empty() : Optional.of(ldapModelList.get(0));
}
I tried putting it in a loop (like in the commented out code above), but I don't know how create a list. I tried removing the get(0), but there was a syntax error... There are many things I tried, but it just did not help.
Anyone can help?
Update/Edit: Thank you all for your answers. I posted a follow up question here. If you have a chance, please help me out. Thanks.

First of all I would like to point out that your JSON isn't formatted properly. When you want to represent multiple objects in JSON you should use square brackets and separate each object with a comma:
{
"result": [
{
"userId": "abcde123",
"telephoneNumber": "1-555-5555555"
},
{
"userId": "fghi456",
"telephoneNumber": "1-333-3333333"
}
],
"error": null
}
The reason your Java code does not work when you try and remove get(0) is because the method public Optional<LDAPModel> getDirectReports(String cdsID) returns an Optional<LDAPModel> type and by removing get(0) your are effectively trying to return an Optional<List<LDAPModel>>. If you want the method to return a list instead of a single object you can change the return type to Optional<List<LDAPModel>> and then safely remove get(0).
public Optional<List<LDAPModel>> getDirectReports(String cdsID) {
LdapQuery ldapQuery = LdapQueryBuilder.query()
.searchScope(SearchScope.SUBTREE)
.where("objectclass").is("person")
.and("managerID").like(cdsID);
List<LDAPModel> ldapModelList = ldapTemplate.search(ldapQuery, (Attributes attrs) ->
LDAPModel.builder()
.userId(getValue(attrs, "userid"))
.telephoneNumber(getValue(attrs, "phoneNumber"))
.build());
return ldapModelList.isEmpty() ? Optional.empty() : Optional.of(ldapModelList);
}

The structure looks strange to me. What you have looks like you want result to be an array of objects:
{
"result": [
{ "userId": "abcde123",
"telephoneNumber": "1-555-5555555" }
{ "userId": "fghi456",
"telephoneNumber": "1-333-3333333" }
],
"error": null
}
Given a reasonable JSON library, then the value of the "result" member of the JSON object is a JSON array, from which you can then pick out each element in turn by indexing, and each element is a JSON object with 2 members.

I assume you already managed to get all the list/array of LDAPModel i.e. List ldapModelList
If so, you just need to return this ldapModelList in your getDirectReports method.
public List<LDAPModel> getDirectReports(String cdsID) {
LdapQuery ldapQuery = LdapQueryBuilder.query()
.searchScope(SearchScope.SUBTREE)
.where("objectclass").is("person")
.and("managerID").like(cdsID);
List<LDAPModel> ldapModelList = ldapTemplate.search(ldapQuery, (Attributes attrs) ->
LDAPModel.builder()
.userId(getValue(attrs, "userid"))
.telephoneNumber(getValue(attrs, "phoneNumber"))
.build());
return ldapModelList;
}
Then just use your library to return the json array. I suppose you use jackson.
Just make sure in LDAPModel you have
getters and setters
empty constructor if you add your own constructor having params. But if you don't add any constructor, then no need to add this default empty constructor as java will automatically create it for you.
LDAPModel class is as follows:
public class LDAPModel {
String userId;
String telephoneNumber;
public String getUserId() {
return userId;
}
public void setUserId(String userId) {
this.userId = userId;
}
public String getTelephoneNumber() {
return telephoneNumber;
}
public void setTelephoneNumber(String telephoneNumber) {
this.telephoneNumber = telephoneNumber;
}
}
For the object to JSON string conversion using Jackson, I assume you already know it or can find out how.

Jackson deserialization of union object with sibling type

I'm trying to combine a few features of Jackson such that I can deserialize a {type,value} pair in json into a Union type in java representing all the information but can't work out how to do it. Help would be greatly appreciated.
Here is what I'm working with:
The java Union type enforces that only a single value can be set at any one time, think of this like an enum but with dynamic data.
The value of the union can take any number of types, some scalar and some object or collection.
Multiple options for the union can share the same type, i.e. noyes and offon are both boolean types in the example below.
I'm not in control of the json structure in any way (it's the data passed to/from an external API) so can't change it at all.
The java classes are generated code from Thrift idl so I can't add annotations to them or adjust their structure drastically. I am in control of the idl though, but would like to keep it fairly clean and free of leaky patterns like the {type, value} that is needed for json but not strongly typed languages.
The type names used in the json (and some of the field names) conflict with java (and other language) keywords which is why each value is suffixed: noyes -> noYesValue, float -> floatValue
A concrete example
I have a json document that looks like this:
{
"obj": [
{
"type": "noyes",
"value": true,
"id": 1
}, {
"type": "offon",
"value": false,
"id": 2
}, {
"type": "text",
"value": "hello",
"id": 3
}, {
"type": "float",
"value": 1.2,
"id": 4
}, {
"type": "times",
"value": [{"s": "12:22", "e": "16:00"}]
"id": 5
}
]
}
And java classes that look like this:
class Response {
List<Item> obj;
}
class Item {
int id;
Value value;
}
class Value {
enum Fields { NO_YES_VALUE, OFF_ON_VALUE, TEXT_VALUE, FLOAT_VALUE, TIMES_VALUE }
static Item noYesValue(boolean noYes) {...}
static Item offOnValue(boolean offOn) {...}
static Item textValue(String text) {...}
static Item floatValue(float value) {...}
static Item timesValue(List<TimesValue> times) {...}
Fields setField;
Object fieldValue;
Value() {}
Value(Fields field, Object value) {setFieldValue(field, value);}
Fields getSetField() { return setField; }
Object getFieldValue() { return fieldValue; }
void setFieldValue(Fields field, Object value) {
// checkType(field, value);
this.setField = field;
this.fieldValue = value;
}
// these do have checks for the set field, types, null, etc
boolean getNoYesValue() { return (Boolean) fieldValue; }
void setNoYesValue(boolean v) { setField = NO_YES_VALUE; fieldValue = v; }
boolean getOffOnValue() { return (Boolean) fieldValue; }
void setOffOnValue(boolean v) { setField = OFF_ON_VALUE; fieldValue = v; }
// ...
}
class TimesValue {
String startTime;
String endTime;
}
With the java classes the following are equivalent:
Value.noYesValue(false);
new Value().setNoYesValue(false);
new Value().setFieldValue(NO_YES_VALUE, false);
new Value(NO_YES_VALUE, false);
Similarly the following are equivalent:
value.getNoYesValue();
(Boolean) value.getFieldValue();
Partially working code (does what I want but only with simple types)
After some more digging and trial and error I've managed to get farther than I have before.
First things first, I flatten out the Value into the Item
interface ItemMixin {
#JsonUnwrapped Value getValue();
}
This hoists all my Value properties so they become part of the Item class, the equivalent of {"value": {"type": "offon", "value": false}} becoming {"type": "offon", "value": false}
Next I need to deal with that Fields enum, to do this I wrote a custom Deserializer which looked like this and register it with a module
new StdDeserializer<Value.Fields>(Value.Fields.class) {
#Override
public Value.Fields deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
String type = p.getText();
switch (type) {
case "noyes": return Value.Fields.NO_YES_VALUE;
case "offon": return Value.Fields.OFF_ON_VALUE;
case "float": return Value.Fields.FLOAT_VALUE;
case "text": return Value.Fields.TEXT_VALUE;
case "times": return Value.Fields.TIMES_VALUE;
// ...
}
ctxt.handleWeirdStringValue(Value.Fields.class, type, "Unsupported Value type");
return null;
}
}
Finally I tell the Value type to use the constructor to create the object
static abstract class ValueMixin {
#JsonCreator
ValueMixin(#JsonProperty("type") Value.Fields type, #JsonProperty("value") Object value) {}
}
This gives me 90% of what I need, however the issue I now have is that this only works for primitive types (boolean, String, etc), my times field just gets deserialized as a String causing exceptions in my code.
I've tried using #JsonTypeInfo and #JsonSubTypes on the value parameter, creating a property for that field and putting the annotations there but can't get the creator to correctly resolve the type needed.

Deseralize JSON with Jackson into a list of heterogeneous elements

In my JSON I have an element with the following contents:
{
...
"locations": [
[
{
"location_type": "permanent",
"position": "at",
"accuracy": "exact"
},
"and",
{
"location_type": "permanent",
"position": "in",
"accuracy": "exact"
}
],
"or",
{
"location_type": "temporary",
"position": "at",
"accuracy": "exact"
}
],
...
}
As shown, an element of locations can be:
a location
a logical operator
a list of locations (allowing for complex locations)
I'm getting "Cannot deserialize instance of com.example.processor.transformation.json.Location out of START_ARRAY token".
How can I consume this into a data structure using Jackson?
What I tried so far:
Providing a Location(String logicalOperator) constructor helps for a flat list case. (I basically turn the operator into a special value of Location.)
Adding a Location(List<Location> subLocations) or a Location(Location[] subLocations) constructor doesn't help for this case.
Note: I am not in control of the JSON format so I cannot encode it in a more Jackson-friendly way.

You're going to need a custom de-serializer for that. You can't just add a constructor.
Here's a self-contained example with class Foo, that can be either represented by its own property "foo" : "someString" or by some logical operator "and" or "or", etc. as a String literal, intended to represent a Foo instance whose foo property will be the value of that literal.
This may or may not fit your case exactly, but you can adjust.
In other words:
{"foo": "a"} --> new Foo("a")
"or" --> new Foo("or")
Example
// given...
#JsonDeserialize(using=MyDeserializer.class)
class Foo {
String foo;
public void setFoo(String s) {
foo = s;
}
public String getFoo() {
return foo;
}
public Foo(String s) {
setFoo(s);
}
}
// and custom de-serializer...
class MyDeserializer extends JsonDeserializer<Foo> {
#Override
public Foo deserialize(JsonParser jp, DeserializationContext ct)
throws IOException, JsonProcessingException {
ObjectCodec oc = jp.getCodec();
JsonNode node = oc.readTree(jp);
// this JSON object has a "foo" property, de-serialize
// injecting its value in Foo's constructor
if (node.has("foo")) {
return new Foo(node.get("foo").asText());
}
// other case, assuming literal (e.g. "and", "or", etc.)
// inject actual node as String value into Foo's constructor
else {
return new Foo(node.asText());
}
}
}
// here's a quick example
String json = "[{\"foo\": \"a\"}, \"or\", {\"foo\": \"b\"}]";
ObjectMapper om = new ObjectMapper();
List<Foo> list = om.readValue(json, new TypeReference<List<Foo>>(){});
list.forEach(f -> System.out.println(f.foo));
Output
a
or
b
Note for clarity
This represents a very simple example.
In your case, you're probably going to want a polymorphic collection of Location POJOs mixed with LogicalOperator POJOs (or something similar), sharing a common marker interface.
You can then decide what object to de-serialize based on whether the JSON node features contents (i.e. a location) or the JSON node is its contents (e.g. the logical operators).

Custom Jackson serializer for a generic tree

Say I have a parametrized tree implemented in Java as follows:
public class Tree<E> {
private static class Node {
E element;
List<Node> children.
}
Node root;
//... You get the idea.
}
The idea here is that the implementation above is only concerned with the topology of the tree, but does not know anything about the elements that will be stored in the tree by an instantiation.
Now, say I want my tree elements to be geographies. The reason they are organized in trees is because continents contain countries, countries contain states or a provinces, and so on. For simplicity, a geography has a name and a type:
public class GeoElement { String name; String type; }
So that, finally, the geo hierarchy looks like so:
public class Geography extends Tree<GeoElement> {}
Now to Jackson serialization. Assuming the Jackson serializer can see the fields, the direct serialization of this implementation will look like this:
{
"root": {
"element": {
"name":"Latin America",
"type":"Continent"
}
"children": [
{
"element": {
"name":"Brazil",
"type":"Country"
},
"children": [
// ... A list of states in Brazil
]
},
{
"element": {
"name":"Argentina",
"type":"Country"
},
"children": [
// ... A list of states in Argentina
]
}
]
}
This JSON rendering is no good because it contains the unnecessary artifacts from the Tree and Node classes, i.e. "root" and "element". What I need instead is this:
{
"name":"Latin America",
"type":"Continent"
"children": [
{
"name":"Brazil",
"type":"Country"
"children": [
// ... A list of states in Brazil
]
},
{
"name":"Argentina",
"type":"Country"
"children": [
// ... A list of states in Argentina
]
}
]
}
Any help is most appreciated. -Igor.

What you need is #JsonUnwrapped.
Annotation used to indicate that a property should be serialized "unwrapped"; that is, if it would be serialized as JSON Object, its properties are instead included as properties of its containing Object
Add this annotation to the root field of Tree & element field of Node classes as follows:
public class Tree<E> {
private static class Node {
#JsonUnwrapped
E element;
List<Node> children.
}
#JsonUnwrapped
Node root;
//... You get the idea.
}
And it will give you your desired output:
{
"name": "Latin America",
"type": "Continent",
"children": [{
"name": "Brazil",
"type": "Country",
"children": []
}, {
"name": "Argentina",
"type": "Country",
"children": []
}]
}

Perhaps use #JsonValue like so:
public class Tree<E> {
#JsonValue
Node root;
}
if all you need is to just "unwrap" your tree?

Your best bet will be to build and register a custom serializer for your objects.
Define your serializer:
public class NodeSerializer extends StdSerializer<Node> {
Then on your Node class:
#JsonSerialize(using = NodeSerializer.class)
public class Node {
}
And inside of the NodeSerializer
#Override
public void serialize(
Node node, JsonGenerator jgen, SerializerProvider provider)
throws IOException, JsonProcessingException {
jgen.writeStartObject();
jgen.writeStringField("name", node.element.name);
jgen.writeStringField("type", node.element.type);
//Process children
serializeFields(node, jgen, provider);
jgen.writeEndObject();
}
This general framework will let you control how the elements get serialized. You may need to #JsonIgnore the element object inside of the Node as well since your custom serializer is taking care of pushing that info into the resulting JSON. There is a lot online about custom serializers and overriding default JSON export.
You can get rid of root in a similar way with a serializer for the Tree implementation.
If you don't want to register the serializer on the class you can also do it on a one at a time basis using the ObjectMapper:
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(Node.class, new NodeSerializer());
mapper.registerModule(module);
String serialized = mapper.writeValueAsString(tree);
The annotation approach will apply globally. This approach allows some control of how/where your custom serializer is used.

For removing the element type, one possibility would be to change your structure so that the name and the type will be directly included in each node:
public class TreeGeo {
private static class Node {
String name;
String type;
List<Node> children.
}
Node root;
}
For removing the root type, I don't know. I suppose that you could extract a sub-object from the jsonObject but I don't know much about Jackson. However, you could give it a better name like world or manipulate the resulting string to remove it manually with some string manipulations.