We Keep Coding

Jackson: deserialize JSON extract deep attribute into parent class

I have some trouble wording my title, so if my question should be re-worded, I'd be happy to repost this question for clarification. :)
Problem: I have this JSON structure
{
"name": "Bob",
"attributes": {
"evaluation": {
"stats": [
{
"testDate": "2020-02-04",
"score": 50
},
{
"testDate": "2020-04-01",
"score": 90
},
{
"testDate": "2020-05-10",
"score": 85
}
],
"survey": {...}
},
"interests": {...},
"personality": [...],
"someRandomUnknownField": {...}
}
}
attributes is any random number of fields except for evaluation.stats that we want to extract out. I want to be able to deserialize into the following classes:
public class Person {
String name;
Map<String, Object> attributes;
List<Stat> stats;
}
public class Stat {
LocalDate date;
int score;
}
When I serialize it back to JSON, I should expect something like this:
{
"name": "Bob",
"attributes" : {
"evaluation": {
"survey": {...}
},
"interests" : {...},
"personality": {...},
"someRandomUnknownField": {...}
},
"stats": [
{
"testDate": "2020-02-04",
"score": 50
},
{
"testDate": "2020-04-01",
"score": 90
},
{
"testDate": "2020-05-10",
"score": 85
}
]
}
I could technically map the whole Person class to its own custom deserializer, but I want to leverage the built-in Jackson deserializers and annotations as much as possible. It's also imperative that stats is extracted (i.e., stats shouldn't also exist under attributes). I'm having trouble finding a simple and maintainable serialization/deserialization scheme. Any help would be appreciate!

I'm not sure if this meets your criterion for a simple and maintainable serialization/deserialization scheme, but you can manipulate the JSON tree to transform your starting JSON into the structure you need:
Assuming I start with a string containing your initial JSON:
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
JsonNode root = mapper.readTree(inputJson);
// copy the "stats" node to the root of the JSON:
ArrayNode statsNode = (ArrayNode) root.path("attributes").path("evaluation").path("stats");
((ObjectNode) root).set("stats", statsNode);
// delete the original "stats" node:
ObjectNode evalNode = (ObjectNode) root.path("attributes").path("evaluation");
evalNode.remove("stats");
This now gives you the JSON you need to deserialize to your Person class:
Person person = mapper.treeToValue(root, Person.class);
When you serialize the Person object you get the following JSON output:
{
"name" : "Bob",
"attributes" : {
"evaluation" : {
"survey" : { }
},
"interests" : { },
"personality" : [ ],
"someRandomUnknownField" : { }
},
"stats" : [ {
"score" : 50,
"testDate" : "2020-02-04"
}, {
"score" : 90,
"testDate" : "2020-04-01"
}, {
"score" : 85,
"testDate" : "2020-05-10"
} ]
}
Just to note, to get this to work, you need the java.time module:
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>2.11.3</version>
</dependency>
And you saw how this was registered in the above code:
mapper.registerModule(new JavaTimeModule());
I also annotated the LocalDate field in the Stat class, as follows:
#JsonProperty("testDate")
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
private LocalDate date;
Very minor note: In your starting JSON (in the question) you showed this:
"personality": [...],
But in your expected final JSON you had this:
"personality": {...},
I assumed this was probably a typo, and it should be an array, not an object, in both cases.

Spring hateoas jaxrslinkbuilder adds null fields

I am trying to add HATEOAS links to my Controller results.
Link creation:
Link link = JaxRsLinkBuilder.linkTo(MainRestService.class)
.slash(this::getVersion)
.slash(LinkMappingConstants.TAXPAYERS)
.slash(taxPayerId)
.slash(LinkMappingConstants.ESTIMATIONS)
.slash(estimationId)
.withSelfRel();
The link itself is created fine, but there are superfluous entries in the resulting JSON:
"links": [
{
"rel": "self",
"href": "http://localhost:8080/blablabla",
"hreflang": null,
"media": null,
"title": null,
"type": null,
"deprecation": null,
"template": {
"variables": [
],
"variableNames": [
]
}
}
]
How can i get this format? (without use property spring.jackson.default-property-inclusion=NON_NULL)
"links": [
{
"rel": "self",
"href": "http://localhost:8080/blablabla"
}
]
Thanks.

As you mention, if you want NON_NULL property inclusion on the whole JSON and not just links you can use spring.jackson.default-property-inclusion=NON_NULL. This won't fix the empty template fields however.
If you want to have NON_NULL property inclusion on just the Link object using Jackson for serialization you can achieve this by using a Jackson MixIn for the Link object with the #JsonInclude(Include.NON_NULL) annotation.
As an example:
#JsonInclude(Include.NON_NULL)
abstract class LinkMixIn {
}
mapper.addMixIn(Link.class, LinkMixIn.class);
To hide the template fields you can either add #JsonIgnore if you never want the template section to be serialized, or try NON_DEFAULT property inclusion on the above answer which creates a new instance of the object and compares it to what is to be serialized to determine if it should be included.
E.g. something like the following would not serialize the result of getTemplate at all
#JsonInclude(Include.NON_NULL)
abstract class LinkMixIn {
#JsonIgnore abstract Template getTemplate();
}

How to tell a JSON schema is compatible with another in Java?

For example, I have a JSON schema looks as following:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"billing_address": { "$ref": "#/definitions/address" },
"shipping_address": { "$ref": "#/definitions/address" }
}
"definitions": {
"address": {
"type": "object",
"properties": {
"street_address": { "type": "string" },
"city": { "type": "string" },
"state": { "type": "string" }
},
"required": ["street_address", "city", "state"]
}
}
}
This schema indicate an object with two vairable billing_address and shipping_address, both of them are of type address, which contains three properties: street_address, city and state.
Now I got another "larger" schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"billing_address": { "$ref": "#/definitions/address" },
"shipping_address": { "$ref": "#/definitions/address" },
"new_address": { "$ref": "#/definitions/address" }
}
"definitions": {
"address": {
"type": "object",
"properties": {
"street_address": { "type": "string" },
"city": { "type": "string" },
"state": { "type": "string" },
"zip_code": { "type": "string" }
},
"required": ["street_address", "city", "state"]
}
}
}
As you can see, I added a new property new_address into the schema, and in address there is a new property called zip_code, which is not a required property.
So if I created an object from the old JSON schema, it should also be available for the new JSON schema. In this case, we will call the new schema is compatible with the old one. (In another word, the new schema is extension of the old one, but no modification.)
The question is how can I judge if a schema is compatible with another in Java? Complicated case should also be taken care, for example "minimum" property for a number field.

Just test it. In my current project, I am writing following contract tests:
1) having Java domain object, I serialize it to JSON and compare it to reference JSON data. I use https://github.com/skyscreamer/JSONassert for comparing two JSON strings.
For reference JSON data, you need to use 'smaller schema' object.
2) having sample JSON data, I deserialize it to my domain object, and verify if deserialization was succesfull. I compare deserialization result with model object. For sample JSON data, you shoud use your 'larger schema' object.
This test verifies if 'larger schema' JSON data is backward compatible with your 'smaller schema' domain.
I write those test at each level of my domain model -one for top-level object, and another one for each non-trivial nested object. That requires more test code and more JSON sample data, but gives much better confidence. If something fails, error messages will be fine-tuned, you will know exactly what level of hierarchy is broken (JSONAssert error messages may have many errors and be non trivial to read for deeply nested object hierarchies). So it's a trade-off between
* time spend to maintain test code and data
* quality of error messages
Such tests are fast- they need just JSON serialization/deserialization.
https://github.com/spring-cloud/spring-cloud-contract will help you writing contract test for REST APIs, messaging, etc- but for simple cases procedure I given above may be good enough

Is it possible to use Spring's annotations to define Completion Suggester for a mapping in Elasticsearch?

I currently have the following POJO.
#Document(indexName="ws",type="vid")
public class Vid {
#Id
private String id;
#Field(type=FieldType.String, index=FieldIndex.not_analyzed)
private List<String> tags;
}
A JSON that represents this POJO is as follows.
{
"id" : "someId",
"tags" : [ "one", "two", "three" ]
}
What I want is to define the mapping for the tags field so that I can use the values in an auto-complete search box. This is supported by Elasticsearch's Completion Suggester. The documentation at https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html seem to suggest to me that I have to set up the mapping as follows.
{
"vid": {
"properties": {
"id": {
"type": "string"
},
"tags": {
"type": "completion",
"index_analyzer": "simple",
"search_analyzer": "simple",
"payloads": true
}
}
}
}
However, that would mean that I would have to revise my POJO and JSON representation.
{
"id": "someId",
"tags": {
"input": [ "one", "two", "three" ]
}
}
I found another good page talking about Completions Suggesters here http://blog.qbox.io/quick-and-dirty-autocomplete-with-elasticsearch-completion-suggest. However, that page seem to suggest redundancy with the tags.
{
"id": "someId",
"tags": [ "one", "two", "three" ],
"tags_suggest": {
"input": [ "one", "two", "three" ]
}
}
Lastly, I found this javadoc page from spring-data-elasticsearch at http://docs.spring.io/spring-data/elasticsearch/docs/current/api/index.html?org/springframework/data/elasticsearch/core/completion/Completion.html. I am sure this class has something to do with Completion Suggesters but I don't know how to use it.
Is there any way I can just use Spring annotations to define the Elasticsearch mapping for Completion Suggester?

Absolutely yes..
you can configure your entity like this:
...
import org.springframework.data.elasticsearch.core.completion.Completion;
...
#Document(indexName = "test-completion-index", type = "annotated-completion-type", indexStoreType = "memory", shards = 1, replicas = 0, refreshInterval = "-1")
public class YoutEntity {
#Id
private String id;
private String name;
#CompletionField(payloads = true, maxInputLength = 100)
private Completion suggest;
...
}
Check this link for example.

I am not experienced with that, but maybe this annotation can be helpful for you:
Link to Spring Data Elasticsearch documentation

Converting a specific JSON to Java Object

I am able to convert simple Json into java objects using Google Gson. The issue is I am not able to convert this specific type of response to a java object. I think this is due to the fact that I am not sure how to actually model the java classes for it. The following is the Json:
{
"List": {
"Items": [
{
"comment": "comment",
"create": "random",
"ID": 21341234,
"URL": "www.asdf.com"
},
{
"comment": "casdf2",
"create": "rafsdfom2",
"ID": 1234,
"Url": "www.asdfd.com"
}
]
},
"related": {
"Book": "ISBN",
"ID": "id"
}}

I haven't worked with Gson specifically, but I have worked with JSON / Java conversion in Jersey and JAXB.
I would say that the easiest way to translate the JSON text is to map every {..} to a Class and [..] to a List.
For example:
Class Data {
HistoryListClass HistoryList;
Relation related;
}
Class HistoryListClass {
List<History> history;
}
Class History {
String comment;
String create;
int ID;
String ttURL;
}
Class Relation {
String book;
String ID;
}
See also:
Converting JSON to Java
The "HistoryList" element in the JSON code should probably be written "historyList", and just looking at the code given, I suggest you change it to contatain the list of "History" objects directly.