We Keep Coding

How to Parse untyped, nested JSON with Polymorphism?

I am using Feign to hit external APIs, and then to process the externally generated JSON (aka, the response data cannot be modified in any way), and I am trying to bundle these together into an extensible super type. At this point, I am not even sure if what I am trying to do is possible with Jackson / Feign. If it would be much easier to abandon (or heavily restructure) the polymorphism, I think I am also ready to give up on it and just create a bunch of sub classes.
Here are my two main questions, with more context below.
Should I just separate the easily deduced types from the complex types, and have a little more duplicated boiler plate?
How can I create a custom deserializer for the list object I linked? Ideally I would like to have some way to populate the more boiler plate fields less manually -- as an example, it would be great if I could call default deserializers inside it, which would rely more on the standard annotations in other objects.
Ideally, I would like one class, like this:
public final class BillApiResponse {
#Valid
#JsonProperty("response_status")
private boolean responseStatus;
#Valid
#JsonProperty("response_message")
private String responseMessage;
#JsonProperty("response_data")
private BillApiResponseData responseData;
//getters and setters, etc.
}
and then I would to have Jackson automatically map the simpler objects in whatever way is easiest (LoginResponse, LoginError), while I would try to implement a custom handler for the more complex objects (UpdateObject, ListOfObjects).
So, something like this:
#JsonTypeInfo(use = Id.DEDUCTION)
#JsonSubTypes({
#Type(value = BillLoginSuccess.class),
#Type(value = BillErrorResponse.class),
//#Type(value = BillResponseObject[].class) <--- This breaks things when added
})
// #JsonTypeResolver(value = BillResponseTypeResolver.class) <--- Open to using one of
// these if I can figure out how
// #JsonDeserialize(using = BillResponseDeserializer.class) <--- Also open to using a
// custom deserializer, but I would like to keep it only for certain parts
public interface BillApiResponseData {}
Here is a link to the API specification I am trying to hit:
Get a List of Objects
This returns an untyped array of untyped objects. Jackson does not seem to like that the array is untyped, and stops parsing everything there. Once inside, we would have to grab the type from a property.
{
"response_status" : 0,
"response_message" : "Success",
"response_data" : [{
"entity" : "SentPay",
"id" : "stp01AUXGYKCBGFMaqlc"
// More fields
} // More values]
}
Login
This returns a totally new object. Generally not having issues handling this one (until I add support for the above list, and then all of the parsing breaks down as Jackson throws errors).
Update Object
This returns an untyped object. Once again, we would have to go inside and look at the property.
I have tried a number of things, but generally I was not successful (hence I am here!).
These include:
Trying to hook into the lifecycle and take over if I detect an array object. I believe this fails because Jackson throws an error when it sees the array does not have a type associated with it.
SimpleModule customDeserializerModule = new SimpleModule()
.setDeserializerModifier(new BeanDeserializerModifier() {
#Override
public JsonDeserializer<?> modifyDeserializer(
DeserializationConfig config,
BeanDescription beanDesc,
JsonDeserializer<?> defaultDeserializer) {
if (beanDesc.getBeanClass().isArray()) {
return new BillResponseDeserializer(defaultDeserializer);
} else {
return defaultDeserializer;
}
}
});
Custom Deserializers. The issue I have is that it seems to want to route ALL of my deserialization calls into the custom one, and I don't want to have to handle the simpler items, which can be deduced.
TypeIdResolvers / TypeResolvers. Frankly these are confusing me a little bit, and I cannot find a good example online to try out.

FasterJackson - fails to convert a key (String of JSON) within a Map into an Object that is using AnyGetter/AnySetter annotations

I am trying to deserialize an Object from a Map(String, Object) via FasterJackson's ObjectMapper.convertValue(map, ComplexRecord.class) method.
My destination object contains many other objects within it. The only problematic object is the embedded "AnyObject" object that uses FasterJackson's (Jackson's) AnyGetter/AnySetter methods.
The AnyObject instance works for every other use case with FasterJackson except, at the moment, for this one where it involves a more "ComplexRecord" deserialization.
This is what it looks like:
#Data
public class ComplexRecord {
private String id;
private AnyObject data;
private String status;
private Instant createdDateTime;
}
#Data
public class AnyObject {
#Getter(AccessLevel.NONE)
#Setter(AccessLevel.NONE)
private final Map<String, Object> data;
public AnyObject() {
this(new LinkedHashMap<>());
}
public AnyObject(Map<String, Object> sourceData) {
this.data = new LinkedHashMap<>(sourceData);
}
#JsonAnySetter
public void setData(String name, Object value) {
this.data.put(name, value);
}
#JsonAnyGetter
public Map<String, Object> getData() {
return Collections.unmodifiableMap(this.data);
}
}
When I try to use ObjectMapper.convertValue(map, ComplexRecord.class) it fails to deserialize the "data" field due to this error:
Cannot construct instance of `AnyObject`
(although at least one Creator exists): no String-argument
constructor/factory method to deserialize from String value
('{"id":"123","v":"anything"}')
at [Source: UNKNOWN; line: -1, column: -1]
I'd like to find a very clean workaround for this. This issue seems to stem from the fact that I am using the ObjectMapper.convertValue method from a complex source of Map where the key "data" is originally available as a String. If I perform a similar operation but with ObjectMapper.readValue() instead of ObjectMapper.convertValue() then the deserialization into AnyObject works just fine.
Since I do not have the luxury of changing the source object of Map into something that would work with ObjectMapper.readValue() method I may be left with only a few choices.
One option I found is using FasterJackson's Custom Deserializer. The only catch I see is that there is no clear way to access the internal ObjectMapper that is provided to the Deserializer. When I debug the deserializer, I do see that JsonParser.getCodec() is the ObjectMapper, but even when trying to do a readValue within the the custom Deserializer the deserialization fails with the same error. i.e.
AnyObject value = jsonParser.getCodec().readValue(p, AnyObject.class);
However following set of calls work just fine:
String stringValue = jsonParser.getCodec().readValue(p, String.class);
AnyObject anyObject = objMapper.readValue(stringValue, AnyObject.class);
Other than this being a 2 step process to deserialize, rather than a 1 step process; the only other issue is that I do not have a clean way to use the "objMapper" (ObjectMapper) instance I am referring to above without casting the codec into an ObjectMapper instance.
This seems like a hack to me but I'd like to get your thoughts and also see if there any other friendlier solution available.
There are a few more thoughts / options but I'd like to get an unbiased opinion on this to see if there is a much simpler way of handling this type of "complex" conversion.
The most ideal outcome is to force coercion of the String into my desired type -- AnyObject. Via annotation or some very simple strategy.
But if I do have to deal with this via a Custom Deserializer, my preference would be to be able to get the handle to the in-process ObjectMapper in a clean way.
Thoughts?

Tried a few things and landed on the following by implementing a custom FasterJackson deserializer:
JsonNode node = jsonParser.readValueAsTree();
AnyObject val = jsonParser.getCodec().getFactory().createParser(node.asText().getBytes()).readValueAs(AnyObject.class);
I am still open to solutions that may be simpler or lean towards better performance with less intermediate garbage generation during parsing.

How to customize ModelMapper

I want to use ModelMapper to convert entity to DTO and back. Mostly it works, but how do I customize it. It has has so many options that it's hard to figure out where to start. What's best practice?
I'll answer it myself below, but if another answer is better I'll accept it.

First here are some links
modelmapper getting started
api doc
blog post
random code examples
My impression of mm is that it is very well engineered. The code is solid and a pleasure to read. However, the documentation is very terse, with very few examples. Also the api is confusing because there seems to be 10 ways to do anything, and no indication of why you’d do it one way or another.
There are two alternatives: Dozer is the most popular, and Orika gets good reviews for ease of use.
Assuming you still want to use mm, here’s what I’ve learned about it.
The main class, ModelMapper, should be a singleton in your app. For me, that meant a #Bean using Spring. It works out of the box for simple cases. For example, suppose you have two classes:
class DogData
{
private String name;
private int mass;
}
class DogInfo
{
private String name;
private boolean large;
}
with appropriate getters/setters. You can do this:
ModelMapper mm = new ModelMapper();
DogData dd = new DogData();
dd.setName("fido");
dd.setMass(70);
DogInfo di = mm.map(dd, DogInfo.class);
and the "name" will be copied from dd to di.
There are many ways to customize mm, but first you need to understand how it works.
The mm object contains a TypeMap for each ordered pair of types, such as <DogInfo, DogData> and <DogData, DogInfo> would be two TypeMaps.
Each TypeMap contains a PropertyMap with a list of mappings. So in the example the mm will automatically create a TypeMap<DogData, DogInfo> that contains a PropertyMap that has a single mapping.
We can write this
TypeMap<DogData, DogInfo> tm = mm.getTypeMap(DogData.class, DogInfo.class);
List<Mapping> list = tm.getMappings();
for (Mapping m : list)
{
System.out.println(m);
}
and it will output
PropertyMapping[DogData.name -> DogInfo.name]
When you call mm.map() this is what it does,
see if the TypeMap exists yet, if not create the TypeMap for the <S,
D> source/destination types
call the TypeMap Condition, if it returns FALSE, do nothing and STOP
call the TypeMap Provider to construct a new destination object if necessary
call the TypeMap PreConverter if it has one
do one of the following:
if the TypeMap has a custom Converter, call it
or, generate a PropertyMap (based on Configuration flags plus any custom mappings that were added), and use it
(Note: the TypeMap also has optional custom Pre/PostPropertyConverters that I think will run at this point before and after each mapping.)
call the TypeMap PostConverter if it has one
Caveat: This flowchart is sort of documented but I had to guess a lot, so it might not be all correct!
You can customize every single step of this process. But the two most common are
step 5a. – write custom TypeMap Converter, or
step 5b. – write custom Property Mapping.
Here is a sample of a custom TypeMap Converter:
Converter<DogData, DogInfo> myConverter = new Converter<DogData, DogInfo>()
{
public DogInfo convert(MappingContext<DogData, DogInfo> context)
{
DogData s = context.getSource();
DogInfo d = context.getDestination();
d.setName(s.getName());
d.setLarge(s.getMass() > 25);
return d;
}
};
mm.addConverter(myConverter);
Note the converter is one-way. You have to write another if you want to customize DogInfo to DogData.
Here is a sample of a custom PropertyMap:
Converter<Integer, Boolean> convertMassToLarge = new Converter<Integer, Boolean>()
{
public Boolean convert(MappingContext<Integer, Boolean> context)
{
// If the dog weighs more than 25, then it must be large
return context.getSource() > 25;
}
};
PropertyMap<DogData, DogInfo> mymap = new PropertyMap<DogData, DogInfo>()
{
protected void configure()
{
// Note: this is not normal code. It is "EDSL" so don't get confused
map(source.getName()).setName(null);
using(convertMassToLarge).map(source.getMass()).setLarge(false);
}
};
mm.addMappings(mymap);
The pm.configure function is really funky. It’s not actual code. It is dummy EDSL code that gets interpreted somehow. For instance the parameter to the setter is not relevant, it is just a placeholder. You can do lots of stuff in here, such as
when(condition).map(getter).setter
when(condition).skip().setter – safely ignore field.
using(converter).map(getter).setter – custom field converter
with(provider).map(getter).setter – custom field constructor
Note the custom mappings are added to the default mappings, so you do not need, for example, to specify
map(source.getName()).setName(null);
in your custom PropertyMap.configure().
In this example, I had to write a Converter to map Integer to Boolean. In most cases this will not be necessary because mm will automatically convert Integer to String, etc.
I'm told you can also create mappings using Java 8 lambda expressions. I tried, but I could not figure it out.
Final Recommendations and Best Practice
By default mm uses MatchingStrategies.STANDARD which is dangerous. It can easily choose the wrong mapping and cause strange, hard to find bugs. And what if next year someone else adds a new column to the database? So don't do it. Make sure you use STRICT mode:
mm.getConfiguration().setMatchingStrategy(MatchingStrategies.STRICT);
Always write unit tests and ensure that all mappings are validated.
DogInfo di = mm.map(dd, DogInfo.class);
mm.validate(); // make sure nothing in the destination is accidentally skipped
Fix any validation failures with mm.addMappings() as shown above.
Put all your mappings in a central place, where the mm singleton is created.

I faced a problem while mapping with ModelMapper. Not only properties but also My source and destination type were different. I solved this problem by doing this ->
if the source and destination type are different. For example,
#Entity
class Student {
private Long id;
#OneToOne
#JoinColumn(name = "laptop_id")
private Laptop laptop;
}
And Dto ->
class StudentDto {
private Long id;
private LaptopDto laptopDto;
}
Here, the source and destination types are different. So, if your MatchingStrategies are STRICT, you won't able to map between these two different types.
Now to solve this, Just simply put this below code in the constructor of your controller class or any class where you want to use ModelMapper->
private ModelMapper modelMapper;
public StudentController(ModelMapper modelMapper) {
this.modelMapper = modelMapper;
this.modelMapper.typeMap(Student.class, StudentDto.class).addMapping(Student::getLaptop, StudentDto::setLaptopDto);
}
That's it. Now you can use ModelMapper.map(source, destination) easily. It will map automatically
modelMapper.map(student, studentDto);

I've been using it from last 6 months, I'm going to explain some of my thoughts about that:
First of all, it is recommended to use it as an unique instance (singleton, spring bean,...), that's explained in the manual, and I think all agree with that.
ModelMapper is a great mapping library and wide flexible. Due to its flexibility, there are many ways to get the same result, and that's why it should be in the manual of best practices of when to use one or other way to do the same thing.
Starting with ModelMapper is a little bit difficult, it has a very tight learning curve and sometimes it is not easy to understand the best ways to do something, or how to do some other thing. So, to start it is required to read and understand the manual precisely.
You can configure your mapping as you want using the next settings:
Access level
Field matching
Naming convention
Name transformer
Name tokenizer
Matching strategy
The default configuration is simply the best (http://modelmapper.org/user-manual/configuration/), but if you want to customise it you are able to do it.
Just one thing related to the Matching Strategy configuration, I think this is the most important configuration and is need to be careful with it. I would use the Strict or Standard but never the Loose, why?
Due Loose is the most flexible and intelligent mapper it could be map some properties you can not expect. So, definitively, be careful with it. I think is better to create your own PropertyMap and use Converters if it is needed instead of configuring it as Loose.
Otherwise, it is important to validate all property matches, you verify all it works, and with ModelMapper it's more need due with intelligent mapping it is done via reflection so you will not have the compiler help, it will continue compiling but the mapping will fail without realising it. That's one of the things I least like, but it needs to avoid boilerplate and manual mapping.
Finally, if you are sure to use ModelMapper in your project you should use it using the way it proposes, don't mix it with manual mappings (for example), just use ModelMapper, if you don't know how to do something be sure is possible (investigate,...). Sometimes is hard to do it with model mapper (I also don't like it) as doing by hand but is the price you should pay to avoid boilerplate mappings in other POJOs.

import org.modelmapper.ModelMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
#Service
public class EntityDtoConversionUtil {
#Autowired
private ModelMapper modelMapper;
public Object convert(Object object,Class<?> type) {
Object MapperObject=modelMapper.map(object, type);
return MapperObject;
}
}

Here is how you can make a customize Conversion Class and can then autowire it where you would like to convert the object to dto and vice versa.
#Component
public class ConversionUtil {
#Bean
public ModelMapper modelMapper() {
return new ModelMapper();
}
public <T,D> D mapItem(T item,Class<D> cl){
return modelMapper().map(item,cl);
}
public <T,D> List<D> map(List<T> list, Class<D> cl){
return list.stream()
.map(item -> modelMapper().map(item, cl))
.collect(Collectors.toList());
}
}

Order matters with class metadata in Genson - Is there a work-around?

I'm using Genson to serialize + deserialize json in my android app into polymorphic objects. The JSON is coming from a variety of sources though and I can't guarantee that the #class metadata will be the first line item in the json. Walking through the Genson code and writing test cases it looks like the #class metadata has to be the first entry in the dictionary.
Has anyone had luck working around this constraint? Is it time to switch to something else, and if so, what?
public class Message {
Payload payload;
// getters & setters
}
public abstract class Payload {
//
}
public class Notification1 extends Payload {
String text;
// getters & setters
}
public class Notification2 extends Payload {
String otherText
// getters & setters
}
String correctOrder = {"#class":"Message","payload":{"#class":"Notification1","text":"Text"}}
String modifiedOrder = {"#class":"Message","payload":{"text":"Text", "#class":"Notification1"}}
Genson g = Genson.Builder()
.addAlias("Notification1", Notification1.class)
.addAlias("Notification2", Notification2.class)
.useRuntimeType(true)
.useClassMetadata(true)
.useMetadata(true)
.useFields(false)
.useIndentation(false)
.create();
g.deserialize(correctOrder, Message.class) // This works
g.deserialize(modifiedOrder, Message.class) // This barfs with the error: com.owlike.genson.JsonBindingException: Could not deserialize to type class com.ol.communication.messages.Message

Indeed the order matters. This was choosed on purpose, see the remarks in the user guide.
If we allow the #class property anywhere in the json object, then we will have to first deserialize all the json object (and its sub properties obj/arr etc) to an intermediary data structure and then to the correct type.
This would incur additional memory overhead and less speed but greater flexibility, true.
A solution would be to mark classes that are polymorphic (annotation/config in the builder), for whom Genson would search/produce the #class property in the stream. This would allow to have this overhead only for the polymorphic objects in the stream.
At the moment it is not implemented, but I opened an issue. It will come in a future release.
Outside of the technical aspects, I don't think you should have polymorphic logic (or any other fancy stuff) when you are dealing with multiple external API. I mean this kind of features is library specific, so if you don't use the same tool on both sides you can run into troubles. Usually people have a layer that will be used to communicate with the APIs and map the data to YOUR model. If you don't own the code on both ends, I think this would be a good solution on the long term.

Use class name as root key for JSON Jackson serialization

Suppose I have a pojo:
import org.codehaus.jackson.map.*;
public class MyPojo {
int id;
public int getId()
{ return this.id; }
public void setId(int id)
{ this.id = id; }
public static void main(String[] args) throws Exception {
MyPojo mp = new MyPojo();
mp.setId(4);
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationConfig.Feature.WRAP_ROOT_VALUE, true);
System.out.println(mapper.getSerializationConfig().isEnabled(SerializationConfig.Feature.WRAP_ROOT_VALUE));
System.out.println(mapper.writeValueAsString(mp));
}
}
When I serialize using the Jackson ObjectMapper, I just get
true
{"id":4}
but I want
true
{"MyPojo":{"id":4}}
I've searched all over, Jacksons documentation is really unorganized and mostly out of date.

By adding the jackson annotation #JsonTypeInfo in class level you can have the expected output. i just added no-changes in your class.
package com.test.jackson;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.SerializationConfig;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeInfo.As;
import com.fasterxml.jackson.annotation.JsonTypeInfo.Id;
#JsonTypeInfo(include=As.WRAPPER_OBJECT, use=Id.NAME)
public class MyPojo {
// Remain same as you have
}
output:
{
"MyPojo": {
"id": 4
}
}

I'm not using jackson, but searching I found this configuration that seems to be what you want: WRAP_ROOT_VALUE
Feature that can be enabled to make root value (usually JSON Object but can be any type) wrapped within a single property JSON object, where key as the "root name", as determined by annotation introspector (esp. for JAXB that uses #XmlRootElement.name) or fallback (non-qualified class name). Feature is mostly intended for JAXB compatibility.
Default setting is false, meaning root
value is not wrapped.
So that you can configure mapper:
objectMapper.configure(SerializationConfig.Feature.WRAP_ROOT_VALUE, true);
I hope it helps you...

Below is a way to achieve this
Map<String, MyPojo> singletonMap = Collections.singletonMap("mypojo", mp);
System.out.println(mapper.writeValueAsString(singletonMap));
Output
{ "mypojo" : { "id" : 4}}
Here the advantage is that we can give our on name for the root key of json object. By the above code, mypojo will be the root key. This approach will be most useful when we use java script template like Mustache.js for iteration of json objects

To achieve this you need to use the JsonTypeInfo annotation on your class and in particular WRAPPER_OBJECT
#JsonTypeName("foo")
#JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT ,use = JsonTypeInfo.Id.NAME)
public class Bar(){
)

There is also a nice annotation for this:
#JsonRootName(value = "my_pojo")
public class MyPojo{
...
}
will generate:
{
"my_pojo" : {...}
}

How about simplest possible solution; just use a wrapper class like:
class Wrapper {
public MyPojo MyPojo;
}
and wrapping/unwrapping in your code?
Beyond this, it would help to know WHY you would like additional json object entry like this? I know this is done by libs that emulate json via xml api (because of impedance between xml and json, due to conversion from xml to json), but for pure json solutions it is usually not needed.
Is it to allow you do figure out what actual type is?
If so, perhaps you could consider enabled polymorphic type information, to let Jackson handle it automatically? (see 1.5 release notes, entry for PTH, for details).

there is another way i used and that worked for me.
I am working with a third party jar, so i have no control for annotations.
So i had to write through bit of hack.
Override: org.codehaus.jackson.map.ser.BeanSerializerFactory.findBeanProperties(SerializationConfig, BasicBeanDescription)
Add your property as below
List<BeanPropertyWriter> props = super.findBeanProperties(config, beanDesc);
BeanPropertyWriter bpw = null;
try {
Class cc = beanDesc.getType().getRawClass();
Method m = cc.getMethod("getClass", null);
bpw = new BeanPropertyWriter("$className", null, null, m, null,true, null);
} catch (SecurityException e) {
// TODO
} catch (NoSuchMethodException e) {
// TODO
}
props.add(bpw);
return props;
This way i get more control and can do other kind of filters too.

#JsonTypeInfo(include=As.WRAPPER_OBJECT, use=Id.NAME)
This annotation works perfectly, as suggested by Arun Prakash. I was trying to get json in this form:
{"Rowset":{"ROW":{"receiptno":"881604199388936","status":"SUCCESS"}}}
but getting like this:
{"ROW":{"receiptno":"881604199388936","status":"SUCCESS"}}
Now that annotation resolved my problem.

I would be interested in hearing the OP's solution for this. I'm having similar issues where my RESTful web service is serializing objects as either XML or JSON for clients. The Javascript clients need to know the wrapping type so that can parse it. Coupling the type to a URI pattern is not an option.
Thanks.
Edit: I noticed that Spring MappingJacksonJsonMarshaller adds the wrapping class when marshalling, so I stepped through the code in debug and noticed that Spring passes in a HashMap with a single key-value pair such that the key is the wrapping name and the value is the object. So, I extended JacksonJaxbJsonProvider, override the writeTo() method and added the following:
HashMap<String, Object> map = new HashMap<String, Object>();
map.put(value.getClass().getSimpleName(), value);
super.writeTo(map, type, genericType, annotations, mediaType, httpHeaders,entityStream);
It's a bit of a hack, but it works nicely.

use withRootName.
objectMapper.writer().withRootName(MyPojo.class.getName());

I have found through experience that it is a good idea for all JSON to include both the backend type (as a string) and the component type used to render it in the front end (if using something like angular or Vue).
The justification for doing this is so that you can process various types with a single set of code.
In vue, for example, having the name of the UI component in the data allows you, among other things, to have a screen rendering a list of children of different types using only a single tag in the parent template.
<component :is="child.componentType"/>.
For backend systems and web services - I prefer to use a single web service processor class that provides logging, auditing and exception handling for all web services by looking up the appropriate processor class based on the incoming payload. That makes the implementation of all my web services look exactly the same (about 3 lines of code), and I get detailed event logging through the lifecycle of the call without writing any per service code to do so.
Having the type wrapping the JSON makes it self documenting. If all you see are the properties, you have no idea what you are looking at until you find the corresponding end point.
If you want to write data driven software, being able to identify what you are processing is a basic requirement.