I'm trying to write some code that will deserialize JSON into somebody elses class. That is, I don't own the target class so I can't annotate it.
In particular, this class has some helper methods that are complicating the deserialization process. Something like this:
class Result {
private List<String> ids;
public List<String> getIds() {
return ids;
}
public void setIds(List<String> ids) {
this.ids = ids;
}
// Helpers
public String getId() {
return this.ids.isEmpty() ? null : this.ids.get(0);
}
public void setId(String id) {
this.ids = List.of(id);
}
}
When serialized, we get both ids and id come through as fields:
{
"ids": ["1", "2", "3"],
"id": "1"
}
And then when deserializing this JSON, Jackson is calling both setters - reasonably enough - and thus the object is wrong. The end result is that the ids field is set to ["1"] and not ["1", "2", "3"] as it should be.
So what I want to be able to do is fix this. And I'm thinking that the easiest/safest/best way is to be able to modify the JSON AST somewhere in the deserializing process. Specifically by just removing the "id" field from the JSON so that it doesn't get seen by the standard deserializer. (I know this works by doing it manually with string manipulation, but that's awful)
I could write a full deserializer for all the fields, but then I'm beholden to maintaining it if any new fields are added in the future, when all I actually want to do is ignore one single field and have everything else processed as normal. The real class actually has about a dozen fields at present, and I can't guarantee that it won't change in the future.
What I can't work out is how to do this. I was really hoping there was just some standard JsonDeserializer subclass that would let me do this, but I can't find one. The best I've been able to work out is a normal StdDeserializer that then uses parser.getCodec().treeToValue() - courtesty of this answer - except that this results in an infinite loop as it calls back into the exact same deserializer every time!
Frustratingly, most answers to this problem are "Just annotate the class" - and that's not an option here!
Is there a standard way to achieve this?
Cheers
There are the Jacksons mixins for exactly this case, a super useful feature!
For your case, define the mixin class and annotate it as if you were annotating the original; you only need to include the overrides, e.g.:
#JsonIgnoreProperties({ "id" })
public class ResultMixin {
// nothing else required!
}
Now, you will have to hook on the ObjectMapper creation to define the mixin. This depends on the framework you are using, but in the end it should look like this:
ObjectMapper om = ...
om.addMixIn(Result.class, ResultMixin.class);
Now this ObjectMapper will take into account the information from you mixin to serialize objects of type Result (and ignore the synthetic id property).
And, of course, I've just found out how to do it :)
I can use BeanDeserializerModifier instead to make things act as needed.
public class MyDeserializerModifier extends BeanDeserializerModifier {
#Override
public List<BeanPropertyDefinition> updateProperties(final DeserializationConfig config,
final BeanDescription beanDesc, final List<BeanPropertyDefinition> propDefs) {
if (beanDesc.getBeanClass().equals(Result.class)) {
propDefs.removeIf(prop -> prop.getName().equals("id"));
}
return super.updateProperties(config, beanDesc, propDefs);
}
#Override
public BeanDeserializerBuilder updateBuilder(final DeserializationConfig config, final BeanDescription beanDesc,
final BeanDeserializerBuilder builder) {
if (beanDesc.getBeanClass().equals(Result.class)) {
builder.addIgnorable("id");
}
return super.updateBuilder(config, beanDesc, builder);
}
}
Related
I'm trying to deserialize a JSON structure with Jackson and I'm working with a DTO that looks like this:
public class RootLevelDTO {
private List<ComplexEntry> complexEntries;
// ... other fields, not relevant
}
Now, the ComplexEntry can have sub-types, those have properties of enum types etc. A lot can go wrong here if the other side of the communication updates their API and e.g. adds another sub type or adds an enum literal.
What I would like to do is to tell Jackson:
if you encounter any databinding error during deserialization of the complexEntries field...
... do not throw an exception, but instead ignore this entry and continue with the next.
What I tried so far is to use a delegating deserializer for ComplexEntry:
public class ComplexEntryDeserializer extends StdDeserializer<ComplexEntry> {
private StdDeserializer<ComplexEntry> delegate;
public ComplexEntryDeserializer(StdDeserializer<ComplexEntry> delegate){
this.delegate = delegate;
}
public ComplexEntry deserialize(JsonParser p, DeserializationContext ctxt){
try {
return this.delegate.deserialize(p, ctxt);
}catch(Exception e){
// the list entry failed to deserialize, but we have to return *something* here
return null;
}
}
// ... other mandatory methods, not relevant here
}
This solution has the problem that it will introduce null values to the complexEntries list, which I then have to explicitly get rid of with a Converter.
Is there a more elegant solution to this problem?
After a lot of tinkering I've ended up with the following solution. It doesn't require any additional jackson modules or other magic, only a single (specific) deserializer.
DTO:
public class RootLevelDTO {
// use a custom deserializer for the list
#JsonDeserialize(using = ListOfComplexEntryDeserializer.class)
private List<ComplexEntry> complexEntries;
}
Deserializer:
public class ListOfComplexEntryDeserializer extends JsonDeserializer<List<ComplexEntry>> {
#Override
public List<ComplexEntry> deserialize(JsonParser p, DeserializationContext ctxt) {
List<ComplexEntry> resultList = new ArrayList<>();
while(p.nextToken() != JsonToken.END_ARRAY){
try {
// delegate the deserialization of the individual list entries to the standard deserializers
resultList.add(ctxt.readValue(p, ComplexEntry.class))
}catch(Exception e){
// log that the entry wasn't deserialized properly
System.out.println("ComplexEntry could not be read and will be ignored.");
}
}
return resultList;
}
}
Big disclaimer: While the code above works, it's not something you should go for by design. I'm really with my back to the wall here and have no other choice (due to external factors beyond my control), and for that case it works.
I have facade interface where users can ask for information about lets say Engineers. That information should be transferred as JSON of which we made a DTO for. Now keep in mind that I have multiple datasources that can provide an item to this list of DTO.
So I believe right now that I can use Decorative Pattern by adding handler of the datasource to the myEngineerListDTO of type List<EngineerDTO>. So by that I mean all the datasources have the same DTO.
This picture below shows that VerticalScrollbar and HorizontalScrollBar have different behaviours added. Which means they add behaviour to the WindowDecorator interface.
My question, does my situation fit the decorator pattern? Do I specifically need to add a behaviour to use this pattern? And is there another pattern that does fit my situation? I have already considered Chain of Responsibility pattern, but because I don't need to terminate my chain on any given moment, i thought maybe Decorator pattern would be better.
Edit:
My end result should be: List<EngineersDTO> from all datasources. The reason I want to add this pattern is so that I can easily add another datasource behind the rest of the "pipeline". This datasource, just like the others, will have addEngineersDTOToList method.
To further illustrate on how you can Chain-of-responsibility pattern I put together a small example. I believe you should be able to adapt this solution to suit the needs of your real world problem.
Problem Space
We have an unknown set of user requests which contain the name of properties to be retrieved. There are multiple datasources which each have varying amounts of properties. We want to search through all possible data sources until all of the properties from the request have been discovered. Some data types and data sources might look like bellow (note I am using Lombok for brevity):
#lombok.Data
class FooBarData {
private final String foo;
private final String bar;
}
#lombok.Data
class FizzBuzzData {
private final String fizz;
private final String buzz;
}
class FooBarService {
public FooBarData invoke() {
System.out.println("This is an expensive FooBar call");
return new FooBarData("FOO", "BAR");
}
}
class FizzBuzzService {
public FizzBuzzData invoke() {
System.out.println("This is an expensive FizzBuzz call");
return new FizzBuzzData("FIZZ", "BUZZ");
}
}
Our end user might require multiple ways to resolve the data. The following could be a valid user input and expected response:
// Input
"foobar", "foo", "fizz"
// Output
{
"foobar" : {
"foo" : "FOO",
"bar" : "BAR"
},
"foo" : "FOO",
"fizz" : "FIZZ"
}
A basic interface and simple concrete implementation for our property resolver might look like bellow:
interface PropertyResolver {
Map<String, Object> resolve(List<String> properties);
}
class UnknownResolver implements PropertyResolver {
#Override
public Map<String, Object> resolve(List<String> properties) {
Map<String, Object> result = new HashMap<>();
for (String property : properties) {
result.put(property, "Unknown");
}
return result;
}
}
Solution Space
Rather than using a normal "Decorator pattern", a better solution may be a "Chain-of-responsibility pattern". This pattern is similar to the decorator pattern, however, each link in the chain is allowed to either work on the item, ignore the item, or end the execution. This is helpful for deciding if a call needs to be made, or terminating the chain if the work is complete for the request. Another difference from the decorator pattern is that resolve will not be overriden by each of the concrete classes; our abstract class can call out to the sub class when required using abstract methods.
Back to the problem at hand... For each resolver we need two components. A way to fetch data from our remote service, and a way to extract all the required properties from the data retrieved. For fetching the data we can provide an abstract method. For extracting a property from the fetched data we can make a small interface and maintain a list of these extractors seeing as multiple properties can be pulled from a single piece of data:
interface PropertyExtractor<Data> {
Object extract(Data data);
}
abstract class PropertyResolverChain<Data> implements PropertyResolver {
private final Map<String, PropertyExtractor<Data>> extractors = new HashMap<>();
private final PropertyResolver successor;
protected PropertyResolverChain(PropertyResolver successor) {
this.successor = successor;
}
protected abstract Data getData();
protected final void setBinding(String property, PropertyExtractor<Data> extractor) {
extractors.put(property, extractor);
}
#Override
public Map<String, Object> resolve(List<String> properties) {
...
}
}
The basic idea for the resolve method is to first evaluate which properties can be fulfilled by this PropertyResolver instance. If there are eligible properties then we will fetch the data using getData. For each eligible property we extract the property value and add it to a result map. Each property which cannot be resolved, the successor will be requested to be resolve that property. If all properties are resolved the chain of execution will end.
#Override
public Map<String, Object> resolve(List<String> properties) {
Map<String, Object> result = new HashMap<>();
List<String> eligibleProperties = new ArrayList<>(properties);
eligibleProperties.retainAll(extractors.keySet());
if (!eligibleProperties.isEmpty()) {
Data data = getData();
for (String property : eligibleProperties) {
result.put(property, extractors.get(property).extract(data));
}
}
List<String> remainingProperties = new ArrayList<>(properties);
remainingProperties.removeAll(eligibleProperties);
if (!remainingProperties.isEmpty()) {
result.putAll(successor.resolve(remainingProperties));
}
return result;
}
Implementing Resolvers
When we go to implement a concrete class for PropertyResolverChain we will need to implement the getData method and also bind PropertyExtractor instances. These bindings can act as an adapter for the data returned by each service. This data can follow the same structure as the data returned by the service, or have a custom schema. Using the FooBarService from earlier as an example, our class could be implemented like bellow (note that we can have many bindings which result in the same data being returned).
class FooBarResolver extends PropertyResolverChain<FooBarData> {
private final FooBarService remoteService;
FooBarResolver(PropertyResolver successor, FooBarService remoteService) {
super(successor);
this.remoteService = remoteService;
// return the whole object
setBinding("foobar", data -> data);
// accept different spellings
setBinding("foo", data -> data.getFoo());
setBinding("bar", data -> data.getBar());
setBinding("FOO", data -> data.getFoo());
setBinding("__bar", data -> data.getBar());
// create new properties all together!!
setBinding("barfoo", data -> data.getBar() + data.getFoo());
}
#Override
protected FooBarData getData() {
return remoteService.invoke();
}
}
Example Usage
Putting it all together, we can invoke the Resolver chain as shown bellow. We can observe that the expensive getData method call is only performed once per Resolver only if the property is bound to the resolver, and that the user gets only the exact fields which they require:
PropertyResolver resolver =
new FizzBuzzResolver(
new FooBarResolver(
new UnknownResolver(),
new FooBarService()),
new FizzBuzzService());
Map<String, Object> result = resolver.resolve(Arrays.asList(
"foobar", "foo", "__bar", "barfoo", "invalid", "fizz"));
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
System.out.println(mapper
.writerWithDefaultPrettyPrinter()
.writeValueAsString(result));
Output
This is an expensive FizzBuzz call
This is an expensive FooBar call
{
"foobar" : {
"foo" : "FOO",
"bar" : "BAR"
},
"__bar" : "BAR",
"barfoo" : "BARFOO",
"foo" : "FOO",
"invalid" : "Unknown",
"fizz" : "FIZZ"
}
As usual, there is probaly a very simple solution to my problem:
I have a JSON schema snippet that defines the following enum:
"title" : {
"type": "string",
"enum": ["Mr", "Miss", "Mrs", "Ms"],
"description": "The person's title"
}
My company's framework then uses jsonschema2pojo and maven to create the necessary POJO (Title lives in Clazz, as title is part of clazz in the JSON schema - with clazz as a name being made up - replace it with employee or customer or whatever you like):
Generated POJO
#Generated("org.jsonschema2pojo")
public static enum Title {
MR("Mr"),
MISS("Miss"),
MRS("Mrs"),
MS("Ms");
private final String value;
private static Map<String, Clazz.Title> constants = new HashMap<String, Clazz.Title>();
static {
for (Clazz.Title c: values()) {
constants.put(c.value, c);
}
}
private Title(String value) {
this.value = value;
}
#JsonValue
#Override
public String toString() {
return this.value;
}
#JsonCreator
public static Clazz.Title fromValue(String value) {
Clazz.Title constant = constants.get(value);
if (constant == null) {
throw new IllegalArgumentException(value);
} else {
return constant;
}
}
}
When I run a request containing the following against it:
...
"title" : "Mr",
...
I get this error thrown back at me:
com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not construct instance of com.example.foo.representations.jaxb.Clazz$Title from String value 'Mr': value not one of declared Enum instance names: [Mr, Miss, Mrs, Ms] at [Source: org.apache.cxf.transport.http.AbstractHTTPDestination$1#1a372b7; line: 4, column: 3] (through reference chain: com.example.foo.representations.jaxb.MySchema["Clazz"]->com.example.foo.representations.jaxb.Clazz["title"])
Clearly, "Mr" is in the Enum.
When debugging, I can see that it runs through the following classes (stack):
findEnum():120, EnumResolver (com.fasterxml.jackson.databind.util)
deserialize():79, EnumDeserializer (com.fasterxml.jackson.databind.deser.std)
It looks like they are only interested in the Enum's "keys" (i.e. constants, e.g. "MR" instead of "Mr"). I'm guessing that the #JsonCreator annotation is ignored for some reason.
Any idea how I can fix that issue?
Is there a configuration value that might be set anywhere that might cause this behaviour? (I'm working on a big projects; if I know what I need to look for I can search the code base; maybe another developer "misconfigured" something somewhere...) Or might the issue be that Title lives in Clazz? Do I need to throw an #JsonProperty in for good measure? (If so, how exactly?)
We are using jackson-core, -annotations, and -databind 2.4.2.
Update: I tried this as a stand-alone project, with the following code, and it worked flawlessly - which means there must be some sort of setting that prevents the annotation from being taken into account...
ObjectMapper mapper = new ObjectMapper(); // create once, reuse
Clazz value = mapper.readValue(new File("resources/data.json"), Clazz.class);
So, it turns out we had the following in our Spring config:
<bean id="mapper" class="com.example.foo.jaxb.links.JsonMapper" />
<!-- ... -->
<jaxrs:providers>
<bean id="jsonprovider"
class="com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider">
<constructor-arg ref="mapper" />
<constructor-arg>
<value></value>
</constructor-arg>
</bean>
</jaxrs:providers>
Jsonmapper extends com.fasterxml.jackson.databind.ObjectMapper, and mostly sets Jackson settings on itself in its constructor. The one thing I had missed in my desperation, however, is this bit of code:
// use JAXB annotations (only)
setAnnotationIntrospector(new JaxbAnnotationIntrospector(
com.fasterxml.jackson.databind.type.TypeFactory.defaultInstance()));
And it pretty much does what it says on the tin, it seems: it stops Jackson from evaluating the #JsonCreator and #JsonValue annotations. Once the "Jackson annotation suppressor" is gone, all is good.
What I'd like to do is understand how it actually works, so if anyone has any helpful links to docs/how-tos/manuals/books for either the Spring stuff or the suppression of annotations, that would be much appreciated. But in the meantime, this fixes my problem. :)
I'm currently having an issue with the deserialization of certain inner-objects, in spring, I initialize all of my objects before outputting them using #ResponseBody.
As an example, this is a response:
[{id:1, location:{id:1, ... extra location data}},
{id:2, location:1}
]
Now, GSON throws an error as it is not able to understand that location:1 refers to the location object already deserialized in the previous object.
Deserialization is done in the following method:
#Override
public void handleReader(Reader reader) {
try {
String json = readerToString(reader);
T object = getGson().fromJson(json, returnType);
handleObject(object);
} catch (Exception e) {
Sentry.captureException(e);
}
}
As an example, this is called through a regular generic class, I'd use the type Event[] as the T generic in order to return an array.
How can I either fix this using Gson or make spring output the full data every time? Ideally I'd like to fix with Gson as it would allow for seriously reduced bandwidth but I'm not too fussed at this point.
My Spring returning method is as follows:
#Override
public List<T> list() {
return service.findAll();
}
with the initialization like so:
#Override
#Transactional
public List<Event> findAll() {
List<Event> list = eventRepository.findByArchivedFalse();
for (Event event : list) {
this.initialize(event);
}
return list;
}
#Override
public Event initialize(Event obj) {
Hibernate.initialize(obj.getLocation());
Hibernate.initialize(obj.getLocation().get... inner data here);
return obj;
}
I imagine this is going to require a real structure review but, if I can help it, I'd like to keep the structure roughly the same.
You're going to have to write a custom deserializer, if you're not willing to change the JSon. However, changing the JSon is exactly what I would recommend.
Option 1: Changing the JSon
I think the right thing to do is to have two separate messages, e.g.
{
"uniqueLocations":
[
{"id":1, ... extra location details} ,
],
"locationMap":
[
{"id":1,"location":1},
{"id":2,"location":1}
... etc.
]
}
This is clearer; this separates your json so that you always have the same types of data in the same places.
Option 2: Making Gson able to do more complicated deserializations
However, if you're not willing to do that, you could write a custom deserializer. The most straightforward way to do that, extending TypeAdapter, only uses specific, concrete classes, not parameterized types. However, if you want to use a parameterized type, you must use a TypeAdapterFactory.
You can read more about how to do this here: How do I implement TypeAdapterFactory in Gson?
We want to write an API that looks something like this:
#XmlRootElement(name = "MyData")
#XmlAccessorType(XmlAccessType.FIELD)
public static class MyData {
public String name;
public Map<String,String> data;
}
#GET
#Path("/mydata")
public MyData getMyData() {
MyData ret = new MyData();
ret.name = "map data";
ret.data = new HashMap<>();
ret.data.put("a1", "b1");
ret.data.put("a2", "b2");
return ret;
}
But here's the sticking point we cannot seem to get around: we want this to return a JSON structure something like this:
{
"MyData":{
"name":"map data",
"data":{
"a1": "b1",
"a2": "b2"
}
}
}
and we can't figure out how to get beyond something like
{
"MyData":{
"name":"map data",
"data":{
"entry":[
{
"key":"a1",
"value":"b1"
},
{
"key":"a2",
"value":"b2"
}
]
}
}
}
Any idea how we might do this? I'm pretty sure this can be done, because I once saw a demo of it. We're using tomcat, Java 7, CXF 2.7.3, and Jackson 2.1.2. Two points:
Note that it doesn't have to contain a Map necessarily: all we need is to marsall a bunch of key/values where the keys are not known in advance.
We have to go both directions - we need to implement PUT/POST as well as GET with this syntax in the representation.
If you want to return only JSON (do not support XML), then simply remove all JAXB annotations from MyData and you will get a pretty good structure, but without the wrapping "MyData" element (which seems redundant to me). You could e.g use core Jackson annotations, like #JsonIgnore (instead of JAXB's). If you want to add the root wrapping element, you could set the WRAP_ROOT_VALUE to true.