We Keep Coding

How to Parse untyped, nested JSON with Polymorphism?

I am using Feign to hit external APIs, and then to process the externally generated JSON (aka, the response data cannot be modified in any way), and I am trying to bundle these together into an extensible super type. At this point, I am not even sure if what I am trying to do is possible with Jackson / Feign. If it would be much easier to abandon (or heavily restructure) the polymorphism, I think I am also ready to give up on it and just create a bunch of sub classes.
Here are my two main questions, with more context below.
Should I just separate the easily deduced types from the complex types, and have a little more duplicated boiler plate?
How can I create a custom deserializer for the list object I linked? Ideally I would like to have some way to populate the more boiler plate fields less manually -- as an example, it would be great if I could call default deserializers inside it, which would rely more on the standard annotations in other objects.
Ideally, I would like one class, like this:
public final class BillApiResponse {
#Valid
#JsonProperty("response_status")
private boolean responseStatus;
#Valid
#JsonProperty("response_message")
private String responseMessage;
#JsonProperty("response_data")
private BillApiResponseData responseData;
//getters and setters, etc.
}
and then I would to have Jackson automatically map the simpler objects in whatever way is easiest (LoginResponse, LoginError), while I would try to implement a custom handler for the more complex objects (UpdateObject, ListOfObjects).
So, something like this:
#JsonTypeInfo(use = Id.DEDUCTION)
#JsonSubTypes({
#Type(value = BillLoginSuccess.class),
#Type(value = BillErrorResponse.class),
//#Type(value = BillResponseObject[].class) <--- This breaks things when added
})
// #JsonTypeResolver(value = BillResponseTypeResolver.class) <--- Open to using one of
// these if I can figure out how
// #JsonDeserialize(using = BillResponseDeserializer.class) <--- Also open to using a
// custom deserializer, but I would like to keep it only for certain parts
public interface BillApiResponseData {}
Here is a link to the API specification I am trying to hit:
Get a List of Objects
This returns an untyped array of untyped objects. Jackson does not seem to like that the array is untyped, and stops parsing everything there. Once inside, we would have to grab the type from a property.
{
"response_status" : 0,
"response_message" : "Success",
"response_data" : [{
"entity" : "SentPay",
"id" : "stp01AUXGYKCBGFMaqlc"
// More fields
} // More values]
}
Login
This returns a totally new object. Generally not having issues handling this one (until I add support for the above list, and then all of the parsing breaks down as Jackson throws errors).
Update Object
This returns an untyped object. Once again, we would have to go inside and look at the property.
I have tried a number of things, but generally I was not successful (hence I am here!).
These include:
Trying to hook into the lifecycle and take over if I detect an array object. I believe this fails because Jackson throws an error when it sees the array does not have a type associated with it.
SimpleModule customDeserializerModule = new SimpleModule()
.setDeserializerModifier(new BeanDeserializerModifier() {
#Override
public JsonDeserializer<?> modifyDeserializer(
DeserializationConfig config,
BeanDescription beanDesc,
JsonDeserializer<?> defaultDeserializer) {
if (beanDesc.getBeanClass().isArray()) {
return new BillResponseDeserializer(defaultDeserializer);
} else {
return defaultDeserializer;
}
}
});
Custom Deserializers. The issue I have is that it seems to want to route ALL of my deserialization calls into the custom one, and I don't want to have to handle the simpler items, which can be deduced.
TypeIdResolvers / TypeResolvers. Frankly these are confusing me a little bit, and I cannot find a good example online to try out.

How to consume a Spring HAL/HATEOAS API in Java using purely Jackson, not Spring

We are trying to create a Java client for an API created with Spring Data.
Some endpoints return hal+json responses containing _embedded and _links attributes.
Our main problem at the moment is trying to wrap our heads around the following structure:
{
"_embedded": {
"plans": [
{
...
}
]
},
...
}
When you hit the plans endpoint you get a paginated response the content of which is within the _embedded object. So the logic is that you call plans and you get back a response containing an _embedded object that contains a plans attribute that holds an array of plan objects.
The content of the _embedded object can vary as well, and trying a solution using generics, like the example following, ended up returning us a List of LinkedHashMap Objects instead of the expected type.
class PaginatedResponse<T> {
#JsonProperty("_embedded")
Embedded<T> embedded;
....
}
class Embedded<T> {
#JsonAlias({"plans", "projects"})
List<T> content; // This instead of type T ends up deserialising as a List of LinkedHashMap objects
....
}
I am not sure if the above issue is relevant to this Jackson bug report dating from 2015.
The only solution we have so far is to either create a paginated response for each type of content, with explicitly defined types, or to include a List<type_here> for each type of object we expect to receive and make sure that we only read from the populated list and not the null ones.
So our main question to this quite spread out issue is, how one is supposed to navigate such an API without the use of Spring?
We do not consider using Spring in any form as an acceptable solution. At the same time, and I may be quite wrong here, but it looks like in the java world Spring is the only framework actively supporting/promoting HAL/HATEOAS?
I'm sorry if there are wrongly expressed concepts, assumptions and terminology in this question but we are trying to wrap our heads around the philosophy of such an implementation and how to deal with it from a Java point of view.

You can try consuming HATEOS API using super type tokens. A kind of generic way to handle all kind of hateos response.
For example
Below generic class to handle response
public class Resource<T> {
protected Resource() {
this.content = null;
}
public Resource(T content, Link... links) {
this(content, Arrays.asList(links));
}
}
Below code to read the response for various objects
ObjectMapper objectMapper = new ObjectMapper();
Resource<ObjectA> objectA = objectMapper.readValue(response, new TypeReference<Resource<ObjectA>>() {});
Resource<ObjectB> objectB = objectMapper.readValue(response, new TypeReference<Resource<ObjectB>>() {});
You can refer below
http://www.java-allandsundry.com/2012/12/json-deserialization-with-jackson-and.html
http://www.java-allandsundry.com/2014/01/consuming-spring-hateoas-rest-service.html

What's the common way to deal with Jackson serialization

Currently I have a project that makes use of Spring-Hibernate and also Jackson to deal with JSON. The first time I tried to use Jackson I always got LazyInitializationException and sometimes infinite loop for multiple entities that references each other. Then I found #JsonIgnore and #JsonIdentityInfo.
Now the problem is sometimes it is needed to ignore properties but sometimes I just need those properties to be serializable. Is there a way to sometimes ignore several fields and sometimes serialize the fields at the runtime?
I found "Serialization and Deserialization with Jackson: how to programmatically ignore fields?"
But if I always have to use the mix in annotation, it would be cumbersome if an object dozens of properties to retrieve. Eg. In page1 I need propertyA, propertyB, propertyC; in page2 I need propertyA and propertyC; in page3 I only need propertyB. In those cases alone I would have to create 1 class for each page resulting in 3 classes.
So in that case is there a way to define something like:
objectA.ignoreAllExcept('propertyA');
String[] properties = {'propertyA', 'propertyC'};
objectB.ignoreAllExcept(properties); // Retrieve propertyA and propertyC
objectC.ignore(properties);

What you might be looking for is a Module. The documentation says that Modules are
Simple interface for extensions that can be registered with ObjectMappers to provide a well-defined set of extensions to default functionality.
Following is am example of how you might use them to accomplish what you want. Note, there are other ways using which this can be achieved; this is just one of them.
A simple DTO that can be used for specifying the properties to filter:
public class PropertyFilter {
public Class<?> classToFilter;
public Set<String> propertiesToIgnore = Collections.emptySet();
public PropertyFilter(Class<?> classToFilter, Set<String> propertiesToIgnore) {
this.classToFilter = classToFilter;
this.propertiesToIgnore = propertiesToIgnore;
}
}
A custom module that filters out properties based on some attribute that you store in the current request.
public class MyModule extends Module {
#Override
public String getModuleName() {
return "Test Module";
}
#Override
public void setupModule(SetupContext context) {
context.addBeanSerializerModifier(new MySerializerModifier());
}
#Override
public Version version() {
// Modify if you need to.
return Version.unknownVersion();
}
public static class MySerializerModifier extends BeanSerializerModifier {
public BeanSerializerBuilder updateBuilder(SerializationConfig config,
BeanDescription beanDesc,
BeanSerializerBuilder builder) {
List<PropertyFilter> filters = (List<PropertyFilter>) RequestContextHolder.getRequestAttributes().getAttribute("filters", RequestAttributes.SCOPE_REQUEST);
PropertyFilter filter = getPropertyFilterForClass(filters, beanDesc.getBeanClass());
if(filter == null) {
return builder;
}
List<BeanPropertyWriter> propsToWrite = new ArrayList<BeanPropertyWriter>();
for(BeanPropertyWriter writer : builder.getProperties()) {
if(!filter.propertiesToIgnore.contains(writer.getName())) {
propsToWrite.add(writer);
}
}
builder.setProperties(propsToWrite);
return builder;
}
private PropertyFilter getPropertyFilterForClass(List<PropertyFilter> filters, Class<?> classToCheck) {
for(PropertyFilter f : filters) {
if(f.classToFilter.equals(classToCheck)) {
return f;
}
}
return null;
}
}
}
Note: There is a changeProperties method in the BeanSerializerModifier class that is more appropriate for changing the property list (according to the documentation). So you can move the code written in the updateBuilder to changeProperties method with appropriate changes.
Now, you need to register this custom module with your ObjectMapper. You can get the Jackson HTTP message converter from your application context, and get its object mapper. I am assuming you already know how to do that as you have been dealing with the lazy-initialization issue as well.
// Figure out a way to get the ObjectMapper.
MappingJackson2HttpMessageConverter converter = ... // get the jackson-mapper;
converter.getObjectMapper().registerModule(new MyModule())
And you are done. When you want to customize the serialization for a particular type of object, create a PropertyFilter for that, put it in a List and make it available as an attribute in the current request. This is just a simple example. You might need to tweak it a bit to suit your needs.
In your question, you seem to be looking for a way to specify the properties-to-filter-out on the serialized objects themselves. That, in my opinion, should be avoided as the list of properties to filter-out doesn't belong to your entities. However, if you do want to do that, create an interface that provides setters and getters for the list of properties. Suppose the name of the interface is CustomSerialized Then, you can modify the MyModule class to look for the instances of this CustomSerialized interface and filter out the properties accordingly.
Note: You might need to adjust/tweak a few things based on the versions of the libraries you are using.

I think there is a more flexible way to do it. You can configure Jackson in a such a way that it will silently ignore lazy loaded properties instead of stopping serialization process. So you can reuse the same class. Just load all necessary properties / relations and pass it to Jackson. You can try to do it by declaring your custom ObjectMapper and by turning off SerializationFeature.FAIL_ON_EMPTY_BEANS feature. Hope it helps.

You can filter out properties without modifying classes by creating a static interface for a mixin annotation. Next, annotate that interface with the #JsonFilter annotation. Create a SimpleBeanPropertyFilter and a SimpleFilterProvider. Then create an ObjectWriter with your filter provider by invoking objectMapper.writer(filterProvider)

Is there a way to create the bean class from a json response

Converting JSON to Java
The above question is with reference to what has been described on the above thread. There are so many API(s) which provide the flexibility to return responses either in XML or JSON. **I would like to know if there is a way to automatically construct the java bean corresponding to a JSON response. **

lets say you get an object like
[
{
"name":"Java 6 Greatest Hits",
"Author":"Jim Bob Jones",
"price":10.25
},
{
"name":"How to raise a goat",
"Author":"Sir Paxton",
"price":55.97
},
{
"name":"Snow - It is cold",
"Author":"Dr. White",
"price":9.99
}
]
And you want a class like
public class Book{
private String author;
private String name;
private Number price
}
with getters and setters
One option is to use a service like JSONGen, which will create that class. You need to use it first, and include the generated code in your project.
Another option could be dynamically generate the class using javassist or CGLib, but that class would be useless unless you use reflection to access its members, so even if it would be a class, it will behave like a really annoying Map. In no way will be better that simple using JSONObject

seems a simple Message Type Entity not meet you requirement ?
if you want convert a json to an existed and known java bean class,
many lib can do so, like
http://json-lib.sourceforge.net/apidocs/net/sf/json/class-use/JSONObject.html
JSONObject.toBean(JSONObject jsonObject, Class beanClass)
Creates a bean from a JSONObject, with a specific target class.
btw, if you are communicating with restful webservice, org.springframework.web.client.RestTemplate will help you get direct bean result
insteadof json.
if class does not exists, you need program with java reflect mechanism.
try use CGLIB ,http://cglib.sourceforge.net/, dynamic create some class like BeanMap. i wrote a simple sample,
but be ware, opearting class byte is hard and you may meet strange trouble with JVM . Strongly not encourage to do so.
public static BeanMap generateBean(JSONObject json) {
BeanGenerator generator = new BeanGenerator();
Iterator keys = json.keys();
while (keys.hasNext()) {
Object key = keys.next();
Object value = json.get(key);
Class keyClass = guessValueClass(value);
generator.addProperty(key.toString(), keyClass);
}
Object result = generator.create();
BeanMap bean = BeanMap.create(result);
keys = json.keys();
while (keys.hasNext()) {
Object key = keys.next();
Object value = json.get(key);
bean.put(key, value);
}
return bean;
}
/**
* TODO fix guess
*/
static Class guessValueClass(Object value) {
try {
Integer.parseInt(value.toString());
return Integer.class;
} catch (NumberFormatException e1) {
}
try {
Double.parseDouble(value.toString());
return Double.class;
} catch (NumberFormatException e1) {
}
return String.class;
}

I believe the main issue here is that the JSON response lacks type information and last time I checked :-) in Java you need to declare the type of a class property. So some heuristics will be needed to infer the type form the value in the JSON response.
For a related question here in SO have a look at: Generate Java class from JSON?

Yes check out http://flexjson.sourceforge.net

If you're wanting to generate Java classes from JSON, perhaps you could try Jackson. It provides a lot of JSON-related functionality, including the ability to generate bytecode from arbitrary JSON. See this blog post for details.

If you're using Jackson (the most popular library there), try
https://bitbucket.org/astav/jsontojava/wiki/Home
Its open source and anyone should be able to contribute.
Summary
A JsonToJava source class file generator that deduces the schema based on supplied sample json data and generates the necessary java data structures.
It encourages teams to think in Json first, before writing actual code.
Features
Can generate classes for an arbitrarily complex hierarchy (recursively)
Can read your existing Java classes and if it can deserialize into those structures, will do so
Will prompt for user input when ambiguous cases exist

How can I tell jackson to ignore a property for which I don't have control over the source code?

Long story short, one of my entities has a GeometryCollection that throws an exception when you call "getBoundary" (the why of this is another book, for now let's say this is the way it works).
Is there a way I can tell Jackson not to include that specific getter? I know I can use #JacksonIgnore when I do own/control the code. But this is not case, jackson ends reaching this point through continuous serialization of the parent objects. I saw a filtering option in jackson documentation. Is that a plausible solution?
Thanks!

You can use Jackson Mixins. For example:
class YourClass {
public int ignoreThis() { return 0; }
}
With this Mixin
abstract class MixIn {
#JsonIgnore abstract int ignoreThis(); // we don't need it!
}
With this:
objectMapper.getSerializationConfig().addMixInAnnotations(YourClass.class, MixIn.class);
Edit:
Thanks to the comments, with Jackson 2.5+, the API has changed and should be called with objectMapper.addMixIn(Class<?> target, Class<?> mixinSource)

One other possibility is, if you want to ignore all unknown properties, you can configure the mapper as follows:
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

Using Java Class
new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
Using Annotation
#JsonIgnoreProperties(ignoreUnknown=true)

Annotation based approach is better. But sometimes manual operation is needed. For this purpose you can use without method of ObjectWriter.
ObjectMapper mapper = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
ObjectWriter writer = mapper.writer().withoutAttribute("property1").withoutAttribute("property2");
String jsonText = writer.writeValueAsString(sourceObject);

Mix-in annotations work pretty well here as already mentioned. Another possibility beyond per-property #JsonIgnore is to use #JsonIgnoreType if you have a type that should never be included (i.e. if all instances of GeometryCollection properties should be ignored). You can then either add it directly (if you control the type), or using mix-in, like:
#JsonIgnoreType abstract class MixIn { }
// and then register mix-in, either via SerializationConfig, or by using SimpleModule
This can be more convenient if you have lots of classes that all have a single 'IgnoredType getContext()' accessor or so (which is the case for many frameworks)

I had a similar issue, but it was related to Hibernate's bi-directional relationships. I wanted to show one side of the relationship and programmatically ignore the other, depending on what view I was dealing with. If you can't do that, you end up with nasty StackOverflowExceptions. For instance, if I had these objects
public class A{
Long id;
String name;
List<B> children;
}
public class B{
Long id;
A parent;
}
I would want to programmatically ignore the parent field in B if I were looking at A, and ignore the children field in A if I were looking at B.
I started off using mixins to do this, but that very quickly becomes horrible; you have so many useless classes laying around that exist solely to format data. I ended up writing my own serializer to handle this in a cleaner way: https://github.com/monitorjbl/json-view.
It allows you programmatically specify what fields to ignore:
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(JsonView.class, new JsonViewSerializer());
mapper.registerModule(module);
List<A> list = getListOfA();
String json = mapper.writeValueAsString(JsonView.with(list)
.onClass(B.class, match()
.exclude("parent")));
It also lets you easily specify very simplified views through wildcard matchers:
String json = mapper.writeValueAsString(JsonView.with(list)
.onClass(A.class, match()
.exclude("*")
.include("id", "name")));
In my original case, the need for simple views like this was to show the bare minimum about the parent/child, but it also became useful for our role-based security. Less privileged views of objects needed to return less information about the object.
All of this comes from the serializer, but I was using Spring MVC in my app. To get it to properly handle these cases, I wrote an integration that you can drop in to existing Spring controller classes:
#Controller
public class JsonController {
private JsonResult json = JsonResult.instance();
#Autowired
private TestObjectService service;
#RequestMapping(method = RequestMethod.GET, value = "/bean")
#ResponseBody
public List<TestObject> getTestObject() {
List<TestObject> list = service.list();
return json.use(JsonView.with(list)
.onClass(TestObject.class, Match.match()
.exclude("int1")
.include("ignoredDirect")))
.returnValue();
}
}
Both are available on Maven Central. I hope it helps someone else out there, this is a particularly ugly problem with Jackson that didn't have a good solution for my case.

If you want to ALWAYS exclude certain properties for any class, you could use setMixInResolver method:
#JsonIgnoreProperties({"id", "index", "version"})
abstract class MixIn {
}
mapper.setMixInResolver(new ClassIntrospector.MixInResolver(){
#Override
public Class<?> findMixInClassFor(Class<?> cls) {
return MixIn.class;
}
#Override
public ClassIntrospector.MixInResolver copy() {
return this;
}
});

One more good point here is to use #JsonFilter.
Some details here Feature: JSON Filter