We Keep Coding

How arbitrary JSON string can be deserialized to java POJO?

Lets say we have simple json string json = {"key1":"value1", "key2":"value2"} and java class
class Foo {
private String field1;
private Integer field2;
//setter & getter
}
Moreover we don't want to change the Foo class. Note that json keys don't match with Foo's fields name.
Is there simple way we can deserilize json string to Foo class with Jackson or any other library?

You can use the following json libraries and build a custom deserializer as shown below.
jackson-annotations-2.10.4,
jackson-core-2.10.4,
jackson.databind-2.10.4
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.node.IntNode;
import java.io.IOException;
public class FooDeserializer extends StdDeserializer<Foo> {
public static void main (String [] args) throws JsonProcessingException {
String json = "{\"key1\":\"value1\", \"key2\":100}";
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addDeserializer(Foo.class, new FooDeserializer());
mapper.registerModule(module);
Foo foo = mapper.readValue(json, Foo.class);
System.out.println(foo);
}
public FooDeserializer() {
this(null);
}
public FooDeserializer(Class<?> vc) {
super(vc);
}
#Override
public Foo deserialize(JsonParser jp, DeserializationContext ctx)
throws IOException, JsonProcessingException {
JsonNode node = jp.getCodec().readTree(jp);
String field1 = node.get("key1").asText();
int field2 = (Integer) ((IntNode) node.get("key2")).numberValue();
return new Foo(field1,field2);
}
}

Jackson deserialization SNS message error MismatchedInputException

I'm coding a functionality of handling callbacks from Amazon Simple Email Service via SNS HTTP requests. I would like to parse message provided by Amazon to local object structure. Problem is that SNS is wrapping JSON message into String and it could not be parsed by Jackson. I'm getting an error:
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `xxx.email.domain.aws.ses.Notification` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('{"notificationType":"Delivery","mail":{"timestamp":"2019-10-02T14:43:14.570Z" ... next values of the message ... }}')
Entire message from SNS looks like this one:
{
"Type" : "Notification",
"MessageId" : "4944xxxx-711d-57d4-91b8-8215cxxxxx",
"TopicArn" : "arn:aws:sns:eu-west-1:...",
"Message" : "{\"notificationType\":\"Delivery\",\"mail\":{\"timestamp\":\"2019-10-02T14:43:14.570Z\", ... next values of the message ... },\"delivery\":{\"timestamp\":\"2019-10-02T14:43:16.030Z\", ... next values of the message ... }}",
"Timestamp" : "2019-10-02T14:43:16.062Z",
"SignatureVersion" : "1",
"Signature" : "signature base64",
"SigningCertURL" : "cert url",
"UnsubscribeURL" : "unsubscribe url"
}
My actual local structure looks like this:
#Data
#JsonNaming(PropertyNamingStrategy.UpperCamelCaseStrategy.class)
public class MessageWrapper {
private String type;
private String messageId;
private String topicArn;
private Notification message;
private Date timestamp;
private String signatureVersion;
private String signature;
private String signingCertURL;
private String unsubscribeURL;
}
#Data
public class Notification {
private String notificationType;
private Mail mail;
}
#Data
public class Mail {
private String messageId;
private String source;
private String sourceArn;
private String sourceIp;
private String sendingAccountId;
private String[] destination;
}
I'm looking for some way to tell Jackson that Message should be extracted from a String and treated as a normal JSON.
Edit
deserialization
private MessageWrapper deserializeMessage(String message) throws IOException {
return new ObjectMapper().readValue(message, MessageWrapper.class);
}

I think to solve this you'll need a custom deserializer for Notification field in MessageWrapper class as well as one for the Mail field in the Notification class the like the following:
public class NotificationDeserializer extends JsonDeserializer<Notification> {
#Override
public Notification deserialize(JsonParser p, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
String text = p.getText();
return new ObjectMapper().readValue(text, Notification.class);
}
}
public class MailDeserializer extends JsonDeserializer<Mail> {
#Override
public Mail deserialize(JsonParser p, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
String text = p.getText();
return new ObjectMapper().readValue(text, Mail.class);
}
}
With some annotations on your classes like the following:
#Data
#JsonNaming(PropertyNamingStrategy.UpperCamelCaseStrategy.class)
public class MessageWrapper {
private String type;
private String messageId;
private String topicArn;
#JsonDeserialize(using = NotificationDeserializer.class)
private Notification message;
private Date timestamp;
private String signatureVersion;
private String signature;
private String signingCertURL;
private String unsubscribeURL;
}
#Data
public class Notification {
private String notificationType;
#JsonDeserialize(using = MailDeserializer.class)
private Mail mail;
}
#Data
public class Mail {
private String messageId;
private String source;
private String sourceArn;
private String sourceIp;
private String sendingAccountId;
private String[] destination;
}
EDIT 1
The MailDeserializer isn't actually needed. The NotificationDeserializer alone takes care of the issue.
EDIT 2
Using a new ObjectMapper in the custom deserializer is a must.

message property is of type Notification and Jackson expects JSON Object not string value. In that case you can create custom deserialiser or implement general solution with some kind of loop back implementation. If given payload is not a JSON Object read it as a String and invoke deserialisation again with this String.
To avoid StackOverflowError you need to use another instance of ObjectMapper or use BeanDeserializerModifier to keep BeanDeserializer instance and use it where JSON Object is encountered. Simple example could look like below:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.BeanDescription;
import com.fasterxml.jackson.databind.DeserializationConfig;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.PropertyNamingStrategy;
import com.fasterxml.jackson.databind.annotation.JsonNaming;
import com.fasterxml.jackson.databind.deser.BeanDeserializer;
import com.fasterxml.jackson.databind.deser.BeanDeserializerBase;
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.node.TextNode;
import lombok.Data;
import lombok.ToString;
import java.io.File;
import java.io.IOException;
import java.util.Collections;
import java.util.Date;
import java.util.Objects;
import java.util.Set;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
SimpleModule loopBackModule = new SimpleModule();
loopBackModule.setDeserializerModifier(new LoopBackBeanDeserializerModifier(Collections.singleton(Notification.class)));
ObjectMapper mapper = new ObjectMapper();
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
mapper.registerModule(loopBackModule);
MessageWrapper wrapper = mapper.readValue(jsonFile, MessageWrapper.class);
System.out.println(wrapper.getMessage());
}
}
class LoopBackBeanDeserializerModifier extends BeanDeserializerModifier {
private final Set<Class> allowedClasses;
LoopBackBeanDeserializerModifier(Set<Class> allowedClasses) {
this.allowedClasses = Objects.requireNonNull(allowedClasses);
}
#Override
public JsonDeserializer<?> modifyDeserializer(DeserializationConfig config, BeanDescription beanDesc, JsonDeserializer<?> deserializer) {
if (allowedClasses.contains(beanDesc.getBeanClass())) {
return new LoopBackBeanDeserializer<>((BeanDeserializerBase) deserializer);
}
return deserializer;
}
}
class LoopBackBeanDeserializer<T> extends BeanDeserializer {
private final BeanDeserializerBase baseDeserializer;
protected LoopBackBeanDeserializer(BeanDeserializerBase src) {
super(src);
this.baseDeserializer = src;
}
#Override
public T deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
// if first token is VALUE_STRING we should read it as String and
// run deserialization process again based on this String.
if (p.currentToken() == JsonToken.VALUE_STRING) {
return (T) ((ObjectMapper) p.getCodec()).readValue(p.getText(), _valueClass);
}
// vanilla bean deserialization
return (T) baseDeserializer.deserialize(p, ctxt);
}
}
POJO model is the same. You just need to list classes for which you expect some problems and loop-back mechanism will work for them.

CQEngine In Clause with MultiValueNullableAttribute

I have a class Object1 which has a list of longs called tags. I have another list of longs called tagsToSearch. How can I construct a query using CQEngine that is the following:
Select * from Object1 Where tags in (tagsToSearch)
If anyone knows how this would look using CQEngine please let me know.

This should do the trick:
package com.googlecode.cqengine;
import com.googlecode.cqengine.attribute.*;
import com.googlecode.cqengine.query.Query;
import com.googlecode.cqengine.query.option.QueryOptions;
import com.googlecode.cqengine.query.parser.sql.SQLParser;
import java.util.*;
import static com.googlecode.cqengine.codegen.AttributeBytecodeGenerator.*;
import static com.googlecode.cqengine.query.QueryFactory.*;
import static java.util.Arrays.asList;
public class TagsExample {
static class MyObject {
final String name;
final List<Long> tags;
public MyObject(String name, List<Long> tags) {
this.name = name;
this.tags = tags;
}
static final Attribute<MyObject, Long> TAGS = new MultiValueAttribute<MyObject, Long>("tags") {
public Iterable<Long> getValues(MyObject object, QueryOptions queryOptions) { return object.tags; }
};
}
public static void main(String[] args) {
IndexedCollection<MyObject> collection = new ConcurrentIndexedCollection<>();
collection.add(new MyObject("foo", asList(1L, 2L, 3L)));
collection.add(new MyObject("bar", asList(4L, 5L, 6L)));
collection.add(new MyObject("baz", asList(7L, 8L, 9L)));
// Search via a programmatic query...
Query<MyObject> nativeQuery = in(MyObject.TAGS, asList(3L, 9L));
collection.retrieve(nativeQuery)
.forEach(object -> System.out.println(object.name));
// ..prints: foo, baz
// Search via an SQL query...
String sqlQuery = "SELECT * FROM collection WHERE tags IN (3, 9)";
SQLParser<MyObject> parser = SQLParser.forPojoWithAttributes(MyObject.class, createAttributes(MyObject.class));
parser.retrieve(collection, sqlQuery)
.forEach(object -> System.out.println(object.name));
// ..prints: foo, baz
}
}

Deserializing stringified (quote enclosed) nested objects with Jackson

I am consuming a "RESTful" service (via RestTemplate) that produces JSON as follows:
{
"id": "abcd1234",
"name": "test",
"connections": {
"default": "http://foo.com/api/",
"dev": "http://dev.foo.com/api/v2"
},
"settings": {
"foo": "{\n \"fooId\": 1, \"token\": \"abc\"}",
"bar": "{\"barId\": 2, \"accountId\": \"d7cj3\"}"
}
}
Note the settings.foo and settings.bar values, which cause issues on deserialization. I would like to be able to deserialize into objects (e.g., settings.getFoo().getFooId(), settings.getFoo().getToken()).
I was able to solve this specifically for an instance of Foo with a custom deserializer:
public class FooDeserializer extends JsonDeserializer<Foo> {
#Override
public Foo deserialize(JsonParser jp, DeserializationContext ctx) throws IOException {
JsonNode node = jp.getCodec().readTree(jp);
String text = node.toString();
String trimmed = text.substring(1, text.length() - 1);
trimmed = trimmed.replace("\\", "");
trimmed = trimmed.replace("\n", "");
ObjectMapper mapper = new ObjectMapper();
JsonNode obj = mapper.readTree(trimmed);
Foo result = mapper.convertValue(obj, Foo.class);
return result;
}
}
#Data
#JsonIgnoreProperties(ignoreUnknown = true)
public class Settings {
#JsonDeserialize(using = FooDeserializer.class)
private Foo foo;
private Bar bar;
}
However, now if I want to deserialize settings.bar, I need to implement another custom deserializer. So I implemented a generic deserializer as follows:
public class QuotedObjectDeserializer<T> extends JsonDeserializer<T> implements ContextualDeserializer {
private Class<?> targetType;
private ObjectMapper mapper;
public QuotedObjectDeserializer(Class<?> targetType, ObjectMapper mapper) {
this.targetType = targetType;
this.mapper = mapper;
}
#Override
public JsonDeserializer<T> createContextual(DeserializationContext context, BeanProperty property) {
this.targetType = property.getType().containedType(1).getRawClass();
return new QuotedObjectDeserializer<T>(this.targetType, this.mapper);
}
#Override
public T deserialize(JsonParser jp, DeserializationContext context) throws IOException {
JsonNode node = jp.getCodec().readTree(jp);
String text = node.toString();
String trimmed = text.substring(1, text.length() - 1);
trimmed = trimmed.replace("\\", "");
trimmed = trimmed.replace("\n", "");
JsonNode obj = this.mapper.readTree(trimmed);
return this.mapper.convertValue(obj, this.mapper.getTypeFactory().constructType(this.targetType));
}
}
Now I'm not sure how to actually use the deserializer, as annotating Settings.Foo with #JsonDeserialize(using = QuotedObjectDeserializer.class) obviously does not work.
Is there a way to annotate properties to use a generic custom deserializer? Or, perhaps more likely, is there a way to configure the default deserializers to handle the stringy objects returned in my example JSON?
Edit: The problem here is specifically deserializing settings.foo and settings.bar as objects. The JSON representation has these objects wrapped in quotes (and polluted with escape sequences), so they are deserialized as Strings.

Sorry about the length of the code here. There are plenty of shortcuts here (no encapsulation; added e to defaulte to avoid keyword etc.) but the intent is there
Model class:
package com.odwyer.rian.test;
import java.io.IOException;
import org.apache.commons.lang3.builder.ReflectionToStringBuilder;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class Model {
public String id;
public String name;
public Connections connections;
public Settings settings;
public static class Connections {
public String defaulte;
public String dev;
#Override
public String toString() {
return ReflectionToStringBuilder.toString(this);
}
}
public static class Foo {
public Foo () {}
#JsonCreator
public static Foo create(String str) throws JsonParseException, JsonMappingException, IOException {
return (new ObjectMapper()).readValue(str, Foo.class);
}
public Integer fooId;
public String token;
#Override
public String toString() {
return ReflectionToStringBuilder.toString(this);
}
}
public static class Bar {
public Bar() {}
#JsonCreator
public static Bar create(String str) throws JsonParseException, JsonMappingException, IOException {
return (new ObjectMapper()).readValue(str, Bar.class);
}
public Integer barId;
public String accountId;
#Override
public String toString() {
return ReflectionToStringBuilder.toString(this);
}
}
public static class Settings {
public Foo foo;
public Bar bar;
#Override
public String toString() {
return ReflectionToStringBuilder.toString(this);
}
}
#Override
public String toString() {
return ReflectionToStringBuilder.toString(this);
}
}
The caller:
package com.odwyer.rian.test;
import java.io.File;
import java.io.IOException;
import java.util.Scanner;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class TestClass {
private static ObjectMapper objectMapper = new ObjectMapper();
public static void main(String[] args) throws JsonParseException, JsonMappingException, IOException {
Scanner file = new Scanner(new File("test.json"));
String jsonStr = file.useDelimiter("\\Z").next();
Model model = objectMapper.readValue(jsonStr, Model.class);
System.out.println(model.toString());
}
}
The result (too much hassle to format out but it is all there!):
com.odwyer.rian.test.Model#190083e[id=abcd1234,name=test,connections=com.odwyer.rian.test.Model$Connections#170d1f3f[defaulte=http://foo.com/api/,dev=http://dev.foo.com/api/v2],settings=com.odwyer.rian.test.Model$Settings#5e7e6ceb[foo=com.odwyer.rian.test.Model$Foo#3e20e8c4[fooId=1,token=abc],bar=com.odwyer.rian.test.Model$Bar#6291bbb9[barId=2,accountId=d7cj3]]]
The key, courtesy of Ted and his post (https://stackoverflow.com/a/8369322/2960707) is the #JsonCreator annotation

Jackson Array Elements to POJO

I'm trying to deserialize an existing JSON document using Jackson, and was wondering if it was possible to perform the following translation without resorting to a stack of custom deserialization.
The input JSON looks like this:
{
"type": "foo",
"content": ["a", "b", {"some": "object", "goes": "here"}, 4]
}
The first 3 elements in content don't change, and are always String, String, SomeDataStructure, Integer (optional)
And I would like to deserialize into something like this:
class Foo {
public static class FooContent {
String some;
String goes;
}
String aString;
String bString;
FooContent content;
Integer cInt;
}
Now I've already come across BeanAsArrayDeserializer, which sounds like it might be what I want, but I can't seem to find anything even remotely like a piece of example code to get me started.
So, any ideas?

You can always implement your custom deserializer. See below source code:
import java.io.IOException;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
;
public class JacksonTest {
public static void main(String[] args) throws Exception {
String json = "{\"type\": \"foo\",\"content\": [\"a\", \"b\", {\"some\": \"object\", \"goes\": \"here\"}, 4]}";
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
mapper.configure(SerializationFeature.WRITE_EMPTY_JSON_ARRAYS, true);
Root pojo = mapper.readValue(json, Root.class);
System.out.println(pojo);
}
}
class Root {
public String type;
#JsonDeserialize(using = FooDeserializer.class)
public Foo content;
#Override
public String toString() {
return content.toString();
}
}
class Foo {
public static class FooContent {
public String some;
public String goes;
#Override
public String toString() {
return "{" + some + ", " + goes + "}";
}
}
String aString;
String bString;
FooContent content;
Integer cInt;
#Override
public String toString() {
return "Foo [aString=" + aString + ", bString=" + bString + ", content=" + content
+ ", cInt=" + cInt + "]";
}
}
class FooDeserializer extends JsonDeserializer<Foo> {
#Override
public Foo deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException,
JsonProcessingException {
Foo f = new Foo();
int index = 0;
while (jp.nextToken() != JsonToken.END_ARRAY) {
JsonToken token = jp.getCurrentToken();
if (token.isStructStart()) {
f.content = jp.readValueAs(Foo.FooContent.class);
index++;
} else {
switch (index++) {
case 0:
f.aString = jp.getText();
break;
case 1:
f.bString = jp.getText();
break;
case 3:
f.cInt = Integer.parseInt(jp.getText());
break;
}
}
}
return f;
}
}
Above app prints:
Foo [aString=a, bString=b, content={object, here}, cInt=4]

The feature that allows to use the BeanAsArrayDeserializer (internally) is #JsonFormat. You can read some information about it in this blog post.
I also just learned about it recently, but I think a mapping that works for your case would look something like the following. You'd need a wrapper to hold the content field though, which would then be of type Foo.
#JsonFormat(shape = Shape.ARRAY)
#JsonPropertyOrder({ "aString", "bString", "content", "cInt" })
class Foo {
public static class FooContent {
String some;
String goes;
}
#JsonFormat(shape = Shape.STRING)
String aString;
#JsonFormat(shape = Shape.STRING)
String bString;
FooContent content;
#JsonFormat(shape = Shape.NUMBER)
Integer cInt;
}