We Keep Coding

How to deserialize a JSON object to a Java collection using Jackson in a Spring application that registered the DefaultScalaModule?

I am authoring a Java library that provides REST endpoints through Spring controllers. The payload of one the endpoint is an instance of my JavaRoutine class, for which I provide a JSON serializer/deserializer pair. Here it is (slightly simplified):
#JsonSerialize(using = JavaRoutine.Serializer.class)
#JsonDeserialize(using = JavaRoutine.Deserializer.class)
public class JavaRoutine {
private final String jobId;
private final List<Object> inputValues;
private final List<ExpressionType> inputTypes; // ExpressionType is defined in my lib
public JavaRoutine(String jobId) {
this.jobId = jobId;
this.inputValues = new ArrayList<>();
this.inputTypes = new ArrayList<>();
}
public String getJobId() { return jobId; }
public void addInput(Object value) {
inputValues.add(value);
inputTypes.add(value == null ? null : ExpressionType.getTypeForValue(value));
}
public static class Serializer extends StdSerializer<JavaRoutine> {
private static final ObjectMapper mapper = new ObjectMapper();
public Serializer() {
super(JavaRoutine.class);
}
#Override
public void serialize(JavaRoutine routine, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeStartObject();
gen.writeStringField("jobId", routine.jobId);
gen.writeArrayFieldStart("inputs");
int inputCount = routine.inputValues.size();
for (int i = 0; i < inputCount; i++) {
gen.writeStartObject();
gen.writeStringField("type", mapper.writeValueAsString(routine.inputTypes.get(i)));
gen.writeStringField("value", mapper.writeValueAsString(routine.inputValues.get(i)));
gen.writeEndObject();
}
gen.writeEndArray();
gen.writeEndObject();
}
}
public static class Deserializer extends StdDeserializer<JavaRoutine> {
private static final ObjectMapper mapper = new ObjectMapper();
public Deserializer() {
super(JavaRoutine.class);
}
#Override
public JavaRoutine deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
Map<String, Object> fields = p.readValueAs(new TypeReference<Map<String, Object>>() {});
JavaRoutine routine = new JavaRoutine((String) fields.get("jobId");
List<Map<String, String>> inputs = (List<Map<String, String>>) fields.get("inputs");
for (Map<String, String> input: inputs) {
ExpressionType inputType = mapper.readValue(input.get("type"), ExpressionType.class);
Object inputValue = inputType == null ? null : mapper.readValue(input.get("value"), inputType.getJavaType());
routine.addInput(inputValue);
}
return routine;
}
}
}
This works. Except when the application that links the library has registered the Jackson module for Scala, which it needs for its own purpose. (In short, the aim of this Jackson module is to deserialize JSON structures into Scala collections and not into Java ones.) As a consequence, the call to p.readValueAs() deserializes the array of "inputs" as a Scala list, which causes the cast to List<Map<String, String>> two lines later to fail.
What solution would you recommend?

Have not tried you example. But running on Kubernetes with multiple Nodes in Google and got the the strange scala collections object when jumping between nodes.
This helped me half way.
Try creating mapper like below in (my guess) the Deserializer.
ObjectMapper mapper.registerModule(new DefaultScalaModule());
Also having some problems with the scala mappings. For me now the order is not kept. So Lists and Maps (LinkedHashMap) will loose the original order. :(

ObjectMapper truncates dates in Map

I have an ObjectMapper that is configured with the following DateTimeFormatter:
DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ").withZone(ZoneOffset.UTC);
I want to convert a map with OffsetDateTime in it to json. My problem is that apparently OffsetDateTime is converted correctly when it is the value
Map<Integer, OffsetDateTime> map = ImmutableMap.of(1, offsetDateTime);
System.out.println(mapper.writeValueAsString(map));
// {"1":"2019-03-20T16:46:00.000+0000"}
but not when it is the key
Map<OffsetDateTime, Integer> map = ImmutableMap.of(offsetDateTime, 1);
System.out.println(mapper.writeValueAsString(map).toString());
// {"2019-03-20T16:46Z":1}
Notice that the seconds and milliseconds were truncated. I have tested this with other maps, like HashMap or TreeMap, with the same results.
This is the definition of the ObjectMapper:
ObjectMapper mapper = new ObjectMapper();
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
mapper.setDateFormat(DateConstants.SIMPLE_DATE_FORMATTER);
mapper.registerModule(new JavaTimeModule());
SimpleModule instantModule = new SimpleModule();
instantModule.addDeserializer(OffsetDateTime.class, new OffsetDateTimeDeserializer());
instantModule.addSerializer(OffsetDateTime.class, new OffsetDateTimeSerializer());
mapper.registerModule(instantModule);
mapper.registerModule(offsetDateTimeModule);
and this serialiser:
public class OffsetDateTimeSerializer extends JsonSerializer<OffsetDateTime> {
#Override
public void serialize(OffsetDateTime value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
String str = DateConstants.FORMATTER.format(value);
gen.writeString(str);
}
}
Edit after assylias answer:
I have now changed my SimpleModule to
SimpleModule instantModule = new SimpleModule();
instantModule.addDeserializer(OffsetDateTime.class, new OffsetDateTimeDeserializer());
instantModule.addKeyDeserializer(OffsetDateTime.class, new OffsetDateTimeKeyDeserializer());
instantModule.addSerializer(OffsetDateTime.class, new OffsetDateTimeSerializer());
instantModule.addKeySerializer(OffsetDateTime.class, new OffsetDateTimeSerializer());
mapper.registerModule(instantModule);
with OffsetDateTimeKeyDeserialzer as
public class OffsetDateTimeKeyDeserializer extends KeyDeserializer {
#Override
public OffsetDateTime deserializeKey(String key, DeserializationContext ctxt) {
return OffsetDateTime.from(DateConstants.FORMATTER.parse(key));
}
}
When OffsetDateTime is the value, things continue to work correctly. However if it is the key, I now get
Exception in thread "main" com.fasterxml.jackson.core.JsonGenerationException: Can not write a string, expecting field name (context: Object)
at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:1961)
at com.fasterxml.jackson.core.json.JsonGeneratorImpl._reportCantWriteValueExpectName(JsonGeneratorImpl.java:244)
at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._verifyValueWrite(WriterBasedJsonGenerator.java:866)
at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator.writeString(WriterBasedJsonGenerator.java:368)
at com.brandwatch.signals.commons.util.jackson.OffsetDateTimeSerializer.serialize(OffsetDateTimeSerializer.java:16)
at com.brandwatch.signals.commons.util.jackson.OffsetDateTimeSerializer.serialize(OffsetDateTimeSerializer.java:12)
at com.fasterxml.jackson.databind.ser.std.StdKeySerializers$Dynamic.serialize(StdKeySerializers.java:225)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:707)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3905)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3219)
Edit 2:
This KeySerializer seems to have done the trick for me:
public class OffsetDateTimeKeySerializer extends StdSerializer<Object> {
public OffsetDateTimeKeySerializer() {
super(Object.class);
}
#Override
public void serialize(Object value, JsonGenerator gen, SerializerProvider provider) throws IOException {
if (value instanceof OffsetDateTime) {
String str = DateConstants.FORMATTER.format((OffsetDateTime) value);
gen.writeFieldName(str);
} else {
gen.writeFieldName(value.toString());
}
}
}

As you have discovered, this only applies the (de-)serializers to values. To apply the same serialization rules to keys, you need a key (de-)serializer:
instantModule.addKeyDeserializer(OffsetDateTime.class, new OffsetDateTimeDeserializer());
instantModule.addKeySerializer(OffsetDateTime.class, new OffsetDateTimeSerializer());
Note that the key deserializer needs a KeyDeserializer object, not a JsonDeserializer.
More info in the javadoc.

Jackson JsonDeserializer delegate deserialization of a field back to default deserializer of that field type

Given
public class ConstraintMatch {
protected String constraintName;
protected Score score;
...
}
I have the following serializer in Jackson:
public class ConstraintMatchJacksonJsonSerializer extends JsonSerializer<ConstraintMatch> {
#Override
public void serialize(ConstraintMatch constraintMatch, JsonGenerator generator, SerializerProvider serializers)
throws IOException {
generator.writeStartObject();
generator.writeStringField("constraintName", constraintMatch.getConstraintName());
generator.writeFieldName("score");
// Delegate to serialization to the default Score serializer
serializers.findValueSerializer(Score.class)
.serialize(constraintMatch.getScore(), generator, serializers);
generator.writeEndObject();
}
}
How do I write a deserializer that also delegates to the default deserializer?
public class ConstraintMatchJacksonJsonDeserializer extends JsonDeserializer<ConstraintMatch> {
#Override
public ConstraintMatch deserialize(JsonParser parser, DeserializationContext context) throws IOException {
JsonNode tree = parser.readValueAsTree();
String constraintName = tree.get("constraintName").asText();
JsonNode scoreNode = tree.get("score");
Score score = ...; // How do I delegate to the default deserializer?
return new ConstraintMatch(constraintName, score);
}
}
I've looked at findContextualValueDeserializer() etc, but I can't create a BeanProperty instance.

In a similar situation, I actually found there were two problems to solve. Firstly, as you say, the need to delegate back to the normal deserializer. But the other problem I encountered was how to feed the JsonNode (TreeNode below) into that next deserialize(JsonParser, ...).
The following is a working sample from that situation, where I wanted to do a lookahead to figure out the subclass.
Hopefully the node here is your scoreNode. And it sounds like objectClass is just Score.class for you.
#Override
public T deserialize(JsonParser parser, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
ObjectMapper mapper = (ObjectMapper) parser.getCodec();
TreeNode node = parser.readValueAsTree();
// Select the subclass to deserialize as
Class<? extends T> objectClass = deduceClass(node);
// This based on ObjectMapper._convert()
// - the problem here was the JsonParser (parser) had gone past the current node
TokenBuffer buf = new TokenBuffer(mapper, false);
SerializationConfig config = mapper.getSerializationConfig()
.without(SerializationFeature.WRAP_ROOT_VALUE);
DefaultSerializerProvider serializerProvider = ((DefaultSerializerProvider) mapper
.getSerializerProvider()).createInstance(config,
mapper.getSerializerFactory());
serializerProvider.serializeValue(buf, node);
JsonParser nestedParser = buf.asParser();
nestedParser.nextToken();
JsonDeserializer<Object> deserializer = ctxt
.findRootValueDeserializer(
mapper.getTypeFactory().constructType(objectClass));
#SuppressWarnings("unchecked")
T obj = (T) deserializer.deserialize(nestedParser, ctxt);
return obj;
}
(Just in case, this was with Jackson 2.7.9)
I'd be pleased to hear about a simpler way to create a JsonParser from a node.

Serializing this:
constraintMatch.getConstraintPackage());
generator.writeStringField("constraintName", constraintMatch.getConstraintName());
generator.writeFieldName("score");
// Delegate to PolymorphicScoreJacksonJsonSerializer
JsonSerializer<Object> scoreSerializer = serializers.findValueSerializer(Score.class);
scoreSerializer.serialize(constraintMatch.getScore(), generator, serializers);
generator.writeEndObject();
Can be deserialized with this:
parser.nextToken();
if (!"constraintName".equals(parser.getCurrentName())) {
throw new IllegalStateException(...);
}
parser.nextToken();
String constraintName = parser.getValueAsString();
parser.nextToken();
if (!"score".equals(parser.getCurrentName())) {
throw new IllegalStateException(...);
}
parser.nextToken();
JsonDeserializer<Object> scoreDeserializer = context.findNonContextualValueDeserializer(context.constructType(Score.class));
Score score = (Score) scoreDeserializer.deserialize(parser, context);

Jackson XML: how to serialize empty/null collections as empty node

I'm using Jackson XML 2.8.9 and unfortunately I cannot find any way to serialize empty/null collections as empty nodes.
Method responsible for serializing to XML:
protected byte[] toXml(final Collection<ReportView> reports) throws IOException
{
final XmlMapper mapper = new XmlMapper();
// place for code which will solve my problem
return mapper.writerWithDefaultPrettyPrinter().withRootName("report").writeValueAsBytes(reports);
}
I tried to use:
serialization inclusion:
mapper.setSerializationInclusion(JsonInclude.Include.ALWAYS);
serialization provider:
final XmlSerializerProvider provider = new XmlSerializerProvider(new XmlRootNameLookup());
provider.setNullValueSerializer(new JsonSerializer<Object>()
{
#Override
public void serialize(final Object value, final JsonGenerator jgen, final SerializerProvider provider) throws IOException
{
jgen.writeString("");
}
});
mapper.setSerializerProvider(provider);
Jackson 2.9.0 EMPTY_ELEMENT_AS_NULL feature:
mapper.configure(FromXmlParser.Feature.EMPTY_ELEMENT_AS_NULL, false);
Unfortunately nothing works. Does anybody know how to achieve it?
Test method:
#Test
public void testToXml() throws IOException
{
final Map<String, Object> payload = new LinkedHashMap<>();
payload.put("amp", "&");
payload.put("empty", Collections.emptyList());
final Date date = new Date();
final ReportView reportView = new ReportView(payload, date, "system");
// when
final byte[] xmlBytes = reportService.toXml(Arrays.asList(reportView));
// then
final StringBuilder expected = new StringBuilder();
expected.append("<report>");
expected.append(" <item>");
expected.append(" <payload>");
expected.append(" <amp>&</amp>");
expected.append(" <empty></empty>");
expected.append(" </payload>");
expected.append(" <timestamp>" + date.getTime() + "</timestamp>");
expected.append(" <changingUser>system</changingUser>");
expected.append(" </item>");
expected.append("</report>");
final String xmlText = new String(xmlBytes).replace("\n", "").replace("\r", "");
assertThat(xmlText).isEqualTo(expected.toString());
}
ReportView class:
public class ReportView {
private final Map<String, Object> payload;
private final Date timestamp;
private final String changingUser;
public ReportView(Map<String, Object> payload, Date timestamp, String changingUser) {
this.payload = payload;
this.timestamp= timestamp;
this.changingUser = changingUser;
}
public String getChangingUser() {
return changingUser;
}
public Date getTimestamp() {
return timestamp;
}
public Map<String, Object> getPayload() {
return payload;
}
}
I prepared a repository with example code: https://github.com/agabrys/bugs-reports/tree/master/jackson-xml/empty-elements-serialization
EDIT:
I extended the test toXml method and did some code cleanup.
I also tried to create a solution based on Module and SerializerModifier. Unfortunately both ended with failure. I created an issue in jackson-dataformat-xml backlog:
NPE after overriding map serializer with custom implementation (XmlBeanSerializerModifier.modifyMapSerializer)
EDIT:
I've got a hint how to solve problem with exception (see NPE after overriding map serializer with custom implementation (XmlBeanSerializerModifier.modifyMapSerializer)) but still it does not solve problem with missing empty/null values.

I needed to tackle the same issue, and here's how I got it working:
First I create a serializer that serializes nulls as empty string:
public class NullAsEmptyStringSerializer extends JsonSerializer<Object> {
static final JsonSerializer<Object> INSTANCE = new NullAsEmptyStringSerializer();
private static final String EMPTY_STRING = "";
private final StringSerializer stringSerializer = new StringSerializer();
#Override
public void serialize(Object value, JsonGenerator gen, SerializerProvider serializers)
throws IOException {
stringSerializer.serialize(EMPTY_STRING, gen, serializers);
}
}
Then I create a serializer modifier, that overwrites the null serializer of the bean properties with my new serializer:
public class NullToEmptyPropertySerializerModifier extends BeanSerializerModifier {
#Override
public List<BeanPropertyWriter> changeProperties(SerializationConfig config,
BeanDescription beanDesc, List<BeanPropertyWriter> beanProperties) {
for (BeanPropertyWriter beanProperty : beanProperties) {
beanProperty.assignNullSerializer(NullAsEmptyStringSerializer.INSTANCE);
}
return beanProperties;
}
}
And finally, I configure the xml mapper to use my modifier:
NullToEmptyPropertySerializerModifier modifier = new NullToEmptyPropertySerializerModifier();
SerializerFactory serializerFactory = BeanSerializerFactory.instance.withSerializerModifier(modifier);
XmlMapper xmlMapper = new XmlMapper();
xmlMapper.setSerializerFactory(serializerFactory);
Trying to see if it's working for strings and objects (Person and Dog are dummy data holder objects):
Dog dog = new Dog("bobby");
Person person = new Person("utku", null, 29, null);
String serialized = xmlMapper.writeValueAsString(person);
System.out.println(serialized);
Gives the following output:
<Person><name>utku</name><address></address><age>29</age><dog></dog></Person>

Jackson Modules for Map Serialization

I have a Class that contains a Map (with non String key) and some other fields.
public class MyClass() {
private Map<KeyObject, OtherObject> map;
private String someField;
public MyClass(Map<KeyObject, OtherObject> map, String someField) {
this.map = map;
this.someField = someField;
}
// Getters & Setters
}
I would like to serialize and deserialize this class using Jackson.
I saw a different ways of doing that and decided to try using jackson modules.
I followed this post and extended JsonDeserializer and JsonSerializer. The problem is that those classes should be typed, so it should look like
public class keyDeserializer extends JsonDeserializer<Map<KeyObject, OtherObject>> {
...
}
The same for the KeySerializer.
Then adding to the module:
module.addSerializer(new keySerializer());
module.addDeserializer(Map.class, new keyDeserializer());
But this is wrong apparently since I'm getting an exception:
keySerializer does not define valid handledType() -- must either register with method that takes type argument or make serializer extend 'org.codehaus.jackson.map.ser.std.SerializerBase'
I could have my serializer and deserializer to be typed to MyClass, but then I had to manually parse all of it, which is not reasonable.
UPDATE:
I managed to bypass the module creation in the code by using annotations
#JsonDeserialize(using = keyDeserializer.class)
#JsonSerialize(using = keySerializer.class)
private Map<KeyObject, OtherObject> map;
But then I have to serialize/deserialize the whole map structure on my own from the toString() output. So tried a different annotation:
#JsonDeserialize(keyUsing = MyKeyDeserializer.class)
private Map<KeyObject, OtherObject> map;
Where MyKeyDeserializer extends org.codehaus.jackson.map.KeyDeserializer and overriding the method
public Object deserializeKey(String key, DeserializationContext ctxt) throws IOException, JsonProcessingException {...}
Then manually deserializing my key but again from the toString() output of my key class.
This is not optimal (this dependency on the toString() method). Is there a better way?

Ended up using this serializer:
public class MapKeySerializer extends SerializerBase<Object> {
private static final SerializerBase<Object> DEFAULT = new StdKeySerializer();
private static final ObjectMapper mapper = new ObjectMapper();
protected MapKeySerializer() {
super(Object.class);
}
#Override
public JsonNode getSchema(SerializerProvider provider, Type typeHint) throws JsonMappingException {
return DEFAULT.getSchema(provider, typeHint);
}
#Override
public void serialize(Object value, JsonGenerator jgen, SerializerProvider provider) throws IOException, JsonGenerationException {
if (null == value) {
throw new JsonGenerationException("Could not serialize object to json, input object to serialize is null");
}
StringWriter writer = new StringWriter();
mapper.writeValue(writer, value);
jgen.writeFieldName(writer.toString());
}
}
And this Deserializer:
public class MapKeyDeserializer extends KeyDeserializer {
private static final ObjectMapper mapper = new ObjectMapper();
#Override
public Object deserializeKey(String key, DeserializationContext ctxt) throws IOException, JsonProcessingException {
return mapper.readValue(key, MyObject.class);
}
}
Annotated my Map:
#JsonDeserialize(keyUsing = MapKeyDeserializer.class)
#JsonSerialize(keyUsing = MapKeySerializer.class)
private Map<KeyObject, OtherObject> map;
This is the solution that worked for me, hope this helps other.