I'm attempting to read a JSON file using Jackson and store one of the fields that is stored as a epoch milliseconds as a Java Instant, however deserialization is not behaving as expected.
Here is what I am seeing when trying to read the timestamp:
1503115200000
Jackson is setting the Instant field as +49601-10-28T16:00:00Z.
This appears to be occurring because Jackson's default is to read the timestamp with Instant.ofEpochSecond(Long l) instead of Instant.ofEpochMilli(Long l).
Is there a way to set the Jackson ObjectMapper to use the ofEpochMilli method instead? This is what I currently have for my ObjectMapper:
ObjectMapper om = new ObjectMapper()
.registerModule(new JavaTimeModule())
.configure(SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS, false)
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.setSerializationInclusion(Include.NON_NULL);
Note
If I change the input JSON to ISO date such as 2017-08-19T04:00:00Z or to epoch seconds such as 1503115200 the Instant field is able to set properly.
Unfortunately the JSON input must be epoch milliseconds e.g. 1503115200000.
Solution was to add .configure(DeserializationFeature.READ_DATE_TIMESTAMPS_AS_NANOSECONDS, false) to the ObjectMapper. Complete ObjectMapper looks like:
ObjectMapper om = new ObjectMapper()
.registerModule(new JavaTimeModule())
.configure(SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS, false)
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(DeserializationFeature.READ_DATE_TIMESTAMPS_AS_NANOSECONDS, false)
.setSerializationInclusion(Include.NON_NULL);
For those who use Spring Boot and experience this issue during rest request deserialization - provide this in the application.properties:
spring.jackson.deserialization.read-date-timestamps-as-nanoseconds=false
#JB Nizet answer is correct, but since Spring Boot has it's own ObjectMapper inside - you need to configure this value in the properties.
From https://github.com/FasterXML/jackson-modules-java8/blob/master/datetime/src/main/java/com/fasterxml/jackson/datatype/jsr310/JavaTimeModule.java:
The more ambiguous integer types are read as fractional seconds without a decimal point if {#code READ_DATE_TIMESTAMPS_AS_NANOSECONDS} is enabled (it is by default), and otherwise they are read as milliseconds.
So you need to disable READ_DATE_TIMESTAMPS_AS_NANOSECONDS.
A more flexible solution could be to write your own convertor class, if you don't want to edit or create an object mapper.
public class LongToInstantConverter extends StdConverter<Long, Instant> {
public Instant convert(final Long value) {
return Instant.ofEpochMilli(value);
}
}
You just need to add the annotation in your class
#JsonDeserialize(converter = LongToInstantConverter.class)
private Instant sentDate;
Related
If I make a POJO class and convert the whole objects to Json using the mapper.writeValueAsString() method, or let Spring handle the conversion, the JavaTimeModule is used appropriately and If my POJO had an OffsetDateTime in it, it will get converted to a string like 2018-10-16T13:49:34.564748+02:00.
For some scenarios, I need to use the mapper's T convertValue(Object fromValue, Class<T> toValueType) method and build an ObjectNode by hand. The method does not convert the OffsetDateTime objects the same way as the writeValueAsString.
I construct my Jackson mapper like this:
ObjectMapper defaultMapperObj = new ObjectMapper();
defaultMapperObj.registerModule(new GuavaModule());
defaultMapperObj.registerModule(new Jdk8Module());
defaultMapperObj.registerModule(new JavaTimeModule());
defaultMapperObj.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
Using the mapper, I can easily convert even Lists to their Json strings:
mapper.convertValue(
Arrays.asList(1,2,23), JsonNode.class
)
Results in:
ArrayNode: [1,2,23]
But If I call this:
mappper.convertValue(OffsetDateTime.now(), JsonNode.class)
I get a DecimalNode: 1542287917.2141993
From the JavaTimeModule documentation:
Most java.time types are serialized as numbers (integers or decimals as appropriate) if the SerializationFeature.WRITE_DATES_AS_TIMESTAMPS feature is enabled, and otherwise are serialized in standard ISO-8601 string representation. [...]
So, to achieve the desired result, disable the SerializationFeature.WRITE_DATES_AS_TIMESTAMPS feature in your ObjectMapper instance:
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
I'm looking at a REST API which uses DropWizard. DropWizard appears to somehow magically know how to serialize joda DateTime objects to a string formatted like this:
YYYY-MM-DD HH:MM:SS
But if I write a test which uses an ObjectMapper to serialize a joda DateTime object, I get something like:
{"month_of_year":1,"hour_of_day":0,"minute_of_hour":15,"second_of_minute":3,"millis_of_second":0,"week_of_weekyear":3,"weekyear":2018,"year_of_era":2018,"year_of_century":18,"century_of_era":20,"millis_of_day":903000,"second_of_day":903,"minute_of_day":15,"day_of_year":18,"day_of_week":4,"day_of_month":18,"year":2018,"era":1,"millis":1516259703000,"chronology":{"zone":{"uncached_zone":{"cachable":true,"fixed":false,"id":"America/Denver"},"fixed":false,"id":"America/Denver"}},"zone":{"uncached_zone":{"cachable":true,"fixed":false,"id":"America/Denver"},"fixed":false,"id":"America/Denver"},"after_now":true,"before_now":false,"equal_now":false}
How does DropWizard know how to serialize this?
The field I'm serialized is annotated like this:
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "YYYY-MM-dd HH:mm:ss")
DateTime myDate;
I understand that I'm supplying a format string there, but I'd like to understand why ObjectMapper (in my test) doesn't respect it, while DropWizard does. (The annotation is a Jackson annotation, not a DropWizard one, so it confuses me that ObjectMapper.writeValue() wouldn't respect it).
When looking at the source of the #JsonFormat annotation, I see:
/**
* Datatype-specific additional piece of configuration that may be used
* to further refine formatting aspects. This may, for example, determine
* low-level format String used for {#link java.util.Date} serialization;
* however, exact use is determined by specific <code>JsonSerializer</code>
*/
public String pattern() default "";
This leads me to believe that Jackson doesn't actually supply the JsonSerializer that recognizes this pattern attribute, and at the same time leads me to believe that DropWizard does have such a JsonSerializer. How can I configure/modify/override the JsonSerializer that DropWizard is apparently using?
Your test is not using the right ObjectMapper. DW in its bootstrap will configure the ObjectMapper for you with reasonable defaults (that's DW's whole thing - everything is somewhat reasonable and pre-configured).
So, in your case, your test would need to use Jackson.newObjectMapper(). Try this:
public static void main(String[] args) throws JsonProcessingException {
ObjectMapper myMapper= new ObjectMapper();
ObjectMapper dwMapper = Jackson.newObjectMapper();
System.out.println(myMapper.writeValueAsString(new X()));
System.out.println(dwMapper.writeValueAsString(new X()));
}
public static class X {
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "YYYY-MM-dd HH:mm:ss")
DateTime now = DateTime.now();
}
Will print:
{"now":{"era":1,"dayOfYear":18,"dayOfWeek":4,"dayOfMonth":18,"year":2018,"weekOfWeekyear":3,"millisOfDay":37615193,"monthOfYear":1,"hourOfDay":10,"minuteOfHour":26,"secondOfMinute":55,"millisOfSecond":193,"weekyear":2018,"yearOfEra":2018,"yearOfCentury":18,"centuryOfEra":20,"secondOfDay":37615,"minuteOfDay":626,"zone":{"fixed":false,"uncachedZone":{"cachable":true,"fixed":false,"id":"Europe/London"},"id":"Europe/London"},"millis":1516271215193,"chronology":{"zone":{"fixed":false,"uncachedZone":{"cachable":true,"fixed":false,"id":"Europe/London"},"id":"Europe/London"}},"afterNow":false,"beforeNow":true,"equalNow":false}}
{"now":"2018-01-18 10:26:55"}
And for completeness, these are the modules that are registered for you:
private static ObjectMapper configure(ObjectMapper mapper) {
mapper.registerModule(new GuavaModule());
mapper.registerModule(new LogbackModule());
mapper.registerModule(new GuavaExtrasModule());
mapper.registerModule(new JodaModule());
mapper.registerModule(new AfterburnerModule());
mapper.registerModule(new FuzzyEnumModule());
mapper.registerModules(new Jdk8Module());
mapper.registerModules(new JavaTimeModule());
mapper.setPropertyNamingStrategy(new AnnotationSensitivePropertyNamingStrategy());
mapper.setSubtypeResolver(new DiscoverableSubtypeResolver());
return mapper;
}
These are all overwritable in the bootstrap.
Please not that I might be on an older version of DW, so it might be slightly different in the latest.
Artur
I have such string:
{
"debug":"false",
"switchTime":"2017-04-12 17:04:42.896026"
}
I'm trying to get object in such approach:
new ObjectMapper().readValue(string, MyObject.class);
And MyObject class:
class MyObject {
private Boolean debug;
private Timestamp switchTime;
//...getters, setters, constructors
}
I have such exception:
com.fasterxml.jackson.databind.exc.InvalidFormatException:
Can not deserialize value of type java.sql.Timestamp from String
"2017-04-12 17:04:42.896026": not a valid representation (error:
Failed to parse Date value '2017-04-12 17:04:42.896026':
Can not parse date "2017-04-12 17:04:42.896026Z": while it seems
to fit format 'yyyy-MM-dd'T'HH:mm:ss.SSS'Z'',
parsing fails (leniency? null)) at [Source:
{"debug":"false", "switchTime":"2017-04-12 17:04:42.896026"};
I don't understand why...If i use in debug mode Timestamp.valueOf() with "2017-04-12 17:04:42.896026" - i have success
I think you need to set the expected date/time format using #JsonFormat annotation as shown below.
class MyObject {
private Boolean debug;
#JsonFormat(pattern="yyyy-MM-dd HH:mm:ss.SSS")
private Timestamp switchTime;
//...getters, setters, constructors
}
You can also set timezone as #JsonFormat(pattern="yyyy-MM-dd HH:mm:ss.SSS",timezone="PST")
I faced the similar problem when I was using lombok in the POJO class which has #Builder and #Value annotations
I have also added the annotation AllArgsConstructor and then it is working fine with my custom deserializer code
The value that you see in debug mode is "toString()" version of actual value of timestamp, so don't rely on what you inspect in debug mode.
You can use #JsonFormat annotation that helps you to convert your timestamp with specified format. You need to take care of timezones also while converting!
I'm implementing a RESTful web service where user has to send a signed verification token along with the request so that I could ensure that the request has not been tampered by a middle man. My current implementation is as follows.
Verification token is a VerifData object serialized into a String and then hashed and encrypted.
class VerifData {
int prop1;
int prop2;
}
In my service, I put data to be serialized into an instance of VerifData and then serialize it using Jackson ObjectMapper and passed along to the verification engine along with the verification token.
VerfiData verifData = new VerifData(12345, 67890);
ObjectMapper mapper = new ObjectMapper();
String verifCodeGenerated = mapper.writeValueAsString(verifData);
But it seems that each time the application container is started, the order of properties being mapped into a string by ObjectMapper changes.
Ex: one time it would be
{"prop1":12345,"prop2":67890}
and another time it would be
{"prop2":67890,"prop1":12345}
So if client has serialized the VerifData instance as into the first String, there is 50% chance of it being failed even though it is correct.
Is there a way to get around this? Can I specify the order of properties to map by ObjectMapper (like in ascending order)? Or is there any other way to best implement this verification step. Both client and server implementations are developed by me. I use Java Security API for signing and verifying.
The annotations are useful, but can be a pain to apply everywhere. You can configure your whole ObjectMapper to work this way with
Current Jackson versions:
objectMapper.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
Older Jackson versions:
objectMapper.configure(SerializationConfig.Feature.SORT_PROPERTIES_ALPHABETICALLY, true);
From the Jackson Annotations documentation:
// ensure that "id" and "name" are output before other properties
#JsonPropertyOrder({ "id", "name" })
// order any properties that don't have explicit setting using alphabetic order
#JsonPropertyOrder(alphabetic=true)
The following 2 ObjectMapper configuration are required:
ObjectMapper.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
or
ObjectMapper.enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY)
defines the property serialization order used for POJO fields
Note: does not apply to java.util.Map serialization!
and
ObjectMapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true)
or
ObjectMapper.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS)
Feature that determines whether java.util.Map entries are first sorted by key before serialization
Spring Boot config example (yaml):
spring:
jackson:
mapper:
SORT_PROPERTIES_ALPHABETICALLY: true
serialization:
ORDER_MAP_ENTRIES_BY_KEYS: true
In Spring Boot you can add this behaviour globally by adding the following to your Application entry point class:
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.featuresToEnable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
return builder;
}
In Jackson 2.x, which you are probably using today, use:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
If you care about looks, you may also consider SerializationFeature.INDENT_OUTPUT as well.
Note that you must serialize Maps or Objects for this to sort correctly. If you serialize a JsonNode for example (from readTree), that won't be properly indented.
Example
import com.fasterxml.jackson.databind.*;
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
String input = "{\"hello\": {\"cruel\" : \"world\"} }";
Object pojo = mapper.readValue(input, Object.class);
System.out.println(mapper.writeValueAsString(pojo));
results in:
{
"hello" : {
"cruel" : "world"
}
}
There is an easier way in Spring Boot by specifying a property (in application.properties for example:
spring.jackson.mapper.sort_properties_alphabetically=true
From Duncan McGregor's answer:
Its better to use it like this:
objectMapper.configure(SerializationConfig.Feature.SORT_PROPERTIES_ALPHABETICALLY, true);
as MapperFeature is for XMLs and comes with jackson-databind which is not required...
I discovered yet another way today in case alphabetic is not your desired sorting order. It turns out adding a #JsonProperty annotation on a field places it last when writing if the rest of the fields are not annotated. I discovered that when I wanted to specify a property name which did not conform to java naming conventions.
By Adding an index attribute you can define the order. Lowest index is placed first.
#JsonProperty(index=20)
String prop1;
#JsonProperty(index=10)
String prop2;
Would render:
{"prop2": "valueProp2", "prop1": "valueProp1"}
You can use mix-in and specify the order of properties as you like:
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
#Component
public final class ObjectMapperUtils {
private static final ObjectMapper MAPPER = new ObjectMapper();
static {
MAPPER.addMixIn(Object.class, IdFirst.class);
}
#Bean
public ObjectMapper objectMapper() {
return MAPPER;
}
#JsonPropertyOrder({"id", "...", "..."})
private abstract static class IdFirst {}
}
I realize this is an old thread, but since I was looking or an answer and landed here, some additional info could be handy for other people.
The #JsonProperty annotation I am using currently (jackson-annotations-2.11.2) accepts, besides the "value" argument, an "index" (numeric) argument that specifies the order of the fields during serialization.
Instead of using flag argument:
objectMapper.enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
As #Gary Rowe mentioned, we can use Jackson2ObjectMapperBuilder to sort the properties globally.
However for this to work, you must have Jackson2ObjectMapperBuilder in your classpath. It is not part of the Jackson library.
As per this documentation, spring-web dependency has Jackson2ObjectMapperBuilder file and should be in your classpath.
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.featuresToEnable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
return builder;
}
You can refer to this for other possible solutions
I'm implementing a RESTful web service where user has to send a signed verification token along with the request so that I could ensure that the request has not been tampered by a middle man. My current implementation is as follows.
Verification token is a VerifData object serialized into a String and then hashed and encrypted.
class VerifData {
int prop1;
int prop2;
}
In my service, I put data to be serialized into an instance of VerifData and then serialize it using Jackson ObjectMapper and passed along to the verification engine along with the verification token.
VerfiData verifData = new VerifData(12345, 67890);
ObjectMapper mapper = new ObjectMapper();
String verifCodeGenerated = mapper.writeValueAsString(verifData);
But it seems that each time the application container is started, the order of properties being mapped into a string by ObjectMapper changes.
Ex: one time it would be
{"prop1":12345,"prop2":67890}
and another time it would be
{"prop2":67890,"prop1":12345}
So if client has serialized the VerifData instance as into the first String, there is 50% chance of it being failed even though it is correct.
Is there a way to get around this? Can I specify the order of properties to map by ObjectMapper (like in ascending order)? Or is there any other way to best implement this verification step. Both client and server implementations are developed by me. I use Java Security API for signing and verifying.
The annotations are useful, but can be a pain to apply everywhere. You can configure your whole ObjectMapper to work this way with
Current Jackson versions:
objectMapper.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
Older Jackson versions:
objectMapper.configure(SerializationConfig.Feature.SORT_PROPERTIES_ALPHABETICALLY, true);
From the Jackson Annotations documentation:
// ensure that "id" and "name" are output before other properties
#JsonPropertyOrder({ "id", "name" })
// order any properties that don't have explicit setting using alphabetic order
#JsonPropertyOrder(alphabetic=true)
The following 2 ObjectMapper configuration are required:
ObjectMapper.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
or
ObjectMapper.enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY)
defines the property serialization order used for POJO fields
Note: does not apply to java.util.Map serialization!
and
ObjectMapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true)
or
ObjectMapper.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS)
Feature that determines whether java.util.Map entries are first sorted by key before serialization
Spring Boot config example (yaml):
spring:
jackson:
mapper:
SORT_PROPERTIES_ALPHABETICALLY: true
serialization:
ORDER_MAP_ENTRIES_BY_KEYS: true
In Spring Boot you can add this behaviour globally by adding the following to your Application entry point class:
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.featuresToEnable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
return builder;
}
In Jackson 2.x, which you are probably using today, use:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
If you care about looks, you may also consider SerializationFeature.INDENT_OUTPUT as well.
Note that you must serialize Maps or Objects for this to sort correctly. If you serialize a JsonNode for example (from readTree), that won't be properly indented.
Example
import com.fasterxml.jackson.databind.*;
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
String input = "{\"hello\": {\"cruel\" : \"world\"} }";
Object pojo = mapper.readValue(input, Object.class);
System.out.println(mapper.writeValueAsString(pojo));
results in:
{
"hello" : {
"cruel" : "world"
}
}
There is an easier way in Spring Boot by specifying a property (in application.properties for example:
spring.jackson.mapper.sort_properties_alphabetically=true
From Duncan McGregor's answer:
Its better to use it like this:
objectMapper.configure(SerializationConfig.Feature.SORT_PROPERTIES_ALPHABETICALLY, true);
as MapperFeature is for XMLs and comes with jackson-databind which is not required...
I discovered yet another way today in case alphabetic is not your desired sorting order. It turns out adding a #JsonProperty annotation on a field places it last when writing if the rest of the fields are not annotated. I discovered that when I wanted to specify a property name which did not conform to java naming conventions.
By Adding an index attribute you can define the order. Lowest index is placed first.
#JsonProperty(index=20)
String prop1;
#JsonProperty(index=10)
String prop2;
Would render:
{"prop2": "valueProp2", "prop1": "valueProp1"}
You can use mix-in and specify the order of properties as you like:
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
#Component
public final class ObjectMapperUtils {
private static final ObjectMapper MAPPER = new ObjectMapper();
static {
MAPPER.addMixIn(Object.class, IdFirst.class);
}
#Bean
public ObjectMapper objectMapper() {
return MAPPER;
}
#JsonPropertyOrder({"id", "...", "..."})
private abstract static class IdFirst {}
}
I realize this is an old thread, but since I was looking or an answer and landed here, some additional info could be handy for other people.
The #JsonProperty annotation I am using currently (jackson-annotations-2.11.2) accepts, besides the "value" argument, an "index" (numeric) argument that specifies the order of the fields during serialization.
Instead of using flag argument:
objectMapper.enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
As #Gary Rowe mentioned, we can use Jackson2ObjectMapperBuilder to sort the properties globally.
However for this to work, you must have Jackson2ObjectMapperBuilder in your classpath. It is not part of the Jackson library.
As per this documentation, spring-web dependency has Jackson2ObjectMapperBuilder file and should be in your classpath.
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.featuresToEnable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
return builder;
}
You can refer to this for other possible solutions