So for this little program I'm writing I'm looking to parse Twitter's tweet stream. Im using the Gson library which works nice. Gson couldn't parse Twitters created_at datetime field, so I had to write a custom JsonDserializer that needs to be registered with the parser through the GsonBuilderas follows:
new GsonBuilder().registerTypeAdatapter(DateTime.class, <myCustomDeserializerType>)
Now my deserializer works well, and I am able to parse Twitter's stream.
However, I'm trying to cover as much of my program with unit tests, so this custom deserializer should be included.
Since a good unit test is a nicely isolated test, I do not want to register it with a Gson object after which I would parse a json string. What I do want is to create an instance of my deserializer and just pass a generic string representing a datetime, so that I could test the deserializer without it being integrated with anything else.
The signature of the deserialize method of a JsonDeserializer is as follows:
deserialize(JsonElement jsonElement, Type type, JsonDeserializationContext jsonDeserializationContext)
Let's say I want to parse the following data: 'Mon Mar 27 14:09:47 +0000 2017'. How would I have to transform my input data in order to correctly test my deserializer.
I'm not looking for code that actually parses this date, I already have that part covered. I'm asking how I can meet the deserialize method's signature so that I can simulate it's use in a Gson it is used in.
JsonSerializer and JsonDeserializer are tightly bound to Gson JSON tree model and a specific Gson configuration (de)serialization context that provides a set of types that can be (de)serialized. Because of this, accomplishing unit tests for JsonSerializer and JsonDeserializer are rather hard than easy.
Consider the following JSON document somewhere in your src/test/resources/.../zoned-date-time.json:
"Mon Mar 27 14:09:47 +0000 2017"
This is a perfectly valid JSON document, and it has nothing except of a single string for simplicity. A date/time formatter for the format above can be implemented in Java 8 as follows:
final class CustomPatterns {
private CustomPatterns() {
}
private static final Map<Long, String> dayOfWeek = ImmutableMap.<Long, String>builder()
.put(1L, "Mon")
.put(2L, "Tue")
.put(3L, "Wed")
.put(4L, "Thu")
.put(5L, "Fri")
.put(6L, "Sat")
.put(7L, "Sun")
.build();
private static final Map<Long, String> monthOfYear = ImmutableMap.<Long, String>builder()
.put(1L, "Jan")
.put(2L, "Feb")
.put(3L, "Mar")
.put(4L, "Apr")
.put(5L, "May")
.put(6L, "Jun")
.put(7L, "Jul")
.put(8L, "Aug")
.put(9L, "Sep")
.put(10L, "Oct")
.put(11L, "Nov")
.put(12L, "Dec")
.build();
static final DateTimeFormatter customDateTimeFormatter = new DateTimeFormatterBuilder()
.appendText(DAY_OF_WEEK, dayOfWeek)
.appendLiteral(' ')
.appendText(MONTH_OF_YEAR, monthOfYear)
.appendLiteral(' ')
.appendValue(DAY_OF_MONTH, 1, 2, NOT_NEGATIVE)
.appendLiteral(' ')
.appendValue(HOUR_OF_DAY, 2)
.appendLiteral(':')
.appendValue(MINUTE_OF_HOUR, 2)
.appendLiteral(':')
.appendValue(SECOND_OF_MINUTE, 2)
.appendLiteral(' ')
.appendOffset("+HHMM", "+0000")
.appendLiteral(' ')
.appendValue(YEAR)
.toFormatter();
}
Now consider the following JSON deserializer for ZonedDateTime:
final class ZonedDateTimeJsonDeserializer
implements JsonDeserializer<ZonedDateTime> {
private static final JsonDeserializer<ZonedDateTime> zonedDateTimeJsonDeserializer = new ZonedDateTimeJsonDeserializer();
private ZonedDateTimeJsonDeserializer() {
}
static JsonDeserializer<ZonedDateTime> getZonedDateTimeJsonDeserializer() {
return zonedDateTimeJsonDeserializer;
}
#Override
public ZonedDateTime deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
throws JsonParseException {
try {
final String s = context.deserialize(jsonElement, String.class);
return ZonedDateTime.parse(s, customDateTimeFormatter);
} catch ( final DateTimeParseException ex ) {
throw new JsonParseException(ex);
}
}
}
Note that I'm deserialiazing a string via the context by intention to accent that more complex JsonDeserializer instances may depend on it heavily. Now let's make some JUnit tests to test it:
public final class ZonedDateTimeJsonDeserializerTest {
private static final TypeToken<ZonedDateTime> zonedDateTimeTypeToken = new TypeToken<ZonedDateTime>() {
};
private static final ZonedDateTime expectedZonedDateTime = ZonedDateTime.of(2017, 3, 27, 14, 9, 47, 0, UTC);
#Test
public void testDeserializeIndirectlyViaAutomaticTypeAdapterBinding()
throws IOException {
final JsonDeserializer<ZonedDateTime> unit = getZonedDateTimeJsonDeserializer();
final Gson gson = new GsonBuilder()
.registerTypeAdapter(ZonedDateTime.class, unit)
.create();
try ( final JsonReader jsonReader = getPackageResourceJsonReader(ZonedDateTimeJsonDeserializerTest.class, "zoned-date-time.json") ) {
final ZonedDateTime actualZonedDateTime = gson.fromJson(jsonReader, ZonedDateTime.class);
assertThat(actualZonedDateTime, is(expectedZonedDateTime));
}
}
#Test
public void testDeserializeIndirectlyViaManualTypeAdapterBinding()
throws IOException {
final JsonDeserializer<ZonedDateTime> unit = getZonedDateTimeJsonDeserializer();
final Gson gson = new Gson();
final TypeAdapterFactory typeAdapterFactory = newFactoryWithMatchRawType(zonedDateTimeTypeToken, unit);
final TypeAdapter<ZonedDateTime> dateTypeAdapter = typeAdapterFactory.create(gson, zonedDateTimeTypeToken);
try ( final JsonReader jsonReader = getPackageResourceJsonReader(ZonedDateTimeJsonDeserializerTest.class, "zoned-date-time.json") ) {
final ZonedDateTime actualZonedDateTime = dateTypeAdapter.read(jsonReader);
assertThat(actualZonedDateTime, is(expectedZonedDateTime));
}
}
#Test
public void testDeserializeDirectlyWithMockedContext()
throws IOException {
final JsonDeserializer<ZonedDateTime> unit = getZonedDateTimeJsonDeserializer();
final JsonDeserializationContext mockContext = mock(JsonDeserializationContext.class);
when(mockContext.deserialize(any(JsonElement.class), eq(String.class))).thenAnswer(iom -> {
final JsonElement jsonElement = (JsonElement) iom.getArguments()[0];
return jsonElement.getAsJsonPrimitive().getAsString();
});
final JsonParser parser = new JsonParser();
try ( final JsonReader jsonReader = getPackageResourceJsonReader(ZonedDateTimeJsonDeserializerTest.class, "zoned-date-time.json") ) {
final JsonElement jsonElement = parser.parse(jsonReader);
final ZonedDateTime actualZonedDateTime = unit.deserialize(jsonElement, ZonedDateTime.class, mockContext);
assertThat(actualZonedDateTime, is(expectedZonedDateTime));
}
verify(mockContext).deserialize(any(JsonPrimitive.class), eq(String.class));
verifyNoMoreInteractions(mockContext);
}
}
Note that each test here requires some Gson configuration to be built in order to let the deserialization context work, or the latter must be mocked. Pretty much to test a simple unit.
An alternative to the JSON tree model in Gson is stream-oriented type adapters that that do not require the entire JSON tree to be constructed, so you can easily read or write directly from/to JSON streams making your (de)serialization faster and less memory consuming. Especially, for simple cases like what trivial string<==>FooBar conversions are.
final class ZonedDateTimeTypeAdapter
extends TypeAdapter<ZonedDateTime> {
private static final TypeAdapter<ZonedDateTime> zonedDateTimeTypeAdapter = new ZonedDateTimeTypeAdapter().nullSafe();
private ZonedDateTimeTypeAdapter() {
}
static TypeAdapter<ZonedDateTime> getZonedDateTimeTypeAdapter() {
return zonedDateTimeTypeAdapter;
}
#Override
public void write(final JsonWriter out, final ZonedDateTime zonedDateTime) {
throw new UnsupportedOperationException();
}
#Override
public ZonedDateTime read(final JsonReader in)
throws IOException {
try {
final String s = in.nextString();
return ZonedDateTime.parse(s, customDateTimeFormatter);
} catch ( final DateTimeParseException ex ) {
throw new JsonParseException(ex);
}
}
}
And here is a simple unit test for the type adapter above:
public final class ZonedDateTimeTypeAdapterTest {
private static final ZonedDateTime expectedZonedDateTime = ZonedDateTime.of(2017, 3, 27, 14, 9, 47, 0, UTC);
#Test(expected = UnsupportedOperationException.class)
public void testWrite() {
final TypeAdapter<ZonedDateTime> unit = getZonedDateTimeTypeAdapter();
unit.toJsonTree(expectedZonedDateTime);
}
#Test
public void testRead()
throws IOException {
final TypeAdapter<ZonedDateTime> unit = getZonedDateTimeTypeAdapter();
try ( final Reader reader = getPackageResourceReader(ZonedDateTimeTypeAdapterTest.class, "zoned-date-time.json") ) {
final ZonedDateTime actualZonedDateTime = unit.fromJson(reader);
assertThat(actualZonedDateTime, is(expectedZonedDateTime));
}
}
}
For simple cases I would definitely go with type adapters however they may be somewhat harder to implement. You could also refer the Gson unit tests for more information.
Related
I've got a json file which has got a Date field. I'm transforming this to a required Date format(dd/MM/yyyy) using JsonDeserializer and registering the class as a TypeAdapter in GsonBuilder. Now, I want to transform this field to different date formats(say ISO8601) for different use cases. Instead of creating new JsonDeserializer's for different date formats, Is there anyway I can pass date format as one of the parameters to JsonSerializer class to make it extensible.
Deserializer class:
public class DateDeserializer implements JsonDeserializer<Date> {
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("dd/MM/yyyy");
#Override
public Date deserialize(JsonElement jsonElement, Type type, JsonDeserializationContext jsonDeserializationContext) {
String dateString = jsonElement.getAsString();
if (!StringUtils.isEmpty(dateString)) {
try {
return DATE_FORMAT.parse(dateString);
} catch (ParseException e) {
}
}
}
GsonBuilder:
public static final Gson GSON = new GsonBuilder()
.registerTypeAdapter(Date.class, new DateDeserializer())
.serializeNulls()
.create();
Does Gson have a way to read in non-standard JSON files?
Instead of a typical file like:
[{obj1},{objN}]
I have a file like this:
{obj1}
{objN}
Where there are no square brackets or commas and each object is separated by a newline character.
Yes, it has. Gson supports lenient reading. For example, the following JSON document (non-standard.json):
{
"foo": 1
}
{
"bar": 1
}
you can use the following reading way:
private static final Gson gson = new Gson();
private static final TypeAdapter<JsonElement> jsonElementTypeAdapter = gson.getAdapter(JsonElement.class);
public static void main(final String... args)
throws IOException {
try ( final Reader reader = getPackageResourceReader(Q43528208.class, "non-standard.json") ) {
final JsonReader jsonReader = new JsonReader(reader);
jsonReader.setLenient(true); // this makes it work
while ( jsonReader.peek() != END_DOCUMENT ) {
final JsonElement jsonElement = jsonElementTypeAdapter.read(jsonReader);
System.out.println(jsonElement);
}
}
}
Output:
{"foo":1}
{"bar":1}
I'm not sure if you can write a robust deserializer this way though.
Update
In order to simplify the Gson support, we can implement a few convenient reading methods:
// A shortcut method for the below implementation: aggregates the whole result into a single list
private static <T> List<T> parseToListLenient(final JsonReader jsonReader, final IMapper<? super JsonReader, ? extends T> mapper)
throws IOException {
final List<T> list = new ArrayList<>();
parseLenient(jsonReader, in -> list.add(mapper.map(in)));
return list;
}
// A convenient strategy-accepting method to configure a JsonReader instance to make it lenient and do read
// The consumer defines the strategy what to do with the current JsonReader token
private static void parseLenient(final JsonReader jsonReader, final IConsumer<? super JsonReader> consumer)
throws IOException {
final boolean isLenient = jsonReader.isLenient();
try {
jsonReader.setLenient(true);
while ( jsonReader.peek() != END_DOCUMENT ) {
consumer.accept(jsonReader);
}
} finally {
jsonReader.setLenient(isLenient);
}
}
// Since Java 8 Consumer inteface does not allow checked exceptions to be rethrown
private interface IConsumer<T> {
void accept(T value)
throws IOException;
}
private interface IMapper<T, R> {
R map(T value)
throws IOException;
}
Then simple reading is really simple, and we can just use the methods above:
final Gson gson = new Gson();
final TypeToken<Map<String, Integer>> typeToken = new TypeToken<Map<String, Integer>>() {
};
final TypeAdapter<Map<String, Integer>> typeAdapter = gson.getAdapter(typeToken);
try ( final JsonReader jsonReader = getPackageResourceJsonReader(Q43528208.class, "non-standard.json") ) {
final List<Map<String, Integer>> maps = parseToListLenient(jsonReader, typeAdapter::read);
System.out.println(maps);
}
Deserialization via Gson directly would require more complicated implementation:
// This is just a marker not meant to be instantiated but to create a sort of "gateway" to dispatch types in Gson
#SuppressWarnings("unused")
private static final class LenientListMarker<T> {
private LenientListMarker() {
throw new AssertionError("must not be instantiated");
}
}
private static void doDeserialize()
throws IOException {
final Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new TypeAdapterFactory() {
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
// Check if the given type is the lenient list marker class
if ( !LenientListMarker.class.isAssignableFrom(typeToken.getRawType()) ) {
// Not the case? Just delegate the job to Gson
return null;
}
final Type listElementType = getTypeParameter0(typeToken.getType());
final TypeAdapter<?> listElementAdapter = gson.getAdapter(TypeToken.get(listElementType));
#SuppressWarnings("unchecked")
final TypeToken<List<?>> listTypeToken = (TypeToken<List<?>>) TypeToken.getParameterized(List.class, listElementType);
final TypeAdapter<List<?>> listAdapter = gson.getAdapter(listTypeToken);
final TypeAdapter<List<?>> typeAdapter = new TypeAdapter<List<?>>() {
#Override
public void write(final JsonWriter out, final List<?> value)
throws IOException {
// Always write a well-formed list
listAdapter.write(out, value);
}
#Override
public List<?> read(final JsonReader in)
throws IOException {
// Delegate the job to the reading method - we only have to tell how to obtain the list values
return parseToListLenient(in, listElementAdapter::read);
}
};
#SuppressWarnings("unchecked")
final TypeAdapter<T> castTypeAdapter = (TypeAdapter<T>) typeAdapter;
return castTypeAdapter;
}
// A simple method to resolve actual type parameter
private Type getTypeParameter0(final Type type) {
if ( !(type instanceof ParameterizedType) ) {
// List or List<?>
return Object.class;
}
return ((ParameterizedType) type).getActualTypeArguments()[0];
}
})
.create();
// This type declares a marker specialization to be used during deserialization
final Type type = new TypeToken<LenientListMarker<Map<String, Integer>>>() {
}.getType();
try ( final JsonReader jsonReader = getPackageResourceJsonReader(Q43528208.class, "non-standard.json") ) {
// This is where we're a sort of cheating:
// We tell Gson to deserialize LenientListMarker<Map<String, Integer>> but the type adapter above will return a list
final List<Map<String, Integer>> maps = gson.fromJson(jsonReader, type);
System.out.println(maps);
}
}
The output is now for Map<String, Integer>s, not JsonElements:
[{foo=1}, {bar=1}]
Update 2
TypeToken.getParameterized workaround:
#SuppressWarnings("unchecked")
final TypeToken<List<?>> listTypeToken = (TypeToken<List<?>>) TypeToken.get(new ParameterizedType() {
#Override
public Type getRawType() {
return List.class;
}
#Override
public Type[] getActualTypeArguments() {
return new Type[]{ listElementType };
}
#Override
public Type getOwnerType() {
return null;
}
});
We can have one more program to introduce comma(,) and construct a well formed JSON
With spark 2, we can add multiline as read option.
spark.df.option("multiline","true").json("data.json")
I have a class that deserializes a JSON element.
public class DateTimeConverter implements JsonSerializer<DateTime>, JsonDeserializer<DateTime>
{
private static final DateTimeFormatter DATE_FORMAT = ISODateTimeFormat.dateHourMinuteSecondMillis();
#Override
public JsonElement serialize(DateTime src, Type typeOfSrc, JsonSerializationContext context)
{
final DateTimeFormatter fmt = ISODateTimeFormat.dateHourMinuteSecondMillis();
return new JsonPrimitive(fmt.print(src));
}
#Override
public DateTime deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
throws JsonParseException
{
final String dateAsString = json.getAsString();
System.out.println(dateAsString);
if (json.isJsonNull() || dateAsString.length()==0)
{
return null;
}
else
{
return DATE_FORMAT.parseDateTime(json.getAsString());
}
}
}
However, my Deserialize method when I input:
2015-07-29T11:00:00.000Z
I receive:
2015-07-29T11
from the System.out.println(dateAsString); Why is it truncating my input?
I think my issue is within my test class:
I constructed a DateTime object to be used with Google's Gson. However, I think the default constructor for DateTimeType doesn't support minute/second/millisecond. Is there a way I can extend the DateTimeType to support it?
Here is my test class:
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.reflect.TypeToken;
import org.joda.time.DateTime;
import org.junit.Test;
import java.lang.reflect.Type;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
/**
* Tests {#link DateTimeConverter}.
*/
public class DateTimeConverterTest {
String testTime = "2015-07-29T11:00:00.001Z";
#Test
public void testDateTimeConverter() throws Exception {
final Gson gson = initCustomGSON();
Type DateTimeType = new TypeToken<DateTime>() {
}.getType();
System.out.println(testTime);
DateTimeConverter timeConverter = new DateTimeConverter();
DateTime m = (gson.fromJson(testTime, DateTimeType));
assertThat("11", is(m.hourOfDay().getAsText()));
}
public Gson initCustomGSON() {
final GsonBuilder builder = new GsonBuilder();
JodaTimeConverters converter = new JodaTimeConverters();
converter.registerAll(builder);
return builder.create();
}
}
You have a few issues with this code.
Your first problem is that : is an operator in Json. You are interpreting an unescaped String with a : in it, so Gson is interpreting it as key : value. Your test string needs to surround the entire text date with quotes to prevent this from happening, e.g.
String testTime = "\"2015-07-29T11:00:00.001Z\"";
You were using ISODateTimeFormat.dateHourMinuteSecondMillis() in your code. However, the format pattern for this is yyyy-MM-dd'T'HH:mm:ss.SSS, which as you can see does not include a time zone. You want to be using ISODateTimeFormat.dateTime(), whose pattern is yyyy-MM-dd'T'HH:mm:ss.SSSZZ, which does have a time zone.
private static final DateTimeFormatter DATE_FORMAT = ISODateTimeFormat.dateTime();
Once these two changes are made, the DateTime object is finally properly created... but it will be created in your local time zone, not in UTC (it will correctly adjust the time to your zone. You can easily switch this back to UTC by doing:
DateTime m = ((DateTime) (gson.fromJson(testTime, DateTimeType))).withZone(DateTimeZone.UTC);
Once you make these three changes, your tests will pass. However: I strongly advise against using JsonSerializer and JsonDeserializer, they have been deprecated in favor of TypeAdapter, whose streaming API is significantly more performant:
New applications should prefer TypeAdapter, whose streaming API is more efficient than this interface's tree API.
I am aware the user guide provides code for how to do it with the JsonSerializer / JsonDeserializer API, but that's just because they haven't yet updated it.
It would simply be something like this:
public class DateTimeAdapter extends TypeAdapter<DateTime> {
private static final DateTimeFormatter FORMAT = ISODateTimeFormat.dateTime();
public DateTime read(JsonReader reader) throws IOException {
if (reader.peek() == JsonToken.NULL) {
reader.nextNull();
return null;
}
String dateString = reader.nextString();
if(dateString.length() == 0) return null;
return FORMAT.parseDateTime(dateString);
}
public void write(JsonWriter writer, DateTime value) throws IOException {
if (value == null) {
writer.nullValue();
return;
}
writer.value(FORMAT.print(value));
}
}
I have mapped a custom deserializer to convert Strings on dd/MM/yyyy pattern to LocalDate so I can call my services with a more readable signature..
This is my dto class that is used as a Jersey #BeanParam to transport data between layers:
public class ProdutoFilterDto implements FilterDto {
private static final long serialVersionUID = -4998167328470565406L;
#QueryParam("dataInicial")
#JsonDeserialize(using = CustomLocalDateDeserializer.class)
private LocalDate dataInicial;
#QueryParam("dataInicial")
#JsonDeserialize(using = CustomLocalDateDeserializer.class)
private LocalDate dataFinal;
public LocalDate getDataInicial() {
return dataInicial;
}
public void setDataInicial(LocalDate dataInicial) {
this.dataInicial = dataInicial;
}
public LocalDate getDataFinal() {
return dataFinal;
}
public void setDataFinal(LocalDate dataFinal) {
this.dataFinal = dataFinal;
}
}
and this is my custom deserializer:
public class CustomLocalDateDeserializer extends JsonDeserializer<LocalDate> {
#Override
public LocalDate deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
final DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd/MM/yyyy");
final String data = p.getValueAsString();
return (LocalDate) formatter.parse(data);
}
}
Its being used on this jersey service:
#Path("produto")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_JSON)
public class ProdutoService {
...
#GET
#Path("query")
#Override
public Response query(
#QueryParam("offset") #DefaultValue(value = "0") Integer offSet,
#QueryParam("limit") #DefaultValue(value = "10") Integer limit,
#BeanParam ProdutoFilterDto filter) { ... }
...
}
I am calling like this:
${host goes here}/produto/query?dataInicial=11/09/1992
The problem is that the deserializer method is never called and the bean param variable remains null..
MessageBodyReaders aren't used for #QueryParam. You seem to be expecting the Jackson MessageBodyReader to handle this deserialization, but it doesn't work like that.
Instead you will want to use a ParamConverter, which will need to be registered through a ParamConverterProvider. For example:
#Provider
public class LocalDateParamConverterProvider implements ParamConverterProvider {
final DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd/MM/yyyy");
#Override
public <T> ParamConverter<T> getConverter(
Class<T> rawType, Type genericType, Annotation[] antns) {
if (LocalDate.class == rawType) {
return new ParamConverter<T>() {
#Override
public T fromString(String string) {
try {
LocalDate localDate = LocalDate.parse(string, formatter);
return rawType.cast(localDate);
} catch (Exception ex) {
throw new BadRequestException(ex);
}
}
#Override
public String toString(T t) {
LocalDate localDate = (LocalDate) t;
return formatter.format(localDate);
}
};
}
return null;
}
}
Now LocalDate will work with #QueryParam and other #XxxParams also.
Some things to note:
If your goal is to parse both your #XxxParams and your JSON body into a bean this will not work. I'm not sure how that would work, but I'm sure it would involve a lot of hacking, and I wouldn't recommend it.
Your cast to (LocalDate) won't work. It's an illegal cast to java.time.format.Parsed. See correct way in code example.
Related to the above point. I was pulling out my hair for a good hour trying to figure out why I was getting a 404, using your parse code. With a 404, the last place I thought to look was in the ParamConverter. But it seems any uncaught exceptions that are thrown in the ParamConverter, will cause a 404. Doesn't make much sense right? The head pounding led me to this, which led me to this, which seems to be a poor specification
"if the field or property is annotated with
#MatrixParam, #QueryParam or #PathParam then an implementation MUST generate an instance of
NotFoundException (404 status) that wraps the thrown exception and no entity
"
Moral of the story: make sure to catch any possible exceptions in the ParamConverter!
See Also:
Good article on ParamConverters
There are many examples of Jackson to/from java.util.Date code but they all seem to leverage POJO annotation. I have generic Maps of scalars that I wish to de/serialize to JSON. This is the current deserializer setup; very simple:
public class JSONUtils {
static {
DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS, true);
mapper.setDateFormat(df); // this works for outbounds but has no effect on inbounds
mapper.getDeserializationConfig().with(df); // Gave this a shot but still does not sniff strings for a format that we declare should be treated as java.util.Date
}
public static Map<String,Object> parseJSON(InputStream is) {
Map<String,Object> data = null;
try {
data = mapper.readValue(is, Map.class);
} catch(Exception e) {
// ...
}
return data;
}
I grok that a dateserializer can turn java.util.Date into a ISO 8601-ish string. It's going the other way that puzzles me. Clearly, in a JSON doc with no context, a string is a string so I cannot know if it was once a date. So I am prepared to duck type this and examine all strings being deserialized and if they smell like YYYY-MM-DDTHH:MM:SS.sss datetimes, then I will make a java.util.Date instead of just passing back a String. So given:
{ "name": "buzz",
"theDate": "2013-09-10T12:00:00.000"
}
will yield
Map<String,Object> m = mapper.readValue(is, Map.class);
Object o1 = m.get("name"); // o1 is instanceof String
Object o2 = m.get("theDate"); // o2 is instanceof Date
But this means that the deserializer has to return two different types and I have not been able to figure out how to do this in Jackson. Does anyone know of a good, compact example that will sniff for date-like strings and turn them into Dates, leaving others as Strings?
I've been looking for the answer on a related subject recently and come up with the following solution, thanks to Justin Musgrove and his article Custom jackson date deserializer.
Basically, the idea is to replace standard deserializer for Object.class that will convert any string in the specified format to the Date object or fallback to the standard behaviour otherwise. Obviously, this operation comes at cost of extra processing, so you'd want to keep a dedicated instance of ObjectMapper configured for this and only use it when absolutely necessary or if prepared doing second pass anyway.
Note that the Date string format in your example has no timezone component, which may cause some issues, but I leave the format as requested. You can use a parser of your choice in place of the FastDateFormat from Apache Commons Lang. I actually use Instant in my case.
CustomObjectDeserializer.java
import java.io.IOException;
import org.apache.commons.lang3.time.FastDateFormat;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.JsonTokenId;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer;
public class CustomObjectDeserializer extends UntypedObjectDeserializer {
private static final long serialVersionUID = 1L;
private static final FastDateFormat format = FastDateFormat.getInstance("yyyy-MM-dd'T'HH:mm:ss.SSS");
public CustomObjectDeserializer() {
super(null, null);
}
#Override
public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
if (p.getCurrentTokenId() == JsonTokenId.ID_STRING) {
try {
String value = p.getText();
// put your own parser here
return format.parse(value);
} catch (Exception e) {
return super.deserialize(p, ctxt);
}
} else {
return super.deserialize(p, ctxt);
}
}
}
JSONUtils.java
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
public class JSONUtils {
private static final ObjectMapper mapper = new ObjectMapper();
static {
mapper.configure(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS, true);
SimpleModule module = new SimpleModule("DateConverter");
// register a new deserializer extending and replacing UntypedObjectDeserializer
module.addDeserializer(Object.class, new CustomObjectDeserializer());
mapper.registerModule(module);
}
public static Map<String, Object> parseJSON(InputStream is) {
Map<String, Object> data = null;
try {
data = mapper.readValue(is, Map.class);
} catch (Exception e) {
// ...
e.printStackTrace();
}
return data;
}
public static void main(String[] args) throws Exception {
String input = "{\"name\": \"buzz\", \"theDate\": \"2013-09-10T12:00:00.000\"}";
InputStream is = new ByteArrayInputStream(input.getBytes(StandardCharsets.UTF_8));
Map<String, Object> m = mapper.readValue(is, Map.class);
Object o1 = m.get("name"); // o1 is instanceof String
Object o2 = m.get("theDate"); // o2 is instanceof Date
System.out.println(o1.getClass().getName() + " : " + o1);
System.out.println(o2.getClass().getName() + " : " + o2);
}
}
If you have a POJO, you can easy use annotation on get and set method with serializer and deserializer.
following an example that serialize and deserialize objects in different ways: List<POJO> to String, String to Map and Map to List<POJO> again. Obviously, in the map the Date values are as String.
This solution is thread safe because uses org.joda.time.format.DateTimeFormat and org.joda.time.format.DateTimeFormatter, you can find more info herein this post How to deserialize JS date using Jackson? and this link http://fahdshariff.blogspot.co.uk/2010/08/dateformat-with-multiple-threads.html
My POJO:
#JsonAutoDetect
public class QueueTask implements Serializable {
private static final long serialVersionUID = -4411796657106403937L;
public enum ActivitiQueueStatus {
IN_PROGRESS(AsyncProcessingWorkflowContentModel.InProgressTask.TYPE.getLocalName()), //
IN_QUEUE(AsyncProcessingWorkflowContentModel.InQueueTask.TYPE.getLocalName());
private String value;
private ActivitiQueueStatus(final String value) {
this.value = value;
}
public static ActivitiQueueStatus enumOf(final String value) {
for (ActivitiQueueStatus enum_i : values()) {
if (enum_i.value.equals(value))
return enum_i;
}
throw new IllegalArgumentException("value '" + value + "' is not a valid enum");
}
}
private String user;
private Date creationDate;
private int noRowsSelected;
private ActivitiQueueStatus status;
public String getUser() {
return user;
}
public void setUser(String user) {
this.user = user;
}
#JsonSerialize(using = JsonDateSerializer.class)
public Date getCreationDate() {
return creationDate;
}
#JsonDeserialize(using = JsonDateDeSerializer.class)
public void setCreationDate(Date creationDate) {
this.creationDate = creationDate;
}
public int getNoRowsSelected() {
return noRowsSelected;
}
public void setNoRowsSelected(int noRowsSelected) {
this.noRowsSelected = noRowsSelected;
}
public ActivitiQueueStatus getStatus() {
return status;
}
public void setStatus(ActivitiQueueStatus status) {
this.status = status;
}
}
My Serializer:
#Component
public class JsonDateDeSerializer extends JsonDeserializer<Date> {
// use joda library for thread safe issue
private static final DateTimeFormatter dateFormat = DateTimeFormat.forPattern("dd/MM/yyyy hh:mm:ss");
#Override
public Date deserialize(final JsonParser jp, final DeserializationContext ctxt) throws IOException, JsonProcessingException {
if (jp.getCurrentToken().equals(JsonToken.VALUE_STRING))
return dateFormat.parseDateTime(jp.getText().toString()).toDate();
return null;
}
}
and Deserializer:
#Component
public class JsonDateSerializer extends JsonSerializer<Date> {
// use joda library for thread safe issue
private static final DateTimeFormatter dateFormat = DateTimeFormat.forPattern("dd/MM/yyyy hh:mm:ss");
#Override
public void serialize(final Date date, final JsonGenerator gen, final SerializerProvider provider) throws IOException, JsonProcessingException {
final String formattedDate = dateFormat.print(date.getTime());
gen.writeString(formattedDate);
}
}
My Service:
public class ServiceMock {
// mock this parameter for usage.
public List<QueueTask> getActiveActivities(QName taskStatus) {
final List<QueueTask> listToReturn = new LinkedList<QueueTask>();
final SimpleDateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");
Date d1 = null, d2 = null, d3 = null, d4 = null, d5 = null;
try {
d1 = dateFormat.parse("01/02/2013 12:44:44");
d2 = dateFormat.parse("21/12/2013 16:44:44");
d3 = dateFormat.parse("21/12/2013 16:45:44");
d4 = dateFormat.parse("21/12/2013 16:44:46");
d5 = dateFormat.parse("11/09/2013 16:44:44");
} catch (ParseException e) {
}
QueueTask dataSet = new QueueTask();
dataSet = new QueueTask();
dataSet.setUser("user_b");
dataSet.setStatus(ActivitiQueueStatus.enumOf("placeInQueue"));
dataSet.setNoRowsSelected(500);
dataSet.setCreationDate(d1);
listToReturn.add(dataSet);
dataSet = new QueueTask();
dataSet.setUser("user_d");
dataSet.setStatus(ActivitiQueueStatus.enumOf("placeInQueue"));
dataSet.setNoRowsSelected(300);
dataSet.setCreationDate(d2);
listToReturn.add(dataSet);
dataSet = new QueueTask();
dataSet.setUser("user_a");
dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
dataSet.setNoRowsSelected(700);
dataSet.setCreationDate(d3);
listToReturn.add(dataSet);
dataSet = new QueueTask();
dataSet.setUser("user_k");
dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
dataSet.setNoRowsSelected(700);
dataSet.setCreationDate(d4);
listToReturn.add(dataSet);
dataSet = new QueueTask();
dataSet.setUser("user_l");
dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
dataSet.setNoRowsSelected(700);
dataSet.setCreationDate(d5);
listToReturn.add(dataSet);
return listToReturn;
}
}
MAIN usage:
public class SerializationServiceTest {
private static final Logger LOGGER = LoggerFactory.getLogger(OUPQueueStatusServiceIT.class);
public void testGetActiveActivitiesSerialization() throws Exception {
LOGGER.info("testGetActiveActivitiesSerialization - start");
ServiceMock mockedService = new ServiceMock();
// AsyncProcessingWorkflowContentModel.InProgressTask.TYPE is an QName, mock this calling
List<QueueTask> tasks = mockedService.getActiveActivities(AsyncProcessingWorkflowContentModel.InProgressTask.TYPE);
assertNotNull(tasks);
assertTrue(tasks.size() == 5);
assertNotNull(tasks.get(0).getUser());
assertNotNull(tasks.get(0).getCreationDate());
assertNotNull(tasks.get(0).getStatus());
assertNotNull(tasks.get(0).getNoRowsSelected());
final ObjectMapper mapper = new ObjectMapper();
final String jsonString = mapper.writeValueAsString(tasks);
assertNotNull(jsonString);
assertTrue(jsonString.contains("creationDate"));
// test serialization from string to Map
final List<Map<String, Object>> listOfMap = mapper.readValue(jsonString, new TypeReference<List<Map<String, Object>>>() {
});
assertNotNull(listOfMap);
final DateFormat formatter = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");
for (Map<String, Object> map_i : listOfMap) {
// check date value
assertTrue(map_i.containsKey("creationDate"));
final Date date = formatter.parse("" + map_i.get("creationDate"));
assertNotNull(date);
assertNotNull(map_i.get("user"));
assertNotNull(map_i.get("status"));
assertNotNull(ActivitiQueueStatus.valueOf("" + map_i.get("status")));
assertNotNull(map_i.get("noRowsSelected"));
}
// test de-serialization
List<QueueTask> deserializedTaskList = mapper.convertValue(listOfMap, new TypeReference<List<QueueTask>>() {
});
assertNotNull(deserializedTaskList);
assertTrue(deserializedTaskList.size() == 5);
for (QueueTask t : deserializedTaskList) {
assertNotNull(t.getUser());
assertNotNull(t.getCreationDate());
assertNotNull(t.getDownloadType());
assertNotNull(t.getStatus());
}
LOGGER.info("testGetActiveActivitiesSerialization - end");
}
public static void main(String[] args) throws Exception {
new SerializationServiceTest().SerializationServiceTest();
}
}
After some weeks poking around on this (and no other comments or answers), I now believe what I seek is NOT possible in Jackson. Deserialization of JSON into a Map with ducktyping for dates must occur after-the-fact. There is no way to interpose the parse stream, sniff the string for YYYY-MM-DDTHH:MM:SS.SSS and upon match substitute a Date object instead of String. You must let Jackson build the Map, then outside of Jackson go back to the top and walk the Map, sniffing for dates.
I will add that since I have a very specific duck I am looking for, the fastest implementation to turn the String into a Date is a hand-rolled thing about 120 lines long that validates and sets up the proper integer m-d-y-h-m-s-ms for Calendar then calls getTime(). 10,000,000 conversions takes 4240 millis, or about 2.3m/sec.
Before the joda-time lobby pipes up, yes, I tried that first:
// This is set up ONCE, outside the timing loop:
DateTimeFormatter format = ISODateTimeFormat.dateHourMinuteSecondMillis();
// These are in the timing loop:
while(loop) {
DateTime time = format.parseDateTime("2013-09-09T14:45:00.123");
Date d = time.toDate();
}
takes about 9630 millis to run, about 1.04m/sec; half the speed. But that's still WAY faster than the "out of the box use javax" option:
java.util.Calendar c2 = javax.xml.bind.DatatypeConverter.parseDateTime(s);
Date d = c2.getTime();
This takes 30428 mills to run, about .33m/sec -- almost 7x slower than the handroll.
SimpleDateFormat is not thread-safe so therefore was not considered in for use in converter utility where I cannot make any assumptions about the callers.
Here is a basic example on how to use Jackson to serialize deserialize a date from an object
public class JacksonSetup {
private static class JacksonSerializer {
private static JacksonSerializer instance;
private JacksonSerializer() {
}
public static JacksonSerializer getInstance() {
if (instance == null)
instance = new JacksonSerializer();
return instance;
}
public <E extends ModelObject> void writeTo(E object, Class<E> type, OutputStream out) throws IOException {
ObjectMapper mapper = getMapper();
mapper.writeValue(out, object);
}
public <E extends ModelObject> void writeTo(E object, Class<E> type, Writer out) throws IOException {
ObjectMapper mapper = getMapper();
mapper.writeValue(out, object);
}
public <E extends ModelObject> E read(String input, Class<E> type) throws IOException {
ObjectMapper mapper = getMapper();
E result = (E) mapper.readValue(input, type);
return result;
}
private ObjectMapper getMapper() {
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
AnnotationIntrospector introspector = new JaxbAnnotationIntrospector(mapper.getTypeFactory());
mapper.setAnnotationIntrospector(introspector);
return mapper;
}
}
private static class JaxbDateSerializer extends XmlAdapter<String, Date> {
private SimpleDateFormat dateFormat = new SimpleDateFormat("MM-dd-yyyy");
#Override
public String marshal(Date date) throws Exception {
return dateFormat.format(date);
}
#Override
public Date unmarshal(String date) throws Exception {
return dateFormat.parse(date);
}
}
private static abstract class ModelObject {
}
private static class Person extends ModelObject {
private String name;
private Date bday;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
#XmlElement(name = "birth-day")
#XmlJavaTypeAdapter(JaxbDateSerializer.class)
public Date getBday() {
return bday;
}
public void setBday(Date bday) {
this.bday = bday;
}
}
public static void main(String[] args) {
try {
Person person = new Person();
person.setName("Jhon Doe");
person.setBday(new Date());
Writer writer = new StringWriter();
JacksonSerializer.getInstance().writeTo(person, Person.class, writer);
System.out.println(writer.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}