ElasticSearch Requests are failing when hit in parallel - java

In My project Dashboard we fetch data using elasticsearch, I send parallel requests for different section of dashboard. Now strange behaviour has started coming. Out of this bunch of request I send almost 1-2 requests always fails. If I reload the page then chances are that the same request will fail or some other.
Error is always this
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.elasticsearch.ElasticsearchException: failed to map source
Stacktrace always starts with below.
java.lang.NumberFormatException: For input string: ""
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
Anyone could figure out by looking at this at error is coming when formatting date attribute when reading via elastic.
Below is how attribute looks in dto.
#Field(type = FieldType.Date, format = DateFormat.date, pattern = "uuuu-MM-dd HH:mm:ss")
#JsonDeserialize(using = CustomDateDeserializer.class)
private Date lastUpdateDate;
This is CustomDateDeserializer class,
public class CustomDateDeserializer extends StdDeserializer<Date> {
private SimpleDateFormat formatter = new SimpleDateFormat("uuuu-MM-dd HH:mm:ss");
public CustomDateDeserializer() {
this(null);
}
public CustomDateDeserializer(Class<?> vc) {
super(vc);
}
#Override
public Date deserialize(JsonParser jsonparser, DeserializationContext ctxt) throws IOException, JsonProcessingException {
String date = jsonparser.getText();
try {
return formatter.parse(date);
} catch (ParseException e) {
throw new RuntimeException(e);
}
}
}
Issue is failing request has no logical behaviour, because of which I am not able to figure out the cause.
Any input on this ?

Related

How to parse properly Date string to java.util.Date when deserializing JSON to Java POJO with Jackson ObjectMapper

I am implementing an application that gets the data from certin endponts in json formats and tries to deserialize tem to Java Objects but I have problems with the parsing of the date in the JSON.This is how the Date looks like in the JSON: "/Date(1633122000000+0300)/" and I cannot find information in Google how to successfully parse this format.
{
"Date": "/Date(1633122000000+0300)/",
"Filled": 0,
"Needed": 0,
"Paid": 0
}
This is the pojo I use to deserialize the data to using Jackson ObjectMapper:
import java.util.Date;
#Data
#AllArgsConstructor
#NoArgsConstructor
public class TimeByDateSheet {
#JsonProperty("Date")
#JsonFormat(timezone = "GMT+03:00")
#JsonDeserialize(using = DateDeserializer.class, as=Date.class)
private Date date;
#JsonProperty("Filled")
private Long filled;
#JsonProperty("Needed")
private Long needed;
#JsonProperty("Paid")
private Integer paid;
}
And here is my DateDeserializer:
#SuppressWarnings("serial")
public class DateDeserializer extends JsonDeserializer<Date> {
#Override
public Date deserialize(JsonParser jsonParser, DeserializationContext context)
throws IOException, JsonProcessingException {
SimpleDateFormat simpleDateFormat =
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss zzz", Locale.getDefault());
String dateStr = jsonParser.getText();
Date date;
try{
date = simpleDateFormat.parse(dateStr);
}catch(ParseException e){
throw new RuntimeException(e);
}
return date;
}
}
But it does not work correctly. I get the following exception:
Connected to the target VM, address: '127.0.0.1:52760', transport: 'socket'
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: java.text.ParseException: Unparseable date: "/Date(1633035600000+0300)/" (through reference chain: java.util.ArrayList[0]->com.dataart.forecasts.pojo.timebydate.TimeByDateSheet["Date"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:392)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1821)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:315)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:176)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4675)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3630)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3613)
at com.dataart.forecasts.DataProcessor.deserializeTimeByDateSheetsList(DataProcessor.java:198)
at com.dataart.forecasts.ForecastReportApplication.main(ForecastReportApplication.java:50)
Caused by: java.lang.RuntimeException: java.text.ParseException: Unparseable date: "/Date(1633035600000+0300)/"
at com.dataart.forecasts.DateDeserializer.deserialize(DateDeserializer.java:28)
at com.dataart.forecasts.DateDeserializer.deserialize(DateDeserializer.java:16)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:313)
... 10 more
Caused by: java.text.ParseException: Unparseable date: "/Date(1633035600000+0300)/"
at java.base/java.text.DateFormat.parse(DateFormat.java:395)
at com.dataart.forecasts.DateDeserializer.deserialize(DateDeserializer.java:26)
... 13 more
Could someone help me, please. I searched a lot in internet but could not find a solution.
Thank you in advance! :)
It looks like there is a problem generating the JSON. I really don't think you want to have the dates formatted like that. Right now, you have some odd text surrounding a unix timestamp in milliseconds followed by a zone offset. You are also using the old and rather frowned-upon Date and SimpleDateFormat classes rather than the newer java.time API. However, it is possible to deserialize your date format. Here is one way:
public class DateDeserializer extends JsonDeserializer<Date> {
#Override
public Date deserialize(JsonParser jsonParser, DeserializationContext context)
throws IOException, JsonProcessingException {
Pattern pattern = Pattern.compile("/Date\\((\\d+)([+-]\\d+)\\)/");
Matcher matcher = pattern.matcher(jsonParser.getText());
if (matcher.find()) {
String timestamp = matcher.group(1);
String offset = matcher.group(2);
Instant instant = Instant.ofEpochMilli(Long.parseLong(timestamp));
ZonedDateTime zdt = instant.atZone(ZoneId.of(offset));
instant = zdt.toInstant();
return Date.from(instant);
} else {
throw new RuntimeException("Invalid format: " + jsonParser.getText());
}
}
}
java.time
For this answer I am assuming:
The time in your JSON may come with or without the UTC offset.
You can go all-in on java.time, the modern Java date and time API, and declare your variable to be of type Instant or OffsetDateTime, for example (not Date).
For JSON that comes with an offset such as +0300 declare your variable an OffsetDateTime. Then use the following deserializer.
public class OdtDeserializer extends JsonDeserializer<OffsetDateTime> {
private static final DateTimeFormatter JSON_DATE_FORMATTER = new DateTimeFormatterBuilder()
.appendLiteral("/Date(")
.appendValue(ChronoField.INSTANT_SECONDS)
.appendValue(ChronoField.MILLI_OF_SECOND, 3)
.appendOffset("+HHMM", "Z")
.appendLiteral(")/")
.toFormatter();
#Override
public OffsetDateTime deserialize(JsonParser jsonParser, DeserializationContext context)
throws IOException {
String dateStr = jsonParser.getText();
return OffsetDateTime.parse(dateStr, JSON_DATE_FORMATTER);
}
}
For JSON that comes without offset like /Date(1636510000000)/ declare your variable Instant. Use a similar deserializer. Leave out the offset from the formatter. Parse into an Instant — the syntax is a bit different.
public class InstantDeserializerWIthoutOffset extends JsonDeserializer<Instant> {
private static final DateTimeFormatter JSON_DATE_FORMATTER = new DateTimeFormatterBuilder()
.appendLiteral("/Date(")
.appendValue(ChronoField.INSTANT_SECONDS)
.appendValue(ChronoField.MILLI_OF_SECOND, 3)
.appendLiteral(")/")
.toFormatter();
#Override
public Instant deserialize(JsonParser jsonParser, DeserializationContext context)
throws IOException {
String dateStr = jsonParser.getText();
return JSON_DATE_FORMATTER.parse(dateStr, Instant::from);
}
}
For JSON that may come with or without the offset still use Instant and just modify the formatter of the latter deserializer to include an optional offset:
private static final DateTimeFormatter JSON_DATE_FORMATTER = new DateTimeFormatterBuilder()
.appendLiteral("/Date(")
.appendValue(ChronoField.INSTANT_SECONDS)
.appendValue(ChronoField.MILLI_OF_SECOND, 3)
.optionalStart()
.appendOffset("+HHMM", "Z")
.optionalEnd()
.appendLiteral(")/")
.toFormatter();
If you cannot modify your POJO class and need to stay with Date, modify my Instant deserializer into a Date deserializer by changing the declaration and returning a Date like this:
String dateStr = jsonParser.getText();
Instant inst = JSON_DATE_FORMATTER.parse(dateStr, Instant::from);
return Date.from(inst);
Final solution catching both: /Date(1633035600000+0300)/ and /Date(-62135596800000)/ (the latter was also present at one place in oneof the JSONs). Thank you #DavidConrad
#Override
public Date deserialize(JsonParser jsonParser, DeserializationContext context) throws IOException, JsonProcessingException {
String dateString = jsonParser.getText();
Pattern pattern = Pattern.compile("/Date\\((-)?(\\d+)([+-]\\d+)?\\)/");
Matcher matcher = pattern.matcher(dateString);
if (!matcher.find()) {
throw new RuntimeException("Invalid format: " + dateString);
}
String timestamp = matcher.group(2);
String offset = matcher.group(3);
Instant instant = Instant.ofEpochMilli(Long.parseLong(timestamp));
if (nonNull(offset)) {
ZonedDateTime zdt = instant.atZone(ZoneId.of(offset));
instant = zdt.toInstant();
}
return Date.from(instant);
}
#DavidConrad Thank you, I will try your solution.
By the way, for now I made a workaround that works for me for now:
#Override
public Date deserialize(JsonParser jsonParser, DeserializationContext context) throws IOException, JsonProcessingException {
SimpleDateFormat dateFormattter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
String dateStr = jsonParser.getText();
String timeZone = dateStr.substring(dateStr.indexOf("+") + 1, dateStr.indexOf(")"));
String timeZoneShift = String.format("%s:%s",
timeZone.substring(0, timeZone.length()/2),
timeZone.substring(timeZone.length()/2));
dateFormattter.setTimeZone(TimeZone.getTimeZone(String.format("GMT+%s", timeZoneShift)));
Long millis = 0L;
if (dateStr.contains("+") && !dateStr.contains("-")) {
millis = Long.parseLong(dateStr.substring(dateStr.indexOf("(") + 1, dateStr.indexOf("+")));
} else if (dateStr.contains("+") && !dateStr.contains("-")) {
millis = Long.parseLong(dateStr.substring(dateStr.indexOf("(") + 1, dateStr.indexOf(")")));
}
Date date = new Date(millis);
String stringDate= dateFormattter.format(date);
try {
date = dateFormattter.parse(stringDate);
} catch (ParseException e) {
throw new RuntimeException(e);
}
return date;
}

Spring Boot opencsv

I am trying to use opencsv to parse a csv file like this:
name,purchase,date
TEST,TEST,2020-10-20T00:37:53.562000000Z
TEST,TEST,2020-10-20T00:37:53.562000000Z
I am trying to add the parsed data to a firebase database following this tutorial: https://attacomsian.com/blog/spring-boot-upload-parse-csv-file. This is my class for the data:
public class Records {
#CsvBindByName
private String name;
#CsvBindByName
private String purchase;
#CsvBindByName
private Timestamp date;
// get and setters left out for brevity pls comment if needed
}
When I parse the file I get this error:
Exception in thread "pool-6-thread-2" Exception in thread "pool-6-thread-1" Exception in thread "pool-6-thread-4" Exception in thread "pool-6-thread-3" java.lang.RuntimeException: com.opencsv.exceptions.CsvDataTypeMismatchException: Conversion of 2022-10-20T00:37:53.562000000Z to com.google.cloud.Timestamp failed.
at com.opencsv.bean.concurrent.ProcessCsvLine.run(ProcessCsvLine.java:99)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.opencsv.exceptions.CsvDataTypeMismatchException: Conversion of 2022-10-20T00:37:53.562000000Z to com.google.cloud.Timestamp failed.
at com.opencsv.bean.ConverterPrimitiveTypes.convertToRead(ConverterPrimitiveTypes.java:128)
at com.opencsv.bean.BeanFieldSingleValue.convert(BeanFieldSingleValue.java:98)
at com.opencsv.bean.AbstractBeanField.setFieldValue(AbstractBeanField.java:180)
at com.opencsv.bean.AbstractMappingStrategy.setFieldValue(AbstractMappingStrategy.java:581)
at com.opencsv.bean.AbstractMappingStrategy.populateNewBean(AbstractMappingStrategy.java:328)
at com.opencsv.bean.concurrent.ProcessCsvLine.processLine(ProcessCsvLine.java:128)
at com.opencsv.bean.concurrent.ProcessCsvLine.run(ProcessCsvLine.java:83)
... 3 more
Caused by: org.apache.commons.beanutils.ConversionException: Can't convert value '2022-10-20T00:37:53.562000000Z' to type class com.google.cloud.Timestamp
at org.apache.commons.beanutils.converters.AbstractConverter.conversionException(AbstractConverter.java:474)
at org.apache.commons.beanutils.converters.StringConverter.convertToType(StringConverter.java:96)
at org.apache.commons.beanutils.converters.AbstractConverter.convert(AbstractConverter.java:169)
at org.apache.commons.beanutils.converters.ConverterFacade.convert(ConverterFacade.java:61)
at org.apache.commons.beanutils.ConvertUtilsBean.convert(ConvertUtilsBean.java:491)
at com.opencsv.bean.ConverterPrimitiveTypes.convertToRead(ConverterPrimitiveTypes.java:118)
... 9 more
How can i fix this error? Do I need to change the format of the date column? I copied the date format from a record in the database so that format is how it should be stored in the database
I changed the csv format to:
name,purchase,date
TEST,TEST,2018-09-16T08:00:00
TEST,TEST,2018-09-16T08:00:00
I modified the class that binds to the csv to look like this:
public class CsvRecords {
#CsvBindByName
private String name;
#CsvBindByName
private String purchase;
#CsvBindByName
private String date;
// get and setters left out for brevity pls comment if needed
}
The POJO class for the data in db:
public class Records {
private String name;
private String purchase;
private Timestamp date;
// get and setters left out for brevity pls comment if needed
}
When uploading in the controller class I then convert the string to LocalDateTime and then again to Timestamp like this:
#PostMapping("/upload-csv-file")
public String uploadCSVFile(#RequestParam("file") MultipartFile file, Model model) {
// validate file
if (file.isEmpty()) {
model.addAttribute("message", "Please select a CSV file to upload.");
model.addAttribute("status", false);
} else {
// parse CSV file to create a list of `User` objects
try (Reader reader = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
// create csv bean reader
CsvToBean<Records> csvToBean = new CsvToBeanBuilder(reader)
.withType(Records.class)
.withIgnoreLeadingWhiteSpace(true)
.build();
// convert `CsvToBean` object to list of records
List<Records> records = csvToBean.parse();
// save users in DB?
for(int i = 0; i<records.size(); i++){
LocalDateTime localDateTime = LocalDateTime.parse(records.get(i).getDate);
Timestamp timestamp = Timestamp.valueOf(localDateTime);
Records rec = new Records(records.get(i).getName(), records.get(i).getPurchase(), timestamp)
firebaseServices.saveDetails(rec);
}
} catch (Exception ex) {
model.addAttribute("message", "An error occurred while processing the CSV file.");
model.addAttribute("status", false);
}
}
return "file-upload-status";
}
For details on the implementation of the firebaseServices class (saveDetails method) I used this tutorial
Could you have a try in this case that replaces the Timestamp to Date?
like follow this:
#CsvBindByName private Date date;
The timestamp field object is an instance from com.google.cloud.Timestamp package.
As #Allen suggested, you might have to look into java.util.date or event better, the Java 8 time APIs java.time.LocalTime to get support from ConverterPrimitiveTypes to be converted automatically.

Cannot save datetime to MongoDB using Spring Boot

I use Spring Boot and I try to save some date in MongoDB. My input date is
"2017-08-14T12:59"
I get this error while saving:
Failed to read HTTP message: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Failed to parse Date value '2017-08-14T12:59': Can not parse date "2017-08-14T12:59.000Z": while it seems to fit format 'yyyy-MM-dd'T'HH:mm:ss.SSS'Z'', parsing fails (leniency? null); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Failed to parse Date value '2017-08-14T12:59': Can not parse date "2017-08-14T12:59.000Z": while it seems to fit format 'yyyy-MM-dd'T'HH:mm:ss.SSS'Z'', parsing fails (leniency? null) (through reference chain:
In my POJO i tried like this:
#JsonDeserialize(using= CustomDateDeserialize.class)
private Date inputDateTime;
and I've implemented Deserializer like this :
private SimpleDateFormat dateFormat = new SimpleDateFormat(
"yyyy-MM-dd HH:mm");
#Override
public Date deserialize(JsonParser paramJsonParser,
DeserializationContext paramDeserializationContext)
throws IOException, JsonProcessingException {
String str = paramJsonParser.getText().trim();
try {
return dateFormat.parse(str);
} catch (ParseException e) {
}
return paramDeserializationContext.parseDate(str);
}
What else I miss here? Any help appreciated.
you need to modify format in your deserializer.
SimpleDateFormat dateFormat = new SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm");
any way simpledatetimeformat is not thread safe. if you java8 use DateTimeFormat.
Why don't you try Instant library
#Field("your_db_id_name")
private Instant inputDateTime;
public void setInputDateTime(Instant inputDateTime) {
this.inputDateTime = inputDateTime;
}
public void getInputDateTime() {
return inputDateTime;
}
You can set Filed by using Instant.now()

Spring Rest Controller date calculation

Below is a generalized version of my Spring RestController implementation. It's purpose is to respond to HTTP requests at a specific base URI ("/rest/my/uri/to/resource") with a resource based on the URI ID input. It works fine, however due to the structure of the data it returns I have had to add an optional date param. I have the controller calculate today's date and use that in my database query when the user does not supply one in the URI.
My question to you is if I use the todaySqlDate variable for each response where the user does not supply a date as I am below, will it recalculate the date each time it responds to a request? Obviously if the user inputs a date like
/rest/my/uri/to/resource/identifier/666?date=2016-03-15
this will not be an issue. My concern is that when the date is not included, a la
/rest/my/uri/to/resource/identifier/666
it will use the default date. Will the default date stop being current if the service is left running for more than 24 hours?
#RestController
#RequestMapping(value = "/rest/my/uri/to/resource")
public class ResourceController
{
private static final Logger LOGGER = LoggerFactory.getLogger(ResourceController.class);
#Autowired
private ResourceService resourceService;
public String todaySqlDate = getTodaySqlDate();
#RequestMapping(value = "/identifier/{id}", method = RequestMethod.GET, produces = {MediaType.APPLICATION_JSON_VALUE, MediaType.APPLICATION_XML_VALUE})
public Resource getResource(#PathVariable("id") String id,
#RequestParam(value="date", defaultValue="", required=false) String resourceDate)
throws InvalidParameterException, DataNotFoundException, Exception
{
LOGGER.trace("In ResourceController.getResouce() with {}", id);
try
{
if(!isValidIdentifier(id))
{
throw new InvalidParameterException("id is not valid: " + id);
}
if(resourceDate.isEmpty())
{
resourceDate = todaySqlDate;
}
else if(!isValidSqlDateString(resourceDate))
{
throw new InvalidParameterException("resourceDate is present but not valid: " + resourceDate);
}
ResourceList resourceList = resourceService.getResource(id, resourceDate);
if (resourceList == null)
{
LOGGER.trace("No resources found for given input");
throw new DataNotFoundException("ResourceList for " + id + " not found");
}
return resourceList;
}
catch (Exception e)
{
LOGGER.error(e.getMessage(), e);
throw e;
}
}
public String getTodaySqlDate()
{
java.util.Date utilDate = new java.util.Date();
java.sql.Date sqlDate = new java.sql.Date(utilDate.getTime());
return sqlDate.toString();
}
}
Yes, every new request will be handled by a new separate instance (thread) and hence it will re-calculate the date every time.
You can have a look at Spring/REST Documentation for more information.
Useful Link:
How are Threads allocated to handle Servlet request?

How to store Date field as ISODate() using jackson in MongoDb

I am trying to persist a java object having java.util.Date field in mongo collection using fasterxml jackson.
The problem is the default nature of objectMapper is to store Date as NumberLong type.
For e.g , a createdTime field of java.util.Date type gets stored as below:
"createdTime" : NumberLong("1427728445176")
I want to store it in ISODate format which is available in mongo Shell.
Now, i know there is way to format object mapper to store Date in a String dateformat.
But I am ONLY looking for ISODate() format.
For e.g
"createdTime" : ISODate("2015-01-20T16:39:42.132Z")
Is there a way to do that ?
Please advise gurus .
Thanks in advance for help.
What you need is the Jackson Joda Module. If you import that into your classpath, you can do the following on your mapper to write it as your desired Timestamp:
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JodaModule());
mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, true);
mapper.writeValueAsString(date);
You can replace date in the code sample above with your POJO as necessary.
Edit:
It looks like what you really want is a custom serializer. That would look something like this:
public class IsoDateSerializer extends JsonSerializer<DateTime> {
#Override
public void serialize(DateTime value, JsonGenerator jgen, SerializerProvider provider) {
String isoDate = ISODateTimeFormat.dateTime().print(value);
jgen.writeRaw("ISODATE(\"" + isoDate + "\")");
}
Then you'll either register it on the mapper for all DateTime types
mapper.addSerializer(DateTime.class, new IsoDateSerializer());
or specify it on the function using annotations
#JsonSerializer(using = IsoDateSerializer.class)
public DateTime createdTime;
I was able to serialize the date string to ISODate format. I wrote a customer date serializer like below.
public void serialize(Date date, JsonGenerator jgen, SerializerProvider provider) throws IOException {
String dateValue = getISODateString(date);
String text = "{ \"$date\" : \""+ dateValue +"\"}";
jgen.writeRawValue(text);
}
Based on request from user #mmx73, I am adding code for customer Date DeSeriaizer.
public class IsoDateDeSerializer extends JsonDeserializer<Date> {
#Override
public Date deserialize(JsonParser jsonParser, DeserializationContext deserializationContext)
throws IOException, JsonProcessingException {
ObjectCodec oc = jsonParser.getCodec();
JsonNode node = oc.readTree(jsonParser);
String dateValue = node.get("$date").asText();
//DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
Date date = null;
try {
date = df.parse(dateValue);
} catch (ParseException e) {
e.printStackTrace();
}
return date;
}
}
None of these answers accomplished what I wanted. I was having trouble because when I serialized the JSON string to MongoDB, it was stored as a String. A nicely formatted string, but a string none the less.
I use the com.fasterxml.jackson.databind.ObjectMapper to convert my objects to/from JSON and I wanted to continue using this class. I have the following method:
public enum JsonIntent {NONE, MONGODB};
public static ObjectMapper getMapper(final JsonIntent intent) {
ObjectMapper mapper = new ObjectMapper();
// Setting to true saves the date as NumberLong("1463597707000")
// Setting to false saves the data as "2016-05-18T19:30:52.000+0000"
mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);
mapper.registerModule(new JodaModule());
if (intent == JsonIntent.MONGODB) {
// If you want a date stored in MONGO as a date, then you must store it in a way that MONGO
// is able to deal with it.
SimpleModule testModule = new SimpleModule("MyModule", new Version(1, 0, 0, null, null, null));
testModule.addSerializer(Date.class, new StdSerializer<Date>(Date.class) {
private static final long serialVersionUID = 1L;
#Override
public void serialize(Date value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
try {
if (value == null) {
jgen.writeNull();
} else {
jgen.writeStartObject();
jgen.writeFieldName("$date");
String isoDate = ISODateTimeFormat.dateTime().print(new DateTime(value));
jgen.writeString(isoDate);
jgen.writeEndObject();
}
} catch (Exception ex) {
Logger.getLogger(JsonUtil.class.getName()).log(Level.SEVERE, "Couldn't format timestamp " + value + ", writing 'null'", ex);
jgen.writeNull();
}
}
});
testModule.addDeserializer(Date.class, new StdDeserializer<Date>(Date.class) {
private static final long serialVersionUID = 1L;
#Override
public Date deserialize(JsonParser jp, DeserializationContext dc) throws IOException, JsonProcessingException {
JsonNode tree = jp.readValueAsTree();
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ");
try {
return ISODateTimeFormat.dateTime().parseDateTime(tree.get("$date").textValue()).toDate();
} catch (Throwable t) {
throw new IOException(t.getMessage(), t);
}
}
});
mapper.registerModule(testModule);
}
return mapper;
}
Now, I can run the following test code:
BObjectMapper mapper = getMapper(JsonUtil.JsonIntent.NONE);
Date d1 = new Date();
String v = mapper.writeValueAsString(d1);
System.out.println("Joda Mapping: " + v);
Date d2 = mapper.readValue(v, Date.class);
System.out.println("Decoded Joda: " + d2);
mapper = getMapper(JsonUtil.JsonIntent.MONGODB);
v = mapper.writeValueAsString(d1);
System.out.println("Mongo Mapping: " + v);
d2 = mapper.readValue(v, Date.class);
System.out.println("Decoded Mongo: " + d2);
The output is as follows:
Joda Mapping: "2016-06-13T14:58:11.937+0000"
Decoded Joda: Mon Jun 13 10:58:11 EDT 2016
Mongo Mapping: {"$date":"2016-06-13T10:58:11.937-04:00"}
Decoded Mongo: Mon Jun 13 10:58:11 EDT 2016
Note that the JSON that will be sent to MONGODB defines the value containing a field named "$date". This tells MongoDB that this is a date object it seems.
When I look at Mongo, I see the following:
"importDate" : ISODate("2016-05-18T18:55:07Z")
Now, I can access the field as a date rather than as a string.
To add an encoded JSON string to Mongo, my code is as follows:
MongoDatabase db = getDatabase();
Document d = Document.parse(json);
db.getCollection(bucket).insertOne(d);
In this case, "json" is the encoded JSON string. Because it is coming from a JSON string, it has no way of knowing the types unless it infers this, which is why we needed the "$date" portion. The "bucket" is just a string indicating which table to use.
As a side note, I found out that if I pull a BSON object from Mongo and convert it to a JSON string by calling doc.toJson() (where doc is of type org.bison.Document as returned from a query), the date object is stored with a long value rather than a formatted text string. I did not check to see if I could push data into mongo after formatting in this way, but, you can modify the deserializer shown above to support this as follows:
testModule.addDeserializer(Date.class, new StdDeserializer<Date>(Date.class) {
private static final long serialVersionUID = 1L;
#Override
public Date deserialize(JsonParser jp, DeserializationContext dc) throws IOException, JsonProcessingException {
JsonNode tree = jp.readValueAsTree();
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ");
try {
// Mongo will return something that looks more like:
// {$date:<long integer for milliseconds>}
// so handle that as well.
JsonNode dateNode = tree.get("$date");
if (dateNode != null) {
String textValue = dateNode.textValue();
if (!Util.IsNullOrEmpty(textValue)) {
return ISODateTimeFormat.dateTime().parseDateTime(textValue).toDate();
}
return Util.MillisToDate(dateNode.asLong());
}
return null;
} catch (Throwable t) {
Util.LogIt("Exception: " + t.getMessage());
throw new IOException(t.getMessage(), t);
}
}
});
You can convert milliseconds to a Date or DateTime as follows:
/**
* Convert milliseconds to a date time. If zero or negative, just return
* null.
*
* #param milliseconds
* #return
*/
public static Date MillisToDate(final long milliseconds) {
if (milliseconds < 1) {
return null;
}
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(milliseconds);
return calendar.getTime();
}
public static DateTime MillisToDateTime(final long milliseconds) {
if (milliseconds < 1) {
return null;
}
return new DateTime(milliseconds);
}
In case you get some message like
com.fasterxml.jackson.core.JsonGenerationException: Can not write a field name, expecting a value.
Make sure you use writeRawValue instead within the accepted answer. That ends the field correctly, otherwise the next field to be serialize may throw this error.
You can solve this issue by reading/writing bson instead of json. Here is a test class:
package com.nagra.jongo.mapper;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import de.undercouch.bson4jackson.BsonFactory;
import de.undercouch.bson4jackson.deserializers.BsonDateDeserializer;
import de.undercouch.bson4jackson.serializers.BsonDateSerializer;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
import java.util.Date;
/**
* Uses Bson4Jackson 2.9.0
*
* <!-- https://mvnrepository.com/artifact/de.undercouch/bson4jackson -->
* <dependency>
* <groupId>de.undercouch</groupId>
* <artifactId>bson4jackson</artifactId>
* <version>2.9.2</version>
* </dependency>
*/
public class ObjectMapperTest {
private ObjectMapper mapper = new ObjectMapper(new BsonFactory());
private static class WrappedDate {
private Date Date = new Date(0);
public WrappedDate() {
}
public Date getDate() {
return Date;
}
public void setDate(Date Date) {
this.Date = Date;
}
}
#Before
public void setUp() {
SimpleModule module = new SimpleModule();
module.addSerializer(Date.class, new BsonDateSerializer());
module.addDeserializer(Date.class, new BsonDateDeserializer());
mapper.registerModule(module);
}
#Test
public void testDate() throws IOException {
WrappedDate date = new WrappedDate();
byte[] b = mapper.writeValueAsBytes(date);
WrappedDate i = mapper.readValue(b, WrappedDate.class);
Assert.assertEquals(date.getDate(), i.getDate());
System.out.println(i.getDate());
}}

Categories