I'm unsure how YAML "timestamps" are supposed to be represented. From the YAML 1.2 specification, it would seem that YAML just assumes that if you have a value in string format that looks like an ISO 8601 date, then it is parsed as a date unless you say differently. Here are a couple of examples from the spec:
date: 2002-12-14
not-date: !!str 2002-04-28
The Timestamp Language-Independent Type for YAML™ Version 1.1 working draft (from 2005, currently 15 years ago!) seems to indicate that a special tag in the form tag:yaml.org,2002:timestamp should be used. It also indicates a shorthand of !!timestamp.
In Java using SnakeYAML Engine v2.0 (org.snakeyaml:snakeyaml-engine:2.0) I tried parsing the 2002-12-14 form, and got a string as the parsed value, not any sort of date object. I see that the SnakeYAML Engine repository has an example using the !!timestamp approach (e.g. !!timestamp 2020-03-24T12:34:00.333), but this is a recent change and I'm sure if this support has been released yet.
I tried both the form fooBar: !!timestamp 2020-03-24 and also fooBar: !!timestamp 2020-03-24T12:34:00.333, but SnakeYAML Engine reported:
could not determine a constructor for the tag tag:yaml.org,2002:timestamp
So what is the official way to represent a date (specifically a local date with YYYY-MM-DD) in YAML, and is the correct approach reflected in the latest YAML specification? Does SnakeYAML Engine support the official YAML date approach?
From the YAML 1.2 specification, it would seem that YAML just assumes that if you have a value in string format that looks like an ISO 8601 date, then it is parsed as a date unless you say differently.
No. The YAML spec gives three schemas (Failsafe, JSON and Core) that should be supported; none of them includes a timestamp type. However, a scalar looking like a timestamp can be parsed as such if a schema is used that supports it. The spec only tells you that if you want to ensure that a scalar is not loaded as timestamp, prefix it with !!str.
So what is the official way to represent a date (specifically a local date with YYYY-MM-DD) in YAML.
The !!timestamp definition you linked is closest to what an official way would be. However, the tag repository containing it is not part of the spec and implementations are not required to support it. Furthermore, it is defined for outdated YAML 1.1.
This means that SnakeYAML isn't required to support timestamps at all. You can see in the example you give that timestamp support is not included; the example implements loading timestamps itself. You can modify that code to use with the normal public interface:
class TimestampConstructor extends Constructor {
public static final Pattern TIMESTAMP = Pattern
.compile("^(?:[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]|[0-9][0-9][0-9][0-9]-[0-9][0-9]?-[0-9][0-9]?(?:[Tt]|[ \t]+)[0-9][0-9]?:[0-9][0-9]:[0-9][0-9](?:\\.[0-9]*)?(?:[ \t]*(?:Z|[-+][0-9][0-9]?(?::[0-9][0-9])?))?)$");
public static final Tag TAG = new Tag(Tag.PREFIX + "timestamp");
public TimestampConstructor() {
this.yamlConstructors.put(TAG, new ConstructTimestamp());
}
private class ConstructTimestamp extends AbstractConstruct {
public Object construct(Node node) {
String val = (String) constructScalar(node);
return LocalDateTime.parse(val);
}
}
}
Then, use it like this:
Yaml yaml = new Yaml(new TimestampConstructor());
yaml.addImplicitResolver(TimestampConstructor.TAG,
TimestampConstructor.PATTERN, "0123456789");
I am having some issue with date logic which I've isolated to Jackson, the JSON serializer.
In the database and in a debug point in the application, dates are correct and everything is written using default timezone. However, in serialization 4 hours are being added. I found this could be remedied by telling Jackson specifically to use EST (it was defaulting to UTC). As below:
#JsonFormat(shape= JsonFormat.Shape.STRING, pattern="yyyy-MM-dd HH:mm:ss.SSSZ", timezone="America/New_York")
private Date startDate;
However, the problem is that only local is using EST and the server will be using UTC. I need Jackson to use system defaults.
Luckily, I found this similar question which is backed up by the documentation. New solution:
#JsonFormat(shape= JsonFormat.Shape.STRING, pattern="yyyy-MM-dd HH:mm:ss.SSSZ", timezone=JsonFormat.DEFAULT_TIMEZONE)
private Date startDate;
However, it doesn't work! I tried also timezone='DEFAULT_TIMEZONE' and a variety of other things but in all cases the api output still seems to be 4 hours ahead of the number in the database.
Other things I have tried:
logging out JsonFormat.DEFAULT_TIMEZONE returns ##default.
logging TimeZone.getDefault().getDisplayName() returns Eastern Standard Time.
Jackson version is 2.9.
Any suggestions?
Solved my own question. Here is what I found:
JsonFormat.DEFAULT_TIMEZONE is NOT the system default, as the documentation and SO answer suggest, but actually defaults to UTC.
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder
/**
* Override the default {#link TimeZone} to use for formatting.
* Default value used is UTC (NOT local timezone).
* #since 4.1.5
*/
public Jackson2ObjectMapperBuilder timeZone(TimeZone timeZone) {
com.fasterxml.jackson.annotation.JsonFormat
/**
* Value that indicates that default {#link java.util.TimeZone}
* (from deserialization or serialization context) should be used:
* annotation does not define value to use.
*<p>
* NOTE: default here does NOT mean JVM defaults but Jackson databindings
* default, usually UTC, but may be changed on <code>ObjectMapper</code>.
*/
public final static String DEFAULT_TIMEZONE = "##default";
Solution:
#Autowired
com.fasterxml.jackson.databind.ObjectMapper objectMapper;
and objectMapper.setTimeZone(TimeZone.getDefault()) in a config class, like so:
package path.to.config;
import ...
#Configuration
public class JacksonConfiguration {
#Autowired
public JacksonConfiguration(ObjectMapper objectMapper){
objectMapper.setTimeZone(TimeZone.getDefault());
}
}
This should set the Jackson ObjectMapper to use system default instead of Jackson default (UTC).
How can I configure JPA/Hibernate to store a date/time in the database as UTC (GMT) time zone? Consider this annotated JPA entity:
public class Event {
#Id
public int id;
#Temporal(TemporalType.TIMESTAMP)
public java.util.Date date;
}
If the date is 2008-Feb-03 9:30am Pacific Standard Time (PST), then I want the UTC time of 2008-Feb-03 5:30pm stored in the database. Likewise, when the date is retrieved from the database, I want it interpreted as UTC. So in this case 530pm is 530pm UTC. When it's displayed it will be formatted as 9:30am PST.
Since Hibernate 5.2, you can now force the UTC time zone by adding the following configuration property into the properties.xml JPA configuration file:
<property name="hibernate.jdbc.time_zone" value="UTC"/>
If you're using Spring Boot, then add this property to your application.properties file:
spring.jpa.properties.hibernate.jdbc.time_zone=UTC
To the best of my knowledge, you need to put your entire Java app in UTC timezone (so that Hibernate will store dates in UTC), and you'll need to convert to whatever timezone desired when you display stuff (at least we do it this way).
At startup, we do:
TimeZone.setDefault(TimeZone.getTimeZone("Etc/UTC"));
And set the desired timezone to the DateFormat:
fmt.setTimeZone(TimeZone.getTimeZone("Europe/Budapest"))
Hibernate is ignorant of time zone stuff in Dates (because there isn't any), but it's actually the JDBC layer that's causing problems. ResultSet.getTimestamp and PreparedStatement.setTimestamp both say in their docs that they transform dates to/from the current JVM timezone by default when reading and writing from/to the database.
I came up with a solution to this in Hibernate 3.5 by subclassing org.hibernate.type.TimestampType that forces these JDBC methods to use UTC instead of the local time zone:
public class UtcTimestampType extends TimestampType {
private static final long serialVersionUID = 8088663383676984635L;
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
#Override
public Object get(ResultSet rs, String name) throws SQLException {
return rs.getTimestamp(name, Calendar.getInstance(UTC));
}
#Override
public void set(PreparedStatement st, Object value, int index) throws SQLException {
Timestamp ts;
if(value instanceof Timestamp) {
ts = (Timestamp) value;
} else {
ts = new Timestamp(((java.util.Date) value).getTime());
}
st.setTimestamp(index, ts, Calendar.getInstance(UTC));
}
}
The same thing should be done to fix TimeType and DateType if you use those types. The downside is you'll have to manually specify that these types are to be used instead of the defaults on every Date field in your POJOs (and also breaks pure JPA compatibility), unless someone knows of a more general override method.
UPDATE: Hibernate 3.6 has changed the types API. In 3.6, I wrote a class UtcTimestampTypeDescriptor to implement this.
public class UtcTimestampTypeDescriptor extends TimestampTypeDescriptor {
public static final UtcTimestampTypeDescriptor INSTANCE = new UtcTimestampTypeDescriptor();
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
public <X> ValueBinder<X> getBinder(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicBinder<X>( javaTypeDescriptor, this ) {
#Override
protected void doBind(PreparedStatement st, X value, int index, WrapperOptions options) throws SQLException {
st.setTimestamp( index, javaTypeDescriptor.unwrap( value, Timestamp.class, options ), Calendar.getInstance(UTC) );
}
};
}
public <X> ValueExtractor<X> getExtractor(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicExtractor<X>( javaTypeDescriptor, this ) {
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance(UTC) ), options );
}
};
}
}
Now when the app starts, if you set TimestampTypeDescriptor.INSTANCE to an instance of UtcTimestampTypeDescriptor, all timestamps will be stored and treated as being in UTC without having to change the annotations on POJOs. [I haven't tested this yet]
With Spring Boot JPA, use the below code in your application.properties file and obviously you can modify timezone to your choice
spring.jpa.properties.hibernate.jdbc.time_zone = UTC
Then in your Entity class file,
#Column
private LocalDateTime created;
Adding an answer that's completely based on and indebted to divestoclimb with a hint from Shaun Stone. Just wanted to spell it out in detail since it's a common problem and the solution is a bit confusing.
This is using Hibernate 4.1.4.Final, though I suspect anything after 3.6 will work.
First, create divestoclimb's UtcTimestampTypeDescriptor
public class UtcTimestampTypeDescriptor extends TimestampTypeDescriptor {
public static final UtcTimestampTypeDescriptor INSTANCE = new UtcTimestampTypeDescriptor();
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
public <X> ValueBinder<X> getBinder(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicBinder<X>( javaTypeDescriptor, this ) {
#Override
protected void doBind(PreparedStatement st, X value, int index, WrapperOptions options) throws SQLException {
st.setTimestamp( index, javaTypeDescriptor.unwrap( value, Timestamp.class, options ), Calendar.getInstance(UTC) );
}
};
}
public <X> ValueExtractor<X> getExtractor(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicExtractor<X>( javaTypeDescriptor, this ) {
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance(UTC) ), options );
}
};
}
}
Then create UtcTimestampType, which uses UtcTimestampTypeDescriptor instead of TimestampTypeDescriptor as the SqlTypeDescriptor in the super constructor call but otherwise delegates everything to TimestampType:
public class UtcTimestampType
extends AbstractSingleColumnStandardBasicType<Date>
implements VersionType<Date>, LiteralType<Date> {
public static final UtcTimestampType INSTANCE = new UtcTimestampType();
public UtcTimestampType() {
super( UtcTimestampTypeDescriptor.INSTANCE, JdbcTimestampTypeDescriptor.INSTANCE );
}
public String getName() {
return TimestampType.INSTANCE.getName();
}
#Override
public String[] getRegistrationKeys() {
return TimestampType.INSTANCE.getRegistrationKeys();
}
public Date next(Date current, SessionImplementor session) {
return TimestampType.INSTANCE.next(current, session);
}
public Date seed(SessionImplementor session) {
return TimestampType.INSTANCE.seed(session);
}
public Comparator<Date> getComparator() {
return TimestampType.INSTANCE.getComparator();
}
public String objectToSQLString(Date value, Dialect dialect) throws Exception {
return TimestampType.INSTANCE.objectToSQLString(value, dialect);
}
public Date fromStringValue(String xml) throws HibernateException {
return TimestampType.INSTANCE.fromStringValue(xml);
}
}
Finally, when you initialize your Hibernate configuration, register UtcTimestampType as a type override:
configuration.registerTypeOverride(new UtcTimestampType());
Now timestamps shouldn't be concerned with the JVM's time zone on their way to and from the database. HTH.
You would think this common problem would be taken care of by Hibernate. But its not! There are a few "hacks" to get it right.
The one I use is to store the Date as a Long in the database. So I am always working with milliseconds after 1/1/70. I then have getters and setters on my Class that return/accept only Dates. So the API remains the same. The down side is that I have longs in the database. SO with SQL I can pretty much only do <,>,= comparisons -- not fancy date operators.
Another approach is to user a custom mapping type as described here:
http://www.hibernate.org/100.html
I think the correct way to deal with this is to use a Calendar instead of a Date though. With the Calendar you can set the TimeZone before persisting.
NOTE: Silly stackoverflow won't let me comment, so here is a response to david a.
If you create this object in Chicago:
new Date(0);
Hibernate persists it as "12/31/1969 18:00:00". Dates should be devoid of timezone, so I'm not sure why the adjustment would be made.
There are several timezones in operation here:
Java's Date classes (util and sql), which have implicit timezones
of UTC
The timezone your JVM is running in, and
the default timezone of your database server.
All of these can be different. Hibernate/JPA has a severe design deficiency in that a user cannot easily ensure that timezone information is preserved in the database server (which allows reconstruction of correct times and dates in the JVM).
Without the ability to (easily) store timezone using JPA/Hibernate then information is lost and once information is lost it becomes expensive to construct it (if at all possible).
I would argue that it is better to always store timezone information (should be the default) and users should then have the optional ability to optimize the timezone away (although it only really affects display, there is still an implicit timezone in any date).
Sorry, this post doesn't provide a work-around (that's been answered elsewhere) but it is a rationalization of why always storing timezone information around is important. Unfortunately it seems many Computer Scientists and programming practitioners argue against the need for timezones simply because they don't appreciate the "loss of information" perspective and how that makes things like internationalization very difficult - which is very important these days with web sites accessible by clients and people in your organization as they move around the world.
Please take a look at my project on Sourceforge which has user types for standard SQL Date and Time types as well as JSR 310 and Joda Time. All of the types try to address the offsetting issue. See http://sourceforge.net/projects/usertype/
EDIT: In response to Derek Mahar's question attached to this comment:
"Chris, do your user types work with Hibernate 3 or greater? – Derek Mahar Nov 7 '10 at 12:30"
Yes these types support Hibernate 3.x versions including Hibernate 3.6.
Date is not in any time zone (it is a millisecond office from a defined moment in time same for everyone), but underlying (R)DBs generally store timestamps in political format (year, month, day, hour, minute, second, ...) that is time-zone sensitive.
To be serious, Hibernate MUST be allow being told within some form of mapping that the DB date is in such-and-such timezone so that when it loads or stores it it does not assume its own...
I encountered just the same problem when I wanted to store the dates in the DB as UTC and avoid using varchar and explicit String <-> java.util.Date conversions, or setting my whole Java app in the UTC time zone (because this could lead to another unexpected issues, if the JVM is shared across many applications).
So, there is an open source project DbAssist, which allows you to easily fix the read/write as UTC date from the database. Since you are using JPA Annotations to map the fields in the entity, all you have to do is to include the following dependency to your Maven pom file:
<dependency>
<groupId>com.montrosesoftware</groupId>
<artifactId>DbAssist-5.2.2</artifactId>
<version>1.0-RELEASE</version>
</dependency>
Then you apply the fix (for Hibernate + Spring Boot example) by adding #EnableAutoConfiguration annotation before the Spring application class. For other setups installation instructions and more use examples, just refer to the project's github.
The good thing is that you don't have to modify the entities at all; you can leave their java.util.Date fields as they are.
5.2.2 has to correspond to the Hibernate version you are using. I am not sure, which version you are using in your project, but the full list of provided fixes is available on the wiki page of the project's github. The reason why the fix is different for various Hibernate versions is because Hibernate creators changed the API a couple of times between the releases.
Internally, the fix uses hints from divestoclimb, Shane and a few other sources in order to create a custom UtcDateType. Then it maps the standard java.util.Date with the custom UtcDateType which handles all the necessary time zone handling.
The mapping of the types is achieved using #Typedef annotation in the provided package-info.java file.
#TypeDef(name = "UtcDateType", defaultForType = Date.class, typeClass = UtcDateType.class),
package com.montrosesoftware.dbassist.types;
You can find an article here which explains why such a time shift occurs at all and what are the approaches to solve it.
Hibernate does not allow for specifying time zones by annotation or any other means. If you use Calendar instead of date, you can implement a workaround using HIbernate property AccessType and implementing the mapping yourself. The more advanced solution is to implement a custom UserType to map your Date or Calendar. Both solutions are explained in my blog post here: http://www.joobik.com/2010/11/mapping-dates-and-time-zones-with.html
Here is the complete checklist for storing dates in your database in a proper way:
first, make sure your OS timezone is st properly: either set TZ env variable in your Dockerfile or start your Java application with -Duser.timezone=“UTC”.
As mentioned earlier: also set the timezone for JPA with hibernate.jdbc.time_zone = UTC
Use OffsetDateTime for storing dates as UTC in your entities. Don’t use Date, Calendar & ZonedDateTime !
I use Struts 1.
And have next bean
import java.util.Date;
public class Event {
private String name;
private Date date;
// getters & setters
}
And form
public class Holiday {
private Event event;
// getters & setters
}
I have jsp for adding new holiday
<html:text name="holidayForm" property="event.date" altKey="date.pattern" maxlength="10" size="10" /></label>
And I cant get date or put default date as Date date = new Date(); in particular format on jsp.
How can I do this?
Rather than making your field a Date, make it a String that represents the date in the correct format. You can use the SimpleDateFormatter class to convert between the two. You could also take a look at the jQuery UI Datepicker module (Javascript, so it will run client-side) to allow your users to select a date (in the correct format), rather than typing it in themselves.
Struts 1 form bean properties are almost always Strings because the built-in type conversion is quite limited. If you map a text input directly to a Date field in your form bean class, the value is expected to be in the format YYYY-MM-DD and you'll get an Error 500: beanUtils.populate error if it doesn't match.
I'm using Spring MVC with JSR303 to do my input validation.
A form I've created has a couple of date fields that are bound to Date objects within the object backing the form. I'm using JSR303 to do the validation for the Date using #Future. I'm also using #DateTimeFormat(pattern="dd/MM/yyyy"), (I know it's not validation).
How do I validate the date format of the String on the form? If I leave the other required fields blank (#NotEmpty) and enter a non-valid date in the form 'dd/MM/yy' it gets converted to 'dd/MM/yyyy' on re-presentation (e.g. 12/03/12 is re-presented as 12/03/0012). Which means I will get duff data in my system. If I enter "aaa" for I get a conversion Exception. Correctly formatted Strings get converted to Date objects.
Additionally should the 'required field' annotation for Date fields be #NotNull or #NotEmpty?
Many thanks in advance for any advice provided.
Thanks Ralph. I did some further digging around and came up with this (Which goes in my form controller):
#InitBinder
public void initBinder(WebDataBinder binder) {
String format = "dd/MM/yyyy";
SimpleDateFormat dateFormat = new SimpleDateFormat(format);
dateFormat.setLenient(false);
CustomDateEditor customDateEditor = new CustomDateEditor(dateFormat,true,format.length());
binder.registerCustomEditor(Date.class, customDateEditor);
}
With the properties file having the following key:
typeMismatch.java.util.Date : Some nice calm reassuring message to assist all negligent users
Maybe there are some other ways to do this but this will do for now.
You can not do this with JSR303, because the validation runs on the already poplulated (form baching) object.
So you need to implement your own custom converter, that is a bit more strickt than the one shipped with spring.
#See Spring Reference: Chapter 6.5 Spring 3 Type Conversion