We Keep Coding

Hibernate retrieve timestamps in UTC

I am using hibernate + spring and want to store/load timestamps in UTC. I've read that I should add a property, so I added this to my application.properties
spring.jpa.properties[hibernate.jdbc.time_zone]=UTC
This worked for one part of the problem - now dates are saved in utc in the database. But when I retrieve timestamps, they are transformed into default timezone. How can I fix this without setting default time zone to UTC?
The property of the entity has type LocalDateTime.
I ran the code, and noticed that the proper result set method is used during get(the one that accepts calendar) with instance that has zone info storing UTC. But after setting calendar's values to the one retrieved from the database, the calendar is transformed into Timestamp with this code
Timestamp ts = new Timestamp(c.getTimeInMillis());
In debug mode, I see that ts stores cdate field with value of timestamp in default time zone(not UTC).

First of all, if we are talking about Hibernate 5 (5.2.3 - 5.6.x if to be precise) the purpose of hibernate.jdbc.time_zone setting is not to give the ability for application developer to implement some kind of sophisticated date/time logic, but to synchronize persistence provider with underlying database, that is clearly stated in the corresponding CR:
Currently my database has implicit date times in UTC. No zoned data is appended to the end of the string (e.g. "2013-10-14 04:00:00").
When Hibernate reads this as a ZonedDateTime, it incorrectly reads it in as EST, as that is the TimeZone of the JVM.
It would be nice to be able to specify the TimeZone of a field by an annotation perhaps.
basically: you definitely need to set up hibernate.jdbc.time_zone if (mine: and only if) SQL statement like SELECT SYSDATE FROM DUAL (SELECT LOCALTIMESTAMP for PostgreSQL, etc) returns something, what you do not expect, in that case Hibernate will start adjusting non-timezone-aware JDBC data to something more or less reliable for application - that is exactly what you are observing (when I retrieve timestamps, they are transformed into default timezone)
At second, any speculations around JSR-310 and JDBC 4.2 (like for timezone-aware java types you need to define DB columns as timestamp with time zone), are not correct in case of Hibernate 5, that is mentioned in the corresponding CR as well:
The whole idea of "stored TZ" really depends on how the database/driver treats TIMESTAMP and whether it supports a "TIMESTAMP_WITH_TIMEZONE" type. I personally think it is a huge mistake to save the specific TZ differences to the DB, so I would personally continue to not support TIMESTAMP_WITH_TIMEZONE types. This would mean we never have to bind the Calendar because we could simply convert the value to to the JVM/JDBC TZ ourselves. Specifically I would suggest that we (continue to) assume that the driver has been set up such that the same TZ is used when ...
And indeed, if you try to find usage of java.sql.Types#TIMESTAMP_WITH_TIMEZONE in Hibernate 5 sources you will find nothing, just because that time Hibernate developers didn't get a common opinion about how timezone conversions should work in cases of different Java versions, Java types, DB engines and JDBC drivers (they are developing the most popular (mine: the only one) JPA implementation, that is definitely not the same as develop microservice), however, there are a lot of related changes in Hibernate 6 (check TimeZoneStorageType for example). In Hibernate 5 all timezone conversion logic passes through TimestampTypeDescriptor:
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return options.getJdbcTimeZone() != null ?
javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance( options.getJdbcTimeZone() ) ), options ) :
javaTypeDescriptor.wrap( rs.getTimestamp( name ), options );
}
and as you can see, Hibernate 5 just gives a hint to JDBC driver, how the last should process #getTimestamp call:
Retrieves the value of a JDBC TIMESTAMP parameter as a java.sql.Timestamp object, using the given Calendar object to construct the Timestamp object. With a Calendar object, the driver can calculate the timestamp taking into account a custom timezone and locale. If no Calendar object is specified, the driver uses the default timezone and locale.
in regard to your case:
you either need to use timezone-aware java types (ZonedDateTime/OffsetDateTime, or even Instant) or code your own Hibernate type, which will handle timezone conversions - that is not so hard as it might seem.

we can also set it up` per-session basis:
session = HibernateUtil.getSessionFactory().withOptions()
.jdbcTimeZone(TimeZone.getTimeZone("UTC"))
.openSession();

My database timezone was in UTC and in my application timezone I solved this problem by having both the Entity and the table have a date in UTC so that there will need to be no conversion between them. Then I did the conversions between timestamps in code in the getters and setters. Then I think you do it manually.
Setter and getter for that field:
public void setCreatedDate(LocalDateTime createdAt)
{
this.createdAt = createdAt.atZone(ZoneId.systemDefault()).withZoneSameInstant(ZoneOffset.UTC).toLocalDateTime();
}
public LocalDateTime getCreatedDate()
{
return createdAt.atZone(ZoneId.systemDefault()).withZoneSameInstant(ZoneOffset.UTC).toLocalDateTime();
}

As alluded to be Andrey in his answer, in Hibernate 6 the way to normalize dates/times to UTC is to use the java.time types OffsetDateTime or ZonedDateTime, and either:
annotate the field #TimeZoneStorage(NORMALIZE_UTC), or
set the property hibernate.timezone.default_storage=NORMALIZE_UTC.
I'm not very certain what you mean about using LocalDateTime here. What would it even mean to normalize a local datetime to UTC? That statement just doesn't really make sense: you can't move a local datetime to a new time zone because it doesn't have an associated time zone to begin with.
I think what you mean is that your "local" date times are actually zoned datetimes in the current JVM time zone. But if that's the case, it's very easy to use localDateTime.atZone(ZoneId.systemDefault()) to represent that situation correctly.

Why does the timezone specified in the JDBC connection string effect how Instants are stored in MySQL?

In our project (Spring Boot 2.2.3, MySQL 5.7, Hibernate, Java 14) we are having all date-related fields as datatype java.time.Instant. In our MySQL all fields are of type DATETIME.
When I specify a connectionString like jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=Europe/Paris for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T02:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
The display of the time in the table view in IntelliJ doesn't seem to effected by the serverTimeZone I set, so I hope it displays the plain value as stored in the DB.
When I change the connectionString to jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=UTC for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T00:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
So it looks like I have an Instant, Java/MySQL assumes it's UTC and converts it to the timezone specified in the connectionstring and therefore adding the one hour during winter time/two hours during summer time for my timezone.
What I would like to understand is who performs these timezone adaptions and why. Because I understood from the MySQL documentation, that these adaptions should not happen for type DATETIME.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.) By default, the current time zone for each connection is the server's time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored.

MySQL 'DATETIME' stores sign, year, day, hour, minute, second, and fractional second. It is the equivalent of LocalDateTime.
Yet, you are writing epoch millis (in new time API terms: java.time.Instant) to it. Traditionally, datetime values in databases were presumed to be instant-based (as in, epoch-millis, not human YMDhms based), hence why java.sql.Timestamp extends java.util.Date (note that j.u.Date is a bald-faced lie. It represents epoch-millis, not date at all). The proof of that is also found in the API: All methods except those that fetch/set epoch-millis are deprecated in j.u.Date.
Note that the modern JDBC spec in fact requires support for the new java.time types such as LocalDate, and I can confirm that e.g. the JDBC drivers for postgres get this right.
In the past, the JDBC APIs would grow new methods - there'd be a .setLocalTime(idxOfQuestionMark, localTimeInstance) method in PreparedStatement, and a .getLocalTime(idxOrNameOfColumn) in ResultSet, for example. But, no longer. Any newly added types are to be used thusly:
LocalTime lt = resultSet.getObject(idxOrNameOfColumn, LocalTime.class);
preparedStatement.setObject(paramIndex, lt);
The first thing to try to do is to get the way mysql stores data, and the way java represents the data it got from mysql / sends to mysql, to line up. Because if the MySQL db has a YMDhms value and the only way your java code can observe this value is via an object whose inner storage allows it to represent only epoch-millis, well, guess what? Somebody somewhere is doing a timezone-based conversion because you can't go from epochmillis to YMDhms or vice versa without it. If you then convert right back you're just introducing opportunities for error.
However, it's mysql, and mysql is not a very good database, so odds are good that the above doesn't work (even though the JDBC spec more or less demands support for LocalTime, LocalDate, LocalDateTime, and ZonedDateTime, and LDT is an excellent match for what MySQL's DATETIME columns actually contain). So, if that doesn't work....
you're going to have to dance around it, and accept that conversion occurs. That will mean that what you're observing (connection timezone has an effect on what you read) will remain. One solution to that is to forget about DATETIME (after all, if indeed LDT instances can't be sent to/received from the JDBC MySQL driver), you have no way to reliably set or get such columns at all. Redesign your DB definitions to use TIMESTAMP WITH TIMEZONE instead (this is like ZonedDateTime, which is close enough to Instant (epoch-millis) that conversion shouldn't be an issue any more, though it's still suboptimal). Lock down the zone on both sides, and you can reliably convert from epoch-millis to that zone and back.
Or, better yet, find a better DB engine :)

Hibernate Ignores DB Timezone Info when Binding to Java Timestamp Object

I have some Dates stored in Oracle with Oracle's TIMESTAMP(3) as its datatype. Now I'm writing a Spring boot app to read those values back. The code is like:
HibernateCallback callback = new HibernateCallback() {
public Object doInHibernate(Session session) throws HibernateException {
Query query = session.createSQLQuery("SELECT date_field FROM some_table WHERE some_conditions");
return query.list();
}
};
So:
List results = (List)getHibernateTemplate().execute(callback);
// suppose there's only one row and one column returned
Timestamp ts = result.get(0)[0];
returns me the Java Timestamp object automatically created by Hibernate. The problem is that, when constructing the object, it ignores the timezone stored in Oracle, but instead uses JVM's default timezone. I tested is by setting different timezones for the JVM, and each time it generates a different timestamp.
It's obviously wrong. The Date should be unique on the time line. It shouldn't depend on JVM's timezone. I'm wondering what's the correct way to include the DB's timezone info when parsing the date. Right now it seems it's just using the String representation stored in Oracle and parse it with JMV's timezone. I'm using Hibernate 4.3.4.Final.
PS: The actual query is high customized so I have to write raw SQL.

Basically, that's not even an issue with Hibernate but with JDBC. By default, JDBC Driver will use system time zone on which JVM is running. If you are connecting to the DB server at a different time zone or even if you want to be independent of the current timezone of the system it is a good idea to set JDBC Time Zone explicitly.
You can use hibernate.jdbc.time_zone property to set the timezone or do it at runtime via.
session = sessionFactory.withOptions()
.jdbcTimeZone(TimeZone.getTimeZone("UTC"))
.openSession();
Also for Oracle, I would say you can use the TIMESTAMP WITH LOCAL TIME ZONE which will respect you JDBC client time zone.

The problem is with Oracle column data type you are using, If you go to official oracle docs, link, you will notice the TIMESTAMP doesn't respect timezone, so you should go with with either TIMESTAMP WITH TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE.

It seems the issue is with few concepts of date handling
Dates in DB should be in UTC, if they are to be used in multiple timezone.
Java program should convert the dates to required timezone.
If you take your date in database as based on UTC, and then check the output of your hibernate query, you should see that date is changed as per JVM's timezone.
If you want the same date back (as was in DB), maybe you should convert the date to UTC timezone.
Now, you can do either of following:
Set the JVM's timezone same as the database record timezone. Not recommended
Change your dates in database as per UTC, and let the dates be changed to JVM's timezone. Recommended

Copying TIMESTAMP to DATETIME on MySQL with Hibernate

I have these two classes
class Source {
// mapped to TIMESTAMP
#Version
#Column(columnDefinition="TIMESTAMP(3) DEFAULT '2016-01-01'")
Instant myInstant;
}
class Destination {
// mapped to DATETIME
#Basic(optional=true)
Instant myInstant;
}
When using Hibernate, I assign
destination.myInstant = source.myInstant;
and then the stored value is smaller by one hour than the original - both according to the command line MySQL client and in Java. My current timezone is UTC+1, so the reason is obviously a timezone conversion.
There are a few places where this can be fixed, but I'm looking for the best practice. The server should work world-wide, so it should continue to use UTC internally, right?
Should I just change the column type to TIMESTAMP? Then, why does Instant by default map to DATETIME?
According to this article, Instant does map to TIMESTAMP, but in my case it did not. Why?

If you want to work with timezones and Java 8 I would recommend using ZonedDateTime or OffsetTimeZone (the latter being prefered when working with Hibernate). For older versions use Calendar.
When you instance it should go by default with the timezone of your computer.
Check if the database is timestamp with or without timezone.
The default you set is also without timezone, and if it is "with timezone" it should automatically add the database's offset.
I hope some of this works. Here's how I did in one of my projects.
#Column(name = "registration_time")
private OffsetDateTime registrationTime;
[...]
subscriber.setRegistrationTime(OffsetDateTime.now());

In MySQL 5 & above, TIMESTAMP values are converted from the current time zone to UTC for storage, and converted back from UTC to the current time zone for retrieval. This occurs only for the TIMESTAMP data type but not for DATETIME. This is the reason you are seeing the difference while assigning a TIMESTAMP to DATETIME. So, having both the columns of same type should work. Hibernate by default maps InstantType to database TIMESTAMP type. Though you could use it for both TIMESTAMP and DATETIME in MYSQL, they are handled differently.

Why is my timestamp shifted in timezone?

I have this date in a PostgreSQL 9.1 database in a timestamp without time zone column:
2012-11-17 13:00:00
It's meant to be in UTC, and it is, which I've verified by selecting it as a UNIX timestamp (EXTRACT epoch).
int epoch = 1353157200; // from database
Date date = new Date((long)epoch * 1000);
System.out.println(date.toGMTString()); // output 17 Nov 2012 13:00:00 GMT
However, when I read this date using JPA/Hibernate, things go wrong. This is my mapping:
#Column(nullable=true,updatable=true,name="startDate")
#Temporal(TemporalType.TIMESTAMP)
private Date start;
The Date I get, however, is:
17 Nov 2012 12:00:00 GMT
Why is this happening, and more importantly, how can I stop it?
Note that I just want to store points in time, universally (as java.util.Date does), and I couldn't care less about timezones, except that I obviously don't want them to corrupt my data.
As you've probably deduced, the client application which connects to the database is in UTC+1 (Netherlands).
Also, the choice for the column type timestamp without time zone was made by Hibernate when it automatically generated the schema. Should that maybe be timestamp with time zone instead?

If a database does not provide timezone information, then the JDBC driver should treat it as if it is in the local timezone of the JVM (see PreparedStatement.setDate(int, Date)):
Sets the designated parameter to the given java.sql.Date value using the default time zone of the virtual machine that is running the application.
The Javadoc and the JDBC specification do not explicitly say anything about ResultSet etc, but to be consistent most drivers will also apply that rule to dates retrieved from the database. If you want explicit control over the timezone used, you will need to use the various set/getDate/Time/Timestamp methods that also accept a Calendar object in the right timezone.
Some drivers also provide a connection property allowing you to specify the timezone to use when converting to/from the database.

I've found that changing the column type to timestamp with time zone fixed the problem. I'll have to convert all my other timestamp columns to that as well then.
From this I conclude that Hibernate does not read timestamp without time zone columns as in UTC, but as in the local timezone. If there is a way to make it interpret them as UTC, please let me know.

There are several solutions to solve the problems:
1) The easiest way is to set the time zone in the JDBC connect string - as long as the database supports this.
For MySQL you can use useGmtMillisForDatetimes=true to force the use of UTC in the database. As far as I know Postgres does not support such an option, but I might be wrong because I don't use Postgres.
2) Set the default time zone in your Java client program with
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
Disadvantage: There you also change the timezone where you don't want, for example if your program has an UI part.
3) Use a mapping with a special getter only for Hibernate:
In your Mapping file
<property name="myTimeWithTzConversion" type="timestamp" access="property">
<column name="..." />
</property>
and in your program you have get/setMyTimeWithTzConversion() for the access only by hibernate to the member variable Timestamp myTime, and in this getter/setter you do the timezone conversion.
We finally decided for 3) (which is a bit more programming work), because there we didn't have to change the existing database, our database was in UTC+1 (which forbids the JDBC connect string solution) and that didn't interfere with the existing UI.