We Keep Coding

Hibernate retrieve timestamps in UTC

I am using hibernate + spring and want to store/load timestamps in UTC. I've read that I should add a property, so I added this to my application.properties
spring.jpa.properties[hibernate.jdbc.time_zone]=UTC
This worked for one part of the problem - now dates are saved in utc in the database. But when I retrieve timestamps, they are transformed into default timezone. How can I fix this without setting default time zone to UTC?
The property of the entity has type LocalDateTime.
I ran the code, and noticed that the proper result set method is used during get(the one that accepts calendar) with instance that has zone info storing UTC. But after setting calendar's values to the one retrieved from the database, the calendar is transformed into Timestamp with this code
Timestamp ts = new Timestamp(c.getTimeInMillis());
In debug mode, I see that ts stores cdate field with value of timestamp in default time zone(not UTC).

First of all, if we are talking about Hibernate 5 (5.2.3 - 5.6.x if to be precise) the purpose of hibernate.jdbc.time_zone setting is not to give the ability for application developer to implement some kind of sophisticated date/time logic, but to synchronize persistence provider with underlying database, that is clearly stated in the corresponding CR:
Currently my database has implicit date times in UTC. No zoned data is appended to the end of the string (e.g. "2013-10-14 04:00:00").
When Hibernate reads this as a ZonedDateTime, it incorrectly reads it in as EST, as that is the TimeZone of the JVM.
It would be nice to be able to specify the TimeZone of a field by an annotation perhaps.
basically: you definitely need to set up hibernate.jdbc.time_zone if (mine: and only if) SQL statement like SELECT SYSDATE FROM DUAL (SELECT LOCALTIMESTAMP for PostgreSQL, etc) returns something, what you do not expect, in that case Hibernate will start adjusting non-timezone-aware JDBC data to something more or less reliable for application - that is exactly what you are observing (when I retrieve timestamps, they are transformed into default timezone)
At second, any speculations around JSR-310 and JDBC 4.2 (like for timezone-aware java types you need to define DB columns as timestamp with time zone), are not correct in case of Hibernate 5, that is mentioned in the corresponding CR as well:
The whole idea of "stored TZ" really depends on how the database/driver treats TIMESTAMP and whether it supports a "TIMESTAMP_WITH_TIMEZONE" type. I personally think it is a huge mistake to save the specific TZ differences to the DB, so I would personally continue to not support TIMESTAMP_WITH_TIMEZONE types. This would mean we never have to bind the Calendar because we could simply convert the value to to the JVM/JDBC TZ ourselves. Specifically I would suggest that we (continue to) assume that the driver has been set up such that the same TZ is used when ...
And indeed, if you try to find usage of java.sql.Types#TIMESTAMP_WITH_TIMEZONE in Hibernate 5 sources you will find nothing, just because that time Hibernate developers didn't get a common opinion about how timezone conversions should work in cases of different Java versions, Java types, DB engines and JDBC drivers (they are developing the most popular (mine: the only one) JPA implementation, that is definitely not the same as develop microservice), however, there are a lot of related changes in Hibernate 6 (check TimeZoneStorageType for example). In Hibernate 5 all timezone conversion logic passes through TimestampTypeDescriptor:
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return options.getJdbcTimeZone() != null ?
javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance( options.getJdbcTimeZone() ) ), options ) :
javaTypeDescriptor.wrap( rs.getTimestamp( name ), options );
}
and as you can see, Hibernate 5 just gives a hint to JDBC driver, how the last should process #getTimestamp call:
Retrieves the value of a JDBC TIMESTAMP parameter as a java.sql.Timestamp object, using the given Calendar object to construct the Timestamp object. With a Calendar object, the driver can calculate the timestamp taking into account a custom timezone and locale. If no Calendar object is specified, the driver uses the default timezone and locale.
in regard to your case:
you either need to use timezone-aware java types (ZonedDateTime/OffsetDateTime, or even Instant) or code your own Hibernate type, which will handle timezone conversions - that is not so hard as it might seem.

we can also set it up` per-session basis:
session = HibernateUtil.getSessionFactory().withOptions()
.jdbcTimeZone(TimeZone.getTimeZone("UTC"))
.openSession();

My database timezone was in UTC and in my application timezone I solved this problem by having both the Entity and the table have a date in UTC so that there will need to be no conversion between them. Then I did the conversions between timestamps in code in the getters and setters. Then I think you do it manually.
Setter and getter for that field:
public void setCreatedDate(LocalDateTime createdAt)
{
this.createdAt = createdAt.atZone(ZoneId.systemDefault()).withZoneSameInstant(ZoneOffset.UTC).toLocalDateTime();
}
public LocalDateTime getCreatedDate()
{
return createdAt.atZone(ZoneId.systemDefault()).withZoneSameInstant(ZoneOffset.UTC).toLocalDateTime();
}

As alluded to be Andrey in his answer, in Hibernate 6 the way to normalize dates/times to UTC is to use the java.time types OffsetDateTime or ZonedDateTime, and either:
annotate the field #TimeZoneStorage(NORMALIZE_UTC), or
set the property hibernate.timezone.default_storage=NORMALIZE_UTC.
I'm not very certain what you mean about using LocalDateTime here. What would it even mean to normalize a local datetime to UTC? That statement just doesn't really make sense: you can't move a local datetime to a new time zone because it doesn't have an associated time zone to begin with.
I think what you mean is that your "local" date times are actually zoned datetimes in the current JVM time zone. But if that's the case, it's very easy to use localDateTime.atZone(ZoneId.systemDefault()) to represent that situation correctly.

Why does the timezone specified in the JDBC connection string effect how Instants are stored in MySQL?

In our project (Spring Boot 2.2.3, MySQL 5.7, Hibernate, Java 14) we are having all date-related fields as datatype java.time.Instant. In our MySQL all fields are of type DATETIME.
When I specify a connectionString like jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=Europe/Paris for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T02:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
The display of the time in the table view in IntelliJ doesn't seem to effected by the serverTimeZone I set, so I hope it displays the plain value as stored in the DB.
When I change the connectionString to jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=UTC for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T00:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
So it looks like I have an Instant, Java/MySQL assumes it's UTC and converts it to the timezone specified in the connectionstring and therefore adding the one hour during winter time/two hours during summer time for my timezone.
What I would like to understand is who performs these timezone adaptions and why. Because I understood from the MySQL documentation, that these adaptions should not happen for type DATETIME.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.) By default, the current time zone for each connection is the server's time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored.

MySQL 'DATETIME' stores sign, year, day, hour, minute, second, and fractional second. It is the equivalent of LocalDateTime.
Yet, you are writing epoch millis (in new time API terms: java.time.Instant) to it. Traditionally, datetime values in databases were presumed to be instant-based (as in, epoch-millis, not human YMDhms based), hence why java.sql.Timestamp extends java.util.Date (note that j.u.Date is a bald-faced lie. It represents epoch-millis, not date at all). The proof of that is also found in the API: All methods except those that fetch/set epoch-millis are deprecated in j.u.Date.
Note that the modern JDBC spec in fact requires support for the new java.time types such as LocalDate, and I can confirm that e.g. the JDBC drivers for postgres get this right.
In the past, the JDBC APIs would grow new methods - there'd be a .setLocalTime(idxOfQuestionMark, localTimeInstance) method in PreparedStatement, and a .getLocalTime(idxOrNameOfColumn) in ResultSet, for example. But, no longer. Any newly added types are to be used thusly:
LocalTime lt = resultSet.getObject(idxOrNameOfColumn, LocalTime.class);
preparedStatement.setObject(paramIndex, lt);
The first thing to try to do is to get the way mysql stores data, and the way java represents the data it got from mysql / sends to mysql, to line up. Because if the MySQL db has a YMDhms value and the only way your java code can observe this value is via an object whose inner storage allows it to represent only epoch-millis, well, guess what? Somebody somewhere is doing a timezone-based conversion because you can't go from epochmillis to YMDhms or vice versa without it. If you then convert right back you're just introducing opportunities for error.
However, it's mysql, and mysql is not a very good database, so odds are good that the above doesn't work (even though the JDBC spec more or less demands support for LocalTime, LocalDate, LocalDateTime, and ZonedDateTime, and LDT is an excellent match for what MySQL's DATETIME columns actually contain). So, if that doesn't work....
you're going to have to dance around it, and accept that conversion occurs. That will mean that what you're observing (connection timezone has an effect on what you read) will remain. One solution to that is to forget about DATETIME (after all, if indeed LDT instances can't be sent to/received from the JDBC MySQL driver), you have no way to reliably set or get such columns at all. Redesign your DB definitions to use TIMESTAMP WITH TIMEZONE instead (this is like ZonedDateTime, which is close enough to Instant (epoch-millis) that conversion shouldn't be an issue any more, though it's still suboptimal). Lock down the zone on both sides, and you can reliably convert from epoch-millis to that zone and back.
Or, better yet, find a better DB engine :)

Does Oracle persist Java Calendar into TIMESTAMP with timezone?

We are persisting a Java Calendar object into an Oracle TIMESTAMP column using the getTimeInMillis() method. Does Oracle persist this data with a time zone?
I'm assuming not as we are not using the TIMESTAMP WITH TIMEZONE data type. If not, when you query SELECT MY_TIMESTAMP FROM MYTABLE what timezone will Oracle associate with this field?
And what is the correct way to 'cast' this to the timezone you are interested in (ie: UTC)?

getTimeInMillis()
is independent of the calendar system and the timezone. So in whichever timezone you specify the time in, the time is not gonna vary and getTimeInMillis() will get the accurate time.
And on the persistence -
yes the oracle persist the data with the timezone. But it will add the timezone offset to the value saved.
So that's brings us to the next question of choosing the right
timezone -
it is ideal to choose the native timezone. As this scenario is a combination of oracle and java its best to use UTC
Cheers!

How to get UTC timestamps from JDBC+postgreSql timestamp?

I created a table like like this in PostgreSQL:
create table myTable (
dateAdded timestamp(0) without time zone null default (current_timestamp at time zone 'UTC');
)
I choose "without time zone" because I know that all timestamps that my application works with are always UTC. As far as I got the documentation the only difference to "with timestamp" is that I can supply values in other time zones which will then be converted to UTC. However I want to avoid such automatic conversions because they could hardly do any good if I know that my values are UTC.
When I add a new record in my test table and view the table's content with pgAdmin I can see that the insertion date has been correctly saved in UTC format.
However when I try to select values using JDBC the value gets 2 hours subtracted. I am located at UTC+2, so it looks like that JDBC assumes that the date in the table is not a UTC timestamp, but a UTC+2 timestamp instead and tries to convert to UTC.
Some googling revealed that the JDBC standard dictates something about conversion to/from the current time zone, but that this could be prevented by supplying a Calander to getTimestamp/setTimestamp calls. However supplying a calendar did not make any difference at all. Here is my MyBatis/Jodatime converter:
#MappedTypes(DateTime.class)
public class DateTimeTypeHandler extends BaseTypeHandler<DateTime> {
private static final Calendar UTC_CALENDAR = Calendar.getInstance(DateTimeZone.UTC.toTimeZone());
#Override
public void setNonNullParameter(PreparedStatement ps, int i,
DateTime parameter, JdbcType jdbcType) throws SQLException {
ps.setTimestamp(i, new Timestamp(parameter.getMillis()), UTC_CALENDAR);
}
#Override
public DateTime getNullableResult(ResultSet rs, String columnName)
throws SQLException {
return fromSQLTimestamp(rs.getTimestamp(columnName, UTC_CALENDAR));
}
/* further get methods with pretty much same content as above */
private static DateTime fromSQLTimestamp(final Timestamp ts) {
if (ts == null) {
return null;
}
return new DateTime(ts.getTime(), DateTimeZone.UTC);
}
}
What's the correct way to get UTC timestamps from JDBC+PostgreSQL timestamp source?

Solution
Set UTC as default timezone of your JVM -Duser.timezone=UTC or set your whole OS to UTC.
Background
In Postgres both TIMESTAMP and TIMESTAMP WITH TIMEZONE are stored the same way - number of seconds since Postgres epoch (2000-01-01). The main difference is what Postgres do when it saves timestamp value such as 2004-10-19 10:23:54+02:
without TZ the +02 is just stripped away
with TZ a -02 correction is performed to make it UTC
Now the interesting thing is when JDBC driver loads the value:
without TZ the stored value is shifted by the user's (JVM / OS) TZ
with TZ the value is considered to be UTC
In both cases you will end up with java.sql.Timestamp object with user's default TZ.
Time Zones
Timestamps without TZ are pretty limited. If you have two systems connected to your database, both with different TZ, they will interpret timestamps differently. Therefore, I strongly advice you to use TIMESTAMP WITH TIMEZONE.
Update
You can tell JDBC what kind of TZ it should use when reading timestamp via ResultSet#getTimestamp(String, Calendar). Excerpt from JavaDoc:
This method uses the given calendar to construct an appropriate millisecond value for the timestamp if the underlying database does not store timezone information.

There are a few tricks specific for the Postgres JDBC driver
See https://jdbc.postgresql.org/documentation/head/java8-date-time.html
So when reading you can do
Instant utc =resultSet.getObject("dateAdded",LocalDateTime.class).toInstant(ZoneOffset.UTC);
If you use a connection pool such as Hikari, you can also specify the time time-zone used by each connection by setting connectionInitSql=set time zone 'UTC'

The solution proposed by Pavel (adding the jvm param '-Duser.timezone=UTC') is for sure the best option, if you don't have system access, this isn't always possible to do.
The only way I found is to convert the timestamp to epoch in the query and read it as a long.
SELECT extract(epoch from my_timestamp_colum at time zone 'utc')::bigint * 1000
AS my_timestamp_as_epoc
FROM my_table
Then read it in plain JDBC with
ResultSet rs = ...
long myTimestampAsEpoc = rs.getLong("my_timestamp_as_epoc");
Date myTimestamp = new Date(myTimestampAsEpoc);
With iBatis/MyBatis, you need to handle the column as a Long, then convert it manually to Date/Timestamp. Inconvenient and ugly.
The reverse operation in PostgreSQL can be done with:
SELECT TIMESTAMP WITHOUT TIME ZONE 'epoch' +
1421855729000 * INTERVAL '1 millisecond'
That said, the JDBC specification doesn't say that the returned timestamp shall or shall not shift the timestamp to the user time zone; BUT since you defined the table column as 'timestamp without time zone' I would expect no time shifts, but the recorded epoch being just wrapped in a java.sql.Timestamp. For my opinion, this is a bug in the PostgreSQL JDBC driver. Being the problem at this level, then, probably there is not much more that can be done, without system access.

How to store a java.util.Date into a MySQL timestamp field in the UTC/GMT timezone?

I used a new Date() object to fill a field in a MySQL DB, but the actual value stored in that field is in my local timezone.
How can I configure MySQL to store it in the UTC/GMT timezone?
I think, configuring the connection string will help but I don't know how. There are many properties in the connection string like useTimezone, serverTimzone, useGmtMillisForDatetimes, useLegacyDatetimeCode, ...

The short answer is:
add "default-time-zone=utc" to my.cnf
in your code, always "think" in UTC, except when displaying dates for your users
when getting/setting dates or timestamps with JDBC, always use the Calendar parameter, set to UTC:
resultset.getTimestamp("my_date", Calendar.getInstance(TimeZone.getTimeZone("UTC")));
either synchronize your servers with NTP, or rely only on the database server to tell you what time it is.
The long answer is this:
When dealing with dates and timezones in any database and with any client code, I usually recommend the following policy:
Configure your database to use UTC timezone, instead of using the server's local timezone (unless it is UTC of course).
How to do so depends on your database server. Instructions for MySQL can be found here: http://dev.mysql.com/doc/refman/5.0/en/time-zone-support.html. Basically you need to write this in my.cnf: default-time-zone=utc
This way you can host your database servers anywhere, change your hosting location easily, and more generally manipulate dates on your servers without any ambiguity.
If you really prefer to use a local timezone, I recommend at least turning off Daylight Saving Time, because having ambiguous dates in your database can be a real nightmare.
For example, if you are building a telephony service and you are using Daylight Saving Time on your database server then you are asking for trouble: there will be no way to tell whether a customer who called from "2008-10-26 02:30:00" to "2008-10-26 02:35:00" actually called for 5 minutes or for 1 hour and 5 minutes (supposing Daylight Saving occurred on Oct. 26th at 3am)!
Inside your application code, always use UTC dates, except when displaying dates to your users.
In Java, when reading from the database, always use:
Timestamp myDate = resultSet.getTimestamp("my_date", Calendar.getInstance(TimeZone.getTimeZone("UTC")));
If you do not do this, the timestamp will be assumed to be in your local TimeZone, instead of UTC.
Synchronize your servers or only rely on the database server's time
If you have your Web server on one server (or more) and your database server on some other server, then I strongly recommend you synchronize their clocks with NTP.
OR, only rely on one server to tell you what time it is. Usually, the database server is the best one to ask for time. In other words, avoid code such as this:
preparedStatement = connection.prepareStatement("UPDATE my_table SET my_time = ? WHERE [...]");
java.util.Date now = new java.util.Date(); // local time! :-(
preparedStatement.setTimestamp(1, new Timestamp(now.getTime()));
int result = preparedStatement.execute();
Instead, rely on the database server's time:
preparedStatement = connection.prepareStatement("UPDATE my_table SET my_time = NOW() WHERE [...]");
int result = preparedStatement.execute();
Hope this helps! :-)

MiniQuark gave some good answers for databases in general, but there are some MySql specific quirks to consider...
Configure your database to use UTC timezone
That actually won't be enough to fix the problem. If you pass a java.util.Date to MySql as the OP was asking, the MySql driver will change the value to make it look like the same local time in the database's time zone.
Example: Your database if configured to UTC. Your application is EST. You pass a java.util.Date object for 5:00 (EST). The database will convert it to 5:00 UTC and store it. Awesome.
You'd have to adjust the time before you pass the data to "undo" this automatic adjustment. Something like...
long originalTime = originalDate.getTime();
Date newDate = new Date(originalTime - TimeZone.getDefault().getOffset(originalTime));
ps.setDate(1, newDate);
Reading the data back out requires a similar conversion..
long dbTime = rs.getTimestamp(1).getTime();
Date originalDate = new Date(dbTime + TimeZone.getDefault().getOffset(dbTime));
Here's another fun quirk...
In Java, when reading from the database, always use: Timestamp myDate
= resultSet.getTimestamp("my_date", Calendar.getInstance(TimeZone.getTimeZone("UTC")));
MySql actually ignores that Calendar parameter. This returns the same value regardless of what calendar you pass it.

I had the same problem, and it took me nearly a day to track down. I'm storing DateTime columns in MySQL. The RDS instance, running in Amazon's Cloud, is correctly set to have a UTC timestamp by default.
The Buggy Code is:
String startTime = "2013-02-01T04:00:00.000Z";
DateTime dt = ISODateTimeFormat.dateTimeParser().parseDateTime(startTime);
PreparedStatement stmt = connection.prepareStatement(insertStatementTemplate);
Timestamp ts = new Timestamp(dt.getMillis());
stmt.setTimestamp(1, ts, Calendar.getInstance(TimeZone.getTimeZone("UTC")));
In the code above, the ".setTimestamp" call would NOT take the date as a UTC date!
After hours of investigating, this turns out to be a known bug in the Java / MySQL Driver. The call to setTimestamp listerally just ignores the Calendar parameter.
To fix this add the "useLegacyDatetimeCode=false" to your database URI.
private final static String DatabaseName =
"jdbc:mysql://foo/?useLegacyDatetimeCode=false";
As soon as i did that, the date stored in the MySQL database was in proper UTC form, rather than in the timezone of my local workstation.

Well, if we're talking about using PreparedStatements, there's a form of setDate where you can pass in a Calendar set to a particular time zone.
For instance, if you have a PreparedStatement named stmt, a Date named date, and assuming the date is the second parameter:
stmt.setDate(2, date, Calendar.getInstance(TimeZone.getTimeZone("GMT")));
The best part is, even if GMT is an invalid time zone name, it still returns the GMT time zone because of the default behavior of getTimeZone.

A java Date is timezone agnostic. It ALWAYS represents a date in GMD(UTC) in milliseconds from the Epoch.
The ONLY time that a timezone is relevant is when you are emitting a date as a string or parsing a data string into a date object.