Spring Boot 3 (Hibernate 6.1) in combination with PostgreSQL seems to have a problem with Instant values mapped to SQL timestamps (without time zone), as the following test case fails. Values read differ from the written values by the amount of the local time zone offset.
The test case executes successfully when using the H2 test database or when switching back to Spring Boot 2.7.6 (Hibernate 5.6).
JPA entity:
#Entity
public class MyEntity {
#Id
UUID id = UUID.randomUUID();
//#JdbcType(TimestampJdbcType.class) //work-around to get it somehow working
//(alternative: declare this globally in a custom Hibernate dialect)
//but why is this neccessary only with spring-boot-3 and postgres?
#Column(columnDefinition = "timestamp")
Instant createdTimestamp = Instant.now();
}
Spring Data repository:
public interface MyEntityRepository extends CrudRepository<MyEntity, UUID> {
}
JUnit integration test:
#DataJpaTest
#AutoConfigureTestDatabase(replace = Replace.NONE) //comment out to use the H2 database
class MyEntityRepositoryITest {
#Autowired
MyEntityRepository myEntityRepository;
#Test
#Transactional(propagation = NOT_SUPPORTED) //do not use a transaction
void readInstantSameAsPersisted() {
var entity = new MyEntity();
myEntityRepository.save(entity);
var read = myEntityRepository.findById(entity.id).orElseThrow();
assertEquals(entity.createdTimestamp.getEpochSecond(), read.createdTimestamp.getEpochSecond());
}
}
Maven pom.xml:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
Spring application.properties:
spring.jpa.hibernate.ddl-auto=create
spring.datasource.url=jdbc:postgresql://localhost:5432/dbname
spring.datasource.username=username
spring.datasource.password=password
Notes:
Setting spring.jpa.properties.hibernate.jdbc.time_zone to any value doesn't help.
I don't see switching to LocalDateTime as an option, as I want the createdTimestamp to represent an unambiguous point in time.
I don't see switching to a SQL type with zone information as an option, as I don't want to allow values with varying zones in the database.
spring.datasource.hikari.connection-init-sql=SET TIME ZONE 'UTC' also works as a work-around, but I don't see why this is neccessary -- reading a value that is based on seconds since 1970-01-01 should give the same result as the written value, regardless of which zone is used behind the scenes.
pgJDBC does not support Instant, but Hibernate does.
This answer could be related, but then why does it work with the older Hibernate?
Update
This answer says timestamp with time zone does (unlike the name suggests) in fact not carry additional zone information and just "stores a point on the UTC time line" that can be represented in different zones, which seems like a better fit for my use case.
The PostgreSQL documentation says:
For timestamp with time zone, the internally stored value is always in UTC... An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone.
If you declare the field as timestamp with time zone, or briefly as timestamptz, then you will reach exactly what you want. It will also fit your requirement not to have multiple timezones in the database.
Related
I am having an issue upgrading Spring Boot Starter Parent from 2.0.9.RELEASE to 2.3.3.RELEASE. To be more specific, I've traced the issue down to migrating from 2.0.9.RELEASE to 2.1.0.RELEASE. Prior to 2.1.0.RELEASE all of our criteria builder predicates that were doing an equals comparison like the below example were working. With the upgrade to 2.1.0.RELEASE we are no longer able to do equals comparisons as no matching row is returned.
Predicate foreverDatePredicate = criteriaBuilder.equal(root.get("deactivatedAt"), LocalDateTime.of(9999, Month.DECEMBER, 31, 0, 0, 0))
It does appear that with these two Spring Boot releases they went from Hibernate Core 5.2.18.FINAL to 5.3.7.FINAL and possibly JPA 2.1 to 2.2. I'm not sure if these version changes are related to the below problem.
The issue appears to be happening because of a timezone mismatch between what is being used as the timezone locally (PST) and the timezone that is being stored in the database (UTC). In my sql editor I was able to verify the hibernate query works just fine and returns the appropriate rows when a timezone is not supplied in the where clause. All of our datetime columns are being stored in our MySQL database in UTC time. I tried using the following timezone related properties in application.properties and our Hibernate JPA properties but both appeared to either not work or just be completely ignored:
# application.properties
spring.jpa.properties.hibernate.jdbc.time_zone=UTC
# Used in jpaProperties property under entityManagerFactory
hibernate.jdbc.time_zone=UTC
As some other threads have suggested I can set the JVM timezone to UTC, and after trying it this works; however, I would rather avoid doing this as it seems like a pretty far-reaching fix for something that should be able to be resolved at a more granular level:
#PostConstruct
public void started() {
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
}
I eventually converted our Hibernate entity types that are responsible for mapping to the MySQL datetime columns from LocalDateTime to OffsetDateTime as shown below. OffsetDateTime stores the full timestamp with timezone in UTC time by default and this has resolved the issue and our datetime equals predicates are working again but this has caused a major refactoring headache.
Changed from
#MappedSuperclass
public class BitemporalEntity implements Serializable {
private static final long serialVersionUID = 2966895098986638738L;
#Column(name = "deactivated_at")
protected LocalDateTime deactivatedAt;
To
#MappedSuperclass
public class BitemporalEntity implements Serializable {
private static final long serialVersionUID = 2966895098986638738L;
#Column(name = "deactivated_at")
protected OffsetDateTime deactivatedAt;
I'm just wondering if I'm missing something here. Is using OffsetDateTime instead of LocalDateTime a good resolution to the issue or is there another way?
I want to use a datetimeoffset column in SQL Server 2008 that stores both the date & time plus the timezone.
The DDL definition of the column in the database:
LastUpdatedDateTime datetimeoffset DEFAULT SysDateTimeOffset()
The Java definition of the field on my entity mapped using Hibernate:
private ZonedDateTime lastUpdatedDateTime = ZonedDateTime.now();
I'm using Hibernate 5.1.0.Final (via Spring boot 1.3.2.RELEASE) and including org.hibernate:hibernate-java8.
Observed behavior (by querying database using WinSQL):
Insert data via a SQL insert statement results in storing the correct date&time and correct timezone: 2016-03-03 13:41:17.5358944 -07:00
Insert data via saving Java entity (with field initialized as per Java code fragment above).
Java reports the date/time value (before save) as: 2016-03-04T14:18:17.076-07:00[America/Denver]
After the save, WinSQL reports the value stored in the database as: 2016-03-04 14:18:17.0760000 +00:00
This has the same date&time, but the wrong timezone (UTC rather than -07:00).
When I declared the field in Java using Timestamp instead of ZonedDateTime, I got the same behavior.
How do I get the timezone to be correctly stored? I don't really care if it is stored as UTC or -07:00 time zone as long as the time is correct based on the time zone. I would think that Hibernate would provide support for this (in the hibernate-java8 library) and that I wouldn't have to code a custom converter or custom user data type.
I finally found a solution:
Use java.time.OffsetDateTime instead of ZonedDateTime for the entity field. As per the class Javadoc, OffsetDateTime is intended for use in database persistence.
Revert back to Hibernate 4 (due to other issues I was having). So I don't know if the next steps are necessary if you are using Hibernate 5 with the hibernate-java8 library.
Add a Hibernate Converter from OffsetDateTime to String. Apparently the datetimeoffset column is being treated by default by JDBC as a String (and not a microsoft.sql.DateTimeOffset class as suggested by the Microsoft JDBC driver documentation). The logic in this converter had to deal with the complication that SQL Server only stores 7 digits for nanoseconds while OffsetDateTime provides 9.
Ensure the Converter is included in the Hibernate EntityManager.
The details on these steps are as follows:
DDL column definition is unchanged.
Entity field definition:
private OffsetDateTime lastUpdatedDateTime;
Converter class:
#Converter(autoApply = true)
public class OffsetDateTimeConverter implements AttributeConverter<OffsetDateTime, String> {
private static DateTimeFormatter FORMATTER_FROM_DB = DateTimeFormatter.ofPattern(
"yyyy-MM-dd HH:mm:ss.nnnnnnn xxx");
private static DateTimeFormatter FORMATTER_TO_DB = DateTimeFormatter.ofPattern(
"yyyy-MM-dd HH:mm:ss.nnnnnnnnn xxx");
#Override
public String convertToDatabaseColumn(OffsetDateTime attribute) {
if (attribute == null) {
return null;
}
return attribute.format(FORMATTER_TO_DB);
}
#Override
public OffsetDateTime convertToEntityAttribute(String dbData) {
if (dbData == null) {
return null;
}
return OffsetDateTime.parse(dbData, FORMATTER_FROM_DB);
}
}
I've switched to Java 8 and Hibernate 5 to overcome the problem of not being able to store milliseconds in Hibernate.
private LocalDateTime date = LocalDateTime.now();
public LocalDateTime getDate() {
return date;
}
Maven dependencies
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.0.1.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-java8</artifactId>
<version>5.0.1.Final</version>
</dependency>
Despite this Hibernate is still storing the LocalDateTime object as tinyblob. What am I missing? I'm using MySQL 5.6.19
According to this
Hibernate Issue, the hibernate-java8 module should map this to TIMESTAMP. I've tried placing a #Column(columnDefinition="TIMESTAMP) on the getter, but this led to a DataIntegrityViolationException.
Placed #Type(type="org.hibernate.type.LocalDateTimeType") on the getter. Database table still persisting as tinyblob.
As of this comment:
Hibernate is at version 5.2.10.Final
hibernate-java8 is deprecated and all functionality is included in hibernate-core
Hibernate provides type converters for all the JDK date/time types:
OffsetDateTime getters on entities should be annotated with #Type(type= "org.hibernate.type.OffsetDateTimeType")
LocalDateTime getters on entities should be annotated with #Type(type= "org.hibernate.type.LocalDateTimeType")
...etc.
Mysql's timestamp does not have millisecond precision so it wouldn't be able to store it as a timestamp, and probably detects this and saves it as a blob.
The reason it stores it as a TINYBLOB is that JPA 2.1 was released before Java 8 and the Date and Time API didn’t exist at that point in time.
To overcome this, you will need to define the mapping to java.sql.Date or java.sql.Timestamp yourself.
In my hibernate-4 entity, I am mapping a joda-time DateTime property using the recommended jadira usertypes:
#Entity
#Table(name="timing")
public class TimingEntity {
...
#Basic(optional=false)
#Column(name="moment")
#Type(type="org.jadira.usertype.dateandtime.joda.PersistentDateTime")
public DateTime getMoment() {
...
My database is MySQL. With hibernate property hbm2ddl.auto set to create value, I have the following column generated in my timing table:
CREATE TABLE `timing` (
...
`moment` DATETIME NOT NULL,
...
)
The generated CREATE TABLE contains the DATETIME column. The DATETIME in MySQL has only seconds precision, without fractional part. In order to enable fractional part, up to microseconds, MySQL 5.6.4 and higher enables DATETIME(precision) columns, for example DATETIME(3) to have milliseconds precision.
My question is -- is there way to specify precision for my temporal fields generated with hbm2ddl? At least, is this a matter of jadira usertypes, or java.sql, or jdbc driver machinery?
P.S. When I manually modify the DB table to have the exact column precision I want, say, DATETIME(3), everything works OK - joda DateTimes are written and read from the DB with milliseconds precision.
I've found one more solution that allows not to hardcode MySQL column definition snippet in your #Column annotation. Define your own hibernate dialect by overriding org.hibernate.dialect.MySQLDialect:
package org.yourproject;
import java.sql.Types;
import org.hibernate.dialect.MySQL5Dialect;
public class MySQL564PlusDialect extends MySQL5Dialect {
public MySQL564PlusDialect() {
super();
registerColumnType( Types.TIMESTAMP, 6, "datetime($l)" );
}
}
and specify it as hibernate property hibernate.dialect=org.yourproject.MySQL564PlusDialect (the dialect you'll want to extend may vary, e.g. org.hibernate.dialect.MySQL5InnoDBDialect instead).
Now you can adjust precision of your DATETIME from within #Column annotation by using length attribute:
#Basic(optional=false)
#Column(name="moment", length=3)
#Type(type="org.jadira.usertype.dateandtime.joda.PersistentDateTime")
public DateTime getMoment() {
...
which will produce DATETIME(3) column definition meaning milliseconds precision. If you need the simple DATETIME (no fractional seconds), just don't specify length. You can use value of length up to 6 which would mean microseconds precision.
If you happen to use a dialect different from the above one (for example the standard org.hibernate.dialect.MySQLDialect or maybe some other database), that will not break your code: the length attribute on #Column will be ignored.
P.S. It would be more sensible to exploit the precision attribute of #Column instead of length, but simple replacing of "datetime($l)" pattern with "datetime($p)" one in my own dialect implementation does not work offhand.
Use can use #Column#columnDefinition property for it
#Basic(optional=false)
#Column(name="moment" columnDefinition="DATETIME(3) NOT NULL")
#Type(type="org.jadira.usertype.dateandtime.joda.PersistentDateTime")
public DateTime getMoment()
{
...
Also you can try #Column#precision property, but in documentation is written that this working only for decimals.
I have a problem about the management of dates with milliseconds. I understand the need to use the TIMESTAMP to store milliseconds:
#Temporal(TIMESTAMP)
#Column(name="DATE_COLUMN", nullable = false)
#Override public java.util.Date getDate() { return this.date; }
But if I can't compare this date to another instance of java.util.Date, unless I pay attention to the order of equals() call, because this.date instance is a java.sql.Timestamp. How to get a java.util.Date from JPA ? Because the date that comes from JPA, even if the method signature is a java.util.Date is actually an instance of java.sql.Timestamp.
java.util.Date newDate = new Date(this.date.getTime());
this.date.equals(newDate) == false
newDate.equals(this.date) == true
I've try to modify my method in the persistence class:
#Override
public Date getDate() {
return this.date == null ? null : new Date(this.date.getTime());
}
It's working, but it's not efficient with lots of data.
There are other options :
I could modify the design of my persistence class, using #PostLoad in order to create a java.util.Date from the persited date after I retrieve it.
I wonder if I can not get a result using a ClassTransformer?
Have you ever been confronted with this problem? What I do not correctly? What is the best way to handle this problem?
TBH, I'm not sure of the exact status of this but there might indeed be a problem with the way Hibernate (which is your JPA provider, right?) handles TIMESTAMP columns.
To map a SQL TIMESTAMP to a java.util.Date, Hibernate uses the TimestampType which will actually assign a java.sql.Timestamp to your java.util.Date attribute. And while this is "legal", the problem is that Timestamp.equals(Object) is not symmetric (why on earth?!) and this breaks the semantics of Date.equals(Object).
As a consequence, you can't "blindly" use myDate.equals(someRealJavaUtilDate) if myDate is mapped to a SQL TIMESTAMP, which is of course not really acceptable.
But although this has been extensively discussed on the Hibernate forums, e.g. in this thread and this one (read all pages), it seems that Hibernate users and developers never agreed on the problem (see issues like HB-681) and I just don't understand why.
Maybe it's just me, maybe I just missing something simple for others, but the problem looks obvious to me and while I consider this stupid java.sql.Timestamp to be the culprit, I still think that Hibernate should shield users from this issue. I don't understand why Gavin didn't agree on this.
My suggestion would be to create a test case demonstrating the issue (should be pretty simple) and to report the problem (again) to see if you get more positive feedback from the current team.
Meanwhile, you could use a custom type to "fix" the problem yourself, using something like this (taken from the forum and pasted as is):
public class TimeMillisType extends org.hibernate.type.TimestampType {
public Date get(ResultSet rs, String name) throws SQLException {
Timestamp timestamp = rs.getTimestamp(name);
if (timestamp == null) return null;
return
new Date(timestamp.getTime()+timestamp.getNanos()/1000000);
}
}
java.sql.Timestamp overrides the compareTo(Date) method, so it should be no problem using compareTo(..)
In short - java.util.Date and java.sql.Timestamp are mutually comparable.
Furthermore, you can always compare the date.getTime(), rather than the objects themselves.
And even further - you can use a long field to store the date. Or even a DateTime (from joda-time)
In my experience you don't want the java.sql.Timestamp out into your logic - it creates a lot of strange errors just as you pointed out, and it does not get any better if your application does serialization.
If it works with the override that returns a new java.util.Date then go for that one. Or even better, go for JodaTime. You'll find lots of examples out on the net doing that. I would not worry about performance here as your database is in magnitude more slow than the creation of a new java.util.Date object.
EDIT:
I see that you are using Hibernate. If you use annotations you can do:
#Type(type = "org.joda.time.contrib.hibernate.PersistentDateTime")
public DateTime getProvisionByTime() {
return provisionByTime;
}
Then you will get nice DateTime objects from Jodatime in your persistent objects. If you want to only have a date, you can use LocalDate like this:
#Type(type = "org.joda.time.contrib.hibernate.PersistentLocalDate")
public LocalDate getCloudExpireDate() {
return cloudExpireDate;
}
IF you use maven, the following dependencies should get this set up for you (you might need to update the hibernate versions)
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate</artifactId>
<version>3.2.6.ga</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.3.1.GA</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time-hibernate</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>1.6.1</version>
</dependency>
The problem is critical for DAO tests:
Employer employer1 = new Employer();
employer1.setName("namenamenamenamenamename");
employer1.setRegistered(new Date(1111111111)); // <- Date field
entityManager.persist(employer1);
assertNotNull(employer1.getId());
entityManager.flush();
entityManager.clear();
Employer employer2 = entityManager.find(Employer.class, employer1.getId());
assertNotNull(employer2);
assertEquals(employer1, employer2); // <- works
assertEquals(employer2, employer1); // <- fails !!!
So the result is really surprising and writing tests became tricky.
But in the real business logic you will never use entity as a set/map key because it is huge and it is mutable. And you will never compare time values by equal comparison. And comparing whole entities should be avoided too.
The usual scenario uses immutable entity ID for map/set key and compares time values with compareTo() method or just using getTime() values.
But making tests is a pain so I implemented my own type handlers
http://pastebin.com/7TgtEd3x
http://pastebin.com/DMrxzUEV
And I have overridden the dialect I use:
package xxx;
import org.hibernate.dialect.HSQLDialect;
import org.hibernate.type.AdaptedImmutableType;
import xxx.DateTimestampType;
import java.util.Date;
public class CustomHSQLDialect extends HSQLDialect {
public CustomHSQLDialect() {
addTypeOverride(DateTimestampType.INSTANCE);
addTypeOverride(new AdaptedImmutableType<Date>(DateTimestampType.INSTANCE));
}
}
I haven't decided yet - would I use this approach both for tests and production or for tests only.
JPA should return a java.util.Date for an attribute of type java.util.Date, the #Temporal(TIMESTAMP) annotation should only affect how the date is stored. You should not get a java.sql.Timestamp back.
What JPA provider are you using? Have you tried this in EclipseLink, the JPA reference implementation?