In our web application we need to show and enter
date time information for different countries in different time zone. Right now, we are maintaining separate web server and separate database (oracle 11g) for each country.
We are planning to merge all into one portal with single database (oracle 11g).
This portal should capture/display date and time in user local time zone.
So far, I have searched about this, I got below suggestion.
1) set web server's and database server's time zone to UTC and while fetching data (data and time) convert into user local time zone.
If you suggest this approach then please clarify the following specific
questions.
most of the time we are capturing date alone, is it require to
capture date and time along with time zone always?
while storing date and time where we need to convert user local time
zone to UTC in javascript/java/oracle?
while fetching date and time where we need to convert UTC to user
local time zone query itself/java/java script?
many place we have reports to show based on date column such as
today/current month/date range.how we can handle this(input - user
local time zone - database in UTC)?
which data type we have to use for date field
(date/timestamp/timestamp with time zone/timestamp with local time
zone)?
2) capture date and time in both user local time zone and UTC. Stored as separate columns, user local time zone will be used for display purpose and UTC will be used for business logic.
If you suggest this approach then please clarify the following specific
questions.
Is it common practice to store the user local time zone and UTC?
which column i have to check condition while fetching reports to show
based on date column such as today/current month/date range?
which data type we have to use for date column
(date/timestamp/timestamp with time zone/timestamp with local time
zone)?
thanks in advance
Read the Question Daylight saving time and time zone best practices. Yours is basically a duplicate.
Servers in UTC
Yes, generally servers should have their OS set to UTC as the time zone, or if not provided use GMT or the Reykjavík Iceland time zone. Your Java implementation probably picks up this setting as its own current default time zone.
Specify time zone
But do not depend on the time zone being set to UTC. A sysadmin could change it. And any Java code in any thread of any app within your JVM can change the JVM’s current default time zone at runtime by calling TimeZone.setDefault. So instead, make a habit of always specifying the desired/expected time zone by passing the optional argument in your Java code.
I consider it a design flaw that any date-time framework would make the time zone optional. Being optional creates endless amounts of confusion because programmers, like everybody else, unconsciously think in terms of their own personal time zone unless prompted. So all too often in date-time work no attention is paid to the issue. Add on the problem that the JVM default varies. By the way, ditto for Locale, same problems, should always be specified explicitly.
UTC
Your business logic, data storage, and data exchange should almost always be done in UTC. Nearly every database has a feature for adjusting any input into UTC and storing in UTC.
When presenting a date-time to a user, adjust into the expected time zone. When serializing a date-time value, use the ISO 8601 string formats. See the Answer by VickyArora for Oracle specifically (I am a Postgres person). Be sure to read the doc carefully, and practice by experimenting to fully understand your database's behavior. The SQL spec does not spell out very much in this regard, and behavior varies widely.
java.sql
Remember that when using Java and JDBC, you will be using the java.sql.Timestamp and related data types. They are always in UTC, automatically. In the future expect to see JDBC drivers updated to directly use the new data types defined in the java.time framework built into Java 8 and later.
java.time
The old classes are outmoded by java.time. Learn to use java.time while avoiding the old java.util.Date/.Calendar and make your programming life much more pleasant.
Until your JDBC driver is updated, you can use the conversion convenience methods built into java.time. See examples next, where Instant is a moment in UTC and ZonedDateTime is an Instant adjusted into a time zone.
Instant instant = myJavaSqlTimestamp.toInstant();
ZoneId zoneId = ZoneId.of( "America/Montreal" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
To go the other direction.
java.sql.Timestamp myJavaSqlTimestamp = java.sql.Timestamp.from( zdt.toInstant() );
If you need original time zone, store it
If your business requirements consider the original input data’s time zone to be important, to be remembered, then store that explicitly as a separate column in your database table. You can use an offset-from-UTC, but that does not provide full information. A time zone is an offset plus a set of rules for the past, present, and future handling of anomalies such as Daylight Saving Time. So a proper time zone name is most appropriate such as America/Montreal.
Date-only is ambiguous
You said you collect many date-only values, without time-of-day and without time zone. The class for that in java.time is LocalDate. As with LocalTime and LocalDateTime, the “Local…” part means no particular locality, so therefore no time zone, and so not a point on the timeline -- has no real meaning.
Keep in mind that a date-only value is ambiguous by definition. At any given moment, the date varies around the world. For example, just after midnight in Paris France is a new day but in Montréal Québec the date is still “yesterday”.
Usually in business some time zone is implicit, even unconsciously intuited. Unconscious intuition about data points tends not to work well over the long term, especially in software. Better to make explicit what time zone was intended. You could store the intended zone alongside the date such as another column in database table, or your could make a comment in your programming code. I believe it would vastly better and safer to store a date-time value. So how do we transform a date-only into a date-time?
Often a new day is the moment after midnight, the first moment of the day. You might think that means the time-of-day 00:00:00.0 but not always. Daylight Saving Time (DST) and possibly other anomalies may push the first moment to a different wall-clock time. Let java.time determine the correct time-of-day for first moment going through the LocalDate class and its atStartOfDay method.
ZoneId zoneId = ZoneId.of( "America/Montreal" );
LocalDate today = LocalDate.now( zoneId );
ZonedDateTime todayStart = today.atStartOfDay( zoneId );
In some business contexts a new day may be defined (or assumed) to be business hours. For example, say a publisher in New York means 9 AM in their local time when they say “the book draft is due by January 2nd”. Let's get that time-of-day for that date in that time zone.
ZoneId zoneId = ZoneId.of( "America/New_York" );
ZonedDateTime zdt = ZonedDateTime.of( 2016 , 1 , 2 , 9 , 0 , 0 , 0 , zoneId );
What does that mean for the author working in New Zealand? Adjust into her particular time zone for presentation to her by calling withZoneSameInstant.
ZoneId zoneId_Pacific_Auckland = ZoneId.of( "Pacific/Auckland" );
ZonedDateTime zdt_Pacific_Auckland = zdt.withZoneSameInstant( zoneId_Pacific_Auckland );
Database
For database storage we transform into an Instant (a moment on the timeline in UTC) and pass as a java.sql.Timestamp as seen earlier above.
java.sql.Timestamp ts = java.sql.Timestamp.from( zdt.toInstant() );
When retrieved from the database, transform back to a New York date-time. Convert from java.sql.Timestamp to an Instant, then apply a time zone ZoneId to get a ZonedDateTime.
Instant instant = ts.toInstant();
ZoneId zoneId = ZoneId.of( "America/New_York" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
If your database driver complies with JDBC 4.2 or later, you may be able to pass/fetch the java.time types directly rather than convert to/from java.sql types. Try the PreparedStatement::setObject and ResultSet::getObject methods.
Use TIMESTAMP WITH LOCAL TIME ZONE
if you want the database to automatically
convert a time between the database and session time zones.
Stores a date and time with up to 9 decimal places of precision. This datatype is
sensitive to time zone differences. Values of this type are automatically converted
between the database time zone and the local (session) time zone. When values are
stored in the database, they are converted to the database time zone, but the local
(session) time zone is not stored. When a value is retrieved from the database, that
value is converted from the database time zone to the local (session) time zone.
Here I have clarified the following specific questions.
Q. Most of the time we are capturing date alone, is it require to capture date and time along with time zone always?
A. Yes
Q. while storing date and time where we need to convert user local time zone to UTC in javascript/java/oracle?
A. Not convert during data save, save as it with source date+time+zone
Q. While fetching date and time where we need to convert UTC to user
local time zone query itself/java/java script?
A. Always convert to display in local time zone OR UTC format where application opened.
Q. Many place we have reports to show based on date column such as
today/current month/date range.how we can handle this(input - user local time zone - database in UTC)?
A. The system should provide setting option to user for date time display in what format, either local where application opened or UTC. All is done on front end only.
Q. Which data type we have to use for date field (date/timestamp/timestamp with time zone/timestamp with local time zone)?
A. Timestamp
So in short, save datetime in source time zone and convert based upon user preferences either in local where page opened or UTC format. Means, conversion will be done through script for display only. The region where product is being popular can also be find.
I would simply transform the existing dates stored in the DB into Long, and persist (ETL process) this Long value, along with the known (or deducted) pattern, Locale and TimeZone (default meta). And persist any new Date as long as with the default meta.
ETL example
Let say 2015-11-29 10:07:49.500 UTC is stored in the DB:
// Known or deducted format of the persisted date
String pattern = "yyyy-MM-dd HH:mm:ss.SSS";
Locale locale = Locale.ENGLISH;
TimeZone zone = "UTC";
// Date to ms
SimpleDateFormat sdf = new SimpleDateFormat(pattern, locale);
sdf.setTimeZone(TimeZone.getTimeZone(zone));
Date date = sdf.parse(pattern);
// ETL: Can now be persisted in Long, along with default META (pattern, Locale, TZ)
Long dateL = date.getTime(); // for e.g. 1448827660720
...
The persisted Long value can also be transformed in any other format, if needed
pattern | locale | tz | result
============================================
yyyy/MM/dd | null | null | 2015/11/29
dd-M-yyyy hh:mm:ss | null | null | 29-11-2015 10:07:40
dd MMMM yyyy zzzz | ENGLISH | null | 29 November 2015 Central European Time
yyyy-MM-dd HH:mm:ss.SSS | null | UTC | 2015-11-29 10:07:49 UTC
Feasible and logical approach is ; Convert user entered time to GMT/UTC +00 and store that in db with or without timezone identifier it does not matter. When you need to display the time to user convert GMT/UTC time in java to local time of user.
You should consider JodaTime and should follow the first suggestion.JodaTime has many classes like LocalDate,LocalDateTime which you can use for your different use cases.
Related
I have seen a lot of debates on the following date conversion:
timeStamp.toLocalDateTime().toLocalDate();
Some people say that it is not appropriate because the timezone has to be specified for proper conversion, otherwise the result may be unexpected. My requirement is that I have an object that contains Timestamp fields and another object that contains LocalDate fields. I have to take the date difference between both so I think that the best common type to use is LocalDate. I don't see why the timezone has to be specified as either timestamp or LocalDate just represent dates. The timezone is already implied. Can someone give an example when this conversion fails?.
It’s more complicated than that. While it’s true that a Timestamp is a point in time, it also tends to have a dual nature where it sometimes pretends to be a date and time of day instead.
BTW, you probably already know, the Timestamp class is poorly designed and long outdated. Best if you can avoid it completely. If you are getting a Timestamp from a legacy API, you are doing the right thing: immediately converting it to a type from java.time, the modern Java date and time API.
Timestamp is a point in time
To convert a point in time (however represented) to a date you need to decide on a time zone. It is never the same date in all time zones. So the choice of time zone will always make a difference. So one correct conversion would be:
ZoneId zone = ZoneId.of("Africa/Cairo");
LocalDate date = timestamp.toInstant().atZone(zone).toLocalDate();
The Timestamp class was designed for use with your SQL database. If your datatype in SQL is timestamp with time zone, then it unambiguously denotes a point in time, and you need to see it as a point in time as just described. Even when to most database engines timestamp with time zone really just means “timestamp in UTC”, it’s still a point in time.
And then again: sometimes to be thought of as date and time of day
From the documentation of Timestamp:
A Timestamp also provides formatting and parsing operations to support
the JDBC escape syntax for timestamp values.
The JDBC escape syntax is defined as
yyyy-mm-dd hh:mm:ss.fffffffff, where fffffffff indicates
nanoseconds.
This doesn’t define any point in time. It’s a mere date and time of day. What the documentation doesn’t even tell you is that the date and time of day is understood in the default time zone of the JVM.
I suppose that the reason for seeing a Timestamp in this way comes from the SQL Timestamp datatype. In most database engines this is a date and time without time zone. It’s not a timestamp, despite the name! It doesn’t define a point in time, which is the purpose of and is in the definition of timestamp.
I have seen a number of cases where the Timestamp prints the same date and time as in the database, but doesn’t represent the point in time implied in the database. For example, there may be a decision that “timestamps” in the database are in UTC, while the JVM uses the time zone of the place where it’s running. It’s a bad practice, but it is not one that will go away within a few years.
This must also have been the reason why Timestamp was fitted with the toLocalDateTime method that you used in the question. It gives you that date and time that were in the database, right? So in this case your conversion in the question ought to be correct, or…?
Where this can fail miserably without us having a chance to notice is, as others have mentioned already, when the default time zone of the JVM is changed. The JVM’s default time zone can be changed at any time from any place in your program or any other program running in the same JVM. When this happens, your Timestamp objects don’t change their point in time, but they do tacitly change their time of day, sometimes also their date. I’ve read horror stories — in Stack Overflow questions and elsewhere — about the wrong results and the confusion coming out of this.
Solution: don’t use Timestamp
Since JDBC 4.2 you can retrieve java.time types out of your SQL database. If your SQL datatype is timestamp with time zone (recommended for timestamps), fetch an OffsetDateTime. Some JDBC drivers also let you fetch an Instant, that’s fine too. In both cases no time zone change will play any trick on you. If the SQL type is timestamp without time zone (discouraged and all too common), fetch a LocalDateTime. Again you can be sure that your object doesn’t change its date and time no matter if the JVM time zone setting changes. Only your LocalDateTime never defined a point in time. Conversion to LocalDate is trivial, as you have already demonstrated in the question.
Links
java.sql.Timestamp documentation
Wikipedia article: Timestamp
Question: Getting the date from a ResultSet for use with java.time classes
Question: Java - Convert java.time.Instant to java.sql.Timestamp without Zone offset
As you can see here(taken from https://stackoverflow.com/a/32443004/1398418):
Timestamp represents a moment in UTC and is the equivalent of the modern Instant.
When you do:
timeStamp.toLocalDateTime().toLocalDate();
the timeStamp is converted from UTC to the system timezone. It's the same as doing:
timeStamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate()
For example:
Timestamp stamp = new Timestamp(TimeUnit.HOURS.toMillis(-1)); // UTC 1969-12-31
System.setProperty("user.timezone", "EET"); // Set system time zone to Eastern European EET - UTC+2
stamp.toLocalDateTime().toLocalDate(); // represents EET 1970-01-01
stamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate(); // represents EET 1970-01-01
That result (getting the date in the system time zone) is expected and if that's what you want, doing timeStamp.toLocalDateTime().toLocalDate() is appropriate and correct.
You're saying that you have a LocalDate field in some object and you want to get a period between it and a Timestamp, well that's just not possible without aditional information. LocalDate just represents a date, it has no time zone information, you need to know how it was created and what time zone was used.
If it represent a date in the system time zone then getting the period by using timeStamp.toLocalDateTime().toLocalDate() would be correct, if it represents a date in UTC or any other time zone then you might get a wrong result.
For example if the LocalDate field represents a date in UTC you will need to use:
timeStamp.toInstant().atZone(ZoneId.of("UTC")).toLocalDate();
Example: the 23rd of January becomes the 24th
You asked:
Can someone give an example when this conversion fails?.
Yes, I can.
Start with the 23rd of January.
LocalDate ld = LocalDate.of( 2020 , Month.JANUARY , 23 );
LocalTime lt = LocalTime.of( 23 , 0 );
ZoneId zMontreal = ZoneId.of( "America/Montreal" );
ZonedDateTime zdt = ZonedDateTime.of( ld , lt , zMontreal );
Instant instant = zdt.toInstant();
zdt.toString() = 2020-01-23T23:00-05:00[America/Montreal]
instant.toString() = 2020-01-24T04:00:00Z
The Instant class represents a moment as seen in UTC. Let's convert to the terribly legacy class java.sql.Timestamp using the new conversion method added to that old class.
// Convert from modern class to troubled legacy class `Timestamp`.
java.sql.Timestamp ts = Timestamp.from( instant );
ts.toString() = 2020-01-23 20:00:00.0
Unfortunately, the Timestamp::toString method dynamically applies the JVM’s current default time zone while generating text.
ZoneOffset defaultOffset = ZoneId.systemDefault().getRules().getOffset( ts.toInstant() );
System.out.println( "JVM’s current default time zone: " + ZoneId.systemDefault() + " had an offset then of: " + defaultOffset );
JVM’s current default time zone: America/Los_Angeles had an offset then of: -08:00
So Timestamp::toString misreports the object’s UTC value after adjusting back eight hours from 4 AM to 8 PM. This anti-feature is one of several severe problems with this poorly designed class. For more discussion of the screwy behavior of Timestamp, see the correct Answer by Ole V.V.
Let's run your code. Imagine at runtime the JVM’s current default time zone is Asia/Tokyo.
TimeZone.setDefault( TimeZone.getTimeZone( "Asia/Tokyo" ) );
LocalDate localDate = ts.toLocalDateTime().toLocalDate();
Test for equality. Oops! We ended up with the 24th rather than the 23rd.
boolean sameDate = ld.isEqual( localDate );
System.out.println( "sameDate = " + sameDate + " | ld: " + ld + " localDate: " + localDate );
sameDate = false | ld: 2020-01-23 localDate: 2020-01-24
See this code run live at IdeOne.com.
So what is wrong with your code?
Never use java.sql.Timestamp. It is one of several terrible date-time classes shipped with the earliest versions of Java. Never use these legacy classes. They have been supplanted entirely by the modern java.time classes defined in JSR 310.
You called toLocalDateTime which strips away vital information. Any time zone or offset-from-UTC is removed, leaving only a date and a time-of-day. So this class cannot be used to represent a moment, is not a point on the timeline. Ex: 2020-12-25 at noon — is that noon in Delhi, noon in Düsseldorf, or noon in Detroit, three different moments several hours apart? A LocalDateTime is inherently ambiguous.
You ignored the crucial issue of time zone in determining a date. For any given moment, the date varies around the globe. At one moment it may be “tomorrow” in Australia while simultaneously “yesterday” in Mexico.
The problem lies in what is being represented by these objects. Your question forgets a crucial aspect, which is: What is the type of timeStamp?
I'm guessing it's a java.sql.Timestamp object.
Timestamp, just like java.util.Date, is old API equivalent to Instant.
It represents an instant in time, in the sense that it is milliseconds since jan 1st 1970 UTC. The system has no idea which timezone that was supposed to be in. You're supposed to know; the error, if an error is going to occur here, already occurred before you get to this code. Here's a trivial explanation of how it COULD go wrong:
you start off with a user entering a date in a date field on a webform; it's 2020-04-01.
Your server, running in Amsterdam, saves it to a DB column that is internally represented as UTC, no zone. This is a mistake (you're not saving an instant in time, you're saving a date, these two are not the same thing). What is actually stored in the DB is the exact moment in time that it is midnight, 2020-04-01 in amsterdam (in UTC, that'd be 22:00 the previous day!).
Later, you query this moment in time back into a java.sql.Timestamp object, and you're doing this when the server's tz is elsewhere (say, London time). You then convert this to a localdatetime, and from there to a localdate, and.... you get 2020-03-31 out.
Whoops.
Dates should remain dates. Never convert LocalX (be it Time, Date, or DateTime) to Instant (or anything that effectively is an instant, including j.s.Timestamp, or j.u.Date - yes, j.u.Date does NOT represent a date, it is very badly named), or vice versa, or pain will ensue. If you must because of backward APIs take extreme care; it's hard to test that 'moving the server's timezone around' breaks stuff!
My requirement is to store all dates & date-times in UTC timezone in the database. I am using Java 8's LocalDate & LocalDateTime in my Hibernate entities.
Is that correct as LocalDate & LocalDateTime doesn't have timezone associated with them?
If not, should I fall back to using good old (or legacy?) Date & Timestamp?
Or should I be using Java 8's Instant? If using Instant, will there be a possibility to store only the date part, without time?
The database is MySQL & SQL Server and this is a Spring Boot application.
The “Local…” types purposely have no concept of time zone. So they do not represent a moment on the timeline. A LocalDateTime represents a vague range of possible moments but has no real meaning until assigning an offset or time zone. That means applying a ZoneId to get a ZonedDateTime.
For example, to say that Christmas this year starts at the first moment of December 25, we say:
LocalDateTime ldt = LocalDateTime.of( 2017 , 12 , 25 , 0 , 0 , 0 , 0 );
But that stroke of midnight happens earlier in the east than in the west.
That is why the elves’ Logistics department maps out Santa’s route starting at Kiribati in the Pacific, the earliest time zone in the world at 14 hours ahead of UTC. After delivering there, they route Santa westward to places like New Zealand for its midnight later. Then on to Asia for their midnight later. Then India, and so on, reaching Europe for their midnight several hours later, and then the east coast of North America for their midnight a few hours more later. All of these places experienced that same LocalDateTime at different moments, each delivery represented by a different ZonedDateTime object.
So…
If you want to record the concept of Christmas starting after midnight on the 25th, use a LocalDateTime and write into a database column of type TIMESTAMP WITHOUT TIME ZONE.
If you want to record the exact moment of each delivery Santa makes, use a ZonedDateTime and write into a database column of type TIMESTAMP WITH TIME ZONE.
About that second bullet, be aware that nearly every database system will use the zone information to adjust the date-time to UTC and store that UTC value. Some save the zone information as well, but some such as Postgres discard the zone info after using it to adjust into UTC. So “with time zone” is something of a misnomer, really meaning “with respect for time zone”. If you care about remembering that original zone, you may need to store its name in a separate column alongside.
Another reason to use Local… types is for future appointments. Politicians enjoy frequently changing their time zone(s) of their jurisdiction. They like to adopt Daylight Saving Time (DST). The like to change the dates of their DST cutovers. They like to drop their adoption of DST. They like to redefine their time zones, changing the boundaries. They like to redefine their offset-from-UTC sometimes by amounts like 15 minutes. And they rarely give advance notice, making such changes with as little as a month or two of warning.
So to make medical check-up appointment for next year or in six months, the time zone definition cannot be predicted. So if you want an appointment of 9 AM, you should use a LocalTime or LocalDateTime recorded in a database column of type TIMESTAMP WITHOUT TIME ZONE. Otherwise that 9 AM appointment, if zoned where the DST cutover is postponed, may appear as 8 AM or 10 AM.
When generating a projected schedule, you can apply a time zone (ZoneId) to those “local” (unzoned) values to create ZonedDateTime objects. But do not rely on those too far out in time when politicians may ruin their meaning by changing the zone(s).
Tip: These frequent changes to DST and time zones means you must keep your time zone tzdata data base up to date. There is a tzdata in your host OS, your JVM, and perhaps in your database system such as Postgres. All three should be frequently updated. Sometimes the zones change faster than the planned update cycles of those products such as Turkey last year deciding to stay on DST with only several weeks notice. So you may occasionally need to manually update those tzdata files. Oracle provides a tool for updating the tzdata of their Java implementations.
The general best practice in handling exact moments is to track them in UTC. Apply a time zone only where necessary such as in presentation to a user where they expect to see values in their own parochial time zone. In java.time, the Instant class represents a moment in the timeline. In UTC with a resolution of nanoseconds.
Instant instant = Instant.now() ; // Current moment on the timeline in UTC.
ZonedDateTime zdt = instant.atZone( z ) ; // Assign a time zone to view the same moment through the lens of a particular region’s wall-clock time.
Instant instant = zdt.toInstant(); // revert back to UTC, stripping away the time zone. But still the same moment in the timeline.
By the way, drivers that comply with JDBC 4.2 and later can deal directly with the java.time types via:
PreparedStatement::setObject
ResultSet::getObject
Oddly, the JDBC 4.2 spec does not require support for the two most common java.time types: Instant & ZonedDateTime. The spec does require support for OffsetDateTime. So you can easily convert back-and-forth.
Avoid the old legacy data types such as java.util.Date and java.sql.Timestamp whenever possible. They are poorly designed, confusing, and flawed.
Understand that all of these four are representations of a moment on the timeline in UTC:
Modern
java.time.Instant
java.time.OffsetDateTime with an assigned offset of ZoneOffset.UTC
Legacy
java.util.Date
java.sql.Timestamp
If you want a date-only value without time-of-day and without time zone, use java.time.LocalDate. This class supplants java.sql.Date.
As for specific databases, be aware that the SQL standard barely touches on the topic of date-time types and their handling. Furthermore, the various databases vary widely, and I really mean widely, in their support of date-time features. Some have virtually no support. Some mix SQL standard types with proprietary types that either predate the standard types or are intended as alternatives to the standard types. In addition, JDBC drivers differ in their behavior with marshaling date-time values to/from the database. Be sure to study the documentation and practice, practice, practice.
Or should I be using Java 8's Instant? If using Instant, will there be
a possibility to store only the date part, without time?
Instant should be suitable for the most operations.
Add hibernate-java8 to pom.xml to support Java 8 time API:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-java8</artifactId>
<version>${version.hibernate}</version>
</dependency>
Then you can use LocalDate or LocalDateTime or Instant for Hibernate entity fields. You need to remove #Temporal(TemporalType.TIMESTAMP).
My requirement is to store all dates & date-times in UTC timezone in
the database. I am using Java 8's LocalDate & LocalDateTime in my
Hibernate entities.
Is that correct as LocalDate & LocalDateTime doesn't have timezone
associated with them?
You can set default JVM timezone somewhere in a configurational code:
#PostConstruct
void setUTCTimezone() {
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
}
Then you'll operate UTC time in your code.
To use Java 8 date types in DTO, you need to add Jsr310JpaConverters:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
And:
#EntityScan(basePackageClasses = { Application.class, Jsr310JpaConverters.class })
SpringBootApplication
public class Application { … }
More options:
Force Java timezone as GMT/UTC
How to set java timezone?
How to set a JVM TimeZone Properly
Part of the new Date API they split the types of dates up. The correct class that includes timezones is ZonedDateTime
// Get the current date and time
ZonedDateTime date1 = ZonedDateTime.parse("2007-12-03T10:15:30+05:30[Asia/Karachi]");
System.out.println("date1: " + date1);
ZonedDateTime zonedDateTime = ZonedDateTime.now();
System.out.println("Zoned Date Time: " + zonedDateTime);
ZoneId id = ZoneId.of("Europe/Paris");
System.out.println("ZoneId: " + id);
ZoneId currentZone = ZoneId.systemDefault();
System.out.println("CurrentZone: " + currentZone);
Prints:
date1: 2007-12-03T10:15:30+05:00[Asia/Karachi]
Zoned Date Time: 2017-04-18T11:36:09.126-04:00[America/New_York]
ZoneId: Europe/Paris
CurrentZone: America/New_York
Follow-up question to my original issue.
Because I know that any date time used is against a known timezone, rather than where the user may be submitting their request from, I take a LocalDateTime, convert to UTC and persist. Then, when the appointment is retrieved I convert the saved time to the meeting location timezone (stored in db). However, it would seem that the values I save are actually being saved in my local timezone.
I receive a date time value in the Rest Controller such as:
startLocalDateTime: 2016-04-11T10:00
endLocalDateTime: 2016-04-11T10:30
Appointment has two ZoneDateTime fields:
#Column(name = "startDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
private ZonedDateTime startDateTime;
#Column(name = "endDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
private ZonedDateTime endDateTime;
Then I change the values to UTC and store on my entity to store to Postgres:
appointment.setStartDateTime(startLocalDateTime.atZone(ZoneId.of( "UTC" )))
appointment.setEndDateTime(endLocalDateTime.atZone(ZoneId.of( "UTC" )))
and I store that in Postgres (columnDefinition= "TIMESTAMP WITH TIME ZONE") When I look at the record in pgadminIII I see:
startDateTime "2016-04-11 04:00:00-06"
endDateTime "2016-04-11 04:30:00-06"
So these appear to be stored properly in UTC format (please correct me if I am doing anything wrong so far). I then retrieve them from the database and they are returned as:
Appointment
startdatetime: 2016-04-11T04:00-06:00[America/Denver]
enddatetime: 2016-04-11T04:30-06:00[America/Denver]
Those values are sent back as JSON:
{
"appointmentId":50,
"startDateTime":"2016-04-11T04:00",
"endDateTime":"2016-04-11T04:30"
}
So even though I am saving them as UTC, when I retrieve them they are in MST (my local) timezone, rather than UTC, and I am unable to convert them back to the actual time.
Still struggling with the persistence. I have tried using the java.sql.timestamp, java.sql.Date, java.util.Date, and java.time.ZonedDateTime on my entity. My Postgres is still a "timestamp with time zone". But because I am using Spring-Data-JPA and need to query with the same type. If I use Date - should that be sql.Date or util.Date?
You said:
I take a LocalDateTime, convert to UTC and persist.
Nope. Stick with that LocalDateTime for booking appointments, no UTC involved for appointments.
When booking appointments that should appear as a certain time-of-day regardless of how politicians redefine the offset used within the time zone(s) under their jurisdiction, store a date with time-of-day but keep the time zone separate.
Politicians around the world have shown a strange penchant for frequently redefining the offset of their zones, adjusting to match or differ their neighbors, or to adopt or drop the foolishness known as Daylight Saving Time (DST). These changes are done with little forewarning, or even none at all. So what is right now looks like 2:30 PM on a future day could become 1:30 PM, 2:00 PM, 3:30 PM, or who knows what the politicians might come up with, if we stored as a moment (date, time-of-day, zone/offset, all together).
Java
To represent a date and time-of-day only, use LocalDateTime. This class purposely lacks any concept of time zone or offset-from-UTC. This lack means this class does not track moments, is not a point on the timeline. So we generally do not use this class in business apps. Booking future appointments is one of the few cases where we do want this class.
LocalDate ld = LocalDate.of( 2020 , Month.JANUARY , 23 ) ; // 2020-01-23.
LocalTime lt = LocalTime.of( 14 , 30 ) ; // 2:30 PM.
LocalDateTime ldt = LocalDateTime.of( ld , lt ) ;
See this code run live at IdeOne.com.
ldt.toString(): 2020-01-23T14:30
Store that in your database using JDBC 4.2 and later.
myPreparedStatement.setObject( … , ldt ) ;
Retrieve from database.
LocalDateTime ldt = myResultSet.getObject( … , LocalDateTime.class );
We need to track the intended time zone. Is this an appointment for a dentist office in Québec? Represent that fact as well. Keep in mind that a time zone is a history of past, present, and future (planned) changes to the offset-from-UTC used by the people of a particular region.
Use the ZoneId class to represent a time zone.
ZoneId z = ZoneId.of( "America/Montreal" ) ;
Save that zone name as text to your database. Get its identifying name by calling ZoneId::toString. Do not use ZoneId::getDisplayName as that is for generating localized text for display to a user, but is not a formal identifier for the zone.
myPreparedStatement.setString( … , z.toString() ) ;
When your retrieve from database, reconstitute the ZoneId.
String zoneIdString = myResultSet.getString( … ) ;
ZoneId z = ZoneId.of( zoneIdString ) ;
When you are generating a calendar for these appointments, if you need to determine (tentatively) points on the timeline, combine the date-with-time-of-day and the time zone together. Applying the ZoneId to the LocalDateTime produces a ZonedDateTime. This ZonedDateTime is a moment, is a point on the timeline, unlike LocalDateTime.
ZonedDateTime zdt = ldt.atZone( z ) ;
See this code run live at IdeOne.com.
zdt.toString(): 2020-01-23T14:30-05:00[America/Montreal]
Of course, we do not store that ZonedDateTime. As discussed above this moment that right now appears to be 2:30 PM on the 23rd would turn into 1:30 PM or 3:30 PM if those busy politicians changed the time zone rules between now and then. We only use this ZonedDateTime tentatively, temporarily.
Database
We store a LocalDateTime only in a column of type TIMESTAMP WITHOUT TIME ZONE. You were incorrectly using WITH rather than WITHOUT, a major problem. For the WITHOUT type, Postgres stores the date and the time-of-day as given in the input. Even if a zone or offset were included with the input, Postgres would ignore the zone or offset.
Be aware that TIMESTAMP WITH TIME ZONE is just the opposite in Postgres. The time zone or offset info accompanying an input is used to adjust the date and time to UTC. That UTC value is then stored. So values in a column of this type are all in UTC. Unfortunately, many tools have the anti-feature of dynamically applying a default time zone to the retrieved value before passing on to you. This creates the false illusion of a time zone being stored when in fact the value is in UTC. Retrieving as a OffsetDateTime will always tell you the truth. I say OffsetDateTime because, oddly, the JDBC 4.2 spec requires support for that class but not for the more commonly used Instant & ZonedDateTime classes. Your JDBC driver may support the other two classes, optionally. But all of this paragraph is neither here nor there, as we are not using WITH columns for future appointments.
As mentioned above, for future appointments should be stored in three (3) separate columns:
TIMESTAMP WITHOUT TIME ZONE for the date and the time-of-day.
TEXT (or similar type) for the identifying name of the intended time zone by which we want to view that appointment.
TEXT (or similar) for the duration of the appointment, in standard ISO 8601 format. (see below)
Points in your Question
You said:
I receive a date time value in the Rest Controller such as:
startLocalDateTime: 2016-04-11T10:00
So parse as a LocalDateTime. Such ISO 8601 compliant strings can be parsed directly by the java.time classes.
LocalDateTime ldt = LocalDateTime.parse( "2016-04-11T10:00" ) ;
You said:
endLocalDateTime: 2016-04-11T10:30
No. Do not store the ending time. The ending time you expect now could be different if time zone anomalies occur at that point. For example, what if your politicians adopt Daylight Saving Time (DST), then during the time of this appointment the "Spring ahead" 1-hour change of DST occurred? Then your half-hour meeting should run from 10:00 AM to 11:30 AM, while your recorded end-time would incorrectly read "10:30 AM" ending.
Generally the best way to handle appointments is to record the duration of the appointment, the span-of-time, rather than on the clock. As mentioned above, an appointment consists of three separate pieces of data: (1) starting date with time-of-day, (2) the intended time zone, and (3) duration of appointment. The ending should be dynamically calculated as needed, using fresh current time zone data, for temporary use.
The ISO 8601 standard includes a textual format for recording such durations: PnYnMnDTnHnMnS where P marks the beginning, and T separates any years-months-days from any hours-minutes-seconds. The java.time classes Period and Duration can parse such strings.
Duration d = Duration.ofHours( 1 ) ;
String output = d.toString() ;
See this code run live at IdeOne.com.
output: PT1H
You can dynamically apply this duration to the LocalDateTime.
LocalDateTime ending = ldt.plus( d ) ;
Ditto for ZonedDateTime.
ZonedDateTime ending = zdt.plus( d ) ;
You said:
Appointment has two ZoneDateTime fields:
Nope. Make that a LocalDateTime field, not ZonedDateTime.
You said:
#Column(name = "startDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
Nope. Make that a column of type "TIMESTAMP WITHOUT TIME ZONE" to track a date and time-of-day without the context of zone/offset.
You said:
Then I change the values to UTC and store on my entity to store to Postgres:
Nope. Booking appointments is one of the few cases where we do not want UTC. When tracking a moment, we generally do want to use UTC. But future appointments are not moments. We do not know the moment until we apply a time zone to the date-with-time-of-day, and we cannot do that ahead of time because we cannot trust politicians.
You said:
Still struggling with the persistence.
Cannot help you there. I do not use either Spring or JPA, as they solve problems I do not have. I use straight JDBC.
You said:
So these appear to be stored properly in UTC format (please correct me if I am doing anything wrong so far).
Nope, do not use UTC, do not store in UTC, not for appointments.
Do use UTC for moments though. For example, your logs should all be reporting each moment of an incident in UTC. Programmers & sysadmins would do well to learn to think in UTC while on the job. Keep a second clock on your desk set to UTC.
You said:
when I retrieve them they are in MST (my local) timezone, rather than UTC, and I am unable to convert them back to the actual time.
We are not using UTC, nor any other time zone or offset-from-UTC in our appointment booking. So your problem vaporizes.
Tip: As I said above, your tools are likely lying to you about time zones. Set the tool and your session to UTC, if need be, to defeat this anti-feature.
You said:
I have tried using the java.sql.timestamp, java.sql.Date, java.util.Date, and java.time.ZonedDateTime on my entity.
Never use java.sql.Timestamp. Replaced by OffsetDateTime (and Instant).
Never use java.sql.Date. Replaced by LocalDate.
Never use java.util.Date. Replaced by Instant.
Use only the modern java.time classes that years ago supplanted those terrible date-time classes bundled with the earliest versions of Java. Those awful classes became legacy as of the adoption of JSR 310.
The jdbc driver has some knowledge about the timezone you are currently in. Generally I have gotten around this in the past by having the database do the timezone conversion for me, some derivative of "timestamp without time zone AT TIME ZONE zone" or "timestamp with time zone at time zone 'UTC'". It is in the guts of the postgres jdbc driver that it is figuring out what timezone the JVM is at and is using it in the save.
I am searching for an standard SQL way to ensure independently from the database to have a column with a timestamp in UTC or as timestamp with the information of the timezone.
UTC example with stored timestamp UTC in DB-column:
stored UTC value = 2014-01-01 15:30:00.000
local time India 21:00/9pm as local timestamp "2014-01-01 21:00:00.000"
stored UTC value = 2013-12-31 23:30:00.000
local time India 05:00/5am as local timestamp "2014-01-01 05:00:00.000"
In the UTC way the application has to cope with the timezoning
And now the way I do not really know if possible, with timestamp and timezone with the above values
stored local value = 2014-01-01 21:00:00.000
local time India 21:00/9pm as local timestamp "2014-01-01 21:00:00.000"
but there is the timezone information?
or how to get the timezone information of this timestamp?
stored local value = 2014-01-01 05:00:00.000
local time India 05:00/5am as local timestamp "2014-01-01 05:00:00.000"
but there is the timezone information?
or how to get the timezone information of this timestamp?
Hopefully someone can help me out of the dark?
Are there good practices on timezoning issues and storing the data for international application?
tl;dr
To store a moment, a point on the timeline, define a database column of SQL-standard type TIMESTAMP WITH TIME ZONE.
Store a moment in UTC.
Instant instant = Instant.now() ; // Capture the current moment in UTC.
myPreparedStatement.setObject( … , instant ) ;
Retrieve a moment in UTC.
Instant instant = myResultSet.getObject( … , Instant.class ) ;
Adjust from UTC to a time zone. Same moment, different wall-clock time.
ZoneId z = ZoneId.of( "Asia/Kolkata" ) ; // Specify time zone in proper `Continent/Region` name, never the ambiguous non-standard 3-4 letter pseudo-zone such as `IST` or `PST`.
ZonedDateTime zdt = instant.atZone( z ) ;
Adjust from a time zone to UTC. Same moment, different wall-clock time.
Instant instant = zdt.toInstant() ;
Date-Time Is Not A String
Serious databases do not store date-time values as strings. Do not confuse a string representation generated by either the database or a Java library with the date-time value itself.
Store UTC
Best practice is to use UTC for your database and other storage, as well as the bulk of your business logic. Present in local time zones only as expected by the user.
See Wikipedia for standard date-time types. Even if you do not use Postgres, consult the outstanding documentation on date-time data types. But I do recommend Postgres, for many reasons including its excellent support of date-time.
Use TIMESTAMPZ
This Postgres expert makes the simple and wise recommendation: Always Use TIMESTAMP WITH TIME ZONE
The name of that data type is a misnomer, causing so much confusion. The time zone information is not stored. What it means is that the time zone indicated with incoming data is respected, and the stored value will be adjusted to UTC. Think of TIMESTAMP WITH TIME ZONE as TIMESTAMP WITH RESPECT FOR TIME ZONE. Reread this paragraph three times out loud, then read the above links, and do some experimenting to be sure you understand.
You may want to also store separately the original time zone information. Not for use in your business logic, but for use as logging info for debugging.
Java
As for Java, be sure to avoid the java.util.Date and .Calendar classes. They are notoriously troublesome. They are supplanted in Java 8 by the new java.time package. Use either that package and/or the Joda-Time 2.4 library which inspired java.time.
Time Zone
And in Java always specify the desired time zone. If omitted you'll implicitly be using the JVM's current default time zone. That implicit default means your results will vary across time and space. This is the root cause of much of the trouble in date-time work. If you want UTC, use the constant in Joda-Time, DateTimeZone.UTC.
Ignoring time zones will not make your life easier.
ISO 8601
This standard is extremely useful and sensible. Study the excellent Wikipedia page. Should be your first choice for a String representation.
Search StackOverflow
These issues have been covered on hundreds, if not thousands, of answers.
I have a table which stores both unix time and the equivalent time stamp.
CREATE TABLE tbl_time
(
time_unix BIGINT,
time_timestamp TIMESTAMP WITHOUT TIME ZONE
);
The database is in PostgreSQL. database has been configured with Asia/Tehran time zone.
for example:
1333436817, 2012-04-03 11:36:57
When I convert the unix time into string format in python with:
datetime.datetime.fromtimestamp(1333436817)
it gives me: datetime.datetime(2012, 4, 3, 11, 36, 57) which is correct and equal to database. But when I do this conversion with java using:
Calendar c = Calendar.getInstance(TimeZone.getTimeZone("Asia/Tehran"));
c.setTimeInMillis(1333436817 * 1000);
System.out.println(c.getTime());
It gives: Sat Jan 24 06:12:35 IRST 1970. The system itself is running under Asia/Tehran time zone. I'm using PostgreSQL 8.4.11 on Debian 6.0.5 with python 3.1 and openjdk 6. Can anyone help?
The result of 1333436817 * 1000 is too big for an integer so it overflows. Java will not promote the type for you automatically in this case.
Try that:
c.setTimeInMillis(1333436817 * 1000L);
Notice the L that force your calculation to use long integer.
The Answer by tibo is correct. My additional thoughts follow.
Use TIMESTAMP WITH TIME ZONE
You mentioned using the data type TIMESTAMP WITHOUT TIME ZONE in Postgres. That type is only for a date-time not tied to any specific time zone. For example, "Christmas starts at midnight on December 25, 2015" translates to a different moment in any particular time zone. Christmas starts earlier, for example, in Paris than in Montréal. This data type is rarely appropriate in business apps. See this Postgres expert’s post, Always use TIMESTAMP WITH TIME ZONE.
In Postgres, the other type TIMESTAMP WITH TIME ZONE means "with respect for time zone". Any offset from UTC or time zone information with incoming data is used to adjust to UTC. That accompanying offset or time zone info is then discarded. Some databases preserve this info, but not Postgres.
Your statement:
The database is in PostgreSQL. database has been configured with Asia/Tehran time zone.
…makes no sense. The data type TIMESTAMP WITHOUT TIME ZONE has no time zone (though you may regard it as UTC), and the data type TIMESTAMP WITH TIME ZONE is always in UTC. With regard to storing date-time values, there is no such time zone configuration.
What you may have meant is that the default time zone for a database session is set to Tehran time zone. See the SET TIME ZONE command. But that setting is mere window-dressing, applied when generating a string representation of the date-time value. When using JDBC and the java.sql.Timestamp class, that session setting is irrelevant as no Strings are being generated by Postgres. Your attention to time zones should be on the Java side (see code below) rather than Postgres.
Generally speaking, your host server operating system should be set to UTC. But your app’s code should never depend on that, instead specifying any desired/expected time zone.
java.time
In Java 8 and later, the new java.time package supplants the old java.util.Date/.Calendar classes. These new classes were inspired by Joda-Time library, defined by JSR 310, and extended by the ThreeTen-Extra project.
Eventually JDBC drivers will be updated to directly handle these new types. In the mean time use the conversion methods added to both the old and new classes.
java.sql.Timestamp ts = myResultSet.getTimestamp( 1 );
Instant instant = ts.toInstant();
ZoneId zoneId = ZoneId.of( "Asia/Tehran" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
Or given your count of whole seconds from the Unix Time epoch, construct an Instant.
long secondsSinceUnixEpoch = 1_333_436_817L ;
Instant instant = Instant.ofEpochSecond( secondsSinceUnixEpoch );
ZoneId zoneId = ZoneId.of( "Asia/Tehran" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
The java Date library has a bad design and not that functional. I can't really help you with your problem but i can give you an advice to try Joda calendar