I have a date field in our PostgreSQL table where date is being saved including time stamp and zone (using "localdatetime" in java side when capturing). But requirement is to pull up record checking only the date part (ignore the time and zone), so that all users from all over the world can see the same result if filter by date.
Please let me know should I provide more explanation.
Wrong data types means lost information
postgres table where date is being saved including time stamp and zone ( using "localdatetime" in java side when capturing )
That is a contradiction in types.
The TIMESTAMP WITH TIME ZONE type in PostgreSQL does not save a time zone. Read the documentation carefully. It explains that any time zone or offset information provided along with the date and time-of-day is used to adjust to an offset from UTC of zero hours-minutes-seconds. In other words, the date and time are adjusted “to UTC”, as we say in shorthand.
That zone or offset information you may have provided is then discarded. Postgres does not remember the original offset or zone. Any value retrieved from a column of type TIMESTAMP WITH TIME ZONE is always in UTC. If you care about the original zone or offset, you must store that yourself in a second column.
But you have a problem. You did not provide any indicator of time zone or offset when sending the date-time to the database. You incorrectly used the LocalDateTime class which purposely lacks any indicator of time zone or offset. The LocalDateTime class represents a date with time-of-day, and nothing else. Thus the contradiction mentioned above. You provided two things (date, time-of-day) to a column that needs three things (date, time-of-day, and zone/offset).
I presume that Postgres took your LocalDateTime value and stored it as if it it were a date and time in UTC. So you have lost information. For example, if one row’s input was meant to be noon on the 23rd of January 2022 in Tokyo Japan 🇯🇵, while another row’s input was meant to be noon on that same 23rd date in Toulouse France 🇫🇷, and yet a third row’s input was meant for noon also on that 23rd but as seen in Toledo Ohio US 🇺🇸, then you will be incorrectly recording three different moments that happened several hours apart as if they all happened at another moment, a fourth moment, noon on the 23rd as seen in UTC, 2022-02-23T12:00Z.
After recording such a mish-mash of erroneous values, there is no going back. Unless you have a way of knowing for certain the intended zone of each row, your information is lost, and your stored column is worthless.
Querying for a day
Let’s set aside the issue of having invalidly stored moments. Let’s now focus on the issue of how to query for a day’s worth of rows with a column of the type TIMESTAMP WITH TIME ZONE.
You need to understand that for any given moment, the date varies by time zone. At any moment it may be “tomorrow” in Sydney Australia 🇦🇺 while simultaneously “yesterday” in Edmonton Alberta Canada 🇨🇦. So you need the context of a time zone (or offset) to perceive a date.
If you want to query for a day as seen in Edmonton, specify the time zone.
ZoneId z = ZoneId.of( "America/Edmonton" ) ;
Specify the desired date.
LocalDate ld = LocalDate.of( 2022 , Month.JANUARY , 23 ) ;
Determine a span of time representing all the moments of that day. The best approach to define such a span is the Half-Open approach. In Half-Open, the beginning is inclusive while the ending is exclusive. So the span starts with the first moment of the day and runs up to, but does not include, the first moment of the following day.
To get start of day, let java.time determine that. Days do not always start at 00:00.
ZonedDateTime start = ld.atStartOfDay( z ) ;
ZonedDateTime end = ld.plusDays( 1 ).atStartOfDay( z ) ;
The ZonedDateTime does not map to any data type in SQL. So convert to an OffsetDateTime object for exchanging with the database.
OffsetDateTime odtStart = start.toOffsetDateTime() ;
OffsetDateTime odtEnd = end.toOffsetDateTime() ;
Pass those through your prepared statement.
myPreparedStatement.setObject( … , odtStart ) ;
myPreparedStatement.setObject( … , odtEnd ) ;
In your SQL, do not use BETWEEN. That command is fully closed. For Half-Open, write your SQL query as looking for values that (a) are not before the start (“not before” is an abbreviated way of saying “is equal to or later than”), and (b) are before the end.
Assuming 24.12.2021 15:00 Berlin (Germany) is in the database.
People from Germany will see 15:00.
But people from London must see either 16:00 or "15:00 wall clock in Berlin".
Since you said
from all over the world can see the same resut
There will be a difference between 15:00 and 16:00: the number 5 and the number 6, they are different.
You could give everyone the same results by printing
15:00 wall clock in Berlin
But this needs to transport the Timezone Berlin beside the datetime.
Related
I have seen a lot of debates on the following date conversion:
timeStamp.toLocalDateTime().toLocalDate();
Some people say that it is not appropriate because the timezone has to be specified for proper conversion, otherwise the result may be unexpected. My requirement is that I have an object that contains Timestamp fields and another object that contains LocalDate fields. I have to take the date difference between both so I think that the best common type to use is LocalDate. I don't see why the timezone has to be specified as either timestamp or LocalDate just represent dates. The timezone is already implied. Can someone give an example when this conversion fails?.
It’s more complicated than that. While it’s true that a Timestamp is a point in time, it also tends to have a dual nature where it sometimes pretends to be a date and time of day instead.
BTW, you probably already know, the Timestamp class is poorly designed and long outdated. Best if you can avoid it completely. If you are getting a Timestamp from a legacy API, you are doing the right thing: immediately converting it to a type from java.time, the modern Java date and time API.
Timestamp is a point in time
To convert a point in time (however represented) to a date you need to decide on a time zone. It is never the same date in all time zones. So the choice of time zone will always make a difference. So one correct conversion would be:
ZoneId zone = ZoneId.of("Africa/Cairo");
LocalDate date = timestamp.toInstant().atZone(zone).toLocalDate();
The Timestamp class was designed for use with your SQL database. If your datatype in SQL is timestamp with time zone, then it unambiguously denotes a point in time, and you need to see it as a point in time as just described. Even when to most database engines timestamp with time zone really just means “timestamp in UTC”, it’s still a point in time.
And then again: sometimes to be thought of as date and time of day
From the documentation of Timestamp:
A Timestamp also provides formatting and parsing operations to support
the JDBC escape syntax for timestamp values.
The JDBC escape syntax is defined as
yyyy-mm-dd hh:mm:ss.fffffffff, where fffffffff indicates
nanoseconds.
This doesn’t define any point in time. It’s a mere date and time of day. What the documentation doesn’t even tell you is that the date and time of day is understood in the default time zone of the JVM.
I suppose that the reason for seeing a Timestamp in this way comes from the SQL Timestamp datatype. In most database engines this is a date and time without time zone. It’s not a timestamp, despite the name! It doesn’t define a point in time, which is the purpose of and is in the definition of timestamp.
I have seen a number of cases where the Timestamp prints the same date and time as in the database, but doesn’t represent the point in time implied in the database. For example, there may be a decision that “timestamps” in the database are in UTC, while the JVM uses the time zone of the place where it’s running. It’s a bad practice, but it is not one that will go away within a few years.
This must also have been the reason why Timestamp was fitted with the toLocalDateTime method that you used in the question. It gives you that date and time that were in the database, right? So in this case your conversion in the question ought to be correct, or…?
Where this can fail miserably without us having a chance to notice is, as others have mentioned already, when the default time zone of the JVM is changed. The JVM’s default time zone can be changed at any time from any place in your program or any other program running in the same JVM. When this happens, your Timestamp objects don’t change their point in time, but they do tacitly change their time of day, sometimes also their date. I’ve read horror stories — in Stack Overflow questions and elsewhere — about the wrong results and the confusion coming out of this.
Solution: don’t use Timestamp
Since JDBC 4.2 you can retrieve java.time types out of your SQL database. If your SQL datatype is timestamp with time zone (recommended for timestamps), fetch an OffsetDateTime. Some JDBC drivers also let you fetch an Instant, that’s fine too. In both cases no time zone change will play any trick on you. If the SQL type is timestamp without time zone (discouraged and all too common), fetch a LocalDateTime. Again you can be sure that your object doesn’t change its date and time no matter if the JVM time zone setting changes. Only your LocalDateTime never defined a point in time. Conversion to LocalDate is trivial, as you have already demonstrated in the question.
Links
java.sql.Timestamp documentation
Wikipedia article: Timestamp
Question: Getting the date from a ResultSet for use with java.time classes
Question: Java - Convert java.time.Instant to java.sql.Timestamp without Zone offset
As you can see here(taken from https://stackoverflow.com/a/32443004/1398418):
Timestamp represents a moment in UTC and is the equivalent of the modern Instant.
When you do:
timeStamp.toLocalDateTime().toLocalDate();
the timeStamp is converted from UTC to the system timezone. It's the same as doing:
timeStamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate()
For example:
Timestamp stamp = new Timestamp(TimeUnit.HOURS.toMillis(-1)); // UTC 1969-12-31
System.setProperty("user.timezone", "EET"); // Set system time zone to Eastern European EET - UTC+2
stamp.toLocalDateTime().toLocalDate(); // represents EET 1970-01-01
stamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate(); // represents EET 1970-01-01
That result (getting the date in the system time zone) is expected and if that's what you want, doing timeStamp.toLocalDateTime().toLocalDate() is appropriate and correct.
You're saying that you have a LocalDate field in some object and you want to get a period between it and a Timestamp, well that's just not possible without aditional information. LocalDate just represents a date, it has no time zone information, you need to know how it was created and what time zone was used.
If it represent a date in the system time zone then getting the period by using timeStamp.toLocalDateTime().toLocalDate() would be correct, if it represents a date in UTC or any other time zone then you might get a wrong result.
For example if the LocalDate field represents a date in UTC you will need to use:
timeStamp.toInstant().atZone(ZoneId.of("UTC")).toLocalDate();
Example: the 23rd of January becomes the 24th
You asked:
Can someone give an example when this conversion fails?.
Yes, I can.
Start with the 23rd of January.
LocalDate ld = LocalDate.of( 2020 , Month.JANUARY , 23 );
LocalTime lt = LocalTime.of( 23 , 0 );
ZoneId zMontreal = ZoneId.of( "America/Montreal" );
ZonedDateTime zdt = ZonedDateTime.of( ld , lt , zMontreal );
Instant instant = zdt.toInstant();
zdt.toString() = 2020-01-23T23:00-05:00[America/Montreal]
instant.toString() = 2020-01-24T04:00:00Z
The Instant class represents a moment as seen in UTC. Let's convert to the terribly legacy class java.sql.Timestamp using the new conversion method added to that old class.
// Convert from modern class to troubled legacy class `Timestamp`.
java.sql.Timestamp ts = Timestamp.from( instant );
ts.toString() = 2020-01-23 20:00:00.0
Unfortunately, the Timestamp::toString method dynamically applies the JVM’s current default time zone while generating text.
ZoneOffset defaultOffset = ZoneId.systemDefault().getRules().getOffset( ts.toInstant() );
System.out.println( "JVM’s current default time zone: " + ZoneId.systemDefault() + " had an offset then of: " + defaultOffset );
JVM’s current default time zone: America/Los_Angeles had an offset then of: -08:00
So Timestamp::toString misreports the object’s UTC value after adjusting back eight hours from 4 AM to 8 PM. This anti-feature is one of several severe problems with this poorly designed class. For more discussion of the screwy behavior of Timestamp, see the correct Answer by Ole V.V.
Let's run your code. Imagine at runtime the JVM’s current default time zone is Asia/Tokyo.
TimeZone.setDefault( TimeZone.getTimeZone( "Asia/Tokyo" ) );
LocalDate localDate = ts.toLocalDateTime().toLocalDate();
Test for equality. Oops! We ended up with the 24th rather than the 23rd.
boolean sameDate = ld.isEqual( localDate );
System.out.println( "sameDate = " + sameDate + " | ld: " + ld + " localDate: " + localDate );
sameDate = false | ld: 2020-01-23 localDate: 2020-01-24
See this code run live at IdeOne.com.
So what is wrong with your code?
Never use java.sql.Timestamp. It is one of several terrible date-time classes shipped with the earliest versions of Java. Never use these legacy classes. They have been supplanted entirely by the modern java.time classes defined in JSR 310.
You called toLocalDateTime which strips away vital information. Any time zone or offset-from-UTC is removed, leaving only a date and a time-of-day. So this class cannot be used to represent a moment, is not a point on the timeline. Ex: 2020-12-25 at noon — is that noon in Delhi, noon in Düsseldorf, or noon in Detroit, three different moments several hours apart? A LocalDateTime is inherently ambiguous.
You ignored the crucial issue of time zone in determining a date. For any given moment, the date varies around the globe. At one moment it may be “tomorrow” in Australia while simultaneously “yesterday” in Mexico.
The problem lies in what is being represented by these objects. Your question forgets a crucial aspect, which is: What is the type of timeStamp?
I'm guessing it's a java.sql.Timestamp object.
Timestamp, just like java.util.Date, is old API equivalent to Instant.
It represents an instant in time, in the sense that it is milliseconds since jan 1st 1970 UTC. The system has no idea which timezone that was supposed to be in. You're supposed to know; the error, if an error is going to occur here, already occurred before you get to this code. Here's a trivial explanation of how it COULD go wrong:
you start off with a user entering a date in a date field on a webform; it's 2020-04-01.
Your server, running in Amsterdam, saves it to a DB column that is internally represented as UTC, no zone. This is a mistake (you're not saving an instant in time, you're saving a date, these two are not the same thing). What is actually stored in the DB is the exact moment in time that it is midnight, 2020-04-01 in amsterdam (in UTC, that'd be 22:00 the previous day!).
Later, you query this moment in time back into a java.sql.Timestamp object, and you're doing this when the server's tz is elsewhere (say, London time). You then convert this to a localdatetime, and from there to a localdate, and.... you get 2020-03-31 out.
Whoops.
Dates should remain dates. Never convert LocalX (be it Time, Date, or DateTime) to Instant (or anything that effectively is an instant, including j.s.Timestamp, or j.u.Date - yes, j.u.Date does NOT represent a date, it is very badly named), or vice versa, or pain will ensue. If you must because of backward APIs take extreme care; it's hard to test that 'moving the server's timezone around' breaks stuff!
I have one question about zones in java.
I have a user case when user can set time zone and some schedule. For example run task on Mondays and Sundays in 11pm with timezone America/Los_Angeles.
If my server time zone is UTC+0:00 I have a problem how to detect a day of week and new time correctly (it will be about Tuesday and Monday on 6am). So my question is how to run user tasks correctly according to server timezone and user timezone.
Update
I have a cron expression where I set hours, minutes and day of week.
When user creates a new task he can set custom timezone(for example to run task on mondays 11pm with custom timezone UTC-7:00)
So if my understanding is right I need to convert his hour setting (11pm) to corresponding server time. So if my server timezone is UTC+3:00 I need to convert 11pm to 9am and it will be not Monday (but Tuesday) on my server. Then if I will run cron on Tuesday 9am it will look to user like task runs on 11pm on Monday. Is my approach correct? I hope you understand my question. Thanks in advance.
The short answer: Count in milliseconds since the epoch.
The long answer:
First, it is very important to understand how computer time works. I would recommend this article., but the gist is, that everything is computed since the epoch, 1.1.1970. Second, I figured an example will tell you more than 1000 words:
ZonedDateTime dateTime = ZonedDateTime.parse("2016-10-20T11:34:57+02:00[Europe/Zurich]"); // UTC + 2
long millisEurope = dateTime.toInstant().toEpochMilli();
System.out.println(millisEurope); // 1476956097000
dateTime = dateTime.withZoneSameLocal(ZoneId.of("America/Los_Angeles")); // UTC - 7
long millisAmerica = dateTime.toInstant().toEpochMilli();
System.out.println(millisAmerica); // 1476988497000
// The difference between UTC + 2 and UTC - 7 == -9
System.out.println((millisEurope - millisAmerica) / 1000 / 60 / 60);
The ZonedDateTime will always use UTC (+ 0) plus the defined time zone.
EDIT (after the update):
Yes, if you get the time and time zone set by the user, you would have to convert it to your own time zone to be executed on the same point in time.
The time zone setting of your server’s operating system and JVM should be irrelevant to your programming. Both can change at any moment, during runtime, so do not rely on it. Always specify the desired/expected time zone in the optional argument to the various date-time methods.
Be aware that you cannot schedule time-zone-sensitive moments far in advance. Politicians are notorious for often changing the time zone definitions, sometimes with very little advance notice.
If you want to set an alarm for the next Monday at 11 AM in the context of the user’s time zone, first you need the user’s time zone. You may be able to detect a default time zone, but ultimately the only reliable way is to ask the user for their desired/expected time zone.
Specify a proper time zone name in the format of continent/region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 3-4 letter abbreviation such as EST or IST as they are not true time zones, not standardized, and not even unique(!).
Keep the alarm’s day-of-week as a DayOfWeek enum object.
DayOfWeek alarmDow = DayOfWeek.MONDAY ;
Keep the alarm’s time-of-day as a LocalTime.
LocalTime alarmTimeOfDay = LocalTime.parse( "11:00:00" );
This class lacks a date and lacks any offset-from-UTC or time zone. So it has no meaning until you adjust into a time zone.
Keep the alarm’s time zone as a ZoneId.
ZoneId z = ZoneId.of( "America/Montreal" );
With those parts in hand, you can schedule the alarm.
Get the current moment as an Instant. The Instant class represents a moment on the timeline in UTC with a resolution of nanoseconds (up to nine (9) digits of a decimal fraction).
Instant now = Instant.now();
Apply the desired time zone to get a ZonedDateTime.
ZonedDateTime zdtNow = instant.atZone( z );
To determine the future moment of the alarm, use the TemporalAdjuster interface to manipulate date-time values. The class TemporalAdjusters (note the plural s) provides implementations. First extract a date-only value, as we will be assigning the alarm’s LocalTime as the time-of-day.
LocalDate today = zdtNow.toLocalDate();
LocalDate dateOfNextOrSameDow = today.with( TemporalAdjusters.withNextOrSame( alarmDow ) );
Apply the desired time-of-day.
ZonedDateTime zdtAlarm = ZonedDateTime.of( dateOfNextOrSameDow , alarmTimeOfDay , z ) ;
This particular alarm date-time may have already passed earlier today. So test. If so, add a week to get the next occurrence of the desired day-of-week.
if( zdtAlarm.isBefore( zdtNow ) ) { // If already passed…
zdtAlarm = zdtAlarm.plusWeeks( 1 ); // …go to next day-of-week occurrence.
}
Follow-up question to my original issue.
Because I know that any date time used is against a known timezone, rather than where the user may be submitting their request from, I take a LocalDateTime, convert to UTC and persist. Then, when the appointment is retrieved I convert the saved time to the meeting location timezone (stored in db). However, it would seem that the values I save are actually being saved in my local timezone.
I receive a date time value in the Rest Controller such as:
startLocalDateTime: 2016-04-11T10:00
endLocalDateTime: 2016-04-11T10:30
Appointment has two ZoneDateTime fields:
#Column(name = "startDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
private ZonedDateTime startDateTime;
#Column(name = "endDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
private ZonedDateTime endDateTime;
Then I change the values to UTC and store on my entity to store to Postgres:
appointment.setStartDateTime(startLocalDateTime.atZone(ZoneId.of( "UTC" )))
appointment.setEndDateTime(endLocalDateTime.atZone(ZoneId.of( "UTC" )))
and I store that in Postgres (columnDefinition= "TIMESTAMP WITH TIME ZONE") When I look at the record in pgadminIII I see:
startDateTime "2016-04-11 04:00:00-06"
endDateTime "2016-04-11 04:30:00-06"
So these appear to be stored properly in UTC format (please correct me if I am doing anything wrong so far). I then retrieve them from the database and they are returned as:
Appointment
startdatetime: 2016-04-11T04:00-06:00[America/Denver]
enddatetime: 2016-04-11T04:30-06:00[America/Denver]
Those values are sent back as JSON:
{
"appointmentId":50,
"startDateTime":"2016-04-11T04:00",
"endDateTime":"2016-04-11T04:30"
}
So even though I am saving them as UTC, when I retrieve them they are in MST (my local) timezone, rather than UTC, and I am unable to convert them back to the actual time.
Still struggling with the persistence. I have tried using the java.sql.timestamp, java.sql.Date, java.util.Date, and java.time.ZonedDateTime on my entity. My Postgres is still a "timestamp with time zone". But because I am using Spring-Data-JPA and need to query with the same type. If I use Date - should that be sql.Date or util.Date?
You said:
I take a LocalDateTime, convert to UTC and persist.
Nope. Stick with that LocalDateTime for booking appointments, no UTC involved for appointments.
When booking appointments that should appear as a certain time-of-day regardless of how politicians redefine the offset used within the time zone(s) under their jurisdiction, store a date with time-of-day but keep the time zone separate.
Politicians around the world have shown a strange penchant for frequently redefining the offset of their zones, adjusting to match or differ their neighbors, or to adopt or drop the foolishness known as Daylight Saving Time (DST). These changes are done with little forewarning, or even none at all. So what is right now looks like 2:30 PM on a future day could become 1:30 PM, 2:00 PM, 3:30 PM, or who knows what the politicians might come up with, if we stored as a moment (date, time-of-day, zone/offset, all together).
Java
To represent a date and time-of-day only, use LocalDateTime. This class purposely lacks any concept of time zone or offset-from-UTC. This lack means this class does not track moments, is not a point on the timeline. So we generally do not use this class in business apps. Booking future appointments is one of the few cases where we do want this class.
LocalDate ld = LocalDate.of( 2020 , Month.JANUARY , 23 ) ; // 2020-01-23.
LocalTime lt = LocalTime.of( 14 , 30 ) ; // 2:30 PM.
LocalDateTime ldt = LocalDateTime.of( ld , lt ) ;
See this code run live at IdeOne.com.
ldt.toString(): 2020-01-23T14:30
Store that in your database using JDBC 4.2 and later.
myPreparedStatement.setObject( … , ldt ) ;
Retrieve from database.
LocalDateTime ldt = myResultSet.getObject( … , LocalDateTime.class );
We need to track the intended time zone. Is this an appointment for a dentist office in Québec? Represent that fact as well. Keep in mind that a time zone is a history of past, present, and future (planned) changes to the offset-from-UTC used by the people of a particular region.
Use the ZoneId class to represent a time zone.
ZoneId z = ZoneId.of( "America/Montreal" ) ;
Save that zone name as text to your database. Get its identifying name by calling ZoneId::toString. Do not use ZoneId::getDisplayName as that is for generating localized text for display to a user, but is not a formal identifier for the zone.
myPreparedStatement.setString( … , z.toString() ) ;
When your retrieve from database, reconstitute the ZoneId.
String zoneIdString = myResultSet.getString( … ) ;
ZoneId z = ZoneId.of( zoneIdString ) ;
When you are generating a calendar for these appointments, if you need to determine (tentatively) points on the timeline, combine the date-with-time-of-day and the time zone together. Applying the ZoneId to the LocalDateTime produces a ZonedDateTime. This ZonedDateTime is a moment, is a point on the timeline, unlike LocalDateTime.
ZonedDateTime zdt = ldt.atZone( z ) ;
See this code run live at IdeOne.com.
zdt.toString(): 2020-01-23T14:30-05:00[America/Montreal]
Of course, we do not store that ZonedDateTime. As discussed above this moment that right now appears to be 2:30 PM on the 23rd would turn into 1:30 PM or 3:30 PM if those busy politicians changed the time zone rules between now and then. We only use this ZonedDateTime tentatively, temporarily.
Database
We store a LocalDateTime only in a column of type TIMESTAMP WITHOUT TIME ZONE. You were incorrectly using WITH rather than WITHOUT, a major problem. For the WITHOUT type, Postgres stores the date and the time-of-day as given in the input. Even if a zone or offset were included with the input, Postgres would ignore the zone or offset.
Be aware that TIMESTAMP WITH TIME ZONE is just the opposite in Postgres. The time zone or offset info accompanying an input is used to adjust the date and time to UTC. That UTC value is then stored. So values in a column of this type are all in UTC. Unfortunately, many tools have the anti-feature of dynamically applying a default time zone to the retrieved value before passing on to you. This creates the false illusion of a time zone being stored when in fact the value is in UTC. Retrieving as a OffsetDateTime will always tell you the truth. I say OffsetDateTime because, oddly, the JDBC 4.2 spec requires support for that class but not for the more commonly used Instant & ZonedDateTime classes. Your JDBC driver may support the other two classes, optionally. But all of this paragraph is neither here nor there, as we are not using WITH columns for future appointments.
As mentioned above, for future appointments should be stored in three (3) separate columns:
TIMESTAMP WITHOUT TIME ZONE for the date and the time-of-day.
TEXT (or similar type) for the identifying name of the intended time zone by which we want to view that appointment.
TEXT (or similar) for the duration of the appointment, in standard ISO 8601 format. (see below)
Points in your Question
You said:
I receive a date time value in the Rest Controller such as:
startLocalDateTime: 2016-04-11T10:00
So parse as a LocalDateTime. Such ISO 8601 compliant strings can be parsed directly by the java.time classes.
LocalDateTime ldt = LocalDateTime.parse( "2016-04-11T10:00" ) ;
You said:
endLocalDateTime: 2016-04-11T10:30
No. Do not store the ending time. The ending time you expect now could be different if time zone anomalies occur at that point. For example, what if your politicians adopt Daylight Saving Time (DST), then during the time of this appointment the "Spring ahead" 1-hour change of DST occurred? Then your half-hour meeting should run from 10:00 AM to 11:30 AM, while your recorded end-time would incorrectly read "10:30 AM" ending.
Generally the best way to handle appointments is to record the duration of the appointment, the span-of-time, rather than on the clock. As mentioned above, an appointment consists of three separate pieces of data: (1) starting date with time-of-day, (2) the intended time zone, and (3) duration of appointment. The ending should be dynamically calculated as needed, using fresh current time zone data, for temporary use.
The ISO 8601 standard includes a textual format for recording such durations: PnYnMnDTnHnMnS where P marks the beginning, and T separates any years-months-days from any hours-minutes-seconds. The java.time classes Period and Duration can parse such strings.
Duration d = Duration.ofHours( 1 ) ;
String output = d.toString() ;
See this code run live at IdeOne.com.
output: PT1H
You can dynamically apply this duration to the LocalDateTime.
LocalDateTime ending = ldt.plus( d ) ;
Ditto for ZonedDateTime.
ZonedDateTime ending = zdt.plus( d ) ;
You said:
Appointment has two ZoneDateTime fields:
Nope. Make that a LocalDateTime field, not ZonedDateTime.
You said:
#Column(name = "startDateTime", columnDefinition= "TIMESTAMP WITH TIME ZONE")
Nope. Make that a column of type "TIMESTAMP WITHOUT TIME ZONE" to track a date and time-of-day without the context of zone/offset.
You said:
Then I change the values to UTC and store on my entity to store to Postgres:
Nope. Booking appointments is one of the few cases where we do not want UTC. When tracking a moment, we generally do want to use UTC. But future appointments are not moments. We do not know the moment until we apply a time zone to the date-with-time-of-day, and we cannot do that ahead of time because we cannot trust politicians.
You said:
Still struggling with the persistence.
Cannot help you there. I do not use either Spring or JPA, as they solve problems I do not have. I use straight JDBC.
You said:
So these appear to be stored properly in UTC format (please correct me if I am doing anything wrong so far).
Nope, do not use UTC, do not store in UTC, not for appointments.
Do use UTC for moments though. For example, your logs should all be reporting each moment of an incident in UTC. Programmers & sysadmins would do well to learn to think in UTC while on the job. Keep a second clock on your desk set to UTC.
You said:
when I retrieve them they are in MST (my local) timezone, rather than UTC, and I am unable to convert them back to the actual time.
We are not using UTC, nor any other time zone or offset-from-UTC in our appointment booking. So your problem vaporizes.
Tip: As I said above, your tools are likely lying to you about time zones. Set the tool and your session to UTC, if need be, to defeat this anti-feature.
You said:
I have tried using the java.sql.timestamp, java.sql.Date, java.util.Date, and java.time.ZonedDateTime on my entity.
Never use java.sql.Timestamp. Replaced by OffsetDateTime (and Instant).
Never use java.sql.Date. Replaced by LocalDate.
Never use java.util.Date. Replaced by Instant.
Use only the modern java.time classes that years ago supplanted those terrible date-time classes bundled with the earliest versions of Java. Those awful classes became legacy as of the adoption of JSR 310.
The jdbc driver has some knowledge about the timezone you are currently in. Generally I have gotten around this in the past by having the database do the timezone conversion for me, some derivative of "timestamp without time zone AT TIME ZONE zone" or "timestamp with time zone at time zone 'UTC'". It is in the guts of the postgres jdbc driver that it is figuring out what timezone the JVM is at and is using it in the save.
In our web application we need to show and enter
date time information for different countries in different time zone. Right now, we are maintaining separate web server and separate database (oracle 11g) for each country.
We are planning to merge all into one portal with single database (oracle 11g).
This portal should capture/display date and time in user local time zone.
So far, I have searched about this, I got below suggestion.
1) set web server's and database server's time zone to UTC and while fetching data (data and time) convert into user local time zone.
If you suggest this approach then please clarify the following specific
questions.
most of the time we are capturing date alone, is it require to
capture date and time along with time zone always?
while storing date and time where we need to convert user local time
zone to UTC in javascript/java/oracle?
while fetching date and time where we need to convert UTC to user
local time zone query itself/java/java script?
many place we have reports to show based on date column such as
today/current month/date range.how we can handle this(input - user
local time zone - database in UTC)?
which data type we have to use for date field
(date/timestamp/timestamp with time zone/timestamp with local time
zone)?
2) capture date and time in both user local time zone and UTC. Stored as separate columns, user local time zone will be used for display purpose and UTC will be used for business logic.
If you suggest this approach then please clarify the following specific
questions.
Is it common practice to store the user local time zone and UTC?
which column i have to check condition while fetching reports to show
based on date column such as today/current month/date range?
which data type we have to use for date column
(date/timestamp/timestamp with time zone/timestamp with local time
zone)?
thanks in advance
Read the Question Daylight saving time and time zone best practices. Yours is basically a duplicate.
Servers in UTC
Yes, generally servers should have their OS set to UTC as the time zone, or if not provided use GMT or the Reykjavík Iceland time zone. Your Java implementation probably picks up this setting as its own current default time zone.
Specify time zone
But do not depend on the time zone being set to UTC. A sysadmin could change it. And any Java code in any thread of any app within your JVM can change the JVM’s current default time zone at runtime by calling TimeZone.setDefault. So instead, make a habit of always specifying the desired/expected time zone by passing the optional argument in your Java code.
I consider it a design flaw that any date-time framework would make the time zone optional. Being optional creates endless amounts of confusion because programmers, like everybody else, unconsciously think in terms of their own personal time zone unless prompted. So all too often in date-time work no attention is paid to the issue. Add on the problem that the JVM default varies. By the way, ditto for Locale, same problems, should always be specified explicitly.
UTC
Your business logic, data storage, and data exchange should almost always be done in UTC. Nearly every database has a feature for adjusting any input into UTC and storing in UTC.
When presenting a date-time to a user, adjust into the expected time zone. When serializing a date-time value, use the ISO 8601 string formats. See the Answer by VickyArora for Oracle specifically (I am a Postgres person). Be sure to read the doc carefully, and practice by experimenting to fully understand your database's behavior. The SQL spec does not spell out very much in this regard, and behavior varies widely.
java.sql
Remember that when using Java and JDBC, you will be using the java.sql.Timestamp and related data types. They are always in UTC, automatically. In the future expect to see JDBC drivers updated to directly use the new data types defined in the java.time framework built into Java 8 and later.
java.time
The old classes are outmoded by java.time. Learn to use java.time while avoiding the old java.util.Date/.Calendar and make your programming life much more pleasant.
Until your JDBC driver is updated, you can use the conversion convenience methods built into java.time. See examples next, where Instant is a moment in UTC and ZonedDateTime is an Instant adjusted into a time zone.
Instant instant = myJavaSqlTimestamp.toInstant();
ZoneId zoneId = ZoneId.of( "America/Montreal" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
To go the other direction.
java.sql.Timestamp myJavaSqlTimestamp = java.sql.Timestamp.from( zdt.toInstant() );
If you need original time zone, store it
If your business requirements consider the original input data’s time zone to be important, to be remembered, then store that explicitly as a separate column in your database table. You can use an offset-from-UTC, but that does not provide full information. A time zone is an offset plus a set of rules for the past, present, and future handling of anomalies such as Daylight Saving Time. So a proper time zone name is most appropriate such as America/Montreal.
Date-only is ambiguous
You said you collect many date-only values, without time-of-day and without time zone. The class for that in java.time is LocalDate. As with LocalTime and LocalDateTime, the “Local…” part means no particular locality, so therefore no time zone, and so not a point on the timeline -- has no real meaning.
Keep in mind that a date-only value is ambiguous by definition. At any given moment, the date varies around the world. For example, just after midnight in Paris France is a new day but in Montréal Québec the date is still “yesterday”.
Usually in business some time zone is implicit, even unconsciously intuited. Unconscious intuition about data points tends not to work well over the long term, especially in software. Better to make explicit what time zone was intended. You could store the intended zone alongside the date such as another column in database table, or your could make a comment in your programming code. I believe it would vastly better and safer to store a date-time value. So how do we transform a date-only into a date-time?
Often a new day is the moment after midnight, the first moment of the day. You might think that means the time-of-day 00:00:00.0 but not always. Daylight Saving Time (DST) and possibly other anomalies may push the first moment to a different wall-clock time. Let java.time determine the correct time-of-day for first moment going through the LocalDate class and its atStartOfDay method.
ZoneId zoneId = ZoneId.of( "America/Montreal" );
LocalDate today = LocalDate.now( zoneId );
ZonedDateTime todayStart = today.atStartOfDay( zoneId );
In some business contexts a new day may be defined (or assumed) to be business hours. For example, say a publisher in New York means 9 AM in their local time when they say “the book draft is due by January 2nd”. Let's get that time-of-day for that date in that time zone.
ZoneId zoneId = ZoneId.of( "America/New_York" );
ZonedDateTime zdt = ZonedDateTime.of( 2016 , 1 , 2 , 9 , 0 , 0 , 0 , zoneId );
What does that mean for the author working in New Zealand? Adjust into her particular time zone for presentation to her by calling withZoneSameInstant.
ZoneId zoneId_Pacific_Auckland = ZoneId.of( "Pacific/Auckland" );
ZonedDateTime zdt_Pacific_Auckland = zdt.withZoneSameInstant( zoneId_Pacific_Auckland );
Database
For database storage we transform into an Instant (a moment on the timeline in UTC) and pass as a java.sql.Timestamp as seen earlier above.
java.sql.Timestamp ts = java.sql.Timestamp.from( zdt.toInstant() );
When retrieved from the database, transform back to a New York date-time. Convert from java.sql.Timestamp to an Instant, then apply a time zone ZoneId to get a ZonedDateTime.
Instant instant = ts.toInstant();
ZoneId zoneId = ZoneId.of( "America/New_York" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
If your database driver complies with JDBC 4.2 or later, you may be able to pass/fetch the java.time types directly rather than convert to/from java.sql types. Try the PreparedStatement::setObject and ResultSet::getObject methods.
Use TIMESTAMP WITH LOCAL TIME ZONE
if you want the database to automatically
convert a time between the database and session time zones.
Stores a date and time with up to 9 decimal places of precision. This datatype is
sensitive to time zone differences. Values of this type are automatically converted
between the database time zone and the local (session) time zone. When values are
stored in the database, they are converted to the database time zone, but the local
(session) time zone is not stored. When a value is retrieved from the database, that
value is converted from the database time zone to the local (session) time zone.
Here I have clarified the following specific questions.
Q. Most of the time we are capturing date alone, is it require to capture date and time along with time zone always?
A. Yes
Q. while storing date and time where we need to convert user local time zone to UTC in javascript/java/oracle?
A. Not convert during data save, save as it with source date+time+zone
Q. While fetching date and time where we need to convert UTC to user
local time zone query itself/java/java script?
A. Always convert to display in local time zone OR UTC format where application opened.
Q. Many place we have reports to show based on date column such as
today/current month/date range.how we can handle this(input - user local time zone - database in UTC)?
A. The system should provide setting option to user for date time display in what format, either local where application opened or UTC. All is done on front end only.
Q. Which data type we have to use for date field (date/timestamp/timestamp with time zone/timestamp with local time zone)?
A. Timestamp
So in short, save datetime in source time zone and convert based upon user preferences either in local where page opened or UTC format. Means, conversion will be done through script for display only. The region where product is being popular can also be find.
I would simply transform the existing dates stored in the DB into Long, and persist (ETL process) this Long value, along with the known (or deducted) pattern, Locale and TimeZone (default meta). And persist any new Date as long as with the default meta.
ETL example
Let say 2015-11-29 10:07:49.500 UTC is stored in the DB:
// Known or deducted format of the persisted date
String pattern = "yyyy-MM-dd HH:mm:ss.SSS";
Locale locale = Locale.ENGLISH;
TimeZone zone = "UTC";
// Date to ms
SimpleDateFormat sdf = new SimpleDateFormat(pattern, locale);
sdf.setTimeZone(TimeZone.getTimeZone(zone));
Date date = sdf.parse(pattern);
// ETL: Can now be persisted in Long, along with default META (pattern, Locale, TZ)
Long dateL = date.getTime(); // for e.g. 1448827660720
...
The persisted Long value can also be transformed in any other format, if needed
pattern | locale | tz | result
============================================
yyyy/MM/dd | null | null | 2015/11/29
dd-M-yyyy hh:mm:ss | null | null | 29-11-2015 10:07:40
dd MMMM yyyy zzzz | ENGLISH | null | 29 November 2015 Central European Time
yyyy-MM-dd HH:mm:ss.SSS | null | UTC | 2015-11-29 10:07:49 UTC
Feasible and logical approach is ; Convert user entered time to GMT/UTC +00 and store that in db with or without timezone identifier it does not matter. When you need to display the time to user convert GMT/UTC time in java to local time of user.
You should consider JodaTime and should follow the first suggestion.JodaTime has many classes like LocalDate,LocalDateTime which you can use for your different use cases.
What api do I have to use to get the time difference between 2 timezones without inputing any datetime.
What I am doing right now, is make a temporary date (midnight) from one of the timezone then convert to utz and then convert again to the other timezone and then compute the duration of the two. It is working but is there a simpler api to get the time difference by just giving the name of the zones?
Too long for comment, so I put it here.
server is on diff. timezone, data/informations come from clients from different timezones and save in the db in utc. Problem arise when the client request a transaction history in a certain date. It seems it is not a simple conversion of the requested date (client timezone) to utc, I need to add the time difference of the server timezone and the client timezone to the converted utc time to get the correct date. (so i do it like I said above). Now I found out that the time difference should be added or subtracted, depending on whos timezone is ahead. Well anyway thanks for everybody's inputs. Need the project to run asap so they decided to use just one timezone for the meantime. Implemented or not I will seek a solution to this for future projects. :)
I'm not sure if I understood your question correctly. Are you trying to get offsets of different timezone without using any additional API?
Below code will do that using plain Java:
String[] ids = TimeZone.getAvailableIDs();
HashMap<String, Integer> map = new HashMap<String, Integer>();
for (String id : ids) {
TimeZone tz = TimeZone.getTimeZone(id);
map.put(id, tz.getOffset(new Date().getTime()) / 1000 / 60); //in minutes
}
System.out.println(map.toString());
private final static int tzDiff = TimeZone.getDefault().getRawOffset() - TimeZone.getTimeZone("America/New_York").getRawOffset();
Joda-Time in maintenance mode
Since you asked your Question, the creator of Joda-Time went on to lead JSR 310 to bring a new set of date-time classes to Java. Those classes arrived in Java 8.
Since then, the Joda-Time project was put into maintenance mode, with the project recommending migration to java.time.
Comparing time zones requires a moment
get the time difference between 2 timezones without inputing any datetime.
That makes no sense. You must specify a moment, a point on the time line, to compare offsets in effect in two different time zones.
Understand that an offset is merely a number of hours-minutes-seconds ahead or behind the prime meridian of time-keeping, UTC.
A time zone is much more. A time zone is a history of the past, present, and future changes to the offset in use by the people of a particular region as decided by their politicians.
By definition, the offset in use varies over time for any particular time zone. So it makes no sense to attempt a comparison of time zones without a specific point in time.
What I am doing right now, is make a temporary date (midnight) from one of the timezone then convert to utz and then convert again to the other timezone and then compute the duration of the two.
I am not sure of your goal, but here is code for doing something similar.
Specify your pair of time zones in which you are interested.
ZoneId zTokyo = ZoneId.of( "Asia/Tokyo" ) ;
ZoneId zEdmonton = ZoneId.of( "America/Edmonton" ) ;
Specify a date.
LocalDate ld = LocalDate.of( 2022 , Month.JANUARY , 23 ) ;
Determine when the day of that date begins in Tokyo time zone. Do not assume the day starts at 00:00. Some dates in some zones may start at a different time. Let java.time determine the first moment of the day.
ZonedDateTime firstMomentOfDayTokyo = ld.atStartOfDay( zTokyo ) ;
Adjust to see that some moment as it looks in the wall-clock time of Edmonton Alberta Canada.
ZonedDateTime zdtEdmonton = firstMomentOfDayTokyo.atZoneSameInstant( zEdmonton ) ;
See that same moment again with an offset of zero by extracting an Instant object.
Instant instant = firstMomentOfDayTokyo.toInstant() ;
Here is the crucial point to understand: firstMomentOfDayTokyo, zdtEdmonton, and instant all represent the very same moment, the simultaneous same point on the time line. Imagine three people in a conference call originating from Japan, Canada, and Iceland (where their time zone is always zero offset). If they all simultaneously looked up at the clock hanging on their respective wall, they would all see different local time-of-day yet be experiencing the same simultaneous moment.
Now that we have an Instant object in hand, we can proceed to solving your challenge: Getting the offset in use for each of those two time zones.
Fetch the time zone rules for each.
ZoneRules rulesTokyo = zTokyo.getRules() ;
ZoneRules rulesEdmonton = zEdmonton.getRules() ;
Get the offset in effect in each zone at that moment. Notice the use of ZoneOffset class here rather than ZoneId.
ZoneOffset offsetTokyo = rulesTokyo.getOffset( instant ) ;
ZoneOffset offsetEdmonton = rulesEdmonton.getOffset( instant ) ;
Calculate the difference between each zone.
Duration d = Duration.ofSeconds( offsetTokyo.getTotalSeconds() - offsetEdmonton.getTotalSeconds() ) ;
But this offset comparisons between time zones is not the best solution to your underlying problem. Read on for a better solution.
Querying database for a day
You said:
server is on diff. timezone
You should always do your Java programming is such a way as to not be affected by the current default time zone of your servers. Always specify the otherwise optional time zone (or offset) arguments to the various date-time related Java methods.
save in the db in utc
Good. It is generally best to store moments after adjusting to UTC. Some relational databases such as Postgres make such an adjustment when receiving inputs to a column of a type akin to the SQL standard type TIMESTAMP WITH TIME ZONE.
Retrieve such a value in JDBC 4.2 and later using java.time classes.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
Write such a value to the database.
myPreparedStatement.setObject( … , odt ) ;
You said:
problem arise when the client request a transaction history in a certain date. It seems it is not a simple conversion of the requested date (client timezone) to utc, I need to add the time difference of the server timezone and the client timezone to the converted utc time to get the correct date.
No, not the best approach. You are thinking in terms of local time zones.
When you go to work as a programmer, DBA, or system-admin, you should forget about your local time zone. Learn to think in UTC, with an offset of zero. Logging, data storage, data exchange, and most of your business logic should all be done in UTC. Keep a second clock on your desk set to UTC; I’m serious, your life will be easier.
As an example, let's say your user wants to query for all records with a timestamp the occurred at some point during one full day as seen in Tokyo Japan.
To query for a day's worth of records, do much the same as we saw above. There we got the first moment of the day in Tokyo.
ZoneId zTokyo = ZoneId.of( "Asia/Tokyo" ) ;
LocalDate ld = LocalDate.of( 2022 , Month.JANUARY , 23 ) ;
ZonedDateTime firstMomentOfDayTokyo = ld.atStartOfDay( zTokyo ) ;
That is the beginning of the span of time (a day) over which we want to query.
Generally best to define a span of time using Half-Open approach. In this approach the beginning is inclusive while the ending is exclusive. So a day starts at first moment of a date and runs up to, but does not include, the first moment of the following date.
ZonedDateTime firstMomentOfFollowingDayTokyo = ld.plusDays( 1 ).atStartOfDay( zTokyo ) ;
The OffsetDateTime is the class that maps to the TIMESTAMP WITH TIME ZONE type defined in standard SQL. So extract a OffsetDateTime from each.
OffsetDateTime start = firstMomentOfDayTokyo.toOffsetDateTime() ;
OffsetDateTime end = firstMomentOfFollowingDayTokyo.toOffsetDateTime() ;
Write your SQL like this:
SELECT *
FROM event_
WHERE when_ >= ?
AND when_ < ?
;
Do not use the SQL command BETWEEN for this work. That command is Fully-Closed rather than Half-Open.
In JDBC:
myPreparedStatement.setObject( 1 , start ) ;
myPreparedStatement.setObject( 2 , end ) ;
Keep in mind that a full day in a particular time zone is not necessarily 24-hours long. A day may be 23, 23.5, 25, or some other number of hours long because of anomalies in political time such as Daylight Saving Time (DST).