My apologies for the extremely basic question, I think I know the answer but would like to verify:
When refering to time zones and how they are usually stored in a database and in a java.util.Date:
When saving a date field in a database, it is timezone agnostic (e.g. always saves in UTC)
When using a Java Date object the date it is also timezone agnostic
Time zone is depeneded only when formatting and parsing dates (DB and Java)
If using Java - it will use the JVM user.timezone to format / parse dates
If using Java on Windows - Java will take this from the Regional settings automatically
The database timezone (and it's machine's timezone) is irrelevant to the Java JDBC client
The timezone of the Database Server is relevant only for direct SQL parsing and formating
My questions are:
Are all of the above assumptions correct? some are? all incorect?
Is there a reference / official source that verifies this more "officially"?
The assumption are mostly correct for Java. They are not necessarily correct for databases, as there are variations.
Java handles time zones with Calendar objects. The java.util.Date object always contains the UTC value.
Databases generally store and return dates and timestamps (with hour, minutes, etc.) as they are written, regardless of the internal format used by storage. If you store 2010-12-25, you will retrieve the same value regardless of the time zone of the clients or server.
Some databases have the TIMESTAMP WITH TIMEZONE data type which stores both the timestamp and the time zone separately.
Dates and timestamps are converted between Java and Database, usually in a manner that the subclasses of java.util.Date that are used are interpreted in the JDBC client's time zone.
Related
In our project (Spring Boot 2.2.3, MySQL 5.7, Hibernate, Java 14) we are having all date-related fields as datatype java.time.Instant. In our MySQL all fields are of type DATETIME.
When I specify a connectionString like jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=Europe/Paris for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T02:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
The display of the time in the table view in IntelliJ doesn't seem to effected by the serverTimeZone I set, so I hope it displays the plain value as stored in the DB.
When I change the connectionString to jdbc:mysql://localhost/mydb?characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=UTC for my JDBC connection and I have a value of 2020-07-13T00:00:00Z in my Entity, in the database 2020-07-13T00:00:00Z gets persisted (viewed via IntelliJ/DataGrip).
When I read it again with the JDBC connection I receive it correctly with 2020-07-13T00:00:00Z.
So it looks like I have an Instant, Java/MySQL assumes it's UTC and converts it to the timezone specified in the connectionstring and therefore adding the one hour during winter time/two hours during summer time for my timezone.
What I would like to understand is who performs these timezone adaptions and why. Because I understood from the MySQL documentation, that these adaptions should not happen for type DATETIME.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.) By default, the current time zone for each connection is the server's time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored.
MySQL 'DATETIME' stores sign, year, day, hour, minute, second, and fractional second. It is the equivalent of LocalDateTime.
Yet, you are writing epoch millis (in new time API terms: java.time.Instant) to it. Traditionally, datetime values in databases were presumed to be instant-based (as in, epoch-millis, not human YMDhms based), hence why java.sql.Timestamp extends java.util.Date (note that j.u.Date is a bald-faced lie. It represents epoch-millis, not date at all). The proof of that is also found in the API: All methods except those that fetch/set epoch-millis are deprecated in j.u.Date.
Note that the modern JDBC spec in fact requires support for the new java.time types such as LocalDate, and I can confirm that e.g. the JDBC drivers for postgres get this right.
In the past, the JDBC APIs would grow new methods - there'd be a .setLocalTime(idxOfQuestionMark, localTimeInstance) method in PreparedStatement, and a .getLocalTime(idxOrNameOfColumn) in ResultSet, for example. But, no longer. Any newly added types are to be used thusly:
LocalTime lt = resultSet.getObject(idxOrNameOfColumn, LocalTime.class);
preparedStatement.setObject(paramIndex, lt);
The first thing to try to do is to get the way mysql stores data, and the way java represents the data it got from mysql / sends to mysql, to line up. Because if the MySQL db has a YMDhms value and the only way your java code can observe this value is via an object whose inner storage allows it to represent only epoch-millis, well, guess what? Somebody somewhere is doing a timezone-based conversion because you can't go from epochmillis to YMDhms or vice versa without it. If you then convert right back you're just introducing opportunities for error.
However, it's mysql, and mysql is not a very good database, so odds are good that the above doesn't work (even though the JDBC spec more or less demands support for LocalTime, LocalDate, LocalDateTime, and ZonedDateTime, and LDT is an excellent match for what MySQL's DATETIME columns actually contain). So, if that doesn't work....
you're going to have to dance around it, and accept that conversion occurs. That will mean that what you're observing (connection timezone has an effect on what you read) will remain. One solution to that is to forget about DATETIME (after all, if indeed LDT instances can't be sent to/received from the JDBC MySQL driver), you have no way to reliably set or get such columns at all. Redesign your DB definitions to use TIMESTAMP WITH TIMEZONE instead (this is like ZonedDateTime, which is close enough to Instant (epoch-millis) that conversion shouldn't be an issue any more, though it's still suboptimal). Lock down the zone on both sides, and you can reliably convert from epoch-millis to that zone and back.
Or, better yet, find a better DB engine :)
I'm including timezone identifiers as strings, in data that's being stored in Google Cloud DataStore, and I want to get the current time offset from UTC (including any adjustment for daylight saving time), for that timezone identifier. For example I want to return something like -08:00, or +6:00
The identifiers I'm using are the (Supported Time Zone Values) (these appear to be timezone identifiers defined by Trifacta, which is a partner of Google that provides data prep capabilities) - the Google documentation appears to be a dupe of Trifacta's own documentation - https://docs.trifacta.com/display/SS/Supported+Time+Zone+Values.
The Google documentation says this in relation to the Supported Time Zone Values:
For the functions that support use of specified time zones, you can
apply the [Supported Time Zone Values] as parameters to specify the time
zone
So, I'm wondering if there might be functions (either in Java, or in GQL) that will accept a Supported Time Zone Value and return an offset, or allow it to be derived? For example I'm wondering if there's something available in Trifacta or GQL that's like the AT TIME ZONE in SQL Server 2016.
How can I return the current offset to UTC based on the Supported Time Zone Value?
The only GCP product that uses the Supported Time Zone Values afaik is Cloud Dataprep by Trifacta. The time zone values are taken by functions NOW and TODAY. In java, you can take a look at ZoneOffset class, but I'm not sure if you can incorporate all the identifiers form Trifacta.
if I understand correctly you are trying to modify most current dates at your timezone based on an offset. This is achievable with the combination of functions using NOW and DATADD in Trifacta.
Refer the article here to get more clarity on this https://community.trifacta.com/s/article/Function-NOW-DATEADD-and-DATEDIF
Best
Vardan
I have some Dates stored in Oracle with Oracle's TIMESTAMP(3) as its datatype. Now I'm writing a Spring boot app to read those values back. The code is like:
HibernateCallback callback = new HibernateCallback() {
public Object doInHibernate(Session session) throws HibernateException {
Query query = session.createSQLQuery("SELECT date_field FROM some_table WHERE some_conditions");
return query.list();
}
};
So:
List results = (List)getHibernateTemplate().execute(callback);
// suppose there's only one row and one column returned
Timestamp ts = result.get(0)[0];
returns me the Java Timestamp object automatically created by Hibernate. The problem is that, when constructing the object, it ignores the timezone stored in Oracle, but instead uses JVM's default timezone. I tested is by setting different timezones for the JVM, and each time it generates a different timestamp.
It's obviously wrong. The Date should be unique on the time line. It shouldn't depend on JVM's timezone. I'm wondering what's the correct way to include the DB's timezone info when parsing the date. Right now it seems it's just using the String representation stored in Oracle and parse it with JMV's timezone. I'm using Hibernate 4.3.4.Final.
PS: The actual query is high customized so I have to write raw SQL.
Basically, that's not even an issue with Hibernate but with JDBC. By default, JDBC Driver will use system time zone on which JVM is running. If you are connecting to the DB server at a different time zone or even if you want to be independent of the current timezone of the system it is a good idea to set JDBC Time Zone explicitly.
You can use hibernate.jdbc.time_zone property to set the timezone or do it at runtime via.
session = sessionFactory.withOptions()
.jdbcTimeZone(TimeZone.getTimeZone("UTC"))
.openSession();
Also for Oracle, I would say you can use the TIMESTAMP WITH LOCAL TIME ZONE which will respect you JDBC client time zone.
The problem is with Oracle column data type you are using, If you go to official oracle docs, link, you will notice the TIMESTAMP doesn't respect timezone, so you should go with with either TIMESTAMP WITH TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE.
It seems the issue is with few concepts of date handling
Dates in DB should be in UTC, if they are to be used in multiple timezone.
Java program should convert the dates to required timezone.
If you take your date in database as based on UTC, and then check the output of your hibernate query, you should see that date is changed as per JVM's timezone.
If you want the same date back (as was in DB), maybe you should convert the date to UTC timezone.
Now, you can do either of following:
Set the JVM's timezone same as the database record timezone. Not recommended
Change your dates in database as per UTC, and let the dates be changed to JVM's timezone. Recommended
I am unable to find a way to read timezone value in PostgreSQL column of type timestamp with time zone.
JDBC offers method java.sql.ResultSet#getTimestamp(int, java.util.Calendar)
but I must provide my own calendar. I have see no way to obtain that calendar from timestamp field I am reading.
I am working on system that stores time data of multiple timezones. I need to save data with timezone information, and be able to read that timezone back from database.
Is it possible without hacks like
storing timezone value in another field
storing date as string, like 'Wed 17 Dec 07:37:16 1997 PST'
I am using JDBC 41 (JDBC4 Postgresql Driver, Version 9.4-1201), java 8.
The PostgreSQL documentation here says that:
For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT).
So there is no need to "store the time zone [that corresponds to the timestamp value]" per se; the timestamp with time zone values are always stored as UTC.
Also, there is no need to "obtain the Calendar from the timestamp field". The purpose of the Calendar is for you to define the particular timezone that you want to work with in your Java application.
In other words, timestamp with timezone does not store values from various timezones (e.g., some in EST, others in PST, etc.), it converts everything to UTC when the timestamp values are inserted.
Accepted answer is true and accurate. timestamp with time type does not store timezone information in the field, and it is not possible to extract it.
If interested in timezone of the timestamp, it must be stored in separately (in other field, or in custom column type).
At first glance, it looks like that timezone may be extracted from timestamp with timezone using function extract(timezone from field), but it is not the case.
That function just gives 'time zone offset from UTC, measured in seconds'. Important (and not stated in documentation) part is that the offset is measured from current timezone (set by session SET SESSION TIME ZONE, or server timezone if not set). It is not offset that was used when saving field.
Which is the most beneficial in Java and a DB for DateTime? (Using JodaTime as Date)
(DateTime object (Java) + TIMESTAMP (DB) ) VS (Milliseconds long (Java) + BIGINT(DB)
for the use of DateTime information in Java Web application backed by an underlying Database
Areas of interest
manipulating, processing and memory usage in Java
saving using efficient storage space in
a MySQL database
ease of porting a BIGINT/TIMESTAMP column to other DBs
ease of searching the DB for a BIGINT/TIMESTAMP or between two BIGINTs/TIMESTAMPs
E.g. Say I had an event with a start & end DateTime.
Is it faster to search for events on dates using BIGINT in the DB than TIMESTAMPS
I might be swapping the underlying DB as scalability and retrieval issues arise.
Would saving the DateTime as a TIMESTAMP in a MySQL DB lead problems when porting to another DB like Oracle?
I currently use the Joda DateTime in java then storing the millisecond of that value.
When retrieving it, I convert the milliseconds back to a DateTime object and display it.
I’m always using the “milliseconds since 1970” approach. This way I don’t have to worry about which timezone the date belongs to because the date in the database is always UTC.
There are really two questions here. First, what abstraction should you use in Java to represent time? Joda Time is definitely better than java.util.Date. If you are wondering whether to simply use a long -- I imagine you can't get away with that if you need to do any date manipulation or comparison. So, Joda Time.
And then it is definitely best to use TIMESTAMP for this in MySQL as it will be nearly identical storage-wise and MySQL will treat the value appropriately, as a date, when you want to use date functions on the column. JDBC drivers will also understand that it should be mapped to a date type.
I can't imagine you will have trouble porting a date type, represented correctly as a date in your schema, to another database, should you need to. I can imagine problems if you treat the date type as a bigint, which is less correct.
So, simply choose the most correct types here. I doubt there is any performance win available from choosing a less suitable type anyway.
Of course it depends on what you want to do with the data, but I would recommend using the DB's native type for time/date (whatever that may be) for storage in the DB. That is the standard for databases, and most date/time functions in DBs expect data in that form.
Special note about MySQL:
While you can use TIMESTAMP for date/time values, be aware that TIMESTAMP cannot record dates earlier than 1970-01-01. So while it's ok for dates close to "now" (such as creation/modification dates), it is not appropriat for possibly historical dates such as date of birth. So only use TIMESTAMP if you are completely certain you will never need historical dates.