I have an Oracle table which contains the following column definition:
COLUMN_NAME : RESERVATIONDATE
DATA_TYPE : TIMESTAMP(6)
NULLABLE : Yes
DATA_DEFAULT: null
Then, from within Java I execute the following command:
insert into my_table (col1, col2, reservationdate) values (:np1, :np2, systimestamp)
Then elsewhere I perform the following command, the aim being to return rows added less than X seconds ago.
select * from my_table where reservationDate >= systimestamp - NUMTODSINTERVAL( :seconds, 'SECOND' )
But no row is returned despite being the reservation date being later than the point identified.
Therefore, I have also run the following command from the same application:
select col1,
col2,
reservationdate,
systimestamp as b,
systimestamp - NUMTODSINTERVAL( 5, 'SECOND' ) as c
from my_table
Which gives the following output:
col1: value1
col2: value2
reservationdate:2017-06-14 14:31:00.746173
b :2017-06-14 15:31:00.905617
c :2017-06-14 15:30:55.905617
Note that the values returned for b and c are basically one hour ahead of reservationdate.
Running the same request on SQL Developer running on the same machine as the Java application gives the correct values:
reservationdate:14-JUN-17 02.31.00.746173000 PM
b :14-JUN-17 02.58.32.863300000 PM +00:00
c :14-JUN-17 02.58.27.863300000 PM +00:00
Oracle is running on one virtual machine, where the output of Unix date is:
Wed Jun 14 14:18:11 UTC 2017
And on the machine where Java is running:
Wed Jun 14 15:21:24 BST 2017
Obviously this is a timezone problem, but I don't see exactly where it is coming from. I'm using systimestamp throughout after all, the aim being to do all timestamp calculations on the database server.
My requests pass through Spring's NamedParameterJdbcTemplate, and I'm using Oracle Database 11g Express Edition 11.2.0.2.0.
Not a fully qualified answer but here are some information:
Data type TIMESTAMP does not store any time zone information. When you insert timestamp with time zone (e.g. SYSTIMESTAMP) then time zone information is cut off.
SYSTIMESTAMP returns TIMESTAMP WITH TIME ZONE data type in time zone of database servers' operating system time zone (UTC in your case)
Whenever you convert/cast a TIMESTAMP (without time zone) to TIMESTAMP WITH TIME ZONE then Oracle takes SESSIONTIMEZONE into account - unless you explicitly set the time zone with FROM_TZ or similar.
Try TIMESTAMP WITH TIMEZONE or TIMESTAMP WITH LOCAL TIMEZONE insted of simple TIMEZONE...
I think you're seeing two things; when you do your debugging queries the way you are pulling, formatting and displaying the b and c values is causing Java to convert the time zone. But I think that's a red herring, as that won't directly affect your original query to find recently-changed rows. It may be related to your Java locale though, and so be related to the main query issue.
If you trace the query you were originally running, you should see that the filter it actually uses is:
SYS_EXTRACT_UTC(INTERNAL_FUNCTION(RESERVATIONDATE))
>=SYS_EXTRACT_UTC(SYSTIMESTAMP(6)-NUMTODSINTERVAL(TO_NUMBER(:SECONDS),'SECOND'))
The function calls are there because "During SELECT FROM operations, Oracle converts the data from the column to the type of the target variable."; so the column value is converted to a timestamp with timezone to be compared against systimestamp, and then to compare two timestamp with zone values they are both converted to UTC.
Then this goes back to what #WernfriedDomscheit said. The sys_extract_utc() function takes as its argument a datetime_with_timezone value, so when a plain timestamp is passed in it is implicitly being converted using the SESSIONTIMEZONE. You are seeing different results for your query because the session time zones are different between your Java code and SQL Developer.
You can select sessiontimezone from dual in both to see what those are, or see how timestamps are actually being converted.
You can avoid the problem by casting the system time back to a time-zone-free timestamp:
... where reservationDate >= cast(systimestamp as timestamp) - NUMTODSINTERVAL( :seconds, 'SECOND' );
Tracing that shows the simpler:
RESERVATIONDATE
>=CAST(SYSTIMESTAMP(6) AS timestamp)-NUMTODSINTERVAL(TO_NUMBER(:SECONDS),'SECOND')
Changing your column definition from timestamp to timestamp with [local] time zone would also work; with local the conversion to UTC still happens, without local it doesn't, according to my traces. This is an interesting read too.
If you look at how the sys_extract_utc() calls are resolved with different session time zones you can see the discrepancy, using a freshly-inserted row and omitting the date elements for brevity:
insert into my_table (col1, col2, reservationdate) values (:np1, :np2, systimestamp);
alter session set nls_timestamp_format = 'HH24:MI:SS.FF1';
alter session set nls_timestamp_tz_format = 'HH24:MI:SS.FF1 TZR';
alter session set time_zone = 'Europe/London';
select reservationdate a,
systimestamp b,
systimestamp(6)-numtodsinterval(to_number(:seconds),'SECOND') c,
cast(reservationdate as timestamp with time zone) d,
sys_extract_utc(reservationdate) e,
sys_extract_utc(systimestamp) f,
sys_extract_utc(systimestamp(6)-numtodsinterval(to_number(:seconds),'SECOND')) g
from my_table;
A B C D E F G
----------- ----------------- ----------------- ------------------------ ----------- ----------- -----------
17:12:13.9 17:12:15.0 +01:00 17:12:10.0 +01:00 17:12:13.9 EUROPE/LONDON 16:12:13.9 16:12:15.0 16:12:10.0
alter session set time_zone = 'UTC';
select reservationdate a,
systimestamp(6) b,
systimestamp(6)-numtodsinterval(to_number(:seconds),'SECOND') c,
cast(reservationdate as timestamp with time zone) d,
sys_extract_utc(reservationdate) e,
sys_extract_utc(systimestamp) f,
sys_extract_utc(systimestamp(6)-numtodsinterval(to_number(:seconds),'SECOND')) g
from my_table;
A B C D E F G
----------- ----------------- ----------------- -------------- ----------- ----------- -----------
17:12:13.9 17:12:15.4 +01:00 17:12:10.4 +01:00 17:12:13.9 UTC 17:12:13.9 16:12:15.4 16:12:10.4
My set-up seems to be different to yours - my database server is on UK time, but DBTIMEZONE is +00:00 - but even so, notice the difference between the two values in the e column. In my case it looks like it will find too much data rather than too little - as e >= g is true. In your case I believe that if you run those from SQL Developer and Java you'll see the discrepancy going the other way, because the session time zone from your Java application (based on its locale) is causing the shift the other way.
Related
I would like to map my postgresql query to jpql query in java. And I can't map "created AT time zone 'UTC' AT time zone 'Europe/Paris'" .
My postgresql query:
SELECT count(*), to_char(created AT time zone 'UTC' AT time zone 'Europe/Paris','DD-MM-YYYY')
from my_table GROUP BY to_char(created AT time zone 'UTC' AT time zone 'Europe/Paris','DD-MM-YYYY');
My jpql query:
Query query = em.createQuery("select count(z), to_char(z.created, 'DD-MM-YYYY'), z.formType from MyTableEntity z where z.created >= :startFrom and z.created <= :endTo GROUP BY to_char(z.created, 'DD-MM-YYYY'), z.formType ");
How can I group by my created field in timestamp 'Europe/Paris'? In database created field is saved in timestamp. AT time zone in jpql doesn't work.
You shouldn't use to_char to convert dates to a string at all. Let JPA handle the conversion. Also PostgreSQL can store a timestamp, but it does not store a time zone. It only stores the offset of the timezone at the given moment.
What you can do to get these results by date is casting to date, rather than using formatting. Here's how it would look as a native query:
SELECT count(*), cast(created AS date), form_type from my_table
GROUP BY cast(created AS date), form_type;
And in JPQL:
SELECT count(z), cast(z.created as date), z.formType from MyTableEntity z
where z.created between :startFrom and :endTo
GROUP BY cast(z.created as date), z.formType
The a between x and y statement is equivalent to a >= x and a <= y.
And if your database runs on a server who's default timezone is Europe/Paris, you don't need that timezone conversion. Alternatively you can run a native query to set the timezone for your current connection https://www.postgresql.org/docs/9.1/sql-set.html and then run your other queries.
SET SESSION TIME ZONE 'Europe/Paris'
Given this table:
CREATE TABLE a(
t TIMESTAMP WITH TIME ZONE
);
And this simple JDBC code snippet:
DriverManager.getConnection(
"jdbc:postgresql://localhost:5432/dbname", "user", "password"
).use { connection ->
val nowSomeTimeZone = OffsetDateTime.now(ZoneOffset.of("+4"))
connection.prepareStatement("insert into a(t) values (?)").use { insertStmt ->
insertStmt.setObject(1, nowSomeTimeZone)
insertStmt.executeUpdate()
}
connection.createStatement().use { stmt ->
stmt.executeQuery("select * from a").use { resultSet ->
resultSet.next()
val t = resultSet.getObject(1, OffsetDateTime::class.java)
println("$nowSomeTimeZone -> $t")
}
}
}
Somewhere inside the JDBC stack an automatic conversion from +04:00 to UTC must be happening, because this is the println output:
2018-08-30T10:35:33.594+04:00 -> 2018-08-30T06:35:33.594Z
What's even more weird, when I look into the table using the psql console client, it shows me the timestamp in yet another time zone (which is my local time zone):
$ psql -h localhost -U username
dbname=> select * from a;
t
----------------------------
2018-08-30 08:35:33.594+02
Why does this conversion happen, and how can I disable it?
Disabling the conversion is not possible, because the PostgreSQL server strips the time zone information and stores timestamps always in UTC, even when you are explicitly using the type TIMESTAMP WITH TIME ZONE.
Quoting the PostgreSQL documentation:
For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's TimeZone parameter, and is converted to UTC using the offset for the timezone zone.
Moreover, the documentation states:
We do not recommend using the type time with time zone (though it is supported by PostgreSQL for legacy applications and for compliance with the SQL standard). PostgreSQL assumes your local time zone for any type containing only date or time.
The weird behaviour described in the question is because
The JDBC driver always returns the timestamp in UTC
The psql console client converts the timestamp to the user's local time zone before displaying it, in this case German Time (+02:00)
Thanks to #RobbyCornelissen for the insights.
We have an audit table( Columns/Types: ID/Number,.. Audited_Date/Date) which logs audit entries using prepared statements. Until now, for different contexts we set the database session timezone for the connection, after which we were using the CURRENT_DATE attribute for the audited_date column. THIS MEANT THAT THE DATE INSERTED INTO THE COLUMN IS IN THE TIMEZONE OF THE CONNECTION WHICH IS IMPORTANT.
Now, we have a new requirement to add different dates based on the supplied timestamps for the audit logs. Similar to the previous approach where the auditing engine didn't have to worry about the timezone, is there a way to set the date for the column, WITHOUT having to do something like this:
TimeZone timeZone = TimeZone.getTimeZone(timezone);
calendar.setTimeZone(timeZone);
preparedStatement.setDate(4, new java.sql.Date(userTimestampMillis), calendar);
I would really like NOT to do this because the timezone attribute is decided based on multiple attributes like system environments, and other parameters. The application uses ALTER SESSION SET TIME_ZONE="CONTEXT_TIMEZONE" in a global scope of the application where connections are fetched from.
Is there any way to let the DB session handle the timezone conversion?
These two approaches fail to convert the the timestamp to the DB session timezone. If i'm not wrong, they are using the JVM timezone.
FAIL1.
Timestamp timestamp = new Timestamp(userTimestampMillis);
preparedStatement.setTimestamp(4, timestamp);
FAIL2.
preparedStatement.setObject(4, new java.sql.Date(userTimestampMillis));
Any documentation is greatly appreciated.
DATEs are not time zone aware, so you probably want to work with something in the TIMESTAMP WITH TIME ZONE data type. You say you can correctly set the database session time zone, so you're mostly there. Say you're in Los Angeles and my database session is in Chicago:
alter session set time_zone = 'America/Chicago';
Session altered.
select current_timestamp from dual;
CURRENT_TIMESTAMP
---------------------------------------------
2018-07-27 20:30:28.672000000 AMERICA/CHICAGO
select cast( current_timestamp at time zone 'America/Los_Angeles' as date ) as d from dual;
D
-------------------
2018-07-27 18:30:28
So you basically need to use AT TIME ZONE to convert the TIMESTAMP WITH TIME ZONE into the correct time zone, then use CAST ( ... AS DATE ) to turn that into a DATE, which basically just trucates off the time zone information and doesn't do any conversion.
I have created sample table in oracle DB as below
"CREATED_ON" TIMESTAMP (6),
"CREATED_ON_TIMEZONE" TIMESTAMP (6) WITH TIME ZONE,
"TIMEZONE_GMT" TIMESTAMP (6) WITH TIME ZONE
and inserted values from java as below
preparedStatement.setTimestamp(1, new Timestamp(new Date().getTime()));
preparedStatement.setTimestamp(2, new Timestamp(new Date().getTime()));
preparedStatement.setTimestamp(3, new Timestamp(new Date().getTime()) ,Calendar.getInstance(TimeZone.getTimeZone("UTC")));
JVM timezone in ASIA/CALCUTTA. I have used SQL developer to query data.
I just wanted to clear my understanding
The first column stored value as per local JVM without timezone since dataType is only timestamp i.e 29-NOV-17 07.04.28.014000000 PM. so for column with timstamp datatype DB stores value as of local JVM which is passed by JDBC driver and there is no conversion happening either JDBC side or DB side ?
Second column store value with TIMEZONE i.e 29-NOV-17 07.04.28.014000000 PM
ASIA/CALCUTTA. So does it mean DB stores value for column with timezone information provided by JDBC driver and there is no convrsion at DB side?
I want to store value in GMT so I set third parameter as GMT , it store value in GMT but timezone was still showing as of local JVM . i.e 29-NOV-17 01.34.28.014000000 PM ASIA/CALCUTTA
I was refering below article but my observations looks totally diffrent.
http://brian.pontarelli.com/2011/08/16/database-handling-for-timezones/
Problem is Java Timestamp does not contain any time zone information.
So you insert a TIMESTAMP value into a column of TIMESTAMP WITH TIME ZONE. In such case Oracle makes an implicit cast with FROM_TZ:
FROM_TZ(<your value>, SESSIONTIMEZONE)
Command preparedStatement.setTimestamp(3, new Timestamp(new Date().getTime()) ,Calendar.getInstance(TimeZone.getTimeZone("UTC"))); would be correct only after an ALTER SESSION SET TIME_ZONE = 'UTC';
I'm using Spring with apache commons BasicDataSource.
The time zone shows as GMT via:
SELECT ##global.time_zone, ##session.time_zone;
My input is in epoch time, 1386831420000 and 1386833220000, so the query should be like this:
SELECT * FROM table WHERE AND arrival_time BETWEEN '2013-12-12 06:57:00' AND '2013-12-12 07:27:00';
I enabled SQL profiing, and this is the query that actually gets executed, so I don't get the correct results:
SELECT * FROM table WHERE AND arrival_time BETWEEN '2013-12-12 01:57:00' AND '2013-12-12 02:27:00';
Notice that the times are off by 5 hours, since I am EST-5, and the time should be in GMT.
My question is: How can I tell MySQL or Spring JDBC not to use the client time zone, and simply to always use GMT?
Please comment if there is any detail I could add to solve the issue.
Try explicitly converting the date string into a date type using TO_DATE (or implementation specific date converter function)
SELECT *
FROM table
WHERE arrival_time BETWEEN
TO_DATE('2013-12-12 06:57:00', 'YYYY-MM-DD HH24:MI:SS')
AND
TO_DATE('2013-12-12 07:27:00', 'YYYY-MM-DD HH24:MI:SS')
The above example is in Oracle SQL but the principle of explicitly casting a date string to a date type is common throughout SQL implementations.
It sounds like the JDBC driver isn't properly detecting Mysql's time zone setting. You may need to specify the server's time zone in the connection string. Try adding
serverTimezone=UTC&useTimezone=true
to the connection string. For more information, refer to the JDBC doc at http://cs.wellesley.edu/~cs304/jdbc/connector-j.html#connector-j-reference