We Keep Coding

Wrong date selection in SQL

I'm working with Hibernate 4.3.8.Final, Primefaces 6.0 and MySQL Database 5.7.13.
I have a table in the database with this structure:
CREATE TABLE `rents` (
`rent_code` INT NOT NULL AUTO_INCREMENT,
`rent_daystart` datetime default NULL,
`rent_dayend` datetime default NULL,
PRIMARY KEY (`rent_code`)
) ENGINE = innodb CHARACTER SET utf8 COLLATE utf8_unicode_ci;
And the following data extracted with Squirrel with the following SQL:
select * from rents
rent_code | rent_daystart | rent_dayend
1 | 2016-11-30 16:03:00.0 | 2016-12-01 16:03:00.0
In my Java bean I have the following function:
public List<Object> getRents(java.util.Date iniDate, java.util.Date endDate){
String SQL="select rent_code from rents where rent_daystart < :inidate and rent_dayend > :enddate";
List<Object> allRecords = null;
Session sesion=HibernateUtil.getSessionFactory().openSession();
try {
sesion.beginTransaction();
Query query = sesion.createSQLQuery(SQL).setDate("inidate", iniDate).setDate("enddate", endDate);
allRecords = query.list();
sesion.getTransaction().commit();
sesion.close();
}
catch (HibernateException he) {
//exception control code
};
return allRecords;
}
I execute the web APP debugging and the dates that the function receives are:
**inidate** = 'Wed Nov 30 17:54:00 CET 2016'
**enddate** = 'Wed Nov 30 18:54:00 CET 2016'
And it returns NO RECORD AT ALL.
If I execute the same SQL in squirrel that way:
select rent_code from rents where rent_daystart < '2016-11-30 17:54:00' and rent_dayend > '2016-11-30 18:54:00'
It returns one record.
I suspect that this is a data type problem or something like that, but after researching in the web it is not clear to me.
May someone help me?
Thanks in advance!

The java.util.Date class doesn't handle time zones. Unless you are using the same time zone in both your Date instance and the database fields, you can easily mismatch the time zones creating the behavior that you are seeing. Try and display the java Date in UTC format, and look at your database date in UTC format, and see if they line up like you expect.

If it is a date format mismatch between the values passed from the code and the values expected in database then use SimpleDateFormat to change the format of the date values before passing on to databse
SimpleDateFormat dt = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss");
Date date = dt.parse(iniDate);

Well, I have found the problem. As stated in Compare Date Java to DateTime Column in database table using SQL I was using the ".setDate" function to set the dates, such function truncates the time part, that was the problem. Now I'm using the ".setTimestamp" that uses both, date and time parts.
Thanks to all for your help, I have learned a lot about TimeZones!

Retrieving UTC DATETIME field from MySQL in Java when server timezone is not UTC

I'm trying to write code to interoperate with a third-party-developed database using Java and MySQL. This database has a field that stores a time stamp in a DATETIME field as a UTC date. The timezone for the server on which both the database and client run is set to a non-UTC zone (Europe/London), so by default the timestamp is read back incorrectly as if it were a local time. I'm trying to write code to read it back as UTC.
I have read several similar questions here, but none of them have an answer that works for me:
MySQL - how to store time with correct timezone? (from Java)
How to store a java.util.Date into a MySQL timestamp field in the UTC/GMT timezone?
Date in UTC in mysql
How do I set the time zone of MySQL?
Unfortunately, I cannot change any server settings, so I have tried using the connection's "time_zone" variable to set the database server to use UTC and the optional Calendar parameter to ResultSet.getTimestamp to retrieve the date, but this has no effect on the result. Here is my code:
private static final Calendar UTCCALENDAR = Calendar.getInstance (TimeZone.getTimeZone (ZoneOffset.UTC));
public Date getDate ()
{
try (Connection c = dataSource.getConnection ();
PreparedStatement s = c
.prepareStatement ("select datefield from dbmail_datefield where physmessage_id=?"))
{
fixTimeZone (c);
s.setLong (1, getPhysId ());
try (ResultSet rs = s.executeQuery ())
{
if (!rs.next ()) return null;
return new Date (rs.getTimestamp(1,UTCCALENDAR).getTime ()); // do not use SQL timestamp object, as it fucks up comparisons!
}
}
catch (SQLException e)
{
throw new MailAccessException ("Error accessing dbmail database", e);
}
}
private void fixTimeZone (Connection c)
{
try (Statement s = c.createStatement ())
{
s.executeUpdate ("set time_zone='+00:00'");
}
catch (SQLException e)
{
throw new MailAccessException ("Unable to set SQL connection time zone to UTC", e);
}
}
The database field I'm trying to read has a value stored in it as follows:
mysql> select * from dbmail_datefield where physmessage_id=494539;
+----------------+--------+---------------------+
| physmessage_id | id | datefield |
+----------------+--------+---------------------+
| 494539 | 494520 | 2015-04-16 10:30:30 |
+----------------+--------+---------------------+
But unfortunately, the result comes out as BST not UTC:
java.lang.AssertionError: expected:<Thu Apr 16 11:30:30 BST 2015> but was:<Thu Apr 16 10:30:30 BST 2015>

Your client getDate() code looks correct as far as it goes. I think you also need to get the MySQL Connector/J JDBC driver to treat the dates stored in the table as UTC dates, to avoid a spurious time zone conversion. This means setting the effective server time zone, in addition to the client session time zone and Calendar used for JDBC getTimestamp calls as you're doing.
Take a look at the values you got in your failed assertion, and which direction the error is in:
expected:<Thu Apr 16 11:30:30 BST 2015> but was:<Thu Apr 16 10:30:30 BST 2015>
What you got back was 10:30 BST, which is 9:30 GMT. This is consistent with the database treating that 10:30 in the table as a BST value and spuriously converting it to GMT for you, before you parse it as a GMT date. That's the opposite direction of a GMT value being spuriously converted to BST.
This may be a JDBC-specific issue, because JDBC requires that time times be converted to the local zone. (Where the MySQL C API doesn't, probably because C's classic time types are not zone-aware the way Java's are.) And it needs to know what zone it's converting from, as well. The MySQL TIMESTAMP type is always stored as UTC. But that's not stated for the DATETIME type. I think that implies that MySQL is going to interpret DATETIME column values as being in the server's time zone. Which you mentioned as being set to BST, and that's consistent with the direction of the shift shown in your assertion error message.
The time_zone session variable you set is telling the MySQL server what your client machine's time zone is, but it doesn't affect what the server thinks its own time zone is. That can be overridden with the serverTimezone JDBC connection property. On your connection, set the serverTimezone to UTC, and make sure useLegacyDatetimeCode is off. (And look through the other zone-related properties if that doesn't work.) See if that gets your dates to come through as UTC with the same calendar field values as in the database.
Be aware that this is going to change the interpretation of other DATETIME values in your database: they're all going to look like UTC dates now (in the context of your JDBC connection). Whether that's correct is going to depend on how they were populated initially. While your client code will have the behavior you want, I don't know if this system as a whole can be made to behave fully consistently without setting the server's time zone to UTC at the server level. Basically, if it doesn't have its zone set to UTC, it's not fully configured for the behavior you want, and you're kludging around it.

Maybe you can use JodaTime as follows;
private static final Calendar UTCCALENDAR = Calendar.getInstance (TimeZone.getTimeZone (ZoneOffset .UTC));
public Date getDate ()
{
try (Connection c = dataSource.getConnection ();
PreparedStatement s = c
.prepareStatement ("select datefield from dbmail_datefield where physmessage_id=?"))
{
s.setLong (1, getPhysId ());
try (ResultSet rs = s.executeQuery ())
{
if (!rs.next ()) return null;
DateTime dt = new LocalDateTime(rs.getTimestamp(1,UTCCALENDAR).getTime ()).toDateTime(DateTimeZone.forID("Europe/London"));
return dt.toDate(); }
}
catch (SQLException e)
{
throw new MailAccessException ("Error accessing dbmail database", e);
}
}
EDIT:
java.util.Date is not TimeZone agnostic. The method toDateTime takes care of TimeZone and DST so you don't care about it
The following code:
public static void main(String[] args) {
// 29/March/2015 1:05 UTC
DateTime now = new DateTime(2015, 3,29,1,5,DateTimeZone.UTC);
// Pre DST 29/March/2015 0:30 UTC
DateTime preDst = new DateTime(2015, 3,29,0,30,DateTimeZone.UTC);
System.out.println("1:05 UTC:"+now);
System.out.println("0:30 UTC:"+preDst);
DateTimeZone europeDTZ = DateTimeZone.forID("Europe/London");
DateTime europeLondon = now.toDateTime(europeDTZ);
System.out.println("1:05 UTC as Europe/London:"+europeLondon);
DateTime europeLondonPreDst = preDst.toDateTime(europeDTZ);
System.out.println("0:30 UTC as Europe/London:"+europeLondonPreDst);
}
Will print:
1:05 UTC:2015-03-29T01:05:00.000Z
0:30 UTC:2015-03-29T00:30:00.000Z
1:05 UTC as Europe/London:2015-03-29T02:05:00.000+01:00
0:30 UTC as Europe/London:2015-03-29T00:30:00.000Z
If you can see JodaTime takes care of DST.

Your best bet, in my view, is to tell MySQL to use GMT and handle all local time issues in your application code, not your database. The values in the database would always be GMT, full stop, which is unambiguous. As you say, with daylight savings time (summer time) adjustments, you can end up with the same value in your database for what is, to us humans, two different times.
This also makes the database portable. If you move to North America and start using MySQL set to (say) Central time, all of a sudden the values in your database seem to have moved several hours. I had that issue with a database I inherited which was using the server's local time, when I moved it from the east coast of the U.S. to the west coast, not having thought to check whether MySQL was slaved to the machine's zone...
long t = 1351382400000; // the timestamp in UTC
String insert = "INSERT INTO my_table (timestamp) VALUES (?)";
PreparedStatement stmt = db.prepareStatement(insert);
java.sql.Timestamp date = new Timestamp(t);
stmt.setTimestamp(1, date);
stmt.executeUpdate();
.....
TimeZone timezone = TimeZone.getTimeZone("MyTimeZoneId");
Calendar cal = java.util.Calendar.getInstance(timezone);
String select = "SELECT timestamp FROM my_table";
// some code omitted....
ResultSet rs = stmt.executeQuery();
while (rs.next()) {
java.sql.Timestamp ts = rs.getTimestamp(1);
cal.setTimeInMillis(ts.getTime());
System.out.println("date in db: " + cal.getTime());
}

If you want to use timezone you can read column as UTC.
ZonedDateTime zdt = ZonedDateTime.of(rs.getTimestamp(1).toLocalDateTime(), ZoneOffset.UTC);
Next you can change to whatever timezone you want using:
zdt = zdt.withZoneSameInstant(ZoneId.of(
TARGET_ZONE));
If you want only to read Date and do not care about zones at all use only:
LocalDateTime ldt = rs.getTimestamp(1).toLocalDateTime()
You will obtain LocalDateTime without timezone.
If you have to return java.util.Date use:
Date.from(ldt.atZone(ZoneOffset.UTC).toInstant());

Don't think about converting or adapting time zone. Don't think about the TZ the mysql uses to store your timestamps or anythink like that. Those things are already handled.
There are three things that you must handle: INPUT, OUTPUT and bugs.
INPUT
When a user enters a date (in a form) without an explicit time zone you have to know what TZ did he intend to use. You can use a SimpleDateFormat object with time zone set to solve this. You don't have to convert the input date, you have to 'interpret' it correctly. Once you have a correctly interpreted Date or timestamp you are done with input.
Input is not only user input, includes configuration files too.
OUTPUT
The same here. Forget about what TZ have your Date objects and timestamps have none, they are just milliseconds since epoch. You have to format your dates to the TZ the user expects so he understand them.
Bugs
You may have bugs in your code related to TZ, but libraries may have them too!!
I noticed mysql java driver failed to communicate the client timezone to the server.
This command s.executeUpdate ("set time_zone='+xx:yy'"); is the workaround but you are using it wrong. You have to tell the server with it the TZ the client is using, before both inserting and querying. The variable is stored in the session. Maybe you may automatize it on your connection pool config.
This is needed so the server know what TZ the client need to use to read or write. This is not dependent on server TZ. It does not mean "store this date in UTC", it does mean "this date I am giving to you is UTC" and "Send me result sets in UTC". No matter you are using Date class with it's internal TZ, the driver screws it up, you would need to set that session variable.
By default it assumes client TZ is the same as server TZ so you shouldn't need to worry about it as you said they are the same.

SQL insert date into database

I use java.util.Date to get the date and I get the date format like this
Sat Jun 29 11:07:25 CDT 2013
When I try to insert it into the database using
String QueryString = "INSERT INTO db (day) Values ('"+d+"');";
st.executeUpdate(QueryString);
I get this
"Conversion failed when converting date and/or time from character
string."
How can I insert that type of date into the db?
Should I declare it as a string?

how can i insert that type of date into the db? should i declare it as a string?
No - you should avoid even the string conversion you currently have. You shouldn't build your SQL dynamically like that - it's a recipe for SQL injection attacks, hard-to-read code, and conversion failures.
Instead, use a PreparedStatement and set the parameter using setDate:
// TODO: Closing the statement cleanly in a finally block or try-with-resources
PreparedStatement pst = conn.prepareStatement("INSERT INTO db (day) Values (?)");
pst.setDate(1, new java.sql.Date(d.getTime()));
pst.executeUpdate();
Note that java.sql.Date is a subclass of java.util.Date, but they're somewhat different. It's never been clear to me which time zone is used to convert the given instant in time into a real date - and the documentation is less than helpful. It's broken by design in my view, but that's a different matter. You can use another setDate overload which accepts a Calendar - which is used for the time zone. It's still all horribly unclear, but hopefully you can get the result you want.

Use java.sql.Date to be compatible with the databse

Alternatively, use an updatable ResultSet:
ResultSet rs = conn.createStatement("select day from db", ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_UPDATABLE);
rs.moveToInsertRow();
rs.updateDate(1, yourJavaSqlDateObject);
rs.insertRow();
rs.first();
Though, depending on your driver/database, there may be a write-lock on the table.
Date formatting should be done on output, not input. Let the database decide how best to store the date.

You could use SimpleDateFormat to format your Date object
Date d = Calendar.getInstance().getTime(); //Your date
String dateString = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(d);
String QueryString = "INSERT INTO db (day) Values ('"+dateString+"');";
st.executeUpdate(QueryString);

Default JDBC date format when reading date as a string from ResultSet

I'm looking at some code that basically does the following:
ResultSet rs = ps.executeQuery();
String myDateStr = rs.getString("MY_DATE"); //field is of type Date
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss'.0'");
Date myDate = sdf.parse(myDateStr);
On some environments the last line works, and on others it throws an Unparseable date exception. It looks like on some systems the default date format is 2013-01-25 00:00:00.0, and on others 2013-01-25 00:00:00.
The JVM, OS and Oracle version are different between the environments (all use Oracle and run on a unix variant though).
Changing the code might be complex. I'm wondering if there is an environment variable or similar that can be set to make the date format returned from rs.getString() consistent?

try this:
ResultSet rs = ps.executeQuery();
Date myDate = rs.getDate("MY_DATE");
or this :
ResultSet rs = ps.executeQuery();
String myDateStr = rs.getString("MY_DATE");
Date myDate = valueOf(myDateStr);
More about date: http://docs.oracle.com/javase/7/docs/api/java/sql/Date.html
More about ResultSet : http://docs.oracle.com/javase/7/docs/api/java/sql/ResultSet.html

Instead of using
String myDateStr = rs.getString("MY_DATE")
you should use
Timestamp timestamp = rs.getTimestamp("MY_DATE");
JDBC / Database will handle the date transformation for you.

If the field is of type Date, then read it as a java.sql.Date and do whatever conversion you need after that. Otherwise you're at the mercy of the database implementation.

For the Oracle JDBC driver I am using, the format is hard-coded in the driver library, so should only differ across systems if different driver versions are in use.
See my attempt to get an answer to the same question here: Where is the date format specified when reading a date as a string from JDBC ResultSet.
(Apologies for asking a separate question, but as your question had been answered multiple times with the ever-helpful "just don't do that" response, I tried again...).

java.sql.timestamp versus date

I am using Derby database with Java and Eclipse. I have a table which has a TIMESTAMP field and I use a model to populate a JTable from it. I am finding timestamp and data and java.sql and java.utils very confusing. The following line of code errors with cannot cast date to timestamp. mod is the model interfacing Derby table and JTable.
int rowcount = mod.getRowCount();
java.sql.Timestamp ts = (java.sql.Timestamp) mod.getValueAt(rowcount-1,1);
My objective is to get the date of the most recent record and subtract 30 days then run an sql query on the same database to find all the records more recent than that date. How do I recover the first timestamp, subtract the 30 days, then construct a query with the result of the subtraction as the condition in a WHERE clause. Sounds simple but I am having such difficulty that I feel I must be missing some fundamental principal. I thought conversion to long and back again might be the route but came up against the same cast problem.

Timestamp is declared as
public class Timestamp extends java.util.Date { ... }
Therefore you can't cast date to timstamp, you could create a timestamp from a date.
Timstamp ts = new Timestamp( date.getTime() );
To subtract 30 days this sequence might be helpful:
Calendar cal = new GregorianCalendar();
cal.setTime( date.getTime() );
cal.add( Calendar.DAY_OF_MONTH, -30 );
Date d30 = cal.getTime();
Anyway I would try to perform this using only SQL.