Part 1:
I have a file which contains date as string in the following format: 2006-02-16T21:36:32.000+0000
I need to write this value to Postgres in a column which is of type Timestamp. How do I go about doing this in Java?
Part 2:
I need to read the value saved in Part 1 from Postgres, and convert it to a string of the following format "2006-02-16T21:36:32.000+0000"
This is what I have tried:
Timestamp lastSyncDate = Timestamp.valueOf(2006-02-16T21:36:32.000+0000)
When I try to write it to postgres, it gives me the following error:
java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
tl;dr
myPreparedStatement.setObject(
… ,
OffsetDateTime.parse( "2006-02-16T21:36:32.000+0000" )
)
With and without zone/offset
You said your column is of type TIMESTAMP which is short for TIMESTAMP WITHOUT TIME ZONE. This type purposely lacks any concept of time zone or offset-from-UTC. Not usually appropriate for common business purposes.
You are using wrong type
That is the wrong type for your input. Your input has an offset-from-UTC of zero hours which means UTC itself. Having an offset or zone, your data should only be stored in a TIMESTAMP WITH TIME ZONE type column. In this …WITH… type in Postgres, any submitted offset/zone info is used to adjust into UTC for storage, and then discarded.
Storing a date-time value with an offset or zone in a column of type TIMESTAMP WITHOUT TIME ZONE is like storing a price/cost with a designated currency( USD, Euros, Rubles, etc.) in a column of numeric type. You are losing vital data, the currency. This renders your data worthless.
See the Postgres documentation page for these types.
Smart objects, not dumb strings
Whenever possible, use objects to exchange data with your database rather than passing meter strings. Let your JDBC driver do it’s job in marshaling the data back-and-forth.
Parse the input string as an OffsetDateTime object.
ISO 8601
Your input strings are in a format defined by the ISO 8601 standard. The java.time classes use ISO 8601 formats by default when parsing/generating strings. No need to specify a formatting pattern.
String input = "2006-02-16T21:36:32.000+0000" ;
OffsetDateTime odt = OffsetDateTime.parse( input ) ;
Define your SQL as a prepared statement.
With JDBC 4.2 and later, you can directly exchange java.time objects with your database. No need to ever again use the troublesome legacy classes such as java.sql.Timestamp.
You could pass an OffsetDateTime. I like to extract an Instant to demonstrate to the reader that I understand how Postgres always stores a TIMESTAMP WITH TIME ZONE value in UTC.
Instant instant = odt.toInstant() ;
myPreparedStatement.setObject( … , instant ) ;
Retrieval.
Instant instant = myResultSet.getObject( … , Instant.class ) ;
2006-02-16T21:36:32.000+0000 looks like a ISO-8601 timestamp.
postgres understands this format. just treat it like any other string.
don't try to convert it into a java timestamp
Related
I have a "Don't ask why just do it this way" question I hope someone can help me with.
I am working on updating a Java app and I cannot change certain classes. The problem is this:
I read a java.sql.Timestamp from an Oracle DB.
The Timestamp needs to be converted to the standard ISO_OFFSET_DATE_TIME as a String.
This String then needs to be parsed into a java.time.OffsetDateTime value.
I cannot get the Timestamp to be readable by any of the java.time classes.
Is there a way around this? How does a java.sql.Timestamp get imported into a java.time.OffsetDateTime?
tl;dr
myTimestamp
.toInstant()
.atOffset(
ZoneOffset.UTC
)
Details
I read a java.sql.Timestamp from an Oracle DB.
When handed an object from the terribly flawed legacy date-time classes, immediately convert to the modern java.time replacement class. That class would be Instant, for representing a moment as seen with an offset fromUTC of zero hours-minutes-seconds.
Conversion methods
To convert, call the new to…/from…/valueOf conversion methods added to the old classes.
Instant instant = myTimestamp.toInstant() ;
The Timestamp needs to be converted to the standard ISO_OFFSET_DATE_TIME as a String.
“converted” is the wrong word to use here. ISO_OFFSET_DATE_TIME refers to a constant holding a predefined formatter. A formatter up is used to parse text to get a date-time object, and is used to generate text representing the content of a date-time object. But a date-time object itself has no “format” as it does not consist of text.
Indeed, you have no need for text at all here in your scenario. As mentioned above, the old legacy class have been gifted with conversion methods.
This String then needs to be parsed into a java.time.OffsetDateTime value.
Apply an offset (ZoneOffset) to your Instant to get a OffsetDateTime.
I assume you want to stick with an offset of zero. So use the constant ZoneOffset.UTC.
OffsetDateTime odt = instant.atOffset( ZoneOffset.UTC ) ;
Avoid Timestamp
You can skip this conversion chore by extracting an OffsetDateTime object from your database. No need to ever use Timestamp again in JDBC 4.2+.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
And writing an OffsetDateTime.
myPreparedStatement.setObject( … , odt ) ;
I have a "Don't ask why just do it this way" question I hope someone can help me with.
Okay. But what you want is farcically crazy.
Timestamp is specifically equivalent to Instant, in that they both represent 'milliseconds since the epoch'.
This in sharp contrast to an OffsetDateTime which is different; that represents a combination of year, month, day, hour, minute, second, millis in the second, and a timezone offset.
Thus, the strategy to convert is not to 'silently' swap from one fundamental concept to another, only convert from different fundamentals with explicit calls. Thus, we first convert the Timestamp to Instant as that's still the same fundamental concept, and then we use the nice java.time API to explicitly swap on over to OffsetDateTime.
NB: Timestamp extends java.util.Date. Be aware that this class is a complete and utter lie - j.u.Date instances do not represent dates. This can, naturally, be confusing! Date is of the same fundamental concept as Instant, just the same: It's millis-since-epoch. Hence why all the methods in Date (such as .getYear()) are deprecated. This is asking a question to a fundamental that cannot properly answer it. Instant doesn't have a getYear either, because you can't answer that question with zone info which instants do not have.
on to the code!
// This should probably be a 'private static final' constant:
ZoneOffset DESIRED_OFFSET = ZoneOffset.ofHours(5);
// the actual code:
Instant i = yourSqlTimestamp.toInstant();
OffsetDateTime odt = i.atOffset(DESIRED_OFFSET);
String result = odt.format(FORMAT_PATTERN);
OffsetDateTime thisIsPointless = OffsetDateTime.parse(result, FORMAT_PATTERN);
As you might imagine, those last 2 lines are pointless.
I came upon a problem with java.time classes and their mapping to DB types. I want to store Instant type, but it behaves quite unintuitively.
Simple project example here: https://gitlab.com/Gobanit/jpa-time-example
I have basic entity class:
#Entity
#Getter
#Setter
public class AbstractEntity {
#Id
#GeneratedValue
private Long id;
// #Column(columnDefinition = "TIMESTAMP WITH TIME ZONE")
private Instant createdAt;
// #Column(columnDefinition = "TIMESTAMP WITH TIME ZONE")
private ZonedDateTime createdAtZoned;
private LocalDateTime createdAtLocal;
private Instant modifiedAt;
#PrePersist
protected void prePersist() {
createdAt = Instant.now();
modifiedAt = createdAt;
createdAtZoned = createdAt.atZone(ZoneId.systemDefault());
createdAtLocal = createdAtZoned.toLocalDateTime();
System.out.println(String.format("Created: instant=%s, zoned=%s, local=%s", createdAt, createdAtZoned, createdAtLocal));
}
#PreUpdate
protected void preUpdate() {
modifiedAt = Instant.now();
}
}
Now, I will create and persist the entity:
#Transactional
#GetMapping("create")
public AbstractEntity createNew() {
System.out.println("create");
System.out.println("ZoneId=" + ZoneId.systemDefault());
var e = new AbstractEntity();
em.persist(e);
return e;
}
The data in Java are as expected - system output below:
ZoneId=Europe/Bratislava
Created: instant=2021-12-29T14:13:55.624902400Z, zoned=2021-12-29T15:13:55.624902400+01:00[Europe/Bratislava], local=2021-12-29T15:13:55.624902400
However, the data in DB are not. They are all stored as the same value. The default type of the column is set to TIMESTAMP for all of them. And they are all stored as if they were LocalDateTime in application JVM timezone.
I dont like this at all, there is no difference between these types and the "meaning" of the value is dependent on the application zone. If I look at the database without knowing which application wrote the data and what is its zone, i cannot say, what the datetime means. Even worse, if I run the application with different zone, it would assume that the values were written in its timezone, which they were not.
Some people say, the time data should be stored in UTC (which i partially agree) and they recommend setting following property, so that the values are converted to UTC zone before storing in DB by JDBC:
spring.jpa.properties.hibernate.jdbc.time_zone: UTC
However, this results in even worse behavior, since it applies not only to Instant/ZonedDateTime but also to LocalDateTime, which seems logically wrong to me. LocalDateTime value should not be affected by any timezone conversions at all, that is why it is LocalDateTime.
System Output:
ZoneId=Europe/Bratislava
Created: instant=2021-12-29T14:15:13.614799900Z, zoned=2021-12-29T15:15:13.614799900+01:00[Europe/Bratislava], local=2021-12-29T15:15:13.614799900
DB Result:
I would expect one of these to happen:
Mapping Instant and ZonedDateTime to TIMESTAMP_WITH_TIMEZONE by default. Also, Instant type should be always serialized with UTC zone - just as when printing to string or serializing with jackson. (ideal)
Mapping Instant and ZonedDateTime to TIMESTAMP but convert them to UTC when serializing, and back to JVM timezone when deserializing. (not ideal, but good enough)
I can override the default mapping explicitly, by uncommenting the line below.
// #Column(columnDefinition = "TIMESTAMP WITH TIME ZONE")
That works fine, only Instant are not converted to UTC but take the JVM zone, just like ZonedDateTime. Technically it is totally fine, i cant see any inconsistency result happen because of that, but personally i would prefer them in UTC, so that they are always unified.
The second problem with this, is that i need to explicitly configure it for each field.
ZoneId=Europe/Bratislava
Created: instant=2021-12-29T14:19:14.600583400Z, zoned=2021-12-29T15:19:14.600583400+01:00[Europe/Bratislava], local=2021-12-29T15:19:14.600583400
In the end, I have few questions, I would really like someone to answer me :)
**
Is there any way to get the ideal solution (the first mentioned)?
Is there any way to at least globally configure the type mapping?
Can you see any risks in this approach?
Is there a better or more standardized approach, that I am not aware of? What is your strategy for working with time?
Thanks!
I cannot speak to JPA, but I can explain some of the JDBC issues.
I want to store Instant type
The JDBC specification does not map Instant.
For a column of a type akin to the SQL standard type TIMESTAMP WITHOUT TIME ZONE, use the Java class LocalDateTime.
For a column of a type akin to the SQL standard type TIMESTAMP WITH TIME ZONE, use the Java class OffsetDateTime.
In JDBC, use setObject & getObject to exchange data-time values as java.time values in JDBC 4.2 and later.
myPreparedStatement.setObject( … , myOffsetDateTime ) ;
Some databases such as Postgres automatically adjust the incoming data to UTC, an offset of zero hours-minutes-seconds. For portable code across databases, you may want to make that adjustment in Java code.
OffsetDateTime myOdt = otherOdt.withOffsetSameInstant( ZoneOffset.UTC ) ;
Capture the current moment as a OffsetDateTime.
OffsetDateTime myOdt = OffsetDateTime.now( ZoneOffset.UTC ) ;
If you have an Instant in hand, convert to OffsetDateTime for storage in a database via JDBC.
OffsetDateTime myOdt = myInstant.atOffset( ZoneOffset.UTC ) ;
There is no need for you to store the zoned or unzoned values in your database. You can easily adjust to time zones after database retrieval, in your Java code, as you would do for other localization needs.
ZoneId z = ZoneId.of( "America/Edmonton" ) ;
ZonedDateTime zdt = myOdt.atZoneSameInstant( z ) ;
I see no benefit generally to using LocalDateTime when dealing with moments, specific points on the timeline. That class purposely lacks the context of a time zone or offset. Such objects contain a date with time of day only, so are inherently ambiguous. You can convert from OffsetDateTime to LocalDateTime but would be stripping away vital information while gaining nothing in return.
they are all stored as if they were LocalDateTime in application JVM timezone.
That is a contradiction in terms. A LocalDateTime has no time zone or offset.
The default type of the column is set to TIMESTAMP for all of them.
There is no default for the data type of a column. You specify the data type when creating the column. The data type does not change while inserting rows. The data type can only change if you issue an ALTER TABLE command.
Also, we have no idea what type you mean with the word TIMESTAMP since you neglected to mention your specific database. As mentioned above, the SQL standard uses four words for each of the two very different timestamp types.
Mapping Instant and ZonedDateTime to TIMESTAMP but convert them to UTC when serializing,
No, JDBC does not require any such mapping of Instant nor ZonedDateTime.
You need to do such conversions in your Java code, outside of the JDBC calls.
The exception is that your particular JDBC driver may choose to go beyond what is required by the JDBC spec. A driver is free to handle the Instant and ZonedDateTime types if it’s creators saw fit. But beware that any such code you wrote may not be portable across other JDBC drivers.
and back to JVM timezone when deserializing.
I believe you’ll find this work much easier if, as a programmer/DBA/SysAdmin, you do most of your thinking, logging, debugging, data storage, and data exchange, in UTC (offset of zero).
Converting to Instant is a simple way of adjusting to UTC.
Instant instant = odt.toInstant() ;
Instant instant = zdt.toInstant() ;
Tip: Generally write your code to not depend on the default time zone of your JVM, your database session, or host OS. Those defaults are not under your control as a programmer. And those defaults can change at any moment during runtime. So use a default only where called for, such as localizing to default zone of a user’s mobile device for presentation in the user-interface. And make your use of such a default explicit, always specifying the otherwise optional zone/offset arguments to various calls.
Tip: Recording the moment a row is inserted or updated is generally best left to the database. Use a trigger where the database server captures the current moment and assigns it to the field. Then you are covered for operations outside your Java app, such as bulk data loads.
I have stored in a String type variable my_date a value retrieved from an XML file.
The my_date is in GMT and has a timezone offset that needs to be considered for the UTC conersion.
I would like to convert it (another String) but in UTC format without the timezone - for example:
String my_date = "2020-02-16T20:40:55.000+01:00"
//Convertion
String my_date_utc = "2020-02-16 21:40:55.000"
Parse as a OffsetDateTime object. Adjust the offset to UTC, producing a second OffsetDateTime object.
OffsetDateTime
.parse
(
"2020-02-16T20:40:55.000+01:00"
)
.withOffsetSameInstant
(
ZoneOffset.UTC
)
Understand that date-time objects are not String objects. They parse and generate strings as inputs and outputs.
Search Stack Overflow to learn about producing strings in various formats using DateTimeFormatter. That has been covered many hundreds of times already.
So Our database has a column BigInt REVISIONTS used as part of java Hibernate Envers
It initially contained timestamps from Java Date.
E.g) ts=1561637560383
I used to convert to Date using new Date(ts)
But since Date cannot contain timezones and we needed UTC date, we had to store UTC directly as BigInt and applied a fix suggested by hibernate. Because of this now our timestamps are like this
E.g) ts=20190827202449 now this is not a timestamp anymore but an actual UTC LocaleDateTime stored as bigint
Now querying this i get long and if I use new Date(ts) i am getting incorrect date of course since this is not a timestamp but Hibernate Date with Temporal.Timestamp stored the UTC as is.
I am thinking of converting Long to string and use formatting to convert back when retrieving.
Are there any other cleaner method of converting ?
UTC fix for Envers
How to save UTC (instead of local) timestamps for Hibernate Envers revision info?
You shouldn't be using Date at all. Never. Ever.
I think storing the time as a Unix Timestamp is pretty fine. They're always in UTC and represent a unique instant on the timeline.
Envers supports both Date and Long/long to be defined as revision timestamp. You should use Long.
Formatting it using a timezone or timezone offset can be easily done with the newer Java Date and Time API available in the java.time package.
With Instant.ofEpochSecond(yourTimestamp) you can create an Instant. With atOffset or atZone you can combine the bare timestamp with a certain timezone or timezone offset.
Your Question is quite unclear. But this might help.
Avoid legacy date-time classes
to convert to Date using new Date(ts)
Never use java.util.Date. That terrible class was supplanted years ago by the java.time classes, specifically by Instant.
Instant
E.g) ts=1561637560383
You are not clear about exactly what that value represents. I will guess it is a count of milliseconds since the epoch reference of first moment of 1970 in UTC, 1970-01-01T00:00Z.
long count = 1_561_637_560_383L ;
If that is a textual value, parse using Long class.
long count = Long.parseLong( "1561637560383" ) ;
Instant instant = Instant.ofEpochMilli( count ) ;
See this code run live at IdeOne.com.
instant.toString(): 2019-06-27T12:12:40.383Z
Tip: In your database, store date-time values using date-time data type.
If your database is too primitive to support date-time types, store as text in UTC using ISO 8601 format.
String output = instant.toString() ; // Ex: 2019-06-27T12:12:40.383Z
…and…
Instant instant = Instant.parse( "2019-06-27T12:12:40.383Z" ) ;
Get count of milliseconds since epoch reference.
long count = instant.toEpochMilli() ;
Convert
When you must use Date to interoperate with old code not yet updated to java.time, convert. Call new to…/from… methods added to the old classes.
java.util.Date d = Date.from( instant ) ;
Instant instant = d.toInstant() ;
I have the following code. I have stored a utc time in my mySQL db and would like to convert it to local time when I pull it from the database.
Instant startInstant = rs.getTimestamp(5).toInstant();
Instant endInstant = rs.getTimestamp(6).toInstant();
ZoneId z = ZoneId.of(TimeZone.getDefault().getID());
ZonedDateTime start = startInstant.atZone(z);
ZonedDateTime end = startInstant.atZone(z);
System.out.println(start.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss VV")));
System.out.println(end.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss VV")));
This is the output I get:
2017-11-01 01:40:00 America/Chicago
2017-11-01 01:40:00 America/Chicago
The date and time are the same as in my db: This is the field value in the table:
2017-11-01 01:40:00
The only thing that seems to have updated is just the zoneID to America/Chicago which is functionally useless for what I'm needing.
Any pointers as to where I've gone wrong?
You have not supplied enough information to give an exact Answer. You did not explain exactly the data type of the column around your date-time value. I suspect you did not use the MySQL equivalent to the SQL standard type of TIMESTAMP WITH TIME ZONE.
I also wonder why your example of a stored value lacks any offset-from-UTC or time zone. That is suspicious.
Lastly, you may be looking st your stored value by using an interactive SQL session with a default time zone bring dynamically applied to the value retrieved from the database before displaying in your session. While well-intentioned, I consider this an anti-feature as it masks the true nature of the stored data. This behavior may explain your confusion. Try explicitly setting your session’s default time zone to UTC.
Other issues with your code…
Skip the use of java.sql.Timestamp altogether. If your JDBC driver supports JDBC 4.2 or later, you can work directly with the java.time types.
Instant instant = myResultSet.getObject( … , Instant.class ) ;
Do not mix an old legacy class like TimeZone with the modern java.time classes. The legacy classes are completely supplanted by java.time; good riddance.
ZoneId z = ZoneId.systemDefault() ;
You are getting the same value that you have entered in a database.
You have timestamp values in database, but how you can say that the values stored
are in UTC (the values might be in UTC but not in UTC format, so by looking into the value, you can't say that the value is in UTC -> '2017-11-01 01:40:00').
If you want to fetch the value in the different time zone, you can also do like this :
eg:
SET time_zone ='+03:00';
SELECT value
FROM table;
It will be providing you the value in specified time zone.