I'm writing a app that exposes a REST API. Some of the query parameters will be date/time (accurate to second), and some of the responses will be timestamps (accurate to millisecond).
The API implementation on the server is in Java. The client apps can be anything - java, javascript, .NET. The API returns XML or JSON data. Date/Time data is stored in a Oracle database.
Does anyone have recommendations, based on prior pain, of what the best format format is for passing these date/time values. I'm thinking myself to just use a good old fashioned long to store the number of milliseconds since January 1, 1970, 00:00:00 GMT.
Edit The date range covered in the API is for real time events, so there will be nothing before 2010, and (setting myself up for abuse here) nothing after 2038.
I guess best would be determined by
a) Wide variety of languages support converting this long into internal date object, without having to write code to do it.
b) Lowest CPU overhead (on server app)
ISO 8601 all the way
Using any epoch-based method means you are bound to the range (in most systems) of a signed 32-bit INT (1901-12-13T20:45:52+00:00 through 2038-01-19T03:14:07+00:00) which, is really more of a timestamp than a date, since it can't handle far-reaching historical or future dates.
Related
I'm having trouble deciding what data type to use to store dates. I think using the Date type will make it easier for the code to be processed by the backend because there are many functions that can be used. But I'm having trouble when it's used for the response API. Each property of data type Date will return a value in the form of a timestamp. Of course this will require effort for the frontend developer to convert it to the actual date.
I've tried several Rest APIs from well-known vendors out there and then I found they use the String data type to process their Rest API requests/responses. Is using the String data type proper to use on date and time?
REST doesn't have a recommended date format, you should select what works best for your end-user and your system.
Generally, people prefer to use ISO 8601 standard for the date-time values in text. The java.time classes use ISO 8601 formats by default when parsing/generating text.
RFC 3339 Date and Time on the Internet is the document to look at that says:
date and time format for use in Internet protocols that is a profile
of the ISO 8601 standard for representation of dates and times using
the Gregorian calendar.
In your case, you are using epoch for a date which is totally fine and easy to convert to an actual human-readable date using java libraries.
A simple solution would be:
LocalDate ld = Instant.ofEpochMilli(epoch)
.atZone(ZoneId.systemDefault()).toLocalDate();
It all boils down to arbitrary personal preference. I would personally use string since it is easier to read before sending, in transit, and upon receipt.
Quick summary of my issue first, then details further below.
I have a Calendar date with 00:00:00 as the time as it's not important to the business. This value is sent to a webservice which generates a XML in which the value ends up in the following format : 2014-09-12T07:55:07.000Z. I have noticed that this is the original value converted to a +0 timezone (UTC) (ours is CET, +1, but currently CEST, +2, because of DST).
I currently have no idea whether the system reading the XML takes timezones into account or would extract 2014-09-12 and assume it's in the +2 timezone.
What I've noticed is that sending "2014-09-12 00:00:00" local time (tz +2) ends up as 2014-09-11T22:00:00.000Z in the XML. No big surprise, it converted it... but if it's interpreted as is by the other system, it will think the date is a day earlier than it should be.
What can be done to ensure this gets interpreted as intended?
I was thinking of using noon instead of midnight to make sure timezone shifts wouldn't impact interpretation, but it feels like a dirty trick. Or maybe I should cheat and have the Calendar timezone be +0 so it's not time-shifted when put in the XML?
Q&A
Why do you "send a Calendar to a webservice"?
The application is in Coldfusion 7. To communicate with SOAP webservices, the server generates a set of Java classes that fit the definition of the argument expected by the webservice. The argument is apparently a single big object with a lot of attributes and sub-attributes. So one instantiates the main Java class and uses setters and further instanciations of other classes to "fill out" all the attributes.
Do you have to use Calendar?
Yes, the Java object definition cannot be changed. It expects Calendar for all dates.
What's this 2014-09-11T22:00:00.000Z format?
I have no idea. This seems to be what the end system expects for dates.
You should use JODA
Unless JODA classes extend Calendar and are compatible with Java 1.3 (current Java version on the Coldfusion server -- yes it's old), I doubt it will work.
How much can you do on the other system?
The other system is the responsibility of a different team and is apparently very hard to change. I expect the solution will have to be found on the side of our application.
Although the time value in your Calendar object is not important to your business, it is important to the webservice that you use and have no control over. The calendar object specifies an instant in time, so you must make sure that instant is in the day that is important to you. I recommend you use midday, as you suggested already. I also recommend that you create your Calendar object in the UTC timezone:
Calendar myCalendar=Calendar.getInstance(TimeZone.getTimeZone("UTC"));
I am using Joda-Time and I have noticed that DateTime is stored as a java.util.Date in the Google App Engine Datastore for Java, while LocalDateTime is stored as a ISO 8601 compliant String.
http://code.google.com/p/objectify-appengine/source/browse/src/com/googlecode/objectify/impl/translate/opt/joda/?r=2d48a85eae3a679c0dc0d01631de99f3b4775b29
I know that java.util.Date is a native type of the Datastore.
Is there any particular advantages is storing date/times as a java.util.Date as compared to a ISO 8601 compliant String or is it all the same. When I say advantage I might consider differences in regards to ...
Inequality queries
Storage size
Read/write cost
etc.
The accepted answer is not wrong, but I would like to add extra details which give a more balanced view.
a) Stable queries: ISO-8601 is stable as long as you assert that
you only use one date format for storage (ISO defines three: calendar date, ordinal date and week date)
and that you always use one precision degree for the time part (for example always in milliseconds)
and that you always use UTC with respect to global timestamps (that is zero offset with symbol Z).
Confirming this kind of stability can be application-dependent while java.util.Date does not require the same care.
b) Precision: ISO-8601 can express more precision beyond milliseconds while java.util.Date and Joda-Time are limited here. This is particularly true if you might later think of other new time libraries like JSR-310 in Java 8 or my own one which provide nanosecond precision. Then you will have precision issues with all JDBC types, java.util.Date and the database columns as far as they are not CHAR or VARCHAR.
A striking example is the JDBC-type java.sql.Time whose precision is limited to seconds, not better. This is pretty much in contrast to the new Java8-type java.time.LocalTime which offers nanoseconds. And worse: This aspect is also relevant for Joda-Time and java.util.Date in your application layer.
For rather academic purposes: Leapseconds can only be stored in ISO-8601-format, not with java.util.Date or similar.
c) Storage size: Of course, java.util.Date has a more compact representation, but else I have to say that disk space is cheap nowadays, so this is not an item to worry about so much.
d) Read-Write-costs: This item is in favor of compact data types like java.util.Date. But you have also to consider that even in this case you have to represent it in a human-readable format in any other layer sooner or later (most in logging or in representation layer). So for data exchange with other proprietary applications which expect java.util.Date this native type is okay, but for logging purposes or XML-data exchange ISO-8601 is probably the better format.
And if you really care so much about performance costs, you might even consider a number type (long - 64 bit) to avoid garbage burdens by unnecessary object creation (in extreme edge cases). Remember: java.util.Date is just a wrapper around a long.
The advantage of java.util.Date: Stable queries (both inequality and equality), storage size, and interoperability with other GAE languages that have native date representations.
This question has few sub-questions!
What is best data type for storing date?
java-s Date, or just long - in miliseconds, formated String?
Client/Server time-zone problem
a) How to show date on client side (correct time zone)
b) If client inserts date, server needs to insert date in servers timezone or better fixed app timezone? (correct api?)
Sorting is easy with long data type, but is it working with Date?
Manipulating (for example select records for last 4 days, or between two dates, etc...).
Is there any good manual for this topic!
Currently, I am using long for date data type, but it somehow does't feel right (for example browsing in admin console is confusing).
Thanks!
java.util.Date is a supported type
How you show this to the user is a more generic java problem. All dates are stored in UTC. I suggest you take a look at joda-time for handling timezones and dates.
Yes, sorting of Date objects is supported in queries etc.
Filtering is also supported, eg, date > yesterday && date < today. Don't forget to bear in mind there are limitations when you impose filters. For example, those inequality filters can only apply to one property at a time. See restrictions on queries
I'm setting the standards for our application.
I've been wondering, what default date format should I choose to use ?
It should be:
Internationalization & timezone aware, the format should be able to represent user local time
Can be efficiently parsed by SimpleDataFormat (or alike, jdk classes only)
Programming Language agnostic (can parse in java, python, god forbid C++ :) and co.)
Preferably ISO based or other accepted standard
Easy to communicate over HTTP (Should such need arises, JSON or YAML or something in this nature)
Can represent time down to seconds resolution (the more precise the better, micro seconds if possible).
Human readable is a plus but not required
Compact is a plus but not required
Thank you,
Maxim.
yyyy-MM-ddThh:mmZ (See ISO 8601) You can add seconds, etc
You can read it easily, it will not be a problem for SimpleDateFormat.
The most canonical and standard form is probably "Unix Time": The number of seconds elapsed since midnight Coordinated Universal Time (UTC) of January 1, 1970.
If you set that as the default time-format you can easily parse it, store it in memory, write it to disk, easily communicate it over HTTP and so on. It is also definitely an accepted standard, and in a sense it is "time-zone aware", since it is well-defined regardless of time-zones.
(This is the format in which I always store all my time stamps; in databases, in memory, on disk, ...)
The "right" default format really depends on what you're doing with it. The formats for parsing, storing, and displaying can all be different.
For storing the date you're (almost) always going to want to use UTC as aioobe says, even when you want to display it in user local time. I say "(almost)" but I really can't think of a case where I would not want UTC for a saved date. You may want to store the TZ information for where the date originated also, so you can report it in that local time, but more often you want to display the local time for the whoever is currently looking at the date. That means having a way to determine the current user's local time regardless of what the original local time was.
For displaying it, the "default format" should usually be determined by the viewers locale. 08/09/10 usually means 2010-Aug-9 in the U.S. ("Middle endian") but normally means 2010-Sep-8 in most of the rest of the world ("Little endian"). The ISO-8601 format "2010-09-10" is safe and unambiguous but often not what people expect to see. You can also look over RFC-3339 for Date and Time on the internet and RFC-2822 for message format (transmitting the date)
For parsing a date, you'll want to parse it and convert it to UTC, but you should be fairly flexible on what you accept. Again, the end users Locale and timezone, if discoverable, can help you determine what format(s) of string to accept as input. This is assuming user-typed strings. If you're generating a date/time stamp you can control the form and parsing will be no problem.
I also second BalusC link which I hadn't seen before and have now favorited.