How can I obtain the current TAI time in milliseconds in Linux using either Java or C++?
The reason I need this is to be able to accurately take timestamps over a long period of time (on the order of years) and still be able to compare them, without worrying about leap seconds. It is possible for multiple measurements to take place during a leap second and all measurements need to be unambiguous, monotonically increasing, and linearly increasing. This will be a dedicated Linux server. This is for a scientific project which needs precision of about .5 seconds.
I do not currently wish to invest in a GPS timekeeper and hope to use NTP to pool.ntp.org in order to keep the system clock on track.
I have looked into the following solutions:
Java 8 or the ThreeTen Project
The only way to obtain a TAIInstant is to use an Instant and then convert it which, according to the specs, "Conversion from an Instant will not be completely accurate near a leap second in accordance with UTC-SLS." That in and of itself is not a big deal (in fact, using UTC-SLS would also be acceptable). However, using now() in the Instant class also seems to just be a wrapper for System.currentTimeMillis(), which makes me think that during the leap second, the time will still be ambiguous and the project will not actually give me TAI time. The Java 8 specifications also state:
Implementations of the Java time-scale using the JSR-310 API are not
required to provide any clock that is sub-second accurate, or that
progresses monotonically or smoothly. Implementations are therefore
not required to actually perform the UTC-SLS slew or to otherwise be
aware of leap seconds.
Using a right/? timezone
This seems like it would work, however I am not sure if the implementation is smart enough to continue working during a leap second or if System.currentTimeMillis() would even give TAI time. In other words, would the underlying implementation still use UTC, thus giving an ambiguous time during the leap second which is then converted to TAI, or does using a right/ timezone actually work with TAI using System.currentTimeMillis() always (ie even during leap second)?
Using CLOCK_TAI
I tried using CLOCK_TAI in the Linux kernel but found it to be completely identical to CLOCK_REALTIME in my test:
Code:
#include <iostream>
#include <time.h>
long sec(int clock)
{
struct timespec gettime_now;
clock_gettime(clock, &gettime_now);
return gettime_now.tv_sec;
}
int main()
{
std::cout << sec(0) << std::endl; // CLOCK_REALTIME
std::cout << sec(1) << std::endl; // CLOCK_MONOTONIC
std::cout << sec(11) << std::endl; // CLOCK_TAI
return 0;
}
The output was simply:
1427744797
6896
1427744797
Using CLOCK_MONOTONIC
The problem with this is that the timestamps need to remain valid and comparable even if the computer restarts.
CLOCK_REALTIME and CLOCK_TAI return the same because the kernel parameter tai_offset is zero.
Check by using adjtimex(timex tmx) and read the value. I think that ntpd will set it if it is new enough (>4.2.6) and has a leap second file. It may also be able to get it from upstream servers but I haven't been able to verify. The call adjtimex() can set tai_offset manually when run as root. You will need a new-ish man page for adjtimex to see the parameters to set. My debian man page was too old but the command worked.
In addition to the correct accepted answer I would also mention the free Java library Time4J (min version v4.1) as possible solution because
I have written it to fill a gap in Java world (java.time cannot do all),
other answers given so far only talk about C++ (but you also asked for Java),
it works according to the same principles described by #user3427419.
It uses a monotonic clock based on System.nanoTime() but even allows custom implementations via the interface TickProvider. For the purpose of calibration, you can either use net.time4j.SystemClock.MONOTONIC, or you use an SNTP-clock named SntpConnector which just needs some simple configuration to connect to any NTP-time-server you want. And thanks to the built-in leap-second-table Time4J can even show you the announced leap second at the end of this month - in ISO-8601-notation or even as formatted local timestamp string in any timezone (using i18n-module).
A recalibration (in case of NTP - reconnect) of the clocks is possible meaning the clocks can be adapted to intermediate time adjustments (although I strongly recommend not to do it during your measurements or during a leap second). Although such a reconnect of an SNTP clock would normally cause the time stepping back in some cases Time4J tries to apply a smoothing algorithm (if activated in clock configuration) to ensure monotone behaviour. Detailed documentation is available online.
Example:
// Step 0: configure your clock
String ntpServer = "ptbtime1.ptb.de";
SntpConnector clock = new SntpConnector(ntpServer);
// Step 1: Timestamp start of the program and associate it with a counter
clock.connect();
// Step 2: Use the counter for sequential measurements at fixed intervals
Moment m = clock.currentTime();
System.out.println(m); // possible output = 2015-06-30T23:59:60,123456789Z
// Step 3: Timestamp new counter value(s) as necessary to keep your data adequately synced
clock.connect();
I doubt if any C++-based solution is more simple. More code demonstrations can also be studied on DZone.
Update (answer to question in comment):
A slightly simplified solution how to automatically download the given IETF-resource for new leap seconds and to translate it into a Time4J-specific format might look like this:
URL url = new URL("https://www.ietf.org/timezones/data/leap-seconds.list");
BufferedReader br =
new BufferedReader(
new InputStreamReader(url.openStream(), "US-ASCII"));
String line;
PlainDate expires = null;
Moment ntpEpoch = PlainTimestamp.of(1900, 1, 1, 0, 0).atUTC();
List<PlainDate> events = new ArrayList<PlainDate>();
try {
while ((line = br.readLine()) != null) {
if (line.startsWith("##")) {
long expraw = Long.parseLong(line.substring(2).trim());
expires = ntpEpoch.plus(
expraw, TimeUnit.SECONDS)
.toZonalTimestamp(ZonalOffset.UTC).toDate();
continue;
} else if (line.startsWith("#")) {
continue; // comment line
}
// this works for some foreseeable future
long epoch = Long.parseLong(line.substring(0, 10));
// this is no leap second
// but just the official introduction of modern UTC scale
if (epoch == 2272060800L) {
continue;
}
// -1 because we don't want to associate
// the leap second with the following day
PlainDate event =
ntpEpoch.plus(epoch - 1, TimeUnit.SECONDS)
.toZonalTimestamp(ZonalOffset.UTC).toDate();
events.add(event); // we don't assume any negative leap seconds here for simplicity
}
} finally {
br.close();
}
// now let's write the result into time4j-format
// use a location relative to class path of main program (see below)
String path = "C:/work/leapseconds.txt";
Writer writer = new FileWriter(new File(path));
String sep = System.getProperty("line.separator");
try {
for (PlainDate event : events) {
writer.write(event + ", +" + sep);
}
writer.write("#expires=" + expires + sep);
} finally {
writer.close();
}
System.out.println(
"Leap second file was successfully written from IETF-resource.");
// And finally, we can start the main program in a separate process
// with the system property "net.time4j.scale.leapseconds.path"
// set to our leapsecond file path (must be relative to class path)
Some notes:
I recommend to write this code as subprogram called by a simple batch program in order to avoid the main program being dependent on internet connectivity. This batch file would finally call the main program with the mentioned system property. If you set this property then the leap seconds will be read from the file specified there, and any eventually available tzdata-module would then stop to yield any concurrent leap second informations.
The reason I need this is to be able to accurately take timestamps
over a long period of time (on the order of years) and still be able
to compare them, without worrying about leap seconds. It is possible
for multiple measurements to take place during a leap second and all
measurements need to be unambiguous, monotonically increasing, and
linearly increasing.
Then your design is suboptimal. You cannot use time and then somehow meddle through leap seconds. This actually comes up often enough and people fall into the same trap of timestamping measurements using wall clock.
Timestamp start of the program and associate it with a counter
Use the counter for sequential measurements at fixed intervals
Timestamp new counter value(s) as necessary to keep your data adequately synced
If you avoid timestamping for the 1 second that leapsecond can occur (midnight!), you are home free because those can be adjusted later.
Now if you insist on using TAI without counter, all you need is a table with leap seconds that need to be accounted for. Then just use monotonic time. There is also libraries that can do this for you, but they may be out of date so you'll have to maintain them yourself,
http://skarnet.org/software/skalibs/libstddjb/tai.html
You have to implement a TAI clock based on C++ std::steady_clock or similar. To synchronize your TAI clock you could rely on GPS or NTP.
Option TAI from NTP: Your TAI implementation would need knowledge about leap seconds. Probably NTP protocol or referenced resources are the most reliable sources of current and future leap seconds.
Option TAI from GPS: GPS clock has a fixed offset to TAI, you do not have to mess with leap seconds
Related
I'm working with an agenda in Java. I have stored in my database the day of the week, the start and end time of some labs availability.
Now I need to provide a service for a schedule system by showing only the unavailable times of the day. For example, if day one has start time 13:00 and end time 19:00, I need to return a range just like this:
[00:00 - 13:00, 19:00 - 23:59] . Remembering that a day can have more than a range available.
Is there any Java Class or API that could help me on subtracting these ranges?
My lib Time4J offers following solution for the subtraction problem:
ClockInterval fullDay = ClockInterval.between(PlainTime.of(0), PlainTime.of(24));
ClockInterval slot = ClockInterval.between(PlainTime.of(13, 0), PlainTime.of(19, 0));
IntervalCollection<PlainTime> icoll = IntervalCollection.onClockAxis().plus(fullDay);
List<ChronoInterval<PlainTime>> result = icoll.minus(slot).getIntervals();
The resulting list of half-open intervals (with open end) can be easily iterated through and gives the expected result {[T00:00/T13:00), [T19:00/T24:00)}. Every result interval can be converted to a standard ClockInterval, too. There are also various methods to print such intervals in a localized way. Furthermore, you might find the class DayPartitionBuilder interesting which allows to connect weekdays and time schedules in streaming, see the example given in the documentation.
About compatibility with java.time:
The between()-methods of ClockInterval also accept instances of java.time.LocalTime.
Every instance of PlainTime can be converted back to LocalTime by help of the method toTemporalAccessor() with the exception of the value 24:00 which exists in Time4J but not in java.time.LocalTime.
First of all I'm new to java.time package.
I'm writing a webapp that need to work with specific times of the day and with several durations of events.
So I wrote my code using LocalTime and Duration classes of java.time package.
When I need to render their value in JSP it is very simple for LocalTime object (because .toString() returns a human readable vale), so I can just write ${startTime} and everything goes in the right way (e.g. it is rendered as 9:00).
The same approach doesn't work for Duration, because its representation is something like PT20M (in this case for 20 minutes).
Does it exist an elegant way to perform a human-readable conversion in JSP directly by EL?
Yes, I know I can convert the object in a string in my classes (before JSP), but I'm searching for the suggested approach (that I'm not able to find)... another point is that I not see an official "convert()" (or whatever else) method in Duration object... so I'm thinking I'm using the wrong object to map a duration of time (to add or subtract on LocalTimes).
Thank you.
Unfortunately there exists no elegant builtin way to format a Duration in Java 8. The best i have found is to use the method bobince describes in this answer:
Duration duration = Duration.ofHours(1).plusMinutes(20);
long s = duration.getSeconds();
System.out.println(String.format("%d:%02d:%02d", s/3600, (s%3600)/60, (s%60)));
Which prints:
1:20:00
The code will have to be tuned if you need longer durations.
I'm not sure what you mean that you are missing a convert method, but Duration is well suited for adding/subtracting on LocalTime. The methods LocalTime.plus() and LocalTime.minus() accepts Duration as argument.
If you're interested in words, apache commons will do the trick:
DurationFormatUtils.formatDurationWords(System.currentTimeMillis() - start, true, false))
2 days 1 hour 5 minutes 20 seconds
https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/time/DurationFormatUtils.html#formatDurationWords-long-boolean-boolean-
I have requirement in which I continuously receive messages that needs to be written in a file. Every time a new message is received it needs to be written in a separate file. What I want is to generate an unique identifier to be used as a file-name. I also want to preserve the order of the messages as well. By this I mean, the identifier generated as a file-name should always be incremental.
I was using UUID.randomUUID() to generate file-names but the problem with this approach is that UUID only assures randomness of the identifier but is not incremental. As a result I am losing the ordering of the file (I want file generated first should appear first in the list).
Approaches known
Can use System.currentTimeMillis() but I can receive multiple messages at same time stamp.
2.Another approach could be to implement static long value and increment it whenever a file is to be created and use the long value as a file-name. But I am not sure about this approach. Also it doesn't seem to be a proper solution to my problem. I think there could be far better solutions than this one.
If someone could suggest me a better solution to this problem, will be highly appreciated.
If you want your id value to uniformly rise even between server restarts, then you must either base it on the system time or have some elaborately robust logic that persists the last ID used. Note that achieving robustness on its own is not hard, but achieving it in a performant and scalable way is.
If you additionally need the id to be unique across multiple nodes in a redundant server cluster, then you need even more elaborate logic, which definitely involves a persistent store to which all the boxes synchronize access. Making this performant is, of course, even harder.
The best option I can see is to have a quite long ID so there's room for these parts:
System.currentTimeMillis for long-term uniqueness (across restarts);
System.nanotime for finer granularity;
a unique id of each server node (determined in a platform-specific way).
The method will still have to remember the last value generated and retry in case of a duplicate. It won't have to retry too many times, though, just until the next nanoTime clock tick—it could even busy-wait for it.
Sketch of code without point 3 (single-node implementation):
private static long lastNanos;
public static synchronized String uniqueId() {
for (;/*ever*/;) {
final long n = System.nanoTime();
if (n == lastNanos) continue;
lastNanos = n;
return "" + System.currentTimeMillis() + n;
}
}
Ok, my hands up. My last answer was fairly flaky and I've deleted it.
Keeping with the spirit of the site, I thought I'd try a different tac.
If you say you are keeping these messages in a single file then you could try something like creating an unique Id out of the size of the file?
Before you write the message to the file it's id could be the current size of the file.
You could add the filename + size as the id if these messages need to be unique across a number of files.
I'll leave the hot potato of synchronization to another day. But you could wrap all of this up in a syncronized object that keeps track of things.
Also, I am assuming that any messages written to the file will not be removed in the future.
ADDITIONAL NOTE:
You could create an message processing object that opens the file on construction (or via a create method).
This object will get the initial size of the file and this will be used as the unique id.
As each message is added (in a synchronized manner), the id is incremented by the size of the message.
This would address the performance issues. Will not work if more than one JVM/Node accesses the same file.
Skeletal Idea:
public class MessageSink {
private long id = 0;
public MessageSink(String filename) {
id = ... get file size ..
}
public synchronized addMessage(Message msg) {
msg.setId(id);
.. write to file + flush ..
.. or add to stack of messages that need to be written to file
.. at a later stage.
id = id + msg.getSize();
}
public void flushMessages() {
.. open file
.. for each message in stack write ...
.. flush and close file
}
}
I had the same requirement and found a suitable solution. Twitter Snowflake uses a simple algorithm to generate sortable 64bit (long) ids. Snowflake is written on Scala but the approach is simple and could be easily used in a Java code.
id is composed of:
timestamp - 41 bits (millisecond precision w/ a custom epoch gives us 69 years);
machine id - 10 bits (MAC address could be used as a hardware id);
sequence number - 12 bits - rolls over every 4096 per machine (with protection to avoid rollover in the same ms)
Formula looks like: ((timestamp - customEpoch) << timestampShift) | (machineId << machineIdShift) | sequenceNumber;
Shift for each component depends on it's bits position in ID.
Detailed description and source code could be found at github:
Twitter Snowflake
Basic Java implementation of the Snowflake algorithm
What is most efficient way to compare date range in case of open end date i.e. optional end date? I want to compare effective date and optional end date given by user with any existing overlapping effective date and optional end date in database.If they overlap I want to display error message. Effective date is required and end date is optional.
Important:
effective and end date situation can be implemented in two general ways.
1) By having end date as DB column
For example, Mortgage or saving account rate. The rate becomes effective at certain point of time and then it stays in effect till next rate becomes effective and ends previous rate's effect.At given point of time at least one record will be in effect.
2) By not having end date in database
For example, Discount, coupoun, promotion or special offer. These all can become effective and end at certain point of time. It is possible that at given time no special offer or discount is running.
Scenario 1 is easy to implement. Every time you insert or edit the record you have to check there is no equivelent record in db with exact same effective date(and time).
Scenario 2 may have two further flavors.
2.1) End date is always required.(user entered or default with year 9999)
In such case, if you find any record that has (start1 <= end1 and start2 <= end2) then you have overlap.
2.2) End date is optional in that case null means positive infinity. User can enter end date or leave it blank.
This can be tricky to validate as more possible combination. You may need to generate query dynamically based on user has given end date or not
if(userEnd != null) {
query.append(dbStart<=userEnd)
}
query.append(dbEnd is null || dbStart>=userStart && dbEnd>=userStart)
If this query finds any result then you are overlaping range. Going one step further if you have requirement to automatically end date previous record if previous record's end date is null then you may want to modify above query as below to pass validation.
if(userEnd != null) {
query.append(dbStart<=userEnd)
}
query.append((dbEnd is null && dbStart>=userStart) || (dbStart>=userStart && dbEnd>=userStart))
Depending on other requirements you may need deleteDate to mark record invalid. Possible combination can be
Effective Date(Required) | Deleted Date (Optional)
OR
Effective Date(Required) | End Date(Required or Optional) | Deleted Date (Optional)
I made a schematic image about a reference interval, which might be open ended (gradient), and a timespan to compare:
The 5 basic cases a-e are without open end. Let's consider the timespan to compare not being open ended in the beginning.
Let's further define, that no two dates matches exactly - maybe because they are measured in microseconds. It doesn't really matter, because you will just switch from < to <= or not, whatever you consider to be a valid assumption.
From the basic cases, we see, they always overlap, except the sample.end is < reference.start or sample.start > ref.end.
Who would have thought it is that easy?
Well - let's see, what happens, if ref.end is open. Case a is not affected, but case e will overlap then too.
That was an easy one, wasn't it?
Now we get difficult: What, if the sample is open ended? Case a will now overlap, but e not being affected.
Hardcore experience: Both dates are open ended: Then an overlap occured.
Conclusion: If in doubt, do an image. Here it was inkscape.
ok not as simple as title may make it sound. I tried this in a very primal way with c# and it worked, but I have a feeling a better job could be achieved with Java and Oracle as database. So the thing is:
I have a reservation system. multiple bookings could be made on the same day for period between date X and date Y as long as each day in the range can accommodate the requested number. Maximum number of clusters to reserve is 46. Hence logically you would look at each day as a holder of 46 cluster reservation and deduce from that.
Now what I have difficulty working out is:
when there are n number of bookings stored and valid in database, then I want to make new booking. So how do I check if this new date range falls within any of the previously booked days or not. Not talking simply here about x falling in y (as ranges). More like:
X_______________________________________Y
X________________________________y
X________________________________Y
X________________________________Y
as u can see the overlap is happening.
Please let me know how could I do this as it will affect early design of objects
Regards
Assume your date has two methods: isBefore(Date other) and isAfter(Date other). Obviously if they don't you can cure this with an external method or wrapping or something. Edit: java.util.Date has compareTo method you could use.
You do this:
public boolean overlapsWithExisting(Booking booking) {
final Date early = booking.getStart();
final Date late = booking.getEnd();
for(Booking existing : existingBookings) {
if(!(early.isAfter(existing.getEnd()) || late.isBefore(existing.getStart()))
return true;
}
return false;
}
We compare this booking to all existing bookings. If this booking ends before the existing booking even starts, or if this booking starts after the existing booking ends, then it doesn't conflict. Any other condition and they will overlap.
Do this to each booking.
Joda-Time – Interval
Rather than roll your own, use the Interval class in the Joda-Time library. An Interval is a pair of specific points along the timeline, each defined as a DateTime instance.
The Interval class offers overlap, gap, and abuts methods.
Half-Open
Those methods wisely use the Half-Open approach to spans of time where the beginning is inclusive while the ending is exclusive. Search StackOverflow for more info.