We Keep Coding

Scheduling periodic tasks and clock drift

I would like to schedule a periodic task which executes every X hours. I have a service which is written in Java and I was thinking of creating a long running background thread that runs forever as long as the service is up. How can I ensure that we are executing the task once every X hours? Is clock drift on my host an issue I should be worried about? I know that frequency of the clock ticks may change if the CPUs are working hard.
Edit: I was thinking of adding a bean to my spring configuration to spin up the thread which will periodically perform my task.

Java provides a java.util.Timer class that is designed to execute a task on a background thread. One of the modes of operation is "repeated execution at regular intervals". There are fixed-delay and fixed-rate execution methods that can be used, depending on your exact needs.
Java 5 added a java.util.concurrent.ScheduledThreadPoolExecutor class that is more flexible than Timer, but also offers fixed-delay and fixed-rate execution methods.
If you need such precise timing that these aren't suitable, I'm not sure that Java is an appropriate solution. You would be starting to enter the realm of a real-time system. At this point, you should likely be looking for other options.

If you are worried, write a test process and run it on the target platform. Using the feature you plan to use for the real process (like ScheduledExecutorService), schedule a task to log the host time every 24 hours. If the host doesn't use NTP to keep its clock synchronized, perhaps you could also make call to a time-keeping web service and log that too. After a few days, you should have a good sense of whether you need a method to correct for drift.
My guess is that the built-in scheduler will be accurate to less than a second per day.

Is clock drift on my host an issue I should be worried about?
Yes, clock drift can be an issue when using ScheduledThreadPoolExecutor.
CronScheduler is specifically designed to be proof against clock drift.
Example usage:
Duration syncPeriod = Duration.ofMinutes(1);
CronScheduler cron = CronScheduler.create(syncPeriod);
// If you need just precisely "once every X hours", irrespective of the
// starting time
cron.scheduleAtFixedRate(0, X, TimeUnit.HOURS, runTimeMillis -> {
// Do the task
});
// If you need "once every X hours" in terms of wall clock time,
// in some time zone:
ZoneId myTZ = ZoneId.systemDefault();
cron.scheduleAtRoundTimesInDay(Duration.ofHours(X), myTZ, runTimeMillis -> {
// Do the task
});
See Javadocs for scheduleAtRoundTimesInDay.

What should Timertask.scheduleAtFixedRate do if the clock changes?

We want to run a task every 1000 seconds (say).
So we have
timer.scheduleAtFixedRate(task, delay, interval);
Mostly, this works fine. However, this is an embedded system and the user can change the real time clock. If they set it to a time in the past after we set up the timer, it seems the timer doesn't execute until the original real-time date/time. So if they set it back 3 days, the timer doesn't execute for 3 days :(
Is this permissible behaviour, or a defect in the Java library? The Oracle javadocs don't seem to mention anything about the dependency or not on the underlying value of the system clock.
If it's permissible, how do we spot this clock change and reschedule our timers?

Looking at the source of Timer for Java 1.7, it appears that is uses System.currentTimeMillis() to determine the next execution of a task.
However, looking at the source of ScheduledThreadPoolExecutor, it uses System.nanoTime().
Which means you won't see that behaviour if you use one in place of a Timer. To create one, use, for instance, Executors.newScheduledThreadPool().
Why you wouldn't see this behaviour is because of what the doc for System.nanoTime() says:
This method can only be used to measure elapsed time and is not related to any other notion of system or wall-clock time. The value returned represents nanoseconds since some fixed but arbitrary origin time [emphasis mine].
As to whether this is a bug in Timer, maybe...
Note that unlike a ScheduledExecutorService, a Timer supports absolute time, and maybe this explains its use of System.currentTimeMillis(); also, Timer has been there since Java 1.3 while System.nanoTime() only appears in 1.5.
But a consequence of using System.currentTimeMillis() is that Timer is sensitive to the system date/time... And that is not documented in the javadoc.

It is reported here http://bugs.sun.com/view_bug.do?bug_id=4290274
Similarly, when the system clock is set to a later time, the task may be run multiple times without any delay to "catch up" the missed executions. Exactly this happens when the computer is set to standby/hibernate and the application is resumed (this is how I found out).
This behavior can also be seen in a Java debugger by suspending the timer thread and resuming it.

Android game loop timing discrepancy [duplicate]

When programming animations and little games I've come to know the incredible importance of Thread.sleep(n); I rely on this method to tell the operating system when my application won't need any CPU, and using this making my program progress in a predictable speed.
My problem is that the JRE uses different methods of implementation of this functionality on different operating systems. On UNIX-based (or influenced) OS:es such as Ubuntu and OS X, the underlying JRE implementation uses a well-functioning and precise system for distributing CPU-time to different applications, and so making my 2D game smooth and lag-free. However, on Windows 7 and older Microsoft systems, the CPU-time distribution seems to work differently, and you usually get back your CPU-time after the given amount of sleep, varying with about 1-2 ms from target sleep. However, you get occasional bursts of extra 10-20 ms of sleep time. This causes my game to lag once every few seconds when this happens. I've noticed this problem exists on most Java games I've tried on Windows, Minecraft being a noticeable example.
Now, I've been looking around on the Internet to find a solution to this problem. I've seen a lot of people using only Thread.yield(); instead of Thread.sleep(n);, which works flawlessly at the cost of the currently used CPU core getting full load, no matter how much CPU your game actually needs. This is not ideal for playing your game on laptops or high energy consumption workstations, and it's an unnecessary trade-off on Macs and Linux systems.
Looking around further I found a commonly used method of correcting sleep time inconsistencies called "spin-sleep", where you only order sleep for 1 ms at a time and check for consistency using the System.nanoTime(); method, which is very accurate even on Microsoft systems. This helps for the normal 1-2 ms of sleep inconsistency, but it won't help against the occasional bursts of +10-20 ms of sleep inconsistency, since this often results in more time spent than one cycle of my loop should take all together.
After tons of looking I found this cryptic article of Andy Malakov, which was very helpful in improving my loop: http://andy-malakov.blogspot.com/2010/06/alternative-to-threadsleep.html
Based on his article I wrote this sleep method:
// Variables for calculating optimal sleep time. In nanoseconds (1s = 10^-9ms).
private long timeBefore = 0L;
private long timeSleepEnd, timeLeft;
// The estimated game update rate.
private double timeUpdateRate;
// The time one game loop cycle should take in order to reach the max FPS.
private long timeLoop;
private void sleep() throws InterruptedException {
// Skip first game loop cycle.
if (timeBefore != 0L) {
// Calculate optimal game loop sleep time.
timeLeft = timeLoop - (System.nanoTime() - timeBefore);
// If all necessary calculations took LESS time than given by the sleepTimeBuffer. Max update rate was reached.
if (timeLeft > 0 && isUpdateRateLimited) {
// Determine when to stop sleeping.
timeSleepEnd = System.nanoTime() + timeLeft;
// Sleep, yield or keep the thread busy until there is not time left to sleep.
do {
if (timeLeft > SLEEP_PRECISION) {
Thread.sleep(1); // Sleep for approximately 1 millisecond.
}
else if (timeLeft > SPIN_YIELD_PRECISION) {
Thread.yield(); // Yield the thread.
}
if (Thread.interrupted()) {
throw new InterruptedException();
}
timeLeft = timeSleepEnd - System.nanoTime();
}
while (timeLeft > 0);
}
// Save the calculated update rate.
timeUpdateRate = 1000000000D / (double) (System.nanoTime() - timeBefore);
}
// Starting point for time measurement.
timeBefore = System.nanoTime();
}
SLEEP_PRECISION I usually put to about 2 ms, and SPIN_YIELD_PRECISION to about 10 000 ns for best performance on my Windows 7 machine.
After tons of hard work, this is the absolute best I can come up with. So, since I still care about improving the accuracy of this sleep method, and I'm still not satisfied with the performance, I would like to appeal to all of you java game hackers and animators out there for suggestions on a better solution for the Windows platform. Could I use a platform-specific way on Windows to make it better? I don't care about having a little platform specific code in my applications, as long as the majority of the code is OS independent.
I would also like to know if there is anyone who knows about Microsoft and Oracle working out a better implementation of the Thread.sleep(n); method, or what's Oracle's future plans are on improving their environment as the basis of applications requiring high timing accuracy, such as music software and games?
Thank you all for reading my lengthy question/article. I hope some people might find my research helpful!

You could use a cyclic timer associated with a mutex. This is IHMO the most efficient way of doing what you want. But then you should think about skipping frames in case the computer lags (You can do it with another nonblocking mutex in the timer code.)
Edit: Some pseudo-code to clarify
Timer code:
While(true):
if acquireIfPossible(mutexSkipRender):
release(mutexSkipRender)
release(mutexRender)
Sleep code:
acquire(mutexSkipRender)
acquire(mutexRender)
release(mutexSkipRender)
Starting values:
mutexSkipRender = 1
mutexRender = 0
Edit: corrected initialization values.
The following code work pretty well on windows (loops at exactly 50fps with a precision to the millisecond)
import java.util.Date;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.Semaphore;
public class Main {
public static void main(String[] args) throws InterruptedException {
final Semaphore mutexRefresh = new Semaphore(0);
final Semaphore mutexRefreshing = new Semaphore(1);
int refresh = 0;
Timer timRefresh = new Timer();
timRefresh.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
if(mutexRefreshing.tryAcquire()) {
mutexRefreshing.release();
mutexRefresh.release();
}
}
}, 0, 1000/50);
// The timer is started and configured for 50fps
Date startDate = new Date();
while(true) { // Refreshing loop
mutexRefresh.acquire();
mutexRefreshing.acquire();
// Refresh
refresh += 1;
if(refresh % 50 == 0) {
Date endDate = new Date();
System.out.println(String.valueOf(50.0*1000/(endDate.getTime() - startDate.getTime())) + " fps.");
startDate = new Date();
}
mutexRefreshing.release();
}
}
}

Your options are limited, and they depend on what exactly you want to do. Your code snippet mentions the max FPS, but the max FPS would require that you never sleep at all, so I'm not entirely sure what you intend with that. None of that sleep or yield checking is going to make any difference in most of the problem situations however - if some other app needs to run now and the OS doesn't want to switch back soon, it doesn't matter which one of those you call, you'll get control back when the OS decides to do so, which will almost certainly be more than 1ms in the future. However, the OS can certainly be coaxed into making switches more often - Win32 has the timeBeginPeriod call for precisely this purpose, which you may be able to use somehow. But there is a good reason for not switching too often - it's less efficient.
The best thing to do, although somewhat more complex, is usually to go for a game loop that doesn't require real-time updates, but instead performs logic updates at fixed intervals (eg. 20x a second) and renders whenever possible (perhaps with arbitrary short sleeps to free up CPU for other apps, if not running in full-screen). By buffering a past logic state as well as the current one you can interpolate between them to make the rendering appear as smooth as if you were doing logic updates each time. For more information on this approach, you can see the Fix Your Timestep article.
I would also like to know if there is anyone who knows about Microsoft and Oracle working out a better implementation of the Thread.sleep(n); method, or what's Oracle's future plans are on improving their environment as the basis of applications requiring high timing accuracy, such as music software and games?
No, this won't be happening. Remember, sleep is just a method saying how long you want your program to be asleep for. It is not a specification for when it will or should wake up, and never will be. By definition, any system with sleep and yield functionality is a multitasking system, where the requirements of other tasks have to be considered, and the operating system always gets the final call on the scheduling of this. The alternative wouldn't work reliably, because if a program could somehow demand to be reactivated at a precise time of its choosing it could starve other processes of CPU power. (eg. A program that spawned a background thread and had both threads performing 1ms of work and calling sleep(1) at the end could take turns to hog a CPU core.) Thus, for a user-space program, sleep (and functionality like it) will always be a lower bound, never an upper bound. To do better than that requires the OS itself to allow certain apps to pretty much own the scheduling, and this is not a desirable feature in operating systems for consumer hardware (while being a common and useful feature for industrial applications).

Thread.Sleep says you're app needs no more time. This means that in a worst case scenario you'll have to wait for an entire thread slice (40ms or so).
Now in bad cases when a driver or something takes up more time it could be you have to wait for 120ms (3*40ms) so Thread.Sleep is not the way to go. Go another way, like registering a 1ms callback and starting draw code very X callbacks.
(This is on windows, i'd use MultiMedia tools to get those 1ms resolution callbacks)

Timing stuff is notoriously bad on windows. This article is a good place to start. Not sure if you care, but also note that there can be worse problems (especially with System.nanoTime) on virtual systems as well (when windows is the guest operating system).

Thread.sleep is inaccurate and makes the animation jittery most of the time.
If you replace it completely with Thread.yield you'll get a solid FPS without lag or jitter, however the CPU usage increases greatly. I moved to Thread.yield a long time ago.
This problem has been discussed on Java Game Development forums for years.

Investigation of optimal sleep time calculation in game loop

When programming animations and little games I've come to know the incredible importance of Thread.sleep(n); I rely on this method to tell the operating system when my application won't need any CPU, and using this making my program progress in a predictable speed.
My problem is that the JRE uses different methods of implementation of this functionality on different operating systems. On UNIX-based (or influenced) OS:es such as Ubuntu and OS X, the underlying JRE implementation uses a well-functioning and precise system for distributing CPU-time to different applications, and so making my 2D game smooth and lag-free. However, on Windows 7 and older Microsoft systems, the CPU-time distribution seems to work differently, and you usually get back your CPU-time after the given amount of sleep, varying with about 1-2 ms from target sleep. However, you get occasional bursts of extra 10-20 ms of sleep time. This causes my game to lag once every few seconds when this happens. I've noticed this problem exists on most Java games I've tried on Windows, Minecraft being a noticeable example.
Now, I've been looking around on the Internet to find a solution to this problem. I've seen a lot of people using only Thread.yield(); instead of Thread.sleep(n);, which works flawlessly at the cost of the currently used CPU core getting full load, no matter how much CPU your game actually needs. This is not ideal for playing your game on laptops or high energy consumption workstations, and it's an unnecessary trade-off on Macs and Linux systems.
Looking around further I found a commonly used method of correcting sleep time inconsistencies called "spin-sleep", where you only order sleep for 1 ms at a time and check for consistency using the System.nanoTime(); method, which is very accurate even on Microsoft systems. This helps for the normal 1-2 ms of sleep inconsistency, but it won't help against the occasional bursts of +10-20 ms of sleep inconsistency, since this often results in more time spent than one cycle of my loop should take all together.
After tons of looking I found this cryptic article of Andy Malakov, which was very helpful in improving my loop: http://andy-malakov.blogspot.com/2010/06/alternative-to-threadsleep.html
Based on his article I wrote this sleep method:
// Variables for calculating optimal sleep time. In nanoseconds (1s = 10^-9ms).
private long timeBefore = 0L;
private long timeSleepEnd, timeLeft;
// The estimated game update rate.
private double timeUpdateRate;
// The time one game loop cycle should take in order to reach the max FPS.
private long timeLoop;
private void sleep() throws InterruptedException {
// Skip first game loop cycle.
if (timeBefore != 0L) {
// Calculate optimal game loop sleep time.
timeLeft = timeLoop - (System.nanoTime() - timeBefore);
// If all necessary calculations took LESS time than given by the sleepTimeBuffer. Max update rate was reached.
if (timeLeft > 0 && isUpdateRateLimited) {
// Determine when to stop sleeping.
timeSleepEnd = System.nanoTime() + timeLeft;
// Sleep, yield or keep the thread busy until there is not time left to sleep.
do {
if (timeLeft > SLEEP_PRECISION) {
Thread.sleep(1); // Sleep for approximately 1 millisecond.
}
else if (timeLeft > SPIN_YIELD_PRECISION) {
Thread.yield(); // Yield the thread.
}
if (Thread.interrupted()) {
throw new InterruptedException();
}
timeLeft = timeSleepEnd - System.nanoTime();
}
while (timeLeft > 0);
}
// Save the calculated update rate.
timeUpdateRate = 1000000000D / (double) (System.nanoTime() - timeBefore);
}
// Starting point for time measurement.
timeBefore = System.nanoTime();
}
SLEEP_PRECISION I usually put to about 2 ms, and SPIN_YIELD_PRECISION to about 10 000 ns for best performance on my Windows 7 machine.
After tons of hard work, this is the absolute best I can come up with. So, since I still care about improving the accuracy of this sleep method, and I'm still not satisfied with the performance, I would like to appeal to all of you java game hackers and animators out there for suggestions on a better solution for the Windows platform. Could I use a platform-specific way on Windows to make it better? I don't care about having a little platform specific code in my applications, as long as the majority of the code is OS independent.
I would also like to know if there is anyone who knows about Microsoft and Oracle working out a better implementation of the Thread.sleep(n); method, or what's Oracle's future plans are on improving their environment as the basis of applications requiring high timing accuracy, such as music software and games?
Thank you all for reading my lengthy question/article. I hope some people might find my research helpful!

You could use a cyclic timer associated with a mutex. This is IHMO the most efficient way of doing what you want. But then you should think about skipping frames in case the computer lags (You can do it with another nonblocking mutex in the timer code.)
Edit: Some pseudo-code to clarify
Timer code:
While(true):
if acquireIfPossible(mutexSkipRender):
release(mutexSkipRender)
release(mutexRender)
Sleep code:
acquire(mutexSkipRender)
acquire(mutexRender)
release(mutexSkipRender)
Starting values:
mutexSkipRender = 1
mutexRender = 0
Edit: corrected initialization values.
The following code work pretty well on windows (loops at exactly 50fps with a precision to the millisecond)
import java.util.Date;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.Semaphore;
public class Main {
public static void main(String[] args) throws InterruptedException {
final Semaphore mutexRefresh = new Semaphore(0);
final Semaphore mutexRefreshing = new Semaphore(1);
int refresh = 0;
Timer timRefresh = new Timer();
timRefresh.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
if(mutexRefreshing.tryAcquire()) {
mutexRefreshing.release();
mutexRefresh.release();
}
}
}, 0, 1000/50);
// The timer is started and configured for 50fps
Date startDate = new Date();
while(true) { // Refreshing loop
mutexRefresh.acquire();
mutexRefreshing.acquire();
// Refresh
refresh += 1;
if(refresh % 50 == 0) {
Date endDate = new Date();
System.out.println(String.valueOf(50.0*1000/(endDate.getTime() - startDate.getTime())) + " fps.");
startDate = new Date();
}
mutexRefreshing.release();
}
}
}

Your options are limited, and they depend on what exactly you want to do. Your code snippet mentions the max FPS, but the max FPS would require that you never sleep at all, so I'm not entirely sure what you intend with that. None of that sleep or yield checking is going to make any difference in most of the problem situations however - if some other app needs to run now and the OS doesn't want to switch back soon, it doesn't matter which one of those you call, you'll get control back when the OS decides to do so, which will almost certainly be more than 1ms in the future. However, the OS can certainly be coaxed into making switches more often - Win32 has the timeBeginPeriod call for precisely this purpose, which you may be able to use somehow. But there is a good reason for not switching too often - it's less efficient.
The best thing to do, although somewhat more complex, is usually to go for a game loop that doesn't require real-time updates, but instead performs logic updates at fixed intervals (eg. 20x a second) and renders whenever possible (perhaps with arbitrary short sleeps to free up CPU for other apps, if not running in full-screen). By buffering a past logic state as well as the current one you can interpolate between them to make the rendering appear as smooth as if you were doing logic updates each time. For more information on this approach, you can see the Fix Your Timestep article.
I would also like to know if there is anyone who knows about Microsoft and Oracle working out a better implementation of the Thread.sleep(n); method, or what's Oracle's future plans are on improving their environment as the basis of applications requiring high timing accuracy, such as music software and games?
No, this won't be happening. Remember, sleep is just a method saying how long you want your program to be asleep for. It is not a specification for when it will or should wake up, and never will be. By definition, any system with sleep and yield functionality is a multitasking system, where the requirements of other tasks have to be considered, and the operating system always gets the final call on the scheduling of this. The alternative wouldn't work reliably, because if a program could somehow demand to be reactivated at a precise time of its choosing it could starve other processes of CPU power. (eg. A program that spawned a background thread and had both threads performing 1ms of work and calling sleep(1) at the end could take turns to hog a CPU core.) Thus, for a user-space program, sleep (and functionality like it) will always be a lower bound, never an upper bound. To do better than that requires the OS itself to allow certain apps to pretty much own the scheduling, and this is not a desirable feature in operating systems for consumer hardware (while being a common and useful feature for industrial applications).

Thread.Sleep says you're app needs no more time. This means that in a worst case scenario you'll have to wait for an entire thread slice (40ms or so).
Now in bad cases when a driver or something takes up more time it could be you have to wait for 120ms (3*40ms) so Thread.Sleep is not the way to go. Go another way, like registering a 1ms callback and starting draw code very X callbacks.
(This is on windows, i'd use MultiMedia tools to get those 1ms resolution callbacks)

Timing stuff is notoriously bad on windows. This article is a good place to start. Not sure if you care, but also note that there can be worse problems (especially with System.nanoTime) on virtual systems as well (when windows is the guest operating system).

Thread.sleep is inaccurate and makes the animation jittery most of the time.
If you replace it completely with Thread.yield you'll get a solid FPS without lag or jitter, however the CPU usage increases greatly. I moved to Thread.yield a long time ago.
This problem has been discussed on Java Game Development forums for years.

ScheduledThreadPoolExecutor executing a wrong time because of CPU time discrepancy

I'm scheduling a task using a ScheduledThreadPoolExecutor object. I use the following method:
public ScheduledFuture<?> schedule(Runnable command, long delay,TimeUnit unit)
and set the delay to 30 seconds (delay = 30,000 and unit=TimeUnit.MILLISECONDS). Sometimes my task occurs immediately and other times it takes 70 seconds.
I believe the ScheduledThreadPoolExecutor uses CPU specific clocks. When i run tests comparing System.currentTimeMillis(), System.nanoTime() [which is CPU specific] i see the following
schedule: 1272637682651ms, 7858346157228410ns
execute: 1272637682667ms, 7858386270968425ns
difference is 16ms but 4011374001ns (or 40,113ms)
so it looks like there is discrepancy between two CPU clocks of 40 seconds
How do i resolve this issue in java code? Unfortunately this is a clients machine and i can't modify their system.

Yes, you're right that ScheduledThreadPoolExecutor uses System.nanoTime(). And you're also right that System.nanoTime() is dependent on the particular system instance. If your process happens to migrate between schedule and execute, then you're out of luck. (I wouldn't think that migrating between CPUs on a multi-CPU system would matter, but maybe it does? Certainly it would matter if you're running in a VM and the VM migrated between hosts).
I think the only real solution in this case is to use something other than ScheduledThreadPoolExecutor... It's not a simple as just changing ScheduledThreadPoolExecutor.now() either. AbstractQueuedSynchronizer$ConditionObject.awaitNanos() uses System.nanoTime() too.
One of my projects uses Quartz for job scheduling and I've never seen the problem you describe with that library. I don't know the implementation details (maybe it just uses System.nanoTime() too, but maybe not?).