Can anyone advise? I have a job which runs on the hour, every hour, and which is controlled in a Java webapp by Quartz 1.8.3.
What I'm seeing though is that Quartz seems to start the job a second early sometimes. But as there's no apparent pattern to this it is difficult to correct.
My code is:
Scheduler_Job sj = Scheduler_Job_DB.get(1);
sj.setJobRunning(Valid.TRUE);
Scheduler_Job_DB.update(sj);
SQLDateTime currPollDateTime = SQLDateTime.getSQLDateTimeNow();
Where currDateTimeNow is being set to 17:59:59 when it should be 18:00:00.
The cron trigger for the Quartz job is 0 0 * * * ?
The method SQLDateTime.getSQLDateTimeNow(); simply gets the current count in milliseconds and converts it into java.sql.Date and java.sql.time elements as follows:
public static SQLDateTime getSQLDateTimeNow() {
long milliSecs = DateTime_Utils.getTimeInMilliseconds();
Date date = new Date(milliSecs);
Time time = new Time(milliSecs);
SQLDateTime sqldt = new SQLDateTime(Date.valueOf(date.toString()), Time.valueOf(time.toString()));
return sqldt;
}
Class SQLDateTime simply has two variables as follows:
private Date sqlDate;
private Time sqlTime;
And is convenient when using dates and times stored in the database in yyyy-mm-dd HH:mm:ss format.
In 99.9% of cases this is fine but I don't understand why the job seems to fire one second early?
I should add that upgrading to Quartz 2.x is not an option.
Any ideas anyone?
My solution is to keep it simple. So if the Quartz job fires at a given hour, and if I have the job's details stored in a database table, then I can also store the current run time and date, e.g. YYYY-MM-DD 18:00:00.
So the job fires, it reads the details including the date and time from the database, and makes all the updates for 18:00:00.
Then at the end of the job, the current run time and date get incremented by an hour and are written back to the database for use as the next run date and time. So 18:00:00 becomes 19:00:00 and the date stays the same unless the job is running at 23:00:00.
So everything is in synch and there's no playing with milliseconds.
It sounds good in theory anyway...
It could be that the different clocks are marginally offset, so when the quartz scheduler starts on 18:00:00 005 it could be that the database still thinks its 17:59:59 980.
If its really an issue to you, you might want to sleep the thread some hundred millis so its always correct.
Related
First, definition of "occur at midnight" is that when task is run, new DateTime() or similar will show 00:00:00 or later for the time portion when converted to a human readable format. Important point is that it must not show 23:59:59 of the previous day.
A common way to achieve this would be to calculate how many milliseconds are between now and the desired point in time, and then use a ScheduledExecutorService to execute the task at the correct time. However, when a leap second is inserted this will result in the task running a second early (or a few milliseconds early depending on how the leap second is 'smeared' and when you scheduled the task):
Runnable task = ...
long numberOfMillisUntilMidnight = ...
ScheduledExecutorService executor = ...
// task runs too early when leap seconds are inserted
executor.schedule(task, numberOfMillisUntilMidnight, TimeUnit.MILLISECONDS);
The reason is that executor.schedule() is based on System.nanoTime() which obviously ignores the leap seconds. I guess what I need is some scheduler based on "run at this time" rather than "run after this amount of time".
For those who are interested, the reason the task must run at midnight related to the fact that all events in my system must be categorized according to which day they occurred on, and in so far as is possible, this needs to be in sync with another system. Of course it would be better if the other system stamped each event with what day it is, but we are not there.
I guess what I need is some scheduler based on "run at this time" rather than "run after this amount of time"
That would be the all-singing, all-dancing solution. But:
First, definition of "occur at midnight" is that when task is run, new DateTime() or similar will show 00:00:00 or later for the time portion...Important point is that it must not show 23:59:59 of the previous day.
(my emphasis)
The simple way is always add a second, or even two. So it'd be 00:00:01 (00:00:00 or later) in the common case, and 00:00:00 (not 23:59:59) in the leap second case.
Based on the resulting discussions it seems clear that, in general, it is unwise to rely on your scheduler to run a task at the "correct" time if "wall time" is important to you. This is also true when running daily tasks at the same "wall time" across daylight savings shifts, although unlike the leap second case, the daylight savings case is well supported by existing tools (by Quartz for example).
Instead I think the best approach for such "wall time sensitive" processes is that when the task is run, check the system clock at that point. If your schedule was inaccurate for whatever reason (leap seconds are not the only time your system clock is adjusted relative to the elapsed time measured by System.nanoTime()) and the time has not yet been reached, then do nothing and reschedule the task for the correct time. This approach would also work for schedules that respond to daylight savings changes but as mentioned above this is already supported by common tools.
This approach was inspired by the comment by Jonathon Reinhart above. Rescheduling rather than sleeping though seems better.
Assuming that your concrete ScheduledExecutorService-implementation relies on System.nanoTime() (as you said) and taking into account your requirement/configuration that the initial delay parameter of the method schedule(...) counts the elapsed milliseconds until next midnight including a possible leap second,
I suggest you to use a leap-second-aware solution. An example using my library Time4J shows how to calculate the delay parameter:
Moment now = SystemClock.currentMoment(); // should not be called during a leap second
Moment nextMidnight =
now.with(
PlainTime.COMPONENT.setToNext(PlainTime.midnightAtStartOfDay()).in(
Timezone.ofSystem().with(
GapResolver.NEXT_VALID_TIME.and(OverlapResolver.EARLIER_OFFSET)
)
)
);
long delayInMilliseconds = SI.NANOSECONDS.between(now, nextMidnight) / 1_000_000;
This code will also choose the earliest valid local time after midnight in case of daylight-saving-change (standard-Java would rather push the time forward by the size of the gap possibly resulting in a local time later than first valid time). For most zones, this is only relevant if choosing an arbitrary start time after midnight (dependent on business requirements).
What so ever, you should also care about connecting your systems to the same NTP-clock. Either you rely on OS-NTP-configuration, or you can use a Java-based clock connecting to NTP (Time4J offers here a solution, too).
If your chosen clock is just doing arbitrary jumps (i.e. if someone has manually adjusted it or in case of bad NTP-configuration) then rescheduling the task after having checked the local walltime again is probably safer. However, I still think that calculating the delay parameter by Time4J-code above is a good idea because the chance to match midnight is higher than just to run the task and rechecking the local time.
You could also combine both approaches: exact calculation of delay AND check/reschedule.
We need to run a batch job each night at 03:00. We have a few processes each built with some tasks.
The idea is to let the processes run each night and download some data files and update other systems.
What I don't understand is how to implement a scheduler (DMScheduler)
that will start a process every night at a specific time.
There exits general information about plugin development but nothing
specific about schedulers as far as I can see.
Can someone explain how DataMinder schedulers work and how to implement one as above?
Perhaps some implementation examples?
The 2 scheduler methods bellow work with "machine" time (as opposed to "human" time):
getNextRunTimeInMilliseconds(long nowInMilliseconds)
setLastStartedTimeInMilliseconds(long lastStartedTimeInMilliseconds)
Please see discussion about "machine" and "human" time at https://docs.oracle.com/javase/tutorial/datetime/iso/instant.html
Basically "machine" time is a time stamp in time without any relation to dates, time zones.
"Human" time is a specific date and time in a specific time zone.
This means that in this case you need to transform between the "machine" time and "human" time.
Calls to method getNextRunTimeInMilliseconds(nowInMilliseconds) may be called multiple times by DataMinder and is just a check
to see if next run time has changed because, for example, plugin/scheduler parameters may have changed.
When a process has been started and finished running the method
setLastStartedTimeInMilliseconds(lastStartedTimeInMilliseconds)
is called with the last time the process was started and run.
Let's take an example:
Here are some date/times and corresponding time stamps:
2016-08-06 03:00:00 = 1470445200000
2016-08-07 03:00:00 = 1470531600000
Assume the process was last run "6:th of August 2016, 03:00 CET". This corresponds to time stamp (as Java long) 1470445200000.
When
setLastStartedTimeInMilliseconds(1470445200000)
is called with above value you need to transfer it to "human" time
"6:th of August 2016, 03:00 CET" and then figure out what the next run time should be.
In this case the next runtime in "human" time would be the day after, that is "7:th of August 2016, 03:00 CET" and this corresponds to
timestamp 1470531600000.
The
getNextRunTimeInMilliseconds(nowInMilliseconds)
should return the new value
of 1470531600000 which tells DataMinder to run the processes on "7:th of August 2016, 03:00 CET". And when the process
has run a call will be made to
setLastStartedTimeInMilliseconds(lastStartedTimeInMilliseconds)
and the scheduler must again figure
out what the next run time should be based on the received lastStartedTimeInMilliseconds.
Hope it make things more clear.
iam trying to make a task to be executed EVERYDAY at 8am (example).
Currently what i have done is make a task, execute it automatically but only for a period of time (in this case 50 seconds) i want it to be executed every day at 8am.
my current code :
Timer time = new Timer();
certif_envia_notificacion st = new certif_envia_notificacion();
time.schedule(st, 0, 50000);
where certif_envia_notificacion is the task that i want to execute, and it does execute every 50 seconds.
Currently the only way i can figure this out is in the task certif_envia_notificacion ask for the time and execute if its 8am, but that seems like a huge waste of server resources, since ill have to execute the task at least 24 times a day.
Thanks in advance.
This is kind of a hack solution that I thought up, but feel free to give comments/feedback based on your requirements.
Basically, from my understanding you have a Java program that runs automatically when the computer starts up (if it restarts), and will then check every hour to see if it is the correct time to run the application. Why not keep it running this way, but instead of running every hour, calculate the time required to get to the next time required (e.g. 8PM the next day).
So for example, lets say you restart it at 3:15PM. When the java program runs, get it to get the current time (System.currentTimeMil...) and then calculate the time required to get to 8PM. You can do some simple math on this (e.g. 20 - current time = time required to wait) and if its negative, simply add 24 to it (e.g. if its 20 - 23, the answer is 24 - 3 = 21, thus wait 21 hours for the next 8PM).
This process can be used every time you run the code, or on startup. I'd recommend just creating a simple function to calculate the required sleep time.
What i finally did was execute the procedure every one hour and make a question of the hour and select the hour that i wanted, i know its not the best way but i needed to finish it fast....
Calendar cal = Calendar.getInstance();
int hour = cal.get(Calendar.HOUR_OF_DAY);
if(hour == DesiredHour){
//do stuff
}else if(hour == DesiredHour2){
//do stuff 2
}
Thanks for the comments.
I am designing a scheduling web app.
I expect events to be added by users in several different timezones and locales. The challenge is correct presentation of these events.
So, as an example:
if a user is in EST timezone and is looking at a webinar event that was added by another user in PST, I want to convert the actual PST time of event into a local time for viewer. So, if an event is scheduled for 2 PM PST, then it should shown as 5 PM EST.
I also want to be careful that performance does not take a hit if there are thousands of events which may require a conversion from actual event time to the viewer's local time.
All thoughts and comments are appreciated.
TIA
In general, scheduling future events is a complex subject. You have to make a distinction in the context of what is going to be scheduled:
Does the event occur at a specific universal instant in time? If so, you should record the event time in terms of UTC.
For example, a task that runs every 24 hours would be scheduled by UTC time and not by the local time. It might start off at some local midnight, but as daylight saving time changes take effect it could be running at 23:00 or 01:00 by the local clock.
However, if the event is scheduled by human beings, it is likely to be in terms of a local time, so you should record it that way.
For example, a meeting that occurs at 08:00 Eastern Time will always occur at that local time. In the winter, that would be 13:00 UTC, and in the summer it would be at 12:00 UTC.
So in this context, you cannot record the scheduled start time in terms of UTC. This is a very common mistake, as there is a ton of advice on the Internet that says "always store using UTC", which would be wrong in this scenario.
Instead, you should store two values - the local time such as 08:00 and its IANA time zone identifier, such as America/New_York. You may also need to store a recurrence pattern or specific date depending on how the event is scheduled.
Consider using Joda Time instead of Java's Calendar or Date classess. It will save you from many headaches. Make sure you read the Joda Time documentation and understand how it works.
Joda Time has all of the functions you will need for converting between one time zone and another - which I believe was the primary concern of your question.
Be sure to have a procedure in place for updating the time zone data regularly. Updates are pushed out multiple times a year, as the governments of the world make changes to the legal definitions of their time zones. You cannot just deploy it once and forget about it.
Also be sure you understand that conversion from local time to a specific UTC moment is not a perfect function due to daylight saving time. If an event is scheduled during an invalid or ambiguous local time, you should have a strategy for detecting and dealing with that in your application. You might just apply some assumptions, or you might want to go out of your way to ask the user what to do.
For example, if I schedule an event at 2:00 AM Eastern Time every day, then on March 10th 2013, that time does not exist. Should the event occur at 3:00 AM? Or should it not occur at all?
Another example, if I schedule an event at 1:00 AM Eastern Time every day, then on November 3rd, 2013, that time occurs twice. Should the event happen at the first (daylight time) instance? Or at the second (standard time) instance? Or both? Should I assume one or the other, or should I ask the user which they mean?
Only you can decide what to do, as it is your application. But ignoring the problem will likely lead to errors.
Once an event has passed, you can record it in UTC if you wish, or record it with the full local date time and offset. Either are acceptable. This works just fine for singular past events, just not for recurring future ones.
In quartz scheduler I have a doubt
For example
If we schedule job for every 5 minutes or 10 minutes or 2 hours it is ok we call scheduler using regular time interval like below no problem,
SimpleTrigger simpleTrigger = new SimpleTrigger("mytrigger",sched.DEFAULT_GROUP,new Date(),null,SimpleTrigger.REPEAT_INDEFINITELY,30L*5000L);
But,If I schedule job # 2'0 clock and 3'0 clock today then,tommorow I change it to 5'0 clock and 7'0 clock and 11'0 clock, my doubt is if fixed interval we call it in above way but it not fixed or periodic interval how do we handle it. Any suggestions or Ideas or examples regarding my query....
What you can do is define the intervals in a file or DB and read it from there.
Also define a JobListener.
Store the interval values in the JobDataMap and every time the job starts, use the listeners method jobToBeExecuted() and read the values from the file/DB and compare it with the values in the JobDataMap.
If they are the same, then no update was done and the job runs, if it was changed, reschedule the job using Scheduler.rescheduleJob(TriggerKey triggerKey,Trigger newTrigger).
Note that there is a little problem with this method, the configuration will only change once the job runs, so even though you update the file/DB the next run would still be in the old time, and only the run after that would be in the new time.