Quartz intermittently triggering the job multiple times when using JobStoreTX - java

I'm using java quartz(2.3.1) and I have a setup where I'm using postgres as the job store and I have 3-4 machines all running the scheduler (vertical scaling). I want the rds to act as a source of truth and if I have a job with the schedule of repeat every 1 hour, I want it to run on any one of the machines. I don't care which one it runs on as long as it is one machine triggered in that hour.
I noticed that this works really well most of the time but I have recently had a trigger which runs once ever hour and about once every two days I see two of my machines getting triggered. I have noticed that my isClustered property is false which I have now set to true, but I'm not sure how this would help since if this was the problem, this issue would be happening a 100% of the time rather than rarely. Could anyone tell me what I should be looking into to actually fix this issue?

org.quartz.jobStore.isClustered = true
ensures proper database row locks are applied to the trigger before picking it, if that property was false both instances can pick up one trigger(race condition) before one could change status of that trigger.
http://www.quartz-scheduler.org/documentation/quartz-1.8.6/configuration/ConfigJDBCJobStoreClustering.html

Related

is it possible to restart a job after stopping?

I am working on a project in which I can hit maximum 15k hit a day to Google API. So I want to stop the job after 15k and resume it next day. Please let me know how can I do the same.
Please let me know how can I achieve the same. Right now I am thinking of using quartz scheduler to schedule the job every day.
If anyone needs full explanation, I can explain it more.
Thanks in advance.
You can stop a step execution (and its surrounding job) using StepExecution#setTerminateOnly. So in your case, you can use for example a ItemReadListener#afterRead or ItemWriteListener#afterWrite that has access to the step execution and set the terminateOnly flag after processing 15k items. When you stop the job gracefully like this, its status will be STOPPED and you will be able to restart it again the next day as you mentioned.
You can find an example in the Stopping a Job Manually for Business Reasons section of the reference documentation.
Hope this helps.
I had something similar where I needed to stop a 24/7 job 5 minutes before server maintenance was scheduled to start.
The easiest I found was to use the Reader and return null to indicate the job should stop. In your case, return null when 15k API requests were processed.
This will likely mean you'll need a bean (could be just an AtomicInteger) available to the Reader and updated by the Processor. But also a Job Listener (sorry, I don't have the code) which also knows about the bean. If the maximum is reached the Listener sets up a custom job exit value to be returned to the scheduler when the job stops. The scheduler has to be configurable enough to know the particular exit value means to start the job again the next day. (Any other non-zero value was treated as an error.)
This means there is a small possibility the job hits 15k but also that it is the last item, so the job is scheduled again for the next day even though there is nothing more to be processed. It shouldn't matter though - the job will start the next day and stop immediately with a normal complete status so the scheduler will not schedule again.

Android Things - UpdatePolicy not applied

I'm programming an update interface in my Android Things project. I can do manual update, with an user input. But I'm trying to schedule an auto-update every night at midnight. I want to use a custom UpdatePolicy with a deadline but I failed to use it.
I tried this in the onCreate method in my activity :
mUpdateManager.setPolicy(
new UpdatePolicy.Builder()
.setPolicy(POLICY_APPLY_AND_REBOOT)
.setUpdateDeadline(10, TimeUnit.SECONDS)
.build());
But there isn't any update after 10 seconds.
Maybe, I don't understand the deadline.
Do I use it wrong ?
The deadline has nothing to do with when an update check is performed. The usual schedule of update checks is
once shortly after boot
once every 5 hours (approximately) thereafter
(These times are not exact for reasons that aren't relevant to this discussion.)
The deadline reflects how long the device will let an available update sit without being applied before the device will force it to apply and reboot. The device doesn't know about an available update until it performs a check, so you could be waiting up to 5 hours for that.
The deadline is meant to operate on a longer timescale (for instance, 5 days, a week, etc). This is useful as a fallback in case there's some kind of bug with the update scheduler, or in case you allow users to postpone the update but don't want them to be able to do that forever.
To achieve what you want, you should schedule (using WorkManager, JobScheduler, etc) a task that runs at midnight each day and calls UpdateManager.performUpdateNow(UpdatePolicy.POLICY_APPLY_AND_REBOOT)
TL,DR: Update checks are very much a background thing. If you care about timing at all, use UpdateManager.performUpdateNow, but no more than once every 5 hours.

Quartz How do I put triggers in a state where they wont re-fire a misfire

I have an interesting issue with our web application. We recently had a huge concurrency problem with Quartz firing off the same java class in hundreds of threads at once causing contention on the database and completely locking our application. Before we were able to address the underlying issue, I had to set all misfire_instr to "1" (Do not re-fire a misfire) on the database table qrtz_triggers to get the application to come up and serve normal traffic. We then manually fired off the missed schedules slowly to avoid the concurrency issue.
I have fixed the concurrency issue and am ready to hot-deploy our application, but am afraid if I set the misfire_instr back to 2 (re-fire misfires), the missed schedules will re-fire again based on prev_fire_time?
Proposed solution - set prev_fire_time to a couple minutes before the next_fire_time so it thinks the last schedules went off. Is my assumption here correct that a schedule is considered missed if the prev_fire_time is before the last time represented by the cron expression?
I also see that schedules which have yet to run have -1 in the column, is this the default for a new schedule, perhaps I should be using this value instead?
Since we needed this to go in immediately, I decided to just try my luck with using -1 as the prev_fire_time value (and updating the triggers back to 2). This worked as expected.

execute job immediately

I try to run emergency job immediately after scheduled with quartz.
my code is below.I give current time as startTime.
Bıt it takes 30- 40 seconds to run job after schedule.How can run immediately.
// Trigger the job to run now, and then repeat every 40 seconds
jobTrigger= newTrigger()
.withIdentity(Long.toString(emergencyJob.getId()), Long.toString(emergencyJob.getVariant().getId()))
.withPriority(emergencyJob.getPriority())
.startAt(new Date(ctime))
.withSchedule(simpleSchedule().withMisfireHandlingInstructionFireNow()
)
.build();
scheduler.scheduleJob(jobDetail, jobTrigger);
You can also fire it with:
scheduler.triggerJob(jobDetail.getKey());
This is just a guess (it's been a while since I've used quartz), but since you create the Date instance before you actually call build(), it may not be able to meet that time constraint and simply fires 40 seconds later when the next scheduled trigger fires. Try something like this to confirm:
.startAt(new Date(System.currentTimeMillis() + 1000))
Play with the 1000ms value to suit your needs. This is to give it a bit more time to meet the first scheduled trigger.
Probably bit late, but maybe someone will find this useful. I had same issue with quartz on JBoss AS (triggers executed late - approximately 20-30 seconds, for no obvious reason). I came to conclusion that this is caused by some bug in JBoss. Same application worked fine on glassfish. I have changed only PU in order to work with eclipse link and other persistence worked fine on JBoss, so I don't suppose issue there. This behaviour occured only when using quartz database task store, with RAM store it worked fine.
To answear the question. If you use database taskstore, consider changing it to RAM store for me that caused triggers to be fired at time.

Scheduling tasks, making sure task is ever being executed

I have an application that checks a resource on the internet for new mails. If there is are new mails it does some processing on them. This means that depending on the amount of mails it might take just a few seconds to hours of processing.
Now the object/program that does the processing is already a singleton. So right now I already took care of there really only being 1 instance that's handling the checking and processing.
However I only have it running once now and I'd like to have it continuously running, checking for new mails more or less every 10 minutes or so to handle them in a timely manner.
I understand I can take care of this with Timer/Timertask or even better I found a resource here: http://www.ibm.com/developerworks/java/library/j-schedule/index.html that uses Scheduler/SchedulerTask. But what I am afraid of.. is if I set it to run every 10 minutes and a previous session is already processing data it will put the new task in a stack waiting to be executed once the previous one is done. So what I'm afraid of is for instance the first run running for 5 hours and then, because it was busy all the time, after that it will launch 5*6-1=29 runs immediately after each other checking for mails and/do some processing without giving the server a break.
Does anyone know how I can solve this?
P.S. the way I have my application set up right now is I'm using a Java Servlet on my tomcat server that's launched upon server start where it creates a Singleton instance of my main program, then calls some method to do the fetching/processing. And what I want is to repeat that fetching/processing every "x" amount of time (10 minutes or so), making sure that really only 1 instance is doing this and that really after each run 10 minutes or so are given to rest.
Actually, Timer + TimerTask can deal with this pretty cleanly. If you schedule something with Timer.scheduleAtFixedRate() You will notice that the docs say that it will attempt to "make up" late events to maintain the long-term period of execution. However, this can be overcome by using TimerTask.scheduledExecutionTime(). The example therein lets you figure out if the task is too tardy to run, and you can just return instead of doing anything. This will, in effect, "clear the queue" of TimerTask.
Of note: TimerTask uses a single thread to execute, so it won't spawn two copies of your task side-by-side.
On the side note part, you don't have to process all 10k emails in the queue in a single run. I would suggest processing for a fixed amount of time using TimerTask.scheduledExecutionTime() to figure out how long you have, then returning. That keeps your process more limber, cleans up the stack between runs, and if you are doing aggregates, ensures that you don't have to rebuild too much data if, for example, the server is restarted in the middle of the task. But this recommendation is based on generalities, since I don't know what you're doing in the task :)

Categories