Related
I have a couple controller methods that must not be allowed to run at the same time:
#Scheduled(cron = "0 * * * * *")
public void first() {
// Acts on database
}
#RequestMapping(value = "/second", method = RequestMethod.POST)
public void second() {
// Also acts on same database
}
First one runs only as a scheduled job. Second one only runs as an HTTP request.
When second() gets called I want it to wait until first() finishes if it is running, then run immediately afterwards. If first() is not running, want second() to run and block first() from running while second() is still running. By block I mean don't allow first() to run at all, don't wait for second() to finish or queue it to run later either. It will attempt to run again only on its next scheduled run.
Edit:
If second() gets requested again while the previous request to second() has not yet completed, want that new request to be ignored.
If you have to maintain this only on one instance of your application, then you can use for example AtomicBoolean:
Let's create additional method, where you make something like this:
private AtomicBoolean isSecondRunning = new AtomicBoolean();
#Scheduled(cron = "0 * * * * *")
public void first() {
if (isSecondRunning.get()) {
return; // 1
}
execute();
}
#RequestMapping(value = "/second", method = RequestMethod.POST)
public void second() {
isSecondRunning.set(true); // 2
try {
execute();
} finally {
isRunning.set(false); // 3
}
}
public synchronized void execute(){
// here execute the code
}
Code explanation:
if isSecondRunning is true, then return from first without execution, if is false, then skip if and go to execute() method
when second is executed, then set isSecondRunning to true and then execute
set isSecondRunning to false, and do it inside finally block, so we can be sure, that it is set to false even if some exception occurs in your execution
The execute method is synchronized, so if first is running, then second will wait
The easiest way would to make both call a method in another layer (e.g. a service). That method, if declared on a singleton bean, can be synchronized so only one thread will be able to execute it at the same time in the same server.
class ScheduledTasks{
#Autowired private Service service;
#Scheduled(cron = "0 * * * * *")
public void first() {
service.doStuff();
}
}
class MyController{
#Autowired private Service service;
#RequestMapping(value = "/second", method = RequestMethod.POST)
public void second() {
service.doStuff();
}
}
#Service
class Service{
public synchronized void doStuff(){...}
}
Be aware, though, that it will cause concurrent requests to your endpoint to seemingly "halt" until the previous ones have completed, when they attempt to call that method.
As an alternative, you may want to convert your Scheduled method to a Quartz job and modify the trigger when your controller is called. This would also require some degree of synchronization so the triggers are modified atomically among concurrent requests, and also you may still need a synchronized method to guarantee that if first() is already running you don't execute the changes from second().
The problem is you can't really do anything simple because the outcome of whether a job is allowed to be queued or not depends on what jobs are already in the queue.
I would suggest you need a JobManager that controls the queue, and a JobRunner that takes any jobs from the queue and runs them. You need to both check the contents of the queue and add to the queue under the same exclusive lock.
public class JobManager {
private final Queue<Jobs> queue;
private final JobRunner jobRunner;
public JobManager() {
this.queue = new LinkedList<Jobs>();
this.jobRunner = new JobRunner(this);
jobRunner.start();
}
public synchronized void requestFirst() {
if (queue.isEmpty()) {
queue.add(Jobs.FIRST);
notifyAll();
}
}
public synchronized void requestSecond() {
if (!queue.contains(Jobs.SECOND)) {
queue.add(Jobs.SECOND);
notifyAll();
}
}
public synchronized Jobs getJob() throws InterruptedException {
while (queue.isEmpty()) {
wait();
}
return queue.peek();
}
public synchronized void notifyFinished(Jobs job) {
queue.remove(job);
}
public void startRunner() {
jobRunner.start();
}
public void stopRunner() {
jobRunner.stopRunner();
}
}
public class JobRunner extends Thread {
private final JobManager jobManager;
private volatile boolean stopping;
public JobRunner(JobManager jobManager) {
this.jobManager = jobManager;
this.stopping = false;
}
public void stopRunner() {
stopping = true;
this.interrupt();
}
#Override
public void run() {
while (!stopping) {
try {
Jobs job = jobManager.getJob();
if (job.equals(Jobs.FIRST)) {
// run first job
} else if (job.equals(Jobs.SECOND)) {
// run second job
}
jobManager.notifyFinished(job);
} catch (InterruptedException ignored) {}
}
}
}
public enum Jobs {
FIRST,
SECOND
}
#Controller
public class WebAccess {
private final JobManager jobManager;
public WebAccess() {
jobManager = new JobManager();
}
#Scheduled(cron = "0 * * * * *")
public void first() {
jobManager.requestFirst();
}
#RequestMapping(value = "/second", method = RequestMethod.POST)
public void second() {
jobManager.requestSecond();
}
#EventListener(ContextClosedEvent.class)
public void stopRunner() {
jobManager.stopRunner();
}
}
Unfortunately because of your complicated requirements for choosing first or second job, you need to hold a synchronization lock whilst checking the queue and deciding whether to add the job or not.
This kind of approach may be extremely difficult to test so probably it should be a last resort.
If you want to get Spring to do the autowiring you could annotate it accordingly.
I have a requirement in which, I will plan multiple tasks for certain interval. And for each task a notification has to be pushed at certain interval of time.
I have two approaches, Either to write a single scheduler which will poll at each one minute and push the notification accordingly. Or I can schedule initialize a scheduler for each task. With the former method the solution is pretty much simple, with the later I can get more control on the scheduler, for instance I can put initial delay for each task specifically(Which is a requirement), then stop an individual task, resume etc. So far I am proceeding with the latter method. But I would like to know If it could be better to use so many scheduler in a single application. Or is it better to go with a single scheduler with 1 minute polling?. On an average I will have about 200+ tasks live at a time. Or for this can I depend on any other library?
So far my code
Sheduler which is an ExecutorService
//Constructor
public TaskScheduler(String taskName) {
this.taskName = taskName;
this.taskResult = new TaskResult();
this.taskResult.setStartTime(getNewDate());
scheduledExecutorService = Executors.newScheduledThreadPool(1);
//DB Operation
}
// To stop an individual task
public TaskResult stop() throws InterruptedException {
try {
System.out.println("Stopping : " + this.taskName);
this.taskResult.setTaskName(this.taskName);
this.taskResult.setEndTime(new Date());
scheduledFuture.cancel(false);
scheduledExecutorService.shutdown();
//DB Operation
System.out.println("Stopping : finished - " + this.taskName + " # "+ new Date());
} catch (Exception e) {
e.printStackTrace();
}
return this.taskResult;
}
//Portion to add task
public TaskScheduler schedule(Runnable task, long initialDelay, long frequency) throws Exception{
this.taskResult.setFrequencyInSeconds(frequency);
scheduledFuture = scheduledExecutorService.scheduleAtFixedRate(task, initialDelay, frequency, TimeUnit.SECONDS);
return this;
}
Task thread with business logic
public class TaskModel implements Runnable {
private String taskName;
public TaskModel() {
}
public TaskModel(String taskName) {
this.taskName = taskName;
}
#Override
public void run() {
// DB operations
.
.
It is better to use a single scheduler, but you don't need to write your own scheduler for this. You could use single instance Executors.newScheduledThreadPool(1) with some number of threads to schedule all your tasks.
Think about the following code:
class TaskScheduler {
private ScheduledExecutorService scheduledExecutorService;
public TaskScheduler(int threads) {
this.scheduledExecutorService = Executors.newScheduledThreadPool(threads);
}
//Portion to add task
public TaskExecutionContext schedule(String taskName, Runnable task, long initialDelay, long frequency) {
TaskExecutionContext context = new TaskExecutionContext(taskName);
context.getTaskResult().setFrequencyInSeconds(frequency);
ScheduledFuture scheduledFuture = scheduledExecutorService.scheduleAtFixedRate(task, initialDelay, frequency, TimeUnit.SECONDS);
context.setScheduledFuture(scheduledFuture);
return context;
}
}
class TaskExecutionContext {
private String taskName;
private TaskResult taskResult;
private ScheduledFuture scheduledFuture;
public TaskExecutionContext(String taskName) {
this.taskName = taskName;
this.taskResult = new TaskResult();
this.taskResult.setTaskName(taskName);
this.taskResult.setStartTime(new Date());
//DB Operation on creation
}
public TaskResult stop() {
try {
System.out.println("Stopping : " + this.taskName);
this.taskResult.setTaskName(this.taskName);
this.taskResult.setEndTime(new Date());
scheduledFuture.cancel(false);
//DB Operation on stopping
System.out.println("Stopping : finished - " + this.taskName + " # " + new Date());
} catch (Exception e) {
e.printStackTrace();
}
return this.taskResult;
}
public TaskResult getTaskResult() {
return this.taskResult;
}
public void setScheduledFuture(ScheduledFuture scheduledFuture) {
this.scheduledFuture = scheduledFuture;
}
}
If you need some extra operations on scheduling, creation and stopping tasks - probably, it's better to have separate TaskExecutionContext, which will perform all your needs.
Create a context when you need to schedule a task and pass it to scheduler.
Is it possible to implement some kind of "preemptive" behavior linked to trigger priorities?
I mean, I want a high-priority trigger to interrupt the currently running low-priority job, and run in its place.
I'd like to go further and not just compare trigger priorities on the same job, but on different jobs trying to work on the same "resource", not at the same time but at overlapping times (assuming the "work" takes time to complete).
I didn't find anything "out of the box". Did anyone ever implement something similar?
This is my solution so far (imports removed). Any warnings or improvements?
//
// AN EXAMPLE JOB CLASS
//
public class DevJob implements InterruptableJob {
private final transient Logger log = LoggerFactory.getLogger(getClass());
AtomicReference<Thread> jobThreadHolder = new AtomicReference<Thread>();
#Override
public void execute(JobExecutionContext jobExecutionContext) throws JobExecutionException {
String triggerName = jobExecutionContext.getTrigger().getKey().toString();
String jobName = jobExecutionContext.getJobDetail().getKey().toString();
log.debug("Executing Job {}-{} ", triggerName, jobName);
jobThreadHolder.set(Thread.currentThread());
JobDataMap jobDataMap = jobExecutionContext.getJobDetail().getJobDataMap();
int duration = jobDataMap.getInt("duration");
try {
log.debug("Job {}-{} works for {}s...", triggerName, jobName, duration);
Thread.sleep(duration*1000);
} catch (InterruptedException e) {
log.debug("Job {}-{} interrupted", triggerName, jobName);
PreemptiveVolatileQueueQuartzListener.setInterrupted(jobExecutionContext);
} finally {
log.debug("Job {}-{} terminates", triggerName, jobName);
}
}
#Override
public void interrupt() throws UnableToInterruptJobException {
Thread thread = jobThreadHolder.getAndSet(null);
if (thread != null) {
thread.interrupt();
}
}
}
//
// IMPLEMENTATION FOR JOB PREEMPTION
//
/**
* This class implements a priority policy for jobs in the same thread group.
* When a new job starts, it will check if another job in the same thread group is already running.
* In such a case it compares trigger priorities. The job with lower priority is put in a wait queue for the thread group,
* but only if another instance with the same jobKey is not in the queue already.
* When the running job terminates, a new job is pulled from the queue based on priority and timestamp: for equal
* priorities, the older one is executed.
* If a job has been interrupted MAX_RESCHEDULINGS times, it will ignore any further interruptions.
* A job must implement InterruptableJob and periodically call checkInterruptRequested() if it can be interrupted by a
* higher priority job; it could ignore interruptions, in which case the higher priority job will execute only after
* its natural termination.
*/
public class PreemptiveVolatileQueueQuartzListener implements JobListener, TriggerListener {
private final transient Logger log = LoggerFactory.getLogger(getClass());
// The number of times that a low priority job can be preempted by any high priority job before it ignores preemption
private static final int MAX_RESCHEDULINGS = 20;
// This map holds the pointer to the current running job and its deferred queue, for a given thread group
private Map<String, RunningJobHolder> runningJobs = new HashMap<>(); // triggerGroup -> RunningJob
private static final String INTERRUPTED_FLAG = "PREEMPT_INTERRUPTED";
private static final String INTERRUPTREQUESTED_FLAG = "PREEMPT_INTERRUPTREQUESTED";
static final String JOB_ORIG_KEY = "PREEMPT_JOBORIGKEY";
/**
* Call this method to notify a job that an interruption has been requested. It should tipically be called
* in the InterruptableJob.interrupt() method. The job will then have to programmatically check this flag with checkInterruptRequested()
* and exit if the result is true.
*/
public final static void requestInterrupt(JobExecutionContext jobExecutionContext) {
jobExecutionContext.getJobDetail().getJobDataMap().put(INTERRUPTREQUESTED_FLAG, true);
}
/**
* Call this method in a job to check if an interruption has been requested. If the result is true, the "interrupted" flag
* will be set to true and the job should exit immediately
* because it will be rescheduled after the interrupting job has finished.
* #param jobExecutionContext can be null if the check should not be performed
* #return true if the interruption was requested
*/
public final static boolean checkInterruptRequested(JobExecutionContext jobExecutionContext) {
boolean result = false;
if (jobExecutionContext!=null) {
try {
result = jobExecutionContext.getJobDetail().getJobDataMap().getBoolean(INTERRUPTREQUESTED_FLAG);
} catch (Exception e) {
// Ignore, stay false
}
if (result) {
setInterrupted(jobExecutionContext);
}
}
return result;
}
/**
* Call this method in a job when catching an InterruptedException if not rethrowing a JobExecutionException
* #param jobExecutionContext
*/
public final static void setInterrupted(JobExecutionContext jobExecutionContext) {
jobExecutionContext.getJobDetail().getJobDataMap().put(INTERRUPTED_FLAG, true);
}
private final boolean isInterrupted(JobExecutionContext jobExecutionContext) {
try {
return true==jobExecutionContext.getJobDetail().getJobDataMap().getBoolean(INTERRUPTED_FLAG);
} catch (Exception e) {
return false;
}
}
private final void clearInterrupted(JobExecutionContext jobExecutionContext) {
jobExecutionContext.getJobDetail().getJobDataMap().remove(INTERRUPTREQUESTED_FLAG);
jobExecutionContext.getJobDetail().getJobDataMap().remove(INTERRUPTED_FLAG);
}
/**
* This method decides if a job has to start or be queued for later.
*/
#Override
public boolean vetoJobExecution(Trigger startingTrigger, JobExecutionContext startingJobContext) {
log.debug("Calculating veto for job {}", makeJobString(startingTrigger));
boolean veto = false;
String preemptedGroup = startingTrigger.getKey().getGroup();
synchronized (runningJobs) {
veto = calcVeto(startingTrigger, preemptedGroup, startingJobContext);
}
log.debug("veto={} for job {}", veto, makeJobString(startingTrigger));
return veto;
}
private boolean calcVeto(Trigger startingTrigger, String preemptedGroup, JobExecutionContext startingJobContext) {
final boolean VETO = true;
final boolean NOVETO = false;
int startingJobPriority = startingTrigger.getPriority();
RunningJobHolder runningJobHolder = runningJobs.get(preemptedGroup);
if (runningJobHolder==null) {
// No conflicting job is running - just start it
runningJobHolder = new RunningJobHolder();
runningJobs.put(preemptedGroup, runningJobHolder);
PrioritizedJob newJob = runningJobHolder.setActiveJob(startingJobPriority, startingTrigger, startingJobContext);
log.debug("Starting new job {} with nothing in the same group", newJob);
return NOVETO;
}
// Check that the current job isn't a job that has just been pulled from the queue and activated
boolean sameTrigger = startingTrigger.equals(runningJobHolder.activeJob.trigger);
if (sameTrigger) {
// runningJobHolder.activeJob has been set in triggerComplete but the trigger didn't fire until now
log.debug("Starting trigger {} is the same as the active one", startingTrigger.getKey());
return NOVETO;
}
// Check that the starting job is not already running and is not already queued, because we don't want
// jobs to accumulate in the queue (a design choice)
if (runningJobHolder.isInHolder(startingTrigger)) {
log.debug("Starting job {} is queued already (maybe with a different trigger)", makeJobString(startingTrigger));
return VETO;
}
// A job for this triggerGroup is already running and is not the same as the one trying to start and is not in the queue already.
// The starting job is therefore queued, ready to be started, regardless of the priority.
PrioritizedJob newJob = runningJobHolder.queueJob(startingJobPriority, startingTrigger, startingJobContext);
log.debug("New job {} queued", newJob);
printQueue(runningJobHolder); // Debug
if (startingJobPriority>runningJobHolder.activeJob.priority) {
if (runningJobHolder.activeJob.reschedulings >= MAX_RESCHEDULINGS) {
// When a job has been preempted too many times, it is left alone even if at lower priority
log.debug("New job {} does not interrupt job {} of lower priority because its reschedulings are {}", newJob, runningJobHolder.activeJob, runningJobHolder.activeJob.reschedulings);
} else {
// The starting job has a higher priority than the current running job, which needs to be interrupted.
// The new job will take the place of the running job later, because it has been added to the queue already.
// If the running job doesn't react to the interruption, it will complete normally and let the next job in
// the queue to proceed.
log.debug("New job {} interrupts job {} of lower priority", newJob, runningJobHolder.activeJob);
try {
Scheduler scheduler = startingJobContext.getScheduler();
scheduler.interrupt(runningJobHolder.getJobKey());
} catch (UnableToInterruptJobException e) {
log.error("Can't interrupt job {} for higher-priority job {} that will have to wait", runningJobHolder.activeJob, newJob);
}
}
}
return VETO;
}
/**
* The interrupt() method of a InterruptableJob should issue a thread.interrupt() on the job thread,
* and if not handled already, the resulting InterruptedException will be handled here.
*/
#Override
public void jobWasExecuted(JobExecutionContext context, JobExecutionException jobException) {
// Just in case a job throws InterruptedException when interrupted.
// Raise a flag if the job was interrupted with InterruptedException
if (jobException!=null && jobException.getCause() instanceof InterruptedException) {
PreemptiveVolatileQueueQuartzListener.setInterrupted(context);
}
}
// Debug method
private void printQueue(RunningJobHolder runningJobHolder) {
if (log.isDebugEnabled()) {
PriorityQueue<PrioritizedJob> clone = new PriorityQueue<PrioritizedJob>();
clone.addAll(runningJobHolder.jobQueue);
log.debug("Priority Queue: {}", clone.isEmpty()?"empty":"");
while (!clone.isEmpty()) {
PrioritizedJob job = clone.poll();
String jobKey = (String) job.trigger.getJobDataMap().getOrDefault(PreemptiveVolatileQueueQuartzListener.JOB_ORIG_KEY, job.trigger.getJobKey().toString());
log.debug("- {} [{}] reschedulings={}", job, jobKey, job.reschedulings);
}
}
}
// When a job finishes execution, a new one is started if the queue is not empty
private boolean startNextJobInQueue(PrioritizedJob terminatedJob, RunningJobHolder runningJobHolder, String preemptedGroup, Trigger usedTrigger, JobExecutionContext context) {
PrioritizedJob queuedJob = runningJobHolder.jobQueue.poll(); // Remove from queue
if (queuedJob!=null) { //
log.debug("Starting next job in queue {} after job {} finished", queuedJob, terminatedJob);
// The job must be cloned with a new trigger to execute immediately.
// Can't reuse the existing jobDetail with a new trigger because for some reason when the original trigger has
// finished all invocations, the following exception is thrown when trying to start the new trigger:
// org.quartz.JobPersistenceException: The job (xxx) referenced by the trigger does not exist.
JobDataMap jobDataMap = queuedJob.jobDetail.getJobDataMap();
JobDataMap triggerJobDataMap = queuedJob.trigger.getJobDataMap();
JobDataMap newTriggerJobDataMap = new JobDataMap(triggerJobDataMap);
// Need to store the original jobKey, used to check if a starting job is already in the queue. I can't use the normal
// jobKey because, when a job is cloned here, its key must be changed in order to store it without a "job already exists" exception.
String jobOrigKey = (String) triggerJobDataMap.getOrDefault(JOB_ORIG_KEY, queuedJob.jobDetail.getKey().toString());
newTriggerJobDataMap.put(JOB_ORIG_KEY, jobOrigKey);
JobDetail newJob = JobBuilder.newJob(queuedJob.jobDetail.getJobClass())
.withIdentity(makePreemptedId(queuedJob.jobDetail.getKey().getName()), queuedJob.jobDetail.getKey().getGroup())
.requestRecovery(queuedJob.jobDetail.requestsRecovery())
.storeDurably(queuedJob.jobDetail.isDurable())
.withDescription(queuedJob.jobDetail.getDescription())
.usingJobData(jobDataMap)
.build();
Trigger newTrigger = newTrigger()
.withPriority(queuedJob.priority)
.withIdentity(makePreemptedId(queuedJob.trigger.getKey().getName()), preemptedGroup)
.usingJobData(newTriggerJobDataMap)
.withDescription(queuedJob.trigger.getDescription())
.startNow()
.withSchedule(simpleSchedule()
// A misfire occurs if a persistent trigger “misses” its firing time because
// of the scheduler being shutdown, or because there are no available threads
.withMisfireHandlingInstructionFireNow() // (Not sure is correct)
)
.build();
try {
context.getScheduler().scheduleJob(newJob, newTrigger);
log.debug("Job {} from queue rescheduled to start now as {}", queuedJob, makeJobString(newTrigger));
queuedJob.reschedulings++;
queuedJob.trigger = newTrigger;
queuedJob.jobDetail = newJob;
runningJobHolder.activeJob = queuedJob;
return true;
} catch (SchedulerException e) {
log.error("Failed to start queued job {}", queuedJob, e);
runningJobHolder.activeJob = null;
return false;
}
}
return false;
}
private String makeJobString(Trigger trigger) {
StringBuilder sb = new StringBuilder(trigger.getKey().toString());
sb.append("-").append(trigger.getJobKey().toString());
return sb.toString();
}
// Each time a job is rescheduled with a new trigger, their names are changed to a (hopefully) unique string
private String makePreemptedId(String oldName) {
final String marker = "_p_r_e_";
long random = ThreadLocalRandom.current().nextLong(999888777L);
StringBuffer result = new StringBuffer(Long.toString(random)); // nnn
int pos = oldName.indexOf(marker);
if (pos>-1) {
result.append(oldName.substring(pos));
} else {
result.append(marker).append(oldName);
}
return result.toString();
}
// Called when a job finishes execution
#Override
public void triggerComplete(Trigger usedTrigger, JobExecutionContext context, CompletedExecutionInstruction completedExecutionInstruction) {
boolean interruptedJob = isInterrupted(context);
if (log.isDebugEnabled()) {
if (interruptedJob) {
log.debug("Interrupted job {}", makeJobString(usedTrigger));
} else {
log.debug("Terminated job {}", makeJobString(usedTrigger));
}
}
String preemptedGroup = usedTrigger.getKey().getGroup();
synchronized (runningJobs) {
RunningJobHolder runningJobHolder = runningJobs.get(preemptedGroup);
// Check that the activeJob is also the one that just terminated - for consistency
if (runningJobHolder==null || !runningJobHolder.getJobKey().equals(context.getJobDetail().getKey())) {
// Should never happen if there aren't any bugs
log.error("Internal Error: the job in triggerComplete {} is not the active job {} for group {} (skipping)",
makeJobString(usedTrigger),
runningJobHolder==null?null:runningJobHolder.activeJob,
preemptedGroup);
return;
}
printQueue(runningJobHolder);
PrioritizedJob terminatedJob = runningJobHolder.activeJob;
clearInterrupted(context);
runningJobHolder.activeJob = null;
// Start the next queued job if any. Do it in a loop because the next job might not start
// properly and this would otherwise prevent the other queued jobs from starting.
boolean started = false;
while (!started && !runningJobHolder.jobQueue.isEmpty()) {
started = startNextJobInQueue(terminatedJob, runningJobHolder, preemptedGroup, usedTrigger, context);
if (interruptedJob && (started || runningJobHolder.jobQueue.isEmpty())) {
// It was an interrupted lower-priority job, so put it in the queue
log.debug("Interrupted job {} added to job queue for rescheduling", terminatedJob);
runningJobHolder.addToQueue(terminatedJob);
interruptedJob=false;
}
}
printQueue(runningJobHolder);
if (runningJobHolder.jobQueue.isEmpty() && runningJobHolder.activeJob == null) {
// The current terminated job was the last one in the trigger group, so we can clean up
log.debug("Job {} ended with an empty proprity queue", terminatedJob);
runningJobs.remove(preemptedGroup);
}
}
}
#Override
public void triggerFired(Trigger trigger, JobExecutionContext context) {
}
#Override
public void triggerMisfired(Trigger trigger) {
}
#Override
public String getName() {
return this.getClass().getSimpleName();
}
#Override
public void jobToBeExecuted(JobExecutionContext context) {
}
#Override
public void jobExecutionVetoed(JobExecutionContext context) {
}
}
//
// RELATED CLASSES
//
/**
* A job with associated priority and timestamp
*
*/
class PrioritizedJob implements Comparable<PrioritizedJob> {
int priority;
long creationTimestamp = System.currentTimeMillis(); // To prevent same-priority elements to age
Trigger trigger;
JobDetail jobDetail; // Needed to create a new job when rescheduling after pulling from the queue
int reschedulings = 0; // Number of times the job has been put back in the queue because preempted by a higher-priority job
#Override
public int compareTo(PrioritizedJob o) {
// Smallest PrioritizedJob goes first, so priority check must be inverted because higher priority goes first
int comparison = -(new Integer(this.priority).compareTo(o.priority));
if (comparison==0) {
// lower timestamp is higher priority
comparison = new Long(this.creationTimestamp).compareTo(o.creationTimestamp);
}
return comparison;
}
#Override
public String toString() {
StringBuffer result = new StringBuffer();
result.append(trigger.getKey()).append("-").append(trigger.getJobKey()).append("(").append(priority).append(")");
return result.toString();
}
}
/**
* Holds the current running job definition for a given trigger group
*/
class RunningJobHolder {
PrioritizedJob activeJob; // The running job
// This queue holds all jobs of the same thread group that tried to start while this job was running.
// They are sorted by priority and timestamp.
// The head of the queue might contain a higher-priority job that has been put in the queue while waiting for
// the active job to handle the interruption
PriorityQueue<PrioritizedJob> jobQueue = new PriorityQueue<>();
JobKey getJobKey() {
return activeJob.trigger.getJobKey();
}
/**
* Create a new PrioritizedJob and set it as active
* #param startingJobPriority
* #param startingTrigger
* #param startingJobContext
* #return the new PrioritizedJob
*/
PrioritizedJob setActiveJob(int startingJobPriority, Trigger startingTrigger, JobExecutionContext startingJobContext) {
PrioritizedJob newJob = new PrioritizedJob();
newJob.priority = startingJobPriority;
newJob.trigger = startingTrigger;
newJob.jobDetail = startingJobContext.getJobDetail();
this.activeJob = newJob;
return newJob;
}
/**
* Create a new PrioritizedJob and add it to the queue
* #param startingJobPriority
* #param startingTrigger
* #param startingJobContext
* #return the new PrioritizedJob
*/
PrioritizedJob queueJob(int startingJobPriority, Trigger startingTrigger, JobExecutionContext startingJobContext) {
PrioritizedJob newJob = new PrioritizedJob();
newJob.priority = startingJobPriority;
newJob.trigger = startingTrigger;
newJob.jobDetail = startingJobContext.getJobDetail();
addToQueue(newJob);
return newJob;
}
/**
* Compares job keys, first by fetching the original job key stored in the trigger JobDataMap, then by using the job's own key
* #param trigger
* #param prioritizedJob
* #return
*/
private boolean equalKeys(Trigger trigger, PrioritizedJob prioritizedJob) {
String triggerJobKeyToCheck = (String) trigger.getJobDataMap().getOrDefault(PreemptiveVolatileQueueQuartzListener.JOB_ORIG_KEY, trigger.getJobKey().toString());
String prioritizedJobKeyToCheck = (String) prioritizedJob.trigger.getJobDataMap().getOrDefault(PreemptiveVolatileQueueQuartzListener.JOB_ORIG_KEY, prioritizedJob.trigger.getJobKey().toString());
return triggerJobKeyToCheck.equals(prioritizedJobKeyToCheck);
}
/**
* Check if the job in a trigger has already been queued (or is the active one) by comparing the job key
* #param trigger
* #return
*/
boolean isInHolder(Trigger trigger) {
if (equalKeys(trigger, activeJob)) {
return true;
}
for (PrioritizedJob prioritizedJob : jobQueue) {
if (equalKeys(trigger, prioritizedJob)) {
return true;
}
}
return false;
}
void addToQueue(PrioritizedJob prioritizedJob) {
jobQueue.add(prioritizedJob);
}
}
I'm struggling with Quartz's triggers. I've created a service which is invoked through a cron expression but also it's a org.quartz.Job itself, the idea behind this is that once the service is invoked I check some variables and based on that I decide to delay (or not) the execution of the service for 1 day; for that, I get the DefaultScheduler, create a new org.quartz.JobDetail and a org.quartz.SimpleTrigger, and schedule them. As you may guess, it's not working, the new trigger is never started.
public class MyService implements MyInterface, org.quartz.Job {
#Override
//method declared in MyInterface
public void generateFile() {
org.quartz.Scheduler scheduler;
try {
scheduler = org.quartz.impl.StdSchedulerFactory.getDefaultScheduler();
} catch (SchedulerException e1) {
scheduler = null;
}
if (condition) {
//regular processing
} else {
try {
org.quartz.JobDetail jobDetail;
SimpleTrigger simpleTrigger;
String name = "jobName";
if (scheduler != null && scheduler.isInStandbyMode()){
jobDetail = new org.quartz.JobDetail(name, org.quartz.Scheduler.DEFAULT_GROUP, MyService.class);
jobDetail.getJobDataMap().put("myService", this);
Calendar c = Calendar.getInstance();
int date = c.get(Calendar.DATE);
c.set(Calendar.DATE, date + 1);
simpleTrigger = new SimpleTrigger("simpleTrig", c.getTime());
simpleTrigger.setJobName(name);
jobDetail.setDurability(true);
scheduler.addJob(jobDetail, true);
scheduler.scheduleJob(simpleTrigger);
//also tested with
//scheduler.scheduleJob(jobDetail, simpleTrigger);
//and same result
scheduler.start();
}
else{
scheduler.shutdown();
}
} catch (java.text.ParseException e) {
}
catch (org.quartz.SchedulerException ex) {
}
}
}
#Override
public void execute(JobExecutionContext jobContext) throws JobExecutionException {
MyInterface jobParamLogService = (MyInterface)jobContext.getJobDetail().getJobDataMap().get("myService");
jobParamLogService.generateFile();
}
As i said, my problem is that the SimpleTrigger never gets invoked (off course, I've tested adding a few minutes, not a day), does any have any tip about what's going on? Any help would be really appreciated.
As #GuillaumePolet suggested, I've tried removing the isInStandByMode condition, but still didn't work. The trick was to replace the constructor with arguments for the default one (empty), and set the variables through the setter methods.Thank you all for your help, kind regards
I have a Java application that makes use of a Quartz Scheduler in the guise of a SchedulerFactoryBean. The main() method gets the application context, retrieves the root bean, and commences scheduling jobs.
The issue is that the Scheduler runs in its own thread, so when the main thread is done submitting jobs, it returns and the Scheduler goes on without it. When the Scheduler is finally done (or even if you explicitly call shutdown() on it), the application just hangs there for all eternity.
I have two solutions:
Keep track of the job/trigger count, incrementing it whenever you add a job to the Scheduler. Attach a simple SchedulerListener to the Scheduler that decrements this count with every call to triggerFinalized(), and set up a while loop with a Thread.sleep() inside it that constantly checks to see if the count has hit 0. When it does, it will return up to the main() method and the application will exit normally.
Take the custom SchedulerListener from option 1, and keep track of the job count inside of it. Increment for every call to jobAdded(), and decrement for every call to triggerFinalized(). When the count hits 0, call shutdown() on the Scheduler (or not, it doesn't actually matter) and then call System.exit(0).
I have implemented both of these independently in turn, so I know they both actually function. The problem is that they are both terrible. An infinite while loop polling a value? System.exit(0)? Bleargh.
Does someone have a better way, or are these seriously my only options here?
Edit: While thinking about this on the way home, I came to the conclusion that this may be caused by the fact that I'm using SchedulerFactoryBean. This auto-starts when Spring initializes the application context - that seems to put it outside the scope of the main thread. If I went with a slightly different Scheduler that I manually initialized and called start() on in the code, would this run the Scheduler in the main thread, thus blocking it until the Scheduler completes running all jobs? Or would I still have this problem?
Edit: Son of a...http://quartz-scheduler.org/documentation/quartz-2.x/examples/Example1
To let the program have an opportunity to run the job, we then sleep for 90 seconds. The scheduler is running in the background and should fire off the job during those 90 seconds.
Apparently, that will not work, because the scheduler seems to always run in the background.
In your SchedulerListener add an object solely for synchronization and locking. Call it exitLock or something. You main thread retrieves the scheduler, sets up the listener, submits all the jobs and then just before returning executes
Object exitLock = listener.getExitLock();
synchronized (exitLock) {
exitLock.wait(); // wait unless notified to terminate
}
On every triggerFinalized() call your listener decrements the counter for pending jobs. Once all the jobs have finished executing your listener shuts the scheduler down.
if (--pendingJobs == 0)
scheduler.shutdown(); // notice, we don't notify exit from here
Once the scheduler shuts down it invokes one last callback on the listener where we notify the main thread to terminate and hence the program exits gracefully.
void schedulerShutdown() {
// scheduler has stopped
synchronized (exitLock) {
exitLock.notify(); // notify the main thread to terminate
}
}
The reason we didn't notify in triggerFinalized() when all the pending jobs were finished is that in case the scheduler was shutdown prematurely and not all the jobs were finished we would have left our main thread hanging. By notifying in response to the shutdown event we make sure our program exits successfully.
I think here can be another solution.
Key points:
When task was executed the last time context.getNextFireTime() returns null.
Scheduler.getCurrentlyExecutingJobs == 1 indicate that it is the last executed job.
So when point 1 and 2 is true we can shutdown Scheduler and call System.exit(0).
Here is the code:
Listener
public class ShutDownListenet implements JobListener {
#Override
public String getName () { return "someName"; }
#Override
public void jobToBeExecuted (JobExecutionContext context) {}
#Override
public void jobExecutionVetoed (JobExecutionContext context) {}
#Override
public void jobWasExecuted (JobExecutionContext context, JobExecutionException jobException) {
try {
if (context.getNextFireTime() == null && context.getScheduler().getCurrentlyExecutingJobs().size() == 1) {
context.getScheduler().shutdown();
System.exit(0);
}
} catch (SchedulerException e) {
e.printStackTrace();
}
}
}
Code in the main function
public static void main (String[] args) {
Trigger trigger = ...
Job job = ...
JobListener listener = new ShutDownListenet();
scheduler.getListenerManager().addJobListener(listener);
scheduler.scheduleJob(job, trigger);
}
NOTE
I do not write synchronized blocks, but I tested this code with 100 concurent jobs, it works.
Did not tested in "complex" enviroment: clusters or RMI. (behavior can be differ).
Any comments are wellcome.
If your Quartz schedules/triggers are based on the database then you program needs to be alive till you would want to stop it. This can be doable like below. The idea is hook SchedulerListener and wait in the main thread. You need to hook your own way to terminate the program gracefully which completely a different topic itself.
public static void main(String[] args) {
AnnotationConfigApplicationContext appContext = // initialize the your spring app Context
// register the shutdown hook for JVM
appContext.registerShutdownHook();
SchedulerFactoryBean schedulerFactory = appContext.getBean(SchedulerFactoryBean.class);
scheduler = schedulerFactory.getScheduler();
final Lock lock = new ReentrantLock();
final Condition waitCond = lock.newCondition();
try {
scheduler.getListenerManager().addSchedulerListener(new SchedulerListener() {
#Override
public void jobAdded(JobDetail arg0) {
}
#Override
public void jobDeleted(JobKey arg0) {
}
#Override
public void jobPaused(JobKey arg0) {
}
#Override
public void jobResumed(JobKey arg0) {
}
#Override
public void jobScheduled(Trigger arg0) {
}
#Override
public void jobUnscheduled(TriggerKey arg0) {
}
#Override
public void jobsPaused(String arg0) {
}
#Override
public void jobsResumed(String arg0) {
}
#Override
public void schedulerError(String arg0, SchedulerException arg1) {
}
#Override
public void schedulerInStandbyMode() {
}
#Override
public void schedulerShutdown() {
lock.lock();
try {
waitCond.signal();
}
finally {
lock.unlock();
}
}
#Override
public void schedulerShuttingdown() {
}
#Override
public void schedulerStarted() {
}
#Override
public void schedulerStarting() {
}
#Override
public void schedulingDataCleared() {
}
#Override
public void triggerFinalized(Trigger arg0) {
}
#Override
public void triggerPaused(TriggerKey arg0) {
}
#Override
public void triggerResumed(TriggerKey arg0) {
}
#Override
public void triggersPaused(String arg0) {
}
#Override
public void triggersResumed(String arg0) {
}
});
// start the scheduler. I set the SchedulerFactoryBean.setAutoStartup(false)
scheduler.start();
lock.lock();
try {
waitCond.await();
}
finally {
lock.unlock();
}
} finally {
scheduler.shutdown(true);
}
}
If it helps someone else. I solved this by adding a shutdown-hook that triggers on Ctrl-C or normal kill (15) from script. A new Thread is spawned and polls the getCurrentlyExecutingJobs().size() every 3 seconds and exits when jobs counter has reached zero meaning all jobs finished.
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
try {
while (jobScheduler.getScheduler().getCurrentlyExecutingJobs().size() > 0) {
Thread.sleep(3000);
}
jobScheduler.getScheduler().clear();
} catch (Exception e) {
e.printStackTrace();
}
}));
while (!scheduler.isShutdown())
{
Thread.sleep(2L * 1000L);//Choose reasonable sleep time
}