I am using a Quartz Job for executing specific tasks.
I am also scheduling its execution in my Main application class and what i am trying to accomplish is not to allow simultaneous instances of this job to be executed.
So the scheduler should only execute the job if its previous instance is finished.
Here is my Job class:
public class MainJob implements Job {
static Logger log = Logger.getLogger(MainJob.class.getName());
#Override
public void execute(JobExecutionContext arg0) throws JobExecutionException {
GlobalConfig cfg = new GlobalConfig();
ProcessDicomFiles processDicomFiles = new ProcessDicomFiles();
ProcessPdfReportFiles processPdf = new ProcessPdfReportFiles();
try {
log.info("1. ---- SCHEDULED JOB -- setStudiesReadyToProcess");
processDicomFiles.setStudiesReadyToProcess();
log.info("2. ---- SCHEDULED JOB --- distributeToStudies");
processDicomFiles.distributeToStudies(cfg.getAssocDir());
...
//process any incoming PDF file
log.info("11. ---- SCHEDULED JOB --- processPdfFolder");
processPdf.processPdfFolder();
} catch (Exception ex) {
Logger.getLogger(FXMLDocumentController.class.getName()).log(Level.ERROR, null, ex);
}
log.info(">>>>>>>>>>> Scheduled Job has ended .... <<<<<<<<<<<<<<<<<<<<");
}
}
So in my application's main class i am starting the scheduler:
...
//start Scheduler
try {
startScheduler();
} catch (SchedulerException ex) {
log.log(Level.INFO, null, ex);
}
...
public void startScheduler () throws SchedulerException {
//Creating scheduler factory and scheduler
factory = new StdSchedulerFactory();
scheduler = factory.getScheduler();
schedulerTimeWindow = config.getSchedulerTimeWindow();
JobDetailImpl jobDetail = new JobDetailImpl();
jobDetail.setName("First Job");
jobDetail.setJobClass(MainJob.class);
SimpleTriggerImpl simpleTrigger = new SimpleTriggerImpl();
simpleTrigger.setStartTime(new Date(System.currentTimeMillis() + 1000));
simpleTrigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
simpleTrigger.setRepeatInterval(schedulerTimeWindow);
simpleTrigger.setName("FirstTrigger");
//Start scheduler
scheduler.start();
scheduler.scheduleJob(jobDetail,simpleTrigger);
}
I would like to prevent scheduler from starting a second MainJob instance if another one is still running ...
Just use the #DisallowConcurrentExecution Annotation on top of the Job class.
See this official example or this tutorial about concurrent job execution.
#DisallowConcurrentExecution can do your job but please consider that it would only prevent your class from being run twice on the same node.
Please see #ReneM comment in Quartz 2.2 multi scheduler and #DisallowConcurrentExecution
You can implement StatefulJob or annotate DisallowConcurrentExecution
Be cereful when execute from Job with #DisallowConcurrentExecution method with #Async
The second thread may conflict with the first
Related
I have a scheduler which picks up the jobs at regular interval of times, each job takes long time to process. While a job is processing, we send heartbeat which updates the last_update_time column in database on a background thread every 5mins until the job is completely finished.
Question: If runtime exception is thrown on background thread while sending heartbeat, how to notify the scheduler thread which is processing a job in runJob method to abort current job immediately and move to next job in the jobs list?
JobRunner.class
#Scheduled
public void pipelineRunner() {
List<Job> jobs = getJobs(); //fetches tasks from database
for(Job job: jobs){
try{
heartBeatSender.startHeartBeat(job); // updates last_update_time column for the current job in database on background thread
runJob(job); // contains logic of processing a job, how to abort current job it is processing if heartbeat sender throws a runtime exception or how to notify or make scheduler thread throw exception so that it can move to next job in the list?
}catch(Exception e){}
finally{
heartBeatSender.stopHeartBeat(); //stops updating last_update_time col.
}
}
}
HeartBeatSender.class
public void startHeartBeat(Job job) {
Runnable heartBeat = () -> {
if (job != null) {
job.setLastUpdateTime(System.currentTimeMillis());
jobRepository.save(job);//runtime exception can be thrown here.
}
}
executor = Executors.newSingleThreadScheduledExecutor();
executor.scheduleAtFixedRate(heartBeat, 0L, heartBeatInterval, TimeUnit.MILLISECONDS); //heartBeatInterval every 5mins
}
The code you posted is a bit inconsistent in that it seems the HeartbeatSender can send heartbeats to several jobs, but the client code always only triggers one. However, if you have another class also using this HeartbeatSender, you will have scheduled two heartbeat runnables, so how would it decide which one stopHeartbeat() will cancel? Not to mention I wouldn't be able to tell how it cancels it in the first place, because it doesn't keep track of the scheduled task beyond the startHeartbeat() call.
So what you may want to do is to have HeartbeatSender track its heartbeats by job, so it can cancel them individually.
class HeartbeatSender {
private final Map<Job, ScheduledFuture<?>> heartbeats = new HashMap<>();
// you can create a new executor in start, but I don't see why
private final ScheduledExecutorService heartbeatExecutor = newSingleThreadScheduledExecutor();
public void startHeartbeat(Job job) {
Runnable heartBeat= ...;
ScheduledFuture<?> schedule = executor.scheduleAtFixedRate(heartBeat, 0L, heartBeatInterval, TimeUnit.MILLISECONDS);
heartbeats.put(job, schedule); // remember so you can cancel later
}
public void stopHeartbeat(Job job) {
ScheduledFuture<?> schedule = heartbeats.remove(job);
if (schedule != null) {
schedule.cancel(false);
}
}
}
Now you can stop heartbeats by job, so you just have to do that in you client code.
for(Job job: jobs){
try {
heartBeatSender.startHeartBeat(job);
runJob(job);
} catch(Exception e) {
} finally{
heartBeatSender.stopHeartBeat(job); // stops heartbeat for this job
}
}
I have an implementation like this which doesn't work. As you see, job takes ~5sec and should run on fixedRate 1sec. That means there should be ~5jobs running in parallel but Spring wait to finish a job before starts another one...
If I add second #Scheduled job 'schedule2' with the same and parameters, I have 2 different jobs running in parallel but never the same job. Is it somehow possible to achieve this?
#Scheduled(fixedRate = 1000)
private void schedule1() {
int index = atomicInteger1.addAndGet(1);
logger.info("Run Schedule1 nr.{} started at: {}", index, LocalDateTime.now());
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
logger.info("Schedule1 nr.{} finished at: {}", index, LocalDateTime.now());
}
}
#Bean(destroyMethod = "shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(10);
}
Each scheduled task will never run in parallel in this case. That's because the task takes longer than the given fixedRate. Why? Because ScheduledExecutorService#scheduleAtFixedRate is called, and as the documentation says (bolded is mine):
... If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute.
One way of solving this is by using #Async and #EnableAsync. Many examples are available in the Spring docs:
#EnableAsync
public class Example {
#Async
#Scheduled(fixedRate = 1000)
public void schedule1() throws InterruptedException {
Thread.sleep(5000);
}
}
If you really want to achieve what you want, you should manage the threads by your own, calling to a service from the job in a separate thread.. but I don't see a reason to do that at least you're only testing and playing with Jobs at home for pet projects.
Anyway, have a look at this:
https://www.baeldung.com/java-future
I have the following method that builds a new JobDetail instance;
private JobDetail getJob(JobID jobID) throws SchedulerException {
Class<? extends Job> jobClass = jobID.getJobClass();
if(jobClass != null){
return JobBuilder
.newJob(jobClass)
.withIdentity(jobID.jobName(), jobID.jobGroup())
.requestRecovery(true)
.build();
}
return null;
}
Note: JobID is just a helper class of mine that provides all the necessary raw data required to build a JobDetail
Then another method that does the actual scheduling;
doSchedule(Scheduler scheduler, JobDetail job, String triggerName){
Trigger trigger = buildSomeTrigger(triggerName);
//check whether job exists in scheduler instance
if(scheduler.checkExists(job.getKey())){
//If trigger also exists, then it probably holds new info
//So, reschedule the existing job with the trigger
if(scheduler.checkExists(trigger.getKey())){
System.out.println("update job with Trigger");
job = job.getJobBuilder().storeDurably(true).build();
scheduler.addJob(job, true);
Trigger oldTrigger = scheduler.getTrigger(trigger.getKey());
scheduler.rescheduleJob(oldTrigger.getKey(), trigger);
}else{
//If trigger does not exist, it's an entirely new trigger
//Add the new trigger to the existing job
System.out.println("save new trigger for job");
trigger = trigger.getTriggerBuilder().forJob(job).build();
scheduler.scheduleJob(trigger);
}
}else{
//If job does not exist, schedule new job with the trigger
System.out.println("schedule new job with trigger");
scheduler.scheduleJob(job, trigger);
}
}
EDIT: Here's what getNewSchedulerInstance() looks like;
private Scheduler getNewSchedulerInstance() {
Scheduler scheduler = null;
try {
scheduler = new StdSchedulerFactory("quartz.properties").getScheduler();
scheduler.setJobFactory(cdiJobFactory); //<== Utilizes the JobFactory implementation specified in the CdiJobFactory.java class
scheduler.getListenerManager().addJobListener(new SimpleJobListener());
scheduler.getListenerManager().addTriggerListener(new SimpleTriggerListener());
} catch (SchedulerException ex) {
System.out.println("Scheduler instantiation failed!");
ex.printStackTrace();
}
return scheduler;
}
Then finally in another method, I have the following;
//This returns a new instance of Scheduler
Scheduler scheduler = getNewSchedulerInstance();
if(scheduler != null){
JobID jobID = new ExtendedJobID(); // <== extends JobID
JobDetail job = getJob(jobID);
doSchedule(scheduler, job);
}
Normally I expect this to print;
schedule new job with trigger
But instead it's printing;
save new trigger for job
EDIT: And hence throws the following exception;
org.quartz.JobPersistenceException: Couldn't store trigger 'GOALS.ACTIVATE_GOALS_0:0' for 'GOALS.ACTIVATE_GOALS' job:
The job (GOALS.ACTIVATE_GOALS) referenced by the trigger does not exist.
[See nested exception: org.quartz.JobPersistenceException: The job (GOALS.ACTIVATE_GOALS) referenced by the trigger does not exist.]
This suggests that the scheduler.checkExists(job.getKey()) returns true
scheduler.checkExists(trigger.getKey()) returns false
Yet exception says that The job (*jobName*) referenced by the (*trigger*) does not exist
So, this issuse only gets me confused as it seems that scheduler.checkExists(...) is failing to appropriately check the existence of the job or not in the scheduler
The Exception shouldn't even get thrown if the check did what I expect it to do;
But as you can see, the JobDetail is newly built and has not in any way been previously associated with the scheduler...
So I'm wondering whether i'm missing some Quartz Scheduler points?...
If so, can someone please explain what I might be doing wrong here?...
Or could this actually be a bug?...
Are there any work-arounds?
I have a started scheduler and have one job running on it. This is the main scheduler thread I shall be running.
public class MyApp {
Scheduler scheduler1;
public static void main(String[] args) {
run();
}
public static void run(){
try {
JobDetail job = JobBuilder.newJob(Job.class)
.withIdentity("JoeyJob", "group1").build();
Trigger trigger1 = TriggerBuilder.newTrigger()
.withIdentity("cronTrigger1", "group1")
.withSchedule(CronScheduleBuilder.cronSchedule("0/10 * * * * ?"))
.build();
scheduler1 = new StdSchedulerFactory().getScheduler();
scheduler1.start();
System.out.println(scheduler1.getSchedulerInstanceId());
scheduler1.scheduleJob(job,trigger1);
Thread.sleep(1000000);
scheduler1.shutdown();
}
catch(Exception e){
e.printStackTrace();
}
}
I wish to run another another job with a trigger on the very same scheduler but I need to access it from a different java executable using probably the scheduler name or any such parameter. I realize that the scheduler name returns something like 'defaultScheduler' and the Instance ID returns 'NON_CLUSTERED' I need to develop an application to run a single scheduler thread and constantly add/remove update jobs of sending emails. As this will be initialized used by a servlet. Is there a way I can access this scheduler from the memory from a different executable instance. This is what I am looking for.
public class Test {
public static void main(String[] args) throws SchedulerException {
run();
}
public static void run()throws SchedulerException{
JobDetail job = JobBuilder.newJob(Job2.class)
.withIdentity("Jake", "group2").build();
Trigger trigger1 = TriggerBuilder.newTrigger()
.withIdentity("cronTrigger2", "group1")
.withSchedule(CronScheduleBuilder.cronSchedule("0/2 * * * * ?"))
.build();
Scheduler scheduler= new StdSchedulerFactory().getScheduler("scheduler-name-something");
scheduler.scheduleJob(job,trigger1);
}
}
Is there a way to use the Scheduler Instance ID and the Scheduler Name to do this?
I checked the documentation, there is no way to do what I was looking for.
JobDetail job = new JobDetail();
job.setName("dummyJ");
job.setJobClass(NotificationCreater.class);
SimpleTrigger trigger = new SimpleTrigger();
trigger.setName("mn");
trigger.setStartTime(new Date(System.currentTimeMillis() + 1000));
trigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
trigger.setRepeatInterval(30000);
Scheduler scheduler = new StdSchedulerFactory().getScheduler();
scheduler.start();
scheduler.scheduleJob(job, trigger);
i am using above code to schedule my activity in NotificationCreater.class but i get error message
error:-Unable to store Job with name: 'dummyJ' and group: 'DEFAULT', because one already exists with this identification.
You can use the init method in Servlet to initialise and start of the schedule. You should also use the destroy method in Servlet to remove the scheduled job from the pool once you application is removed to avoid the same error happening during re-deployment. You can do something like scheduler.unscheduleJob() and scheduler.shutdown() to remove the job and stop the scheduler from destroy method.
If using servlets, and want to run your job on application startup, I guess this is how you should proceed to achieve.
The Job Class
public class DummyJob{
public DummyJob() throws ParseException, SchedulerException {
JobDetail job = new JobDetail();
job.setName("dummyJ");
job.setJobClass(NotificationCreater.class);
SimpleTrigger trigger = new SimpleTrigger();
trigger.setName("mn");
trigger.setStartTime(new Date(System.currentTimeMillis() + 1000));
trigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
trigger.setRepeatInterval(30000);
Scheduler scheduler = new StdSchedulerFactory().getScheduler();
scheduler.start();
scheduler.scheduleJob(job, trigger);
}
}
The servlet
public class JobInitializerServlet extends HttpServlet {
/**
*
*/
private static final long serialVersionUID = 5102955939315248840L;
/**
* Application logger to log info, debug, error messages.
*/
private static final Logger APP_LOGGER = Logger.getLogger("appLogger");
/**
* #see Servlet#init(ServletConfig) Initializes DummyJob
*/
public void init(ServletConfig config) throws ServletException {
try {
DummyJob scheduler = new DummyJob();
} catch (java.text.ParseException e) {
APP_LOGGER.error(e.getLocalizedMessage(), e);
} catch (SchedulerException e) {
APP_LOGGER.error(e.getLocalizedMessage(), e);
}
}
}
And servlet Mapping
<servlet>
<description>
</description>
<display-name>JobInitializerServlet</display-name>
<servlet-name>JobInitializerServlet</servlet-name>
<servlet-class>com.job.servlet.JobInitializerServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
This will initiate the job as soon as you deploy or start your application. Hope this helps.
trigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
The trigger is being set for the indefinite repeat counts.
Meaning, the trigger will be there in database forever.
And as a result, the job associated with the trigger would also exist in database
forever.
So, you executed your program for the first time and become glad to see it running.
You stopped the execution and had a coffee break.
You then come back and want to show this to your manager and ##$%## BOOM #$%#$%#$5.
You trying to create the job and trigger with the name which are already in
database. And scheduler will offcourse prevents you from doing this.
Solutions :
Wipe out all the data from the quartz database tables before you start the next execution of
program. OR
Don't use an indefinite trigger . Use a simple one . A one time execution or two or three but not ~. OR
Use RAMJobStore.