I have a Java program that executes from Spring Qquartz every 20 seconds. Sometimes it takes just few seconds to execute, but as data gets bigger I'm sure it run for 20 seconds or more.
How can I prevent Quartz from firing/triggering the job while one instance is still being executed? Firing 2 jobs performing same operations on a database would not be so good. Is there a way I can do some kind of synchronization?
Quartz 1
If you change your class to implement StatefulJob instead of Job, Quartz will take care of this for you. From the StatefulJob javadoc:
stateful jobs are not allowed to
execute concurrently, which means new
triggers that occur before the
completion of the execute(xx) method
will be delayed.
StatefulJob extends Job and does not add any new methods, so all you need to do to get the behaviour you want is change this:
public class YourJob implements org.quartz.Job {
void execute(JobExecutionContext context) {/*implementation omitted*/}
}
To this:
public class YourJob implements org.quartz.StatefulJob {
void execute(JobExecutionContext context) {/*implementation omitted*/}
}
Quartz 2
In version 2.0 of Quartz, StatefulJob is deprecated. It is now recommended to use annotations instead, e.g.
#DisallowConcurrentExecution
public class YourJob implements org.quartz.Job {
void execute(JobExecutionContext context) {/*implementation omitted*/}
}
If all you need to do is fire every 20 seconds, Quartz is serious overkill. The java.util.concurrent.ScheduledExecutorService should be perfectly sufficient for that job.
The ScheduledExecutorService also provides two semantics for scheduling. "fixed rate" will attempt to run your job every 20 seconds regardless of overlap, whereas "fixed delay" will attempt to leave 20 seconds between the end of the first job and the start of the next. If you want to avoid overlap, then fixed-delay is safest.
Just in case anyone references this question, StatefulJob has been deprecated. They now suggest you use annotations instead...
#PersistJobDataAfterExecution
#DisallowConcurrentExecution
public class TestJob implements Job {
This will explain what those annotations mean...
The annotations cause behavior just as
their names describe - multiple
instances of the job will not be
allowed to run concurrently (consider
a case where a job has code in its
execute() method that takes 34 seconds
to run, but it is scheduled with a
trigger that repeats every 30
seconds), and will have its JobDataMap
contents re-persisted in the
scheduler's JobStore after each
execution. For the purposes of this
example, only
#PersistJobDataAfterExecution
annotation is truly relevant, but it's
always wise to use the
#DisallowConcurrentExecution
annotation with it, to prevent
race-conditions on saved data.
if you use spring quartz, i think you have to configure like this
<bean id="batchConsumerJob"class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="myScheduler" />
<property name="targetMethod" value="execute" />
<property name="concurrent" value="false" />
</bean>
I'm not sure you want synchronisation, since the second task will block until the first finishes, and you'll end up with a backlog. You could put the jobs in a queue, but from your description it sounds like the queue may grow indefinitely.
I would investigate ReadWriteLocks, and let your task set a lock whilst it is running. Future tasks can inspect this lock, and exit immediately if an old task is still running. I've found from experience that that's the most reliable way to approach this.
Perhaps generate a warning as well so you know you're encountering problems and increase the time interval accordingly ?
put them in a queue
Even if the time exceeds 20 second current job should be finished & then the next should be fetched from the queue.
Or you can also increase time to some reasonable amount.
You can use a semaphore. When the semaphore is taken, abandon the 2nd job and wait until the next fire time.
Related
I need to implement a Quartz job that behaves differently depending on how many times it has been run.
What would you say is the best (or at least a good) way to do that ?
Keeping a counter as state doesn't seem to work since it looks like Quartz recreates it's jobs every time.
Thanks.
This is what the Quartz PersistJobDataAfterExecution annotation allows you to easily implement without using any extra framework / logic.
From PersistJobDataAfterExecution (Quartz 2.2.0) javadoc:
An annotation that marks a Job class as one that makes updates to its
JobDataMap during execution, and wishes the scheduler to re-store the
JobDataMap when execution completes.
So when your job starts you simply read your job execution counter from the JobDataMap that is passed to the job's execute method. If the counter is not present in the JobDataMap, you initialize it with 0. At the end of your job execution you increment the counter in the JobDataMap. This updated counter value will then be available during the next job execution.
I am using Quartz for scheduling parallel tasks, How can I get job running time in Quartz?
JobExecutionContext expose a some useful methods:
getJobRunTime: returns the time only after the job has actually completed
(you may want to use a JobListener to call it when job finished the
execution).
getFireTime: get the actual time the job started, so you can the current Date to calculate the elapsed time (you can call this method even inside the Job itself).
Note: To know "how long it WILL takes to run one job" you have to implement on your own doing some simple math to get the % of completion. Quartz itself doesn't have such a feature.
I have a number of tasks that I would like to execute periodically at different rates for most tasks. Some of the tasks may be scheduled for simultaneous execution though. Also, a task may need to start executing while another is currently executing.
I would also like to customize each task by setting an object for it, on which the task will operate while it is being executed.
Usually, the tasks will execute in periods of 2 to 30 minutes and will take around 4-5 seconds, sometimes up to 30 seconds when they are executed.
I've found Executors.newSingleThreadedScheduledExecutor(ThreadFactory) to be almost exactly what I want, except that it might cause me problems if a new task happens to be scheduled for execution while another is already executing. This is due to the fact that the Executor is backed up by a single execution thread.
The alternative is to use Executors.newScheduledThreadPool(corePoolSize, ThreadFactory), but this requires me to create a number of threads in a pool. I would like to avoid creating threads until it is necessary, for instance if I have two or more tasks that happen to need parallell executing due to their colliding execution schedules.
For the case above, the Executors.newCachedThreadPool(ThreadFactory) appears to do what I want, but then I can't schedule my tasks. A combination of both cached and scheduled executors would be best I think, but I am unable to find something like that in Java.
What would be the best way to implement the above do you think?
Isn't ScheduledThreadPoolExecutor.ScheduledThreadPoolExecutor(int):
ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(0);
what you need? 0 is the corePoolSize:
corePoolSize - the number of threads to keep in the pool, even if they are idle, unless allowCoreThreadTimeOut is set
I guess you will not able to do that with ScheduledExecutor, because it uses DelayedWorkQueue where as newCachedThreadPool uses ThreadPoolExecutor SynchronousQueue as a work queue.
So you can not change implementation of ScheduledThreadPoolExecutor to act like that.
I need to perform a task every few hours and I'm looking for most efficient solution for that. I thought about two approaches:
1) busy waiting
while(true){
doMyJob()
wait(2*hour);
}
2) executor scheduling:
executor.schedule(new MyJobTask(),2,TimeUnit.HOUR);
...
class MyJobTask implements Runnable{
void run(){
doMyJob();
...
executor.schedule(new MyJobTask(),2,TimeUnit.HOUR);
}
Could you please advise me which solution is more efficient and in what situation each of them is more preferable (if any). Intuitively, I would go for second solution but I couldn't find anything to prove my intuition. If you have some other solutions - please share. Solution should be also memory efficient (that's why I have a dilemma - do I need to create and keep a ThreadPool object only to do a simple job every two hours).
None of the proposed solutions is really advisable inside an EE container (where you should avoid messing with threads), which you seem to target according to the tags of your question.
Starting with Java EE 5 there is the timer service which according to my tests works quite nicely with longer timeouts like the 2 hours in your example. There is one point that you really shouldn't forget though - quoting from the aforementioned tutorial:
Timers are persistent. If the server is shut down (or even crashes), timers are saved and will become active again when the server is restarted. If a timer expires while the server is down, the container will call the #Timeout method when the server is restarted.
If for whatever reason this solution is not acceptable you should have a look at the Quartz Scheduler. Its possibilities exceed your requirements by far, but at least it gives you a ready to use solution whose compatibility with a wide range of application servers is guaranteed.
Both should have about the same efficiency but I would suggest using ScheduledExecutorService
Executors.newSingleThreadScheduledExecutor()
.scheduleAtFixedRate(new MyJobTask(), 0, 2, TimeUnit.HOUR);
There are several reasons, detailed here: A better way to run code for a period of time
But importantly, ScheduledExecutorService allows you to use multiple threads so that tasks which take a long time don't necessarily have to back-up your queue of tasks (the service can be running 2 of your tasks simultaneously). Also, if doMyJob throws an exception, ScheduledExecutorService will continue to schedule your task rather than being cancelled because it failed to reschedule the task.
How can I schedule time to stop and kill a single job
from the begining of execution task ?
For example determine that if after an hour the task is still working, it will stop and remove. (I don't mean to Repeated task)
I use with org.quartz.Scheduler and org.quartz.JobDetail in java
many thanks
You might want to look into the org.quartz.InterruptableJob interface (which extends Job) and the interrupt() method on the scheduler.
If you want to actually call Thread.interrupt() on the threads in the worker thread pool you'd likely need your implementation of org.quartz.spi.ThreadPool.
I think it would probably be easier for you if you coded this functionality within the job itself. I say this for a few reasons:
Its very easy to track how long the job has been running from within the job
If the job needs to stop, its easy to stop gracefully and set aside the work it was doing
Your job may be able to tell progress on the works its doing, and if its close to being done let it go beyond the kill-time + 10% or something
If you change your class to implement StatefulJob instead of Job, Quartz will take care of this for you. From the StatefulJob javadoc:
stateful jobs are not allowed to execute concurrently, which means new triggers that occur before the completion of the execute(xx) method will be delayed.
StatefulJob extends Job and does not add any new methods, so all you need to do to get the behaviour you want is change this:
public class YourJob implements org.quartz.Job {
void execute(JobExecutionContext context) {/*implementation omitted*/}
}
To this:
public class YourJob implements org.quartz.StatefulJob {
void execute(JobExecutionContext context) {/*implementation omitted*/}
}
Couple more options are here