We Keep Coding

Spring Boot - #EnableScheduling combined with spring-boot-starter-quartz

We have a project that uses somewhere around ten Quartz jobs via the aforementioned dependency. Everything seemed to work fine for a while but we have two jobs that just randomly stop working (all jobs are scheduled to run every five seconds). Our mitigation strategy was to migrate these jobs to Spring's 'own' scheduling as they do not require any input data. After adding #EnableScheduling and the appropriate #Scheduled annotation, they work fine and run every five seconds. The problem is that now, the 'old' Quartz jobs seemed to have stopped working (at least in our integration tests, they wait 20 seconds for an execution). The Quartz jobs just never fire. When increasing the timeout, the Quartz jobs sometimes start about 30 seconds after being scheduled. While they are running, the Spring jobs seem to wait. We have tried setting the thread count for both Quartz and Spring to > 50 but nothing seems to help. We're somewhat out of ideas, does anybody know a solution?
We're using Spring Boot 2.3.3. and the latest Quartz, 2.3.2.
Thanks.

Ok so the problem seems to be with our #Transactional management - we seem to have somehow acquired a lock that blocks Quartz's scheduler from finding triggers that were scheduled for immediate execution (Quartz trigger does not fire immediately seems to have the same problem, but unrelated to Spring scheduling). We 'fixed' the problem by setting
spring.quartz.properties.org.quartz.scheduler.idleWaitTime=5000
This is obviously not a fix but instead of waiting for 30 seconds (the default for Quartz's idle wait time which initially made the jobs only being executed after said time), the scheduler only waits for five seconds to search for triggers after not finding one. This makes our immediate jobs trigger five seconds late but that isn't a problem in our use case.
Of course, this isn't the real solution, refining our #Transactionals would probably be the correct fix but we cannot do that right now, maybe sometime in the future.
Why the problem started ocurring after #EnableScheduling and not beforehand, we have no idea.

Running Only Once A Schedule Job Across Multiple Instances

I have a schedule job that run every end of the month. After running it saves some data to database.
When i scale the app(for example with 2 instances) both instances run the schedule job and both save the data and at the end of day my database has the same data.
So i want the schedule job only run one time regardless of instances numbers at cloud.

In my project, I have maintained a database table to hold a lock for each job which needs to be executed only once in the cluster.
When a Job gets triggered then it first tries to acquire lock from the database and if it gets that lock only then it will get executed. If it fails to acquire the lock then job will not get executed.
You can also look at the clustering feature of Quartz job.
http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/introduction.html

I agree with the comments. If you can utilize a scheduler that's going to be your best, most flexible option. In addition, a scheduler should be executing your job as a "task" on Cloud Foundry. The task will only run on one instance, so you won't need to worry about how many instances your application is using (the two are separate in that regard).
If you're using Pivotal Cloud Foundry/Tanzu Cloud Foundry there is a scheduler you can ask your operations team to install. I don't know about other variants of CF, but I assume there are other schedulers.
https://network.pivotal.io/products/p-scheduler/
If using a scheduler is not an option then this is a concern you'll need to handle in your application. The solution of using a shared lock is a good one, but there is also a little trick you can do on Cloud Foundry that I feel is a little simpler.
When your application runs, certain environment variables are set by the platform. There is one called INSTANCE_INDEX which has a number indicating the instance on which the app is running. It's zero-based, so your first app instance will be running on instance zero, the second instance one, etc.
In your code, simply look at the instance index and see if it's zero. If the index is non-zero, have your task end without doing anything. If it's zero, then let the task proceed and do its work. The task will execute on every application instance, but it will only do work on the first instance. It's an easy way to guarantee something like a database migration or background job only runs once.
One final option would be to use multiple processes. This is a feature of Cloud Foundry that enables you to have different processes running, like your web process and a background worker process.
https://docs.cloudfoundry.org/devguide/multiple-processes.html
The interesting thing about this feature is that you can scale the different processes independently of each other. Thus you could have as many web processes running, but only one background worker which would guarantee that your background process only runs once.
That said, the downside of this approach is that you end up with separate containers for each process and the background process would need to continue running. The foundation expects it to be a long-running process, not a finite duration batch job. You could get around this by wrapping your periodic task a loop or something which keeps the process running forever.
I wouldn't really recommend this option but I wanted to throw it out there just in case.

You can use #SnapLock annotation in your method which guarantees that task only runs once. See documentation in this repo https://github.com/luismpcosta/snap-scheduler
Example:
Import maven dependency
<dependency>
<groupId>io.opensw.scheduler</groupId>
<artifactId>snap-scheduler-core</artifactId>
<version>0.3.0</version>
</dependency>
After importing maven dependency, you'll need to create the required tables tables.
Finally, see bellow how to annotate methods which guarantees that only runs once with #SnapLock annotation:
import io.opensw.scheduler.core.annotations.SnapLock;
...
#SnapLock(key = "UNIQUE_TASK_KEY", time = 60)
#Scheduled(fixedRate = 30000)
public void reportCurrentTime() {
...
}
With this approach you also guarantee audit of the tasks execution.

Does the quartz scheduler run on its own thread?

I'd like to use quartz in a project of mine. I know that there is a scheduler and a threadpool for the jobs.
Does the scheduler continuously run in its own thread and fires off the jobs?
If I run the app on a 2 core CPU will one core be busy with the scheduling thread?

Yes, quartz uses at least n+1 threads where 1 is the scheduler thread that is running in an infinite loop sleeping before next task that should be triggered. N is the number of worker threads in the threadpool. You can configure this number using the property org.quartz.threadPool.threadCount.

You can see it work by running the stuff through for example Eclipse and then in the debug view you can see the active and sleeping threads of the application.
Quartz will create a configurable pool of threads. Each job will fire in its own thread (of course, otherwise they can't run concurrently). And no, its not a busy loop so the scheduler won't claim a CPU for itself.

Schedule task to run once per week in java

What would be the best way to schedule an event to run every Monday at 00:00?
Note though the app is running 24/7. So OS based schedulers such as cron are not needed.

You could embed a Quartz Scheduler in your Java app if it's a long running app. Quartz is a very flexible/powerful scheduler that can be configured in many ways.
Alternatively, if you simply want to run a Java process at a particular time then you should look at cron.
Edit: Given that your app is 24/7 (noting comments above), Quartz is definitely the way forward.

I think cron4J may be your best bet. You can then configure it to run at ceratin times.

Us the OS scheduler; either cron, Windows task scheduler or similar

Sling scheduler periodic job -- will the job overlap?

I am using Sling's scheduler to schedule periodic jobs and I'm wondering if I'm scheduling job A to run every 5 minute. In the unlikely occasion, the job start to run and took more than 5 minute what will happen? I have specified the job cannot run in parallel.
Job A will run again immediately after the previous run finishes.
Job A will run 5 minute after the previous run finishes.
Under the hood, Sling's scheduler is using QuartzScheduler, so if you know how QuartzScheduler will behave in this case please do share your knowledge as well.
Any help is much appreciated!

In Quartz Scheduler 2.1.x the annotation DisallowConcurrentExecution is used to prevent concurrent execution of the same Job.
In Quartz Scheduler 2.0.x in order to void the concurrent execution of a Job you have to implement the StatefulJob interface.
The decision on whether the misfired execution will execute when the previous job completes or it will be ignored depends on the trigger's misfire policy. By default when the scheduler starts, it searches for any persistent triggers that have misfired, and it then updates each of them based on their individually configured misfire instructions.
So in my opinion Job A will run again immediately after the previous run finishes. I suppose that Sling uses the default misfire policy. Otherwise the answer depends on the misfire policy selection.
That's how Quartz Scheduler works. I don't know how the Sling's scheduler works.
I hope this helps.