Why is my #Scheduled Spring Boot Task running inconsistently in Azure? - java

I have a Spring Boot microservice deployed in Azure that is supposed to be running on a fixed rate with the #Scheduled Spring Annotation.
When I run it locally, it performs exactly as expected.
When deployed in Azure, it seems to be a mixed bag as to when it will run as scheduled.
During off-peak hours (~00:00 - 8:00AM) it seems to work as scheduled with a little variation here and there.
However, during peak business hours (12:00PM - 18:00PM) the scheduled times can vary DRASTICALLY.
A service that should be running once every minute will run potentially every 5 minutes during this time.
It's required that the service stay up and running (can't just kick off the service anew when scheduled), it has a list of customers that it loops through (the list is fetched from a DB whenever it first starts or gets through the list). It works a certain number of customers every time it is scheduled and moves on to the next fixed set of customers until it goes through them all and then starts the process anew.
Is this due to throttling during peak hours?
Does anyone know of a good way to keep my service firing on its schedule, or an Azure alternative to the #Scheduled annotation?
Thanks

Use fixedRate
fixedRate : makes Spring run the task on periodic intervals even if the last invocation may still be running.
fixedDelay : specifically controls the next execution time when the last execution finishes. In code:
#Scheduled(fixedDelay=5000)
public void updateEmployeeInventory(){
}
#Scheduled(fixedRate=5000)
public void updateEmployeeInventory(){
}
I've got the answer from there How wait #Scheduled till previous task is not finished?

Related

#scheduled annotation in spring boot

I have created a scheduled job in spring boot, in which I have also used #schdeulerlock.
The problem is scheduler is not working properly.
If the job is scheduled to execute after every 10 min, so if the process takes more than 10 mins then what will happen, although there is a #schdeulerlock which release lock acter 15 mins
Scheduled task will never run in parallel. It waits for previous invocation to finish before it starts next one
See this thread for more details

How to kill quartz scheduled threads?

I had added quartz scheduler code earlier in our application and now I have removed that code.
The job was scheduled to run every hour.
But now I have removed that code and deployed new war which does not have that code.
But now also I can see logs like below every one hour,
10:00:00.001 [org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-4]
How to kill these threads?
If you are making use of quartz jdbcstore, you need to delete the trigger and job detail records from database.

Shutting down spring batch jobs gracefully in tomcat

We run several spring batch jobs within tomcat in the same web application that serves up our UI. Lately we have been adding many more jobs and we are noticing that when we patch our app, several jobs may get stuck in a STARTING or STARTED status. Many of those jobs ensure that another job is not running before they start up, so this means after we patch the server, some of our jobs are broken until we manually run SQL to update the statuses of the jobs to ABANDONED or STOPPED.
I have read here that JobScope and StepScope jobs don't play nicely with shutting down.
That article suggests not using JobScope or StepScope but I can't help but think that this is a solved problem where people must be doing something when the application exits to prevent this problem.
Are there some best practices for handling this scenario? What are you doing in your applications?
We are using spring-batch version 3.0.3.RELEASE
I will provide you an idea on how to solve this scenario. Not necessarily a spring-batch solution.
Everytime I need to add jobs in an application I do as this:
Create a table to control the jobs (queue, priority, status, etc.)
Create a JobController class to manage all jobs
All jobs are defined by the status R-running, F-Finished, Q-Queue (you can add more as you need like aborted, cancelled, etc) (the jobs control these statuses)
The jobController must be loaded only once, you can define it as a spring bean for this
Add a boolean attribute to JobController to inform if you already checked the jobs when you instantiate it. Set it to false
Check if there are jobs with the R status which means that in the last stop of the server they were running so you update every job with this R status to Q and increase their priority so it will get executed first after a restart of the server. This check is inside the if for that boolean attribute, after the check set it to true.
That way every time you call the JobController for the first time and there are unfinished jobs from a server crash you will be able to set then all to a status where it can be executed again. And this check will happens only once since you will be checking that boolean attribute.
A thing that you should be aware of is caution with your jobs priority, if you manage it wrong you may run into a starvation problem.
You can easily adapt this solution to spring-batch.
Hope it helps.

How to check whether #Schedule annotation (Spring 3.1) cron job is running.

In my application I have one cron job which connects to a FTP server and transfer files, a very simple functionality and it is configured using spring #Schedule annotation with cron expression as a parameter.
It was running fine for few months and then suddenly it stopped, got the connectException.
May be the FTP server was down or something happened which causes the cron thread to stop.
I looked (google) for the reasons but didnt get any ( Nothing much in the logs also - Just the exception name ).It may be a one time thing :)
my question is that can I put some check or watcher on the #Schedule cron job to know whether it is running or not ?
Sorry for my bad explanation/english
Thanks
my question is that can I put some check or watcher on the #Schedule
cron job to know whether it is running or not ?
Basically, you can't.
When you use #Scheduled, Spring uses a ScheduledAnnotationBeanPostProcessor to register the tasks you specify (annotated methods). It registers them with a ScheduledTaskRegistrar. The ScheduledAnnotationBeanPostProcessor is an ApplicationListener<ContextRefreshEvent>. When it receives the ContextRefreshEvent from the ApplicationContext, it schedules the tasks registered in the ScheduledTaskRegistrar.
During this step, these tasks are scheduled with a TaskScheduler which typically wraps a ScheduledExecutorService. If an exception is uncaught in a submitted task, then the task is removed from the ScheduledExecutorService queue.
The TaskScheduler class does not provide a public API to retrieve the scheduled tasks, ie. the ScheduledFuture objects. So you can't use it to find out if your tasks are running or not.
And you probably shouldn't. Develop your tasks, your #Scheduled methods, to be able to withstand an exception being thrown. Some times, obviously, that's not possible. With a network error, for example, you would probably have to restart your application. Without knowing anything else about your application, I would say more logging is your best bet.

How does the JDBCJobStore work .?

So I started to tinker around with JDBCJobStore in Quartz. Firstly, I could not find a single good resource on how to configure it from scratch. After looking for it for a while and singling out a good resource for beginners, I downloaded the sample application at Job scheduling with Quartz. I have a few doubts regarding it.
How does JDBCJobStore capture jobs.? I mean in order for the job to get stored in the database does the job have to run manually once.? Or will JDBCJobStore automatically detect the jobs and their details..?
How does JDBCJobStore schedule the jobs.? Does it hit the database at a fixed interval like a heartbeat to check if there are any scheduled jobs.? Or does it keep the triggers in the memory while the application is running.?
In order to run the jobs will I have to manually specify the details of the job like like name and group and fetch the trigger accordingly.? Is there any alternative to this.?
On each application restart how can I tell the scheduler to start automatically..? Can it be specified somehow.?
If you are using servlet/app server you can start it during startup:
http://quartz-scheduler.org/documentation/quartz-2.2.x/cookbook/ServletInitScheduler
If you are running standalone you have to initialize it manually i think.
You can read more about JobStores here:
http://quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-09
And about jobs and triggers:
http://quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-02
http://quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-03
http://quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-04
I guess that quartz checks jobs based on time interval to proper work in clusters and distributed systems.

Categories