We Keep Coding

Scheduler in springboot for huge data alternate of spring batch processing

I have to run a schduler(any schduled jobs) where i have to fetch 300,000-400,000 in average twice daily from database and apply business logics one by one where each process requests to thirdparty which takes 3-4 seconds to respond.
what are alternates of spring batch processing to process such huge data in efficient ways?
Note: fetched data are not static, data may vary everyday.

May be #Scheduled annotation can help you out.

You can use the Spring Batch schedular. It execute spring batch jobs periodically on fixed schedule using some cron expression passed to Spring TaskScheduler
To configure, batch job scheduling is done in two steps:
Enable scheduling with #EnableScheduling annotation.
Create method annotated with #Scheduled and provide recurrence details using cron job. Add the job execution logic inside this method.
Here is the link of an example : https://howtodoinjava.com/spring-batch/job-scheduler-example/

Why is my #Scheduled Spring Boot Task running inconsistently in Azure?

I have a Spring Boot microservice deployed in Azure that is supposed to be running on a fixed rate with the #Scheduled Spring Annotation.
When I run it locally, it performs exactly as expected.
When deployed in Azure, it seems to be a mixed bag as to when it will run as scheduled.
During off-peak hours (~00:00 - 8:00AM) it seems to work as scheduled with a little variation here and there.
However, during peak business hours (12:00PM - 18:00PM) the scheduled times can vary DRASTICALLY.
A service that should be running once every minute will run potentially every 5 minutes during this time.
It's required that the service stay up and running (can't just kick off the service anew when scheduled), it has a list of customers that it loops through (the list is fetched from a DB whenever it first starts or gets through the list). It works a certain number of customers every time it is scheduled and moves on to the next fixed set of customers until it goes through them all and then starts the process anew.
Is this due to throttling during peak hours?
Does anyone know of a good way to keep my service firing on its schedule, or an Azure alternative to the #Scheduled annotation?
Thanks

Use fixedRate
fixedRate : makes Spring run the task on periodic intervals even if the last invocation may still be running.
fixedDelay : specifically controls the next execution time when the last execution finishes. In code:
#Scheduled(fixedDelay=5000)
public void updateEmployeeInventory(){
}
#Scheduled(fixedRate=5000)
public void updateEmployeeInventory(){
}
I've got the answer from there How wait #Scheduled till previous task is not finished?

Quartz scheduler executes multiple times concurrently on a scheduled time

I have used quartz scheduler in my spring boot application . I am using java programmatic scheduling not a spring based annotated one. Here's the snippet
Scheduler scheduler =
StdSchedulerFactory.getDefaultScheduler();
JobDetail jobDetail = newJob(ExPJob.class) .build();
CronScheduleBuilder cronSchedule =
cronSchedule (mycronExpression)
.withMisfireHandlingInstructionDoNothing();
I am shutting down this as bellow
#PreDestroy
Public void shutdownSch(){
Scheduler scheduler =
StdSchedulerFactory.getDefaultScheduler();
scheduler.shutdown();
}
Whats my problem ?
-- if I do a re/deployment for my app.war, In Weblogic thread pool does not get killed, and after scheduler starts,
It creates another thread pool it seems.
For Example : if I schedule a cron Job at 4 PM lot of jobs are firing at the same time. I have tried to find the PID in Linux box I'm not getting any regd this zombie threads.
What are the solutions tried ? --
I am able to kill jobs successfully via jobkeys
scheduler.deletejob(jobKey) .
So why issue still persists ?
-- I have done a series of deployments without the above mentioned job killing code before.
So, I suspect those jobs/threads are already running inside JVM. I tried a weblogic restart via console through MWare folks. But still no luck. Kindly pour your thoughts.

How to kill quartz scheduled threads?

I had added quartz scheduler code earlier in our application and now I have removed that code.
The job was scheduled to run every hour.
But now I have removed that code and deployed new war which does not have that code.
But now also I can see logs like below every one hour,
10:00:00.001 [org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-4]
How to kill these threads?

If you are making use of quartz jdbcstore, you need to delete the trigger and job detail records from database.

How to check whether #Schedule annotation (Spring 3.1) cron job is running.

In my application I have one cron job which connects to a FTP server and transfer files, a very simple functionality and it is configured using spring #Schedule annotation with cron expression as a parameter.
It was running fine for few months and then suddenly it stopped, got the connectException.
May be the FTP server was down or something happened which causes the cron thread to stop.
I looked (google) for the reasons but didnt get any ( Nothing much in the logs also - Just the exception name ).It may be a one time thing :)
my question is that can I put some check or watcher on the #Schedule cron job to know whether it is running or not ?
Sorry for my bad explanation/english
Thanks

my question is that can I put some check or watcher on the #Schedule
cron job to know whether it is running or not ?
Basically, you can't.
When you use #Scheduled, Spring uses a ScheduledAnnotationBeanPostProcessor to register the tasks you specify (annotated methods). It registers them with a ScheduledTaskRegistrar. The ScheduledAnnotationBeanPostProcessor is an ApplicationListener<ContextRefreshEvent>. When it receives the ContextRefreshEvent from the ApplicationContext, it schedules the tasks registered in the ScheduledTaskRegistrar.
During this step, these tasks are scheduled with a TaskScheduler which typically wraps a ScheduledExecutorService. If an exception is uncaught in a submitted task, then the task is removed from the ScheduledExecutorService queue.
The TaskScheduler class does not provide a public API to retrieve the scheduled tasks, ie. the ScheduledFuture objects. So you can't use it to find out if your tasks are running or not.
And you probably shouldn't. Develop your tasks, your #Scheduled methods, to be able to withstand an exception being thrown. Some times, obviously, that's not possible. With a network error, for example, you would probably have to restart your application. Without knowing anything else about your application, I would say more logging is your best bet.