We Keep Coding

SpringFramework #Scheduled is executing the method twice but console log prints record of only one execution

The scheduled task[payment transfer] is executing twice but console logs has track of only one execution.
The only proof for the duplicate run is that the transfer history in database has two entry with the same schedule id with difference of 1second in executed time.
Main class
#EnableScheduling
#SpringBootApplication(scanBasePackages = {
"com.`*`",
"com.`*`"
},
exclude = {SecurityAutoConfiguration.class})<br>
public class PaymentsSchedulerApplication {
public static void main(String[] args) {
SpringApplication.run(PaymentsSchedulerApplication.class, args);
}
}
Here is my component class where the method is annotated with #Scheduled. It is intended to run at 00:30 Hrs Local time every day.
#Component
#Slf4j
#RequiredArgsConstructor
public class ScheduledTask {
private final ISchedulerService iSchedulerService;
this is the method that is running as a ghost as we couldn't find any information in logs on the duplicated execution.
#Scheduled(cron = "${scheduler.jobs.repeating-transfer.cron}")
public void scheduleTask() {
// cron: '0 30 0 * * *
log.info("Execute")<br>
}

A blunder by devops team, they had run the scheduler service in DR environment connected to prod db. Hence both Prod and DR service were running the batch which lead to duplication.

How to test proper working of Quartz job in Spring Boot application

I want to test if my Quartz trigger is working as it supposes in practice.
My Quartz configuration looks like:
#Configuration
public class QuartzConfiguration {
#Bean
public JobDetail verificationTokenRemoverJobDetails() {
return
JobBuilder
.newJob(VerificationTokenQuartzRemoverJob.class)
.withIdentity("Job for verification token remover")
.storeDurably()
.build();
}
#Bean
public Trigger verificationTokenRemoverJobTrigger(JobDetail jobADetails) {
return
TriggerBuilder
.newTrigger()
.forJob(jobADetails)
.withIdentity("Trigger for verification token remover")
.withSchedule(CronScheduleBuilder.cronSchedule("0 0 0/2 1/1 * ? *"))
.build();
}
}
and my Job class looks like:
#AllArgsConstructor
public class VerificationTokenQuartzRemoverJob implements Job {
private VerificationTokenRepository verificationTokenRepository;
#Override
public void execute(JobExecutionContext context) {
verificationTokenRepository.deleteAllByCreatedLessThan(LocalDateTime.now().minusMinutes(30));
}
}
When I am starting my Spring Boot application in logs I can realize that Job is working and triggered cyclical but it's not enough to confirm the proper working.
That's why I decided to create a JUnit test. I found a tutorial: click but an owner used a clause while(true) which according to this topic: click is not a preferable option. Here occurs a question, is there any other option to verify the Job class name, the identity of the trigger and check if CRON expression and the concrete job are called as often as possible?
If it possible I will be grateful for suggestions on how to reach a desirable effect.

Please not that the above answer on using the Spring Scheduling has one big drawback: This works nicely if you just run a single instance of your application, but as soon as you scale up to multiple instances it becomes more complex:
You might want to run the job only once at a certain interval but if two nodes run simultaneously the job might run on both nodes (so basically twice). Quartz can handle these kind of situations because it can have a central database through which it can coordinate if a job is already started.
With spring scheduling the job will run on both nodes.

With SpringBoot, You could do easier doing the following
--- Option 1 ---
#Configuration
// This is important!
#EnableScheduling
public class Configuration{
// TODO Change to 0 0 0/2 1/1 * ? *
#Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {
long now = System.currentTimeMillis() / 1000;
System.out.println(
"schedule tasks using cron jobs - " + now);
}
}
Full Example:
https://www.baeldung.com/spring-scheduled-tasks
--- Option 2-> Programatically ---
#Configuration
#EnableScheduling
public class Configuration implements SchedulingConfigurer {
#Bean(destroyMethod = "shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(100);
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
CronTrigger cronTrigger
= new CronTrigger("* * * * * *");
taskRegistrar.setScheduler(taskExecutor());
taskRegistrar.addTriggerTask(
new Runnable() {
#Override public void run() {
System.out.println("RUN!!!");
}
},
cronTrigger
);
}
}

ThreadPoolTaskScheduler change poolSize in runtime

I create scheduler:
#Bean
TaskScheduler taskScheduler(){
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setPoolSize(5);
threadPoolTaskScheduler.setAwaitTerminationSeconds(60);
threadPoolTaskScheduler.setThreadNamePrefix("Test-");
return threadPoolTaskScheduler;
}
I wait next:
Each 1 second triggered my scheduled method and start 5 threads(PoolSize(5)) and each thread will make my logic. For that I create scheduled method in #Component bean:
#Slf4j
#Component
public class MyScheduler {
private final TaskScheduler taskScheduler;
public MyScheduler(TaskScheduler taskScheduler) {
this.taskScheduler = taskScheduler;
}
#Scheduled(fixedDelay = 1000L)
public void test(){
taskScheduler.schedule(() -> {
try {
Thread.sleep(9000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
log.info("test");
}, new Date());
}
}
It work fine.
each 1 second start test() method and work 9 seconds. On 5 secon I have full threadPool and wait first free thread. If I set sleep(5000L) - threadPool can not fill up.
But now I need change poolSize in runtime. for example from 5 to 10. How can I do it?

According to the documentation you can resize the scheduler at runtime by calling setPoolSize() (see: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/concurrent/ThreadPoolTaskScheduler.html#setPoolSize-int-)
The problem is: you need to have a reference to the specific class ThreadPoolTaskScheduler instead of the interface TaskScheduler.
You need to decide if you want to change the references from TaskScheduler to ThreadPoolTaskScheduler without breaking anyhting in your code

Spring batch trigger process after all jobs complete

I have a spring batch application with a series of jobs. I want to send an email after ALL the jobs have completed, but I'm not sure the best way to do this. The options I am considering are:
Run the jobs in a certain order, and amend the JobListener of the last job, so that it will send the email. Downside with this is it won't work if a further job is added to the end of the batch.
Add a new job which will send the email and order the jobs, making sure this additional job is run last.
Are there any built in spring-batch constructs that will be triggered on completion of the entire batch?
The final option would be my preferred solution, so my question is, are there any spring-batch classes that listen for batch completion (similar to JobExecutionListenerSupport or a Step Listener)?

No. I am not aware of any batch listener that listen for the whole batch's completion.
I have two alternatives for you. Both allows you to stick to Spring.
(1) If your application is designed as perpetual (i.e. like a web-server), you can inject a custom jobLauncher, grab the TaskExecutor and wait its completion (either through simple counters that counts call-backs from afterJob functions or through a certain amount of time that it requires all jobs to be submitted -- not necessarily started).
Add a configuration class like this:
#Configuration
class JobConfiguration implements InitializingBean {
TaskExecutor taskExecutor;
#Bean TaskExecutor taskExecutor () {
// here, change to your liking, in this example
// I put a SyncTaskExecutor
taskExecutor = new SyncTaskExecutor();
return taskExecutor;
}
#Bean
public JobLauncher jobLauncher(#Autowired JobRepository jobRepository,
#Autowired TaskExecutor taskExecutor) {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
launcher.setTaskExecutor(taskExecutor);
return launcher;
}
List<Job> jobs;
// I don't use this in this example
// however, jobs.size() will help you with the countdown latch
#Autowired public void setJobs (List<Job> jobs) {
this.jobs = jobs;
}
#AfterPropertiesSet
public void init () {
// either countdown until all jobs are submitted
// or sleep a finite amount of time
// in this example, I'll be lazy and just sleep
Thread.sleep(1 * 3600 * 1000L); // wait 1 hour
taskExecutor.shutdown();
try {
taskExecutor.awaitTermination();
} catch (Exception e) {}
finally {
// send your e-mail here.
}
}
}
(2) If your application stops when all jobs are done, you can simply follow this to send out an e-mail.
I repeat a few lines of code for completeness:
public class TerminateBean {
#PreDestroy
public void onDestroy() throws Exception {
// send out e-mail here
}
}
We also have to add a bean of this type:
#Configuration
public class ShutdownConfig {
#Bean
public TerminateBean getTerminateBean() {
return new TerminateBean();
}
}

If I understand correctly, you have a job of jobs. In this case, you can define an enclosing job with a series of steps of type JobStep (which delegates to a job). Then, you can register a JobExecutionListener on the enclosing job. This listener will be called once all steps (ie sub-jobs) have completed.
More details about the JobStep here: https://docs.spring.io/spring-batch/4.0.x/api/org/springframework/batch/core/step/job/JobStep.html

Spring Task Async and Delayed

I need to do onething that I don't know wich is the best practice to this.
After I send one request to an especific service, this one returns OK and queue my request. I have a callback service that is used to notify when it ends.
The problem is that the whole process can take a long time and over without notify anything and after that I need to consider a timeout.
The application is SpringBoot APP and I was considering to use the annotations #EnableAsync and #Async on a service method with a Sleep time.
#Configuration
#EnableAsync
public class AsyncConfiguration implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("TIMCLL-");
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
// TODO Auto-generated method stub
return null;
}
}
.
.
.
#Async
public void verifyStatusTimPayment() throws InterruptedException {
Thread.sleep(5000);
logger.info( "Executed after 5s " + new SimpleDateFormat("dd/MM/yyyy hh:mm:ss").format(new Date()));
}
The verification needs to be done 15 minutes after the request and have to occur just one time per request.
How can I do this without make a Thread.sleep ?????

You could use the ScheduledExecutorService to schedule the Task
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
...
scheduler.schedule(() -> {yourtaskhere}, 15, TimeUnit.MINUTES);
But this is not what you want. What if the server dies between the scheduling of the task and its execution? you will lose your task.
It would be better if you persist the message in a queue, and retrieve it later, or use any scheduler that uses persistence (a la Quartz)

you can add #EnableScheduling annotation to your configuration:
#Configuration
#EnableAsync
#EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("TIMCLL-");
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
// TODO Auto-generated method stub
return null;
}
}
if you want to do schedule once and delayed you can call the taskScheduler :
#Autowired
private TaskScheduler taskScheduler;
and execute the task:
taskScheduler.schedule(
() -> {//your task},
//Your Delay task
);

i think we can use #Scheduled in latest spring. it will run every 15 minutes
like method is annotated as below
#Scheduled(cron = "0 0/15 * * * *")
public void verifyStatusTimPayment() throws InterruptedException {
logger.info( "Executed after 5s " + new SimpleDateFormat("dd/MM/yyyy hh:mm:ss").format(new Date()));
}
i know i am late but might help someone who is going through thread

You can use Redis backed delayed scheduler, which would guarantee you wouldn't lose your tasks. This can be done using Rqueue very easily.
In Rqueue you can enqueue a task which would run after 15 minutes as
public class Verification {
private String id;
}
#Component
class VerificationListener {
#RqueueListener(
value = "verification-queue")
public void onMessage(Verification verification) {
// do verification
}
}
#Service
class DelayedTaskService {
#Autowired private RqueueMessageSender rqueueMessageSender
public void enqeueVerification(Verification verification) {
rqueueMessageSender.enqueuIn("verification-queue", verification, Duration.ofMinutes(15);
}
}
P.S. I'm a developer of the Rqueue library.