We Keep Coding

Spring sleuth not creating new traceid inside a spring scheduler

I am scheduling the spring scheduler with SchedulingConfigurer as follows. However, new traceid is not getting created every time the "ProcessJob" method is getting called.
Even following method always logs with the same traceid.
log.info("Performing task");
What is the issue here? and how do i ensure new traceid everytime this job is triggered.
I have even tried wrapping "processJob" method call inside newSpan as follows: but no luck.
Fix 1: not working:
private void setSchedule() {
future =
taskScheduler.schedule(
() -> {
Span newSpan = tracer.nextSpan().name("newSpan").start();
try (SpanInScope ws = tracer.withSpanInScope(newSpan.start())) {
log.info("Performing task");
taskManager.processJob();
} finally {
newSpan.finish();
}
},
dynamicTrigger);
}
Original class that needs fix:
public class SchedulerConfig
implements SchedulingConfigurer, ApplicationListener<RefreshScopeRefreshedEvent> {
private final DynamicTrigger dynamicTrigger;
private final TaskManager taskManager;
private TaskScheduler taskScheduler;
private ScheduledFuture<?> future;
#Bean(destroyMethod = "shutdown")
public ExecutorService taskExecutor() {
return Executors.newScheduledThreadPool(1);
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
taskScheduler = taskRegistrar.getScheduler();
setSchedule();
}
private void setSchedule() {
future =
taskScheduler.schedule(
() -> {z
log.info("Performing task");
taskManager.processJob();
},
dynamicTrigger);
}
#Override
public void onApplicationEvent(RefreshScopeRefreshedEvent event) {
log.info("Rescheduling due to change in cron expression");
future.cancel(false);
setSchedule();
}

The way you start the span is not how you suppose to do it (e.g.: you call start twice). Please check the docs to see how to do it properly: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#using-creating-and-ending-spans
The easiest way to start a new span is using #NewSpan on a method that belongs to a Spring Bean, please see the docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#using-annotations-new-spans
For scheduling, I think it is way simpler using #Scheduled, see the docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#sleuth-async-scheduled-integration
This is also instrumented out of the box by Sleuth so you don't need to do anything to start a new Span:
#Scheduled(fixedDelay = 1_000)
public void scheduled() {
log.info("Hey, look what I'm doing");
}
If you don't want to use #Scheduled, you can use a TraceableScheduledExecutorService as your ExecutorService, docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#sleuth-async-executor-service-integration

How to test proper working of Quartz job in Spring Boot application

I want to test if my Quartz trigger is working as it supposes in practice.
My Quartz configuration looks like:
#Configuration
public class QuartzConfiguration {
#Bean
public JobDetail verificationTokenRemoverJobDetails() {
return
JobBuilder
.newJob(VerificationTokenQuartzRemoverJob.class)
.withIdentity("Job for verification token remover")
.storeDurably()
.build();
}
#Bean
public Trigger verificationTokenRemoverJobTrigger(JobDetail jobADetails) {
return
TriggerBuilder
.newTrigger()
.forJob(jobADetails)
.withIdentity("Trigger for verification token remover")
.withSchedule(CronScheduleBuilder.cronSchedule("0 0 0/2 1/1 * ? *"))
.build();
}
}
and my Job class looks like:
#AllArgsConstructor
public class VerificationTokenQuartzRemoverJob implements Job {
private VerificationTokenRepository verificationTokenRepository;
#Override
public void execute(JobExecutionContext context) {
verificationTokenRepository.deleteAllByCreatedLessThan(LocalDateTime.now().minusMinutes(30));
}
}
When I am starting my Spring Boot application in logs I can realize that Job is working and triggered cyclical but it's not enough to confirm the proper working.
That's why I decided to create a JUnit test. I found a tutorial: click but an owner used a clause while(true) which according to this topic: click is not a preferable option. Here occurs a question, is there any other option to verify the Job class name, the identity of the trigger and check if CRON expression and the concrete job are called as often as possible?
If it possible I will be grateful for suggestions on how to reach a desirable effect.

Please not that the above answer on using the Spring Scheduling has one big drawback: This works nicely if you just run a single instance of your application, but as soon as you scale up to multiple instances it becomes more complex:
You might want to run the job only once at a certain interval but if two nodes run simultaneously the job might run on both nodes (so basically twice). Quartz can handle these kind of situations because it can have a central database through which it can coordinate if a job is already started.
With spring scheduling the job will run on both nodes.

With SpringBoot, You could do easier doing the following
--- Option 1 ---
#Configuration
// This is important!
#EnableScheduling
public class Configuration{
// TODO Change to 0 0 0/2 1/1 * ? *
#Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {
long now = System.currentTimeMillis() / 1000;
System.out.println(
"schedule tasks using cron jobs - " + now);
}
}
Full Example:
https://www.baeldung.com/spring-scheduled-tasks
--- Option 2-> Programatically ---
#Configuration
#EnableScheduling
public class Configuration implements SchedulingConfigurer {
#Bean(destroyMethod = "shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(100);
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
CronTrigger cronTrigger
= new CronTrigger("* * * * * *");
taskRegistrar.setScheduler(taskExecutor());
taskRegistrar.addTriggerTask(
new Runnable() {
#Override public void run() {
System.out.println("RUN!!!");
}
},
cronTrigger
);
}
}

Is there a better approach to reduce execution time of a long running quartz job?

I have a cron job that will be triggered sometime in the night, that gets a large number of productIds around 100k from the DB,
and get the productInfo for all of them from a service that takes around 700ms for an API call for 1 productId.
CronJob
public class GetProducts extends QuartzJobBean {
#Autowired
private ProductClient productClient;
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
List<Long> ProductsIds = fetchAllProductIdsFromDb();
Response<ProductClientResponse> response = null;
for (Long productId : ProductsIds) {
ProductClientRequestBody requestBody = new ProductClientRequestBody();
requestBody.putIdInsideTheRequestBody(productId);
response = productClient.getResult(requestBody).execute();
if (response != null && response.isSuccessful()) {
log.info("We have got successful response for {}", i);
}
}
}
}
Here productClient is the Retrofit client of the service.
So this job will take 5 hours technically to complete.
#============================================================================
# Configure Main Scheduler Properties
#============================================================================
org.quartz.scheduler.instanceName=QuartzScheduler
org.quartz.scheduler.instanceId=AUTO
#============================================================================
# Configure ThreadPool
#============================================================================
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=80
org.quartz.threadPool.threadPriority=5
#============================================================================
# Configure JobStore
#============================================================================
org.quartz.jobStore.misfireThreshold=60000
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties=true
org.quartz.jobStore.dataSource=myDS
org.quartz.jobStore.tablePrefix=QRTZ_
org.quartz.jobStore.isClustered=true
org.quartz.jobStore.clusterCheckinInterval=20000
#============================================================================
# Configure Datasources
#============================================================================
org.quartz.dataSource.myDS.driver=com.mysql.jdbc.Driver
org.quartz.dataSource.myDS.maxConnections=10
org.quartz.dataSource.myDS.validationQuery=select 0 from dual
org.quartz.scheduler.batchTriggerAcquisitionMaxCount =10
org.quartz.dataSource.myDS.URL=jdbc:mysql://127.0.0.1:3306/quartz
org.quartz.dataSource.myDS.user=root
org.quartz.dataSource.myDS.password=root
This is my quartz properties file.
I wanted to know if there any better approach to get the ProductInfo for all the 100k products.
One approach
I schedule 100k jobs for all the ProductIds. and while quartz running on a clustered environment will schedule according to the instance available.
org.quartz.threadPool.threadCount=80 - this property states that in one instance of the service at most 80 threads can take up the jobs. Is it?
And if I have 2 instances running then at least 100-160 jobs can run concurrently. Am I correct in my approach? which can reduce the time by a huge margin.
Is there any other better approach than this?

Note :- This is to demonstrate the approach the code can be further improvised(I have not tested the code and there can be syntactical errors)
public class ProductsJob extends QuartzJobBean {
#Autowired
private ProductClient productClient;
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
List<Long> ProductsIds = fetchAllProductIdsFromDb();
ProductsExecutor productsExecutor=new ProductsExecutor();
productsExecutor.runTasks(ProductsIds);
}
}
public class ProductsExecutor {
private int threadPoolSize=10;
private ExecutorService executorService;
public ProductsExecutor()
{
executorService =Executors.newFixedThreadPool(threadPoolSize);
}
public void runTasks(List<Integer> productIds)
{
//Loop and execute the task and wait for them it necessary
ProductsWorker worker=new ProductsWorker();
worker.setProductId(productId);
executorService.invokeAll(worker);
}
}
public class ProductsWorker implements Runnable {
private int productId;
#Override
public void run() {
ProductClientRequestBody requestBody = new ProductClientRequestBody();
requestBody.putIdInsideTheRequestBody(productId);
response = productClient.getResult(requestBody).execute();
if (response != null && response.isSuccessful()) {
log.info("We have got successful response for {}");
}
}
public void setProductId(int productId)
{
this.productId=productId;
}
}
Don't forget to destroy your executor service as with each quartz run a new executor service will be created.Also you can add awaitTermination for all the submitted tasks.

Spring Task Async and Delayed

I need to do onething that I don't know wich is the best practice to this.
After I send one request to an especific service, this one returns OK and queue my request. I have a callback service that is used to notify when it ends.
The problem is that the whole process can take a long time and over without notify anything and after that I need to consider a timeout.
The application is SpringBoot APP and I was considering to use the annotations #EnableAsync and #Async on a service method with a Sleep time.
#Configuration
#EnableAsync
public class AsyncConfiguration implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("TIMCLL-");
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
// TODO Auto-generated method stub
return null;
}
}
.
.
.
#Async
public void verifyStatusTimPayment() throws InterruptedException {
Thread.sleep(5000);
logger.info( "Executed after 5s " + new SimpleDateFormat("dd/MM/yyyy hh:mm:ss").format(new Date()));
}
The verification needs to be done 15 minutes after the request and have to occur just one time per request.
How can I do this without make a Thread.sleep ?????

You could use the ScheduledExecutorService to schedule the Task
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
...
scheduler.schedule(() -> {yourtaskhere}, 15, TimeUnit.MINUTES);
But this is not what you want. What if the server dies between the scheduling of the task and its execution? you will lose your task.
It would be better if you persist the message in a queue, and retrieve it later, or use any scheduler that uses persistence (a la Quartz)

you can add #EnableScheduling annotation to your configuration:
#Configuration
#EnableAsync
#EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("TIMCLL-");
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
// TODO Auto-generated method stub
return null;
}
}
if you want to do schedule once and delayed you can call the taskScheduler :
#Autowired
private TaskScheduler taskScheduler;
and execute the task:
taskScheduler.schedule(
() -> {//your task},
//Your Delay task
);

i think we can use #Scheduled in latest spring. it will run every 15 minutes
like method is annotated as below
#Scheduled(cron = "0 0/15 * * * *")
public void verifyStatusTimPayment() throws InterruptedException {
logger.info( "Executed after 5s " + new SimpleDateFormat("dd/MM/yyyy hh:mm:ss").format(new Date()));
}
i know i am late but might help someone who is going through thread

You can use Redis backed delayed scheduler, which would guarantee you wouldn't lose your tasks. This can be done using Rqueue very easily.
In Rqueue you can enqueue a task which would run after 15 minutes as
public class Verification {
private String id;
}
#Component
class VerificationListener {
#RqueueListener(
value = "verification-queue")
public void onMessage(Verification verification) {
// do verification
}
}
#Service
class DelayedTaskService {
#Autowired private RqueueMessageSender rqueueMessageSender
public void enqeueVerification(Verification verification) {
rqueueMessageSender.enqueuIn("verification-queue", verification, Duration.ofMinutes(15);
}
}
P.S. I'm a developer of the Rqueue library.

Spring : Schedule a task which takes a parameter

I have a class with the following function:
public class classA{
...
...
void function_to_be_scheduled(String param){
...
...
}
}
I want to schedule the function using the scheduled-tasks element of the task namespace.
<task:scheduled-tasks>
<task:scheduled ref="beanA" method="function_to_be_scheduled" cron="${cron}"/>
</task:scheduled-tasks>
How do i pass the parameter to the function which i want to schedule?

According to the docs you cant.
Notice that the methods to be scheduled must have void returns and
must not expect any arguments.

The Spring doc about scheduling says:
Notice that the methods to be scheduled must have void returns and must not expect any arguments
Since the parameter comes from the Spring config file you can declare a bean (es beanB which wraps beanA) in the spring file, inject the parameter you need in the bean and the schedule the execution of a method of the bean which knows the parameter (it could be a simple wrapper of your beanA)

You can use TaskScheduler and encapsule your logic with a parameter in Runnable:
#Autowired
private TaskScheduler scheduler;
public void scheduleRules() {
MyTask task = new MyTaskImpl(someParam);
// new CronTrigger
scheduler.scheduleAtFixedRate(task, Duration.ofMinutes(1));
}

I've found that the only way to do this is to have a façade method that is #Scheduled and knows the default value required. The useful side-effect of this is that you can also provide an API through a #Controller to provide manual triggering with a specific parameter - useful if you need to re-run an activity.
#Scheduled(cron = "${myChronSchedule}")
public void generateActivities() {
this.generateActivities(LocalDate.now());
}
public void generateActivities(LocalDate theDate) {
// do the work
...
}
If you don't need the façade to be public, there's no reason why it can't be private and no-one is the wiser.

The Task Scheduler did the trick for me
First create a configuration class called ThreadPoolTaskScheduler class. Find details Here!
Then create a class where the magic happens
#Component
public class ThreadPoolTaskSchedulerExample {
#Autowired
private ThreadPoolTaskScheduler taskScheduler;
class EmailWatch implements Runnable{
private String userEmail;
public EmailWatch(String userEmail){
this.userEmail = userEmail;
}
#Override
public void run() {
System.out.println("This is the email "+ userEmail);
}
}
public void watchEmail(String userEmail) {
//refresh watch every minute
CronTrigger cronTrigger = new CronTrigger("0 * * ? * *");
taskScheduler.schedule(new EmailWatch(userEmail));
}
}