I have a cron expression which will run everyday at 7 PM .I am using spring boot latest version.
#Scheduled(cron = "${my.cron.expression}")
public void scheduleTask(){
//call service layer where business logic resides
//other autowired beans here
}
I have 2 doubts.
Q1) How can i make sure that CRON JOB is executed only if old instance has finished running.
Q2) How to reload/refresh application context and reload all the beans afresh for every CRON JOB call?
For second point take a look at spring-cloud-config and client library spring-cloud-config-client. There is an HTTP endpoint to refresh beans
I have to create storage for any distributed operation like this, i.e. message queues, databases.
For your Q1: suggest you to have class level boolean variable like isRunning like below
public class scheduler {
private boolean isRunning = false;
#Scheduled(cron = "${my.cron.expression}")
public void scheduleTask(){
if(!isRunning) {
isRunning = true;
// executions goes here
}
isRunning = false;
}
}
Q2: To Reload beans
public class scheduler {
private boolean isRunning = false;
#Scheduled(cron = "${my.cron.expression}")
public void scheduleTask(){
if(!isRunning) {
isRunning = true;
// executions goes here
}
isRunning = false;
DefaultSingletonBeanRegistry registry = (DefaultSingletonBeanRegistry)
context.getBeanFactory();
registry.destroySingleton({yourbean}) //destroys the bean object
registry.registerSingleton({yourbeanname}, {newbeanobject}) //add to singleton beans cache
}
}
The fixedDelay property makes sure that there is a delay of n millisecond between the finish time of an execution of a task and the start time of the next execution of the task.
#Schedules({
#Scheduled(fixedDelay = 1000),
#Scheduled(cron = "${my.cron.expression}")
})
Related
I've defined 3 jobs using fixedDelayString=300000 (5 minutes) and I did that 3 these jobs will be executed independently. For that reason, I created an Async implementation. At first, each job worked fine, but in the time they started to delay a lot.
Each execution is about 5seg, but the next execution started to run after 10minutes. And occasionally 15 or 18minutes.
An example could be:
#EnableScheduling
public class AppConfig {
#Async('threadPoolTaskExecutor')
#Scheduled(fixedDelayString=15000)
public void doSomething1() {
// something that should run periodically
}
#Async('threadPoolTaskExecutor')
#Scheduled(fixedDelayString=300000)
public void doSomething2() {
// something that should run periodically
}
#Async('threadPoolTaskExecutor')
#Scheduled(fixedDelayString=300000)
public void doSomething3() {
// this job begins to have interval larger in each execution
}
}
#Configuration
#EnableAsync
public class AsyncConf {
#Bean("threadPoolTaskExecutor")
public TaskExecutor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(3);
executor.setMaxPoolSize(1000);
executor.setThreadNamePrefix("Async-");
return executor;
}
}
ยดยดยด
To mention the fixed delay, it should be fixedDelay instead of fixedDelayString. Check below code:
#Async('threadPoolTaskExecutor')
#Scheduled(fixedDelay = 300000)
public void doSomething3() { ... }
Also you should write #EnableScheduling annotation on your configuration class.
Also note that fixedDelay specifies that job should run next after specified amount of time once execution is completed. If you want to run your jobs at fixed intervals, you should try fixedRate instead of fixedDelay. Check more about scheduling here - https://www.baeldung.com/spring-scheduled-tasks
I am scheduling the spring scheduler with SchedulingConfigurer as follows. However, new traceid is not getting created every time the "ProcessJob" method is getting called.
Even following method always logs with the same traceid.
log.info("Performing task");
What is the issue here? and how do i ensure new traceid everytime this job is triggered.
I have even tried wrapping "processJob" method call inside newSpan as follows: but no luck.
Fix 1: not working:
private void setSchedule() {
future =
taskScheduler.schedule(
() -> {
Span newSpan = tracer.nextSpan().name("newSpan").start();
try (SpanInScope ws = tracer.withSpanInScope(newSpan.start())) {
log.info("Performing task");
taskManager.processJob();
} finally {
newSpan.finish();
}
},
dynamicTrigger);
}
Original class that needs fix:
public class SchedulerConfig
implements SchedulingConfigurer, ApplicationListener<RefreshScopeRefreshedEvent> {
private final DynamicTrigger dynamicTrigger;
private final TaskManager taskManager;
private TaskScheduler taskScheduler;
private ScheduledFuture<?> future;
#Bean(destroyMethod = "shutdown")
public ExecutorService taskExecutor() {
return Executors.newScheduledThreadPool(1);
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
taskScheduler = taskRegistrar.getScheduler();
setSchedule();
}
private void setSchedule() {
future =
taskScheduler.schedule(
() -> {z
log.info("Performing task");
taskManager.processJob();
},
dynamicTrigger);
}
#Override
public void onApplicationEvent(RefreshScopeRefreshedEvent event) {
log.info("Rescheduling due to change in cron expression");
future.cancel(false);
setSchedule();
}
The way you start the span is not how you suppose to do it (e.g.: you call start twice). Please check the docs to see how to do it properly: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#using-creating-and-ending-spans
The easiest way to start a new span is using #NewSpan on a method that belongs to a Spring Bean, please see the docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#using-annotations-new-spans
For scheduling, I think it is way simpler using #Scheduled, see the docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#sleuth-async-scheduled-integration
This is also instrumented out of the box by Sleuth so you don't need to do anything to start a new Span:
#Scheduled(fixedDelay = 1_000)
public void scheduled() {
log.info("Hey, look what I'm doing");
}
If you don't want to use #Scheduled, you can use a TraceableScheduledExecutorService as your ExecutorService, docs: https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/htmlsingle/#sleuth-async-executor-service-integration
I want to test if my Quartz trigger is working as it supposes in practice.
My Quartz configuration looks like:
#Configuration
public class QuartzConfiguration {
#Bean
public JobDetail verificationTokenRemoverJobDetails() {
return
JobBuilder
.newJob(VerificationTokenQuartzRemoverJob.class)
.withIdentity("Job for verification token remover")
.storeDurably()
.build();
}
#Bean
public Trigger verificationTokenRemoverJobTrigger(JobDetail jobADetails) {
return
TriggerBuilder
.newTrigger()
.forJob(jobADetails)
.withIdentity("Trigger for verification token remover")
.withSchedule(CronScheduleBuilder.cronSchedule("0 0 0/2 1/1 * ? *"))
.build();
}
}
and my Job class looks like:
#AllArgsConstructor
public class VerificationTokenQuartzRemoverJob implements Job {
private VerificationTokenRepository verificationTokenRepository;
#Override
public void execute(JobExecutionContext context) {
verificationTokenRepository.deleteAllByCreatedLessThan(LocalDateTime.now().minusMinutes(30));
}
}
When I am starting my Spring Boot application in logs I can realize that Job is working and triggered cyclical but it's not enough to confirm the proper working.
That's why I decided to create a JUnit test. I found a tutorial: click but an owner used a clause while(true) which according to this topic: click is not a preferable option. Here occurs a question, is there any other option to verify the Job class name, the identity of the trigger and check if CRON expression and the concrete job are called as often as possible?
If it possible I will be grateful for suggestions on how to reach a desirable effect.
Please not that the above answer on using the Spring Scheduling has one big drawback: This works nicely if you just run a single instance of your application, but as soon as you scale up to multiple instances it becomes more complex:
You might want to run the job only once at a certain interval but if two nodes run simultaneously the job might run on both nodes (so basically twice). Quartz can handle these kind of situations because it can have a central database through which it can coordinate if a job is already started.
With spring scheduling the job will run on both nodes.
With SpringBoot, You could do easier doing the following
--- Option 1 ---
#Configuration
// This is important!
#EnableScheduling
public class Configuration{
// TODO Change to 0 0 0/2 1/1 * ? *
#Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {
long now = System.currentTimeMillis() / 1000;
System.out.println(
"schedule tasks using cron jobs - " + now);
}
}
Full Example:
https://www.baeldung.com/spring-scheduled-tasks
--- Option 2-> Programatically ---
#Configuration
#EnableScheduling
public class Configuration implements SchedulingConfigurer {
#Bean(destroyMethod = "shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(100);
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
CronTrigger cronTrigger
= new CronTrigger("* * * * * *");
taskRegistrar.setScheduler(taskExecutor());
taskRegistrar.addTriggerTask(
new Runnable() {
#Override public void run() {
System.out.println("RUN!!!");
}
},
cronTrigger
);
}
}
I have a cron job that will be triggered sometime in the night, that gets a large number of productIds around 100k from the DB,
and get the productInfo for all of them from a service that takes around 700ms for an API call for 1 productId.
CronJob
public class GetProducts extends QuartzJobBean {
#Autowired
private ProductClient productClient;
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
List<Long> ProductsIds = fetchAllProductIdsFromDb();
Response<ProductClientResponse> response = null;
for (Long productId : ProductsIds) {
ProductClientRequestBody requestBody = new ProductClientRequestBody();
requestBody.putIdInsideTheRequestBody(productId);
response = productClient.getResult(requestBody).execute();
if (response != null && response.isSuccessful()) {
log.info("We have got successful response for {}", i);
}
}
}
}
Here productClient is the Retrofit client of the service.
So this job will take 5 hours technically to complete.
#============================================================================
# Configure Main Scheduler Properties
#============================================================================
org.quartz.scheduler.instanceName=QuartzScheduler
org.quartz.scheduler.instanceId=AUTO
#============================================================================
# Configure ThreadPool
#============================================================================
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=80
org.quartz.threadPool.threadPriority=5
#============================================================================
# Configure JobStore
#============================================================================
org.quartz.jobStore.misfireThreshold=60000
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties=true
org.quartz.jobStore.dataSource=myDS
org.quartz.jobStore.tablePrefix=QRTZ_
org.quartz.jobStore.isClustered=true
org.quartz.jobStore.clusterCheckinInterval=20000
#============================================================================
# Configure Datasources
#============================================================================
org.quartz.dataSource.myDS.driver=com.mysql.jdbc.Driver
org.quartz.dataSource.myDS.maxConnections=10
org.quartz.dataSource.myDS.validationQuery=select 0 from dual
org.quartz.scheduler.batchTriggerAcquisitionMaxCount =10
org.quartz.dataSource.myDS.URL=jdbc:mysql://127.0.0.1:3306/quartz
org.quartz.dataSource.myDS.user=root
org.quartz.dataSource.myDS.password=root
This is my quartz properties file.
I wanted to know if there any better approach to get the ProductInfo for all the 100k products.
One approach
I schedule 100k jobs for all the ProductIds. and while quartz running on a clustered environment will schedule according to the instance available.
org.quartz.threadPool.threadCount=80 - this property states that in one instance of the service at most 80 threads can take up the jobs. Is it?
And if I have 2 instances running then at least 100-160 jobs can run concurrently. Am I correct in my approach? which can reduce the time by a huge margin.
Is there any other better approach than this?
Note :- This is to demonstrate the approach the code can be further improvised(I have not tested the code and there can be syntactical errors)
public class ProductsJob extends QuartzJobBean {
#Autowired
private ProductClient productClient;
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
List<Long> ProductsIds = fetchAllProductIdsFromDb();
ProductsExecutor productsExecutor=new ProductsExecutor();
productsExecutor.runTasks(ProductsIds);
}
}
public class ProductsExecutor {
private int threadPoolSize=10;
private ExecutorService executorService;
public ProductsExecutor()
{
executorService =Executors.newFixedThreadPool(threadPoolSize);
}
public void runTasks(List<Integer> productIds)
{
//Loop and execute the task and wait for them it necessary
ProductsWorker worker=new ProductsWorker();
worker.setProductId(productId);
executorService.invokeAll(worker);
}
}
public class ProductsWorker implements Runnable {
private int productId;
#Override
public void run() {
ProductClientRequestBody requestBody = new ProductClientRequestBody();
requestBody.putIdInsideTheRequestBody(productId);
response = productClient.getResult(requestBody).execute();
if (response != null && response.isSuccessful()) {
log.info("We have got successful response for {}");
}
}
public void setProductId(int productId)
{
this.productId=productId;
}
}
Don't forget to destroy your executor service as with each quartz run a new executor service will be created.Also you can add awaitTermination for all the submitted tasks.
I have a question about a general design pattern in EJB. I hava Java EE application (EJBs and Web) and I need a kind of background process which is permanently scanning and processing specific data via JPA.
One solution, I think about, is to implement a #Singleton EJB. In a method annotated with #PostConstruct I can start my process.
#Singleton
#Startup
public class MyUpdateService {
#PostConstruct
void init() {
while(true) {
// scann for new data...
// do the job.....
}
}
}
But is this a recommended pattern? Or is there a better way to run such a class in an EJB Container?
In EJBs there are the other patterns like #TimerService and the new Java EE7 batch processing. But both concepts - I think - are used for finite Processes?
Using EJB TimerService in current project for tasks like periodic data pruning, or back-end data synchronization. It allows not only single time execution, but also interval timers and timers with calendar based schedule.
Smth like:
#Startup
#Singleton
public class SyncTimer {
private static final long HOUR = 60 * 60 * 1000L;
#Resource
private TimerService timerService;
private Timer timer;
#PostConstruct
public void init() {
TimerConfig config = new TimerConfig();
config.setPersistent(false);
timer = timerService.createIntervalTimer(HOUR, HOUR, config);
}
#Timeout
private synchronized void onTimer() {
// every hour action
}
}
As an alternative to the TimerSerivce mentioned by #devmind since Java EE 7 it is possible to use the ManagedScheduledExecutorService:
#Startup
#Singleton
public class Scheduler {
static final long INITIAL_DELAY = 0;
static final long PERIOD = 2;
#Resource
ManagedScheduledExecutorService scheduler;
#PostConstruct
public void init() {
this.scheduler.scheduleAtFixedRate(this::invokePeriodically,
INITIAL_DELAY, PERIOD,
TimeUnit.SECONDS);
}
public void invokePeriodically() {
System.out.println("Don't use sout in prod " + LocalTime.now());
}
}
In difference to the TimerSerivce the ExecutorService can be run in parallel in separate tasks. See also a blog post form Adam Bien.