We Keep Coding

Spring - Dynamically add and remove scheduled task

I am working on spring based server application. Basically it will poll scores of various sporting events in very short interval and save in db. For polling there will be many(can be around 100) calls to different apis concurrently at regular interval for example some api call will have 3 seconds interval some have 5 seconds etc., server will keep polling for latest data at frequent interval.
These calls will be added and removed dynamically. I have little experience in using spring. I think I have to use some scheduler. Can anyone point in right direction what approach or which scheduler is best in this scenario.

In essence you want to inject an instance of a scheduling task executor
#Configuration
public class MyApplicationConfiguration {
#Bean
public ThreadPoolTaskScheduler threadPoolTaskScheduler() {
ThreadPoolTaskScheduler tpts = new ThreadPoolTaskScheduler();
// maybe configure it a little?
return tpts;
}
#Bean
public MyService myService() {
return new MyService();
}
}
class MyService {
#Autowired
private ThreadPoolTaskScheduler tpts;
public void doSomething() {
Runnable task = ...
tpts.scheduleWithFixedDelay(task, 1000);
}
}
You can see a reasonable guide here, or the SchedulingTaskExecutor Javadoc and the Spring Task Execution and Scheduling Reference

You can use #Scheduled Spring Annotation for this. Refer this link for examples.

java scheduler spring vs quartz

Currently I am building a spring standalone program in order to learn new methods and architectures.
The last few days I tried to learn scheduler. I never used them before so I read some articles handling the different possible methods. Two of them are especially interesting: The spring nativ #Scheduler and Quartz.
From what I read, Spring is a little bit smaller then Quartz and much more basic. And quartz is not easy to use with spring (because of the autowired and components).
My problem now is, that there is one thing I do not understand:
From my understanding, both methods are creating parallel Threads in order to asynchronously run the jobs. But what if I now have a spring #Service in my main Application, that is holding a HashMap with some information. The data is updated and changed with user interaction. Parallel there are the scheduler. And a scheduler now whants to use this HashMap from the main application as well. Is this even possible?
Or do I understand something wrong? Because there is also the #Async annotation and I did not understand the difference. Because a scheduler itself is already parallel to the main corpus, isn't it?
(summing up, two questions:
can a job that is executed every five seconds, implemented with a scheduler, use a HashMap out of a service inside the main program? (in spring #Scheduler and/or in Quartz?)
Why is there a #Async annotation. Isn't a scheduler already parallel to the main process?
)

I have to make a few assumptions about which version of Spring you're using but as you're in the process of learning, I would assume that you're using spring-boot or a fairly new version, so please excuse if the annotations don't match your version of Spring. This said, to answer your two questions the best I can:
can a job that is executed every five seconds, implemented with a scheduler, use a HashMap out of a service inside the main program? (in spring #Scheduler and/or in Quartz?)
Yes, absolutely! The easiest way is to make sure that the hashmap in question is declared as static. To access the hashmap from the scheduled job, simply either autowire your service class or create a static get function for the hashmap.
Here is an example of a recent Vaadin project where I needed a scheduled message sent to a set of subscribers.
SchedulerConfig.class
#Configuration
#EnableAsync
#EnableScheduling
public class SchedulerConfig {
#Scheduled(fixedDelay=5000)
public void refreshVaadinUIs() {
Broadcaster.broadcast(
new BroadcastMessage(
BroadcastMessageType.AUTO_REFRESH_LIST
)
);
}
}
Broadcaster.class
public class Broadcaster implements Serializable {
private static final long serialVersionUID = 3540459607283346649L;
private static ExecutorService executorService = Executors.newSingleThreadExecutor();
private static LinkedList<BroadcastListener> listeners = new LinkedList<BroadcastListener>();
public interface BroadcastListener {
void receiveBroadcast(BroadcastMessage message);
}
public static synchronized void register(BroadcastListener listener) {
listeners.add(listener);
}
public static synchronized void unregister(BroadcastListener listener) {
listeners.remove(listener);
}
public static synchronized void broadcast(final BroadcastMessage message) {
for (final BroadcastListener listener: listeners)
executorService.execute(new Runnable() {
#Override
public void run() {
listener.receiveBroadcast(message);
}
});
}
}
Why is there a #Async annotation. Isn't a scheduler already parallel to the main process?
Yes, the scheduler is running in its own thread but what occurs to the scheduler on long running tasks (ie: doing a SOAP call to a remote server that takes a very long time to complete)?
The #Async annotation isn't required for scheduling but if you have a long running function being invoked by the scheduler, it becomes quite important.
This annotation is used to take a specific task and request to Spring's TaskExecutor to execute it on its own thread instead of the current thread. The #Async annotation causes the function to immediately return but execution will be later made by the TaskExecutor.
This said, without the #EnableAsync or #Async annotation, the functions you call will hold up the TaskScheduler as they will be executed on the same thread. On a long running operation, this would cause the scheduler to be held up and unable to execute any other scheduled functions until it returns.
I would suggest a read of Spring's Documentation about Task Execution and Scheduling It provides a great explanation of the TaskScheduler and TaskExecutor in Spring

Scheduling a Task in Spring MVC

In my Spring MVC application i need to schedule a task with specific date & Time. Like- i have to schedule to send a email which will be configured dynamically by customer. In Spring #Schedule annotation is there but how can i change value dynamically every time with any date & Time.
Any help is appreciated.

You should try TaskScheduler, see the javadoc here:
private TaskScheduler scheduler = new ConcurrentTaskScheduler();
#PostConstruct
private void executeJob() {
scheduler.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
// your business here
}
}, INTERVAL);
}

Refer Spring Task Execution and Scheduling
Example Annotations
#Configuration
#EnableAsync
#EnableScheduling
public class MyComponent {
#Async
#Scheduled(fixedDelay=5000, repeatCount=0)
public void doSomething() {
// something that should execute periodically
}
}
I think the repeatCount=0 will make the function execute only once (yet to test)
Full example with Quartz scheduler http://www.mkyong.com/spring/spring-quartz-scheduler-example/
You need to introduce XML configuration as follows
<task:annotation-driven executor="myExecutor" scheduler="myScheduler"/>
<task:executor id="myExecutor" pool-size="5"/>
<task:scheduler id="myScheduler" pool-size="10"/>}

You can achieve this easily within standard java API, by scheduling task for time difference between the time task is created and target date entered by customers. Simply provide this difference as parameter delay.
ScheduledThreadPoolExecutor
schedule(Callable<V> callable, long delay, TimeUnit unit)
Creates and executes a ScheduledFuture that becomes enabled after the given delay.
ScheduledFuture<?> schedule(Runnable command, long delay, TimeUnit unit)
Creates and executes a one-shot action that becomes enabled after the given delay.
So you either have to submit Runnable or Callable to this service.
You can refer to this answer for calculation between dates:
time difference

Does spring #Scheduled annotated methods runs on different threads?

I have several methods annotated with #Scheduled(fixedDelay=10000).
In the application context, I have this annotation-driven setup:
<task:annotation-driven />
The problem is, sometimes some of the method executions get delayed by seconds and even minutes.
I'm assuming that even if a method takes a while to finish executing, the other methods would still execute. So I don't understand the delay.
Is there a way to maybe lessen or even remove the delay?

For completeness, code below shows the simplest possible way to configure scheduler with java config:
#Configuration
#EnableScheduling
public class SpringConfiguration {
#Bean(destroyMethod = "shutdown")
public Executor taskScheduler() {
return Executors.newScheduledThreadPool(5);
}
...
When more control is desired, a #Configuration class may implement SchedulingConfigurer.

The documentation about scheduling says:
If you do not provide a pool-size attribute, the default thread pool will only have a single thread.
So if you have many scheduled tasks, you should configure the scheduler, as explained in the documentation, to have a pool with more threads, to make sure one long task doesn't delay all the other ones.

If you're using Spring Boot:
There is also a property you can set in your application properties file that increases the pool size:
spring.task.scheduling.pool.size=10
Seems to be there since Spring Boot 2.1.0.

A method annotated with #Scheduled is meant to be run separately, on a different thread at a moment in time.
If you haven't provided a TaskScheduler in your configuration, Spring will use
Executors.newSingleThreadScheduledExecutor();
which returns an ScheduledExecutorService that runs on a single thread. As such, if you have multiple #Scheduled methods, although they are scheduled, they each need to wait for the thread to complete executing the previous task. You might keep getting bigger and bigger delays as the the queue fills up faster than it empties out.
Make sure you configure your scheduling environment with an appropriate amount of threads.

The #EnableScheduling annotation provides the key information and how to resolve it:
By default, will be searching for an associated scheduler definition:
either a unique TaskScheduler bean in the context, or a TaskScheduler
bean named "taskScheduler" otherwise; the same lookup will also be
performed for a ScheduledExecutorService bean. If neither of the two
is resolvable, a local single-threaded default scheduler will be
created and used within the registrar.
When more control is desired, a #Configuration class may implement
SchedulingConfigurer. This allows access to the underlying
ScheduledTaskRegistrar instance. For example, the following example
demonstrates how to customize the Executor used to execute scheduled
tasks:
#Configuration
#EnableScheduling
public class AppConfig implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
}
#Bean(destroyMethod="shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(100);
}
}
(emphasis added)

you can use:
#Bean()
public ThreadPoolTaskScheduler taskScheduler(){
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.setPoolSize(2);
return taskScheduler;
}

Use below link for the reference..great explanation and implementation:
https://crmepham.github.io/spring-boot-multi-thread-scheduling/#:~:text=By%20default%20Spring%20Boot%20will,there%20is%20enough%20threads%20available).

Using XML file add below lines..
<task:scheduler id="taskScheduler" pool-size="15" />
<task:scheduled-tasks scheduler="taskScheduler" >
....
</task:scheduled-tasks>

default spring using a single thread for schedule task. you can using #Configuration for class implements SchedulingConfigurer . referce: https://crmepham.github.io/spring-boot-multi-thread-scheduling/

We need to pass our own thread pool scheduler, otherwise it will use default single threaded executor. Have added below code to fix-
#Bean
public Executor scheduledTaskThreadPool() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(10);
executor.setThreadNamePrefix("name-");
executor.initialize();
return executor;
}

Running a Job only once Using Quartz

Is there a way I could run a job only once using Quartz in Java? I understand it does not make sense to use Quartz in this case. But, the thing is, I have multiple jobs and they are run multiple times. So, I am using Quartz.
Is this even possible?

You should use SimpleTrigger that fires at specific time and without repeating. TriggerUtils has many handy methods for creating these kind of things.

Yes, it's possible!
JobKey jobKey = new JobKey("testJob");
JobDetail job = newJob(TestJob.class)
.withIdentity(jobKey)
.storeDurably()
.build();
scheduler.addJob(job, true);
scheduler.triggerJob(jobKey); //trigger a job inmediately

In quartz > 2.0, you can get the scheduler to unschedule any job after work is done:
#Override
protected void execute(JobExecutionContext context)
throws JobExecutionException {
...
// process execution
...
context.getScheduler().unscheduleJob(triggerKey);
...
}
where triggerKey is the ID of the job to run only once. After this, the job wouldn't be called anymore.

Here is an example of how to run a TestJob class immediately with Quartz 2.x:
public JobKey runJob(String jobName)
{
// if you don't call startAt() then the current time (immediately) is assumed.
Trigger runOnceTrigger = TriggerBuilder.newTrigger().build();
JobKey jobKey = new JobKey(jobName);
JobDetail job = JobBuilder.newJob(TestJob.class).withIdentity(jobKey).build();
scheduler.scheduleJob(job, runOnceTrigger);
return jobKey;
}
see also Quartz Enterprise Job Scheduler Tutorials → SimpleTriggers

I'm not sure how much similar is Quartz in Mono and Java but this seems working in .Net
TriggerBuilder.Create ()
.StartNow ()
.Build ();

I had to ask myself if it made sense to try to configure a job and add checks if it had been run already as suggested in Marko Lahma's answer (since scheduling a job to run once results in it being run once, every time we start the app). I found examples of CommandLineRunner apps which didn't quite work for me, mostly because we already had an ApplicationRunner which was used for other jobs which use Quartz scheduling / cron. I wasn't happy with having Quartz initialize this job using a SimpleTrigger, so I had to find something else.
Using some ideas from the following articles:
Multiple Spring boot CommandLineRunner based on command line argument
Run Spring Batch Job programmatically?
Firing Quartz jobs manually
Is there any way to get job keys in Quartz by job name
How to list all Jobs in the Quartz Scheduler
Spring Boot CommandLineRunner and ApplicationRunner
I was able to piece together a working implementation which allows me to do the following:
run existing jobs via Quartz, on a timer
run new job, one time programmatically (single use Quartz job using the SimpleTrigger didn't satisfy my requirements, since it would be run once on every application load)
I came up with the following CommandLineRunner class:
public class BatchCommandLineRunner implements CommandLineRunner {
#Autowired
private Scheduler scheduler;
private static final Logger LOGGER = LoggerFactory.getLogger(BatchCommandLineRunner.class);
public void run(final String... args) throws SchedulerException {
LOGGER.info("BatchCommandLineRunner: running with args -> " + Arrays.toString(args));
for (final String jobName : args) {
final JobKey jobKey = findJobKey(jobName);
if (jobKey != null) {
LOGGER.info("Triggering job for: " + jobName);
scheduler.triggerJob(jobKey);
} else {
LOGGER.info("No job found for jobName: " + jobName);
}
}
}
private JobKey findJobKey(final String jobNameToFind) throws SchedulerException {
for (final JobKey jobKey : scheduler.getJobKeys(GroupMatcher.jobGroupEquals("DEFAULT"))) {
final String jobName = jobKey.getName();
if (jobName.equals(jobNameToFind)) {
return jobKey;
}
}
return null;
}
}
In one of my configuration classes I added a CommandLineRunner bean which calls the custom CommandLineRunner I created:
#Configuration
public class BatchConfiguration {
private static final Logger LOGGER = LoggerFactory.getLogger(BatchConfiguration.class);
#Bean
public BatchCommandLineRunner batchCommandLineRunner() {
return new BatchCommandLineRunner();
}
#Bean
public CommandLineRunner runCommandLineArgs(final ApplicationArguments applicationArguments) throws Exception {
final List<String> jobNames = applicationArguments.getOptionValues("jobName");
LOGGER.info("runCommandLineArgs: running the following jobs -> " + ArrayUtils.toString(jobNames));
batchCommandLineRunner().run(jobNames.toArray(ArrayUtils.EMPTY_STRING_ARRAY));
return null;
}
}
Later, I am able to initiate these jobs via the CLI without affecting my current Quartz scheduled jobs, and as long as no one runs the command via CLI multiple times, it will never be run again. I have to do some juggling of types since I accept ApplicationArguments, and then convert them into String[].
Finally, I am able to call it like this:
java -jar <your_application>.jar --jobName=<QuartzRegisteredJobDetailFactoryBean>
The result is that the job is initialized only when I call it, and it is excluded from my CronTriggerFactoryBean triggers which I used for my other jobs.
There are several assumptions being made here, so I'll try to summarize:
the job must be registered as a JobDetailFactoryBean (e.g.: scheduler.setJobDetails(...))
everything is essentially the same as a job with CronTriggerFactoryBean, excepting the lacking scheduler.setTriggers(...) call
Spring knows to execute the CommandLineRunner classes after the application has booted
I hardcoded the parameter being passed into the application to "jobName"
I assumed a group name of "DEFAULT" for all jobs; if you want to use differing groups this would need to be adjusted when fetching JobKey, which is used to actually run the job
there is nothing which prevents this job from being run multiple times via CLI, but it was triggered on every application load using SimpleTrigger approach, so this is better for me; if this is not acceptable, perhaps using StepListener and ExitStatus, etc. can prevent it from being executed twice

Another solution: There is a method .withRepeatCount(0) in SimpleSchedulerBuilder:
public final int TEN_SECONDS = 10;
Trigger trigger = newTrigger()
.withIdentity("myJob", "myJobGroup")
.startAt(new Date(System.currentMillis()+TEN_SECONDS*1000)
.withSchedule(SimpleScheduleBuilder.simpleSchedule()
.withRepeatCount(0)
.withIntervalInMinutes(1))
.build();