I have a Spring Boot Application with a bunch of background jobs. I have added the following Annotation on my main application class:
#SpringBootApplication
#EnableScheduling
public class MyApplication {
In the job class, I have following configuration:
#Component
public class MyTask {
#Scheduled(fixedDelay = 14400000)
public void doSomething()
Right now, Spring Boot is executing the jobs in a sequential manner, i.e., one job at a time. This seems most likely due to a single thread based pool.
Is there any Annotation/property that can be used to increase the thread pool size?
Till now, I have found a solution here, but it requires writing a new Configuration class.
Ideally, it should be a property in application.properties file.
I usually don't put business logic inside a #Scheduled method, instead, I call another method in other component and this method has the #Async annotation. When your scheduled job is fired, then it calls the async method in another thread and you scheduler is free to run other jobs.
Check more how to do it here: https://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html#scheduling-annotation-support
I don't see a property for this in https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html and I don't see any annotation in the docs.
If you want it configurable at that level, just create your own custom properties, which you inject into the other solution you found.
Related
I am coming from a (mostly) Python Django/Celery background and starting with Spring Boot.
I am having a hard time to understand how do you separate the roles.
For example when having a Django/Celery project, I will have one one side the web backends started as gunicorn, and on the other side the workers started as celery (so, different commands, but pointing at the same code).
But on Spring Boot, you only have a single entrypoint, and as soon as the scheduler is injected, the jobs will start being processed.
What is the correct way do separate those like in a Django/Celery application ?
Should I put almost all my code as a library and then create 2 final applications, one that will setup the #DispatcherServlet and another one that will setup #EnableScheduling, or is there some kind of configuration to be injected at runtime ?
In my opinion, if 'the web' and 'the scheduler' are both the important function in the application, then we don't need to separate them as long as you are creating a monolithic application.
Because you are using spring boot, so #DispatcherServlet and all the other web component that a web application needed will be injected and configured automatically. The only thing you have to do is creating some class that annotated with #Controller or #RestController and set up the #RequestMapping methods inside those class.
How about Scheduler? you need to add #EnableScheduling in one of the #Configuration class first, then create Scheduler class in scheduler package like below code sample.
You can use cron property to set up the specify execute time just like Linux crontab. The jobs will start being processed only if the cron time is up.
#Component
public class PlatformScheduler {
#Autowired
private BatchService batchService;
#Scheduled(cron = "0 0 12 * * *")
public void dailyInitialize() {
clearCompletedBatches();
queryBatchesToRunToday();
}
#Scheduled(fixedRate = 10000, initialDelay = 10000)
private void harvestCompletedBatches() {
batchService.harvestCompletedBatches();
}
#Scheduled(fixedRate = 10000, initialDelay = 10000)
private void executeWaitingBatches() {
batchService.executeWaitingBatches(new DateTime());
}
}
The most simple project hierarchy will be like below, 'the web' and 'the scheduler' can be in the same project safely and share the same #Service components without harm.
I need to implement Spring process which checks database table for new rows and makes calculations. I'm thinking to implement infinite loop which is triggered every 10 minutes.
Is there someway to implement this with Spring Boot? I can always use Java Thread but it's for sure it's better to let Spring to manage this.
You can try scheduling with Spring Schedule
Here is a official example
Technically, you enable scheduling with #EnableScheduling and annotated your task with #Scheduled(fixedRate=600000).
Another config you can used to tune your scheduler:
fixedRate: Execute the annotated method with a fixed period in milliseconds between invocations.
fixedDelay: Execute the annotated method with a fixed period in milliseconds between the end of the last invocation and the start of the next.
cron: A cron-like expression, extending the usual UN*X definition to include triggers on the second as well as minute, hour, day of month, month and day of week.
find the below sample code
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
#Component
public class AppScheduler {
#Scheduled(fixedRate = 10000)
public void myScheduler() {
System.out.println("Test print");
}
}
It looks like this questions is old, but I would like to answer.
See, you can make an object of ThreadPoolTaskExecutor and assign the background process to it.
Here is my detailed code if anyone wants to see.
https://github.com/xbox2204/SpringBoot-JPA-Swagger-Angular
First thing to do would be, add these two annotation in your application starter class.
#EnableAsync
#EnableScheduling
Now, create you TaskExceutor in the same class and assign it to bean with a name, so that background tasks can be created anywhere in your application and attached with this bean.
#Bean(name="thPlExectr")
public Executor getExecutor(){
return new ThreadPoolTaskExecutor();
}
Now create a component class somewhere in the project, where you you will create the background task.
Here, you will mention the frequency with which you want your task to be exceuted. I wanted to print a statement every 5 second, you can choose your own frequency and give your own method definiton. Also, make it async and attach it with the TaskExecutor bean mentioned above, so that it doesn't interrupt the normal flow of your application.
#Component
public class BackgroundTasks {
private static final Logger log= LoggerFactory.getLogger(BackgroundTasks.class);
#Async("thPlExectr")
#Scheduled(fixedRate = 5000)
public void backTask(){
log.info("Hi Vineet! I am a background task");
}
}
Currently I am building a spring standalone program in order to learn new methods and architectures.
The last few days I tried to learn scheduler. I never used them before so I read some articles handling the different possible methods. Two of them are especially interesting: The spring nativ #Scheduler and Quartz.
From what I read, Spring is a little bit smaller then Quartz and much more basic. And quartz is not easy to use with spring (because of the autowired and components).
My problem now is, that there is one thing I do not understand:
From my understanding, both methods are creating parallel Threads in order to asynchronously run the jobs. But what if I now have a spring #Service in my main Application, that is holding a HashMap with some information. The data is updated and changed with user interaction. Parallel there are the scheduler. And a scheduler now whants to use this HashMap from the main application as well. Is this even possible?
Or do I understand something wrong? Because there is also the #Async annotation and I did not understand the difference. Because a scheduler itself is already parallel to the main corpus, isn't it?
(summing up, two questions:
can a job that is executed every five seconds, implemented with a scheduler, use a HashMap out of a service inside the main program? (in spring #Scheduler and/or in Quartz?)
Why is there a #Async annotation. Isn't a scheduler already parallel to the main process?
)
I have to make a few assumptions about which version of Spring you're using but as you're in the process of learning, I would assume that you're using spring-boot or a fairly new version, so please excuse if the annotations don't match your version of Spring. This said, to answer your two questions the best I can:
can a job that is executed every five seconds, implemented with a scheduler, use a HashMap out of a service inside the main program? (in spring #Scheduler and/or in Quartz?)
Yes, absolutely! The easiest way is to make sure that the hashmap in question is declared as static. To access the hashmap from the scheduled job, simply either autowire your service class or create a static get function for the hashmap.
Here is an example of a recent Vaadin project where I needed a scheduled message sent to a set of subscribers.
SchedulerConfig.class
#Configuration
#EnableAsync
#EnableScheduling
public class SchedulerConfig {
#Scheduled(fixedDelay=5000)
public void refreshVaadinUIs() {
Broadcaster.broadcast(
new BroadcastMessage(
BroadcastMessageType.AUTO_REFRESH_LIST
)
);
}
}
Broadcaster.class
public class Broadcaster implements Serializable {
private static final long serialVersionUID = 3540459607283346649L;
private static ExecutorService executorService = Executors.newSingleThreadExecutor();
private static LinkedList<BroadcastListener> listeners = new LinkedList<BroadcastListener>();
public interface BroadcastListener {
void receiveBroadcast(BroadcastMessage message);
}
public static synchronized void register(BroadcastListener listener) {
listeners.add(listener);
}
public static synchronized void unregister(BroadcastListener listener) {
listeners.remove(listener);
}
public static synchronized void broadcast(final BroadcastMessage message) {
for (final BroadcastListener listener: listeners)
executorService.execute(new Runnable() {
#Override
public void run() {
listener.receiveBroadcast(message);
}
});
}
}
Why is there a #Async annotation. Isn't a scheduler already parallel to the main process?
Yes, the scheduler is running in its own thread but what occurs to the scheduler on long running tasks (ie: doing a SOAP call to a remote server that takes a very long time to complete)?
The #Async annotation isn't required for scheduling but if you have a long running function being invoked by the scheduler, it becomes quite important.
This annotation is used to take a specific task and request to Spring's TaskExecutor to execute it on its own thread instead of the current thread. The #Async annotation causes the function to immediately return but execution will be later made by the TaskExecutor.
This said, without the #EnableAsync or #Async annotation, the functions you call will hold up the TaskScheduler as they will be executed on the same thread. On a long running operation, this would cause the scheduler to be held up and unable to execute any other scheduled functions until it returns.
I would suggest a read of Spring's Documentation about Task Execution and Scheduling It provides a great explanation of the TaskScheduler and TaskExecutor in Spring
Got a pretty standard Spring Boot (1.3.5) application.
Enabled scheduling with #EnableScheduling (tried on main application entry point and a #Configuration annotated class.
Created a simple class with a #Scheduled method (simple fixedDelay schedule).
Scheduled task executes twice (always).
From what I have gathered so far, it is probably because two contexts are being loaded, and thusly picking up my beans twice.
Ok.
So how do I fix/prevent this double execution, since all the config is basically hidden Spring Boot magic?
Framework versions:
Spring Boot 1.3.5
Spring Cloud Brixton SR1
Main application:
#SpringBootApplication
#EnableDiscoveryClient
#EnableAsync
#EnableCircuitBreaker
public class AlertsApplication {
public static void main(final String[] args) {
SpringApplication.run(AlertsApplication.class, args);
}
}
My task class (HookCreateRequest list is pulled in from application.yml - I do not believe that to be relevant currently, but if required, can be provided):
#ConditionalOnProperty(name = "init.runner", havingValue = "InitRunner")
#ConfigurationProperties(prefix = "webhook")
public class InitRunner /*implements CommandLineRunner*/ {
private final List<HookCreateRequest> receivers = new ArrayList<>();
#Autowired
private WebHookService hookService;
#Scheduled (fixedRate = 300000)
public void run() throws Exception {
getReceivers().stream().forEach(item -> {
log.debug("Request : {}", item);
hookService.create(item);
});
}
public List<HookCreateRequest> getReceivers() {
return receivers;
}
}
There is zero xml configuration.
Not sure what else might be relevant?
EDIT 2016/07/04
I have modified to output the scheduled instance when it runs (I suspected that two different instances were being created). However, the logs seem to indicate it is the SAME instance of the task object.
logs:
15:01:16.170 DEBUG - scheduled.ScheduleHookRecreation - Schedule task running: scheduled.ScheduleHookRecreation#705a651b
...task stuff happening
...first run completes, then:
15:01:39.050 DEBUG - scheduled.ScheduleHookRecreation - Schedule task running: scheduled.ScheduleHookRecreation#705a651b
So it would seem it is the same task instance (#705a651b). Now why would in the name of sweet things would it be executed twice?
EDIT 2016/07/05
I added a #PostConstruct method to the class that carries the scheduled method, with just some logging output in. By doing that I could verify that the #PostConstruct method is being called twice - which seems to confirm that the bean is being picked up twice, which which presumably means it is fed to the scheduler twice. So how to prevent this?
Had the same problem, in my case the reason was in #Scheduled annotation's initialDelay parameter absence - method was called on application start.
I wanted to introduce #Async methods (for sending mails in parallel) in my SpringBoot application.
But when I put the #EnableAsync annotation on our application's main #Configuration class (annotated with #SpringBootApplication), the Flyway DB migrations are executed before the DataSourceInitializer (which runs schema.sql and data.sql for my tests) executed.
The first operation involving a 'should-be-migrated' database table fails.
Removing the #EnableAsync puts everything back to normal. Why does this happen and how could I fix this (or work around the issue)?
Update Some more findings: #EnableAsync(mode = AdviceMode.ASPECTJ) keeps the original order of DB setup, but the #Async method runs on the same thread as caller thread then. I also saw that the Bean 'objectPostProcessor' is created early (3rd bean) when #EnableAsync is not present, or #EnableAsync(mode = AdviceMode.ASPECTJ) is used. When only #EnableAsync is used, this bean is created much later.
Update 2 While I wasn't able to create a minimal project which reproduces the problem yet, I found out that the proper DB setup order is restored in my affected application when I comment out the #EnableWebSocketMessageBroker in the following:
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer
{
...
}
Bean 'webSocketConfig' is the first bean created (as per INFO-level console output) if #EnableWebSocketMessageBroker is present.
It turned out that having both #EnableAsync and #EnableWebSocketMessageBroker present in my application caused the described effect.
Removing one of it, restored the expected behavior, in which case the DataSourceInitializerPostProcessor created the DataSourceInitializer which triggered execution of schema.sql and data.sql, before flyway migrations took place.
When both annotations were present, the registration of the BeanPostProcessor named internalAsyncAnnotationProcessor happened before the DataSourceInitializerPostProcessor was registered.
The cause of the problem was that the registration of internalAsyncAnnotationProcessor caused the creation of the dataSource bean as a side effect. This side effect was caused by spring looking for a TaskExecutor bean to use, for the #Async method execution. spring unexpectedly picked up the clientInboundChannelExecutor bean which was present because of the #EnableWebSocketMessageBroker. Using this bean caused the instantiation of WebSocketMessagingAutoConfiguration which created the objectMapper bean (for json-serialization) which uses services that use DAO-repositories which depend on dataSource. So all those beans got created.
Because DataSourceInitializerPostProcessor wasn't even registered at that time, DataSourceInitializer was created much later, after the flyway migration took place.
The javadoc for #EnableAsync says the following:
By default, a SimpleAsyncTaskExecutor will be used to process async method invocations. Besides, annotated methods having a void return type cannot transmit any exception back to the caller. By default, such uncaught exceptions are only logged.
I assumed, that a SimpleAsyncTaskExecutor will be created to run the #Async methods, but instead spring picked up an existing bean with a matching type.
So the solution for this issue was to implement AsyncConfigurer, and provide my own Executor. This is also suggested in the javadoc of #EnableAsync:
To customize all this, implement AsyncConfigurer and provide:
* your own Executor through the getAsyncExecutor() method, and
* your own AsyncUncaughtExceptionHandler through the getAsyncUncaughtExceptionHandler() method.
With this tweak the DB setup is again executed as expected.