I am coming from a (mostly) Python Django/Celery background and starting with Spring Boot.
I am having a hard time to understand how do you separate the roles.
For example when having a Django/Celery project, I will have one one side the web backends started as gunicorn, and on the other side the workers started as celery (so, different commands, but pointing at the same code).
But on Spring Boot, you only have a single entrypoint, and as soon as the scheduler is injected, the jobs will start being processed.
What is the correct way do separate those like in a Django/Celery application ?
Should I put almost all my code as a library and then create 2 final applications, one that will setup the #DispatcherServlet and another one that will setup #EnableScheduling, or is there some kind of configuration to be injected at runtime ?
In my opinion, if 'the web' and 'the scheduler' are both the important function in the application, then we don't need to separate them as long as you are creating a monolithic application.
Because you are using spring boot, so #DispatcherServlet and all the other web component that a web application needed will be injected and configured automatically. The only thing you have to do is creating some class that annotated with #Controller or #RestController and set up the #RequestMapping methods inside those class.
How about Scheduler? you need to add #EnableScheduling in one of the #Configuration class first, then create Scheduler class in scheduler package like below code sample.
You can use cron property to set up the specify execute time just like Linux crontab. The jobs will start being processed only if the cron time is up.
#Component
public class PlatformScheduler {
#Autowired
private BatchService batchService;
#Scheduled(cron = "0 0 12 * * *")
public void dailyInitialize() {
clearCompletedBatches();
queryBatchesToRunToday();
}
#Scheduled(fixedRate = 10000, initialDelay = 10000)
private void harvestCompletedBatches() {
batchService.harvestCompletedBatches();
}
#Scheduled(fixedRate = 10000, initialDelay = 10000)
private void executeWaitingBatches() {
batchService.executeWaitingBatches(new DateTime());
}
}
The most simple project hierarchy will be like below, 'the web' and 'the scheduler' can be in the same project safely and share the same #Service components without harm.
Related
My java web application that uses spring for dependency injection is packaged in an EAR and is deployed in Jboss 7, but has no EJB. The application is installed on two load balancing machines. I need to schedule a method to run daily, but that method can't run at the same time on both instances.
I tried to use Spring's Scheduling annotations, but the problem is that, as there is load balancing, the scheduled method runs twice (once in each cluster).
What is the best way to do this in Jboss 7? Can someone help me ?
The method to be scheduled looks like the one below.
public synchronized void processor() {
LOGGER.info("start");
//processing logic
LOGGER.info("the end");
}
Thanks a lot!!!
Well, considering the requirements: two or more apps and they need to be synchronized, you need either #Singleton or #Stateless EJB, described here.
Invoking it via the timer service, then it needs to be an EJB with #Timer on some method and if you use #Scheduled or such on a method, then it will invoke that method
On this case a Singleton is recommended, otherwise, you might end up with multiple instances of the same timer running.
Example
#Example
private void init()
{
ScheduleExpression Expression = new ScheduleExpression();
#This means twice per hour {0,2,4, ... 22} ~ since it ends on 23h:
expression.second(0).minute(0).hour(*/2).month(*).dayOfWeek(*);
stopTimer();
Timer timer = service.createCalendarTimer(exp);
}
Any other suggestion seems to add too much complexity.
I need to implement Spring process which checks database table for new rows and makes calculations. I'm thinking to implement infinite loop which is triggered every 10 minutes.
Is there someway to implement this with Spring Boot? I can always use Java Thread but it's for sure it's better to let Spring to manage this.
You can try scheduling with Spring Schedule
Here is a official example
Technically, you enable scheduling with #EnableScheduling and annotated your task with #Scheduled(fixedRate=600000).
Another config you can used to tune your scheduler:
fixedRate: Execute the annotated method with a fixed period in milliseconds between invocations.
fixedDelay: Execute the annotated method with a fixed period in milliseconds between the end of the last invocation and the start of the next.
cron: A cron-like expression, extending the usual UN*X definition to include triggers on the second as well as minute, hour, day of month, month and day of week.
find the below sample code
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
#Component
public class AppScheduler {
#Scheduled(fixedRate = 10000)
public void myScheduler() {
System.out.println("Test print");
}
}
It looks like this questions is old, but I would like to answer.
See, you can make an object of ThreadPoolTaskExecutor and assign the background process to it.
Here is my detailed code if anyone wants to see.
https://github.com/xbox2204/SpringBoot-JPA-Swagger-Angular
First thing to do would be, add these two annotation in your application starter class.
#EnableAsync
#EnableScheduling
Now, create you TaskExceutor in the same class and assign it to bean with a name, so that background tasks can be created anywhere in your application and attached with this bean.
#Bean(name="thPlExectr")
public Executor getExecutor(){
return new ThreadPoolTaskExecutor();
}
Now create a component class somewhere in the project, where you you will create the background task.
Here, you will mention the frequency with which you want your task to be exceuted. I wanted to print a statement every 5 second, you can choose your own frequency and give your own method definiton. Also, make it async and attach it with the TaskExecutor bean mentioned above, so that it doesn't interrupt the normal flow of your application.
#Component
public class BackgroundTasks {
private static final Logger log= LoggerFactory.getLogger(BackgroundTasks.class);
#Async("thPlExectr")
#Scheduled(fixedRate = 5000)
public void backTask(){
log.info("Hi Vineet! I am a background task");
}
}
I have an application that uses Spring Batch to define a preset number of jobs, which are currently all defined in the XML.
We add more jobs over time which requires updating the XML, however these jobs are always based on the same parent and can easily be predetermined using a simple SQL query.
So I've been trying to switch to use some combination of XML configuration and Java-based configuration but am quickly getting confused.
Even though we have many jobs, each job definition falls into essentially one of two categories. All of the jobs inherit from one or the other parent job and are effectively identical, besides having different names. The job name is used in the process to select different data from the database.
I've come up with some code much like the following but have run into problems getting it to work.
Full disclaimer that I'm also not entirely sure I'm going about this in the right way. More on that in a second; first, the code:
#Configuration
#EnableBatchProcessing
public class DynamicJobConfigurer extends DefaultBatchConfigurer implements InitializingBean {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private JobRegistry jobRegistry;
#Autowired
private DataSource dataSource;
#Autowired
private CustomJobDefinitionService customJobDefinitionService;
private Flow injectedFlow1;
private Flow injectedFlow2;
public void setupJobs() throws DuplicateJobException {
List<JobDefinition> jobDefinitions = customJobDefinitionService.getAllJobDefinitions();
for (JobDefinition jobDefinition : jobDefinitions) {
Job job = null;
if (jobDefinition.getType() == 1) {
job = jobBuilderFactory.get(jobDefinition.getName())
.start(injectedFlow1).build()
.build();
} else if (jobDefinition.getType() == 2) {
job = jobBuilderFactory.get(jobDefinition.getName())
.start(injectedFlow2).build()
.build();
}
if (job != null) {
jobRegistry.register(new ReferenceJobFactory(job));
}
}
}
#Override
public void afterPropertiesSet() throws Exception {
setupJobs();
}
public void setInjectedFlow1(Flow injectedFlow1) {
this.injectedFlow1 = injectedFlow1;
}
public void setInjectedFlow2(Flow injectedFlow2) {
this.injectedFlow2 = injectedFlow2;
}
}
I have the flows that get injected defined in the XML, much like this:
<batch:flow id="injectedFlow1">
<batch:step id="InjectedFlow1.Step1" next="InjectedFlow1.Step2">
<batch:flow parent="InjectedFlow.Step1" />
</batch:step>
<batch:step id="InjectedFlow1.Step2">
<batch:flow parent="InjectedFlow.Step2" />
</batch:step>
</batch:flow>
So as you can see, I'm effectively kicking off the setupJobs() method (which is intended to dynamically create these job definitions) from the afterPropertiesSet() method of InitializingBean. I'm not sure that's right. It is running, but I'm not sure if there's a different entry point that's better intended for this purpose. Also I'm not sure what the point of the #Configuration annotation is to be honest.
The problem I'm currently running into is as soon as I call register() from JobRegistry, it throws the following IllegalStateException:
To use the default BatchConfigurer the context must contain no more than one DataSource, found 2.
Note: my project actually has two data sources defined. The first is the default dataSource bean which connects to the database that Spring Batch uses. The second data source is an external database, and this second one contains all the information I need to define my list of jobs. But the main one does use the default name "dataSource" so I'm not quite sure how else I can tell it to use that one.
First of all - I don't recommend using a combination of XML as well as Java Configuration. Use only one, preferably Java one as its not much of an effort to convert XML config to Java config. (Unless you have some very good reasons to do it - that you haven't explained)
I haven't used Spring Batch alone as I have always used it with Spring Boot and I have a project where I have defined multiple jobs and it always worked well for similar code that you have shown.
For your issue, there are some answers on SO like this OR this which are basically trying to say that you need to write your own BatchConfigurer and not rely on default one.
Now coming to solution using Spring Boot
With Spring Boot, You should try segregate job definitions and job executions.
You should first try to just define jobs and initialize Spring context without enabling jobs (spring.batch.job.enabled=false)
In your Spring Boot main method, when you start app with something like - SpringApplication.run(Application.class, args); ...you will get ApplicationContext ctx
Now you can get your relevant beans from this context & launch specif jobs by getting names from property or command line etc & using JobLauncher.run(...) method.
You can refer my this answer if willing to order job executions. You can also write job schedulers using Java.
Point being, you separate your job building / bean configurations & job execution concerns.
Challenge
Keeping multiple jobs in a single project can be challenging when you try to have different settings for each job as application.properties file is environment specific and not job specific i.e. spring boot properties will apply to all jobs.
In my particular case, the solution was to actually eliminate the #Configuration
and #EnableBatchProcessing annotations from my class above. Something about these caused it to try and use the DefaultBatchConfigurer which fails when you have more than one data source defined (even if you've identified them clearly with "dataSource" as the primary and some other name for the secondary).
The #Configuration class in particular wasn't necessary because all it really does is lets your class get auto-instantiated without having to define it as a bean in the app context. But since I was doing that anyway this one was superfluous.
One of the downsides of removing #EnableBatchProcessing was that I could no longer auto-wire the JobBuilderFactory bean. So instead I just had to do to create it:
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(dataSource);
factory.setTransactionManager(transactionManager);
factory.afterPropertiesSet();
jobRepository = factory.getObject();
jobBuilderFactory = new JobBuilderFactory(jobRepository);
Then it seems I was on the right track already by using jobRegistry.register(...) to define my jobs. So essentially once I removed those annotations above everything started working. I'm going to mark Sabir's answer as the correct one however because it helped me out.
I have a Spring Boot Application with a bunch of background jobs. I have added the following Annotation on my main application class:
#SpringBootApplication
#EnableScheduling
public class MyApplication {
In the job class, I have following configuration:
#Component
public class MyTask {
#Scheduled(fixedDelay = 14400000)
public void doSomething()
Right now, Spring Boot is executing the jobs in a sequential manner, i.e., one job at a time. This seems most likely due to a single thread based pool.
Is there any Annotation/property that can be used to increase the thread pool size?
Till now, I have found a solution here, but it requires writing a new Configuration class.
Ideally, it should be a property in application.properties file.
I usually don't put business logic inside a #Scheduled method, instead, I call another method in other component and this method has the #Async annotation. When your scheduled job is fired, then it calls the async method in another thread and you scheduler is free to run other jobs.
Check more how to do it here: https://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html#scheduling-annotation-support
I don't see a property for this in https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html and I don't see any annotation in the docs.
If you want it configurable at that level, just create your own custom properties, which you inject into the other solution you found.
I have two tomee instances clustered.
Each one have a method annotated like
#Schedule(dayOfWeek = "*")
public void runMeDaily() {...}
I'd like to run this method only once a day. Not twice a day (one on each instance)
I could use a flag as described here Run #Scheduled task only on one WebLogic cluster node? or just elect some node, but I wonder if there's a more elegant way to do that.
This question is somewhat related to EJB3.1 #Schedule in clustered environment but I am not using JBOSS. (and it's not answered)
Im using same approach as in other thread - checking that particular host is the correct one to run job. But..
Im not very info ee tools, but in spring you can use profiles for that. Probably you can find similar solution for your needs. Take a look at http://spring.io/blog/2011/06/21/spring-3-1-m2-testing-with-configuration-classes-and-profiles
You can define two seperate beans:
#Configuration
#Profile("dev")
public class StandaloneDataConfig {
#Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.HSQL)
.addScript("classpath:com/bank/config/sql/schema.sql")
.addScript("classpath:com/bank/config/sql/test-data.sql")
.build();
}
}
#Configuration
#Profile("production")
public class JndiDataConfig {
#Bean
public DataSource dataSource() throws Exception {
Context ctx = new InitialContext();
return (DataSource) ctx.lookup("java:comp/env/jdbc/datasource");
}
}
and decide which one to turn on by switching profile. So your class with method annotated #Scheduled would be loaded only for specific profile. Ofcourse then you need to configure your app to turn on profile on of the nodes only. In spring app it would be as simple as passing -Dspring.profiles.active=profile to one of them.
I could only solve this using a non-Java EE solution, specific to the platform (proprietary). In my case, I am using TomEE+ and Quartz. Running Quartz in the clustered mode (org.quartz.jobStore.isClustered = true) and persisting the timers in a single database forces Quartz to choose an instance to trigger the timer, so it will only run once.
This link was very useful -- http://rmannibucau.wordpress.com/2012/08/22/tomee-quartz-configuration-for-scheduled-methods/
It's a shame Java EE does not specify a behavior for that. (yet, I hope) :-)
I solved this problem by making one of the box as master. basically set an environment variable on one of the box like master=true.
and read it in your java code through system.getenv("master"). if its present and its true then run your code.
basic snippet
#schedule()
void process(){
boolean master=Boolean.parseBoolean(system.getenv("master"));
if(master)
{
//your logic
}
}