I am using spring scheduler tasks for calling a method in class after fixed interval like below
<task:scheduled-tasks scheduler="scheduler">
<task:scheduled ref="processScheduledJobs" method="init" fixed-delay=5000/>
Once the scheduler triggers the init method. init method is going to use the thread pool executor to execute all the jobs in the queue.
<bean id="processScheduledJobs" class="XXXX.VV.ProcessScheduledJobs">
<property name="pool" ref="jobExecutorService"" />
</bean>
<bean id="scheduler" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="threadFactory">
<bean class="XXX..VVV.NamingThreadFactory">
<constructor-arg value="thread" />
</bean>
</property>
<property name="corePoolSize" value="16" />
<property name="maxPoolSize" value="64" />
<property name="keepAliveSeconds" value="4" />
<property name="queueCapacity" value="512" />
</bean>
Questions:
is the initial thread which executed the init method wait till all the processing (done by executor service by spawning new threads) in the init method is finished?
Is the pool size attribute for scheduler task is only used for triggering the tasks not for executing or processing the logic inside the triggered task.
Thread belonging to scheduler will submit all jobs to jobExecutorService. Up to 64 of them will begin executing immediately, the rest, up to 512 will end up in queue. As soon as all are submitted - not executed - the init method will exit. This shouldn't take longer than mere milliseconds.
If scheduler is not the same as jobExecutorService - I can't tell this since part of your xml is missing - its threads won't be used for execution of jobs logic.
Related
I have JVM1 sending requests to JVM2 synchronously via MQ. This is achieved using JmsInvokerServiceExporter ,SimpleMessageListenerContainer and JmsInvokerProxyFactoryBean. The listener method in JVM2 is written such that should any exception arise during processing, it will fabricate a default response indicating that something went wrong. If everything goes fine, it will send successful response.
Now what I am looking for is if the listener method is taking more time than a predefined period, an exception be thrown so that the listener can stop further procesing and send the default response back to JVM1. Basically JVM2 listener should not process beyond time X sec and should be able to send default response back.
I can make the JVM1 time out after X secs but then I won't get the default response that I am expecting from JVM2. Can someone please let me know if such a thing can be achieved or not? The code snippet for setting up the communication between jvm1 and 2 is given below.
Expose a pojo Calculator.java service for remoting over JMS using JMSInvokerServiceExporter
<bean id="calculatorService" class="com.spring.remoting.service.impl.SalaryCalculator" />
<bean id="remoteCalculatorService" class="org.springframework.jms.remoting.JmsInvokerServiceExporter">
<property name="service" ref="calculatorService">
<property name="serviceInterface" value="com.spring.remoting.service.Calculator" />
</bean>
Listener and Invoker
<bean class="org.springframework.jms.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="myQueue"/>
<property name="concurrentConsumers" value="3"/>
<property name="messageListener" ref="remoteCalculatorService"/>
</bean>
<bean id="remoteCalculatorService" class="org.springframework.jms.remoting.JmsInvokerProxyFactoryBean">
<property name="serviceInterface" value="com.spring.remoting.service.Calculator" />
<property name="connectionFactory" ref="connectionFactory"/>
<property name="queue" ref="myQueue"/>
</bean>
I'm following this tutorial to schedule jobs with Spring.
In the tutorial scheduling is started by the following code:
public static void main(String args[]){
AbstractApplicationContext context = new ClassPathXmlApplicationContext("app-config.xml");
}
Instead of using main, i'd like to start jobs with a method that can be called from anywhere in my application, for example:
public void startJobs() {
// what should this method do to start the jobs?
}
Can the following work?
public void startJobs() {
AbstractApplicationContext context = new ClassPathXmlApplicationContext("app-config.xml");
}
Is this considered good practice?
Basically what i want to achieve is to be able to start the jobs whenever i want (whenever i call the startJobs() method), not on startup in the main method.
How can i do this?
Have you done with scheduling using quartz and spring. If yes, is it running fine? The example link you shared belongs to "The Task Namespace"
Quartz uses Trigger, Job and JobDetail objects to realize scheduling of all kinds of jobs. For the basic concepts behind Quartz, have a look at
http://quartz-scheduler.org/documentation/quartz-2.2.x/quick-start
To integrate it with Spring, Please have a look at this also
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html#scheduling-quartz-jobdetail
XML Configuration of Spring and quartz.
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<bean name="csvLoaderJobDetail"
class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.example.CSVloader.ScheduledJob" />
<property name="jobDataMap">
<map>
<entry key="csvData" value="value1" />
<entry key="date" value="25/09/2015" />
<entry key="csvId" value="1" />
<entry key="jobName" value="csvLoadJob" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
</property>
<property name="durability" value="true" />
</bean>
<bean id="csvLoaderTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="csvLoaderJobDetail" />
<property name="cronExpression" value="0 0 12 * * ?" />
</bean>
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="jobDetails">
<list>
<ref bean="csvLoaderJobDetail" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="csvLoaderTrigger" />
</list>
</property>
<property name="quartzProperties">
<props>
<prop key="org.quartz.scheduler.skipUpdateCheck">true</prop>
</props>
</property>
</bean>
To fire the job manually, you need to inject SchedulerFactoryBean in your spring bean. First you need to get all the jobs created in quartz scheduler, then you can trigger any job manually by using Job Key and Job group of each job.
#Autowired
private SchedulerFactoryBean schedulerFactory;
org.quartz.Scheduler scheduler = schedulerFactory.getScheduler();
// loop jobs by group
for (String groupName : scheduler.getJobGroupNames()) {
// get jobkey
for (JobKey jobKey : scheduler.getJobKeys(GroupMatcher
.jobGroupEquals(groupName))) {
String jobName = jobKey.getName();
String jobGroup = jobKey.getGroup();
scheduler.triggerJob(jobName, jobGroup);
}
}
Now you can create a list of Object, which contain jobName and jobGroup to trigger any job manually.
A better and easy way , use #Scheduled annotation.
Method 1) Task scheduling using fixed delay attribute in #Scheduled annotation
#Scheduled(fixedDelay = 5000)
Method 2) Task scheduling using cron expression in #Scheduled annotation
#Scheduled(cron="*/5 * * * * ?")
Method 3) Task scheduling using cron expression from properties file
#Scheduled(cron = "${cron.expression}")
You can get complete example here
I'm using quartz scheduler for scheduling a spring batch job.
The application starts without any exception but it never fires any job.
Just let me to explain my scenario:
If I run the job(with scheduler) through a main method using MapJobRepositoryFactoryBean it works perfectly, but after integration of the scheduler with spring-mvc web app it shows some version update error, after that I used "JobRepositoryFactoryBean" which uses database for storing job states.
So I added JobRepositoryFactoryBean bean and other DB changes, but it never triggers the job.
bellow is a snippet of log
2015-02-10 19:14:45 INFO context.support.XmlWebApplicationContext - Bean 'jobRegistry' of type [class org.springframework.batch.core.configuration.support.MapJobRegistry] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2015-02-10 19:14:45 INFO jdbc.datasource.DriverManagerDataSource - Loaded JDBC driver: com.mysql.jdbc.Driver
2015-02-10 19:14:45 INFO launch.support.SimpleJobLauncher - No TaskExecutor has been set, defaulting to synchronous executor.
2015-02-10 19:14:46 INFO context.support.DefaultLifecycleProcessor - Starting beans in phase 2147483647
2015-02-10 19:14:46 INFO scheduling.quartz.SchedulerFactoryBean - Starting Quartz Scheduler now
2015-02-10 19:14:46 INFO web.servlet.DispatcherServlet - FrameworkServlet 'mvc-dispatcher': initialization completed in 2155 ms
Here is my job configuration
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"
p:dataSource-ref="dataSource" p:transactionManager-ref="transactionManager">
<property name="databaseType" value="reconConfig!{batch.databaseType}" />
<property name="isolationLevelForCreate" value="ISOLATION_DEFAULT" />
</bean>
<bean id="mapJobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"
lazy-init="true" autowire-candidate="false" />
<bean id="jobOperator"
class="org.springframework.batch.core.launch.support.SimpleJobOperator"
p:jobLauncher-ref="jobLauncher" p:jobExplorer-ref="jobExplorer"
p:jobRepository-ref="jobRepository" p:jobRegistry-ref="jobRegistry" />
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"
p:dataSource-ref="dataSource" />
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="appDataSource" />
</bean>
<bean class="org.springframework.batch.core.scope.StepScope" />
<bean id="reconConfigPlaceholderProperties"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="location" value="classpath:batchDb.properties" />
<property name="placeholderPrefix" value="reconConfig!{" />
<property name="placeholderSuffix" value="}" />
</bean>
</beans>
It was running successfully , but after some more development it stopped working. I'm unable to figure out what exactly I changed in configuration which caused this.
Can any one please suggest the check points in using "JobRepositoryFactoryBean", If I'm missing or the problem is else where.
If this is your entire configuration for job scheduling, I believe you are missing the Cron scheduling part entirely...
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="*/10 * * * * ?" />
</bean>
</property>
</bean>
Please read through the spring doc and the quartz scheduling section here.
We have had the similar or the same problem. Look into DB repository. Repository is not resistant from different instances of application server (e.g. testing and development environment). It means, when two or more applications are connected into the same DB, you can have a problem. Applications begin to pull on the time and jobs. Unregistered jobs in one application are signed as ERROR and blocked and vice versa.
Two tables are important in this case.
Select XXX_SCHEDULER_STATE. Is there more than one row? Than there can be conflict. (Are you not able to distinguish your APP Server? If yes, you are connected into another DB than you suppose. It is very often but trivial problem.)
Select XXX_TRIGGERS.TRIGGER_STATE is there ERROR? If yes, try to change it from any SQL tool:
update TRIGGERS set TRIGGER_STATE = 'WATING' where TRIGGER_STATE = 'ERROR';
Restart application server. If you have a luck, the failed trigger started and work after restart. If not, try to shutdown concurrent App Server or change the repository.
I am trying this and get exception:
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
Is there anyone who have encountered this problem?
#Transactional(propagation = Propagation.REQUIRED)
public void method1() // this method must be Transactional
{
... /*code to call JMS services*/
method2();
}
#Transactional(propagation = Propagation.NOT_SUPPORTED)
public void method2()
{
batchService.runJobWithId(123L);
}
Try removing #Transactional(propagation = Propagation.NOT_SUPPORTED) from method2()
I solved this issue like this:
Create a new bean exclusively to launch the batch job.
Add #Transaction(propagation = Propagation.NOT_SUPPORTED) on that bean.
Configure a task executor for spring batch. something like:
spring-batch.xml:
<beans:bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<beans:property name="jobRepository" ref="jobRepository" />
<beans:property name="taskExecutor" ref="taskExecutor" />
</beans:bean>
Application-context.xml
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="50" />
<property name="queueCapacity" value="10" />
<property name="keepAliveSeconds" value="120" />
</bean>
What does the trick is the task executor. If you check your logs spring batch will complain that, since no task executor is defined, it reverts to synchronous task execution, which is on the same thread. Using the configuration above jobs will be launched on a different thread.
Important note
Altough I managed for a while without configuring a taskExecutor, I ran into the twilight zone when I added a JobListener to update my own tables with start and end times for the job. Any DB Updates the jobListener performed would be reverted when the StartJob() call returned. So, I recommend that you execute your jobs in a separated thread. Simpler. Cleaner.
i have a spring batch job which needs to be scheduled at specific hour of the day. I have setup a Quartz CRON scheduler to accomplish this. However i see that the job is getting triggered only once.
What could be wrong ?
following is the XML file snippet -
<batch:job id="getFleetUpdatesJob" job-repository="jobRepository">
<batch:step id="step0">
<batch:tasklet ref="fleetUpdatesID" transaction-manager="jobRepository-transactionManager" />
</batch:step>
<!-- <batch:step id="step1" next="step2">
<batch:tasklet ref="world" transaction-manager="jobRepository-transactionManager" />
</batch:step> -->
</batch:job>
<!-- Quartz related beans START -->
<bean name="updateDataFeedJobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="<package>.schedule.UpdateDataFeedJob" />
</bean>
<bean id="cronTriggerId" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="updateDataFeedJobDetail" />
<!-- run every morning at 3AM -->
<!-- <property name="cronExpression" value="0 0 3 * * ?" /> -->
<!-- run the job at 8pm everyday -->
<property name="cronExpression" value="0 0 20 * * ?" />
</bean>
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="cronTriggerId" />
</list>
</property>
</bean>
<!-- Quartz related beans END -->
I suspect the cron expression is working correctly, however a Spring Batch job will only execute once unless it's parameters change.
You can verify this if you get a stack trace similar to:
2011-08-18 00:40:26,155 INFO [org.springframework.batch.core.launch.support.SimpleJobLauncher] - <Job: [FlowJob: [name=job1]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED]>
2011-08-18 00:40:30,002 INFO [com.beny23.test.JobLauncherDetails] - <Quartz trigger firing with Spring Batch jobName=job1>
2011-08-18 00:40:30,015 ERROR [com.beny23.test.JobLauncherDetails] - <Could not execute job.>
org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={run.id=1}. If you want to run this job again, change the parameters.
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
In order to ensure that the parameters for each job invokation are different, you could modify the UpdateDataFeedJob class to call your job like this:
JobParametersBuilder builder = new JobParametersBuilder();
builder.addLong("run.ts", System.currentTimeMillis());
JobParameters jobParameters = builder.toJobParameters();
Job job = jobLocator.getJob(jobName);
jobLauncher.run(job, jobParameters);
I dont now spring but i use Quartz .I think problem is ,you are giving two cron expression.You must give a single cron expression like 0 0 3-20 * * ? (some thing like that ) ,crontrigger can have only one cron expression rewrite cron expression that fire at 3 and 20 .