I use Quartz scheduler with cron expressions for few tasks to be executed on a regular intervals in my Java Spring MVC application. In my root-context.xml file, I have the following:
<!--Quartz Scheduler Beans -->
<bean id="emailNotificationJob"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="eventsService" />
<property name="targetMethod" value="sendEventEmailNotification" />
<property name="concurrent" value="false" /> <!-- this is the property to prevent concurrent execution -->
</bean>
<bean id="deleteWebContentsJob"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="webContentDefinitionService" />
<property name="targetMethod" value="deleteWebContents" />
<property name="concurrent" value="false" />
</bean>
<bean id="saveStaticContentsJob"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="webContentDefinitionService" />
<property name="targetMethod" value="saveStaticContents" />
<property name="concurrent" value="false" />
</bean>
<bean id="emailTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="emailNotificationJob" />
<property name="cronExpression" value="0 0/1 * 1/1 * ? *" />
</bean>
<bean id="deleteWebContentsTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="deleteWebContentsJob"/>
<property name="cronExpression" value="0 1 0 1/1 * ? *" />
</bean>
<bean id="staticContentsUploadTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="saveStaticContentsJob" />
<property name="cronExpression" value="0 0/1 * 1/1 * ? *" />
</bean>
<!-- Scheduler factory bean to glue together jobDetails and triggers to Configure Quartz Scheduler -->
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="jobDetails">
<list>
<ref bean="emailNotificationJob" />
<ref bean="deleteWebContentsJob" />
<ref bean="saveStaticContentsJob" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="emailTrigger" />
<ref bean="deleteWebContentsTrigger" />
<ref bean="staticContentsUploadTrigger" />
</list>
</property>
</bean>
<!-- Quartz Bean End -->
I use Quartz Scheduler 2.2.1 in my application.
I want to set priorities for various triggers. Here I have two triggers that are set to fire at same time intervals. I am trying to set one of them as priority 1 and the other one as priority 2. Is there a way to do that.
CronTriggerFactoryBean has a property called priority (spring-context-support:4.2.4.RELEASE). This is in turn used to set the priority of the CronTriggerImpl, which has a javadoc that states:
The priority of a Trigger acts as a tie breaker such that if two Triggers have the same scheduled fire time, then Quartz will do its best to give the one with the higher priority first access to a worker thread.
If not explicitly set, the default value is 5.
Then again, if there are enough threads in the thread pool for both processes, I'm not sure if the priority will have any effect (depending on whether the priority is also assigned to the actual worker thread).
Related
It is mentioned in the documentation of DefaultMessageListenerContainer class that it is not recommended to use CachingConnectionFactory with dynamic scaling. While searching, I have encountered following link:
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
Here found a comment from Gary Russell that
the problem is with caching consumers when using variable concurrency in the container; we can end up with a live consumer "stuck" in the cache".
We have used DefaultMessageListenerContainer and CachingConnectionFactory together so this is surely a problem from above link.
We are encountering problems with our application having following behaviour:
TCP ZeroWindow network congestion
TCP RESET from application server to MQ
DB connection grows during the issue while different transactions halt
Messages in certain queues gets built up
We have following code configuration :
In ibmmq-context.xml file:
<!-- WebSphere MQ Connection Factory -->
<bean id="appMqConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="hostName">
<value>${ibmmq.ip}</value>
</property>
<property name="port">
<value>${ibmmq.port}</value>
</property>
<property name="queueManager">
<value>${ibmmq.queuemanager}</value>
</property>
<property name="channel">
<value>${ibmmq.channel}</value>
</property>
<property name="clientReconnectOptions">
<util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CLIENT_RECONNECT"/>
</property>
<property name="transportType" ref="appTransport"/>
</bean>
<!-- A cached connection -->
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="appMqConnectionFactory"/>
<property name="sessionCacheSize" value="${jms.session.cachesize}"/>
</bean>
<!-- Use native MQ classes. -->
<bean id="appTransport" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean">
<property name="staticField">
<value>com.ibm.mq.jms.JMSC.MQJMS_TP_CLIENT_MQ_TCPIP</value>
</property>
</bean>
In jms-context file:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
There are 6 such listeners like bankListener and each of the listeners has concurrency value, varies from 10-40
<bean id="listenerExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="maxPoolSize" value="140"/>
<property name="corePoolSize" value="100"/>
<property name="queueCapacity" value="30"/>
<property name="threadNamePrefix" value="jms-listener-task-"/>
<property name="threadGroupName" value="jms-listener-tasks"/>
</bean>
and jms-context.xml file uses ibmmq-context.xml file.
And to note, we have used IBM MQ 7.1, Spring 4.2.8, spring-integration-core as 4.3.1.RELEASE and JBoss EAP 6.4.10
We are planning to fix this by following way:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="appMqConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
My request:
Please review the configuration and let me know is there anything else to be changed.
Could you please also explain our application behaviour(above 4 points - a to d) with our current configuration with CachingConnectionFactory and DefaultMessageListenerContainer
Thanks in advance for your help.
Try setting:
cachingConnectionFactory.setCacheConsumers(false);
Or in Spring it should be:
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
...
<property name="cacheConsumers" value="false"/>
...
</bean>
I am trying to implement Spring batch job for database cleanup.It just delete entry from a table in scheduled way.
First we fetch 10 rows from table.(ItemReader)
Removing these 10 entries from table(ItemWriter)
I have scheduled the batch at 15 minute interval.
When we launch the batch,surprisingly 10 thread tries to read the data from table.
Below is the configuration.
<!-- spring batch context -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="batchTransactionManager" />
</bean>
<bean id="batchTransactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<!--<property name="taskExecutor">
<bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
</property>-->
</bean>
<bean
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<!-- spring batch context -->
<!--<bean id="completionPolicy" class="org.springframework.batch.repeat.policy.DefaultResultCompletionPolicy"/>-->
<batch:job id="csrfTokenCleanUpBatchJob">
<batch:step id="step">
<tasklet>
<chunk reader="csrfTokenReader" writer="csrfTokenWriter" commit-interval="10"></chunk>
</tasklet>
</batch:step>
</batch:job>
<!-- run every 10 seconds -->
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="* 0/15 * * * ?" />
</bean>
</property>
</bean>
<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.test.oauth.batch.job.CSRFTokenJobLauncher" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="csrfTokenCleanUpBatchJob" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
</property>
</bean>
</beans>
It is all by design you want to process each record. The ItemWriter gets as many records as you want but is bound by the commit-interval. Yours is 1 which means each record is individually committed, I suggest you set it to 50. The processor processes each record by it self until the commit interval is reached then the writer is called. As mentioned yours is 1.
Also, make the read method of ItemReader as synchronised.
I am using Activiti BPM to implement a simple workflow. This workflow sends email to users and awaits user feedback. If feedback is not received for 24 hrs, it will again send the email and await for response.
I used boundary event like this:
<boundaryEvent id="boundarytimer1" name="Timer" attachedToRef="userTask" cancelActivity="true">
<timerEventDefinition>
<timeDuration>PT24H</timeDuration>
</timerEventDefinition>
</boundaryEvent>
I can see timer job created in ACT_RU_JOB table. However this job is not triggerred.
Probably you didn't set jobExecutorActivate=true inside your Activiti configuration.
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="true" />
<property name="jobExecutorActivate" value="true" />
<property name="history" value="activity"/>
</bean>
Hope it helps.
From activiti 6.0.0 Version
need to use springConfiguration.setAsyncExecutorActivate(true);
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="true" />
<property name="asyncExecutorActivate" value="true" />
<property name="history" value="activity"/>
</bean>
I have a Quartz trigger created through Spring Framework which I occasionally need to pause (TRIGGER_STATE=PAUSED). However, if I do a deploy of the application while the trigger is paused it gets un-paused due to the trigger being recreated. Is there a Quartz setting to retain the trigger state if the trigger already exists?
<bean id="updateJobQuartzTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="name" value="updateTrigger" />
<property name="group" value="updateDaily" />
<property name="jobDetail" ref="updateJob" />
<property name="cronExpression" value="0 0 4 * * ?" />
</bean>
<bean name="updateJob" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="name" value="updateJob" />
<property name="group" value="updateDaily" />
<property name="jobClass" value="com.project.quartz.SendMessageJob" />
<property name="jobDataMap">
<map>
<entry key="messageContent">
<map>
<entry key="job" value="UpdateLoad"/>
</map>
</entry>
</map>
</property>
<property name="durability" value="true"/>
</bean>
We've recently rolled out a new Java/Spring/Quartz/MyBatis stack to handle our large, data-intensive jobs. Mostly it's been working well, but we've noticed something interesting.
We have a pair of similar jobs that run once a day, a half hour apart. Each performs the same business logic, executing over its own discreet type of users in our database. The jobs have run successfully every day since we released them to production (about a week or so ago). But on two separate occasions, our logs have shown that the trigger had fired right on time, but the actual job didn't execute until 15 minutes later. Once it was the first trigger/job that exhibited this behavior, and once it was the second.
Our logs are written out by the GlobalTriggerListener and GlobalJobListener, as well as periodically by the business logic performed by the job itself, and they all indicate that, on those days, the job ran 15 minutes after the trigger fired. There are no indications whatsoever of any errors occurring around that time.
Below is from our spring/quartz context XML file:
<bean name="AnnualXXXRenewals" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.aaa.cron.base.TaskFactory" />
<property name="jobDataAsMap">
<map>
<entry key="taskClassName" value="com.aaa.cron.tasks.AnnualRenewalTask" />
<entry key="arg.mode" value="xxx" />
</map>
</property>
</bean>
<bean name="AnnualYYYRenewals" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.aaa.cron.base.TaskFactory" />
<property name="jobDataAsMap">
<map>
<entry key="taskClassName" value="com.aaa.cron.tasks.AnnualRenewalTask" />
<entry key="arg.mode" value="yyy" />
</map>
</property>
</bean>
<!-- Daily Trigger 8 PM PST (3AM GMT)-->
<bean id="dailyTrigger0800PM" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="AnnualXXXRenewals"/>
<property name="cronExpression" value="0 0 13 ? * * *"/>
</bean>
<!-- Daily Trigger 8:30 PM PST (3:30AM GMT)-->
<bean id="dailyTrigger0830PM" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="AnnualYYYRenewals"/>
<property name="cronExpression" value="0 30 13 ? * * *"/>
</bean>
<bean id="schedulerFactory"
class="com.aaa.wrapper.spring.UpdatingSchedulerFactoryBean"
lazy-init="false">
<property name="configLocation"
value="classpath:com/aaa/config/quartz.properties" />
<property name="autoStartup" value="true" />
<property name="applicationContextSchedulerContextKey" value="applicationContext" />
<property name="overwriteExistingJobs" value="true" />
<property name="dataSource">
<ref bean="dataSource" />
</property>
<property name="transactionManager">
<ref bean="transactionManager" />
</property>
<property name="jobFactory">
<bean class="com.aaa.wrapper.spring.AutowiringSpringBeanJobFactory" />
</property>
<property name="jobDetails">
<list>
<ref bean="AnnualXXXRenewals" />
<ref bean="AnnualYYYRenewals" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="dailyTrigger0800PM" />
<ref bean="dailyTrigger0830PM" />
</list>
</property>
</bean>
And below is from our quartz.properties:
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.scheduler.skipUpdateCheck = true
#org.quartz.jobStore.useProperties=true
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
# Vendor-specific; MySql uses StdJDBCDelegate
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
# Needed to manage cluster instances
org.quartz.scheduler.instanceId=AUTO
org.quartz.scheduler.instanceName=AAA_SCHEDULER
org.quartz.scheduler.jmx.export=true
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 2
org.quartz.threadPool.threadPriority = 5
org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true
# Job and Trigger Listener
org.quartz.jobListener.GlobalJobListener.class = com.aaa.cron.base.GlobalJobListener
org.quartz.jobListener.GlobalJobListener.name = GlobalJobListener
org.quartz.triggerListener.GlobalTriggerListener.class = com.aaa.cron.base.GlobalTriggerListener
org.quartz.triggerListener.GlobalTriggerListener.name = GlobalTriggerListener
Other info: Our server is set to UTC time. The jobs have been taking a matter of seconds to run. The jobs are clustered, currently across 5 Tomcat instances.
Has anyone run into anything like this before? So far it's not caused any problems, but it would be good to figure out what's going on.