With the intent to improve performance of quartz, I intend to use batching in quartz as suggested in Performance Tuning on Quartz Scheduler
We create our quartz scheduler in integration with spring as below.
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<!-- Quartz requires a separate 'quartz.properties' file -->
<property name="configLocation" value="classpath:/quartz.properties"/>
<!-- Naturally, Quartz with the DB requires references
to the data source and transaction manager beans -->
<property name="dataSource" ref="quartzDataSource"/>
<!--<property name="transactionManager" ref="transactionManager"/>-->
<!-- reference to our 'autowiring job factory bean', defined above: -->
<property name="jobFactory" ref="quartzJobFactory"/>
<!-- Boolean controlling whether you want to override
the job definitions in the DB on the app start up.
We'll talk about it more in the next section. -->
<property name="overwriteExistingJobs" value="true"/>
<!-- I will not explain the next three properties, just use it as shown: -->
<property name="autoStartup" value="${isQuartzEnabled}" />
<property name="schedulerName" value="quartzScheduler"/>
<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
<!-- Controls whether to wait for jobs completion on app shutdown, we use 'true' -->
<property name="waitForJobsToCompleteOnShutdown"
value="true"/>
<!-- Tell the Quartz scheduler about the triggers.
We have implemented the 'quartzTriggers' bean in the 'Jobs and triggers' section.
No need to pass job definitions, since triggers created via Spring know their jobs.
Later we'll see a case when we'll have to disable this and pass the jobs explicitly.-->
<property name="triggers" ref="quartzTriggers"/>
</bean>
How do I specify maxBatchSize & batchTimeWindow of createScheduler in DirectSchedulerFactory ?
I found that we can achieve by configuring the below property in quartz.properties file
org.quartz.scheduler.batchTriggerAcquisitionMaxCount
Reference : Configure Main Scheduler Settings
org.quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow will set the batchTimeWindow.
org.quartz.scheduler.batchTriggerAcquisitionMaxCount will set the maxBatchSize property.
Source : https://github.com/quartz-scheduler/quartz/blob/master/quartz-core/src/main/java/org/quartz/impl/StdSchedulerFactory.java
Related
I'm using quartz scheduler for scheduling a spring batch job.
The application starts without any exception but it never fires any job.
Just let me to explain my scenario:
If I run the job(with scheduler) through a main method using MapJobRepositoryFactoryBean it works perfectly, but after integration of the scheduler with spring-mvc web app it shows some version update error, after that I used "JobRepositoryFactoryBean" which uses database for storing job states.
So I added JobRepositoryFactoryBean bean and other DB changes, but it never triggers the job.
bellow is a snippet of log
2015-02-10 19:14:45 INFO context.support.XmlWebApplicationContext - Bean 'jobRegistry' of type [class org.springframework.batch.core.configuration.support.MapJobRegistry] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2015-02-10 19:14:45 INFO jdbc.datasource.DriverManagerDataSource - Loaded JDBC driver: com.mysql.jdbc.Driver
2015-02-10 19:14:45 INFO launch.support.SimpleJobLauncher - No TaskExecutor has been set, defaulting to synchronous executor.
2015-02-10 19:14:46 INFO context.support.DefaultLifecycleProcessor - Starting beans in phase 2147483647
2015-02-10 19:14:46 INFO scheduling.quartz.SchedulerFactoryBean - Starting Quartz Scheduler now
2015-02-10 19:14:46 INFO web.servlet.DispatcherServlet - FrameworkServlet 'mvc-dispatcher': initialization completed in 2155 ms
Here is my job configuration
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"
p:dataSource-ref="dataSource" p:transactionManager-ref="transactionManager">
<property name="databaseType" value="reconConfig!{batch.databaseType}" />
<property name="isolationLevelForCreate" value="ISOLATION_DEFAULT" />
</bean>
<bean id="mapJobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"
lazy-init="true" autowire-candidate="false" />
<bean id="jobOperator"
class="org.springframework.batch.core.launch.support.SimpleJobOperator"
p:jobLauncher-ref="jobLauncher" p:jobExplorer-ref="jobExplorer"
p:jobRepository-ref="jobRepository" p:jobRegistry-ref="jobRegistry" />
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"
p:dataSource-ref="dataSource" />
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="appDataSource" />
</bean>
<bean class="org.springframework.batch.core.scope.StepScope" />
<bean id="reconConfigPlaceholderProperties"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="location" value="classpath:batchDb.properties" />
<property name="placeholderPrefix" value="reconConfig!{" />
<property name="placeholderSuffix" value="}" />
</bean>
</beans>
It was running successfully , but after some more development it stopped working. I'm unable to figure out what exactly I changed in configuration which caused this.
Can any one please suggest the check points in using "JobRepositoryFactoryBean", If I'm missing or the problem is else where.
If this is your entire configuration for job scheduling, I believe you are missing the Cron scheduling part entirely...
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="*/10 * * * * ?" />
</bean>
</property>
</bean>
Please read through the spring doc and the quartz scheduling section here.
We have had the similar or the same problem. Look into DB repository. Repository is not resistant from different instances of application server (e.g. testing and development environment). It means, when two or more applications are connected into the same DB, you can have a problem. Applications begin to pull on the time and jobs. Unregistered jobs in one application are signed as ERROR and blocked and vice versa.
Two tables are important in this case.
Select XXX_SCHEDULER_STATE. Is there more than one row? Than there can be conflict. (Are you not able to distinguish your APP Server? If yes, you are connected into another DB than you suppose. It is very often but trivial problem.)
Select XXX_TRIGGERS.TRIGGER_STATE is there ERROR? If yes, try to change it from any SQL tool:
update TRIGGERS set TRIGGER_STATE = 'WATING' where TRIGGER_STATE = 'ERROR';
Restart application server. If you have a luck, the failed trigger started and work after restart. If not, try to shutdown concurrent App Server or change the repository.
I have a Java EE web application (hibernate3, seam) that I'm using in Weblogic container.
I want to introduce Liquibase for schema migrations.
Currently we use
<property name="hibernate.hbm2ddl.auto" value="update"/>
which we want to drop because it can be dangerous.
I want the migration to automatically happen at deployments, so I'm using the servlet listener integration.
In web.xml, the first listener is:
<listener>
<listener-class>liquibase.integration.servlet.LiquibaseServletListener</listener-class>
</listener>
Sadly, this listener comes into play after the Hibernate initialization and it throws missing table errors (because the schema is empty).
I'm google-ing like a boss for hours and I'm a bit confused now.
Thanks in advance
UPDATE
If I set <property name="hibernate.hbm2ddl.auto" value="none" />, liquibase finishes it's job successfully and the app starts up as expected. If I set validate, it seems like hibernate schema validation takes place before liquibase and it cries because of missing tables.
UPDATE
It seems like Seam initializes Hibernate, but Liquibase listener is listed before SeamListener, so I have no clue on how to enable schema validation and liquibase at the same time...
My understanding is that the LiquibaseServletListener requires the path to change log file which is passed using liquibase.changelog context param. So you already have a change log generated or am I missing something here ?
You can take a look at the liquibase hibernate integration library provided by Liquibase.
This library works with both the classic hibernate configuration (via .cfg and .xml files) as well as JPA configuration via persistence.xml.
AFAIK, generating the changelog and running the change log are two seperate process. Liquibase hibernate integration library helps in generating the change log from the diff of current state of entities in persistence unit and the current database state.
How to determine the order of listeners in web.xml
You should place:
<listener>
<listener-class>liquibase.integration.servlet.LiquibaseServletListener</listener-class>
</listener>
before ORM or framework other related listeners.
I use Spring beans LiquiBase activation to reduce DB authentication data duplication by using already provided datasource bean:
<bean id="liquibase" class="liquibase.integration.spring.SpringLiquibase">
<property name="dataSource" ref="dataSource" />
<property name="changeLog" value="classpath:sql/master.sql" />
<property name="defaultSchema" value="PRODUCT" />
</bean>
To restrict order use depends-on attribute:
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"
depends-on="liquibase">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="packagesToScan" value="product.domain" />
<property name="jpaProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.hbm2ddl.auto">validate</prop>
</props>
</property>
</bean>
I implemented a Scheduled Job in Spring, I'm using Spring 3.1.1 with Hibernate and Struts2. The configuration works fine, but I want to change de cron dynamically, so I found several examples but I could not implement them, everywhere are different configurations, I only need to read cron values from the database instead of configuration file. Is this possible somehow?
My configuration now looks like that:
<!-- Scheduler Tasks -->
<bean name="statTask" class="com.bvc.spring.schedulers.MarketStatusJob"></bean>
<!-- Scheduler jobs -->
<bean id="statJobDetail" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="statTask" />
<property name="targetMethod" value="execute" />
</bean>
<!-- Cron Triggers -->
<bean id="statCronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="statJobDetail" />
<property name="cronExpression" value="0 30 12 1/1 * ? *"/>
</bean>
<!-- Triggers -->
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="statCronTrigger"/>
</list>
</property>
</bean>
Thanks in advance for help guys.
The possibly easiest approach would be, to subclass CronTriggerBean and implement database property resolution in an overridden setCronExpression(..) method, where you go to the database, fetch the desired cron, and call super.setCronExpression(dbValue)
An alternative, harder approach is to implement a PropertyPlaceholderConfigurer that reads them from a database rather than a properties file. It may not be trivial, though. There is no support for that, because it is more customary to read the values from a properties file. Also note that you won't be able to change the cron dynamically during execution.
You dont need to have statCronTrigger, need to implement quartz trigger in your main class
Job detail is fine.
CronTrigger trigger = null;
JobDetail jobD;
//Load context
ApplicationContext context = new ClassPathXmlApplicationContext("YOUR_CONTEXT_FILES.xml");
//Setup JobDetail
jobD = (JobDetail) context.getBean("statJobDetail");
//Setup CronTrigger
try {
trigger = new CronTrigger();
trigger.setName("AppTrigger");
trigger.setGroup(jobD.getGroup());
trigger.setJobName(jobD.getName());
trigger.setJobGroup(jobD.getGroup());
trigger.setCronExpression("*/10 * * * * ?");// you can read this from DB as well or any other configured string
} catch (ParseException e1) {
e1.printStackTrace();
}
//Scheduler
try{
Scheduler scheduler = (Scheduler) context.getBean("Scheduler");
scheduler.scheduleJob(jobD, trigger);
You may add quartz scheduler bean in context
<bean id="Scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean"></bean>
I'm using Spring's SchedulerFactoryBean to run some Quartz jobs within a Spring based java application. At present, this is a single instance application in development, but as soon as we start horizontally scaling this we will want to use a jdbc based JobStore for Quartz so no more than one app will run a given job.
Right now, SchedulerFactoryBean is configured as follows:
<bean id="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" >
<property name="taskExecutor" ref="taskExecutor"/>
<property name="triggers">
<list>
<!-- a bunch of triggers here -->
</list>
<property name="applicationContextSchedulerContextKey">
<value>applicationContext</value>
</property>
</bean>
and with using a jdbc based JobStore it will look like this
<bean id="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" >
<property name="dataSource" ref="mysqlJobDataSource"/>
<property name="taskExecutor" ref="taskExecutor"/>
<property name="triggers">
<list>
<!-- a bunch of triggers here -->
</list>
</property>
<property name="applicationContextSchedulerContextKey">
<value>applicationContext</value>
</property>
<property name="quartzProperties">
<props>
<prop key="org.quartz.jobStore.class">org.quartz.impl.jdbcjobstore.JobStoreTX</prop>
<prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.StdJDBCDelegate</prop>
<!-- and a bunch of other quartz props -->
</props>
</property>
</bean>
Ideally, I'd like to continue using the default RAMJobStore version (the first one) for developers, but use the jdbc version for deployed environments. However, there doesn't seem to be a very good way to switch between the two through something like a property, since the jdbc store involves lots more configuration and the mere existence of the dataSource property on SchedulerFactoryBean means it tries to a JDBC based job store.
Also, Since SchedulerFactoryBean is an initializing bean where the initializing basically starts running all of the jobs, so I can't have both of those beans defined in a config file loaded into the spring context either, which means I'll have parallel jobs running.
I've also read through this answer, but this situtation differs in that I'm dealing with two InitializingBeans that should never be in the same context at the same time.
What would be the simplest way to configure switching between these two configurations of SchedulerFactoryBean?
From Spring 3.1 you can use Spring profiles:
<bean name="schedulerFactoryBean" profile="dev" ...
<bean name="schedulerFactoryBean" profile="prd" ...
Then you can instruct Spring container which profile to use, see How to set active spring 3.1 environment profile via a properites file and not via an env variable or system property and Spring autowire a stubbed service - duplicate bean.
If you can't use 3.1 or profiles, the old-school of solving such issues is to have two context files: schedulerContext-dev.xml and schedulerContext-prd.xml`. Then you can import them selectively:
<import resource="schedulerContext-${some.property}"/>
A better option would be using quartz properties file. As part of your release you can have different files per environment. The context that way is the same for all the environments, the only thing that changes is the configuration file. Using maven profiles you can solve it
We're building an app around MongoDB, and have a need to run cron-like jobs periodically. I've used Quartz before for this kind of thing when projects were based around an RDBMS with JDBC.
Since we're already using MongoDB for our main datastore in this project, I'd prefer to not introduce an RDBMS simply to persist Quartz jobs, but there doesn't seem to be any kind of JobStore implementatiom for MongoDB.
Can anyone recommend either a way to back Quartz with MongoDB, or a simple alternative to Quartz? My needs are fairly simple (run various java jobs with some manner of configuration, à la cron).
Edit: Latest implementation https://github.com/michaelklishin/quartz-mongodb forked from below repo
I wrote a MongoDB JobStore for Quartz which is located here: https://github.com/mulesoft/quartz-mongodb It doesn't support everything, but it works for a bunch of use cases.
We run quartz with Spring and it's just an XML file with the jobs defined and cron expressions.
Declare a job in Spring:
<bean name="myJob" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="concurrent" value="false"/>
<property name="targetBeanName" value="myBean"/>
<property name="targetMethod" value="myScheduledMethod"/>
</bean>
<bean id="myJobTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="myJob"/>
<!-- every 30s -->
<property name="cronExpression" value="0/30 * * * * ?"/>
</bean>
Quartz Wiring:
<bean id="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<!-- List of batch jobs to be fed to the scheduler. -->
<list>
<ref bean="myTrigger"/>
</list>
</property>
</bean>
Run it with:
import org.springframework.context.support.ClassPathXmlApplicationContext;
public class App
{
public static void main( String[] args ) throws Exception
{
new ClassPathXmlApplicationContext("jobs-context.xml");
}
}