Trying hands on with Spring Batch to read data which is created only yesterday. Below is the bean I am trying to use, using JdbcPagingItemReader & SqlPagingQueryProviderFactoryBean. However, the query isn't getting executed.
Appreciate your help!
<bean id="customersPagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="dataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause" value="SELECT CUST_ID, CREATED " />
<property name="fromClause" value=" from CUSTOMERS" />
<property name="whereClause" value=" where CREATED >= trunc(SYSDATE-1) and CREATED < trunc(SYSDATE)" />
</bean>
</property>
<property name="pageSize" value="5" />
<property name="fetchSize" value="5" />
<property name="rowMapper">
<bean class="com.yahoo.affiliationapi.api.CustomerRowMapper" />
</property>
</bean>
I was able to figure this out. When I looked at the Job Step Exit Message says - 'sortKey must be specified'
I just added the below property to the above code & it started working fine.
<property name="sortKey" value="CUST_ID" />
you have to set sort key for queryProvider.
you can refer this example:
#Bean
public SqlPagingQueryProviderFactoryBean queryProvider() {
SqlPagingQueryProviderFactoryBean provider = new SqlPagingQueryProviderFactoryBean();
provider.setSelectClause("select id, name, credit");
provider.setFromClause("from customer");
provider.setWhereClause("where status=:status");
provider.setSortKey("id");
return provider;
}
Related
I am new to Spring batch and trying to write multiple XML files for each record that I am going to read from database table. Suppose if I read 10 records, I need to create 10 XML files. One for each record.
The name for each XML file should be unique. For that, I am planning to use "column_name1" value but I am not sure how to achieve that. If anyone can help me in this then that would be a great help.
Updated:
Added the #{formsPMVRowMapper.id} to resource property which can point to DefaultOutboundIFMRowMapper(Custom implementation of RowMapper) where I created a class level variable to set the row id. but still it's not working since it tries to call getter of ID even before getting into mapRow method which I think correct behviour but I am not sure how to get hold of that ID which I can use as resource name for my file in multiXmlFileItemWriter.
Could someone please let me know what could be the possibly correct way to do this?
Below is my Spring batch configuration file.
<util:properties id="batchProperties">
<prop key="batch.output.file">${outbound.pmv.filename}</prop>
</util:properties>
<bean id="itemReader" parent="pagingItemReader">
<property name="queryProvider" ref="outboundQueryProvider" />
<property name="rowMapper" ref="pmvRowMapper" />
</bean>
<bean id="pmvRowMapper"
class="tx.oag.cs.txcses.arch.batch.readers.DefaultOutboundIFMRowMapper">
<property name="idName" value="outbound_locate_record_staging_id" />
</bean>
<bean id="outboundQueryProvider" class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause"
value="select column_name1" />
<property name="fromClause" value="from table_name" />
<property name="whereClause"
value="where column_name1='AAA' and column_name1='bbbb'" />
<property name="sortKey" value="column_name1" />
</bean>
<bean id="batchProcessor" parent="outboundStagingBatchProcessor">
<property name="entityClass"
value="Class_Name" />
</bean>
<bean id="itemWriter" parent="multiXmlFileItemWriter"/>
<bean id="multiXmlFileItemWriter"
class="org.springframework.batch.item.file.MultiResourceItemWriter">
<property name="resource" value="${outbound.ifm.outbound}/#{pmvRowMapper.id}">
</property>
<property name="delegate">
<bean class="org.springframework.batch.item.xml.StaxEventItemWriter">
<property name="marshaller">
<bean class="tx.oag.cs.txcses.arch.batch.utils.XMLStringMarshaller" />
</property>
</bean>
</property>
<property name="itemCountLimitPerResource" value="1" />
</bean>
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location"
value="classpath:config/env/#{#env}/batch-outbound.properties" />
<property name="properties" ref="batchProperties" />
<property name="localOverride" value="true" />
</bean>
I understand that above "resource" property can only write files with same name but I am not sure how to use "resource" property well in co-ordination with "resourceSuffixCreator" property.
I am having some trouble configuring Spring to use BATCH_* tables hosted by MySQL.
I created the tables ok according to docs however it looks like the code is trying to get a sequence number using the Oracle flavour function.
The error I get is:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown table 'BATCH_JOB_SEQ' in field list
But this is hiding the real problem. I debugged it and its trying to run this code:
select " + getIncrementerName() + ".nextval from dual";
Which is obviously Oracle dialect. I notice that there exists the correct incrementer in my environment here:
org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer()
but its calling
org.springframework.jdbc.support.incrementer.OracleMaxValueIncrementer()
I have setup my data source thus:
<bean id="springDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://10.252.205.5:3306/MASKNG" />
<property name="username" value="MASKNG" />
<property name="password" value="maskng" />
</bean>
Anyone have an ideas as this is a show stopper for us atm
Well, well, I really should RTM a little more...you just have to tell the jobRepository bean what type of DB you are using
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="springDataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="validateTransactionState" value="${jobRepository.validationTransactionState:true}" />
<property name="isolationLevelForCreate" value="${jobRepository.isolationLevelForCreate}" />
<!-- <property name="databaseType" value="oracle" /> -->
<property name="databaseType" value="mysql" />
<property name="tablePrefix" value="BATCH_" />
<property name="lobHandler" ref="lobHandler"/>
</bean>
I am using Activiti BPM to implement a simple workflow. This workflow sends email to users and awaits user feedback. If feedback is not received for 24 hrs, it will again send the email and await for response.
I used boundary event like this:
<boundaryEvent id="boundarytimer1" name="Timer" attachedToRef="userTask" cancelActivity="true">
<timerEventDefinition>
<timeDuration>PT24H</timeDuration>
</timerEventDefinition>
</boundaryEvent>
I can see timer job created in ACT_RU_JOB table. However this job is not triggerred.
Probably you didn't set jobExecutorActivate=true inside your Activiti configuration.
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="true" />
<property name="jobExecutorActivate" value="true" />
<property name="history" value="activity"/>
</bean>
Hope it helps.
From activiti 6.0.0 Version
need to use springConfiguration.setAsyncExecutorActivate(true);
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="true" />
<property name="asyncExecutorActivate" value="true" />
<property name="history" value="activity"/>
</bean>
I have a Spring application, i want to change the data source dynamically,ie. when input a DS URL, the Spring beans and all dependency will get updated automatically.I know this is somewhat strange, but anyway i want to achieve that.
My Spring configuration as following:
<bean id="majorDataSource" class="org.postgresql.ds.PGSimpleDataSource">
<property name="serverName" value="${jdbc.serverName}" />
<property name="portNumber" value="${jdbc.portNumber}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="databaseName" value="${jdbc.databaseName}" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="majorDataSource"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager"/>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="majorDataSource"/>
<property name="configLocation" value="classpath:sqlmap-config.xml"/>
</bean>
The questions are:
The JDBC URL is stored in properties, which could be changed runtime.
Once the URL is changed, i need to re-create the data source and maybe the dependent objects. I could not figure out how to do it elegantly in Spring?
I have known that Spring did could dynamically route data source based on one key, but the data source URL is predefined in Spring and will not change runtime. It is not my case.
You can use spring's AbstractRoutingDataSource by extending it and overriding the method determineCurrentLookupKey() that should return the key referencing the datasource's spring bean to be used.
Take a look at this blog article on spring source's blog which will show you an example of how to use that feature.
Basically to answer your questions, what you will need to do is to define the two datasources as different spring bean in your XML config. There is no need to create one dynamically, spring will load both, and use one or the other dynamically depending on your criteria in the determineCurrentLookupKey() method.
This would lead to something like:
XML config
<!-- first data source -->
<bean id="majorDataSource" class="org.postgresql.ds.PGSimpleDataSource">
<property name="serverName" value="${jdbc.major.serverName}" />
<property name="portNumber" value="${jdbc.major.portNumber}" />
<property name="user" value="${jdbc.major.username}" />
<property name="password" value="${jdbc.major.password}" />
<property name="databaseName" value="${jdbc.major.databaseName}" />
</bean>
<!-- second data source -->
<bean id="minorDataSource" class="org.postgresql.ds.PGSimpleDataSource">
<property name="serverName" value="${jdbc.minor.serverName}" />
<property name="portNumber" value="${jdbc.minor.portNumber}" />
<property name="user" value="${jdbc.minor.username}" />
<property name="password" value="${jdbc.minor.password}" />
<property name="databaseName" value="${jdbc.minor.databaseName}" />
</bean>
<!-- facade data source -->
<bean id="dataSource" class="blog.datasource.CustomerRoutingDataSource">
<property name="targetDataSources">
<map>
<entry key="MINOR" value-ref="minorDataSource"/>
<entry key="MAJOR" value-ref="majorDataSource"/>
</map>
</property>
<property name="defaultTargetDataSource" ref="majorDataSource"/>
</bean>
<!-- wiring up -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager"/>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="configLocation" value="classpath:sqlmap-config.xml"/>
</bean>
Java
public class MyRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
// get the current url
HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.currentRequestAttributes()).getRequest();
if (request.getRequestURL().toString().endsWith("/minor"))
return "MINOR";
else
return "MAJOR";
}
}
I am not sure if this is the correct Way of doing this but what you can do is something like this.
<bean id="majorDataSource" class="org.postgresql.ds.PGSimpleDataSource">
<property name="serverName" value="dummydata" />
<property name="portNumber" value="dummydata" />
<property name="user" value="dummydata" />
<property name="password" value="dummydata" />
<property name="databaseName" value="dummydata" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="majorDataSource"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager"/>
and in Java Class
public class TestTransaction {
#Autowired
private DataSourceTransactionManager manager;
private PlatformTransactionManager transactionManager;
public testExecution(DataSource ds) {
manager.setDataSource(ds);
transactionManager = manager;
TransactionDefinition def = new DefaultTransactionDefinition();
TransactionStatus status = transactionManager.getTransaction(def);
try {
jdbcTemplate.update();
transactionManager.commit(status);
} catch (Exception ex) {
transactionManager.rollback(status);
}
}
}
Please suggest if this Approach could Work as i am Still new to Spring
For some reason the ReloadablePropertiesFactoryBean doens't seem to do what I expect. When I change a property in the test.properties file I get a log message saying:
9979 [timer] INFO com.krest.core.properties.ReloadablePropertiesFactoryBean - Reloading Properties...
But for some reason nothing happens. The old property value is still set on my test bean. Does anybody know why this is? I use Spring 3 and my configuration looks like this:
<bean id="configproperties"
class="com.krest.core.properties.ReloadablePropertiesFactoryBean">
<property name="location"
value="classpath:test.properties" />
</bean>
<bean id="propertyConfigurer"
class="com.krest.core.properties.ReloadingPropertyPlaceholderConfigurer">
<property name="properties" ref="configproperties" />
<property name="reloadingPlaceholderPrefix" value="rel{" />
<property name="reloadingPlaceholderSuffix" value="}" />
</bean>
<bean id="mybean" class="nl.mycompany.TestBean">
<property name="message" value="rel{timer-delay}" />
</bean>
<!-- regularly reload property files. -->
<bean id="timer" class="org.springframework.scheduling.timer.TimerFactoryBean">
<property name="scheduledTimerTasks">
<bean id="reloadProperties"
class="org.springframework.scheduling.timer.ScheduledTimerTask">
<property name="period" value="1000" />
<property name="runnable">
<bean class="com.krest.core.properties.ReloadConfiguration">
<property name="reconfigurableBeans">
<list>
<ref bean="configproperties" />
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>