I am new to Spring batch and trying to write multiple XML files for each record that I am going to read from database table. Suppose if I read 10 records, I need to create 10 XML files. One for each record.
The name for each XML file should be unique. For that, I am planning to use "column_name1" value but I am not sure how to achieve that. If anyone can help me in this then that would be a great help.
Updated:
Added the #{formsPMVRowMapper.id} to resource property which can point to DefaultOutboundIFMRowMapper(Custom implementation of RowMapper) where I created a class level variable to set the row id. but still it's not working since it tries to call getter of ID even before getting into mapRow method which I think correct behviour but I am not sure how to get hold of that ID which I can use as resource name for my file in multiXmlFileItemWriter.
Could someone please let me know what could be the possibly correct way to do this?
Below is my Spring batch configuration file.
<util:properties id="batchProperties">
<prop key="batch.output.file">${outbound.pmv.filename}</prop>
</util:properties>
<bean id="itemReader" parent="pagingItemReader">
<property name="queryProvider" ref="outboundQueryProvider" />
<property name="rowMapper" ref="pmvRowMapper" />
</bean>
<bean id="pmvRowMapper"
class="tx.oag.cs.txcses.arch.batch.readers.DefaultOutboundIFMRowMapper">
<property name="idName" value="outbound_locate_record_staging_id" />
</bean>
<bean id="outboundQueryProvider" class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause"
value="select column_name1" />
<property name="fromClause" value="from table_name" />
<property name="whereClause"
value="where column_name1='AAA' and column_name1='bbbb'" />
<property name="sortKey" value="column_name1" />
</bean>
<bean id="batchProcessor" parent="outboundStagingBatchProcessor">
<property name="entityClass"
value="Class_Name" />
</bean>
<bean id="itemWriter" parent="multiXmlFileItemWriter"/>
<bean id="multiXmlFileItemWriter"
class="org.springframework.batch.item.file.MultiResourceItemWriter">
<property name="resource" value="${outbound.ifm.outbound}/#{pmvRowMapper.id}">
</property>
<property name="delegate">
<bean class="org.springframework.batch.item.xml.StaxEventItemWriter">
<property name="marshaller">
<bean class="tx.oag.cs.txcses.arch.batch.utils.XMLStringMarshaller" />
</property>
</bean>
</property>
<property name="itemCountLimitPerResource" value="1" />
</bean>
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location"
value="classpath:config/env/#{#env}/batch-outbound.properties" />
<property name="properties" ref="batchProperties" />
<property name="localOverride" value="true" />
</bean>
I understand that above "resource" property can only write files with same name but I am not sure how to use "resource" property well in co-ordination with "resourceSuffixCreator" property.
Related
Trying hands on with Spring Batch to read data which is created only yesterday. Below is the bean I am trying to use, using JdbcPagingItemReader & SqlPagingQueryProviderFactoryBean. However, the query isn't getting executed.
Appreciate your help!
<bean id="customersPagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="dataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause" value="SELECT CUST_ID, CREATED " />
<property name="fromClause" value=" from CUSTOMERS" />
<property name="whereClause" value=" where CREATED >= trunc(SYSDATE-1) and CREATED < trunc(SYSDATE)" />
</bean>
</property>
<property name="pageSize" value="5" />
<property name="fetchSize" value="5" />
<property name="rowMapper">
<bean class="com.yahoo.affiliationapi.api.CustomerRowMapper" />
</property>
</bean>
I was able to figure this out. When I looked at the Job Step Exit Message says - 'sortKey must be specified'
I just added the below property to the above code & it started working fine.
<property name="sortKey" value="CUST_ID" />
you have to set sort key for queryProvider.
you can refer this example:
#Bean
public SqlPagingQueryProviderFactoryBean queryProvider() {
SqlPagingQueryProviderFactoryBean provider = new SqlPagingQueryProviderFactoryBean();
provider.setSelectClause("select id, name, credit");
provider.setFromClause("from customer");
provider.setWhereClause("where status=:status");
provider.setSortKey("id");
return provider;
}
For some reason the ReloadablePropertiesFactoryBean doens't seem to do what I expect. When I change a property in the test.properties file I get a log message saying:
9979 [timer] INFO com.krest.core.properties.ReloadablePropertiesFactoryBean - Reloading Properties...
But for some reason nothing happens. The old property value is still set on my test bean. Does anybody know why this is? I use Spring 3 and my configuration looks like this:
<bean id="configproperties"
class="com.krest.core.properties.ReloadablePropertiesFactoryBean">
<property name="location"
value="classpath:test.properties" />
</bean>
<bean id="propertyConfigurer"
class="com.krest.core.properties.ReloadingPropertyPlaceholderConfigurer">
<property name="properties" ref="configproperties" />
<property name="reloadingPlaceholderPrefix" value="rel{" />
<property name="reloadingPlaceholderSuffix" value="}" />
</bean>
<bean id="mybean" class="nl.mycompany.TestBean">
<property name="message" value="rel{timer-delay}" />
</bean>
<!-- regularly reload property files. -->
<bean id="timer" class="org.springframework.scheduling.timer.TimerFactoryBean">
<property name="scheduledTimerTasks">
<bean id="reloadProperties"
class="org.springframework.scheduling.timer.ScheduledTimerTask">
<property name="period" value="1000" />
<property name="runnable">
<bean class="com.krest.core.properties.ReloadConfiguration">
<property name="reconfigurableBeans">
<list>
<ref bean="configproperties" />
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
I have the bean definitions below. If I change the placeholderPrefix for the "exposeSystemProperties" bean to "${" and use that in the properties path of the second bean, it works. If I change it to anything but "%{" it doesn't work. I can't use any other string (e.g. "$sys{", "#[", etc.). I'm currently on 3.0.5.RELEASE.
Any thoughts as to why this is? To compound it all, I have a 3rd PropertyPlaceHolderConfigure, so only having two prefixes does not work.
<bean id="exposeSystemProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
<property name="placeholderPrefix"><value>$sys{</value></property>
<property name="order" value="10" />
</bean>
<bean id="localFileProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_NEVER" />
<property name="placeholderPrefix" value="%{" />
<property name="placeholderSuffix" value="}" />
<property name="order" value="20" />
<property name="locations">
<array>
<bean class="java.lang.String">
<constructor-arg><value>classpath:properties/$sys{deploy.env}/client.properties</value></constructor-arg>
</bean>
</array>
</property>
</bean>
Since what you need the prefix for is to control environment specific properties, this can be done by using system variables ( instead of a deploy.env property in your example ):
<value>classpath:properties/${ENV_SYSTEM:dev}/client.properties</value>
In this case it will always look under:
<value>classpath:properties/dev/client.properties</value>
by default, unless a ENV_SYSTEM system variable is set. If it is set to "qa", for example, it will automatically look under:
<value>classpath:properties/qa/client.properties</value>
Another approach, in case you are open to "look into the future" a bit, is to use Spring 3.1's PROFILE feature, where beans can be profile specific. For example:
<beans profile="dev">
<jdbc:embedded-database id="dataSource">
<jdbc:script location="classpath:com/bank/config/sql/schema.sql"/>
<jdbc:script location="classpath:com/bank/config/sql/test-data.sql"/>
</jdbc:embedded-database>
</beans>
This dataSource will only be loaded in case a profile is set to dev:
GenericXmlApplicationContext ctx = new GenericXmlApplicationContext();
ctx.getEnvironment().setActiveProfiles( "dev" );
ctx.load( "classpath:/org/boom/bang/config/xml/*-config.xml" );
ctx.refresh();
I'm using spring 2.0.6 and quartz 1.5.2 in a java based webapp. Want to know how to configure my applicationContext-quartz.xml such that i can read values from a properties file. i.e. I would like to have my file read
<property name="imageFolder" value="${config.imageFolder}" />
<property name="rawImageFolder" value="${config.rawImageFolder}" />
<property name="imageOutputFolder" value="${config.imageOutputFolder}" />
instead of
<property name="imageFolder" value="/path/to/dir1" />
<property name="rawImageFolder" value="/path/to/dir2" />
<property name="imageOutputFolder" value="/path/to/dir3" />
Use the PropertyPlaceholderConfigurer mechanism:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<!-- change this to your property location -->
<value>classpath:quartz.properties</value>
</property>
</bean>
I haven't gotten my head wrapped around Spring yet, so correct me if this question doesn't make sense...
I have a PropertyPlaceholderConfigurer
<bean id="rdbmPropertiesPlacholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" lazy-init="false">
<property name="location" value="classpath:/properties/rdbm.properties" />
</bean>
And I have a bean being injected I guess?
<bean id="PortalDb" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${hibernate.connection.driver_class}" />
<property name="url" value="${hibernate.connection.url}" />
<property name="username" value="${hibernate.connection.username}" />
<property name="password" value="${hibernate.connection.password}" />
...
What I want is a second placeholder pointing to a different properties file with the username/password so that I can split up the properties into two different files. Then the database connection information can be separate from the db username/password, and I can source control one and not the other.
I've tried basically copying the rdbmPropertiesPlaceholder with a different id and file and trying to access the properties, but it doesn't work.
This code is from the uPortal open source web portal project.
Using this notation lets you specify multiple files:
<bean id="rdbmPropertiesPlacholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" lazy-init="false">
<property name="locations">
<list>
<value>classpath:/properties/rdbm.properties</value>
<value>classpath:/properties/passwords.properties</value>
</list>
</property>
</bean>
The propertyplaceholderconfigurerer just merges all of these to look like there's only one, so your bean definitions do not know where the properties come from.
The org.springframework.beans.factory.config.PropertyPlaceholderConfigurer can do this (as already answered. What you may want to do is make use of the name spacing so that you can refer to same-named properties from both files without ambiquity. For your example, you can do this:
<bean id="generalPropertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="classpath:/properties/general.properties"/>
</bean>
<bean id="db.PropertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="classpath:/properties/rdbm.properties" />
<property name="placeholderPrefix" value="$db{" />
<property name="placeholderSuffix" value="}" />
</bean>
In your context files, you can now refer to general properties with ${someproperty}, and refer to rdbm properties with $db{someproperty}.
This will make your context files much cleaner and clearer to the developer.