For a Spring batch project, I need to get the date from a file and then I need to pass this date to a procedure and then run the procedure.
Then the result of the procedure must be written to a csv file.
I tried using listeners but couldn't do this.
Can anyone please tell how this can be achieved or if possible can you share any sample code in github.
First of all, you will need to get the date from your file and store it in the JobExecutionContext. One of the most simple solution would be to create a custom Tasklet to read the text file and store the result String in the context via a StepExecutionListener
This tasklet takes a file parameter and stores the result string with the key file.date :
public class CustomTasklet implements Tasklet, StepExecutionListener {
private String date;
private String file;
#Override
public void beforeStep(StepExecution stepExecution) {}
#Override
public ExitStatus afterStep(StepExecution stepExecution) {
stepExecution.getJobExecution().getExecutionContext().put("file.date", date);
}
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
// Read from file using FileUtils (Apache)
date = FileUtils.readFileToString(file);
}
public void setFile(String file) {
this.file = file;
}
}
Use it this way :
<batch:step>
<batch:tasklet>
<bean class="xx.xx.xx.CustomTasklet">
<property name="file" value="${file.path}"></property>
</bean>
</batch:tasklet>
</batch:step>
Then you will use a Chunk with late binding to retrieve the previously stored value (i.e. using #{jobExecutionContext['file.date']}).
The reader will be a StoredProcedureItemReader :
<bean class="org.springframework.batch.item.database.StoredProcedureItemReader">
<property name="dataSource" ref="dataSource" />
<property name="procedureName" value="${procedureName}" />
<property name="fetchSize" value="50" />
<property name="parameters">
<list>
<bean class="org.springframework.jdbc.core.SqlParameter">
<constructor-arg index="0" value="#{jobExecutionContext['file.date']}" />
</bean>
</list>
</property>
<property name="rowMapper" ref="rowMapper" />
<property name="preparedStatementSetter" ref="preparedStatementSetter" />
</bean>
The writer will be a FlatFileItemWriter :
<bean class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
<property name="resource" value="${file.dest.path}" />
<property name="lineAggregator" ref="lineAggregator" />
</bean>
Related
I've a context listener where I'm loading all the properties. These properties I'm trying to set in my spring-web.xml, but it throws an exception
Because its not able to fetch and set the property to the xml
Here is my spring-web.xml
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.ibm.as400.access.AS400JDBCDriver" />
<property name="url" value="jdbc:as400://localhost/BB" />
<property name="username" value="{as400.username}" />
<property name="password" value="{as400.password}" />
</bean>
My class of loading properties
public class LoadProperties implements ServletContextListener {
private static Properties properties = null;
private static Logger logger = Logger.getLogger(LoadProperties .class);
#Override
public void contextDestroyed(ServletContextEvent arg0) { }
#Override
public void contextInitialized(ServletContextEvent arg0) {
properties = BBUtil.getProperties("datasource-cfg.properties");
for (String prop : properties.stringPropertyNames()) {
logger.info("Property Loaded :"+properties.getProperty(prop));
if (System.getProperty(prop) == null) {
System.setProperty(prop, properties.getProperty(prop));
}
}
}
}
This class is getting executed and setting the properties under System.
This is my properties file
as400.username=ROOT
as400.password=ROOT
How can I set the values into my spring-web.xml
Any ideas would be greatly appreciated.
we load it using context:property-placeholder
<context:property-placeholder location="classpath:db.properties" />
<bean id="datasource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="dbHost" value="${db.url}"/>
<property name="dbPort" value="${db.number}"/>
<property name="dbService" value="${db.name}"/>
<property name="dbUrl" value="${db.user}"/>
<property name="dbPassword" value="${db.password}"/>
</bean>
The properties file can also be loaded using prefixes such as http:,file:
The placeholder should be used with a $ sign as follows
<property name="username" value="${as400.username}" />
<property name="password" value="${as400.password}" />
ExtendedIllegalArgumentException essentially indicates that the argument passed is invalid.
I have bean class as below -
public class Account {
private String strAccNumber = "";
private List<Account> accountList = new ArrayList<Account>();
// getter setter
....
...
#Override
public String toString() {
// code for PassThroughLineAggregator
String data="";
for (int j=0; j<accountList.size(); j++) {
Account bean = accountList.get(j);
data = data + bean.getStrAccNumber();
if (j<(accountList.size()-1)) {
data = data + "\n";
}
}
return data;
}
}
to write data I want to set accountList in my config file.
I'm using below code to set bean property.
<bean id="flatFileReader" class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
<property name="resource" value="classpath:springBatch.csv" />
<property name="strict" value="false"></property>
<property name="linesToSkip" value="1" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="ACC#" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="com.abc.reader.AccountDetailsFieldSetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="flatFileProcessor"
class="com.abc.processor.AccountItemProcessor"
scope="step"></bean>
<bean id="flatFileWriter"
class="com.abc.FlatFileItemWriterListener"
scope="step">
<property name="resource" value="classpath:springBatch1.csv" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="strAccNumber" />
</bean>
</property>
</bean>
</property>
</bean>
AccountItemProcessor -
public class AccountItemProcessor implements ItemProcessor<Account, List<Account>>{
#Override
public List<Account> process(Account accObj) throws Exception {
// logic..
}
After processing single record in the process step I want to write multiple items(list of item) , how can I write multiple item at a time using arraylist. In my case I want to write data from accountlist.
You'll want to implement your own LineAggregator. That is what provides the String to be written to the file. In your case, you'll write one that returns a String that is really multiple lines.
You can read more about the LineAggregator interface in the documentation here: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/file/transform/LineAggregator.html
I have a problem the InOnly exchange pattern what I use with aciveMq.
I wrote a module what run in ServiceMix. It works correctly except that it send every message to dead letter queue (ActiveMQ.DLQ). If I check the message then dlqDeliveryFailureCause contains this message: java.lang.Throwable: Message Expired.
I checked the JMSExpiration = 0.
The route:
from("direct:" + reqOutQueue).id("reqInEnritch")
.log("Start dispatch")
.setExchangePattern(ExchangePattern.InOnly)
.recipientList().method(EsbDynamicRouter.class, "systemRoute").parallelProcessing();
The function, what gives back the endpoint list:
#RecipientList
public String[] systemRoute(#Body Object body) throws Exception
{
String[] result = null;
List<DispConfView> targetList;
int cikl = 0;
SystemQueueHelper systemInfo;
MessageHeaderHelper msgHeadHelp = new MessageHeaderHelper(body);
// The object contains the affected elements
targetList = dispHelp.getDispConfByMsgType(msgHeadHelp.getMsgType());
result = new String[targetList.size()];
for (DispConfView element : targetList)
{
// It builds the target andpoints
systemInfo = new SystemQueueHelper(element.getSystemCode(), null, msgHeadHelp.getDirection());
result[cikl] = systemInfo.getQueuName();
cikl++;
}
return result;
}
The list contains these values:
activemq:queue:ERP.req.in?exchangePattern=InOnly
activemq:queue:WF.req.in?exchangePattern=InOnly
As you see, I try to set the correct pattern, but every messages go to dead letter queue.
Please help, what I have to set up!
Thank you!
The solution:
It is settable in context file:
<!-- JMS configuration -->
<bean id="pooledJmsConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="1" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="failover:(tcp://localhost:61616)" />
<property name="redeliveryPolicy">
<bean class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="5"/>
</bean>
</property>
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledJmsConnectionFactory"/>
<property name="cacheLevelName" value="CACHE_CONSUMER" />
<property name="disableReplyTo" value="true" />
</bean>
The "jmsConfig bean ""diasbleReplayTo" property solve the problem.
I am new to Spring LDAP and would like to know the following. Is it possible to "compound" 2 or more searches in a single one, while at the same time using an iterator?
For example, my code right now looks something like this:
List result = ldapTemplate.search(baseDn, encode, searchControls2,
genericContextMapper);
BaseDn points to a certain branch of the LDAP tree, and genericContextMapper has an iterator that I am using throught my code to gradually fetch the results.
But what if I want to pass more than 1 BaseDN, each pointing to completely different LDAP branches, and I expect the iterator to start with the next BaseDN as soon as it is done with the previous one? Can I do this kind of chaining through Spring LDAP?
I know I can achieve this by creating my own custom class that just "sticks together" the results, but I was wondering if there is an easier out of the box solution for this.
Thanks!
You can try this
<bean id="authenticator" class="org.springframework.security.providers.ldap.authenticator.BindAuthenticator">
<constructor-arg ref="contextSource"/>
<property name="userSearch" ref="userSearch" />
</bean>
<bean id="userSearch" class="com.security.OUUserSearch">
<property name="userSearches" ref="userSearches" />
</bean>
<util:list id="userSearches">
<ref bean="userSearch1" />
<ref bean="userSearch2" />
</util:list>
<bean id="userSearch1" class="org.springframework.security.ldap.search.FilterBasedLdapUserSearch">
<constructor-arg value="OU=test1" />
<constructor-arg value="(uid={0})" />
<constructor-arg ref="contextSource"/>
<property name="searchSubtree" value="true" />
</bean>
<bean id="userSearch2" class="org.springframework.security.ldap.search.FilterBasedLdapUserSearch">
<constructor-arg value="OU=test2" />
<constructor-arg value="(uid={0})" />
<constructor-arg ref="contextSource"/>
<property name="searchSubtree" value="true" />
And OUUserSearch is
public class OUUserSearch implements LdapUserSearch
{
List<LdapUserSearch> myUserSearches = null;
public DirContextOperations searchForUser(String theUsername)
{
DirContextOperations theResult = null;
for(LdapUserSearch aSearch : myUserSearches)
{
try
{
theResult = aSearch.searchForUser(theUsername);
if (theResult != null)
{
// we are done
break;
}
}
catch (Exception e)
{
String aMessage = e.getLocalizedMessage();
task.debug(aMessage);
}
}
if (theResult == null)
{
throw new UsernameNotFoundException("Unable to authenticate user (" + theUsername + ")");
}
return theResult;
}
public List<LdapUserSearch> getUserSearches() { return myUserSearches;}
public void setUserSearches(List<LdapUserSearch> theSearches) {myUserSearches = theSearches;}
I am looking for a guidance/solution with Spring batch integration. I have a directory to which external application will send xml files. My application should read the file content and move the file to another directory.
The application should be able to process the files in parallel.
Thanks in advance.
You can use Spring Integration ftp / sftp combined with Spring Batch:
1.Spring Integration Ftp Configuration :
<bean id="ftpClientFactory"
class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="${host.name}" />
<property name="port" value="${host.port}" />
<property name="username" value="${host.username}" />
<property name="password" value="${host.password}" />
<property name="bufferSize" value="100000"/>
</bean>
<int:channel id="ftpChannel" />
<int-ftp:outbound-channel-adapter id="ftpOutbound"
channel="ftpChannel" remote-directory="/yourremotedirectory/" session-factory="ftpClientFactory" use-temporary-file-name="false" />
2.Create your reader and autowire a service to provide your items if needed :
#Scope("step")
public class MajorItemReader implements InitializingBean{
private List<YourItem> yourItems= null;
#Autowired
private MyService provider;
public YourItem read() {
if ((yourItems!= null) && (yourItems.size() != 0)) {
return yourItems.remove(0);
}
return null;
}
//Reading Items from Service
private void reloadItems() {
this.yourItems= new ArrayList<YourItem>();
// use the service to provide your Items
if (yourItems.isEmpty()) {
yourItems= null;
}
}
public MyService getProvider() {
return provider;
}
public void setProvider(MyService provider) {
this.provider = provider;
}
#Override
public void afterPropertiesSet() throws Exception {
reloadItems();
}
}
3. Create Your Own Item Processor
public class MyProcessor implements
ItemProcessor<YourItem, YourItem> {
#Override
public YourItem process(YourItem arg0) throws Exception {
// Apply any logic to your Item before transferring it to the writer
return arg0;
}
}
4. Create Your Own Writer :
public class MyWriter{
#Autowired
#Qualifier("ftpChannel")
private MessageChannel messageChannel;
public void write(YourItem pack) throws IOException {
//create your file and from your Item
File file = new File("the_created_file");
// Sending the file via Spring Integration Ftp Channel
Message<File> message = MessageBuilder.withPayload(file).build();
messageChannel.send(message);
}
5.Batch Configuration :
<bean id="dataSourcee"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="" />
<property name="url" value="" />
<property name="username" value="" />
<property name="password" value="" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSourcee" />
<property name="transactionManager" ref="transactionManagerrr" />
<property name="databaseType" value="" />
</bean>
<bean id="transactionManagerrr"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
6.Another ApplicationContext file to Configure Your Job :
<context:annotation-config />
<bean id="provider" class="mypackage.MyService" />
<context:component-scan base-package="mypackage" />
<bean id="myReader" class="mypackage.MyReader"
<property name="provider" ref="provider" />
</bean>
<bean id="myWriter" class="mypackage.MyWriter" />
<bean id="myProcessor" class="mypackage.MyProcessor" />
<bean id="mReader"
class="org.springframework.batch.item.adapter.ItemReaderAdapter">
<property name="targetObject" ref="myReader" />
<property name="targetMethod" value="read" />
</bean>
<bean id="mProcessor"
class="org.springframework.batch.item.adapter.ItemProcessorAdapter">
<property name="targetObject" ref="myProcessor" />
<property name="targetMethod" value="process" />
</bean>
<bean id="mWriter"
class="org.springframework.batch.item.adapter.ItemWriterAdapter">
<property name="targetObject" ref="myWriter" />
<property name="targetMethod" value="write" />
</bean>
<batch:job id="myJob">
<batch:step id="step01">
<batch:tasklet>
<batch:chunk reader="mReader" writer="mWriter"
processor="mProcessor" commit-interval="1">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="myRunScheduler" class="mypackage.MyJobLauncher" />
<task:scheduled-tasks>
<task:scheduled ref="myJobLauncher" method="run"
cron="0 0/5 * * * ?" />
<!-- this will maker the job runs every 5 minutes -->
</task:scheduled-tasks>
7.Finally Configure A launcher to launch your job :
public class MyJobLauncher {
#Autowired
private JobLauncher jobLauncher;
#Autowired
#Qualifier("myJob")
private Job job;
public void run() {
try {
String dateParam = new Date().toString();
JobParameters param = new JobParametersBuilder().addString("date",
dateParam).toJobParameters();
JobExecution execution = jobLauncher.run(job, param);
execution.stop();
} catch (Exception e) {
e.printStackTrace();
}
}