Spring-Batch Invalid JobExecution, ID not found - java
I am trying to run few spring-batch jobs in a loop. Some of the jobs gets successfully completed. But others do not , and the below exception is thrown in the logs:
<ERROR> [org.springframework.batch.core.job.AbstractJob] :Encountered fatal error executing job
org.springframework.batch.core.repository.dao.NoSuchObjectException: **Invalid JobExecution, ID <theid> not found.**
at org.springframework.batch.core.repository.dao.JdbcJobExecutionDao.updateJobExecution(JdbcJobExecutionDao.java:197)
at
The applicationContext.xml has the below configurations:
<beans:bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<beans:property name="databaseType" value="POSTGRES" />
<beans:property name="dataSource" ref="dataSource" />
<beans:property name="transactionManager" ref="transactionManager" />
<beans:property name="isolationLevelForCreate" value="ISOLATION_DEFAULT" />
</beans:bean>
<beans:bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<beans:property name="jobRepository" ref="jobRepository" />
</beans:bean>
<beans:bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
<beans:property name="dataSource" ref="dataSource" />
</beans:bean>
<beans:bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager" >
<beans:property name="dataSource"
ref="dataSource" />
</beans:bean>
Related
No message found under code 'application.title' for locale 'en_US'
I am having my properties file in the src/main/resource folder. messages.properties messages_en.properties messages_en_US.properties Below is the entry in my spring-servlet.xml file. <resources mapping="/resources/**" location="/resources/" /> <beans:bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <beans:property name="prefix" value="/" /> <beans:property name="suffix" value=".jsp" /> </beans:bean> <!-- Localization Settings --> <beans:bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource"> <beans:property name="basename" value="classpath:messages" /> <beans:property name="defaultEncoding" value="UTF-8" /> </beans:bean> <beans:bean id="localeResolver" class="org.springframework.web.servlet.i18n.CookieLocaleResolver"> <beans:property name="defaultLocale" value="en" /> <beans:property name="cookieName" value="appLocaleCookie"></beans:property> <beans:property name="cookieMaxAge" value="3600"></beans:property> </beans:bean> <interceptors> <beans:bean class="org.springframework.web.servlet.i18n.LocaleChangeInterceptor"> <beans:property name="paramName" value="locale" /> </beans:bean> </interceptors> <!-- Property place holder configuration --> <beans:bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <beans:property name="location" value="classpath:config.properties" /> </beans:bean> But I keep getting the following error javax.servlet.jsp.JspTagException: No message found under code 'application.title' for locale 'en_US'. at org.springframework.web.servlet.tags.MessageTag.doEndTag(MessageTag.java:200) at org.apache.jsp.login_jsp._jspx_meth_spring_005fmessage_005f0(login_jsp.java:224) I had literally tried everything for the basename value with classpath, without classpath but nothing works.
Removing duplicate code from Spring job configuration
Below is my step configuration - <beans:bean id="myInputFileReader" class="com.rbos.fm.risk.batch.spring.reader.InputFileReader" scope="step"> <beans:property name="delegate"> <beans:bean class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <beans:property name="resource" ref="inputFileSystemResource" /> <beans:property name="linesToSkip" value="1" /> <beans:property name="lineMapper"> <beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <beans:property name="lineTokenizer"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <beans:property name="delimiter" value="|"/> </beans:bean> </beans:property> <beans:property name="fieldSetMapper"> <beans:bean class="com.test.MyFieldMapper1" scope="prototype"/> </beans:property> </beans:bean> </beans:property> </beans:bean> </beans:property> <beans:property name="param1" value="#{jobParameters['param1']}"/> <beans:property name="param2" value="#{jobParameters['param2']}"/> <beans:property name="param3" value="#{jobParameters['param3']}"/> </beans:bean> <beans:bean id="fileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> <beans:property name="resource" ref="outputFileSystemResource" /> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value="|"/> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="outcolomn1, outcolomn2, outcolomn3, outcolomn4"/> </beans:bean> </beans:property> </beans:bean> </beans:property> </beans:bean> <step id="myPreProcessing1"> <tasklet> <chunk reader="myInputFileReader" processor="myFileProcessor1" writer="fileItemWriter" commit-interval="10000"/> </tasklet> </step> I have to add other (similar) steps. I would like to use same readers and writers as only change in reader will be a new FieldSetMapper and writer will be a new BeanWrapperFieldExtractor. So ideally, I would like to specify a reader and writer on parent level and would like to refer them in all the steps by just overriding new mapper and extractor.
Use Bean Definition Inheritance. Create an abstract bean definition and have your actual definitions use this as a parent. <beans:bean id="parentInputFileReader" class="com.rbos.fm.risk.batch.spring.reader.InputFileReader" scope="step" abstract="true"> <beans:property name="delegate"> <beans:bean class="org.springframework.batch.item.file.FlatFileItemReader"> <beans:property name="resource" ref="inputFileSystemResource" /> <beans:property name="linesToSkip" value="1" /> </beans:bean> </beans:property> <beans:property name="param1" value="#{jobParameters['param1']}"/> <beans:property name="param2" value="#{jobParameters['param2']}"/> <beans:property name="param3" value="#{jobParameters['param3']}"/> </beans:bean> <beans:bean id="parentLineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper" abstract="true"> <beans:property name="lineTokenizer"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <beans:property name="delimiter" value="|"/> </beans:bean> </beans:property> </beans:bean> Now that you have parent beans you can reference them using the parent attribute and only configure those properties that are needed or need to be modified from the default. <bean id="myInputFileReader" parent="parent"> <beans:property name="lineMapper"> <bean parent="parentLineMapper" > <beans:property name="fieldSetMapper"> <beans:bean class="com.test.MyFieldMapper1" /> </beans:property> </bean> </beans:property /> </bean>
Spring batch is getting stopped while updating more than 1000 records through itemwriter class
I'm using Spring batch to read and update same DB table through JPAPagingItemReader and customized Itemwriter class using Open JPA. But, while trying to update more than 1000 records, some of the records are not updated and batch is getting stopped with either of batch statuses STOPPED, FAILED, UNKNOWN (with no Exceptions during batch process) The below is what I'm doing in my code 1) Reader - <beans:bean id="dataReader" class="org.springframework.batch.item.database.JpaPagingItemReader" scope="step"> <beans:property name="entityManagerFactory" ref="entityManagerFactory" /> <beans:property name="pageSize" value="1"/> <beans:property name="queryString" value="select b from Test b where b.testStatus = #{jobParameters[status]}"/> <!—here status value is ‘No’ --> </beans:bean> 2) Processor – Doing some business logic depends on some condition 3) Writer – <beans:bean id="dataWriter" class="com.example.batch..DataWriter"> <beans:property name="entityManager" ref="sharedEntityManager" /> </beans:bean> public class DataWriter implements ItemWriter<TestData> { private EntityManager entityManager; public EntityManager getEntityManager() { return this.entityManager; } public void setEntityManager(EntityManager entityManager) { this.entityManager = entityManager; } #Transactional public void write (List<? extends TestData> items) { try { for (TestData testData : items) { updateTestDataStatus(testData.getTestDataKey()); } } catch (Exception e) { e.printStackTrace(); } } private void updateTestDataStatus(String testDataKey) { Test test = getEntityManager().find(Test.class, testDataKey); //getEntityManager is sharedEntityManager test.setTestStatus("Yes"); getEntityManager().persist(test); } } 4) DB config <tx:annotation-driven transaction-manager="transactionManager" /> <beans:bean id="sharedEntityManager" class="org.springframework.orm.jpa.support.SharedEntityManagerBean"> <beans:property name="entityManagerFactory" ref="entityManagerFactory" /> </beans:bean> <!-- Configuration for JPATransaction Manager --> <beans:bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"> <beans:property name="entityManagerFactory" ref="entityManagerFactory" /> </beans:bean> <!-- Configuration for EntityManager --> <beans:bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"> <beans:property name="dataSource" ref="dataSource" /> <beans:property name="persistenceUnitName" value="testdata-pu" /> <beans:property name="jpaVendorAdapter"> <beans:bean class="org.springframework.orm.jpa.vendor.OpenJpaVendorAdapter"> <beans:property name="databasePlatform" value="org.apache.openjpa.jdbc.sql.OracleDictionary" /> <beans:property name="database" value="ORACLE" /> <beans:property name="showSql" value="true" /> </beans:bean> </beans:property> </beans:bean> <!-- Configuration for Data Source --> <beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <beans:property name="driverClassName" value="${database.driverClassName}" /> <beans:property name="url" value="${database.url}" /> <beans:property name="username" value="${database.username}" /> <beans:property name="password" value="${database.password}" /> </beans:bean> <beans:bean id="dataSourcePropertiesplaceholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" > <beans:property name="location" value="classpath:datasource.properties"/> </beans:bean> <!-- jobRepository - in-memory mapping configuration --> <beans:bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> <beans:property name="transactionManager" ref="jobRepository-transactionManager" /> </beans:bean> <beans:bean id="jobRepository-transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
Too many Connections Exception in Spring Batch
In my web application I am using Spring Batch Framework and mysql database. In DAO class I am closing all the connection but still I am getting "Too many Connections" Exception.Plz help how to resolve this. This is my Spring Batch configuration files: EramBatchJob.xml: <?xml version="1.0" encoding="UTF-8"?> <beans:beans xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd"> <beans:bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <beans:property name="location"> <beans:value>eramBatchApp.properties</beans:value> </beans:property> </beans:bean> <beans:import resource="EramGBT-JOBREPOSITORY.xml"/> <!-- Headers --> <beans:bean id="accountHeader" class="com.order.batch.AccountHeader" scope="step"> <beans:property name="seperator" value="#{jobParameters['Delimiter']}"> </beans:property> </beans:bean> <beans:bean id="customerHeader" class="com.order.batch.CustomerHeader" scope="step"> <beans:property name="seperator" value="#{jobParameters['Delimiter']}"></beans:property> </beans:bean> <beans:bean id="finstatHeader" class="com.order.batch.FinStatHeader" scope="step"> <beans:property name="seperator" value="#{jobParameters['Delimiter']}"></beans:property> </beans:bean> <!-- Headers --> <!-- Asynchronous Executor --> <beans:bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" scope="step"> <!-- <beans:property name="concurrencyLimit" value="5"/> --> <beans:property name="concurrencyLimit" value="#{jobParameters['concurrencyLimit']}"/> </beans:bean> <!-- Reader --> <beans:bean id="accountInfoFileReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <beans:property name="resource" value="file:#{jobParameters['resource']}" /> <beans:property name="lineMapper"> <beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <beans:property name="lineTokenizer"> <beans:bean class="org.springframework.batch.item.file.transform.FixedLengthTokenizer"> <!-- <beans:property name="delimiter" value=" "/> --> <!-- <beans:property name="columns" value="1-30,31-120,121-190,191-260,261-310,311-360,361-370,371-380,381-382,420-423,400-419,385-393,424-451" /> Correct --> <!-- <beans:property name="columns" value="1-30,31-120,121-190,191-260,261-310,311-360,361-370,371-380,381-382,371-380,400-419,385-393,420-451" /> --> <beans:property name="columns" value="#{jobParameters['InputDeLimiter']}" /> <beans:property name="names" value="accountNumber,businessName,addr1,addr2,city,state,zip,phone,countryCode,phoneCode,bizID,duns,dummy" /> </beans:bean> </beans:property> <beans:property name="fieldSetMapper"> <beans:bean class="com.order.batch.AccountInfoFiledSetMap" /> </beans:property> </beans:bean> </beans:property> </beans:bean> <!-- Processor --> <beans:bean id="productOrderProccesor" class="com.order.batch.ProductOrderProccesor" scope="step"> <beans:property name="toolKitId" value="#{jobParameters['toolKitId']}"/> <beans:property name="toolKitPwd" value="#{jobParameters['toolKitPwd']}"/> </beans:bean> <!-- Wrtiter --> <!-- Account File Writer --> <beans:bean id="accountFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step"> <beans:property name="resource" value="file:#{jobParameters['AccountFilePath']}" /> <!-- <beans:property name="appendAllowed" value="true" />--> <!--<beans:property name="shouldDeleteIfExists" value="true"/> --> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value="#{jobParameters['Delimiter']}"/> <!-- <beans:property name="columns" value="1-30,31-120,121-190,191-260,261-310,311-360,361-370,371-380,381-382,371-380,400-419,385-393,420-452" /> --> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="sacct_Nbr_Cust,scountryCode_Cust,sCrcy_Cd,sDuns_Nbr,sEnq_Duns,sPrim_Name,sAddr_Line,sPost_Town,sPrim_Geo_Area,sPost_Cd,sCntry_Cd,sTlcm_Nbr,sCurr_Cntl_Yr,sTotl_Emp,sPrim_Sic,sLoc_Stat,sLgl_Form,sAddr_Tenr_Type_Cd,sCeo_Nme,sPayd_Scr,sPays_3_Mo_Ago,sPayd_Norm,sAvg_High_Cr,sHigh_Cr,sTot_Pmt,sFinl_Embt_Ind,sOut_Bus_In,sCrim_Ind,sHist_Ind,sImpt_Ind,sExpt_Ind,sDnb_Ratg,sRat_Cd,sAaa_Rat,sBus_Stru,sHq_Duns,sHq_Nme,sHq_Cntry_Cd,sPnt_Nme,sPnt_Duns,sPnt_Ctry_Cd,sDom_Ult_Pnt_Nme,sDom_Ult_Pnt_Duns,sDom_Ult_Ctry_Cd,sGbl_Ult_Pnt_Nme,sGbl_Ult_Pnt_Duns,sGbl_Ult_Ctry_Cd,sDelq_Scr_Entr_Natl_Pctl,sDelq_Scr_Entr_Inds_Pctl,sDelq_Scr_Entr_Inds_Defu_Indn,sFail_Scr_Entr_Natl_Pctl,sFail_Scr_Entr_Defu_Indn,sFail_Scr_Entr_Inds_Pctl,sFail_Scr_Entr_Inds_Defu_Indn,sCurr_Rato,sQk_Rato,sClm_Ind,sSuit_Jdgt_Ind,sTrdg_Styl,sTrdg_Styl1,sTrdg_Styl2,sTrdg_Styl3,sTrdg_Styl4,sPrim_Sic,sPrim_Sic_Type_Cd,sLcl_Atv_Cd,sLcl_Actv_Cd_Type,sStrt_Yr,sBus_Regn_Nbr,sMax_Cr,sIncn_Yr,sEu_Admin_App,sEu_Burgalary_Ind,sEu_Bus_Cease_Ind,sEu_Bus_Windup_Ind,sFire_Ind,sEu_Insol_Stmt,sEu_Liqd_It,sEu_Meet_Crdtr,sEu_Min_Dat_Ind,sEu_Misc_Ovrd,sEu_Neg_Inf_Ind,sFail_Scr_Entr_Scr_Cmty_Cd,sFail_Scr_Entr_Scr_Cmty_Cd1,sFail_Scr_Entr_Scr_Cmty_Cd2,sFail_Scr_Entr_Scr_Cmty_Cd3,sFail_Scr_Entr_Scr_Cmty_Cd4,sFail_Scr_Entr_Scr_Cmty_Cd5,sFail_Scr_Entr_Scr_Cmty_Cd6,sFail_Scr_Entr_Scr_Cmty_Cd7,sFail_Scr_Entr_Scr_Cmty_Cd8,sFail_Scr_Entr_Scr_Cmty_Cd9,sFail_Scr_Entr_Scr_Cmty_Cd10,sFail_Scr_Entr_Scr_Cmty_Cd11,sFail_Scr_Entr_Scr_Cmty_Cd12,sFail_Scr_Entr_Scr_Cmty_Cd13,sFail_Scr_Entr_Scr_Cmty_Cd14,sFail_Scr_Entr_Scr_Cmty_Cd15,sFail_Scr_Entr_Scr_Cmty_Cd16,sFail_Scr_Entr_Scr_Cmty_Cd17,sFail_Scr_Entr_Scr_Cmty_Cd18,sFail_Scr_Entr_Scr_Cmty_Cd19,sFail_Scr_Entr_Scr_Cmty_Cd20,sFail_Scr_Entr_Scr_Cmty_Cd21,sFail_Scr_Entr_Scr_Cmty_Cd22,sFail_Scr_Entr_Scr_Cmty_Cd23,sFail_Scr_Entr_Scr_Cmty_Cd24,sFail_Scr_Entr_Scr_Cmty_Cd25,sFail_Scr_Entr_Scr_Cmty_Cd26,sFail_Scr_Entr_Scr_Cmty_Cd27,sFail_Scr_Entr_Scr_Cmty_Cd28,sFail_Scr_Entr_Scr_Cmty_Cd29,sFail_Scr_Entr_Scr_Ovrd_Cd,sFail_Scr_Entr_Scr_Ovrd_Cd1,sFail_Scr_Entr_Scr_Ovrd_Cd2,sFail_Scr_Entr_Scr_Ovrd_Cd3,sFail_Scr_Entr_Scr_Ovrd_Cd4,sDelq_Scr_Entr_Scr_Cmty_Cd,sDelq_Scr_Entr_Scr_Cmty_Cd1,sDelq_Scr_Entr_Scr_Cmty_Cd2,sDelq_Scr_Entr_Scr_Cmty_Cd3,sDelq_Scr_Entr_Scr_Cmty_Cd4,sDelq_Scr_Entr_Scr_Cmty_Cd5,sDelq_Scr_Entr_Scr_Cmty_Cd6,sDelq_Scr_Entr_Scr_Cmty_Cd7,sDelq_Scr_Entr_Scr_Cmty_Cd8,sDelq_Scr_Entr_Scr_Cmty_Cd9,sDelq_Scr_Entr_Scr_Cmty_Cd10,sDelq_Scr_Entr_Scr_Cmty_Cd11,sDelq_Scr_Entr_Scr_Cmty_Cd12,sDelq_Scr_Entr_Scr_Cmty_Cd13,sDelq_Scr_Entr_Scr_Cmty_Cd14,sDelq_Scr_Scr_Ovrd_Cd,sDelq_Scr_Scr_Ovrd_Cd1,sDelq_Scr_Scr_Ovrd_Cd2,sDelq_Scr_Scr_Ovrd_Cd3,sDelq_Scr_Scr_Ovrd_Cd4,sDast_Ref_Dt,sRate_Id,sRpt_Net_Wrth,sRpt_Net_Sls,sRpt_Net_Income,sCustomer_Bs_Name,dummydelimiter"/> </beans:bean> </beans:property> </beans:bean> </beans:property> <beans:property name="headerCallback" ref="accountHeader"></beans:property> </beans:bean> <!-- Customer File Writer --> <beans:bean id="customerFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step"> <beans:property name="resource" value="file:#{jobParameters['CustomerFilePath']}" /> <!-- <beans:property name="appendAllowed" value="true" /> --> <!--<beans:property name="shouldDeleteIfExists" value="true"/> --> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value="#{jobParameters['Delimiter']}"/> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="sacct_Nbr_Cust,sbusiness_Nme_Cust,saddr1_Cust,saddr2_Cust,scity_Cust,sstate_Cust,szip_Cust,sphoneCode_Cust,sphone_Cust,scountryCode_Cust,sDuns_cust"/> </beans:bean> </beans:property> </beans:bean> </beans:property> <beans:property name="headerCallback" ref="customerHeader"></beans:property> </beans:bean> <!-- Finance Statement Writer --> <beans:bean id="financeStatFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step"> <beans:property name="resource" value="file:#{jobParameters['FinStatFilePath']}" /> <!-- <beans:property name="appendAllowed" value="true" /> --> <!-- <beans:property name="shouldDeleteIfExists" value="true"/> --> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value="#{jobParameters['Delimiter']}"/> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="sacct_Nbr_Cust,sDnb_Ind,sCash_Liq_Aset,sAct_Rec,sAct_Pay,sStk,sTot_Curr_Aset,sTot_Curr_Liab,sTot_Aset,sTot_Liab,sNet_Wrth,sItng_Aset,sSls,sNet_Incm,sStmt_Dt,sStmt_Type,sRate_Id,sStmt_Crcy_Cd,dummydelimiter"/> </beans:bean> </beans:property> </beans:bean> </beans:property> <beans:property name="headerCallback" ref="finstatHeader"></beans:property> </beans:bean> <!-- Composite Writer --> <beans:bean id="compositeWriter" class="org.springframework.batch.item.support.CompositeItemWriter"> <beans:property name="delegates"> <beans:list> <beans:ref bean="accountFileWriter" /> <beans:ref bean="customerFileWriter" /> <beans:ref bean="financeStatFileWriter" /> </beans:list> </beans:property> </beans:bean> <!-- Writers fro Skip Plocies --> <beans:bean id="excludeWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> <beans:property name="shouldDeleteIfExists" value="true"/> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value=","/> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="errormessage,accountNumber,businessName,addr1,addr2,city,state,zip,phone,countryCode,phoneCode,bizID,duns"/> </beans:bean> </beans:property> </beans:bean> </beans:property> </beans:bean> <beans:bean id="writeWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> <beans:property name="shouldDeleteIfExists" value="true"/> <beans:property name="lineAggregator"> <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <beans:property name="delimiter" value=","/> <beans:property name="fieldExtractor"> <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <beans:property name="names" value="sAcct_Nbr,sDuns_Nbr,sCustomer_Bs_Name"/> </beans:bean> </beans:property> </beans:bean> </beans:property> </beans:bean> <!-- Listeners --> <beans:bean id="eramBatchListener" class="com.order.batch.EramJobListener"> <beans:property name="excludeWriter" ref="excludeWriter" ></beans:property> <beans:property name="writeWriter" ref="writeWriter" ></beans:property> </beans:bean> <!-- Job --> <job id="EramBatchJob" job-repository="jobRepository" > <step id="step2"> <tasklet transaction-manager="jobRepository-transactionManager" task-executor="taskExecutor"> <chunk reader="accountInfoFileReader" processor="productOrderProccesor" writer="compositeWriter" commit-interval="#{jobParameters['CommitInterval']}" skip-limit="10000" retry-limit="1"> <streams> <stream ref="compositeWriter"/> <stream ref="accountInfoFileReader"/> <stream ref="excludeWriter"/> <stream ref="writeWriter"/> </streams> <retryable-exception-classes> <include class="java.lang.Exception"/> </retryable-exception-classes> <skippable-exception-classes> <include class="org.springframework.batch.item.file.FlatFileParseException"/> <include class="org.springframework.batch.item.file.transform.IncorrectLineLengthException"/> <include class="org.springframework.batch.core.step.skip.NonSkippableReadException"/> <include class="java.io.IOException"/> <include class="org.springframework.beans.NotReadablePropertyException"/> <include class="org.springframework.batch.item.ItemStreamException"/> <include class="com.order.exception.InvalidDunsOrCountryCodeException"/> </skippable-exception-classes> <listeners> <listener ref="eramBatchListener" /> </listeners> </chunk> </tasklet> <listeners> <listener ref="eramBatchListener"/> </listeners> </step> <listeners> <listener ref="eramBatchListener"/> </listeners> </job> </beans:beans> EramGBT-JOBREPOSITORY.xml: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd"> <bean id="jobRepository-dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="${EramBatch.App.DataBase.DriverClass}" /> <property name="url" value="${EramBatch.App.DataBase.DataBaseURL}" /> <property name="username" value="${EramBatch.App.DataBase.UserName}" /> <!-- your user id. e.g. root--> <property name="password" value="${EramBatch.App.DataBase.Password}" /> <!-- your password--> <property name="maxIdle" value="10"/> <property name="maxActive" value="100"/> <property name="maxWait" value="10000"/> <property name="validationQuery" value="select 1"/> <property name="testOnBorrow" value="false"/> <property name="testWhileIdle" value="true"/> <property name="timeBetweenEvictionRunsMillis" value="1200000"/> <property name="minEvictableIdleTimeMillis" value="1800000"/> <property name="numTestsPerEvictionRun" value="5"/> <property name="defaultAutoCommit" value="false"/> </bean> <bean id="jobRepository-transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager" lazy-init="true"> <property name="dataSource" ref="jobRepository-dataSource" /> </bean> <bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name="dataSource" ref="jobRepository-dataSource" /> <property name="transactionManager" ref="jobRepository-transactionManager"/> <property name="isolationLevelForCreate" value="ISOLATION_READ_UNCOMMITTED" /> <property name="databaseType" value="mysql" /> <property name="tablePrefix" value="batch_"/> </bean> <bean id="asyncTaskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor"/> <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository"/> <property name="taskExecutor"> <bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" /> </property> </bean> <bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean" p:dataSource-ref="jobRepository-dataSource" p:tablePrefix="batch_" /> <bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry"/> <bean id="jobOperator" class="org.springframework.batch.core.launch.support.SimpleJobOperator"> <property name="jobExplorer" ref="jobExplorer"/> <property name="jobRepository" ref="jobRepository"/> <property name="jobRegistry" ref="jobRegistry" /> <property name="jobLauncher" ref="jobLauncher" /> </bean> </beans>
This looks like a MySQL error. Try increasing max_connections in /etc/my.cnf. Also you may look into your app's multiprocessing and threading model. Exhaustion of MySQL connections could indicate a problem where the application is spawning new threads/processes all the time.
Moving Spring Web Project using LDAP Authentication and Authorities to Spring and CAS
I am trying to move a Spring Web Project using LDAP for Authentication and Authorities to Spring and CAS. My project was working great using LDAP but now I have to use CAS.. Once I changed the XML file everything stopped. XML using LDAP: <?xml version="1.0" encoding="UTF-8"?> <beans:beans xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:jee="http://www.springframework.org/schema/jee" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.1.xsd http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.1.xsd " xmlns="http://www.springframework.org/schema/security"> <http auto-config="true" use-expressions="true"> <intercept-url access="hasRole('ROLE_MEMBER_INQUIRY')" pattern="/requests/**" /> <form-login default-target-url="/requests/add.html" /> </http> <authentication-manager> <ldap-authentication-provider user-search-base="ou=webusers" user-search-filter="(uid={0})"> <password-compare> <password-encoder ref="passwordEncoder"> </password-encoder> </password-compare> </ldap-authentication-provider> </authentication-manager> <beans:bean id="passwordEncoder" class="org.springframework.security.authentication.encoding.Md5PasswordEncoder"> </beans:bean> <beans:bean id="contextSource" class="org.springframework.security.ldap.DefaultSpringSecurityContextSource"> <beans:constructor-arg value="ldaps://dvldap01.uftwf.dev:636/dc=uftwf,dc=dev" /> <beans:property name="userDn" value="cn=Manager,dc=uftwf,dc=dev" /> <beans:property name="password" value="uftwf" /> </beans:bean> <beans:bean id="ldapAuthProvider" class="org.springframework.security.ldap.authentication.LdapAuthenticationProvider"> <beans:constructor-arg> <beans:bean class="org.springframework.security.ldap.authentication.BindAuthenticator"> <beans:constructor-arg ref="contextSource" /> <beans:property name="userDnPatterns"> <beans:list> <beans:value> uid={0},ou=webusers </beans:value> </beans:list> </beans:property> </beans:bean> </beans:constructor-arg> <beans:constructor-arg> <beans:bean class="org.springframework.security.ldap.userdetails.DefaultLdapAuthoritiesPopulator"> <beans:constructor-arg ref="contextSource" /> <beans:constructor-arg value="ou=groups" /> <beans:property name="groupRoleAttribute" value="ou" /> </beans:bean> </beans:constructor-arg> </beans:bean> <ldap-server url="ldaps://dvldap01.uftwf.dev:636/dc=uftwf,dc=dev" /> <beans:bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <beans:property name="location" value="classpath:jdbc.properties2" /> </beans:bean> <beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" > <beans:property name="driverClassName" value="${database.driver}" /> <beans:property name="url" value="${database.url}" /> <beans:property name="username" value="${database.user}" /> <beans:property name="password" value="${database.password}" /> <beans:property name="initialSize" value="5" /> <beans:property name="maxActive" value="10" /> </beans:bean> <!-- <beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" > <beans:property name="driverClassName" value="${database.driver}" /> <beans:property name="url" value="${database.url}" /> <beans:property name="username" value="${database.user}" /> <beans:property name="password" value="${database.password}" /> <beans:property name="initialSize" value="5" /> <beans:property name="maxActive" value="10" /> </beans:bean> <jee:jndi-lookup id="dataSourcejndi" jndi-name="dataSourcejndi" lookup-on-startup="false" proxy-interface="javax.sql.DataSource" cache="true" resource-ref="true" /> <beans:bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean" lazy-init="true"> <beans:property name="dataSource" ref="dataSourcejndi" /> </beans:bean> <beans:bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean"> <beans:property name="jndiName" value="java:dataSourcejndi" /> </beans:bean> <beans:bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean"> <beans:property name="jndiName" value="java:comp/env/jdbc/mi"/> </beans:bean> <mvc:annotation-driven /> --> <!-- <beans:bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean"> <beans:property name="jndiName" value="java:dataSourcejndi" /> </beans:bean> --> </beans:beans> XML using CAS: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:sec="http://www.springframework.org/schema/security" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.1.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.1.xsd"> <http auto-config="true" use-expressions="true"> <intercept-url access="hasRole('ROLE_MEMBER_INQUIRY')" pattern="/requests/**" /> <form-login default-target-url="/requests/add.html" /> </http> <bean id="securityFilter" class="org.springframework.security.util.FilterChainProxy"> <sec:filter-chain-map path-type="ant"> <sec:filter-chain pattern="/images/**" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/css/**" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/js/**" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/403.jsp" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/404.jsp" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/error.jsp" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/**/cas/changePassword.htm*" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/**/cas/login.htm*" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/**/cas/passwordExpired.htm*" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/**/*.html*" filters="channelProcessingFilter"/> <sec:filter-chain pattern="/**" filters="channelProcessingFilter,httpSessionContextIntegrationFilter,logoutFilter,casSingleSignOutFilter,casProcessingFilter,securityContextHolderAwareRequestFilter,exceptionTranslationFilter,filterInvocationInterceptor"/> </sec:filter-chain-map> </bean> <!-- this is what hooks up the CAS entry point --> <bean id="exceptionTranslationFilter" class="org.springframework.security.ui.ExceptionTranslationFilter"> <property name="authenticationEntryPoint"> <ref local="casProcessingFilterEntryPoint"/> </property> </bean> <!-- where do I go when I need authentication from CAS--> <bean id="casProcessingFilterEntryPoint" class="org.springframework.security.ui.cas.CasProcessingFilterEntryPoint"> <property name="loginUrl" value="https://dvjvm11.uftwf.dev:8443/cas-server-webapp/login"/> <property name="serviceProperties" ref="serviceProperties"/> </bean> <!-- defines which roles are allowed to access http resources --> <bean id="filterInvocationInterceptor" class="org.springframework.security.intercept.web.FilterSecurityInterceptor"> <property name="authenticationManager" ref="authenticationManager"/> <property name="accessDecisionManager" ref="accessDecisionManager"/> <property name="objectDefinitionSource"> <value> PATTERN_TYPE_APACHE_ANT **=ROLE_ALLOWED_ROLES_HERE </value> </property> </bean> <!-- hooks up CAS ticket validator and user details loader --> <bean id="authenticationManager" class="org.springframework.security.providers.ProviderManager"> <property name="providers"> <list> <ref bean="casAuthenticationProvider"/> </list> </property> </bean> <!-- supporting class for filterInvocationInterceptor --> <bean id="accessDecisionManager" class="org.springframework.security.vote.AffirmativeBased"> <property name="allowIfAllAbstainDecisions" value="false"/> <property name="decisionVoters"> <list> <ref local="roleVoter"/> </list> </property> </bean> <bean id="roleVoter" class="org.springframework.security.vote.RoleVoter"> <property name="rolePrefix" value=""/> </bean> <!-- setup method level security using annotations --> <sec:global-method-security jsr250-annotations="enabled" secured-annotations="enabled"/> <alias name="authenticationManager" alias="_authenticationManager"/> <bean id="passwordEncoder" class="org.springframework.security.providers.encoding.ShaPasswordEncoder"/> <!-- which service (application) am I authenticating --> <bean id="serviceProperties" class="org.springframework.security.ui.cas.ServiceProperties"> <property name="service" value="https://dvjvm11.uftwf.dev:8443/cas-server-webapp/j_spring_cas_security_check"/> <property name="sendRenew" value="false"/> </bean> <!-- handles a logout request from the CAS server --> <bean id="casSingleSignOutFilter" class="org.jasig.cas.client.session.SingleSignOutFilter"/> <!-- performs CAS authentication --> <bean id="casProcessingFilter" class="org.springframework.security.ui.cas.CasProcessingFilter"> <property name="authenticationManager" ref="authenticationManager"/> <property name="authenticationFailureUrl" value="/403.jsp"/> <property name="alwaysUseDefaultTargetUrl" value="false"/> <property name="defaultTargetUrl" value="/"/> </bean> <!-- Does the CAS ticket validation and user details loading --> <bean id="casAuthenticationProvider" class="org.springframework.security.providers.cas.CasAuthenticationProvider"> <property name="userDetailsService" ref="pickYourUserDetailsServiceImplementation"/> <property name="serviceProperties" ref="serviceProperties"/> <property name="ticketValidator"> <bean class="org.jasig.cas.client.validation.Cas20ServiceTicketValidator"> <constructor-arg index="0" value="https://dvjvm11.uftwf.dev:8443/cas-server-webapp/"/> </bean> </property> <property name="key" value="my_password_for_this_auth_provider_only"/> </bean> <!-- Log failed authentication attempts to commons-logging --> <bean id="loggerListener" class="org.springframework.security.event.authentication.LoggerListener"/> <bean id="httpSessionContextIntegrationFilter" class="org.springframework.security.context.HttpSessionContextIntegrationFilter"/> <bean id="securityContextHolderAwareRequestFilter" class="org.springframework.security.wrapper.SecurityContextHolderAwareRequestFilter"/> <!-- ===================== SSL SWITCHING ==================== --> <bean id="channelProcessingFilter" class="org.springframework.security.securechannel.ChannelProcessingFilter"> <property name="channelDecisionManager" ref="channelDecisionManager"/> <property name="filterInvocationDefinitionSource"> <value> PATTERN_TYPE_APACHE_ANT **=REQUIRES_SECURE_CHANNEL </value> </property> </bean> <bean id="channelDecisionManager" class="org.springframework.security.securechannel.ChannelDecisionManagerImpl"> <property name="channelProcessors"> <list> <bean class="org.springframework.security.securechannel.SecureChannelProcessor"> <property name="entryPoint" ref="channelEntryPoint"/> </bean> <bean class="org.springframework.security.securechannel.InsecureChannelProcessor"> <property name="entryPoint" ref="channelEntryPoint"/> </bean> </list> </property> </bean> <bean id="channelEntryPoint" class="org.springframework.security.securechannel.RetryWithHttpsEntryPoint"> <property name="portMapper" ref="portMapper"/> </bean> <bean id="portMapper" class="org.springframework.security.util.PortMapperImpl"> <property name="portMappings"> <map> <entry key="80" value="443"/> <entry key="8080" value="8443"/> <entry key="5580" value="5543"/> </map> </property> </bean> <!-- Invoked when the user clicks logout --> <bean id="logoutFilter" class="org.springframework.security.ui.logout.LogoutFilter"> <!-- URL redirected to after logout success --> <constructor-arg value="https://dvjvm11.uftwf.dev:8443/cas-server-webapp/logout"/> <constructor-arg> <list> <bean class="org.springframework.security.ui.logout.SecurityContextLogoutHandler"> <property name="invalidateHttpSession" value="false"/> </bean> </list> </constructor-arg> </bean> <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"> <property name="driverClassName" value="${database.driver}" /> <property name="url" value="${database.url}" /> <property name="username" value="${database.user}" /> <property name="password" value="${database.password}" /> <property name="initialSize" value="5" /> <property name="maxActive" value="10" /> </bean> </beans> can someone please tell me why everything stopped working
I'm not really sure why you use securityFilter and http tags. We use it like this <http use-expressions="true" request-matcher="ciRegex" auto-config="false" disable-url-rewriting="true" access-denied-page="/denied.page" entry-point-ref="casProcessingFilterEntryPoint"> [..] <intercept-url pattern="/.*" access="isAuthenticated()" /> [..] <custom-filter ref="casSingleSignOutFilter" before="CAS_FILTER" /> <custom-filter ref="casAuthenticationFilter" after="CAS_FILTER" /> <custom-filter ref="sessionRegistryFixFilter" after="SESSION_MANAGEMENT_FILTER" /> <logout invalidate-session="true" logout-success-url="{cas.url}/logout?service=[..]" /> </http> Andd this redirects your calls (if you request somethig that is actually behind an "isAuthenticated()" clause) to the CAS server. And from that point on it works like described here