We Keep Coding

Proxy Error - at com.sun.proxy.$Proxy43.run(Unknown Source)

My Spring Batch job failed but I am not sure what is wrong with it. There is not any valid code which I wrote is shown in the Exception. Below is what I got in the logs:
2021-12-28 10:39:35.062 ERROR 30652 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step masterStep in job importUserJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:137)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy43.run(Unknown Source)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:782)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332)
at alfresco.bulkimport.folders.Application.main(Application.java:11)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
My Spring Boot App Class:
#SpringBootApplication(exclude = { DataSourceAutoConfiguration.class })
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
}
My Spring Batch Job Config:
#Slf4j
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Value("file:input/*.txt")
private Resource[] inputResources;
#Autowired
private FlatFileItemReader<FolderNodePojo> nodeItemReader;
#Bean("partitioner")
#StepScope
public Partitioner partitioner() {
log.info("In Partitioner");
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
partitioner.setResources(inputResources);
partitioner.partition(10);
return partitioner;
}
#Bean
public NodeItemProcessor processor() {
return new NodeItemProcessor();
}
#Bean
public NodeItemWriter<FolderNodePojo> writer() {
return new NodeItemWriter<FolderNodePojo>();
}
#Bean
public Job importUserJob(JobFinishListener listener) {
return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener)
.flow(masterStep()).end().build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1").<FolderNodePojo, FolderNodePojo>chunk(5).reader(nodeItemReader)
.processor(processor()).writer(writer()).build();
}
#Bean
public ThreadPoolTaskExecutor taskExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(10);
taskExecutor.setMaxPoolSize(20);
taskExecutor.setQueueCapacity(10);
taskExecutor.afterPropertiesSet();
taskExecutor.setThreadNamePrefix("Thread");
return taskExecutor;
}
#Bean
#Qualifier("masterStep")
public Step masterStep() {
return stepBuilderFactory.get("masterStep").partitioner("step1", partitioner()).step(step1())
.taskExecutor(taskExecutor()).build();
}
#Bean
#StepScope
#Qualifier("nodeItemReader")
#DependsOn("partitioner")
public FlatFileItemReader<FolderNodePojo> NodeItemReader(#Value("#{stepExecutionContext['fileName']}") String filename)
throws MalformedURLException {
return new FlatFileItemReaderBuilder<FolderNodePojo>().name("NodeItemReader").delimited().delimiter("<#|FooBar|#>")
.names(new String[] { "id" }).fieldSetMapper(new BeanWrapperFieldSetMapper<FolderNodePojo>() {
{
setTargetType(FolderNodePojo.class);
}
}).linesToSkip(0).resource(new UrlResource(filename)).build();
}
}
I tried to follow this SO Thread and some similar threads but could not relate them to my situation. My situation is while using Spring Batch/Job/Step. Any idea/clue will be greatly appreciated. I am clueless and not sure where to start debugging. Kindly let me know if any other details/aspects are required to look into. I can provide.

Commit failed while step execution data was already updated. Reverting to old version. Spring batch

I was having problem starting up spring batch application, which I mentioned here. Now I have update the code to use HibernateTransactionManager and my job has 6 steps, and first step works fine now, but second steps fails. Following is update code in which I have added HibernateTransactionManager.
BatchScheduler.Class
import org.springframework.batch.core.launch.support.SimpleJobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean;
import org.springframework.batch.support.transaction.ResourcelessTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.annotation.EnableScheduling;
//#Configuration
#EnableScheduling
public class BatchScheduler {
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(
ResourcelessTransactionManager txManager) throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(txManager);
factory.afterPropertiesSet();
return factory;
}
#Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean factory) throws Exception {
return factory.getObject();
}
#Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
return launcher;
}
}
#Configuration
#EnableBatchProcessing
#Import({BatchScheduler.class})
public class LogBatchConfiguration {
private static final Logger log = LoggerFactory.getLogger(LogBatchConfiguration.class);
#Autowired private SimpleJobLauncher jobLauncher;
//**********New added
#Bean
public PlatformTransactionManager mytransactionManager() {
HibernateTransactionManager transactionManager = new HibernateTransactionManager();
transactionManager.setSessionFactory(createLocalSessionFactory(mainDataSource).getObject());
return transactionManager;
}
#Autowired
#Qualifier(value = "mainDataSource")
public DataSource mainDataSource;
#Autowired
#Qualifier(value = "toolsDataSource")
public DataSource toolsDataSource;
#Autowired public StepBuilderFactory stepBuilderFactory;
#Autowired public JobBuilderFactory jobBuilderFactory;
#Autowired private RestTemplate restTemplate;
#Qualifier(value = "createLocalSessionFactory")
#Autowired
private SessionFactory createLocalSessionFactory;
#Qualifier(value = "createToolSessionFactory")
#Autowired
private SessionFactory createToolSessionFactory;
#Bean
RestTemplate restTemplate(RestTemplateBuilder builder) {
RestTemplate restTemplate = new RestTemplate();
List<HttpMessageConverter<?>> converters = restTemplate.getMessageConverters();
for (HttpMessageConverter<?> converter : converters) {
if (converter instanceof MappingJackson2HttpMessageConverter) {
MappingJackson2HttpMessageConverter jsonConverter =
(MappingJackson2HttpMessageConverter) converter;
jsonConverter.setObjectMapper(new ObjectMapper());
jsonConverter.setSupportedMediaTypes(
ImmutableList.of(
new MediaType(
"application", "json", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET),
new MediaType(
"text", "javascript", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET)));
}
}
return restTemplate;
}
#Bean
public LocalSessionFactoryBean createLocalSessionFactory(
#Qualifier("mainDataSource") DataSource dataSource) {
LocalSessionFactoryBean factory = new LocalSessionFactoryBean();
factory.setDataSource(dataSource);
factory.setPackagesToScan(
new String[] {
"com.abc.def..entities",
"com.abc.entities"
});
factory.setHibernateProperties(hibernateVCommitProperties());
return factory;
}
#Bean
public LocalSessionFactoryBean createToolSessionFactory(
#Qualifier("toolsDataSource") DataSource dataSource) {
LocalSessionFactoryBean factory = new LocalSessionFactoryBean();
factory.setDataSource(dataSource);
factory.setPackagesToScan(
new String[] {
"com.abc.def..entities",
"com.abc.entities"
});
factory.setHibernateProperties(hibernateCommitProperties());
return factory;
}
#Scheduled(cron = "${server.cron}")
public void SyncJobTrigger() throws Exception {
if (log.isInfoEnabled()) {
}
JobParameters param =
new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
JobExecution execution = jobLauncher.run(populateServerDataJob(), param);
if (log.isInfoEnabled()) {
log.info("BATCH tools Server job finished with status: " + execution.getStatus());
}
}
//--------------------------------- JOBS ----------------------------------------------
#Bean
public Job populateServerDataJob() throws Exception {
return jobBuilderFactory
.get("populateData")
.start(serverRequestData()) // this step's writer is failing
.next(a)
.build();
}
#Bean
#StepScope
public HibernateCursorItemReader<Request> myRequestReader() throws Exception {
// returns reader and works fine as seen in logs, data is fetched
}
#Bean
#StepScope
public requestToServerProcessor requestToServerRequestProcessor() {
return new requestToServerProcessor(mainDataSource);
}
// WRITER
#Bean
#StepScope
public HibernateItemWriter<ServerRequestDetails> serverRequestWriter() {
HibernateItemWriter<ServerRequestDetails> writer = new HibernateItemWriter();
writer.setSessionFactory(createLocalSessionFactory(mainDataSource).getObject());
return writer;
}
#Bean
#JobScope
public Step serverRequestData() throws Exception {
return stepBuilderFactory
.get("getServerRequestData")
.transactionManager(mytransactionManager())/***** added manager here.
.<Request, ServerRequestDetails>chunk(100)
.reader(mylRequestReader())
.processor(requestToServerRequestProcessor())
.writer(serverRequestWriter())
.build();
}
}
DataSourceConfiguration.class
#Configuration
public class DataSourceConfiguration {
private static final Logger log = LoggerFactory.getLogger(DataSourceConfiguration.class);
#Value("${spring.datasource.usern}")
private String insightsDbUser;
#Value("${spring.datasource.pass}")
private String insightsDbPass;
#Value("${spring.datasource.url}")
private String insightsDbConnection;
#Value("${tools.datasource.user}")
private String toolsDbUser
#Value("${tools.datasource.pass}")
private String toolsDbPass;
#Value("${tools.datasource.url}")
private String toolsDbConnection;
#Bean(name = "mainDataSource")
#Primary
public DataSource mainDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl(insightsDbConnection);
dataSource.setUsername(insightsDbUser);
dataSource.setPassword(insightsDbPass);
if (log.isInfoEnabled()) {
//
}
return dataSource;
}
#Bean(name = "toolsDataSource")
public DataSource toolsDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl(toolsDbConnection);
dataSource.setUsername(toolsDbUser);
dataSource.setPassword(toolsDbPass);
if (log.isInfoEnabled()) {
//
}
return dataSource;
}
}
Now in this serverRequestData, is the first step of job, and step "a" is second step, step "a" code I have not put because it is exactly similar to step "serverRequestData". I have created a HibernateTransactionManager and specified that while running step, but now first step works fine.Earlier problem with first step was here. Now step "a" fails because of following error which seems similar to earlier failure but I do see extra stackTrace :
2021-04-16 09:56:52.986 INFO 47235 --- [main] o.s.batch.core.job.SimpleStepHandler : Executing step: [a]
2021-04-16 09:56:59.305 INFO 47235 --- [main] o.s.batch.core.step.tasklet.TaskletStep : Commit failed while step execution data was already updated. Reverting to old version.
2021-04-16 09:56:59.308 ERROR 47235 --- [main] o.s.batch.core.step.AbstractStep : Encountered an error executing step getVsyncDetails in job populateVtoolsServerDataJob
javax.persistence.TransactionRequiredException: no transaction is in progress
at org.hibernate.internal.AbstractSharedSessionContract.checkTransactionNeededForUpdateOperation(AbstractSharedSessionContract.java:413)
at org.hibernate.internal.SessionImpl.checkTransactionNeededForUpdateOperation(SessionImpl.java:3397)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1354)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1349)
at org.springframework.orm.hibernate5.SessionFactoryUtils.flush(SessionFactoryUtils.java:148)
at org.springframework.orm.hibernate5.SpringSessionSynchronization.beforeCommit(SpringSessionSynchronization.java:95)
at org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:96)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:919)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:727)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy118.execute(Unknown Source)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:410)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:319)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at com.CoreApplication.main(CoreApplication.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
2021-04-16 09:56:59.309 INFO 47235 --- [main] o.s.batch.core.step.AbstractStep : Step: [a] executed in 6s323ms
2021-04-16 09:56:59.309 ERROR 47235 --- [main] o.s.batch.core.step.AbstractStep : Encountered an error saving batch meta data for step a in job populateServerDataJob. This job is now in an unknown state and should not be restarted.
org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=2 with wrong version (1), where current version is 2
at org.springframework.batch.core.repository.dao.MapStepExecutionDao.updateStepExecution(MapStepExecutionDao.java:106)
at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy93.update(Unknown Source)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:275)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy118.execute(Unknown Source)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:410)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:319)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at com.CoreApplication.main(CoreApplication.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
2021-04-16 09:56:59.313 INFO 47235 --- [main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=populateServerDataJob]] completed with the following parameters: [{}] and the following status: [UNKNOWN] in 9s178ms
I see one extra log
2021-04-16 09:56:59.305 INFO 47235 --- [main] o.s.batch.core.step.tasklet.TaskletStep : Commit failed while step execution data was already updated. Reverting to old version.
Which I did not see earlier.
Now since after adding HibernateTransactionManager, the first step writer is working fine, so I added same Transaction Manage while configuring step a. But now I am getting same error but with some extra log, so I think there can be some another reason behind this?

You are declaring a ResourcelessTransactionManager bean that will be used by default in Spring Batch. Since this transaction manager does not create any transaction, your hibernate item writer fails with this error.
You need to configure Spring Batch to use the HibernateTransactionManager (and remove the ResourcelessTransactionManager). This can be done with a custom Batch configurer as mentioned in the documentation here. In your case, it could be something like:
#Bean
public BatchConfigurer batchConfigurer(DataSource ds, SessionFactory sf) {
return new DefaultBatchConfigurer(ds) {
#Override
public PlatformTransactionManager getTransactionManager() {
return new HibernateTransactionManager(sf);
}
};
}

Spring Batch - Unable to create metadata tables on Postgres and load actual data to mysql

I am developing Spring Boot v2.2.6.RELEASE + Spring Batch. In this example, I am looking to read csv file, load the data into mysql and spring batch metadata tables into the Postgres database.
But This gives me error:
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:787) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:768) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at com.example.SpringBatchMetadataInDiffSchemaApplication.main(SpringBatchMetadataInDiffSchemaApplication.java:27) [classes/:na]
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?]; nested exception is org.postgresql.util.PSQLException: ERROR: relation "batch_job_instance" does not exist
Position: 39
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:235) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1443) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:669) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:712) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:768) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.getJobInstance(JdbcJobInstanceDao.java:151) ~[spring-batch-core-4.2.1.RELEASE.jar:4.2.1.RELEASE]
at org.springframework.batch.core.repository.support.SimpleJobRepository.getLastJobExecution(SimpleJobRepository.java:271) ~[spring-batch-core-4.2.1.RELEASE.jar:4.2.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ~[spring-tx-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99) ~[spring-tx-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at com.sun.proxy.$Proxy61.getLastJobExecution(Unknown Source) ~[na:na]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:104) ~[spring-batch-core-4.2.1.RELEASE.jar:4.2.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.2.1.RELEASE.jar:4.2.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at com.sun.proxy.$Proxy58.run(Unknown Source) ~[na:na]
at com.example.SpringBatchMetadataInDiffSchemaApplication.run(SpringBatchMetadataInDiffSchemaApplication.java:36) [classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:784) [spring-boot-2.2.2.RELEASE.jar:2.2.2.RELEASE]
... 5 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: relation "batch_job_instance" does not exist
Position: 39
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2497) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2233) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:310) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149) ~[postgresql-42.2.8.jar:42.2.8]
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:108) ~[postgresql-42.2.8.jar:42.2.8]
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-3.4.1.jar:na]
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) ~[HikariCP-3.4.1.jar:na]
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:678) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617) ~[spring-jdbc-5.2.2.RELEASE.jar:5.2.2.RELEASE]
... 37 common frames omitted
JobConfig.java
#Configuration
public class JobConfig {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Qualifier("secondaryDS")
#Autowired
private DataSource dataSource;
#Bean
public FlatFileItemReader<Customer> customerItemReader(){
FlatFileItemReader<Customer> reader = new FlatFileItemReader<>();
reader.setLinesToSkip(1);
reader.setResource(new ClassPathResource("/data/customer.csv"));
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
tokenizer.setNames(new String[] {"id", "firstName", "lastName", "birthdate"});
DefaultLineMapper<Customer> customerLineMapper = new DefaultLineMapper<>();
customerLineMapper.setLineTokenizer(tokenizer);
customerLineMapper.setFieldSetMapper(new CustomerFieldSetMapper());
customerLineMapper.afterPropertiesSet();
reader.setLineMapper(customerLineMapper);
return reader;
}
#Bean
public JdbcBatchItemWriter<Customer> customerItemWriter(){
JdbcBatchItemWriter<Customer> writer = new JdbcBatchItemWriter<>();
writer.setDataSource(this.dataSource);
writer.setSql("INSERT INTO CUSTOMER VALUES (:id, :firstName, :lastName, :birthdate)");
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>());
writer.afterPropertiesSet();
return writer;
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Customer, Customer> chunk(1000)
.reader(customerItemReader())
.writer(customerItemWriter())
.build();
}
#Bean
public Job job() {
return jobBuilderFactory.get("job")
.start(step1())
.build();
}
}
DatabaseConfig.java
#Configuration
public class DatabaseConfig {
#Autowired
private Environment env;
// For Test schema
#Bean(name="secondaryDS")
#ConfigurationProperties(prefix="spring.datasource")
public DataSource getSeconadaryBatchDataSource(){
return DataSourceBuilder.create()
.url(env.getProperty("spring.datasource.url"))
.driverClassName(env.getProperty("spring.datasource.driver-class-name"))
.username(env.getProperty("spring.datasource.username"))
.password(env.getProperty("spring.datasource.password"))
.build();
}
// For "batchmetadata" tables
#Bean(name="primaryDS")
#Primary
#ConfigurationProperties(prefix="spring.hello.datasource")
public DataSource getPrimaryBatchDataSource(){
return DataSourceBuilder.create()
.url(env.getProperty("spring.hello.datasource.url"))
.driverClassName(env.getProperty("spring.hello.datasource.driver-class-name"))
.username(env.getProperty("spring.hello.datasource.username"))
.password(env.getProperty("spring.hello.datasource.password"))
.build();
}
//Ref: https://stackoverflow.com/questions/28275448/multiple-data-source-and-schema-creation-in-spring-boot
#Bean(name = "primaryEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean primaryEntityManagerFactory( EntityManagerFactoryBuilder builder) {
Map<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto", "update");
return builder
.dataSource(getSeconadaryBatchDataSource())
.packages("com.example.model")
.persistenceUnit("default")
.properties(properties)
.build();
}
/*#Bean(name = "secondaryEntityManagerFactory")
#Primary
public LocalContainerEntityManagerFactoryBean secondaryEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("hibernate.hbm2ddl.auto", "create");
return builder
.dataSource(getPrimaryBatchDataSource())
.packages("com.example.model")
.persistenceUnit("default")
.properties(properties)
.build();
}*/
}
application.properties
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=root
spring.datasource.password=root
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.jpa.hibernate.ddl-auto=create
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
# Postgres
spring.hello.datasource.driver-class-name=org.postgresql.Driver
spring.hello.datasource.url=jdbc:postgresql://localhost:5432/program?currentSchema=BATCH
spring.hello.datasource.username=postgres
spring.hello.datasource.password=admin
#spring.batch.table-prefix=batchmetadata.BATCH_
spring.batch.initialize-schema=always
spring.batch.job.enabled=false
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.jpa.hibernate.naming.implicit-strategy=org.hibernate.boot.model.naming.ImplicitNamingStrategyLegacyJpaImpl

Maybe you are creating table using quote like "BATCH_JOB_INSTANCE".
Also check is schema is specified in datasource.url since you don't use schema name in query.

Thanks Abinash for great help. I simply reverse Primary and Secondary. It worked.
# Datasource - Postgres - Primary
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/program?currentSchema=BATCH
spring.datasource.username=postgres
spring.datasource.password=admin
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
spring.jpa.hibernate.ddl-auto=create
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
# MySQL- Secondary
spring.hello.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.hello.datasource.url=jdbc:mysql://localhost:3306/test
spring.hello.datasource.username=root
spring.hello.datasource.password=root
#spring.batch.table-prefix=batchmetadata.BATCH_
spring.batch.initialize-schema=always
spring.batch.job.enabled=false
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.jpa.hibernate.naming.implicit-strategy=org.hibernate.boot.model.naming.ImplicitNamingStrategyLegacyJpaImpl
Also, using below worked.
writer.setSql("INSERT INTO \"BATCH\".customer VALUES (:id, :firstName, :lastName, :birthdate)");

ItemStreamException: Failed to initialize the reader Caused by:IllegalStateException: Stream is already initialized. Close before re-opening

I've repeated the example provided in the Creating a Batch Service tutorial
But I want to add second step to the job so it should do following:
1 step(it is already provided in tutorial): read people names from csv and save it in databse in upperCase
2 step: read from database names convert it to lowercase and insert into another database table.
Currently my configuration looks like this:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
// tag::readerwriterprocessor[]
#Bean
public FlatFileItemReader<Person> reader() {
return new FlatFileItemReaderBuilder<Person>()
.name("personItemReader")
.resource(new ClassPathResource("users.csv"))
.delimited()
.names(new String[]{"firstName", "lastName"})
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}})
.build();
}
#Bean
public JdbcCursorItemReader<Person> dbReader(DataSource dataSource) {
JdbcCursorItemReader<Person> reader = new JdbcCursorItemReader<>();
reader.setDataSource(dataSource);
BeanWrapperFieldSetMapper<Person> personBeanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}};
reader.setRowMapper(new RowMapper<Person>() {
#Override
public Person mapRow(ResultSet resultSet, int i) throws SQLException {
Person person = new Person();
person.setFirstName(resultSet.getString(0));
person.setLastName(resultSet.getString(1));
return person;
}
});
reader.setSql("SELECT first_name, last_name FROM people");
reader.open(new ExecutionContext());
return reader;
}
#Bean
public PersonItemProcessor processor() {
return new PersonItemProcessor();
}
#Bean
public ToLowerCaseProcessor processor2() {
return new ToLowerCaseProcessor();
}
#Bean
public JdbcBatchItemWriter<Person> writer(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Person>()
.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
.sql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)")
.dataSource(dataSource)
.build();
}
// end::readerwriterprocessor[]
// tag::jobstep[]
#Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1, Step step2) {
return jobBuilderFactory.get("importUserJob")
//.incrementer(new RunIdIncrementer())
.listener(listener)
.listener(new JobExecutionListener() {
#Override
public void beforeJob(JobExecution jobExecution) {
System.out.println("!!!!!!!!!!!!!SECOND_LISTENER_BEFORE!!!!!!!!!!!!!!!!");
}
#Override
public void afterJob(JobExecution jobExecution) {
System.out.println("!!!!!!!!!!!!!SECOND_LISTENER_AFTER!!!!!!!!!!!!!!!!");
}
})
.flow(step1)
.next(step2)
.end()
.build();
}
#Bean
public Step step1(JdbcBatchItemWriter<Person> writer) {
return stepBuilderFactory.get("step1")
.<Person, Person>chunk(10)
.reader(reader())
.processor(processor())
.writer(writer)
.build();
}
#Bean
public Step step2(JdbcBatchItemWriter<Person> writer, DataSource dataSource) {
return stepBuilderFactory.get("step2")
.<Person, Person>chunk(10)
.reader(dbReader(dataSource))
.processor(processor2())
.writer(writer)
.build();
}
// end::jobstep[]
}
But when I start my spring boot application on the second step I experince the following error:
2019-08-02 14:33:54.164 ERROR 26972 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step step2 in job importUserJob
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:152) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at com.sun.proxy.$Proxy44.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:206) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:180) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:167) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:162) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:779) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:763) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:318) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at hello.Application.main(Application.java:10) ~[classes/:na]
Caused by: java.lang.IllegalStateException: Stream is already initialized. Close before re-opening.
at org.springframework.util.Assert.state(Assert.java:73) ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.item.database.AbstractCursorItemReader.doOpen(AbstractCursorItemReader.java:424) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:149) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
... 34 common frames omitted
I have no idea what do I wrong. What the stream has mentioned ?
Could please explain how to correct it ?

in your dbReader method you should'nt call :
reader.open(new ExecutionContext());
It is called by springbatch.

Spring Data Jpa repository in spring batch tasklet throws TransactionRequiredException

I'm having problems using a Spring Data Jpa repository inside a Spring Batch tasklet.
I'd expect to have a valid hibernate transaction in the MyOrderTasklet's execute method using the myTransactionManager that is configured for this step. But as soon as flush() is called or if the execute method is left (without explicitly calling flush()). I get a
TransactionRequiredException"no transaction is in progress".
When debugging I see that a transaction gets created by spring batch before entering the tasklet's execute method and that a valid hibernateTransaction is created and put in the org.hibernate.jdbc.JDBCContext instance by calling getJpaDialect().beginTransaction() in org.springframework.orm.jpa.JpaTransactionManager#doBegin().
When methods of the orderRepository are called, I see that the call to AbstractPlatformTransactionManager#getTransaction finds an existing transaction and calls handleExistingTransaction. But later on org.hibernate.ejb.AbstractEntityManagerImpl#isTransactionInProgress returns false because no hibernateSession can be found.
I see that different Hibernate sessions and EntityManagers are created for transaction around the tasklet's execute method and for the calls to the repository. The outer hibernate transaction which is bound to the outer hibernate session can't be found by the inner hibernate session.
Any ideas how to solve this? How can the same hibernate session be used for the tasklet's execute method and the calls to the repository? Can the hibernateTemplate maybe be propagated to other sessions somehow?
Here's some code extract to show my setup:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackageClasses = MyRepositoryConfig.class)
#ComponentScan(basePackageClasses=MyRepositoryConfig.class)
public class MyRepositoryConfig {
#Autowired
private InfrastructureConfiguration infrastructureConfiguration;
#Bean
public EntityManagerFactory entityManagerFactory() throws SQLException {
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setJpaVendorAdapter(infrastructureConfiguration.hibernateJpaVendorAdapter());
factory.setPackagesToScan("com.example.model");
factory.setDataSource(infrastructureConfiguration.dataSource());
if (StringUtils.hasText(infrastructureConfiguration.getSchema())) {
factory.getJpaPropertyMap().put("hibernate.default_schema", infrastructureConfiguration.getSchema());
}
factory.afterPropertiesSet();
return factory.getObject();
}
#Bean(name= { "transactionManager", "myTransactionManager"})
public PlatformTransactionManager transactionManager() throws SQLException {
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(entityManagerFactory());
return txManager;
}
#Bean
public HibernateExceptionTranslator hibernateExceptionTranslator() {
return new HibernateExceptionTranslator();
}
#Bean
public MyRepositoryService myRepositoryService() {
return new myRepositoryServiceImpl();
}
}
#Configuration
public class DefaultInfrastructureConfiguration implements InfrastructureConfiguration {
#Value("${my.schema:MYSCHEMA}")
private String defaultSchema;
#Value("${novis.jdbc.url:jdbc:oracle:thin:#//example.com/example}")
private String jdbcUrl;
#Value("${novis.jdbc.username:scott}")
private String jdbcUsername;
#Value("${novis.jdbc.password:tiger}")
private String jdbcPassword;
#Value("${novis.jdbc.driverClassName:oracle.jdbc.driver.OracleDriver}")
private String jdbcDriverClassName;
#Value("${novis.hibernate.database:ORACLE}")
private String hibernateDatabase;
#Bean
#Override
public DataSource dataSource() {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(jdbcDriverClassName);
ds.setUrl(jdbcUrl);
ds.setUsername(jdbcUsername);
ds.setPassword(jdbcPassword);
ds.setTestWhileIdle(true);
ds.setValidationQuery("SELECT 1 FROM DUAL");
return ds;
}
#Override
public HibernateJpaVendorAdapter hibernateJpaVendorAdapter() {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setGenerateDdl(false);
vendorAdapter.setDatabase(Database.valueOf(Database.class, hibernateDatabase));
return vendorAdapter;
}
#Override
public String getSchema() {
return defaultSchema;
}
}
#Configuration
#Import(MyRepositoryConfig.class)
public class OrderManagerConfig {
#Autowired
#Qualifier("myTransactionManager")
private PlatformTransactionManager myTransactionManager;
....
#Bean
public Tasklet MyOrderTasklet() {
return new MyOrderTasklet();
}
....
#Bean
public Step processMyOrderErrorsStep() {
return steps.get("processMyOrderErrorsStep").
transactionManager(myTransactionManager).
tasklet(myOrderProcessor()).
listener(stepExecutionLoggerListener()).
build();
}
....
}
public class MyOrderTasklet implements Tasklet {
#Autowired
private OrderRepository orderRepository;
#Autowired
private PlatformTransactionManager myTransactionManager;
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
for (Order order : orderRepository.findByErrorIsNotNullAndErrorHandledFalse()) {
// Handle the error...
order.setErrorHandled(true);
orderRepository.saveAndFlush(order);
}
return RepeatStatus.FINISHED;
}
}
Exception:
03 Sep 2014 09:53:39:902 ERROR AbstractStep:225 - Encountered an error executing step processMyOrderErrorsStep in job processMyOrderErrors
javax.persistence.TransactionRequiredException: no transaction is in progress
at org.hibernate.ejb.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:959) ~[hibernate-entitymanager-3.6.10.Final.jar:3.6.10.Final]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:342) ~[spring-orm-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy153.flush(Unknown Source) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:289) ~[spring-orm-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy153.flush(Unknown Source) ~[?:?]
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.flush(SimpleJpaRepository.java:436) ~[spring-data-jpa-1.6.1.RELEASE.jar:?]
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.saveAndFlush(SimpleJpaRepository.java:404) ~[spring-data-jpa-1.6.1.RELEASE.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.executeMethodOn(RepositoryFactorySupport.java:405) ~[spring-data-commons-1.8.1.RELEASE.jar:?]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:390) ~[spring-data-commons-1.8.1.RELEASE.jar:?]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:344) ~[spring-data-commons-1.8.1.RELEASE.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke(CrudMethodMetadataPostProcessor.java:111) ~[spring-data-jpa-1.6.1.RELEASE.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy198.saveAndFlush(Unknown Source) ~[?:?]
at ch.local.ordermanager.MyOrderTasklet.execute(MyOrderTasklet.java:41) ~[classes/:?]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368) ~[spring-batch-infrastructure-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:386) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobOperator.start(SimpleJobOperator.java:314) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at ch.local.batchmanager.BatchJobRunner.runJob(BatchJobRunner.java:20) [classes/:?]
at ch.local.batchmanager.BatchManager.run(BatchManager.java:30) [classes/:?]
at ch.local.common.base.AbstractBatchApplication.execute(AbstractBatchApplication.java:38) [classes/:?]
at ch.local.batchmanager.BatchManager.main(BatchManager.java:9) [classes/:?]

I suspect the following piece of code:
#Bean
public EntityManager entityManager(EntityManagerFactory entityManagerFactory) {
return entityManagerFactory.createEntityManager();
}
Usually you don't configure an entityManager singleton, you let Spring decide when to create a new instance at transaction boundary.
The entity manager is usually being inserted using:
#PersistenceContext
EntityManager entityManager;
Even if you don't declare such bean, Spring will still create one and will provide you the one that's suitable for your current running transaction.
Using a singleton EntityManager is problematic, since the EntityManager is not thread-safe and reusing it without clearing might also lead to memory leaks and stale entity versions.

Although I defined the hibernate verison in the pom.xml to be 4.3.5, another library in our project has a dependency to hibernate 3.6.10, which was then used (and I did not notice). The error disappeared after getting rid of Hibernate 3 and using the latest hibernate 4 version (4.3.5 at the moment).