I'm using Spring Batch version 2.2.4.RELEASE
I tried to write a simple example with stateful ItemReader, ItemProcessor and ItemWriter beans.
public class StatefulItemReader implements ItemReader<String> {
private List<String> list;
#BeforeStep
public void initializeState(StepExecution stepExecution) {
this.list = new ArrayList<>();
}
#AfterStep
public ExitStatus exploitState(StepExecution stepExecution) {
System.out.println("******************************");
System.out.println(" READING RESULTS : " + list.size());
return stepExecution.getExitStatus();
}
#Override
public String read() throws Exception {
this.list.add("some stateful reading information");
if (list.size() < 10) {
return "value " + list.size();
}
return null;
}
}
In my integration test, I'm declaring my beans in an inner static java config class like the one below:
#ContextConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
public class SingletonScopedTest {
#Configuration
#EnableBatchProcessing
static class TestConfig {
#Autowired
private JobBuilderFactory jobBuilder;
#Autowired
private StepBuilderFactory stepBuilder;
#Bean
JobLauncherTestUtils jobLauncherTestUtils() {
return new JobLauncherTestUtils();
}
#Bean
public DataSource dataSource() {
EmbeddedDatabaseBuilder embeddedDatabaseBuilder = new EmbeddedDatabaseBuilder();
return embeddedDatabaseBuilder.addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql")
.addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql")
.setType(EmbeddedDatabaseType.HSQL)
.build();
}
#Bean
public Job jobUnderTest() {
return jobBuilder.get("job-under-test")
.start(stepUnderTest())
.build();
}
#Bean
public Step stepUnderTest() {
return stepBuilder.get("step-under-test")
.<String, String>chunk(1)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
#Bean
public ItemReader<String> reader() {
return new StatefulItemReader();
}
#Bean
public ItemProcessor<String, String> processor() {
return new StatefulItemProcessor();
}
#Bean
public ItemWriter<String> writer() {
return new StatefulItemWriter();
}
}
#Autowired
JobLauncherTestUtils jobLauncherTestUtils;
#Test
public void testStepExecution() {
JobExecution jobExecution = jobLauncherTestUtils.launchStep("step-under-test");
assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus());
}
}
This test passes.
But as soon as I define my StatefulItemReader as a step scoped bean (which is better for a stateful reader), the "before step" code is no longer executed.
...
#Bean
#StepScope
public ItemReader<String> reader() {
return new StatefulItemReader();
}
...
And I notice the same issue with processor and my writer beans.
What's wrong with my code? Is it related to this resolved issue: https://jira.springsource.org/browse/BATCH-1230
My whole Maven project with several JUnit tests can be found on GitHub: https://github.com/galak75/spring-batch-step-scope
Thank you in advance for your answers.
When you configure a bean as follows:
#Bean
#StepScope
public MyInterface myBean() {
return new MyInterfaceImpl();
}
You are telling Spring to use the proxy mode ScopedProxyMode.TARGET_CLASS. However, by returning the MyInterface, instead of the MyInterfaceImpl, the proxy only has visibility into the methods on the MyInterface. This prevents Spring Batch from being able to find the methods on MyInterfaceImpl that have been annotated with the listener annotations like #BeforeStep. The correct way to configure this is to return MyInterfaceImpl on your configuration method like below:
#Bean
#StepScope
public MyInterfaceImpl myBean() {
return new MyInterfaceImpl();
}
We have added a warning log message on startup that points out, as we look for the annotated listener methods, if the object is proxied and the target is an interface, we won't be able to find methods on the implementing class with annotations on them.
as suggested by pojo-guy
Solution is to implement StepExecutionListener and Override beforeStep method to set stepExecution
#Override
public void beforeStep(StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
Related
(Note this issue might be connected to this question, but it has a much smaller scope.)
I have the simplest of jobs defined like this:
#Configuration
#EnableBatchProcessing
public class FileTransformerConfiguration {
private JobBuilderFactory jobBuilderFactory;
private StepBuilderFactory stepBuilderFactory;
#Autowired
public FileTransformerConfiguration(JobBuilderFactory jobBuilderFactory,
StepBuilderFactory stepBuilderFactory) {
this.jobBuilderFactory = jobBuilderFactory;
this.stepBuilderFactory = stepBuilderFactory;
}
#Bean
public Job transformJob() {
return this.jobBuilderFactory.get("transformJob").incrementer(new RunIdIncrementer())
.flow(transformStep()).end().build();
}
#Bean
public Step transformStep() {
return this.stepBuilderFactory.get("transformStep")
.<String, String>chunk(1).reader(new ItemReader())
.processor(processor())
.writer(new ItemWriter()).build();
}
#Bean
public ItemProcessor<String, String> processor() {
return item -> {
System.out.println("Converting item (" + item + ")...");
return item;
};
}
}
public class ItemReader implements ItemStreamReader<String> {
private Iterator<String> it;
#Override
public void open(ExecutionContext executionContext) throws ItemStreamException {
this.it = Arrays.asList("A", "B", "C", "D", "E").iterator();
}
#Override
public String read() throws Exception {
return this.it.hasNext() ? this.it.next() : null;
}
#Override
public void close() throws ItemStreamException { }
#Override
public void update(ExecutionContext executionContext) throws ItemStreamException {}
}
#JobScope
public class ItemWriter implements ItemStreamWriter<String> {
#Override
public void open(ExecutionContext executionContext) throws ItemStreamException { }
#Override
public void write(List<? extends String> items) throws Exception {
items.forEach(item -> System.out.println("Writing item: " + item));
}
#Override
public void update(ExecutionContext executionContext) throws ItemStreamException { }
#Override
public void close() throws ItemStreamException { }
}
There is no fancy logic, just strings being moved through the pipeline.
The code is called like this:
#SpringBootApplication
public class TestCmpsApplication {
}
#SpringBootTest(classes = {TestCmpsApplication.class})
public class FileTransformerImplIT {
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job transformJob;
#Test
void test1() throws Exception {
String id = UUID.randomUUID().toString();
JobParametersBuilder jobParameters = new JobParametersBuilder();
jobParameters.addLong("PARAM_START_TIME", System.currentTimeMillis());
jobParameters.addString("PARAM_MAPPING_RULE_DEFINITION_ID", id, true);
this.jobLauncher.run(this.transformJob, jobParameters.toJobParameters());
}
#Test
void test2() throws Exception {
String id = UUID.randomUUID().toString();
JobParametersBuilder jobParameters = new JobParametersBuilder();
jobParameters.addLong("PARAM_START_TIME", System.currentTimeMillis());
jobParameters.addString("PARAM_MAPPING_RULE_DEFINITION_ID", id, true);
this.jobLauncher.run(this.transformJob, jobParameters.toJobParameters());
}
}
(Note there need to be two tests, even though they are identical. The first one will always work.)
So this works fine. However, once I add this:
#Bean
public Step transformStep() {
return this.stepBuilderFactory.get("transformStep")
.<String, String>chunk(1).reader(new ItemReader())
.processor(processor())
.writer(new ItemWriter())
.transactionAttribute(transactionAttribute()).build();
}
private TransactionAttribute transactionAttribute() {
DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
attribute.setPropagationBehavior(Propagation.NEVER.value());
return attribute;
}
Now the second test fails. The test itself says
TransactionSuspensionNotSupportedException: Transaction manager [org.springframework.batch.support.transaction.ResourcelessTransactionManager] does not support transaction suspension
While the log helpfully provides this error:
IllegalTransactionStateException: Existing transaction found for transaction marked with propagation 'never'
Okay. I directly told the job to never use a transaction, but somehow, somebody creates one anyway. So let's try MANDATORY. Now the test has the same error as above, the log now says:
IllegalTransactionStateException: No existing transaction found for transaction marked with propagation 'mandatory'
Somehow, somebody creates a transaction, but not for all two jobs? Surely SUPPORTS will work then. No, then the test will fail with the same exception, and the log will have this:
OptimisticLockingFailureException: Attempt to update step execution id=1 with wrong version (2), where current version is 3
I have no idea what is happening. Clearly someone creates transactions outside the step, but I have no idea how to stop them. Because I'd rather have no transactions. Or at least a working transaction management were transactions will work the same when called twice in a row.
I tried Spring Batch 4.2, 4.2.5, 4.3 and 4.3.1.
What did I do wrong? How can I make this work?
The problem is with the default job repository. It seems its transaction handling is buggy. To fix this, replace this with the JDBC job repository with an in-memory database. Just add this class to the Spring context:
#Configuration
#EnableBatchProcessing
public class InMemoryBatchContextConfigurer extends DefaultBatchConfigurer {
#Override
protected JobRepository createJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDatabaseType(DatabaseType.H2.getProductName());
factory.setDataSource(dataSource());
factory.setTransactionManager(getTransactionManager());
return factory.getObject();
}
public DataSource dataSource() {
EmbeddedDatabaseBuilder embeddedDatabaseBuilder = new EmbeddedDatabaseBuilder();
return embeddedDatabaseBuilder
.addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
.addScript("classpath:org/springframework/batch/core/schema-h2.sql")
.setType(EmbeddedDatabaseType.H2).build();
}
}
I have a code snippet that looks like this one
Service with #Transactional method
public class XService {
private Repo1 repo1;
private Repo2 repo2;
private Repo3 repo3;
XService(Repo1 repo1, Repo2 repo2, Repo3 repo3) {
this.repo1 = repo1;
this.repo2 = repo2;
this.repo3 = repo3;
}
#Transactional(rollbackFor = Exception.class)
public SomeObject method(Arg1 arg1, Arg2 arg2) {
repo1.method1();
repo2.method2();
repo3.method3(); // probability of exception here, in which case rollback is needed
}
}
Class from where method is invoked
public class YService {
private XService xService;
public YService(XService xService) {
this.xService = xService;
}
public SomeObject method(Arg1 arg1, Arg2 arg2) {
xService.method(arg1, arg2);
}
}
I have also added the #EnableTransactionManagement on my SpringBootApplication class. But the database operations from repo1 and repo2 are not rolled back in case of Exception from repo3.
Every repository is using Spring JDBCTemplate for querying the database.
Configuration class
#Configuration
public class ConfigurationClass {
#Bean
#Inject
public PlatformTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
#Bean
#Inject
public DataSource dataSource() {
// setting config properties here
return new HikariDataSource(config);
}
#Bean
#Inject
public Repo1 repo1(JDBCTemplate template) {
return new Repo1(template);
}
#Bean
#Inject
public Repo2 repo2(JDBCTemplate template) {
return new Repo2(template);
}
#Bean
#Inject
public Repo3 repo3(JDBCTemplate template) {
return new Repo3(template);
}
#Bean
#Inject
public XService XService(Repo1 repo1, Repo2 repo2, Repo3 repo3) {
return new XService(repo1, repo2, repo3);
}
#Bean
#Inject
public YService YService(XService xService) {
return new YService(xService);
}
}
Using a TransactionTemplate and enclosing my repo calls in the template worked.
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus transactionStatus) throws TransactionException {
repo1.method1();
repo2.method2();
repo3.method3();
}
});
I don't know the reason this one worked. It will be great if anyone can help in understanding the reason behind this.
I need to be able to create new bean instance - not on every call to method like in proxyMode=*, but only when needed (eg. to call .getObject()).
I know that there is ObjectFactory and ServiceFactory, but problem with this is that I cannot define factories in java config, but have to use hardcoded string inside bean. So this is what I want to achieve:
#Configuration
class Config {
#Bean
public MessageListenerContainerFactory listenerContainerFactory() {
MessageListenerContainerFactory listenerContainerFactory = new MessageListenerContainerFactory();
listenerContainerFactory.setMessageListener(rabbitProcessor());
return listenerContainerFactory;
}
#Bean
#Scope(SCOPE_PROTOTYPE)
public MessageListener rabbitProcessor() {
return new RabbitProcessor();
}
#Bean
#Scope(SCOPE_PROTOTYPE)
public MessageListener notThisOne() {
return new NotThisOne();
}
}
class MessageListenerContainerFactory {
private MessageListener messageListener;
public void setMessageListener(MessageListener messageListener) {
this.messageListener = messageListener;
}
public SimpleMessageListenerContainer createListenerContainer(){
SimpleMessageListenerContainer setMessageListener= new SimpleMessageListenerContainer();
//THIS HERE IS NEEDED!!!
Object needed = SPRINGCONTEXT.GETBEANNAMEOF(this.messageListener).getObject();
listenerContainer.setMessageListener(needed);
return setMessageListener;
}
}
You can try with #Qualifier annotation.
Ref - http://zetcode.com/articles/springbootqualifier/
#Bean
#Qualifier("rabbitProcessor")
#Scope(SCOPE_PROTOTYPE)
public MessageListener rabbitProcessor() {
return new RabbitProcessor();
}
#Bean
#Qualifier("notThisOne")
#Scope(SCOPE_PROTOTYPE)
public MessageListener notThisOne() {
return new NotThisOne();
}
Then you can create object by calling getBean() on applicationContext whenever you need like ..
applicationContext.getBean("rabbitProcessor");
//or
applicationContext.getBean("notThisOne");
I am using scheduled task to update my database like this:
public interface UserRatingManager {
public void updateAllUsers();
}
#Service
public class DefaultUserRatingManager implements UserRatingManager {
#Autowired
UserRatingDAO userRatingDAO;
#Override
#Transactional("txName")
public void updateAllUsers() {
List<String> userIds = userRatingDAO.getAllUserIds();
for (String userId : userIds) {
updateUserRating(userId);
}
}
}
public interface UserRatingDAO extends GenericDAO<UserRating, String> {
public void deleteAll();
public List<String> getAllUserIds();
}
#Repository
public class HibernateUserRatingDAO extends BaseDAO<UserRating, String> implements UserRatingDAO {
#Override
public List<String> getAllUserIds() {
List<String> result = new ArrayList<String>();
Query q1 = getSession().createQuery("Select userId from UserRating");
}
}
I configured the persistence like this:
#Configuration
#ComponentScan({ "com.estartup" })
#PropertySource("classpath:jdbc.properties")
#EnableTransactionManagement
#EnableScheduling
public class PersistenceConfig {
#Autowired
Environment env;
#Scheduled(fixedRate = 5000)
public void run() {
userRatingManager().updateAllUsers();
}
#Bean
public DataSource dataSource() {
DriverManagerDataSource driverManagerDataSource = new DriverManagerDataSource(env.getProperty("connection.url"), env.getProperty("connection.username"), env.getProperty("connection.password"));
driverManagerDataSource.setDriverClassName("com.mysql.jdbc.Driver");
return driverManagerDataSource;
}
public PersistenceConfig() {
super();
}
#Bean
public UserRatingUpdate userRatingUpdate() {
return new UserRatingUpdate();
}
#Bean
public UserRatingManager userRatingManager() {
return new DefaultUserRatingManager();
}
#Bean
public LocalSessionFactoryBean runnableSessionFactory() {
LocalSessionFactoryBean factoryBean = null;
try {
factoryBean = createBaseSessionFactory();
} catch (Exception e) {
e.printStackTrace();
}
return factoryBean;
}
private LocalSessionFactoryBean createBaseSessionFactory() throws IOException {
LocalSessionFactoryBean factoryBean;
factoryBean = new LocalSessionFactoryBean();
Properties pp = new Properties();
pp.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQLDialect");
pp.setProperty("hibernate.max_fetch_depth", "3");
pp.setProperty("hibernate.show_sql", "false");
factoryBean.setDataSource(dataSource());
factoryBean.setPackagesToScan(new String[] { "com.estartup.*" });
factoryBean.setHibernateProperties(pp);
factoryBean.afterPropertiesSet();
return factoryBean;
}
#Bean(name = "txName")
public HibernateTransactionManager runnableTransactionManager() {
HibernateTransactionManager htm = new HibernateTransactionManager(runnableSessionFactory().getObject());
return htm;
}
}
However, when I get to:
Query q1 = getSession().createQuery("Select userId from UserRating");
in the above HibernateUserRatingDAO I get an exception:
org.hibernate.HibernateException: createQuery is not valid without active transaction
at org.hibernate.context.internal.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:352)
at com.sun.proxy.$Proxy63.createQuery(Unknown Source)
at com.estartup.dao.impl.HibernateUserRatingDAO.getAllUserIds(HibernateUserRatingDAO.java:36)
How can I configure to include my scheduled tasks in transactions ?
EDITED:
Here is the code for BaseDAO
#Repository
public class BaseDAO<T, ID extends Serializable> extends GenericDAOImpl<T, ID> {
private static final Logger logger = LoggerFactory.getLogger(BaseDAO.class);
#Autowired
#Override
public void setSessionFactory(SessionFactory sessionFactory) {
super.setSessionFactory(sessionFactory);
}
public void setTopAndForUpdate(int top, Query query){
query.setLockOptions(LockOptions.UPGRADE);
query.setFirstResult(0);
query.setMaxResults(top);
}
EDITED
Enabling Spring transaction prints the following log:
DEBUG [pool-1-thread-1] org.springframework.transaction.annotation.AnnotationTransactionAttributeSource - Adding transactional method 'updateAllUsers' with attribute: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; 'txName'
What is happening in this case is that since you are using userRatingManager() inside the configuration (where the actual scheduled method exists), the proxy that Spring creates to handle the transaction management for UserRatingUpdate is not being used.
I propose you do the following:
public interface WhateverService {
void executeScheduled();
}
#Service
public class WhateverServiceImpl {
private final UserRatingManager userRatingManager;
#Autowired
public WhateverServiceImpl(UserRatingManager userRatingManager) {
this.userRatingManager = userRatingManager;
}
#Scheduled(fixedRate = 5000)
public void executeScheduled() {
userRatingManager.updateAllUsers()
}
}
Also change your transaction manager configuration code to:
#Bean(name = "txName")
#Autowired
public HibernateTransactionManager runnableTransactionManager(SessionFactory sessionFactory) {
HibernateTransactionManager htm = new HibernateTransactionManager();
htm.setSessionFactory(sessionFactory);
return htm;
}
and remove factoryBean.afterPropertiesSet(); from createBaseSessionFactory
As I already mentioned, I used your code and created a small sample that works for me. Judging by the classes used, I assumed you are using Hibernate Generic DAO Framework. It's a standalone sample, the main() class is Main. Running it you can see the transactional related DEBUG messages in logs that show when a transaction is initiated and committed. You can compare my settings, jars versions used with what you have and see if anything stands out.
Also, as I already suggested you might want to look in the logs to see if proper transactional behavior is being used and compare that with the logs my sample creates.
I tried to replicate your problem so I integrated it in my Hibernate examples on GitHub:
You can run my CompanySchedulerTest and see it's working so this is what I did to run it:
I made sure the application context is aware of our scheduler
<task:annotation-driven/>
The scheduler is defined in its own bean:
#Service
public class CompanyScheduler implements DisposableBean {
private static final Logger LOG = LoggerFactory.getLogger(CompanyScheduler.class);
#Autowired
private CompanyManager companyManager;
private volatile boolean enabled = true;
#Override
public void destroy() throws Exception {
enabled = false;
}
#Scheduled(fixedRate = 100)
public void run() {
if (enabled) {
LOG.info("Run scheduler");
companyManager.updateAllUsers();
}
}
}
My JPA/Hibernate configs are in applicationContext-test.xml and they are configured for JPA according to the Spring framework indications, so you might want to double check your Hibernate settings as well.
Given the following service:
public interface MyService {
void method();
}
And it's implementation:
#Service
public class MyServiceImpl implements MyService {
#Transactional
#CustomAnnotation
#Override
public void method() {
...
}
}
I would like to use a StaticMethodMatcherPointcutAdvisor in the following manner:
public class MyPointcutAdvisor extends StaticMethodMatcherPointcutAdvisor {
...
#Override
public boolean matches(Method method, Class targetClass) {
Method m = method;
if(annotationPresent(method)) {
return true;
}
Class<?> userClass = ClassUtils.getUserClass(targetClass);
Method specificMethod = ClassUtils.getMostSpecificMethod(method, userClass);
specificMethod = BridgeMethodResolver.findBridgedMethod(specificMethod);
if(annotationPresent(specificMethod )) {
return true;
}
return false;
}
...
}
The problem is that Spring uses an InfrastructureAdvisorAutoProxyCreator to create a Proxy of that class, whereas the DefaultAdvisorAutoProxyCreator would create the proxy for the MyPointcutAdvisor, but the MyPointcutAdvisor is only given the proxy as targetClass parameter. Thus, the PointcutAdvisor cannot find the annotation and therefore does not match.
For completion this is my Configuration-class:
#Configuration
#EnableTransactionManagement
public class MyConfiguration {
#Bean
public DefaultAdvisorAutoProxyCreator defaultAdvisorAutoProxyCreator() {
return new DefaultAdvisorAutoProxyCreator();
}
#Bean
public MyPointcutAdvisor myPointcutAdvisor() {
return new MyPointcutAdvisor();
}
...
}
My question is: Is there a way to use #EnableTransactionManagement in combination with a StaticMethodMatcherPointcutAdvisor ?
Workarounds:
Put #CustomAnnotation into the service interface: I want to have clean interfaces.
Add #Role(BeanDefinition.ROLE_INFRASTRUCTURE) to MyPointCutAdvisor bean configuration, thus, the InfrastructureAdvisorAutoProxyCreator will create the proxy. This seems like the wrong way, since this bean is not infrastructure
Copy the beans from ProxyTransactionManagementConfiguration, remove #EnableTransactionManagement and remove #Role(BeanDefinition.ROLE_INFRASTRUCTURE), thus the DefaultAdvisorAutoProxyCreator will create the proxy, which is my current workaround and results in the following configuration:
#Configuration
public class MyWorkaroundConfiguration {
#Bean
public DefaultAdvisorAutoProxyCreator defaultAdvisorAutoProxyCreator() {
return new DefaultAdvisorAutoProxyCreator();
}
#Bean
public MyPointcutAdvisor myPointcutAdvisor() {
return new MyPointcutAdvisor();
}
#Bean
public TransactionAttributeSource transactionAttributeSource() {
return new AnnotationTransactionAttributeSource();
}
#Bean(name = TransactionManagementConfigUtils.TRANSACTION_ADVISOR_BEAN_NAME)
public BeanFactoryTransactionAttributeSourceAdvisor transactionAdvisor(
TransactionInterceptor transactionInterceptor) {
BeanFactoryTransactionAttributeSourceAdvisor advisor =
new BeanFactoryTransactionAttributeSourceAdvisor();
advisor.setTransactionAttributeSource(transactionAttributeSource());
advisor.setAdvice(transactionInterceptor);
return advisor;
}
#Bean
public TransactionInterceptor transactionInterceptor(
PlatformTransactionManager transactionManager) {
TransactionInterceptor interceptor = new TransactionInterceptor();
interceptor.setTransactionAttributeSource(transactionAttributeSource());
interceptor.setTransactionManager(transactionManager);
return interceptor;
}
...
}
Using #EnableAspectJAutoProxy instead of the DefaultAutoProxyCreator works for me.
#Configuration
#EnableAspectJAutoProxy
#EnableTransactionManagement
public class MyConfiguration {
}
This also allows using #Aspect like M. Deinum suggested.