I have a data source configuration class that looks as follows, with separate DataSource beans for testing and non-testing environments using JOOQ. In my code, I do not use DSLContext.transaction(ctx -> {...} but rather mark the method as transactional, so that JOOQ defers to Spring's declarative transactions for transactionality. I am using Spring 4.3.7.RELEASE.
I have the following issue:
During testing (JUnit), #Transactional works as expected. A single method is transactional no matter how many times I use the DSLContext's store() method, and a RuntimeException triggers a rollback of the entire transaction.
During actual production runtime, #Transactional is completely ignored. A method is no longer transactional, and TransactionSynchronizationManager.getResourceMap() holds two separate values: one showing to my connection pool (which is not transactional), and one showing the TransactionAwareDataSourceProxy).
In this case, I would have expected only a single resource of type TransactionAwareDataSourceProxy which wraps my DB CP.
After much trial and error using the second set of configuration changes I made (noted below with "AFTER"), #Transactional works correctly as expected even during runtime, though TransactionSynchronizationManager.getResourceMap() holds the following value:
In this case, my DataSourceTransactionManager seems to not even know the TransactionAwareDataSourceProxy (most likely due to my passing it the simple DataSource, and not the proxy object), which seems to completely 'skip' the proxy anyway.
My question is: the initial configuration that I had seemed correct, but did not work. The proposed 'fix' works, but IMO should not work at all (since the transaction manager does not seem to be aware of the TransactionAwareDataSourceProxy).
What is going on here? Is there a cleaner way to fix this issue?
BEFORE (not transactional during runtime)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
#Primary
public DSLContext dslContext(org.jooq.Configuration configuration) throws SQLException {
return new DefaultDSLContext(configuration);
}
#Bean
#Primary
public org.jooq.Configuration defaultConfiguration(DataSourceConnectionProvider dataSourceConnectionProvider) {
org.jooq.Configuration configuration = new DefaultConfiguration()
.derive(dataSourceConnectionProvider)
.derive(SQLDialect.POSTGRES_9_5);
configuration.set(new DeleteOrUpdateWithoutWhereListener());
return configuration;
}
#Bean
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(DataSource dataSource) {
return new DataSourceConnectionProvider(dataSource);
}
#Configuration
#ConditionalOnClass(EmbeddedPostgres.class)
static class EmbeddedDataSourceConfig {
#Value("${spring.jdbc.port}")
private int dbPort;
#Bean(destroyMethod = "close")
public EmbeddedPostgres embeddedPostgres() throws Exception {
EmbeddedPostgres embeddedPostgres = EmbeddedPostgresHelper.startDatabase(dbPort);
return embeddedPostgres;
}
#Bean
#Primary
public DataSource dataSource(EmbeddedPostgres embeddedPostgres) throws Exception {
DataSource dataSource = embeddedPostgres.getPostgresDatabase();
return new TransactionAwareDataSourceProxy(dataSource);
}
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPool();
DataSource dataSource = new HikariDataSource(hikariConfig);
return new TransactionAwareDataSourceProxy(dataSource);
}
private HikariConfig buildPool() {
HikariConfig config = new HikariConfig();
config.setJdbcUrl(url);
config.setUsername(username);
config.setPassword(password);
config.setDriverClassName(driverClass);
config.setConnectionTestQuery("SELECT 1");
config.setMaximumPoolSize(maxPoolSize);
return config;
}
}
AFTER (transactional during runtime, as expected, all non-listed beans identical to above)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(TransactionAwareDataSourceProxy dataSourceProxy) {
return new DataSourceConnectionProvider(dataSourceProxy);
}
#Bean
public TransactionAwareDataSourceProxy transactionAwareDataSourceProxy(DataSource dataSource) {
return new TransactionAwareDataSourceProxy(dataSource);
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPoolConfig();
DataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource; // not returning the proxy here
}
}
}
I'll turn my comments into an answer.
The transaction manager should NOT be aware of the proxy. From the documentation:
Note that the transaction manager, for example
DataSourceTransactionManager, still needs to work with the underlying
DataSource, not with this proxy.
The class TransactionAwareDataSourceProxy is a special purpose class that is not needed in most cases. Anything that is interfacing with your data source through the Spring framework infrastructure should NOT have the proxy in their chain of access. The proxy is intended for code that cannot interface with the Spring infrastructure. For example, a third party library that was already setup to work with JDBC and did not accept any of Spring's JDBC templates. This is stated in the same docs as above:
This proxy allows data access code to work with the plain JDBC API and
still participate in Spring-managed transactions, similar to JDBC code
in a J2EE/JTA environment. However, if possible, use Spring's
DataSourceUtils, JdbcTemplate or JDBC operation objects to get
transaction participation even without a proxy for the target
DataSource, avoiding the need to define such a proxy in the first
place.
If you do not have any code that needs to bypass the Spring framework then do not use the TransactionAwareDataSourceProxy at all. If you do have legacy code like this then you will need to do what you already configured in your second setup. You will need to create two beans, one which is the data source, and one which is the proxy. You should then give the data source to all of the Spring managed types and the proxy to the legacy types.
Related
I was setting up a basic CRUD web app with JPA in plain Spring (no Spring Boot or Spring Data JPA) for educational purposes and faced a strange problem: Spring doesn't translate exceptions for my repository. According to the Spring documentation (here and here), it is sufficient to mark the repository with the #Repository annotation and Spring will automatically enable exception translation for this repository.
However, when I did so and triggered a UNIQUE constraint violation, I still was getting a JPA PersistenceException (with a Hibernate ConstraintViolationException inside) instead of the Spring DataIntegrityViolationException.
I used pure Java Spring configuration and it took me quite some time to realize that I should compare it with the XML configuration in the documentation. Compared to the pure Java configuration, the XML configuration adds a PersistenceExceptionTranslationPostProcessor into the context. When I added it manually with #Bean, it worked, but now I have a question.
Have I misconfigured something? The Spring documentation doesn't require registering that post-processor manually for pure Java configuration. Maybe there is another way to register it, say an #EnableXXX annotation?
Here is the summary of my configuration.
#Configuration
#ComponentScan("com.example.secured_crm")
public class SpringConfiguration {
// the problem is solved if I uncomment this
//#Bean
//public PersistenceExceptionTranslationPostProcessor exceptionTranslation() {
// return new PersistenceExceptionTranslationPostProcessor();
//}
}
#Configuration
#PropertySource("classpath:db.properties")
#EnableTransactionManagement
public class DataSourceConfiguration {
#Value("${jdbc.driver}")
private String driverClass;
#Value("${jdbc.url}")
private String url;
// ...
#Value("${hibernate.debug}")
private String hibernateDebug;
#Bean
public DataSource dataSource() {
var dataSource = new ComboPooledDataSource();
// ...
return dataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
var emFactory = new LocalContainerEntityManagerFactoryBean();
emFactory.setDataSource(dataSource());
emFactory.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
emFactory.setPackagesToScan("com.example.secured_crm.entities");
var properties = new Properties();
properties.setProperty("hibernate.dialect", hibernateDialect);
properties.setProperty("hibernate.show_sql", hibernateDebug);
properties.setProperty("hibernate.format_sql", "true");
emFactory.setJpaProperties(properties);
return emFactory;
}
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
var txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(entityManagerFactory);
return txManager;
}
}
public interface UserRepository {
User findByName(String username);
List<User> findAll();
void save(User user);
boolean deleteById(int id);
User findById(int id);
}
#Repository
public class UserJpaRepository implements UserRepository {
#PersistenceContext
EntityManager em;
#Override
public void save(User user) {
if (user.getId() == null) {
em.persist(user);
} else {
em.merge(user);
}
}
// and so on...
}
By the way, when I tried to add the post-processor in DataSourceConfiguration, it disabled #PropertySource effect. So far my impression of Spring is that it's one big hack...
It requires to manually register PersistenceExceptionTranslationPostProcessor in order for the exception translation to take effect.
The documentation you mentioned simply does not updated yet to show a fully working java configuration. It should mention to register this post processor. ( So feel free to provide a PR to update the docs.).
If you check from its javadoc , it already mentioned PersistenceExceptionTranslationPostProcessor is necessary to be registered :
As a consequence, all that is usually needed to enable automatic
exception translation is marking all affected beans (such as
Repositories or DAOs) with the #Repository annotation, along with
defining this post-processor as a bean in the application context.
P.S. If you are using spring-boot , and if it detects PersistenceExceptionTranslationPostProcessor is in the class-path , it will automatically register it by default such that you do not need to register manually.
I have seen a lot of examples of Spring Batch projects where either (a) a dataSource is defined, or (b) no dataSource is defined.
However, in my project, I would like my business logic to have access to a dataSource, but I want Spring Batch to NOT use the dataSource. Is this possible?
This guy has a similar problem: Spring boot + spring batch without DataSource
Generally, using spring-batch without a database is not a good idea, since there could be concurrency issues depending on the kind of job you define. So at least an using an inmemory db is strongly advised, especially if you plan to use the job in production.
Using SpringBatch with SpringBoot will initialize an inmemory datasource, if you do not configure your own datasource(s).
Taking this into account, let me redefine your question as follows: Can my businesslogic use another datasource than springbatch is using to update its BATCH-tables?
Yes, it can. As a matter of fact, you can use as many datasources as you want inside your SpringBatch Jobs. Just use by-name autowiring.
Here is how I do it:
I always use Configuration class, which defines all the datasources I have to use in my Jobs
Configuration
public class DatasourceConfiguration {
#Bean
#ConditionalOnMissingBean(name = "dataSource")
public DataSource dataSource() {
// create datasource, that is used by springbatch
// for instance, create an inmemory datasource using the
// EmbeddedDatabaseFactory
return ...;
}
#Bean
#ConditionalOnMissingBean(name = "bl1datasource")
public DataSource bl1datasource() {
return ...; // your first datasource that is used in your businesslogic
}
#Bean
#ConditionalOnMissingBean(name = "bl2datasource")
public DataSource bl2datasource() {
return ...; // your second datasource that is used in your businesslogic
}
}
Three points to note:
SpringBatch is looking for a datasource with the name "dataSource", if you do not provide this EXACT (uppercase 'S') name as the name, spring batch will try to autowire by type and if it finds more than one instance of DataSource, it will throw an exception.
Put your datasource configuration in its own class. Do not put them in the same class as your jobdefinitions are. Spring needs to be able to instantiate the datasource-SpringBean with the name "dataSource" very early when it loads the context. Before it starts to instantiate your Job- and Step-Beans. Spring will not be able to do it correctly, if you put your datasource definitions in the same class as you have your job/step definitions.
Using #ConditionalOnMissingBean is not mandatory, but I found it a good practics. It makes it easy to change the datasources for unit/integration tests. Just provide an additional test configuration in the ContextConfiguration of your unit/IT test which, for instance, overwrites the "bl1Datasource" with an inMemoryDataSource:
Configuration
public class TestBL1DatasourceConfiguration {
// overwritting bl1datasource with an inMemoryDatasource.
#Bean
public DataSource bl1datasource() {
return new EmbeddedDatabaseFactory.getDatabase();
}
}
In order to use the businesslogic datasources, use injection by name:
#Component
public class PrepareRe1Re2BezStepCreatorComponent {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private DataSource bl1datasource;
#Autowired
private DataSource bl2datasource;
public Step createStep() throws Exception {
SimpleStepBuilder<..., ...> builder =
stepBuilderFactory.get("astep") //
.<..., ...> chunk(100) //
.reader(createReader(bl1datasource)) //
.writer(createWriter(bl2datasource)); //
return builder.build();
}
}
Furthermore, you probably want to consider using XA-Datasources if you'd like to work with several datasources.
Edited:
Since it seems that you really don't want to use a datasource, you have to implement your own BatchConfigurer (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html) (as Michael Minella - the SpringBatch project lead - pointed out above).
You can use the code of org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer as a starting point for your own implementation. Simply remove all the datasource/transactionmanager code and keep the content of the if (datasource === null) part in the initialize method. This will initialize a MapBasedJobRepository and MapBasedJobExplorer. But again, this is NOT a useable solution in a productive environment, since it is not threadsafe.
Edited:
How to implement it:
Configuration class that defines the "businessDataSource":
#Configuration
public class DataSourceConfigurationSimple {
DataSource embeddedDataSource;
#Bean
public DataSource myBusinessDataSource() {
if (embeddedDataSource == null) {
EmbeddedDatabaseFactory factory = new EmbeddedDatabaseFactory();
embeddedDataSource = factory.getDatabase();
}
return embeddedDataSource;
}
}
The implementation of a specific BatchConfigurer:
(of course, the methods have to be implemented...)
public class MyBatchConfigurer implements BatchConfigurer {
#Override
public JobRepository getJobRepository() throws Exception {
return null;
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception {
return null;
}
#Override
public JobLauncher getJobLauncher() throws Exception {
return null;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return null;
}
}
And finally the main configuration and launch class:
#SpringBootApplication
#Configuration
#EnableBatchProcessing
// Importing MyBatchConfigurer will install your BatchConfigurer instead of
// SpringBatch default configurer.
#Import({DataSourceConfigurationSimple.class, MyBatchConfigurer.class})
public class SimpleTestJob {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
public Job job() throws Exception {
SimpleJobBuilder standardJob = this.jobs.get(JOB_NAME)
.start(step1());
return standardJob.build();
}
protected Step step1() throws Exception {
TaskletStepBuilder standardStep1 = this.steps.get("SimpleTest_step1_Step")
.tasklet(tasklet());
return standardStep1.build();
}
protected Tasklet tasklet() {
return (contribution, context) -> {
System.out.println("tasklet called");
return RepeatStatus.FINISHED;
};
}
public static void main(String[] args) throws Exception {
SpringApplication.run(SimpleTestJob.class, args);
}
}
I am trying to configure multiple JPA entity/transaction managers within the same application context using Spring's #Configuration class.
When the context loads, Spring is having difficulties auto-wiring the beans because they implement the same interfaces.
Unfortunately, I'm using legacy code so I can't auto-wire the beans directly and use the #Qualifier annotations, which is why I'm trying to do it using the configuration class.
Within a #Bean declaration, is there any way to qualify which bean should be injected? I thought that using a direct method call would be enough, but it typically results in errors such as
NoUniqueBeanDefinitionException: No qualifying bean of type
[javax.sql.DataSource] is defined: expected single matching bean but
found 4
Here's an example of what I'm trying to do below:
#Configuration
public class ApplicationConfig {
#Bean(name = "transactionManager1")
public PlatformTransactionManager transactionManager1() {
return new JpaTransactionManager(entityManagerFactory1());
}
#Bean(name = "entityManagerFactory1")
public EntityManagerFactory entityManagerFactory1() {
...
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setDataSource(dataSource1());
...
}
#Bean(destroyMethod = "")
#ConfigurationProperties(prefix = "datasource.test1")
public JndiObjectFactoryBean jndiObjectFactoryBean1() {
return new JndiObjectFactoryBean();
}
#Bean(name = "dataSource1")
public DataSource dataSource1() {
JndiDataSourceLookup lookup = new JndiDataSourceLookup();
return lookup.getDataSource(jndiObjectFactoryBean1().getJndiName());
}
#Bean(name = "transactionManager2")
public PlatformTransactionManager transactionManager2() {
return new JpaTransactionManager(entityManagerFactory2());
}
#Bean(name = "entityManagerFactory2")
public EntityManagerFactory entityManagerFactory2() {
...
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setDataSource(dataSource2());
...
}
#Bean(destroyMethod = "")
#ConfigurationProperties(prefix = "datasource.test2")
public JndiObjectFactoryBean jndiObjectFactoryBean2() {
return new JndiObjectFactoryBean();
}
#Bean(name = "dataSource2")
public DataSource dataSource2() {
JndiDataSourceLookup lookup = new JndiDataSourceLookup();
return lookup.getDataSource(jndiObjectFactoryBean2().getJndiName());
}
I suppose I could try to inject the beans directly via the Spring context's getBean() method, but is there a cleaner way of doing this?
I'm not too familiar with the #Primary annotation, but based on what I've read I don't know how spring would autowire the secondary data source in this case since it looks like it would always pick the beans with #Primary first.
If you cannot change the injection sites to add qualifiers, then you're going to have to create a delegating DataSource based on some logic (which you haven't detailed in the question).
Something like this.
#Primary #Bean
public DelegatingDataSource delegatingDataSource(List<DataSource> sources) {
return new DelegatingDataSource() {
#Override
public DataSource getTargetDataSource() {
// decide which dataSource to delegate to
return sources.get(0);
}
}
}
I've used DelegatingDataSource, but that may not be able to provide what you need. You may need to get more advanced with some kind of interceptor/aspect to get details of the caller on which to base the DataSource selection.
However it's implemented, you need to specify a #Primary bean and use it as a proxy.
Good day. My Spring Boot app uses Postgress database. For tests it uses H2 database. When running in non-test mode beans need to be initialized in this order:
1) Init DataSource
2) Init JPA beans
When running in test mode I need to create and populate H2 database before JPA beans initialization:
1) Init DataSource
2) Init DataSourceInitializer
3) Init JPA beans
The problem is that JPA beans get initialized before DataSourceInitializer (step 3 precedes step 2) and test fails on missing tables (hibernate.hbm2ddl.auto=validate).
Step 1
#Configuration
#EnableTransactionManagement
public class DataSourceConfig {
#Primary
#Bean
#ConfigurationProperties(prefix = "datasource.runtime")
public DataSource runtimeDataSource() {
return DataSourceBuilder.create().build();
}
}
Step 2
#Configuration
#Profile(Profiles.INTEGRATION_TEST)
public class DataSourceTestConfig {
#Autowired
private ResourceLoader resourceLoader;
#Bean
public DataSourceInitializer runtimeDataSourceInitializer(#Qualifier("runtimeDataSource") DataSource dataSource) {
DataSourceInitializer initializer = new DataSourceInitializer();
initializer.setDataSource(dataSource);
initializer.setDatabasePopulator(new ResourceDatabasePopulator(
resourceLoader.getResource("classpath:runtime/schema.sql")
));
return initializer;
}
}
Step 3
#Configuration
#EnableTransactionManagement
public class JpaConfig {
#Autowired
private Environment environment;
#Autowired
#Qualifier(value = "runtimeDataSource")
private DataSource runtimeDataSource;
#Primary
#Bean
public LocalContainerEntityManagerFactoryBean runtimeEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(runtimeDataSource)
.properties(hibernateSettings())
.packages(
"cz.adx.anx.car.cases.domain",
"cz.adx.anx.car.lib.domain",
"org.springframework.data.jpa.convert.threeten" // Hibernate support for Java 8 date and time classes
)
.persistenceUnit("runtimePersistenceUnit")
.build();
}
}
I need beans from class DataSourceTestConfig get initialized before JpaConfig and after DataSourceConfig but only in test mode. In non-test mode beans from JpaConfig should be initialized after DataSourceConfig and beans from DataSourceTestConfig must be omited. Therefore I cannot annotate class JpaConfig with #DependsOn beans from class DataSourceTestConfig because this class is located in test packages and not present in non-test mode. I could duplicate config classes and make them conditional on profile but I don't feel comfortable with this solution. Please, is there a better solution? Thanks in advance!
PS: My app uses two databases/datasources but I shortened the code above to make it easier to read. I'm using Spring Boot 1.3.1.RELEASE.
UPDATE 1:
I tried to use approach suggested by #luboskrnac. I placed annotation ActiveProfiles on my integration test classes:
#ActiveProfiles("IT")
public abstract class IntegrationTest {...}
And I used annotation Profile on relevant beans in class JpaConfig shown below:
#Configuration
#EnableTransactionManagement
public class JpaConfig {
#Autowired
private Environment environment;
#Autowired
#Qualifier(value = "runtimeDataSource")
private DataSource runtimeDataSource;
#Autowired
#Qualifier(value = "configDataSource")
private DataSource configDataSource;
#Profile("!IT")
#Bean(name = "runtimeEntityManagerFactory")
#DependsOn("runtimeDataSource")
public LocalContainerEntityManagerFactoryBean runtimeEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return createRuntimeEntityManagerFactory(builder);
}
#Profile("IT")
#Bean(name = "runtimeEntityManagerFactory")
#DependsOn("runtimeDataSourceInitializer")
public LocalContainerEntityManagerFactoryBean testRuntimeEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return jpaConfig.createRuntimeEntityManagerFactory(builder);
}
public LocalContainerEntityManagerFactoryBean createRuntimeEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(runtimeDataSource)
.properties(hibernateSettings())
.packages(
"cz.adx.anx.car.cases.domain",
"cz.adx.anx.car.lib.domain",
"org.springframework.data.jpa.convert.threeten" // Hibernate support for Java 8 date and time classes
)
.persistenceUnit("runtimePersistenceUnit")
.build();
}
}
And I'm creating the transaction managers the same way. Because I use two datasources (two different databases) I use bean names in the EnableJpaRepositories annotation.
#Configuration
#EnableJpaRepositories(
entityManagerFactoryRef = "runtimeEntityManagerFactory",
transactionManagerRef = "runtimeTransactionManager",
basePackages = "cz.adx.anx.car.lib.repository"
)
public class JpaCarLibRepositoryConfig {
}
So I need the non-test bean and test bean registered under the same name. But Spring gives me an exception:
org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type [javax.persistence.EntityManagerFactory] is defined: expected single matching bean but found 2: runtimeEntityManagerFactory
Any advices please?
I would suggest to drop any considerations about explicit bean creation ordering or bean dependencies.
Simply populate database in test based on Spring #Sql annotation. Test may look something like this:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration
#Sql("/test-schema.sql")
public class DatabaseTests {
#Test
public void emptySchemaTest {
// execute code that uses the test schema without any test data
}
#Test
#Sql({"/test-schema.sql", "/test-user-data.sql"})
public void userTest {
// execute code that uses the test schema and test data
}
}
If you'll need to swap datasource (e.g. using PostgereSQL in PROD and H2 in tests), just use Spring #Profile, #ActiveProfiles annotations.
I am trying to configure a couple of datasources within Spring Batch. On startup, Spring Batch is throwing the following exception:
To use the default BatchConfigurer the context must contain no more thanone DataSource, found 2
Snippet from Batch Configuration
#Configuration
#EnableBatchProcessing
public class BatchJobConfiguration {
#Primary
#Bean(name = "baseDatasource")
public DataSource dataSource() {
// first datasource definition here
}
#Bean(name = "secondaryDataSource")
public DataSource dataSource2() {
// second datasource definition here
}
...
}
Not sure why I am seeing this exception, because I have seen some xml based configuration for Spring batch that declare multiple datasources. I am using Spring Batch core version 3.0.1.RELEASE with Spring Boot version 1.1.5.RELEASE. Any help would be greatly appreciated.
You must provide your own BatchConfigurer. Spring does not want to make that decision for you
#Configuration
#EnableBatchProcessing
public class BatchConfig {
#Bean
BatchConfigurer configurer(#Qualifier("batchDataSource") DataSource dataSource){
return new DefaultBatchConfigurer(dataSource);
}
...
AbstractBatchConfiguration tries to lookup BatchConfigurer in container first, if it is not found then tries to create it itself - this is where IllegalStateException is thrown where there is more than one DataSource bean in container.
The approach to solving the problem is to prevent from creation the DefaultBatchConfigurer bean in AbstractBatchConfiguration.
To do it we hint to create DefaultBatchConfigurer by Spring container using #Component annotation:
The configuration class where #EnableBatchProcessing is placed we can annotate with #ComponentScan that scan the package that contains the empty class that is derived from DefaultBatchConfigurer:
package batch_config;
...
#EnableBatchProcessing
#ComponentScan(basePackageClasses = MyBatchConfigurer.class)
public class MyBatchConfig {
...
}
the full code of that empty derived class is here:
package batch_config.components;
import org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer;
import org.springframework.stereotype.Component;
#Component
public class MyBatchConfigurer extends DefaultBatchConfigurer {
}
In this configuration the #Primary annotation works for DataSource bean as in the example below:
#Configuration
public class BatchTestDatabaseConfig {
#Bean
#Primary
public DataSource dataSource()
{
return .........;
}
}
This works for the Spring Batch version 3.0.3.RELEASE
The simplest solution to make #Primary annotation on DataSource work might be just adding #ComponentScan(basePackageClasses = DefaultBatchConfigurer.class) along with #EnableBatchProcessing annotation:
#Configuration
#EnableBatchProcessing
#ComponentScan(basePackageClasses = DefaultBatchConfigurer.class)
public class MyBatchConfig {
I would like to provide a solution here, which is very similar to the one answered by #vanarchi, but I managed to put all the necessary configurations into one class.
For the sake of completeness, the solution here assumes that primary datasource is hsql.
#Configuration
#EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {
#Bean
#Primary
public DataSource batchDataSource() {
// no need shutdown, EmbeddedDatabaseFactoryBean will take care of this
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
EmbeddedDatabase embeddedDatabase = builder
.addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql")
.addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql")
.setType(EmbeddedDatabaseType.HSQL) //.H2 or .DERBY
.build();
return embeddedDatabase;
}
#Override
protected JobRepository createJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(batchDataSource());
factory.setTransactionManager(transactionManager());
factory.afterPropertiesSet();
return (JobRepository) factory.getObject();
}
private ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
//NOTE: the code below is just to provide developer an easy way to access the in-momery hsql datasource, as we configured it to the primary datasource to store batch job related data. Default username : sa, password : ''
#PostConstruct
public void getDbManager(){
DatabaseManagerSwing.main(
new String[] { "--url", "jdbc:hsqldb:mem:testdb", "--user", "sa", "--password", ""});
}
}
THREE key points in this solution:
This class is annotated with #EnableBatchProcessing and #Configuration, as well as extended from DefaultBatchConfigurer. By doing this, we instruct spring-batch to use our customized batch configurer when AbstractBatchConfiguration tries to lookup BatchConfigurer;
Annotate batchDataSource bean as #Primary, which instruct spring-batch to use this datasource as its datasource of storing the 9 job related tables.
Override protected JobRepository createJobRepository() throws Exception method, which makes the jobRepository bean to use the primary datasource, as well as use a different transactionManager instance from the other datasource(s).
The simplest solution is to extend the DefaultBatchConfigurer and autowire your datasource via a qualifier:
#Component
public class MyBatchConfigurer extends DefaultBatchConfigurer {
/**
* Initialize the BatchConfigurer to use the datasource of your choosing
* #param firstDataSource
*/
#Autowired
public MyBatchConfigurer(#Qualifier("firstDataSource") DataSource firstDataSource) {
super(firstDataSource);
}
}
Side Note (as this also deals with the use of multiple data sources): If you use autoconfig to run data initialization scripts, you may notice that it's not initializing on the datasource you'd expect. For that issue, take a look at this: https://github.com/spring-projects/spring-boot/issues/9528
You can define below beans and make sure you application.properties file has entries needed for
#Configuration
#PropertySource("classpath:application.properties")
public class DataSourceConfig {
#Primary
#Bean(name = "abcDataSource")
#ConfigurationProperties(prefix = "abc.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
#Bean(name = "xyzDataSource")
#ConfigurationProperties(prefix = "xyz.datasource")
public DataSource xyzDataSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
}
application.properties
abc.datasource.jdbc-url=XXXXX
abc.datasource.username=XXXXX
abc.datasource.password=xxxxx
abc.datasource.driver-class-name=org.postgresql.Driver
...........
...........
...........
...........
Here you can refer: Spring Boot Configure and Use Two DataSources
First, create a custom BatchConfigurer
#Configuration
#Component
public class TwoDataSourcesBatchConfigurer implements BatchConfigurer {
#Autowired
#Qualifier("dataSource1")
DataSource dataSource;
#Override
public JobExplorer getJobExplorer() throws Exception {
...
}
#Override
public JobLauncher getJobLauncher() throws Exception {
...
}
#Override
public JobRepository getJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
// use the autowired data source
factory.setDataSource(dataSource);
factory.setTransactionManager(getTransactionManager());
factory.afterPropertiesSet();
return factory.getObject();
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception {
...
}
}
Then,
#Configuration
#EnableBatchProcessing
#ComponentScan("package")
public class JobConfig {
// define job, step, ...
}