I have 1 spring context named PersistenceJPAConfig. Now I want to configure a spring batch and for that I have added a new class with #Configuration annotation and #EnableBatchProcessing. After adding new configuration class, I got error trying to use repository methods: nested exception is javax.persistence.TransactionRequiredException: no transaction is in progress. I know this is because I have a parent spring context and a child context which means I will have 2 instance for every repository and every service. I have tried to exclude repository scanning and service scanning with:
#ComponentScan(useDefaultFilters = false,
excludeFilters = {#ComponentScan.Filter(Repository.class), #ComponentScan.Filter(Service.class), #ComponentScan.Filter(Configuration.class)})
but it's not working. The only solution until now is to move all the beans from the second configuration to the first one, but I don't want that. How to solve this conflict between the contexts?
Main context:
package com.netoptics.server;
import java.util.Properties;
import javax.persistence.EntityManagerFactory;
import javax.sql.DataSource;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.JpaVendorAdapter;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import com.websilicon.security.SecurityGlobals;
import com.websilicon.util.AppConfig;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories({"com.whitelist.manager.repositories", "com.wsnms", "com.websilicon"})
public class PersistenceJPAConfig {
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setPackagesToScan("com.whitelist.manager", "com.wsnms", "com.websilicon");
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
entityManagerFactoryBean.setJpaVendorAdapter(vendorAdapter);
entityManagerFactoryBean.setJpaProperties(additionalProperties());
return entityManagerFactoryBean;
}
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory);
return transactionManager;
}
#Bean
public DataSource dataSource() {
String databaseDriver = AppConfig.getInstance().getString("dataDatabaseDriver", "");
String databaseUrl = AppConfig.getInstance().getString("dataDatabaseUrl", "");
String databaseUsername = AppConfig.getInstance().getString("dataDatabaseUsername", "");
String dataDatabasePassword = AppConfig.getInstance().getPassword("dataDatabasePassword", SecurityGlobals.KEY, "");
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(databaseDriver);
dataSource.setUrl(databaseUrl);
dataSource.setUsername(databaseUsername);
dataSource.setPassword(dataDatabasePassword);
return dataSource;
}
#Bean
public PersistenceExceptionTranslationPostProcessor exceptionTranslation() {
return new PersistenceExceptionTranslationPostProcessor();
}
private Properties additionalProperties() {
Properties properties = new Properties();
properties.setProperty("hibernate.hbm2ddl.auto", "none");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.PostgreSQL82Dialect");
properties.setProperty("hibernate.show_sql", "false");
properties.setProperty("hibernate.jdbc.batch_size", "1000");
return properties;
}
}
Second context for configuring spring batch:
#Configuration
#EnableBatchProcessing
#ComponentScan(useDefaultFilters = false,
excludeFilters = {#ComponentScan.Filter(Repository.class), #ComponentScan.Filter(Service.class)})
public class SaveImsiCSVBatchConfig {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Autowired
public DataSource dataSource;
#Autowired
private Environment environment;
#Autowired
private WmAdminImsisResourceHelper wmAdminImsisResourceHelper;
#Bean
public JobRepository jobRepository() {
MapJobRepositoryFactoryBean factoryBean = new MapJobRepositoryFactoryBean(new ResourcelessTransactionManager());
try {
return factoryBean.getObject();
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
#Bean
public JobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
return jobLauncher;
}
#Bean
#StepScope
public JdbcCursorItemReader<WmPushedImsi> reader(#Value("#{jobParameters['sortProperty']}") String sortProperty,
#Value("#{jobParameters['sortValue']}") String sortValue, #Value("#{jobParameters['username']}") String usernameFilter,
#Value("#{jobParameters['imsinumber']}") String imsiNumberFilter) {
JdbcCursorItemReader<WmPushedImsi> reader = new JdbcCursorItemReader<>();
reader.setDataSource(dataSource);
String sql =
"select us.username, wp.imsinumber, wp.startdate, wp.expiredate, case when wp.failedpushbatchid is not null or wp.faileddeletebatchid is not null then 'Yes' ELSE 'No' end as dirty from\n"
+ "wm_pushed_imsi wp INNER JOIN wm_admin_user wu on wp.userid = wu.id INNER JOIN users us on wu.userid = us.id";
if (usernameFilter != null && imsiNumberFilter != null) {
sql += " AND us.username LIKE '%" + usernameFilter + "%' AND wp.imsinumber LIKE '%" + imsiNumberFilter + "%'";
} else if (usernameFilter != null) {
sql += " AND us.username LIKE '%" + usernameFilter + "%'";
} else if (imsiNumberFilter != null) {
sql += " AND wp.imsinumber LIKE '%" + imsiNumberFilter + "%'";
}
if (sortProperty != null) {
sql += " order by " + sortProperty + " " + sortValue;
}
reader.setSql(sql);
reader.setRowMapper(new ImsiRowMapper());
return reader;
}
#Bean
public ImsiProcessor processor() {
return new ImsiProcessor();
}
#Bean
#StepScope
public FlatFileItemWriter<WmPushedImsi> writer(#Value("#{jobParameters['currentDate']}") Date currentDate) {
wmAdminImsisResourceHelper.createDirectoryForSavingCsv();
String fileName = wmAdminImsisResourceHelper.createFileNameForCsv(currentDate) + environment.getProperty("CSVEXTENSION");
String columnsTitle = Arrays.toString(new String[] {environment.getProperty("CSV_IMSINUMBER"), environment.getProperty("CSV_USERNAME"),
environment.getProperty("CSV_STARTDATE"), environment.getProperty("CSV_EXPIREDATE"), environment.getProperty("CSV_DIRTY")});
FlatFileItemWriter<WmPushedImsi> writer = new FlatFileItemWriter<>();
writer.setResource(new FileSystemResource(fileName));
writer.setHeaderCallback(writerHeader -> writerHeader.write(columnsTitle.substring(1, columnsTitle.length() - 1)));
writer.setLineAggregator(new DelimitedLineAggregator<>() {
{
setDelimiter(",");
setFieldExtractor(new BeanWrapperFieldExtractor<>() {
{
setNames(new String[] {WmPushedImsi_.IMSI_NUMBER, "username", WmPushedImsi_.START_DATE, WmPushedImsi_.EXPIRE_DATE, "dirty"});
}
});
}
});
return writer;
}
#Bean
public Step stepToCreateCsvFile() {
return stepBuilderFactory.get(Objects.requireNonNull(environment.getProperty("CSV_STEP_CREATE_FILE"))).<WmPushedImsi, WmPushedImsi>chunk(50000)
.reader(reader("", "", "", "")).processor(processor()).writer(writer(null)).build();
}
#Bean
public Step stepToDeleteFileAndCreateArchive() {
FileArchiveAndDeletingTasklet task = new FileArchiveAndDeletingTasklet();
task.setWmAdminImsisResourceHelper(wmAdminImsisResourceHelper);
task.setDateString(environment.getProperty("CSV_DATE"));
return stepBuilderFactory.get(Objects.requireNonNull(environment.getProperty("CSV_STEP_CREATE_ARCHIVE"))).tasklet(task).build();
}
#Bean
public Job exportImsiCSVJob() {
return jobBuilderFactory.get(Objects.requireNonNull(environment.getProperty("CSV_EXPORT_JOB"))).incrementer(new RunIdIncrementer())
.start(stepToCreateCsvFile()).next(stepToDeleteFileAndCreateArchive()).build();
}
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor(JobRegistry jobRegistry) {
JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor = new JobRegistryBeanPostProcessor();
jobRegistryBeanPostProcessor.setJobRegistry(jobRegistry);
return jobRegistryBeanPostProcessor;
}
public class ImsiRowMapper implements RowMapper<WmPushedImsi> {
#Override
public WmPushedImsi mapRow(ResultSet rs, int rowNum) throws SQLException {
WmPushedImsi wmPushedImsi = new WmPushedImsi();
wmPushedImsi.setImsiNumber(rs.getString(WmPushedImsi_.IMSI_NUMBER));
wmPushedImsi.setUsername(rs.getString("username"));
wmPushedImsi.setStartDate(rs.getDate(WmPushedImsi_.START_DATE));
wmPushedImsi.setExpireDate(rs.getDate(WmPushedImsi_.EXPIRE_DATE));
wmPushedImsi.setDirty(rs.getString("dirty"));
return wmPushedImsi;
}
}
}
You are using the MapJobRepository with a ResourcelessTransactionManager. With this configuration, there are no transactions on Spring Batch side. Hence the error no transaction is in progress.
You need to configure a JobRepository with the transaction manager you defined in your PersistenceJPAConfig. To do this, you have to define a bean of type BatchConfigurer and override getTransactionManager. Here is an example:
#Bean
public BatchConfigurer batchConfigurer() {
return new DefaultBatchConfigurer() {
#Override
public PlatformTransactionManager getTransactionManager() {
return new MyTransactionManager();
}
};
}
For more details, please check the Java config section of the reference documentation. Please note that this requires Spring Batch v4.1+.
Related
I am getting the error below when I try configure multiple datasources in Spring boot:
Property: xyz.integration.multidatasources.connections.datasource1.username
Value: safuser
Origin: class path resource [application.properties]:14:67
Reason: org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType
When I try to load multiple datasource configurations. The properties are as below:
xyz.integration.multidatasources.connections.datasource1.url=jdbc:postgresql://localhost:5432/safdb
xyz.integration.multidatasources.connections.datasource1.driver-class-name=org.postgresql.Driver
xyz.integration.multidatasources.connections.datasource1.username=auser
xyz.integration.multidatasources.connections.datasource1.password=apassword
The java class is as below:
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties(prefix = "xyz.integration.multidatasources")
public class MultipleDatasourcesProperties {
private Map<String, DataSourceProperties> connections = new HashMap<>();
public Map<String, DataSourceProperties> getConnections() {
return connections;
}
public void setConnections(Map<String, DataSourceProperties> connections) {
this.connections = connections;
}
}
Am I missing anything in my configuration?
Can you try this ?
application.properties
primary.url={primary-database-url}
primary.username={primary-database-username}
primary.password={primary-database-password}
primary.driver-class-name=com.mysql.jdbc.Driver
primary.test-on-borrow=true
primary.validation-query=SELECT 1
secondary.url={secondary-database-url}
secondary.username={secondary-database-username}
secondary.password={secondary-database-password}
secondary.driver-class-name=com.mysql.jdbc.Driver
secondary.test-on-borrow=true
secondary.validation-query=SELECT 1
secondary.validation-interval=25200000
Create the configuration beans
#Bean(name = "primaryDataSource")
#ConfigurationProperties("primary")
#Primary
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "secondaryDataSource")
#ConfigurationProperties("secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Next, create the JdbcTemplate beans that we are going to use for accessing the data sources in our data access layer.
#Bean(name = "primaryJdbcTemplate")
public JdbcTemplate primaryJdbcTemplate(#Qualifier("primary") DataSource primaryDs) {
return new JdbcTemplate(writeDs);
}
#Bean(name = “secondaryJdbcTemplate")
public JdbcTemplate secondaryJdbcTemplate(#Qualifier("secondary") DataSource secondaryDs) {
return new JdbcTemplate(secondaryDs);
}
Try accessing properties like this:
#Resource(name = "primaryJdbcTemplate")
private JdbcTemplate primaryJdbcTemplate;
#Resource(name = "secondaryJdbcTemplate")
private JdbcTemplate secondaryJdbcTemplate;
primaryJdbcTemplate.query(“any-query-to-apply-on-primary-data-source”);
I am trying to set up a second data source in my Spring application.
Below are the 2 configuration classes for the 2 data sources:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.XYXYale.persistence.XY",
entityManagerFactoryRef = "awEntityManagerFactory",
transactionManagerRef= "awTransactionManager"
)
public class Datasource1DataSourceConfig {
#Value("${spring.first-datasource.url}")
private String url;
#Value("${spring.first-datasource.username}")
private String username;
#Value("${spring.first-datasource.password}")
private String pw;
#Value("${spring.first-datasource.driver-class-name}")
private String driver;
#Bean
#Primary
#ConfigurationProperties("spring.first-datasource")
public DataSourceProperties awDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#Qualifier("awEntityManagerFactory")
public DataSource awDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
driver);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(pw);
return dataSource;
}
#Primary
#Bean(name = "awEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean awEntityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(awDataSource());
em.setPersistenceUnitName("XY");
");
em.setPackagesToScan(new String[] { "com.XY.XY.domain.XY" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setDatabasePlatform("org.hibernate.dialect.MySQLDialect");
em.setJpaVendorAdapter(vendorAdapter);
return em;
}
#Primary
#Bean
public PlatformTransactionManager awTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
awEntityManagerFactory().getObject());
return transactionManager;
}
}
The second Config class:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.XYXYale.persistence.YX",
entityManagerFactoryRef = "sndEntityManagerFactory",
transactionManagerRef= "sndTransactionManager"
)
public class SndDataSourceConfig {
#Value("${spring.second-datasource.jdbcUrl}")
private String url;
#Value("${spring.second-datasource.username}")
private String username;
#Value("${spring.second-datasource.password}")
private String pw;
#Value("${spring.second-datasource.driver-class-name}")
private String driver;
#Bean
#ConfigurationProperties("spring.second-datasource")
public DataSourceProperties sndDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Qualifier("sndEntityManagerFactory")
public DataSource sndDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
driver);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(pw);
return dataSource;
}
#Bean(name = "sndEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean sndEntityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(sndDataSource());
em.setPersistenceUnitName("snd");
em.setPackagesToScan(new String[] { "com.XY.XY.domain.YX" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setDatabasePlatform("org.hibernate.dialect.MySQLDialect");
em.setJpaVendorAdapter(vendorAdapter);
return em;
}
#Bean
public PlatformTransactionManager sndTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
sndEntityManagerFactory().getObject());
return transactionManager;
}
}
I have in com.XYXYale.persistence.XY a Spring data JPA Repo defined as follows
DemoRepo
#Repository
public interface DemoRepo extends CrudRepository<Demo, String>, DemoRepoCustom{
}
DemoRepoCustom
#NoRepositoryBean
public interface DemoRepoCustom {
Demo returnDemoContent();
}
DemoRepoImpl
public class DemoRepoImpl extends QuerydslRepositorySupport implements DemoRepoCustom {
public DemoRepoImpl() {
super(Demo.class);
}
public Demo returnDemoContent(){
return something;
}
}
The Repo is used in the following way
#Autowired
DemoRepo demoRepo;
I am getting this exception:
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'demoRepositoryImpl': Unsatisfied dependency expressed through method 'setEntityManager' parameter 0; nested exception is org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type 'javax.persistence.EntityManager' available: expected single matching bean but found 2: org.springframework.orm.jpa.SharedEntityManagerCreator#0,org.springframework.orm.jpa.SharedEntityManagerCreator#1
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:678)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:90)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:376)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1411)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:592)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:307)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1255)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1175)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:595)
... 55 more
Caused by: org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type 'javax.persistence.EntityManager' available: expected single matching bean but found 2: org.springframework.orm.jpa.SharedEntityManagerCreator#0,org.springframework.orm.jpa.SharedEntityManagerCreator#1
at org.springframework.beans.factory.config.DependencyDescriptor.resolveNotUnique(DependencyDescriptor.java:221)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1233)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1175)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:670)
... 70 more
Anyone having a suggestion on how to solve that? I could think of injecting the correct entitymanager to each repo, however I have no idea how to do that.
Thanks in advance. Cant find any solution on here or other sites.
if you will need to connect to more than one data source.
and assuming that you have already your spring app, then use this code 👇:
in your application.properties add this code:
# DEFAULT CONFIG
spring.main.banner-mode=off
server.port=2020
# CUSTOM PREFIX FOR SQL SERVER
sqlserver.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
sqlserver.datasource.jdbcUrl=jdbc:sqlserver://192.168.9.44;databaseName=myAwesomeDB
sqlserver.datasource.username=someUser
sqlserver.datasource.password=somePass
# DEFAULT PREFIX FOR POSTGRES
spring.datasource.platform=postgres
spring.datasource.url=jdbc:postgresql://192.168.6.125:5432/wilmer
spring.datasource.username=admWinter
spring.datasource.password=wilmer43nlp
now create a class called DataSourceDBConfig.java and add this code:
package com.example.demo.config;
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.env.Environment;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import javax.sql.DataSource;
/**
* #author Wilmer Villca
* #version 0.0.1
*/
#Configuration
public class DataSourceDBConfig {
private final Environment env;
#Autowired
public DataSourceDBConfig(Environment env) {
this.env = env;
}
#Bean
#Primary
#ConfigurationProperties(prefix = "sqlserver.datasource")
public DataSource dataSourceFromSqlServer() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public HikariDataSource dataSourceFromPostgres() {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setDriverClassName("org.postgresql.Driver");
hikariConfig.setJdbcUrl(env.getProperty("spring.datasource.url"));
hikariConfig.setUsername(env.getProperty("spring.datasource.username"));
hikariConfig.setPassword(env.getProperty("spring.datasource.password"));
return new HikariDataSource(hikariConfig);
}
#Bean
#Qualifier("jdbcTemplateSqlServer")
public JdbcTemplate jdbcTemplateSqlServer(#Qualifier("dataSourceFromSqlServer") DataSource ds) {
return new JdbcTemplate(ds);
}
#Bean
#Qualifier("jdbcTemplatePostgres")
public JdbcTemplate jdbcTemplatePostgres(#Qualifier("dataSourceFromPostgres") DataSource ds) {
return new JdbcTemplate(ds);
}
}
that is all you need to to access multiple databases 😉
NOTE: Do not forget and
remember that this only is an alternative, exist anothers many ways 🎉
i hope you understand, 👀
These are my classes and configurations,
ServiceImpl.java
#Override
#Transactional(propagation = Propagation.REQUIRED,rollbackFor = Exception.class)
public InsertUpdateSuccessBean insertNewSiteEntry(SiteEntry newSiteBean) throws SQLException {
dao.insert1();
dao.insert2();
except();
return new InsertUpdateSuccessBean(true,LoggerConstants.NEW_SITE_ENTRY_SUCCESS);
}
private void except()throws SQLException{
throw new SQLException();
}
DAOImpl.java
#Repository
public class DAOImpl implements DAO
{
#Autowired
private DataSourceTransactionManager transManager;
#Autowired
private CommonDAOUtils commonUtils;
private static final Logger logger = Logger.getLogger(HunterDAOImpl.class) ;
#Override
public Integer insert1() throws SQLException {
Integer insertNumRows = 0;
Connection connectionObject = transManager.getDataSource().getConnection();
PreparedStatement psObject = connectionObject.prepareStatement(
SQLQuery );
insertNumRows = psObject.executeUpdate();
commonUtils.closer(null,psObject,connectionObject);
// Function to close open connection resultsets statement objects
return insertNumRows;
}
#Override
public Integer insert2() throws SQLException {
Integer insertNumRows = 0;
Connection connectionObject = transManager.getDataSource().getConnection();
PreparedStatement psObject = connectionObject.prepareStatement(
SQLQuery );
insertNumRows = psObject.executeUpdate();
commonUtils.closer(null,psObject,connectionObject);
// Function to close open connection resultsets statement objects
return insertNumRows;
}
}
AppConfig.java
import com.interceptors.AppInterceptor;
import com.utils.Constants;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor;
import org.springframework.beans.factory.annotation.Configurable;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.*;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.core.env.Environment;
import org.springframework.core.io.ClassPathResource;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.servlet.config.annotation.DefaultServletHandlerConfigurer;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
import org.springframework.web.servlet.view.InternalResourceViewResolver;
#Component
#Configuration
#EnableWebMvc
#Configurable
#ComponentScan(basePackages = {Constants.BASE_PACKAGE})
#EnableAspectJAutoProxy(proxyTargetClass = true)
#PropertySource(Constants.DB_PROPERTIES_PATH)
#EnableTransactionManagement
public class AppConfig extends WebMvcConfigurerAdapter implements EnvironmentAware{
#Autowired
Environment environmentObject;
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer(){
PropertySourcesPlaceholderConfigurer placeHolder =new PropertySourcesPlaceholderConfigurer();
placeHolder.setLocation(new ClassPathResource(Constants.PROP_FILE_NAME));
return placeHolder;
}
#Bean
public InternalResourceViewResolver viewResolver(){
InternalResourceViewResolver internalResourceViewResolver = new InternalResourceViewResolver();
internalResourceViewResolver.setPrefix(Constants.ROOT_PATH);
internalResourceViewResolver.setSuffix(Constants.JSP_DOT_EXTENSION);
return internalResourceViewResolver;
}
#Override
public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer) {
configurer.enable();
}
#Bean
public DriverManagerDataSource dataSource(){
DriverManagerDataSource driverManagerDataSource = new DriverManagerDataSource();
driverManagerDataSource.setDriverClassName(environmentObject.getProperty(Constants.JDBC_DRIVERCLASS_NAME_PROP_FILE));
driverManagerDataSource.setUrl(environmentObject.getProperty(Constants.JDBC_URL_PROP_FILE));
driverManagerDataSource.setUsername(environmentObject.getProperty(Constants.JDBC_USERNAME_PROP_FILE));
driverManagerDataSource.setPassword(new String(Constants.PASSWORD));
return driverManagerDataSource;
}
#Bean
public AutowiredAnnotationBeanPostProcessor postProcessorBean(){
return new AutowiredAnnotationBeanPostProcessor();
}
private ClientHttpRequestFactory getClientHttpRequestFactory() {
int timeout = 5000;
HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory();
clientHttpRequestFactory.setConnectTimeout(timeout);
return clientHttpRequestFactory;
}
#Bean
public DataSourceTransactionManager transactionManager(){
final DataSourceTransactionManager transactionManager = new DataSourceTransactionManager(dataSource());
transactionManager.setRollbackOnCommitFailure(true);
return transactionManager;
}
}
And I also have an around advice for my project:
#Aspect
#Component
public class AroundAdvice
{
private static final Logger logger = Logger.getLogger(AroundAdvice.class) ;
// Add pointcuts here - package names here for around advice
#Around("execution(* com.beans.*.*(..)) || " +
"execution(* com.config.*.*(..)) || " +
"execution(* com.controller.*.*(..)) || " +
"execution(* com.dao.*.*(..)) || " +
"execution(* com.service.*.*(..)) || " +
"execution(* com.utils.*.*(..)) || "+
"execution(* com.interceptors.*.*(..))")
public Object aroundAdviceMethod(ProceedingJoinPoint proceedingObject)throws Throwable{
MethodSignature methodSignObject = (MethodSignature) proceedingObject.getSignature();
logger.debug(Constants.EXECUTING_METH_STR + methodSignObject.getMethod());
Object value = null;
try {
value = proceedingObject.proceed();
} catch (Exception e) {
e.printStackTrace();
logger.error(Constants.EXCEPTION_ASPECT_STR+methodSignObject.getMethod());
logger.error(Constants.EXCEPTION_MESSAGE,e);
throw e;
}
logger.debug(Constants.RETURN_STR+value);
return value;
}
}
On executing this flow, the inserts are successful, however when the exception is thrown, it is not rolling back. However, my logger reads that rolling back is initialized and done as follows,
14:11:51 DEBUG DataSourceTransactionManager:851 - Initiating transaction rollback
14:11:51 DEBUG DataSourceTransactionManager:325 - Rolling back JDBC transaction on Connection [org.postgresql.jdbc4.Jdbc4Connection#3b467b21]
14:11:51 DEBUG DataSourceTransactionManager:368 - Releasing JDBC Connection [org.postgresql.jdbc4.Jdbc4Connection#3b467b21] after transaction
Please let me know if I am missing something
The problem is your dao. You are opening connections yourself and therefor bypass all transaction management. Your dao is to complex just use a JdbcTemplate instead of your current code.
#Repository
public class DAOImpl implements DAO {
private static final Logger logger = Logger.getLogger(HunterDAOImpl.class) ;
private final JdbcTemplate jdbc;
public DAOImpl(DataSource dataSource) {
this.jdbc = new JdbcTemplate(dataSource);
}
#Override
public Integer insert1() throws SQLException {
return jdbc.update(SQLQuery);
}
#Override
public Integer insert2() throws SQLException {
return jdbc.update(SQLQuery);
}
}
This will do exactly the same as your code, with one main difference it will use the Connection opened when the transaction was started. Your sample used 3 separate connections and thus 3 individual transactions instead of 1 single transaction.
I am using AbstractRoutingDataSource to create multitenancy in my application. I noticed that after a few redeployments of the webapp from my IDE I eventually get the MySQL error "Too many connections".
After further investigations, I found out that when I run the MySQL command show processlist;, I see that the open connection amount is increased by 10 after each deployment, which probably means that the connection pool is somehow still alive somewhere.
Before I used AbstractRoutingDataSource I used the default spring datasource configuration (using application.properties) and it worked fine.
Here's the Multitenant configuration class:
import org.apache.commons.io.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.sql.DataSource;
import java.io.IOException;
import java.io.InputStream;
import java.util.*;
import java.util.stream.Collectors;
/**
* Created by Alon Segal on 16/03/2017.
*/
#Configuration
public class MultitenantConfiguration {
#Autowired
private DataSourceProperties properties;
/**
* Defines the data source for the application
*
* #return
*/
#Bean
#ConfigurationProperties(
prefix = "spring.datasource"
)
public DataSource dataSource() {
//Creating datasources map "resolvedDataSources" here
MultitenantDataSource dataSource = new MultitenantDataSource();
dataSource.setDefaultTargetDataSource(defaultDataSource());
dataSource.setTargetDataSources(resolvedDataSources);
// Call this to finalize the initialization of the data source.
dataSource.afterPropertiesSet();
return dataSource;
}
/**
* Creates the default data source for the application
*
* #return
*/
private DataSource defaultDataSource() {
.
.
.
}
}
And the datasource class:
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
/**
* Created by Alon Segal on 16/03/2017.
*/
public class MultitenantDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return TenantContext.getCurrentTenant();
}
}
I also tried to use #Bean(destroyMethod = "close") but there is not close method defined on AbstractRoutingDataSource.
I searched everywhere but could not find and answer. Can someone help me understand what's preventing the connection pool from being released between redeployments?
Thanks in advance.
Ok so I eventually solved the issue by giving up on using Spring's AbstractRoutingDataSource and instead using Hibernate's mechanism for multitenancy based on the solution that can be found in this article.
Long story short
You need to do 3 steps:
Step 1: Create a CurrentTenantIdentifierResolver
#Component
public class TenantIdentifierResolver implements CurrentTenantIdentifierResolver {
#Override
public String resolveCurrentTenantIdentifier() {
String tenantId = TenantContext.getCurrentTenant();
if (tenantId != null) {
return tenantId;
}
return DEFAULT_TENANT_ID;
}
#Override
public boolean validateExistingCurrentSessions() {
return true;
}
}
Step 2: Create a MultiTenantConnectionProvider
#Component
public class MultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifie) throws SQLException {
String tenantIdentifier = TenantContext.getCurrentTenant();
final Connection connection = getAnyConnection();
try {
if (tenantIdentifier != null) {
connection.createStatement().execute("USE " + tenantIdentifier);
} else {
connection.createStatement().execute("USE " + DEFAULT_TENANT_ID);
}
}
catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" + tenantIdentifier + "]",
e
);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try {
connection.createStatement().execute( "USE " + DEFAULT_TENANT_ID );
}
catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" + tenantIdentifier + "]",
e
);
}
connection.close();
}
#SuppressWarnings("rawtypes")
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
}
Step 3: Wire it up
#Configuration
public class HibernateConfig {
#Autowired
private JpaProperties jpaProperties;
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
return new HibernateJpaVendorAdapter();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource,
MultiTenantConnectionProvider multiTenantConnectionProviderImpl,
CurrentTenantIdentifierResolver currentTenantIdentifierResolverImpl) {
Map<String, Object> properties = new HashMap<>();
properties.putAll(jpaProperties.getHibernateProperties(dataSource));
properties.put(Environment.MULTI_TENANT, MultiTenancyStrategy.SCHEMA);
properties.put(Environment.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProviderImpl);
properties.put(Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, currentTenantIdentifierResolverImpl);
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource);
em.setPackagesToScan("com.autorni");
em.setJpaVendorAdapter(jpaVendorAdapter());
em.setJpaPropertyMap(properties);
return em;
}
}
I have a java app with redis, and it throws the exception.
Here are classes.
Main class:
public class App {
public static void main( String[] args ) {
ManipulatingData manData = new ManipulatingData();
manData.addData();
}
}
ApplicationConfig:
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package com.mycompany.springredisdatabook;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.StringRedisSerializer;
#Configuration
public class ApplicationConfig {
#Bean
public JedisConnectionFactory connectionFactory() {
JedisConnectionFactory connectionFactory = new JedisConnectionFactory();
connectionFactory.setHostName("localhost");
connectionFactory.setPort(6379);
return connectionFactory;
}
#Bean
public StringRedisTemplate redisTemplate() {
StringRedisTemplate redisTemplate = new StringRedisTemplate();
redisTemplate.setConnectionFactory(connectionFactory());
return redisTemplate;
}
#Bean
public RedisTemplate<String, Long> longTemplate() {
StringRedisSerializer STRING_SERIALIZER = new StringRedisSerializer();
RedisTemplate<String, Long> redisTemplate = new RedisTemplate<String, Long>();
redisTemplate.setConnectionFactory(connectionFactory());
redisTemplate.setKeySerializer(STRING_SERIALIZER);
redisTemplate.setValueSerializer(LongSerializer.INSTANCE);
return redisTemplate;
}
}
ManipulatingData:
package com.mycompany.springredisdatabook;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.StringRedisTemplate;
public class ManipulatingData {
public ManipulatingData() {}
#Autowired
StringRedisTemplate redisTemplate;
public void addData() {
double start = System.currentTimeMillis();
for (int i=1; i<=1000; i++) {
redisTemplate.opsForSet().add("k" + i, "v" + i);
}
double end = System.currentTimeMillis();
System.out.println("Add data time: " + (end-start));
}
public String getData (String key) {
return redisTemplate.opsForValue().get(key);
}
public void deleteData(String key) {
redisTemplate.opsForValue().getOperations().delete(key);
}
}
The Exception:
Exception in thread "main" java.lang.NullPointerException
at com.mycompany.springredisdatabook.ManipulatingData.addData(ManipulatingData.java:25)
at com.mycompany.springredisdatabook.App.main(App.java:11)
Java Result: 1
So, what is it? I have no idea. I'm using spring, by the way
your redisTemplate instance is null. check your spring.xml for the configuration of redisTemplate and possibilities which causes it to be null