We Keep Coding

Dynamic Datasource using SpringBoot [duplicate]

This question already has answers here:
implement dynamically datasource in spring data jpa
(2 answers)
Closed 2 years ago.
I'm writing a Microservice using SpringBoot and I have a requirement to select the datasource dynamically. I will select the datasource based on the parameter. Each datasources will point to Oracle Database that has the same schema (same tables, triggers, stored procedures and etc). How can I implement this requirement?

Configure all datasources at startup, then:
either:
a. Have a different repo class that implements each datasource, check the parameter before calling the corresponding repo.
b. Have one repo class that checks the parameter and uses the corresponding datasource for its queries.

I did implement something similar...
I used application.properties to store datasource connection
Datasource drive name
spring.datasource.driver-class-name=
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
there is some code example on how to read and write to properties
https://github.com/evandbrown/amediamanager/blob/d42228a924cfbf14832e774a77c03eb0e9c2dba1/src/main/java/com/amediamanager/config/ConfigurationProviderChain.java
create an endpoint to update your properties
#PutMapping("/update")
public ResponseEntity<?> updateConnection(#RequestBody final List<ConfigurationProperty> properies) {
LOGGER.trace("Updating data source properties ");
for (final ConfigurationProperty configurationProperty : properies) {
config.getConfigurationProvider().persistDatabaseProperty(configurationProperty.getPropertyName(),
configurationProperty.getPropertyValue());
}
}
}
return new ResponseEntity<> (HttpStatus.OK);
}
One more tip from experience, ping the connection before executing update endpoint. :)
The only conclusion with my implementation, the user will need to restart the server for making changes to pickup.

You must implement two configuration datasource in your application.properties and after that configure two EntityManagerFactory and TransactionManager. You can switch between both datasources using diferent Repositories for every one.
application.properties
first.datasource.url=jdbc:oracle:thin:#//host:1521/firstdb
first.datasource.username=first
first.datasource.password=first
first.datasource.driver-class-name=oracle.jdbc.OracleDriver
second.datasource.url=jdbc:oracle:thin:#//host:1521/firstdb
second.datasource.username=second
second.datasource.password=second
second.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.jpa.database=default
Two packages to every Entity and repository and two configuration to setup.
First:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "entityManagerFactory",
basePackages = { "com.system.first.repo" }
)
public class FirstDbConfig {
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties(prefix = "first.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("dataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.system.first.domain")
.persistenceUnit("first")
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("entityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Second:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "secondEntityManagerFactory",
transactionManagerRef = "secondTransactionManager",
basePackages = { "com.system.second.repo" }
)
public class SecondDbConfig {
#Bean(name = "secondDataSource")
#ConfigurationProperties(prefix = "second.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "secondEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
secondEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("secondDataSource") DataSource dataSource
) {
return
builder
.dataSource(dataSource)
.packages("com.system.second.domain")
.persistenceUnit("second")
.build();
}
#Bean(name = "secondTransactionManager")
public PlatformTransactionManager secondTransactionManager(
#Qualifier("secondEntityManagerFactory") EntityManagerFactory
secondEntityManagerFactory
) {
return new JpaTransactionManager(secondEntityManagerFactory);
}
}

try this AbstractRoutingDataSource in springboot.
https://spring.io/blog/2007/01/23/dynamic-datasource-routing/

Spring boot is not using the right datasource for one of my repositories (I have two)

I have the following configuration for the main datasource.
#Configuration
#PropertySource({ "classpath:application.properties" })
#EnableJpaRepositories(
basePackages = "com.my.proj.datastores.authentication",
entityManagerFactoryRef = "userEntityManager",
transactionManagerRef = "userTransactionManager"
)
public class SpringDatabaseConfig {
#Autowired
private Environment env;
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean userEntityManager() {
LocalContainerEntityManagerFactoryBean em
= new LocalContainerEntityManagerFactoryBean();
em.setDataSource(userDataSource());
em.setPackagesToScan(
new String[] { "com.my.proj.datastores.authentication" });
HibernateJpaVendorAdapter vendorAdapter
= new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto",
env.getProperty("hibernate.hbm2ddl.auto"));
properties.put("hibernate.dialect",
env.getProperty("hibernate.dialect"));
em.setJpaPropertyMap(properties);
return em;
}
#Primary
#Bean
public DataSource userDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("spring.datasource.driver-class-name"));
dataSource.setUrl(env.getProperty("spring.datasource.jdbc-url"));
dataSource.setUsername(env.getProperty("spring.datasource.username"));
dataSource.setPassword(env.getProperty("spring.datasource.password"));
return dataSource;
}
#Primary
#Bean
public PlatformTransactionManager userTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
userEntityManager().getObject());
return transactionManager;
}
}
I also have this configuration for my second datasource :
#Configuration
#PropertySource({ "classpath:application.properties" })
#EnableJpaRepositories(
basePackages = "com.my.proj.datastores.ngl",
entityManagerFactoryRef = "otherTypeEntityManager",
transactionManagerRef = "otherTypeTransactionManager"
)
public class OracleDatabaseConfig {
#Autowired
private Environment env;
#Bean
public LocalContainerEntityManagerFactoryBean otherTypeEntityManager() {
LocalContainerEntityManagerFactoryBean em
= new LocalContainerEntityManagerFactoryBean();
em.setDataSource(otherTypeDataSource());
em.setPackagesToScan(
new String[] { "com.my.proj.datastores.ngl" });
HibernateJpaVendorAdapter vendorAdapter
= new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto",
env.getProperty("hibernate.hbm2ddl.auto"));
properties.put("hibernate.dialect",
env.getProperty("hibernate.dialect"));
em.setJpaPropertyMap(properties);
return em;
}
#Bean
public DataSource otherTypeDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("spring.second.datasource.driver-class-name"));
dataSource.setUrl(env.getProperty("spring.second.datasource.jdbc-url"));
dataSource.setUsername(env.getProperty("spring.second.datasource.username"));
dataSource.setPassword(env.getProperty("spring.second.datasource.password"));
return dataSource;
}
#Bean
public PlatformTransactionManager otherTypeTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
otherTypeEntityManager().getObject());
return transactionManager;
}
}
The primary one is for a database that handles auth, the second one is where I want to get data from. In an ideal world, these would be one database, but this is the way this project is.
I also have this application.properties :
## Spring DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.jdbc-url= ${DATASOURCE_URL}
spring.datasource.username= ${DATASOURCE_USERNAME}
spring.datasource.password= ${DATASOURCE_PASSWORD}
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.second.datasource.jdbc-url=jdbc:oracle:thin:#localhost:1521:XE
spring.second.datasource.username=admin
spring.second.datasource.password=password
spring.second.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
The application runs, and the #Primary data source seems to work.
However, I am unsure of how to "hook up" that second one so that a repository class uses it over the primary one.
I do have the repositories in different packages because I read that I need to.
The query I am trying to run in my repository is a simple "select all" on a table, and I get an error saying that the table doesn't exist... But I know it is trying to use the #Primary, because the error is a MySql exception, and the second data source is an Oracle one.
Any ideas?

What you need is to create 2 configuration classes, separate the model/repository packages etc to make the config easy like this.
com.packagename.multidatasources
├── first
│ ├── model
│ └── repository
└── second
├── model
└── repository
And then define configs for both the datasources and specify the #EnableJpaRepositories annotation to use the datasource and scan the base package where the respective repository is.
Use the following link to understand how to do it, you may find the entityManager and transactionManager a little extra but its best practice to include them:
http://fabiomaffioletti.me/blog/2014/04/15/distributed-transactions-multiple-databases-spring-boot-spring-data-jpa-atomikos/
http://xantorohara.blogspot.com/2013/11/spring-boot-jdbc-with-multiple.html
See the spring doc for the same here

This situation is complex but you can make it very simple by creating another application.properties which contains application-dev.properties and application-staging.properties. With the new implementation, have each data source in one of the created environments.
For example, application-dev.properties can contain the following config:
## Spring DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.jdbc-url= ${DATASOURCE_URL}
spring.datasource.username= ${DATASOURCE_USERNAME}
spring.datasource.password= ${DATASOURCE_PASSWORD}
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
and application-staging.properties can contain:
spring.second.datasource.jdbc-url=jdbc:oracle:thin:#localhost:1521:XE
spring.second.datasource.username=admin
spring.second.datasource.password=password
spring.second.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
Once that's done, you can then add spring.active.profile=dev to the top of the application-dev.properties file and spring.active.profile=staging to the top of the application-staging.properties file.
Finally, you can set the environment you intend to use on system startup application.properties file as spring.active.profile=dev or staging as the case may be. This approach should be able to solve the issues without much sweat.
In conclusion, you will be having 3 .properties file all together as shown below
1. application.properties
2. application-dev.properties
3. application-staging.properties
However, you will need to comment out your data configuration class.
I hope this help. Happy coding.

Spring-jpa-data: more than one DataSource and one single Spring config

I am trying to use Crud Repository from Spring-jpa-data:
My config with data access beans looks like:
#Configuration
#EnableTransactionManagement
#ComponentScan (basePackages = {"com.comp.olme"})
#PropertySource("classpath:OlmeSmb-${env}.properties")
#EnableJpaRepositories(basePackages="com.comp.olme", entityManagerFactoryRef ="emGapSort", transactionManagerRef = "txManagerGapSort")
#EnableScheduling
public class OlmeSmbConfig {
#Bean
public BasicDataSource olmeDataSource() {
BasicDataSource olmeDataSource = new BasicDataSource();
...
return olmeDataSource;
}
#Bean
public BasicDataSource gapSortDataSource() {
BasicDataSource gapSortDataSource = new BasicDataSource();
...
return gapSortDataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean emFactoryOLME(#Qualifier("olmeDataSource") BasicDataSource olmeDataSource) {
LocalContainerEntityManagerFactoryBean localConnectionFactoryBean = new LocalContainerEntityManagerFactoryBean();
...
return localConnectionFactoryBean;
}
#Bean
public LocalContainerEntityManagerFactoryBean emFactoryGapSort(#Qualifier("gapSortDataSource") BasicDataSource gapSortDataSource) {
LocalContainerEntityManagerFactoryBean localConnectionFactoryBeanGapSort = new LocalContainerEntityManagerFactoryBean();
...
return localConnectionFactoryBeanGapSort;
}
#Bean
public EntityManager emOLME(#Qualifier("emFactoryOLME") EntityManagerFactory entityManagerFactory) {
return entityManagerFactory.createEntityManager();
}
#Bean
public EntityManager emGapSort(#Qualifier("emFactoryGapSort") EntityManagerFactory entityManagerFactory) {
return entityManagerFactory.createEntityManager();
}
#Bean
public JpaTransactionManager txManagerOLME(#Qualifier("emFactoryOLME") EntityManagerFactory entityManagerFactoryOLME) {
JpaTransactionManager txManagerOLME = new JpaTransactionManager();
...
return txManagerOLME;
}
#Bean
public JpaTransactionManager txManagerGapSort(#Qualifier("emFactoryGapSort") EntityManagerFactory entityManagerFactoryGapSort) {
JpaTransactionManager txManagerGapSort = new JpaTransactionManager();
....
return txManagerGapSort;
}
}
So, as you can see, i have two datasources, two EntityManagerFactories, two TransactionManagers and others...
But i pass only one EntityManagerFactory into #EnableJpaRepositories annotation (entityManagerFactoryRef ="emGapSort").
The question is: how two use more than one DataSources (entityManagerFactory) with Spring-jpa-data?
I read one example where splitting config described as a solution, but i would like to use one single Spring Config. Is it possibe?
Thank you.

From spring boot - #5401
After a careful consideration we've decided not to implement this. That annotation proposal of yours is a mix of configuration key prefix, bean prefix and other bean-related settings. While it sounds appealing on paper, it would be quite hard to implement and probably even harder to keep it consistent with user's customizations.
So, you may try put it on two #Configuration classes (one #EnableJpaRepositories per #Configuration).

Spring & Hibernate without jpa

For my new project I plan to use Hibernate 5 and Spring 4 and as always like to separate into different layers / projects.
Gradle dependencies:
"org.springframework:spring-webmvc:4.2.1.RELEASE",
"org.springframework:spring-orm:4.2.1.RELEASE",
'org.hibernate:hibernate-core:5.0.2.Final',
'mysql:mysql-connector-java:5.1.36'
There is an API project, that contains a User.class. From my opinion this user class must not use any annotations for a database layer. It must not specify #Table(name = "users") or other things. It should be a simple Objects with getters and setters.
The database layer should decide how to store the data and this depends strongly on the database (e.g. MongoDB or MySQL).
I followed some tutorials for Hibernate and ended up with the following #Configuration class
#Configuration
#ComponentScan("de.pentos.proto")
#EnableWebMvc
#EnableTransactionManagement
public class AppConfig {
private static final Logger log = LoggerFactory.getLogger(AppConfig.class);
private static Properties getHibernateProperties() {
final Properties properties = new Properties();
properties.put("hibernate.show_sql", "true");
// properties.put("hibernate.dialect", "org.hibernate.dialect.MySQLDialect");
properties.put("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
properties.put("hbm2ddl.auto", "create");
return properties;
}
{
log.debug("Here am I: {}");
}
#Bean(name = "dataSource")
public DataSource getDataSource() {
final DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/myschema");
dataSource.setUsername("user");
dataSource.setPassword("password");
return dataSource;
}
#Inject
#Bean(name = "sessionFactory")
public SessionFactory getSessionFactory(final DataSource dataSource) {
final LocalSessionFactoryBuilder sessionBuilder = new LocalSessionFactoryBuilder(dataSource);
sessionBuilder.addAnnotatedClasses(User.class);
sessionBuilder.addProperties(getHibernateProperties());
return sessionBuilder.buildSessionFactory();
}
#Inject
#Bean(name = "transactionManager")
public HibernateTransactionManager getTransactionManager(final SessionFactory sessionFactory) {
final HibernateTransactionManager transactionManager = new HibernateTransactionManager(
sessionFactory);
return transactionManager;
}
}
It works very well, except it uses an annotated class.
How can I add my hbm/user.xml to the sessionBuilder?
I tried with Configuration class, that I found in some examples, but the Method buildSessionFactory() is deprecated.
I also tried the ServiceRegistry described here but then lost my datasource approach and without the datasource the system was not able to setup the HibernateTransactionManager.
Without HibernateTransactionManager I was not able to use #Transactional and I don't like to open and close me transactions manually.
Currently I'm spinning in circles and really need help to get it to work. I already thought about throwing Hibernate away and use my good old MyBatis approach, but you know, I like to learn something new...

Add xml files as a resource to SessionFactory, as follows:
#Inject
#Bean(name = "sessionFactory")
public SessionFactory getSessionFactory(final DataSource dataSource) {
final LocalSessionFactoryBuilder sessionBuilder = new LocalSessionFactoryBuilder(dataSource);
sessionBuilder.addResource("/path-to-/hbm/user.xml");
sessionBuilder.addAnnotatedClasses(User.class);
sessionBuilder.addProperties(getHibernateProperties());
return sessionBuilder.buildSessionFactory();
}

how can i parameterize datasource properties on Spring 4?

I am using Spring 4.16. I want to parameterize my persistence data. This is my config right now:
#Configuration
#EnableTransactionManagement
public class PersistenceConfiguration {
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManager = new LocalContainerEntityManagerFactoryBean();
entityManager.setDataSource(this.dataSource());
entityManager.setPackagesToScan(new String[] {"com.example.movies.domain"});
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
entityManager.setJpaVendorAdapter(vendorAdapter);
entityManager.setJpaProperties(this.properties());
return entityManager;
}
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/sarasa_db");
dataSource.setUsername("root");
dataSource.setPassword("mypassword");
return dataSource;
}
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
#Bean
public PersistenceExceptionTranslationPostProcessor exceptionTranslation() {
return new PersistenceExceptionTranslationPostProcessor();
}
private Properties properties() {
Properties properties = new Properties();
properties.setProperty("hibernate.hbm2ddl.auto", "update");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
properties.setProperty("hibernate.show_sql", "false");
return properties;
}
}
And i want to parameterize on my application.properties all things i can. First of all, i want to put in datasource properties (so as i was reading, is possible that spring builds my datasource automatically, but apparently that is only using JdbcTemplate...):
spring.datasource.driverClassName=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/sarasa_db
spring.datasource.username=root
spring.datasource.password=mypassword
And, if it's possible, all the Properties's properties, which i couldn't find nothing in doc
Do you know how could i do it ?
EDIT
This is my DAO implementation
#Configuration
#Import(PersistenceConfiguration.class)
public class DAOConfiguration {
#PersistenceContext
private EntityManager entityManager;
#Bean
public ClientDAO clientDAO() {
SimpleJpaRepository<Client, String> support = this.getSimpleJpaRepository(Client.class);
return new MySQLClientDAO(support);
}
#Bean
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
#Description("Hibernate repository helper")
protected <T> SimpleJpaRepository<T, String> getSimpleJpaRepository(Class<T> domainClass) {
return new SimpleJpaRepository<T, String>(domainClass, this.entityManager);
}
}

You could do something like this:
First define PropertySourcesPlaceholderConfigurer bean somewhere in your Spring configuration:
#Bean
public static PropertySourcesPlaceholderConfigurer propertyPlaceholderConfigurer() {
PropertySourcesPlaceholderConfigurer ppc = new PropertySourcesPlaceholderConfigurer();
ppc.setLocation(new ClassPathResource("application.properties"));
return ppc;
}
This configuration assumes that application.properties file is placed at the root of your classpath.
After setting up the property placeholder configurer you can access the properties in your database configuration class like so:
#Configuration
#EnableTransactionManagement
public class PersistenceConfiguration {
#Value("${spring.datasource.url}")
private String jdbcUrl;
// ...
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl(jdbcUrl);
// ...
}
}
If you want an easy way to parametrize all properties, you should take a look at Spring Boot. It uses the application.properties file to automatically create data source with those properties, and many other things. This is probably the automatic datasource creation you mentioned in your question.