I have two MySQL datasources in Spring Boot, therefore I have two config classes. But it looks like its only using the primary datasource. All entities are created for the primary datasource, so both crawlerdb and userdb entities are created in the userdb.
My userdb primary config:
/**
* Data source for the MySQL User Database schema
*/
#Configuration
#EntityScan(basePackages = "cs.crawler.server.projectcs.domain.userdb")
#EnableJpaRepositories(basePackages = "cs.crawler.server.projectcs.repos.userdb")
#EnableTransactionManagement
public class UserDomainConfig {
#Bean
#Primary
#ConfigurationProperties("spring.datasource.users")
public DataSourceProperties userDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.users.configuration")
public HikariDataSource firstDataSource(DataSourceProperties firstDataSourceProperties) {
return firstDataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
}
}
My secondary crawlerdb config:
/**
* Data source for the MySQL crawler database schema
*/
#Configuration
#EntityScan(basePackages = "cs.crawler.server.projectcs.domain.crawlerdb")
#EnableJpaRepositories(basePackages = "cs.crawler.server.projectcs.repos.crawlerdb")
#EnableTransactionManagement
public class CrawlerDomainConfig {
#Bean
#ConfigurationProperties("spring.datasource.crawler")
public DataSourceProperties crawlerDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.datasource.crawler.configuration")
public HikariDataSource secondDataSource(DataSourceProperties secondDataSourceProperties) {
return secondDataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
}
}
The entity classes for both schema's are in different packages as shown in #EntityScan above the class name. But when I check MySQL workbench for the created schema's I see that all entities are created in the userdb.
I fixed my issues by changing the config files like this aticle:
https://springframework.guru/how-to-configure-multiple-data-sources-in-a-spring-boot-application/
Related
I have a component in jar fole which has a autowired DataSource
#Service
Class ReadService{
#Autowire
DataSource readDataSource
}
#Service
Class WriteService{
#Autowire
DataSource writeDataSource
}
I have to configure two different severs, one to read and one to write.. how do I inject those DataSource to these beans created from Jar plugin...
You can read more here
in two words, you can specify different datasource configurations in properties file and create datasource beans separately
app.datasource.member.url=jdbc:mysql://localhost:3306/memberdb?createDatabaseIfNotExist=true
app.datasource.member.username=root
app.datasource.member.password=P#ssw0rd#
app.datasource.member.driverClassName=com.mysql.cj.jdbc.Driver
#card number (cardholder id, cardnumber)
app.datasource.cardholder.url=jdbc:mysql://localhost:3306/cardholderdb?createDatabaseIfNotExist=true
app.datasource.cardholder.username=root
app.datasource.cardholder.password=P#ssw0rd#
app.datasource.cardholder.driverClassName=com.mysql.cj.jdbc.Driver
#expiration date (card id, expiration month, expiration year)
app.datasource.card.url=jdbc:mysql://localhost:3306/carddb?createDatabaseIfNotExist=true
app.datasource.card.username=root
app.datasource.card.password=P#ssw0rd#
app.datasource.card.driverClassName=com.mysql.cj.jdbc.Driver
For primary Datasource
#Bean
#Primary
#ConfigurationProperties("app.datasource.member")
public DataSourceProperties memberDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("app.datasource.member.configuration")
public DataSource memberDataSource() {
return memberDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
For secondary datasource
/*cardholder data source */
#Bean
#ConfigurationProperties("app.datasource.cardholder")
public DataSourceProperties cardHolderDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("app.datasource.cardholder.configuration")
public DataSource cardholderDataSource() {
return cardHolderDataSourceProperties().initializeDataSourceBuilder()
.type(BasicDataSource.class).build();
}
/*card data source*/
#Bean
#ConfigurationProperties("app.datasource.card")
public DataSourceProperties cardDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("app.datasource.card.configuration")
public DataSource cardDataSource() {
return cardDataSourceProperties().initializeDataSourceBuilder()
.type(BasicDataSource.class).build();
}
And specify schema in entity declaration
#Table(name = "member", schema = "memberdb")
If you build 2 separate applications based on same component, you can simply use different database connection settings in application.yml (or application.properties) of both applications
For "Write" app:
spring:
datasource:
url: jdbc:postgresql://localhost:5433/writeDB
username: "username"
password: "password"
For "Read" app:
spring:
datasource:
url: jdbc:postgresql://localhost:5433/readDB
username: "username"
password: "password"
This question already has answers here:
implement dynamically datasource in spring data jpa
(2 answers)
Closed 2 years ago.
I'm writing a Microservice using SpringBoot and I have a requirement to select the datasource dynamically. I will select the datasource based on the parameter. Each datasources will point to Oracle Database that has the same schema (same tables, triggers, stored procedures and etc). How can I implement this requirement?
Configure all datasources at startup, then:
either:
a. Have a different repo class that implements each datasource, check the parameter before calling the corresponding repo.
b. Have one repo class that checks the parameter and uses the corresponding datasource for its queries.
I did implement something similar...
I used application.properties to store datasource connection
Datasource drive name
spring.datasource.driver-class-name=
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
there is some code example on how to read and write to properties
https://github.com/evandbrown/amediamanager/blob/d42228a924cfbf14832e774a77c03eb0e9c2dba1/src/main/java/com/amediamanager/config/ConfigurationProviderChain.java
create an endpoint to update your properties
#PutMapping("/update")
public ResponseEntity<?> updateConnection(#RequestBody final List<ConfigurationProperty> properies) {
LOGGER.trace("Updating data source properties ");
for (final ConfigurationProperty configurationProperty : properies) {
config.getConfigurationProvider().persistDatabaseProperty(configurationProperty.getPropertyName(),
configurationProperty.getPropertyValue());
}
}
}
return new ResponseEntity<> (HttpStatus.OK);
}
One more tip from experience, ping the connection before executing update endpoint. :)
The only conclusion with my implementation, the user will need to restart the server for making changes to pickup.
You must implement two configuration datasource in your application.properties and after that configure two EntityManagerFactory and TransactionManager. You can switch between both datasources using diferent Repositories for every one.
application.properties
first.datasource.url=jdbc:oracle:thin:#//host:1521/firstdb
first.datasource.username=first
first.datasource.password=first
first.datasource.driver-class-name=oracle.jdbc.OracleDriver
second.datasource.url=jdbc:oracle:thin:#//host:1521/firstdb
second.datasource.username=second
second.datasource.password=second
second.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.jpa.database=default
Two packages to every Entity and repository and two configuration to setup.
First:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "entityManagerFactory",
basePackages = { "com.system.first.repo" }
)
public class FirstDbConfig {
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties(prefix = "first.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("dataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.system.first.domain")
.persistenceUnit("first")
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("entityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Second:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "secondEntityManagerFactory",
transactionManagerRef = "secondTransactionManager",
basePackages = { "com.system.second.repo" }
)
public class SecondDbConfig {
#Bean(name = "secondDataSource")
#ConfigurationProperties(prefix = "second.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "secondEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
secondEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("secondDataSource") DataSource dataSource
) {
return
builder
.dataSource(dataSource)
.packages("com.system.second.domain")
.persistenceUnit("second")
.build();
}
#Bean(name = "secondTransactionManager")
public PlatformTransactionManager secondTransactionManager(
#Qualifier("secondEntityManagerFactory") EntityManagerFactory
secondEntityManagerFactory
) {
return new JpaTransactionManager(secondEntityManagerFactory);
}
}
try this AbstractRoutingDataSource in springboot.
https://spring.io/blog/2007/01/23/dynamic-datasource-routing/
My current project needs to connect to multiple databases. I set
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=update
in application.properties.
and I have some dbConfig as below:
#Configuration
public class DBSourceConfiguration {
public final static String DATA_SOURCE_PRIMARY = "dataSource";
public final static String DATA_SOURCE_PROPERTIES = "propertiesDataSource";
public final static String DATA_SOURCE_REPORT = "reportDataSource";
public final static String DATA_SOURCE_NEW_DRAGON = "newDragonDataSource";
#Primary
#Bean(name = DATA_SOURCE_PRIMARY)
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = DATA_SOURCE_REPORT)
#ConfigurationProperties(prefix = "externaldatasource.report")
public DataSource reportDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = DATA_SOURCE_NEW_DRAGON)
#ConfigurationProperties(prefix = "externaldatasource.newdragon")
public DataSource newDragonDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = DATA_SOURCE_PROPERTIES)
#ConfigurationProperties(prefix = "externaldatasource.properties")
public DataSource propertiesDataSource() {
return DataSourceBuilder.create().build();
}
}
and
<!-- language: Java -->
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = PrimaryDbConfig.ENTITY_MANAGER_FACTORY,
transactionManagerRef = PrimaryDbConfig.TRANSACTION_MANAGER,
basePackageClasses = { _TbsRepositoryBasePackage.class })
public class PrimaryDbConfig extends AbstractDbConfig {
public final static String ENTITY_MANAGER_FACTORY = "entityManagerFactoryPrimary";
public final static String ENTITY_MANAGER = "entityManagerPrimary";
public final static String TRANSACTION_MANAGER = "transactionManagerPrimary";
#Autowired
#Qualifier(DBSourceConfiguration.DATA_SOURCE_PRIMARY)
private DataSource dataSource;
#Primary
#Bean(name = ENTITY_MANAGER_FACTORY)
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder.dataSource(dataSource).properties(getVendorProperties(dataSource)).packages(_TbsEntityBasePackage.class).persistenceUnit("primaryPersistenceUnit").build();
}
#Primary
#Bean(name = ENTITY_MANAGER)
public EntityManager entityManager(EntityManagerFactoryBuilder builder) {
return entityManagerFactory(builder).getObject().createEntityManager();
}
#Primary
#Bean(name = TRANSACTION_MANAGER)
public PlatformTransactionManager transactionManager(EntityManagerFactoryBuilder builder) {
return new JpaTransactionManager(entityManagerFactory(builder).getObject());
}
}
and
<!-- language: Java -->
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = PropertiesDbConfig.ENTITY_MANAGER_FACTORY,
transactionManagerRef = PropertiesDbConfig.TRANSACTION_MANAGER,
basePackageClasses = { _PropertiesRepositoryBasePackage.class })
public class PropertiesDbConfig extends AbstractDbConfig {
public final static String ENTITY_MANAGER_FACTORY = "entityManagerFactoryProperties";
public final static String ENTITY_MANAGER = "entityManagerProperties";
public final static String TRANSACTION_MANAGER = "transactionManagerProperties";
#Autowired
#Qualifier(DBSourceConfiguration.DATA_SOURCE_PROPERTIES)
private DataSource dataSource;
#Bean(name = ENTITY_MANAGER_FACTORY)
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder.dataSource(dataSource).properties(getVendorProperties(dataSource)).packages(_PropertiesEntityBasePackage.class).persistenceUnit("propertiesPersistenceUnit").build();
}
#Bean(name = ENTITY_MANAGER)
public EntityManager entityManager(EntityManagerFactoryBuilder builder) {
return entityManagerFactory(builder).getObject().createEntityManager();
}
#Bean(name = TRANSACTION_MANAGER)
public PlatformTransactionManager transactionManager(EntityManagerFactoryBuilder builder) {
return new JpaTransactionManager(entityManagerFactory(builder).getObject());
}
}
and two more DBConfig classes(just like two DbConfig classes above).
My problem is every time I run this web application, Entities (under different packages) will generate to all databases. In other words, Tbs's(Primary) entities will generate tables to newDragon and all other databases.
For instance, Entity A belongs to primary data source, Entity B belongs to properties datasouce. But framework generates table A, B to both primary database and newDragon database and other two database.
Update 2018/06/01 - 1
Although framework generate all entities to all databases, but I can still access tables from the right database. All my web application functionalities work very well. This is very odd, isn't it?
I guess my configuration is fine, so that there is no any problems while my application access database (like read from wrong database and get empty result or insert data to wrong database, etc). Probably something else cause this gernerte all tables to all databases problem.
Based on the configuration you provided, CRUD tables from the right database shouldn't be problem. But generating tables into the right database, sometimes you may want to check whether the configuration picks entity/package names correctly or not.
Each LocalContainerEntityManagerFactoryBean is set with unique package class, the framework will then scan entities under this package name and generate tables accordingly at target datasource; however, there's a situation the packageToScan will be changed. As you have #EntityScan annotation, it would
overrides packagesToScan on all defined LocalContainerEntityManagerFactoryBean, reference code as follow: EntityScanRegistrar.java
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if (bean instanceof LocalContainerEntityManagerFactoryBean) {
LocalContainerEntityManagerFactoryBean factoryBean = (LocalContainerEntityManagerFactoryBean) bean;
factoryBean.setPackagesToScan(this.packagesToScan);
this.processed = true;
}
return bean;
}
As a result, even you have provided each LocalContainerEntityManagerFactoryBean with unique package class, the final result may still be overriden by the framework if you've #EntityScan somewhere at your application. Your configuration seems ok to me, so try to find and resolve the package names between #EntityScan and LocalContainerEntityManagerFactoryBean first, it should solve the issue.
reference: https://github.com/spring-projects/spring-boot/issues/6830
I have two datasources that I'm trying to assign a specific datasource to each JpaRepositories. I'm using the spring boot framework. The primary datasource is used about 90% of the time while the secondary datasource is used about 10% so it would be nice to default to the primary and only assign the secondary datasource when required. I tried using the docs here https://docs.spring.io/spring-boot/docs/current/reference/html/howto-data-access.html , but I dont think its exactly what I need. Any tips would be great!
spring.datasource.configuration.url=jdbc:postgresql://localhost:5432/mydatabase
spring.datasource.configuration.username=dockerusername
spring.datasource.configuration.password=dockerpassword
spring.datasource.configuration.driver-class-name=org.postgresql.Driver
spring.datasource.cached.url=jdbc:hsqldb:mem:main
spring.datasource.cached.driver-class-name=org.hsqldb.jdbc.JDBCDriver
spring.datasource.initialize=false
spring.jpa.database=default
spring.jpa.hibernate.ddl-auto=update
config file
#Configuration
#ComponentScan({"com.praeses.gov"})
public class Config {
#Bean
#ConfigurationProperties("spring.datasource.configuration")
public DataSourceProperties configDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.datasource.configuration")
public DataSource configDataSource() {
return configDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.cached")
public DataSourceProperties cachedDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.cached")
public DataSource cachedDataSource() {
return cachedDataSourceProperties().initializeDataSourceBuilder().build();
}
repository that uses the primary datasource
#Qualifier("spring.datasource.cached")
#Repository("spring.datasource.cached")
public interface GeoitemRepository extends JpaRepository<Geoitem, String> {
}
repository that uses the secondary datasource
#Qualifier("spring.datasource.configuration")
#Repository("spring.datasource.configuration")
public interface GeoitemhistoryRepository extends JpaRepository<Geoitemhistory, String> {
}
You can do something on similar lines as below link. Just create two config files instead of one(one each for each datasource) and add both of them to your main application.
Also, in the properties file, give as below:
datasource.local.url=
datasource.local.driver-class-name=
datasource.local.username=
datasource.local.password=
datasource.primary.url=
datasource.primary.driver-class-name=
datasource.primary.username=
datasource.primary.password=
Also, give #Primary annotation to the LocalContainerEntityManagerFactoryBean and platformTransactionManager in the config file for primary datasource.
See this answer for more details:
https://stackoverflow.com/a/44971911/6775742
I have a Spring Boot app for which I have configured two data sources. So far I've configured the data sources in my Application class (annotated with #EnableAutoConfiguration):
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.db1")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.db2")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
I also added the configuration values to application.properties:
datasource.db1.url=...
...
datasource.db2.url=...
...
Since db1 is the #Primary data source, it is chosen by default. How do I tell an interface extending JpaRepository that it should use db2 instead?
UPDATE: mentioning that my repository is an interface.
You can get the bean associated to the secondary data source from the application context.
For example in Application.java (I'm also using Spring Boot) you define:
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
and in a service (here for calling a stored procedure) you have:
#Service
public class EngineImpl implements EngineDao {
private SetScartiProcedure setScarti;
#Autowired
public void init(ApplicationContext ctx) {
DataSource dataSource = (DataSource) ctx.getBean("secondaryDataSource");
this.setScarti = new SetScartiProcedure(dataSource);
}
public class SetScartiProcedure extends StoredProcedure {
...
}
based on this you can define several DataSourcethis way
#Bean
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(customerDataSource())
.packages(Customer.class)
.persistenceUnit("customers")
.build();
}
#Bean
public LocalContainerEntityManagerFactoryBean orderEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(orderDataSource())
.packages(Order.class)
.persistenceUnit("orders")
.build();
}
and then bind each one of them to different classes that each one of them manages
#Configuration
#EnableJpaRepositories(basePackageClasses = Customer.class,
entityManagerFactoryRef = "customerEntityManagerFactory")
public class CustomerConfiguration {
...
}
#Configuration
#EnableJpaRepositories(basePackageClasses = Order.class,
entityManagerFactoryRef = "orderEntityManagerFactory")
public class OrderConfiguration {
...
}
the repositories should know which database to use by the DataSource that was bidden to the class