I have ONE spring boot (1.5.4.RELEASE) project using java 8 deployed on AWS HPC. This project architect scope works for Spring Web Application(Website), Rest API Services(Mobile Developer) & Account Administration for Company.
So there is 3 different respective Database like (2-SQL Server & 1-MySQL).
Here on stack-overflow, I'm posting my question for find a best way to implementation this Spring-Boot Project by help of talented stack-overflow users.
Here is my configure properties files.
application.properties
#For Public Website
spring.datasource.clone.web.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.clone.web.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName=PROD_WEB;
# Username and password
spring.datasource.web.username = web
spring.datasource.web.password = ED5FLW64ZU976Q36
#For Rest API
spring.datasource.clone.url = jdbc:mysql://localhost:3306/PROD_REST;
# Username and password
spring.datasource.clone.username = rest
spring.datasource.clone.password = Firewall77#
#For Account Administration for Company Users
spring.datasource.admin.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName=PROD_ADMIN;
# Username and password
spring.datasource.admin.username = admin
spring.datasource.admin.password = Firewall77#
# Backup & Cron Policy
...
I would greatly appreciate for some very good suggestion to implement it. your knowledge on this subject would help me, Thanks.
You need to implement two different beans, one for each datasource and make them take the corresponding configuration properties respectively:
First bean will be responsible for the first datasource configuration, and should be daclared as primary datasource with #Primary, so it can be setup as the main datasource for the project.
The second bean will configure the second datasource.
This is how they should be implemented in Spring:
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource.web")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.datasource.rest")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Here's how should be your application.properties configured, to take these two beans configuration into account:
#For Public Website
spring.datasource.web.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.web.url = jdbc:sqlserver://127.0.0.01\\dbo:1433;databaseName= PROD_WEB;
# Username and password
spring.datasource.web.username = web
spring.datasource.web.password = ED5FLW64ZU976Q36
#For Rest API
spring.datasource.rest.driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.rest.url = jdbc:mysql://localhost:3306/PROD_REST;
# Username and password
spring.datasource.rest.username = rest
spring.datasource.rest.password = Firewall77#
Note:
I changed the configuration properties here so they can be differentiated between web and rest datasources:
Properties starting with spring.datasource.web will be dedicated to configure the first datasource.
Properties starting with spring.datasource.rest will be dedicated to configure the second datasource.
try to use DataSource configuration specified at Configuration for each data sources for more help check this Using multiple datasources with Spring Boot and Spring Data
Related
I have a Spring Boot app that uses Hibernate and Spring Batch. I am using PostgreSQL for my backend database.
My project has 2 different data sources configured: one for Hibernate and one for Spring Batch. They are both in the same database but in different schemas.
My spring batch connection string is the following:
spring.batch.datasource.url=jdbc:postgresql://localhost:5432/mytestapp?currentSchema=springbatch
I have the Datasource configured in the following way:
#Bean("b_ds_prop")
#ConfigurationProperties("spring.batch.datasource")
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
#Bean("b_ds")
#DependsOn({"b_ds_prop"})
public DataSource batchFrameworkDatasource(DataSourceProperties dataSourceProperties) {
System.out.println("start print");
// dataSourceProperties.getSchema().stream().forEach(System.out::println);
// System.out.println("datasrcprop is null : " + (dataSourceProperties==null));
// System.out.println("datasrcprop.schema is null : " + (dataSourceProperties.getSchema()==null));
System.out.println("end print");
// dataSourceProperties.setSchema(Arrays.asList("spring_batch"));
return dataSourceProperties.initializeDataSourceBuilder().build();
}
#Bean
#DependsOn({"b_ds"})
public BatchConfigurer defaultBatchConfigurer(#Qualifier("b_ds") DataSource dataSource) {
return new DefaultBatchConfigurer(dataSource);
}
The currentSchema=springbatch setting is not respected, no matter what I do. All my Spring Batch tables keep ending up in public in Postgres.
The schema named springbatch is already present in my database : mytestapp.
I have tried currentSchema=springbatch, public, I have even tried spring.batch.table-prefix=springbatch..
I have tried everything, but still I cannot understand why my batch tables keep ending up in the public schema.
Note I have another connection string in my project like:
spring.hib.datasource.url=jdbc:postgresql://localhost:5432/mytestapp
I have also tried programmatically checking what is the value of schema in DataSourceProperties.
System.out.println("datasrcprop.schema is null : " + (dataSourceProperties.getSchema()==null));
This evaluates to true.
How can I set the schema to be different for Spring Batch ?
By default, Spring boot automatically creates the spring batch metadata tables at starting. It executes the default script which is located in org.springframework.batch.core package. In your situtation, this script is schema-postgresql.sql. This script creates tables in the postgre default schema so public if nothing is specified (and as we don't know which datasource spring boot uses to create spring batch metadata tables, we don't know what is the default schema).
When executing creation script, spring boot doesn't take into account spring.batch.table-prefix property which is only used to say spring batch where metadata tables are.
If you want to control this spring boot feature, you should :
modify the creation script (add the springbatch schema) and give spring boot the path for the modified script (spring.batch.schema property)
OR
create by yourself spring batch metadata tables before starting application and disable spring boot auto creation (spring.batch.initialize-schema=never property)
Good luck
Have you tried this configration:
spring.datasource.url=jdbc:postgresql://localhost:5432/mytestapp
#spring batch properties
spring.batch.job.enabled=false
spring.batch.initializer.enabled=false
spring.batch.initialize-schema=never
spring.batch.table-prefix=springbatch.
I have a spring boot application which I have configured most of the properties through the properties file. However, I was looking if there is a way to set the TRANSACTION_ISOLATION_LEVEL through the Spring boot properties. Could someone help me on this.
I'm initializing the data source bean in the following way:
#Bean
#ConfigurationProperties(prefix="spring.datasource")
public DataSource dataSource() {
return buildDataSource("spring.datasource");
}
private DataSource buildDataSource(String propPrefix) {
Stirng driverClassName = env.getProperty(propPrefix + ".driver-class-name");
return DataSourceBuilder.create()
.driverClassName(driverClassName)
.build();
}
Could someone please help me on how to specify the TRANSACTION_ISOLATION_LEVEL either through properties or during the data source initialization.
So, the javax.sql.DataSource does not provide a way to set default Transaction Isolation Level. Still, you can do it, but you must strict to particular implementation. Let me give you c couple of examples:
In case you use Apache BasicDataSource implementation of DataSource, then you can use this. This is for DBCP.
If you are using Apache BasicDataSource, but for DBCP2, you can do something like this.
But in most cases, if we are talking about Spring, we use Hikari Connection Pool. So, in case of HikariCP, you can use this method.
The same is applicable to Spring Boot. Let me explain - using Spring Boot properties file, you can set default transaction isolation level, but for specific DataSource, I mean this property:
spring.datasource.hikari.transaction-isolation
as you probably noticed, let you set default transaction isolation level for HikariCP (if you are using one) And this property:
spring.datasource.dbcp2.default-transaction-isolation
allow you to configure default isolation level for DBCP2.
Hope it helped, have a nice day! :)
I am working in spring hibernate project where I am connecting to multiple database. These database details are loaded in a properties file and is imported into hibernate.xml file. Each key was mapped as below.
dbName= dbHost
Mapping as below :
<bean id="dbId" parent="abstractDataSource">
<property name="url" value="${dbName}" />
</bean>
But now i need to include a condition such that the property key to be mapped to multiple values like below
E.g. dbName= statusFlag,dbHost,dbBackupHostName
I need to modify the hibernate configuration file such that using the keyName, corresponding values should be retrieved and splitted with comma separated.
Then I need to check for the statusFlag. If it is true, then i should set the dbValue to 'dbHost', else if the flag is false, it should be set to 'dbBackupHostName'.
Please help me how to implement this condition check in hibernate configuration file.
Also please let me know if this approach is fine to proceed or am i making it too difficult to implement?
Waiting for some valueable answers..
~~~Suriya
To use complex logic in bean wiring and creation in spring you can use Java Configuration
It should work for Spring 3. Sample pseudo code is below.
#Configuration
public class MyCustomConfig {
#Value
String statusFlag;
#Value
String dbHost;
#Value
String dbBackupHostName;
#Bean
public BasicDataSource datasource () {
BasicDataSource datasource = new BasicDataSource();
//Do some custom processing to create datasource
...
return datasource;//return after creation
}
}
Use PropertyPlaceholderConfigurer to load properties as #PropertySource is not available in 3.0.5.
If you can change the spring version to 3.1 which shouldn't make much difference then you can use #Profile along with #PropertySource.
If the logic is not too complex and you can separate set of properties that have to be active at a time. (Local db, Dev db, Prod db, Custom etc.) Then you can try using #Profile.
#Profile annotated bean is created only if that profile is active. A profile can be activated by setting spring.profiles.active.
So to activate the Dev profile we can set in the properties files
spring.profiles.active=dev
//activating 2 profiles
spring.profiles.active=dev,mvc
Below is the sample pseudo code.
#Configuration
#PropertySource("bootstrap.properties")//set spring.profiles.active in it
public class MyCustomConfig {
#Profile("profile1")
public BasicDataSource datasource1 () {
//config using ${profile1.dbName} etc
...
}
#Profile("profile2")
public BasicDataSource datasource2 () {
//config using ${profile2.dbName} etc
...
}
#Profile("profile3")
public BasicDataSource datasource3 () {
//config using ${profile3.dbName} etc
...
}
}
Our application allows our customer to have multiple databases, all running on one instance of the database server.
For example, databases might be dbcommon, dbLive, dbStaging, dbUAT, dbDev, dbSandbox. The common database and the Production database always exists, but the others are optional (and there is no limit). In dbcommon there is a table that tells us all the databases....so that's where I would need to start. The tables in common are different from the others, and the others all have the same schema (subscriber data)
Using Hibernate, how can I dynamically create/use a connection to either Live or Staging (or any of the others)? I am using Spring if that helps.
I have come across answers that suggest creating different connections in configuration, but because the number of subscriber databases can vary (per install, not while the app is running), this isn't an option for me.
As I discovered after posting this question, and as the user manish suggested, Hibernate's Multi Tenancy support (using the Database MultiTenancyStrategy) works for me. I had to piece together a solution using various resources (listed below).
http://www.ticnfae.co.uk/blog/2014/07/16/hibernate-multi-tenancy-with-spring/
Setting up a MultiTenantConnectionProvider using Hibernate 4.2 and Spring 3.1.1
Multi-Tenancy with Spring + Hibernate: "SessionFactory configured for multi-tenancy, but no tenant identifier specified"
I'm still looking for a way to be able to reference the common (shared) database at the same time as tenant databases...and will try to add that to this answer when complete.
The simplest way I can see to do this is to manage everything via profiles in Spring.
I accomplished this by using application.yml. I'm also using a Hikari connection pool, but that doesn't effect the configuration too much.
Here is an example of a application.yml with 3 profiles listed, and I've defined two of them as an example.
spring:
profiles:
include: dev,test,production
active: dev
---
spring:
profiles: dev
oms:
omsDataSource:
driverClassName: com.informix.jdbc.IfxDriver
jdbcUrl: jdbc:informix-sqli://devdb:9000/hol:INFORMIXSERVER=m_tcp_1;client_deve=en_US.8859-1;db_deve=en_US.8859-1;LOBCACHE=-1
password: oms
username: oms
connectionTestQuery: select count(*) from systables
maximumPoolSize: 5
---
spring:
profiles: test
oms:
omsDataSource:
driverClassName: com.informix.jdbc.IfxDriver
jdbcUrl: jdbc:informix-sqli://testdb:9000/hol:INFORMIXSERVER=m_tcp_1;client_deve=en_US.8859-1;db_deve=en_US.8859-1;LOBCACHE=-1
password: oms
username: oms
connectionTestQuery: select count(*) from systables
maximumPoolSize: 5
In my DB config class, I set the JPA repositories, and tell it what entityManager to use. I also setup the configuration properties to pull frmo the application.yml . This means it will swap out the details based on the profile the app is using on launch.
#EnableJpaRepositories(entityManagerFactoryRef = "entityManagerFactoryOms",
transactionManagerRef = "transactionManagerOms",
basePackages= "persistence.oms")
#Configuration
#ConfigurationProperties(prefix = "oms.omsDataSource")
public class omsDbConfig extends HikariConfig {
//This will automatically fill in the required fields from the application.yml.
#Bean
public HikariDataSource orcaDataSource() throws SQLException {
return new HikariDataSource(this);
}
//I use that datasource to define my entityMangerFactory
#Bean(name = "entityManagerFactoryOms")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryOrca() throws SQLException {
JpaVendorAdapter adapter = new HibernateJpaVendorAdapter();
Properties props = new Properties();
props.setProperty("hibernate.dialect","org.hibernate.dialect.InformixDialect");
LocalContainerEntityManagerFactoryBean emfb =
new LocalContainerEntityManagerFactoryBean();
emfb.setDataSource(orcaDataSource());
emfb.setPackagesToScan("persistence.oms");
emfb.setJpaProperties(props);
emfb.setJpaVendorAdapter(adapter);
return emfb;
}
}
The entities and repositories are defined normally, there's nothing special there. The DB will switch connection based on whatever profile I tell it to run.
I just switch out the active profile in the application.yml to whichever one I need.
Safety note: Define a production profile, don't have production as a default profile.
I need to create a connection pool from a spring application running in a tomcat server.
This application has many catalogs, the main catalog (its is static) called 'db' has just one table with all existing catalog names and a boolean flag for the "active" one.
When the application starts I need to choose from the main catalogs the active one, then I have to select it as default catalog.
How can I accomplish this?
Until now I used a custom class DataSourceSelector extends DriverManagerDataSource but now I need to improve the db connection using a pool, then I thought about a tomcat dbcp pool.
I would suggest the following steps:
Extend BasicDataSourceFactory to produce customized BasicDataSources.
Those customized BasicDataSources would already know which catalog is active and have the defaultCatalog property set accordingly.
Use your extended BasicDataSourceFactory in the Tomcat configuration.
#Configuration
public class DataAccessConfiguration {
#Bean(destroyMethod = "close")
public javax.sql.DataSource dataSource() {
org.apache.tomcat.jdbc.pool.DataSource ds = new org.apache.tomcat.jdbc.pool.DataSource();
ds.setDriverClassName("com.mysql.jdbc.Driver");
ds.setUrl("jdbc:mysql://localhost/db");
ds.setUsername("javauser");
ds.setPassword("");
ds.setInitialSize(5);
ds.setMaxActive(10);
ds.setMaxIdle(5);
ds.setMinIdle(2);
ds.get
return ds;
}
}