I have an application which uses four databases in different geographical locations. All the databases contains same tables and only the database name is different according to the location.
I have to create some reports in my application which uses data from each database. What would be the proper way to create those database connection from a java application and is there a suitable design pattern for this task which I could use?
As you have not tagged your question with any of this, hibernate, JPA, ORM, I assume you are dealing with plain JDBC.
Having said that, I suggest you to have a DAO layer to deal with underlying databases, and leave the connection details to specific implementations. You can configure your connection strings in some .properties files, lets say.
[Complement]
You can also make use of DAO factory, an implementation of Abstract Factory or Factory Method pattern, whichever suits here.
[Links]
A very fine implementation of DAO and DAO Factory, by BalusC
Core J2EE Patterns -- arguably dated but might provide some idea.
There are multiple ways you can achieve this:
If you are using any Java EE container which supports distributed transaction then you can use there functionality.
If you are with plain JDBC then you will have to maintain your own connection for every database.
For JDBC:
Provide all connection details
Have an Facade which gives you desired object by calling a abstract generic DAO.
Have a factory which creates dao based on connection.
Use ORM tools like Hibernate, where you can use configuration for multiple database. Tutorial.
If you are using Spring, then you can configure one datasource per database. Docs
Design Patterns:
Facade Pattern - for hiding the complexity and multiple database usage.
Factory - In case you manage the database connection yourself.
Singleton - For datasources
You can handle multiple connections easily using a ORM tool like Hibernate.. You can specify each connection in a separate configuration file and instantiate the required connection by getting a new session factory each time.
Other way would be to use datasource and JNDI : Java connecting to multiple databases
I think you can use a combination of Factory pattern and Singleton pattern for the purpose.
The Ideal way to achieve this is by using a multi-dimensional system like OLAP. But see if you can create a view out of those databases. Then you just need to query the view (i.e. just a single database connection). Also you can still use multiple database connections if you want.
is very easy :)
1.Create a Data Source to try connection to DB
public DataSource getDataSource(String db) throws Exception {
DataSource dt = null;
InitialContext ic = null;
try {
if(db.trim().equals("you_database_name")) {
dt = (DataSource)ic.lookup("jdbc/connection_name");
} else if(db.trim().equals("you_database_name")) {
dt = (DataSource) ic.lookup("jdbc/connection_name");
}
return dt;
} catch(NamingException n) {
throw new Exception("Err getDataSource (ServiceLocator) NamingException - " + n.getMessage());
}
2.Create a class DataBase, remember close all connection in this point.
public class DataBases {
public YouNameDataSourceClass dataSrc;
public DataBases() throws Exception {
super();
dataSrc = new YouNameDataSourceClass.getDataSource();
}
public Connection getConnectionAS400() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
public Connection getConnectionOracle() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
public Connection getConnectionSQLServer() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
}
Good look.
Assuming you are using Spring MVC with Hibernate with XML configurations, follow these steps:
Create beans of all the databases in your spring-servlet file.
<bean id="dataSource1" Class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
<property name="url" value="jdbc:sqlserver://localhost:1433;databaseName=database1"/>
<property name="username" value="abc" />
<property name="password" value="abc#123" />
</bean>
Create sessionFactory beans of all the databases you want in the Spring-servlet file.
<bean id="datasource1SessionFactory" class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
<property name="dataSource"
ref="database1"/>
<property name="packagesToScan"
value="com.id4.iprod.entity"/>
<property name="hibernateProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">
</prop>
<prop key="hibernate.dialect">
org.hibernate.dialect.SQLServer2012Dialect
</prop>
</props>
</property>
</bean>
Now you just need to open session of the database you want to in DAO and access the desired results from desired database.
Session datasource1= this.datasource1SessionFactory.openSession();
Related
I am working in a project using Spring, Spring Data JPA, Spring Security, Primefaces...
I was following this tutorial about dynamic datasource routing with spring.
In this tutorial, you can only achieve dynamic datasource switching between a pre-defined datasources.
Here is a snippet of my code :
springContext-jpa.xml
<bean id="dsCgWeb1" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName.Cargest_web}"></property>
<property name="url" value="${jdbc.url.Cargest_web}"></property>
<property name="username" value="${jdbc.username.Cargest_web}"></property>
<property name="password" value="${jdbc.password.Cargest_web}"></property>
</bean>
<bean id="dsCgWeb2" class="org.apache.commons.dbcp.BasicDataSource">
// same properties, different values ..
</bean>
<!-- Generic Datasource [Default : dsCargestWeb1] -->
<bean id="dsCgWeb" class="com.cargest.custom.CargestRoutingDataSource">
<property name="targetDataSources">
<map>
<entry key="1" value-ref="dsCgWeb1" />
<entry key="2" value-ref="dsCgWeb2" />
</map>
</property>
<property name="defaultTargetDataSource" ref="dsCgWeb1" />
</bean>
What i want to do is to make the targetDataSources map dynamic same as its elements too.
In other words, i want to fetch a certain database table, use properties stored in that table to create my datasources then put them in a map like targetDataSources.
Is there a way to do this ?
Nothing in AbstractRoutingDataSource forces you to use a static map of DataSourceS. It is up to you to contruct a bean implementing Map<Object, Object>, where key is what you use to select the DataSource, and value is a DataSource or (by default) a String referencing a JNDI defined data source. You can even modify it dynamically since, as the map is stored in memory, AbstractRoutingDataSource does no caching.
I have no full example code. But here is what I can imagine. In a web application, you have one database per client, all with same structure - ok, it would be a strange design, say it is just for the example. At login time, the application creates the datasource for the client and stores it in a map indexed by sessionId - The map is a bean in root context named dataSources
#Autowired
#Qualifier("dataSources");
Map<String, DataSource> sources;
// I assume url, user and password have been found from connected user
// I use DriverManagerDataSource for the example because it is simple to setup
DataSource dataSource = new DriverManagerDataSource(url, user, password);
sources.put(request.getSession.getId(), dataSource);
You also need a session listener to cleanup dataSources in its destroy method
#Autowired
#Qualifier("dataSources");
Map<String, DataSource> sources;
public void sessionDestroyed(HttpSessionEvent se) {
// eventually cleanup the DataSource if appropriate (nothing to do for DriverManagerDataSource ...)
sources.remove(se.getSession.getId());
}
The routing datasource could be like :
public class SessionRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
HttpServletRequest request = ((ServletRequestAttributes)
RequestContextHolder.getRequestAttributes()).getRequest();
return request.getSession().getId();
}
#Autowired
#Qualifier("dataSources")
public void setDataSources(Map<String, DataSource> dataSources) {
setTargetDataSources(dataSources);
}
I have not tested anything because it would be a lot of work to setting the different database, but I thing that it should be Ok. In real world there would not be a different data source per session but one per user with a count of session per user but as I said it is an over simplified example.
The datasource used by a thread might change from time to time.
Should pay attention to concurrency, applications might get concurrency issues in concurrent environment.
thread-bound AbstractRoutingDataSource sample
It can be achieved with AbstractRoutingDataSource and keeping the information in the thread-local Variable. Here is a beautiful working example you can refer to:
Multi-tenancy: Managing multiple datasources with Spring Data JPA
Currently,I am storing database details in a property file and then creating an datasource using
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName">
<value>${driverClassName}</value>
</property>
<property name="url">
<value>${url}</value>
</property>
<property name="username">
<value>${username}</value>
</property>
<property name="password">
<value>${password}</value>
</property>
</bean>
My Client asked us to place a config database and this DB will store all the i18keys and the main database values.
So I need to create one two datasources one is for Configs and other is the main database.
I can create the config data sources using the same. But How I can create an second datasource as all the database details are stored in config database.
can you pointers will be very helpful.
You might take a look into the Java-configuration for Spring. You can combine that with your current XML-configuration using <context:component-scan base-package="..."/>.
The general approach would be to configure the first datasource for configuration (like in your current setup) using XML. The XML should also refer to a 'configuration class'.
That is a special class, annotated with #Configuration, which gets the first datasource injected (or maybe some DAO), and then defines a method like so:
#Bean
public DataSource secondDataSource() {
// Construct the second datasource using the configuration
// retrieved from the first datasource.
return new BasicDataSource();
}
Note that you might want to add a qualifier to either (or even both) datasources so you can distinguish between the two datasources when you want to have them injected into other beans using #Injector #Autowired.
I'm trying to implement the solution outlined in this answer. The short of it is: I want to set the role for each database connection in order to provide better data separation for different customers. This requires intercepting JDBC queries or transactions, setting the user before the query runs and resetting it afterwards. This is mainly done to comply with some regulatory requirements.
Currently I'm using Tomcat and Tomcat's JDBC pool connecting to a PostgreSQL database. The application is built with Spring and Hibernate. So far I couldn't find any point for intercepting the queries.
I tried JDBC interceptors for Tomcat's built in pool but they have to be global and I need to access data from my Web appliation in order to correlate requests to database users. As far as I see, Hibernate's interceptors work only on entities which is too high level for this use case.
What I need is something like the following:
class ConnectionPoolCallback {
void onConnectionRetrieved(Connection conn) {
conn.execute("SET ROLE " + getRole()); // getRole is some magic
}
void onConnectionReturned(Connection conn) {
conn.execute("RESET ROLE");
}
}
And now I need a place to register this callback... Does anybody have any idea how to implement something like this?
Hibernate 4 has multitenancy support. For plain sql you will need datasource routing which I believe spring has now or is an addon.
I would not mess ( ie extend) the pool library.
Option 1:
As Adam mentioned, use Hibernate 4's multi-tenant support. Read the docs on Hibernate multi-tenancy and then implement the MultiTenantConnectionProvider and CurrentTenantIdentifierResolver interfaces.
In the getConnection method, call SET ROLE as you've done above. Although it's at the Hibernate level, this hook is pretty close in functionality to what you asked for in your question.
Option 2:
I tried JDBC interceptors for Tomcat's built in pool but they have to
be global and I need to access data from my Web appliation in order to
correlate requests to database users.
If you can reconfigure your app to define the connection pool as a Spring bean rather than obtain it from Tomcat, you can probably add your own hook by proxying the data source:
<!-- I like c3p0, but use whatever pool you want -->
<bean id="actualDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="jdbcUrl" value="${db.url}"/>
<property name="user" value="${db.user}" />
.....
<!-- uses the actual data source. name it "dataSource". i believe the Spring tx
stuff looks for a bean named "dataSource". -->
<bean id="dataSource" class="com.musiKk.RoleSettingDSProxy">
<property name="actualDataSource"><ref bean="actualDataSource" /></property>
</bean>
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource"><ref bean="dataSource" /></property>
....
And then build com.musiKk.RoleSettingDSProxy like this:
public class RoleSettingDSProxy implements DataSource {
private DataSource actualDataSource;
public Connection getConnection() throws SQLException {
Connection con = actualDataSource.getConnection();
// do your thing here. reference a thread local set by
// a servlet filter to get the current tenant and set the role
return con;
}
public void setActualDataSource(DataSource actualDataSource) {
this.actualDataSource = actualDataSource;
}
Note that I haven't actually tried option 2, it's just an idea. I can't immediately think of any reason why it wouldn't work, but it may unravel on you for some reason if you try to implement it.
One solution that comes to mind is to utilize the Hibernate listeners/callbacks. But do beware that is very low level and quite error-prone. I use it myself to get a certain degree of automated audit logging going; it was not a pretty development cycle to get it to work reliably. unfortunately I can't share code since I don't own it.
http://docs.jboss.org/hibernate/entitymanager/3.6/reference/en/html/listeners.html
I'm using Spring with DBCP and need to refresh my datasource when some configuration on operation environment changes, without restart all application.
If I do it with no use of DBCP, I force this refresh closing current opened datasource in use and Start a new instance of DataSource.
Using DBCP+Spring, I can't do that.
Somebody knows if it is possible?
I don't think there is such a support in plain DBCP, mostly because database connection properties are very rarely changing during the lifetime of the application. Also you will have to consider transition time, when some connections served by the old data source are still opened while others are already served from the new (refreshed) one.
Decorator/proxy approach
I would suggest you to write custom implementation of DataSource leveraging Decorator/Proxy design pattern. Your implementation would simply call target data source (created by DBCP), most of the time doing nothing more. But when you call some sort of refresh() method, your decorator will close previously created data source and create new one with fresh configuration. Remember about multi-threading!
#Service
public class RefreshableDataSource implements DataSource {
private AtomicReference<DataSource> target = new AtomicReference<DataSource>();
#PostConstruct
public void refresh() {
target.set(createDsManuallyUsingSomeExternalConfigurationSource());
}
#Override
public Connection getConnection() throws SQLException {
return target.get().getConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return target.get().getConnection(username, password);
}
//Rest of DataSource methods
}
The createDsManuallyUsingSomeExternalConfigurationSource() method might look like this:
private DataSource createDsManuallyUsingSomeExternalConfigurationSource() {
DataSource ds = new org.apache.commons.dbcp.BasicDataSource();
ds.setDriverClassName("org.h2.Driver");
ds.setUrl(/*New database URL*/);
ds.setUsername(/*New username*/);
ds.setPassword(/*New password*/);
return ds;
}
This is a rough equivalent of Spring bean:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:mem:" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
You can't just inject such a target bean into your proxy/decorator RefreshableDataSource as you want data source configuration to be dynamic/refreshable, while Spring only allows you to inject static properties. This means that it is your responsibility to create an instance of target BasicDataSource, but as you can see, it is nothing scary.
Actually, I have a second thought: Spring SpEL AFAIK allows you to call other beans' methods from XML configuration. But this is a very wide topic.
JNDI approach
Another approach might be to use JNDI to fetch DataSource and use hot-deployment (it works with JBoss and its *-ds.xml files.
I have a web application which connects to an Oracle database. The application is now going to have a new set of users. A new db is being planned for this new set of users. Is it possible to connect to the appropriate db based on the user who logs in. As of now the database configuration is done through JNDIName entry in an xml file.
Absolutely. For a given DAO class (assuming you're using DAOs), create two bean definitions, one for each database, and then pick which DAO bean you want to use in your business logic:
<bean id="dao1" class="com.app.MyDaoClass">
<property name="dataSource" ref="dataSource1"/>
</bean>
<bean id="dao2" class="com.app.MyDaoClass">
<property name="dataSource" ref="dataSource2"/>
</bean>
Where dao1 and dao2 are the DataSource beans representing your two different databases.
At runtime, your business logic selects dao1 or dao2 appropriately.
I'd suggest injecting both the data sources into your DAOs and then within your DAO decide the correct data source to use based on the current user. The current user can be passed to the DAO from your presentation/service layer.