I'm using Spring with DBCP and need to refresh my datasource when some configuration on operation environment changes, without restart all application.
If I do it with no use of DBCP, I force this refresh closing current opened datasource in use and Start a new instance of DataSource.
Using DBCP+Spring, I can't do that.
Somebody knows if it is possible?
I don't think there is such a support in plain DBCP, mostly because database connection properties are very rarely changing during the lifetime of the application. Also you will have to consider transition time, when some connections served by the old data source are still opened while others are already served from the new (refreshed) one.
Decorator/proxy approach
I would suggest you to write custom implementation of DataSource leveraging Decorator/Proxy design pattern. Your implementation would simply call target data source (created by DBCP), most of the time doing nothing more. But when you call some sort of refresh() method, your decorator will close previously created data source and create new one with fresh configuration. Remember about multi-threading!
#Service
public class RefreshableDataSource implements DataSource {
private AtomicReference<DataSource> target = new AtomicReference<DataSource>();
#PostConstruct
public void refresh() {
target.set(createDsManuallyUsingSomeExternalConfigurationSource());
}
#Override
public Connection getConnection() throws SQLException {
return target.get().getConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return target.get().getConnection(username, password);
}
//Rest of DataSource methods
}
The createDsManuallyUsingSomeExternalConfigurationSource() method might look like this:
private DataSource createDsManuallyUsingSomeExternalConfigurationSource() {
DataSource ds = new org.apache.commons.dbcp.BasicDataSource();
ds.setDriverClassName("org.h2.Driver");
ds.setUrl(/*New database URL*/);
ds.setUsername(/*New username*/);
ds.setPassword(/*New password*/);
return ds;
}
This is a rough equivalent of Spring bean:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:mem:" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
You can't just inject such a target bean into your proxy/decorator RefreshableDataSource as you want data source configuration to be dynamic/refreshable, while Spring only allows you to inject static properties. This means that it is your responsibility to create an instance of target BasicDataSource, but as you can see, it is nothing scary.
Actually, I have a second thought: Spring SpEL AFAIK allows you to call other beans' methods from XML configuration. But this is a very wide topic.
JNDI approach
Another approach might be to use JNDI to fetch DataSource and use hot-deployment (it works with JBoss and its *-ds.xml files.
Related
I am working in a project using Spring, Spring Data JPA, Spring Security, Primefaces...
I was following this tutorial about dynamic datasource routing with spring.
In this tutorial, you can only achieve dynamic datasource switching between a pre-defined datasources.
Here is a snippet of my code :
springContext-jpa.xml
<bean id="dsCgWeb1" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName.Cargest_web}"></property>
<property name="url" value="${jdbc.url.Cargest_web}"></property>
<property name="username" value="${jdbc.username.Cargest_web}"></property>
<property name="password" value="${jdbc.password.Cargest_web}"></property>
</bean>
<bean id="dsCgWeb2" class="org.apache.commons.dbcp.BasicDataSource">
// same properties, different values ..
</bean>
<!-- Generic Datasource [Default : dsCargestWeb1] -->
<bean id="dsCgWeb" class="com.cargest.custom.CargestRoutingDataSource">
<property name="targetDataSources">
<map>
<entry key="1" value-ref="dsCgWeb1" />
<entry key="2" value-ref="dsCgWeb2" />
</map>
</property>
<property name="defaultTargetDataSource" ref="dsCgWeb1" />
</bean>
What i want to do is to make the targetDataSources map dynamic same as its elements too.
In other words, i want to fetch a certain database table, use properties stored in that table to create my datasources then put them in a map like targetDataSources.
Is there a way to do this ?
Nothing in AbstractRoutingDataSource forces you to use a static map of DataSourceS. It is up to you to contruct a bean implementing Map<Object, Object>, where key is what you use to select the DataSource, and value is a DataSource or (by default) a String referencing a JNDI defined data source. You can even modify it dynamically since, as the map is stored in memory, AbstractRoutingDataSource does no caching.
I have no full example code. But here is what I can imagine. In a web application, you have one database per client, all with same structure - ok, it would be a strange design, say it is just for the example. At login time, the application creates the datasource for the client and stores it in a map indexed by sessionId - The map is a bean in root context named dataSources
#Autowired
#Qualifier("dataSources");
Map<String, DataSource> sources;
// I assume url, user and password have been found from connected user
// I use DriverManagerDataSource for the example because it is simple to setup
DataSource dataSource = new DriverManagerDataSource(url, user, password);
sources.put(request.getSession.getId(), dataSource);
You also need a session listener to cleanup dataSources in its destroy method
#Autowired
#Qualifier("dataSources");
Map<String, DataSource> sources;
public void sessionDestroyed(HttpSessionEvent se) {
// eventually cleanup the DataSource if appropriate (nothing to do for DriverManagerDataSource ...)
sources.remove(se.getSession.getId());
}
The routing datasource could be like :
public class SessionRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
HttpServletRequest request = ((ServletRequestAttributes)
RequestContextHolder.getRequestAttributes()).getRequest();
return request.getSession().getId();
}
#Autowired
#Qualifier("dataSources")
public void setDataSources(Map<String, DataSource> dataSources) {
setTargetDataSources(dataSources);
}
I have not tested anything because it would be a lot of work to setting the different database, but I thing that it should be Ok. In real world there would not be a different data source per session but one per user with a count of session per user but as I said it is an over simplified example.
The datasource used by a thread might change from time to time.
Should pay attention to concurrency, applications might get concurrency issues in concurrent environment.
thread-bound AbstractRoutingDataSource sample
It can be achieved with AbstractRoutingDataSource and keeping the information in the thread-local Variable. Here is a beautiful working example you can refer to:
Multi-tenancy: Managing multiple datasources with Spring Data JPA
I have a spring based web application and in my application context xml file, I have defined a bean which has all the parameters to connect to database. As part of this bean, for one of the parameters, I have a password key, as shown in the below example and I wanted the value should come from a /vault/password file. This /vault/password is not part of the project/application. This /vault/password will be there in host machine by default.
What is the syntax in applicationContext.xml bean definition, to read a value from a file outside of application context.
<bean class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close" id="dataSource">
<property name="url" value="jdbc:postgresql://postgres:5432/" />
<property name="username" value="postgres" />
<property name="password" value="/vault/password" />
</bean>
Something like this is probably your best bet:
How to correctly override BasicDataSource for Spring and Hibernate
PROBLEM:
Now I need to provide custom data source based on server environment
(not config), for which I need to calculate driverClassName and url
fields based on some condition.
SOLUTION:
Create a factory (since you need to customize only the creation phase
of the object, you don't need to control the whole lifetime of it).
public class MyDataSourceFactory {
public DataSource createDataSource() {
BasicDataSource target = new BasicDataSource();
if (condition) {
target.setDriverClassName("com.mysql.jdbc.Driver");
target.setUrl("jdbc:mysql://localhost/test?relaxAutoCommit=true");
} else { ... }
return target;
}
}
In your case, your customization would do some I/O to set target.password.
I have a web application running under Tomcat 7 using Spring with c3po as the connection pool manager. I have also used dbcp and have the same result.
I initiate a long running single threaded process which makes a large number of database calls using jdbcTemplate.update(), etc, in various dao's. As each of these updates is simple and independent, no transaction manager is being used.
For some reason, I am running out of connections. What appears to be happening is that each dao is holding onto its own connection and not returning it to the pool.
Is this normal behaviour? I had expected that the connection was tied to the jdbcTemplate.update() and released back as soon as this had finished.
...
In the context file...
<bean id="enquiryDataSource" destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${enquiry.drivername}"/>
<property name="url" value="${enquiry.jdbc}"/>
<property name="username" value="${enquiry.username}"/>
<property name="password" value="${enquiry.password}"/>
<property name="maxWait" value="30000"/>
<property name="maxActive" value="50"/>
</bean>
In a typical dao constructor...
#Autowired
public XXXCountryDao(#Qualifier("enquiryDataSource") DataSource dataSource,
#Qualifier("sqlUpdaterFactoryImpl") SqlUpdaterFactory sqlUpdaterFactory, #Qualifier("sqlFormatterFactoryImpl") SqlFormatterFactory sqlFormatterFactory) {
super("Country", dataSource, sqlUpdaterFactory, sqlFormatterFactory);
// ...other constructor stuff
}
All dao's inherit from...
public abstract class AbstractFileProcessorDao<ImportRecType, QueryRecType> extends JdbcDaoSupport {
// ...
}
In a typical dao method...
protected boolean runUpdateToSqlDatabase(Map<String, Object> values, Map<String, Object> whereValues) {
if (values.isEmpty())
return true;
String sql = updateUpdaterServer.getSql(values, whereValues);
if (logger.isDebugEnabled())
logger.debug("Server SQL -> " + sql);
getJdbcTemplate().update(sql);
return false;
}
Please check your application for "rogue" calls to DataSource#getConnection (you can use your IDE to search for method references). Connection leaks are usually caused by obtaining a connection which is then never closed via Connection#close.
When working with Spring's JdbcTemplate all JDBC resource handling (opening / closing connections, statements, result sets) is done automatically. But with legacy code you never know.
I have an application which uses four databases in different geographical locations. All the databases contains same tables and only the database name is different according to the location.
I have to create some reports in my application which uses data from each database. What would be the proper way to create those database connection from a java application and is there a suitable design pattern for this task which I could use?
As you have not tagged your question with any of this, hibernate, JPA, ORM, I assume you are dealing with plain JDBC.
Having said that, I suggest you to have a DAO layer to deal with underlying databases, and leave the connection details to specific implementations. You can configure your connection strings in some .properties files, lets say.
[Complement]
You can also make use of DAO factory, an implementation of Abstract Factory or Factory Method pattern, whichever suits here.
[Links]
A very fine implementation of DAO and DAO Factory, by BalusC
Core J2EE Patterns -- arguably dated but might provide some idea.
There are multiple ways you can achieve this:
If you are using any Java EE container which supports distributed transaction then you can use there functionality.
If you are with plain JDBC then you will have to maintain your own connection for every database.
For JDBC:
Provide all connection details
Have an Facade which gives you desired object by calling a abstract generic DAO.
Have a factory which creates dao based on connection.
Use ORM tools like Hibernate, where you can use configuration for multiple database. Tutorial.
If you are using Spring, then you can configure one datasource per database. Docs
Design Patterns:
Facade Pattern - for hiding the complexity and multiple database usage.
Factory - In case you manage the database connection yourself.
Singleton - For datasources
You can handle multiple connections easily using a ORM tool like Hibernate.. You can specify each connection in a separate configuration file and instantiate the required connection by getting a new session factory each time.
Other way would be to use datasource and JNDI : Java connecting to multiple databases
I think you can use a combination of Factory pattern and Singleton pattern for the purpose.
The Ideal way to achieve this is by using a multi-dimensional system like OLAP. But see if you can create a view out of those databases. Then you just need to query the view (i.e. just a single database connection). Also you can still use multiple database connections if you want.
is very easy :)
1.Create a Data Source to try connection to DB
public DataSource getDataSource(String db) throws Exception {
DataSource dt = null;
InitialContext ic = null;
try {
if(db.trim().equals("you_database_name")) {
dt = (DataSource)ic.lookup("jdbc/connection_name");
} else if(db.trim().equals("you_database_name")) {
dt = (DataSource) ic.lookup("jdbc/connection_name");
}
return dt;
} catch(NamingException n) {
throw new Exception("Err getDataSource (ServiceLocator) NamingException - " + n.getMessage());
}
2.Create a class DataBase, remember close all connection in this point.
public class DataBases {
public YouNameDataSourceClass dataSrc;
public DataBases() throws Exception {
super();
dataSrc = new YouNameDataSourceClass.getDataSource();
}
public Connection getConnectionAS400() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
public Connection getConnectionOracle() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
public Connection getConnectionSQLServer() throws Exception {
return locator.getDataSource("you_database_name").getConnection();
}
}
Good look.
Assuming you are using Spring MVC with Hibernate with XML configurations, follow these steps:
Create beans of all the databases in your spring-servlet file.
<bean id="dataSource1" Class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
<property name="url" value="jdbc:sqlserver://localhost:1433;databaseName=database1"/>
<property name="username" value="abc" />
<property name="password" value="abc#123" />
</bean>
Create sessionFactory beans of all the databases you want in the Spring-servlet file.
<bean id="datasource1SessionFactory" class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
<property name="dataSource"
ref="database1"/>
<property name="packagesToScan"
value="com.id4.iprod.entity"/>
<property name="hibernateProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">
</prop>
<prop key="hibernate.dialect">
org.hibernate.dialect.SQLServer2012Dialect
</prop>
</props>
</property>
</bean>
Now you just need to open session of the database you want to in DAO and access the desired results from desired database.
Session datasource1= this.datasource1SessionFactory.openSession();
I have a Jersey resource that access the database. Basically it opens a database connection in the initialization of the resource. Performs queries on the resource's methods.
I have observed that when I do not use #Singleton, the database is being open at each request. And we know opening a connection is really expensive right?
So my question is, should I specify that the resource be singleton or is it really better to keep it at per request especially when the resource is connecting to the database?
My resource code looks like this:
//Use #Singleton here or not?
#Path(/myservice/)
public class MyResource {
private ResponseGenerator responser;
private Log logger = LogFactory.getLog(MyResource.class);
public MyResource() {
responser = new ResponseGenerator();
}
#GET
#Path("/clients")
public String getClients() {
logger.info("GETTING LIST OF CLIENTS");
return responser.returnClients();
}
...
// some more methods
...
}
And I connect to the database using a code similar to this:
public class ResponseGenerator {
private Connection conn;
private PreparedStatement prepStmt;
private ResultSet rs;
public ResponseGenerator(){
Class.forName("org.h2.Driver");
conn = DriverManager.getConnection("jdbc:h2:testdb");
}
public String returnClients(){
String result;
try{
prepStmt = conn.prepareStatement("SELECT * FROM hosts");
rs = prepStmt.executeQuery();
...
//do some processing here
...
} catch (SQLException se){
logger.warn("Some message");
} finally {
rs.close();
prepStmt.close();
// should I also close the connection here (in every method) if I stick to per request
// and add getting of connection at the start of every method
// conn.close();
}
return result
}
...
// some more methods
...
}
Some comments on best practices for the code will also be helpful.
Rather than thinking about making the resource a singleton, focus more on managing backend, service type objects like your ResponseGenerator class as singletons, which obviously shouldn't be instantiated every request.
Making the resource a singleton as well is one way of managing ResponseGenerator as a singleton, but it's not the only or necessarily the best way, see Access external objects in Jersey Resource class and How to wire in a collaborator into a Jersey resource? for ways to inject this into non-singleton resources.
Note that your ResponseGenerator class would need work before it would function as a singleton, whether injected into a per-request resource or instantiated in a singleton resource. It's not thread safe, and you would open a single connection on startup and reuse it across requests, which won't work, you should use a connection pool to do the heavy lifting of efficiently + safely reusing connections across requests.
Some comments on best practices for the code will also be helpful.
You'll get better responses on http://codereview.stackexchange.com,
but:
ResponseGenerator is a poor name for a class (just about everything in a web application is a response generator).
don't use String as the return type of your service and object, use proper typed objects (eg it sounds like you're returning a java.util.List of something).
Don't swallow your SQLException, bubble it up to allow Jersey to generate a 5xx series response code in your resource.
Use final member variables.
Your log object should be static.
You best option is to use a framework like Spring with Jersey which I outlined in a similar post. The only difference is that instead of injecting a service bean you would inject a pooled DataSource and this can easily be configured using c3p0.
Example applicationContext.xml, notice the "scope" is set to prototype which is equivalent to a singleton in Spring parlance.
<bean id="pooledDataSource" scope="prototype" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="jdbcUrl" value="${jpa.url}" />
<property name="user" value="${jpa.username}" />
<property name="password" value="${jpa.password}" />
<property name="initialPoolSize" value="1" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="3" />
<property name="idleConnectionTestPeriod" value="500" />
<property name="acquireIncrement" value="1" />
<property name="maxStatements" value="50" />
<property name="numHelperThreads" value="1" />
</bean>
In your MyResource.java you would simply add the following and Spring would inject it appropriately.
private DataSource pooledDataSource;
public void setPooledDataSource(DataSource pooledDataSource) {
this.pooledDataSource = pooledDataSource;
}
Then you could change your ResponseGenerator to accept the DataSource and use this to query the database.