We Keep Coding

Multi connection pool configurations for different modules in application

Our application has multiple modules, each module use its own schema in the same mysql database. Now I need to make different connection pool configurations for each module because of their different db resource consuming nature, i.e. some module may have 20 active connections at a point of time, but others may just have 1 max. I have searched here and other forums, couldn't find a solution, just this topic is not about multi-tenancy or multi-database, all schemas are in the same db.
Here's the config we have:
<bean id="dataSource" class="our.own.package.RoutingDataSource"> <!-- RoutingDataSource extends spring AbstractDataSource -->
<property name="master" ref="masterDS"/>
</bean>
<bean id="abstractDataSource" abstract="true">
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="initialPoolSize" value="#initial.pool.size#" />
<property name="minPoolSize" value="#min.pool.size#" />
<property name="maxPoolSize" value="#max.pool.size#" /> <!-- I want to have different configs for each module in our application -->
</bean>
<bean id="masterDS" class="com.mchange.v2.c3p0.ComboPooledDataSource" parent="abstractDataSource">
<property name="jdbcUrl" value="jdbc:mysql://#host#/" />
<property name="user" value="#user#" />
<property name="password" value="#pwd#" />
<property name="dataSourceName" value="#dbName#" />
</bean>
So now I have two questions:
1) Is it possible to have different connection pool configurations for one datasource in Spring?
2) If I have to go with the multiple datasource way(one datasource for one module), is implementing Spring's AbstractRoutingDataSource the correct way to go?
Thank you!

Ad.1 Your data source is in the fact connection pool so you want to have multiple pools on the top of another pool. You can do it but you will face many other problems.
Ad.2. Yes definitely. You already have RoutingDataSource so this one should be implementation of AbstractRoutingDataSource. Probably you already have logic there to determine current the data source routing key which should be used to do the lookup.

DataSource in an OSGI container

I have a simple Spring App that connects to a DB via an EntityManager
So I have to following configuration:
<bean id="domainEntityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="TheManager" />
<property name="dataSource" ref="domainDataSource" />
<property name="packagesToScan" value="com.conztanz.persistence.stock.model" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">create-drop</prop>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQL5Dialect</prop>
</props>
</property>
</bean>
<bean id="domainDataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5433/dbName" />
<property name="username" value="xxxx" />
<property name="password" value="xxxx" />
</bean>
This works fine when lunched via a main class (loading the AppContext manually)
But, once deployed into ServiceMix I get the following error :
Property 'driverClassName' threw exception; nested exception is java.lang.IllegalStateException: Could not load JDBC driver class [org.postgresql.Driver]
I've read somewhere that OSGI and DriverManager don't mix well but I fail to understand why.
The solution that seems to be a good practice is to expose the dataSource as an OSGI bundle, do you agree ? and in that case how would you have access to it from a spring context to be able to have an EntityManager for example ?

DriverManager does not work well in OSGi. The easiest way is to use a DataSource directly. Most DB drivers have such a class. If you instantiate it in your app context then it will work. The downside is though that it binds your application to the DB driver as it then will import the packages for the DataSource impl.
A more loosely coupled way is to use ops4j pax jdbc. It allows to create a DataSource as an OSGi service from a config in config admin. So in your app context you just have to add a dependency to a DataSource service. So your application is not bound to the specific DB driver. One typical use case is to use H2 in tests and oracle in production.

How to handle Connection pool size established by EntityManger through a Spring Application

We have a web application implementing Spring MVC 3.2 using JPA as a framework for ORM. Now the problem is that EntityManager is creating a lot of open connections with the database. We want to handle it in such a way that for every query a connection should be established and closed after completion.
As per the spring implementation EntityManager is created once. But the problem here is we in some way want to handle the client connections that EntityManager is creating for querying the database.
Whenever the query is completed in the database, that connection goes into sleep, instead we want to close it once the query returns the result.
DB type: MySQL
My configuration for JPA is :
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="packagesToScan" value="com.reppify" />
<property name="jpaPropertyMap" ref="jpaPropertyMap" />
<property name="dataSource" ref="dataSourceLocal" />
<property name="persistenceUnitName" value="cron-jpa" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
</bean>
<bean id="dataSourceLocal"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${database.driver}" />
<property name="url" value="${database.url}" />
<property name="username" value="${database.user}" />
<property name="password" value="${database.password}" />
</bean>
We are using hibernate-jpa-api-2.0 jar as a dependency to the project.
And my JAVA Base DAO implementation for injecting EntityManager looks like:
protected EntityManager entityManager;
#PersistenceContext
public void setEntityManager(EntityManager entityManager) {
this.entityManager = entityManager;
}
Please suggest us an optimum solution for the same.

DriverManagerDataSource is not a connections pool, it creates a new connection on every call. This class is useful for testing but you shouldn't use it in production, choose a connection pool instead. There are many connection pools to choose from:
HikariCP
Apache Commons DBCP
c3p0
...

Spring #Transactional on one service method spanning over two Hibernate transaction managers

I was wondering if it is possible to use two transaction manager in one service methods.
Because due to the limitation of client's mysql db configuration, we have got 2 different datasources within one database, i.e., one user/pwd/url per schema. Thats why i have to configured two transaction managers. Now I got problem when it comes to the service implementation. See the following code:
public class DemoService{
...
#Transactional(value = "t1")
public doOne(){
doTwo();
}
#Transactional(value = "t2")
public doTwo(){
}
...
}
if I using this code pattern, i always got the exception
org.hibernate.HibernateException: No Session found for current thread
If i run the two methods seperately, it workd fine.
Did i miss something? Or there is other work around here?
Any advice would be appreciated.
btw: some of my configuration
<bean id="sessionFactorySso" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="mappingLocations">
<list>
<value>classpath*:sso.vo/*.hbm.xml</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.show_sql">true</prop>
<prop key="generateDdl">true</prop>
<prop key="hibernate.dialect">${dialect} </prop>
</props>
</property>
<property name="dataSource" ref="dataSourceSso"/>
</bean>
<bean id="dataSourceSso" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${driver}"/>
<property name="jdbcUrl" value="${sso.url}"/>
<property name="user" value="${sso.username}"/>
<property name="password" value="${sso.password}"/>
<!-- these are C3P0 properties -->
<property name="acquireIncrement" value="2" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="2" />
<property name="automaticTestTable" value="test_c3p0" />
<property name="idleConnectionTestPeriod" value="300" />
<property name="testConnectionOnCheckin" value="true" />
<property name="testConnectionOnCheckout" value="true" />
<property name="autoCommitOnClose" value="true" />
<property name="checkoutTimeout" value="1000" />
<property name="breakAfterAcquireFailure" value="false" />
<property name="maxIdleTime" value="0" />
</bean>
<bean id="transactionManagerSso" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactorySso"/>
<qualifier value="sso" />
</bean>
<tx:annotation-driven transaction-manager="transactionManagerSso" />

Because you want to enlist two data sources in one transaction you need XA(Global) Transaction.
Therefore you need to:
Set the Spring JTA transaction manager
You Hibernate properties should use the JTA platform settings
Your data source connections should be XA complaint
You need an application server JTA transaction manager or a stand-alone tarnsaction manager (Bitronix, Atomikos, JOTM)
You will need two session factory configurations, one for each individual data source.
And you won't have two transaction managers: t1 and t2, but instead you will enlist two transactional XA data sources that will be automatically enlisted in the same global transaction, meaning you will have two XA connections being enlisted in the same global transaction. The XA transaction will use the 2PC protocol to commit both resources upon commit time.
Checkout this Bitronix Hibernate example.

You have a few options:
Inject the bean into itself and use the reference to call doTwo(). This really goes against the whole idea of IoC and AOP so I don't recommend it.
Switch to compile time weaving. Rather than using proxies, Spring (actually the AspectJ compiler) will add the bytecode to start/stop transactions to your class at compile time. There are pros and cons to this approach. See this page for more details.
Use load time weaving. Same as #2 except that your classes are modified as they are loaded rather than at compile time. IMO, Java classloading is complicated enough. I'm sure this works great for some folks but I personally would avoid this.
As Vlad pointed out, you can use JTA and XA.
Start a new transaction against transaction manager 2 within doOne() before calling doTwo(). RTFM on programmatic transaction management.
Check out ChainedTransactionManager. It essentially aggregates multiple transaction managers and does a "best effort" with commit/rollback. This is NOT a true two-phase commit like Vlad's solution.
All of these except for Vlad's solution (#4) have the potential to leave the databases in an inconsistent state. You need to use JTA/XA/two-phase commit to ensure consistency in the event that one of the TX managers throws an exception at commit time.

Atomikos / Spring - Global Transaction over two DBs

I am using Spring and trying to setup a global transaction spanning over two MS SQL Server DBs. The app is running inside Tomcat 6.
I have these definitions:
<bean id="dataSource1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
....
</bean>
<bean id="sessionFactory1"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource1"/>
....
</bean>
<bean id="hibernateTransactionManager1"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref local="sessionFactory1"/>
</property>
</bean>
<bean id="dataSource2" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
....
</bean>
<bean id="sessionFactory2"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource2"/>
....
</bean>
<bean id="hibernateTransactionManager2"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref local="sessionFactory2"/>
</property>
</bean>
Then also, each DAO is linked either to sessionFactory1 or to sessionFactory2.
<bean name="stateHibernateDao" class="com.project.dao.StateHibernateDao">
<property name="sessionFactory" ref="sessionFactory1"/>
</bean>
Also, I recently added these two.
<bean id="atomikosTransactionManager" class="com.atomikos.icatch.jta.UserTransactionManager" init-method="init" destroy-method="close">
<property name="forceShutdown" value="false" />
<property name="transactionTimeout" value="300" />
</bean>
<bean id="atomikosUserTransaction" class="com.atomikos.icatch.jta.UserTransactionImp">
<property name="transactionTimeout" value="300" />
</bean>
I am trying to programmatically manage the global transaction
(this is some old legacy code and I don't want to change it too
much so I prefer keeping this managed programmatically).
So now I have this UserTransaction ut (injected from Spring), so I call ut.begin(), do some DB/DAO operations to the two DBs through the DAOs, then I call ut.commit().
The thing is that even before the ut.commit() call, I can see the data is already committed to the DBs?!
I don't think Atomikos is aware of my two DBs, their data sources, session factories, etc. I don't think it starts any transactions on them. Looks like they are not enlisted at all in the global transaction.
To me it seems that each DB/DAO operation goes to the SQL Server on its own, so SQL Server creates an implicit transaction for just that DAO/DB operation, applies the operation and commits the implicit the transaction.
But 1) and 2) are just guesses of mine.
My questions:
Do I need to start the two DB transactions myself (but OK, this is what I am currently doing and I am trying to get rid of; that's why I am trying to use Atomikos to start with)?
How I can configure all this correctly so that when I call ut.begin() it begins a global transaction to the two DBs and when I call ut.commit() it commits it?
I haven't played with JTA recently so seems to me I am missing something quite basic here. What is it?
Edit 1
<bean id="globalTransactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="userTransaction" ref="atomikosUserTransaction"/>
<property name="transactionManager" ref="atomikosTransactionManager" />
<property name="allowCustomIsolationLevels" value="true" />
<property name="transactionSynchronization" value="2" />
</bean>