We Keep Coding

How-to configure Spring data source to avoid participation in transaction synchronization?

I'm observing seemingly weird behavior related to how Spring handles operations with multiple data sources, especially participation in transaction synchronization.
My Spring context has two data sources configured:
1) dataSource1 - MySQL database
2) dataSource2 - Apache Impala database
Both data sources have a jdbc template configured for them. Additionally dataSource1 has a transaction manager configured. dataSource2 has no transaction manager configured (Apache Impala does not have transactions by design, it just happens to expose SQL-like querying functionality via JDBC connector).
I'm using this configuration in a spring-batch application - dataSource1 and its transactionManager is set to be used to store data related to spring-batch job meta-info.
Next I have a spring-batch job configured with a single custom Tasklet step. In this step I'm accessing dataSource2 via its jdbc template. And this is where the problem occurs - to my surprise connection to dataSource2 starts to participate in a transaction synchronization.
2018-06-06 10:41:08,179 DEBUG [main] org.springframework.jdbc.datasource.DataSourceUtils - Registering transaction synchronization for JDBC Connection
I understand that upon a start of a spring-batch step a transaction related to dataSource1 is started however why should dataSource2 be anyhow involved in that?
A negative side-effect is that once connection to dataSource2 starts to participate in transaction synchronization the connection is not closed after e.g. jdbcTemplate.execute(...) command completes. Essentially what I'm observing is that open connections to dataSource2 are closed only when the outer transaction completes.
Is there a way to configure Spring context and dataSource2 to not participate in a transaction synchronization?
Configuration
<bean id="dataSource1" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">
<constructor-arg ref="hikariConfig" />
</bean>
<bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig">
<property name="poolName" value="springHikariCP" />
<property name="dataSourceClassName" value="com.mysql.jdbc.jdbc2.optional.MysqlDataSource" />
...
</bean>
<bean id="dataSource1JdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource1"/>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager" p:entityManagerFactory-ref="entityManagerFactory"/>
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" p:dataSource-ref="dataSource1">
<property name="packagesToScan" value="..."/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="sharedCacheMode" value="DISABLE_SELECTIVE"/>
</bean>
<bean id="dataSource2" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="org.apache.hive.jdbc.HiveDriver"/>
...
</bean>
<bean id="dataSource2JdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource2"/>
</bean>

Spring #Transactional on one service method spanning over two Hibernate transaction managers

I was wondering if it is possible to use two transaction manager in one service methods.
Because due to the limitation of client's mysql db configuration, we have got 2 different datasources within one database, i.e., one user/pwd/url per schema. Thats why i have to configured two transaction managers. Now I got problem when it comes to the service implementation. See the following code:
public class DemoService{
...
#Transactional(value = "t1")
public doOne(){
doTwo();
}
#Transactional(value = "t2")
public doTwo(){
}
...
}
if I using this code pattern, i always got the exception
org.hibernate.HibernateException: No Session found for current thread
If i run the two methods seperately, it workd fine.
Did i miss something? Or there is other work around here?
Any advice would be appreciated.
btw: some of my configuration
<bean id="sessionFactorySso" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="mappingLocations">
<list>
<value>classpath*:sso.vo/*.hbm.xml</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.show_sql">true</prop>
<prop key="generateDdl">true</prop>
<prop key="hibernate.dialect">${dialect} </prop>
</props>
</property>
<property name="dataSource" ref="dataSourceSso"/>
</bean>
<bean id="dataSourceSso" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${driver}"/>
<property name="jdbcUrl" value="${sso.url}"/>
<property name="user" value="${sso.username}"/>
<property name="password" value="${sso.password}"/>
<!-- these are C3P0 properties -->
<property name="acquireIncrement" value="2" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="2" />
<property name="automaticTestTable" value="test_c3p0" />
<property name="idleConnectionTestPeriod" value="300" />
<property name="testConnectionOnCheckin" value="true" />
<property name="testConnectionOnCheckout" value="true" />
<property name="autoCommitOnClose" value="true" />
<property name="checkoutTimeout" value="1000" />
<property name="breakAfterAcquireFailure" value="false" />
<property name="maxIdleTime" value="0" />
</bean>
<bean id="transactionManagerSso" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactorySso"/>
<qualifier value="sso" />
</bean>
<tx:annotation-driven transaction-manager="transactionManagerSso" />

Because you want to enlist two data sources in one transaction you need XA(Global) Transaction.
Therefore you need to:
Set the Spring JTA transaction manager
You Hibernate properties should use the JTA platform settings
Your data source connections should be XA complaint
You need an application server JTA transaction manager or a stand-alone tarnsaction manager (Bitronix, Atomikos, JOTM)
You will need two session factory configurations, one for each individual data source.
And you won't have two transaction managers: t1 and t2, but instead you will enlist two transactional XA data sources that will be automatically enlisted in the same global transaction, meaning you will have two XA connections being enlisted in the same global transaction. The XA transaction will use the 2PC protocol to commit both resources upon commit time.
Checkout this Bitronix Hibernate example.

You have a few options:
Inject the bean into itself and use the reference to call doTwo(). This really goes against the whole idea of IoC and AOP so I don't recommend it.
Switch to compile time weaving. Rather than using proxies, Spring (actually the AspectJ compiler) will add the bytecode to start/stop transactions to your class at compile time. There are pros and cons to this approach. See this page for more details.
Use load time weaving. Same as #2 except that your classes are modified as they are loaded rather than at compile time. IMO, Java classloading is complicated enough. I'm sure this works great for some folks but I personally would avoid this.
As Vlad pointed out, you can use JTA and XA.
Start a new transaction against transaction manager 2 within doOne() before calling doTwo(). RTFM on programmatic transaction management.
Check out ChainedTransactionManager. It essentially aggregates multiple transaction managers and does a "best effort" with commit/rollback. This is NOT a true two-phase commit like Vlad's solution.
All of these except for Vlad's solution (#4) have the potential to leave the databases in an inconsistent state. You need to use JTA/XA/two-phase commit to ensure consistency in the event that one of the TX managers throws an exception at commit time.

Spring Data + JPA with multiple datasources but only one set of Repositories

I've been researching this a bunch today and I'm starting to think that what I want to do may not be possible, so I am turning to you, o mighty Stackoverflow, for help.
I'm building a RESTful services platform in Java, with Spring Data 3.1.2 + JPA as my persistence layer (as documented here). My data model objects are all implemented as interfaces that extend the Spring JpaRepository interface. I've got everything wired up and working nicely with a single datasource, as shown in this example (note that the datasource shown is Derby, but that's just for development purposes; in production, we'll be using Oracle):
<jpa:repositories base-package="com.my.cool.package.repository"/>
<bean id="emf" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
</property>
<property name="packagesToScan" value="com.my.cool.package" />
<property name="jpaVendorAdapter" ref="jpaVendorAdapter" />
</bean>
<bean name="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="emf" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.apache.derby.jdbc.EmbeddedDriver" />
<property name="url" value="jdbc:derby:derbyDB" />
<property name="username" value="dev" />
<property name="password" value="notARealPassword" />
</bean>
<bean id="jpaVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="org.hibernate.dialect.DerbyTenSevenDialect" />
</bean>
The problem is that this application will need to connect to several (Oracle) databases. The credentials included with each incoming request will contain a field that tells the application which database to go to in order to fulfill that request. The schemas for each database are the same, so there's no need for separate repository interfaces for each database.
After a fair amount of Googling, it's clear that this is a common scenario. To wit:
multiple databases with Spring Data JPA
Spring + Hibernate + JPA + multiple databases
how to setup spring data jpa with multiple datasources
And here's a blog post by a (former?) Spring developer, which isn't actually relevant to the topic at hand, but someone brings it up in the comments, and the author responds with some info:
http://blog.springsource.org/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/#comment-198835
The theme that seems to be emerging is that the way to solve this problem is to define multiple EntityManagerFactories, and to wire each one to the appropriate repositories like so:
<jpa:repositories base-package="com.my.cool.package.repository1" entity-manager-factory-ref="firstEntityManagerFactory" />
<jpa:repositories base-package="com.my.cool.package.repository2" entity-manager-factory-ref="secondEntityManagerFactory" />
However, as I've mentioned, I want to reuse my repository across all of the datasources, so this approach doesn't seem like it would work.
I know that there's no way around having logic in my code that takes the relevant piece of information from the request, and uses it to determine which datasource (or EntityManagerFactory) to use. The part I'm struggling with is how to get a handle to that datasource/EntityManagerFactory and "inject" it into my repository objects. Any ideas?

If you're really using the different DataSourcees in a multi-tenant kind of way (essentially assigning a request to a DataSource and sticking with it for the entire request) you should have a look at AbstractRoutingDataSource. It essentially provides a way to keep a Map of DataSourcees as well as a callback method to return a key to be used to lookup the DataSource to be used eventually. The implementation of this method usually looks up some thread bound key and return that (or even maps that onto a DataSource map key in turn). You just have to make sure some web component binds that key to the thread in the first place.
If you have that in place your Spring configuration just sets up a bean for your sub-class of AbstractRoutingDataSource and pipes the map of DataSources into it. Your Spring Data JPA setup stays the default way. The EntityManagerFactoryBean refers to the AbstractRoutingDataSource and you have a single <jpa:repositories /> element only.

How to use same connection for two queries in Spring?

I have the following code in a Spring JdbcTemplate based dao -
getJdbcTemplate().update("Record Insert Query...");
int recordId = getJdbcTemplate().queryForInt("SELECT last_insert_id()");
The problem is that my sometimes my update and queryForInt queries get executed using different connections from the connection pool.
This results in an incorrect recordId being returned since MySql last_insert_id() is supposed to be called from the same connection that issued insert query.
I have considered the SingleConnectionDataSource but do not want to use it since it degrades the application performance. I only want single connection for these two queries. Not for all the requests for all the services.
So I have two questions:
Can I manage the connection used by the template class?
Does JdbcTemplate perform automatic transaction management? If i manually apply a transaction to my Dao method does that mean two transactions will be created per query?
Hoping that you guys can shed some light on the topic.
Update - I tried nwinkler's approach and wrapped my service layer in a transaction. I was surprised to see the same bug pop up again after sometime. Digging into the Spring source code i found this -
public <T> T execute(PreparedStatementCreator psc, PreparedStatementCallback<T> action)
throws DataAccessException {
//Lots of code
Connection con = DataSourceUtils.getConnection(getDataSource());
//Lots of code
}
So contrary to what I thought, there isn't necessarily one database connection per transaction, but one connection for each query executed.
Which brings me back to my problem. I want to execute two queries from the same connection. :-(
Update -
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${db.driver}" />
<property name="url" value="${db.jdbc.url}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.password}" />
<property name="maxActive" value="${db.max.active}" />
<property name="initialSize" value="20" />
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate"
autowire="byName">
<property name="dataSource">
<ref local="dataSource" />
</property>
</bean>
<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<tx:advice id="transactionAdvice" transaction-manager="transactionManager">
<tx:attributes>
<tx:method name="*" propagation="REQUIRES_NEW" rollback-for="java.lang.Exception" timeout="30" />
</tx:attributes>
</tx:advice>
<aop:config>
<aop:pointcut id="pointcut" expression="execution(* service.*.*(..))" />
<aop:pointcut id="pointcut2" expression="execution(* *.ws.*.*(..))" />
<aop:advisor pointcut-ref="pointcut" advice-ref="transactionAdvice" />
<aop:advisor pointcut-ref="pointcut2" advice-ref="transactionAdvice" />
</aop:config>

Make sure your DAO is wrapped in a transaction (e.g. by using Spring's Interceptors for Transactions). The same connection will then be used for both calls.
Even better would be to have the transactions one level higher, at the service layer.
Documentation: http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/transaction.html
Update:
If you take a look at the JavaDoc of the DataSourceUtils.getConnection() method that you referenced in your update, you will see that it obtains the connection associated with the current thread:
Is aware of a corresponding Connection bound to the current thread, for example
when using {#link DataSourceTransactionManager}. Will bind a Connection to the
thread if transaction synchronization is active, e.g. when running within a
{#link org.springframework.transaction.jta.JtaTransactionManager JTA} transaction).
According to this, it should work like you have set it up. I have used this pattern plenty of times, and never ran into any issues like you described...
Please also take a look at this thread, someone was dealing with similar issues there: Spring Jdbc declarative transactions created but not doing anything

This is my approach to do this:
namedJdbcTemplate.execute(savedQuery, map, new PreparedStatementCallback<Object>() {
#Override
public Object doInPreparedStatement(PreparedStatement paramPreparedStatement)
throws SQLException, DataAccessException {
paramPreparedStatement.execute("SET #userLogin = 'blabla123'");
paramPreparedStatement.executeUpdate();
return null;
}
});

How to initialize in-memory HSQLDB using script via Spring

I am attempting to do unit testing of my DAO (using Spring and Hibernate). I am using HSQLDB per this tutorial. The tutorial states that the in-memory HSQLDB database can be initialized using a SQL script but I cannot find information on how to do so in Spring. Here is the pertinent Spring context config:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:mem:mydb" />
<property name="username" value="sa" />
<property name="password" value="" />
<property name="initialSize" value="5" />
<property name="maxActive" value="10" />
<property name="poolPreparedStatements" value="true" />
<property name="maxOpenPreparedStatements" value="10" />
</bean>
Any help would be appreciated. Thanks.

If you are trying to work with in-memory databases and Spring, there is a new jdbc namespace for Spring 3 that makes working with embedded databases very easy.
The best part is that it acts as a DataSource, so it can easily be dropped in to replace your existing dataSource bean.
<jdbc:embedded-database id="dataSource" type="HSQL">
<jdbc:script location="classpath:schema.sql"/>
<jdbc:script location="classpath:test-data.sql"/>
</jdbc:embedded-database>
If you are more interested in doing this with Java Config, take a look at the EmbeddedDatabaseBuilder (new in Spring 3.0).
#Configuration
public class DatabaseTestConfig {
#Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.HSQL)
.addScript("classpath:schema.sql")
.addScript("classpath:test-data.sql")
.build();
}
}

Nicholas answer is perfectly fine, but you can use jdbc namespace to initialize external database as well:
<jee:jndi-lookup id="dataSource" jndi-name="java:comp/env/jdbc/DS"/>
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:/META-INF/database/init.sql"/>
</jdbc:initialize-database>

In the tutorial you link to, one of the ways of setting things up is this (after obvious correction):
In-memory from a script: jdbc:hsqldb:file:path-to-file
I think that that would appear to be relevant. I suggest replacing path-to-file with something that looks like a fully-qualified filename…

You could get around this by creating a subclass of BasicDataSource with getters/setters for two new properties, initExecuteSqlFile and destroyExecuteSqlFile, that can have a comma-seperated list of SQL files to execute. The subclass would have init() and destroy() methods that handle the init/destroy SQL files.
Then use the following bean definition:
<bean
id="datasource"
class="com.example.MyBasicDataSource"
destroy-method="destroy"
init-method="init"
>
<property name="destroyExecuteSqlFile">
<value>h2-destroy-01.sql</value>
</property>
<property name="initExecuteSqlFile">
<value>h2-init-01.sql,h2-init-02.sql,h2-init-03.sql</value>
</property>
<!-- Other properties -->
</bean>

With embedded-database we would only be able to connect to the database from the same JVM. If we have two JVMs, for performance or other constraints, we can:
Instead of using an embedded-database, you can use the datasource suggested in this answer.
Then initialize like Poitrek De suggested (and suggested in previous answer too). You may want to create tables only if they do not exist (as suggested here).