We Keep Coding

Issue with CachingConnectionFactory and DefaultMessageListenerContainer

It is mentioned in the documentation of DefaultMessageListenerContainer class that it is not recommended to use CachingConnectionFactory with dynamic scaling. While searching, I have encountered following link:
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
Here found a comment from Gary Russell that
the problem is with caching consumers when using variable concurrency in the container; we can end up with a live consumer "stuck" in the cache".
We have used DefaultMessageListenerContainer and CachingConnectionFactory together so this is surely a problem from above link.
We are encountering problems with our application having following behaviour:
TCP ZeroWindow network congestion
TCP RESET from application server to MQ
DB connection grows during the issue while different transactions halt
Messages in certain queues gets built up
We have following code configuration :
In ibmmq-context.xml file:
<!-- WebSphere MQ Connection Factory -->
<bean id="appMqConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="hostName">
<value>${ibmmq.ip}</value>
</property>
<property name="port">
<value>${ibmmq.port}</value>
</property>
<property name="queueManager">
<value>${ibmmq.queuemanager}</value>
</property>
<property name="channel">
<value>${ibmmq.channel}</value>
</property>
<property name="clientReconnectOptions">
<util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CLIENT_RECONNECT"/>
</property>
<property name="transportType" ref="appTransport"/>
</bean>
<!-- A cached connection -->
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="appMqConnectionFactory"/>
<property name="sessionCacheSize" value="${jms.session.cachesize}"/>
</bean>
<!-- Use native MQ classes. -->
<bean id="appTransport" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean">
<property name="staticField">
<value>com.ibm.mq.jms.JMSC.MQJMS_TP_CLIENT_MQ_TCPIP</value>
</property>
</bean>
In jms-context file:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
There are 6 such listeners like bankListener and each of the listeners has concurrency value, varies from 10-40
<bean id="listenerExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="maxPoolSize" value="140"/>
<property name="corePoolSize" value="100"/>
<property name="queueCapacity" value="30"/>
<property name="threadNamePrefix" value="jms-listener-task-"/>
<property name="threadGroupName" value="jms-listener-tasks"/>
</bean>
and jms-context.xml file uses ibmmq-context.xml file.
And to note, we have used IBM MQ 7.1, Spring 4.2.8, spring-integration-core as 4.3.1.RELEASE and JBoss EAP 6.4.10
We are planning to fix this by following way:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="appMqConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
My request:
Please review the configuration and let me know is there anything else to be changed.
Could you please also explain our application behaviour(above 4 points - a to d) with our current configuration with CachingConnectionFactory and DefaultMessageListenerContainer
Thanks in advance for your help.

Try setting:
cachingConnectionFactory.setCacheConsumers(false);
Or in Spring it should be:
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
...
<property name="cacheConsumers" value="false"/>
...
</bean>

Reading Dual/Multiple Properties in Spring

I'm trying to read the application specific properties from database, and I'm trying to keep the database credentials in properties file.
So I need to load both properties (File and DB), while the application is
loading.
I'm having my configuration as follows.
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="WEB-INF/config/db.properties"/>
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
<property name="properties">
<bean class="org.apache.commons.configuration.ConfigurationConverter" factory-method="getProperties">
<constructor-arg>
<bean class="org.apache.commons.configuration.DatabaseConfiguration">
<constructor-arg type="javax.sql.DataSource" ref="postgresqlDataSource"/>
<constructor-arg value="TBL_APP_SETTINGS"/>
<constructor-arg value="PROP_KEY"/>
<constructor-arg value="PROP_VALUE"/>
</bean>
</constructor-arg>
</bean>
</property>
</bean>
<bean id="postgresqlDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="url" value="${db.url}"/>
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="username" value="${db.user}"/>
<property name="password" value="${db.pass}"/>
<property name="initialSize" value="1"/>
<property name="maxActive" value="5"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="timeBetweenEvictionRunsMillis" value="5000"/>
<property name="removeAbandonedTimeout" value="60"/>
<property name="minEvictableIdleTimeMillis" value="30000"/>
<property name="minIdle" value="3"/>
</bean>
It's throwing error for ${db.url}
please help how to do this? thanks in advance.

The easiest way is to externalize some of your config properties the guide is on spring.io site. You can pass properies as java system properties or command line arguments
The second way is to write your own PropertyPlaceholderConfigurer which will read properties from application.yml and from the database

I believe the easiest way you can do it is writing credentials into the spring xml and import it to your main xml and reference them as beans:
<bean id="dbUrl" class="java.lang.String">
<constructor-arg type="java.lang.String" value="jdbc://..."/>
</bean>
... same for dbUser and dbPassword
And then refer to the value using ref.

Advantage of org.springframework.jdbc.datasource.DriverManagerDataSource over oracle.jdbc.pool.OracleDataSource

I currently use this configuration for my projects:
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource"
destroy-method="close">
<property name="URL" ...
<property name="user" ...
<property name="password" ...
<property name="connectionCachingEnabled" value="true" />
And it works fine, pretty fast.
I happened to see, on an old project (spring 2.5) this configuration:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName">
<value>oracle.jdbc.driver.OracleDriver</value>
</property>
<property name="url"...
<property name="username" ...
<property name="password" ...
</bean>
From documentation it would seem that this last option does not make use of a connection pool. I see no reason to use this configuration over mine, but it still exists so I am curious: where's the advantage/limitation?

1st configuration is oracle specific, whereas 2nd configuration is generic. You can explicitly define driver class.
This is the only major difference I can see in them other than connection pool support of OracleDataSource.
You can use it for generic behavior as mentioned below:
<bean id="baseDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource"
abstract="true">
<property name="username" value="user"/>
<property name="password" value="pwd" />
</bean>
<bean id="mySqlDataSource" parent="baseDataSource">
<property name="driverClassName" value="${mySQL.driver}" />
<property name="url" value="${mySQL.url}"/>
</bean>
<bean id="oracleDataSource" parent="baseDataSource">
<property name="driverClassName" value="${oracle.driver}" />
<property name="url" value="${oracle.url}"/>
</bean>
Property values you can externalized.
You can explore Apache Jakarta Commons DBCP which has all the features of DriverManagerDataSource
along with connection pool feature.

Atomikos / Spring - Global Transaction over two DBs

I am using Spring and trying to setup a global transaction spanning over two MS SQL Server DBs. The app is running inside Tomcat 6.
I have these definitions:
<bean id="dataSource1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
....
</bean>
<bean id="sessionFactory1"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource1"/>
....
</bean>
<bean id="hibernateTransactionManager1"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref local="sessionFactory1"/>
</property>
</bean>
<bean id="dataSource2" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
....
</bean>
<bean id="sessionFactory2"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource2"/>
....
</bean>
<bean id="hibernateTransactionManager2"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref local="sessionFactory2"/>
</property>
</bean>
Then also, each DAO is linked either to sessionFactory1 or to sessionFactory2.
<bean name="stateHibernateDao" class="com.project.dao.StateHibernateDao">
<property name="sessionFactory" ref="sessionFactory1"/>
</bean>
Also, I recently added these two.
<bean id="atomikosTransactionManager" class="com.atomikos.icatch.jta.UserTransactionManager" init-method="init" destroy-method="close">
<property name="forceShutdown" value="false" />
<property name="transactionTimeout" value="300" />
</bean>
<bean id="atomikosUserTransaction" class="com.atomikos.icatch.jta.UserTransactionImp">
<property name="transactionTimeout" value="300" />
</bean>
I am trying to programmatically manage the global transaction
(this is some old legacy code and I don't want to change it too
much so I prefer keeping this managed programmatically).
So now I have this UserTransaction ut (injected from Spring), so I call ut.begin(), do some DB/DAO operations to the two DBs through the DAOs, then I call ut.commit().
The thing is that even before the ut.commit() call, I can see the data is already committed to the DBs?!
I don't think Atomikos is aware of my two DBs, their data sources, session factories, etc. I don't think it starts any transactions on them. Looks like they are not enlisted at all in the global transaction.
To me it seems that each DB/DAO operation goes to the SQL Server on its own, so SQL Server creates an implicit transaction for just that DAO/DB operation, applies the operation and commits the implicit the transaction.
But 1) and 2) are just guesses of mine.
My questions:
Do I need to start the two DB transactions myself (but OK, this is what I am currently doing and I am trying to get rid of; that's why I am trying to use Atomikos to start with)?
How I can configure all this correctly so that when I call ut.begin() it begins a global transaction to the two DBs and when I call ut.commit() it commits it?
I haven't played with JTA recently so seems to me I am missing something quite basic here. What is it?
Edit 1
<bean id="globalTransactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="userTransaction" ref="atomikosUserTransaction"/>
<property name="transactionManager" ref="atomikosTransactionManager" />
<property name="allowCustomIsolationLevels" value="true" />
<property name="transactionSynchronization" value="2" />
</bean>

Usage of P6Spy with datasource in Spring applicationContext.xml

I am using Hibernate 4, Spring 3, JSF 2.0 and Weblogic 10.3.6 as server.
I have created datasource on Weblogic server and in applicationContext.xml I have defined datasource as
<!-- Data Source Declaration -->
<bean id="DataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/​myDS"/>
</bean>
If I would want to use the P6Spy for logging SQL parameters, how can and where I should add the following in applicationcontext.xml?
<property name="hibernate.connection.driver_class">com.p6spy.engine.spy.
P6SpyDriver</property>
Any help is highly appreciable.
Thanks

The easiest way to integrate p6spy using spring is to use the P6DataSource class. The P6DataSource class is just a proxy for the real data source. This lets you obtain the real data source using any of the spring data source factory implementations.
<bean id="dataSource" class="com.p6spy.engine.spy.P6DataSource">
<constructor-arg>
<bean id="DataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/​myDS"/>
</bean>
</constructor-arg>
</bean>
If you are using an XADatasource, just change the classname to P6ConnectionPoolDataSource as shown below. Note: P6ConnectionPoolDataSource implements the ConnectionPoolDataSource and XADataSource interfaces.
<bean id="dataSource" class="com.p6spy.engine.spy.P6ConnectionPoolDataSource">
<constructor-arg>
<bean id="DataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/​myDS"/>
</bean>
</constructor-arg>
</bean>

You need to create bean of session factory in applicationContext.xml file as follows:
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource">
<ref bean="dataSource" />
</property>
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.p6spy.engine.spy.
P6SpyDriver" />
<property name="url" value="jdbc\:mysql\://localhost\:3306/testdb" />
<property name="username" value="my_username" />
<property name="password" value="my_password" />
</bean>
Please refer to: http://www.mkyong.com/hibernate/how-to-display-hibernate-sql-parameter-values-solution/ for more about P6Spy library.
We can omit "dataSource" bean and directly write properties. Ref: how to configure hibernate config file for sql server