We Keep Coding

Issue with CachingConnectionFactory and DefaultMessageListenerContainer

It is mentioned in the documentation of DefaultMessageListenerContainer class that it is not recommended to use CachingConnectionFactory with dynamic scaling. While searching, I have encountered following link:
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
Here found a comment from Gary Russell that
the problem is with caching consumers when using variable concurrency in the container; we can end up with a live consumer "stuck" in the cache".
We have used DefaultMessageListenerContainer and CachingConnectionFactory together so this is surely a problem from above link.
We are encountering problems with our application having following behaviour:
TCP ZeroWindow network congestion
TCP RESET from application server to MQ
DB connection grows during the issue while different transactions halt
Messages in certain queues gets built up
We have following code configuration :
In ibmmq-context.xml file:
<!-- WebSphere MQ Connection Factory -->
<bean id="appMqConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="hostName">
<value>${ibmmq.ip}</value>
</property>
<property name="port">
<value>${ibmmq.port}</value>
</property>
<property name="queueManager">
<value>${ibmmq.queuemanager}</value>
</property>
<property name="channel">
<value>${ibmmq.channel}</value>
</property>
<property name="clientReconnectOptions">
<util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CLIENT_RECONNECT"/>
</property>
<property name="transportType" ref="appTransport"/>
</bean>
<!-- A cached connection -->
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="appMqConnectionFactory"/>
<property name="sessionCacheSize" value="${jms.session.cachesize}"/>
</bean>
<!-- Use native MQ classes. -->
<bean id="appTransport" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean">
<property name="staticField">
<value>com.ibm.mq.jms.JMSC.MQJMS_TP_CLIENT_MQ_TCPIP</value>
</property>
</bean>
In jms-context file:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
There are 6 such listeners like bankListener and each of the listeners has concurrency value, varies from 10-40
<bean id="listenerExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="maxPoolSize" value="140"/>
<property name="corePoolSize" value="100"/>
<property name="queueCapacity" value="30"/>
<property name="threadNamePrefix" value="jms-listener-task-"/>
<property name="threadGroupName" value="jms-listener-tasks"/>
</bean>
and jms-context.xml file uses ibmmq-context.xml file.
And to note, we have used IBM MQ 7.1, Spring 4.2.8, spring-integration-core as 4.3.1.RELEASE and JBoss EAP 6.4.10
We are planning to fix this by following way:
<bean id="bankListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="appMqConnectionFactory" />
<property name="destination" ref="transactionResponseDestination" />
<property name="messageListener" ref="thirdpartyService" />
<property name="autoStartup" value="false"/>
<property name="taskExecutor" ref="listenerExecutor"/>
<property name="concurrency" value="20-30"/>
</bean>
My request:
Please review the configuration and let me know is there anything else to be changed.
Could you please also explain our application behaviour(above 4 points - a to d) with our current configuration with CachingConnectionFactory and DefaultMessageListenerContainer
Thanks in advance for your help.

Try setting:
cachingConnectionFactory.setCacheConsumers(false);
Or in Spring it should be:
<bean id="appCachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
...
<property name="cacheConsumers" value="false"/>
...
</bean>

Advantage of org.springframework.jdbc.datasource.DriverManagerDataSource over oracle.jdbc.pool.OracleDataSource

I currently use this configuration for my projects:
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource"
destroy-method="close">
<property name="URL" ...
<property name="user" ...
<property name="password" ...
<property name="connectionCachingEnabled" value="true" />
And it works fine, pretty fast.
I happened to see, on an old project (spring 2.5) this configuration:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName">
<value>oracle.jdbc.driver.OracleDriver</value>
</property>
<property name="url"...
<property name="username" ...
<property name="password" ...
</bean>
From documentation it would seem that this last option does not make use of a connection pool. I see no reason to use this configuration over mine, but it still exists so I am curious: where's the advantage/limitation?

1st configuration is oracle specific, whereas 2nd configuration is generic. You can explicitly define driver class.
This is the only major difference I can see in them other than connection pool support of OracleDataSource.
You can use it for generic behavior as mentioned below:
<bean id="baseDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource"
abstract="true">
<property name="username" value="user"/>
<property name="password" value="pwd" />
</bean>
<bean id="mySqlDataSource" parent="baseDataSource">
<property name="driverClassName" value="${mySQL.driver}" />
<property name="url" value="${mySQL.url}"/>
</bean>
<bean id="oracleDataSource" parent="baseDataSource">
<property name="driverClassName" value="${oracle.driver}" />
<property name="url" value="${oracle.url}"/>
</bean>
Property values you can externalized.
You can explore Apache Jakarta Commons DBCP which has all the features of DriverManagerDataSource
along with connection pool feature.

Creating a Spring DataSource for connecting to Google Cloud SQL

I'm trying to find the best way to create a dataSource in Spring for connecting to a Google Cloud SQL instance.
I'm currently using:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.GoogleDriver" />
<property name="url" value="jdbc:google:mysql://myappid:instanceId/mydb?user=myuser" />
<property name="username" value="myuser" />
<property name="password" value="mypassword" />
</bean>
However, I'm a little concerned about using the DriverManagerDataSource provided by Spring as it's documentation says it creates a new connection for every call.
Before migrating over to App Engine I was using a connection pool called BoneCP - however it uses classes that are restricted by App Engine. Is there a connection pool or some other data source class that is recommended to be used with Google Cloud SQL?

Try c3p0 or commons-dbcp. They both implement javax.sql.Datasource which is whitelisted by app-engine.
Example on commons-dbcp:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.GoogleDriver" />
<property name="url" value="jdbc:google:mysql://myappid:instanceId/mydb?user=myuser" />
<property name="username" value="myuser" />
<property name="password" value="mypassword" />
<property name="validationQuery" value="SELECT 1"/>
</bean>

expose c3po dataSourceName in jmx

Is there a way to make c3p0 to register to jmx with its dataSourceName? Currently my c3p0 data create a random name to register to jmx even though it has its dataSourceName. For example my configuration in spring is:
<bean id="services" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="dataSourceName" value="mySQLDataSource"/>
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="jdbcUrl">
<value>${jdbcUrl}</value>
</property>
<property name="user">
<value>${user}</value>
</property>
<property name="password">
<value>${password}</value>
</property>
<property name="initialPoolSize" value="1" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="10" />
<property name="maxIdleTime" value="10" />
</property>
</bean>11

Not sure if C3P0 allows that, but that feature is present in BoneCP (http://jolbox.com).

It's definitely possible!
I've gotten it working with the .properties file, myself, but the documentation says it will also work with an XML file.
With your Spring approach it might be less confusing to make a properties file.
Mine is at webapps/[appname]/WEB-INF/classes/c3p0.properties
References:
- http://www.mchange.com/projects/c3p0/#jmx_configuration_and_management
- http://www.mchange.com/projects/c3p0/#c3p0_conf

Configure spring datasource for hibernate and #Transactional

At this moment I'm using DriverManagerDataSource with #Transactional annotation to manage transactions. But all transactions are very very slow, probably because data source open and close connection to db each time.
What data source should I use to speed up transaction?

I am using in my application combination of two approaches. the first one is c3p0 connection pooling, its almost the same solution as chkal sugested. The second approach is to use Spring lazyConnectionDataSourceProxy, which creates lazy loading proxy that loads connection only if you hit the database. This is very useful, when you have second level cache and you are only reading cached data and queries - database wont be hit, and you don't need to acquire connection (which is pretty expensive).
<bean name="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="${jdbc.driverClassName}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<!-- Pool properties -->
<property name="minPoolSize" value="5" />
<property name="initialPoolSize" value="10" />
<property name="maxPoolSize" value="50" />
<property name="maxStatements" value="50" />
<property name="idleConnectionTestPeriod" value="120" />
<property name="maxIdleTime" value="1200" />
</bean>
<bean name="lazyConnectionDataSourceProxy" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
<property name="targetDataSource" ref="dataSource" />
</bean>

DriverManagerDataSource isn't actually a connection pool and should only be used for testing. You should try BasicDataSource from Apache Commons DBCP. Something like:
<bean id="dataSource" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
</bean>