JBoss crashes: datasource cannot reuse connections - java

We have JBOSS 4.0.3.SP1 over TOMCAT 5.25.26 application server.
Usually it works fine, but about once a month we have "No ManagedConnections available"
Caused by: org.jboss.util.NestedSQLException: No ManagedConnections available within configured blocking timeout ( 5000 [ms] ); - nested throwable: (javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 5000 [ms] ))
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:79)
at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:82)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:423)
... 73 more
Caused by: javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 5000 [ms] )
at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:246)
at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:529)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:410)
at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:342)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:462)
at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:894)
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:73)
When I look in oracle sessions table I can see that there ara all 100 available connections are INACTIVE and the last sql was the one we use to check connection (select sysdate from dual).
Connection pool just cannot reuse them Why this can happen?
We have JNDI datasource defined in oracle-ds.xml like this:
<datasources>
<local-tx-datasource>
<jndi-name>ROT</jndi-name>
<connection-url>jdbc:oracle:thin:#ttt.tt.tt:1521:usr</connection-url>
<driver-class>oracle.jdbc.driver.OracleDriver</driver-class>
<user-name>usr</user-name>
<password>usrpass</password>
<min-pool-size>5</min-pool-size>
<max-pool-size>100</max-pool-size>
<idle-timeout-minutes>10</idle-timeout-minutes>
<track-statements>true</track-statements>
<blocking-timeout-millis>5000</blocking-timeout-millis>
<!-- Uses the pingDatabase method to check a connection is still valid before handing it out from the pool -->
<check-valid-connection-sql>select sysdate from dual</check-valid-connection-sql>
<metadata>
<type-mapping>Oracle9i</type-mapping>
</metadata>
</local-tx-datasource>
</datasources>
Here is the hibernate config for datasource:
<bean id="rotSessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.OracleDialec</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.format_sql">false</prop>
<prop key="hibernate.cglib.use_reflection_optimizer">true</prop>
<prop key="hibernate.cache.provider_class">org.hibernate.cache.HashtableCacheProvider</prop>
<prop key="hibernate.jdbc.batch_size">20</prop>
<prop key="hibernate.connection.release_mode">after_statement</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.max_size">20</prop>
<prop key="hibernate.c3p0.timeout">300</prop>
<prop key="hibernate.c3p0.max_statements">50</prop>
<prop key="hibernate.c3p0.idle_test_period">3000</prop>
</props>
</property>
<property name="dataSource" ref="dataSource"/>
Data source in hibernate:

Related

MySQL : EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost

I have an Spring application with quartz scheduler for batch processing. It was working fine from last 4 years. There is a batch which runs on every 1 minute. But from last few days suddenly when the batch is trying to fetch some data we are getting org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Could not open connection
This batch runs throughout the day but some times gives this exception and all other batches fails to connect to MySQL. So we are restarting the application and it is working fine.
Data source config
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="url">
<value>${DATABASE_URL}</value>
</property>
<property name="driverClassName">
<value>com.mysql.jdbc.Driver</value>
</property>
<property name="username">
<value>${DATABASE_USER}</value>
</property>
<property name="password">
<value>${DATABASE_PASSWORD}</value>
</property>
</bean>
Session factory connection pooling configuration
<bean id="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.show_sql">false</prop>
<prop key="hibernate.enable_lazy_load_no_trans">true</prop>
<!-- Configurations specific to c3p0 connection pooling -->
<prop key="hibernate.c3p0.acquire_increment">5</prop>
<prop key="hibernate.c3p0.idle_test_period">1800</prop>
<prop key="hibernate.c3p0.max_size">600</prop>
<prop key="hibernate.c3p0.max_statements">50</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.timeout">1800</prop>
</props>
</property>
<property name="annotatedClasses">
<list> .... </list>
</property>
</bean>
May be it is the problem with the mysql running your db-server.
change the mysql configuration in your db-server
add the bind-address in mysql.cnf under [mysqld] section
bind-address=your_spring_application_ip
also change the accessing host for the mysql user.
run the following query in mysql
update mysql.user set host='%' where user='your_username';
flush privileges;
then restart the mysql service
sudo systemctl restart mysql
Recently we had kind of the same issue. Our application failed to connect to aws Aurora instance. What we ended up doing changing sql driver to mariadb one. Hope this will help.

getting jdbc transaction failed at first hit

I am getting JDBC transaction failed at first hit to DB. I found one problem is that. My mysql is idle for 10 to 12 hours. so may be after 10 hours when i am going to hit at login time getting exception
my configuration is
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQL5Dialect</prop>
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.hbm2ddl.auto">update</prop>
<prop key="hibernate.jdbc.batch_size">50</prop>
<prop key="hibernate.current_session_context_class">thread</prop>
<prop key="hibernate.connection.isolation">2</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.max_size">20</prop>
<prop key="hibernate.c3p0.timeout">300</prop>
<prop key="hibernate.c3p0.max_statements">50</prop>
<prop key="hibernate.c3p0.idle_test_period">3000</prop>
</props>
</property>
<property name="packagesToScan" value="com.perennialsys.salesoptimization.vo" />
</bean>
please help for this. thanks in advanced
one more thing that i got from googling is idle_test_period should not exceeds timeout. is this the reason?
The database server may close a connection on its side after a certain amount of time - causing some error in your application, because it'll attempt to send a query on a connection which is no longer available on the server side.
In order to avoid this you can let the pool periodically check a connection for it's validity. This is what idle_test_period is for.
timeout is the timespan after which the pool will remove a connection from the pool, because the connection wasn't checked out (used) for a while and the pool contains more connections than c3pO.min_size.
Please keep in mind, that hibernate.c3p0.idle_test_period value must never exceed that of hibernate.c3p0.timeout. Otherwise C3P0 will never detect connections that has been closed.
Thanks,
Rajesh

Whats the difference between hibernate.connection.autocommit and hibernate.c3p0.autocommit?

I have following bean:
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql">
<value>${hibernate.show_sql}</value>
</property>
<property name="generateDdl">
<value>${generateDdl}</value>
</property>
<property name="databasePlatform">
<value>${databasePlatform}</value>
</property>
</bean>
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">${hibernate.hbm2ddl.auto}</prop>
<prop key="hibernate.connection.autocommit">${hibernate.connection.autocommit}</prop>
<prop key="hibernate.archive.autodetection">${hibernate.archive.autodetection}</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
<prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
<prop key="hibernate.comment_sql">${hibernate.comment_sql}</prop>
<!-- optimization settings -->
<prop key="hibernate.max_fetch_depth">${hibernate.max_fetch_depth}</prop>
<prop key="hibernate.jdbc.batch_size">${hibernate.jdbc.batch_size}</prop>
<!-- c3p0 connection pool settings -->
<prop key="hibernate.c3p0.acquire_increment">${hibernate.c3p0.acquire_increment}</prop>
<prop key="hibernate.c3p0.idle_test_period">${hibernate.c3p0.idle_test_period}</prop>
<prop key="hibernate.c3p0.timeout">${hibernate.c3p0.timeout}</prop>
<prop key="hibernate.c3p0.maxIdleTime">${hibernate.c3p0.maxIdleTime}</prop>
<prop key="hibernate.c3p0.min_size">${hibernate.c3p0.min_size}</prop>
<prop key="hibernate.c3p0.max_size">${hibernate.c3p0.max_size}</prop>
<prop key="hibernate.c3p0.max_statements">${hibernate.c3p0.max_statements}</prop>
<prop key="hibernate.c3p0.min_pool_size">${hibernate.c3p0.min_pool_size}</prop>
<prop key="hibernate.c3p0.max_pool_size">${hibernate.c3p0.max_pool_size}</prop>
<prop key="hibernate.c3p0.preferredTestQuery">${hibernate.c3p0.preferredTestQuery}</prop>
<prop key="hibernate.c3p0.idleConnectionTestPeriod">${hibernate.c3p0.idleConnectionTestPeriod}</prop>
<prop key="hibernate.c3p0.autocommit">${hibernate.c3p0.autocommit}</prop>
<!-- second level cache settings -->
<prop key="hibernate.cache.provider_class">${hibernate.cache.provider_class}</prop>
<prop key="net.sf.ehcache.configurationResourceName">${net.sf.ehcache.configurationResourceName}</prop>
<prop key="hibernate.cache.use_query_cache">${hibernate.cache.use_query_cache}</prop>
<prop key="hibernate.cache.use_second_level_cache">${hibernate.cache.use_second_level_cache}</prop>
</props>
</property>
</bean>
Can anyone explain to me whats the difference between hibernate.connection.autocommit and hibernate.c3p0.autocommit? Also there are another equal properties . Whats the difference? Also what happens if they are different which one will prevail?
According to the c3p0 documentation :
autoCommitOnClose Must be set in c3p0.properties, C3P0 default: false
The JDBC spec is unfortunately silent on what should happen to unresolved, pending transactions on Connection close. C3P0's default policy is to rollback any uncommitted, pending work. (I think this is absolutely, undeniably the right policy, but there is no consensus among JDBC driver vendors.) Setting autoCommitOnClose to true causes uncommitted pending work to be committed, rather than rolled back on Connection close.
As hibernate.c3o0.autocommit drives this settings, we can say that the difference is that hibernate.connection.autocommit is just like wrapping every query with begin transaction; and then commit.
Both are not recommended.

Application not respond on tomcat after abour 30min on c3p0 configuration

I have a problem with c3p0. Last time I configured c3p0 as dataSource in application because Quartz Scheduler opened new connection for every check, now is ok but from this time application doesnt respond after about 30mins, sometimes more. There is no logs in catalina or application logs file. Other applications work normally on the same server. On this application I got 404. I returned to default Spring data source and removed configuration and everything works.
What can cause the problem ?
Tomcat 8.0.21
c3p0.version 0.9.5.1
Hibernate version 4.3.1.Final
Configuration
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="${db.driver}" />
<property name="jdbcUrl" value="${db.url}" />
<property name="user" value="${db.username}" />
<property name="password" value="${db.password}" />
</bean>
<property name="hibernateProperties">
<props>
<!-- HIBERNATE CONFIGURATION -->
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
<!-- <prop key="hibernate.hbm2ddl.auto">${hibernate.hbm2ddl.auto}</prop> -->
<prop key="hibernate.connection.characterEncoding">UTF-8</prop>
<prop key="hibernate.connection.useUnicode">true</prop>
<prop key="hibernate.generate_statistics">true</prop>
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.cache.use_second_level_cache">true</prop>
<prop key="hibernate.cache.region.factory_class">org.hibernate.cache.ehcache.EhCacheRegionFactory
</prop>
<prop key="hibernate.cache.use_query_cache">true</prop>
<prop key="hibernate.cache.provider_class">net.sf.ehcache.hibernate.EhCacheProvider</prop>
<prop key="hibernate.cache.provider_configuration_file_resource_path">ehcache.xml</prop>
<!-- HIBERNATE SEARCH CONFIGURATION -->
<prop key="hibernate.search.default.directory_provider">filesystem</prop>
<prop key="hibernate.search.default.indexBase">${hibernate.search.local}</prop>
<!-- CONNECTIONS POOLING CONFIGURATION -->
<prop key="hibernate.c3p0.acquire_increment">1</prop>
<prop key="hibernate.c3p0.idle_test_period">1</prop>
<prop key="hibernate.c3p0.max_size">60</prop>
<prop key="hibernate.c3p0.max_statements">0</prop>
<prop key="hibernate.c3p0.min_size">30</prop>
<prop key="hibernate.c3p0.timeout">0</prop>
</props>
</property>
</bean>
Update with new settings:
<prop key="hibernate.c3p0.acquire_increment">1</prop>
<prop key="hibernate.c3p0.idle_test_period">1000</prop>
<prop key="hibernate.c3p0.max_size">60</prop>
<prop key="hibernate.c3p0.max_statements">0</prop>
<prop key="hibernate.c3p0.min_size">20</prop>
<prop key="hibernate.c3p0.timeout">100</prop>
Still same, one user, 1h:40min

Hibernate 4 default transaction timeout value

I'm getting this error:
2014-11-14 17:39:44 [WARN]SqlExceptionHelper:143 SQL Error: 0, SQLState: 08S01
2014-11-14 17:39:44 [ERROR]SqlExceptionHelper:144 Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2014-11-14 17:39:44 [ERROR]BackupService:245 org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Could not open connection
Seems that Hibernate has a small transaction timeout for my job, but I can not find what is the default timeout value. I did not find it also in hibernate documentation, I find how to change it, but not the default value.
This is my hibernate configuration:
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan">
<list>
<value>com.test.db.entity</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.HSQLDialect</prop>
<prop key="hibernate.show_sql">false</prop>
<prop key="hibernate.bytecode.use_reflection_optimizer">false</prop>
<prop key="hibernate.cglib.use_reflection_optimizer">true</prop>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.search.autoregister_listeners">false</prop>
<prop key="hibernate.transaction.factory_class">org.hibernate.transaction.JDBCTransactionFactory</prop>
</props>
</property>
</bean>
Do you know what is the default timeout value for an hibernate transaction with this configuration?
I got the same exception. I googled and found your question without answer :)
I changed "Timeout Seconds:" from 300 to 1000 (possible to increase more, it depends on your transaction timeout) on "mydomain->Configuration->JTA" on Weblogic Console or increase JTA timeout on your application server.
Then, for hibernate.cfg.xml you may check this answer: Hibernate/MySQL Connection Timeout
It works.

Categories