We Keep Coding

MySQL : EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost

I have an Spring application with quartz scheduler for batch processing. It was working fine from last 4 years. There is a batch which runs on every 1 minute. But from last few days suddenly when the batch is trying to fetch some data we are getting org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Could not open connection
This batch runs throughout the day but some times gives this exception and all other batches fails to connect to MySQL. So we are restarting the application and it is working fine.
Data source config
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="url">
<value>${DATABASE_URL}</value>
</property>
<property name="driverClassName">
<value>com.mysql.jdbc.Driver</value>
</property>
<property name="username">
<value>${DATABASE_USER}</value>
</property>
<property name="password">
<value>${DATABASE_PASSWORD}</value>
</property>
</bean>
Session factory connection pooling configuration
<bean id="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.show_sql">false</prop>
<prop key="hibernate.enable_lazy_load_no_trans">true</prop>
<!-- Configurations specific to c3p0 connection pooling -->
<prop key="hibernate.c3p0.acquire_increment">5</prop>
<prop key="hibernate.c3p0.idle_test_period">1800</prop>
<prop key="hibernate.c3p0.max_size">600</prop>
<prop key="hibernate.c3p0.max_statements">50</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.timeout">1800</prop>
</props>
</property>
<property name="annotatedClasses">
<list> .... </list>
</property>
</bean>

May be it is the problem with the mysql running your db-server.
change the mysql configuration in your db-server
add the bind-address in mysql.cnf under [mysqld] section
bind-address=your_spring_application_ip
also change the accessing host for the mysql user.
run the following query in mysql
update mysql.user set host='%' where user='your_username';
flush privileges;
then restart the mysql service
sudo systemctl restart mysql

Recently we had kind of the same issue. Our application failed to connect to aws Aurora instance. What we ended up doing changing sql driver to mariadb one. Hope this will help.

Spring 4 + JPA (Hibernate 4) + JTA transaction manager doesn't flush automatically

I am migrating an application from
Spring 3.0.5 + JPA 2.0
to
Spring 4 + JPA (Hibernate 4)
I have followed the migration guide : https://github.com/spring-projects/spring-framework/wiki/Migrating-from-earlier-versions-of-the-spring-framework.
The application is using a JTA transaction manager : a Jencks / GeronimoPlatformTransactionManager (because of transactions distributed on datasources and ESB).
The Spring / JPA configuration is :
<bean id="rduEntityManagerFactory" class="ch.vd.dsas.rdu.repository.crud.service.ExtendedLocalContainerEntityManagerFactoryBean"
depends-on="rduTransactionManagerLocator,jGroupsCacheManagerPeerProviderFactoryLocator">
<property name="persistenceUnitName" value="rduPersistenceUnit" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="${rdu.jpa.database}" />
</bean>
</property>
<property name="persistenceUnitPostProcessors">
<bean class="ch.vd.dsas.rdu.commons.tx.spring.JtaPersistenceUnitPostProcessor">
<property name="jtaDataSource" ref="rduDataSource" />
</bean>
</property>
<property name="jpaProperties" ref="jpaProperties"/>
</bean>
<util:properties id="jpaProperties">
<prop key="javax.persistence.transactionType">JTA</prop>
<prop key="javax.persistence.validation.mode">CALLBACK</prop>
<prop key="hibernate.hbm2ddl.auto">${rdu.jpa.hbm2ddl.auto}</prop>
<prop key="hibernate.current_session_context_class">jta</prop>
<!-- Transaction properties -->
<prop key="hibernate.transaction.jta.platform">ch.vd.dsas.rdu.ref.transaction.jencks.JencksTransactionManagerLookup</prop>
<prop key="hibernate.transaction.manager_lookup_class">ch.vd.dsas.rdu.transaction.jencks.JencksTransactionManagerLookup</prop>
<prop key="hibernate.default_schema">${rdu.datasource.schemaMetier}</prop>
<!-- Debug properties -->
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.show_sql">true</prop>
<!-- Cache properties -->
<prop key="hibernate.cache.use_second_level_cache">true</prop>
<prop key="hibernate.cache.use_query_cache">true</prop>
<prop key="hibernate.cache.region.factory_class">net.sf.ehcache.hibernate.ReplicatedSingletonEhCacheRegionFactory</prop>
<prop key="hibernate.cache.cluster_name">${rdu.hibernate.cache.jgroups.cluster.name}</prop>
<prop key="net.sf.ehcache.configurationResourceName">/hibernate-ehcache.xml</prop>
</util:properties>
Transactions are annotation driven :
<tx:annotation-driven transaction-manager="rduJtaTransactionManager" />
The transaction manager is declared like that :
<!-- From Jencks org.jencks:jencks:2.2 -->
<bean id="rduJencksTransactionManager" class="org.jencks.factory.TransactionManagerFactoryBean" />
<bean id="rduJtaTransactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<qualifier value="rdu" />
<property name="transactionManager" ref="rduJencksTransactionManager" />
<property name="userTransaction" ref="rduJencksTransactionManager" />
</bean>
<bean id="rduTransactionManagerLocator" class="ch.vd.dsas.rdu.transaction.jencks.TransactionManagerLocator" factory-method="getInstance">
<property name="transactionManager" ref="rduJencksTransactionManager"/>
</bean>
The application is starting and accessing data and displaying it.
However no insert / update are performed.
If I change data and submit the change the application receives the change but the data does not get flushed to the database.
I have activated logs and I see the transaction :
rdu 2015-06-18 20:28:01,817 [ http-8080-1] DEBUG [ o.s.t.j.JtaTransactionManager] Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.save]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; 'rdu'
rdu 2015-06-18 20:28:01,817 [ http-8080-1] DEBUG [ o.s.t.j.JtaTransactionManager] Participating in existing transaction
rdu 2015-06-18 20:28:01,823 [ http-8080-1] DEBUG [ o.s.t.j.JtaTransactionManager] Initiating transaction commit
But nothing is sent to the database.
If I intercept the execution through debugging and manually flush the Hibernate session, the data gets updated.
It seems the JPA / Hibernate session is not coordinated with the transaction.
I can't figure out what is missing in the configuration and why the session is not flushed automatically.
Hope you can help me with this issue.
Best regards,
Eric

The problem is due to this property:
<prop key="hibernate.transaction.jta.platform">ch.vd.dsas.rdu.ref.transaction.jencks.JencksTransactionManagerLookup</prop>
The hibernate.transaction.jta.platform property is not the same with hibernate.transaction.manager_lookup_class and it should point to an AbstractJtaPlatform implementation:
<property name="hibernate.transaction.jta.platform" value="org.hibernate.service.jta.platform.internal.SunOneJtaPlatform"/>

HikariCP with PostgreSQL: setQueryTimeout(int) is not yet implemented

I tried to use HikariCP with PostgreSQL but it doesn't work. It throws an exception:
WARN [2014-10-24 14:38:54,195] com.zaxxer.hikari.pool.HikariPool: Exception during keep alive check, that means the connection must be dead.
! org.postgresql.util.PSQLException: Method org.postgresql.jdbc4.Jdbc4Statement.setQueryTimeout(int) is not yet implemented.
! at org.postgresql.Driver.notImplemented(Driver.java:753) ~[postgresql-9.0-801.jdbc4.jar:na]
! at org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:656) ~[postgresql-9.0-801.jdbc4.jar:na]
! at com.zaxxer.hikari.pool.HikariPool.isConnectionAlive(HikariPool.java:473) [HikariCP-java6-2.1.0.jar:na]
! at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:178) [HikariCP-java6-2.1.0.jar:na]
! at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:94) [HikariCP-java6-2.1.0.jar:na]
I tried all possible driver versions (including org.postgresql:postgresql:9.3-1102-jdbc41) - nothing helped. Did anybody use it with PostgreSQL?
Here is my config file:
<bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig">
<property name="connectionTestQuery" value="SELECT 1" />
<property name="dataSourceClassName" value="org.postgresql.ds.PGSimpleDataSource" />
<property name="maximumPoolSize" value="${jdbc.maximumPoolSize:20}" />
<property name="idleTimeout" value="${jdbc.idleTimeout:600000}" />
<property name="dataSourceProperties">
<props>
<prop key="databaseName">vidsearch_test</prop>
<prop key="user">${hibernate.connection.username}</prop>
<prop key="password">${hibernate.connection.password}</prop>
</props>
</property>
</bean>
<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">
<constructor-arg ref="hikariConfig"/>
</bean>

Don't use a connectionTestQuery. Allow HikariCP to follow its default behavior of using the Connection.isValid() method. This requires a JDBC4 driver.

Version 42.2.0 (or higher) implements set/get network timeout.

Hibernate doesn't start

I have the following hibernate config on spring and the server starts after a long time but doesn't connect to DB (no schema on DB). So It was supose to give a error message or create the schema with <prop key="hibernate.hbm2ddl.auto">create</prop>.
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method = "close">
<property name="driverClass" value="com.mysql.jdbc.Driver"/>
<property name="jdbcUrl" value="jdbc:mysql://dburl:3306"/>
<property name="user" value="user"/>
<property name="password" value="pass!"/>
</bean>
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="mappingResources">
<list>
<value>waf/resources/User.hbm.xml</value>
<value>waf/resources/Post.hbm.xml</value>
<value>waf/resources/Position.hbm.xml</value>
<value>waf/resources/Comment.hbm.xml</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQL5Dialect</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.hbm2ddl.auto">validate</prop>
<!-- C3P0 CONNECTION POOL -->
<prop key="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</prop>
<prop key="c3p0.acquire_increment">1</prop>
<prop key="c3p0.idle_test_period">100</prop>
<prop key="c3p0.max_size">20</prop>
<prop key="c3p0.max_statements">50</prop>
<prop key="c3p0.min_size">1</prop>
<prop key="c3p0.timeout">10</prop>
</props>
</property>
<property name="dataSource">
<ref bean="dataSource"/>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory">
<ref bean="sessionFactory"/>
</property>
</bean>
Can you guys help me out?

Hibernate does not create schemas with hbm2ddl.auto. It just creates | creates-drop | etc tables.

Going through the HBM files, you have given as below..
Validate instead of Create
I hope this will not create the DDL. can you please check that? Or is it a typo in question?

Adding to what others have proposed if you are using connection pool (which you are as c3po is mentioned) then while the session Factory is created it will try to use the connection pool backed datasource which in turn will connect to database to pre-create and pool connection. When you say that it does not connect to database - how do you know that ? Is there error in logs? I have seen that if Hibernate session factory is not able to configure itself it throws an error in the logs.

Connection pool problem with Spring and programmatic transaction management

I need your help in order to solve connection pool problem with Spring.
I’m using Spring with Java 1.4 (no annotation).
Here is the datasource and the connection pool definition:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName">
<value>${database.jdbcName}</value>
</property>
<property name="url" value="jdbc:sybase:${database.url}">
</property>
<property name="username">
<value>${database.username}</value>
</property>
<property name="password">
<value>${database.password}</value>
</property>
<property name="maxActive">
<value>${database.maxActive}</value>
</property>
<property name="maxIdle" >
<value>${database.maxIdle}</value>
</property>
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource">
<ref bean="dataSource" />
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.SybaseDialect</prop>
<prop key="show_sql">true</prop>
<prop key="hibernate.connection.release_mode">after_transaction</prop>
<prop key="hibernate.dbcp.maxWait">100000</prop>
<prop key="hibernate.connection.autocommit">false</prop>
<prop key="cache.provider_class">org.hibernate.cache.NoCacheProvider</prop>
</props>
</property>
...
When all the connections are active, if the system need a new connection I got a message saying “opening JDBC connection” and all the process are stopped. I don’t understand why all the threads are locked.
All services and DAO classes are defined as singletons in Spring application context.
Any idea?
Thanks for your help.
Béranger

What values are you using for maxActive and maxIdle? I can give you a more definitive answer if I know those values, but in the meantime you can also try changing the value of hibernate.dbcp.maxWait from 100000 to 1 and test your application again.
Also make sure you haven't you accidentally made some of your DAO methods synchronized.

This was a weird one but I finally solved my problem.
The reason of this connection locking was that I used both declarative and programmatic transaction management with Spring. I think that my programmatic transaction management was wrong.
Here is the code used to begin a transaction:
//Return Spring context
ApplicationContext context = ApplicationContextHolder.getContext();
// transactionManager
PlatformTransactionManager transactionManager = (PlatformTransactionManager) context.getBean("txManager");
//Set propagation required
DefaultTransactionDefinition td = new DefaultTransactionDefinition();
td.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
// Begin
TransactionStatus transactionStatus = transactionManager.getTransaction(td);
As you can see I set the programmatic transactions as “required”. But when those transactions were started in a transactional context, Spring didn’t recognized the current transaction and started a new one. It leaded to a connection locking...
I solved this mystery by removing the programmatic transaction management.