We in the process of migrating from MySql to MariaDB due to licensing/commercial usage reasons.
We have successfully replaced the MySql connector jar with MariaDB client jar (first change) and are now trying to replace MySql server with MariaDB server without changing the data files.
All our applications run perfectly for about 8-12 hours after which we see the following exception:
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Cannot open connection
Caused by:
org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:74)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:29)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:426)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:144)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:494)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:315)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:257)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:102)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
at $Proxy4.getMessageCountByStatus(Unknown Source)
at com.onmobile.cmfweb.monitoring.CmfMessagesMonitor.getMessageCounts(CmfMessagesMonitor.java:56)
at sun.reflect.GeneratedMethodAccessor625.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:270)
at com.onmobile.cmfshare.MethodInvockingBean.invoke(MethodInvockingBean.java:28)
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:212)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:79)
at org.quartz.core.JobRunShell.run(JobRunShell.java:203)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to localhost:3306: unexpected end of stream, read 0bytes from 4
at org.mariadb.jdbc.internal.SQLExceptionMapper.get(SQLExceptionMapper.java:136)
at org.mariadb.jdbc.internal.SQLExceptionMapper.throwException(SQLExceptionMapper.java:106)
at org.mariadb.jdbc.Driver.connect(Driver.java:114)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:37)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:544)
at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:81)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:423)
... 21 more
Caused by: org.mariadb.jdbc.internal.common.QueryException: Could not connect to localhost:3306: unexpected end of stream, read 0bytes from 4
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:509)
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:669)
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.<init>(MySQLProtocol.java:264)
at org.mariadb.jdbc.Driver.connect(Driver.java:110)
... 28 more
Caused by: java.io.EOFException: unexpected end of stream, read 0bytes from 4
at org.mariadb.jdbc.internal.common.packet.buffer.ReadUtil.readFully(ReadUtil.java:84)
at org.mariadb.jdbc.internal.common.packet.buffer.ReadUtil.readFully(ReadUtil.java:92)
at org.mariadb.jdbc.internal.common.packet.RawPacket.nextPacket(RawPacket.java:77)
at org.mariadb.jdbc.internal.common.packet.SyncPacketFetcher.getRawPacket(SyncPacketFetcher.java:67)
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:467)
... 31 more
The exceptions disappear if MySql server is used again.
From what I have debugged so far MariaDB server is closing the client connections for some reason. I initially suspected of idle connections but we are using Hibernate configurations like 'testOnBorrow' so it shouldn't be the case.
Can anyone help us to find out the cause for this and help us fix the problem? Is there any specific configuration in MariaDB that I should be doing?
I've even run our applications by increasing the value of the param 'interactive_timeout' in the MariaDB server but it did not help.
Btw, we are using Spring-Hibernate and pooling our connections using Commons-pool jar.
Any kind of help will be deeply appreciated.
We encountered the same problem recently, due to a combination of incorrect parameters.
This error is caused by your web instance trying to use a connection that is no longer a valid one.
This can be resolved by making sure the following parameters are correct:
You have a validationQuery configured for your database connection. i.e. validationQuery="SELECT 1" in case of mariadb, in the server configuration.
You have the wait_timeout set to a reasonable value. 8 hours keepAlive seems a bit optimistic, we are now using wait_timeout=180
Make sure the validationInterval, which you can also set in the server configuration (server.xml in case of a tomcat setup), is set to a value that is lower than the wait_timeout value. I reduced the wait_timeout to 15s in one case, with the validationInterval value being higher than that, which caused the error to still appear at times.
Now it is set to validationInterval=60, combined with the wait_timeout=180 which should catch any broken connections in time.
You can try to increase 'wait_timeout' instead of 'interactive_timeout'. 8-12 hours until error correlates quite well with the default value of wait_timeout which is 10 hours. JDBC driver does not set interactive client flag when connecting, so change to 'interactive_timeout' might have no effect. I also replied to the cross-post in Maria KB ;)
According to the findings in this article you could get away with two options:
You should try turning innodb_file_per_table off
or...
Increasing the open file limits, this tells you the current value on yout machine $ ulimit -n, should you be running on high spec machines
I also had an error like this one Could not connect to localhost:3306: unexpected end of stream, read 0 bytes from 4
My architecture was an webapp of Tomcat that was inserting data into MariaDB.
I was using the connector version 1.5.1RC!
I switched to the 1.3.1 connector version and now runs fine!
I also faced this issue. The issue goes away with lowering the wait_timeout.
SHOW SESSION VARIABLES LIKE 'wait_timeout'; ->> would be 8 hours. which is huge.
SET session wait_timeout=300; ->> this should fix the issue.
Look at
SHOW VARIABLES LIKE 'max_connections';
SHOW GLOBAL STATUS LIKE '%connect%';
Compare the my.cnf values between the two systems; you may see a difference in things like above that make a difference. If the problem is still not obvious, please provide those values for further discussion.
I think that the problem is with MariaDB Connector/J. Try to use a stable one like MariaDB Connector/J 1.2.3 for example. Also, study Failover behaviour with Basic failover when autoReconnect is set to true or Standard failover.
Related
I fail to establish a connection to a MySql database on an external server when I use the mysql-connector-java version 8.0.11+ (I was currently testing with 8.0.25). However I am able to create a connection when I use the older MySql connector mysql-connector-java version 5.1.49.
The version of the MySql Database is "8.0.25-15"
The code fails on the first line:
try (Connection conn = DriverManager.getConnection(DB_URL, USER, PASS);
{...}
Since I can connect though mysql-connector-java version 5.1.49, I first thought I could solve this problem by adding parameters to the DB_URL. I have experimented with a lot of parameters, but no luck:
useSSL=false
serverTimezone=GMT
useUnicode=true
characterEncoding=utf-8
passwordCharacterEncoding=utf-8
connectionCollation=utf8mb4_bin
autoReconnect=true
failOverReadOnly=false
maxReconnects=10
cacheServerConfiguration=false
The error I always receive is:
java.sql.SQLNonTransientConnectionException: Could not create connection to database server.
Caused by: java.lang.NullPointerException: Cannot invoke "String.toUpperCase(java.util.Locale)" because "javaEncoding" is null
at com.mysql.cj.CharsetMapping.getMysqlCharsetForJavaEncoding(CharsetMapping.java:552)
at com.mysql.cj.CharsetMapping.getCollationIndexForJavaEncoding(CharsetMapping.java:585)
at com.mysql.cj.protocol.a.NativeServerSession.configureCharacterSets(NativeServerSession.java:452)
at com.mysql.cj.jdbc.ConnectionImpl.initializePropsFromServer(ConnectionImpl.java:1329)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:866)
A month ago, I did not have this problem, so I suspect the cause is a change in the database (I am not tbe database Administrator).
When I debug, the error seems to be caused because the program requests the "character_set_system" variable from MySql, which is"utf8mb3" (instead "utf8" or "utf8mb4").
I suspect that changing the database variable "utf8mb3" to "utf8" might solve this problem? But I am not certain and I cannot simply change this value because the database is hosted by an external company. I have typed "Show Variables" and "Show Global Variables" in a MySql editor for your information:
A screenshot of the DB variables concerning language.
Alternatively I would like to tell java to ignore "character_set_system". It seems that Java first looks for "local.character_set_results", but this variable returns null despite that "character_set_results" is defined in the database.
I hope someone can help me with this problem.
Kind Regards
Steven
SOLVED
The latest version of MySQL JDBC driver evidently attempts to create an SSL connection. The app which WORKED did NOT define javax.net.ssl.keystore (and other) values, so (I assume) it reverts to a non-SSL connection. The app which FAILED did define all of the keystore/certificate parameters and then the connection attempt failed - still not sure of exactly why, if someone could enlighten would appreciate. The exception thrown could also be more informative.
Anyway, adding "&useSSL=false&allowPublicKeyRetrieval=true" to the connection parameters works for both applications. (Although not a very good setup for any production app, can use for this POC!)
============= Original Posting Follows =========
This has had me stumped for a few days! AWS server (86_64), Fedora 29, Java 8 (1.8.0_201-b09), MySQL - all upgraded to the latest release. MySQL is running on the same server. Driver: com.mysql.cj.jdbc.Driver
Connection params: jdbc:mysql://172.31.10.222:3306?user=xxxx&password=xxxx. (IP is the AWS private IP for this server.)
Have 2 different utilities - separate 'main' routines, but all in the same JAR file. The first one works perfectly, the second one aborts with the stack pasted below. This is consistent over dozens of trials with minor variations.
The different main routines both use the 'connectToDB' method shown in the stack trace. Instrumented this to log all parameters and the class name of the driver in both cases and verified that they are identical.
Have already tried/verified:
Multiple Google searches of the same error
MySql is running and OK (connect thru another terminal window w no problem),
Same JAR used with all local code
Classpath is identical
Running under the same Linux user, using the same MySQL user/password
Verified the IP above (172.31.10.222)
Also changed the IP to 'localhost', 127.0.0.1, and a symbolic name from /etc/hosts
Added -Xmx2G to make sure not an OOM issue
These are 2 different 'main' methods, but don't do much more than load some properties (from the same property files) before trying to connect to the DB. Then they execute the same connectToDB routine.
Something in the context must be different! Hoping that someone here can at least provide some hints on where to look!
Thanks
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207)
at spc.tools.spcConnection.connectToDB(spcConnection.java:692)
at spc.tools.spcConnection.connectToLocalDB(spcConnection.java:1418)
at spc.tools.spcConnection.loadAllConnectionsOK(spcConnection.java:1112)
at spc.tools.spcConnection.initOneTime(spcConnection.java:1012)
at spc.tools.spcConnection.init(spcConnection.java:194)
at spc.ops.spcServerBase.environmentOK(spcServerBase.java:367)
at spc.ops.spcServerBase.environmentOK(spcServerBase.java:342)
at spc.run.daemon.spcDaemon.main(spcDaemon.java:1082)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:351)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.negotiateSSLConnection(NativeAuthenticationProvider.java:777)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:486)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:202)
at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1442)
at com.mysql.cj.NativeSession.connect(NativeSession.java:165)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:955)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)
... 11 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:879)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:850)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:720)
at sun.security.ssl.Handshaker.sendChangeCipherSpec(Handshaker.java:1144)
at sun.security.ssl.ClientHandshaker.sendChangeCipherAndFinish(ClientHandshaker.java:1280)
at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:1190)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:369)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:315)
====================================================== New Information =====
Here are the key lines from the connectToDB method (responding to comment below):
Driver dvr = (Driver)Class.forName(info.sDBDriver).newInstance();
if(dvr!=null) conn = dvr.connect(accessString(), null);
verified the expected driver and that accessString() returns the 'jdbc:mysql:...' string as above. The exception is thrown executing the 2nd line above.
Since the original post, put in code to run a 'netstat -ln' immediately before these lines and post the result to the log. This shows an active listener for port 3306.
It would be great if you can post connectToDB() method.
According to this sentence " The first one works perfectly, the second one aborts with the stack pasted below.", you must have closed the db connection in the first connectToDB() call.
So that second call is broken as the DB connection was terminated by first call.
The good way to setup is to use dependency injection for database object.
Hope this helps!
I am trying to get a JDBC connection pool and resource setup in a local GlassFish server to connect to a Heroku PostgreSQL instance.
I have tried it in Tomcat setup as a resource and it works ok, but in GlassFish I have had some issues.
First up I had to downgrade my version of Java 8 to 151 to stop one error I was getting. Now when I try and execute the query in my app I get:
java.net.ConnectException: Connection refused (Connection refused)
I think this is related to connections to Heroku PostgreSQL having to use SSL, but I am not sure what I need to do in GlassFish to configure that.
Does anyone have any experience doing this?
After a lot of head scratching and Googling, I finally got this working. The exception above is a red herring. It is complaining it cannot connect to a local Derby database which is the GlassFish default resource. This means the problem was actually related to not being able to find the resource I specified.
Here are the steps I needed to get the connection working in GlassFish 5:
Set JAVA_HOME to be JDK 1.8.0_151 (any newer version of 1.8 does not work) GlassFish needs to run on this version.
In your Bean class use #Resource(mappedName = "jdbc/postgresql"). This was the only way I could get it to find my jdbc/postgresql resource (#Resource(name = "jdbc/postgresql"), #Resource(name = "java:comp/env/jdbc/postgresql") and #Resource(lookup = "java:comp/env/jdbc/postgresql") did not work).
The connection Url must be like this:jdbc:postgresql://<host>:<port>/<databse>?sslmode=require (originally i had this jdbc:postgresql://<host>:<port>/<databse>?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory which had been working for me in other places).
So the key points are to use JDK 1.8.0_151, mappedName in the resource attribute and sslmode=require in the connection string.
Life should not be this complicated...
I'm having a problem when connecting to an Oracle database, it takes a long time (about ~5 minutes) and it sends the below shown exception. Most of the time, after the first error, the next connections for the same process work correctly.
It is a RHEL 6 machine, with two different network interfaces and ip addresses.
NOTE:
I am not using an url like: "jdbc:oracle:thin:#xxxx:yyy, it is actually: "jdbc:oracle:thin:#xxxx:yyyy:zzz.
The SID is not missing, sorry for that :(
This is roughly what I've isolated:
bin/java -classpath ojdbc6_g.jar -Djavax.net.debug=all
-Djava.util.logging.config.file=logging.properties
Class.forName ("oracle.jdbc.OracleDriver")
DriverManager.getConnection("jdbc:oracle:thin:#xxxx:yyyy", "aaaa", "bbbb")
Error StackTrace:
java.sql.SQLRecoverableException: IO Error: Connection reset
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:533)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:557)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:233)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:29)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:556)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:221)
at test.jdbc.Main(Test.java:120)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at oracle.net.ns.DataPacket.send(DataPacket.java:248)
at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:227)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:309)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:257)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:182)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:99)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:121)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:77)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1173)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:309)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:200)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:404)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:430)
... 35 more
There's a very verbose log of what happens over here: http://pastebin.com/MMFKU26z
The line that says GET STUCK HERE represents the 5 minute waiting time
You are probably running into an issue with the Oracle JDBC driver which uses a blocking random number generator by default on Linux. Try running Java with the following argument:
-Djava.security.egd=file:/dev/./urandom
Alternatively you could start up a daemon to feed the random number generator. The Linux "rngd" daemon is an example.
Sources:
http://www.usn-it.de/index.php/2009/02/20/oracle-11g-jdbc-driver-hangs-blocked-by-devrandom-entropy-pool-empty/
http://bugs.java.com/view_bug.do;?bug_id=6521844
https://bugs.openjdk.java.net/browse/JDK-6202721
https://community.oracle.com/message/3701989
Oracle JDBC intermittent Connection Issue
Oracle getConnection slow
How to solve performance problem with Java SecureRandom?
Looking at your connection string, it seems that Oracle service name or SID is missing. The connection string should look like "jdbc:oracle:thin:#xxxx:yyyy/zzz", where zzz is the SID. You may also want to have a look at the answer to this question.
I have a transaction which runs for many hours in the Java Layer and the when Hibernate tries persist collated data, Exception Stack trace shown below is thrown.
Note: I have also tried the configurations specified at..
http://forums.mysql.com/read.php?39,52805,205216#msg-205216 and http://forums.mysql.com/read.php?39,52805,273371#msg-273371
++++++
ERROR org.hibernate.transaction.JDBCTransaction (JDBCTransaction.java:232) - Could not toggle autocommit
java.sql.SQLException: No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error:
** BEGIN NESTED EXCEPTION **
com.mysql.jdbc.CommunicationsException
MESSAGE: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.io.EOFException
STACKTRACE:
java.io.EOFException
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1903)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2349)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2860)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1571)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1666)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2972)
at com.mysql.jdbc.Connection.commit(Connection.java:2147)
at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:301)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:200)
at org.hibernate.transaction.JDBCTransaction.commitAndResetAutoCommit(JDBCTransaction.java:170)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:146)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:656)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:625)
MySQL has a default timeout period after which it terminates idle connections. By default the period is 8 hrs.
The DBCP however is unaware of when the connections are terminated by the MySQL server and so these stale connections are still handed out on demand to the application. Hence you get an Exception.
A fix is to add the following properties to your dbcp config:
validationQuery=”SELECT 1″
testOnBorrow=”true”
This tells DBCP to validate the connection with that query.
The connection was most likely dropped by the database. Specify a validation query on DBCP to ensure that the connections in the pool are still alive.
http://commons.apache.org/dbcp/configuration.html
We were having a very similar stacktrace at work when trying to index data, using Lucene, from our Spring application. The problem was the wait_timeout setting in my.cnf. We had just upgraded our MySQL installation and our DBA had forgotten to change wait_timeout from the default setting to what it was previously, 3600.