We have one of the customer java websphere applications pointing to oracle database encountering connection close error while doing the plsql stored procedure execution. Sometimes after a few minutes into the stored procedure call and a few times after ~2 hours into execution. In a normal scenario, this job succeeds, within <10 seconds . So during failure time, it may have been waiting for some resource or other bottleneck that resulted in long execution time. So want to understand if anybody encountered such an error and what is the cause and fix of this?
The database is oracle version 11.2.0.4.
Below is the application log.
com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Connection is closed.
at com.ibm.ws.rsadapter.jdbc.WSJdbcWrapper.createClosedException(WSJdbcWrapper.java:122)
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.activate(WSJdbcConnection.java:2936)
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.setAutoCommit(WSJdbcConnection.java:3512)
We found below oracle doc matching the error but it seems if it would have been this bug then rerun also would have been failed. So our case might be different.
Protocol Violation and Connection is Closed exceptions When Using JDBC 12.1.0.2 with WebSphere (Doc ID 2359390.1)
I am currently using Spring's Mongo persistence layer for querying MongoDB. The collection I query contains about 4G of data. When I run the find code on my IDE it retrieves the data. However, when I run the same code on my server, it freezes for about 15 to 20 minutes and eventually throws the error below. My concern is that it runs without a hitch on my IDE running on my 4G Ram windows PC and fails on my 14G ram server. I have looked through the Mongo Log, and there's nothing there that points to the problem. I also assumed that the problem might be an environmental issue since it works on my local spring IDE, however the libraries on both my local pc are the same as the ones on my server. Has anyone had this kind of issue or can any one point me to what I'm doing wrong. Also weirdly, the find operation works when I revert to Mongo's java driver find methods.
I'm using mongo-java-driver - 2.12.1
spring-data-mongodb - 1.7.0.RELEASE
See below sample find operation code and error message.
List<HTObject> empObjects =mongoOperations.find(new Query(Criteria.where("date").gte(dateS).lte(dateE)),HTObject.class);
The exception I get is:
09:42:01.436 [main] DEBUG o.s.data.mongodb.core.MongoDbUtils - Getting Mongo Database name=[Hansard]
Exception in thread "main" org.springframework.dao.DataAccessResourceFailureException: Cursor 185020098546 not found on server 172.30.128.155:27017; nested exception is com.mongodb.MongoException$CursorNotFound: Cursor 185020098546 not found on server 172.30.128.155:27017
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:73)
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2002)
at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:1885)
at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:1696)
at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTempate.java:1679)
at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:598)
at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:589)
at com.sa.dbObject.TestDb.main(TestDb.java:74)
Caused by: com.mongodb.MongoException$CursorNotFound: Cursor 185020098546 not found on server 172.30.128.155:27017
at com.mongodb.QueryResultIterator.throwOnQueryFailure(QueryResultIterator.java:218)
at com.mongodb.QueryResultIterator.init(QueryResultIterator.java:198)
at com.mongodb.QueryResultIterator.initFromQueryResponse(QueryResultIterator.java:176)
at com.mongodb.QueryResultIterator.getMore(QueryResultIterator.java:141)
at com.mongodb.QueryResultIterator.hasNext(QueryResultIterator.java:127)
at com.mongodb.DBCursor._hasNext(DBCursor.java:551)
at com.mongodb.DBCursor.hasNext(DBCursor.java:571)
at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:1871)
... 5 more
In short
The MongoDB result cursor is not available anymore on the server.
Explanation
This can happen when using Sharding and a connection to a mongos fails over or if you run into timeouts (see http://docs.mongodb.org/manual/core/cursors/#closure-of-inactive-cursors).
You're performing a query that loads all objects into one list (mongoOperations.find). Depending on the result size, this may take a long time. Using an Iterator can help to leverage but even loading huge amounts using Iterators is limited at a certain point.
You should partition the results if you have to query very large data amounts using either paging (paging gets slower the more records you skip) or by querying with splits of your range (you have already a date range, so this could work).
I am attempting to add an Interbase connection pool to GlassFish v3 to use EJB 3.1 in a project. The glassfish log appears to be connecting to my database properly, it spits out all my table names and indices. However, I get an error
INFO: fetching database metadata
SEVERE: could not complete schema update
java.lang.NullPointerException
at interbase.interclient.ResultSet.local_Close(Unknown Source)
...
And when I ping the connection pool from within Glassfish I receive "Ping failed Exception - null".
I have the following properties set with my connectionpool:
resource type: javax.sql.DataSource
Datasource Classname: interbase.interclient.DataSource
portNumber: 3050
as well as my database info.
I can't seem to find information elsewhere. This question is similar but did not receive an answer.
thanks.
If the ping fails at the connection pool level then things are very likely not going to work. Any chances to use another database? If yes, really do it because Interbase seems to be an outdated product and I'm not very confident with the quality of available JDBC drivers (drivers listed in this page are all so old and dusty). It looks like time has been suspended with J2EE 1.3... If not, then maybe try another driver (the Firebird one for example).
Update: As pointed out by Craig in a comment, InterBase is maintained by Embarcadero and includes a Type 4 JDBC Driver. But, still, the Borland/CodeGear/Embarcadero products are IMHO on a dying trend.
This question already has answers here:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
(51 answers)
Closed 6 years ago.
My program that connects to a MySQL database was working fine. Then, without changing any code used to set up the connection, I get this exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
What happened?
The code used to get the connection:
private static Connection getDBConnection() throws SQLException, InstantiationException, IllegalAccessException, ClassNotFoundException {
String username = "user";
String password = "pass";
String url = "jdbc:mysql://www.domain.com:3306/dbName?connectTimeout=3000";
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection(url, username, password);
return conn;
}
This is a wrapped exception and not really interesting. It is the root cause of the exception which actually tells us something about the root cause. Please look a bit further in the stacktrace. The chance is big that you'll then face a SQLException: Connection refused or SQLException: Connection timed out.
If this is true in your case as well, then all the possible causes are:
IP address or hostname in JDBC URL is wrong.
Hostname in JDBC URL is not recognized by local DNS server.
Port number is missing or wrong in JDBC URL.
DB server is down.
DB server doesn't accept TCP/IP connections.
Something in between Java and DB is blocking connections, e.g. a firewall or proxy.
To solve the one or the either, follow the following advices:
Verify and test them with ping.
Refresh DNS or use IP address in JDBC URL instead.
Verify it based on my.cnf of MySQL DB.
Start it.
Verify if mysqld is started without the --skip-networking option.
Disable firewall and/or configure firewall/proxy to allow/forward the port.
By the way (and unrelated to the actual problem), you don't necessarily need to load the JDBC driver on every getConnection() call. Just only once during startup is enough.
check your wait timeout set on the DB server.
Some times it defaults to 10 seconds. This looses the connection in 10 seconds.
mysql> show global variables like '%time%' ;
update it make it something like 28800
mysql> SET GLOBAL wait_timeout = 28800;
I've been having this issue also for about 8-9 days.
Here's some background: I'm developing a simple Java application that runs in bash.
Details:
Spring 2.5.6
Hibernate3.2.3.ga
With maven.
(The base of the project is from mkyong.com , the spring tutorial without anotations )
MySQL version:
[jvazquez#archbox ~]$ mysql --version
mysql Ver 14.14 Distrib 5.5.9, for Linux (i686) using readline 5.1
Linux archbox 2.6.37-ARCH #1 SMP PREEMPT Fri Feb 18 16:58:42 UTC 2011 i686 Intel(R) Core(TM)2 Quad CPU Q8200 # 2.33GHz GenuineIntel GNU/Linux
The application works fine in Arch Linux, Mac OS X 10.6, and FreeBSD 7.2.
When I moved the jar file to another arch linux in a different host, using the same mysql, a similar my.cnf, and the similar kernel version, the connection died and obtained the same error as the original poster:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure
I tried every possible combination for this that I found on so and the forums (http://forums.mysql.com/read.php?39,180347,180347#msg-180347 for example, which is closed now and I can't post .. ), specifically:
Triple check that I wasn't using skip
networking. (verified with ps aux
and the my.cnf)
Tried enable log_warnings=1 in the my.cnf but obviously, I wasn't hitting the
server so I didn't saw anything while using the app
SHOW ENGINE innodb STATUS didn't show anything at all; during the tests I could connect via shell, and php also connected to the mysql server
/etc/hosts has localhost 127.0.0.1
Tried the jdbc properties using localhost and 127.0.0.1 with no results
Tried adding c3p0 and changed the max_wait
Max connections in the my.cnf was changed to 900 , 2000 and still nothing my.cnf
Added wait_timeout = 60 my.cnf
Added net_wait_timeout = 360 my.cnf
Added the destroy-method="close" spring.xml
As it was pointed out (if you look up for the same exception , you will find several so threads about the issue Reproduce com.mysql.jdbc.exceptions.jdbc4.CommunicationsException with a setup of Spring, hibernate and C3P0
for example ).
If you are using tomcat, please check the security exception (again, it is on SO, you will find it )
Check that you can resolve that url that you are using
Try adding c3p0.
Verify that there isn't a firewall rejecting your connections
Finally , if you are using GNU/Linux ( ARch linux for example and you indeed obtain this exception )
Try
MySQL Forums :: JDBC and Java :: EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost
If the link get's removed, just add mysqld:ALL to /etc/hosts.allow
I know that is a bit extense, but it may help anybody using GNU/Linux and having this exception and this thread seemed the best place to post my research.
Hope it helps
I got the same error
but then I figured it out its because the Mysql server is not running at that time.
So to change the status of the server
Go to Task Manager
Go to Services
then search for your Mysql server(eg:for my case its MYSQL56)
then you will see under the status column it says its not running
by right clicking and select start
Hope this will help.
We have a piece of software (webapp with Tomcat) using Apache commons connection pooling, and worked great for years. In the last month I had to update the libraries due to an old bug we were encountering. The bug had been fixed in a recent version.
Shortly after deploying this, we started getting exactly these messages. Out of the thousands of connections we'd get a day, a handful (under 10, usually) would get this error message. There was no real pattern, except they would sometimes cluster in little groups of 2 to 5.
I changed the options to on the pool to validate the connection every time one is taken from or put back in the pool (if one is found bad, a new one is generated instead) and the problem went away.
Have you updated your MySQL jar lately? It seems like there may be a new setting that didn't used to be there in our (admittedly very old) jar.
I agree with BalusC to try some other options on your config, such as those you're passing to MySQL (in addition to the connection timeout).
If this failure is transient like mine was, instead of permanent, then you could use a simple try/catch and a loop to keep trying until things succeed or use a connection pool to handle that detail for you.
Other random idea: I don't know what happens why you try to use a closed connection (which exception you get). Could you be accidentally closing the connection somewhere?
Ensure skip-networking is commented out in my.cnf/my.ini
As BalusC mentioned, it would be very useful to post the full stacktrace (always post a full stacktrace, it is useless and frustrating to have only the first lines of a stacktrace).
Anyway, you mentioned that your code was working fine and that this problem started suddenly to occur without any code change so I'm wondering if this could be related to you other question Problem with not closing db connection while debugging? Actually, if this problem started while debugging, then I think it is (you ran out of connections). In that case, restart you database server (and follow the suggestions of the other question to avoid this situation).
I encountered same problem. I am using spring & dbcp & mysql 5.5But If I change localhost to 192.168.1.110 then everything works. What make things more weird is mysql -h localhost just works fine.
update: Finally found a solution. Changing bindaddress to localhost or 127.0.0.1 in my.conf will fix the problem.
In my case, the local loopback interface wasn't started, so "localhost" couldn't be resolved.
You can check this by running "ifconfig" and you should see an interface called "lo". If it is not up, you can activate it by running "ifup lo" or "ifconfig lo up".
In my case, the mysql.com downloaded Connector/J 5.1.29 .jar had this error whereas the 5.1.29 .jar downloaded from the MvnRepository did not.
This happened when building a Google appengine application in Android Studio (gradle, Windows x64), communicating to a Linux MySQL server on the local network/local VM.
I see you are connecting to a remote host. Now the question is what type of a network are you using to connect to the internet?
WINDOWS
If it's a mobile broadband device then get your machines IP address and add it to your hosting server so that your host server can allow connections coming from your machine.[your host might have turned this off due to security reasons].
Note that every time you use a different network device your IP changes.
If you are using a LAN then set a static IP address on your machine then add it to your host.
I hope this helps!! :)
I got the communications failure error when using a java.sql.PreparedStatement with a specific statement.
This was running against MySQL 5.6, Tomcat 7.0.29 and JDK 1.7.0_67 on a Windows 7 x64 machine.
The cause turned out to be setting an integer to a string parameter and a string to an integer parameter then trying to perform executeQuery on the prepared statement. After I corrected the order of parameter setting the statement performed correctly.
This had nothing to do with network issues as the wording of the error message suggested.
The escential problem is that Mysql JDBC pool connections is not used, then the Timeout from Mysql, close the Connections. You need change the pool Parameters to get restart connection when the connection has failures, on this way:
Connection Validation: Required (Check)
Validation Method: autocommit
You can change the Validation Method if you cannot get it works!
If you use WAMP, make sure it is online. What I did was, first turned my firewall off, then it worked, so after that I allowed connection for all local ports, specially port 80. Than I got rid of this problem. For me it was the Firewall who was blocking the connection.
I had the same problem and I used most of the params (autoreconnect etc..), but didn't try the (test_on_idle, or test_on_connect) , I am going to do them next.
However, I had this hack that got me through this:
I have a cron job called Healthcheck, It wakes up every 10 mins and makes a REST API call to the server. The web / app server picks this up, connects to the db, makes a small change and comes back with a 'yes all quiet on western front' or 'shitshappening'. When the latter, it sends a pager / email to the right people.
It has the side effect of always keeping the db connection pool fresh. So long as this cron is running, I don't have the db connection timeout issues. otherwise, they crop up.
I am having a problem with Java DB that I just don't know how to resolve. I am creating a DB and connecting to it using Java DB's native JDBC driver. If I relocate that database physically and try to connect to it using its new path, I consistently get XJ004 errors:
ERROR XJ004: Database 'blahblah' not found.
I am sure I am using the correct connection string. Is there any possibility the DB is somehow getting corrupted? Or is there some encoding of the DB path in the DB such that if you relocate a Java DB it gets confused?
I'm really at a loss here. :( Please help!
Jim
Have you verified that this error message isn't also used when there's no listener on the host machine ... and were you using JavaDB on your local machine before the relocation? Many database systems (and I'm not that familiar with JavaDB) ship set-up to only allow connections from localhost for security reasons. On PostgreSQL for instance, you have to allow TCP connections and bounce the daemon to obtain a remote connection.
Anyway ... since the problem started when you when remote, look for issues related to that first! (And if you can run your application on the remote machine, does that work?)
There must be a file named derby.log somewhere. Check the error there. If it is not detailed enough, try setting derby.stream.error.logSeverityLevel to a lower value. See the manual for more information.