We Keep Coding

Mysql exception from execute timeout

I have the following java code fragment:
Connection conn = DriverManager.getConnection(connString, user, password);
Statement stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);
stmt.setMaxRows(100);
stmt.setQueryTimeout(2);
try {
stmt.execute("SELECT SLEEP(100)"); // (1)
} finally {
stmt.close();
conn.close();
}
It is designed to simulate a long running query that will time out. However, it raises an exception that is not the expected one:
Streaming result set com.mysql.jdbc.RowDataDynamic# is still
active. No statements may be issued when any streaming result sets are
open and in use on a given connection. Ensure that you have called
.close() on any active streaming result sets before attempting more
queries. java.sql.SQLException: Streaming result set
com.mysql.jdbc.RowDataDynamic# is still active. No
statements may be issued when any streaming result sets are open and
in use on a given connection. Ensure that you have called .close() on
any active streaming result sets before attempting more queries.
Looking closer, it appears the error happens as the close method attempts to reset the MaxRows to its default value. However, if I replace line (1) with
stmt.executeQuery("SELECT SLEEP(100)");
The proper exception, com.mysql.jdbc.exceptions.MySQLTimeoutException is raised.
I would like to run execute, stream the results, have a limit on the number of rows, and not have this error crop up. Any suggestions?

warning messages with hikaricp

I am using hikaricp as my database connection pool. I am closing the connection when I am done with my SQL statement by calling close on the connection which I believe you should return the connection proxy back to the pool . Yet, I see the following warning (not error) message and I have to wonder whether its an issue that needs to be addressed because I'm not cleaning up my connection resources properly. I am not using a try with resources but using a try catch finally (I close the connection in the finally & I realize that in certain conditions, the finally may actually not get called but it is being called in my case). any thoughts?
2014-09-15 13:59:26,083 WARN c.z.h.p.LeakTask [Hikari Housekeeping Timer (pool wmHikariCp)] Connection leak detection triggered, stack trace follows java.lang.Exception
at com.abc.test.DBConnPool.getConnection(DBConnPool.java:71)
at com.abc.test.TestProj.callDB(TestApp.java:30)
at com.abc.test.TestProj.main(TestApp.java:81)

Can you confirm whether you are indeed releasing the connection back to the pool? Can you use try with resources (in addition to increasing threshold as Brett mentioned):
try (Connection conn = DBConnectionPool.getConnection();
PreparedStatement ps = conn.prepareStatement(preparedQuery);) {
and the resultset (because some drivers/databases may not clean out resultset when connection is cleaned though not sure if this is still valid for new drivers/databases but I'm not sure what you are using):
try (ResultSet rs = ps.executeQuery();) {
Hope it helps.

There is a possibility that you are merely using the connection longer than the leak detection timeout. The leak detection is a threshold for how long a connection is out of the pool. Try increasing the threshold to a longer timeout.

Closing connection in BoneCP

We have BoneCP library managing our connection pools. I'd like to know if the following statement will return the connection back to the pool.
statement = conn.createStatement();
....
lots of code.
....
Connection conn = statement.getConnection();
statement.close();
conn.close();
Will the above code close the connection and put the connection back into the pool?
Update:
I'm asking this question because when I run the statistics on connectionpool, i still see the conPool.getTotalLeased() showing that 2 connections are being used. But, I've closed the connections using the above mechanism.

The whole sense of a pool is to hold already established connections to your database. When you retrieve a connection from your pool you save the time to connect to your database.
What you are seeing is the pool holding your connections so it is all fine.
When you close your connection, it is only returned to the pool and marked as available for your next retrieval.

Yes it does moves the connection back to the pool. There was a mistake when getting the connection from the pool, fixed it, now i'm not seeing the 2 connections in the totalLeased() method.
Mistake that I found
conPool.getConnection(); // loitering connection which we see in the getTotalLeased();
..
Statement st = conPool.getConnection().getStatement(); //we have handle to this connection.
return st;

MySQL "No operations allowed after statement closed"

I'm running into a strange situation with a prepared statement hitting a MySQL database using MySQL Connector/J. In certain environments, I periodically have issues with longer existing (> 5 minutes) prepared statements. I frequently get an exception when calling executeBatch that reads:
"No operations allowed after statement closed"
However, there is no code that could be closing the statement that I can see. The code looks something like the following:
private void execute(MyClass myObj, List<MyThing> things) throws SQLException {
Connection con = null;
PreparedStatement pstmt = null;
try {
con = ConnectionHelper.getConnection();
pstmt = con.prepareStatement(INSERT_SQL);
int c = 0;
for (MyThing thing : things) {
pstmt.setInt(1, myObj.getA());
pstmt.setLong(2, thing.getB());
pstmt.addBatch();
if (++c % 500 == 0) {
pstmt.executeBatch();
}
}
pstmt.executeBatch();
}
finally {
ConnectionHelper.close(pstmt, con);
}
}
ConnectionHelper.close essentially just calls close on the statement and the connection. ConnectionHelper.getConnection is a bit of a rabbit hole -- it roughly retrieves a connection from a pool using java.sql.DriverManager and proxool, then wraps it with Spring DataSourceUtils.
Usually it will fail on the last pstmt.executeBatch(), but will sometimes fail in other places. I've checked and wait_timeout and interactive_timeout are configured to defaults (definitely > 5 minutes). Moreover, in most cases, the connection and statement are used in the loop, but the a few seconds later the statement fails outside of the loop. The DB server and the app server are running on the same subnet, so network issues seem unlikely.
Looking for any tips on how to debug this issue -- at the moment, I'm trying to dig in to the MySQL Connector/J code to see if I can somehow get some additional debugging statements out. Unfortunately I can't attach a debugger, as it can only be reproduced in a select couple environments at the moment.

Take a look at the line:
if (++c % 500 == 0) {
pstmt.executeBatch();
}
What happens if that gets executed, but the loop terminates. Then you call pstmt.executeBatch again with nothing in the batch.

JDBC Query excecution

I am facing an issue while executing queries.I use the same resultSet and statement for excecuting all the queries.Now I face an intermittent SQlException saying that connection is already closed.Now we have to either have separate resultSet for each query or have lock like structure.Can anyone tell which is better.I think introducing locks will slow down the process.Am I right?
Update:
To be more clear.The error may happen because the finally block gets called before all the queries get executed and the connection gets closed and exception will be thrown.
This is the exception I get
java.sql.SQLException: Connection has
already been closed. at
weblogic.jdbc.wrapper.PoolConnection.checkConnection(PoolConnection.java:81)
at
weblogic.jdbc.wrapper.ResultSet.preInvocationHandler(ResultSet.java:68)
at
weblogic.jdbc.wrapper.ResultSet_com_informix_jdbc_IfxResultSet.next(Unknown
Source) at
com.test.test.execute(test.java:76)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:413)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:225)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1858)
at
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:459)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1077)
at
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:7047)
at
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at
weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3902)
at
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2773)
at
weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at
weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
Sample code:
ResultSet rst=null;
Statement stmt=null;
Connection con=DBConnection.getConnection();
stmt=con.createStatement();
rst=stmt.executeQuery("select * from dual");
while(rst.next())
{ : ://Some code }
rst=stmt.executeQuery("select * from doctor where degree="BM");
while(rst.next())
{ //blah blah }
finally
{
//close con,rst and stmt
}

you are not reusing the resultset, you are leaking resultsets.
rst=stmt.executeQuery... generates a new resultset and the previous resultset is never closed :(

It appears that the code in question has issues in multi-threaded environment.
DBConnection.getConnection() is probably returning the same connection to all threads. When multiple threads are processing multiple requests, the first thread that finishes execution of the method will close the connection, leaving all other threads high and sundry.
I'm speculating here, but is appears that the connection object returned by DBConnection is an instance member of the DBConnection object, and that would qualify as a bad practice for a connection manager in a multi-threaded environment.
A code fix would avoid the usage of instance members for Connection, Statement (and the like), and the ResultSet objects.

I'm not sure what's going on without knowing more about your code. Is it threaded ? Is the underlying database going down (or are you losing connectivity to it).
One thing I would do is to implement connection pooling (via Apache DBCP, say). This framework will maintain a pool of connections to your database and validate these connections before handing them out to you. You would ask for a new connection each time you make a query (or perhaps set of queries) but because they're pooled this shouldn't be a major oeverhead.

Unless your connection to the database has really been closed I think you did something more like this:
try {
return resultSet.getBoolean("SUCCESS");
} finally {
resultSet.close();
}
This code will actually close the connection before your result set is being evaluated, resulting in the exception you show.