We Keep Coding

Result Set resource leak even though I am closing the RS?

I am getting 'Resource leak: 'rsHP' is not closed at this location' everywhere I use a rsHP = stmt.executeQuery(query);
Here is a basic layout of what this method does...
public static void method(String x, Connection conn){
Statement stmtHP = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_UPDATABLE);
ResultSet rsHP = null;
try{
----ALGORITHM IN HERE------
****This is the general form of this method*****
queryHP = "select * from SOMETABLE where SOMETHING = 'blah'";
rsHP = stmtHP.executeQuery(queryHP);
while(rsHP.next()){
List.add(rsHP.getString("COLNAME"));
}
.
.
repeats for 8 different queries
.
.
queryHP = "select * from SOMEOTHERTABLE where SOMETHINGELSE = 'blah2'";
rsHP = stmtHP.executeQuery(queryHP);
while(rsHP.next()){
List.add(rsHP.getString("NEWCOLNAME"));
}
}catch(Exception e){
System.out.println("Hey dumbo you suck, Exception Found");
rsHP.close();
stmtHP.close();
conn.close();
}finally{
rsHP.close();
stmtHP.close();
// connection gets closed later if no exceptions thrown
}
}// end method
At the end here I am clearly closing all my stuff. I am confused as to how I have a memory leak if it is literally impossible for my method to terminate without closing the RS outside of an error being thrown.

Connection#createStatement() throws an SQLException so this code would not compile at all.
I suggest you change the signature of the method to
public static void method(String x, Connection conn) throws SQLException
For the resource leak, I guess using the following logic will help you
try{
// code
rsHP.close();
conn.close();
}catch(Exception e){
// StackTrace
}finally{
if (rsHP != null) rsHP.close();
if (conn != null) conn.close();
}

What is the preferred method of ensuring that database objects are closed?

I'm curious about best practices when it comes to database interaction. I've been using a pattern that I believe handles making sure all of the appropriate objects are closed when I'm done with them. However, a coworker recently refactored my code with a comment along the lines of, "making sure we always close database objects". I need to know if one pattern is "better" than the other for some reason. Is the pattern that I've been using wrong somehow? Does one pattern have advantages over the other?
The pattern that I've been following:
public void doStuff() {
try {
final Connection connection = this.getConnection();
try {
final PreparedStatement ps = connection.prepareStatement("SELECT COLA, COLB FROM TBL WHERE COLC = ?");
try {
ps.setString(1, "asdf");
final ResultSet rs = ps.executeQuery();
try {
// get data from rs
} finally {
rs.close();
}
} finally {
ps.close();
}
} finally {
connection.close();
}
} catch (SQLException e) {
// do something with the error
}
}
The pattern that my coworker modifed my code to:
public void doStuff() {
Connection connection = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
connection = this.getConnection();
ps = connection.prepareStatement("SELECT COLA, COLB FROM TBL WHERE COLC = ?");
ps.setString(1, "asdf");
rs = ps.executeQuery();
// get data from rs
} finally {
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
// do something with the error
}
}
if (ps!= null) {
try {
ps.close();
} catch (SQLException e) {
// do something with the error
}
}
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
// do something with the error
}
}
}
}

If you're using Java 6 or prior, then use the latter because it's easier to read and maintain. Note that the latter can be improved with some refactoring to handle the cumbersome try-catch for every call to close method.
If you're using Java 7 or higher, then use try-with-resources:
try (Connection con = ...;
PreparedStatement pstmt = ...) {
pstmt.setXyz(...);
ResultSet rs = pstmt.executeQuery();
//read data from resultset
//and then close it
rs.close();
} catch (Exception e) {
//handle the exception properly...
}
In case you want to make sure about closing the ResultSet, you may use a nested try-with-resources:
try (Connection con = ...;
PreparedStatement pstmt = ...) {
pstmt.setXyz(...);
try(ResultSet rs = pstmt.executeQuery()) {
//read data from resultset
}
} catch (Exception e) {
//handle the exception properly...
}

The latter is easier to read; deep nesting is hard to reason about.
I prefer safe wrappers around closeables, e.g., they do nothing if the closeable is null. This also makes the mainline code easier to read.
Luigi's answer makes the most sense from Java 7 on, of course.

It's often simpler and cleaner to abstract the closure of your database resources to a dedicated manager object, which will contain any NPE's and such that might be thrown.
A pretty well written one exists as part of the open source project, OpenFire:
https://github.com/igniterealtime/Openfire/blob/master/src/java/org/jivesoftware/database/DbConnectionManager.java#L243
Sample helper method from this DbConnectionManager:
public static void closeResultSet(ResultSet rs) {
if (rs != null) {
try {
rs.close();
}
catch (SQLException e) {
Log.error(e.getMessage(), e);
}
}
}
So in your finally block you just pass your resources back into your manager, and it handles the ugly logic to test for nulls and catch exceptions, etc.
Like:
Connection con = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
con = DbConnectionManager.getConnection();
ps = con.prepareStatement(yourStatement);
rs = ps.executeQuery();
if (rs != null) {
while (rs.next()) {
// do stuff
}
}
} catch (SQLException e) {
LOG.error(e.getMessage(), e);
} finally {
DbConnectionManager.closeConnection(rs, ps, con);
}

Simulating Go's defer statement :D
try(Defer defer = new Defer())
{
Connection connection = ...;
defer.add( connection::close );
....
Path tmpFile = ...;
defer.add( ()->Files.delete(tmpFile) );
....
} // Defer.close() => executing registered actions, from last to first
How Defer is implemented is left as an exercise to readers:)

Cancel SQL Statement with JDBC

I am stuck with this problem here. I am running an application on my Tomcat Application Server. As a frontend I am using an HTML site with javascript in it, in the backend i am using Java.
When the user clicks on a button several sql queries are made, one after another. Now I want to provide the ability to cancel this query if the user wants to.
I already checked if my jdbc driver and the database are compatible for the cancel() method and this is just fine.
Here is my code :
PreparedStatement stmt = null;
public void runQuery(String query) {
Connection con = getConnection();
try {
stmt = con.prepareStatement(query);
stmt.execute();
} catch(SQLException e) {
e.printStackTrace();
} finally {
if(stmt != null && !stmt.isClosed()) {
stmt.close();
}
if(con != null) {
con.close();
}
}
}
public void cancelQuery() {
try {
if(stmt != null && !stmt.isClosed()) {
stmt.cancel();
}
} catch (SQLException e) {
e.printStackTrace();
}
}
So the user clicks the run button => runQuery is executed and stmt is initialized/overriden with the query which needs to execute.
Then the user clicks the cancel button => cancelQuery is executed.
Unfortunately I sometimes get a NullPointerException because stmt is null. But it should not even call cancelQuery if the stmt is null ?!
Here is the stacktrace:
Stacktrace:] with root cause
java.lang.NullPointerException
at com.sybase.jdbc3.jdbc.SybStatement.doCancel(SybStatement.java:646)
at com.sybase.jdbc3.jdbc.SybStatement.cancel(SybStatement.java:614)
at org.apache.tomcat.dbcp.dbcp2.DelegatingStatement.cancel(DelegatingStatement.java:269)
at org.apache.tomcat.dbcp.dbcp2.DelegatingStatement.cancel(DelegatingStatement.java:269)
at de.package.util.DBHelper.cancelQuery(DBHelper.java:82)
.....
Any idea why this keeps producing an exception ? How can I cancel the statement the right way ?
EDIT:
I had allook at the link in the comments and now running the cancel() method from a different thread. However the NullPointer still happens. This is how i call the cancel() method now:
public void cancelQuery() {
Thread thread = new Thread(new SQLCancelRunnable(stmt));
thread.start();
}
public class SQLCancelRunnable implements Runnable {
PreparedStatement stmt;
public SQLCancelRunnable(PreparedStatement stmt) {
this.stmt = stmt;
}
#Override
public void run() {
if(stmt != null) {
try {
System.out.println(stmt);
System.out.println(stmt.toString());
stmt.cancel();
System.out.println("canceled");
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
EDIT2
Found my answer the problem was the finally block of the runQuery() method. Because I closed statement & connection the NullPointer was thrown.
I now removed this block but this, of course, leads to huge resource leaking. Anyone who can guide me in the right direction how to close my resources properly ?

PreparedStatement stmt = null;
public void runQuery(String query) {
Connection con = getConnection();
try {
stmt = con.prepareStatement(query);
stmt.execute();
}
catch(SQLException e) {
e.printStackTrace();
}
finally {
if(stmt != null && !stmt.isClosed()) {
stmt.close();
}
if(con != null) {
con.close();
}
}
}
public void cancelQuery() {
try {
if(stmt != null && !stmt.isClosed()) {
stmt.cancel();
}
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
Try this. I have added a Generic Exception just after the SQLException.
Cannot say this is a very clean solution but it will ignore the null pointer exception which is possiblly raised by stmt.close() statement.

You'll need to synchronize statement closing:
public void runQuery(String query) {
...
try {
stmt = con.prepareStatement(query);
stmt.execute();
...
finally {
synchronized(this) {
if(stmt != null && !stmt.isClosed()) {
stmt.close();
}
}
}
}
public void cancelQuery() {
synchronized(this) {
if(stmt != null && !stmt.isClosed()) {
stmt.cancel();
}
}
}
between every statement another thread may execute some kind of code, so a simple if is not sufficient to be sure the state of the world is as you expect it to be.
Synchronizing on this may not be the best option, but as the stmt may be null we can't use this object.
Edit: If the code starting the query is asynchronous you'll also have to be prepared for calls to cancelQuery even before your statement is prepared.

You can use Statement.cancel()
As Java Docs Says
void cancel()
throws SQLException
Cancels this Statement object if both the DBMS and driver support
aborting an SQL statement. This method can be used by one thread to
cancel a statement that is being executed by another thread.
You can also set setQueryTimeout if query execution passes a threshhold time
java.sql.Statement.setQueryTimeout(seconds)
Update
Don't forget to Rollback the transation
Anyone who can guide me in the right direction how to close my
resources properly ?
this is what for finally block was invented
finally{
//Release All Resources
}
The finally block always executes when the try block exits. This
ensures that the finally block is executed even if an unexpected
exception occurs. But finally is useful for more than just exception
handling — it allows the programmer to avoid having cleanup code
accidentally bypassed by a return, continue, or break. Putting cleanup
code in a finally block is always a good practice, even when no
exceptions are anticipated.

You should take a look at Apache DB-Utils it makes this kind of problems disappear and you can simply write something like:
} finally {
DbUtils.closeQuietly(resutlSet);
DbUtils.closeQuietly(preparedStatement);
DbUtils.closeQuietly(connection);
}

What's wrong with my code Data source rejected establishment of connection, message from server: "Too many connections"

I am getting this error from the database
Data source rejected establishment of connection, message from
server: "Too many connections"
I am closing my connections properly, I don't know why i am getting this error when my thread pool is only 5 and i think i am closing the connections. Anyone know why.
Below is my code. I have changed a few things to stay anonymous
public class myMain {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int i = 0; i < (569743 / 100); i++) {
try {
MyClass mclass = new MyClass();
executor.execute(mclass);
} catch (Exception e) {
e.printStackTrace();
}
}
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Finished all threads");
}
}
.
public class MyClass implements Runnable {
public MyClass(){
Class.forName("com.mysql.jdbc.Driver");
connect = DriverManager.getConnection("jdbc:mysql://localhost/mytable?" + "user=&password=");
}
public void run() {
try {
String sql = "My select STATEMENT";
PreparedStatement stmt = connect.prepareStatement(sql);
ResultSet rs = stmt.executeQuery(sql);
while (rs.next()) {
try {
int id = rs.getInt("movie_id");
String title = rs.getString("title");
int year = rs.getInt("year");
String trailerUrl = rs.getString("trailerUrl");
if (trailerUrl == null || (trailerUrl != null && trailerUrl.length() == 0)) {
YoutubeTrailerFetcher youtube = new YoutubeTrailerFetcher(title, year);
String trailer = youtube.getVideoId();
String updateSql = "my UPDATE Statement";
Statement updateStm = connect.createStatement();
updateStm.executeUpdate(updateSql);
updateStm.close();
}
} catch (Exception e) {
e.printStackTrace();
}
}
connect.close();
stmt.close();
rs.close();
}catch(Exception e){
e.printStackTrace();
}
}
}

The main problem with your code is that you open a connection in the main thread, when calling the MyClass constructor, but the connection is only closed after the thread pool has finished executing the task. So you're trying to open 5697 connections, which is a whole lot. And these connections wait for being closed that one of the 5 threads in the pool has finished executing the task.
As I said in my comment: the method opening the connection should be the one closing it. And it should be closed in a finally block to ensure it is closed, whatever happens in the method, or better, you should use the try-with-resources statement to ensure that

Handling all exceptions when executing SQL in Java

There are many steps involved in executing one SQL statement in Java:
Create connection
Create statement
Execute statement, create resultset
Close resultset
Close statement
Close connection
At each of these steps SQLException can be thrown. If we to handle all exception and release all the resources correctly, the code will will look like this with 4 levels of TRY stacked on the top of each other.
try {
Connection connection = dataSource.getConnection();
try {
PreparedStatement statement = connection.prepareStatement("SELECT 1 FROM myTable");
try {
ResultSet result = statement.executeQuery();
try {
if (result.next()) {
Integer theOne = result.getInt(1);
}
}
finally {
result.close();
}
}
finally {
statement.close();
}
}
finally {
connection.close();
}
}
catch (SQLException e) {
// Handle exception
}
Can you propose a better (shorter) way to execute a statement while still release all the consumed resources?

If you are using Java 7, the try with resources statement will shorten this quite a bit, and make it more maintainable:
try (Connection conn = ds.getConnection(); PreparedStatement ps = conn.prepareStatement(queryString); ResultSet rs = ps.execute()) {
} catch (SQLException e) {
//Log the error somehow
}
Note that closing the connection closes all associated Statements and ResultSets.

Check out Apache Commons DbUtils, and in particular the closeQuietly() method. It will handle the connection/statement/result set closing correctly, including the cases where one or more are null.
An alternative is Spring JdbcTemplate, which abstracts a lot of work away from you, and you handle your database queries in a much more functional fashion. You simply provide a class as a callback to be called on for every row of a ResultSet. It'll handle iteration, exception handling and the correct closing of resources.

I create a utility class with static methods I can call:
package persistence;
// add imports.
public final class DatabaseUtils {
// similar for the others Connection and Statement
public static void close(ResultSet rs) {
try {
if (rs != null) {
rs.close();
}
} catch (Exception e) {
LOGGER.error("Failed to close ResultSet", e);
}
}
}
So your code would be:
Integer theOne = null;
Connection connection = null;
PreparedStatement statment = null;
ResultSet result = null;
try {
connection = dataSource.getConnection();
statement = connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
while (result.next()) {
theOne = result.getInt(1);
}
} catch (SQLException e) {
// do something
} finally {
DatabaseUtils.close(result);
DatabaseUtils.close(statement);
DatabaseUtils.close(connection);
}
return theOne;
I'd recommend instantiating the Connection outside this method and passing it in. You can handle transactions better that way.

Connection connection = null;
PreparedStatement statement = null;
ResultSet result = null;
try {
connection = dataSource.getConnection();
statement = connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
if (result.next()) {
Integer theOne = result.getInt(1);
}
}
catch (SQLException e) { /* log error */ }
finally {
if (result != null) try { result.close(); } catch (Exception e) {/*log error or ignore*/}
if (statement != null) try { statement.close(); } catch (Exception e) {/*log error or ignore*/}
if (connection != null) try { connection.close(); } catch (Exception e) {/*log error or ignore*/}
}

Just close the Connection, this releases all resources*. You don't need to close Statement and ResultSet.
*just make sure you don't have any active transactions.

Your code can be shortened and written in this way...
Connection connection = dataSource.getConnection();
PreparedStatement statement = null;
ResultSet result = null;
try {
statement= connection.prepareStatement("SELECT 1 FROM myTable");
result = statement.executeQuery();
if (result.next()) {
Integer theOne = result.getInt(1);
}
} catch (SQLException e) {
// Handle exception
} finally {
if(result != null) result.close();
if(statement != null) statement.close();
if(connection != null) connection.close();
}