We Keep Coding

HSQLDB never closes DB connection

I'm playing with this kind of database, and I've tried to close the HSQLDB connection after I used it, but it's still opened at the end.
Code:
//----This methods are in a specific connection class file
public static Connection conn = null;
public static Connection getConnection(){
try {
input = new FileInputStream("PathToMyPropertiesFile");
prop.load(input);
//The properties constants are correctly checked
Class.forName(prop.getProperty("DRIVER_HSQLDB"));
conn = DriverManager.getConnection(prop.getProperty("CONN_HSQLDB"));
}
catch(ClassNotFoundException | SQLException e) {
LOG.log(null,"Error: "+e);
}
catch (IOException ex) {
LOG.log(null,"FILE ERROR: "+ex);
}
finally {
if (input != null) {
try {
input.close();
} catch (Exception e) {
LOG.log(null,"CLOSE ERROR: "+e);
}
}
}
return conn;
}
public static boolean stopConn() {
try {
if(conn != null) {
conn.close();
System.err.println("\nCLOSE CONN\n"+conn);
return true;
}
}
catch (SQLException e) {
e.printStackTrace();
return false;
}
return false;
}
//========= the other class file with the methods to use the conneciton
public static boolean insertUser(String uName, String uEmail){
Connection con;
con = ConnectionDB.getConnection();
PreparedStatement ps = null;
try {
String consulta = "insert into USERS (\"NICK\",\"EMAIL\") VALUES(?,?);";
ps = con.prepareStatement(consulta);
System.err.println(ps);
ps.setString(1,uName);
ps.setString(2,uEmail);
System.err.println("\nASSIGNATION\n"+ps);
if(ps.executeUpdate() == 1) {
System.err.println("\nTRUE\n");
return true;
}
}
catch(SQLException e) {
e.printStackTrace();
}
finally {
try {
System.err.println("\nFINALLY\n"+ps);
if(ps != null) {
ps.close();
System.err.println("\nCLOSE PS\n"+ps);
}
if(con != null) {
con.close();
System.err.println("\nCLOSE CON\n"+con);
if(ConnectionDB.stopConn()) {
System.err.println("\nALL IS OK\n"+ConnectionDB.conn);
}
else {
System.err.println("\nMEEEEKKKK!!!\n"+ConnectionDB.conn);
}
}
}
}
return false;
}
The console give me this results, and I don't know why never the connection is closed because I tried to close it twice. If someone has an idea please tell me.
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[null], [null]]]
ASSIGNATION
org.hsqThis is my cldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[extra], [extra#mail.com]]]
TRUE
FINALLY
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[extra], [extra#mail.com]]]
CLOSE PS
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[closed]
CLOSE CON
org.hsqldb.jdbc.JDBCConnection#3e5b87f5
CLOSE CONN
org.hsqldb.jdbc.JDBCConnection#3e5b87f5
ALL IS OK
org.hsqldb.jdbc.JDBCConnection#3e5b87f5

Closing a JDBC connections does not close an in-process database. This allows you to open and close different connections during the runtime of your application.
You need to execute a JDBC Statement to shutdown the database. The SQL statement to execute is "SHUTDOWN".
It is possible to add a connection property "shutdown=true" to the JDBC connection URL to force a quick shutdown when the last connection to the in-process database is closed. But this is mainly useful for readonly or test databases. A full SHUTDOWN allows the database to open quickly the next time a connection is made.
See the Guide http://hsqldb.org/doc/2.0/guide/running-chapt.html#rgc_inprocess

Java - Tomcat GC won't free up.. always cause crash, can't find any memory leak

I have few issues with my tomcat application.
I'm using a linux server with 1024M memory.
I deploy my app on the tomcat, and everything working great.
Recently i notice that the 'Tenured Gen' heap memory not free up when it should...
It reach 99% and then crash tomcat..
I check my application with VisualVM, and the same result.
it fill up the memory and the 'Old Gen' never free up.
This is when the application run few minutes with no request:
IMG: Everything looks normal
And when I start to send requests with 200 Thread on a loop
this what happened:
IMG: all memories are full
So then I check data on the MAT, and this is my result:
IMG: look like a memory leak
IMG: Problem with the sql jdbc?
IMG: Can't understand what is wrong
this is my ConnectionPool class:
public class ConnectionPool {
private static ConnectionPool singleton = null;
private ArrayList<Connection> freeConnections;
private ArrayList<Connection> allConnections;
private int MAX_CONNECTIONS = 1;
private final String shema = "Topic";
private final String url = "jdbc:mysql://localhost:3306/"+shema+"? autoReconnect=true&useSSL=false";
private final String username = "root";
private final String password = "password";
public static ConnectionPool getInstance(){
if (singleton == null)
{
synchronized (ConnectionPool.class) {
if (singleton == null){
System.out.println("ConnectionPool get instance");
try {
singleton = new ConnectionPool();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
} //this will invoke the constructor
}
}
}
return singleton;
}
private ConnectionPool() throws Exception {
freeConnections = new ArrayList<Connection>();
allConnections = new ArrayList<Connection>();
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
for (int i = 0; i < MAX_CONNECTIONS; i++) {
try {
addNewConnection();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
private void addNewConnection() throws SQLException {
try {
Connection conn = DriverManager.getConnection(url, username, password);
freeConnections.add(conn);
allConnections.add(conn);
} catch (SQLException e) {
throw e;
}
}
public Connection getConnection()
{
while (true) {
synchronized (freeConnections) {
if (freeConnections.size() != 0) { // free connection is available
Connection conn = freeConnections.get(0);
freeConnections.remove(0);
try {
conn.setAutoCommit(true);
} catch (SQLException e) {
e.printStackTrace();
}
return conn;
}
try {
freeConnections.wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
public void returnConnection(Connection conn)
{
if (null == conn) { // ignore invalid value
return;
}
if (!allConnections.contains(conn)) {
return;
}
synchronized (freeConnections) {
if (freeConnections.contains(conn)) {
return;
}
freeConnections.add(conn);
freeConnections.notifyAll();
return;
}
}
public void closeAllconnections()
{
for (Connection conn : allConnections) {
try {
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
System.out.println("ConnectionPool all connection closed");
deregisterDriver();
}
public void deregisterDriver() {
try {
java.sql.Driver driver = DriverManager.getDriver(url);
DriverManager.deregisterDriver(driver);
} catch (SQLException e) {
e.printStackTrace();
}
System.out.println("ConnectionPool deregister driver");
}
}
Please help me to understand what is wrong and explain me.
1.Why the GC won't free up or why he can't do his job?
2.Is something wrong with my ConnectionPool Class?
3.why tomcat not saying anything about OutOfMemoryException in my logs(just crashing)?

See the connection details, apparently you have 10 connections and each retains cca 66 MB if the heap summing up to 660 MB required RAM.
I don't know what data you select, however when returning a connection you may want to close all resultsets and statements (why are you creating your own pool? dbcp, c3p0 or commons-pool ain't good enough? for learning?) and seems it may be not enough. I really don't know what the pool implementations do to release all resources properly.
And seems it is not so straightforward to share open connections between multiple threads Java MySQL JDBC Memory Leak so I would suggest to use a working pool solution (dbcp)

JAVA JDBC reusing connections

I have a Java program in which I am doing some JDBC for select queries. Will it be advisable to call testDataBase() each time which inturns calls DBConnection() each time or I should reuse one connection for all the queries. Thanks in advance.
private void testDataBase(String query){
Connection con = DBConnection();
Statement st = null;
ResultSet rs = null;
try {
st = con.createStatement();
rs = st.executeQuery(query);
boolean flag = true;
while (rs.next()) {
String resultString = "";
for(int i = 1; i <=rs.getMetaData().getColumnCount();i++){
resultString=resultString+" "+ rs.getString(i);
}
System.out.println(resultString);
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
if (st != null) {
try {
st.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (con != null) {
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
private Connection DBConnection() {
final String method_name = "DBConnection";
Connection conn = null;
try{
Class.forName(driver).newInstance();
conn = java.sql.DriverManager.getConnection(url,userName,password);
}catch (ClassNotFoundException e) {
System.out.println(e.getMessage());
} catch (SQLException e) {
System.out.println(e.getMessage());
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return conn;
}

Opening a DB connection is an expensive operation in terms of perfofmance. You should use a ConnectionPool for sharing connections among different requests.

Connections are not thread safe, so sharing them across requests is not a good idea.
A better idea is to pool connections and keep their scope as narrow as possible: check the connection out of the pool, use it, close it in transaction scope.

Database connections are long-running and should be re-used, unless you have a very low query rate.

Getting a database connection is quite an expensive operation, so it is advisable to re-use a connection if possible. Consider also using connection pooling, which will maintain a number of connections for you, so you can just grab one from the pool when needed. The method shown above might not need to change, it depends on the DBConnection() method you call.

I completely agree with #Amir Kost, in terms of performances, opening a DB connection in one of the slowest operation that you can do, and if you have restrictive real time constraints it could be a big issue.
I do not know if you are using a framework or not, but a good practice is to publish a bean which wrap a pool of connection and every time that you need to interact directly with the db, you get the current open connection (which usually corresponds to a so called "session").
I suggest to you, (even if you are not using any framework) to reproduce this technicality.

If you want only one instance of Connection, you can make use of the Singleton pattern, you can consider :
public class Connector {
private static final String URL = "jdbc:mysql://localhost/";
private static final String LOGIN = "root";
private static final String PASSWORD = "azerty";
private static final String DBNAME = "videotheque";
private static Connector connector;
private static Connection connection;
private Connector() {
}
public synchronized static Connector getInstance() {
if (connector == null) {
connector = new Connector();
}
return connector;
}
public static Connection getConnection() {
if (connection == null) {
Connection c = null;
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
c = DriverManager.getConnection(URL + DBNAME, LOGIN, PASSWORD);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return c;
}
return connection;
}
}
And then, you can call : Connector.getInstance().getConnection()

Java 1.6 java.sql try/catch/finally crazy coding

I'm currently creating a lot of classes that will access database using a connection pool.
So I get a connection, create a statement and get the result set. (I can't use Java 1.7 and the fantastic Automatic Resource Management)
When finishing my method I must finish with a finally block:
if (rs != null) {
try {
rs.close();
} catch (SQLException sqle) {
logger.error("Couldn't close result set", sqle);
}
}
if (st != null) {
try {
st.close();
} catch (SQLException sqle) {
logger.error("Couldn't close statement", sqle);
}
}
if (connection != null) {
try {
connection.close();
} catch (SQLException sqle) {
logger.error("Couldn't close connection", sqle);
}
}
I'm already seeing the nightmare it will be for XX classes having 4/5 methods each.
Would it be good practice to make an helper class which would got a special close method for each object type like :
public static void closeResource(Connection connection) {
if (connection != null) {
try {
connection.close();
} catch (SQLException sqle) {
logger.error("Couldn't close connection", sqle);
}
}
And then just doing my usual finally with xx.close(connection);xx.close(statement);xx.close(rs);
Or in the same thinking (I know at this point I'll shock some people as I myself find that a bit ackward), having a method like public static void closeResources(Object... obj) {} with an awful list of instanceof ?
Or in your experience, coding the whole thing everywhere is better ?

Use Apache commons project : http://commons.apache.org/dbutils/apidocs/org/apache/commons/dbutils/DbUtils.html
DbUtils.closeQuietly() is probably what you need

Use overloading.
private void close(ResultSet rSet) throws SQLException {
if (rSet != null) {
rSet.close();
}
}
private void close(Statement statement) throws SQLException {
if (statement != null) {
statement.close();
}
}
private void close(Connection conn) throws SQLException {
if (conn != null) {
conn.close();
}
}
Usage will be much cleaner now:
try {
// do db stuff
} catch (Exception e) {
logger.error("log it", e);
} finally {
close(rs);
close(cs);
close(conn);
}

Just one more example. Suitable for simple small projects.
Object doRequest() throws SQLException {
PreparedStatement ps = ... // initialize statement
try {
ResultSet rs = ps.executeQuery();
try {
// use ResultSet
return someResult;
} finally {
rs.close();
}
} finally {
ps.close();
}
}
Although it is not pretends to be complete solution (many nested try-finally are quite unreadable), there are several advantages:
Method itself not deals with exception handling. Often only caller may decide what to do with exception.
As follows, method always returns correct result or throws exception. No magic "error values" required.
Resources closed only if they were initialized. No need to check for null berode close().

You could also exploit the fact that for every class you want to close, the close method has no args, and make a reflective helper method like this:
public static final void tryClose(Object o){
if(o != null){
Method[] m = o.getClass().getMethods();
for (Method method : m) {
if("close".equals(method.getName())){
if(!method.isAccessible()) method.setAccessible(true);
try {
method.invoke(o);
} catch (Exception e) {
System.err.println(e);
}
break;
}
}
}
}
EDIT: Tested with FileWriter, works fine in my machine.

Try-catch-finally and then again a try catch

I have often come across situations like :-
try{
...
stmts
...
}
catch(Exception ex) {
...
stmts
...
} finally {
connection.close // throws an exception
}
which still needs a try - catch block inside finally.
What is the best practice to overcome this?

Write a SQLUtils class that contains static closeQuietly methods that catch and log such exceptions, then use as appropriate.
You'll end up with something that reads like this:
public class SQLUtils
{
private static Log log = LogFactory.getLog(SQLUtils.class);
public static void closeQuietly(Connection connection)
{
try
{
if (connection != null)
{
connection.close();
}
}
catch (SQLExcetpion e)
{
log.error("An error occurred closing connection.", e);
}
}
public static void closeQuietly(Statement statement)
{
try
{
if (statement!= null)
{
statement.close();
}
}
catch (SQLExcetpion e)
{
log.error("An error occurred closing statement.", e);
}
}
public static void closeQuietly(ResultSet resultSet)
{
try
{
if (resultSet!= null)
{
resultSet.close();
}
}
catch (SQLExcetpion e)
{
log.error("An error occurred closing result set.", e);
}
}
}
And your client code will be something like:
Connection connection = null;
Statement statement = null;
ResultSet resultSet = null;
try
{
connection = getConnection();
statement = connection.prepareStatement(...);
resultSet = statement.executeQuery();
...
}
finally
{
SQLUtils.closeQuietly(resultSet);
SQLUtils.closeQuietly(statment);
SQLUtils.closeQuietly(connection);
}
Update: since Java 7, the various JDBC interfaces extend java.lang.AutoCloseable and while the above code answers the original question, if you're writing code directly against the JDBC API, it can now be structured:
try (
Connection connection = getConnection();
PreparedStatement statement = connection.prepareStatement(...);
ResultSet resultSet = statement.executeQuery()
)
{
...
}

As others have mentioned, a static closeQuietly utility is the way to go. One thing to add - if you are in the world of java.io rather than java.sql then there is a useful interface for exactly this purpose - java.io.Closeable
All the data sources and sinks in java.io implement this interface - all streams, channels, writers and readers. That way you can create a single utility to cope with the same "exception on close()" issue without requiring many overloaded versions.
e.g.
public class IoUtils {
public static closeQuietly (Closeable closeable) {
try {
closeable.close();
} catch (IOException logAndContinue) {
...
}
}
}

I usually did it this way:
try {
try {
..
stmts
...
}
finally {
connection.close():
}
} catch (Exception ex) {
..
stmts
..
}
I usually only used this when I wasn't using a library that took care of this plumbing for me.
As Imagist points out, this isn't technically the same as the finally will run before the catch but I think it solves the problem you were trying to solve.

Commons-io also has closeQuietly() for in and output streams. I'm using it all the time. It makes your code much more readable.

In Java 10 you can write:
public void java10() throws SQLException {
try (var connection = Connections.openConnection();
var callableStatement = connection.prepareCall("my_call");
var resultSet = callableStatement.executeQuery()) {
while (resultSet.next()) {
var value = resultSet.getString(1);
System.out.println(value);
}
}
}
In Java 7, 8 and 9 you can write:
public void java7() throws SQLException {
try (Connection connection = Connections.openConnection();
CallableStatement callableStatement = connection.prepareCall("my_call");
ResultSet resultSet = callableStatement.executeQuery()) {
while (resultSet.next()) {
String value = resultSet.getString(1);
System.out.println(value);
}
}
}
In Java 6 you need to write all these lines:
public void java6() throws SQLException {
Connection connection = Connections.openConnection();
try {
CallableStatement callableStatement = connection.prepareCall("my_call");
try {
ResultSet resultSet = callableStatement.executeQuery();
try {
while (resultSet.next()) {
String value = resultSet.getString(1);
System.out.println(value);
}
} finally {
try {
resultSet.close();
} catch (Exception ignored) {
}
}
} finally {
try {
callableStatement.close();
} catch (Exception ignored) {
}
}
} finally {
try {
connection.close();
} catch (Exception ignored) {
}
}
}

Don't hesitate use one more try ... catch inside finally.

Generally you don't want to do anything more than log an exception which happens when closing a resource, so it should really go in its own try/catch. However, this is generic code that will happen often, so Don't Repeat Yourself, and put the close in a static method (as Nick Holt suggests) that way you won't have the two try/catch items in the same method, making the code easier to read and follow.

There is also handy Closeables#closeQuitely method in Google Guava library - it can be used for any Closeable

Can we have try block followed by finally bock and catch block later to that?

just remember .. finally always get execute either with try or catch ..