We Keep Coding

Java abstract class for dynamic JDBC requests

I am currently reworking an ADF Fusion application, that uses a lot of Java nested in Beans to actually manage JDBC requests. As the code ermerged from the pre-Java8 era there is a bunch of deprecated technologies in it and I neither have the time nor the knowledge to rework everything (which describes the percentage of the code that is outdated and hard to debug).
Something very regularly is that inside the backing bean classes manual JDBC requests with our inhouse DB are handled (often uncannily nested in other methods). As I began to outsource them I realized I wrote the same block of code over and over again:
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
try {
conn = CC.getConn(); //CC is of type "CustomConnection",
//a static assist class that fetches the connection
stmt = conn.createStatement();
rs = stmt.executeQuery("Some SQL");
while(rs.next()) {
//handle the result
}
} catch(Exception e){
e.printStackTrace();
} finally {
try {
rs.close();
stmt.close();
conn.close();
} catch(Exception e){
e.printStackTrace();
}
}
or for PreparedStatement respectively:
Connection conn = null;
PreparedStatement pstmt = null;
ResultSet rs = null;
try {
conn = CC.getConn(); //CC is of type "CustomConnection",
//a static assist class that fetches the connection
pstmt = conn.prepareStatement("Some SQL");
//populate the pstmt with params
rs = pstmt.executeUpdate();
while(rs.next()) {
//handle the result
}
} catch(Exception e){
e.printStackTrace();
} finally {
try {
rs.close();
pstmt.close();
conn.close();
} catch(Exception e){
e.printStackTrace();
}
}
While I'm aware that this is not the best practice it worked reliable so far but writing multiple methods like this with real difference only in the handling of the ResultSet became very tedious. So my approach was to write an abstract superclass that provides a request() method and let the extending classes define the parameters to populate a PreparedStatement and the handling of the ResultSet.
public abstract class Requestable {
public void request(String SQL, HashMap<String, Integer> args) {
Connection conn = null;
PreparedStatement pstmt = null;
try {
conn = CC.getConn();
pstmt = conn.prepareStatement(SQL);
pstmt = fill(args);
onResponse(pstmt.executeUpdate());
//handle result
} catch(Exception e) {
e.printStackTrace();
} finally {
try {
pstmt.close();
conn.close();
} catch(Exception e) {
e.printStackTrace();
}
}
}
public abstract PreparedStatement fill(HashMap args);
public abstract void onResponse(ResultSet rs);
public Requestable() {
super();
}
}
This would be an example for PreparedStatement. Statements would get an own separate method.
Writing this draft, I came across the issue that some classes which are intended to extend Requestable currently have multiple different requests that they perform (which all need a specific handling of the result). With my approach, I could define the methods fill() and onResponse() only once per class. Is there a way to like pass a function reference to request() that gets defined in the extending class and executed at the position of fill() and onResponse()?

Can return connection object inside a try with resources

I have connection provider class as bleow to return connection.
public class ConnectionProvider {
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static Connection ConnectDB() throws ClassNotFoundException, SQLException {
try (Connection connection = DriverManager
.getConnection("jdbc:mysql://localhost:3306/jspservlet_test","root", "root");
) {
return connection;
}
}
}
Here is main method to call connection provider.
public void Test() {
try {
Connection con = ConnectionProvider.ConnectDB();
PreparedStatement ps = con.prepareStatement("");
} catch (SQLException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
But "com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed." error are always show at below line of code.
PreparedStatement ps = con.prepareStatement("");
Because, according to Oracle documentation, If use try with resources java 7 features, resources are auto close after try block even it's errors occurred or not. So even I returned the connection it's already closed.
Let me know, my usage logic is wrong?
How can I return this connection inside try with resource?
I tried many time googling for solution but does not get convenience answers for me.
Let me know your suggestion and feedback please.

What you can't do...
With a try-with-resources as you have it after you return the connection you return(d) is close(d). You can't return the connection from inside the try with resources.
What you can do...
Pass the connection (inside your try-with-resources) to a method that takes a connection. You can also use a ConnectionPool, and get the Connection when you need it (to create and execute a query).

Let me know, my usage logic is wrong?
The usage of 'try-with-resources' logic is wrong in this context, because the intention of ConnectDB() is to return a connection instance which could be actually used by the caller to send a SQL statement, but instead, the connection instance is getting auto-closed, before it could be used by the caller, because of using 'try-with-resources' construct of Java.

Quick how-to on try-with-resource and JDBC
Your ConnectionProvider's ConnectDB already declares it is throwing SQLException - so no need to catch it in here: (You should consider replacing this code with connection pool maybe)
public class ConnectionProvider {
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static Connection ConnectDB() throws SQLException {
return DriverManager.getConnection("jdbc:mysql://localhost:3306/jspservlet_test","root", "root");
}
}
Instead use try-with-resource in your test-class to clean up your code and focus on errors your SQL code
might have:
public void Test() {
try (Connection con = ConnectionProvider.ConnectDB();
PreparedStatement ps = con.prepareStatement("SELECT 1")) {
//Prepare your Statement
ps.setInt(1, 1);
//And another try-with-resource for the result - note the closing brace
try(ResultSet rs = ps.executeQuery()) {
while(rs.next()) {
//Handle your Result
System.out.println(rs.getString(1));
}
} // This closes try-with-resource. Exception will be rethron to be caught in outer catch!
} catch (SQLException e) {
//SQL is Broken - but only ONE catch to catch them all
e.printStackTrace();
}
}
That way you gain
Better readability for your code (no calls to close surrounded by finally and if != null)
Centralized error handling if anything in your SQL code breaks (so you can focus on functional error of "statement didn't run")
Better code quality: No need to worry about Cursors, Statements, Connections not being propery closed.

JAVA JDBC reusing connections

I have a Java program in which I am doing some JDBC for select queries. Will it be advisable to call testDataBase() each time which inturns calls DBConnection() each time or I should reuse one connection for all the queries. Thanks in advance.
private void testDataBase(String query){
Connection con = DBConnection();
Statement st = null;
ResultSet rs = null;
try {
st = con.createStatement();
rs = st.executeQuery(query);
boolean flag = true;
while (rs.next()) {
String resultString = "";
for(int i = 1; i <=rs.getMetaData().getColumnCount();i++){
resultString=resultString+" "+ rs.getString(i);
}
System.out.println(resultString);
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
if (st != null) {
try {
st.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (con != null) {
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
private Connection DBConnection() {
final String method_name = "DBConnection";
Connection conn = null;
try{
Class.forName(driver).newInstance();
conn = java.sql.DriverManager.getConnection(url,userName,password);
}catch (ClassNotFoundException e) {
System.out.println(e.getMessage());
} catch (SQLException e) {
System.out.println(e.getMessage());
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return conn;
}

Opening a DB connection is an expensive operation in terms of perfofmance. You should use a ConnectionPool for sharing connections among different requests.

Connections are not thread safe, so sharing them across requests is not a good idea.
A better idea is to pool connections and keep their scope as narrow as possible: check the connection out of the pool, use it, close it in transaction scope.

Database connections are long-running and should be re-used, unless you have a very low query rate.

Getting a database connection is quite an expensive operation, so it is advisable to re-use a connection if possible. Consider also using connection pooling, which will maintain a number of connections for you, so you can just grab one from the pool when needed. The method shown above might not need to change, it depends on the DBConnection() method you call.

I completely agree with #Amir Kost, in terms of performances, opening a DB connection in one of the slowest operation that you can do, and if you have restrictive real time constraints it could be a big issue.
I do not know if you are using a framework or not, but a good practice is to publish a bean which wrap a pool of connection and every time that you need to interact directly with the db, you get the current open connection (which usually corresponds to a so called "session").
I suggest to you, (even if you are not using any framework) to reproduce this technicality.

If you want only one instance of Connection, you can make use of the Singleton pattern, you can consider :
public class Connector {
private static final String URL = "jdbc:mysql://localhost/";
private static final String LOGIN = "root";
private static final String PASSWORD = "azerty";
private static final String DBNAME = "videotheque";
private static Connector connector;
private static Connection connection;
private Connector() {
}
public synchronized static Connector getInstance() {
if (connector == null) {
connector = new Connector();
}
return connector;
}
public static Connection getConnection() {
if (connection == null) {
Connection c = null;
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
c = DriverManager.getConnection(URL + DBNAME, LOGIN, PASSWORD);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return c;
}
return connection;
}
}
And then, you can call : Connector.getInstance().getConnection()

Returning a ResultSet

I am trying to create a method from where I can query my database and retrieve a whole table.
Currently, it works just fine if I use the data inside the method. However, I want the method to return the results.
I'm getting a java.sql.SQLException: Operation not allowed after ResultSet closed on the current code.
How can I achieve this?
public ResultSet select() {
con = null;
st = null;
rs = null;
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM biler");
/*
if (rs.next()) {
System.out.println(rs.getString("model"));
}*/
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
return rs;
}

You should never pass a ResultSet around through public methods. This is prone to resource leaking because you're forced to keep the statement and the connection open. Closing them would implicitly close the result set. But keeping them open would cause them to dangle around and cause the DB to run out of resources when there are too many of them open.
Map it to a collection of Javabeans like so and return it instead:
public List<Biler> list() throws SQLException {
Connection connection = null;
PreparedStatement statement = null;
ResultSet resultSet = null;
List<Biler> bilers = new ArrayList<Biler>();
try {
connection = database.getConnection();
statement = connection.prepareStatement("SELECT id, name, value FROM Biler");
resultSet = statement.executeQuery();
while (resultSet.next()) {
Biler biler = new Biler();
biler.setId(resultSet.getLong("id"));
biler.setName(resultSet.getString("name"));
biler.setValue(resultSet.getInt("value"));
bilers.add(biler);
}
} finally {
if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
if (statement != null) try { statement.close(); } catch (SQLException ignore) {}
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return bilers;
}
Or, if you're on Java 7 already, just make use of try-with-resources statement which will auto-close those resources:
public List<Biler> list() throws SQLException {
List<Biler> bilers = new ArrayList<Biler>();
try (
Connection connection = database.getConnection();
PreparedStatement statement = connection.prepareStatement("SELECT id, name, value FROM Biler");
ResultSet resultSet = statement.executeQuery();
) {
while (resultSet.next()) {
Biler biler = new Biler();
biler.setId(resultSet.getLong("id"));
biler.setName(resultSet.getString("name"));
biler.setValue(resultSet.getInt("value"));
bilers.add(biler);
}
}
return bilers;
}
By the way, you should not be declaring the Connection, Statement and ResultSet as instance variables at all (major threadsafety problem!), nor be swallowing the SQLException at that point at all (the caller will have no clue that a problem occurred), nor be closing the resources in the same try (if e.g. result set close throws an exception, then statement and connection are still open). All those issues are fixed in the above code snippets.

If you don't know what you want of the ResultSet on retrieving time I suggest mapping the complete thing into a map like this:
List<Map<String, Object>> resultList = new ArrayList<Map<String, Object>>();
Map<String, Object> row = null;
ResultSetMetaData metaData = rs.getMetaData();
Integer columnCount = metaData.getColumnCount();
while (rs.next()) {
row = new HashMap<String, Object>();
for (int i = 1; i <= columnCount; i++) {
row.put(metaData.getColumnName(i), rs.getObject(i));
}
resultList.add(row);
}
So basically you have the same thing as the ResultSet then (without the ResultSetMetaData).

Well, you do call rs.close() in your finally-block.
That's basically a good idea, as you should close all your resources (connections, statements, result sets, ...).
But you must close them after you use them.
There are at least three possible solutions:
don't close the resultset (and connection, ...) and require the caller to call a separate "close" method.
This basically means that now the caller needs to remember to call close and doesn't really make things easier.
let the caller pass in a class that gets passed the resultset and call that within your method
This works, but can become slightly verbose, as you'll need a subclass of some interface (possibly as an anonymous inner class) for each block of code you want to execute on the resultset.
The interface looked like this:
public interface ResultSetConsumer<T> {
public T consume(ResultSet rs);
}
and your select method looked like this:
public <T> List<T> select(String query, ResultSetConsumer<T> consumer) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery(query);
List<T> result = new ArrayList<T>();
while (rs.next()) {
result.add(consumer.consume(rs));
}
} catch (SQLException ex) {
// logging
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
return rs;
}
do all the work inside the select method and return some List as a result.
This is probably the most widely used one: iterate over the resultset and convert the data into custom data in your own DTOs and return those.

As everyone before me said its a bad idea to pass the result set. If you are using Connection pool library like c3p0 then you can safely user CachedRowSet and its implementation CachedRowSetImpl. Using this you can close the connection. It will only use connection when required. Here is snippet from the java doc:
A CachedRowSet object is a disconnected rowset, which means that it makes use of a connection to its data source only briefly. It connects to its data source while it is reading data to populate itself with rows and again while it is propagating changes back to its underlying data source. The rest of the time, a CachedRowSet object is disconnected, including while its data is being modified. Being disconnected makes a RowSet object much leaner and therefore much easier to pass to another component. For example, a disconnected RowSet object can be serialized and passed over the wire to a thin client such as a personal digital assistant (PDA).
Here is the code snippet for querying and returning ResultSet:
public ResultSet getContent(String queryStr) {
Connection conn = null;
Statement stmt = null;
ResultSet resultSet = null;
CachedRowSetImpl crs = null;
try {
Connection conn = dataSource.getConnection();
stmt = conn.createStatement();
resultSet = stmt.executeQuery(queryStr);
crs = new CachedRowSetImpl();
crs.populate(resultSet);
} catch (SQLException e) {
throw new IllegalStateException("Unable to execute query: " + queryStr, e);
}finally {
try {
if (resultSet != null) {
resultSet.close();
}
if (stmt != null) {
stmt.close();
}
if (conn != null) {
conn.close();
}
} catch (SQLException e) {
LOGGER.error("Ignored", e);
}
}
return crs;
}
Here is the snippet for creating data source using c3p0:
ComboPooledDataSource cpds = new ComboPooledDataSource();
try {
cpds.setDriverClass("<driver class>"); //loads the jdbc driver
} catch (PropertyVetoException e) {
e.printStackTrace();
return;
}
cpds.setJdbcUrl("jdbc:<url>");
cpds.setMinPoolSize(5);
cpds.setAcquireIncrement(5);
cpds.setMaxPoolSize(20);
javax.sql.DataSource dataSource = cpds;

You can use the CachedRowSet object that is just for what you want:
public CachedRowSetImpl select(String url, String user, String password) {
CachedRowSetImpl crs = null;
try (Connection con = DriverManager.getConnection(url, user, password);
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("SELECT * FROM biler");) {
crs = new CachedRowSetImpl();
crs.populate(rs);
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQL.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
return crs;
}
You can read the documentation here:
https://docs.oracle.com/javase/7/docs/api/javax/sql/rowset/CachedRowSet.html

You're closing the ResultSet and consequently you can't use it anymore.
In order to return the contents of the table, you'll have to iterate through the ResultSet and build a per-row representation (in a List, perhaps?). Presumably each row represents some entity, and I would create such an entity for each row.
while (rs.next()) {
list.add(new Entity(rs));
}
return list;
The alternative is to provide some callback object, and your ResultSet iteration would call on that object for each ResultSet row. That way you don't need to build an object representing the whole table (which may be a problem if it's sizable)
while (rs.next()) {
client.processResultSet(rs);
}
I would be reluctant to let clients close the result set/statement/connection. These need to be managed carefully to avoid resource leaks, and you're much better off handling this in one place (preferably close to where you open them!).
Note: You can use Apache Commons DbUtils.closeQuietly() to simply and reliably close the connect/statement/resultset tuple (handling nulls and exceptions properly)

It is bad practice to return result set ,secondly you are already closing it so after closing it you can not use it anymore.
I would suggest using Java 7 with multiple resource in try block will helpful you as suggested above.
If you want entire table result ,you should return its output rather than resultSet.

Assuming you can afford storing the entire result in memory, you may simply return some table-like structure. Using Tablesaw for instance, simply do
Table t = Table.read().db(rows);
with rows a standard java.sql.ResultSet. For details see here. Tablesaw becomes especially useful if you intend to slice-and-dice your data further as it gives you Pandas-like functionality.

Problem with static variables in java

I am using static variables pretty much heavily in my application. Now after the application status is finished I am facing a problem in garbage collection. The variables that are declares as static are never garbage collected and my memory runs out quickly.
The specific problem is on mysql connection. I am storing the connection variable in a static variable and so I don't have to open the connection every time I run a query. This leads to a problem of usage of all memory every time I use the connection variable to execute the query and the used memory is not released. Is it a good idea to store the connection variable in static variable ? when I tried to open and close the connection every time without static variable I solved the memory management problem but the responsiveness of the application is slowed down by 10 to 20 times.
Do you need more information to understand this problem ? If yes please ask me without down voting. Thanks!
EDIT
This is my connector class
import java.sql.*;
public class connect {
public Connection conn = null;
public connect() {
try {
if (conn == null) {
String userName = "root";
String password = "password";
String url = "jdbc:mysql://localhost/pos?zeroDateTimeBehavior=convertToNull";
Class.forName("com.mysql.jdbc.Driver").newInstance();
conn = DriverManager.getConnection(url, userName, password);
System.out.println("Database connection established");
}
} catch (Exception e) {
System.err.println("Cannot connect to database server");
}
}
}
This is my class where i am storing the connection
public class variables {
public static connect con = new connect();
}
And this method i use to execute the query
public class mysql_query {
public static ResultSet execute_mysql(Connection con, String sqlStatement) {
try {
//ResultSet result = null;
java.sql.Statement cs = con.createStatement();
ResultSet result = cs.executeQuery(sqlStatement);
return result;
} catch (SQLException ex) {
Logger.getLogger(mysql_query.class.getName()).log(Level.SEVERE, null, ex);
return null;
}
}
public static void main(String args[]){
String sql = "SELECT * FROM pos_user_login WHERE moderator='1' AND "
+ "company_id='1'";
ResultSet rs = execute_mysql(variables.con.conn, sql);
}
}

Just an idea: You might not be closing your ResultSet and Statement objects, correctly. If you don't do that, the MySQL JDBC driver might keep a hold on many resources that you don't need anymore. Especially ResultSet can be very painful, as some parts of the database cursor are still in memory.
An example to give you an idea is this:
PreparedStatement stmt = null;
ResultSet rs = null;
try {
stmt = connection.prepareStatement(...);
rs = stmt.executeQuery();
}
// Close your resources in a finally block! Because the finally block
// is executed even if you have exceptions in the try block.
// If you do this a lot of times, write utility methods...
finally {
try {
if (rs != null) {
rs.close();
}
} catch (SQLException ignore) {}
try {
if (stmt != null) {
stmt.close();
}
} catch (SQLException ignore) {}
}

Maybe it'd be better to look at using a connection pool rather than the static variable... Connection pools maintain a bunch of open connections and serve them out when they're needed. Should solve your performance problem and your memory problem.

a static variable will not garbage collected but if you are just storing a some connection data it should not be a problem. What are you exactly storing?
Matteo

Well, judging by what you say, you have an object (let's call it Obj) class which contains the static variable with the connection. Due to you creates a new Obj each time and you stores it at that moment, I think you are doing a lot of copies of the connection which the JVM is unable to clean because they are static.
You could consider the possibility of store this kind of information in a Model class, or remove the static mark in order to let the JVM collect this objects properly.