Traditional DB singleton connection works poorly - java

I am using singleton database connection inside my java application, here is code of my connection manager class:
public abstract class DatabaseManager {
//Static instance of connection, only one will ever exist
private static Connection connection = null;
private static String dbName="SNfinal";
//Returns single instance of connection
public static Connection getConnection(){
//If instance has not been created yet, create it
if(DatabaseManager.connection == null){
initConnection();
}
return DatabaseManager.connection;
}
//Gets JDBC connection instance
private static void initConnection(){
try{
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
String connectionUrl = "jdbc:sqlserver://localhost:1433;" +
"databaseName="+dbName+";integratedSecurity=true";
DatabaseManager.connection =
DriverManager.getConnection(connectionUrl);
}
catch (ClassNotFoundException e){
System.out.println(e.getMessage());
System.exit(0);
}
catch (SQLException e){
System.out.println(e.getMessage());
System.exit(0);
}
catch (Exception e){
}
}
public static ResultSet executeQuery(String SQL, String dbName)
{
ResultSet rset = null ;
try {
Statement st = DatabaseManager.getConnection().createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,ResultSet.CONCUR_READ_ONLY);
rset = st.executeQuery(SQL);
//st.close();
}
catch (SQLException e) {
System.out.println(e.getMessage());
System.exit(0);
}
return rset;
}
public static void executeUpdate(String SQL, String dbName)
{
try {
Statement st = DatabaseManager.getConnection().createStatement();
st.executeUpdate(SQL);
st.close();
}
catch (SQLException e) {
System.out.println(e.getMessage());
System.exit(0);
}
}
}
The problem is my code work perfect at the start but when time past it becomes really slow. What caused that problem and how can i fix that?
At starting time my application handles around 20 queries per second, after 1 hour of running it reaches to 10 queries per second and after 3 days of running it reaches to 1 query per 10 seconds!!
P.S: My application is a single user application that makes many queries through database.
P.S: Here is my JVM parameters in eclipse.ini:
--launcher.XXMaxPermSize
512M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
512m
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.6
-Xms500m
-Xmx4G
-XX:MaxHeapSize=4500m
Unfortunately database is remote and I have not any monitoring access to it for finding out what is going on there.
Here is the example of my usage:
String count="select count(*) as counter from TSN";
ResultSet rscount=DatabaseManager.executeQuery(count, "SNfinal");
if(rscount.next()) {
numberofNodes=rscount.getInt("counter");
}

What caused that problem and how can i fix that?
The main problem that you have here is in the executeQuery() method.
You are not closing the Statement, I suppose that you have commented the line st.close() because you need the ResultSet open
for further processing.
I can see that your idea is to avoid see duplicate JDBC code in your application, but this is not the right approach.
The rule is: close the ResultSet and after that, close the Statement,
otherwise you are not releasing resources correctly and you expose to the kind of problem that you are describing.
Here you can find a good explanation about how to close resources correctly (take in mind that in your case you donĀ“t need
to close the connection)
Edit:
An example could be
try{
Statement st = DatabaseManager.getConnection().createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,ResultSet.CONCUR_READ_ONLY);
ResultSet rsCount = st.executeQuery(count); //count="select count(*) as counter from TSN";
if(rsCount.next()) {
numberofNodes=rscount.getInt("counter");
}
} catch (SQLException e) {
//log exception
} finally {
rsCount.close();
st.close();
}

You should consider using a disconnected resultset like a CachedRowSet http://docs.oracle.com/javase/1.5.0/docs/api/javax/sql/rowset/CachedRowSet.html
public static ResultSet executeQuery(String SQL, String dbName)
{
CachedRowSetImpl crs = new CachedRowSetImpl();
ResultSet rset = null ;
Statement st = null;
try {
st = DatabaseManager.getConnection().createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,ResultSet.CONCUR_READ_ONLY);
rset = st.executeQuery(SQL);
crs.populate(rset);
}
catch (SQLException e) {
System.out.println(e.getMessage());
System.exit(0);
}finally{
rset.close();
st.close();
}
return crs;
}
CachedRowSet implements ResultSet so it should behave like a ResultSet.
http://www.onjava.com/pub/a/onjava/2004/06/23/cachedrowset.html
In addition to these changes, I would recommend you use a pooled datasource to get connections and close them instead of holding on to one open connection.
http://brettwooldridge.github.io/HikariCP/
Or if you arent java7, bonecp or c3po.
EDIT:
To answer your question, this solves your problem because CachedRowSetImpl doesnt stay connected to the database while in use.
This allows you to close your Resultset and Statement after you've populated the CachedRowSetImpl.
Hope that answers your question.

Although Connection Manager would close Statement and Resultset automatically, but it would be better if you close them immediately.
There's nothing else in your code will effect your single thread task, so I bet there must be something wrong in your database. Try to find out if there's any database locking or wrong column index. And also have a look at database query status, find out where's the bottleneck.

Related

Can return connection object inside a try with resources

I have connection provider class as bleow to return connection.
public class ConnectionProvider {
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static Connection ConnectDB() throws ClassNotFoundException, SQLException {
try (Connection connection = DriverManager
.getConnection("jdbc:mysql://localhost:3306/jspservlet_test","root", "root");
) {
return connection;
}
}
}
Here is main method to call connection provider.
public void Test() {
try {
Connection con = ConnectionProvider.ConnectDB();
PreparedStatement ps = con.prepareStatement("");
} catch (SQLException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
But "com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed." error are always show at below line of code.
PreparedStatement ps = con.prepareStatement("");
Because, according to Oracle documentation, If use try with resources java 7 features, resources are auto close after try block even it's errors occurred or not. So even I returned the connection it's already closed.
Let me know, my usage logic is wrong?
How can I return this connection inside try with resource?
I tried many time googling for solution but does not get convenience answers for me.
Let me know your suggestion and feedback please.
What you can't do...
With a try-with-resources as you have it after you return the connection you return(d) is close(d). You can't return the connection from inside the try with resources.
What you can do...
Pass the connection (inside your try-with-resources) to a method that takes a connection. You can also use a ConnectionPool, and get the Connection when you need it (to create and execute a query).
Let me know, my usage logic is wrong?
The usage of 'try-with-resources' logic is wrong in this context, because the intention of ConnectDB() is to return a connection instance which could be actually used by the caller to send a SQL statement, but instead, the connection instance is getting auto-closed, before it could be used by the caller, because of using 'try-with-resources' construct of Java.
Quick how-to on try-with-resource and JDBC
Your ConnectionProvider's ConnectDB already declares it is throwing SQLException - so no need to catch it in here: (You should consider replacing this code with connection pool maybe)
public class ConnectionProvider {
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static Connection ConnectDB() throws SQLException {
return DriverManager.getConnection("jdbc:mysql://localhost:3306/jspservlet_test","root", "root");
}
}
Instead use try-with-resource in your test-class to clean up your code and focus on errors your SQL code
might have:
public void Test() {
try (Connection con = ConnectionProvider.ConnectDB();
PreparedStatement ps = con.prepareStatement("SELECT 1")) {
//Prepare your Statement
ps.setInt(1, 1);
//And another try-with-resource for the result - note the closing brace
try(ResultSet rs = ps.executeQuery()) {
while(rs.next()) {
//Handle your Result
System.out.println(rs.getString(1));
}
} // This closes try-with-resource. Exception will be rethron to be caught in outer catch!
} catch (SQLException e) {
//SQL is Broken - but only ONE catch to catch them all
e.printStackTrace();
}
}
That way you gain
Better readability for your code (no calls to close surrounded by finally and if != null)
Centralized error handling if anything in your SQL code breaks (so you can focus on functional error of "statement didn't run")
Better code quality: No need to worry about Cursors, Statements, Connections not being propery closed.

Is this open/close JDBC connection code ok?

I'm not sure if this code is correct. I get it from an example of my Java course but I see that in fact it never closes the connection and the exceptions doesn't look to be catched correctly. (I call the query methods from my business tier).
public class Persistence {
static final String JDBC_DRIVER = "com.mysql.jdbc.Driver";
static final String DB_URL = "jdbc:mysql://localhost/myDB";
static final String USER = "user";
static final String PASS = "pass";
private static Connection con;
static {
openConnection();
}
private static boolean openConnection() {
try {
Class.forName(JDBC_DRIVER).newInstance();
con = DriverManager.getConnection(DB_URL, USER, PASS);
return true;
} catch (InstantiationException ex) {
ex.printStackTrace();
return false;
} catch (IllegalAccessException ex) {
ex.printStackTrace();
return false;
} catch (ClassNotFoundException e) {
e.printStackTrace();
return false;
} catch (SQLException e) {
System.err.println("SQL problem: " + e.getMessage());
return false;
}
}
//----------EXAMPLE QUERY-----------
public static String someQuery() throws SQLException {
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("SELECT column FROM myDB");
String data;
while (rs.next()) {
data = rs.getString("column");
}
rs.close();
st.close();
return data;
}
}
Should I open and close the connection inside every query method and delete the "static{}" expression?
Like this? (still not sure about the exceptions):
public static String someQuery() throws SQLException {
openConnection();
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("SELECT column FROM myDB");
String data;
while (rs.next()) {
data = rs.getString("column");
}
rs.close();
st.close();
con.close();
return data;
}
Thanks for your answers.
The static block is only executed once, so you open a single connection and then keep it open for the duration of the program.
That does work but is flawed for a number of reasons. For example as soon as you start multi-threading it is completely useless.
Your second example is better but still flawed, the best approach would be:
Use a connection pool to keep the connections open, request one from the pool when you need it.
Use a try-finally block to ensure you always close the connection and/or return it to the pool when done.
You do not need to create a new instance of JDBC Driver class
1) Change this to Class.forName(JDBC_DRIVER).newInstance() to Class.forName(JDBC_DRIVER)
You just need to register the class with the JVM(which involves intialization of static variables and blocks)
2)Try creating connection thread pool as suggested above
3)Use a try with resources block for Connection,Statement and ResultSet as all the three interfaces extend java.lang.AutoCloseable.As such,your resources are always closed without you having to write boiler plate code.Also,the exception in your business logic does not get masked by any exception occuring while closing a resource in finally block.
But of course you know JDK7 for that.These were implemnted as part of Project Coin.Just google it and you will get more information

Problem with static variables in java

I am using static variables pretty much heavily in my application. Now after the application status is finished I am facing a problem in garbage collection. The variables that are declares as static are never garbage collected and my memory runs out quickly.
The specific problem is on mysql connection. I am storing the connection variable in a static variable and so I don't have to open the connection every time I run a query. This leads to a problem of usage of all memory every time I use the connection variable to execute the query and the used memory is not released. Is it a good idea to store the connection variable in static variable ? when I tried to open and close the connection every time without static variable I solved the memory management problem but the responsiveness of the application is slowed down by 10 to 20 times.
Do you need more information to understand this problem ? If yes please ask me without down voting. Thanks!
EDIT
This is my connector class
import java.sql.*;
public class connect {
public Connection conn = null;
public connect() {
try {
if (conn == null) {
String userName = "root";
String password = "password";
String url = "jdbc:mysql://localhost/pos?zeroDateTimeBehavior=convertToNull";
Class.forName("com.mysql.jdbc.Driver").newInstance();
conn = DriverManager.getConnection(url, userName, password);
System.out.println("Database connection established");
}
} catch (Exception e) {
System.err.println("Cannot connect to database server");
}
}
}
This is my class where i am storing the connection
public class variables {
public static connect con = new connect();
}
And this method i use to execute the query
public class mysql_query {
public static ResultSet execute_mysql(Connection con, String sqlStatement) {
try {
//ResultSet result = null;
java.sql.Statement cs = con.createStatement();
ResultSet result = cs.executeQuery(sqlStatement);
return result;
} catch (SQLException ex) {
Logger.getLogger(mysql_query.class.getName()).log(Level.SEVERE, null, ex);
return null;
}
}
public static void main(String args[]){
String sql = "SELECT * FROM pos_user_login WHERE moderator='1' AND "
+ "company_id='1'";
ResultSet rs = execute_mysql(variables.con.conn, sql);
}
}
Just an idea: You might not be closing your ResultSet and Statement objects, correctly. If you don't do that, the MySQL JDBC driver might keep a hold on many resources that you don't need anymore. Especially ResultSet can be very painful, as some parts of the database cursor are still in memory.
An example to give you an idea is this:
PreparedStatement stmt = null;
ResultSet rs = null;
try {
stmt = connection.prepareStatement(...);
rs = stmt.executeQuery();
}
// Close your resources in a finally block! Because the finally block
// is executed even if you have exceptions in the try block.
// If you do this a lot of times, write utility methods...
finally {
try {
if (rs != null) {
rs.close();
}
} catch (SQLException ignore) {}
try {
if (stmt != null) {
stmt.close();
}
} catch (SQLException ignore) {}
}
Maybe it'd be better to look at using a connection pool rather than the static variable... Connection pools maintain a bunch of open connections and serve them out when they're needed. Should solve your performance problem and your memory problem.
a static variable will not garbage collected but if you are just storing a some connection data it should not be a problem. What are you exactly storing?
Matteo
Well, judging by what you say, you have an object (let's call it Obj) class which contains the static variable with the connection. Due to you creates a new Obj each time and you stores it at that moment, I think you are doing a lot of copies of the connection which the JVM is unable to clean because they are static.
You could consider the possibility of store this kind of information in a Model class, or remove the static mark in order to let the JVM collect this objects properly.

How to Correctly Close Resources

As I was cleaning up some code, FindBugs pointed me to a bit of JDBC code that uses Connection, CallableStatement, and ResultSet objects. Here's a snippet from that code:
CallableStatement cStmt = getConnection().prepareCall("...");
...
ResultSet rs = cStmt.executeQuery();
while ( rs.next() )
{
...
}
cStmt.close();
rs.close();
con.close();
FindBugs pointed out that these should be within a finally block. I started refactoring my code to do this and I started to wonder how to handle the code within the finally block.
It's possible that the creation of the CallableStatement of Connection objects would throw an exception, leaving my ResultSet object as null. When I try to close the ResultSet, I'd get a NullPointerException and my Connection would, in turn, never get closed. Indeed, this thread brings up the same concept and shows that wrapping your close() invocations in a null check is a good idea.
But what about other possible exceptions? According to the Java API spec, Statement.close() can throw a SQLException "if a database error occurs". So, even if my CallableStatement is not null and I can successfully call close() on it, I might still get an exception and not have a chance to close my other resources.
The only "fail safe" solution I can think of is to wrap each close() invocation in its own try/catch block, like this:
finally {
try {
cStmt.close();
} catch (Exception e) { /* Intentionally Swallow Exception */ }
try {
rs.close();
} catch (Exception e) { /* Intentionally Swallow Exception */ }
try {
con.close();
} catch (Exception e) { /* Intentionally Swallow Exception */ }
}
Boy, if that doesn't look awful. Is there a better way to go about this?
I think the best answer has already being mentioned, but I thought it could be interesing to mention that you could consider the new JDK 7 feature of autoclosable resources.
try{
try(Connection conn = DriverManager.getConnection("jdbc:mysql://localhost/hrdb", "obiwan", "kenobi");
Statement stm = conn.createStatement();
ResultSet rs = stm.executeQuery("select name from department")) {
while(rs.next()){
System.out.println(rs.getString("name"));
}
}
}catch(SQLException e){
//you might wanna check e.getSuppressed() as well
//log, wrap, rethrow as desired.
}
Not all of us can migrate to JDK 7 now, but for those who can start playing with the developer preview, this offer an interesting way of doing things and certainly may deprecate other approaches in the near future.
Use Lombok's cleanup, if you can:
#Cleanup
Connection c = ...
#Cleanup
statement = c.prepareStatement(...);
#Cleanup
rs = statement.execute(...);
This works translates to three nested try-finally blocks and works fine with exception. Never ever swallow an exception without a very good reason!
An alternative:
Write an own utility method like this:
public static void close(ResultSet rs, Statement stmt, Connection con) throws SQLException {
try {
try {
if (rs!=null) rs.close();
} finally {
if (stmt!=null) stmt.close();
}
} finally {
if (con!=null) con.close();
}
}
and use it in
try {
Connection con = ...
Statement stmt = ...
ResultSet rs = ...
} finally {
close(rs, stmt, con);
}
and let the Exception bubble up or handle it as you want.
Well basically that is what you do except you first of all don't necessarily swallow the exception (you can null check and at least LOG the exception). Second, you can set up a nice utility class with things like
public static void close(ResultSet rs) {
try { if (rs != null) rs.close();
} catch (SQLException (e) {
log.error("",e);
}
}
Then you just static import that class.
your finally becomes something like
finally {
close(resultset);
close(statement);
close(connection);
}
which really isn't that hideous.
You only have to close the Connection.
try
{
cStmt.close();
}
catch(Exception e)
{
/* Intentionally Swallow Exception */
}
From docs.oracle.com:
A Statement object is automatically closed when it is garbage collected. When a Statement object is closed, its current ResultSet object, if one exists, is also closed.
Calling close() on a Connection releases its database and JDBC resources.
The only way I know to hide all that ugly try-catch boilerplate code is to utilize something like Spring's JBDC Template.
You can wrap one block into another block:
try{
Connection c = ...
try{
statement = c.prepareStatement(...);
try{
rs = statement.execute(...);
}finally{
rs.close();
}
}finally{
statement.close()
}
}finally{
c.close();
}
}catch(SQLException e){}
Use the lowermost catch-block for everything that might crop up

Which should I close first, the PreparedStatement or the Connection?

When using a PreparedStatement in JDBC, should I close the PreparedStatement first or the Connection first? I just saw a code sample in which the Connection is closed first, but it seems to me more logical to close the PreparedStatement first.
Is there a standard, accepted way to do this? Does it matter? Does closing the Connection also cause the PreparedStatement to be closed, since the PreparedStatement is directly related to the Connection object?
The statement. I would expect you to close (in order)
the result set
the statement
the connection
(and check for nulls along the way!)
i.e. close in reverse order to the opening sequence.
If you use Spring JdbcTemplate (or similar) then that will look after this for you. Alternatively you can use Apache Commons DbUtils and DbUtils.close() or DbUtils.closeQuietly().
The following procedures should be done (in order)
The ResultSet
The PreparedStatement
The Connection.
Also, it's advisable to close all JDBC related objects in the finally close to guarantee closure.
//Do the following when dealing with JDBC. This is how I've implemented my JDBC transactions through DAO....
Connection conn = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
conn = ....
ps = conn.prepareStatement(...);
//Populate PreparedStatement
rs = ps.executeQuery();
} catch (/*All relevant exceptions such as SQLException*/Exception e) {
logger.error("Damn, stupid exception: " , e);
} finally {
if (rs != null) {
try {
rs.close();
rs = null;
} catch (SQLException e) {
logger.error(e.getMessage(), e.fillInStackTrace());
}
}
if (ps != null) {
try {
ps.close();
ps = null;
} catch (SQLException e) {
logger.error(e.getMessage(), e.fillInStackTrace());
}
}
try {
if (conn!= null && !conn.isClosed()){
if (!conn.getAutoCommit()) {
conn.commit();
conn.setAutoCommit(true);
}
conn.close();
conn= null;
}
} catch (SQLException sqle) {
logger.error(sqle.getMessage(), sqle.fillInStackTrace());
}
}
You can see I've checked if my objects are null and for connection, check first if the connection is not autocommited. Many people fail to check it and realise that the transaction hasn't been committed to DB.

Categories