We Keep Coding

Does getConnection always open a new connection?

I have found this method in a Java file and I am wondering what's happening here? Does this really work? I think this method does nothing because the getConnection()-method creates a new connection and then it gets closed. Am I correct?
public void closeAllConnections()
{
for(String db : this.dbList)
try {
DataSource ds = (DataSource) this.applicationContext.getBean(db+"DataSource");
ds.getConnection().close();
} catch (Exception e) {
//...
}
}

It depends how your DataSource is defined.
If it uses connection pooling it can reuse existing connection
for example see oracle explanation
Connection pooling in the JDBC 2.0 extension API is a framework for
caching database connections. This allows reuse of physical
connections and reduced overhead for your application.

sharing a JDBC Connection object between other objects

I have created a Database class which uses a static connection object in order to be used in common between instances of itself. my question is that is there any problem with this approach or not?
class Database {
private static Connection connection = null;
public Database() {
if(connection == null){
...
connection = DriverManager.getConnection(...);
...
}
}
}

If you are going to have many (hundreds) of queries per second then implementing a connection pool is the way to go. See the answer to this question for more details. However, if you a Java novice (we all were one day!) then I don't imagine you will be needing this requirement, and probably will struggle to implement it.
Instead, the simple pattern of creating a new connection if required, and then closing it when finished will be the best way to go forward for you. Below is a modified version of your Database class which I think is a good way to move forward.
class Database {
private Connection con = null;
private final String connectionString;
public Database(String connectionString) {
this.connectionString = connectionString;
}
public void connect() throws SQLException {
if (con != null // if the connection exists
&& !con.isClosed() // and has not been closed
&& con.isValid(0)) { // and appears to be functioning (with a test timeout of 0ms)
return; // skip connection creation
}
// create the connection
con = DriverManager.getConnection(connectionString);
}
public void testFunction() {
try {
connect();
// .. do some stuff with the connection ..
} catch (Exception e) {
// log or otherwise deal with the error
} finally {
try {
con.close();
} catch (Exception e) {
System.err.println("Failed to close connection: " + e.toString());
}
}
}
}
Some things to note about this solution:
It is not very efficient - creating a new connection always takes more time than using an existing one
This class if not thread safe - if you need this requirement, I recommend using a thread pool. However, if you create a new instance of this class per thread then it will be thread safe (as there is not static connection to worry about!)
It does do the job - certainly for simple cases. I use the model for a relatively low volume database which has approx 50-100 connections made/closed per minute and it does not add a noticeable lag
It is very robust - nothing is safer than opening and closing a connection per query. You are guaranteed to be able to handle a connection failure per query, and the connection will always be closed (unless it already has been).
Disclaimer The solution above is not a particularly amazing solution. However, I believe it is simple to implement and a good way for a Java novice to get to know the ropes before jumping into external libraries.

There is nothing wrong with creating an object to manage your connections, however, connections should be opened and closed and can be used in multi-threaded environments, so having a static connection is not a good idea. For a method that needs a connection, get a connection use it, close it. Even if you are not using it in a multi-threaded environment, the connection can time-out, then you need to constantly check if the connection is up and available, instead of just saying, get me a connection, use the connection, close it.

java.sql.SQLException: Listener refused the connection with the following error: ORA-12519, TNS:no appropriate service handler found

I am passing Resultset object to each thread. Each thread is connecting to the database and inserting data. Untill thread 110 it is working fine. After it crosses 111 thread it throws the above exception.
I am using oracle 11g.
My sample Thread code is:
class MyThreadClass implements Runnable
{
public Connection connection;
public Statement statement2;
public ResultSet rs2;
public String cookie;
public MyThreadClass(ResultSet rs1)
{
rs2=rs1;
}
public void run()
{
try
{
cookie=rs2.getString("COOKIE");
driver = "oracle.jdbc.driver.OracleDriver";
url = "jdbc:oracle:thin:#127.0.0.1:1521:xx";
/* connection
statement2.executeUpdate("INSERT INTO visit_header VALUES ('"+cookie+"')");
}
I am not getting how to handle this exception.

Your multi-threaded application is opening too many Connections/Sessions. Hence, the listener is dropping and blocking new connections for a while.
Check your DB resource usage first:
SELECT * FROM v$resource_limit WHERE resource_name IN ('processes','sessions');
Check to see if your MAX_UTILIZATION for either your Processes or Sessions is getting too close to the LIMIT_VALUE. If yes, you should either:
Use DB Connection pooling to share Connection objects between threads. Or,
Increase the number of processes/sessions that Oracle can service simultaneously.
Actually, Connection Pooling (#1) should always be done. An application cannot scale up otherwise. Check Apache Commons DBCP for details. For #2, open a new SQL*Plus session as SYSTEM and run:
ALTER system SET processes=<n-as-per-number-of-threads> scope=spfile;
to increase backend concurrency. Then RESTART the Database. IMPORTANT!

I guess the database just don't accept more connections from your host. If I understand your question right you are making maybe 100 threads which each connects to the database in short time. Maybe you don't even close the connection correctly, or the accesses are lasting so long that a huge amount of connections are opened. The database have a limit to which it accepts connections.
You should definitely reduce the number of connections by some clever technique. Maybe reduce the number of concurrent threads and/or use a connection pool.

Try this solution at your end. It worked for me.
Close the connection in try/catch block and just after closing the connection,
write-
Thread.sleep(1000);
In this case you can write it as-
finally {
try {
if (conn != null && !conn.isClosed())
{
conn.close();
Thread.sleep(1000);
}
}
catch (SQLException e) {
e.printStackTrace();}
}

How to check if another instance of Derby is already booted?

I'm writing some Java application that uses Java DB (i.e. Apache Derby) as database. I use the following method to connect to database:
Connection getConnection() throws SQLException {
EmbeddedDataSource ds = new EmbeddedDataSource();
ds.setDatabaseName(dbUri);
ds.setPassword(password);
ds.setUser(username);
Connection conn = ds.getConnection();
conn.setSchema(schema);
return conn;
}
This works ok, but sometimes I get the following exception:
java.sql.SQLException: Another instance of Derby may have already booted the database
This happens when I run my application and at the same time SQuirreL SQL Client is connected to my database. So everything works as expected, but I would like to be able to check for this in my getConnection() method. I other words, I would like to check if any sessions are opened to my database, and for example, close them, throw my own exception or display error dialog box. I don't know how to do this.
Thx

Rather than declaring that your application "throws SQLException", you can use a "try" block to catch the SQLException, then examine the exception and decide if it is the "Another instance of Derby" exception or not.
Then, you can throw your own exception from your "getConnection" method accordingly.

I modified my getConnection() method to something like below. It does what i want:
Connection getConnection() throws SQLException, DAOConnection {
EmbeddedDataSource ds = new EmbeddedDataSource();
ds.setDatabaseName(dbUri);
ds.setPassword(password);
ds.setUser(username);
Connection conn = null;
try {
conn = ds.getConnection();
} catch (SQLException e) {
// check if cannot connect due to "Another instance of
// Derby may have already booted the database".
// The error code of this exception is 45000.
if (e.getNextException().getErrorCode() == 45000) {
throw new DAOConnection(e.getNextException().getMessage());
}
throw new SQLException(e);
}
conn.setSchema(schema);
return conn;
}

Prevention is better than cure. IMO, catching exception and then realizing it was duplicate Derby server started is not an ideal design. Better would be to prevent duplicate instantiation. If possible you can synchronize your getConnection() method or make it part of a singleton class or load the embedded Derby driver from static initializer block of a startup/main class which is loaded only once by JVM and hence Derby will be started only once. Something like following in the main/startup class should do the trick:
static {
try {
Class.forName("org.apache.derby.jdbc.EmbeddedDriver");
}
catch(Exception e){
.....
}
}
Per the link here http://db.apache.org/derby/docs/10.3/devguide/tdevdvlp38381.html loading the Driver should start the Derby embedded system.

Connection pooling with Java and MySQL in Tomcat web application

I recently wrote and deployed a Java web application to a server and I'm finding an unusual problem which didn't appear during development or testing.
When a user logs in after so long and goes to display data from the database, the page indicates that there are no records to see. But upon page refresh, the first x records are shown according to the pagination rules.
Checking the logs, I find:
ERROR|19 09 2009|09 28 54|http-8080-4|myDataSharer.database_access.Database_Metadata_DBA| - Error getting types of columns of tabular Dataset 12
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.io.EOFException
STACKTRACE:
java.io.EOFException
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1956)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2368)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2867)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616)
And so on for several hundred lines.
The application is currently set for about 100 users but is not yet in full use. It uses connection pooling between the Apache Tomcat servlets / jsps and a MySQL database with the following code example forming the general arrangement of a database operation, of which there are typically several per page:
// Gets a Dataset.
public static Dataset getDataset(int DatasetNo) {
ConnectionPool_DBA pool = ConnectionPool_DBA.getInstance();
Connection connection = pool.getConnection();
PreparedStatement ps = null;
ResultSet rs = null;
String query = ("SELECT * " +
"FROM Dataset " +
"WHERE DatasetNo = ?;");
try {
ps = connection.prepareStatement(query);
ps.setInt(1, DatasetNo);
rs = ps.executeQuery();
if (rs.next()) {
Dataset d = new Dataset();
d.setDatasetNo(rs.getInt("DatasetNo"));
d.setDatasetName(rs.getString("DatasetName"));
...
}
return d;
}
else {
return null;
}
}
catch(Exception ex) {
logger.error("Error getting Dataset " + DatasetNo + "\n", ex);
return null;
}
finally {
DatabaseUtils.closeResultSet(rs);
DatabaseUtils.closePreparedStatement(ps);
pool.freeConnection(connection);
}
}
Is anyone able to advise a way of correcting this problem?
I believe it is due to MySQL leaving connection poll connections open for up to eight hours but am not certain.
Thanks
Martin O'Shea.
Just to clarify one point made about my method of connection pooling, it isn't Oracle that I'm using in my application but a class of my own as follows:
package myDataSharer.database_access;
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.InitialContext;
import org.apache.log4j.Logger;
public class ConnectionPool_DBA {
static Logger logger = Logger.getLogger(ConnectionPool_DBA.class.getName());
private static ConnectionPool_DBA pool = null;
private static DataSource dataSource = null;
public synchronized static ConnectionPool_DBA getInstance() {
if (pool == null) {
pool = new ConnectionPool_DBA();
}
return pool;
}
private ConnectionPool_DBA() {
try {
InitialContext ic = new InitialContext();
dataSource = (DataSource) ic.lookup("java:/comp/env/jdbc/myDataSharer");
}
catch(Exception ex) {
logger.error("Error getting a connection pool's datasource\n", ex);
}
}
public void freeConnection(Connection c) {
try {
c.close();
}
catch (Exception ex) {
logger.error("Error terminating a connection pool connection\n", ex);
}
}
public Connection getConnection() {
try {
return dataSource.getConnection();
}
catch (Exception ex) {
logger.error("Error getting a connection pool connection\n", ex);
return null;
}
}
}
I think the mention of Oracle is due to me using a similar name.

There are a few pointers on avoiding this situation, obtained from other sources, especially from the connection pool implementations of other drivers and from other application servers. Some of the information is already available in the Tomcat documentation on JNDI Data Sources.
Establish a cleanup/reaper schedule that will close connections in the pool, if they are inactive beyond a certain period. It is not good practice to leave a connection to the database open for 8 hours (the MySQL default). On most application servers, the inactive connection timeout value is configurable and is usually less than 15 minutes (i.e. connections cannot be left in the pool for more than 15 minutes unless they are being reused time and again). In Tomcat, when using a JNDI DataSource, use the removeAbandoned and removeAbandonedTimeout settings to do the same.
When a new connection is return from the pool to the application, ensure that it is tested first. For instance, most application servers that I know, can be configured so that connection to an Oracle database are tested with an execute of "SELECT 1 FROM dual". In Tomcat, use the validationQuery property to set the appropriate query for MySQL - I believe this is "SELECT 1" (without quotes). The reason why setting the value of the validationQuery property helps, is because if the query fails to execute, the connection is dropped from the pool, and new one is created in its place.
As far are the behavior of your application is concerned, the user is probably seeing the result of the pool returning a stale connection to the application for the first time. The second time around, the pool probably returns a different connection that can service the application's queries.
Tomcat JNDI Data Sources are based on Commons DBCP, so the configuration properties applicable to DBCP will apply to Tomcat as well.

I'd wonder why you're using ConnectionPool_DBA in your code instead of letting Tomcat handle the pooling and simply looking up the connection using JNDI.
Why are you using an Oracle connection pool with MySQL? When I do JNDI lookups and connection pooling, I prefer the Apache DBCP library. I find that it works very well.
I'd also ask if your DatabaseUtils methods throw any exceptions, because if either of the calls prior to your call to pool.freeConnection() throw one you'll never free up that connection.
I don't like your code much because a class that performs SQL operations should have its Connection instance passed into it, and should not have the dual responsibility of acquiring and using the Connection. A persistence class can't know if it's being used in a larger transaction. Better to have a separate service layer that acquires the Connection, manages the transaction, marshals the persistence classes, and cleans up when it's complete.
UPDATE:
Google turned up the Oracle class with the same name as yours. Now I really don't like your code, because you wrote something of your own when a better alternative was easily available. I'd ditch yours right away and redo this using DBCP and JNDI.

This error indicates server closes connection unexpectedly. This can occur in following 2 cases,
MySQL closes idle connection after certain time (default is 8 hours). When this occurs, no thread is responsible for closing the connection so it gets stale. This is most likely the cause if this error only happens after long idle.
If you don't completely read all the responses, the connection may get returned to the pool in busy state. Next time, a command is sent to MySQL and it closes connection for wrong state. If the error occurs quite frequent, this is probably the cause.
Meanwhile, setting up an eviction thread will help to alleviate the problem. Add something like this to the Data Source,
...
removeAbandoned="true"
removeAbandonedTimeout="120"
logAbandoned="true"
testOnBorrow="false"
testOnReturn="false"
timeBetweenEvictionRunsMillis="60000"
numTestsPerEvictionRun="5"
minEvictableIdleTimeMillis="30000"
testWhileIdle="true"
validationQuery="select now()"

Is there a router between the web server and the database that transparently closes idle TCP/IP connections?
If so, you must have your connection pool either discard unused-for-more-than-XX-minutes connections from the pool, or do some kind of ping every YY minutes on the connection to keep it active.

On the off chance you haven't found your answer I've been dealing with this for the last day. I am essentially doing the same thing you are except that I'm basing my pooling off of apache.commons.pool. Same exact error you are seeing EOF. Check your mysqld error log file which is most likely in your data directory. Look for mysqld crashing. mysqld_safe will restart your mysqld quickly if it crashes so it won't be apparent that this is the case unless you look in its logfile. /var/log is not help for this scenario.
Connections that were created before the crash will EOF after the crash.