We Keep Coding

JDBC Oracle Thin ORA-02396 Connection idle timeout

I am supporting some legacy code and it's chugged along fine until recently. I am looking for if there is a setting for JDBC Oracle thin connection where I can specify idle timeout via Java (no connection pooling)? A lot of resources online refer to connection pooling... is it even possible in my case (to specify idle timeout, in a non-pooling situation)? Or is idle time a setting on the specific DB user account?
Updates + Questions
I was able to log in as the user, and ran a query to try to find out resource limits. select * from USER_RESOURCE_LIMITS; However everything came back "UNLIMITED". Is it possible for another value (say from the JDBC connection) to override the "UNLIMITED"?
So the job holds onto the connection, while we actively query another system via DB links for a good duration of ~2+ hours... Now, why would the idle timeout even come into play?
Update #2
We switched to a different account (that has the same kind of DB link setup) and the job was able to finish like it did before. Which sort of points to something wonky with the Oracle user profile? But like I said, querying USER_RESOURCE_LIMITS shows both users to have "UNLIMITED" idle time. DBA pretty confirmed that too. What else could be causing this difference?
Update #3
Stack trace and such.
java.sql.SQLException: ORA-02396: exceeded maximum idle time, please connect again
ORA-06512: at line 1
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:316)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:282)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:639)
at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:184)
at oracle.jdbc.driver.T4CCallableStatement.execute_for_rows(T4CCallableStatement.java:873)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1086)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:2984)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3076)
at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4273)
at com.grocery.stand.Helper.getAccess(Helper.java:216)
at com.grocery.stand.fruitbasket.Dao.getPriceData(Dao.java:216)
at com.grocery.stand.fruitbasket.Dao.getPricees(Dao.java:183)
at com.grocery.stand.fruitbasket.UpdatePrice.updateAllFruitPrices(UpdatePrice.java:256)
at com.grocery.stand.fruitbasket.UpdatePrice.main(UpdatePrice.java:58)
SQL Exception while getting Data from SYSTEM_B
Exception while updating pricing : ORA-01012: not logged on
Exception in thread "main" java.sql.SQLException: ORA-01012: not logged on
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:316)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:277)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:272)
at oracle.jdbc.driver.T4C7Ocommoncall.receive(T4C7Ocommoncall.java:129)
at oracle.jdbc.driver.T4CConnection.do_rollback(T4CConnection.java:478)
at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:1045)
at com.grocery.stand.Helper.rollBack(Helper.java:75)
at com.grocery.stand.fruitbasket.UpdatePrice.updatePartNumbers(UpdatePrice.java:291)
at com.grocery.stand.fruitbasket.UpdatePrice.main(UpdatePrice.java:58)
Connection Code
public static Connection openConnection() throws SQLException {
String userName = propBundle.getString(DB_UID);
String password = propBundle.getString(DB_PWD);
String url = propBundle.getString(DB_URL);
Connection conn = null;
try {
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
conn = (Connection) DriverManager.getConnection(url, userName,
password);
conn.setAutoCommit(false);
} catch (SQLException sqle) {
sqle.printStackTrace(System.out);
throw sqle;
}
return conn;
}
Error occurs on line execute()
public static void getSystemAccess(Connection dbConnection) throws SQLException {
try {
CallableStatement authStmt = null;
String authorize = "CALL ABC.ACCESS_PROCEDURE#some_db_link()";
authStmt = dbConnection.prepareCall(authorize);
authStmt.execute();
authStmt.close();
} catch (SQLException sqle1) {
sqle1.printStackTrace();
throw new SQLException(sqle1.getMessage());
}
}

I'm not sure that I understand the question you're asking.
The error you are getting indicates that the Oracle user that you are using to connect to the database has a profile configured (in Oracle) that limits the amount of time the connection can be idle. Oracle is killing your connection when the connection remains idle too long. Normally, the solution to this sort of problem would be to go to the DBA and ask for the idle time to be increased or to look through your code and see why the connection is open and unused for so long. If you were using a connection pool (which it doesn't appear you are), it would make sense for some connections to remain open and idle for long periods of time. Since it doesn't appear that you are using a connection pool, the question is whether it makes sense for the application to hold open the connection for long periods of time without doing anything. If the application opens a connection when the user logs in at 9am and doesn't close it until the user shuts down at 5pm, it may make sense to adjust the IDLE_TIME setting for this user in the database. Otherwise, you may want to investigate whether it makes logical sense for the application to hold open the database connection so long without doing something or whether the application can be modified to close the connection when it is no longer needed.

Best way to manage DB connections without JNDI

I have a website, in which currently I am getting 1000 page views. I am expecting it will go around 30k per day in future. Now the problem for me to manage the DB connections.
At present I am just connecting to DB directly from java program. I know it is worst design in the world. But for time being I have written like that.
I have plan to manage connection pooling using JNDI. But the problem is my hosting provider is not supporting JNDI.
Can anyone suggest me how to manage DB connections without jndi?

Connection pooling does not per se require the connections to be obtained by JNDI. You can also just setup and use a connection pool independently from JNDI. Let's assume that you'd like to use C3P0, which is one of the better connection pools, then you can find "raw" JNDI-less setup details in this tutorial.
Here's an extract of the tutorial:
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass( "org.postgresql.Driver" ); //loads the jdbc driver
cpds.setJdbcUrl( "jdbc:postgresql://localhost/testdb" );
cpds.setUser("swaldman");
cpds.setPassword("test-password");
Create the datasource once during application's startup and store it somewhere in the context. The connection can then be acquired and used as follows:
Connection connection = null;
// ...
try {
connection = cpds.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
Yes, closing in finally is still mandatory, else the connection pool won't be able to take the connection back in pool for future reuse and it'll run out of connections.

Connection pooling with Java and MySQL in Tomcat web application

I recently wrote and deployed a Java web application to a server and I'm finding an unusual problem which didn't appear during development or testing.
When a user logs in after so long and goes to display data from the database, the page indicates that there are no records to see. But upon page refresh, the first x records are shown according to the pagination rules.
Checking the logs, I find:
ERROR|19 09 2009|09 28 54|http-8080-4|myDataSharer.database_access.Database_Metadata_DBA| - Error getting types of columns of tabular Dataset 12
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.io.EOFException
STACKTRACE:
java.io.EOFException
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1956)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2368)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2867)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616)
And so on for several hundred lines.
The application is currently set for about 100 users but is not yet in full use. It uses connection pooling between the Apache Tomcat servlets / jsps and a MySQL database with the following code example forming the general arrangement of a database operation, of which there are typically several per page:
// Gets a Dataset.
public static Dataset getDataset(int DatasetNo) {
ConnectionPool_DBA pool = ConnectionPool_DBA.getInstance();
Connection connection = pool.getConnection();
PreparedStatement ps = null;
ResultSet rs = null;
String query = ("SELECT * " +
"FROM Dataset " +
"WHERE DatasetNo = ?;");
try {
ps = connection.prepareStatement(query);
ps.setInt(1, DatasetNo);
rs = ps.executeQuery();
if (rs.next()) {
Dataset d = new Dataset();
d.setDatasetNo(rs.getInt("DatasetNo"));
d.setDatasetName(rs.getString("DatasetName"));
...
}
return d;
}
else {
return null;
}
}
catch(Exception ex) {
logger.error("Error getting Dataset " + DatasetNo + "\n", ex);
return null;
}
finally {
DatabaseUtils.closeResultSet(rs);
DatabaseUtils.closePreparedStatement(ps);
pool.freeConnection(connection);
}
}
Is anyone able to advise a way of correcting this problem?
I believe it is due to MySQL leaving connection poll connections open for up to eight hours but am not certain.
Thanks
Martin O'Shea.
Just to clarify one point made about my method of connection pooling, it isn't Oracle that I'm using in my application but a class of my own as follows:
package myDataSharer.database_access;
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.InitialContext;
import org.apache.log4j.Logger;
public class ConnectionPool_DBA {
static Logger logger = Logger.getLogger(ConnectionPool_DBA.class.getName());
private static ConnectionPool_DBA pool = null;
private static DataSource dataSource = null;
public synchronized static ConnectionPool_DBA getInstance() {
if (pool == null) {
pool = new ConnectionPool_DBA();
}
return pool;
}
private ConnectionPool_DBA() {
try {
InitialContext ic = new InitialContext();
dataSource = (DataSource) ic.lookup("java:/comp/env/jdbc/myDataSharer");
}
catch(Exception ex) {
logger.error("Error getting a connection pool's datasource\n", ex);
}
}
public void freeConnection(Connection c) {
try {
c.close();
}
catch (Exception ex) {
logger.error("Error terminating a connection pool connection\n", ex);
}
}
public Connection getConnection() {
try {
return dataSource.getConnection();
}
catch (Exception ex) {
logger.error("Error getting a connection pool connection\n", ex);
return null;
}
}
}
I think the mention of Oracle is due to me using a similar name.

There are a few pointers on avoiding this situation, obtained from other sources, especially from the connection pool implementations of other drivers and from other application servers. Some of the information is already available in the Tomcat documentation on JNDI Data Sources.
Establish a cleanup/reaper schedule that will close connections in the pool, if they are inactive beyond a certain period. It is not good practice to leave a connection to the database open for 8 hours (the MySQL default). On most application servers, the inactive connection timeout value is configurable and is usually less than 15 minutes (i.e. connections cannot be left in the pool for more than 15 minutes unless they are being reused time and again). In Tomcat, when using a JNDI DataSource, use the removeAbandoned and removeAbandonedTimeout settings to do the same.
When a new connection is return from the pool to the application, ensure that it is tested first. For instance, most application servers that I know, can be configured so that connection to an Oracle database are tested with an execute of "SELECT 1 FROM dual". In Tomcat, use the validationQuery property to set the appropriate query for MySQL - I believe this is "SELECT 1" (without quotes). The reason why setting the value of the validationQuery property helps, is because if the query fails to execute, the connection is dropped from the pool, and new one is created in its place.
As far are the behavior of your application is concerned, the user is probably seeing the result of the pool returning a stale connection to the application for the first time. The second time around, the pool probably returns a different connection that can service the application's queries.
Tomcat JNDI Data Sources are based on Commons DBCP, so the configuration properties applicable to DBCP will apply to Tomcat as well.

I'd wonder why you're using ConnectionPool_DBA in your code instead of letting Tomcat handle the pooling and simply looking up the connection using JNDI.
Why are you using an Oracle connection pool with MySQL? When I do JNDI lookups and connection pooling, I prefer the Apache DBCP library. I find that it works very well.
I'd also ask if your DatabaseUtils methods throw any exceptions, because if either of the calls prior to your call to pool.freeConnection() throw one you'll never free up that connection.
I don't like your code much because a class that performs SQL operations should have its Connection instance passed into it, and should not have the dual responsibility of acquiring and using the Connection. A persistence class can't know if it's being used in a larger transaction. Better to have a separate service layer that acquires the Connection, manages the transaction, marshals the persistence classes, and cleans up when it's complete.
UPDATE:
Google turned up the Oracle class with the same name as yours. Now I really don't like your code, because you wrote something of your own when a better alternative was easily available. I'd ditch yours right away and redo this using DBCP and JNDI.

This error indicates server closes connection unexpectedly. This can occur in following 2 cases,
MySQL closes idle connection after certain time (default is 8 hours). When this occurs, no thread is responsible for closing the connection so it gets stale. This is most likely the cause if this error only happens after long idle.
If you don't completely read all the responses, the connection may get returned to the pool in busy state. Next time, a command is sent to MySQL and it closes connection for wrong state. If the error occurs quite frequent, this is probably the cause.
Meanwhile, setting up an eviction thread will help to alleviate the problem. Add something like this to the Data Source,
...
removeAbandoned="true"
removeAbandonedTimeout="120"
logAbandoned="true"
testOnBorrow="false"
testOnReturn="false"
timeBetweenEvictionRunsMillis="60000"
numTestsPerEvictionRun="5"
minEvictableIdleTimeMillis="30000"
testWhileIdle="true"
validationQuery="select now()"

Is there a router between the web server and the database that transparently closes idle TCP/IP connections?
If so, you must have your connection pool either discard unused-for-more-than-XX-minutes connections from the pool, or do some kind of ping every YY minutes on the connection to keep it active.

On the off chance you haven't found your answer I've been dealing with this for the last day. I am essentially doing the same thing you are except that I'm basing my pooling off of apache.commons.pool. Same exact error you are seeing EOF. Check your mysqld error log file which is most likely in your data directory. Look for mysqld crashing. mysqld_safe will restart your mysqld quickly if it crashes so it won't be apparent that this is the case unless you look in its logfile. /var/log is not help for this scenario.
Connections that were created before the crash will EOF after the crash.

Connection timeout for DriverManager getConnection

I am trying to connect to DB using the standard JDBC way
connection = DriverManager.getConnection(url, username, password);
Is there a maximum value of timeout on the connection, how long does a connection live, can I increase the value. I want in cases the connection to be open forever , is it a good idea.

You can set the Timeout on the DriverManager like this:
DriverManager.setLoginTimeout(10);
Connection c = DriverManager.getConnection(url, username, password);
Which would imply that if the connection cannot open within the given time that it times out.
In terms of keeping a connection open forever, it is possible if you do not close the connection but it may not be a good idea. Connections should be closed as soon as you are finished with them.
If you want to optimise the opening and closing of connections then you can use a connection pool.

The value is usually DB-controlled. You have no control over it using code. It depends on the DB server used. It is usually around 30 minutes up to one hour.
On the other hand, keeping a Connection open forever is a very bad idea. Best practice is to acquire and close Connection, Statement and ResultSet in the shortest possible scope to avoid resource leaks and potential application crashes caused by the leaks and timeouts.
True, connecting the DB is an expensive task. If your application is supposed to run a relatively long time and to connect the DB fairly often, then consider using a connection pool to improve connecting performance. If your application is a webapplication, then take a look in the appserver's documentation, it usually provides a connection pooling facility in flavor of a DataSource. If it is a client application, then look for 3rd party connection pooling libraries which have proven their robustness with years, such as Apache Commons DBCP (commonly used, used in lot appservers), C3P0 (known from Hibernate) and Proxool (if you want XA connections).
Keep in mind, when using a connection pool, you still have to write proper JDBC code, i.o.w. acquire and close all the resources in the shortest possible scope. The connection pool will on its turn worry about actually closing the connection or just releasing it back to pool for further reuse.
You may get some more insights out of this article how to do the JDBC basics the proper way.
Hope this helps and happy coding.

Here is how to do it with Connector/J MYSQL driver:
String qqq = "jdbc:mysql://localhost/Test?connectTimeout=TIME_IN_MILLIS";
conn = DriverManager.getConnection(qqq, db_user, db_pass);
It worked for me after setLoginTimeout() did nothing.

Just reposting a more complete repost of a comment from user flamming_python as answer because it worked for me:
dbConnectionString = "jdbc:mysql://"+dbHost+":"+dbPort+"/"+dbTable+"?user="+dbUser+"&password="+dbPassword;
Properties properties = new Properties();
properties.put("connectTimeout", "2000");
dbConnect = DriverManager.getConnection(dbConnectionString, properties);
Original comment:
"LoL # your threads - you can do it quite simply by creating a Properties object prop, then prop.put("connectTimeout", "2000") (where the "2000" is the timeout in ms), and then pass your prop object to the DriverManager.getConnection() method along with your url"

As bestro suggested, it's possible to use a future with an associated timeout. For example:
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Connection> future = executor.submit(new Callable<Connection>() {
#Override
public Connection call() throws Exception {
Connection con = DriverManager.getConnection(url, username, password);
return con;
}
});
Connection con = null;
try {
con = future.get(5, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException ex) {
Logger.getLogger(Scratch.class.getName()).log(Level.SEVERE, null, ex);
}
executor.shutdownNow();
if (con == null) {
System.out.println("Could not establish connection");
} else {
System.out.println("Connection established!");
}

You can use ExecutorService interface from Java. Below is a sample of what you need to do.
Future<Boolean> future = executor.submit(YOUR_METHOD);
future.get(TIMEOUT_YOU_NEED, TimeUnit.SECONDS);

JDBC Connection Issue

I have create a getDBConnection method in my Java application. This returns a connection object, and hence I haven't closed this connection in this method itself.
Now, I am invoking this method from various methods in my application at regular intervals, and closing them inside a try - finally block. I thought this should free up the connection after use. However, I am seeing a large number of connections opened (about 50) in the MySQL Administrator's Server Connections tab.
//Defining a method to retrieve a database connection
// PropDemo is a properties class that retrieves Database related values from a file
public Connection getDBConnection() {
//Instantiating the Properties object
PropDemo prop = new PropDemo();
Connection con = null;
// Retrieving values from the parameters.properties file
String JdbcDriver = prop.getMessage("JdbcDriver");
String JdbcUrlPrefix = prop.getMessage("JdbcUrlPrefix");
String DBIP = prop.getMessage("DBIP");
String DBName = prop.getMessage("DBName");
String DBUser = prop.getMessage("DBUser");
String DBPassword = prop.getMessage("DBPassword");
try {
// Loading and instantiating the JDBC MySQL connector driver class
Class.forName(JdbcDriver).newInstance();
con = DriverManager.getConnection(JdbcUrlPrefix + DBIP + "/" + DBName, DBUser, DBPassword);
if (con.isClosed())
Logger.log("Connection cannot be established", "vm");
} catch (Exception e) {
Logger.log("Exception: " + e, "vm");
Logger.log(Logger.stack2string(e), "vm");
}
return con;
}
I am also closing the associated ResultSet and Statement Objects. What could be missing here?
I am planning to replace all the Statements with PreparedStatements for efficiency and security reasons. Will that help significantly? What else can be done?
EDIT:
This is just a core java application that is repeatedly quering for changes in some fields in a MySQL database through MySQL-JDBC connector. I am not using any framework like Spring or Hibernate.

Your code looks sane.
That's how you're creating a new connection.
Probably the error is where you close it.
You should close it in a finally block.
Some additional questions.
1) Are you sure those 50 conections come from this program ? Maybe there are some others comming from your same office. To confirm this you would need to stop the program, and look again in your connection monitor.
2) Does your application uses many connection simultaneously? Probably its a peak when you're using 50 at the same time.
If you can post the code where you close the connection. Chances are the problem is there.
Additionally I would suggest you to use a connection pool. You can build one your self or you can see the results from this page:
How many JDBC connections in Java?

Are you closing the connection object when you application closes as well?

Are you using your JDBC connection within a J2EE application server or with Hibernate?
Both of these tend to start out with a fairly high connection pool to begin with, so you would see a large number.
Check out the details on connection pooling.

You could take a Singleton approach to the problem and only create a new Connection object if the current one is null:
If (connectionObject != null){
return connectionObject;
}else {
//create new connection object
}
This will make sure that you only have one non-null connection at any time.