We Keep Coding

Releasing connections from connection pool

In our code (which runs as a schedule job via timer), we have threads running in parallel to perform a database operation. Problem here is each thread is initiating a connection via Hibernate factory. These connections are closed after every database action but stil gets stocked in the connection pool(as INACTIVE). All the connections gets released only after the job/main process is killed.Is there any way to release the connection even from connection pool after the database operation. When we use cron job instead of timer, the process gets killed automatically but we dont need cron here.
Kindly help us to resolve this as we are already nearing production release.
Note : We came to know about this when QA tested with heavy load on the job and for each load new connections are pulled.

You need to restrict the number of threads getting created in the thread pool.
dotConnect for Oracle uses connection pooling. The OracleConnection connection string has the Pooling parameter. If Pooling=true (the default value), the connection is not deleted after closing it, it is placed to the pool instead. When a new connection with the same connection string is opened, it is taken from the pool (if there are free connections) instead of the creating a new one. This provides significant performance improvements. If you use 800 connections that are connected for 10-15 seconds each, and there are only few different connection strings, you may not have 800 actual connections. Closed connections will be placed to the pool, and they will be taken from the pool when a new connection with the same connection string will open. No additional connection will open in such case.
You can disable Pooling by adding 'Pooling=false' to the connection string. In such case, a connection will be deleted from memory and free the session. However this may lead to performance loss.
Most likely, pooling should not cause creating too much sessions. Try testing your application with pooling on. If the session number will be too large, you can disable pooling.
For more information, please refer to http://www.devart.com/dotconnect/oracle/docs/FAQ.html#q54

I have found the root cause for the issue and have also found the solution.
The root cause was number of connections set as minimum and maximum and the time out parameter.
The minimum was 5 and max was 20 and timeout was 800 seconds. But out job was scheduled to run every minute. Due to the configuration, the connections were not released properly within minute.
Another issue was our code was not using the session factory as singleton, but was initializing for each thread. Since the resource was not shared, each session factory creates 5 connections by default and extended to 20 max. Since the timeout also was higher before the connections are released, next set of job starts and creates its own set of new connections.
Finally the pool gets full and oracle becomes unavailable.
We fixed this by sharing the session object across and also setting the timeout to lesser value so that connections are getting released from pool.

Is each connection backed by one Thread in Database Connection Pooling?

In Database connection Pooling,
does each connection correspond to 1 thread in the database ?
Would it really matter ? I mean, there could be a threadPool
executor, that execute whatever a connection Object has to execute ?
I wonder how it works, because that would help in understanding how this things is actually tuned.
My understanding so far is "one connection - one thread". Otherwise why most database would be blocking ?

Connection pooling is what you have on client side (i.e. in Java).
The connection pool is just that - a pool of open connections to the database. These are not bound to threads, any number of threads can request connection from the pool at any given time - the pool will grant the request if a connection is available, and if not, it will either create a new one, block or deny the request (depends on implementation). The main idea here is to have less connections than threads, another purpose is to keep those connections open if there's many short DB operations (the creation of DB connection is an expensive operation).
On the server side, this depends on the DB implementation. I would expect most DB servers to use one thread per connection - someone has to listen on the open socket after all. For many DB engines this can be much more complex though, e.g. there may be one module listening on the socket, and in turn sending the queries to another module that may have different number of threads running.

Should I close a Connection obtained from a DataSource manually?

When I get a Connection from the DataSource, should I close it manually? I mean in case I must close it, how it will be used in future requests?

A connection obtained from a connection pool should be used exactly the same as a normal connection. The JDBC 4.2 specification (section 11.1) says about pooling:
When an application is finished using a connection, it closes the logical connection
using the method Connection.close. This closes the logical connection but does
not close the physical connection. Instead, the physical connection is returned to the
pool so that it can be reused.
Connection pooling is completely transparent to the client: A client obtains a pooled
connection and uses it just the same way it obtains and uses a non pooled
connection.
(emphasis mine)
This means that when you are done with a connection, you always call Connection.close()! It doesn't matter if it is a physical connection, or a logical connection from the pool.
The reason is that whether a connection is a physical (direct) connection or a logical connection should be purely a matter of configuration, not a concern of the application code that merely uses the connection.
In the case of a connection pool, the close() will - details may vary, and some implementations are buggy in this respect - invalidate the logical connection and signal to the connection pool that the underlying physical connection is available for re-use. The connection pool may do some validity checks and then return the (physical) connection into the pool or close it (eg if the pool has too many idle connections, or the connection is too old, etc).
Calling close() is not only allowed, it is even vital for the correct working of a connection pool. Not calling close() usually requires some helper thread to close (reclaim) logical connections that have been in use for too long. As this timeout is usually longer than normal application needs, it might lead to exhaustion of the pool, or to configuration where the pool needs a higher maximum number of connections than is really necessary.

You should close Connection in order to return it to the pool, next time you'll ask for Datasource.getConnection() connection from the pool will be obtained. There is no problem here.
Sometimes you don't want to close connection after each operation and use the same connection for several operations. In this case you shouldn't close it until last operation finished.

Use try with resources to avoid connection problems
try (Connection con = ds.getConnection();
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(...)) {...}

Java Connection Pooling

I searched for connection pooling and read about it. If I understand it correctly, a connection pool is like a collection of open connections. If a connection is established or created it should be added to the connection pool, and if that connection is closed it should be removed in connection pool; while it is open I can use it again and again.
While reading these tutorials and explanations about connection pooling I have some questions:
Can a pool of connections only be used on a certain computer? Like ComputerA
cannot share its connection pool with ComputerB?
Where should connection.close() be placed?
Is it correct to use a connection ONLY when selecting/loading record? After I got the returned records/data I close the connection at finally statement. Same as adding, editing and deleting records. And while it is processing I place a progress bar so the user will have to wait for it to be completed and to do some process again, which means I will only open connection one at a time.
Thanks for the explanation. :)

Note: I assume we're talking about the java.sql.Connection interface.
Can a pool of connections only be used on a certain computer? Like ComputerA cannot share its connection pool with ComputerB?
A connection exists between a running application and a database. Naturally, two different machines can't share the same running application, so they can't share connections with a database.
Where should connection.close() be placed?
You should always make sure to call close() on a Connection instance after using it (typically in a finally block). If pooling is being used, this will actually return the connection to the pool behind the scenes. Reference: Closing JDBC Connections in Pool
Is it correct to use a connection ONLY when selecting/loading record? After I got the returned records/data I close the connection at finally statement.
Yes, that's correct. You don't want to manually hang on to a Connection reference - use it to execute SQL/DML and then check it back into the pool by calling close() in the finally block, just like you're doing.

JDBC Connection Pooling: Connection Reuse?

As per my understanding, JDBC Connection Pooling (at a basic level) works this way:
create connections during app initialization and put in a cache
provide these cached connections on demand to the app
a separate thread maintains the Connection Pool, performing activities like:
discard connections that have been used (closed)
create new connections and add to the cache to maintain a specific count of connections
But, whenever I hear the term "connection reuse" in a JDBC Connection Pooling discussion, I get confused. When does the connection reuse occurs?
Does it means that Connection Pool provides the same connection for two different database interactions (without closing it)? Or, is there a way to continue using a connection even after it gets closed after a DB call?

Connection pooling works by re-using connections. Applications "borrow" a connection from the pool, then "return" it when finished. The connection is then handed out again to another part of the application, or even a different application.
This is perfectly safe as long as the same connection is not is use by two threads at the same time.
The key point with connection pooling is to avoid creating new connections where possible, since it's usually an expensive operation. Reusing connections is critical for performance.

The connection pool does not provide you with the actual Connection instance from the driver, but returns a wrapper. When you call 'close()' on a Connection instance from the pool, it will not close the driver's Connection, but instead just return the open connection to the pool so that it can be re-used (see skaffman's answer).

Connection pooling reuses connections.
Here is how apache dbcp works underline.
Connection poolableConnection= apacheDbcpDataSource.getConnection();
Apache DBCP implementation returns connection wrapper which is of type PoolableConnection.
poolableConnection.close();
PoolableConnection.close() inspects if actual underlying connection is closed or not, if not then it returns this PoolableConnection instance into connection pool (GenericObjectPool in this case).
if (!isUnderlyingConectionClosed) {
// Normal close: underlying connection is still open, so we
// simply need to return this proxy to the pool
try {
genericObjectPool.returnObject(this); //this is PoolableConnection instance in this case
....
}

My understanding is the same as stated above and, thanks to a bug, I have evidence that it's correct. In the application I work with there was a bug, an SQL command with an invalid column name. On execution an exception is thrown. If the connection is closed then the next time a connection is gotten and used, with correct SQL this time, an exception is thrown again and the error message is the same as the first time though the incorrect column name doesn't even appear in the second SQL. So the connection is obviously being reused. If the connection is not closed after the first exception is thrown (because of the bad column name) then the next time a connection is used everything works just fine. Presumably this is because the first connection hasn't been returned to the pool for reuse. (This bug is occurring with Jave 1.6_30 and a connection to a MySQL database.)