I have a java application that gets called from a script every few minutes. The flow is:
Script invokes java process every few minutes.
The java program runs for a few minutes (sometimes less), do some processing and once completed exits by calling System.exit(0).
In step 2, in order to do the processing, we use c3p0 connection pool to connect to MySQL DB.
Everytime we get the connection from the pool, we call close(). We also have unreturnedConnectionTimeout set to 20 so that any time a connection is help too long or not closed, c3p0 takes care of it.
My question is, once all the processing in Step 2 is complete, and before we exit by calling System.exit(0), do we need to do any clean up of the connection pool?
I know that calling close() with connection pool only returns the connection back to pool and do not actually "close" the connection. So before exiting do I need to do any clean up of the conn pool? If I don't do any clean up and does a JVM exit, does the connections to DB still linger around or does c3p0 cleans/force close them? Thanks!
If you are using an instance of com.mchange.v2.c3p0.PooledDataSource (e.g. com.mchange.v2.c3p0.ComboPooledDataSource ), you can use the method com.mchange.v2.c3p0.DataSources.destroy().
This method:
Immediately releases resources (Threads and database Connections) that are held by a C3P0 DataSource.
Only DataSources created by the poolingDataSource() method hold any non-memory resources. Calling this method on unpooled DataSources is effectively a no-op.
You can safely presume that destroying a pooled DataSource that is wrapped around another DataSource created by this library destroys both the outer and the wrapped DataSource. There is no reason to hold a reference to a nested DataSource in order to explicitly destroy it.
PooledDataSource dataSource = new ComboPooledDataSource();
// everything you do
DataSources.destroy(dataSource);
System.exit(0);
In another way, (again, if you are using an instance of com.mchange.v2.c3p0.PooledDataSource), the method com.mchange.v2.c3p0.PooledDataSource.html#hardReset():
Destroys all pooled and checked-out Connections associated with this DataSource immediately. The PooledDataSource is reset to its initial state prior to first Connection acquisition, with no pools yet active, but ready for requests.
PooledDataSource dataSource = new ComboPooledDataSource();
// everything you do
dataSource.hardReset();
System.exit(0);
The method dataSource.close() is the best choice because it is more obvious. It is always advisable to close the resources. The connection might stay open for a moment in the database, but after the timeout, closes. This behavior depends on the database vendor.
Related
I'm using sql server jdbc driver mssql-jdbc-9.2.1.jre8-sources.jar and create an object of com.microsoft.sqlserver.jdbc.SQLServerDataSource.
After setting it up with host-credential-etc, opening Connection and closing Connection, how do I close SQLServerDataSource object itself, I could not find any method in API.
https://javadoc.io/doc/com.microsoft.sqlserver/mssql-jdbc/9.2.1.jre8/com/microsoft/sqlserver/jdbc/SQLServerDataSource.html
Note to closer: I'm NOT using C3P0 connection pool, that may have close option.
The SQLServerDataSource is a simple data source that does not provide any form of connection pooling, it only serves as a factory of physical, non-pooled JDBC connections. As such, you do not need to close, destroy or otherwise dispose of this class (other than normal garbage collection).
The need to close a data source only exists for data sources that are backed by a connection pool (like c3p0), to release those pooled connections and maybe end house-keeping threads.
I have a java program which updates a table in oracle database.
I have tried it using a single JDBC connection and it's very slow and takes hours to complete.
I'm trying to use HikariCP to make a connection pool and have multiple threads get separate connections from the pool.
Suppose I have 6 threads and 5 database connections in the pool and 5 of the threads call the HikariDataSource.getConnection() method. Will each of them get a separate db connection object?
If yes, then, will the thread be in blocked/ waiting state, when it calls the getConnection method or it executes the remaining code with a null connection?
If no, how do I get them separate connections?
Will each of them get a separate db connection object?
Each thread ask connection, if available gets a separate db connection object
If yes, then, will the thread be in blocked/ waiting state, when it calls the getConnection method or it executes the remaining code with a null connection?
If no available connection it will wait until connection is released to pool and take it, if it won't get connection until timeout defined, it will throw a timeout exception
If no, how do I get them separate connections?
Irrelevant, because each thread will get different connection
About HikariCP and concurrency:
HikariCP contains a custom lock-free collection called a ConcurrentBag. The idea was borrowed from the C# .NET ConcurrentBag class, but the internal implementation quite different. The ConcurrentBag provides...
A lock-free design
ThreadLocal caching
Queue-stealing
Direct hand-off optimizations
...resulting in a high degree of concurrency, extremely low latency, and minimized occurrences of false-sharing.
In our code (which runs as a schedule job via timer), we have threads running in parallel to perform a database operation. Problem here is each thread is initiating a connection via Hibernate factory. These connections are closed after every database action but stil gets stocked in the connection pool(as INACTIVE). All the connections gets released only after the job/main process is killed.Is there any way to release the connection even from connection pool after the database operation. When we use cron job instead of timer, the process gets killed automatically but we dont need cron here.
Kindly help us to resolve this as we are already nearing production release.
Note : We came to know about this when QA tested with heavy load on the job and for each load new connections are pulled.
You need to restrict the number of threads getting created in the thread pool.
dotConnect for Oracle uses connection pooling. The OracleConnection connection string has the Pooling parameter. If Pooling=true (the default value), the connection is not deleted after closing it, it is placed to the pool instead. When a new connection with the same connection string is opened, it is taken from the pool (if there are free connections) instead of the creating a new one. This provides significant performance improvements. If you use 800 connections that are connected for 10-15 seconds each, and there are only few different connection strings, you may not have 800 actual connections. Closed connections will be placed to the pool, and they will be taken from the pool when a new connection with the same connection string will open. No additional connection will open in such case.
You can disable Pooling by adding 'Pooling=false' to the connection string. In such case, a connection will be deleted from memory and free the session. However this may lead to performance loss.
Most likely, pooling should not cause creating too much sessions. Try testing your application with pooling on. If the session number will be too large, you can disable pooling.
For more information, please refer to http://www.devart.com/dotconnect/oracle/docs/FAQ.html#q54
I have found the root cause for the issue and have also found the solution.
The root cause was number of connections set as minimum and maximum and the time out parameter.
The minimum was 5 and max was 20 and timeout was 800 seconds. But out job was scheduled to run every minute. Due to the configuration, the connections were not released properly within minute.
Another issue was our code was not using the session factory as singleton, but was initializing for each thread. Since the resource was not shared, each session factory creates 5 connections by default and extended to 20 max. Since the timeout also was higher before the connections are released, next set of job starts and creates its own set of new connections.
Finally the pool gets full and oracle becomes unavailable.
We fixed this by sharing the session object across and also setting the timeout to lesser value so that connections are getting released from pool.
I searched for connection pooling and read about it. If I understand it correctly, a connection pool is like a collection of open connections. If a connection is established or created it should be added to the connection pool, and if that connection is closed it should be removed in connection pool; while it is open I can use it again and again.
While reading these tutorials and explanations about connection pooling I have some questions:
Can a pool of connections only be used on a certain computer? Like ComputerA
cannot share its connection pool with ComputerB?
Where should connection.close() be placed?
Is it correct to use a connection ONLY when selecting/loading record? After I got the returned records/data I close the connection at finally statement. Same as adding, editing and deleting records. And while it is processing I place a progress bar so the user will have to wait for it to be completed and to do some process again, which means I will only open connection one at a time.
Thanks for the explanation. :)
Note: I assume we're talking about the java.sql.Connection interface.
Can a pool of connections only be used on a certain computer? Like ComputerA cannot share its connection pool with ComputerB?
A connection exists between a running application and a database. Naturally, two different machines can't share the same running application, so they can't share connections with a database.
Where should connection.close() be placed?
You should always make sure to call close() on a Connection instance after using it (typically in a finally block). If pooling is being used, this will actually return the connection to the pool behind the scenes. Reference: Closing JDBC Connections in Pool
Is it correct to use a connection ONLY when selecting/loading record? After I got the returned records/data I close the connection at finally statement.
Yes, that's correct. You don't want to manually hang on to a Connection reference - use it to execute SQL/DML and then check it back into the pool by calling close() in the finally block, just like you're doing.
As per my understanding, JDBC Connection Pooling (at a basic level) works this way:
create connections during app initialization and put in a cache
provide these cached connections on demand to the app
a separate thread maintains the Connection Pool, performing activities like:
discard connections that have been used (closed)
create new connections and add to the cache to maintain a specific count of connections
But, whenever I hear the term "connection reuse" in a JDBC Connection Pooling discussion, I get confused. When does the connection reuse occurs?
Does it means that Connection Pool provides the same connection for two different database interactions (without closing it)? Or, is there a way to continue using a connection even after it gets closed after a DB call?
Connection pooling works by re-using connections. Applications "borrow" a connection from the pool, then "return" it when finished. The connection is then handed out again to another part of the application, or even a different application.
This is perfectly safe as long as the same connection is not is use by two threads at the same time.
The key point with connection pooling is to avoid creating new connections where possible, since it's usually an expensive operation. Reusing connections is critical for performance.
The connection pool does not provide you with the actual Connection instance from the driver, but returns a wrapper. When you call 'close()' on a Connection instance from the pool, it will not close the driver's Connection, but instead just return the open connection to the pool so that it can be re-used (see skaffman's answer).
Connection pooling reuses connections.
Here is how apache dbcp works underline.
Connection poolableConnection= apacheDbcpDataSource.getConnection();
Apache DBCP implementation returns connection wrapper which is of type PoolableConnection.
poolableConnection.close();
PoolableConnection.close() inspects if actual underlying connection is closed or not, if not then it returns this PoolableConnection instance into connection pool (GenericObjectPool in this case).
if (!isUnderlyingConectionClosed) {
// Normal close: underlying connection is still open, so we
// simply need to return this proxy to the pool
try {
genericObjectPool.returnObject(this); //this is PoolableConnection instance in this case
....
}
My understanding is the same as stated above and, thanks to a bug, I have evidence that it's correct. In the application I work with there was a bug, an SQL command with an invalid column name. On execution an exception is thrown. If the connection is closed then the next time a connection is gotten and used, with correct SQL this time, an exception is thrown again and the error message is the same as the first time though the incorrect column name doesn't even appear in the second SQL. So the connection is obviously being reused. If the connection is not closed after the first exception is thrown (because of the bad column name) then the next time a connection is used everything works just fine. Presumably this is because the first connection hasn't been returned to the pool for reuse. (This bug is occurring with Jave 1.6_30 and a connection to a MySQL database.)