We are writing a simple standalone java batch. We are using DB2 database. We are trying to do connection pooling using UCP (Version: ucp-11.2.0.3.0).
We have initialized minimum pool size as 5. But when we retrieve one connection and when we are printing the Available and Borrowed connections, we are getting Available as 0 and Borrowed as 1. When retrieving multiple connections, it is still printing the same eventhough the mimimum pool size is 5. we are not getting the any exception eventhough the max limit is crossed. Could you please help us on resolving the issue?
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
pds.setConnectionFactoryClassName("XXXX");
pds.setURL("XXX");
pds.setUser("CCCC");
pds.setPassword("xxx");
pds.setInitialPoolSize(1);
pds.setMinPoolSize(5);
pds.setMaxPoolSize(10);
connection = pds.getConnection();
System.out.println("\nConnection borrowed from the pool");
int avlConnCount = pds.getAvailableConnectionsCount();
System.out.println("\nAvailable connections: " + avlConnCount);
int brwConnCount = pds.getBorrowedConnectionsCount();
System.out.println("\nBorrowed connections: " + brwConnCount);
Output:
-------
Connection borrowed from the pool
Available connections: 0
Borrowed connections: 1
Connection borrowed from the pool
Available connections: 0
Borrowed connections: 1
UCP is the connection pooling library for Oracle Database.
Db2 does not come with its own connection pooling library, instead, it works well with major connection pooling libraries, including Apache DBCP and HikariCP.
Related
I have defined three transaction in which select operations and SELECT operations are happening on the different parameter passed . I try to invoke this method concurrently . I am get an error:
o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: null
Aug 25, 2020 # 12:16:39.000 2020-08-25 06:46:39.388 ERROR 1 --- [o-9003-exec-630] o.h.engine.jdbc.spi.SqlExceptionHelper : Hikari - Connection is not available, request timed out after 60000ms.
And sometimes
org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
I am new to java. Please guide me to solve this issue. Do I need to write multithreading to access number of resources or configuration issue?
hikari:
poolName: Hikari
autoCommit: false
minimumIdle: 5
connectionTimeout: 60000
maximumPoolSize: 80
idleTimeout: 60000
maxLifetime: 240000
leakDetectionThreshold: 300000
Multiple Threads read to the same table in database by using the same connection in java?
This is generally speaking not going to work. The JDBC API types Connection, Statement, ResultSet and so on are not generally thread-safe1. You should not try to use on instance in multiple threads.
If you want to avoid having multiple connections open the normal approach is to use a JDBC connection pool to manage the connections. When a thread needs to talk to the database, it gets a connection from the pool. When it has finished talking to the database, it releases it back to the pool.
In the PostgreSQL / Hikari case:
For PostgreSQL - "Using the driver in a multi-threaded or a servlet environment"
For Hikari - the getConnection() call is thread-safe, but I couldn't find anything that explicitly talked about the thread-safety of the connection object when shared by multiple threads.
1 - I have seen it stated that a spec compliant JDBC driver should be thread-safe, but I could not see where the JDBC spec actually requires this to be so. But even assuming that it does say that somewhere, the threads sharing a connection would need to coordinate very carefully to avoid things like one thread causing another thread's resultset to "spontaneously" close.
I am developing an application using play framework (version 2.8.0), java(version 1.8) with an oracle database(version 12C).
There is only zero or one hit to the database in a day, I am getting below error.
java.sql.SQLRecoverableException: IO Error: Socket read timed out
at oracle.jdbc.driver.T4CConnection.logoff(T4CConnection.java:919)
at oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:2005)
at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:138)
at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:447)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Socket read timed out
at oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:174)
at oracle.net.ns.NIOHeader.readHeaderBuffer(NIOHeader.java:82)
at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:139)
at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:101)
at oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:80)
at oracle.jdbc.driver.T4CMAREngineNIO.prepareForReading(T4CMAREngineNIO.java:98)
at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:534)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:485)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4C7Ocommoncall.doOLOGOFF(T4C7Ocommoncall.java:62)
at oracle.jdbc.driver.T4CConnection.logoff(T4CConnection.java:908)
... 6 common frames omitted
db {
default {
driver=oracle.jdbc.OracleDriver
url="jdbc:oracle:thin:#XXX.XXX.XXX.XX:XXXX/XXXXXXX"
username="XXXXXXXXX"
password="XXXXXXXXX"
hikaricp {
dataSource {
cachePrepStmts = true
prepStmtCacheSize = 250
prepStmtCacheSqlLimit = 2048
}
}
}
}
It seems it is causing due to inactive database connection, How can I solve this?
Please let me know if any other information is required?
You can enable TCP keepalive for JDBC - either be setting directive or by adding "ENABLE=BROKEN" into connection string.
Usually Cisco/Juniper cuts off TCP connection when it is inactive for more that on hour.
While Linux kernel starts sending keepalive probes after two hours(tcp_keepalive_time). So if you decide to turn tcp keepalive on, you will also need root, to change this kernel tunable to lower value(10-15 minutes)
Moreover HikariCP should not keep open any connection for longer than 30 minutes - by default.
So if your FW, Linux kernel and HikariCP all use default settings, then this error should not occur in your system.
See HikariCP official documentation
maxLifetime:
This property controls the maximum lifetime of a connection in the
pool. An in-use connection will never be retired, only when it is
closed will it then be removed. On a connection-by-connection basis,
minor negative attenuation is applied to avoid mass-extinction in the
pool. We strongly recommend setting this value, and it should be
several seconds shorter than any database or infrastructure imposed
connection time limit. A value of 0 indicates no maximum lifetime
(infinite lifetime), subject of course to the idleTimeout setting. The
minimum allowed value is 30000ms (30 seconds). Default: 1800000 (30
minutes)
I have added the below configuration for hickaricp in configuration file and it is
working fine.
## Database Connection Pool
play.db.pool = hikaricp
play.db.prototype.hikaricp.connectionTimeout=120000
play.db.prototype.hikaricp.idleTimeout=15000
play.db.prototype.hikaricp.leakDetectionThreshold=120000
play.db.prototype.hikaricp.validationTimeout=10000
play.db.prototype.hikaricp.maxLifetime=120000
I have a Java application that schedules a cron job after every 1 min. It runs on Glassfish 4. We are using Hibernate with JTA Entity Manager which is container managed for executing the queries on SQL Server database.
JDBC Connection Pool Settings are:
Initial and Minimum Pool Size:16
Maximum Pool Size:64
Pool Resize Quantity:4
Idle Timeout:300
Max Wait Time:60000
JDBC Connection Pool Statistics after 22 Hours run:
NumConnUsed 0count
NumConnAcquired 14404count
NumConnReleased 14404count
NumConnCreated 16count
NumConnFree 16count
The number of acquired connections keeps on incrementing and the Glassfish 4 crashes after around 10 days with below exception.
RAR5117 : Failed to obtain/create connection from connection pool [ com.beonic.tiv5 ]. Reason : com.sun.appserv.connectors.internal.api.PoolingException: java.lang.RuntimeException: Got exception during XAResource.start:
Please suggest how to avoid Glassfish crash.
finally
{
em = null;
ic = null;
}
I think here is the problem you are never commiting or closing the transacction
Giving this example and documentation of JTA check 5.2.2
// BMT idiom
#Resource public UserTransaction utx;
#Resource public EntityManagerFactory factory;
public void doBusiness() {
EntityManager em = factory.createEntityManager();
try {
// do some work
...
utx.commit();
}
catch (RuntimeException e) {
if (utx != null) utx.rollback();
throw e; // or display error message
}
finally {
em.close();
}
This is the correct way of doing a transacction. But you are only nulling the values and nothing more, that's why you your pools and not being closed
Here is more documentation about Transactions
It's hard to tell what is the real cause of the problem, but the problem might be that all your connections have become stale because not used for a long time.
It is a good practice to set up connection validation, which ensures that connections are reopened when closed by the external server.
There is a thorough article about connection pools in Glassfish/Payara, checkout especially the section about Connection validation (using Derby DB in the example):
To turn on connection validation :
asadmin set
resources.jdbc-connection-pool.test-pool.connection-validation-method=custom-validation
asadmin set
resources.jdbc-connection-pool.test-pool.validation-classname=
org.glassfish.api.jdbc.validation.DerbyConnectionValidation
asadmin
set
resources.jdbc-connection-pool.test-pool.is-connection-validation-required=true
I am using Oracle UCP (Universal Connection Pool). I am getting the following error after processing around 100K records.
oracle.ucp.UniversalConnectionPoolException: Invalid life cycle state. Check the status of the Universal Connection Pool
at oracle.ucp.util.UCPErrorHandler.newSQLException(UCPErrorHandler.java:488) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.util.UCPErrorHandler.throwSQLException(UCPErrorHandler.java:163) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:943) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:873) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:863) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
Caused by: oracle.ucp.UniversalConnectionPoolException: Invalid life cycle state. Check the status of the Universal Connection Pool
at oracle.ucp.util.UCPErrorHandler.newUniversalConnectionPoolException(UCPErrorHandler.java:368) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.util.UCPErrorHandler.throwUniversalConnectionPoolException(UCPErrorHandler.java:49) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.util.UCPErrorHandler.throwUniversalConnectionPoolException(UCPErrorHandler.java:80) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.util.UCPErrorHandler.throwUniversalConnectionPoolException(UCPErrorHandler.java:131) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnectionWithoutCountingRequests(UniversalConnectionPoolImpl.java:304) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnectionAndValidate(UniversalConnectionPoolImpl.java:168) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnection(UniversalConnectionPoolImpl.java:143) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.jdbc.JDBCConnectionPool.borrowConnection(JDBCConnectionPool.java:157) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:931) ~[ucp-11.2.0.3.0.jar:11.2.0.3.0]
... 15 more
Here is the code snippet to create the DataSource
PoolDataSource dataSource = PoolDataSourceFactory.getPoolDataSource();
dataSource.setURL("jdbc.url"));
dataSource.setUser("jdbc.user"));
dataSource.setPassword("jdbc.password"));
dataSource.setMaxConnectionReuseCount(100);
dataSource.setInitialPoolSize(50);
dataSource.setMinPoolSize(50);
dataSource.setMaxPoolSize(100);
dataSource.setValidateConnectionOnBorrow(true);
dataSource.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
And then I get the connection from data source as
datasource.getConnection();
In my case, I was using wrong password :|
In my case I encountered this exception during a routine upgrade of a tomcat application. When attempting to interact with an Oracle database (and so getting a connection from the UniversalConnectionPool) the application kept getting this exception.
After reading up on the UniversalConnectionPool (overview-using-ucp-manager), I found this line in some Oracle documentation: "a life cycle state exception occurs if an application attempts to start a pool that has been previously started or if the pool is in a state other than stopped or failed."
That is the closest thing I could find for something related to this exception.
I ended up finding that a mistake in my build config meant that I have an OJDBC driver placed in both my application and externally in the Tomcat and somehow these were then interacting with my datasource and the UCP together at the same time, causing the one initialized last to give this error.
Removing the OJDBC driver from the classpath in my application fixed the issue.
There is on error that appears in my log file every time:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Socket closed
STACKTRACE:
java.net.SocketException: Socket closed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:113)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:160)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1994)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3250)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3179)
at com.mysql.jdbc.Statement.executeQuery(Statement.java:1207)
at com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyStatement.java:35)
It happens always in the same point of the code, when the application does a specific query in the MySQL database.
The query is LIKE THIS:
SELECT M.*, O.id
FROM order_message M
INNER JOIN orders O ON M.order_id = O.id
WHERE O.seller_id = 14224 AND M.sender_user_id <> 14224
ORDER BY M.creation_date DESC
LIMIT 5
I noticed (by EXPLAINING this query) that it always use temporary/filesort to execute. All indexes are properly created and I think these is no way to improve this, but I suspect the query performance or resource utilization is causing the exception error.
I am using amazon RDS
The problem was that my connection pool was configured in a way that any database connection that took longer than 10 seconds would be dropped by the connection pool (c3p0). I was using the unreturnedConnectionTimeout parameter.
The c3p0 documentation page discourage to use this parameter. Ideally, all connections should be properly closed (and thus returned to the pool)
http://www.mchange.com/projects/c3p0/#unreturnedConnectionTimeout
I have increased the parameter to 60 seconds and the problem get solved.