I'm new to c3op, and confused about the use of :
c3p0.idle_test_period
In this link : HowTo configure the C3P0 connection pool
idleTestPeriod : Must be set in hibernate.cfg.xml (or hibernate.properties), Hibernate default:
0, If this is a number greater than 0, c3p0 will test all idle, pooled but unchecked-out
connections, every this number of seconds.
What is the purpose of this kind of test (idel, pooled connections), and the relationship between c3p0.idle_test_period and c3p0.timeout?
The database server may close a connection on its side after a certain amount of time - causing some error in your application, because it'll attempt to send a query on a connection which is no longer available on the server side.
In order to avoid this you can let the pool periodically check a connection (Think of a ping) for it's validity. This is what idle_test_period is for.
timeout is the timespan after which the pool will remove a connection from the pool, because the connection wasn't checked out (used) for a while and the pool contains more connections than c3pO.min_size.
I think this setting is used in hibernate in order to validate pooled connection after every few seconds . Consider a scenario if on database side, password is changed. Hibernate already pooled connections on old password. So it is security breach having pool with wrong password.So when hibernate will validate after few seconds it . It will invalidate that pooled connection.
Related
I run some tomcat application, use jndi connection pool.
In some time connection pool stops to give connections and application hangs.
Seems because some code receives connection and doesn't return it back to the pool.
How can I monitor - which code does it ?
More common - I want to see what all connections do at the moment.
I cannot change application. But I can adjust Tomcat, maybe add some interceptors.
Most connection pool implementations can be configured to detect connections that are not returned to the pool. E.g. for Tomcat's JDBC connection pool there are various configurations options for "abandoned connections" (connections for which the lease expired). If you search for "Abandoned" on this web-page, you'll find the options:
removeAbandoned
removeAbandonedTimeout
logAbandoned
suspectTimeout
As mentioned on the web-page, these settings will add a little overhead but at least your application will not hang. When testing your application, set a low value for removeAbandonedTimeout and a low value for maxActive so that you can catch unreturned connections early.
I never use the connection pool API itself, I always wrap it in a helper.
That way, I can do this in the helper:
private Exception created = (0 == 1) ? new Exception() : null;
When I run into problems like yours, I just change one character (0 -> 1) to have a stack trace of who created this instance in my debugger.
I put Jsch into commons-pool (with spring pool support) with initial success
http://docs.spring.io/spring/docs/3.2.4.RELEASE/spring-framework-reference/htmlsingle/#aop-ts-pool
However:
Should we pool the channels within the Session instead of pooling the sessions? Each Jsch session creates one thread. Pooling Jsch sessions will create x threads. Pooling channels, there will really be only one Jsch thread.
(commons-pool) what happens if the Jsch session went stale? How to regenerate the session in the context of the commons-pool or using spring pool support? How to detect whether it goes stale?
Thanks
Figured out my own question. I will share my project in the next day or two.
Pooling channels are much more effective. There is really no need to create multiple sessions (if the session connects to the same sftp endpoint).
I implemented a JSch connection pool (pooling channels) with spring pool and commons-pool. I will post to the github in the next day or two. The most important question is, what if the connection went stale.
I found out that based on my implementation of 1 Session - multiple channels, and if the connection went stale, the pooled objects (in this case, the channel) will be stale. The pooled object should be invalidated and deleted from the pool. When the connection comes back up, and when new application thread "borrows" from the pool, new pool objects will be created.
To validate my observation, my not-so-automated test:
a) Create a set (say 10) of app threads checking out channel resource from the pool.
b) Have the thread to sleep 20 seconds
c) Create another set of app threads checking out channel resources from the pool.
At a), set breakpoint when i==7, break the connection by "iptable drop (linux) or pfctl -e; pfctl -f /etc/pf.conf (mac, google how to do!)". This first set of app threads will get exception because the channel is broken.
At b), restart the connection
At c), the 2nd set of app threads will be successfully completing the operation because the broken connection has been restored.
I have Java application with Hibernate framework(no spring) connect to MySQL DB , manage connection pooling via c3p0
i try to configure my apllication to read from slave db and write to master db , i have following this link to some extend Master/Slave load balance
let's say if the application already got a session with connection in pool and it need to execute a read-only method , like this
public someReadOnlyMethod()
{
Session session = (get session from current Thread)
//set read-only so that it read from slave db
session.connection().setReadOnly(true);
(...connect to db to do something...)
//set it back in case of this method is followed by write method so that it go to master db
session.connection().setReadOnly(false);
}
Is the pooling create a new connection to connect to db 2 times for read-only and write operation(if so,this will heavily impact performance) or it smart enough to swap the operation to already existing read-only and writable connection pool ?
thx for your advice.
so this has nothing to do with the pool; it's all in the mysql driver. c3p0 will pass your call to setReadOnly (whether true or false) to the underlying Connection, and the Connection will route to the master or the slaves accordingly.
if you don't like how your Connections default (probably by default they are not read only), you can set the read-only property in the onAcquire method of a c3p0 ConnectionCustomizer, and the value use set (true or false) will become th default that c3p0 resets Connections to.
good luck!
tl;dr: It will re-use existing connections whenever you switch setReadOnly(true/false).
JDBC will connect to all servers listed in your connection URL when you do ReplicationDriver().connect(url). Those connections will remain open for re-use no matter how many times you switch setReadOnly().
Source: I just tested Connector/J version 5.1.38 with com.mysql.jdbc.ReplicationDriver.
The database connection is get like below
public Connection getDBConection(){
Context context = new InitialContext();
DataSource dataSource = (javax.sql.DataSource) context.lookup("java:myDataSource");
Connection conn = dataSource.getConnection();
}
For a userA, is each database request should call getDBConnection() once; but no need to control all request use the same connection?
That is, if userA has three database request, then userA should call getDBConnection() three times, and call Connection.closed() after used in each request?
If the userA call getDBConnection() three times (that is, call dataSource.getConnection() three times), is three connection created? Or it is unknown and controlled by weblogic?
I feel very chaos, is it true that there should be one new connection for one database request? or just call DataSource.getConnection() for each database request and the number of new connection created is controlled by web server, no need to think how many connection is actually created.
Every time you call DataSource.getConnection, the data source will retrieve a connection for you. It should be true that the returned connection is not being actively used by anyone else, but it is not necessarily a brand-new connection.
For example, if you use a connection pool, which is a very common practice, then when you call Connection.close, the connection is not actually closed, but instead returns to a pool of available connections. Then, when you call DataSource.getConnection, the connection pool will see if it has any spare connections lying around that it hasn't already handed out. If so, it will typically test that they haven't gone stale (usually by executing a very quick query against a dummy table). If not, it will return the existing connection to the caller. But if the connection is stale, then the connection pool will retrieve a truly new connection from the underlying database driver, and return that instead.
Typically, connection pools have a maximum number of real connections that they will keep at any one time (say, 50). If your application tries to request more than 50 simultaneous connections, DataSource.getConnection will throw an exception. Or in some implementations, it will block for a while until one becomes available, and then throw an exception after that time expires. For a sample implementation, have a look at Apache Commons DBCP.
Hopefully that answers your question!
Does anybody know a way to reestablish/retry again hibernate connection. I mean for example: the remote DB is down and I start my application. Hibernate is not able to establish a connection. it fails. But the application is not closed. Is there a way to say hibernate to try one more time to establish a connecton?
Thanks in advance
You should really go for C3P0 connection pooling: http://www.mchange.com/projects/c3p0/index.html#hibernate-specific
There is a section in C3P0 documentation on that subject: http://www.mchange.com/projects/c3p0/index.html#configuring_recovery
First you have to properly configure c3p0, which in case of using hibernate must happen in c3p0.properties file.
In your c3p0.properties put these properties to retry to reconnect indefinitely every 3 seconds when database is down:
c3p0.acquireRetryAttempts = 0
c3p0.acquireRetryDelay = 3000
c3p0.breakAfterAcquireFailure = false
Also, to avoid broken connections lying in your pool indefinitely, use connection age management:
c3p0.maxConnectionAge = 6000
c3p0.maxIdleTime = 6000
c3p0.maxIdleTimeExcessConnections = 1800
c3p0.idleConnectionTestPeriod = 3600
These may be quite expensive, but helpful if above is not enough:
c3p0.testConnectionOnCheckout = true
c3p0.preferredTestQuery = SELECT 1;
You may also want to check for connection leaks which prevent recovery:
c3p0.debugUnreturnedConnectionStackTraces = true
And finally, make sure that C3P0 is hooked with hibernate correctly, enable debug logging for "com.mchange" package and see if C3P0 tells you anything about itself. It should state configuration properties which are loaded, so see if it's all there.
I hope this helps.
C3P0 is the internal connection-pool implementation for hibernate.
Add "hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvider"
in hibernate properties file. Create a file c3p0.properties setting the parameters accordingly. This file & c3p0-x.jar must be in classpath.
c3p0.properties
c3p0.idleConnectionTestPeriod : If this is a number greater than 0, c3p0 will test all idle, pooled but unchecked-out connections, every this number of seconds.
c3p0.testConnectionOnCheckout : Use only if necessary. Expensive. If true, an operation will be performed at every connection checkout to verify that the connection is valid. Better choice: verify connections periodically using idleConnectionTestPeriod.
There are several other properties that can be configured in hibernate.properties & c3p0.properties.
May be, you try to call method .getCurrentSession() instead .openSession()?
If connection falls you must establish new one.
I hope this helps.