I have implemeted a web application using Struts2 and Hibernate 4.3.5
I am facing some problems and need some clarification on some connection pooling properties.
To make connection pooling with hibernate I have used c3p0-0.9.1.jar
Below are the connection pooling Properties of c3p0 in hibernate.cfg.xml
<!-- Connection Pooling using c3p0 -->
<property name="connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">10</property>
<property name="hibernate.c3p0.timeout">500</property>
<property name="hibernate.c3p0.max_statements">2</property>// I heard somewhere that this property improves performance.. But dont know How it improves
<property name="hibernate.c3p0.idle_test_period">300</property>
<property name="hibernate.c3p0.preferredTestQuery">
select sysdate from dual;
</property>
In my application I have used * session-per-request* patterns by setting following property in hibernte-cfg.xml
<!-- To open the connection when the request hits. -->
<property name="hibernate.current_session_context_class">thread</property>
When I use this I am getting Session as following
//Here MyHiberantaeUtil is my own class having singleton pattern and Static block
// To read hibernate-cfg.xml and will create SessionFactroy object.
SessionFactory sessionFactory = MyHibernateUtil.getSessionFactory();
Session session = sessionFactory.getCurrentSession();
As per the documentation .. When I use this pattern connection should be created when request is requested and should be closed when request completed.
For my case Its creating connection .. i.e I can get Session Using this getCurrentSession() But its not closing when The request ended.
To test connection pooling I have used jconsole tool in my jdk
Where I can see
`numBusyConection = 2(If that is two requests.)`
I would like to know is there any thing I missed . Why thread property is not working as expected ?
And how to say that Request has completed and request has generated ?
Is session-per-request is best pattern or better to go any other patterns ?
I did not find this class `org.hibernate.connection.C3P0ConnectionProvider` in any jar files.. But My application is connecting to DB and working fine.
Please help thanks in advance.
The JDBC connection is opened when first needed and closed when the Session is closed. You need to manually close the session when you go out of your Service layer:
session.close();
Otherwise the connections won't be closed and you'll starve the connection pool.
It's a better idea to use Spring, because it has support for automatically creating/closing sessions for you. Your MyHibernateUtil is no match for what Spring has to offer:
local transactions
global transactions
transaction annotations and automatic commit on success or rollback on runtime exceptions
Setting up Spring with Hibernate is as easy as using the HibernateUtil class.
Related
I am working on an application which is based on Hibernate 5.1.9.Final and Spring 4.3.6.RELEASE
What I am trying to discover is what the default values for connecting to the relational database are, such as connection_timeout, maxConnections, etc.
I am trying to understand this existing project, and I don't have any of this configuration on that, but still, I see several connections on the Database -> Postgres.
How to discover what the defaults are? Are there defaults values? And what is the default connection pool framework? C3p0?
The default connection pooling mechanisms in hibernate are not production, not even performance testing ready. Here is a quotation from hibernate documentation
Hibernate's own connection pooling algorithm is, however, quite
rudimentary. It is intended to help you get started and is not
intended for use in a production system, or even for performance
testing. You should use a third party pool for best performance and
stability. Just replace the hibernate.connection.pool_size property
with connection pool specific settings. This will turn off Hibernate's
internal pool. For example, you might like to use c3p0.
The property setting up the number of pooled connections is:
hibernate.connection.pool_size
Here is an example C3P0 configuration:
hibernate.connection.driver_class = org.postgresql.Driver
hibernate.connection.url = jdbc:postgresql://localhost/mydatabase
hibernate.connection.username = myuser
hibernate.connection.password = secret
hibernate.c3p0.min_size=5
hibernate.c3p0.max_size=20
hibernate.c3p0.timeout=1800
hibernate.c3p0.max_statements=50
hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
recently Hikari is very popular choice for connection pooling.
https://brettwooldridge.github.io/HikariCP/
Here is an example set of Hikari properties:
<property name="hikari.dataSource.cachePrepStmts">true</property>
<property name="hikari.dataSource.prepStmtCacheSize">250</property>
<property name="hikari.dataSource.prepStmtCacheSqlLimit">2048</property>
<property name="hikari.dataSource.useServerPrepStmts">true</property>
<property name="hikari.maximumPoolSize">30</property>
<property name="hikari.idleTimeout">30000</property>
<!-- Database connection properties -->
<property name="hibernate.hikari.dataSourceClassName">com.mysql.jdbc.jdbc2.optional.MysqlDataSource</property>
<property name="hikari.dataSource.url">jdbc:mysql://127.0.0.1/sample</property>
<property name="hikari.dataSource.user">root</property>
<property name="hikari.dataSource.password">tiger</property>
source:
https://self-learning-java-tutorial.blogspot.com/2016/01/hibernate-hikaricp-example.html
My application has long running transactions and hence I tried the option session.close() at the end of every method to ensure that the connection objects are not held indefinitely for long time.
When session.close() option is used, I could see that the Hibernate's session object and the corresponding Connection object obtained from session.connection() are destroyed properly. But the issue is with the connection pool. The connection obtained by the session is not released back to the connection pool even after closing the session. Other requests are found waiting for connection from the pool.
I am using JTA transaction in my application. In hibernate.cfg.xml, I have set connection.release_mode to auto (default) and connection.autocommit to true.
Has anyone faced this issue? Please let me know what am I missing here.
Follow-up: This is my hibernate configuration file details:
<property name="connection.datasource">MXoraDS</property>
<property name="dialect">org.hibernate.dialect.Oracle9Dialect</property>
<property name="connection.release_mode">after_statement</property>
<property name="hibernate.transaction.manager_lookup_class">org.hibernate.transaction.WeblogicTransactionManagerLookup</property>
<property name="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</property>
<property name="hibernate.current_session_context_class">org.hibernate.context.JTASessionContext</property>
<property name="transaction.auto_close_session">true</property>
<property name="max_fetch_depth">2</property>
We use JSF and EJB 2.1 at the application layer connecting to Oracle DB. The after_statement doesn't seem to release the connection to the pool. Please let me know if u need any more details.
I am using JTA transaction in my application. In hibernate.cfg.xml, I have set connection.release_mode to auto (default) and connection.autocommit to true.
Could you try to define the hibernate.connection.release_mode property to after_statement explicitly? I know this is supposed to be the default but, depending on your context (could you be using Spring?), auto might not behave as expected (see here and here).
For reference, here is what the Table 3.4. Hibernate JDBC and Connection Properties writes about the property hibernate.connection.release_mode:
Specifies when Hibernate should
release JDBC connections. By default,
a JDBC connection is held until the
session is explicitly closed or
disconnected. For an application
server JTA datasource, use
after_statement to aggressively
release connections after every JDBC
call. For a non-JTA connection, it
often makes sense to release the
connection at the end of each
transaction, by using
after_transaction. auto will
choose after_statement for the JTA
and CMT transaction strategies and
after_transaction for the JDBC
transaction strategy.
e.g. auto (default) | on_close | after_transaction |
after_statement
This setting only affects Sessions
returned from
SessionFactory.openSession. For
Sessions obtained through
SessionFactory.getCurrentSession, the
CurrentSessionContext implementation
configured for use controls the
connection release mode for those
Sessions. See Section 2.5,
“Contextual sessions”
If it doesn't help, please add more details about your environment and configuration (Spring?), how you get the session, etc.
if you are using the JDBCTransactionManager, the connection will be returned to the connectionpool when the transactions ends.
I am using spring boot and hibernate over jpa with tomcat connection pooling. Can you please help me understanding how spring uses DB connections during transactions. For example consider following scenario:
We have DB connection pool of 2 connections.
Spring starts a transaction i.e. call method decorated with #Transactional annotation.
This method do a DB update
The calls an external service
As response is received from the external service, it updates DB and return.
Spring commits the transaction
Assuming the external service(step 4) takes 1 minute to complete, how many DB connections will be available in the DB pool?. Assuming, spring keeps hold of DB connection until the transaction completes, there will be only 1 DB connection available for any request received during this time and if we received more than 1 requests, they will have to wait for DB connection.
Please confirm my understanding and if it is correct, suggest how I can handle this situation in a high transaction volume system.
Thanks
First your understanding is correct. See the spring documentation about declarative transaction management.
I guess you do the external service call within the transaction, because you want the database changes to be rollbacked in case of an exception. Or in other words you want the db updates to reflect the state of the external service call.
If so you can't move it out the transaction boundary. In this case you should either increase your connection pool size or maybe you can delegate long running transactions to a dedicated server node that handles them. This would keep e.g. a "main" server node that handles user requests away from long running transactions.
And you should think about the data consistency. Is it really necessary that the db update must be synchronized with the external service call? Can the external service call be moved out of the transaction boundary?
You can specify initial size and maximum size of connection pool as per your requirement(depends on the performance of your application).
For example,
<bean id="springDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" >
<property name="url" value="jdbc:oracle:thin:#localhost:1521:SPRING_TEST" />
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="username" value="root" />
<property name="password" value="root" />
<property name="removeAbandoned" value="true"/>
<property name="initialSize" value="20" />
<property name="maxActive" value="30" />
</bean>
this will create 20 database connection as initialSize is 20 and goes up to 30 Database connection if required as maxActive is 30. you can customize your database connection pool by using different properties provided by Apache DBCP library. Above example is creating connection pool with Oracle 11g database and i am using oracle.jdbc.driver.OracleDriver comes along with ojdbc6.jar or ojdbc6_g.jar
I'm using JPA (Hibernate as provider), Glassfish and MySQL. Everything works great in development, but when I deploy the app to a test server and let it run (largely idle) overnight, I'm usually greeted with this in the morning:
[#|2011-03-09T15:06:00.229+0000|INFO|glassfish3.0.1|javax.enterprise.system.std.com.sun.enterprise.v3.services.impl|_ThreadID=23;_ThreadName=Thread-1;|ERROR [htt\
p-thread-pool-8080-(1)] (JDBCTransaction.java:91) - JDBC begin failed
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 41,936,868 milliseconds ago. The last packet \
sent successfully to the server was 41,936,868 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expirin\
g and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connec\
tion property 'autoReconnect=true' to avoid this problem.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1118)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3321)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1940)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2562)
at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:4956)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:87)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1473)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:60)
I tried using the following in my persistence.xml, but it didn't help:
<property name="hibernate.c3p0.min_size" value="5"/>
<property name="hibernate.c3p0.max_size" value="20"/>
<property name="hibernate.c3p0.idleTestPeriod" value="30"/>
<property name="hibernate.c3p0.timeout" value="0"/>
<property name="hibernate.c3p0.max_statements" value="0"/>
So that's the C3p0 configuration; it's entirely possible I'm missing the part that actually tells hibernate "hey, use c3p0".
I'm about to try the suggestion that's right there in the error message: add autoReconnect=true to my JDBC URL, but this is really starting to feel like cargo-cult development at this point. I would appreciate some guidance on the proper way to address this issue. It's hard to debug, because the test cycle is effectively "run it overnight, see what happens in the morning".
I should probably mention how I'm actually using connections in my app. I have a custom Servlet Filter that intercepts all requests. It creates an EntityManager, stores it in a ThreadLocal, and is closed by the filter in a catch/finally block. All my entities obtain a reference to the EntityManager from the ThreadLocal.
It's entirely possible that my filter is at fault, but as it only seems to happen after idle periods, I suspect something else is wrong. I do intend to move to Seam/Weld when I have a chance to catch my breath, but for now I'm relying on this filter.
Edit: here's the TL;DR solution:
use your container's connection pool, if you can (thanks, #partenon)
make sure your connection pool uses connection validation (thanks, #matt b)
In my case, I had to go into the Glassfish console under Resources/JDBC/Connection Pools, Advanced Tab, and then enable Connection Validation:
This was really the crucial step. You also probably want to set Validate At Most Once to something reasonable, say 100 seconds. If you're using C3P0 or similar, make sure you configure idle_test_period and preferredTestQuery.
Whatever you end up doing, it's important to test out your changes to see if they have the desired effect. To make the timeout happen faster in MySQL, you can temporarily set the wait_timeout to something low like 30 seconds by editing my.cnf. This was a tremendous help in debugging this problem, as it allowed me to test changes in seconds, rather than hours.
I think the real question is: why are you using an external connection pooling mechanism instead of using Glassfish' own pooling? Your app server is better suited to provide this kind of service to your application. "External" connection pooling mechanisms are better suited for standalone applications, not in-container applications.
You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
Just a shot in the dark, but have you taken a look at setting the autoReconnect=true property in your JDBC driver? Or consider disabling the server-side setting for client connection timeouts.
I think the property to set the connection test period is idle_test_period, not idleTestPeriod as per C3P0 documentation here. So you should be using:
<property name="hibernate.c3p0.idle_test_period" value="30"/>
instead.
I was getting the same problem and it took time to figure out the solution.
I use Hibernate 4.0.1 and mysql 5.1(no spring framework) and I was facing the issue. First make sure that you configured the c3p0 jars properly which are essential.
I used these properties in hibernate.cfg.xml
<property name="hibernate.c3p0.validate">true</property>
<property name="hibernate.connection.provider_class">org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="hibernate.c3p0.preferredTestQuery">SELECT 1;</property>
<property name="hibernate.c3p0.testConnectionOnCheckout">true</property>
<property name="hibernate.c3p0.idle_test_period">10</property>
<property name="hibernate.c3p0.acquireRetryAttempts">5</property>
<property name="hibernate.c3p0.acquireRetryDelay">200</property>
<property name="hibernate.c3p0.timeout">40</property>
But it's of no use 'cause C3p0 was still taking the default properties not the properties which I set in hibernate.cfg.xml, You can check it in logs. So, I searched many websites for right solution and finally I came up with this. remove the C3p0 properties in cfg.xml and create c3p0-config.xml in the root path(along with cfg.xml) and set properties as follows.
<c3p0-config>
<default-config>
<property name="automaticTestTable">con_test</property>
<property name="checkoutTimeout">40</property>
<property name="idleConnectionTestPeriod">10</property>
<property name="initialPoolSize">10</property>
<property name="maxPoolSize">20</property>
<property name="minPoolSize">5</property>
<property name="maxStatements">50</property>
<property name="preferredTestQuery">SELECT 1;</property>
<property name="acquireRetryAttempts">5</property>
<property name="acquireRetryDelay">200</property>
<property name="maxIdleTime">30</property>
</default-config>
</c3p0-config>
but if you run, ORM takes the jdbc connection but not C3p0 connection pool 'cause we should add these properties in hibernate.cfg.xml
<property name="hibernate.c3p0.validate">true</property>
<property name="hibernate.connection.provider_class">org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider</property>
now everything works fine(At least it worked fine for me) and the issue is solved.
check the following for references.
http://www.mchange.com/projects/c3p0/index.html#configuring_connection_testing
https://community.jboss.org/wiki/HowToConfigureTheC3P0ConnectionPool
I hope this solves your problem.
After reading previous questions about this error, it seems like all of them conclude that you need to enable XA on all of the data sources. But:
What if I don't want a distributed
transaction? What would I do if I want to
start transactions on two different
databases at the same time, but
commit the transaction on one database
and roll back the transaction on
the other?
I'm wondering how my code
actually initiated a distributed
transaction. It looks to me like I'm
starting completely separate
transactions on each of the
databases.
Info about the application:
The application is an EJB running on a Sun Java Application Server 9.1
I use something like the following spring context to set up the hibernate session factories:
<bean id="dbADatasource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/dbA"/>
</bean>
<bean id="dbASessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dbADatasource" />
<property name="hibernateProperties">
hibernate.dialect=org.hibernate.dialect.Oracle9Dialect
hibernate.default_schema=schemaA
</property>
<property name="mappingResources">
[mapping resources...]
</property>
</bean>
<bean id="dbBDatasource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/dbB"/>
</bean>
<bean id="dbBSessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dbBDatasource" />
<property name="hibernateProperties">
hibernate.dialect=org.hibernate.dialect.Oracle9Dialect
hibernate.default_schema=schemaB
</property>
<property name="mappingResources">
[mapping resources...]
</property>
</bean>
Both of the JNDI resources are javax.sql.ConnectionPoolDatasoure's. They actually both point to the same connection pool, but we have two different JNDI resources because there's the possibility that the two, completely separate, groups of tables will move to different databases in the future.
Then in code, I do:
sessionA = dbASessionFactory.openSession();
sessionB = dbBSessionFactory.openSession();
sessionA.beginTransaction();
sessionB.beginTransaction();
The sessionB.beginTransaction() line produces the error in the title of this post - sometimes. I ran the app on two different sun application servers. On one runs it fine, the other throws the error. I don't see any difference in how the two servers are configured although they do connect to different, but equivalent databases.
So the question is
Why doesn't the above code start
completely independent transactions?
How can I force it to start
independent transactions rather than
a distributed transaction?
What configuration could cause the difference in
behavior between the two application
servers?
Thanks.
P.S. the stack trace is:
Local transaction already has 1 non-XA Resource: cannot add more resources.
at com.sun.enterprise.distributedtx.J2EETransactionManagerOpt.enlistResource(J2EETransactionManagerOpt.java:124)
at com.sun.enterprise.resource.ResourceManagerImpl.registerResource(ResourceManagerImpl.java:144)
at com.sun.enterprise.resource.ResourceManagerImpl.enlistResource(ResourceManagerImpl.java:102)
at com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:216)
at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:327)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
at com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:108)
at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:82)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:142)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:85)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1354)
at [application code ...]
1 Why doesn't the above code start completely independent transactions?
The app. server manages the transaction for you which can, if necessary, be a distributed transaction. It enlists all the participants automatically. When there's only one participant, you don't notice any difference with a plain JDBC transaction, but if there are more than one, a distributed transaction is really needed, hence the error.
2 How can I force it to start independent transactions rather than a
distributed transaction?
You can configure the datasource to be XA or Local. The transactional behavior of Spring/Hibernate can also be configured to use either regular JDBC transactions or delegate the management of transactions to the JTA distributed transaction manager.
I suggest you switch the datasource to non-XA and try to configure Spring/Hibernate to use the JDBC transactions. You should find the relevant information in the documentation, here what I suspect is the line to change:
<bean id="txManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager" />
This should essentially means that you are not using the app. server distributed transaction manager.
3 What configuration could cause the difference in behavior between the
two application servers?
If you have really exactly the same app and configuration, this means that in one case only one participant is enlisted in the dist. transaction, while there are two in the 2nd case. One participant corresponds to one physical connection to a database usually. Could it be that in one case, you use two schema on two different databases, while in the 2nd case you use two schema on the same physical database? A more probable explanation would be that the datasource were configured differently on the two app. server.
PS: If you use JTA distributed transactions, you should use UserTransaction.{begin,commit,rollback} rather than their equivalent on the Session.
After reading previous questions about this error, it seems like all of them conclude that you need to enable XA on all of the data sources.
No, not all, all except one (as the exception is saying) if your application server supports Logging Last Resource (LLR) optimization (which allows to enlist one non-XA resource in a global transaction).
Why doesn't the above code start completely independent transactions?
Because you aren't. When using beginTransaction() behind EJB Session Beans, Hibernate will join the JTA transaction (refer to the documentation for full details). So the first call just works but the second call means enlisting another transactional resource in the current transaction. And since none of your resources are XA, you get an exception.
How can I force it to start independent transactions rather than a distributed transaction?
See #ewernli answer.
What configuration could cause the difference in behavior between the two application servers?
No idea. Maybe one of them is using at least one XA datasource.