Does it make more sense to have to connection pool at the JNDI level or at the webapp level? For example, I could create at simply javax.sql.DataSource thusly:
<Context antiJARLocking="true">
<Resource name="jdbc/myDataSource"
auth="Container"
type="javax.sql.DataSource"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/myDataSource" user="user" password="password" />
</Context>
and then configure the pool in Spring thusly:
<bean id="myDataSource" class="com.mchange.v2.c3p0.DataSources"
factory-method="pooledDataSource">
<constructor-arg>
<jee:jndi-lookup jndi-name="java:comp/env/jdbc/myDataSource" />
</constructor-arg>
</bean>
Or, I could configure the pool directly in JNDI itself:
<Resource name="jdbc/myDataSource"
auth="Container"
factory="org.apache.naming.factory.BeanFactory"
type="com.mchange.v2.c3p0.ComboPooledDataSource"
driverClassName="com.mysql.jdbc.Driver"
jdbcUrl="jdbc:mysql://localhost/myDataSource"
user="user" password="password"
minPoolSize="3"
maxPoolSize="15"
maxIdleTime="5000"
idleConnectionTestPeriod="300"
acquireIncrement="3" />
Leaving this spring:
<jee:jndi-lookup id="myDataSource" jndi-name="java:comp/env/jdbc/myDataSource" />
In both cases, the myDataSource spring bean would be a c3p0 connection pooled data source, but which one is better? I am thinking that having the pool in JNDI makes the most sense, but the downside to that is that you must push your c3p0 lib to the servlet container level which could cause conflicts with existing servlets if they currently use a different version. However, putting it in JNDI means your applications dont have to worry about pooling at all. What do y'all think?
You don't need to pool the data source, obtained from JNDI, as it is already pooled :)
The only difference between having a container-manager pool v.s. application pool is that in 1st case you have an ability to monitor the workload on the pool using the standard interfaces (e.g. JBoss console). Then administrator of the application server manages the decision about increasing the pool size, if necessary. He may also switch applications to another DB server (e.g. planned migration from MySQL to Oracle). The disadvantage is that you need slightly more efforts to setup JNDI test data source for your unit tests (see here).
And in 2nd case, yes, you have to package either DBCP or c3p0 plus the JDBC driver together with you application. In this case it is not so easy to collect the statistics about all pools for all application running in Tomcat. Also migration to newer JDBC driver (MySQL 4 to MySQL 5) cannot be done for all applications at once. And connection properties are wired to your application, even if you use a .property file (so changing that needs reassembling and redeployment of the project). Perhaps you don't need all that, as you have only application, one DB and no management overhead.
More topics on this subject:
10 pools with 5 connections v.s. 1 pool with 50 connection (that is what happens when each application starts using it's own pool)
DBCP vs C3P0
Connection pool implementations
Why to use a connection pool?
Related
I have been searching high and low and gathered bits and pieces, apologies if this has already been answered elsewhere but I am unable to find it.
I am writing a web application in Java with Tomcat, and SQL Azure in the backend.
There are multiple servlets accessing the SQL Azure DB. I would like to use Connection Pools as managed by Tomcat 8.5
My application context.xml in META-INF is as follows:
<Context>
<Resource name="jdbc/sqlazure"
auth="Container"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
type="javax.sql.DataSource"
maxIdle="30"
username="[username]"
password="[password]"
url="jdbc:sqlserver://[database]:1433;database=BackInfo;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30"
removeAbandonedTimeout="30"
logAbandoned="true" />
</Context>
In the Java Code, I access the typical way:
InitialContext ic = new InitialContext();ds = (DataSource)ic.lookup("java:comp/env/jdbc/sqlazure");
try(Connection con = ds.getConnection())....
Everything seems to work, so let me confirm my understanding here:
I do not need to specify a separate web.xml since I'm using Tomcat 8.5. Correct ?
Azure will automatically create a pool when I connect in this manner. The number of connections in the pool etc cannot (do not need to?) be configured.
Before I realized I would have other servlets that need to access the database, I had one servlet directly creating a Datasource via SQLServerConnectionPoolDataSource and getting a connection from there. The documentation states:
SQLServerConnectionPoolDataSource is typically used in Java Application Server environments that support built-in connection pooling and require a ConnectionPoolDataSource to provide physical connections, such as Java Platform, Enterprise Edition (Java EE) application servers that provide JDBC 3.0 API spec connection pooling.
Does this mean that when I use SQLServerConnectionPoolDataSource directly to ask for a connection, it will check if Tomcat supports pooling and then is basically using JDBC mechanisms to create a pool of SQL Azure connections managed by Tomcat ?
When getting the DataSource via Tomcat JNDI lookup, using SQLServerDriver as specified in context.xml. When the web app starts up, it will check context.xml and use SQLServerDriver to connect to SQL Azure, check if pooling is supported, if yes then Tomcat is using the driver to automatically creating a connection pool DataSource that it returns ?
I also just thought of one other question. Would it make sense to have a Singleton DataSource class that returns a reference to the pooled connection DataSource, or would it be better to have each servlet lookup the datasource in its init() and store in a private variable ?
Thanks
Based on my understanding, the jdbc connection pool for SQL Server is created by Java Application, not Azure does. My suggestion is that you need to refer to the Tomcat offical documents below to understand the JNDI Resources & JDBC DataSource.
JNDI Resources: http://tomcat.apache.org/tomcat-8.5-doc/jndi-resources-howto.html
JDBC DataSource: http://tomcat.apache.org/tomcat-8.5-doc/jndi-datasource-examples-howto.html
I am using Tomcat 7 connection pooling (as a Tomcat resource in server.xml and context.xml) in a web application and it works.
My question is: Is it possible to "tell"/"force" tomcat to dispose the connection pool after it is created?
The reason I ask is the following:
I am using H2 and run into some "racing" issue on shutdown.
H2 remains open as long as there is a connection open but Tomcat does not dispose the connection pool and so connections remain open. And as a result I have various issues on shutdown.
I found that I could issue an SQL SHUTDOWN command to close H2 but I want to explore all the alternatives for my case.
So is it possible to "tell"/"force" tomcat to dispose the connection pool (at least on shutdown)?
I think you can try to put debug log on and check whether its issue with connection not released by application or something else like datasource configuration parameter in server.xml.
mostly it should be the case where application is not releasing connection.
How about writing a custom ServletContextListener and then having it shut down the pool when the context is destroyed. Here's an article about ServletContextListener being used to create shutdown hooks:
shutdown hook for java web application
The API for context listeners seems pretty straightforward:
http://docs.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html
please see my answer in : Context specific JNDI parameters problem in tomcat 6. There's a lot of stuff you can do with jndi resource.
<Resource name="jdbc/NAME" auth="Container" type="javax.sql.DataSource"
maxActive="100" minIdle="10" maxWait="10000" removeAbandoned="true"
removeAbandonedTimeout="60" logAbandoned="true"
testWhileIdle="true" testOnBorrow="true" testOnReturn="false"
timeBetweenEvictionRunsMillis="5000"
validationQuery="SELECT 1" initialSize="10"
username="usrname" password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/databb?autoReconnect=true"/>
As described in documentation http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Resource_Definitions it possible to set closeMethod of container datasource. I'm not sure about 'disposing' connection pool, but i think it worth a try.
Name of the zero-argument method to call on a singleton resource when
it is no longer required. This is intended to speed up clean-up of
resources that would otherwise happen as part of garbage collection.
This attribute is ignored if the singleton attribute is false. If not
specificed, no default is defined and no close method will be called.
Also you can deploy DataSource programmatically and start/stop it in ServletContextListener, example for dbcp (sorry, it from my unit-tests, but easy to rewrite):
import org.apache.commons.dbcp.BasicDataSource;
static BasicDataSource bds = new BasicDataSource();
#BeforeClass
public void setUp() throws Exception {
bds.setDefaultAutoCommit(false);
bds.setDriverClassName("org.h2.Driver");
bds.setInitialSize(0);
bds.setMaxActive(2);
bds.setMaxWait(10000);
bds.setPassword(null);
bds.setUrl("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1");
bds.setUsername("sa");
bds.setValidationQuery("select 1 as test");
}
#AfterClass
public void tearDown() throws Exception {
bds.close();
}
Maybe I don't understand the question. To be more clear, if you shutdown Tomcat (and hence it's JVM), then neither Tomcat nor it's connection pools, nor any of it's daemon threads would still be alive ... and thus no memory references would be left in tack.
How exactly have you come to the conclusion that a Tomcat resource has a reference to a connection pool, after Tomcat is shutdown? You wouldn't be able to see that in a profiler, and frankly, it doesn't quite make sense. For example, is it possible that the OS keeps connections open for a short period of time before destroying them? Have you tried the same tests using the same OS, but instead with a different container, something like Jetty? Are you using keep-alives, or some form of persistent connections?
If you want to get down-and-dirty with Tomcat's management of a resource exposed through JNDI, you can make your own DataSource implementation that delegates to the H2 DataSource. You can do this my using the factory-method attribute within your server.xml (or context.xml)
Here's a working example on github : https://github.com/tvollmer/connection-factory
Beyond that, it would help if you could further clarify what you mean by "various issues on shutdown". Those details matter, and it's not clear to me how you've logically gone from "various issues" to saying that Tomcat "does not dispose the connection pool". You could be dealing with 2 completely separate issues here, and you may want to check out antiResourceLocking and antiJARLocking settings for Tomcat on Windows.
After reading previous questions about this error, it seems like all of them conclude that you need to enable XA on all of the data sources. But:
What if I don't want a distributed
transaction? What would I do if I want to
start transactions on two different
databases at the same time, but
commit the transaction on one database
and roll back the transaction on
the other?
I'm wondering how my code
actually initiated a distributed
transaction. It looks to me like I'm
starting completely separate
transactions on each of the
databases.
Info about the application:
The application is an EJB running on a Sun Java Application Server 9.1
I use something like the following spring context to set up the hibernate session factories:
<bean id="dbADatasource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/dbA"/>
</bean>
<bean id="dbASessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dbADatasource" />
<property name="hibernateProperties">
hibernate.dialect=org.hibernate.dialect.Oracle9Dialect
hibernate.default_schema=schemaA
</property>
<property name="mappingResources">
[mapping resources...]
</property>
</bean>
<bean id="dbBDatasource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/dbB"/>
</bean>
<bean id="dbBSessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dbBDatasource" />
<property name="hibernateProperties">
hibernate.dialect=org.hibernate.dialect.Oracle9Dialect
hibernate.default_schema=schemaB
</property>
<property name="mappingResources">
[mapping resources...]
</property>
</bean>
Both of the JNDI resources are javax.sql.ConnectionPoolDatasoure's. They actually both point to the same connection pool, but we have two different JNDI resources because there's the possibility that the two, completely separate, groups of tables will move to different databases in the future.
Then in code, I do:
sessionA = dbASessionFactory.openSession();
sessionB = dbBSessionFactory.openSession();
sessionA.beginTransaction();
sessionB.beginTransaction();
The sessionB.beginTransaction() line produces the error in the title of this post - sometimes. I ran the app on two different sun application servers. On one runs it fine, the other throws the error. I don't see any difference in how the two servers are configured although they do connect to different, but equivalent databases.
So the question is
Why doesn't the above code start
completely independent transactions?
How can I force it to start
independent transactions rather than
a distributed transaction?
What configuration could cause the difference in
behavior between the two application
servers?
Thanks.
P.S. the stack trace is:
Local transaction already has 1 non-XA Resource: cannot add more resources.
at com.sun.enterprise.distributedtx.J2EETransactionManagerOpt.enlistResource(J2EETransactionManagerOpt.java:124)
at com.sun.enterprise.resource.ResourceManagerImpl.registerResource(ResourceManagerImpl.java:144)
at com.sun.enterprise.resource.ResourceManagerImpl.enlistResource(ResourceManagerImpl.java:102)
at com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:216)
at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:327)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
at com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:108)
at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:82)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:142)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:85)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1354)
at [application code ...]
1 Why doesn't the above code start completely independent transactions?
The app. server manages the transaction for you which can, if necessary, be a distributed transaction. It enlists all the participants automatically. When there's only one participant, you don't notice any difference with a plain JDBC transaction, but if there are more than one, a distributed transaction is really needed, hence the error.
2 How can I force it to start independent transactions rather than a
distributed transaction?
You can configure the datasource to be XA or Local. The transactional behavior of Spring/Hibernate can also be configured to use either regular JDBC transactions or delegate the management of transactions to the JTA distributed transaction manager.
I suggest you switch the datasource to non-XA and try to configure Spring/Hibernate to use the JDBC transactions. You should find the relevant information in the documentation, here what I suspect is the line to change:
<bean id="txManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager" />
This should essentially means that you are not using the app. server distributed transaction manager.
3 What configuration could cause the difference in behavior between the
two application servers?
If you have really exactly the same app and configuration, this means that in one case only one participant is enlisted in the dist. transaction, while there are two in the 2nd case. One participant corresponds to one physical connection to a database usually. Could it be that in one case, you use two schema on two different databases, while in the 2nd case you use two schema on the same physical database? A more probable explanation would be that the datasource were configured differently on the two app. server.
PS: If you use JTA distributed transactions, you should use UserTransaction.{begin,commit,rollback} rather than their equivalent on the Session.
After reading previous questions about this error, it seems like all of them conclude that you need to enable XA on all of the data sources.
No, not all, all except one (as the exception is saying) if your application server supports Logging Last Resource (LLR) optimization (which allows to enlist one non-XA resource in a global transaction).
Why doesn't the above code start completely independent transactions?
Because you aren't. When using beginTransaction() behind EJB Session Beans, Hibernate will join the JTA transaction (refer to the documentation for full details). So the first call just works but the second call means enlisting another transactional resource in the current transaction. And since none of your resources are XA, you get an exception.
How can I force it to start independent transactions rather than a distributed transaction?
See #ewernli answer.
What configuration could cause the difference in behavior between the two application servers?
No idea. Maybe one of them is using at least one XA datasource.
We have a web application right now that we deploy a copy for each client. Our current deployment strategy is to create a uniquely named jdbc connection pool for each instance.
so say jdbc/client. They are specified like this...
< Context path="/"
reloadable="true"
docBase="\home\client\ROOT"
debug="5" >
< Resource name="jdbc/client"
auth="Container"
type="javax.sql.DataSource"
maxActive="100"
maxIdle="30"
validationQuery="SELECT 1"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="300000"
numTestsPerEvictionRun="6"
minEvictableIdleTimeMillis="1800000"
maxWait="10000"
username="user"
password="pass"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://databaseserver:3306/client ?zeroDateTimeBehavior=convertToNull&jdbcCompliantTruncation=false"/>
< /Context>
The question is, if I were to standardize it so that instead of unique names the connection pool is called jdbc/database on all deployed instances, is there a chance of database crossing, ie one customer in another customer's database, or are these localized to a specific deployed instance?
Thoughts?
Thanks,
Scott
No. The scope of the data source name is one Tomcat instance.
If you are starting a separate Tomcat process for each customer, all that matters is how the data source is configured, not what Tomcat calls it. As long as each data source is configured to use a different database, there won't be any cross talk.
This depends on how you deploy application for each client,
If each client gets their own Tomcat installation (they have different CATALINA_HOME), there is no chance for it to cross.
If they all use the same installation but run as different host in Tomcat, you need to make sure you don't define the datasource in conf/context.xml, which is shared by all hosts.
If all clients share the same Tomcat instances and they are simply different web apps, more attention is required. You need to define the datasource either in META-INF/context.xml or WEB-INF/web.xml. For further isolation, you should copy dbcp.jar to WEB-INF/lib of each application so they use their own DBCP instance.
If you're defining the JNDI DataSource resource within the Context for a deployment of the application, I believe you could even have multiple copies of the same application running in the same Tomcat instance and using the same JNDI name to access different databases. If each application instance is running in a different instance of Tomcat completely, there is certainly no way that one instance would be referring to the database specified for another instance.
No there is no chance of database crossing becoz the scope of the data source name is one Tomcat instance and you can have multiple data source in single tomcat instance .... so as long as data source is different there is no chance of database crossing.....
I am using Apache DBCP with JNDI in my tomcat container to pool connections to a mysql database. Everything works fine but a problem i am seeing is that once a pconnection is pooled it is never released. So after a load spike connection sit their forever sleeping. Is their a way to shrink the pool through context.xml parameters? here is my ocnfiguration:
defaultAutoCommit="false"
defaultTransactionIsolation="REPEATABLE_READ"
auth="Container"
type="javax.sql.DataSource"
logAbandoned="true"
removeAbandoned="true"
removeAbandonedTimeout="300"
maxActive="-1"
initialSize="15"
maxIdle="10"
maxWait="10000"
username="user"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/mydb"/>
Try to set minEvictableIdleTimeMillis to lower value than its default, which is 1000*60*30.
try to use c3p0 connection pool`~~
In comparable situations I've used Tomcat's JMX adapter to tweak the settings of the connection pool. This can be used both, to increase and to decrease thesize of the pool. It is therefore a good idea to enable JMX remote access to the servlet container at least in production environments, just to have some chance to react on exceptional operational situations.