I am using Apache DBCP with JNDI in my tomcat container to pool connections to a mysql database. Everything works fine but a problem i am seeing is that once a pconnection is pooled it is never released. So after a load spike connection sit their forever sleeping. Is their a way to shrink the pool through context.xml parameters? here is my ocnfiguration:
defaultAutoCommit="false"
defaultTransactionIsolation="REPEATABLE_READ"
auth="Container"
type="javax.sql.DataSource"
logAbandoned="true"
removeAbandoned="true"
removeAbandonedTimeout="300"
maxActive="-1"
initialSize="15"
maxIdle="10"
maxWait="10000"
username="user"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/mydb"/>
Try to set minEvictableIdleTimeMillis to lower value than its default, which is 1000*60*30.
try to use c3p0 connection pool`~~
In comparable situations I've used Tomcat's JMX adapter to tweak the settings of the connection pool. This can be used both, to increase and to decrease thesize of the pool. It is therefore a good idea to enable JMX remote access to the servlet container at least in production environments, just to have some chance to react on exceptional operational situations.
Related
This is a bit of a weird one question and I cannot for the life of figure out what's going on. I have two applications (WARS) running on a Tomcat7 instance. Both of them connect to a database and get that connection using JNDI. The datasource reference specified in the tomcat context.xml file as follows.
Tomcat7/conf/context.xml
<Resource name="jdbc/dataSourceOne" auth="Container" type="javax.sql.DataSource" maxActive="100" maxIdle="30" maxWait="10000" username="user" password="pass" driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/databaseone" />
<Resource name="jdbc/dataSourceTwo" auth="Container" type="javax.sql.DataSource" maxActive="100" maxIdle="30" maxWait="10000" username="user" password="pass" driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/databasetwo" />
WEB-INF/lib
Both applications have a copy of the mysql connector jar in their lib folders.
mysql-connector-java-5.1.34.jar
Now here's where it gets weird, application 1 works straight out of the box. I just need to deploy the WAR on my tomcat instance and it's all systems go. It gets its datasource and can connect to it's database with absolutely no problems.
However, when I try and deploy application 2, I get a MySQL class not found exception. Now the fix for this is to add the above mysql connector jar to the tomcat7/lib folder.
My question is why do I need only need to add this for my second application? If tomcat7 requires the mysql connector jar why does my first application work without any modification to the tomcat7 setup at all?
It just doesn't make sense that one application will work but the other won't without the modification.
Any help would be greatly appreciated folks.
#Gimby - Turns out you were right, application 1 wasn't actually using the datasource defined in tomcats context.xml file. Thanks for the input, really appreciate the help.
This is the first time I'm trying to set up a connection pool in tomcat (version 8), and I'm kind of confused. It seems that I have 2 options available to me, 1) I can add something similar to this:
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<!-- Specify a JDBC datasource for oracle -->
<Resource name="jdbc/testdb"
auth="Container"
type="javax.sql.DataSource"
username="DB_USERNAME"
password="DB_PASSWORD"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:#xxx:1525:dbname"
maxActive="10"
maxIdle="4" />
</Context>
In a file called 'context.xml' in my projects WEB-INF folder. OR, 2) I can add something like this:
<Context path="/dbcp" docBase="dbcp" debug="5"
reloadable="true" crossContext="true">
<Resource name="jdbc/TestDB" auth="Container"
type="javax.sql.DataSource" removeAbandoned="true"
removeAbandonedTimeout="30" maxActive="100"
maxIdle="30" maxWait="10000" username="kunal"
password="java_facier"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/dbcptest"/>
</Context>
in TOMCAT_HOME/conf/sever.xml and then add this:
<resource-ref>
<description> DB Connection Pooling</description>
<res-ref-name> jdbc/TestDB</res-ref-name>
<res-type> javax.sql.DataSource</res-type>
<res-auth> Container</res-auth>
</resource-ref>
to my WEB-INF/web.xml file in my project...
If I'm correct in both instances here, which one is better to use? Also where would I put my database vendors jdbc-driver?
Put JDBC libraries to $tomcat/lib folder so that webapp reloads don't touch it. I have found this the best common rule and even an official Apache Tomcat documentation instructs $CATALINA_HOME/lib folder so please follow their wisdom.
$tomcat/webapps/mywebapp/META-INF/context.xml
This provides you an application level pool. You having two or more webapps connecting to a same database each have own pool. It means they obye own private max-limit and such meters. Distributing and installing mywebapp.war file is easy, this is a self-contained application.
$tomcat/conf/server.xml
This provides a global pool where two or more webapps share same max-limit and such meters. If you must control the overall number of connections opened to the db engine this is where you should put jdbc settings. Tomcat admin must put this setting in place before deploying a webapp.
edit: Oh I see you have put jdbc settings inside the <Context> element in server.xml file. Well, in that case I think its still a webapp-private pool. If you need global pool you add it inside <GlobalNamingResources> element at the top of xml file.
Either approach has it pros and cons :
1. Application level packaging creates App local connection pools.
2. Independent from Tomcat's setup
but,
Application on release requires environment specific releases.
While, if tomcat maintains connection pool then app only depends on "Registered Name" and single package can be distributed for all environments.
Just to summarize, App level packaging is developer centric whereas Tomcat-maintained connection pools are Operational friendly.
I am using Tomcat 7 connection pooling (as a Tomcat resource in server.xml and context.xml) in a web application and it works.
My question is: Is it possible to "tell"/"force" tomcat to dispose the connection pool after it is created?
The reason I ask is the following:
I am using H2 and run into some "racing" issue on shutdown.
H2 remains open as long as there is a connection open but Tomcat does not dispose the connection pool and so connections remain open. And as a result I have various issues on shutdown.
I found that I could issue an SQL SHUTDOWN command to close H2 but I want to explore all the alternatives for my case.
So is it possible to "tell"/"force" tomcat to dispose the connection pool (at least on shutdown)?
I think you can try to put debug log on and check whether its issue with connection not released by application or something else like datasource configuration parameter in server.xml.
mostly it should be the case where application is not releasing connection.
How about writing a custom ServletContextListener and then having it shut down the pool when the context is destroyed. Here's an article about ServletContextListener being used to create shutdown hooks:
shutdown hook for java web application
The API for context listeners seems pretty straightforward:
http://docs.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html
please see my answer in : Context specific JNDI parameters problem in tomcat 6. There's a lot of stuff you can do with jndi resource.
<Resource name="jdbc/NAME" auth="Container" type="javax.sql.DataSource"
maxActive="100" minIdle="10" maxWait="10000" removeAbandoned="true"
removeAbandonedTimeout="60" logAbandoned="true"
testWhileIdle="true" testOnBorrow="true" testOnReturn="false"
timeBetweenEvictionRunsMillis="5000"
validationQuery="SELECT 1" initialSize="10"
username="usrname" password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/databb?autoReconnect=true"/>
As described in documentation http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Resource_Definitions it possible to set closeMethod of container datasource. I'm not sure about 'disposing' connection pool, but i think it worth a try.
Name of the zero-argument method to call on a singleton resource when
it is no longer required. This is intended to speed up clean-up of
resources that would otherwise happen as part of garbage collection.
This attribute is ignored if the singleton attribute is false. If not
specificed, no default is defined and no close method will be called.
Also you can deploy DataSource programmatically and start/stop it in ServletContextListener, example for dbcp (sorry, it from my unit-tests, but easy to rewrite):
import org.apache.commons.dbcp.BasicDataSource;
static BasicDataSource bds = new BasicDataSource();
#BeforeClass
public void setUp() throws Exception {
bds.setDefaultAutoCommit(false);
bds.setDriverClassName("org.h2.Driver");
bds.setInitialSize(0);
bds.setMaxActive(2);
bds.setMaxWait(10000);
bds.setPassword(null);
bds.setUrl("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1");
bds.setUsername("sa");
bds.setValidationQuery("select 1 as test");
}
#AfterClass
public void tearDown() throws Exception {
bds.close();
}
Maybe I don't understand the question. To be more clear, if you shutdown Tomcat (and hence it's JVM), then neither Tomcat nor it's connection pools, nor any of it's daemon threads would still be alive ... and thus no memory references would be left in tack.
How exactly have you come to the conclusion that a Tomcat resource has a reference to a connection pool, after Tomcat is shutdown? You wouldn't be able to see that in a profiler, and frankly, it doesn't quite make sense. For example, is it possible that the OS keeps connections open for a short period of time before destroying them? Have you tried the same tests using the same OS, but instead with a different container, something like Jetty? Are you using keep-alives, or some form of persistent connections?
If you want to get down-and-dirty with Tomcat's management of a resource exposed through JNDI, you can make your own DataSource implementation that delegates to the H2 DataSource. You can do this my using the factory-method attribute within your server.xml (or context.xml)
Here's a working example on github : https://github.com/tvollmer/connection-factory
Beyond that, it would help if you could further clarify what you mean by "various issues on shutdown". Those details matter, and it's not clear to me how you've logically gone from "various issues" to saying that Tomcat "does not dispose the connection pool". You could be dealing with 2 completely separate issues here, and you may want to check out antiResourceLocking and antiJARLocking settings for Tomcat on Windows.
Does it make more sense to have to connection pool at the JNDI level or at the webapp level? For example, I could create at simply javax.sql.DataSource thusly:
<Context antiJARLocking="true">
<Resource name="jdbc/myDataSource"
auth="Container"
type="javax.sql.DataSource"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/myDataSource" user="user" password="password" />
</Context>
and then configure the pool in Spring thusly:
<bean id="myDataSource" class="com.mchange.v2.c3p0.DataSources"
factory-method="pooledDataSource">
<constructor-arg>
<jee:jndi-lookup jndi-name="java:comp/env/jdbc/myDataSource" />
</constructor-arg>
</bean>
Or, I could configure the pool directly in JNDI itself:
<Resource name="jdbc/myDataSource"
auth="Container"
factory="org.apache.naming.factory.BeanFactory"
type="com.mchange.v2.c3p0.ComboPooledDataSource"
driverClassName="com.mysql.jdbc.Driver"
jdbcUrl="jdbc:mysql://localhost/myDataSource"
user="user" password="password"
minPoolSize="3"
maxPoolSize="15"
maxIdleTime="5000"
idleConnectionTestPeriod="300"
acquireIncrement="3" />
Leaving this spring:
<jee:jndi-lookup id="myDataSource" jndi-name="java:comp/env/jdbc/myDataSource" />
In both cases, the myDataSource spring bean would be a c3p0 connection pooled data source, but which one is better? I am thinking that having the pool in JNDI makes the most sense, but the downside to that is that you must push your c3p0 lib to the servlet container level which could cause conflicts with existing servlets if they currently use a different version. However, putting it in JNDI means your applications dont have to worry about pooling at all. What do y'all think?
You don't need to pool the data source, obtained from JNDI, as it is already pooled :)
The only difference between having a container-manager pool v.s. application pool is that in 1st case you have an ability to monitor the workload on the pool using the standard interfaces (e.g. JBoss console). Then administrator of the application server manages the decision about increasing the pool size, if necessary. He may also switch applications to another DB server (e.g. planned migration from MySQL to Oracle). The disadvantage is that you need slightly more efforts to setup JNDI test data source for your unit tests (see here).
And in 2nd case, yes, you have to package either DBCP or c3p0 plus the JDBC driver together with you application. In this case it is not so easy to collect the statistics about all pools for all application running in Tomcat. Also migration to newer JDBC driver (MySQL 4 to MySQL 5) cannot be done for all applications at once. And connection properties are wired to your application, even if you use a .property file (so changing that needs reassembling and redeployment of the project). Perhaps you don't need all that, as you have only application, one DB and no management overhead.
More topics on this subject:
10 pools with 5 connections v.s. 1 pool with 50 connection (that is what happens when each application starts using it's own pool)
DBCP vs C3P0
Connection pool implementations
Why to use a connection pool?
For my services in production environment I always set up DB connections pool in Tomcat's context.xml:
<Resource name="jdbc/MyDB" auth="Container" type="javax.sql.DataSource"
maxActive="256" maxIdle="5" maxWait="10000"
removeAbandoned="true" removeAbandonedTimeout="60" logAbandoned="true"
username="xxx" password="xxx" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://host:3306/dbname?autoReconnect=true"
validationQuery="SELECT 1"
/>
Then later in my service I use:
InitialContext ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/MyDB");
Connection db = ds.getConnection();
For development I want to run Axis2 standalone - is there a way how I could set up somewhere DB connections pool in Axis as well so I would not need to modify service code and use it the same way as with Tomcat?
Why not have different context.xml files for different environments.
e.g.
context_DEV.xml
context_UAT.xml
context_PROD.xml
and then use a symlink to point to the correct one.
e.g.
context.xml -> context_DEV.xml
Also, see this thread which recommends using a servlet container (such as Tomcat) rather than axis2 standalone server for stability.