I am using Tomcat 7 connection pooling (as a Tomcat resource in server.xml and context.xml) in a web application and it works.
My question is: Is it possible to "tell"/"force" tomcat to dispose the connection pool after it is created?
The reason I ask is the following:
I am using H2 and run into some "racing" issue on shutdown.
H2 remains open as long as there is a connection open but Tomcat does not dispose the connection pool and so connections remain open. And as a result I have various issues on shutdown.
I found that I could issue an SQL SHUTDOWN command to close H2 but I want to explore all the alternatives for my case.
So is it possible to "tell"/"force" tomcat to dispose the connection pool (at least on shutdown)?
I think you can try to put debug log on and check whether its issue with connection not released by application or something else like datasource configuration parameter in server.xml.
mostly it should be the case where application is not releasing connection.
How about writing a custom ServletContextListener and then having it shut down the pool when the context is destroyed. Here's an article about ServletContextListener being used to create shutdown hooks:
shutdown hook for java web application
The API for context listeners seems pretty straightforward:
http://docs.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html
please see my answer in : Context specific JNDI parameters problem in tomcat 6. There's a lot of stuff you can do with jndi resource.
<Resource name="jdbc/NAME" auth="Container" type="javax.sql.DataSource"
maxActive="100" minIdle="10" maxWait="10000" removeAbandoned="true"
removeAbandonedTimeout="60" logAbandoned="true"
testWhileIdle="true" testOnBorrow="true" testOnReturn="false"
timeBetweenEvictionRunsMillis="5000"
validationQuery="SELECT 1" initialSize="10"
username="usrname" password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/databb?autoReconnect=true"/>
As described in documentation http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Resource_Definitions it possible to set closeMethod of container datasource. I'm not sure about 'disposing' connection pool, but i think it worth a try.
Name of the zero-argument method to call on a singleton resource when
it is no longer required. This is intended to speed up clean-up of
resources that would otherwise happen as part of garbage collection.
This attribute is ignored if the singleton attribute is false. If not
specificed, no default is defined and no close method will be called.
Also you can deploy DataSource programmatically and start/stop it in ServletContextListener, example for dbcp (sorry, it from my unit-tests, but easy to rewrite):
import org.apache.commons.dbcp.BasicDataSource;
static BasicDataSource bds = new BasicDataSource();
#BeforeClass
public void setUp() throws Exception {
bds.setDefaultAutoCommit(false);
bds.setDriverClassName("org.h2.Driver");
bds.setInitialSize(0);
bds.setMaxActive(2);
bds.setMaxWait(10000);
bds.setPassword(null);
bds.setUrl("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1");
bds.setUsername("sa");
bds.setValidationQuery("select 1 as test");
}
#AfterClass
public void tearDown() throws Exception {
bds.close();
}
Maybe I don't understand the question. To be more clear, if you shutdown Tomcat (and hence it's JVM), then neither Tomcat nor it's connection pools, nor any of it's daemon threads would still be alive ... and thus no memory references would be left in tack.
How exactly have you come to the conclusion that a Tomcat resource has a reference to a connection pool, after Tomcat is shutdown? You wouldn't be able to see that in a profiler, and frankly, it doesn't quite make sense. For example, is it possible that the OS keeps connections open for a short period of time before destroying them? Have you tried the same tests using the same OS, but instead with a different container, something like Jetty? Are you using keep-alives, or some form of persistent connections?
If you want to get down-and-dirty with Tomcat's management of a resource exposed through JNDI, you can make your own DataSource implementation that delegates to the H2 DataSource. You can do this my using the factory-method attribute within your server.xml (or context.xml)
Here's a working example on github : https://github.com/tvollmer/connection-factory
Beyond that, it would help if you could further clarify what you mean by "various issues on shutdown". Those details matter, and it's not clear to me how you've logically gone from "various issues" to saying that Tomcat "does not dispose the connection pool". You could be dealing with 2 completely separate issues here, and you may want to check out antiResourceLocking and antiJARLocking settings for Tomcat on Windows.
Related
We have a backup database (which we want tomcat to failover in case primary database is down).
Trying to setup as following, but it is not working, am I missing something ?
<Resource auth="Container"
driverClassName="oracle.jdbc.driver.OracleDriver"
initialSize="5"
maxIdle="10"
maxTotal="15"
maxWaitMillis="15000"
name="jdbc/dbcporacle"
username="username"
password="xxxxxxxx"
testOnBorrow="true"
type="javax.sql.DataSource"
validationQuery="select 1 from dual"
url="jdbc:oracle:thin:#dbhost11111.com:3203/dbsvc111, jdbc:oracle:thin:#dbhost22222.com:3203/dbsvc111"/>
Note 2 urls (comma separated) in the url attribute
I'm not aware that there ever was such an option to provide two URLs and assume that they both have the same username/password. Where did you get this information? Or have you just been blindly trying?
Failing over to a different database is not as trivial as connecting to a different URL. Well, at first it is, but what happens when the original database comes back online? How does it get all the updates that were written to the secondary database? How will you determine that it's time to connect back to the original one?
One proper way to deal with failures is to configure your backend databases as cluster - they'll have a single URL that the application connects to, and transparently handle failover independently of your application.
So: Are you missing something? Yes: Configuring a datasource to two URLs never was supposed to work (let alone to work this way), and it won't solve your problem anyway.
And if your database is read-only, so that you can transparently switch between the alternatives: Handle this on the application level and just configure two databases that your application can failover to. This way your application will have to deal with it. Of course, you could implement your own DataSource to handle such a handover on failure of one of them.
I have been searching high and low and gathered bits and pieces, apologies if this has already been answered elsewhere but I am unable to find it.
I am writing a web application in Java with Tomcat, and SQL Azure in the backend.
There are multiple servlets accessing the SQL Azure DB. I would like to use Connection Pools as managed by Tomcat 8.5
My application context.xml in META-INF is as follows:
<Context>
<Resource name="jdbc/sqlazure"
auth="Container"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
type="javax.sql.DataSource"
maxIdle="30"
username="[username]"
password="[password]"
url="jdbc:sqlserver://[database]:1433;database=BackInfo;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30"
removeAbandonedTimeout="30"
logAbandoned="true" />
</Context>
In the Java Code, I access the typical way:
InitialContext ic = new InitialContext();ds = (DataSource)ic.lookup("java:comp/env/jdbc/sqlazure");
try(Connection con = ds.getConnection())....
Everything seems to work, so let me confirm my understanding here:
I do not need to specify a separate web.xml since I'm using Tomcat 8.5. Correct ?
Azure will automatically create a pool when I connect in this manner. The number of connections in the pool etc cannot (do not need to?) be configured.
Before I realized I would have other servlets that need to access the database, I had one servlet directly creating a Datasource via SQLServerConnectionPoolDataSource and getting a connection from there. The documentation states:
SQLServerConnectionPoolDataSource is typically used in Java Application Server environments that support built-in connection pooling and require a ConnectionPoolDataSource to provide physical connections, such as Java Platform, Enterprise Edition (Java EE) application servers that provide JDBC 3.0 API spec connection pooling.
Does this mean that when I use SQLServerConnectionPoolDataSource directly to ask for a connection, it will check if Tomcat supports pooling and then is basically using JDBC mechanisms to create a pool of SQL Azure connections managed by Tomcat ?
When getting the DataSource via Tomcat JNDI lookup, using SQLServerDriver as specified in context.xml. When the web app starts up, it will check context.xml and use SQLServerDriver to connect to SQL Azure, check if pooling is supported, if yes then Tomcat is using the driver to automatically creating a connection pool DataSource that it returns ?
I also just thought of one other question. Would it make sense to have a Singleton DataSource class that returns a reference to the pooled connection DataSource, or would it be better to have each servlet lookup the datasource in its init() and store in a private variable ?
Thanks
Based on my understanding, the jdbc connection pool for SQL Server is created by Java Application, not Azure does. My suggestion is that you need to refer to the Tomcat offical documents below to understand the JNDI Resources & JDBC DataSource.
JNDI Resources: http://tomcat.apache.org/tomcat-8.5-doc/jndi-resources-howto.html
JDBC DataSource: http://tomcat.apache.org/tomcat-8.5-doc/jndi-datasource-examples-howto.html
I have scoured the internet for over a day and still cannot seem to find a working solution. I am using MySQL Connector/J 5.1.35 and Tomcat JDBC Connection Pool with Tomcat 6.0.32 and I am not able to start/stop/reload an existing Web Application without the tomcat jdbc cleaner pool thread not being able to be stopped and causing memory leaks. I have tried everything that I could find on the net about it and I still have the same issues. If I stop the container, it wouldn't matter since once the container stops, the threads are dead. BUT, since I am not stopping the container, just merely stopping the application, BAM! - Memory Leak.
This is beyond frustrating and you would think that Apache would have a detailed solution for this, but they don't.
Anyone know how to resolve this once and for all?
Oh, and yes - I de-register the drivers and run AbandonedConnectionCleanupThread.shutdown() in a ServletContextListener and have tried both to deploy the drivers in $CATALINA_HOME/lib and WEB-INF/lib, but still have the same problem.
Update:
It looks like the connection pool may NOT be the issue. If I use DBCP, I do not get errors about memory leaks, but if I run "Find Leaks" on the manager page, Tomcat definitely discovers a leak in the application - just from starting it, running an operation that reads from the database, then stopping it. If I just start/stop it, then no issue reported. I have made sure that all connections/statements/resultSets are properly closed. If I switch back to Tomcat JDBC, I still have the same issue, it just gets reported (at least as far as I can tell) during stop/restart of the application as [Pool-Cleaner] Thread not being able to be stopped, and the Manager finding leaks when clicking the "Find Leaks" button.
Your JDBC drivers should be in $CATALINA_HOME/lib, and your database connection pools should be configured in the server.xml file using <Resource> tags. You context.xml file for the webapp should then link to the global data source with a <ResourceLink> tag.
This way connections are global and shared by all webapps connecting to the same database. Connections will not be released when a webapp is stopped or restarted, because they don't belong to the webapp, they belong to Tomcat.
No database connection memory leak, assuming all your webapp servlets/handlers clean up their resources correctly in finally blocks.
FYI: Deregistring a driver doesn't release any resources, such as open Connections, Statements, and ResultSets. It just prevents new connections from being created.
Ok, discovered that this is NOT AT ALL related to JDBC (face-palm). Turns out this is an RMI issue, so I am now going to re-post this question as a new question.
Thanks everyone for your time.
Does it make more sense to have to connection pool at the JNDI level or at the webapp level? For example, I could create at simply javax.sql.DataSource thusly:
<Context antiJARLocking="true">
<Resource name="jdbc/myDataSource"
auth="Container"
type="javax.sql.DataSource"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/myDataSource" user="user" password="password" />
</Context>
and then configure the pool in Spring thusly:
<bean id="myDataSource" class="com.mchange.v2.c3p0.DataSources"
factory-method="pooledDataSource">
<constructor-arg>
<jee:jndi-lookup jndi-name="java:comp/env/jdbc/myDataSource" />
</constructor-arg>
</bean>
Or, I could configure the pool directly in JNDI itself:
<Resource name="jdbc/myDataSource"
auth="Container"
factory="org.apache.naming.factory.BeanFactory"
type="com.mchange.v2.c3p0.ComboPooledDataSource"
driverClassName="com.mysql.jdbc.Driver"
jdbcUrl="jdbc:mysql://localhost/myDataSource"
user="user" password="password"
minPoolSize="3"
maxPoolSize="15"
maxIdleTime="5000"
idleConnectionTestPeriod="300"
acquireIncrement="3" />
Leaving this spring:
<jee:jndi-lookup id="myDataSource" jndi-name="java:comp/env/jdbc/myDataSource" />
In both cases, the myDataSource spring bean would be a c3p0 connection pooled data source, but which one is better? I am thinking that having the pool in JNDI makes the most sense, but the downside to that is that you must push your c3p0 lib to the servlet container level which could cause conflicts with existing servlets if they currently use a different version. However, putting it in JNDI means your applications dont have to worry about pooling at all. What do y'all think?
You don't need to pool the data source, obtained from JNDI, as it is already pooled :)
The only difference between having a container-manager pool v.s. application pool is that in 1st case you have an ability to monitor the workload on the pool using the standard interfaces (e.g. JBoss console). Then administrator of the application server manages the decision about increasing the pool size, if necessary. He may also switch applications to another DB server (e.g. planned migration from MySQL to Oracle). The disadvantage is that you need slightly more efforts to setup JNDI test data source for your unit tests (see here).
And in 2nd case, yes, you have to package either DBCP or c3p0 plus the JDBC driver together with you application. In this case it is not so easy to collect the statistics about all pools for all application running in Tomcat. Also migration to newer JDBC driver (MySQL 4 to MySQL 5) cannot be done for all applications at once. And connection properties are wired to your application, even if you use a .property file (so changing that needs reassembling and redeployment of the project). Perhaps you don't need all that, as you have only application, one DB and no management overhead.
More topics on this subject:
10 pools with 5 connections v.s. 1 pool with 50 connection (that is what happens when each application starts using it's own pool)
DBCP vs C3P0
Connection pool implementations
Why to use a connection pool?
We have a web application right now that we deploy a copy for each client. Our current deployment strategy is to create a uniquely named jdbc connection pool for each instance.
so say jdbc/client. They are specified like this...
< Context path="/"
reloadable="true"
docBase="\home\client\ROOT"
debug="5" >
< Resource name="jdbc/client"
auth="Container"
type="javax.sql.DataSource"
maxActive="100"
maxIdle="30"
validationQuery="SELECT 1"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="300000"
numTestsPerEvictionRun="6"
minEvictableIdleTimeMillis="1800000"
maxWait="10000"
username="user"
password="pass"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://databaseserver:3306/client ?zeroDateTimeBehavior=convertToNull&jdbcCompliantTruncation=false"/>
< /Context>
The question is, if I were to standardize it so that instead of unique names the connection pool is called jdbc/database on all deployed instances, is there a chance of database crossing, ie one customer in another customer's database, or are these localized to a specific deployed instance?
Thoughts?
Thanks,
Scott
No. The scope of the data source name is one Tomcat instance.
If you are starting a separate Tomcat process for each customer, all that matters is how the data source is configured, not what Tomcat calls it. As long as each data source is configured to use a different database, there won't be any cross talk.
This depends on how you deploy application for each client,
If each client gets their own Tomcat installation (they have different CATALINA_HOME), there is no chance for it to cross.
If they all use the same installation but run as different host in Tomcat, you need to make sure you don't define the datasource in conf/context.xml, which is shared by all hosts.
If all clients share the same Tomcat instances and they are simply different web apps, more attention is required. You need to define the datasource either in META-INF/context.xml or WEB-INF/web.xml. For further isolation, you should copy dbcp.jar to WEB-INF/lib of each application so they use their own DBCP instance.
If you're defining the JNDI DataSource resource within the Context for a deployment of the application, I believe you could even have multiple copies of the same application running in the same Tomcat instance and using the same JNDI name to access different databases. If each application instance is running in a different instance of Tomcat completely, there is certainly no way that one instance would be referring to the database specified for another instance.
No there is no chance of database crossing becoz the scope of the data source name is one Tomcat instance and you can have multiple data source in single tomcat instance .... so as long as data source is different there is no chance of database crossing.....