Too many connections , tomcat7 on AWS EC2 - java

I signed up for a new AWS free account, and I am trying to to connect the Tomcat to the RDS database through JNDI Pool connection. The war deployed on the Beanstalk is the default one (which does not have any database connection or queries)
Here is how my context.xml looks like
<Context>
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<Manager pathname="" />
<!-- Uncomment this to enable Comet connection tacking (provides events
on session expiration as well as webapp lifecycle) -->
<!--
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
-->
<Resource name="jdbc/web"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
closeMethod="close"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/storage"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
closeMethod="close"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/secure"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
closeMethod="close"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/event"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
closeMethod="close"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
</Context>
When I restart my tomcat I receive the following exception.
SEVERE: Unable to create initial connections of pool.
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too many connections
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:910)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3923)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1273)
I am not sure how the connections are getting maxed out without having my webapp fire any query or even connect to database

You're facing a quota of allowed connections into your AWS database. By setting up 4 datasources with each 10 connections into that database, you exceed that limit.
By reducing the maxActive to something lower (say... 3?) you should be good to go (for now)
As for performance considerations: This will work fine as long as none of your pools gets too active.
In the long term I'd recommend one or more of those:
get a larger connections quota on your db in AWS
get more users with different permissions to reflect your desired usage
reduce number of pools - if they're on the same db doing similar stuff, you might be better of with one pool with all your quota assigned. So you can assign connections where they're needed most at that time.

Related

Is it possible to inject a context.xml into another context.xml in Tomcat?

I have two context files as main-context.xml and sub-context.xml.
I would like to import/include sub-context.xml into main-context.xml.
Is it possible to inject a context.xml into another context.xml in Tomcat?
main-context.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Valve className="org.apache.catalina.valves.rewrite.RewriteValve"/>
<Resource name="mainDs" auth="Container"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#localdomain:1521/test"
username="test_username" password="test_password"
maxTotal="20"
maxIdle="10"
maxWaitMillis="-1"
validationQuery="SELECT 1 FROM DUAL"
validationQueryTimeout="3"
testOnBorrow="true"
testWhileIdle="true"
defaultTransactionIsolation="2" />
<!-- Default set of monitored resources. If one of these changes, the -->
<!-- web application will be reloaded. -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<WatchedResource>WEB-INF/tomcat-web.xml</WatchedResource>
<WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>
<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<!--
<Manager pathname="" />
-->
</Context>
sub-context.xml:
<Resource name="subDs" auth="Container"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#localdomain:1521/testsub"
username="testsub_username" password="testsub_password"
maxTotal="20"
maxIdle="10"
maxWaitMillis="-1"
validationQuery="SELECT 1 FROM DUAL"
validationQueryTimeout="3"
testOnBorrow="true"
testWhileIdle="true"
defaultTransactionIsolation="2" />

Problems with Tomcat8 using connection pooling to OracleDB

We have an application provided by a 3rd party vendor that runs on Tomcat 8 and JDK 8 to an Oracle 12 DB with ojdbc7.jar & xdb6.jar driver. The application works, but is slower than expected. When investigating it appears the application is configured to use connection pooling, but it appears the application is creating new connections per query, and not using any of the initially created connections to the Database.
Unfortunately, I don't have access to the code of the 3rd party app. But, hoping for an idea on what I am missing in the Tomcat setting to have pooling work.
I've tried going through Apache's documentation for the older Oracle connections, and trying other options found on the web.
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#localhost:1521:XE"
username="myProxyUser" password="myPassword"
initialSize="5" maxTotal="100" maxIdle="-1"
maxWaitMillis="30000"
validationQuery="select 1 from dual"
testOnBorrow="true"
accessToUnderlyingConnectionAllowed = "true"
connectionProperties="defaultRowPrefetch=100"
removeAbandoned = "true"
removeAbandonedTimeout = "30"/>
You can check tomcat docs, mainly use factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
example on how to configure a resource for JNDI lookups
<Resource name="jdbc/TestDB"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
validationInterval="30000"
timeBetweenEvictionRunsMillis="30000"
maxActive="100"
minIdle="10"
maxWait="10000"
initialSize="10"
removeAbandonedTimeout="60"
removeAbandoned="true"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jmxEnabled="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mysql"/>
you can specify pooling by defining type and factory
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"

How to formate JNDI to authenticate with a domain user instead of SQL user

I have a working JNDI with SQL server username and password in my Java application. It throws NameNotFoundException when I try to change the user to my domain user instead of SQL user.
Here it is mentioned I only have to provide integratedSecurity as true. Have I missed anything in my JNDI?
JNDI with SQL Server User:
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://localhost:1433;database=survey;user=myuser;password=mypassword;"
/>
My JNDI with domain user:
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://localhost:1433;database=survey;integratedSecurity=true;"
/>
Another driver is necessary. See this source http://www.bigdata-unleashed.com/20150930/database-ntlm-authentifizierung
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
integratedSecurity="true"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost:1433;database=survey;"
/>
Database driver is available at http://sourceforge.net/projects/jtds/files/jtds/

org.apache.tomcat.jdbc.pool DB password decryption

We are using tomcat jdbc pool in our project. The connection pool configuration is
<Resource name="jdbc/cc" auth="Container" type="javax.sql.DataSource" username="xx" password="plain text"
driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/xx?autoReconnect=true&useUnicode=true&characterEncoding=utf8"
maxActive="50" maxIdle="25" minIdle="10" maxWait="10000" testOnBorrow="true"
validationQuery="SELECT 1" factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"/>
We want to use encrypted DB password in configuration rather than plain text. How we can do this, please help me on this. ( we know how to do this if we are using tomcat dbcp, but the same implementation is not working for tomcat jdbc)

Tomcat jdbc connection pool, does not release connections after using

What may be reason for tomcat does not release connections after using?
Herem is my config
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource" username="))" password="))"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="))"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
initialSize="55"
maxActive="55" maxIdle="55" maxWait="15000"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1 FROM DUAL"
validationInterval="20000"
timeBetweenEvictionRunsMillis="30000"
removeAbandoned="true"
removeAbandonedTimeout="60"
/>
You still need to call connection.close() in your code to have the connection released to the pool. It does not happen automatically.
You can configure the pool to detect "left over connections" wiht:
removeAbandoned="true"
removeAbandonedTimeout="60"
connection inactive for longer than given number of seconds will be closed automatically and recycled by the pool
The most probable is that you don't close connections after using them.
It can happens for example if you don't manage correctly exceptions.

Categories