We have an application provided by a 3rd party vendor that runs on Tomcat 8 and JDK 8 to an Oracle 12 DB with ojdbc7.jar & xdb6.jar driver. The application works, but is slower than expected. When investigating it appears the application is configured to use connection pooling, but it appears the application is creating new connections per query, and not using any of the initially created connections to the Database.
Unfortunately, I don't have access to the code of the 3rd party app. But, hoping for an idea on what I am missing in the Tomcat setting to have pooling work.
I've tried going through Apache's documentation for the older Oracle connections, and trying other options found on the web.
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#localhost:1521:XE"
username="myProxyUser" password="myPassword"
initialSize="5" maxTotal="100" maxIdle="-1"
maxWaitMillis="30000"
validationQuery="select 1 from dual"
testOnBorrow="true"
accessToUnderlyingConnectionAllowed = "true"
connectionProperties="defaultRowPrefetch=100"
removeAbandoned = "true"
removeAbandonedTimeout = "30"/>
You can check tomcat docs, mainly use factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
example on how to configure a resource for JNDI lookups
<Resource name="jdbc/TestDB"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
validationInterval="30000"
timeBetweenEvictionRunsMillis="30000"
maxActive="100"
minIdle="10"
maxWait="10000"
initialSize="10"
removeAbandonedTimeout="60"
removeAbandoned="true"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jmxEnabled="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mysql"/>
you can specify pooling by defining type and factory
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
Related
I have a working JNDI with SQL server username and password in my Java application. It throws NameNotFoundException when I try to change the user to my domain user instead of SQL user.
Here it is mentioned I only have to provide integratedSecurity as true. Have I missed anything in my JNDI?
JNDI with SQL Server User:
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://localhost:1433;database=survey;user=myuser;password=mypassword;"
/>
My JNDI with domain user:
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://localhost:1433;database=survey;integratedSecurity=true;"
/>
Another driver is necessary. See this source http://www.bigdata-unleashed.com/20150930/database-ntlm-authentifizierung
<Resource
name="jdbc/MyJNDI"
auth="Container"
type="javax.sql.DataSource"
maxTotal="100"
maxIdle="30"
maxWaitMillis="1000"
integratedSecurity="true"
validationQuery="select 1"
testOnBorrow="true"
testOnConnect="true"
testOnReturn="true"
testWhileIdle="true"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost:1433;database=survey;"
/>
Database driver is available at http://sourceforge.net/projects/jtds/files/jtds/
We are using tomcat jdbc pool in our project. The connection pool configuration is
<Resource name="jdbc/cc" auth="Container" type="javax.sql.DataSource" username="xx" password="plain text"
driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/xx?autoReconnect=true&useUnicode=true&characterEncoding=utf8"
maxActive="50" maxIdle="25" minIdle="10" maxWait="10000" testOnBorrow="true"
validationQuery="SELECT 1" factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"/>
We want to use encrypted DB password in configuration rather than plain text. How we can do this, please help me on this. ( we know how to do this if we are using tomcat dbcp, but the same implementation is not working for tomcat jdbc)
I signed up for a new AWS free account, and I am trying to to connect the Tomcat to the RDS database through JNDI Pool connection. The war deployed on the Beanstalk is the default one (which does not have any database connection or queries)
Here is how my context.xml looks like
<Context>
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<Manager pathname="" />
<!-- Uncomment this to enable Comet connection tacking (provides events
on session expiration as well as webapp lifecycle) -->
<!--
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
-->
<Resource name="jdbc/web"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
closeMethod="close"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/storage"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
closeMethod="close"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/secure"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
closeMethod="close"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
<Resource name="jdbc/event"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
closeMethod="close"
maxActive="10"
maxIdle="5"
minIdle="1"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://aamhszwo5p1rrw.cf7vbk9ataaz.us-west-2.rds.amazonaws.com:3306/broadleaf"/>
</Context>
When I restart my tomcat I receive the following exception.
SEVERE: Unable to create initial connections of pool.
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too many connections
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:910)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3923)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1273)
I am not sure how the connections are getting maxed out without having my webapp fire any query or even connect to database
You're facing a quota of allowed connections into your AWS database. By setting up 4 datasources with each 10 connections into that database, you exceed that limit.
By reducing the maxActive to something lower (say... 3?) you should be good to go (for now)
As for performance considerations: This will work fine as long as none of your pools gets too active.
In the long term I'd recommend one or more of those:
get a larger connections quota on your db in AWS
get more users with different permissions to reflect your desired usage
reduce number of pools - if they're on the same db doing similar stuff, you might be better of with one pool with all your quota assigned. So you can assign connections where they're needed most at that time.
What may be reason for tomcat does not release connections after using?
Herem is my config
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource" username="))" password="))"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="))"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
initialSize="55"
maxActive="55" maxIdle="55" maxWait="15000"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1 FROM DUAL"
validationInterval="20000"
timeBetweenEvictionRunsMillis="30000"
removeAbandoned="true"
removeAbandonedTimeout="60"
/>
You still need to call connection.close() in your code to have the connection released to the pool. It does not happen automatically.
You can configure the pool to detect "left over connections" wiht:
removeAbandoned="true"
removeAbandonedTimeout="60"
connection inactive for longer than given number of seconds will be closed automatically and recycled by the pool
The most probable is that you don't close connections after using them.
It can happens for example if you don't manage correctly exceptions.
On one developer workstation running Eclipse Helios SR2, Windows 7 and Tomcat 6.0.32 we have a very strange case of duplicate JNDI connection pools
Running tomcat from Eclipse
server.xml
>
<Context docBase="path to web app" path="/ds-web" reloadable="true">
<Resource
name="jdbc/ds"
username="ds"
password="pass"
type="javax.sql.DataSource"
url="jdbc:postgresql://localhost:5432/ds"
auth="Container"
driverClassName="org.postgresql.Driver"
factory="org.apache.commons.dbcp.BasicDataSourceFactory"
logAbandoned="true"
maxActive="30"
maxIdle="10"
maxWait="1000"
removeAbandoned="true"
removeAbandonedTimeout="60"
validationQuery="SELECT 1"
testOnBorrow="true"
testOnReturn="true"/>
</Context>
When start server, on the console we see the following 3 times in a row
AbandonedObjectPool is used
(org.apache.commons.dbcp.AbandonedObjectPool#11aa58b)
LogAbandoned: true
RemoveAbandoned: true
RemoveAbandonedTimeout: 60
Application then fails to find the JNDI resource
If we remove the <Resource> in server.xml, then the console shows no connection pool is created at all
On another developer machine with the same hardware and OS we do not have this problem
Any ideas?
Thanks
Marc
My suggestion, Copy the whole <Context> from server.xml and create a blank context.xml inside your web application META-INF folder and paste the <Context> copied from server.xml there.
Restart your application and see if this works.
We re-installed postgresql on the machine and the problem got solved somehow.
We're thinking something in the original postgresql config had been messed up and tomcat was failing to connect to the BD somehow. I guess tomcat was simply "trying 3 times", thus the triple output.
Another unsolved mystery...
move
<Resource
name="jdbc/ds"
username="ds"
password="pass"
type="javax.sql.DataSource"
url="jdbc:postgresql://localhost:5432/ds"
auth="Container"
driverClassName="org.postgresql.Driver"
factory="org.apache.commons.dbcp.BasicDataSourceFactory"
logAbandoned="true"
maxActive="30"
maxIdle="10"
maxWait="1000"
removeAbandoned="true"
removeAbandonedTimeout="60"
validationQuery="SELECT 1"
testOnBorrow="true"
testOnReturn="true"/>
to server.xml namely the
<GlobalNamingResources>
element
in your conf/context.xml file you would instead specify
<ContextLink name="jdbc/ds" global="jdbc/ds"/>
and this way, the three contexts will share the same pool.