I'm having a little trouble with my MySQL- Connection- Pooling.
This is the case:
Different jobs are scheduled via Quartz. All jobs connect to different databases which works fine the whole day while the nightly scheduled jobs fail with a CommunicationsException...
Quartz-Jobs:
Job1 runs 0 0 6,10,14,18 * * ?
Job2 runs 0 30 10,18 * * ?
Job3 runs 0 0 5 * * ?
As you can see the last job runs at 18 taking about 1 hour to run.
The first job at 5am is the one that fails. I already tried all kinds of parameter-combinations in my resource config this is the one I am running right now:
<!-- Database 1 (MySQL) -->
<Resource
auth="Container"
driverClassName="com.mysql.jdbc.Driver"
maxActive="100"
maxIdle="30"
maxWait="10000"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
type="javax.sql.DataSource"
name="jdbc/appDbProd"
username="****"
password="****"
url="jdbc:mysql://127.0.0.1:3306/appDbProd?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="true"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="1800000"
/>
<!-- Database 2 (MySQL) -->
<Resource
auth="Container"
driverClassName="com.mysql.jdbc.Driver"
maxActive="100"
maxIdle="30"
maxWait="10000"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
type="javax.sql.DataSource"
name="jdbc/prodDbCopy"
username="****"
password="****"
url="jdbc:mysql://127.0.0.1:3306/prodDbCopy?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="true"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="1800000"
/>
<!-- Database 3 (MSSQL)-->
<Resource
auth="Container"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
maxActive="30"
maxIdle="30"
maxWait="100"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
name="jdbc/catalogDb"
username="****"
password="****"
type="javax.sql.DataSource"
url="jdbc:jtds:sqlserver://127.0.0.1:1433;databaseName=catalog;useNdTLMv2=false"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="true"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="1800000"
/>
For obvious reasons I changed IPs, Usernames and Passwords but they can be assumed to be correct, seeing that the application runs successfully the whole day.
The most annoying thing is:
The first job that runs first queries Database2 successfully but fails to query Database1 for some reason (CommunicationsException):
Caused by:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
The last packet successfully received
from the server was 39,376,539
milliseconds ago. The last packet
sent successfully to the server was
39,376,539 milliseconds ago. is longer
than the server configured value of
'wait_timeout'. You should consider
either expiring and/or testing
connection validity before use in your
application, increasing the server
configured values for client timeouts,
or using the Connector/J connection
property 'autoReconnect=true' to avoid
this problem.
Any ideas? Thanks!
As it says, enable connection validation on the connection pool.
I get this error when a connection is left idle for a while. Do your jobs stop hitting the DB at some point, then run some queries?
It also happens when there's a link failure, you might want to check with your network team if the connection doesn't drop.
I have run into a similar issue using Oracle, and JDBC. It ended up I was consuming all of the connections that were available to the database, and I wasn't relinquishing any of them and my code would just halt or timeout. Take a look at how you are handling your connections in your code.
Related
We have an application provided by a 3rd party vendor that runs on Tomcat 8 and JDK 8 to an Oracle 12 DB with ojdbc7.jar & xdb6.jar driver. The application works, but is slower than expected. When investigating it appears the application is configured to use connection pooling, but it appears the application is creating new connections per query, and not using any of the initially created connections to the Database.
Unfortunately, I don't have access to the code of the 3rd party app. But, hoping for an idea on what I am missing in the Tomcat setting to have pooling work.
I've tried going through Apache's documentation for the older Oracle connections, and trying other options found on the web.
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#localhost:1521:XE"
username="myProxyUser" password="myPassword"
initialSize="5" maxTotal="100" maxIdle="-1"
maxWaitMillis="30000"
validationQuery="select 1 from dual"
testOnBorrow="true"
accessToUnderlyingConnectionAllowed = "true"
connectionProperties="defaultRowPrefetch=100"
removeAbandoned = "true"
removeAbandonedTimeout = "30"/>
You can check tomcat docs, mainly use factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
example on how to configure a resource for JNDI lookups
<Resource name="jdbc/TestDB"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
validationInterval="30000"
timeBetweenEvictionRunsMillis="30000"
maxActive="100"
minIdle="10"
maxWait="10000"
initialSize="10"
removeAbandonedTimeout="60"
removeAbandoned="true"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jmxEnabled="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="root"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mysql"/>
you can specify pooling by defining type and factory
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
We are using tomcat jdbc pool in our project. The connection pool configuration is
<Resource name="jdbc/cc" auth="Container" type="javax.sql.DataSource" username="xx" password="plain text"
driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/xx?autoReconnect=true&useUnicode=true&characterEncoding=utf8"
maxActive="50" maxIdle="25" minIdle="10" maxWait="10000" testOnBorrow="true"
validationQuery="SELECT 1" factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"/>
We want to use encrypted DB password in configuration rather than plain text. How we can do this, please help me on this. ( we know how to do this if we are using tomcat dbcp, but the same implementation is not working for tomcat jdbc)
What may be reason for tomcat does not release connections after using?
Herem is my config
<Resource name="jdbc/DataSource" auth="Container"
type="javax.sql.DataSource" username="))" password="))"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="))"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
initialSize="55"
maxActive="55" maxIdle="55" maxWait="15000"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1 FROM DUAL"
validationInterval="20000"
timeBetweenEvictionRunsMillis="30000"
removeAbandoned="true"
removeAbandonedTimeout="60"
/>
You still need to call connection.close() in your code to have the connection released to the pool. It does not happen automatically.
You can configure the pool to detect "left over connections" wiht:
removeAbandoned="true"
removeAbandonedTimeout="60"
connection inactive for longer than given number of seconds will be closed automatically and recycled by the pool
The most probable is that you don't close connections after using them.
It can happens for example if you don't manage correctly exceptions.
I've been trying to configure a connection pool for a SQL Server 2012 database. I currently have Informix and Oracle pools configured and working, only SQL Server is giving me a headache. This is how my resource on Context.xml looks so far:
<Resource name="jdbc/sqlserv"
auth="Container"
factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"
driverClass="com.microsoft.sqlserver.jdbc.SQLServerDriver"
type="javax.sql.DataSource"
maxActive="50"
maxIdle="10"
maxWait="15000"
username="username"
password="password"
url="jdbc:sqlserver://127.0.0.1:1433;databaseName=SQLDB;"
removeAbandoned="true"
removeAbandonedTimeout="30"
logAbandoned="true" />
That's using sqljdbc4 driver, of course. We already tried using jtds-1.3.0 with the driverClass="net.sourceforge.jtds.jdbc.Driver", but no go. All the resource-refs are also being correctly configured. Whenever I try to create a new connection using that Resource, it fails.
For comparison's sake, here's how our Informix and Oracle resources look like:
<Resource name="jdbc/infmx"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"
maxActive="50"
maxIdle="10"
maxWait="15000"
username="username"
password="password"
driverClassName="com.informix.jdbc.IfxDriver"
url="jdbc:informix-sqli://localhost:30091/infmx:informixserver=ol_infmx_soc"
removeAbandoned="true"
removeAbandonedTimeout="30"
logAbandoned="true"/>
<Resource name="jdbc/orcl"
auth="Container"
type="oracle.jdbc.pool.OracleDataSource"
driverClassName="oracle.jdbc.driver.OracleDriver"
factory="oracle.jdbc.pool.OracleDataSourceFactory"
url="jdbc:oracle:thin:#127.0.0.1:1521:orcl"
user="username"
password="password"
maxActive="50"
maxIdle="10"
maxWait="15000" />
So My question is: How can I correctly configure a connection pool for SQL Server 2012 on my tomcat context? I've searched high and low, attempted everything I've found, but nothing worked.
Thanks in advance.
[edit] Here's the stack trace: http://pastebin.com/w3rZSERs
[edit-2] It seems the problem is that Tomcat can't find the driver on his lib folder. We're pretty sure it's there, but we don't know to be sure of that. This happens with both sqljdbc4 and jtds-1.3.0. We're following every guideline we can find, but the problem persists.
We found our problem.
driverClass="com.microsoft.sqlserver.jdbc.SQLServerDriver"
Should have been
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
It seems to me that the java side is correctly configured.
Can you access the server using another JDBC connection (for example SquirrelSQL or similar software)?
If you can't access to the server using Squirrel, maybe you did not enable the TCP/IP access to your server, in this case, follow the accepted answer of Enable remote connections for SQL Server Express 2012
On one developer workstation running Eclipse Helios SR2, Windows 7 and Tomcat 6.0.32 we have a very strange case of duplicate JNDI connection pools
Running tomcat from Eclipse
server.xml
>
<Context docBase="path to web app" path="/ds-web" reloadable="true">
<Resource
name="jdbc/ds"
username="ds"
password="pass"
type="javax.sql.DataSource"
url="jdbc:postgresql://localhost:5432/ds"
auth="Container"
driverClassName="org.postgresql.Driver"
factory="org.apache.commons.dbcp.BasicDataSourceFactory"
logAbandoned="true"
maxActive="30"
maxIdle="10"
maxWait="1000"
removeAbandoned="true"
removeAbandonedTimeout="60"
validationQuery="SELECT 1"
testOnBorrow="true"
testOnReturn="true"/>
</Context>
When start server, on the console we see the following 3 times in a row
AbandonedObjectPool is used
(org.apache.commons.dbcp.AbandonedObjectPool#11aa58b)
LogAbandoned: true
RemoveAbandoned: true
RemoveAbandonedTimeout: 60
Application then fails to find the JNDI resource
If we remove the <Resource> in server.xml, then the console shows no connection pool is created at all
On another developer machine with the same hardware and OS we do not have this problem
Any ideas?
Thanks
Marc
My suggestion, Copy the whole <Context> from server.xml and create a blank context.xml inside your web application META-INF folder and paste the <Context> copied from server.xml there.
Restart your application and see if this works.
We re-installed postgresql on the machine and the problem got solved somehow.
We're thinking something in the original postgresql config had been messed up and tomcat was failing to connect to the BD somehow. I guess tomcat was simply "trying 3 times", thus the triple output.
Another unsolved mystery...
move
<Resource
name="jdbc/ds"
username="ds"
password="pass"
type="javax.sql.DataSource"
url="jdbc:postgresql://localhost:5432/ds"
auth="Container"
driverClassName="org.postgresql.Driver"
factory="org.apache.commons.dbcp.BasicDataSourceFactory"
logAbandoned="true"
maxActive="30"
maxIdle="10"
maxWait="1000"
removeAbandoned="true"
removeAbandonedTimeout="60"
validationQuery="SELECT 1"
testOnBorrow="true"
testOnReturn="true"/>
to server.xml namely the
<GlobalNamingResources>
element
in your conf/context.xml file you would instead specify
<ContextLink name="jdbc/ds" global="jdbc/ds"/>
and this way, the three contexts will share the same pool.