I want to maintain connection pooling into my java/spring application. For this i used spring configuration file. Which looks like this
<bean id="dataSource" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${db.driver}"/>
<property name="url" value="${db.jdbcurl}"/>
<property name="username" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="initialSize" value="3"/> T
<property name="maxActive" value="100"/>
</bean>
This application war is hosted into tomcat. When i saw tomcat context.xml file which is set by some network team. Context.xml file also contain some configuration regarding the connection pooling.It looks like this
<Context>
<Resource name="jdbc/test"
auth="Container"
type="javax.sql.DataSource"
username="naveen"
password="12345"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql:localhost:3306/cyl"
maxActive="15"
maxIdle="3"/>
</Context>
Now i have two question to asked
1) Which connection pool configuration will apply when application runs.
2) If we have multiple war a.war(with connection pool config), b.war(with out CPC), c.war(without CPC) then, will tomcat connection pool setting will apply to all application.
3) what can be max value for maxActive attribute.
Thanks in advance
Related
I have a spring application hosted on to the server (Tomcat 8.5). It goes idle if no one uses it. I already knew that timeout will occur if the DB is in idle state for 8 hours (Default timeout of MySQL). As mentioned in Spring Autoreconnect and Connection lost overnight post i have tried the solution available here.I have tried configuring application.properties but that doesn't bring any solution to the problem.
(PS:I'm not changing anything other than application.properties in my Spring Application).
Well if this
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 60000
spring.datasource.validationQuery = SELECT 1
or this
spring.datasource.testOnBorrow=true
spring.datasource.validationQuery=SELECT 1
didnt work maybe try this
Post SpringBoot 1.4 names have changed
They have defined new specific namespaces for the four connections pools spring supports: tomcat, hikari, dbcp, dbcp2.
spring.datasource.tomcat.testOnBorrow=true
spring.datasource.tomcat.validationQuery=SELECT 1
If problem doesn't solve even after including properties as in application.properties, Then problem will be solved when including testOnBorrow,validationQuery in application-context.xml located in src/main/resources
<bean name="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${database.driver.classname}"/>
<property name="url" value="${database.url}"/>
<property name="username" value="${database.username}"/>
<property name="password" value="${database.password}"/>
<property name="initialSize" value="2"/>
<property name="maxActive" value="50"/>
<property name="maxIdle" value="5"/>
<property name="maxWait" value="-1"/>
<property name="removeAbandoned" value="true"/>
<property name="removeAbandonedTimeout" value="600"/>
<property name="logAbandoned" value="true"/>
<property name="testOnBorrow" value="true" />
<property name="validationQuery" value="SELECT 1" />
</bean>
The solution is to validate connection thread when it is borrowed from thread pool by enabling testOnBorrow and providing validationQuery.
I'm using Spring configuration file to configure C3P0. To monitor DataSource I configured net.bull.javamelody.SpringDataSourceFactoryBean as mentioned in the user guide of javamelody. But my report is showing 0 Active jdbc connections where as my minPoolSize is 10. What did I miss?
In web.xml added monitoring-spring.xml
<context-param>
<param-name>
contextConfigLocation
</param-name>
<param-value>
classpath:net/bull/javamelody/monitoring-spring.xml,
</param-value>
</context-param>
In Spring jdbc Configuration file is:
<bean id="sql2oDatasource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="#{dbProps['ops.jdbc.driverClassName']}"/>
<property name="jdbcUrl" value="#{dbProps['ops.jdbc.url']}"/>
<property name="user" value="#{dbProps['ops.jdbc.username']}"/>
<property name="password" value="#{dbProps['ops.jdbc.password']}"/>
<property name="maxPoolSize" value="#{dbProps['ops.c3p0.max_size']}"/>
<property name="minPoolSize" value="#{dbProps['ops.c3p0.min_size']}"/>
<property name="maxStatements" value="#{dbProps['ops.c3p0.max_statements']}"/>
<property name="checkoutTimeout" value="#{dbProps['ops.c3p0.timeout']}"/>
<property name="preferredTestQuery" value="SELECT 1"/>
</bean>
<!-- Configuring the session factory for SQL-2-O -->
<bean id="sql2oSession" class="org.sql2o.Sql2o">
<constructor-arg ref="wrappedDBDataSource"/>
<constructor-arg value="PostgreSQL" type="org.sql2o.QuirksMode"/>
</bean>
<bean id="wrappedDBDataSource" class="net.bull.javamelody.SpringDataSourceFactoryBean" primary="true">
<property name="targetName" value="sql2oDatasource"/>
</bean>
I tried to pass DriverClass as net.bull.javamelody.JdbcDriver in datasource and driver as:
<property name="properties">
<props>
<prop key="driver">org.postgresql.Driver</prop>
</props>
</property>
But postgresql driver is not getting registered this way.
Your configuration looks ok according to documentation. You see db active connection, pool size values on report that also means your config is ok.
In the other hand active db connection count means how many connections from db connection pool is connection operating with db at that moment. 0 active connections is good for you(if your application is working properly). It means your db operations are quickly done. It is hard to catch active connection while db operations are fast.
I have a working programm with MySQL, Spring and a tomcat server, but the database connection is in spring-database.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="baseController" class="com.afterguard.sailplanner.controller.BaseController">
<property name="dao" ref="daoImpl" />
</bean>
<bean id="daoImpl" class="com.afterguard.sailplanner.dao.DaoImpl">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/sailplanner" />
<property name="username" value="sailplanner" />
<property name="password" value="sailplanner2" />
</bean>
</beans>
In order to deploy this program on a server now, I would need bring this database connection to the server, e.g. the context.xml oder server.xml.
How do I do I setup my tomcat 7 server to connect to the database and tell my application, that the database connection is setup in the tomcat?
You have to create a dataSource with JNDI :
First, in your spring bean configuration (spring-database.xml) add this :
<beans:bean id="dbDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<beans:property name="jndiName" value="java:comp/env/jdbc/myDB"/>
</beans:bean>
Second, create the jndi resource in the file apache-tomcat/conf/server.xml :
<Resource name="jdbc/globalDB"
global="jdbc/globalDB"
auth="Container"
type="javax.sql.DataSource"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/sailplanner"
username="sailplanner"
password="sailplanner2"/>
Finally, create the resource link in the file apache-tomcat/conf/context.xml :
<ResourceLink name="jdbc/myDB"
global="jdbc/globalDB"
auth="Container"
type="javax.sql.DataSource" />
For Sql Server I do the following configuration:
You must be using Tomcat to manage connections. To do so we will be adding data sources to the TOMCAT_HOME/conf/context.xml file
<Context>
<Resource
name="jdbc/Oltp1"
auth="Container"
type="javax.sql.DataSource"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost:3306/sailplanner"
username=user
password=password
testOnBorrow="true"
testOnConnect="true"
validationQuery="SELECT 1"
removeAbandoned="true"
removeAbandonedTimeout="30"
/>
</Context>
SQL Server JDBC driver (jtds-version.jar) must be copied to the TOMCAT_HOME/lib folder (if it is not there already)
If there are multiple data sources (OLTP1, OLTP2, etc) we will be adding multiple <Resource/> tags
You will be using the following JNDI names to name our data sources:
jdbc/Oltp1 – main OLTP database
jdbc/Oltp2 – secondary database
You will be configuring data sources so connection pool will know how to test connections and what to do with closed connections:
testOnBorrow="true"
testOnConnect="true"
validationQuery="SELECT 1"
removeAbandoned="true"
removeAbandonedTimeout="30"
Tomcat must be restarted when new resource is added or existing resource is modified.
application.properties file
Important! This file is deployed as a part of the classpath (src/main/resources/application.properties), not the external application.properties with externalized properties deployed to the TOMCAT_HOME/conf
You will be added the following property to this file:
spring.datasource.jndi-name=java:comp/env/jdbc/Oltp1
You need to remove all other spring.datasource properties from the application.properties if any exist
Important! You had to remove all spring.datasource properties from the external configuration file (application.properties deployed to the COMCAT_HOME/conf)
the database connection properties you should put in a property file and and depending upon your requirement you can change values in property file
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClass" value="${jdbc.driverClassName}" />
<property name="jdbcUrl" value="${jdbc.databaseurl}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
</bean>
the property file
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.databaseurl=jdbc:mysql://localhost:3306/sailplanner
jdbc.username=root
jdbc.password=root
and to include this file in xml use
<bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="locations">
<list>
<value>classpath:applications.properties</value>
</list>
</property>
</bean>
Currently we are loading our JDBC source values from properties file as per following:
<context:property-placeholder location="classpath:master.properties" ignore-unresolvable="true" />
<bean id="mainDataSource" class="com.jolbox.bonecp.BoneCPDataSource"
destroy-method="close">
<property name="driverClass" value="${database.driver}" />
<property name="jdbcUrl" value="${database.url}" />
<property name="username" value="${database.user}" />
<property name="password" value="${database.password}" />
<property name="idleConnectionTestPeriod" value="60" />
<property name="idleMaxAge" value="240" />
<property name="maxConnectionsPerPartition" value="2" />
<property name="minConnectionsPerPartition" value="2" />
<property name="partitionCount" value="3" />
<property name="acquireIncrement" value="10" />
<property name="statementsCacheSize" value="50" />
<property name="releaseHelperThreads" value="3" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy"
scope="singleton">
<property name="targetDataSource">
<ref local="mainDataSource" />
</property>
</bean>
This seg. works fine with classpath based app.properties file and get rid of app.properties.
We would like to load this values from context.xml (either placed in META-INF or $CATALINA_HOME/conf/context.xml). It will help us load proper values at prod/staging servers.
Will appreciate any help or alternate method/suggestions. (If similar question is already answered, please share the link)
Thanks!
As Alan Hay mentioned you could externalize the datasource configuration into Tomcat's own context.xml and then have Spring do a JNDI lookup to retrieve it. This is an approach that I've commonly used on some of the projects I've worked on.
The pieces you need to put in place to achieve would be:
1. Add the datasource configuration to $CATALINA_HOME/conf/context.xml
<GlobalNamingResources>
<Resource type="javax.sql.DataSource"
name="dsName"
factory="com.jolbox.bonecp.BoneCPDataSource"
driverClassName="your.driver.classname"
jdbcUrl="your:driver:url"
username="username"
password="password"
idleMaxAge="240"
idleConnectionTestPeriod="60"
partitionCount="3"
acquireIncrement="10"
maxConnectionsPerPartition="2"
minConnectionsPerPartition="2"
statementsCacheSize="50"
releaseHelperThreads="3" />
</GlobalNamingResources>
2. Add a resource link in the application's META-INF/context.xml
<Context path="/YourApp">
<ResourceLink description="Datasource for YourApp"
global="jdbc/dsName"
name="jdbc/dsName"
type="javax.sql.DataSource" />
</Context>
3. Modify the Spring config to lookup the datasource in JNDI
<beans xmlns:jee="http://www.springframework.org/schema/jee"
xsi:schemaLocation="http://www.springframework.org/schema/jee classpath:/org/springframework/ejb/config/spring-jee-3.0.xsd">
<jee:jndi-lookup id="dataSource"
jndi-name="java:comp/env/jdbc/dsName" />
4. Move the driver and datasource jars
Since the datasource configuration is now container managed, you should place the database driver and datasource jars into $CATALINA_HOME/lib so they are available to Tomcat when it creates the datasource. These jars should no longer need to reside in the WEB-INF/lib of your application.
The way I do it, I have default properties file in my classpath, and I have one properties file that I use to override those default (development) properties:
in my spring file:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="true"/>
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
<value>classpath:camel.properties</value>
<value>classpath:email.properties</value>
<value>${external_config}</value>
</list>
</property>
</bean>
In development, I use the properties file in the classpath.
On the different envionment, staging, integration, production we start the application with the following parameter:
-Dexternal_config=file:c:/staging.properties
That way I have the same build for all environments and can choose the location of my overriding properties file.
So, ${external_config} is a place holder for a configuration file that depends on the environment.
Spring looks for an environment variable named external_config, that variable should point to a configuration file. If the file is not found, Spring ignores it with ignoreResourceNotFound = true;
Spring 3.1 introduced support for environment specific profiles. Define some profiles, associate them with some properties files e.g. test.properties, live.properties and set an environment variable on server start -Dspring.profiles.active=Test. Clean and simple.
http://spring.io/blog/2011/02/11/spring-framework-3-1-m1-released/
Other option as hinted at above is to shift to a container managed Connection Pool looked up via JNDI i.e define a JNDI datasource named xyz the same on each server and then:
I develop an applivation with very load(request).
I used following technologies in my appliation:
Jpa/Hibernate as persistense layer
Spring and Spring Dao
C3p0 as connection pooling
my problem is : I run my application , when number of request increase, throw exception in
persistense layer that"Cannt open connection"
I increase oracle max session but my problem not solve
I indept in C3p0 document and test its options but my problem not solve.
Thank you for your attention
You increased max sessions on Oracle, but you didn't increase the max size of your connection pool. The exception is telling you that your pool is exhausted. Either find what's holding connections open and get them released sooner, or increase the number of max active connections in the pool.
Is it possible for you to post the Spring configuration for your DataSource. I would expect something like:
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="oracle.jdbc.driver.OracleDriver"/>
<property name="jdbcUrl" value="${jdbc.connection.url}"/>
<property name="user" value="${jdbc.connection.username}"/>
<property name="password" value="${jdbc.connection.password}"/>
<property name="initialPoolSize" value="5"/>
<property name="minPoolSize" value="5"/>
<property name="maxPoolSize" value="100"/>
</bean>
With another bean configured where the dataSource is passed by reference:
<bean id="mySampleDao" class="com.example.dao.MySampleDao">
<property name="dataSource" ref="dataSource" />
</bean>
Is this what you have?
What version of Oracle are you using?