I am trying to connect to mysql database using apache dbcp2 connection pooling following its tutorial from here:
https://git-wip-us.apache.org/repos/asf?p=commons-dbcp.git;a=blob;f=doc/PoolingDataSourceExample.java;h=2a12c74898930b9623223db1597b8a8052a6f1df;hb=HEAD
My Database connection class that return the connection looks something like this:
public class DbConnection {
private static interface Singleton {
final DbConnection INSTANCE = new DbConnection();
}
private final PoolingDataSource<PoolableConnection> dataSource;
private DbConnection() {
// A ConnectionFactory that the pool will use to create Connections.
DriverManagerConnectionFactory connectionFactory = new DriverManagerConnectionFactory(url, user, pass); //url,user,pass for db connnection
//implement the pooling functionality.
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory, null);
GenericObjectPoolConfig config = new GenericObjectPoolConfig();
config.setMaxWaitMillis(500);
config.setMaxTotal(20);
config.setMaxIdle(5);
config.setMinIdle(5);
//PoolingDataSource expect an ObjectPool
ObjectPool<PoolableConnection> connectionPool = new GenericObjectPool<>(poolableConnectionFactory, config);
// Set the poolableConnectionFactory's pool property to the owning pool
poolableConnectionFactory.setPool(connectionPool);
this.dataSource = new PoolingDataSource<>(connectionPool);
}
public static Connection getCon() throws SQLException {
return Singleton.INSTANCE.dataSource.getConnection();
}
}
I am using apache jmeter to test the connection and return something from my mysql database. I created 100 users with Ramp-up periods(in seconds) being 2 second. I created a Http request and when I tried to see my response in view results tree I successfully get response for first 20 request. Later requests from (21 to 100) have blank response. I have gone through many of the issues involving:
java.sql.SQLException: Cannot get a connection, pool error Timeout
waiting for idle object
I access my code as:
try (PreparedStatement ps = DbConnection.getCon().prepareStatement("SELECT id FROM test WHERE id =?;")) {
ps.setString(1, id);
try (
ResultSet rs = ps.executeQuery()) {
return rs.next();
}
} catch (Exception ex) {
}
You aren't closing Connection object, you must declare such variable in try-with-resources block :
try (Connection conn = DbConnection.getCon();
PreparedStatement ps = conn.prepareStatement("SELECT id FROM test WHERE id =?;")) {
Also you need to close ResultSet, either call close method ResultSet in method or add it to try-with-resources block using a new method returning it
Related
I'm looking for a way to
configure Hikari Connection Pool for Spring DataSource
explicitly set fetchSize for resultSet.
Here is my application.properties
# Datasource
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.url=jdbc:postgresql://mydb
spring.datasource.username=user
spring.datasource.password=pass
spring.datasource.driverClassName=org.postgresql.Driver
#Hikari
spring.datasource.hikari.connectionTimeout=30000
spring.datasource.hikari.idleTimeout=600000
spring.datasource.hikari.maxLifetime=1800000
spring.datasource.hikari.maximum-pool-size=100
spring.datasource.hikari.auto-commit=true
here is my service code
#Service
#RequiredArgsConstructor
public class PerfService {
private final JdbcTemplate template;
public RowCountResponse getData(String param) {
Connection connection = null;
PreparedStatement preparedStatement = null;
try {
connection = Objects.requireNonNull(template.getDataSource()).getConnection();
preparedStatement = connection.prepareStatement("SELECT whatever");
preparedStatement.setFetchSize(30000); // !!!
preparedStatement.setString(1, param);
ResultSet resultSet = preparedStatement.executeQuery();
int rowCount = 0;
while (resultSet.next()) {
rowCount++;
}
resultSet.close();
return new RowCountResponse()
.setRowCount(rowCount);
} catch (Exception e) {
throw new RuntimeException("Error while fetching rows", e);
} finally {
JdbcUtils.closeStatement(preparedStatement);
DataSourceUtils.releaseConnection(connection, template.getDataSource());
}
}
}
Seems like my pool is leaking. I don't return connection properly to the pool.
Is there more elegant way to use jdbcTemplate and preparedStatement.setFetchSize(30000);
Sorry for being late, But may help others.
There is an issue with postgresql when setting fetchSize - https://github.com/spring-projects/spring-framework/issues/24848
In order to work you also have to set autoCommit=false for your connection so in your example it would be:
spring.datasource.hikari.auto-commit=false
But setting auto-commit to false for your connection pool could be inconvenient so one workaround is to have two connection pools - one readonly with autoCommit=false and second(ordinary) with autoCommit=true.
Also JdbcTemplate have setFetchSize method so you could simply do
template.setFetchSize(30000);
You can use the connection parameter defaultRowFetchSize, see the documentation.
I'm using EJB 2 with container managed transactions (JTA) on WebLogic server. For debugging purposes, I'm trying to get Connection object from the DataSource retrieved from JNDI, execute an update and commit. However, I'm getting the following exception when trying to commit.
java.sql.SQLException: Cannot call commit when using distributed transactions
The DataSource type I'm using is of type weblogic.jdbc.common.internal.RmiDataSource and the Connection object is of type weblogic.jdbc.wrapper.JTAConnection_weblogic_jdbc_wrapper_XAConnection_oracle_jdbc_driver_LogicalConnection-OracleDS-187.
Clearly the Connection object I'm retrieving is trying to participate in JTA transaction, which is not what I want. How do I get non-JTA-managed connection from the DataSource?
Here is how I'm currently retrieving the connection
Connection connection = null;
PreparedStatement ps = null;
try {
DataSource ds = ConnectionFactory.getDataSourceFromJNDI("jdbc/OracleDS");
connection = ds.getConnection();
ps = connection.prepareStatement(...);
ps.setString(1, msg);
ps.executeUpdate();
connection.commit();
}
catch (Exception e) {
if (connection != null)
connection.rollback();
}
}
Im having a problem with an application, I'm using EclipseLink 2.4.1 with oracle 11g, I'm getting a J2CA0045E error, because the pooling does not set free any connections pool from the jndi,
Have you have this problem?, apparently the connection does not get released and it reach the limit I've configured.
Any ideas, this is my Connection pool
Connection timeout: 180
Maximum connections: 10
Minimum connections: 1
Reap time: 60
Unused timeout: 180
Aged timeout: 120
Purge policy: Entire Pool
EDIT
I have a web service in the same service, and the behavior is the same, using same jndi, the connection increase and does not release any connection
initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/abc");
conn = ds.getConnection();
Based on the code provided, it looks like you are storing the Connection reference at a class scope. You should avoid storing references to Connection objects that could span multiple requests.
Instead, store a reference to the DataSource.
To illustrate. This approach is preferred:
public class GoodService {
DataSource ds;
public GoodService() throws NamingException {
// Only the DataSource reference is stored, this is allowed
ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
}
public void doSomething() throws Exception {
// Get the connection in a request.
Connection conn = ds.getConnection();
try {
// do something...
} finally {
// Always close the connection in the finally block
// so it gets returned to the pool no matter what
conn.close();
}
}
}
The following approach is considered bad practice, and is likely to cause the J2CA0045E error you are seeing:
public class BadService {
Connection conn; // BAD: don't store a Connection across requests!
public BadService() throws NamingException, SQLException {
DataSource ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
// BAD: don't leave connections open indefinitely like this
conn = ds.getConnection();
}
public void doSomething() throws Exception {
// do something using conn...
}
}
I want to make a connection pool for my h2 database. But I think my pool opens new connection every time I call getConnection(). I guess there should be a fixed amount of reusable connections, but if I run this code :
Connection conn = DataSource.getInstance().getConnection();
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("SELECT * FROM NODE_USERS;");
while (rs.next()) {
System.out.println(rs.getString("login"));
}
try {
// wait a bit
Thread.sleep(20000);
} catch (InterruptedException e) {
e.printStackTrace();
}
stmt.close();
rs.close();
conn.close();
DataSource:
public class DataSource {
private static volatile DataSource datasource;
private BasicDataSource ds;
private DataSource() throws IOException, SQLException, PropertyVetoException {
ds = new BasicDataSource();
ds.setUsername("sa");
ds.setPassword("sa");
ds.setUrl("jdbc:h2:tcp://localhost/~/test");
ds.setMinIdle(5);
ds.setMaxActive(10);
ds.setMaxIdle(20);
ds.setMaxOpenPreparedStatements(180);
}
public static DataSource getInstance() throws IOException, SQLException, PropertyVetoException {
if (datasource == null) {
synchronized (DataSource.class) {
if (datasource == null) {
datasource = new DataSource();
}
}
datasource = new DataSource();
}
return datasource;
}
public Connection getConnection() throws SQLException {
return this.ds.getConnection();
}
}
and then execute select * from information_schema.sessions;, there will be two rows. What is wrong? Also I was trying H2 tutorial example, but I've got the same result.
You are using a connection pool, namely BasicDataSource. It will create a configured number of connections initially and then when getConnection() is called, it will either reused a free pooled connection or create a new one, up to a configured limit (or no limit if configured so). When the connection obtained is "closed" using Connection.close(), it is actually returned to the pool instead of being closed right away.
You basically have no control over how many open connctions there will be at a given time, apart of configuring the minimum and maximum allowed open connections. You observing two open connections therefore doesn't prove anything. If you want to see whether there is a limit on open connections, configure the BasicDataSource to use at most 2 connections using BasicDataSource.maxActive(), then try to obtain three connections at the same time. And for another test, with the same configuration, try obtaining two connections, returning them using Connection.close() and then obtaining another two connections.
I use c3p0 library of version 0.9.2.1 for creating a connection pool in a web application.
After MySQL timeout I get exception:
"HTTP Status 500 - The last packet successfully received from the
server was 627 301 milliseconds ago. The last packet sent successfully
to the server was 627 302 milliseconds ago. is longer than the server
configured value of 'wait_timeout'. You should consider either
expiring and/or testing connection validity before use in your
application, increasing the server configured values for client
timeouts, or using the Connector/J connection property
'autoReconnect=true' to avoid this problem."
I tried to append an autoReconnect parameter to JDBC url but there is no effect as well. So, in that way I use connection pool in my application:
For testing I have set wait_timeout of MySQL to 180 sec
set ##global.wait_timeout=180;
show global variables like "wait_timeout";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| wait_timeout | 180 |
+---------------+-------+
and further there are next pieces of code
c3p0.properties
# JDBC paramters are ommited
# Basic pool configuration
c3p0.initialPoolSize=5
c3p0.minPoolSize=5
c3p0.maxPoolSize=50
c3p0.acquireIncrement=5
# Managing connection age
c3p0.maxConnectionAge=100
c3p0.maxIdleTime=90
# Configuring connection testing
c3p0.idleConnectionTestPeriod=30
c3p0.testConnectionOnCheckin=true
c3p0.preferredTestQuery=SELECT 1
DBConnectionUtil.java
public class DBConnectionUtil {
// initialized through c3p0.properties
private static ComboPooledDataSource ds = new ComboPooledDataSource();
public static ComboPooledDataSource getConnectionPool() {
return ds;
}
public static void destroyConnectionPool() {
ds.close();
}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
}
UserDAO.java
public class UserDAO {
private Connection connection;
public UserDAO() throws SQLException {
connection = DBConnectionUtil.getConnection();
}
public User find(Integer id) throws SQLException {
User user = null;
PreparedStatement ps = connection.prepareStatement("SELECT * FROM `USERS` WHERE ID = ?");
ps.setInt(1, id);
ResultSet rs = ps.executeQuery();
if (rs.next()) {
user = new User();
user.setId(rs.getInt("ID"));
user.setName(rs.getString("NAME"));
user.setUsername(rs.getString("USERNAME"));
user.setPassword(rs.getString("PASSWORD"));
user.setParentId(rs.getInt("PARENT_ID"));
}
rs.close();
ps.close();
return user;
}
}
DAOUtil.java
public class DAOUtil {
private static UserDAO userDAO;
public static UserDAO getUserDAO() {
return userDAO;
}
static {
try {
userDAO = new UserDAO();
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
}
After expired MySQL wait_timeout, for instance, when I call DAOUtil.userDAO.find(id) it throws exception which was described above.
Could you help me to realize what I'm doing wrong, please?
Note: I can't change MySQL ini file.
Try to close connection in your Dao class and ask for new one from connection pool for every request to database.
Connection con;
try {
con=DBConnectionUtil.getConnection();
//some code here
} finally {
if(con!=null){
con.close();
}
And it's not safe to have Connection as a object field, better to use it as a local variable, because connection is not thread safe.