I'm using postgres jdbc driver to connect to Amazon RedShift. Here are also BasicDataSource from DBCP 2.0.1 and JdbcTemplate from Spring 4. I use DataSourceTransactionManager with Transactional annotations.
It looks like DataSource still keeps on creating new connections!
// that is how dataSource is created
BasicDataSource dataSource = new BasicDataSource() {
public Connection getConnection() throws SQLException {
Connection c = super.getConnection();
System.out.println("New connection: " + c);
return c;
}
};
dataSource.setUsername(env.getProperty(USERNAME));
dataSource.setPassword(env.getProperty(PASSWORD));
dataSource.setDriverClassName(env.getProperty(DRIVER_CLASS));
dataSource.setUrl(env.getProperty(CONNECTION_URL));
and I see in console for each operation another Connection object (they have different hashcodes). If I switch to SingleConnectionDataSource all works as expected, with a single connection object.
Before call to jdbcTemplate#execute I use TransactionSynchronizationManager.isActualTransactionActive to see that transactions are working (they are)...
What could I miss then? Why transactions are closed? Or what more can I do to investigate the problem. The url also have tcpKeepAlive=true parameter...
UPD thanks to Evgeniy, I've changed the code to see when connections are really created:
BasicDataSource dataSource = new BasicDataSource() {
protected ConnectionFactory createConnectionFactory() throws SQLException {
final ConnectionFactory cf = super.createConnectionFactory();
return new ConnectionFactory() {
public Connection createConnection() throws SQLException {
Connection c = cf.createConnection();
System.out.println("New connection from factory: " + c);
return c;
}
};
}
};
//dataSource.setMaxIdle(0);
Now I really see that only two connections were created (and if I add setMaxIdle(0) they are instead recreated before each query).
So my suspicion was wrong and pool works as expected. Thanks a lot!
Different hash codes do not prove they are different physical connections. Try to watch sessions on the database and you will see that close on connection from BasicDataSource does not close a physical connection.
Related
So I am trying to understand how to build connection pools and to connect via java to an Oracle DB. i am trying to use dbcp2 to learn more about how all of this works. If I use the BasicDataSource I can connect and I see 5 connections via the number I set up when I created the pools like this.
private static BasicDataSource getDataSource() {
{
if (dataSource == null)
{
BasicDataSource ds = new BasicDataSource();
ds.setUrl("jdbc:oracle:thin:#pdb_tac");
ds.setUsername("hr");
ds.setPassword("my_password");
//ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setDriverClassName("oracle.jdbc.replay.OracleDataSourceImpl");
//ds.setDriverClass("oracle.jdbc.replay.OracleDataSourceImpl");
ds.setDefaultAutoCommit(false);
ds.setInitialSize(5);
ds.setMinIdle(5);
ds.setMaxIdle(10);
ds.setMaxOpenPreparedStatements(100);
dataSource = ds;
}
return dataSource;
}
}
Well, my issue is I need to be able to create a PoolingDataSource to deal with connecting to an Oracle RAC for high availability.
So I create my data source like this.
private static DataSource setupDataSource(){
String connectURI = "jdbc:oracle:thin:hr/my_password#pdb_tac";
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI,null);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory,null);
ObjectPool objectPool = new GenericObjectPool(poolableConnectionFactory);
poolableConnectionFactory.setPool(objectPool);
PoolingDataSource dataSource = new PoolingDataSource(objectPool);
return dataSource;
}
Well when I connect I only see 1 connection to the database not 5 like I do when I use BasicDataSource. Can someone help me understand what I am doing wrong? Thanks.
In Spring Boot, does jdbcTemplate not close the connection automatically once after the it executes the query?
In this case, I am executing a query using jdbcTemplate(where it connects to teradata) but the session is not closing after the query is executed. How can I close the session?
This is my dao file -
#Component
public class DDLReviewDao {
#Autowired
#Qualifier("devbJdbc")
private JdbcTemplate jdbcTemplate;
public static final Logger logger = LogManager.getLogger(DDLReviewDao.class);
public List<DDLObject> getDDLReviewData(DDLQuery ddlQuery) {
String selectSql = MacroGenerator.generateMacro(ddlQuery);
List<DDLObject> ddlObject = jdbcTemplate.query(selectSql, new DDLMapper());
logger.info(ddlObject);
return ddlObject;
}
}
JdbcTemplate gets its connections from javax.sql.DataSource implementation - which is passed to its constructor link.
The DataSources can be basic (creates Connection object for each request) or pooling (has pool of connections and just 'borrows' one for given request's use).
So, it appears that the connection is not closing because you have passed some pooling datasource to JdbcTemplate named devbJdbc. If you realy want to close every connection opened to do the JdbcTemplate job, you can use a basic DataSource implementation: org.springframework.jdbc.datasource.SimpleDriverDataSource just like that:
#Configuration
class DevbConfig {
#Bean(name = "devbDataSource")
DataSource devbDataSource() {
try {
return new SimpleDriverDataSource(DriverManager.getDriver("jdbc:..."), "jdbc:...", "username", "password");
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
#Bean(name = "devbJdbc")
JdbcTemplate devbJdbc(#Qualifier("devbDataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}
In Spring Boot, does jdbcTemplate not close the connection
automatically once after the it executes the query?
Should it close the connection or return it to the connection pool (in case the DataSource is pooled)?
If you read the source code at http://grepcode.com/file/repo1.maven.org/maven2/org.springframework/spring-jdbc/4.1.7.RELEASE/org/springframework/jdbc/core/JdbcTemplate.java#JdbcTemplate.execute%28org.springframework.jdbc.core.StatementCallback%29it boils down to:
public static void doReleaseConnection(Connection con, DataSource dataSource) throws SQLException {
if (con == null) {
return;
}
if (dataSource != null) {
ConnectionHolder conHolder = (ConnectionHolder) TransactionSynchronizationManager.getResource(dataSource);
if (conHolder != null && connectionEquals(conHolder, con)) {
// It's the transactional Connection: Don't close it.
conHolder.released();
return;
}
}
logger.debug("Returning JDBC Connection to DataSource");
doCloseConnection(con, dataSource);
}
and
public static void doCloseConnection(Connection con, DataSource dataSource) throws SQLException {
if (!(dataSource instanceof SmartDataSource) || ((SmartDataSource) dataSource).shouldClose(con)) {
con.close();
}
}
Most-likely, if the DataSource instance is pooled, connections are release back for reuse and not closed.
According to the spring boot docs you can allocate a maximum number of connections to a spring pool like so:
spring.datasource.tomcat.max-active=50
This will obviously only work with the embedded webserver. If you are deploying it to something like a Jboss you'll have to configure that property in the Server config file.
Im having a problem with an application, I'm using EclipseLink 2.4.1 with oracle 11g, I'm getting a J2CA0045E error, because the pooling does not set free any connections pool from the jndi,
Have you have this problem?, apparently the connection does not get released and it reach the limit I've configured.
Any ideas, this is my Connection pool
Connection timeout: 180
Maximum connections: 10
Minimum connections: 1
Reap time: 60
Unused timeout: 180
Aged timeout: 120
Purge policy: Entire Pool
EDIT
I have a web service in the same service, and the behavior is the same, using same jndi, the connection increase and does not release any connection
initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/abc");
conn = ds.getConnection();
Based on the code provided, it looks like you are storing the Connection reference at a class scope. You should avoid storing references to Connection objects that could span multiple requests.
Instead, store a reference to the DataSource.
To illustrate. This approach is preferred:
public class GoodService {
DataSource ds;
public GoodService() throws NamingException {
// Only the DataSource reference is stored, this is allowed
ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
}
public void doSomething() throws Exception {
// Get the connection in a request.
Connection conn = ds.getConnection();
try {
// do something...
} finally {
// Always close the connection in the finally block
// so it gets returned to the pool no matter what
conn.close();
}
}
}
The following approach is considered bad practice, and is likely to cause the J2CA0045E error you are seeing:
public class BadService {
Connection conn; // BAD: don't store a Connection across requests!
public BadService() throws NamingException, SQLException {
DataSource ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
// BAD: don't leave connections open indefinitely like this
conn = ds.getConnection();
}
public void doSomething() throws Exception {
// do something using conn...
}
}
I want to make a connection pool for my h2 database. But I think my pool opens new connection every time I call getConnection(). I guess there should be a fixed amount of reusable connections, but if I run this code :
Connection conn = DataSource.getInstance().getConnection();
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("SELECT * FROM NODE_USERS;");
while (rs.next()) {
System.out.println(rs.getString("login"));
}
try {
// wait a bit
Thread.sleep(20000);
} catch (InterruptedException e) {
e.printStackTrace();
}
stmt.close();
rs.close();
conn.close();
DataSource:
public class DataSource {
private static volatile DataSource datasource;
private BasicDataSource ds;
private DataSource() throws IOException, SQLException, PropertyVetoException {
ds = new BasicDataSource();
ds.setUsername("sa");
ds.setPassword("sa");
ds.setUrl("jdbc:h2:tcp://localhost/~/test");
ds.setMinIdle(5);
ds.setMaxActive(10);
ds.setMaxIdle(20);
ds.setMaxOpenPreparedStatements(180);
}
public static DataSource getInstance() throws IOException, SQLException, PropertyVetoException {
if (datasource == null) {
synchronized (DataSource.class) {
if (datasource == null) {
datasource = new DataSource();
}
}
datasource = new DataSource();
}
return datasource;
}
public Connection getConnection() throws SQLException {
return this.ds.getConnection();
}
}
and then execute select * from information_schema.sessions;, there will be two rows. What is wrong? Also I was trying H2 tutorial example, but I've got the same result.
You are using a connection pool, namely BasicDataSource. It will create a configured number of connections initially and then when getConnection() is called, it will either reused a free pooled connection or create a new one, up to a configured limit (or no limit if configured so). When the connection obtained is "closed" using Connection.close(), it is actually returned to the pool instead of being closed right away.
You basically have no control over how many open connctions there will be at a given time, apart of configuring the minimum and maximum allowed open connections. You observing two open connections therefore doesn't prove anything. If you want to see whether there is a limit on open connections, configure the BasicDataSource to use at most 2 connections using BasicDataSource.maxActive(), then try to obtain three connections at the same time. And for another test, with the same configuration, try obtaining two connections, returning them using Connection.close() and then obtaining another two connections.
I have main data source for working with database, which uses some connection pool. Now I want to create a second data source, which would use the same connection pool to perform logging operations in a separate transaction, then main database transaction! As far as I understood from the Glassfish docs if multiple data sources are using the same connection pool, then they share a transaction until connection is not closed (I might be wrong, please correct me).
So, is there a way to start a new transaction, when getting a connection to a data source? By setting TransactionIsolation may be?
Connection is retrieved in the following way:
private synchronized Connection getConnection() {
if (connection == null) {
try {
final Context ctx = new InitialContext();
final DataSource ds = (DataSource) ctx.lookup(getDataSourceLookupAddress());
connection = ds.getConnection();
} catch (final NamingException e) {
errorHandler.error("Datasource JNDI lookup failed: " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
} catch (final SQLException e) {
errorHandler.error("Sql connection failed to " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
}
}
return connection;
}
I have figured it out. It's necessary to start a new Thread, then a new transaction will be created. See also Writing to database log in the middle of rollback