We Keep Coding

java dbcp2 connection pooling

So I am trying to understand how to build connection pools and to connect via java to an Oracle DB. i am trying to use dbcp2 to learn more about how all of this works. If I use the BasicDataSource I can connect and I see 5 connections via the number I set up when I created the pools like this.
private static BasicDataSource getDataSource() {
{
if (dataSource == null)
{
BasicDataSource ds = new BasicDataSource();
ds.setUrl("jdbc:oracle:thin:#pdb_tac");
ds.setUsername("hr");
ds.setPassword("my_password");
//ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setDriverClassName("oracle.jdbc.replay.OracleDataSourceImpl");
//ds.setDriverClass("oracle.jdbc.replay.OracleDataSourceImpl");
ds.setDefaultAutoCommit(false);
ds.setInitialSize(5);
ds.setMinIdle(5);
ds.setMaxIdle(10);
ds.setMaxOpenPreparedStatements(100);
dataSource = ds;
}
return dataSource;
}
}
Well, my issue is I need to be able to create a PoolingDataSource to deal with connecting to an Oracle RAC for high availability.
So I create my data source like this.
private static DataSource setupDataSource(){
String connectURI = "jdbc:oracle:thin:hr/my_password#pdb_tac";
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI,null);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory,null);
ObjectPool objectPool = new GenericObjectPool(poolableConnectionFactory);
poolableConnectionFactory.setPool(objectPool);
PoolingDataSource dataSource = new PoolingDataSource(objectPool);
return dataSource;
}
Well when I connect I only see 1 connection to the database not 5 like I do when I use BasicDataSource. Can someone help me understand what I am doing wrong? Thanks.

How to make app still run when the hikaridatasource bean cannot be created due to some external datasource problem?

when there will be an issue with the external database for a live webapp and bean won't be able to get instantiated when retried again by the application then app goes down how can we fix this problem?
#Bean(destroyMethod = "close")
public DataSource dataSource(){
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setDriverClassName("com.mysql.jdbc.Driver");
hikariConfig.setJdbcUrl("jdbc:mysql://localhost:3306/spring-test");
hikariConfig.setUsername("root");
hikariConfig.setPassword("admin");
hikariConfig.setMaximumPoolSize(5);
hikariConfig.setConnectionTestQuery("SELECT 1");
hikariConfig.setPoolName("springHikariCP");
hikariConfig.addDataSourceProperty("dataSource.cachePrepStmts", "true");
hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSize", "250");
hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSqlLimit", "2048");
hikariConfig.addDataSourceProperty("dataSource.useServerPrepStmts", "true");
HikariDataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource;
}

Your code will probably throw an exception if the datasource is down. Wrap it into try/catch like so:
try {
<your code goes here>
} catch (Exception e) {
return null;
}
But code calling for a datasource will have to accept the result may be null rather than throwing NullPointerExceptions.

Spring Batch - Problems Connecting to Postgres Database

Up until now I had been using in-memory H2 DB with Spring Batch. However, now I switched to connecting to external postgres DB. Here was my connection object (with some obfuscation):
#Bean
public DataSource postgresDatasource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.postgresql.Driver");
datasource.setUrl("jdbc:postgresql://x.x.x.x:xxxx/blah");
datasource.setUsername("Joe");
datasource.setPassword("password");
return datasource;
}
When I start my application, I get:
Caused by: org.springframework.jdbc.BadSqlGrammarException:
PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID,
JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?];
nested exception is org.postgresql.util.PSQLException: ERROR: relation
"batch_job_instance" does not exist
I then read that Spring Batch uses the database to save metadata for its recover/retry functionality, and with embedded databases, these are tables Spring Batch sets up by default. Ok, so that would explain why I had never seen this error before.
However, it said I could set this property:
spring.batch.initialize-schema=never
So I put this in my application.properties file. However, I am still getting the error. I would be grateful for any ideas.

I was able to address this myself. Ultimately I needed the Spring Batch repository independent from my actual target relational database. So I found this reference:
https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84
I was able to take the DefaultBatchConfigurer class from that example and make a minor change to the data source by adding the #Qualifier for embedded/local data source:
#Autowired(required = false)
public void setDataSource(#Qualifier("dataSource") DataSource dataSource) {
this.dataSource = dataSource;
this.transactionManager = new DataSourceTransactionManager(dataSource);
}
Then, on my Spring Batch reader (in my other batch config class), I made a minor change to the data source by adding the #Qualifier for postgres data source:
#Bean
public ItemReader<StuffDto> itemReader(#Qualifier("postgresDataSource")DataSource dataSource) {
return new JdbcCursorItemReaderBuilder<StuffDto>()
.name("cursorItemReader")
.dataSource(dataSource)
.sql(GET_DATA)
.rowMapper(new BeanPropertyRowMapper<>(StuffDto.class))
.build();
}
Then lastly (or firstly really as I did these first), I explicitly named my data source beans so java could tell them apart to use as above:
#Configuration
public class PersistenceContext {
#Bean(name = "dataSource")
public DataSource dataSource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.h2.Driver");
datasource.setUrl("jdbc:h2:file:/tmp/test");
datasource.setUsername("sa");
datasource.setPassword("");
return datasource;
}
#Bean(name = "postgresDataSource")
public DataSource postgresDatasource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.postgresql.Driver");
datasource.setUrl("jdbc:postgresql://x.x.x.x:xxxx/blah");
datasource.setUsername("joe");
datasource.setPassword("password");
return datasource; }
}
Once I did all the above, the error disappeared and everything worked.

How to close the connection in Spring Boot after jdbcTemplate executes the query?

In Spring Boot, does jdbcTemplate not close the connection automatically once after the it executes the query?
In this case, I am executing a query using jdbcTemplate(where it connects to teradata) but the session is not closing after the query is executed. How can I close the session?
This is my dao file -
#Component
public class DDLReviewDao {
#Autowired
#Qualifier("devbJdbc")
private JdbcTemplate jdbcTemplate;
public static final Logger logger = LogManager.getLogger(DDLReviewDao.class);
public List<DDLObject> getDDLReviewData(DDLQuery ddlQuery) {
String selectSql = MacroGenerator.generateMacro(ddlQuery);
List<DDLObject> ddlObject = jdbcTemplate.query(selectSql, new DDLMapper());
logger.info(ddlObject);
return ddlObject;
}
}

JdbcTemplate gets its connections from javax.sql.DataSource implementation - which is passed to its constructor link.
The DataSources can be basic (creates Connection object for each request) or pooling (has pool of connections and just 'borrows' one for given request's use).
So, it appears that the connection is not closing because you have passed some pooling datasource to JdbcTemplate named devbJdbc. If you realy want to close every connection opened to do the JdbcTemplate job, you can use a basic DataSource implementation: org.springframework.jdbc.datasource.SimpleDriverDataSource just like that:
#Configuration
class DevbConfig {
#Bean(name = "devbDataSource")
DataSource devbDataSource() {
try {
return new SimpleDriverDataSource(DriverManager.getDriver("jdbc:..."), "jdbc:...", "username", "password");
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
#Bean(name = "devbJdbc")
JdbcTemplate devbJdbc(#Qualifier("devbDataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}

In Spring Boot, does jdbcTemplate not close the connection
automatically once after the it executes the query?
Should it close the connection or return it to the connection pool (in case the DataSource is pooled)?
If you read the source code at http://grepcode.com/file/repo1.maven.org/maven2/org.springframework/spring-jdbc/4.1.7.RELEASE/org/springframework/jdbc/core/JdbcTemplate.java#JdbcTemplate.execute%28org.springframework.jdbc.core.StatementCallback%29it boils down to:
public static void doReleaseConnection(Connection con, DataSource dataSource) throws SQLException {
if (con == null) {
return;
}
if (dataSource != null) {
ConnectionHolder conHolder = (ConnectionHolder) TransactionSynchronizationManager.getResource(dataSource);
if (conHolder != null && connectionEquals(conHolder, con)) {
// It's the transactional Connection: Don't close it.
conHolder.released();
return;
}
}
logger.debug("Returning JDBC Connection to DataSource");
doCloseConnection(con, dataSource);
}
and
public static void doCloseConnection(Connection con, DataSource dataSource) throws SQLException {
if (!(dataSource instanceof SmartDataSource) || ((SmartDataSource) dataSource).shouldClose(con)) {
con.close();
}
}
Most-likely, if the DataSource instance is pooled, connections are release back for reuse and not closed.

According to the spring boot docs you can allocate a maximum number of connections to a spring pool like so:
spring.datasource.tomcat.max-active=50
This will obviously only work with the embedded webserver. If you are deploying it to something like a Jboss you'll have to configure that property in the Server config file.

Spring JdbcTemplate with DBCP BasicDataSource still closes connections

I'm using postgres jdbc driver to connect to Amazon RedShift. Here are also BasicDataSource from DBCP 2.0.1 and JdbcTemplate from Spring 4. I use DataSourceTransactionManager with Transactional annotations.
It looks like DataSource still keeps on creating new connections!
// that is how dataSource is created
BasicDataSource dataSource = new BasicDataSource() {
public Connection getConnection() throws SQLException {
Connection c = super.getConnection();
System.out.println("New connection: " + c);
return c;
}
};
dataSource.setUsername(env.getProperty(USERNAME));
dataSource.setPassword(env.getProperty(PASSWORD));
dataSource.setDriverClassName(env.getProperty(DRIVER_CLASS));
dataSource.setUrl(env.getProperty(CONNECTION_URL));
and I see in console for each operation another Connection object (they have different hashcodes). If I switch to SingleConnectionDataSource all works as expected, with a single connection object.
Before call to jdbcTemplate#execute I use TransactionSynchronizationManager.isActualTransactionActive to see that transactions are working (they are)...
What could I miss then? Why transactions are closed? Or what more can I do to investigate the problem. The url also have tcpKeepAlive=true parameter...
UPD thanks to Evgeniy, I've changed the code to see when connections are really created:
BasicDataSource dataSource = new BasicDataSource() {
protected ConnectionFactory createConnectionFactory() throws SQLException {
final ConnectionFactory cf = super.createConnectionFactory();
return new ConnectionFactory() {
public Connection createConnection() throws SQLException {
Connection c = cf.createConnection();
System.out.println("New connection from factory: " + c);
return c;
}
};
}
};
//dataSource.setMaxIdle(0);
Now I really see that only two connections were created (and if I add setMaxIdle(0) they are instead recreated before each query).
So my suspicion was wrong and pool works as expected. Thanks a lot!

Different hash codes do not prove they are different physical connections. Try to watch sessions on the database and you will see that close on connection from BasicDataSource does not close a physical connection.