We Keep Coding

Connection pooling in multithreaded Java application

I am working on an application that has about 15 threads running the entire time.We recently started using HikariCP for connection pooling.
These threads are restarted every 24 hours. When the threads are restarted, we explicitly close the Hikari datasource by calling dataSource.close() Until before we started to use Connection pooling, One connection object was passed around in the thread to all functions. Now, when the dataSource is closed and if the old connection object was already passed to a method, that returned an error that said dataSource has already been closed which makes sense.
To get around this issue, instead of passing around same connection object in a thread, we started creating them in methods in DBUtils class(Basically functions with queries)
This is how run method of a thread in our application looks like:
#Override
public void run() {
consumer.subscribe(this.topics);
while (!isStopped.get()) {
try {
for (ConsumerRecord<Integer, String> record : records) {
try{
/*some code*/
}catch(JsonProcessingException ex){
ex.printStackTrace();
}
}
DBUtils.Messages(LOGGER.getName(),entryExitList);
} catch (IOException exception) {
this.interrupt();
}
consumer.close();
}
Now, after starting to use HikariCP, instead of passing connection object to DBUtils.Messages, we get a connection from the pool in the method itself
i.e
public static final void Messages(String threadName, List<EntryExit> entryExitMessages) throws SQLException {
Connection connection = DBUtils.getConnection(threadName);
/*code*/
try{
connection.close();
}catch(SQLException se){}
}
This is what getConnection method of DBUtils looks like
public static synchronized Connection getConnection(String threadName) {
Connection connection = null;
try {
if (ds == null || ds.isClosed()) {
config.setJdbcUrl(getProperty("postgres.url"));
config.setUsername(getProperty("postgres.username"));
config.setPassword(getProperty("postgres.password"));
config.setDriverClassName(getProperty("postgres.driver"));
config.setMaximumPoolSize(getProperty("postgres.max-pool-size"));
config.setMetricRegistry(ApplicationUtils.getMetricRegistry());
config.setConnectionTimeout(getProperty("postgres.connection-timeout"));
config.setLeakDetectionThreshold(getProperty("postgres.leak-detection-threshold"));
config.setIdleTimeout(getProperty("postgres.idle-timeout"));
config.setMaxLifetime(getProperty("postgres.max-lifetime"));
config.setValidationTimeout(getProperty("postgres.validation-timeout"));
config.setMinimumIdle(getProperty("postgres.minimum-idle"));
config.setPoolName("PostgresConnectionPool");
ds = new HikariDataSource(config);
}
connection = ds.getConnection();
return connection;
} catch (Exception exception) {
exception.printStackTrace();
}
}
But since call to this method is inside while loop in the thread, PostgresConnectionPool.pool.Wait keeps increasing.
.What's the best way to deal with this?
Edit: PostgresConnection is the pool name . PoolPostgresConnectionPool.pool.Wait is coming from Dropwizard metrics :
https://github.com/brettwooldridge/HikariCP/wiki/Dropwizard-Metrics

Unable to save java object in Redis

I was trying to run the following example in my machine:
https://examples.javacodegeeks.com/enterprise-java/spring/spring-data-redis-example/
public static void main(String[] args) {
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(new ClassPathResource("/spring/spring-config.xml").getPath());
RedisRepo redisRepo = (RedisRepo)context.getBean(“redisRepo");
try{
JedisPool pool = new JedisPool(new JedisPoolConfig(), "x.x.x.x”);
Jedis jedis = pool.getResource();
System.out.println("Connected to Redis”);//connected to Redis
System.out.println("server is running: "+jedis.ping());//PONG
System.out.println("current keys are :"+jedis.keys(“*”));//[ ]
Employee s = new Employee();
s.setId(1);
s.setName(“abc”);
redisRepo.saveState(s);
System.out.println("server is running: "+jedis.ping());//PONG
System.out.println("Finding the One : "+redisRepo.getState(1);//Finding the one :null
}
catch(Exception e){
logger.error(e.getMessage(), e);
}
}
But getting the following error while trying to save the model object:
Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
I would like to add that I can test the connection to redis server by pinging.How can I resolve this issue to persist data in Redis?

I could resolve the issue to save data on local instance by running redis locally.While to save data on remote server I had import resource in the main class.Like:
#ImportResource({classpath:path-to-youXML})

Websphere 7.0.0.17 error with Pool Connection

Im having a problem with an application, I'm using EclipseLink 2.4.1 with oracle 11g, I'm getting a J2CA0045E error, because the pooling does not set free any connections pool from the jndi,
Have you have this problem?, apparently the connection does not get released and it reach the limit I've configured.
Any ideas, this is my Connection pool
Connection timeout: 180
Maximum connections: 10
Minimum connections: 1
Reap time: 60
Unused timeout: 180
Aged timeout: 120
Purge policy: Entire Pool
EDIT
I have a web service in the same service, and the behavior is the same, using same jndi, the connection increase and does not release any connection
initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/abc");
conn = ds.getConnection();

Based on the code provided, it looks like you are storing the Connection reference at a class scope. You should avoid storing references to Connection objects that could span multiple requests.
Instead, store a reference to the DataSource.
To illustrate. This approach is preferred:
public class GoodService {
DataSource ds;
public GoodService() throws NamingException {
// Only the DataSource reference is stored, this is allowed
ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
}
public void doSomething() throws Exception {
// Get the connection in a request.
Connection conn = ds.getConnection();
try {
// do something...
} finally {
// Always close the connection in the finally block
// so it gets returned to the pool no matter what
conn.close();
}
}
}
The following approach is considered bad practice, and is likely to cause the J2CA0045E error you are seeing:
public class BadService {
Connection conn; // BAD: don't store a Connection across requests!
public BadService() throws NamingException, SQLException {
DataSource ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
// BAD: don't leave connections open indefinitely like this
conn = ds.getConnection();
}
public void doSomething() throws Exception {
// do something using conn...
}
}

Single database connection for web application

I am developing a simple CRUD application, using JDBC to establish connection and perform basic CRUD operations. In that process, created a DatabaseListener to create a connection object at startup and storing it in the context attribute for reuse.
Below is the code.
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.log4j.Logger;
public class DatabaseInitListner implements ServletContextListener {
private static final Logger LOG = Logger.getLogger(DatabaseInitListner.class);
private DBUtil databaseUtil = null;
#Override
public void contextDestroyed(ServletContextEvent event) {
databaseUtil.closeConnection();
}
#Override
public void contextInitialized(ServletContextEvent contextinitEvent) {
ServletContext servletContext = contextinitEvent.getServletContext();
String database = servletContext.getInitParameter("db_name");
String url = servletContext.getInitParameter("db_url")
+ database;
String username = servletContext.getInitParameter("db_user");
String password = servletContext.getInitParameter("db_password");
String driverName = servletContext.getInitParameter("db_driver");
databaseUtil = new DBUtil(url, username, password,
driverName);
servletContext.setAttribute("databaseSingleConnectionObject",
databaseUtil.getConnection());
}
}
public class DBUtil {
private Connection connection = null;
private static final Logger LOG = Logger.getLogger(DatabaseUtil.class);
public DatabaseUtil(String url, String username, String password,
String driver) {
try {
Class.forName(driver);
this.connection = DriverManager.getConnection(url, username,
password);
LOG.debug("Connection Established... ");
} catch (ClassNotFoundException | SQLException e) {
LOG.error("Could not create connection... ", e);
}
}
public Connection getConnection() {
return connection;
}
public void closeConnection() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
LOG.error("Unable to close connection... ", e);
}
}
}
}
I am accessing the connection in servlets like this
Connection jdbcConnection = (Connection) getServletContext().getAttribute("databaseSingleConnectionObject");
I am not sure if this is right approach. What are the effects of single database connection?

When you use a single database connection like this you make your application slow and brittle.
Slow: because the connection implementation is synchronized, each user has to wait until the connection is free. If one user's query takes a while to come back that directly increases the time any other concurrent users spend waiting. If there were multiple connections available from a pool then the time spent by one user would not impact other users nearly as greatly (unless a query's results take all the JVM's memory or a big query bogs down the database server).
Brittle: The connection is a network connection, they tend to go down. Without a provision to create new connections any kind of timeout, network hiccup, or period of database non-availability (such as taking the database offline for maintenance) is going to require an application restart. Using a connection pool will mean your application will be able to survive these episodes and recover without outside intervention.

This will not be threadsafe, and if it were, performance would be really poor.
Look into using a Connection Pool, like DBCP or C3PO

You should let your application server manage database connection. Add a JNDI datasource in its configuration file and make a lookup from your application to get a connection when needed (for instance when you instantiate a class that must access your database).
You may configure the datasource to manage a connection pool so that each user session will get its own.
Depending on the AS you use run a search with keywords 'JNDI' and 'datasource' and you will get further details about the AS configuration and how to implement it in your application.

Creating JDBC transaction for second data source with existing connection pool in Glassfish

I have main data source for working with database, which uses some connection pool. Now I want to create a second data source, which would use the same connection pool to perform logging operations in a separate transaction, then main database transaction! As far as I understood from the Glassfish docs if multiple data sources are using the same connection pool, then they share a transaction until connection is not closed (I might be wrong, please correct me).
So, is there a way to start a new transaction, when getting a connection to a data source? By setting TransactionIsolation may be?
Connection is retrieved in the following way:
private synchronized Connection getConnection() {
if (connection == null) {
try {
final Context ctx = new InitialContext();
final DataSource ds = (DataSource) ctx.lookup(getDataSourceLookupAddress());
connection = ds.getConnection();
} catch (final NamingException e) {
errorHandler.error("Datasource JNDI lookup failed: " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
} catch (final SQLException e) {
errorHandler.error("Sql connection failed to " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
}
}
return connection;
}

I have figured it out. It's necessary to start a new Thread, then a new transaction will be created. See also Writing to database log in the middle of rollback