We Keep Coding

HikariCP not closing connections on close() (Connection Leak)

I'm using HikariCP 3.3.1 and PostgreSQL. But I've a problem with closing my connections, in Hikari config I set maximum pool size to 15 and minimum idle connection to 5, but after a few minutes of work with database I've found out connections don't closes, they stack more and more (almost 100 Idle connections right now).
My Connector class:
Connector.java
public class Connector implements IConnector {
private static HikariConfig config = new HikariConfig();
private static HikariDataSource ds;
static {
config.setDriverClassName(org.postgresql.Driver.class.getName());
config.setJdbcUrl("jdbc:postgresql://localhost:5432/vskDB");
config.setUsername("postgres");
config.setPassword("root");
config.setMinimumIdle(5);
config.setMaximumPoolSize(15);
config.setConnectionTimeout(20000);
config.setIdleTimeout(300000);
ds = new HikariDataSource(config);
}
public Connection getConnection() {
log.info("getConnection() invoked");
try {
return ds.getConnection();
} catch (SQLException e) {
log.error("Can't get connection from DataSource.");
log.error(e.getMessage());
System.out.println(e.getMessage());
}
return null;
}
Connector() {
}
}
And here's my DAO class (simplified):
UserDAO.java
public class UserDatabaseDAO implements UserDAO {
private Connector connector = new Connector();
private Connection dbConnection;
#Override
public void removeUser(Long id) {
try {
dbConnection = connector.getConnection();
if (dbConnection == null)
throw new ConnectException();
PreparedStatement preparedStatement = dbConnection.prepareStatement("DELETE FROM users WHERE user_id = ?");
preparedStatement.setLong(1, id);
preparedStatement.execute();
} catch (SQLException | ConnectException e) {
log.error("Can't remove user from database");
log.error(e.getMessage());
System.out.print(e.getMessage());
} finally {
try {
dbConnection.close();
} catch (SQLException e) {
log.error("Can't close connection");
log.error(e.getMessage());
System.out.print(e.getMessage());
}
}
}
}
Here I've found an issue with some facts about Hikari:
You must call close() on the connection instance that HikariCP gives you
Maybe my dbConnection.close() wont work because it's just a copy of Connection which Hikari gives me in getConnection() method.

You forgot to close also PreparedStatement
try {
if (preparedStatement != null) {
preparedStatement.close();
}
if (dbConnection != null) {
dbConnection.close();
}
Releases this Statement object's database and JDBC resources immediately instead of waiting for this to happen when it is automatically closed. It is generally good practice to release resources as soon as you are finished with them to avoid tying up database resources.

JSR 352: Connection is closed Error if connection is closed in the Readers close() in partitioned step

I create a Connection Connection con = ds.getConnection();(where ds is DataSource) in the open of the Reader and close it in the close() of the Reader.
But when i run a job with multiple partitions, in the middle of the job , i get Connection is closed error
Caused by: java.sql.SQLException: [jcc][t4][10335][10366][3.58.82] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003 DSRA0010E: SQL State = 08003, Error Code = -4,470
I assume this happens when one of the partition completes.
So my question is why does this happen? And how should connections be handled? Or does Java take care of closing the connections?
I am using Java Batch on WebSphere Liberty
UPDATE:
<jdbcDriver libraryRef="DB2JCC4Lib"/>
<properties.db2.jcc databaseName="" driverType="4" password="" portNumber="" queryDataSize="65535" serverName="" user=""/>
</dataSource>
public class Reader implements ItemReader {
private DataSource ds = null;
private Connection con = null;
public Reader() {
}
public void close() {
try {
con.close();
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
/**
* #see ItemReader#readItem()
*/
public Object readItem() {
String s="";
try {
if (rs.next()) {
for (int i = 1; i <= 10; i++) {
s+=rs.getString(i);
}
return s;
}
else {
return null;
}
} catch (SQLException e) {
e.printStackTrace();
}
return null;
}
public Serializable checkpointInfo() {
}
public void open(Serializable checkpoint) {
if (ds == null) {
try {
ds = (DataSource) new InitialContext()
.lookup("java:comp/env/jdbc/dataSource");
} catch (Exception e) {
e.printStackTrace();
}
}
try {
con = ds.getConnection();
statement= con
.prepareCall("call abc.xyz(?)");
statement.setString("param", "xxx");
boolean result= statement.execute();
if (result) {
rs = statement.getResultSet();
if (rs == null) {
throw new NullPointerException();
}
} else {
throw new SQLException();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
Complete error message
[ERROR ] J2CA0024E: Method rollback, within transaction branch ID {XidImpl: formatId(57415344), gtrid_length(36), bqual_length(40),
data(0000015645eff4470000000915937ff85f46c3ed056b19010aa5147e1183f8d3ae81c04c0000015645eff4470000000915937ff85f46c3ed056b19010aa5147e1183f8d3ae81c04c00000001)} of resource pool connectionManager[Pool], caught com.ibm.ws.exception.WsException: DSRA0080E: An exception was received by the Data Store Adapter. See original exception message: [jcc][t4][10335][10366][3.58.82] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003. with SQL State : 08003 SQL Code : -4470

I dont't know if JSR-352's Batch handles processing exactly the same way as Spring Batch does but...
In Spring Batch if you have a Reader that uses chunk processing what i think you could do to solve the problem is to put the openConnection() in the beforeRead() and the closeConnection() in the afterRead().
To do that you should implement a Listener. Check these out so you get an idea of what i'm talking about.
Spring Annotation Type BeforeRead
Interface ItemReadListener

DataSource context.lookup after redeploy

after searching for my problem in weblogic and hibernate I found that the problem happens in the jndi lookup, when you install the app, it runs ok but when you redeploy the app this happens :
javax.naming.NameNotFoundException: Unable to resolve 'ds_c719_002'. Resolved ''; remaining name 'ds_c719_002'
My config file (i'm using java config)
#Bean
public DataSource getDatasourceConfiguration() {
System.out.println("empezando a buscar jndi-------------");
try {
Thread.sleep(10000);
} catch (InterruptedException e1) {
}
final JndiDataSourceLookup dsLookup = new JndiDataSourceLookup();
dsLookup.setResourceRef(true);
Hashtable<String, String> h = new Hashtable<String, String>(7);
h.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
InitialContext context=null;
try {
context = new InitialContext(h);
} catch (NamingException e) {
log.error(e);
}
DataSource dataSource;
try {
dataSource = (javax.sql.DataSource) context.lookup("ds_c719_002");
this.ds=dataSource;
context.close();
return dataSource;
} catch (NamingException e) {
log.error(e);
}
return null;
}
Regards

javax.naming.NoInitialContextException: while running JUNIT test cases in webservices

I have JNDI config for all possible database connectivity in my application. Meanwhile I am using JUNIT to test my applications. I find successful connection form my webservices class invocation but error throws while calling it through junit test class.
My JNDI setting is
public static Connection getConnectionPool() {
Connection conn = null;
System.out.println("Creating connection pool.");
try {
Context initContext = new InitialContext();
Context envContext = (Context)initContext.lookup("java:/comp/env");
DataSource ds = (DataSource)envContext.lookup(getJNDIName());
conn = ds.getConnection();
} catch (SQLException e1) {
e1.printStackTrace();
} catch (NamingException e) {
e.printStackTrace();
}
return conn;
}
private static String getJNDIName()
{
if(!readJNDI)
{
try{
Properties prop = new Properties();
prop.load(Thread.currentThread().getContextClassLoader().getResourceAsStream("config.properties"));
JNDIName = prop.getProperty("jndi.name");
System.out.println("JNDIName : "+JNDIName);
readJNDI = true;
}catch(Exception ex){
ex.printStackTrace();
}
}
return JNDIName;
}
Below mentioned is the error.
javax.naming.NoInitialContextException: Need to specify class name in
environment or system property, or as an applet parameter, or in an
application resource file: java.naming.factory.initial
How do I resolve it.?
Thanks in advance.

You should use a local datasource.
Just pull out the DataSource/TransactionManager definition into a separate XML file, and use that instead of the JNDI one during testing.
You can still use all the rest of your configuration.

Where to initialize Connection Pooling, java tomcat

I have read this page: http://www.javaranch.com/journal/200601/JDBCConnectionPooling.html
The approach of Method Scope Connections seems quite good for me.
But i have one Question, when do i init the JDBCServlet class?
Every time i want a connection? Because i thought that everytime i want a connection i just call getConnection()...
public class JDBCServlet extends HttpServlet {
private DataSource datasource;
public void init(ServletConfig config) throws ServletException {
try {
// Look up the JNDI data source only once at init time
Context envCtx = (Context) new InitialContext().lookup("java:comp/env");
datasource = (DataSource) envCtx.lookup("jdbc/MyDataSource");
}
catch (NamingException e) {
e.printStackTrace();
}
}
private Connection getConnection() throws SQLException {
return datasource.getConnection();
}
public void doGet (HttpServletRequest req, HttpServletResponse res) throws ServletException {
Connection connection=null;
try {
connection = getConnection();
..<do JDBC work>..
}
catch (SQLException sqlException) {
sqlException.printStackTrace();
}
finally {
if (connection != null)
try {connection.close();} catch (SQLException e) {}
}
}
}
}

The JDBCServlet servlet is invoked when you navigate to a link where the servlet is mapped to.
So you dont have to do anything apart apart from the mapping from URL to servlet which is done in the web.xml.
the web container then create the instance of the servlet by using init method and then call doGet.
This is a PDF for a servlet tutorial with tomcat, but the basics are the same.
http://www.tutorialspoint.com/servlets/servlets_tutorial.pdf
look at the servlet deployment section
for the DBUtil Class this is a good example.
public class DBUtil {
private static DataSource dataSource;
static {
try {
dataSource = new InitialContext().lookup("jdbc/MyDataSource");
} catch (NamingException e) {
throw new ExceptionInInitializerError("'jdbc/MyDataSource' not found in JNDI", e);
}
}
public static Connection getConnection() {
return dataSource.getConnection();
}
}
This gives your class that can be used by all servlets in your Web Application.
you wil call the DBUtil class by
try {
connection = DBUtil.getConnection();
statement = connection.prepareStatement("SELECT id, foo, bar FROM table");
resultSet = statement.executeQuery();
//Do what you need to do.
} finally {
if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
if (statement != null) try { statement.close(); } catch (SQLException ignore) {}
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}

Servlets are initialised by the Container which will call init as required and defined by the web.xml configuration file.
Use the Apache Tomcat JDBC Connection pools which does the same in a standard way.

There's many connection pooling libraries including MySQL's own. Many people use C3P0 which is very mature.
The common idea is that you define the Datasource in the server container and access the JNDI reference from your code. During the servlet init you're just looking up the datasource so this is the ideal place to do this. This will not use any connections until you actually perform an action.
See Tomcat DBCP for a good intro and if you wish to use C3P0 C3P0 Tomcat Configuration