I'm using HikariCP 3.3.1 and PostgreSQL. But I've a problem with closing my connections, in Hikari config I set maximum pool size to 15 and minimum idle connection to 5, but after a few minutes of work with database I've found out connections don't closes, they stack more and more (almost 100 Idle connections right now).
My Connector class:
Connector.java
public class Connector implements IConnector {
private static HikariConfig config = new HikariConfig();
private static HikariDataSource ds;
static {
config.setDriverClassName(org.postgresql.Driver.class.getName());
config.setJdbcUrl("jdbc:postgresql://localhost:5432/vskDB");
config.setUsername("postgres");
config.setPassword("root");
config.setMinimumIdle(5);
config.setMaximumPoolSize(15);
config.setConnectionTimeout(20000);
config.setIdleTimeout(300000);
ds = new HikariDataSource(config);
}
public Connection getConnection() {
log.info("getConnection() invoked");
try {
return ds.getConnection();
} catch (SQLException e) {
log.error("Can't get connection from DataSource.");
log.error(e.getMessage());
System.out.println(e.getMessage());
}
return null;
}
Connector() {
}
}
And here's my DAO class (simplified):
UserDAO.java
public class UserDatabaseDAO implements UserDAO {
private Connector connector = new Connector();
private Connection dbConnection;
#Override
public void removeUser(Long id) {
try {
dbConnection = connector.getConnection();
if (dbConnection == null)
throw new ConnectException();
PreparedStatement preparedStatement = dbConnection.prepareStatement("DELETE FROM users WHERE user_id = ?");
preparedStatement.setLong(1, id);
preparedStatement.execute();
} catch (SQLException | ConnectException e) {
log.error("Can't remove user from database");
log.error(e.getMessage());
System.out.print(e.getMessage());
} finally {
try {
dbConnection.close();
} catch (SQLException e) {
log.error("Can't close connection");
log.error(e.getMessage());
System.out.print(e.getMessage());
}
}
}
}
Here I've found an issue with some facts about Hikari:
You must call close() on the connection instance that HikariCP gives you
Maybe my dbConnection.close() wont work because it's just a copy of Connection which Hikari gives me in getConnection() method.
You forgot to close also PreparedStatement
try {
if (preparedStatement != null) {
preparedStatement.close();
}
if (dbConnection != null) {
dbConnection.close();
}
Releases this Statement object's database and JDBC resources immediately instead of waiting for this to happen when it is automatically closed. It is generally good practice to release resources as soon as you are finished with them to avoid tying up database resources.
Related
I'm playing with this kind of database, and I've tried to close the HSQLDB connection after I used it, but it's still opened at the end.
Code:
//----This methods are in a specific connection class file
public static Connection conn = null;
public static Connection getConnection(){
try {
input = new FileInputStream("PathToMyPropertiesFile");
prop.load(input);
//The properties constants are correctly checked
Class.forName(prop.getProperty("DRIVER_HSQLDB"));
conn = DriverManager.getConnection(prop.getProperty("CONN_HSQLDB"));
}
catch(ClassNotFoundException | SQLException e) {
LOG.log(null,"Error: "+e);
}
catch (IOException ex) {
LOG.log(null,"FILE ERROR: "+ex);
}
finally {
if (input != null) {
try {
input.close();
} catch (Exception e) {
LOG.log(null,"CLOSE ERROR: "+e);
}
}
}
return conn;
}
public static boolean stopConn() {
try {
if(conn != null) {
conn.close();
System.err.println("\nCLOSE CONN\n"+conn);
return true;
}
}
catch (SQLException e) {
e.printStackTrace();
return false;
}
return false;
}
//========= the other class file with the methods to use the conneciton
public static boolean insertUser(String uName, String uEmail){
Connection con;
con = ConnectionDB.getConnection();
PreparedStatement ps = null;
try {
String consulta = "insert into USERS (\"NICK\",\"EMAIL\") VALUES(?,?);";
ps = con.prepareStatement(consulta);
System.err.println(ps);
ps.setString(1,uName);
ps.setString(2,uEmail);
System.err.println("\nASSIGNATION\n"+ps);
if(ps.executeUpdate() == 1) {
System.err.println("\nTRUE\n");
return true;
}
}
catch(SQLException e) {
e.printStackTrace();
}
finally {
try {
System.err.println("\nFINALLY\n"+ps);
if(ps != null) {
ps.close();
System.err.println("\nCLOSE PS\n"+ps);
}
if(con != null) {
con.close();
System.err.println("\nCLOSE CON\n"+con);
if(ConnectionDB.stopConn()) {
System.err.println("\nALL IS OK\n"+ConnectionDB.conn);
}
else {
System.err.println("\nMEEEEKKKK!!!\n"+ConnectionDB.conn);
}
}
}
}
return false;
}
The console give me this results, and I don't know why never the connection is closed because I tried to close it twice. If someone has an idea please tell me.
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[null], [null]]]
ASSIGNATION
org.hsqThis is my cldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[extra], [extra#mail.com]]]
TRUE
FINALLY
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[sql=[insert into USERS ("NICK","EMAIL") VALUES(?,?);], parameters=[[extra], [extra#mail.com]]]
CLOSE PS
org.hsqldb.jdbc.JDBCPreparedStatement#4501280b[closed]
CLOSE CON
org.hsqldb.jdbc.JDBCConnection#3e5b87f5
CLOSE CONN
org.hsqldb.jdbc.JDBCConnection#3e5b87f5
ALL IS OK
org.hsqldb.jdbc.JDBCConnection#3e5b87f5
Closing a JDBC connections does not close an in-process database. This allows you to open and close different connections during the runtime of your application.
You need to execute a JDBC Statement to shutdown the database. The SQL statement to execute is "SHUTDOWN".
It is possible to add a connection property "shutdown=true" to the JDBC connection URL to force a quick shutdown when the last connection to the in-process database is closed. But this is mainly useful for readonly or test databases. A full SHUTDOWN allows the database to open quickly the next time a connection is made.
See the Guide http://hsqldb.org/doc/2.0/guide/running-chapt.html#rgc_inprocess
I have requirement where I need to read data from multiple rdbms tables which are present different databases. I have a regular connection class which returns a connection from its getConnection() method as below.
public class DbManager {
private static Connection connection;
public static Connection getConnection() {
try {
Class.forName("com.mysql.jdbc.Driver");
if(connection == null) {
connection = DriverManager.getConnection("jdbc:mysql://localhost:portnumber", "username", "password");
return connection;
} else {
return connection;
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
return null;
} catch (SQLException e) {
e.printStackTrace();
return null;
}
}
private DbManager() {
}
}
But this works only for one database. I have multiple databases in MySql, Oracle, Postgres and I have to read all the tables in all of those databases.
Is there a way to create connections for multiple databases using the same class or is there a different way to implement that ?
I create a Connection Connection con = ds.getConnection();(where ds is DataSource) in the open of the Reader and close it in the close() of the Reader.
But when i run a job with multiple partitions, in the middle of the job , i get Connection is closed error
Caused by: java.sql.SQLException: [jcc][t4][10335][10366][3.58.82] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003 DSRA0010E: SQL State = 08003, Error Code = -4,470
I assume this happens when one of the partition completes.
So my question is why does this happen? And how should connections be handled? Or does Java take care of closing the connections?
I am using Java Batch on WebSphere Liberty
UPDATE:
<jdbcDriver libraryRef="DB2JCC4Lib"/>
<properties.db2.jcc databaseName="" driverType="4" password="" portNumber="" queryDataSize="65535" serverName="" user=""/>
</dataSource>
public class Reader implements ItemReader {
private DataSource ds = null;
private Connection con = null;
public Reader() {
}
public void close() {
try {
con.close();
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
/**
* #see ItemReader#readItem()
*/
public Object readItem() {
String s="";
try {
if (rs.next()) {
for (int i = 1; i <= 10; i++) {
s+=rs.getString(i);
}
return s;
}
else {
return null;
}
} catch (SQLException e) {
e.printStackTrace();
}
return null;
}
public Serializable checkpointInfo() {
}
public void open(Serializable checkpoint) {
if (ds == null) {
try {
ds = (DataSource) new InitialContext()
.lookup("java:comp/env/jdbc/dataSource");
} catch (Exception e) {
e.printStackTrace();
}
}
try {
con = ds.getConnection();
statement= con
.prepareCall("call abc.xyz(?)");
statement.setString("param", "xxx");
boolean result= statement.execute();
if (result) {
rs = statement.getResultSet();
if (rs == null) {
throw new NullPointerException();
}
} else {
throw new SQLException();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
Complete error message
[ERROR ] J2CA0024E: Method rollback, within transaction branch ID {XidImpl: formatId(57415344), gtrid_length(36), bqual_length(40),
data(0000015645eff4470000000915937ff85f46c3ed056b19010aa5147e1183f8d3ae81c04c0000015645eff4470000000915937ff85f46c3ed056b19010aa5147e1183f8d3ae81c04c00000001)} of resource pool connectionManager[Pool], caught com.ibm.ws.exception.WsException: DSRA0080E: An exception was received by the Data Store Adapter. See original exception message: [jcc][t4][10335][10366][3.58.82] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003. with SQL State : 08003 SQL Code : -4470
I dont't know if JSR-352's Batch handles processing exactly the same way as Spring Batch does but...
In Spring Batch if you have a Reader that uses chunk processing what i think you could do to solve the problem is to put the openConnection() in the beforeRead() and the closeConnection() in the afterRead().
To do that you should implement a Listener. Check these out so you get an idea of what i'm talking about.
Spring Annotation Type BeforeRead
Interface ItemReadListener
I have a Java program in which I am doing some JDBC for select queries. Will it be advisable to call testDataBase() each time which inturns calls DBConnection() each time or I should reuse one connection for all the queries. Thanks in advance.
private void testDataBase(String query){
Connection con = DBConnection();
Statement st = null;
ResultSet rs = null;
try {
st = con.createStatement();
rs = st.executeQuery(query);
boolean flag = true;
while (rs.next()) {
String resultString = "";
for(int i = 1; i <=rs.getMetaData().getColumnCount();i++){
resultString=resultString+" "+ rs.getString(i);
}
System.out.println(resultString);
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
if (st != null) {
try {
st.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (con != null) {
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
private Connection DBConnection() {
final String method_name = "DBConnection";
Connection conn = null;
try{
Class.forName(driver).newInstance();
conn = java.sql.DriverManager.getConnection(url,userName,password);
}catch (ClassNotFoundException e) {
System.out.println(e.getMessage());
} catch (SQLException e) {
System.out.println(e.getMessage());
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return conn;
}
Opening a DB connection is an expensive operation in terms of perfofmance. You should use a ConnectionPool for sharing connections among different requests.
Connections are not thread safe, so sharing them across requests is not a good idea.
A better idea is to pool connections and keep their scope as narrow as possible: check the connection out of the pool, use it, close it in transaction scope.
Database connections are long-running and should be re-used, unless you have a very low query rate.
Getting a database connection is quite an expensive operation, so it is advisable to re-use a connection if possible. Consider also using connection pooling, which will maintain a number of connections for you, so you can just grab one from the pool when needed. The method shown above might not need to change, it depends on the DBConnection() method you call.
I completely agree with #Amir Kost, in terms of performances, opening a DB connection in one of the slowest operation that you can do, and if you have restrictive real time constraints it could be a big issue.
I do not know if you are using a framework or not, but a good practice is to publish a bean which wrap a pool of connection and every time that you need to interact directly with the db, you get the current open connection (which usually corresponds to a so called "session").
I suggest to you, (even if you are not using any framework) to reproduce this technicality.
If you want only one instance of Connection, you can make use of the Singleton pattern, you can consider :
public class Connector {
private static final String URL = "jdbc:mysql://localhost/";
private static final String LOGIN = "root";
private static final String PASSWORD = "azerty";
private static final String DBNAME = "videotheque";
private static Connector connector;
private static Connection connection;
private Connector() {
}
public synchronized static Connector getInstance() {
if (connector == null) {
connector = new Connector();
}
return connector;
}
public static Connection getConnection() {
if (connection == null) {
Connection c = null;
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
c = DriverManager.getConnection(URL + DBNAME, LOGIN, PASSWORD);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return c;
}
return connection;
}
}
And then, you can call : Connector.getInstance().getConnection()
In my Java application's DAO layer I have two DAO classes EmployeeDAO and BankDAO. I need to control/handle their database transactions. I use connection pooling to get database connections.
EmployeeDAO class:
public class EmployeeDAO {
String name;
String empCode;
int age;
// Getters & Setters
}
BankDAO class:
public class BankDAO {
String bankName;
String acNo;
String empCode;
// Getters & Setters
}
Let's say I am going to store an Employee and Bank account details related to that employee in two database tables. First I save employee and second I save bank details and if an error occurs when storing bank details I need to rollback complete transaction.
How to manage this sort of transaction while using DAOs?
If you are using plain JDBC, what you could do is share the same instance of Connection in the two instances of the DAO classes.
public class EmployeeDAO {
private Connection conn;
public void setConnection(Connection conn) {
this.conn = conn;
}
...
}
public class BankDAO {
private Connection conn;
public void setConnection(Connection conn) {
this.conn = conn;
}
...
}
In the client code, first you need to create a Connection object instance. Next, you need start the transaction, with conn.setAutoCommit(false);. Pass the Connection object instance to the both DAO classes. If no errors occurs in any operation, conn.commit();, otherwise, conn.rollback();
e.g.:
Connection conn = null;
try {
// getConnection from pool
conn.setAutoCommit(false);
EmployeeDAO employeeDAO = new EmployeeDAO();
employeeDAO.setConnection(conn);
BankDAO bankDAO = new BankDAO();
bankDAO.setConnection(conn);
// save employee
// save bank details
conn.commit();
catch(Exception e) {
if (conn != null) {
conn.rollback();
}
} finally {
if (conn != null) {
conn.close();
}
}
When you open a connection from the database, you can start a new transaction using the method [Connection#setAutoCommit][1](false), do all your insert/update/delete operations and execute commit to save all these changes, in case of an error you can rollback all the actions or to a savepoint. Here is an exampleof what I'm saying:
public void saveSomeData(DAOClass daoObject) {
Connection con = null;
try {
con = getConnectionFromDBPool(); //get the connection from the connection pool
con.setAutoCommit(false);
//start your transaction
PreparedStatement ps = con.prepareCall("insert into tablex values(?, ?)");
ps.setInt(1, daoObject.getAttribute1());
ps.setString(2, daoObject.getAttribute2());
ps.execute();
//add another insert/update/delete operations...
//at the end, you commit the transaction
con.commit();
} catch (Exception e) {
//start a rollback
if (con != null) {
try {
con.rollback();
} catch (Exception ex) {
}
}
//handle the exception
e.printStackTrace();
} finally {
if (con != null) {
try {
con.close();
} catch (Exception e) {
}
}
}
}
Another hint: you should close all the resources manually before closing the connection. This code is just explanatory, but I have should close the prepared statement after using it.
More info about handling transactions:
Using Transactions