We Keep Coding

Java ComboPooledDataSource exceeds pool size and does not reuse

I have the following code that gets called by an other application (That I can not change) to read form a Database.
The method is called in a loop very often and DOSes the DB.
At The DB I can see that there are many connecetions opend ... increasing to some hundred ... and than the DB crashed due to the load.
// Called in a loop
private <T> T execute(String query, PostProcessor<T> postProc, PreProcessor preProcs) throws OperationFailedException {
try (Connection conn
= Objects.requireNonNull(dataSourceRef.get(), "No Connection").getConnection();
PreparedStatement smt = conn.prepareStatement(query)) {
preProc.process(smt);
return postProc.process(smt.executeQuery());
} catch (SQLException e) {
throw new OperationFailedException(e.getMessage(), e);
}
}
the date source gets initialised before ...
// class variable
// AtomicReference<PooledDataSource> dataSourceRef = new AtomicReference<PooledDataSource>();
// Init method
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass(config.getConnectionDriver());
cpds.setJdbcUrl(config.getConnectionString());
cpds.setUser(config.getConnectionUser());
cpds.setPassword(config.getConnectionPassword());
// cpds.setMaxPoolSize(10);
dataSourceRef.getAndSet(cpds);
My question why is this happening.
I thought due to the pooling not for each query a new connections should be used.
As well by setting the max pool size this is not working.
As well I tried it with try-catch-finally construct and closing the stm and the conn after use.
(As I've read somewhere that finally might get called delayed under high load scenarios ... I thouht that might be the case)
But still why is the pool size exceeded?
How can I limit that and block the method untill a connection is avalible again before continuing?

During connection polling in c3p0 you have to consider some options. Those are given bellow application.property file:
db.driver: oracle.jdbc.driver.OracleDriver // for Oracle
db.username: YOUR_USER_NAME
db.password: YOUR_USER_PASSWORD
db.url: DATABASE_URL
minPoolSize:5 // number of minimum poolSize
maxPoolSize:100 // number of maximum poolSize
maxIdleTime:5 // In seconds. After that time it will realease the unused connection.
maxStatements:1000
maxStatementsPerConnection:100
maxIdleTimeExcessConnections:10000
Here, maxIdleTime is the main points. It defines how many seconds this will release unused connection. It is in second.
Another is minPoolSize . It defines how many connection it will hold during idle mode.
Another is maxPoolSize . It defines how many maximum connection it will hold during loaded mode.
Now, how you configure ComboPooledDataSource? Here is the code:
#Bean
public ComboPooledDataSource dataSource(){
ComboPooledDataSource dataSource = new ComboPooledDataSource();
try {
dataSource.setDriverClass(env.getProperty("db.driver"));
dataSource.setJdbcUrl(env.getProperty("db.url"));
dataSource.setUser(env.getProperty("db.username"));
dataSource.setPassword(env.getProperty("db.password"));
dataSource.setMinPoolSize(Integer.parseInt(env.getProperty("minPoolSize")));
dataSource.setMaxPoolSize(Integer.parseInt(env.getProperty("maxPoolSize")));
dataSource.setMaxIdleTime(Integer.parseInt(env.getProperty("maxIdleTime")));
dataSource.setMaxStatements(Integer.parseInt(env.getProperty("maxStatements")));
dataSource.setMaxStatementsPerConnection(Integer.parseInt(env.getProperty("maxStatementsPerConnection")));
dataSource.setMaxIdleTimeExcessConnections(10000);
} catch (PropertyVetoException e) {
e.printStackTrace();
}
return dataSource;
}
For details implementation, please check this thread . Here i added practical implementation
Edit (Getting connection)
You can get connection using bellow way:
Session session = entityManager.unwrap(Session.class);
session.doWork(connection -> doSomeStuffWith(connection));
How you get EntityManager?
#PersistenceContext
private EntityManager entityManager;
Hope this will help you.
Thanks :)

create another connection in transaction mysql

I want know if it is possibile create two connection different connections in one transaction, So suppose to have this code( I do a create operation in table "employe" and I create a log in table "log"):
try {
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
PreparateStatement p1 //Preparate statement to put the employe parameter
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
p1.execute();
//This create another connect
LOGStatic.createLog("CREATE EMPLOYE");
connessione_db.commit();
connessione_db.setAutoCommit(true);
}catch(....){
connection_db.rollback();
}
This is the method LOGStatic.createLog("CREATE EMPLOYE");
public static void creaLogException(String messaggio) {
Connection connection_db= null;
InitialContext initialContext;
try {
initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
connection_db= datasource.getConnection();
// the code continues with the save operation
} catch (NamingException n) {
}
// actual jndi name is "jdbc/postgres"
catch (SQLException s) {
}
My question if is it possible this behavior and there aren't problem with commit and rollback operatio or it is not possible to have another connection?
Anyone can help

At least in JDBC, the transaction is tightly coupled with the Connection.
In a Java EE server, if you are writing session beans, then the transaction will be managed my the server. So, in that case, you could call several methods, and the transaction will follow the method calls.
In JDBC the simple solution is to not close the connection, but to pass it around to different methods as an input parameter.
But, be very careful, not closing connections is almost a sure shot way of getting to OutOfMemoryError.
(Instead of passing around the connection as input parameters to various methods, you could use ThreadLocal. But that is another source of memory leak, if you don't clean up ThreadLocal variables.)
For more information on how transaction-lifecycle and Connection are tied in JDBC, refer this: How to start a transaction in JDBC?
Note: Even in JavaEE nested transaction is not possible as nested transactions is not supported in JTA.
Passing the connection around:
public static void createEmployee(){
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
try {
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
PreparateStatement p1 //Preparate statement to put the employe parameter
p1.execute();
//This create another connect
createLog("CREATE EMPLOYE", connessione_db);
connessione_db.commit();
//connessione_db.setAutoCommit(true); //No need
}catch(....){
try{ connection_db.rollback(); }catch(Exception e){ /*You can also check some flags to avoid exception*/ }
}finally{
try{ connection_db.close(); }catch(Exception e){ /*Safe to ignore*/ }
}
}
public static void createLog(String messaggio, Connection connection_db) {
try {
// the code continues with the save operation
} catch (SQLException s) {
}
}

Mysql Stored Procedure per connection or per transaction for thread safe

First let me describe the problem I am facing.
We are developing a web service using Jersey and MySQL at back-end. It's simple that user can view or edit data via Restful API, and the service will load/save data to or from MySQL database.
I created several stored procedure at MySQL server, like SelectAnswer and UpdateAnswer. In the program, we use standard JDBC and MySQL connector to connect to the database.
Firstly I created stored procedure per connection. Like each time the program start a new connection to the database, it creates a bunch of Stored Procedure.
public enum JDBCDataSource {
INSTANCE;
private BasicDataSource dataSource = new BasicDataSource();
private Connection conn;
private CallableStatement answerUpsert;
private JDBCDataSource(){
initConnection();
try{
conn = dataSource.getConnection();
answerUpsert = prepareSP(answerUpsert,"{call upsert_answer(?, ?, ?)}");
} catch (SQLException e) {
e.printStackTrace();
}
}
And the program reuse the stored procedure for each time calling:
private void executeUpsert(String app, String id, String content)
throws SQLException{
try {
CallableStatement callableStatement = JDBCDataSource.INSTANCE.getUpsert();
callableStatement.setString(1,app);
callableStatement.setInt(2,Integer.valueOf(id));
callableStatement.setString(3,content);
callableStatement.execute();
} catch (NumberFormatException e) {
e.printStackTrace();
} finally {
callableStatement.clearParameters();
}
}
So each time calling the procedure, the function set its own parameters, and execute the SP, and finally clear the parameters.
But it is not thread safe, if the user post two request and one request is trying to set parameters, and another to clear parameters, it will cause an exception.
So for stored procedure in MySQL, should I create it per transaction or per connection in order to keep it thread safe? Maybe my understanding of SP and MySQL is not correct, or maybe it is design problem. Please share your though for my question.

Creating JDBC transaction for second data source with existing connection pool in Glassfish

I have main data source for working with database, which uses some connection pool. Now I want to create a second data source, which would use the same connection pool to perform logging operations in a separate transaction, then main database transaction! As far as I understood from the Glassfish docs if multiple data sources are using the same connection pool, then they share a transaction until connection is not closed (I might be wrong, please correct me).
So, is there a way to start a new transaction, when getting a connection to a data source? By setting TransactionIsolation may be?
Connection is retrieved in the following way:
private synchronized Connection getConnection() {
if (connection == null) {
try {
final Context ctx = new InitialContext();
final DataSource ds = (DataSource) ctx.lookup(getDataSourceLookupAddress());
connection = ds.getConnection();
} catch (final NamingException e) {
errorHandler.error("Datasource JNDI lookup failed: " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
} catch (final SQLException e) {
errorHandler.error("Sql connection failed to " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
}
}
return connection;
}

I have figured it out. It's necessary to start a new Thread, then a new transaction will be created. See also Writing to database log in the middle of rollback

Hibernate get Connection object for JasperRunManager

I'm using Hibernate 3.2.5 in a project and Jasper 3.7.2 to generate Some Reports for the application.
Is there any way to get a Connection Object from a HibernateSessionFactory Object? I wanna handle only one connection, because at this time I have to have a properties file for the ODBC connection that is gonna handle the Jasper through the JasperRunManager statics methods, and also Hibernate for other side.
This is the method I need to pass the connection:
byte[] net.sf.jasperreports.engine.JasperRunManager.runReportToPdf(InputStream inputStream, Map parameters, Connection conn) throws JRException
Thanks in advance. :)

I had the same problem with hibernate 4
and here is solution
Session ses = ...
final InputStream finalCompiledReportStream = compiledReportStream;
final OutputStream finalByteArrayOutputStream = byteArrayOutputStream;
ses.doWork(new Work() {
public void execute(Connection connection) throws SQLException {
try {
JasperFillManager.fillReportToStream(finalCompiledReportStream, finalByteArrayOutputStream, parameters, connection);
} catch (JRException e) {
ReportAction.this.logger.error(e);
}
}
});

This works awesomely and you can use the connection object elsewhere where you may need it in the code. doWork may need you to keep doing that many times but creating a connection object nice is a cleaner solution.
SessionImpl sessionImpl = (SessionImpl) session;
Connection conn = sessionImpl.connection();
Where session is the name of the Hibernate Session object