We Keep Coding

SpringBoot how to set connection fetch size for Hikari Pool and jdbcTemplate

I'm looking for a way to
configure Hikari Connection Pool for Spring DataSource
explicitly set fetchSize for resultSet.
Here is my application.properties
# Datasource
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.url=jdbc:postgresql://mydb
spring.datasource.username=user
spring.datasource.password=pass
spring.datasource.driverClassName=org.postgresql.Driver
#Hikari
spring.datasource.hikari.connectionTimeout=30000
spring.datasource.hikari.idleTimeout=600000
spring.datasource.hikari.maxLifetime=1800000
spring.datasource.hikari.maximum-pool-size=100
spring.datasource.hikari.auto-commit=true
here is my service code
#Service
#RequiredArgsConstructor
public class PerfService {
private final JdbcTemplate template;
public RowCountResponse getData(String param) {
Connection connection = null;
PreparedStatement preparedStatement = null;
try {
connection = Objects.requireNonNull(template.getDataSource()).getConnection();
preparedStatement = connection.prepareStatement("SELECT whatever");
preparedStatement.setFetchSize(30000); // !!!
preparedStatement.setString(1, param);
ResultSet resultSet = preparedStatement.executeQuery();
int rowCount = 0;
while (resultSet.next()) {
rowCount++;
}
resultSet.close();
return new RowCountResponse()
.setRowCount(rowCount);
} catch (Exception e) {
throw new RuntimeException("Error while fetching rows", e);
} finally {
JdbcUtils.closeStatement(preparedStatement);
DataSourceUtils.releaseConnection(connection, template.getDataSource());
}
}
}
Seems like my pool is leaking. I don't return connection properly to the pool.
Is there more elegant way to use jdbcTemplate and preparedStatement.setFetchSize(30000);

Sorry for being late, But may help others.
There is an issue with postgresql when setting fetchSize - https://github.com/spring-projects/spring-framework/issues/24848
In order to work you also have to set autoCommit=false for your connection so in your example it would be:
spring.datasource.hikari.auto-commit=false
But setting auto-commit to false for your connection pool could be inconvenient so one workaround is to have two connection pools - one readonly with autoCommit=false and second(ordinary) with autoCommit=true.
Also JdbcTemplate have setFetchSize method so you could simply do
template.setFetchSize(30000);

You can use the connection parameter defaultRowFetchSize, see the documentation.

Apache dbcp connection pooling not working properly

I am trying to connect to mysql database using apache dbcp2 connection pooling following its tutorial from here:
https://git-wip-us.apache.org/repos/asf?p=commons-dbcp.git;a=blob;f=doc/PoolingDataSourceExample.java;h=2a12c74898930b9623223db1597b8a8052a6f1df;hb=HEAD
My Database connection class that return the connection looks something like this:
public class DbConnection {
private static interface Singleton {
final DbConnection INSTANCE = new DbConnection();
}
private final PoolingDataSource<PoolableConnection> dataSource;
private DbConnection() {
// A ConnectionFactory that the pool will use to create Connections.
DriverManagerConnectionFactory connectionFactory = new DriverManagerConnectionFactory(url, user, pass); //url,user,pass for db connnection
//implement the pooling functionality.
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory, null);
GenericObjectPoolConfig config = new GenericObjectPoolConfig();
config.setMaxWaitMillis(500);
config.setMaxTotal(20);
config.setMaxIdle(5);
config.setMinIdle(5);
//PoolingDataSource expect an ObjectPool
ObjectPool<PoolableConnection> connectionPool = new GenericObjectPool<>(poolableConnectionFactory, config);
// Set the poolableConnectionFactory's pool property to the owning pool
poolableConnectionFactory.setPool(connectionPool);
this.dataSource = new PoolingDataSource<>(connectionPool);
}
public static Connection getCon() throws SQLException {
return Singleton.INSTANCE.dataSource.getConnection();
}
}
I am using apache jmeter to test the connection and return something from my mysql database. I created 100 users with Ramp-up periods(in seconds) being 2 second. I created a Http request and when I tried to see my response in view results tree I successfully get response for first 20 request. Later requests from (21 to 100) have blank response. I have gone through many of the issues involving:
java.sql.SQLException: Cannot get a connection, pool error Timeout
waiting for idle object
I access my code as:
try (PreparedStatement ps = DbConnection.getCon().prepareStatement("SELECT id FROM test WHERE id =?;")) {
ps.setString(1, id);
try (
ResultSet rs = ps.executeQuery()) {
return rs.next();
}
} catch (Exception ex) {
}

You aren't closing Connection object, you must declare such variable in try-with-resources block :
try (Connection conn = DbConnection.getCon();
PreparedStatement ps = conn.prepareStatement("SELECT id FROM test WHERE id =?;")) {
Also you need to close ResultSet, either call close method ResultSet in method or add it to try-with-resources block using a new method returning it

create another connection in transaction mysql

I want know if it is possibile create two connection different connections in one transaction, So suppose to have this code( I do a create operation in table "employe" and I create a log in table "log"):
try {
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
PreparateStatement p1 //Preparate statement to put the employe parameter
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
p1.execute();
//This create another connect
LOGStatic.createLog("CREATE EMPLOYE");
connessione_db.commit();
connessione_db.setAutoCommit(true);
}catch(....){
connection_db.rollback();
}
This is the method LOGStatic.createLog("CREATE EMPLOYE");
public static void creaLogException(String messaggio) {
Connection connection_db= null;
InitialContext initialContext;
try {
initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
connection_db= datasource.getConnection();
// the code continues with the save operation
} catch (NamingException n) {
}
// actual jndi name is "jdbc/postgres"
catch (SQLException s) {
}
My question if is it possible this behavior and there aren't problem with commit and rollback operatio or it is not possible to have another connection?
Anyone can help

At least in JDBC, the transaction is tightly coupled with the Connection.
In a Java EE server, if you are writing session beans, then the transaction will be managed my the server. So, in that case, you could call several methods, and the transaction will follow the method calls.
In JDBC the simple solution is to not close the connection, but to pass it around to different methods as an input parameter.
But, be very careful, not closing connections is almost a sure shot way of getting to OutOfMemoryError.
(Instead of passing around the connection as input parameters to various methods, you could use ThreadLocal. But that is another source of memory leak, if you don't clean up ThreadLocal variables.)
For more information on how transaction-lifecycle and Connection are tied in JDBC, refer this: How to start a transaction in JDBC?
Note: Even in JavaEE nested transaction is not possible as nested transactions is not supported in JTA.
Passing the connection around:
public static void createEmployee(){
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
try {
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
PreparateStatement p1 //Preparate statement to put the employe parameter
p1.execute();
//This create another connect
createLog("CREATE EMPLOYE", connessione_db);
connessione_db.commit();
//connessione_db.setAutoCommit(true); //No need
}catch(....){
try{ connection_db.rollback(); }catch(Exception e){ /*You can also check some flags to avoid exception*/ }
}finally{
try{ connection_db.close(); }catch(Exception e){ /*Safe to ignore*/ }
}
}
public static void createLog(String messaggio, Connection connection_db) {
try {
// the code continues with the save operation
} catch (SQLException s) {
}
}

Can not connect to Glassfish connection pool

I can not connect from java to Glassfish connection pool.
There is exception - javax.naming.NoInitialContextException
In Glassfish admin:
1. I created JDBC Connection Pool, named it ‘PaymentSys’, and I can ping it.
2. I created JDBC Resource, named it ‘jdbc/paymentSys’ and link it to connection pool ‘PaymentSys’.
In java code I wrote:
public Connection getConnection(){
Connection conn=null;
String dataSourceContext = "java:comp/env/jdbc/paymentSys";
try{
InitialContext context = new InitialContext();
DataSource ds = (DataSource) context.lookup(dataSourceContext);
conn = ds.getConnection();
}
catch (Exception ex) {
ex.printStackTrace();
}
return conn;
}
I tried in dataSourceContext use 'jdbc/paymentSys', but it was not useful.

Problem was in insert sql command.

Creating JDBC transaction for second data source with existing connection pool in Glassfish

I have main data source for working with database, which uses some connection pool. Now I want to create a second data source, which would use the same connection pool to perform logging operations in a separate transaction, then main database transaction! As far as I understood from the Glassfish docs if multiple data sources are using the same connection pool, then they share a transaction until connection is not closed (I might be wrong, please correct me).
So, is there a way to start a new transaction, when getting a connection to a data source? By setting TransactionIsolation may be?
Connection is retrieved in the following way:
private synchronized Connection getConnection() {
if (connection == null) {
try {
final Context ctx = new InitialContext();
final DataSource ds = (DataSource) ctx.lookup(getDataSourceLookupAddress());
connection = ds.getConnection();
} catch (final NamingException e) {
errorHandler.error("Datasource JNDI lookup failed: " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
} catch (final SQLException e) {
errorHandler.error("Sql connection failed to " + dataSourceLookupAddress + "!");
errorHandler.error(e.toString());
}
}
return connection;
}

I have figured it out. It's necessary to start a new Thread, then a new transaction will be created. See also Writing to database log in the middle of rollback