Datasource pool connection not closed in Wildfly 20 - java

I have a Wildfly Java application running with a MariaDB database. Initially the connection works fine, but after 20 connections (the default) the next time it tries to connect the server hangs and after around one minute it throws the following exception:
javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/IndustryDS
This is how I connect and close the datasource:
private InitialContext context = null;
private DataSource ds = null;
private Connection conn = null;
try {
context = new InitialContext();
ds = (DataSource)context.lookup(pool);
conn = ds.getConnection(); // <--- here is where it hangs
// use the connection
if (conn != null)
conn.close();
if (context != null)
context.close();
}
catch (NamingException e) {
logger.error(e.getMessage());
throw new DAOException(e.getMessage());
}
catch (SQLException e) {
logger.error(e.getMessage());
throw new DAOException(e.getMessage()); // <--- this error is thrown
}
The datasource configuration in standalone.xml
<datasource jta="true" jndi-name="java:jboss/datasources/IndustryDS"
pool-name="IndustryDS" enabled="true" use-java-context="true">
<connection-url>jdbc:mariadb://localhost:3306/industry</connection-url>
<driver>mariadb</driver>
<security>
<user-name>user</user-name>
<password>xxxxxx/password>
</security>
</datasource>
By default, MariaDB supports 150 connections so the database shouldn't be the problem. The default maximum pool size in Wildfly is 20 and I'm the only user in the system. Every time I initiate a function in my application I request two connections and then disconnect.
Why the datasource connections are not available even when I close them?

Here's what worked for me.
Enable cached connection manager in debug mode
<cached-connection-manager debug="true" error="true"/>
Look for this text in your log file - "Closing a connection for you. Please close them yourself". This will help help you find the leak in your code.
In my case, jdbcTemplate.getConnection().createClob() was causing the pool to exhaust.
try {
Connection conn = jdbcTemplate.getConnection()
....
conn.createClob();
...
} catch() {
...
} finally {
conn.close()
}
So properly closing the connection as shown above worked for us.
Hope this saves a lot of time for someone.

One problem with your code is that the context and connection may not be closed if there is an exception.
The old way to solve this was to close the resources in in a finally block. The modern way is to use try with resources. For example:
try (InitialContext context = new InitialContext();
Connection conn = ((DataSource) context.lookup(pool)).getConnection()) {
// use the connection
} catch (NamingException e) {
logger.error(e.getMessage());
throw new DAOException(e.getMessage());
} catch (SQLException e) {
logger.error(e.getMessage());
throw new DAOException(e.getMessage());
}
The try with resources starts with resource declarations where the resources are declared and initialized. Then there is a body where the resources are used. Finally you have (optional) catch and finally blocks.
The secret sauce is that the try with resources construct will automatically close each of the (non-null) resources, in reverse order that they were opened. Exceptions thrown by the close calls will be dealt with appropriately. And so on.
(You can achieve (more or less) the same thing in the old way with finally blocks, but it is complicated.)

I've done this a very different way using JPA and never had an issue. My code looks something like:
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.Query;
public class MyClass {
#PersistenceContext
private EntityManager entityManager;
public SomeObject getSomeObject() {
// as an example query
Query query = entityManager.createQuery("select ...")
}
}
There is some additional configuration needed in META-INF/persistence.xml that looks like:
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://xmlns.jcp.org/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence
http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd"
version="2.1">
<persistence-unit name="somethingPU" transaction-type="JTA">
<jta-data-source>jdbc/yourJNDIName</jta-data-source>
</persistence-unit>
</persistence>
In this way you're not ever dealing with connection management - the container (Wildfly in your case) takes care of it for you.

Related

Java ComboPooledDataSource exceeds pool size and does not reuse

I have the following code that gets called by an other application (That I can not change) to read form a Database.
The method is called in a loop very often and DOSes the DB.
At The DB I can see that there are many connecetions opend ... increasing to some hundred ... and than the DB crashed due to the load.
// Called in a loop
private <T> T execute(String query, PostProcessor<T> postProc, PreProcessor preProcs) throws OperationFailedException {
try (Connection conn
= Objects.requireNonNull(dataSourceRef.get(), "No Connection").getConnection();
PreparedStatement smt = conn.prepareStatement(query)) {
preProc.process(smt);
return postProc.process(smt.executeQuery());
} catch (SQLException e) {
throw new OperationFailedException(e.getMessage(), e);
}
}
the date source gets initialised before ...
// class variable
// AtomicReference<PooledDataSource> dataSourceRef = new AtomicReference<PooledDataSource>();
// Init method
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass(config.getConnectionDriver());
cpds.setJdbcUrl(config.getConnectionString());
cpds.setUser(config.getConnectionUser());
cpds.setPassword(config.getConnectionPassword());
// cpds.setMaxPoolSize(10);
dataSourceRef.getAndSet(cpds);
My question why is this happening.
I thought due to the pooling not for each query a new connections should be used.
As well by setting the max pool size this is not working.
As well I tried it with try-catch-finally construct and closing the stm and the conn after use.
(As I've read somewhere that finally might get called delayed under high load scenarios ... I thouht that might be the case)
But still why is the pool size exceeded?
How can I limit that and block the method untill a connection is avalible again before continuing?
During connection polling in c3p0 you have to consider some options. Those are given bellow application.property file:
db.driver: oracle.jdbc.driver.OracleDriver // for Oracle
db.username: YOUR_USER_NAME
db.password: YOUR_USER_PASSWORD
db.url: DATABASE_URL
minPoolSize:5 // number of minimum poolSize
maxPoolSize:100 // number of maximum poolSize
maxIdleTime:5 // In seconds. After that time it will realease the unused connection.
maxStatements:1000
maxStatementsPerConnection:100
maxIdleTimeExcessConnections:10000
Here, maxIdleTime is the main points. It defines how many seconds this will release unused connection. It is in second.
Another is minPoolSize . It defines how many connection it will hold during idle mode.
Another is maxPoolSize . It defines how many maximum connection it will hold during loaded mode.
Now, how you configure ComboPooledDataSource? Here is the code:
#Bean
public ComboPooledDataSource dataSource(){
ComboPooledDataSource dataSource = new ComboPooledDataSource();
try {
dataSource.setDriverClass(env.getProperty("db.driver"));
dataSource.setJdbcUrl(env.getProperty("db.url"));
dataSource.setUser(env.getProperty("db.username"));
dataSource.setPassword(env.getProperty("db.password"));
dataSource.setMinPoolSize(Integer.parseInt(env.getProperty("minPoolSize")));
dataSource.setMaxPoolSize(Integer.parseInt(env.getProperty("maxPoolSize")));
dataSource.setMaxIdleTime(Integer.parseInt(env.getProperty("maxIdleTime")));
dataSource.setMaxStatements(Integer.parseInt(env.getProperty("maxStatements")));
dataSource.setMaxStatementsPerConnection(Integer.parseInt(env.getProperty("maxStatementsPerConnection")));
dataSource.setMaxIdleTimeExcessConnections(10000);
} catch (PropertyVetoException e) {
e.printStackTrace();
}
return dataSource;
}
For details implementation, please check this thread . Here i added practical implementation
Edit (Getting connection)
You can get connection using bellow way:
Session session = entityManager.unwrap(Session.class);
session.doWork(connection -> doSomeStuffWith(connection));
How you get EntityManager?
#PersistenceContext
private EntityManager entityManager;
Hope this will help you.
Thanks :)

How to connect a JEE app to a MySQL database?

I'm trying to connect a JEE app to a MySQL database. There is a lot of documentations and similar questions on this topic, but none of the solutions i've found works so far (Nonetheless, I may have miss one in particular so feel free to propose).
The server name is db and the database name is joinmyparty
Here is my java code to connect :
public class MySqlConnection {
private static Connection conn;
public static Connection getConn() {
try {
Context initContext = new InitialContext() ;
Context envContext = (Context)initContext.lookup("java:/comp/env") ;
DataSource ds = (DataSource) envContext.lookup("jdbc/joinmyparty") ;
conn = ds.getConnection();
} catch (NamingException e) {
System.out.println("Cannot get connection: " + e);
} catch (SQLException e) {
System.out.println("Cannot get connection: " + e);
}
return conn;
}
Every time I call a DAO, I use this method getConn() to open a connection and i close it in the finally block.
I wrote this into my /tomcat/conf/context.xml file :
<?xml version="1.0" encoding="UTF-8"?>
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<WatchedResource>WEB-INF/tomcat-web.xml</WatchedResource>
<WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>
<Resource url="jdbc:mysql://db:3306/joinmyparty"
driverClassName="com.mysql.jdbc.Driver" password="mypassword"
username="myusername" maxWait="10000" maxIdle="30"
maxActive="100" type="javax.sql.DataSource" auth="Container"
name="jdbc/joinmyparty" validationQuery="select 1" testOnBorrow="true"/>
</Context>
I've tried to put the mysql-connector .jar into the /tomcat/lib or into the /WEB-INF/lib and into the build-path. I've also tried multiple versions of connectors (but only one at a time), especialy to get the one with same level as MySQL database.
When I call a servlet which requires to connect the database, I've a blank page with a POST() method and an error 5OO with a GET() method.
Here is the log error (I tried to translate it my best since i'm not english native) :
description : this server has encountered an internal error which prevents it from fulfilling your request
exception
java.lang.NullPointerException
com.picco.user.dao.UserDao.findByEmail(UserDao.java:40)
com.picco.user.servlet.Index.doGet(Index.java:56)
javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
And here the part of the code concerned (but as I said, all codes using DAO have the same problem).
try {
UserDao udao = new UserDao();
User u = udao.findByEmail(myCookieVal);
SongDao sdao = new SongDao();
ArrayList<Song> list = sdao.getAllSongs(u.getId());
Random rd = new Random();
int count = udao.count();
request.setAttribute("currentSong", list.get(rd.nextInt(list.size())));
request.setAttribute("songList", list);
request.setAttribute("partyCount", count);
this.getServletContext().getRequestDispatcher("/index.jsp").forward(request, response);
} catch(SQLException e) {
e.printStackTrace();
} finally {
HsqlConnection.closeInstance();
}
Dont hesitate to ask me for details if I did not describe enought the problem.
Maybe the problem is that you try to fetch the resource User before having instantiated the connection that is set to null.
The MySQL JDBC JAR belongs in the Tomcat /lib folder. You want the Tomcat class loader to find it first.
Shouldn't your DAO get a connection? I'd add it to the constructor.

create another connection in transaction mysql

I want know if it is possibile create two connection different connections in one transaction, So suppose to have this code( I do a create operation in table "employe" and I create a log in table "log"):
try {
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
PreparateStatement p1 //Preparate statement to put the employe parameter
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
p1.execute();
//This create another connect
LOGStatic.createLog("CREATE EMPLOYE");
connessione_db.commit();
connessione_db.setAutoCommit(true);
}catch(....){
connection_db.rollback();
}
This is the method LOGStatic.createLog("CREATE EMPLOYE");
public static void creaLogException(String messaggio) {
Connection connection_db= null;
InitialContext initialContext;
try {
initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
connection_db= datasource.getConnection();
// the code continues with the save operation
} catch (NamingException n) {
}
// actual jndi name is "jdbc/postgres"
catch (SQLException s) {
}
My question if is it possible this behavior and there aren't problem with commit and rollback operatio or it is not possible to have another connection?
Anyone can help
At least in JDBC, the transaction is tightly coupled with the Connection.
In a Java EE server, if you are writing session beans, then the transaction will be managed my the server. So, in that case, you could call several methods, and the transaction will follow the method calls.
In JDBC the simple solution is to not close the connection, but to pass it around to different methods as an input parameter.
But, be very careful, not closing connections is almost a sure shot way of getting to OutOfMemoryError.
(Instead of passing around the connection as input parameters to various methods, you could use ThreadLocal. But that is another source of memory leak, if you don't clean up ThreadLocal variables.)
For more information on how transaction-lifecycle and Connection are tied in JDBC, refer this: How to start a transaction in JDBC?
Note: Even in JavaEE nested transaction is not possible as nested transactions is not supported in JTA.
Passing the connection around:
public static void createEmployee(){
InitialContext initialContext = new InitialContext();
DataSource datasource = (DataSource) initialContext.lookup("java:/comp/env/jdbc/postgres");
Connection connection_db= datasource.getConnection();
try {
connessione_db.setAutoCommit(false);
connessione_db.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
PreparateStatement p1 //Preparate statement to put the employe parameter
p1.execute();
//This create another connect
createLog("CREATE EMPLOYE", connessione_db);
connessione_db.commit();
//connessione_db.setAutoCommit(true); //No need
}catch(....){
try{ connection_db.rollback(); }catch(Exception e){ /*You can also check some flags to avoid exception*/ }
}finally{
try{ connection_db.close(); }catch(Exception e){ /*Safe to ignore*/ }
}
}
public static void createLog(String messaggio, Connection connection_db) {
try {
// the code continues with the save operation
} catch (SQLException s) {
}
}

Make c3p0 behave like Hibernate built-in connection pool

Probably this is stupid or weird question but I'm facing a strange bug in the app for many weeks and do not know how to solve it.
There are some Quartz jobs that periodically updating database (change the status of orders and stuff). There is only Hibernate 3.1.3 without spring and all transactions handled in code manually with explicit call session.close() in finally block.
It all worked fine with Hibernate built-in connection pool, however after I have changed built-in connection pool to c3p0 pool there are some bugs appeared connected with database transaction/session management. There is no exceptions or anything in the log files, so it is hard to tell what exactly was the reason.
Is there any way to make c3p0 connection pool behave like built-in pool by configuration? I need only one c3p0 feature - checking dead connections. Before this pool was implemented there was idle SQL Connection Reset issue. To keep connections alive I decided to use c3p0 pooling instead of built-in.
here is my current c3p0 configuration:
c3p0.min_size=0
c3p0.max_size=100
c3p0.max_statements=0
c3p0.idle_test_period=600
c3p0.testConnectionOnCheckout=true
c3p0.preferredTestQuery=select 1 from dual
c3p0.autoCommitOnClose=true
c3p0.checkoutTimeout=120000
c3p0.acquireIncrement=2
c3p0.privilegeSpawnedThreads=true
c3p0.numHelperThreads=8
Thanks in advance.
c3p0 is an old version: 0.9.0.4, because I have Java 1.4 environment.
Here is a rough example how transactions are managed by the code:
SessionFactory sessionFactory = null;
Context ctx = null;
Object obj = null;
ctx = new InitialContext();
obj = ctx.lookup("HibernateSessionFactory");
sessionFactory = (SessionFactory) obj;
session = sessionFactory.openSession();
Transaction tx = null;
try {
tx = session.beginTransaction();
// some transactional code
if (!tx.wasRolledBack() && !tx.wasCommitted()) {
tx.commit();
}
} catch (Exception ex) {
if (tx != null && !tx.wasRolledBack() && !tx.wasCommitted()) {
// if it is runtime exception then rollback
if (ex instanceof RuntimeException) {
logger.log(Level.ERROR, "10001", ex);
try {
tx.rollback();
} catch (RuntimeException rtex) {
logger.log(Level.ERROR, "10002", rtex);
}
}
else {
// attempt to commit if there are any other exceptions
try {
tx.commit();
} catch (RuntimeException rtex) {
logger.log(Level.ERROR, "10004", rtex);
// if commit fails then try to rollback
try {
tx.rollback();
} catch (RuntimeException rtex2) {
logger.log(Level.ERROR, "10002", rtex2);
}
// then rethrow this exception
throw rtex;
}
}
}
finally {
session.close();
}

Reload web server with gwt and c3p0 connection pool?

I have a web application written in gwt, and I'm using a PostgreSQL database in the back end. When I make a new session on the server, I set up c3p0 and get a jdbc connection:
ComboPooledDataSource source = new ComboPooledDataSource();
Properties connectionProps = new Properties();
connectionProps.put("user", "username");
connectionProps.put("password", "password");
source.setProperties(connectionProps);
source.setJdbcUrl("some jdbc url that works");
and when I close my session on the server, I close the ComboPooledDataSource.
However... when I press the yellow "reload web server" button in GWT development mode and refresh my page, I get the following warning, and a bunch of subsequent errors preventing me from obtaining a database connection:
WARNING: A C3P0Registry mbean is already registered. This probably means that an application using c3p0 was undeployed, but not all PooledDataSources were closed prior to undeployment. This may lead to resource leaks over time. Please take care to close all PooledDataSources.
Which I assume means that reloading the web server didn't close the ComboPooledDataSource I made (probably a safe assumption). Is there any way I can get it to do that so I can obtain a connection after reloading the web server?
Closing dataSource generally (not only C3P0) is unadvisable because they should to be used from many applications on your server. If you kill this connection pool other can loose data access. In practice you shoud leave pool management to your container and use only JNDI.
Anyway if you neet to ged rid of the warning in your GWT console use this method in your EventListener contextDestroyer:
public abstract class YourListenerimplements EventListener {
//Probably you initialize your dataSource here. I do it with Guice.
#Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
...
}
#Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
try {
connection = dataSource.getConnection(); //Your dataSource (I obtain it from Guice)
} catch (SQLException ex) {
} finally {
try {
if (connection != null) {
connection.close();
}
if (dataSource != null) {
try {
DataSources.destroy(dataSource);
dataSource = null;
} catch (Exception e) {
}
}
} catch (SQLException sQLException) {
XLog.error(sQLException);
}
}
}
}

Categories