I am getting below exception in spring transactional
Could not commit JDBC transaction; nested exception is
com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Connection
is closed.
I am using web sphere connection pool.
We solved this problem by setting "Minimum connections" to 0. This is not performance-wise but much better than exception.
Related
I've got a problem with a Spring web application that periodically runs into an error fetching a connection from my connection pool. Eventually in the logs I see entries like:
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
Only way to recover I've found once it hits this point is to restart Tomcat.
I think the most likely explanation is I have some code somewhere that is not properly cleaning up its connection - returning it to Hikari, leaving something open so Spring can't clean it up, etc.
To troubleshoot I've set my hikari config leakDetectionThreshold to 5000ms and enabled logging. After that, I see log entries like
2018-04-24 19:53:56 WARN ProxyLeakTask:87 - Connection leak detection
triggered for org.postgresql.jdbc.PgConnection#664ec666, stack trace
follows
java.lang.Exception: Apparent connection leak detected
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:99)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:129)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:47)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:146)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:172)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:148)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1940)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1909)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1887)
at org.hibernate.loader.Loader.doQuery(Loader.java:932)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349)
at org.hibernate.loader.Loader.doList(Loader.java:2615)
at org.hibernate.loader.Loader.doList(Loader.java:2598)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430)
at org.hibernate.loader.Loader.list(Loader.java:2425)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:335)
at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:2129)
at org.hibernate.internal.AbstractSharedSessionContract.list(AbstractSharedSessionContract.java:981)
at org.hibernate.query.internal.NativeQueryImpl.doList(NativeQueryImpl.java:147)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1398)
at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1444)
at sun.reflect.GeneratedMethodAccessor191.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.orm.jpa.SharedEntityManagerCreator$DeferredQueryInvocationHandler.invoke(SharedEntityManagerCreator.java:379)
at com.sun.proxy.$Proxy163.getSingleResult(Unknown Source)
at com.mycompany.web.jpa.util.DBHelper.getPagedMappedDbResults(DBHelper.java:76)
at com.mycompany.web.jpa.repository.TaskRepositoryImpl.findTaskDetailsByStepIdAndIdIn(TaskRepositoryImpl.java:245)
......
So it is detecting a possible leak. Could be a false positive I suppose? But this is also the only class in my app that is doing database access outside of the standard service/repository pattern often used in Spring apps, so it seems like a likely culprit, and it's my best lead at the moment.
Anyway, the last piece of non library code I see in the trace (ie stuff I wrote, so most likely to be the cause of the leak!) is my DBHelper::getPagedMappedDbResults method, relevant bit included here:
Query q = entityManager.createNativeQuery(countQueryText);
setQueryParameters(q, parameters);
long numActualResults = 0;
try {
numActualResults = ((Number)q.getSingleResult()).longValue(); // line 76
} catch (Exception e) {
System.out.println("just in case: " + e);
}
So basically I create a Query object from my EntityManager instance, set some parameters, and run it to get some results.
Is there something I need to be doing with a Query object when I'm done with it? q.cleanup()? I don't see anything like this from reading the docs, but am I not doing good housekeeping on this resource?
The entityManager itself is created from an #Autowired annotation. My understanding is if I didn't "new" it to instantiate it and instead let the Spring framework autowire it, then Spring will do whatever cleanup is necessary. Is that right? Or do I need to be doing some cleanup after I use the entityManager?
Version details:
Tomcat 8 / Java 8
Spring 5.0.0.RELEASE
Spring Data Kay-RELEASE
Hibernate 5.2.3.Final
Hikari 2.4.5
Any advice or suggestions would be greatly appreciated, thanks!
What is the query? Is it heavy? Maybe you have deadlock here? Connection management looks fine. You do not acquire connection explicitly, so no need to release it. The query might be long running so Hibernate is not able to complete it and release the connection.
Also, you can check the number of open connections on the DB side. Do some analysis on that side as well.
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
No operations allowed after connection closed.Connection was
implicitly closed due to underlying exception/error:
This exception occured after 7 hrs with out using connection.
what is good solution for this problem
There is on error that appears in my log file every time:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Socket closed
STACKTRACE:
java.net.SocketException: Socket closed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:113)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:160)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1994)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3250)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3179)
at com.mysql.jdbc.Statement.executeQuery(Statement.java:1207)
at com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyStatement.java:35)
It happens always in the same point of the code, when the application does a specific query in the MySQL database.
The query is LIKE THIS:
SELECT M.*, O.id
FROM order_message M
INNER JOIN orders O ON M.order_id = O.id
WHERE O.seller_id = 14224 AND M.sender_user_id <> 14224
ORDER BY M.creation_date DESC
LIMIT 5
I noticed (by EXPLAINING this query) that it always use temporary/filesort to execute. All indexes are properly created and I think these is no way to improve this, but I suspect the query performance or resource utilization is causing the exception error.
I am using amazon RDS
The problem was that my connection pool was configured in a way that any database connection that took longer than 10 seconds would be dropped by the connection pool (c3p0). I was using the unreturnedConnectionTimeout parameter.
The c3p0 documentation page discourage to use this parameter. Ideally, all connections should be properly closed (and thus returned to the pool)
http://www.mchange.com/projects/c3p0/#unreturnedConnectionTimeout
I have increased the parameter to 60 seconds and the problem get solved.
I am using DB2 with JDBC and the below code throws SQLException when I try to run for the first time after I left the connection idle for a few minutes. From the second time onward it would work exactly the way it has to.
ResultSet.next()
Any ideas on what would cause the exception?
Exception trace:
FFDC Exception:com.ibm.db2.jcc.am.lo SourceId:com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next
ProbeId:2624 Reporter:
com.ibm.ws.rsadapter.jdbc.WSJccResultSet#b080b08
com.ibm.db2.jcc.am.lo: The current transaction was rolled back because of error "-30108".. SQLCODE=-1476, SQLSTATE=40506, DRIVER=3.57.110
at com.ibm.db2.jcc.am.bd.a(bd.java:663)
at com.ibm.db2.jcc.am.bd.a(bd.java:60)
at com.ibm.db2.jcc.am.bd.a(bd.java:127)
at com.ibm.db2.jcc.am.am.b(am.java:3760)
at com.ibm.db2.jcc.t4.eb.h(eb.java:278)
at com.ibm.db2.jcc.t4.eb.a(eb.java:239)
at com.ibm.db2.jcc.t4.eb.c(eb.java:31)
at com.ibm.db2.jcc.t4.u.a(u.java:32)
at com.ibm.db2.jcc.t4.j.Zb(j.java:259)
at com.ibm.db2.jcc.am.am.X(am.java:3554)
at com.ibm.db2.jcc.t4.d.f(d.java:1881)
at com.ibm.db2.jcc.am.gc.a(gc.java:200)
at com.ibm.db2.jcc.t4.d.a(d.java:109)
at com.ibm.db2.jcc.am.am.c(am.java:366)
at com.ibm.db2.jcc.am.am.next(am.java:293)
at com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next(WSJdbcResultSet.java:3120)
The "outer" SQL Code, -1476, indicates that the transaction was rolled back because of the "inner" SQL Code, -30108, which says:
A connection failed but has been re-established. Special register
settings might have been replayed. Host name or IP address of the new
connection: host-name. Service name or port number of the new
connection: service-name. Reason code: reason-code.
I would guess that your connection timed out, and it dropped.
I have a simple Web App developed using Spring and recently I'm having some problems with DB connections. My DB is on MS SQL Server 2005.
To retrieve data I implemented several DAO using JDBCTemplate class provided by the framework.
Sometimes I get this exception:
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [SELECT [Campaigns].[CampaignID],[CampaignCode],[CampaignType],[StartDate],[EndDate],[Status],[FirstUpdate],[LastUpdate],[FirstUpdateUserID],[LastUpdateUserID],[CampaignDescriptions].[Description] FROM [Campaigns] INNER JOIN [CampaignDescriptions] ON [Campaigns].[CampaignID] = [CampaignDescriptions].[CampaignID] ORDER BY [StartDate] DESC]; Connection reset by peer: socket write error; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset by peer: socket write error
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:894)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
the exception seems to occur only when the reported query is executed.
I can't understand what's causing this problem. Any idea?
The method executing the query is:
#Transactional(propagation = Propagation.NEVER)
public Campaign[] getAll() {
List<Campaign> campList = getJdbcTemplate()
.query(BASE_QUERY,
new CampaignMapper());
return campList.toArray(new Campaign[0]);
}
where BASE_QUERY is the same in the exception message.
it kinda of smells like a stale connection problem. It might be showing up only in this query if it is quite common or the first to be executed (most of the time).
Things to check:
Server side connection timeout
Connection pool minium idle timeout
Connection pool verify connection before use