How to properly close vertx and vertx mysqlclient on jakartaee application? - java

I use vertx and vertx mysql client in a jakartaEE10 application. I used netbeans to write code, it auto reloads once I make change by redeploying the war file in TomEE. This used to work fine in java ee 7 app (I used only Tomcat9 there, not TomEE).
I closed vertx and mysqlclient. I tried closing in #Disposes, and then tried closing contextDestroyed method of ServletContextLisener. I can see the code triggers. But the next time app loads it doesnt seem to load properly. Every vertx call blocks.
Is there anything more should I do please let me know.
I am getting following error in logs.
08-Jan-2023 12:24:47.937 WARNING [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [app] appears to have started a thread named [vertx-blocked-thread-checker] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [app] appears to have started a thread named [vert.x-eventloop-thread-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.base#17.0.5/sun.nio.ch.KQueue.poll(Native Method)
I would love to provide more information if needed. Plese let me know.
Edit 1:
I was blocking on #postconstruct to load config from database. I spawned a new thread to execute the blocking code. Now it runs fine, not sure how postconstruct thread is linked to event loop thread. This still blocks postconstruct method by calling Thread.join() but somehow it works.

Related

debug java application while startup

It might sound a bit too naive, but I always had a tough time to debug a java class during server startup. Here is the scenario:
The java application is hosted on a tomcat server.
Suppose there is a class which is invoked while the tomcat server starts up.
As soon as I stop the tomcat instance for a restart, the eclipse debug stops and I can start the debug only once the application is up and running.
Now how do I debug this class on eclipse?
Thanks for you help in advance.
You need to pass the "wait for debugger to connect" flag to tomcat. So the startup will wait until you have connected and thus you won't miss the breakpoint.
Take for example those java options to make tomcat listen for a debugger:
-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=8797,server=y,suspend=y
The wait for a debugger connection flag is the suspend=y entry in the above line.
Debugging of classes during startup should work without problems. For example, suppose you implement a ServletContextListener class. The belonging contextInitialized method is called during Tomcat startup (or to be more precise: while your app gets deployed).
You can set breakpoints in this class. At that point, Tomcat is already started completely and there shouldn't be any problems with debugging.

Tomcat shutdown VERY slow after calling shutdown.sh?

My situation is like this:
Everytime before uploading war file to web-app folder, I stop Tomcat by calling sh shutdown.sh. It used to take about 30 seconds for a total shutdown. But now it doesn't work well anymore.
Actually, it did some work, because when I access the application from web-page it throws 503 error (Under Maintenance). But when I use ps aux | grep tomcat to check, the tomcat process is still there. And it will be there for around 5 - 10 mins.
I understand that it may need to take extra times to complete all the tasks, but it is way too slow (5 - 10 minutes), before it is stop totally. I don't understand why this happens, but there must be some reason. Maybe there's something to do with the code, or the new script of deployment we used recently. I just have almost no clue about where to check.
This is important to our team because we are using "auto-deployment", in which we use a script to auto-package war file, uploading and deploy on a specific time. If we started a new tomcat instance before the old one successfully shutdown, it will hang there for eternal, and cleaning up task by "kill -9" is daunting.
Is there anyone who has experimented this issue? Any clue would be appreciated.
Hoàng Long -
Thank you for the update.
1) The fact that you see your Quartz jobs running, and the error message, are both significant:
SEVERE: The web application [/project] appears to have started a
thread named [Resource Destroyer in BasicResourcePool.close()] but has
failed to stop it. This is very likely to create a memory leak.
2) One suggestion is configuration:
http://forum.springsource.org/showthread.php?17833-Spring-Quartz-Tomcat-no-shutdown
I had the same problem. I fixed it by adding
destroy-method="destroy" to the SchedulerFactoryBean definition.
This way spring closes down the scheduler when the application is
stopped.
3) Another suggestion is to add a shutdown listener:
http://forums.terracotta.org/forums/posts/list/15/4341.page
Using a context listener and introducing a timeout on shutdown solves
the issue for me. I just wait a second after shutting down:
public void contextDestroyed(ServletContextEvent sce) {
try {
factory.getScheduler().shutdown();
Thread.sleep(1000);
If this is something that mystically started to happen within the last few days, perhaps you're running into the Linux leap second bug? For more information, see
https://serverfault.com/questions/403732/anyone-else-experiencing-high-rates-of-linux-server-crashes-during-a-leap-second
https://access.redhat.com/knowledge/articles/15145
http://pedroalves-bi.blogspot.fi/2012/07/java-leap-second-bug-how-to-fix-your.html

A web application appears to have started a thread named [22] but has failed to stop it. This is very likely to create a memory leak

I have a web application with Servlets in the back end deployed over tomcat. The application is simple java application.
I see this error frequently in the server logs:
SEVERE: A web application appears to have started a thread named
[22] but has failed to stop it. This is very likely
to create a memory leak.
Are there any potential reasons which might be causing it?
I'd use visualvm 1.3.2 and see what threads are being created. Be sure to add all the plug-ins.
If it's not being done by your code you won't have much control over it.
You also don't know if the message is a red herring or not. Load test your code over a period of time and measure what happens.
I faced similar situation recently Resolved in below steps
I took Thread dump. ( using Kill -QUIT pid )
Found the Runnable/Thread class form dump
Then i put a debug point in run method and started application in
debug mode.
Got the code which starts My Thread and I observed it was not
stopped while stoping application.
Introduced code to stop the thread in contextDestroyed method of
AppContextListener (This is My application class which extends
ServletContextListener ) as this method will be called when i stop
tomcat.
If you set Thread as Dameon Thread it is not going to help , you can visit explanation.
Tomcat waits for all the application's threads (user threads not daemon threads) to stop before it goes down, I guess that in your case this specific thread is a user thread and therefore tomcat generated this error.
I suggest you to change this thread to daemon (assuming this one is yours)

Starting and reconnecting to a spring bean from console

I like to write a little server application being controlled by a little console app (start, pause, stop, etc). For the server spring should be used (part of it already exist). So what I do to start the server is something like this:
start a server thread and exit main method
and then, in the thread:
load application context from xml
start some worker threads connecting to beans doing stuff
Now I want the server to be stopped by another command. How can I connect to the already running context and shut it down?
Thanks for your help,
Alexander
The classical way to manage running Java code is through JMX.
Spring lets you export beans as MBeans.
And for accessing JMX via the command line, look at the answers to this recent question:
Calling JMX MBean method from a shell script
You could create a pid file, when the server starts, it should log the pid to a file, server.pid. When you do a stop, you can read the process and kill it. Or even simpler, have a ever running thread in your main class which keeps looking for a file in some folder. As soon as the file becomes available or gets modified, it will stop the server.

JDBC connection for a background thread being closed accessing in Websphere

I have an application running in Websphere Portal Server inside of Websphere Application Server 6.0 (WAS). In this application for one particular functionality that takes a long time to complete, I am firing a new thread that performs this action. This new thread opens a new Session from Hibernate and starts performing DB transactions with it. Sometimes (haven't been able to see a pattern), the transactions inside the thread work fine and the process completes successfully. Other times however I get the errors below:
org.hibernate.exception.GenericJDBCException: could not load an entity: [OBJECT NAME#218294]
...
Caused by: com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Connection is closed.
Method cleanup failed while trying to execute method cleanup on ManagedConnection WSRdbManagedConnectionImpl#642aa0d8 from resource jdbc/MyJDBCDataSource. Caught exception: com.ibm.ws.exception.WsException: DSRA0080E: An exception was received by the Data Store Adapter. See original exception message: Cannot call 'cleanup' on a ManagedConnection while it is still in a transaction..
How can I stop this from happening? Why does it seem that WAS wants to kill my connections even though they're not done. Is there a way I can stop WAS from attempting to close this particular connection?
Thanks
I mentioned two possible causes in my other answer: 1. the hibernate.connection.release_mode optional parameter or 2. a problem with unmanaged threads. Now that I read this question, I really start to think that your problem may be related to the fact that you're spawning your own threads. Since they aren't managed by the container, connections used in these treads may appear as "leaked" (not closed properly) and I wouldn't be surprised if WAS tries to recover them at some point.
If you want to start a long running job, you should use a WorkManager. Don't spawn threads yourself.

Categories