We use multiple Thread-Pools in our Java Server,
and one of the Thread-Pool should only contain "deamon" Threads.
I use a Thread Factory, which calls setDaemon(true) for that pool,
and I use my own Thread sub-class, which looks like this:
private static final class DaemonThread extends Thread {
public DaemonThread(ThreadGroup group, Runnable target,
String name, long stackSize) {
super(group, target, name, stackSize);
}
#Override
public void run() {
if (!isDaemon()) {
LOG.error("Wrong state for isDaemon(): false");
}
super.run();
}
}
So, I'm expecting only to see something like this,
when I do a Thread-Dump:
"DAEMON-pool-1-thread-1234" Id=61 in WAITING on lock
=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#2030c72c (daemon)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at com.company.thread.task.Task$TaskThreadFactory$DaemonThread.run(Task.java:194)
Locked synchronizers: count = 0
But I also see something like this too (no "(daemon)"):
"DAEMON-pool-1-thread-3287" Id=3383 in TIMED_WAITING on lock
=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#69e4d79b
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.poll(ScheduledThreadPoolExecutor.java:1134)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.poll(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at com.company.thread.task.Task$TaskThreadFactory$DaemonThread.run(Task.java:194)
Locked synchronizers: count = 0
As you might have noticed from the bottom line,
the Thread that does not say "(daemon)" is of my sub-class.
Firstly, it should be daemon, because the thread-pool sets it,
and secondly, if it is not daemon, then I should see an error log,
which I cannot find in our logs either.
So, idk if the thread-dump output is "wrong", and the thread is daemon,
or if it is not daemon, and the log output is somehow lost,
or if somehow the daemon state is changed after the thread is created,
which is, AFAIK, not allowed:
https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#setDaemon-boolean-
This method must be invoked before the thread is started.
Why do I want to know this? Because our Java server often doesn't shut down when it should, and I suspect it might be related to some threads not being daemon threads, when they should.
Related
My flink job, taskManager vm has too many threadPool, every pool contains one thread, thread names:"pool-{poolNumber}-thread-1".
thread dump:
// about 2200 threads like this
"pool-236-thread-1" #153 prio=5 os_prio=0 tid=0x00007f9e1c12d000 nid=0x245 waiting on condition [0x00007f9e40c66000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007a8082938> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
my debug:
I debug the job in my IDE, start the job.
there is one waiting thread names like 'pool-{poolNumber}-thread-1', which is "pool-5-thread-1".
I put a break on Thread.class's constructure.
the thread to create "pool-5-thread-1"'s stack is:
questions:
I don't know what to do next😢
We are using Wildfly 10.1.0 Final.
We encountered an OutOfMemoryError caused by threads kept growing.
After examining the thread dump.
We found that there are thousands of Remoting "endpoint" task-N threads.
What are Remoting "endpoint" task-N threads for?
Are they created by jobss-remoting?
After restarting the server, we found that in the begining, there were only 16 threads of them:
Remoting "endpoint" task-1 ~ Remoting "endpoint" task-16.
After the server run for serveral days or months, there may be hundred or thousands of Remoting threads:
A snippet of thread dump is listed below.
In this thread dump, there are several "Remoting "endpoint" task-11" with different number.
So are other tasks such as task-1 to task-16.
All these threads were doing nothing but waiting.
"Remoting "endpoint" task-11" #55415 daemon prio=5 os_prio=0 tid=0x00007f2b8c0a8000 nid=0x276e waiting on condition [0x00007f280a36c000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabee2b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-11" #55417 daemon prio=5 os_prio=0 tid=0x00007f2ba003f800 nid=0x276d waiting on condition [0x00007f2794818000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabecf40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-11" #55414 daemon prio=5 os_prio=0 tid=0x00007f2b98023800 nid=0x276b waiting on condition [0x00007f2792bfc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabeda50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-10" #55411 daemon prio=5 os_prio=0 tid=0x00007f2ba003e000 nid=0x276a waiting on condition [0x00007f27926f7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabecf40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-10" #55413 daemon prio=5 os_prio=0 tid=0x00007f2b8c0a7000 nid=0x2769 waiting on condition [0x00007f27927f8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabee2b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-10" #55412 daemon prio=5 os_prio=0 tid=0x00007f2b98022800 nid=0x2768 waiting on condition [0x00007f27c4815000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabeda50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-9" #55372 daemon prio=5 os_prio=0 tid=0x00007f2c7408f000 nid=0x41df waiting on condition [0x00007f27907d8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-8" #55369 daemon prio=5 os_prio=0 tid=0x00007f2c7408d000 nid=0x41dd waiting on condition [0x00007f27909da000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-7" #55368 daemon prio=5 os_prio=0 tid=0x00007f2c7408b000 nid=0x41dc waiting on condition [0x00007f2790adb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-6" #55367 daemon prio=5 os_prio=0 tid=0x00007f2c74089000 nid=0x41db waiting on condition [0x00007f2790bdc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-5" #55366 daemon prio=5 os_prio=0 tid=0x00007f2c74087000 nid=0x41da waiting on condition [0x00007f2790cdd000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-4" #55365 daemon prio=5 os_prio=0 tid=0x00007f2c74085000 nid=0x41d9 waiting on condition [0x00007f2790dde000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-9" #55364 daemon prio=5 os_prio=0 tid=0x00007f2bd813c000 nid=0x41d8 waiting on condition [0x00007f2790edf000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabed500> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Remoting "endpoint" task-9" #55363 daemon prio=5 os_prio=0 tid=0x00007f2bf4044000 nid=0x41d7 waiting on condition [0x00007f2790fe0000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006eabee3c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
....
20180903
I found that "Remoting "endpoint" task" threads are created by "xnio".
And I found there is an issue of xnio that is very similar to our scenario:
https://issues.jboss.org/browse/XNIO-285
It says this issue has been fixed in "xnio 3.6.0.Beta1".
Unfortunately, Wildfly 10.1.0 is using xnio 3.4.0.
When I tried to upgrade to xnio 3.6.5, I got an java.lang.NoClassDefFoundError of org/wildfly/common/context/Contextual. After upgrading wildfly-common-1.4.0.Final.jar which contains the class "org/wildfly/common/context/Contextual", NoClassDefFoundError was still there.
Is there any other way to prevent Remoting "endpoint" task threads from growing?
You might be using scoped EJB context for Remote Method Execution.
Every scoped EJB context will create new thread and simply calling context.close() method won't close the context so you are getting OutOfMemoryError
How to close scoped EJB client contexts?
The answer is the same, use the close() method on the EJB client context. But the real question is how do you get the relevant scoped EJB client context which is associated with a JNDI context. Before we get to that, it's important to understand how the ejb: JNDI namespace that's used for EJB lookups and how the JNDI context (typically the InitialContext that you see in the client code) are related. The JNDI API provided by Java language allows "URL context factory" to be registered in the JNDI framework (see this for details http://docs.oracle.com/javase/jndi/tutorial/provider/url/factory.html). Like that documentation states, the URL context factory can be used to resolve URL strings during JNDI lookup. That's what the ejb: prefix is when you do a remote EJB lookup. The ejb: URL string is backed by a URL context factory.
Internally, when a lookup happens for a ejb: URL string, a relevant javax.naming.Context is created for that ejb: lookup. Let's see some code for better understanding:
// JNDI context "A"
Context jndiCtx = new InitialContext(props);
// Now let's lookup a EJB
MyBean bean = jndiCtx.lookup("ejb:app/module/distinct/bean!interface");
So we first create a JNDI context and then use it to lookup an EJB. The bean lookup using the ejb: JNDI name, although, is just one statement, involves a few more things under the hood. What's actually happening when you lookup that string is that a separate javax.naming.Context gets created for the ejb: URL string. This new javax.naming.Context is then used to lookup the rest of the string in that JNDI name.
Let's break up that one line into multiple statements to understand better:
// Remember, the ejb: is backed by a URL context factory which returns a Context for the ejb: URL (that's why it's called a context factory)
final Context ejbNamingContext = (Context) jndiCtx.lookup("ejb:");
// Use the returned EJB naming context to lookup the rest of the JNDI string for EJB
final MyBean bean = ejbNamingContext.lookup("app/module/distinct/bean!interface");
As you see above, we split up that single statement into a couple of statements for explaining the details better. So as you can see when the ejb: URL string is parsed in a JNDI name, it gets hold of a javax.naming.Context instance. This instance is different from the one which was used to do the lookup (jndiCtx in this example). This is an important detail to understand (for reasons explained later). Now this returned instance is used to lookup the rest of the JNDI string ("app/module/distinct/bean!interface"), which then returns the EJB proxy. Irrespective of whether the lookup is done in a single statement or multiple parts, the code works the same. i.e. an instance of javax.naming.Context gets created for the ejb: URL string.
So why am I explaining all this when the section is titled "How to close scoped EJB client contexts"? The reason is because client applications dealing with scoped EJB client contexts which are associated with a JNDI context would expect the following code to close the associated EJB client context, but will be surprised that it won't:
final Properties props = new Properties();
// mark it for scoped EJB client context
props.put("org.jboss.ejb.client.scoped.context","true");
// add other properties
props.put(....);
...
Context jndiCtx = new InitialContext(props);
try {
final MyBean bean = jndiCtx.lookup("ejb:app/module/distinct/bean!interface");
bean.doSomething();
} finally {
jndiCtx.close();
}
Applications expect that the call to jndiCtx.close() will effectively close the EJB client context associated with the JNDI context. That doesn't happen because as explained previously, the javax.naming.Context backing the ejb: URL string is a different instance than the one the code is closing. The JNDI implementation in Java, only just closes the context on which the close was called. As a result, the other javax.naming.Context that backs the ejb: URL string is still not closed, which effectively means that the scoped EJB client context is not closed too which then ultimately means that the connection to the server(s) in the EJB client context are not closed too.
So now let's see how this can be done properly. We know that the ejb: URL string lookup returns us a javax.naming.Context. All we have to do is keep a reference to this instance and close it when we are done with the EJB invocations. So here's how it's going to look:
final Properties props = new Properties();
// mark it for scoped EJB client context
props.put("org.jboss.ejb.client.scoped.context","true");
// add other properties
props.put(....);
...
Context jndiCtx = new InitialContext(props);
Context ejbRootNamingContext = (Context) jndiCtx.lookup("ejb:");
try {
final MyBean bean = ejbRootNamingContext.lookup("app/module/distinct/bean!interface"); // the rest of the EJB jndi string
bean.doSomething();
} finally {
try {
// close the EJB naming JNDI context
ejbRootNamingContext.close();
} catch (Throwable t) {
// log and ignore
}
try {
// also close our other JNDI context since we are done with it too
jndiCtx.close();
} catch (Throwable t) {
// log and ignore
}
}
As you see, we changed the code to first do a lookup on just the "ejb:" string to get hold of the EJB naming context and then used that ejbRootNamingContext instance to lookup the rest of the EJB JNDI name to get hold of the EJB proxy. Then when it was time to close the context, we closed the ejbRootNamingContext (as well as the other JNDI context). Closing the ejbRootNamingContext ensures that the scoped EJB client context associated with that JNDI context is closed too. Effectively, this closes the connection(s) to the server(s) within that EJB client context.
For More Details you can refer Scoped EJB client contexts
I found that the Remoting "endpoint" task threads are created by javax.management.remote.JMXConnector. We opened some javax.management.remote.JMXConnector to access MBeans in the other servers. But didn't close them. After closing those JMXConnector instances, the threads are gone.
javax.management.remote.JMXConnector is using xnio to communicate with MBeans. It will create a XnioWorker when it is opened, and XnioWorker will create Remoting "endpoint" task threads. So the problem is not caused by EJB.
I'm investigating the strange issue with tomcat shutdown process: after runnig shutdown.sh the java process still appears(I check it by using ps -ef|grep tomcat)
The situation is a bit complicated, because I have very limitted access to the server(no debug, for example)
I took thread dump(by using kill -3 <PID>) and heap dump by using remote jConsole and Hotspot features.
After looking into thread dump I found this:
"pool-2-thread-1" #74 prio=5 os_prio=0 tid=0x00007f3354359800 nid=0x7b46 waiting on condition [0x00007f333e55d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c378d330> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
So, my understanding of the problem is follows: There is a resource(DB connection or something else) which is used in CachedThreadpool, and this resource is now locked,
and prevent to thread pool-2-thread-1 to stop. Assuming that this thread isn't deamon - JVM cannot gracefully stop.
Is there a way to find out which resource is locked, from where is it locked and how to avoid that? Another question is - how to prevent this situation?
Another this is: what the adress 0x00007f333e55d000 is for?
Thanks!
After a wail of struggling I found out, that the freezing thread always have the same name pool-2-thread-1. I grep all the sources of the project to find any places where any scheduled thread pool is started: Executors#newScheduledThreadPool. After a huge amount of time I logged all that places, with thread name in thread pool.
One more restart server and gotcha!
I found out one thread pool, that is started with only one thread and used following code:
private void shutdownThreadpool(int tries) {
try {
boolean terminated;
LOGGER.debug("Trying to shutdown thread pool in {} tries", tries);
pool.shutdown();
do {
terminated = pool.awaitTermination(WAIT_ON_SHUTDOWN,TimeUnit.MILLISECONDS);
if (--tries == 0
&& !terminated) {
LOGGER.debug("After 10 attempts couldn't shutdown thread pool, force shutdown");
pool.shutdownNow();
terminated = pool.awaitTermination(WAIT_ON_SHUTDOWN, TimeUnit.MILLISECONDS);
if (!terminated) {
LOGGER.debug("Cannot stop thread pool even with force");
LOGGER.trace("Some of the workers doesn't react to Interruption event properly");
terminated = true;
}
} else {
LOGGER.info("After {} attempts doesn't stop", tries);
}
} while (!terminated);
LOGGER.debug("Successfully stop thread pool");
} catch (final InterruptedException ie) {
LOGGER.warn("Thread pool shutdown interrupted");
}
}
After that the issue was solved.
Could somebody help me and explain what exactly is going on, looking at the following thread dump. It is a web application running on Tomcat 7 and we experience that some requests do not get answered:
"ajp-bio-8012-exec-161" daemon prio=10 tid=0x00007fe170603000 nid=0x344f runnable [0x00007fe174fae000]
java.lang.Thread.State: RUNNABLE
at java.security.AccessController.doPrivileged(Native Method)
at java.io.FilePermission.init(FilePermission.java:209)
at java.io.FilePermission.<init>(FilePermission.java:285)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.io.File.exists(File.java:808)
at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1080)
at sun.misc.URLClassPath$FileLoader.findResource(URLClassPath.java:1047)
at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
at java.lang.ClassLoader.getResource(ClassLoader.java:1142)
at org.apache.catalina.loader.WebappClassLoader.getResource(WebappClassLoader.java:1445)
at java.lang.Class.getResource(Class.java:2142)
at javassist.ClassClassPath.find(ClassClassPath.java:84)
at javassist.ClassPoolTail.find(ClassPoolTail.java:317)
at javassist.ClassPool.find(ClassPool.java:495)
at javassist.ClassPool.createCtClass(ClassPool.java:479)
at javassist.ClassPool.get0(ClassPool.java:445)
- locked <0x00000000db6cdb48> (a javassist.ClassPool)
at javassist.ClassPool.get(ClassPool.java:414)
at javassist.compiler.MemberResolver.lookupClass0(MemberResolver.java:425)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:389)
at javassist.compiler.MemberResolver.lookupClassByJvmName(MemberResolver.java:310)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:327)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:314)
at javassist.compiler.Javac.compileField(Javac.java:122)
at javassist.compiler.Javac.compile(Javac.java:91)
at javassist.CtField.make(CtField.java:163)
at com.mycompany.validation.util.BeanGenerator.addProperty(BeanGenerator.java:157)
...
at java.lang.Thread.run(Thread.java:745)
"ajp-bio-8012-exec-12" daemon prio=10 tid=0x0000000000c49800 nid=0x3d99 waiting for monitor entry [0x00007fe1756b7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at javassist.ClassPool.get0(ClassPool.java:432)
- waiting to lock <0x00000000db6cdb48> (a javassist.ClassPool)
at javassist.ClassPool.get(ClassPool.java:414)
at javassist.compiler.MemberResolver.lookupClass0(MemberResolver.java:425)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:389)
at javassist.compiler.MemberResolver.lookupClassByJvmName(MemberResolver.java:310)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:327)
at javassist.compiler.MemberResolver.lookupClass(MemberResolver.java:314)
at javassist.compiler.Javac.compileField(Javac.java:122)
at javassist.compiler.Javac.compile(Javac.java:91)
at javassist.CtField.make(CtField.java:163)
at com.mycompany.validation.util.BeanGenerator.addProperty(BeanGenerator.java:157)
...
at java.lang.Thread.run(Thread.java:745)
"ajp-bio-8012-exec-5" daemon prio=10 tid=0x0000000001603800 nid=0x7c77 waiting for monitor entry [0x00007fe174aaa000]
java.lang.Thread.State: BLOCKED (on object monitor)
at javassist.ClassPool.makeClass(ClassPool.java:621)
- waiting to lock <0x00000000db6cdb48> (a javassist.ClassPool)
at javassist.ClassPool.makeClass(ClassPool.java:606)
at com.mycompany.validation.util.BeanGenerator.init(BeanGenerator.java:334)
...
at java.lang.Thread.run(Thread.java:745)
I am not a specialist in thread dumps and just would like to know how to get rid of the problem.
Has the first thread locked an object (ClassPool) that two other request threads are waiting for to unlock? Why is it locked? Do I have to synchronize and how?
Thanks for any hint!
You have a single ClassPool in your application which has the monitor id 0x00000000db6cdb48.
If you see the method signature of ClassPool.get0:
protected synchronized CtClass get0(String classname, boolean useCache)
So it is synchronized. I suppose you simply used the ClassPool.getDefault(). You have to know that this method returns a single instance, so all of your calls will be sychronized.
Create a new pool for each thread (with new ClassPool(ClassPool.getDefault()) for example) and it will be fine then.
Apart from that you might check your secrurity manager, why it takes so long.
I've got a UI automation framework that launches tests using TestNG and runs through pages using Selenium/WebDriver. Oftentimes the pages I'm testing make AJAX calls that modify the DOM upon returning. In these cases I use Selenium explicit waits to declare a DOM condition that I want to be met before the automation can proceed (IE: some button gets enabled).
Internally Selenium's FluentWait.until method handles this by polling the DOM for my ExpectedCondition every 500ms and calling Thread.sleep() in-between these checks.
When I run two tests back to back in a TestNG suite this works perfectly fine for the first test, but starts to fail with an InterruptedException about halfway through each subsequent test. This is consistent. The exceptions look like this:
Associated Throwable Type: class org.openqa.selenium.WebDriverException Associated Throwable Message: java.lang.InterruptedException: sleep interrupted
The strange thing is that there's no multi-threading going on here. I've disabled Selenium Grid, BrowserMob Proxy, and every other bit of code that could be conflicting. I've read both of these questions:
https://stackoverflow.com/questions/24495176/why-is-thread-sleep-being-interrupted - Closed for not providing enough detail, but one of the proposed answers states that one should override the Thread.interrupt method for debugging.
Who interrupts my thread? - Accepted answer also states that one should override the Thread.interrupt method for debugging.
My problem with this solution is that placing a breakpoint inside the existing Thread.interrupt method does not reveal any calls around the time that the thread is interrupted. This includes calls from all of my third party dependencies (IE: TestNG and Selenium). Whatever is calling this thread interrupt appears to be external to my framework.
I've also tried calling Thread.currentThread.isInterrupted() at every point prior to the FluentWait.until call and it consistently returns false. I've even used IntelliJ's evaluate function to check for isInterrupted inside the Selenium code itself. This thread is only being interrupted once the Thread.sleep call occurs inside FluentWait.until.
I've seen this happen on multiple Windows build servers as well as on my Macbook, so this does not appear to be machine specific.
I thought for a while that this might be caused by a TestNG timeout, but reducing the TestNG timeout in my suite yielded a different behavior than these interruptions.
Currently I'm working around this issue with the following code which swallows the exception and resumes the explicit wait:
public static boolean waitForElementStatus(Stuff)
{
/* snip - setup for ExpectedCondition (change) */
long startSeconds = new Date().getTime() / 1000;
long currentSeconds = startSeconds;
long remainingSeconds = maxElementStatusChangeSeconds;
WebDriverWait waitForElement = new WebDriverWait(driver, maxElementStatusChangeSeconds);
boolean changed = false;
boolean firstWait = true; // If specified time is 0 we still want to check once.
out:while(firstWait || remainingSeconds > 0)
{
firstWait = false;
Boolean exceptionThrown = false;
try
{
waitForElement.until(change);
}
catch(Throwable t)
{
exceptionThrown = true;
if(t.getCause()) != null
{
t = t.getCause(); // InterruptedException is wrapped inside a WebDriverException
}
if(t.getClass().equals(InterruptedException.class))
{
Thread.interrupted(); // clear interrupt status for this thread
currentSeconds = new Date().getTime() / 1000;
remainingSeconds = startSeconds + maxElementStatusChangeSeconds - currentSeconds;
if(remainingSeconds > 0)
{
String warning = String.format("Caught unidentified interrupt inside Selenium " +
"FluentWait.until call. Swallowing interrupt and repeating call with [%s] seconds " +
"remaining.", remainingSeconds);
CombinedLogger.warn(warning);
waitForElement = new WebDriverWait(driver, remainingSeconds);
}
else
{
// If a timeout exception would have been thrown instead of the interruption then
// we'll allow the WebDriverWait to execute one last time so it can throw the
// timeout instead.
waitForElement = new WebDriverWait(driver, 0);
}
}
else if(haltOnFailure) // for any other exception type such as TimeoutException
{
CombinedLogger.error(stuff + "...FAILURE(HALTING)", t);
break out;
}
else // for any other exception type such as TimeoutException
{
CombinedLogger.info(stuff + "...failure(non-halting)");
break out;
}
}
if(!exceptionThrown)
{
changed = true;
CombinedLogger.info(stuff + "...success ");
break out;
}
}
return changed;
}
This workaround does function, and fortunately these mystery interrupts are only occurring sporadically afterwards (they don't happen repeatedly), so the tests are able to proceed. However, I understand that swallowing InterruptedException is bad form. If possible, I'd like to determine where and why these interrupts are taking place so that I can put an end to them instead of using this hack.
Simply propagating the exceptions is not an option since these tests need to continue running instead of obediently crashing.
Are there any known utilities, JVM arguments, or libraries that I could use which would help me track down Java thread interruptions that are caused by code which is out of my control?
Update 12/10/2014: I've captured two thread dumps. One is from immediately before the interrupt and one is from immediately after it. The only difference between the two is the line number of the interrupted thread (it goes from the try block to the catch block after being interrupted). Not sure what this tells me, but here's the data:
Full thread dump (immediately before interrupt)
"TestNG#1359" prio=5 tid=0xc nid=NA runnable
java.lang.Thread.State: RUNNABLE
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:232)
/* snip - company stuff */
at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:46)
at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
at org.testng.internal.MethodInvocationHelper.invokeWithTimeoutWithNoExecutor(MethodInvocationHelper.java:240)
at org.testng.internal.MethodInvocationHelper.invokeWithTimeout(MethodInvocationHelper.java:229)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:724)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"main#1" prio=5 tid=0x1 nid=NA waiting
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:422)
at java.util.concurrent.FutureTask.get(FutureTask.java:199)
at java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:289)
at org.testng.internal.thread.ThreadUtil.execute(ThreadUtil.java:72)
at org.testng.SuiteRunner.runInParallelTestMode(SuiteRunner.java:367)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:308)
at org.testng.SuiteRunner.run(SuiteRunner.java:254)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:125)
"Thread-8#2432" daemon prio=5 tid=0x15 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(FileInputStream.java:-1)
at java.io.FileInputStream.read(FileInputStream.java:272)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
- locked <0xe08> (a java.lang.UNIXProcess$ProcessPipeInputStream)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.exec.StreamPumper.run(StreamPumper.java:105)
at java.lang.Thread.run(Thread.java:745)
"Thread-7#2431" daemon prio=5 tid=0x14 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(FileInputStream.java:-1)
at java.io.FileInputStream.read(FileInputStream.java:272)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
- locked <0xe09> (a java.lang.UNIXProcess$ProcessPipeInputStream)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.exec.StreamPumper.run(StreamPumper.java:105)
at java.lang.Thread.run(Thread.java:745)
"Thread-6#2424" prio=5 tid=0x13 nid=NA waiting
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:503)
at java.lang.UNIXProcess.waitFor(UNIXProcess.java:261)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:347)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:46)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:188)
"process reaper#2008" daemon prio=10 tid=0x10 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at java.lang.UNIXProcess.waitForProcessExit(UNIXProcess.java:-1)
at java.lang.UNIXProcess.access$500(UNIXProcess.java:54)
at java.lang.UNIXProcess$4.run(UNIXProcess.java:225)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"ReaderThread#645" prio=5 tid=0xb nid=NA runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(SocketInputStream.java:-1)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
- locked <0xe0b> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at org.testng.remote.strprotocol.BaseMessageSender$ReaderThread.run(BaseMessageSender.java:245)
"Finalizer#2957" daemon prio=8 tid=0x3 nid=NA waiting
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Object.java:-1)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
"Reference Handler#2958" daemon prio=10 tid=0x2 nid=NA waiting
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
"Signal Dispatcher#2956" daemon prio=9 tid=0x4 nid=NA runnable
java.lang.Thread.State: RUNNABLE
There is not much that can be inferred from the thread dump as in what caused it.
But in reality you cannot rely on Thread.sleep() too much ,it might be interrupted for known/unknown reason.OS might be the reason in the later case.
Thread.sleep() is one of the few methods which takes interrupt seriously. As a thread cannot handle InterruptedException while it is sleeping ,you need to handle it.
What you are doing right now might not be a workaround but a way to go in such cases,where we cannot do without Thread.sleep().
A bit outdated but I have similar problem and with the help of your previously posted link (https://stackoverflow.com/a/2476246) I put a breakpoint into the Thread.interrupt() method.
It reveals that the interruption was made by StoryManager.waitUntilAllDoneOrFailed() method that triggers future.cancel() method after the timeout set on whole story.
My whole setup is:
page.getPageObject().withTimeoutOf(convertDuration(duration)).waitFor(by);
where duration is about 60 secs. (the minute is due to some async stuff)
and
configuredEmbedder().embedderControls().useStoryTimeouts("30");
And the stackTrace is:
at java.util.concurrent.FutureTask.cancel(FutureTask.java:174)
at org.jbehave.core.embedder.StoryManager.waitUntilAllDoneOrFailed(StoryManager.java:184)
at org.jbehave.core.embedder.StoryManager.performStories(StoryManager.java:121)
at org.jbehave.core.embedder.StoryManager.runStories(StoryManager.java:107)
and that interrupts later Thread.sleep() method in ThucydidesFluentWait.doWait() (basically in the underneath Sleeper instance method sleep())
Increasing the story timeout or proper setup of waitFor(...) timeout vs. story timeout solves the problem on my side.