We Keep Coding

How do I kill a Java Future?

The service I'm working on uses a Future to run multiple tasks in parallel; each task can take up to a minute to complete. However, it seems the external lib is buggy, since in some occasions (2% of the time) it doesn't return. In those cases I would like to give a 2-minute wait time, and if it hasn't returned, I would like to kill the future and re-schedule again later (it will succeed eventually).
How do I kill the Future?
private void run() {
ExecutorService queue = Executors.newFixedThreadPool(1);
Future<Integer> f = queue.submit(new MyTask());
Thread.sleep(500);
try {
Integer r = f.get(120, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
f.cancel(true);
}
// Bad future still running here and I need it dead.
}
private class MyTask implements Callable<Integer> {
private ExternalLibrary extlib = new ExternalLibrary();
#Override
public Integer call() throws Exception {
// step 1 - do a few things
// step 2 - process data
Integer val = this.extlib.doSomething(); // here's the problem!
// step 3 - do other things
return val;
}
}
I can see the external lib running and consuming CPU (for 24 hours)... doing nothing. It's a simple task that should never take more than 60 seconds to complete its work.
So far, I'm killing the whole JVM once a day to get rid of this issue, but I'm sure there must be a better way. I wonder how app servers (Tomcat, JBoss, Weblogic, etc.) do it with rogue processes.

Even if you could kill the future hanging in the buggy library, this does likely not solve your problem. The library might still have acquired some resource which will not be properly clean up. This might be memory allocations, open file handles or even monitors leaving some internal data structures in an inconsistent state. Eventually you will likely be back at the point where you have to restart your JVM.
There's basically two options: Fix or isolate it.
Fix: try to get the library fixed. If this is not possible,
isolate: isolate the library into a external service your application depends on. E.g. implement a REST API for calling the library and wrap everything up into a Docker image. Automate restarting of the Docker container as needed.

As others have mentioned, stopping a Future is cooperative, meaning, the thread running async must respond to cancellation from the waiting thread. If the async task isn't cooperative simply invoking shutdown or shutdownNow won't be enough as the underlying TPE will just interrupt the threads.
If you have no control over extlib, and extlib is not cooperative, I see two options
You can stop the thread currently running. This can cause issues if the thread being stopped currently is holding a lock or some other resource. It can lead to interesting bugs that could be hard to dissect.
This could take some more work, but you could run the async task as a separate process entirely. The TPE can still run the process and, on interruption, can destroy the process. This obviously has more interesting issues like how to load the process with required input.

If I understand your requirement correctly & based on your requirement (i.e. 1 thread), you can look for shutting down executorservice in 2 phases, code is available in java doc of executorservice:
try {
Integer r = f.get(120, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
//f.cancel(true); you can omit this call if you wish.
shutdownAndAwaitTermination(queue);
} ... //remaining method code
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
Please read documentation about shutdown() , shutdownNow() how they behaves because it clearly mentions there is no 100% guarantee that tasks / executorservice will get stopped if its running.

Unfortunately if the external library is not co-operating to thread interrupts, there is nothing you can do to kill the Thread running the task managed by the ExecutorService.
An alternative that I can think of is to run the offending code as a separate process. Using ProcessBuilder and Process, your task can effectively control (or) even kill the offending process after a timeout (https://docs.oracle.com/javase/9/docs/api/java/lang/Process.html#destroyForcibly--).
Also see https://docs.oracle.com/javase/9/docs/api/java/lang/ProcessBuilder.html

#joe That is correct. Unless you have control over the thread and inside the thread you can't kill it.
this.extlib.doSomething();
if this line starts a thread then we need to get hold of that thread to kill it as we don't have reference to stop it.

In your code, the call:
this.extlib.doSomething()
must be synchronous, because if it is not, the code lost sense. With that assumption, you can try:
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Integer> future = executor.submit(new MyTask());
try {
future.get(120, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
} catch (TimeoutException e) {
future.cancel(true);
} finally {
executor.shutdownNow();
}
If this doesn't stop the doSomethig work is because this doSomething function is opening other threads to do the work. In that case, maybe you can check the threads that are running with:
Thread.getAllStackTraces()
And try to kill the right one...

Best practice for interrupting threads that take longer than a threshold

I am using the Java ExecutorService framework to submit callable tasks for execution.
These tasks communicate with a web service and a web service timeout of 5 mins is applied.
However I've seen that in some cases the timeout is being ignored and thread 'hangs' on an API call - hence, I want to cancel all the tasks that take longer than say, 5 mins.
Currently, I have a list of futures and I iterate through them and call future.get until all tasks are complete. Now, I've seen that the future.get overloaded method takes a timeout and throws a timeout when the task doesnt complete in that window. So I thought of an approach where I do a future.get() with timeout and in case of TimeoutException I do a future.cancel(true) to make sure that this task is interrupted.
My main questions
1. Is the get with a timeout the best way to solve this issue?
2. Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Any suggestions would be deeply appreciated.

Is the get with a timeout the best way to solve this issue?
This will not suffice. For instance, if your task is not designed to response to interruption, it will keep on running or be just blocked
Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Yes, You might end up cancelling as task which is never scheduled to run if your thread-pool is not configured properly
Following code snippet could be one of the way you can make your task responsive to interruption when your task contains Non-interruptible Blocking. Also it does not cancel the task which are not scheduled to run. The idea here is to override interrupt method and close running tasks by say closing sockets, database connections etc. This code is not perfect and you need to make changes as per requirements, handle exceptions etc.
class LongRunningTask extends Thread {
private Socket socket;
private volatile AtomicBoolean atomicBoolean;
public LongRunningTask() {
atomicBoolean = new AtomicBoolean(false);
}
#Override
public void interrupt() {
try {
//clean up any resources, close connections etc.
socket.close();
} catch(Throwable e) {
} finally {
atomicBoolean.compareAndSet(true, false);
//set the interupt status of executing thread.
super.interrupt();
}
}
public boolean isRunning() {
return atomicBoolean.get();
}
#Override
public void run() {
atomicBoolean.compareAndSet(false, true);
//any long running task that might hang..for instance
try {
socket = new Socket("0.0.0.0", 5000);
socket.getInputStream().read();
} catch (UnknownHostException e) {
} catch (IOException e) {
} finally {
}
}
}
//your task caller thread
//map of futures and tasks
Map<Future, LongRunningTask> map = new HashMap<Future, LongRunningTask>();
ArrayList<Future> list = new ArrayList<Future>();
int noOfSubmittedTasks = 0;
for(int i = 0; i < 6; i++) {
LongRunningTask task = new LongRunningTask();
Future f = execService.submit(task);
map.put(f, task);
list.add(f);
noOfSubmittedTasks++;
}
while(noOfSubmittedTasks > 0) {
for(int i=0;i < list.size();i++) {
Future f = list.get(i);
LongRunningTask task = map.get(f);
if (task.isRunning()) {
/*
* This ensures that you process only those tasks which are run once
*/
try {
f.get(5, TimeUnit.MINUTES);
noOfSubmittedTasks--;
} catch (InterruptedException e) {
} catch (ExecutionException e) {
} catch (TimeoutException e) {
//this will call the overridden interrupt method
f.cancel(true);
noOfSubmittedTasks--;
}
}
}
}
execService.shutdown();

Is the get with a timeout the best way to solve this issue?
Yes it is perfectly fine to get(timeout) on a Future object, if the task that the future points to is already executed it will return immediately. If the task is yet to be executed or is being executed then it will wait until timeout and is a good practice.
Is there the possibility that I'm waiting with the get call on a task
that hasnt yet been placed on the thread pool(isnt an active worker)
You get Future object only when you place a task on the thread pool so it is not possible to call get() on a task without placing it on thread pool. Yes there is a possibility that the task has not yet been taken by a free worker.

The approach that you are talking about is ok. But most importantly before setting a threshold on the timeout you need to know what is the perfect value of thread pool size and timiout for your environment. Do a stress testing which will reveal whether the no of worker threads that you configured as part of Threadpool is fine or not. And this may even reduce the timeout value. So this test is most important i feel.
Timeout on get is perfectly fine but you should add to cancel the task if it throws TimeoutException. And if you do the above test properly and set your thread pool size and timeout value to ideal than you may not even need to cancel tasks externally (but you can have this as backup). And yes sometimes in canceling a task you may end up canceling a task which is not yet picked up by the Executor.

You can of course cancel a Task by using
task.cancel(true)
It is perfectly legal. But this will interrupt the thread if it is "RUNNING".
If the thread is waiting to acquire an intrinsic lock then the "interruption" request has no effect other than setting the thread's interrupted status. In this case you cannot do anything to stop it. For the interruption to happen, the thread should come out from the "blocked" state by acquiring the lock it was waiting for (which may take more than 5 mins). This is a limitation of using "intrinsic locking".
However you can use explicit lock classes to solve this problem. You can use "lockInterruptibly" method of the "Lock" interface to achieve this. "lockInterruptibly" will allow the thread to try to acquire a lock while remaining responsive to the interruption. Here is a small example to achieve that:
public void workWithExplicitLock()throws InterruptedException{
Lock lock = new ReentrantLock();
lock.lockInterruptibly()();
try {
// work with shared object state
} finally {
lock.unlock();
}
}

ReentrantLock says "unlocked" but first thread stops

We're calling "lock()" on a ReentrantLock and threads are getting stuck there when they apparently shouldn't.
When debugging with a breakpoint just before the call to "lock()", the first thread would stop there with the program pointer going to "Thread.exit()".
The lock object's toString() says "unlocked" and it's "state" attribute is "0".
The behavior is not always the same. Sometimes the first thread goes past the lock as expected.
userLock.lock(); //first thread sometimes gets stuck here (and the following ones as well)
//"userLock" has "state=0" and toString() says "UNLOCKED"
try {
Transaction tr = HibernateConfig.getSessionFactory().getCurrentSession().beginTransaction();
try {
execute();
tr.commit();
} catch (ConstraintViolationException e) {
//probably traces with repeated time
System.err.println(e.getMessage());
if (tr.isActive()) {
tr.rollback();
}
} catch (RuntimeException e) {
e.printStackTrace();
if (tr.isActive()) {
tr.rollback();
}
}
} catch (Throwable e) {
e.printStackTrace();
} finally {
userLock.unlock();
}

try to put a breakpoint after userLock.lock(); then you should get the thread, that gets the lock.
alternatively you could use userLock.getOwner(); right behind .lock() to see wich thread got the lock.

The problem was my breakpoint was not before "lock()" like I said, but on it.
What happened is a bunch of threads would be blocked in that line by the breakpoint, one of them would still acquire the lock, and then the debugger would give me control over a random one of them which hadn't acquired the lock. And I was failing to check every thread blocked by the breakpoint to find the free one.
In the end I put the breakpoint actually before the lock and it behaved as expected.
This was confusing and I hope the question will still help someone.
Note: I'm still confused by the fact the lock's state said "unlocked" in the debugger while I was controlling a locked thread.

Making Daemon Thread to run till JVM Exits

After reading throw Daemon threads and implementing according to my requirements raised several doubts.
Please clarify me
I am using ServletContextListener class to invoke a Daemon thread which needs to run unitl JVM exits
public void contextInitialized (ServletContextEvent event) {
context = event.getServletContext();
//getting from spring context
MyServiceManager serviceManager = (MyServiceManager) ctx.getBean("myServiceManager");
serviceManager.setDaemon(true);
serviceManager.start();
}
in ServiceManager class I am running an infinite loop to run the program foever until JVM exists
public void run() {
try {
startService();
} catch (Exception e) {
logger.error("Error Occured in Background Process Runner");
}
}
private void startService(){
while(true){
try{
//invoke some new threads and do processing jobs until server/jvm stops
}catch(Exception e) {
//log but don't quit
}
}
}
}
The concern is, will daemon thread with the above implmentation runs foever? if not, what should i do to achieve my job. Unless JVM stops (server stopped), tell it to not to quit.
Note: I am trying my level best to format this post. but today something is going, it's not getting formatted :(

Your daemon thread will run until the run() method terminates, either by exiting normally or by throwing an exception (or until the VM exits, which is what you want). Since you catch all exceptions thrown by startService(), it will run until startService() returns or throws an exception, and will then exit normally.
Be aware that if startService() is interrupted while it is waiting for I/O, sleeping, or in a wait state, then it will generally throw an InterruptedException. Well-behaved threads usually exit when they are interrupted, as this is the normal method for telling a thread to exit.
Note that marking the thread as a daemon only means that the VM will not wait until the thread exits before it shuts down. None of the other behavior regarding thread termination is affected by the thread being a daemon or not.

Java: How do I catch InterruptedException on a thread, when interrupted by another thread?

I'm developing a multithreaded application to make connections to external servers - each on separate threads - and will be blocked until there is input. Each of these extends the Thread class. For the sake of explanation, let's call these "connection threads".
All these connection threads are stored in a concurrent hashmap.
Then, I allow RESTful web services method call to cancel any of the threads. (I'm using Grizzly/Jersey, so each call is a thread on its own.)
I retrieve the specific connection thread (from the hashmap) and call the interrupt() method on it.
So, here is the question, within the connection thread, how do I catch the InterruptedException? (I'd like to do something when the connection thread is stopped by an external RESTful command.)

So, here is the question, within the connection thread, how do I catch
the InterruptedException?
You can not. Since if your thread is blocked on a read I/O operation it can not be interrupted. This is because the interrupt just sets a flag to indicate that the thread has been interrupted. But if your thread has been blocked for I/O it will not see the flag.
The proper way for this is to close the underlying socket (that the thread is blocked to), then catch the exception and propagate it up.
So since your connection threads extend Thread do the following:
#Override
public void interrupt(){
try{
socket.close();
}
finally{
super.interrupt();
}
}
This way it is possible to interrupt a thread blocked on the I/O.
Then in your run method do:
#Override
public void run(){
while(!Thread.currentThread().isInterrupted()){
//Do your work
}
}
So in your case don't try to catch an InterruptedException. You can not interrupt the thread blocked on I/O. Just check if your thread has been interrupted and facilitate the interruption by closing the stream.

When you call Thread.interrupt() on some thread, what happens is that 'interruption' flag is set for that thread. Some methods do check this flag (by Thread.interrupted() or Thread.isInterrupted()) and throw InterruptedException, but usually only methods that can block do that. So there is no guarantee that InterruptedException will ever be thrown in interrupted thread. If you don't call any method that throws InterruptedException, there is no point in catching that exception, since it will not be thrown at all. However you can always check if your thread was interrupted by calling Thread.isInterrupted().

the problem it is with blocking.
Hoverer, try this code, maybe it will help you:
try{
yourObject.read();
}catch(InterruptedException ie){
// interrupted by other thread
}
catch(Exception ex){
// io or some other exception happent
}
your read method, should check if there is available buytes at socket for eg, if there are than read it, othervise go to speel mode. When is sleeping than is available the wake up (InterruptedException) at pur socket read ( whatever read have you) it will be blocked. Some API has a value to max waiting, eg 5 sec 60 sec, if nothing o read than it will be next code executed.
class MyReadingObject
{
public read() throws InterruptedException{
while(shouldIread){
if(socket.available() > 0){
byte[] buff = new byte[socket.avaialble()]
socket.read(buff);
return;
}
else{
Thread.currentThread.sleep(whateverMilliseconds);
}
}
}
}
something like that, but with error handling and some design patterns

Calling interrupt() on a thread doesn't stop it, it just switches on the interrupt flag. It's the responsibility of the code to handle the change in the interrupt status of the thread in consideration and act accordingly. If you are performing a blocking operation in that thread, you are pretty much SOL because your thread is "blocking" on the read. Have a look at the answer which I posted here. So basically, unless you are looping over stuff or periodically checking some flags inside that thread, you have no way of breaking out without closing sockets or stuff like that.
One solution here is to "explicitly" expose the underlying connection object and call close() on it, forcing it to throw some sort of exception, which can be then handled in the threaded code. Something like:
class MyAction extends Thread implements Disposable {
public void doStuff() {
try {
byte[] data = this.connection.readFully();
} catch (InterruptedException e) {
// possibly interrupted by forceful connection close
}
#Override
public void dispose() {
this.connection.close();
}
}
// Elsewhere in code
MyAction action = conMap.get("something");
action.dispose();

Use a try-catch like so:
try {
//code
} catch ( InterruptedException e) {
//interrupted
}
I think that should do the trick, you could also keep a boolean variable on whether to exit, so they would check that variable, if it's true, stop