We Keep Coding

CountdownLatch await() is not waiting for last thread to end

I wrote a small peice of program to demonstrate the usage of CountDownLatch class in java.
But, it not working as expected. I created 5 threads and assigned task to each thread. Now, each thread would wait for the start signal. Once the start signal is on, all thread start its work and call countDown(). Now, my main thread wait for all the thread to finish its work till it receives the done signal. But the output is not expected. Please help if I am missing anything in the concept.
Below is the program.
class Task implements Runnable{
private CountDownLatch startSignal;
private CountDownLatch doneSignal;
private int id;
Task(int id, CountDownLatch startSignal, CountDownLatch doneSignal){
this.startSignal = startSignal;
this.doneSignal = doneSignal;
this.id = id;
}
#Override
public void run() {
try {
startSignal.await();
performTask();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void performTask() {
try {
System.out.println("Task started by thread : " + id);
Thread.sleep(5000);
doneSignal.countDown();
System.out.println("Task ended by thread : " + id);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public class CountDownLatchExample {
public static void main(String[] args) {
CountDownLatch startSignal = new CountDownLatch(1);
CountDownLatch doneSignal = new CountDownLatch(5);
for(int i=0; i < 5; ++i) {
new Thread(new Task(i, startSignal, doneSignal)).start();
}
System.out.println("Press enter to start work");
new Scanner(System.in).nextLine();
startSignal.countDown();
try {
doneSignal.await();
System.out.println("All Tasks Completed");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Output
Press enter to start work
Task started by thread : 0
Task started by thread : 4
Task started by thread : 3
Task started by thread : 2
Task started by thread : 1
Task ended by thread : 4
Task ended by thread : 2
Task ended by thread : 1
All Tasks Completed
Task ended by thread : 0
Task ended by thread : 3
Expected output
Press enter to start work
Task started by thread : 0
Task started by thread : 4
Task started by thread : 3
Task started by thread : 2
Task started by thread : 1
Task ended by thread : 4
Task ended by thread : 2
Task ended by thread : 1
Task ended by thread : 0
Task ended by thread : 3
All Tasks Completed

In your Task class, you have:
doneSignal.countDown();
System.out.println("Task ended by thread : " + id);
In other words, you count down the latch before you print "task ended". That allows the main thread to wake up from its call to doneSignal.await() and print "All Tasks Completed" before all the "task ended" print statements complete. Though note the "wrong output" will not always happen; sometimes you'll get your expected output.
Simply switch those two lines of code around to guarantee the output you want:
System.out.println("Task ended by thread : " + id);
doneSignal.countDown();
This ensures the print statement happens-before the doneSignal.countDown() call, which itself happens-before the main thread returns from doneSignal.await(). Thus, now the above "task ended" print statement happens-before the main thread wakes up and prints the "All Tasks Completed" message.

How to correctly implement executor that runs multiple iterations and waits for all tasks to complete and successfully terminates after tasks are done

Cut to the chase short answer ---------------------
Code demonstrating the accepted answer can be found here:
Full example:
https://github.com/NACHC-CAD/thread-example/tree/shutdown-first
Implementation:
https://github.com/NACHC-CAD/thread-example/blob/shutdown-first/src/main/java/com/nachc/examples/threadexample/WidgetFactory.java
Original Post -------------------------------------
There are a number of examples of use of Java threads and Executors:
https://www.baeldung.com/thread-pool-java-and-guava
https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
https://howtodoinjava.com/java/multi-threading/java-thread-pool-executor-example/
https://jenkov.com/tutorials/java-concurrency/thread-pools.html
https://xperti.io/blogs/thread-pools-java-introduction/
https://www.journaldev.com/1069/threadpoolexecutor-java-thread-pool-example-executorservice
https://stackify.com/java-thread-pools/
However, I've not been able to successfully write an example that executes all of the tasks, waits for the tasks to complete, and then correctly terminates.
Working from this example: https://howtodoinjava.com/java/multi-threading/java-thread-pool-executor-example/
The code only calls executor.shutdown(). This does not allow the threads time to complete if they consume any time.
I've created a complete simplest example here: https://github.com/NACHC-CAD/thread-example/tree/await-termination
The shutdown only branch covers this use case (https://github.com/NACHC-CAD/thread-example/tree/shutdown-only):
public void makeWidgets() {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(batchSize);
log.info("Building " + howMany + " widgets...");
for (int i = 0; i < howMany; i++) {
Widget widget = new Widget(lotNumber, i);
WidgetRunnable runnable = new WidgetRunnable(widget);
executor.execute(runnable);
}
log.info("SHUTTING DOWN----------------");
executor.shutdown();
}
This code gives the following output (there should be 1000 widgets created and they should report that they are done after waiting 1 second).
2022-04-23 21:27:05,796 21:27:05.796 [main] INFO (WidgetFactoryIntegrationTest.java:12) - Starting test...
2022-04-23 21:27:05,799 21:27:05.799 [main] INFO (WidgetFactory.java:29) - Building 100 widgets...
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-2] INFO (Widget.java:24) - Starting build: 1/1
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-4] INFO (Widget.java:24) - Starting build: 1/3
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-1] INFO (Widget.java:24) - Starting build: 1/0
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-5] INFO (Widget.java:24) - Starting build: 1/4
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-6] INFO (Widget.java:24) - Starting build: 1/5
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-7] INFO (Widget.java:24) - Starting build: 1/6
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-8] INFO (Widget.java:24) - Starting build: 1/7
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-10] INFO (Widget.java:24) - Starting build: 1/9
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-9] INFO (Widget.java:24) - Starting build: 1/8
2022-04-23 21:27:05,801 21:27:05.801 [main] INFO (WidgetFactory.java:35) - SHUTTING DOWN----------------
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-3] INFO (Widget.java:24) - Starting build: 1/2
2022-04-23 21:27:05,801 21:27:05.801 [main] INFO (WidgetFactoryIntegrationTest.java:18) - Done.
If I add executor.awaitTermination the code runs all threads but never terminates. This example is in the await-termination branch: https://github.com/NACHC-CAD/thread-example/tree/await-termination
public void makeWidgets() {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(batchSize);
log.info("Building " + howMany + " widgets...");
for (int i = 0; i < howMany; i++) {
Widget widget = new Widget(lotNumber, i);
WidgetRunnable runnable = new WidgetRunnable(widget);
executor.execute(runnable);
}
try {
executor.awaitTermination(1000, TimeUnit.HOURS);
} catch(Exception exp) {
throw(new RuntimeException(exp));
}
log.info("SHUTTING DOWN----------------");
executor.shutdown();
}
This code lets all of the runnables finish but never exits. How do I let all of the runnables finish and have the code run to completion (exit)?

With reference to ThreadPoolExecutor documentation. The awaitTermination() method description reads:
Blocks until all tasks have completed execution after a shutdown request
While the shutdown() method descriptin reads
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted
Which indicates that awaitTermination() call is effective after a shutdown() call.
To solve the above problem, shutdown() needs to be called first and then awaitTermination()
NOTE: I have not personally tested this; however, John has, as mentioned in the comment of the original post and the mechanism works

The Answer by Ironluca is correct. Here is additional points and some example code.
For one thing, there is no need to declare & cast ThreadPoolExecutor directly. Just use the more general ExecutorService.
And using a thread pool sized to your batch size seems unwise. In current Java, you generally want an active thread pool to be less than the count of CPU cores. (This calculus will change radically if Project Loom and its virtual threads succeeds, but that is not the reality today, though you can try the early-access build.)
int threadPoolSize = 3 ; // Generally less than number of cores.
ExecutorService executorService = Executors.newFixedThreadPool( threadPoolSize );
Let's simplify your example scenario. We define Widget as a simple record.
record Widget ( UUID id , Instant whenCreated ) {}
Define a task that produces a Widget. We want to get back a Widget object, so we use Callable rather than Runnable.
Callable < Widget > makeWidgetTask = ( ) -> {
Thread.sleep( Duration.ofMillis( 50 ).toMillis() ); // Pretend that we have a long-running task.
Widget widget = new Widget( UUID.randomUUID() , Instant.now() );
return widget;
};
Make a big collection, to be used in running that task many times.
List < Callable < Widget > > tasks = Collections.nCopies( 1_000 , makeWidgetTask );
Actually, we need to wrap in a try-catch.
List < Future < Widget > > futures = null;
try
{
futures = executorService.invokeAll( tasks );
}
catch ( InterruptedException e )
{
throw new RuntimeException( e );
}
Submit all those tasks to the executor service. Notice how we get back a list of Future objects. A Future is our handle to the success or failure of each task’s completion.
As for how to wait for completion, and how to use ExecutorService#shutdown, shutdownNow, and awaitTermination, merely read the Javadoc. 👉 A full example of boilerplate code is provided for you.
To quote the Javadoc:
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ex) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
The key concept is that shutdown does not stop any work-in-progress. All tasks currently under execution will continue. All submitted tasks will eventually be scheduled for execution on a core as a thread becomes available. The shutdown method does only one thing: Stop any further tasks from being submitted to this executor service. To quote the Javadoc:
shutdown() … previously submitted tasks are executed, but no new tasks will be accepted.
To quote further:
This method does not wait for previously submitted tasks to complete execution. Use awaitTermination to do that.
So you need to call awaitTermination after calling shutdown. You pass arguments for a reasonable time in which you expect all submitted tasks to be completed or cancelled or interrupted. If that time limit elapses, then you can presume something has gone wrong.
Notice that the call to shutdown does not block, but the call to awaitTermination does block.
Let's adapt the boilerplate code to our own example.
executorService.shutdown(); // Disable new tasks from being submitted.
try
{
if ( ! executorService.awaitTermination( 60 , TimeUnit.SECONDS ) )
{
executorService.shutdownNow(); // Cancel currently executing tasks.
// Wait a while for tasks to respond to being cancelled.
if ( ! executorService.awaitTermination( 60 , TimeUnit.SECONDS ) )
{ System.err.println( "Executor service did not terminate." ); }
}
}
catch ( InterruptedException ex )
{
executorService.shutdownNow(); // (Re-)Cancel if current thread also interrupted
Thread.currentThread().interrupt(); // Preserve interrupt status
}
Finally, review our results by examining the collection of Future objects.
System.out.println( "Count futures: " + futures.size() );
for ( Future < Widget > future : futures )
{
if ( ! future.isDone() ) { System.out.println( "Oops! Task not done: " + future.toString() ); }
else if ( future.isCancelled() ) { System.out.println( "Bummer. Task cancelled: " + future.toString() ); }
else // Else task must have completed successfully.
{
try
{
Widget widget = future.get();
System.out.println( widget.toString() );
}
catch ( InterruptedException e )
{
throw new RuntimeException( e );
}
catch ( ExecutionException e )
{
throw new RuntimeException( e );
}
}
}
Add some elapsed time code at top and bottom.
long start = System.nanoTime();
…
System.out.println( "Elapsed: " + Duration.ofNanos( System.nanoTime() - start ) );
On my M1 MacBook Pro with 8 real cores, on Java 18, that takes about 18 seconds.
Count futures: 1000
Widget[id=56e594bf-75a6-4cf1-83fc-2b671873c534, whenCreated=2022-04-25T07:00:18.977719Z]
Widget[id=11373948-0689-467a-9ace-1e8d57f40f40, whenCreated=2022-04-25T07:00:18.977721Z]
…
Widget[id=d3b11574-6c11-41cc-9f26-c24ad53aa18c, whenCreated=2022-04-25T07:00:36.747058Z]
Widget[id=017ff453-da92-4296-992e-2c2a2ac44ed8, whenCreated=2022-04-25T07:00:36.748571Z]
Elapsed: PT17.906065583S
Full example code, for your copy-paste convenience.
package work.basil.example.threading;
import java.time.Duration;
import java.time.Instant;
import java.util.Collections;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.*;
public class App
{
public static void main ( String[] args )
{
long start = System.nanoTime();
int threadPoolSize = 3; // Generally less than number of cores.
ExecutorService executorService = Executors.newFixedThreadPool( threadPoolSize );
record Widget( UUID id , Instant whenCreated )
{
}
Callable < Widget > makeWidgetTask = ( ) -> {
Thread.sleep( Duration.ofMillis( 50 ).toMillis() ); // Pretend that we have a long-running task.
Widget widget = new Widget( UUID.randomUUID() , Instant.now() );
return widget;
};
List < Callable < Widget > > tasks = Collections.nCopies( 1_000 , makeWidgetTask );
List < Future < Widget > > futures = null;
try
{
futures = executorService.invokeAll( tasks );
}
catch ( InterruptedException e )
{
throw new RuntimeException( e );
}
executorService.shutdown(); // Disable new tasks from being submitted.
try
{
if ( ! executorService.awaitTermination( 60 , TimeUnit.SECONDS ) )
{
executorService.shutdownNow(); // Cancel currently executing tasks.
// Wait a while for tasks to respond to being cancelled.
if ( ! executorService.awaitTermination( 60 , TimeUnit.SECONDS ) )
{ System.err.println( "Executor service did not terminate." ); }
}
}
catch ( InterruptedException ex )
{
executorService.shutdownNow(); // (Re-)Cancel if current thread also interrupted
Thread.currentThread().interrupt(); // Preserve interrupt status
}
System.out.println( "Count futures: " + futures.size() );
for ( Future < Widget > future : futures )
{
if ( ! future.isDone() ) { System.out.println( "Oops! Task not done: " + future.toString() ); }
else if ( future.isCancelled() ) { System.out.println( "Bummer. Task cancelled: " + future.toString() ); }
else // Else task must have completed successfully.
{
try
{
Widget widget = future.get();
System.out.println( widget.toString() );
}
catch ( InterruptedException e )
{
throw new RuntimeException( e );
}
catch ( ExecutionException e )
{
throw new RuntimeException( e );
}
}
}
System.out.println( "Elapsed: " + Duration.ofNanos( System.nanoTime() - start ) );
}
}

How can I know who caused an interruptedExeption? (Java)

I'm using interrupt() in my code to signal from a thread to another to wake up from "endless" (Maximum time) sleep and verify a condition in a while.
I'm using also monitor (synchronized block, notify and wait) and synchronized method. I wrote my code in the way that some thread sleeps until they got an interrupt but some interrupt wake up thread when they should not be awaken (they must simulate they are doing other things sleeping). The problem is that I'm not able to find the thread that do interrupt() when it should not, how can I found it?
Is a good way to code using interrupt() in this way?
That's the code in which sleep get interrupted but should not
private void medicalVisit(int number) {
long sleepTime = (long) ((Math.random() * 2 + 0.5) * 1000); // 500 <= sleepTime (in msec) <= 2500
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
System.out.println(this.getName()+" ERROR, interrupt from sleep, id: 2 (medicalVisit)");
e.printStackTrace();
}
System.out.println(this.getName()+" - "+number+"° medical visit ended");
}
This is an example of code that launch an interrupt
private void handlerYellowPatient() {
Iterator<Patient> patientIt = yellows.iterator();
while(patientIt.hasNext()) {
Patient p = patientIt.next();
p.itsTurn = true;
p.interrupt();
yellows.remove(p);
}
}
And this an example of code "consuming" interrupt properly
private void waitUntilItsTurn(int number) {
// simulating the period of time before entering in guard
long sleepTime = (long) ((Math.random() * 2 + 0.5) * 1000); // 500 <= sleepTime (in msec) <= 2500
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
// must not be awaken while here
System.out.println(this.getName()+" ERROR MAYBE, interrupt from sleep, id: 1");
e.printStackTrace();
}
WMan.addPatient(this, WMan);
while (!itsTurn) {
try {
Thread.sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
// WMan handlerRedPatient interrupt#1
System.out.println(this.getName()+" - the wait is over, it's my turn for the "+number+"° times");
}
}
itsTurn = false;
}
Hoping these code can help

Active Time of Thread Java

How Can I get active time of Thread for which it was actually in running state. Minus all the Waiting and Sleeping time.
I didn't seem to find anything in Thread API that gives me desired results.
If not exact code any ideas or Pseudo Codes would be great start.
Thanks

Thanks assylias
The ThreadMXBean worked for me.. Here is sample code of what I did.
#Override
public void run() {
System.out.println(Thread.currentThread().getName());
try {
for(int i = 999999999; i > 0; i--){}
Thread.sleep(5000);
ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
long threadCpuTime = threadBean.getCurrentThreadCpuTime();
System.out.println(Thread.currentThread().getName() + " :: " + threadCpuTime);
} catch (InterruptedException e) {
e.printStackTrace();
return;
}
}
I Put up a sleep of 5 seconds just to check if it was added to CPU time. It wasn't added. Also I put up empty for loop so that processing takes some time.
Here's Output
Thread-1 :: 15600200
Thread-4 :: 15700100
Thread-3 :: 15600100
Thread-0 :: 15500100
Thread-2 :: 0

Start and stop Process Thread from Callable

I have a callable which starts a Thread(this Thread runs a ping process) I want to allow the user to cancel the tasks:
public class PingCallable implements Callable<PingResult> {
private ProcThread processThread;
public PingCallable(String ip) {
this.processThread = new ProcThread(ip);
}
#Override
public PingResult call() throws Exception {
log.trace("Checking if the ip " + ip + " is alive");
try {
processThread.start();
try {
processThread.join();
} catch (InterruptedException e) {
log.error("The callable thread was interrupted for " + processThread.getName());
processThread.interrupt();
// Good practice to reset the interrupt flag.
Thread.currentThread().interrupt();
}
} catch (Throwable e) {
System.out.println("Throwable ");
}
return new PingResult(ip, processThread.isPingAlive());
}
}
The ProcThread, looks something like:
#Override
public void run() {
try {
process = Runtime.getRuntime().exec("the long ping", null, workDirFile);
/* Get process input and error stream, not here to keep it short*/
// waitFor is InterruptedException sensitive
exitVal = process.waitFor();
} catch (InterruptedException ex) {
log.error("interrupted " + getName(), ex);
process.destroy();
/* Stop the intput and error stream handlers, not here */
// Reset the status, good practice
Thread.currentThread().interrupt();
} catch (IOException ex) {
log.error("Exception while execution", ex);
}
}
And the test:
#Test
public void test() throws ExecutionException, InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(15);
List<Future<PingResult>> futures = new ArrayList<>();
for (int i= 0; i < 100; i++) {
PingCallable pingTask = new PingCallable("10.1.1.142");
futures.add(executorService.submit(pingTask));
}
Thread.sleep(10000);
executorService.shutdownNow();
// for (Future<PingResult> future : futures) {
// future.cancel(true);
// }
}
I monitor the ping processes using ProcessExplorer, I see 15, then the shutdownNow is executed, or future.cancel(true), only 4-5 max 8 processes are interrupted, the rest are left alive, I almost never see 15 messages saying "The callable thread was interrupted..", and the test does not finish until the processes end. Why is that?

I might not have a complete answer but there are two things to note:
shutdownNow signals a shutdown, to see if threads are actually stopped, use awaitTermination
process.destroy() also takes time to execute so the callable should wait for that to complete after interrupting the process-thread.
I modified the code a little and found that future.cancel(true) will actually prevent execution of anything in the catch InterruptedException-block of ProcThread, unless you use executor.shutdown() instead of executor.shutdownNow(). The unit-test does finish when "Executor terminated: true" is printed (using junit 4.11).
It looks like using future.cancel(true) and executor.shutdownNow() will double-interrupt a thread and that can cause the interrupted-blocks to be skipped.
Below the code I used for testing. Uncomment for (Future<PingResult> f : futures) f.cancel(true); together with shutdown(Now) to see the difference in output.
public class TestRunInterrupt {
static long sleepTime = 1000L;
static long killTime = 2000L;
#Test
public void testInterrupts() throws Exception {
ExecutorService executorService = Executors.newFixedThreadPool(3);
List<Future<PingResult>> futures = new ArrayList<Future<PingResult>>();
for (int i= 0; i < 100; i++) {
PingCallable pingTask = new PingCallable("10.1.1.142");
futures.add(executorService.submit(pingTask));
}
Thread.sleep(sleepTime + sleepTime / 2);
// for (Future<PingResult> f : futures) f.cancel(true);
// executorService.shutdown();
executorService.shutdownNow();
int i = 0;
while (!executorService.isTerminated()) {
System.out.println("Awaiting executor termination " + i);
executorService.awaitTermination(1000L, TimeUnit.MILLISECONDS);
i++;
if (i > 5) {
break;
}
}
System.out.println("Executor terminated: " + executorService.isTerminated());
}
static class ProcThread extends Thread {
static AtomicInteger tcount = new AtomicInteger();
int id;
volatile boolean slept;
public ProcThread() {
super();
id = tcount.incrementAndGet();
}
#Override
public void run() {
try {
Thread.sleep(sleepTime);
slept = true;
} catch (InterruptedException ie) {
// Catching an interrupted-exception clears the interrupted flag.
System.out.println(id + " procThread interrupted");
try {
Thread.sleep(killTime);
System.out.println(id + " procThread kill time finished");
} catch (InterruptedException ie2) {
System.out.println(id + "procThread killing interrupted");
}
Thread.currentThread().interrupt();
} catch (Throwable t) {
System.out.println(id + " procThread stopped: " + t);
}
}
}
static class PingCallable implements Callable<PingResult> {
ProcThread pthread;
public PingCallable(String s) {
pthread = new ProcThread();
}
#Override
public PingResult call() throws Exception {
System.out.println(pthread.id + " starting sleep");
pthread.start();
try {
System.out.println(pthread.id + " awaiting sleep");
pthread.join();
} catch (InterruptedException ie) {
System.out.println(pthread.id + " callable interrupted");
pthread.interrupt();
// wait for kill process to finish
pthread.join();
System.out.println(pthread.id + " callable interrupt done");
Thread.currentThread().interrupt();
} catch (Throwable t) {
System.out.println(pthread.id + " callable stopped: " + t);
}
return new PingResult(pthread.id, pthread.slept);
}
}
static class PingResult {
int id;
boolean slept;
public PingResult(int id, boolean slept) {
this.id = id;
this.slept = slept;
System.out.println(id + " slept " + slept);
}
}
}
Output without future.cancel(true) or with future.cancel(true) and normal shutdown():
1 starting sleep
1 awaiting sleep
2 starting sleep
3 starting sleep
2 awaiting sleep
3 awaiting sleep
1 slept true
3 slept true
2 slept true
5 starting sleep
4 starting sleep
6 starting sleep
5 awaiting sleep
6 awaiting sleep
4 awaiting sleep
4 callable interrupted
Awaiting executor termination 0
6 callable interrupted
4 procThread interrupted
5 callable interrupted
6 procThread interrupted
5 procThread interrupted
Awaiting executor termination 1
6 procThread kill time finished
5 procThread kill time finished
4 procThread kill time finished
5 callable interrupt done
5 slept false
6 callable interrupt done
4 callable interrupt done
6 slept false
4 slept false
Executor terminated: true
Output with future.cancel(true) and shutdownNow():
1 starting sleep
2 starting sleep
1 awaiting sleep
2 awaiting sleep
3 starting sleep
3 awaiting sleep
3 slept true
2 slept true
1 slept true
4 starting sleep
6 starting sleep
5 starting sleep
4 awaiting sleep
5 awaiting sleep
6 awaiting sleep
5 callable interrupted
6 callable interrupted
4 callable interrupted
5 procThread interrupted
6 procThread interrupted
4 procThread interrupted
Executor terminated: true

Yesterday I ran a series of tests, one of the most fruitful involved:
Interrupting the threads which run the procces, checking that it was interrupted, and that the process nevertheless was still hanging on "waitFor",
I decided to investigate why was the process not detecting that the thread in which it was running was interrupted.
I found that it is crucial to handle the streams (output, input and error) correctly otherwise the external process will block on I/O buffer.
I noticed that my error handler was also blocking on reading (no error output), don't know if it's an issue, but I decided to follow the suggestion and redirect the err stream to out stream
Finally I discovered that there is a correct way to invoke and destroy processes in Java
New ProcThread (As #pauli suggests, it does not extend from THREAD anymore! Run's in a callable, I keep the name so the difference can be noticed) looks like:
try {
ProcessBuilder builder = new ProcessBuilder(cmd);
builder.directory(new File(workDir));
builder.redirectErrorStream(true);
process = builder.start();
// any output?
sht= new StreamHandlerThread(process.getInputStream(), outBuff);
sht.start();
// Wait for is InterruptedException sensitive, so when you want the job to stop, interrupt the thread.
exitVal = process.waitFor();
sht.join();
postProcessing();
log.info("exitValue: %d", exitVal);
} catch (InterruptedException ex) {
log.error("interrupted " + Thread.currentThread().getName(), ex);
shutdownProcess();
The shutdown process:
private void shutdownProcess() {
postProcessing();
sht.interrupt();
sht.join();
}
The postProcessing:
private void postProcessing() {
if (process != null) {
closeTheStream(process.getErrorStream());
closeTheStream(process.getInputStream());
closeTheStream(process.getOutputStream());
process.destroy();
}
}