We Keep Coding

Stop ExecutorService on thread failure and exception handling

This is a simplified example I did to expose my problem. I have some task doSomeWork() that I handle in a multihreading fashion using ExecutorService (4 threads at a time max). However, if any of the threads/tasks generates an exception, I would like to:
Stop any further tasks from being processed.
Catch the exception at the main thread level.
public static void main(String[] args) {
final ExecutorService threadPool = Executors.newFixedThreadPool(4);
final ExecutorCompletionService<Void> completionService = new ExecutorCompletionService<>(threadPool);
try {
for (int i = 0; i < 10; i++) {
int b = i;
completionService.submit(() -> doSomeWork(b));
}
threadPool.shutdown();
threadPool.awaitTermination(8, TimeUnit.HOURS);
System.exit(0);
} catch (Exception e) {
System.out.println("Something wrong happened: " + e.getMessage());
}
System.exit(1);
}
//This function have 50% odds of throwing an exception
public static Void doSomeWork(int i) throws Exception {
Thread.sleep(500);
if ((Math.random() > 0.5))
{
System.out.println("I have reached indice: " + i);
}
else
{
throw new Exception("I couldn't handle indice " + i);
}
return null;
}
Currently, an execution would output something like this:
I have reached indice: 0
I have reached indice: 2
I have reached indice: 1
I have reached indice: 4
I have reached indice: 6
I have reached indice: 7
I have reached indice: 5
I have reached indice: 9
As you can see, indice 3 is missing, nevertheless the execution of the remaining threads completed. It also didn't output anything about the exception.
My desired output would be something like this:
I have reached indice: 0
I have reached indice: 2
I have reached indice: 1
Something wrong happened: I couldn't handle indice 3
Other solutions I found around this problem were using a callable with a future but in a blocking fashion. I can't block the execution of other threads while waiting for the future otherwise this whole multithreading is pointless.

You can do that using CompletableFuture. This is your main function I tested:
final ExecutorService executorService = Executors.newFixedThreadPool(4);
final List<CompletableFuture<Void>> all = new ArrayList<>();
try {
for (int i = 0; i < 10; i++) {
int b = i;
CompletableFuture<Void> v = CompletableFuture.runAsync(() -> {
try {
doSomeWork(b);
} catch (Exception e) {
throw new RuntimeException(e);
}
},
executorService);
all.add(v);
}
CompletableFuture<Void> placeholder = CompletableFuture.allOf(all.toArray(new CompletableFuture[0]));
failFast(all, placeholder);
System.out.println("All tasks ended");
} catch (Exception e) {
System.out.println("Something wrong happened: " + e.getMessage());
} finally {
executorService.shutdownNow();
}
Utility function to make the joint future fail as soon as one of them is failed (or when all of them are completed):
private static <T> void failFast(List<CompletableFuture<T>> futures, CompletableFuture<T> joint) {
while (true) {
if (joint.isDone()) {
return;
}
for (CompletableFuture<T> future : futures) {
if (future.isCompletedExceptionally()) {
return;
}
}
}
}
And this is the output I get:
I have reached indice: 1
I have reached indice: 7
I have reached indice: 5
I have reached indice: 4
Something wrong happened: java.lang.RuntimeException: java.lang.Exception: I couldn't handle indice 0
Explanation:
The method CompletableFuture.runAsync() allows you to provide with a Runnable (your doSomeWork) and an executor with a certain number of threads. Here, I pass an executor with 4 threads (as you did in your example).
Inside the runnable, I don't only run the doSomeWork function but I also catch Exception and throw a RuntimeException (need to do that because Lambdas do not support checked exceptions, so I need to wrap it into a runtime one but it will still interrupt execution and be catched by your main).
Each time I create a new CompletableFuture<Void> for the task with the given index i, I will store this result into a list of completable futures.
The for loop will take nothing to execute, since the completable futures run asynchronously.
Hence, I create a joint completable future with CompletableFuture.allOf(...) and then I use the utility function failFast on this future in order to stop as soon as one of the task is failed (or to continue until all of them are complete).
So basically as soon as one of the futures fails throwing an exception, the joint future is considered to be completed and will hence leave the handle to your main thread which is, meanwhile, being hit by the RuntimeException that was thrown inside the lambda expression.
Note: thanks to Thomas' comment, I've updated the code to use an ExecutorService instead of a simple Executor. That allows you to have a call to shutdownNow() inside your finally block after you catch the exception.
As Thomas suggests, also, you may directly throw a RuntimeException inside your doSomeWork function so you don't need to catch and wrap inside the lambda expression.
Other note: #matt made me notice something I didn't know. The .allOf() future will be completed when ALL futures are completed, whether successfully or not.
Hence, as he pointed out, my solution wouldn't work as is. I've edited again the answer to take his comment into account, thanks #matt for making me notice.

It sounds like you've excluded a correct way to do this, based on an incorrect assumption. Keep your futures.
List<Future<?>> futures = new ArrayList<>();
Then when you submit.
futures.add( completionService.submit( () -> doSomeWork(b) ) );
Now, you can check the futures status in your main thread.
for(Future<?> f: futures){
try{
f.get();
} catch( ExecutionException e){
//execution exception handled on the main thread.
completionService.shutdownNow();
} catch( InterruptedException ie){
//what should happen here.
}
}
That way, shutdownNow gets called, so all of the non-started tasks get returned and will not start.
You could use a timeout with get to check every task since some will be running in parallel.
Here is a complete compilable example.
import java.util.concurrent.*;
import java.util.*;
public class ExecutorJunk{
static int count = 0;
static void task(){
int z = count++;
if(z == 3){
throw new RuntimeException("z is 3");
}
System.out.println("z: " + z);
try{ Thread.sleep(1500);} catch(InterruptedException e){};
}
public static void main(String[] args){
ExecutorService service = Executors.newFixedThreadPool(4);
List<Future<?>> all = new ArrayList<>();
for(int i = 0; i<10; i++){
all.add( service.submit(ExecutorJunk::task) );
}
service.shutdown();
try{
while(!service.isTerminated()){
for(Future f: all){
try{
f.get(1, TimeUnit.MILLISECONDS);
} catch( TimeoutException toe){
//pass.
}
}
}
} catch(Exception e){
System.out.println( service.shutdownNow().size() + " tasks not started");
e.printStackTrace();
}
}
}
When I run this I get.
z: 0
z: 1
z: 2
z: 4
5 tasks not started
java.util.concurrent.ExecutionException: java.lang.RuntimeException: z is 3
...
It probably could be done a bit smarter. Such as purging the list of futures as get succeeds, instead of using a timeout just check the futures if they're done, then doing a future.get.

How to enforce timeout and cancel async CompletableFuture Jobs

I am using Java 8, and I want to know the recommended way to enforce timeout on 3 async jobs that I would to execute async and retrieve the result from the future. Note that the timeout is the same for all 3 jobs. I also want to cancel the job if it goes beyond time limit.
I am thinking something like this:
// Submit jobs async
List<CompletableFuture<String>> futures = submitJobs(); // Uses CompletableFuture.supplyAsync
List<CompletableFuture<Void>> all = CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
try {
allFutures.get(100L, TimeUnit.MILLISECONDS);
} catch (TimeoutException e){
for(CompletableFuture f : future) {
if(!f.isDone()) {
/*
From Java Doc:
#param mayInterruptIfRunning this value has no effect in this
* implementation because interrupts are not used to control
* processing.
*/
f.cancel(true);
}
}
}
List<String> output = new ArrayList<>();
for(CompeletableFuture fu : futures) {
if(!fu.isCancelled()) { // Is this needed?
output.add(fu.join());
}
}
return output;
Will something like this work? Is there a better way?
How to cancel the future properly? Java doc says, thread cannot be interrupted? So, if I were to cancel a future, and call join(), will I get the result immediately since the thread will not be interrupted?
Is it recommended to use join() or get() to get the result after waiting is over?

It is worth noting that calling cancel on CompletableFuture is effectively the same as calling completeExceptionally on the current stage. The cancellation will not impact prior stages. With that said:
In principle, something like this will work assuming upstream cancellation is not necessary (from a pseudocode perspective, the above has syntax errors).
CompletableFuture cancellation will not interrupt the current thread. Cancellation will cause all downstream stages to be triggered immediately with a CancellationException (will short circuit the execution flow).
'join' and 'get' are effectively the same in the case where the caller is willing to wait indefinitely. Join handles wrapping the checked Exceptions for you. If the caller wants to timeout, get will be needed.
Including a segment to illustrate the behavior on cancellation. Note how downstream processes will not be started, but upstream processes continue even after cancellation.
public static void main(String[] args) throws Exception
{
int maxSleepTime = 1000;
Random random = new Random();
AtomicInteger value = new AtomicInteger();
List<String> calculatedValues = new ArrayList<>();
Supplier<String> process = () -> { try { Thread.sleep(random.nextInt(maxSleepTime)); System.out.println("Stage 1 Running!"); } catch (InterruptedException e) { e.printStackTrace(); } return Integer.toString(value.getAndIncrement()); };
List<CompletableFuture<String>> stage1 = IntStream.range(0, 10).mapToObj(val -> CompletableFuture.supplyAsync(process)).collect(Collectors.toList());
List<CompletableFuture<String>> stage2 = stage1.stream().map(Test::appendNumber).collect(Collectors.toList());
List<CompletableFuture<String>> stage3 = stage2.stream().map(Test::printIfCancelled).collect(Collectors.toList());
CompletableFuture<Void> awaitAll = CompletableFuture.allOf(stage2.toArray(new CompletableFuture[0]));
try
{
/*Wait 1/2 the time, some should be complete. Some not complete -> TimeoutException*/
awaitAll.get(maxSleepTime / 2, TimeUnit.MILLISECONDS);
}
catch(TimeoutException ex)
{
for(CompletableFuture<String> toCancel : stage2)
{
boolean irrelevantValue = false;
if(!toCancel.isDone())
toCancel.cancel(irrelevantValue);
else
calculatedValues.add(toCancel.join());
}
}
System.out.println("All futures Cancelled! But some Stage 1's may still continue printing anyways.");
System.out.println("Values returned as of cancellation: " + calculatedValues);
Thread.sleep(maxSleepTime);
}
private static CompletableFuture<String> appendNumber(CompletableFuture<String> baseFuture)
{
return baseFuture.thenApply(val -> { System.out.println("Stage 2 Running"); return "#" + val; });
}
private static CompletableFuture<String> printIfCancelled(CompletableFuture<String> baseFuture)
{
return baseFuture.thenApply(val -> { System.out.println("Stage 3 Running!"); return val; }).exceptionally(ex -> { System.out.println("Stage 3 Cancelled!"); return ex.getMessage(); });
}
If it is necessary to cancel the upstream process (ex: cancel some network call), custom handling will be needed.

After calling cancel you cannot join the furure, since you get an exception.
One way to terminate the computation is to let it have a reference to the future and check it periodically: if it was cancelled abort the computation from inside.
This can be done if the computaion is a loop where at each iteration you can do the check.
Do you need it to be a CompletableFuture? Cause another way is to avoid to use a CompleatableFuture, and use a simple Future or a FutureTask instead: if you execute it with an Executor calling future.cancel(true) will terminate the computation if possbile.
Answerring to the question: "call join(), will I get the result immediately".
No you will not get it immediately, it will hang and wait to complete the computation: there is no way to force a computation that takes a long time to complete in a shorter time.
You can call future.complete(value) providing a value to be used as default result by other threads that have a reference to that future.

How to make manual retry thread safe

I am trying to do a manual retry. But I feel the code is not thread safe.
Can anyone please provide suggestion on how to make it thread safe
while (retryCounter < maxRetries) {
try {
//TODO - add delay with config
Thread.sleep(3000);
t.run();
break;
} catch (Exception e) {
retryCounter++;
//TODO - Add audit logs
if (retryCounter >= maxRetries) {
LOG.info("Max retries exceeded");
//TODO - remove exception add audit logs
throw new RuntimeException("Max retry exceeded");
}
}
}
Thanks in advance

As pointed out by # SteffenJacobs, t.run does not execute the run logic on a separate thread but rather on the thread that makes the invocation. If you replace t.run with t.start then the run logic will be executed on a different thread asynchronously which means that exceptions in that new thread will never be handled by your catch block. For example the code below:
public static void main(String[] args) {
int retryCounter = 0;
int maxRetries = 3;
Thread t = new Thread(new Runnable() {
public void run() {
System.out.println("..." + Thread.currentThread());
throw new RuntimeException("Thrown by me!!!");
}});
while (retryCounter < maxRetries) {
try {
Thread.sleep(3000);
System.out.println("***" + Thread.currentThread());
t.start();
break;
} catch (Exception e) {
System.out.println("retrying attempt " + retryCounter);
System.out.println(e);
retryCounter++;
if (retryCounter >= maxRetries) {
throw new RuntimeException("Max retry exceeded");
}
} finally {
System.out.println("in finally");
}
}
}
prints:
***Thread[main,5,main]
in finally
...Thread[Thread-0,5,main]
Exception in thread "Thread-0" java.lang.RuntimeException: Thrown by me!!!
...
Process finished with exit code 0
Bottom line - To detect errors in your threaded tasks, you will need to think of a different approach.
You could consider using Futures (https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html). In this case your tasks will need to return a status of their execution while the code that launched them waits on Future.get , examines the status and reruns the task as needed.

The answer of #David Soroko is very close, but I think it need little modification. As your code would run under multi-threaded environment, we should take care of Synchronization
If multiple thread tries to enter while loop, it may happen that 2 invocations of Retry Thread would get started, with 2 calls of t.start(). Ideally, we should start only single invocation during single iteration of while loop.
Little modification :
Declare LOCK object in your class level
// Object to ensure thread safety between multiple threads
private final LOCK = new Object();
Use LOCK object with synchronized block in while loop
synchronized(LOCK) { // This will ensure single thread entry at a time
while (retryCounter < maxRetries) {
try {
Thread.sleep(3000);
System.out.println("***" + Thread.currentThread());
t.start();
break;
} catch (Exception e) {
System.out.println("retrying attempt " + retryCounter);
System.out.println(e);
retryCounter++;
if (retryCounter >= maxRetries) {
throw new RuntimeException("Max retry exceeded");
}
} finally {
System.out.println("in finally");
}
}
}
The variables retryCounter and maxRetries are read by multiple threads. This means they are shared resources between threads. We need to ensure thread safety there as well. There is concept of Atomicity and Volatility.
Atomicity : When any thread T1 is doing operation on Variable A, at the same time, if other thread T2 tries to do operation on A, either operation of T1 should get completed fully or operation of T1 should be aborted fully, before starting operation of T2.
Volatility : When multiple threads are accessing single variable A, all threads should read the actual value from memory. In multithreaded environment, depending on processor, threads do optimisation while reading variable resources like A. Optimised value of A may not match actual value of A in memory. This may lead to Race condition or misbehaviour of your business logic. In your case, the Retry operation may run more than Max Retry count in case of misbehaviour.
Make both shared resources to comply Volatile and Atomic properties
final AtomicInteger retryCounter = new AtomicInteger(0);
final AtomicInteger maxRetries = new AtomicInteger(3);

Do I need to call join() when stopping a Thread in Java?

I have these 5 simple thread that run a while loop:
flasherThread = new Thread(new Runnable() {
#Override
public void run() {
while(running.get()) {
// do network stuff
}
}
});
running is declared as private final AtomicBoolean running;.
I have this method:
public void stopFlasherThread() {
running.set(false);
}
My question is by setting the flag to false that stops the thread immediately ? Or do I need to call flasherThread.join() to make sure that the thread has stopped ?
The main issue is that I have 4-5 of these at a time.
So I have a loop such as:
for (int i = 0; i < 5; i++) {
ThreadArrayList.get(i).stopFlasherThread();
ThreadArrayList.get(i).join() // should I do this ?
}
Any help would be great! Thanks

According to the official documentation on join:
The join method allows one thread to wait for the completion of another. If t is a Thread object whose thread is currently executing,
t.join();
causes the current thread to pause execution until t's thread terminates.
So, no... or not necessarily, only if you need the result of the work of that thread to do something. The join will not stop / interrupt the thread, it will wait for it to finish its work. The stopFlasherThread will make the loop stop.
I would advise you to follow a different approach on using threads on Java using ExecutorService. For example:
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<AtomicInteger> futureResult = executor.submit(new Callable<AtomicInteger>() {
#Override
public AtomicInteger call() {
// Here I return a random integer, but you can do your proper calculation
AtomicInteger atomicInteger =
new AtomicInteger(ThreadLocalRandom.current().nextInt());
System.out.println(Thread.currentThread().getName() + " " + atomicInteger);
return atomicInteger;
}
});
// Thread returns result, but continues to execute as it is a single thread pool
try {
System.out.println(Thread.currentThread().getName() + " " + futureResult.get());
} catch (InterruptedException e) {
// Handle exception properly
e.printStackTrace();
} catch (ExecutionException e) {
// Handle exception properly
e.printStackTrace();
}
// Stop all threads
executor.shutdownNow();
There I define an inline class that extends the Callable interface and implement the call method to perform a task in another thread. This returns the result of the computation in the variable futureResult which is a Future. Since executor is a thread pool, it continues to be available to take tasks even though our task here has already been resolved. To finish the whole thread pool loop you can do a executor.shutdownNow().

ExecutorService.invokeAll and shutdown

So I have some Callable tasks, sensitive to interruptions, which I submit to the ExecutorService using invokeAll. After 5 seconds from another method I call executorService.shutdownNow after which I call the awaitTermination, which returns true, so all seems good. The problem is the executor never terminates.
Due to logging I know that each one of my tasks finished.
nevertheless the invokeAll still blocks on f.get when i is equal to the number of threads of the executor:
The following code is obtained from AbstractExecutorService + some logging.
#Override
public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks) throws InterruptedException {
if (tasks == null) throw new NullPointerException();
ArrayList<Future<T>> futures = new ArrayList<Future<T>>(tasks.size());
boolean done = false;
try {
List<Callable<T>> list = new ArrayList<Callable<T>>();
for (Callable<T> t : tasks) {
list.add(t);
RunnableFuture<T> f = newTaskFor(t);
futures.add(f);
execute(f);
}
for (int i = 0, size = futures.size(); i < size; i++) {
Future<T> f = futures.get(i);
if (!f.isDone()) {
log.info("Future %s is not done!. Task %s", i, list.get(i));
try {
log.info("Get from future %s", i);
// NEXT LINE BLOCKS FOR i= NUMBER OF THREADS
f.get();
log.info("Got result from future %s", i);
} catch (CancellationException ignore) {
} catch (ExecutionException ignore) {
}
}
}
log.info("Obtained all!");
done = true;
return futures;
} finally {
if (!done) for (int i = 0, size = futures.size(); i < size; i++)
futures.get(i).cancel(true);
}
}
Am I not suppose to use invokeAll with shutdown? I guess not, after all they are in the same class. Why does it get blocked, only when i= the number of threads of the executor?

Yes, you're not suppose to use invokeAll with shutdown. At least this is what I understand, correct me if I'm wrong.
The shutdownNow method:
public List<Runnable> shutdownNow() {
...
checkShutdownAccess();
advanceRunState(STOP);
interruptWorkers();
tasks = drainQueue();
...
}
The only thing is does is interrupt working threads and remove the rest of the runnables from the working queue, see drainQueue. ShutdownNow/Shutdown does not modify the futures in our invokeAll method
So what happens in my case is that for an Executor with N threads, I invoke 300 jobs, each of them take more than 1 minute, after 5 seconds I cancel (interrupt working threads), N threads are interrupted (0 to N-1). What happens with the rest of the futures? Nothing, the next call to f.get() (see corresponding line in the question) will block and you're stuck there. This explains why I'm always blocked on i = Number of threads.