We Keep Coding

Difference between completeExceptionally and obtrudeException

Just going through the CompletableFuture documentation and stumbled upon the completeExceptionally and obtrudeException methods and is having a hard time comprehending the difference and use case. Can the community help understand the difference and the use case with an example?

Explanation
The difference is subtle but important. From the official documentation:
completeExceptionally​
If not already completed, causes invocations of get() and related methods to throw the given exception.
obtrudeException
Forcibly causes subsequent invocations of method get() and related methods to throw the given exception, whether or not already completed. [...]
So they differ in their behavior regarding CompletableFutures that are already completed.
Basically, a future can either be completed or still pending (not completed). When you call completeExceptionally or obtrudeException, the behavior differs depending on the state of the future at that point in time.
Already completed future
Consider this example where the future is already completed at the moment of calling the method:
CompletableFuture<String> future = CompletableFuture.completedFuture("hello world");
future.completeExceptionally(new RuntimeException("Oh noes!"));
System.out.println(future.get()); // Prints "hello world" just fine
versus
CompletableFuture<String> future = CompletableFuture.completedFuture("hello world");
future.obtrudeException(new RuntimeException("Oh noes!"));
System.out.println(future.get()); // Throws the exception
Not completed future
And in case the future is not completed yet, they will both throw an exception:
CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
return "hello world";
});
future.completeExceptionally(new RuntimeException("Oh noes!"));
System.out.println(future.get());
and
CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
return "hello world";
});
future.obtrudeException(new RuntimeException("Oh noes!"));
System.out.println(future.get());
complete and obtrudeValue
Likewise there are also the methods complete and obtrudeValue which behave in the same way, but instead of throwing an exception, you can supply a value instead.
So complete basically completes the future with the given value, in case the future is not done yet, otherwise it does not do anything.
While obtrudeValue will supply the given value regardless, so it resets or cancels whatever the future already computed and replaces it by the given value instead.

completeExceptionally:
completableFuture.completeExceptionally(
new RuntimeException("Calculation failed!"));
//..
completableFuture.get(); //exception will be thrown whether `completableFuture` was not already completed.
obtrudeException:
completableFuture.obtrudeException(
new RuntimeException("Calculation failed!"));
//..
completableFuture.get(); //exception will be thrown **whether or not** `completableFuture` was completed.

Will all futures passed to CompletableFuture.allOf() run?

So I have a couple futures which I want to run, even if some fail I'd like all to have a chance to run. So if I do:
CompletableFuture.allOf(futures).join()
Will that be the case? My reasoning is that every future would have its own queed job in its executor and therefore all would run provided the main thread doesn't finish first. My issue is that I specifically .join() on .allOf() so my application doesnt end before running everything
So allOf() semantics confuse me: Will the future return complete when all of the passed futures complete regardless if successful? Or will it complete a failed future if it sees one failed without waiting for the rest?
EDIT
To illustrate my question further, does .allOf behaves like this:
Stream.of(futures).forEach(future -> {
try {
future.join()
} catch (Throwable e) {
//dont throw, we want to join the rest
}
})
Or does it behaves like the following:
Stream.of(futures).forEach(future -> {
try {
future.join()
} catch (Throwable e) {
throw e; //All other remaining .join() wont run
}
})
Which is it? first or second case? Since I want the first case thats what I'm using on my code temporarily, but I'd like to use allOf() if possible because its more aesthetic
Thanks!

Yes, each future will be independently attempted for completion.
I think you are also trying to understand how the control flows in various scenarios. I have come up with 4 scenarios :
A future where failure shall happens due to an unhandled exception
A future which shall be explicitly marked as failed with a completeExceptionally AND has an exceptionally block at its tail.
A future which shall be explicitly marked as failed with a completeExceptionally AND does not have an exceptionally block at its tail.
A future that shall complete to success.
//CASE 1
// A future that shall fail due to an unandled exception in its run
// and has an exceptionally block at its tail
CompletableFuture<Void> unhandledFailureFutureWithExceptionHandler =
CompletableFuture.runAsync(() -> {
throw new RuntimeException("Exception in unhandledFailureFutureWithExceptionHandler");
});
unhandledFailureFutureWithExceptionHandler = unhandledFailureFutureWithExceptionHandler
.exceptionally(throwable -> {
// Handling exception for this future
// HANDLING POINT 1
System.out.println("Handling exception at HANDLING POINT FOR CASE 1,
failure message is : " + throwable.getMessage());
return null;
});
//CASE 2
//A future that shall fail and has an exceptionally block at its tail
CompletableFuture<Void> failedFutureWithExceptionHandler = new CompletableFuture<>();
failedFutureWithExceptionHandler.completeExceptionally(
new RuntimeException("Exception in failedFutureWithExceptionHandler")
);
failedFutureWithExceptionHandler = failedFutureWithExceptionHandler.exceptionally(throwable -> {
// Handling exception for this future
// HANDLING POINT 2
System.out.println("Handling exception at HANDLING POINT FOR CASE 2,
failure message is : " + throwable.getMessage());
return null;
});
//CASE 3
//A future that shall fail and has no exceptionally block at its tail
CompletableFuture<Void> failedFutureWithoutExceptionHandler = new CompletableFuture<>();
failedFutureWithoutExceptionHandler.completeExceptionally(
new RuntimeException("Exception in failedFutureWithoutExceptionHandler")
);
//CASE 4
//A future that shall succeed and print a message to console
CompletableFuture<Void> successFuture = CompletableFuture.runAsync(() ->
System.out.println("CASE 4 : Running successFuture")
);
CompletableFuture.allOf(unhandledFailureFutureWithExceptionHandler,
failedFutureWithExceptionHandler, failedFutureWithoutExceptionHandler, successFuture)
.exceptionally(throwable -> {
// Handling exception if ANY of the futures that did not have its own exceptionally block
// In this case the exception of `failedFutureWithoutExceptionHandler` will be handled here
// HANDLING POINT 3
System.out.println("Handling exception at HANDLING POINT FOR CASE 3,
failure message is : " + throwable.getMessage());
return null;
}).join();
The output produced on the console is
Handling exception at HANDLING POINT FOR CASE 1, failure message is : java.lang.RuntimeException: Exception in unhandledFailureFutureWithExceptionHandler
Handling exception at HANDLING POINT FOR CASE 2, failure message is : Exception in failedFutureWithExceptionHandler
CASE 4 : Running successFuture
Handling exception at HANDLING POINT FOR CASE 3, failure message is : java.lang.RuntimeException: Exception in failedFutureWithoutExceptionHandler
As you can see if a future throws an unhandled error as in case 1, if it has a exceptionally block chained to its tail, the exception shall be handled at that point
As for case 2, in case where the future is marked as failed with completeExceptionally, if the future has a handler chained to its tail, then the exceptionally block shall be handled by that block
In case 3, the future is marked as failed and does not have an exceptionally block, thus it shall be handled by the exceptionally block at the next level, in this case it is the exceptionally block of the allOf().
As you can see, case 4 runs to completion and the message gets print on the console irrespective of the failures of the other futures.

Will exceptions be caught in following tasks?

try {
for (final Future<Boolean> bool : threadPool.invokeAll(tasks)) {
if (!bool.get()) {
return false;
}
}
return true;
} finally {
threadPool.shutdownNow();
threadPool.awaitTermination(
IMPORT_THREADS_AWAIT_TERMINATION_TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
}
Assume the task can throw an exception. Say there are 5 tasks, and the 5th task throws an exception, but the 1st task in the loop returns false. ( and loop breaks). If that task throws an exception, I want to catch it and do some cleanup logic. I am wondering my logic will never capture exception thrown by task 5 since i break from the loop on task 1
In such a case, will we ever encounter exception thrown by task 5 ?
Show threadPool.shutdownNow() throw the exception of task 5 ?

The exception will be thrown by task5, but the executor will catch it and wrap it as ExecutionException.
In this case, unless you call futureOfTask5.get(), the exception will not be re-thrown.
You can add try...catch block in the call method of task5 to log it.

CompletableFuture exception behavior with join() then get()

My intuition is that the following code is wrong. I believe because join() is being used, any exceptions throw while completing the futures will be unchecked. Then when get() is called, there will be no checked exceptions, no logging of any errors, and difficulty diagnosing errors during failure.
List<CompletableFuture> list = ImmutableList.of(future1, future2);
CompletableFuture.allOf(list.toArray(new CompletableFuture[list.size()])).join();
try {
result1 = future1.get();
result2 = future2.get();
} catch (InterruptedException | ExecutionException e) {
// will this ever run if join() is already called?
}
I have looked through the documentation for CompletableFuture but haven't found the exact answer to my question. I am asking here and will then go read through the source code.
The only why I can see that the catch block code would run is if somehow checked exceptions can be saved in some execution context and not thrown in join() (or thrown wrapped by an unchecked exception), and then throw again in some form after get(). This seems unlikely to me.
So my ultimate question is, will the catch block code ever run?

Both the join and the get method are blocking method that relies on completion signals and returns the result T. Processing the piece of code as in question :-
On one hand, InterruptedException could be thrown while the thread is interrupted in the process of waiting as we do a get, the wait here is already completed by the join method.
Also, as stated in the join method documentation
/**
* ... if a
* computation involved in the completion of this
* CompletableFuture threw an exception, this method throws an
* (unchecked) {#link CompletionException} with the underlying
* exception as its cause.
*/
So, on the other hand, the ExecutionException for futureN.get() in your case could only be thrown when and if the future completed exceptionally. Since the future if executed exceptionally would end up in throwing a CompletionException for the join call, it wouldn't reach the catch block ever or for that sake try block either.

Yes, the code would never be reached, but that doesn't make the "code wrong".
First, let's just try it out...
CompletableFuture<Integer> future1 = CompletableFuture.supplyAsync(() -> {
throw new IllegalArgumentException();
});
try
{
CompletableFuture.allOf(future1).join();
}
catch (Exception e1)
{
System.out.println("I'd exit here."); // *1
}
try
{
future1.get();
}
catch (InterruptedException | ExecutionException e)
{
System.out.println("Entered!");
}
Since you didn't do the try/catch "*1", the Exception would cause the method to exit and the get() would never be reached; so the second catch clause would never be executed.
However, the catch is still necessary because it's for the compiler, which has no way of knowing the previous call sequence.
The more straightforward way of doing this would be like this anyway:
CompletableFuture<Integer> future1 = CompletableFuture.supplyAsync(() -> {
throw new IllegalArgumentException();
});
try
{
CompletableFuture.allOf(future1).join();
future1.get();
}
catch (CompletionException e1) // this is unchecked, of course
{
System.out.println("Exception when joining");
}
catch (InterruptedException | ExecutionException e)
{
System.out.println("Exception when getting");
}

What does 'Thread termination due to failure' refer to?

The javadoc for ExecutorService sometimes refers to the case when a Thread terminates 'due to failure'. However, it is not clear what kind of failure does this refer to.
For instance, the single thread executor documentation says that
if this single thread terminates due to a failure during execution
prior to shutdown, a new one will take its place if needed to execute
subsequent tasks
I would have thought that this situation might happen in case of an Exception, or maybe a RuntimeException, but it does not seem to be the case. Running the following code seems to be giving the same thread name and thread ID.
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(() -> {
System.out.println("Hello from " + Thread.currentThread().getName()+ " " + Thread.currentThread().getId());
throw new NullPointerException("Test");
});
executor.submit(() -> {
System.out.println("Hello 2 from " + Thread.currentThread().getName() + " " + Thread.currentThread().getId());
});
The output of this code is:
Hello from pool-1-thread-1 12
Hello 2 from pool-1-thread-1 12
It seems that the same thread is being reused even in the case of NullPointerException.
So what kind of 'failure' is the Javadoc referring to?

This is an interesting question. Following the code in ThreadPoolExecutor the thread is discarded when a Runnable is passed to the execute() method.
When you call submit() the executor creates a wrapper for the callable/runnable of type FutureTask. FutureTask.run() has some logic to catch exceptions and store them (so then, you can query this from the Future). In this case, the exception never reaches the ThreadPool, so the thread is not discarded.

Augusto is right. Runnable tasks should have discarded the Thread after encountering the exception when they have passed as parameter in execute() method.
I have found concrete evidence regarding swallowing of exceptions by Future tasks at this article and Future Task source code
**Inside FutureTask$Sync**
void innerRun() {
if (!compareAndSetState(READY, RUNNING))
return;
runner = Thread.currentThread();
if (getState() == RUNNING) { // recheck after setting thread
V result;
try {
result = callable.call();
} catch (Throwable ex) {
setException(ex);
return;
}
set(result);
} else {
releaseShared(0); // cancel
}
}
protected void setException(Throwable t) {
sync.innerSetException(t);
}
There are few more interesting questions in SE around this topic.
Catching thread exceptions from Java ExecutorService
Choose between ExecutorService's submit and ExecutorService's execute
EDIT:
Thread failure or termination will happen when an exception is uncaught in the thread code. If you submit task by execute() instead of submit(), exception won't be caught unless you catch the exception. Uncaught exception by the thread code will result thread to terminate or failure and new thread will be created by Executor.
If you submit the task through submit(), a FutureTask will be created and that task will swallow uncaught exception by the code. Since the exception was caught in FutureTask, the thread won't be discarded.