We Keep Coding

Do I have to call join() for a future inside a whenComplete()?

I'm adding some code to an existing endpoint to send an email. We don't need the result of sending an email to return a response to the user, so I'm adding a .whenComplete() at the end of the chain of futures, calling our email service from within. The call to the email service is also async, returning a CompletionStage<Void>.
CompletionStage<SomeResponse> someEndpoint() {
return doThings()
.thenApply(things -> {
return someResponseFormat(things);
})
.whenComplete((someResponse, ex) -> {
if (ex == null) {
emailClient.sendEmail(someResponse); // CompletionStage<Void>
}
});
}
As I understand, that task will be scheduled and executed. Do I need to call join() on sendEmail(...)? Would doing so have a different behavior than not calling them? What is the best practice?
Edit: Originally I asked if I need to call join() or get(), which was misunderstood as "which do I need to call," when I meant, "do I need to call either at all."

The operation associated with emailClient.sendEmail(someResponse) will be scheduled regardless of whether you wait for its completion, so unless the JVM terminates in the meanwhile, it will complete. But
Nobody will notice when the operation completed or be able to wait for its completion.
Nobody will notice when the operation fails with an exception.
So what you probably want to do, is
CompletionStage<SomeResponse> someEndpoint() {
return doThings()
.thenApply(things -> someResponseFormat(things))
.thenCompose(someResponse -> emailClient.sendEmail(someResponse)
.thenApply(_void -> someResponse));
}
Then, when the caller of someEndpoint() invokes join() on it, the join() would wait for the completion of the sendEmail and also report errors when sendEmail fails. Likewise, when the caller of someEndpoint() chains dependent operations, they would start after the completion of sendEmail.

Do while loop behaving unexpectedly, for some inexplicable reason

I've been all over the internet and the Java docs regarding this one; I can't seem to figure out what it is about do while loops I'm not understanding. Here's the background: I have some message handler code that takes some JSON formatted data from a REST endpoint, parses it into a runnable task, then adds this task to a linked blocking queue for processing by the worker thread. Meanwhile, on the worker thread, I have this do while loop to process the message tasks:
do {
PublicTask currentTask = pubMsgQ.poll();
currentTask.run();
} while(pubMsgQ.size() > 0);
pubMsgQ is a LinkedBlockingQueue<PublicTask> (PublicTask implements the Runnable interface). I can't see any problems with this loop (obviously, or else I wouldn't be here), but this is how it behaves during execution: Upon entering the do block, pubMsgQ is polled and returns the runnable task as expected. The task is then run successfully with expected results, but then we get to the while statement. Now, according to the Java docs, poll() should return and remove the head of the queue, so I should expect that pubMsgQ.size() will return 0, right? Wrong I guess, because somehow the while statement passes and the program enters the do block again; of course this time pubMsgQ.poll() returns null (as I would have expected it should) and the program crashes with NullPointerException. What? Please explain like I'm five...
EDIT:
I decided to leave my original post as is above; because I think I actually explain the undesired behavior of that specific piece of the code quite succinctly (the loop is being executed twice while I'm fairly certain there is no way the loop should be executing twice). However, I realize that probably doesn't give enough context for that loop's existence and purpose in the first place, so here is the complete breakdown for what I am actually trying to accomplish with this code as I am sure there is a better way to implement this altogether anyways.
What this loop is actually a part of is a message handler class which implements the MessageHandler interface belonging to my Client Endpoint class [correction from my previous post; I had said the messages coming in were JSON formatted strings from a REST endpoint. This is technically not true: they are JSON formatted strings being received through a web socket connection. Note that while I am using the Spring framework, this is not a STOMP client; I am only using the built-in javax WebSocketContainer as this is more lightweight and easier for me to implement]. When a new message comes in onMessage() is called, which passes the JSON string to the MessageHandler; so here is the code for the entire MessageHandler class:
public class MessageHandler implements com.innotech.gofish.AutoBrokerClient.MessageHandler {
private LinkedBlockingQueue<PublicTask> pubMsgQ = new LinkedBlockingQueue<PublicTask>();
private LinkedBlockingQueue<AuthenticatedTask> authMsgQ = new LinkedBlockingQueue<AuthenticatedTask>();
private MessageLooper workerThread;
private CyclicBarrier latch = new CyclicBarrier(2);
private boolean running = false;
private final boolean authenticated;
public MessageHandler(boolean authenticated) {
this.authenticated = authenticated;
}
#Override
public void handleMessage(String msg) {
try {
//Create new Task and submit it to the message queue:
if(authenticated) {
AuthenticatedTask msgTsk = new AuthenticatedTask(msg);
authMsgQ.put(msgTsk);
} else {
PublicTask msgTsk = new PublicTask(msg);
pubMsgQ.put(msgTsk);
}
//Check status of worker thread:
if(!running) {
workerThread = new MessageLooper();
running = true;
workerThread.start();
} else if(running && !workerThread.active) {
latch.await();
latch.reset();
}
} catch(InterruptedException | BrokenBarrierException e) {
e.printStackTrace();
}
}
private class MessageLooper extends Thread {
boolean active = false;
public MessageLooper() {
}
#Override
public synchronized void run() {
while(running) {
active = true;
if(authenticated) {
do {
AuthenticatedTask currentTask = authMsgQ.poll();
currentTask.run();
if(GoFishApplication.halt) {
GoFishApplication.reset();
}
} while(authMsgQ.size() > 0);
} else {
do {
PublicTask currentTask = pubMsgQ.poll();
currentTask.run();
} while(pubMsgQ.size() > 0);
}
try {
active = false;
latch.await();
} catch (InterruptedException | BrokenBarrierException e) {
e.printStackTrace();
}
}
}
}
}
You may probably see where I'm going with this...what this Gerry-rigged code is trying to do is act as a facsimile for the Looper class provided by the Android Development Kit. The actual desired behavior is as messages are received, the handleMessage() method adds the messages to the queue for processing and the messages are processed on the worker thread separately as long as there are messages to process. If there are no more messages to process, the worker thread waits until it is notified by the handler that more messages have been received; at which point it resumes processing those messages until the queue is once again empty. Rinse and repeat until the user stops the program.
Of course, the closest thing the JDK provides to this is the ThreadPoolExecutor (which I know is probably the actual proper way to implement this); but for the life of me I couldn't figure out how to for this exact case. Finally, as a quick aside so I can be sure to explain everything fully, The reason why there are two queues (and a public and authenticated handler) is because there are two web socket connections. One is an authenticated channel for sending/receiving private messages; the other is un-authenticated and used only to send/receive public messages. There should be no interference, however, given that the authenticated status is final and set at construction; and each Client Endpoint is passed it's own Handler which is instantiated at the time of server connection.

You appear to have a number of concurrency / threading bugs in your code.
Assumptions:
It looks like there could be multiple MessageHandler objects, each with its own pair of queues and (supposedly) at most one MessageLooper thread. It also looks as if a given MessageHandler could be used by multiple request worker threads.
If that is the case, then one problem is that MessageHandler is not thread-safe. Specifically, the handleMessage is accessing and updating fields of the MessageHandler instance without doing any synchronization.
Some of the fields are initialized during object creation and then never changed. They are probably OK. (But you should declare them as final to be sure!) But some of the variables are supposed to change during operation, and they must be handled correctly.
One section that rings particular alarm bells is this:
if (!running) {
workerThread = new MessageLooper();
running = true;
workerThread.start();
} else if (running && !workerThread.active) {
latch.await();
latch.reset();
}
Since this is not synchronized, and the variables are not volatile:
There are race conditions if two threads call this code simultaneously; e.g. between testing running and assigning true to it.
If one thread sets running to true, there are no guarantees that a second thread will see the new value.
The net result is that you could potentially get two or more MessageLooper threads for a given set of queues. That breaks your assumptions in the MessageLooper code.
Looking at the MessageLooper code, I see that you have declared the run method as synchronized. Unfortunately, that doesn't help. The problem is that the run method will be synchronizing on this ... which is the specific instance of MessageLooper. And it will acquire the lock once and release it once. On short, the synchronized is wrong.
(For Java synchronized methods and synchronized blocks to work properly, 1) the threads involved need to synchronize on the same object (i.e. the same primitive lock), and 2) all read and write operations on the state guarded by the lock need to be done while holding the lock. This applies to use of Lock objects as well.)
So ...
There is no synchronization between a MessageLooper thread and any other threads that are adding to or removing from the queues.
There are no guarantees that the MessageLooper thread will notice changes to the running flag.
As I previously noted, you could have two or more MessageLooper polling the same pair of queues.
In short, there are lots of possible explanations for strange behavior in the code in the Question. This includes the specific problem you noticed with the queue size.
Writing correct multi-threaded code is difficult. This is why you should be using an ExecutorService rather than attempting to roll your own code.
But it you do need to roll your own concurrency code, I recommend buying and reading "Java: Concurrency in Practice" by Brian Goetz et al. It is still the only good textbook on this topic ...

How in WebFlux to stop publisher when request is aborted by client?

SpringBoot v2.5.1
There is an endpoint requesting a long running process result and it is created somehow
(for simplicity it is Mono.fromCallable( ... long running ... ).
Client make a request and triggers the publisher to do the work, but after several seconds client aborts the request (i.e. connection is lost). And the process still continues to utilize resources for computation of a result to throw away.
What is a mechanism of notifying Project Reactor's event loop about unnecessary work in progress that should be cancelled?
#RestController
class EndpointSpin {
#GetMapping("/spin")
Mono<Long> spin() {
AtomicLong counter = new AtomicLong(0);
Instant stopTime = Instant.now().plus(Duration.of(1, ChronoUnit.HOURS));
return Mono.fromCallable(() -> {
while (Instant.now().isBefore(stopTime)) {
counter.incrementAndGet();
if (counter.get() % 10_000_000 == 0) {
System.out.println(counter.get());
}
// of course this does not work
if (Thread.currentThread().isInterrupted()){
break;
}
}
return counter.get();
});
}
}

fromCallable doesn't shield you from blocking computation inside the Callable, which your example demonstrates.
The primary mean of cancellation in Reactive Streams is the cancel() signal propagated from downstream via the Subscription.
Even with that, the fundamental requirement of avoiding blocking code inside reactive code still holds, because if the operators are simple enough (ie. synchronous), a blocking step could even prevent the propagation of the cancel() signal...
A way to adapt non-reactive code while still getting notified about cancellation is Mono.create: it exposes a MonoSink (via a Consumer<MonoSink>) which can be used to push elements to downstream, and at the same time it has a onCancel handler.
You would need to rewrite your code to eg. check an AtomicBoolean on each iteration of the loop, and have that AtomicBoolean flipped in the sink's onCancel handler:
Mono.create(sink -> {
AtomicBoolean isCancelled = new AtomicBoolean();
sink.onCancel(() -> isCancelled.set(true));
while (...) {
...
if (isCancelled.get()) break;
}
});
Another thing that is important to note in your example: the AtomicInteger is shared state. If you subscribe a second time to the returned Mono, both subscriptions will share the counter and increment it / check it in parallel, which is probably not good.
Creating these state variables inside the Consumer<MonoSink> of Mono.create ensures that each subscription gets its own separate state.

Why android room inserts an object at the end of the asynctask generating it (the object)?

Summary
Room immediately inserts entities generated through UI, but delays those sent by an asynctask until the (far) end of the generating asynctask : the entity objects received are usable and displayed on UI, but without any id from database, hampering any other operation relying on id.
The insert operation happens only when the generating asynctask is properly stopped: Why? And how to solve this?
More context
The generating asynctask
We use an asynctask to monitor a socket and send back some events (as Room entity) to the application repository (as intended by android architecture components). This asynctask basically runs continuously in background (with some sleep regularly set) and is only stopped a while before the end of use of the application (if done right). So far it hasn't caused any issue for us to deviate so much from the original concept of short-lived asynctask.
I am pretty much aware we could improve the design, but this is another subject/question/time-hole ;-).
Room insert operation
Insertion happens through a dedicated asynctask, where the returned id of the entry in database is affected back to the entity just inserted (see code below). This is logged and entities from UI are persisted "immediately", they get back their ID and all is well. The asynctask-generated entities, well they wait for their "parent" task to stop and are then all inserted.
Entity composition
At first, the entity was generated inside the asynctask and sent through progress message. Then the construction of the object was moved outside of the asynctask and at the same level of the UI event construction, yet same behavior.
These events are some longs (timestamps) and several strings.
From the generating asynctask all starts from here:
#Override
protected void onProgressUpdate(OnProgressObject... values) {
OnProgressObject onProgressObject = values[0];
if (onProgressObject instanceof OnProgressEvent) {
eventRecipient.sendAutoEvent(((OnProgressEvent) onProgressObject).autoEvent);
}
}
The eventRecipient is the EventsRepository:
public void sendAutoEvent(AutoEvent autoEvent) {
Log.d(LOG_TAG, "got an autoevent to treat...");
EventModel newEvent = EventModel.fromCub(
autoEvent.cubTimeStamp,
autoEvent.description,
autoEvent.eventType
);
addEvent(newEvent);
}
public void addEvent(EventModel event) {
new insertEventAsyncTask(event).execute(event);
// other operations using flawlessly the "event"...
}
private class insertEventAsyncTask extends AsyncTask<EventModel, Void, Long> {
private EventModel eventModel;
public insertEventAsyncTask(EventModel eventModel) {
this.eventModel = eventModel;
}
#Override
protected Long doInBackground(EventModel... eventModels) {
// inserting the event "only"
return eventDao.insert(eventModels[0]);
}
#Override
protected void onPostExecute(Long eventId) {
super.onPostExecute(eventId);
// inserting all the medias associated to this event
// only one media is expected this way though.
eventModel.id = eventId;
Log.d(LOG_TAG, "event inserted in DB, got id : " + eventId);
}
}

I am pretty much aware we could improve the design, but this is another subject/question/time-hole
Since I suspect that it is the cause of your current problem, perhaps you should not dismiss this.
My interpretation of your problem is: you have an outer AsyncTask (the one with the onPublishProgress() method shown in the first code listing). You are executing that with execute(). Inside of that outer AsyncTask you have an inner AsyncTask (the one from your repository). You are executing that with execute(). And, your complaint is that the inner AsyncTask does not run until the outer AsyncTask completes.
If so, your problem is that execute() is single-threaded, and you are tying up that thread by having an AsyncTask run indefinitely. Until your outer AsyncTask completes its background work and returns from doInBackground(), the inner AsyncTask is blocked.
The "can we keep using hacks?" solution is to continue using AsyncTask but switch to executeOnExecutor() instead of execute(), supplying a thread pool to use. AsyncTask.THREAD_POOL_EXECUTOR would be a candidate.
The "OK, can we clean this up a little bit?" solution is to replace both AsyncTask instances with either simple Thread objects or the direct use of some multi-thread thread pool (see Executors). AsyncTask is obsolete, but to the extent that it is useful, only use it when you need to do work on the main application thread (onPostExecute()) after completing the background work (doInBackground()). Neither of your AsyncTask implementations need to be doing work on the main application thread after the background work is complete, so you do not need an AsyncTask for either of them. So, for example, your run-forever thread could be a Thread, while you use a thread pool inside of your repository for your DAO calls.
(the "hey, can we get modern on our threading, to go along with our use of Architecture Components?" solution is to switch to RxJava or Kotlin coroutines, in conjunction with LiveData — this is much more work, but they each have their own merits over manual thread management)

Why ?
Basically, it was written in the AsyncTask documentation : all asynctasks are executed serially on a unique background thread.
My code, even without nested asynctask, was blocking this thread with an almost never-ending task, delaying all database operations until its completion (or app crash, hence some data loss).
A quick solution : moving an AsyncTask to a Thread
Other alternatives were nicely listed by (CommonsWare)[https://stackoverflow.com/a/56925864/9138818], here are the steps I followed that solved this issue.
The main difficulty was to redirect code that was executed on UI thread (onPreExecute, onProgressUpdate, onPostExecute) through a Handler associated to the main thread.
First step was get a reference to a handler :
// Inside Runnable task's constructor :
// get the handler of the main thread (UI), needed for sending back data.
this.uiHandler = new Handler(Looper.getMainLooper());
Then, the "doInBackground" is refactored to fit a Runnable main method signature :
// previously "public String doInBackground()"
// returned value handled through publishProgress.
#Override
public void run() {
// recommended by Android Thread documentation
android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
// code previously in doInBackground
Now, code in the onProgressUpdate (that was called by publishProgress inside doInBackground method) was moved into a Runnable posted on the UI thread handler :
// asynctask method onProgressUpdate was renamed publishProgress =>
// doInBackground's body is almost untouched.
private void publishProgress(final OnProgressObject... values) {
uiHandler.post(new Runnable() {
#Override
public void run() {
// move here code previously in the AsyncTask's publishProgress()
}
});
}
At last, I had to change the way the task was created, runned and stopped by using Thread.interrupted instead of isCancelled and by creating the Runnable task before the thread :
public void startCUBMonitoring() {
if (autoEventThread == null) {
Log.d(LOG_TAG, "startCUBMonitoring");
addUIEvent("CUB MONITORING STARTED", "CUB_connexion");
SessionRepository sessionRepository =
ElabsheetApplication.getInstance().getSessionRepository();
// Creation of the task
AutoEventTask autoEventTask = new AutoEventTask(
this,
sessionRepository,
sessionRepository.getCUBConfig()
);
autoEventThread = new Thread(autoEventTask);
autoEventThread.start();
}
}
public void stopCUBMonitoring() {
if (autoEventThread != null) {
Log.d(LOG_TAG, "stopCUBMonitoring");
addUIEvent("CUB MONITORING STOPPED", "CUB_connexion");
autoEventThread.interrupt();
autoEventThread = null;
}
}
Hoped it could help...

Best practice for interrupting threads that take longer than a threshold

I am using the Java ExecutorService framework to submit callable tasks for execution.
These tasks communicate with a web service and a web service timeout of 5 mins is applied.
However I've seen that in some cases the timeout is being ignored and thread 'hangs' on an API call - hence, I want to cancel all the tasks that take longer than say, 5 mins.
Currently, I have a list of futures and I iterate through them and call future.get until all tasks are complete. Now, I've seen that the future.get overloaded method takes a timeout and throws a timeout when the task doesnt complete in that window. So I thought of an approach where I do a future.get() with timeout and in case of TimeoutException I do a future.cancel(true) to make sure that this task is interrupted.
My main questions
1. Is the get with a timeout the best way to solve this issue?
2. Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Any suggestions would be deeply appreciated.

Is the get with a timeout the best way to solve this issue?
This will not suffice. For instance, if your task is not designed to response to interruption, it will keep on running or be just blocked
Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Yes, You might end up cancelling as task which is never scheduled to run if your thread-pool is not configured properly
Following code snippet could be one of the way you can make your task responsive to interruption when your task contains Non-interruptible Blocking. Also it does not cancel the task which are not scheduled to run. The idea here is to override interrupt method and close running tasks by say closing sockets, database connections etc. This code is not perfect and you need to make changes as per requirements, handle exceptions etc.
class LongRunningTask extends Thread {
private Socket socket;
private volatile AtomicBoolean atomicBoolean;
public LongRunningTask() {
atomicBoolean = new AtomicBoolean(false);
}
#Override
public void interrupt() {
try {
//clean up any resources, close connections etc.
socket.close();
} catch(Throwable e) {
} finally {
atomicBoolean.compareAndSet(true, false);
//set the interupt status of executing thread.
super.interrupt();
}
}
public boolean isRunning() {
return atomicBoolean.get();
}
#Override
public void run() {
atomicBoolean.compareAndSet(false, true);
//any long running task that might hang..for instance
try {
socket = new Socket("0.0.0.0", 5000);
socket.getInputStream().read();
} catch (UnknownHostException e) {
} catch (IOException e) {
} finally {
}
}
}
//your task caller thread
//map of futures and tasks
Map<Future, LongRunningTask> map = new HashMap<Future, LongRunningTask>();
ArrayList<Future> list = new ArrayList<Future>();
int noOfSubmittedTasks = 0;
for(int i = 0; i < 6; i++) {
LongRunningTask task = new LongRunningTask();
Future f = execService.submit(task);
map.put(f, task);
list.add(f);
noOfSubmittedTasks++;
}
while(noOfSubmittedTasks > 0) {
for(int i=0;i < list.size();i++) {
Future f = list.get(i);
LongRunningTask task = map.get(f);
if (task.isRunning()) {
/*
* This ensures that you process only those tasks which are run once
*/
try {
f.get(5, TimeUnit.MINUTES);
noOfSubmittedTasks--;
} catch (InterruptedException e) {
} catch (ExecutionException e) {
} catch (TimeoutException e) {
//this will call the overridden interrupt method
f.cancel(true);
noOfSubmittedTasks--;
}
}
}
}
execService.shutdown();

Is the get with a timeout the best way to solve this issue?
Yes it is perfectly fine to get(timeout) on a Future object, if the task that the future points to is already executed it will return immediately. If the task is yet to be executed or is being executed then it will wait until timeout and is a good practice.
Is there the possibility that I'm waiting with the get call on a task
that hasnt yet been placed on the thread pool(isnt an active worker)
You get Future object only when you place a task on the thread pool so it is not possible to call get() on a task without placing it on thread pool. Yes there is a possibility that the task has not yet been taken by a free worker.

The approach that you are talking about is ok. But most importantly before setting a threshold on the timeout you need to know what is the perfect value of thread pool size and timiout for your environment. Do a stress testing which will reveal whether the no of worker threads that you configured as part of Threadpool is fine or not. And this may even reduce the timeout value. So this test is most important i feel.
Timeout on get is perfectly fine but you should add to cancel the task if it throws TimeoutException. And if you do the above test properly and set your thread pool size and timeout value to ideal than you may not even need to cancel tasks externally (but you can have this as backup). And yes sometimes in canceling a task you may end up canceling a task which is not yet picked up by the Executor.

You can of course cancel a Task by using
task.cancel(true)
It is perfectly legal. But this will interrupt the thread if it is "RUNNING".
If the thread is waiting to acquire an intrinsic lock then the "interruption" request has no effect other than setting the thread's interrupted status. In this case you cannot do anything to stop it. For the interruption to happen, the thread should come out from the "blocked" state by acquiring the lock it was waiting for (which may take more than 5 mins). This is a limitation of using "intrinsic locking".
However you can use explicit lock classes to solve this problem. You can use "lockInterruptibly" method of the "Lock" interface to achieve this. "lockInterruptibly" will allow the thread to try to acquire a lock while remaining responsive to the interruption. Here is a small example to achieve that:
public void workWithExplicitLock()throws InterruptedException{
Lock lock = new ReentrantLock();
lock.lockInterruptibly()();
try {
// work with shared object state
} finally {
lock.unlock();
}
}