We Keep Coding

io.smallrye.mutiny.Uni multi threading doesn't work (Quarkus)

I try to perform 2 different operations with different threads each.Here is my code :
Uni.combine().all()
.unis(getItem(), getItemDetails())
.asTuple().subscribe().with(tuple -> {
context.setItem(tuple.getItem1());
context.setItemDetails(tuple.getItem2());
});
Methods :
public Uni<ItemResponse> callGetItem(){
Supplier<ItemResponse> supplier = () -> itemService.getItem("item_id_1");
return Uni.createFrom().item(supplier);
}
public Uni<ItemDetailsResponse> callGetItemDetail(){
Supplier<ItemDetailsResponse> supplier = () -> itemService.getItemDetail("dummy_item_id");
return Uni.createFrom().item(supplier) ;
}
But when i run the code both callGetItem() and callGetItemDetail() methods works in the same thread (executor-thread-0).
Where am i doing wrong?
Edit:
When i give an executor service Executors.newFixedThreadPool(2) for my Unis,
They still work in single thread. I mofified callGetItem() and callGetItemDetail() as :
public Uni<ItemResponse> callGetItem(){
Supplier<ItemResponse> supplier = () -> itemService.getItem("item_id_1");
return Uni.createFrom().item(supplier).emitOn(executor);
}
public Uni<ItemDetailsResponse> callGetItemDetail(){
Supplier<ItemDetailsResponse> supplier = () -> itemService.getItemDetail("dummy_item_id");
return Uni.createFrom().item(supplier).emitOn(executor) ;
}
executor is :
ExecutorService executor = Executors.newFixedThreadPool(2);
but they still works in same thread. Do you have any idea why it happens?

Since you are composing different Unis using Uni.combine().all().unis().asTuple(), the combined Uni will emit its result (combination) after the last element has emitted its item.
The last (upstream) Uni will have its item emitted (as is the case for other Unis as well) on whatever Thread that you have declaratively set it to emit on. Hence the combination Uni will follow execution on the same calling Thread.
As a result, if you are accessing the combined group values, you will be accessing these on the same execution carrier Thread.

How to wrap a Reactor Flux stream with a Redisson lock and unlock?

I have a Flux stream that reads objects from the database.
For each of these objects, I have a processing function to be run.
I want the processing function to execute after acquiring Redis lock on ID of given object, and after processing release the lock (also if the processing function throws an error).
What's the easiest way in Flux to create such a stream?
Here is some code of my failed attempt at doing this with transform.
I could probably make withLock take a function which would be attached as afterLock.flatMap(func), but I am looking for a solution that can avoid that.
I would like this to be as transparent to the rest of the stream as possible, and not require seperate attachment of lock and unlock functions, just one attachment that can do "lock-process-unlock".
private <T> Function<Flux<T>, Publisher<T>> withLock(Function<T, String> keyExtractor) {
return flux -> {
Flux<T> afterLock = flux.flatMap(ev -> redis.getLock(keyExtractor.apply(ev)).lock(1000L, TimeUnit.MILLISECONDS).map(ret -> ev));
// processing logic should be attached somewhere here
afterLock
.flatMap(ret -> redis.getLock(keyExtractor.apply(ret)).unlock()
.thenReturn(ret)
.onErrorResume(e -> redis.getLock(keyExtractor.apply(ret)).unlock().thenReturn(ret)));
return afterLock;
};
}
Flux.just(someObjectFromDatabase)
.transform(withLock(t -> t.id()))
.flatMap(this::objectProcessor)

One of the solution is to use Mono.usingWhen that allows to use async operations for resource supplier, resource closure and cleanup.
Mono.usingWhen(
lockService.acquire(key),
lock -> process(),
lock -> lockService.release(lock)
);
In our case we wrapped Redis lock into LockService that looks like the following
public interface ReactiveLockService {
Mono<LockEntry> acquire(String key, Duration expireAfter);
Mono<Void> release(LockEntry lock);
interface LockEntry {
String getKey();
String getValue();
}
}

Thanks for your answer #Alex, in the meantime I was able to come with something like this which is very flexible in terms of organizing the stream and resilent to failures (took me a while to cover edge cases...)
It can be used as a call to stream.flatMap(withLock(..., processor)
public static <T> Function<T, Flux<T>> withLock(
long tid, String lockPrefix, int lockTimeMs, Function<T, String> keyExtractor, Function<Mono<T>, Flux<T>> processor, RedissonReactiveClient redis) {
// If Redis lock or unlock operations fail, that will on purpose propagate the error.
// If processor throws an error, lock will be unlocked first before error propagation.
// tid has to be unique for each local task, it's a virtual "thread id" so if it's used concurrently locks will not protect the code
return flux -> {
Function<T, RLockReactive> getLock = ev -> redis.getLock(lockPrefix + keyExtractor.apply(ev));
RLockReactive lock = getLock.apply(flux);
Supplier<Mono<T>> unlock = () -> lock.unlock(tid).then(Mono.<T>empty());
Supplier<Mono<T>> doLock = () -> lock.lock(lockTimeMs, TimeUnit.MILLISECONDS, tid).then(Mono.<T>empty());
// Careful not to call map/flatMap on redis.lock/redis.unlock which returns Void and so it won't trigger on empty stream...lol!
return Flux.concat(
Mono.defer(doLock),
Flux.defer(() -> processor.apply(Mono.just(flux))
.onErrorResume(err -> unlock.get()
.onErrorResume(unlockError -> {
err.addSuppressed(unlockError);
// Propagate original processor error, but note the unlock error as suppressed
return Mono.error(err);
})
.then(Mono.error(err)))),
Mono.defer(unlock)
);
};

defer thenApplyAsync execution

I have following scenario.
CompletableFuture<T> result = CompletableFuture.supplyAsync(task, executor);
result.thenRun(() -> {
...
});
// ....
// after some more code, based on some condition I attach the thenApply() to result.
if ( x == 1) {
result.thenApplyAsync(t -> {
return null;
});
}
The question is what if the CompletableFuture thread finishes the execution before the main thread reaches the thenApplyAsync ? does the CompletableFuture result shall attach itself to thenApply. i.e should callback be declared at the time of defining CompletableFuture.supplyAsync() itself ?
Also what is the order of execution ? thenRun() is always executed at last (after thenApply()) ?
Is there any drawback to use this strategy?

You seem to be missing an important point. When you chain a dependent function, you are not altering the future you’re invoking the chaining method on.
Instead, each of these methods returns a new completion stage representing the dependent action.
Since you are attaching two dependent actions to result, which represent the task passed to supplyAsync, there is no relationship between these two actions. They may run in an arbitrary order and even at the same time in different threads.
Since you are not storing the future returned by thenApplyAsync anywhere, the result of its evaluation would be lost anyway. Assuming that your function returns a result of the same type as T, you could use
if(x == 1) {
result = result.thenApplyAsync(t -> {
return null;
});
}
to replace the potentially completed future with the new future that only gets completed when the result of the specified function has been evaluated. The runnable registered at the original future via thenRun still does not depend on this new future. Note that thenApplyAsync without an executor will always use the default executor, regardless of which executor was used to complete the other future.
If you want to ensure that the Runnable has been successfully executed before any other stage, you can use
CompletableFuture<T> result = CompletableFuture.supplyAsync(task, executor);
CompletableFuture<Void> thenRun = result.thenRun(() -> {
//...
});
result = result.thenCombine(thenRun, (t,v) -> t);
An alternative would be
result = result.whenComplete((value, throwable) -> {
//...
});
but here, the code will be always executed even in the exceptional case (which includes cancellation). You would have to check whether throwable is null, if you want to execute the code only in the successful case.
If you want to ensure that the runnable runs after both actions, the simplest strategy would be to chain it after the if statement, when the final completion stage is defined:
if(x == 1) {
result = result.thenApplyAsync(t -> {
return null;
});
}
result.thenRun(() -> {
//...
});
If that is not an option, you would need an incomplete future which you can complete on either result:
CompletableFuture<T> result = CompletableFuture.supplyAsync(task, executor);
//...
CompletableFuture<T> finalStage = new CompletableFuture<>();
finalStage.thenRun(() -> {
//...
});
// ...
if(x == 1) {
result = result.thenApplyAsync(t -> {
return null;
});
}
result.whenComplete((v,t) -> {
if(t != null) finalStage.completeExceptionally(t); else finalStage.complete(v);
});
The finalStage initially has no defined way of completion, but we can still chain dependent actions. Once we know the actual future, we can chain a handler which will complete our finalStage with whatever result we have.
As a final note, the methods without …Async, like thenRun, provide the least control over the evaluation thread. They may get executed in whatever thread completed the future, like one of executor’s threads in your example, but also directly in the thread calling thenRun, and even less intuitive, in your original example, the runnable may get executed during the unrelated thenApplyAsync invocation.

Create CompletableFuture from a sync method call

I would like to know if a one-liner exists for creating a CompletableFuture from a synchron method call. If no, why?
Long version:
final CompletableFuture<ReturnType> future = new CompletableFuture<>();
final String parameters = "hello";
ReturnType result;
try {
result = syncMethodCall(parameters);
} catch (Exception e) {
future.completeExceptionally(e);
}
future.complete(result);
return future;
Short desired version (or kind):
final String parameters = "hello";
return CompletableFuture.superMethod(() -> {syncMethodCall(parameters)});

Since you accepted an answer that performs an asynchronous call, it’s unclear why you asked for a “synchron method call” in the first place. The task of performing an asynchronous method invocation is quite easy with CompletableFuture:
String parameters="hello";
return CompletableFuture.supplyAsync(() -> syncMethodCall(parameters));
If your intention was to enforce the future to be already completed upon returning, it’s easy to enforce:
String parameters="hello";
CompletableFuture<ReturnType> f = CompletableFuture.supplyAsync(
() -> syncMethodCall(parameters));
f.handle((x,y) -> null).join();
return f;
The handle stage before the join ensures that in case syncMethodCall threw an exception, join won’t, as that seems to be your intention. But the handle stage is not returned, instead, the original future with the recorded exception will be returned.
Note that there’s a trick to do everything within the caller’s thread with the current implementation:
return CompletableFuture.completedFuture("hello")
.thenApply(parameters -> syncMethodCall(parameters));
The function passed to thenApply will be evaluated immediately when the future is already completed. But still, exceptions thrown by syncMethodCall are recorded in the returned future. So the outcome is identical to the “long version” of your question.

Since you want that your CompletableFuture is completed with a result of some method call, and you do not want to complete that CompletableFuture yourself - then you need not CompletableFuture - any Future implementation would be ok.
For example,
T function(parameters) {
return new T();
}
T res1 = function(parameters); // sync call
Future<T> f = ForkJoinPool.commonPool.submit(() -> function(parameters)); // async call
T res2 = f.get();
assert(res1.equals(res2));

Call multiple synchronous tasks asynchronously using RxJava

I have an async task represented by Futures executing in a separate threadpool that I want to join using RxJava. The "old" way of doing it using Java 5 constructs would be something like this (omitting collecting the results):
final Future<Response> future1 = wsClient.callAsync();
final Future<Response> future2 = wsClient.callAsync();
final Future<Response> future3 = wsClient.callAsync();
final Future<Response> future4 = wsClient.callAsync();
future1.get();
future2.get();
future3.get();
future4.get();
This would block my current thread until all futures are completed, but the calls would be in parallell and the whole operation would only take the time equal to the longest call.
I want to do the same using RxJava, but I'm a bit noob when it comes to how to model it correctly.
I've tried the following, and it seems to work:
Observable.from(Arrays.asList(1,2,3,4))
.flatMap(n -> Observable.from(wsClient.callAsync(), Schedulers.io()))
.toList()
.toBlocking()
.single();
The problem with this approach is that I introduce the Schedulers.io threadpool which causes unnecessary thread switching as I'm already blocking the current thread (using toBlocking()).
Is there any way I can model the Rx flow to execute the tasks in parallel, and block until all has been completed?

You should use zip function.
For example like this:
Observable.zip(
Observable.from(wsClient.callAsync(), Schedulers.io()),
Observable.from(wsClient.callAsync(), Schedulers.io()),
Observable.from(wsClient.callAsync(), Schedulers.io()),
Observable.from(wsClient.callAsync(), Schedulers.io()),
(response1, response2, response3, response4) -> {
// This is a zipping function...
// You'll end up here when you've got all responses
// Do what you want with them and return a combined result
// ...
return null; //combined result instead of null
})
.subscribe(combinedResult -> {
// Use the combined result
});
Observable.zip can also work with an Iterable so you can wrap your Observable.from(wsClient.callAsync(), Schedulers.io()); around with one (that returns 4 of those).