I have a Flux stream. For each element processed I want to have an action triggered which is an asynchronous/non-blocking one. For example, a method returning back a Mono from a db update.
I want this action to be done on the doOnNext block.
I don't want to affect the Flux, the processing and the back pressure implemented there.
Supposing Mono method to be called is
Mono<Integer> dbUpdate();
should my Flux be like this?
public Flux<Data> processData(PollRequest request)
{
return searchService.search(request)
.doOnNext(data -> dbUpdate(data));
}
Or should be as mentioned on a stack overflow example.
public Flux<Data> processData(PollRequest request)
{
return searchService.search(request)
.doOnNext(data -> dbUpdate(data).subscribe());
}
Won't the above cause blocking issues inside doOnNext?
Also which is the most appropriate scheduler to use for this type of action?
dbUpdate() will be ignored if you do not subscribe to it. The following snippet doesn't print anything because Mono.just("db update") doesn't get subscribed.
Mono<String> dbUpdate() {
return Mono.just("db update")
.doOnNext(System.out::println);
}
public Flux<String> processData() {
return Flux.just("item 1", "item 2")
.doOnNext(data -> dbUpdate());
}
Note that .subscribe() doesn't block your thread, it kicks off the work and returns immediately.
Related
I want to do something finally after stream terminates for any reason including cancellation, and I
found the doFinally method, but it dose not work when cancellation, because https://github.com/reactor/reactor-core/issues/1090#issuecomment-367633241 show :
Cancellation travels only upstream
So, how to capture the cancel signal?
There is my code:
public Mono<Void> myFunction() {
return Mono.just("hello")
.flatMap(s -> foo(s))
.doFinally(signalType -> {
// do something finally, but the doFinally won't be called
System.out.println(signalType);
});
}
// some other library's function that I cant not modify any way
public Mono<Void> foo(String s) {
// return a reactive stream, and will cancel it after it be subscribed, like:
return Mono.just(s)
.doOnSubscribe(subscription -> subscription.cancel())
.then();
}
You can't in that particular arrangement, because the foo() method/library seems to manage the subscription (the cancellation) itself, instead of leaving that responsibility to the consumer. Managing the subscription like that is thus not necessarily a good thing.
CompletableFuture executes a task on a separate thread ( uses a thread-pool ) and provides a callback function. Let's say I have an API call in a CompletableFuture. Is that an API call blocking? Would the thread be blocked till it does not get a response from the API? ( I know main thread/tomcat thread will be non-blocking, but what about the thread on which CompletableFuture task is executing? )
Mono is completely non-blocking, as far as I know.
Please shed some light on this and correct me if I am wrong.
CompletableFuture is Async. But is it non-blocking?
One which is true about CompletableFuture is that it is truly async, it allows you to run your task asynchronously from the caller thread and the API such as thenXXX allows you to process the result when it becomes available. On the other hand, CompletableFuture is not always non-blocking. For example, when you run the following code, it will be executed asynchronously on the default ForkJoinPool:
CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
}
return 1;
});
It is clear that the Thread in ForkJoinPool that executes the task will be blocked eventually which means that we can't guarantee that the call will be non-blocking.
On the other hand, CompletableFuture exposes API which allows you to make it truly non-blocking.
For example, you can always do the following:
public CompletableFuture myNonBlockingHttpCall(Object someData) {
var uncompletedFuture = new CompletableFuture(); // creates uncompleted future
myAsyncHttpClient.execute(someData, (result, exception -> {
if(exception != null) {
uncompletedFuture.completeExceptionally(exception);
return;
}
uncompletedFuture.complete(result);
})
return uncompletedFuture;
}
As you can see, the API of CompletableFuture future provides you with the complete and completeExceptionally methods that complete your execution whenever it is needed without blocking any thread.
Mono vs CompletableFuture
In the previous section, we got an overview of CF behavior, but what is the central difference between CompletableFuture and Mono?
It worth to mention that we can do blocking Mono as well. No one prevents us from writing the following:
Mono.fromCallable(() -> {
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
}
return 1;
})
Of course, once we subscribe to the future, the caller thread will be blocked. But we can always work around that by providing an additional subscribeOn operator. Nevertheless, the broader API of Mono is not the key feature.
In order to understand the main difference between CompletableFuture and Mono, lets back to previously mentioned myNonBlockingHttpCall method implementation.
public CompletableFuture myUpperLevelBusinessLogic() {
var future = myNonBlockingHttpCall();
// ... some code
if (something) {
// oh we don't really need anything, let's just throw an exception
var errorFuture = new CompletableFuture();
errorFuture.completeExceptionally(new RuntimeException());
return errorFuture;
}
return future;
}
In the case of CompletableFuture, once the method is called, it will eagerly execute HTTP call to another service/resource. Even though we will not really need the result of the execution after verifying some pre/post conditions, it starts the execution, and additional CPU/DB-Connections/What-Ever-Machine-Resources will be allocated for this work.
In contrast, the Mono type is lazy by definition:
public Mono myNonBlockingHttpCallWithMono(Object someData) {
return Mono.create(sink -> {
myAsyncHttpClient.execute(someData, (result, exception -> {
if(exception != null) {
sink.error(exception);
return;
}
sink.success(result);
})
});
}
public Mono myUpperLevelBusinessLogic() {
var mono = myNonBlockingHttpCallWithMono();
// ... some code
if (something) {
// oh we don't really need anything, let's just throw an exception
return Mono.error(new RuntimeException());
}
return mono;
}
In this case, nothing will happen until the final mono is subscribed. Thus, only when Mono returned by the myNonBlockingHttpCallWithMono method, will be subscribed, the logic provided to Mono.create(Consumer) will be executed.
And we can go even further. We can make our execution much lazier. As you might know, Mono extends Publisher from the Reactive Streams specification. The screaming feature of Reactive Streams is backpressure support. Thus, using the Mono API we can do execution only when the data is really needed, and our subscriber is ready to consume them:
Mono.create(sink -> {
AtomicBoolean once = new AtomicBoolean();
sink.onRequest(__ -> {
if(!once.get() && once.compareAndSet(false, true) {
myAsyncHttpClient.execute(someData, (result, exception -> {
if(exception != null) {
sink.error(exception);
return;
}
sink.success(result);
});
}
});
});
In this example, we execute data only when subscriber called Subscription#request so by doing that it declared its readiness to receive data.
Summary
CompletableFuture is async and can be non-blocking
CompletableFuture is eager. You can't postpone the execution. But you can cancel them (which is better than nothing)
Mono is async/non-blocking and can easily execute any call on different Thread by composing the main Mono with different operators.
Mono is truly lazy and allows postponing execution startup by the subscriber presence and its readiness to consume data.
Building up on Oleh's answer, a possible lazy solution for CompletableFuture would be
public CompletableFuture myNonBlockingHttpCall(CompletableFuture<ExecutorService> dispatch, Object someData) {
var uncompletedFuture = new CompletableFuture(); // creates uncompleted future
dispatch.thenAccept(x -> x.submit(() -> {
myAsyncHttpClient.execute(someData, (result, exception -> {
if(exception != null) {
uncompletedFuture.completeExceptionally(exception);
return;
}
uncompletedFuture.complete(result);
})
}));
return uncompletedFuture;
}
Then, later on you simply do
dispatch.complete(executor);
That would make CompletableFuture equivalent to Mono, but without backpressure, I guess.
I've code like this in a repository:
return Completable.fromAction {
// Some code
loginService.login(id)
.subscribe(
{ response ->
if(response.isNotSuccessful()) {
throw Exception()
}
// Some code
},
{ e ->
throw e
}
)
}
I've code like this in a ViewModel:
fun onLoginAction(id) {
repository.login(id)
.subscribe(
{
showSuccess()
},
{
showFailure()
}
)
}
Basically, the ViewModel calls the login method in the repository which returns the Completable.
This results in an UndeliverableException when the response is not successful. I want the Completable's subscriber's onError() method to be called. How do I do this?
I don't have enough knowledge to actually say this with certainty, but I still think this has some value to you and it's too big for a comment.
Here's what I think it's happening. When onError fails rx won't run this through the same observable stream. Instead, it will propagate this to the RxPlugins error handler and eventually to the default exception handler in your system. You can find this here.
This is to say that when loginService.login(id) throws the exception in the onError, the Completable stream won't have a chance to catch it and forward it to the onError of the outer subscribe. In other words, the completable stream is independent of the login service one.
Usually, you'd want to create one single stream and let the view model subscribe to it. If you have more than one stream, rx has loads of operators to help you chain these. Try and make the repository return one stream from the service. Something like this:
fun login(id) = loginService.login(id)
And now on the view model, you can check if the call was or not successful using the same method - response.isNotSuccessful()
I have a usecase where I initiallize some global variables in a Completable , and in the next step in the chain (using andThen operator) I make use of those variables.
Following sample explains my usecase in detail
Say you have a class User
class User {
String name;
}
and I have an Observable like this ,
private User mUser; // this is a global variable
public Observable<String> stringObservable() {
return Completable.fromAction(() -> {
mUser = new User();
mUser.name = "Name";
}).andThen(Observable.just(mUser.name));
}
First I'm doing some initiallizations in my Completable.fromAction and I expect andThen operator to start only after completing the Completable.fromAction.
Which means I expect mUser to be initallized when the andThen operator starts.
Following is my subscription to this observable
stringObservable()
.subscribe(s -> Log.d(TAG, "success: " + s),
throwable -> Log.e(TAG, "error: " + throwable.getMessage()));
But when I run this code , I get an error
Attempt to read from field 'java.lang.String User.name' on a null object reference
which means mUser is null , andThen started before executing the code in Completable.fromAction. Whats happening here?
According to documentation of andThen
Returns an Observable which will subscribe to this Completable and once that is completed then will subscribe to the {#code next} ObservableSource. An error event from this Completable will be propagated to the downstream subscriber and will result in skipping the subscription of the Observable.
The issue is not with andThen but with the statement Observable.just(mUser.name) inside andThen . The just operator will try to create the observable immediately though it will emit only after Completable.fromAction.
Problem here is , while trying to create the Observable using just , the mUser is null.
Solution : You need to defer the creation of the String Observable till a subscription happens , till the upstream of andThen starts emission.
Instead of andThen(Observable.just(mUser.name));
use
andThen(Observable.defer(() -> Observable.just(mUser.name)));
Or
andThen(Observable.fromCallable(() -> mUser.name));
I don't think #Sarath Kn's answer is 100% correct. Yes just will create observable as soon as it's called, but andThen is still calling just at an unexpected time.
We can compare andThen with flatMap to get some better understanding. Here is a fully runnable test:
package com.example;
import org.junit.Test;
import io.reactivex.Completable;
import io.reactivex.Observable;
import io.reactivex.observers.TestObserver;
import io.reactivex.schedulers.Schedulers;
public class ExampleTest {
#Test
public void createsIntermediateObservable_AfterSubscribing() {
Observable<String> coldObservable = getObservableSource()
.flatMap(integer -> getIntermediateObservable())
.subscribeOn(Schedulers.trampoline())
.observeOn(Schedulers.trampoline());
System.out.println("Cold obs created... subscribing");
TestObserver<String> testObserver = coldObservable.test();
testObserver.awaitTerminalEvent();
/*
Resulting logs:
Creating observable source
Cold obs created... subscribing
Emitting 1,2,3
Creating intermediate observable
Creating intermediate observable
Creating intermediate observable
Emitting complete notification
IMPORTANT: see that intermediate observables are created AFTER subscribing
*/
}
#Test
public void createsIntermediateObservable_BeforeSubscribing() {
Observable<String> coldObservable = getCompletableSource()
.andThen(getIntermediateObservable())
.subscribeOn(Schedulers.trampoline())
.observeOn(Schedulers.trampoline());
System.out.println("Cold obs created... subscribing");
TestObserver<String> testObserver = coldObservable.test();
testObserver.awaitTerminalEvent();
/*
Resulting logs:
Creating completable source
Creating intermediate observable
Cold obs created... subscribing
Emitting complete notification
IMPORTANT: see that intermediate observable is created BEFORE subscribing =(
*/
}
private Observable<Integer> getObservableSource() {
System.out.println("Creating observable source");
return Observable.create(emitter -> {
System.out.println("Emitting 1,2,3");
emitter.onNext(1);
emitter.onNext(2);
emitter.onNext(3);
System.out.println("Emitting complete notification");
emitter.onComplete();
});
}
private Observable<String> getIntermediateObservable() {
System.out.println("Creating intermediate observable");
return Observable.just("A");
}
private Completable getCompletableSource() {
System.out.println("Creating completable source");
return Completable.create(emitter -> {
System.out.println("Emitting complete notification");
emitter.onComplete();
});
}
}
You can see that when we use flatmap, the just is called after subscribing, which makes sense. If the intermediate observable depended on the items emitted to the flatmap then of course the system can't create the intermediate observable before subscription. It would not yet have any values. You can imagine this wouldn't work if flatmap called just before subscribing:
.flatMap(integer -> getIntermediateObservable(integer))
What is weird is that andThen is able to create it's inner observable (i.e. call just) before subscribing. It makes sense that it can do this. The only thing andThen is going to receive is a complete notification, so there is no reason NOT to create the intermediate observable early. The only problem is that it's not the expected behavior.
#Sarath Kn's solution is correct, but for the wrong reason. If we use defer we can see things working as expected:
#Test
public void usingDefer_CreatesIntermediateObservable_AfterSubscribing() {
Observable<String> coldObservable = getCompletableSource()
.andThen(Observable.defer(this::getIntermediateObservable))
.subscribeOn(Schedulers.trampoline())
.observeOn(Schedulers.trampoline());
System.out.println("Cold obs created... subscribing");
TestObserver<String> testObserver = coldObservable.test();
testObserver.awaitTerminalEvent();
/*
Resulting logs:
Creating completable source
Cold obs created... subscribing
Emitting complete notification
Creating intermediate observable
IMPORTANT: see that intermediate observable is created AFTER subscribing =) YEAY!!
*/
}
I'm using RxJava in and Android application with RxAndroid. I'm using mergeDelayError to combine two retro fit network calls into one observable which will process emitted items if either emits one and the error if either has one. This is not working and it is only firing off the onError action when either encounters an error. Now to test this I shifted to a very simple example and still the successAction is never called when I have an onError call. See example below.
Observable.mergeDelayError(
Observable.error(new RuntimeException()),
Observable.just("Hello")
)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.finallyDo(completeAction)
.subscribe(successAction, errorAction);
The success action will only be called if I use two success observables. Am I missing something with how mergeDelayError is supposed to work?
EDIT:
I've found that if I remove the observeOn and subscribeOn everything works as expected. I need to specify threads and thought that was the whole point of using Rx. Any idea why specifying those Schedulers would break the behavior?
Use .observeOn(AndroidSchedulers.mainThread(), true) instead of .observeOn(AndroidSchedulers.mainThread()
public final Observable<T> observeOn(Scheduler scheduler, boolean delayError) {
return observeOn(scheduler, delayError, RxRingBuffer.SIZE);
}
Above is the signature of observeOn function. Following code works.
Observable.mergeDelayError(
Observable.error(new RuntimeException()),
Observable.just("Hello")
)
.observeOn(AndroidSchedulers.mainThread(), true)
.subscribeOn(Schedulers.io())
.subscribe(new Subscriber<String>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(String s) {
}
});
Got this trick from ConcatDelayError thread: https://github.com/ReactiveX/RxJava/issues/3908#issuecomment-217999009
This still seems like a bug in the mergeDelayError operator but I was able to get it working by duplicating the observerOn and Subscribe on for each observable.
Observable.mergeDelayError(
Observable.error(new RuntimeException())
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io()),
Observable.just("Hello")
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
)
.finallyDo(completeAction)
.subscribe(successAction, errorAction);
I think you don't wait for the terminal event and the main thread quits before the events are delivered to your observer. The following test passes for me with RxJava 1.0.14:
#Test
public void errorDelayed() {
TestSubscriber<Object> ts = TestSubscriber.create();
Observable.mergeDelayError(
Observable.error(new RuntimeException()),
Observable.just("Hello")
)
.subscribeOn(Schedulers.io()).subscribe(ts);
ts.awaitTerminalEvent();
ts.assertError(RuntimeException.class);
ts.assertValue("Hello");
}