We Keep Coding

Executing Mono streams that no one subscribes to in Spring Web flux [duplicate]

This question already has an answer here:
What is the difference between block() , subscribe() and subscribe(-)
(1 answer)
Closed last year.
I have a spring Webflux application. There are two important parts to this application:
A job is scheduled to run at a fixed interval.
The job fetches the data from DB and stores the data in Redis.
void run() {
redisAdapter.getTtl()
.doOnError(RefreshExternalCache::logError)
.switchIfEmpty(Mono.defer(() -> {
log.debug(">> RefreshExternalCache > refreshExternalCacheIfNeeded => Remaining TTL could not be retrieved. Cache does not exist. " +
"Trying to create the cache.");
return Mono.just(Duration.ofSeconds(0));
}))
.subscribe(remainingTtl -> {
log.debug(">> RefreshExternalCache > refreshExternalCacheIfNeeded => original ttl for the cache: {} | ttl for cache in seconds = {} | ttl for cache in minutes = {}",
remainingTtl, remainingTtl.getSeconds(), remainingTtl.toMinutes());
if (isExternalCacheRefreshNeeded(remainingTtl, offerServiceProperties.getExternalCacheExpiration(), offerServiceProperties.getExternalCacheRefreshPeriod())) {
log.debug(">> RefreshExternalCache > refreshExternalCacheIfNeeded => external cache is up-to-date, skipping refresh");
} else {
log.debug(">> RefreshExternalCache > refreshExternalCacheIfNeeded => external cache is outdated, updating the external cache");
offerService.refreshExternalCache();
}
});
}
This basically calls another method called refreshExternalCache(), the implementation below:
public void refreshExternalCache() {
fetchOffersFromSource()
.doOnNext(offerData -> {
log.debug(LOG_REFRESH_CACHE + "Updating local offer cache with data from source");
localCache.put(OFFER_DATA_KEY, offerData);
storeOffersInExternalCache(offerData, offerServiceProperties.getExternalCacheExpiration());
})
.doOnSuccess(offerData -> meterRegistry.counter(METRIC_EXTERNAL_CACHE_REFRESH_COUNTER, TAG_OUTCOME, SUCCESS).increment())
.doOnError(sourceThrowable -> {
log.debug(LOG_REFRESH_CACHE + "Error while refreshing external cache {}", sourceThrowable.getMessage());
meterRegistry.counter(METRIC_EXTERNAL_CACHE_REFRESH_COUNTER, TAG_OUTCOME, FAILURE).increment();
}).subscribe();
}
Also, in the above method, you can see a call to storeOffersInExternalCache
public void storeOffersInExternalCache(OfferData offerData, Duration ttl) {
log.info(LOG_STORING_OFFER_DATA + "Storing the offer data in external cache...");
redisAdapter.storeOffers(offerData, ttl);
}
public void storeOffers(OfferData offerData, Duration ttl) {
Mono.fromRunnable(() -> redisClient.storeSerializedOffers(serializeFromDomain(offerData), ttl)
.doOnNext(status -> {
if (Boolean.TRUE.equals(status)) {
log.info(LOG_STORE_OFFERS + "Data stored in redis.");
meterRegistry.counter(METRIC_REDIS_STORE_OFFERS, TAG_OUTCOME, SUCCESS).increment();
} else {
log.error(LOG_STORE_OFFERS + "Unable to store data in redis.");
meterRegistry.counter(METRIC_REDIS_STORE_OFFERS, TAG_OUTCOME, FAILURE).increment();
}
}).retryWhen(Retry.backoff(redisRetryProperties.getMaxAttempts(), redisRetryProperties.getWaitDuration()).jitter(redisRetryProperties.getBackoffJitter()))
.doOnError(throwable -> {
meterRegistry.counter(METRIC_REDIS_STORE_OFFERS, TAG_OUTCOME, FAILURE).increment();
log.error(LOG_STORE_OFFERS + "Unable to store data in redis. Error: [{}]", throwable.getMessage());
})).subscribeOn(Schedulers.boundedElastic());
}
Redis Client
#Slf4j
#Component
public class RedisClient {
private final ReactiveRedisTemplate<String, String> reactiveRedisTemplate;
private final ReactiveValueOperations<String, String> reactiveValueOps;
public RedisClient(#Qualifier("reactiveRedisTemplate") ReactiveRedisTemplate<String, String> reactiveRedisTemplate) {
this.reactiveRedisTemplate = reactiveRedisTemplate;
this.reactiveValueOps = reactiveRedisTemplate.opsForValue();
}
Mono<Optional<String>> fetchSerializedOffers() {
return reactiveValueOps.get(OFFER_DATA_KEY).map(Optional::ofNullable);
}
Mono<Boolean> storeSerializedOffers(String serializedOffers, Duration ttl) {
return reactiveValueOps.set(OFFER_DATA_KEY, serializedOffers, ttl);
}
Mono<Duration> getTtl() {
return reactiveRedisTemplate.getExpire(OFFER_DATA_KEY);
}
}
Now my concerns are:
If I do not call the subscribe method on these Mono streams, these methods are not even executed. This is fair as they won't execute until someone subscribes to them.
As I understand it correctly, subscribe is a blocking call. This defeats the whole purpose of Reactive programming. Isn't it?
I looked for several ways to make this work, one of them has been shown above. I tried calling one of the methods in Mono.fromRunnable but this also is not a very good approach. (read it on another thread in StackOverflow).
So, is the approach that I am taking above not correct? How do we execute the Mono streams that no one subscribes to?

Answering your concern number 2 (which seems to be the only real doubt in your question). Not really. block() (https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#block--) is the one that subscribes to a Mono or Flux and waits indefinitely until a next signal is received. On the other hand subscribe() (https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#subscribe--) subscribes to a Mono or Flux but it doesn't block and instead reacts when an element is emitted.

Problems with RXJava

I'm adapting some sample code from what3words for accessing their API via their Java SDK. It uses RXJava.
The sample code is:
Observable.fromCallable(() -> wrapper.convertTo3wa(new Coordinates(51.2423, -0.12423)).execute())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(result -> {
if (result.isSuccessful()) {
Log.i("MainActivity", String.format("3 word address: %s", result.getWords()));
} else {
Log.e("MainActivity", result.getError().getMessage());
}
});
First of all. this gives a deprecation warning when building and a IDE warning (Result of 'Observable.subscribe()' is ignored).
To resolve this first issue I have added Disposable myDisposable = in front of the Observable. Is this correct? (See below for where it is added)
Next I need to add a timeout so that I can show a warning etc if the request times out. To do this I have added .timeout(5000, TimeUnit.MILLISECONDS) to the builder.
This works, but the way timeouts seem to work on Observables is that they throw an exception and I cannot figure out how to catch and handle that exception.
What I have right now is:
Disposable myDisposable = Observable.fromCallable(() -> wrapper.convertTo3wa(new Coordinates(51.2423, -0.12423)).execute())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.timeout(5000, TimeUnit.MILLISECONDS)
.subscribe(result -> {
if (result.isSuccessful()) {
Log.i("MainActivity", String.format("3 word address: %s", result.getWords()));
} else {
Log.e("MainActivity", result.getError().getMessage());
}
});
This builds and runs fine, and the API/deprecation warning is not shown, BUT when no network is available this correctly times out and throws the unhandled exception.
So, the code seems to be correct, but how on earth do add the exception handling to catch the timeout TimeoutException that is thrown?
I've tried numerous things, including: adding a try-catch clause around the whole Observable - this warns that TimeoutException is not thrown by the code in the `try; and adding an error handler.
Adding the error handler has got me closest, and so the code below is as far as I have got:
Disposable myDisposable = Observable.fromCallable(() -> wrapper.convertTo3wa(new Coordinates(51.2423, -0.12423)).execute())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.timeout(5000, TimeUnit.MILLISECONDS)
.subscribe(result -> {
if (result.isSuccessful()) {
Log.i("MainActivity", String.format("3 word address: %s", result.getWords()));
} else {
Log.e("MainActivity", result.getError().getMessage());
}
}, error -> {
runOnUiThread(new Runnable() {
#Override
public void run() {
myTextView.setText(R.string.network_not_available);
}
});
});
This catches the Timeout correctly and updates my UI without error, however when the network is restored it seems that the Observable might be trying to return and a null pointer exception is thrown.
(Update, this NPE might actually be being thrown sometimes after a short time whether the network is restored or not... but it is always thrown when the network restores.)
I get FATAL EXCEPTION: RxCachedThreadScheduler-1 and java.lang.NullPointerException: Callable returned a null value. Null values are generally not allowed in 3.x operators and sources.
Do I need to destroy the Observable or something to prevent the NPE?

You need to add an onError handler to your subscribe call:
.subscribe(result -> {
if (result.isSuccessful()) {
Log.i("MainActivity", String.format("3 word address: %s", result.getWords()));
} else {
Log.e("MainActivity", result.getError().getMessage());
}
},
error -> {
// handle error here
});
When a an exception makes it to a subscribe call that does not have an onError handler, it will throw a OnErrorNotImplementedException, like this:
io.reactivex.exceptions.OnErrorNotImplementedException: The exception was not handled due to missing onError handler in the subscribe() method call. Further reading: https://github.com/ReactiveX/RxJava/wiki/Error-Handling | java.util.concurrent.TimeoutException: The source did not signal an event for 1 seconds and has been terminated.
Adding the onError handler will prevent that, and the onError handler will get called instead.

There's a few things going on here:
First of all. this gives a deprecation warning when building and a IDE warning (Result of 'Observable.subscribe()' is ignored).
subscribe() returns a Disposable. The idea is that when you're no longer interested in receiving the output of your observable, you call dispose() on the disposable and the work terminates. This can also prevent memory leaks.
As an example, imagine you have an Activity, and you start an Observable to run a long network query which finally posts something to the Activity UI. If the user navigates away before this task completes, or the Activity is otherwise destroyed, then you're no longer interested in its output because there is no longer a UI to post to. So you may call dispose() in onStop().
So, the code seems to be correct, but how on earth do add the exception handling to catch the timeout TimeoutException that is thrown?
Using the error block in subscribe is one option, but there are others. For example, if you wanted to keep using your Result class, you could use something like onErrorReturn(throwable -> Result.error(throwable)). Obviously I'm guessing what that class looks like:
.timeout(5000, TimeUnit.MILLISECONDS)
.onErrorReturn(throwable -> Result.errorWithMessage(R.string.network_not_available))
.subscribe(result -> {
if (result.isSuccessful()) {
Log.i("MainActivity", String.format("3 word address: %s", result.getWords()));
} else {
myTextView.setText(result.getErrorMessage());
}
});
java.lang.NullPointerException: Callable returned a null value. Null values are generally not allowed in 3.x operators and sources.
This:
wrapper.convertTo3wa(new Coordinates(51.2423, -0.12423)).execute()
is returning null. You can do something like:
Observable.fromCallable(() -> {
Result<?> out = wrapper.convertTo3wa(new Coordinates(51.2423, -0.12423)).execute();
if(out == null)
out = Result.error(/*Returned null*/);
}
return out;
}

Emit single item while update UI about progress in RxJava / RxAndroid

I'm currently trying to learn RxJava in Android. I require some guides.
At the moment, I'm trying to rewrite AsyncTask below to RxJava:
public class MyAsyncTask extends AsyncTask<Void, ProgressInfo, Result> {
#Override
protected Result doInBackground(Void... void) {
//Long running task
publishProgress(progressInfo);
//Long running task
return result;
}
#Override
protected void onProgressUpdate(ProgressInfo... progressInfo) {
//Update the progress to UI using data from ProgressInfo
}
#Override
protected void onPostExecute(Result res) {
//Task is completed with a Result
}
}
In AsyncTask approach shown above, I can update the UI about the progress by making use of onProgressUpdate method, I pack every data I needed into ProgressInfo and reflect the UI in onProgressUpdate. After task ends, the Result will be passed from from doInBackground to onPostExecute.
But, when I'm trying to implement this with RxJava, I have a hard time dealing with it. Since I cannot pass any parameter to onComplete in Observer. And thus, I ended up with following implementation. I merged the pass of the ProgressInfo and Result into onNext.
Observable.create(emitter -> {
//Long running task
emitter.onNext(progressInfo);
//Long running task
emitter.onNext(result);
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(object -> {
if(object instanceof ProgressInfo){
//Update the progress to UI using data from ProgressInfo
}else if(object instanceof Result){
//Task is completed with a Result
}
});
QUESTION 1: Is my implementation/concept in RxJava right or wrong?
Although it works, I personally feels the implementation above strange and wrong to me. Since the task ultimately is just trying to do some calculations and come out with a single item - Result. The emission of ProgressInfo is like a "side" thing but not "main" thing. I should implement it with Single.create(). But if I did this, I cannot think of any way to pass any ProgressInfo to my UI.
QUESTION 2:
Is there a better idea/way to emit single item while updating the UI during the process?
If yes, how would you implement this logic in RxJava? Can you show me your codes/examples?

QUESTION 1: Is my implementation/concept in RxJava right or wrong?
Surely it depends on your use-case. If you want to provide feedback on each progress-step, there is no way, which I am aware of, to do it differently. I would recommand to provide progress feedback, when the task takes quite a few time and you are able to provide meaningful progress-information.
Either use a union of ProgressInfo and Result in one type and test for null or use a marker interface, from which ProgressInfo and Result inherite from.
interface ResultT { }
final class ProgressInfo implements ResultT { }
final class Result implements ResultT { }
When the result is emitted via onNext, I would recommand to complete the observable, in order to give notice to the subscriber, that the task has been done. The subscriber will receive the result via onNext and a onComplete afterwards.
Observable.<ResultT>create(emitter -> {
emitter.onNext(progressInfo);
emitter.onNext(result);
emitter.onComplete();
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(object -> {
if (object instanceof ProgressInfo) {
//Update the progress to UI using data from ProgressInfo
} else if (object instanceof Result) {
//Task is completed with a Result
}
});
If you have no meaningfull progress-information, I would recommend using a Single.
QUESTION 2: Is there a better idea/way to emit single item while updating the UI during the process?
The doOn*-Operators could be used, to update the UI on subscription and termination. This way is one of the easiest, but could cause problems, when events from other subscriptions interleave with UI changes^1
.doOnSubscribe(disposable -> {/* update ui */})
.subscribe(s -> {
// success: update ui
},
throwable -> {
// error happened: update ui
},
() -> {
// complete: update ui
});
My recommandation would be modelling all States (e.g. Success/ Error) via a class and switch-case in the the subscribe-method (see ^1). First emit an StartProgress-event, then the ProgressInformation ones and on finish the SucessResult. Catch any errors with onError*-operators and return a FailureResult, which contains a error-message and maybe the throwable.
Observable.<ResultT>create(emitter -> {
emitter.onNext(progressInfo);
emitter.onNext(result);
emitter.onComplete();
}).startWith(new StartProgress())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.onErrorReturn(throwable -> new FailureResult(throwable))
.subscribe(object -> {
// when StartProgress -> updateUI
// when ProgressInformation -> updateUI
// ...
});
^1 http://hannesdorfmann.com/android/mosby3-mvi-1

1- Create a data class called ProgressInfo
data class ProgressInfo(val progress: Float,val downloadedFile: File, val isCompleted: Boolean = false )
2- Create observable
Observable.create<ProgressInfo> { emitter ->
try {
val url = URL("mediaUrl")
val targetFile = File( "filePath")
if (targetFile.exists().not() && targetFile.createNewFile()) {
val openConnection = url.openConnection()
openConnection.connect()
val totalBytes = openConnection.contentLength
val openStream = openConnection.inputStream
var downloadedBytes = 0f
openStream.use { inStream ->
FileOutputStream(targetFile).use { outStream ->
val streamSlice = ByteArray(1024)
while (true) {
val read = inStream.read(streamSlice)
if (read == -1) {
// file download complete
val progressInfo =
ProgressInfo(
(downloadedBytes / totalBytes) * 100f,
targetFile,
true
)
emitter.onNext(progressInfo)
break
}
downloadedBytes += read
outStream.write(streamSlice)
// update progress
emitter.onNext(
ProgressInfo(
(downloadedBytes / totalBytes) * 100f,
targetFile
)
)
}
}
}
}
emitter.onComplete()
} catch (ex: Exception) {
emitter.onError(ex)
}
}.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
// update your progress here
}, {
// on error
},{
// on complete
})

RxJava zip 3 observables - onSubscribe lag

Please look at this simple code:
private void refreshData(String endpoint)
{
KProgressHUD busy = Utils.showBusyIndicator(MainActivity.this);
MCityEndpoint mcityService = ServiceFactory.createRetrofitService(MCityEndpoint.class, Configuration.getApiUrl());
Disposable disposable = Observable.zip
(
mcityService.getMcityDictionaries(selectedCity.id),
mcityService.getEvents(selectedCity.id, endpoint),
mcityService.getUserDetails(selectedCity.id),
(dictionaries,events, userInfo) ->
{
processData(dictionaries, events, userInfo);
return events.events;
}
)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(result ->
{
if(mFragment instanceof ListFragment)
{
((ListFragment) mFragment).refreshData(result);
}
updateUserDetails();
busy.dismiss();
Log.d(Configuration.tag,"Refresh data complete");
}, throwable ->
{
Log.d(Configuration.tag,throwable.toString());
busy.dismiss();
this.logout(true);
});
mCompositeDisposable.add(disposable);
}
I want to call 3 observables, process their values and finally load data to list. It works, but I'm getting 1 second lag. Result is:
Busy indicator shows up
Busy indicator hides
1 second lag without busy indicator visible
List is refreshed after lag
I can't understand reason of lag in this case. Everything should be refreshed with busy indicator visible. Do you have any idea?

Single observer is not calling onError()

I've written the following code in my MainActivity of an Android app. When I run the following code, it doesn't throw any exception and onError() also doesn't get called. However I see onSuccess: testing starts twice but I don't see onSuccess: testing ends. Why isn't onError() being called and/or why isn't the app crashing?
Single.timer(1000, TimeUnit.MILLISECONDS)
.subscribeOn(Schedulers.computation())
.subscribeWith(new DisposableSingleObserver<Long>() {
#Override
public void onSuccess(Long initiationTimeStamp) {
String s = null;
Log.d(TAG, "onSuccess: testing starts");
Log.d(TAG, "onSuccess:test "+ s.isEmpty());
Log.d(TAG, "onSuccess: testing ends");
}
#Override
public void onError(Throwable e) {
e.printStackTrace();
}
});

You are calling s.isEmpty() on a NULL String, thats why it ends at first print. That said onSuccess() does not throw anything, so it just stops execution when NullPointerException is thrown (it is silently handled inside RxJava for you). As soon as you subscribe to observable, you get initial value in onSuccess(), then if it changes or you resubscribe you get another value in onSuccess(), thats why it gets called twice. And because onError() is for errors occurring along the operation chain, you do not get an error in onSuccess() when exception is thrown.
This behaviour is intentional. According to Rx Contract, an observer should not receive both onSuccess() and onError(). You need to handle the exception in onSuccess() by yourself.
For example:
Single.timer(1000, TimeUnit.MILLISECONDS)
.subscribeOn(Schedulers.computation())
.subscribeWith(new DisposableSingleObserver<Long>() {
#Override
public void onSuccess(Long initiationTimeStamp) {
try {
String s = null;
Log.d(TAG, "onSuccess: testing starts");
Log.d(TAG, "onSuccess:test "+ s.isEmpty());
Log.d(TAG, "onSuccess: testing ends");
}
catch (Throwable ex) {
// tell the upstream we can't accept any more data (OPTIONAL)
dispose();
// pass error to error handler
onError(ex);
}
}
#Override
public void onError(Throwable e) {
e.printStackTrace();
}
});
Good luck :)

onError is for errors that happen along the operator chain. What you do in onSuccess is already at the end of it, and will not be reported in onError.