I'm trying to use RX Java to consume some data coming from a source that keeps sending objects.
I'm wondering how to implement a retry policy for cases in which my own code throws an exception. For example a network exception should trigger a retry with exponential backoff policy.
Some code :
message.map(this::processMessage)
.subscribe((message)->{
//do something after mapping
});
processMessage(message) is the method which contains the risky code that might fail and its the part of code that I want to retry but I dont want to stop the observable from consuming data from the source.
Any thoughts on this?
message
.map(this::processMessage)
.retryWhen(errors -> errors.flatMap(error -> {
if (error instanceof IOException) {
return Observable.just(null);
}
// For anything else, don't retry
return Observable.error(error);
})
.subscribe(
System.out::println,
error -> System.out.println("Error!")
);
or catch the error
message.map(this::processMessage)
.onErrorReturn(error -> "Empty result")
.subscribe((message)->{})
or procses the error
message
.map(this::processMessage)
.doOnError(throwable -> Log.e(TAG, "Throwable " + throwable.getMessage()))
.subscribe(
System.out::println,
error -> System.out.println("Error!")
);
Untested, but retryWhen differs to repeatWhen that is not only called in onComplete.
http://blog.danlew.net/2016/01/25/rxjavas-repeatwhen-and-retrywhen-explained/
-> Each error is flatmapped so that we can either return onNext(null) (to trigger a resubscription) or onError(error) (to avoid resubscription).
Backoff Policy:
source.retryWhen(errors ->
errors
.zipWith(Observable.range(1, 3), (n, i) -> i)
.flatMap(retryCount -> Observable.timer((long) Math.pow(5, retryCount), TimeUnit.SECONDS))
);
flatMap + timer is preferable over delay in this case because it lets us modify the delay by the number of retries. The above retries three times and delays each retry by 5 ^ retryCount, giving you exponential backoff with just a handful of operators!
Take an example from articles:
https://medium.com/#v.danylo/server-polling-and-retrying-failed-operations-with-retrofit-and-rxjava-8bcc7e641a5a#.a6ll8d5bt
http://kevinmarlow.me/better-networking-with-rxjava-and-retrofit-on-android/
They helped me oneday.
Recently I developed the library that exactly suits your needs.
RetrofitRxErrorHandler
If you combine Exponential strategy with backupObservable you will get the expected result.
Related
I have a similar problem to this question and I do not see an accepted answer. I have researched through and did not get a satisfactory answer.
I have a reactive Kafka consumer (Spring Reactor) with a poll amount 'x' and the application pushes the messages polled to a reactive endpoint using reactive webclient. The issue here is that the external service can perform differently overtime and I will have to adjust the Kafka consumer to poll less messages when the circuit breaker opens (Or kick in backpressure) when we see lot of failures. Is there a way in the current reactor to automatically
React when the circuit breaker is in open state and reduce the poll amount or slow down the consumption.
Increase the poll amount to the previous state when the circuit is closed ( External service would scaled up if it goes down ).
I do not want to use delayElements or delayUntil since these are mostly static in nature and want the application to react during runtime. How can I configure these end to end backpressure? I would provide the values for consumers when the circuit is closed, partially closed and open in app configs.
As backpressure is based on the slowness of the consumer, one way to achieve this is to convert certain exception types to delay. You can use the onErrorResume for this purpose as demonstrated below:
long start = System.currentTimeMillis();
Flux.range(1, 1000)
.doOnNext(item -> System.out.println("Elpased " + (System.currentTimeMillis() - start) + " millis for item: " + item))
.flatMap(item -> process(item).onErrorResume(this::slowDown), 5) // concurrency limit for demo
.blockLast();
System.out.println("Flow took " + (System.currentTimeMillis() - start) + " milliseconds.");
private Mono<Integer> process(Integer item) {
// simulate error for some items
if (item >= 50 && item <= 100) {
return Mono.error(new RuntimeException("Downstream failed."));
}
// normal processing
return Mono.delay(Duration.ofMillis(10))
.thenReturn(item);
}
private Mono<Integer> slowDown(Throwable e) {
if (e instanceof RuntimeException) { // you could check for circuit breaker exception
return Mono.delay(Duration.ofMillis(1000)).then(Mono.empty()); // delay to slow down
}
return Mono.empty(); // no delay for other errors
}
If you check the output of this code, you can see there is some slow down between the items 50 and 100 but it works at regular speed before and after.
Note that my example does not use Kafka. As you are using reactor-kafka library which honors backpressure it is supposed to work the same way as this dummy example.
Also, as the Flux might process items concurrently, the slow down is not immediate, it will try to process some additional items before properly slowing down.
I am using RxJava2 Flowables by subscribing to a stream of events from a PublishSubject.It's being used in enterprise level application and we don't have the choice of dropping any events.
I am using version RxJava 2.2.8
I am using BackpressureStrategy.BUFFER as I don't want to lose any of my events.
Also, I buffer again for 50000 or 3 minutes whichever is earlier. This I do as I want to consolidate events and then process them.
But I get the following errors in a few minutes of my run
io.reactivex.exceptions.MissingBackpressureException: Could not emit buffer due to lack of requests
at io.reactivex.internal.subscribers.QueueDrainSubscriber.fastPathOrderedEmitMax(QueueDrainSubscriber.java:121)
at io.reactivex.internal.operators.flowable.FlowableBufferTimed$BufferExactBoundedSubscriber.run(FlowableBufferTimed.java:569)
at io.reactivex.Scheduler$Worker$PeriodicTask.run(Scheduler.java:479)
at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
I tried increasing the buffer size by setting up, but there is no change in the behavior.
System.setProperty("rx2.buffer-size", "524288");
Also If I buffer for a longer time instead of 3 minutes, I get the exception after much longer time probably because my downstream performs better when the events are consolidated more. However, I don't have that choice because these are live events and needs processing immediately(in 3-5 minutes).
I also tried thread.sleep() before invoking the "subscription.next" in case of error but still getting the same results.
keySubject.hide()
.toFlowable(BackpressureStrategy.BUFFER)
.parallel()
.runOn(Schedulers.computation())
.map(e -> e.getContents())
.flatMap(s -> Flowable.fromIterable(s))
.sequential()
.buffer(3,TimeUnit.MINUTES,50000)
.subscribe(new Subscriber<List<String>>() {
#Override
public void onSubscribe(Subscription var1) {
innerSubscription = var1;
innerSubscription.request(1L);
}
#Override
public void onNext(List<String> logs) {
Subscription.request(1L);
/// Do some logic here
}
I want to know How do I handle the backpressure to avoid this exception? Is this exception because of ".buffer" method
Is there a way for me to check the status of these buffers. Also why even if I increase the rx2.buffer-size, I still get the exception in the same amount of time. Ideally, the system should run longer with a higher buffer size if the exception is because if buffer getting full.
Any help on the reason for this message "Could not emit buffer due to lack of requests at " will be great.
The thing is, why do you use a subject that isn't backpressure-aware? Are you using that as a poor man's event bus? Also, assuming e.getContents() is a simple getter I believe you can replace this whole block
.toFlowable(BackpressureStrategy.BUFFER)
.parallel()
.runOn(Schedulers.computation())
.map(e -> e.getContents())
.flatMap(s -> Flowable.fromIterable(s))
.sequential()
.buffer(3,TimeUnit.MINUTES,50000)
.subscribe(new Subscriber<List<String>>() { ... });
with
.flatMapIterable(e -> e.getContents())
.buffer(3,TimeUnit.MINUTES,50000)
.rebatchRequests(1)
.observeOn(Schedulers.computation())
.doOnNext(s -> /* Do some logic here */)
.subscribe();
I need to send some data after user registered. I want to do first attempt in main thread, but if there are any errors, I want to retry 5 times with 10 minutes interval.
#Override
public void sendRegisterInfo(MailData data) {
Mono.just(data)
.doOnNext(this::send)
.doOnError(ex -> logger.warn("Main queue {}", ex.getMessage()))
.doOnSuccess(d -> logger.info("Send mail to {}", d.getRecipient()))
.onErrorResume(ex -> retryQueue(data))
.subscribe();
}
private Mono<MailData> retryQueue(MailData data) {
return Mono.just(data)
.delayElement(Duration.of(10, ChronoUnit.MINUTES))
.doOnNext(this::send)
.doOnError(ex -> logger.warn("Retry queue {}", ex.getMessage()))
.doOnSuccess(d -> logger.info("Send mail to {}", d.getRecipient()))
.retry(5)
.subscribe();
}
It works.
But I've got some questions:
Did I correct to make operation in doOnNext function?
Is it correct to use delayElement to make a delay between executions?
Did the thread blocked when waiting for delay?
And what the best practice to make a retries on error and make a delay between it?
doOnXXX for logging is fine. But for the actual element processing, you must prefer using flatMap rather than doOnNext (assuming your processing is asynchronous / can be converted to returning a Flux/Mono).
This is correct. Another way is to turn the code around and start from a Flux.interval, but here delayElement is better IMO.
The delay runs on a separate thread/scheduler (by default, Schedulers.parallel()), so not blocking the main thread.
There's actually a Retry builder dedicated to that kind of use case in the reactor-extra addon: https://github.com/reactor/reactor-addons/blob/master/reactor-extra/src/main/java/reactor/retry/Retry.java
I have a PublishSubject which receive emits from UI:
myPublishSubject
.map {
...
}
.doOnNext {
// using emitted item
}
.timeout (...) // wait for the gap!
.doOnNext {
// running a function after a specific gap between two item
}
.subscribe()
I want to wait a specific amount of time after last emit (not onComplete, cause it continue emitting later) and run a function. It can be interpreted as a gap between item emotion.
I am looking for something like timeout but this method issue is it kills the Observable with error.
You have to be a bit creative with publish and switchMap for example:
PublishSubject<Integer> ps = PublishSubject.create();
ps.publish(o ->
o.mergeWith(
o.switchMap(e ->
Observable.just(1).delay(200, TimeUnit.MILLISECONDS)
.ignoreElements()
.doOnCompleted(() -> System.out.println("Timeout action: " + e))
)
)
).subscribe(System.out::println);
ps.onNext(1);
ps.onNext(2);
Thread.sleep(100);
ps.onNext(3);
Thread.sleep(250);
ps.onNext(4);
Thread.sleep(250);
It works by sharing a source and routing into two ways, one is directly emitting while the other feeds a switchMap that when receives a new item, starts a delayed Observable and reacts to its completion (ignoring the original trigger element to avoid duplicate events due to mergeWith). When there is a new signal during the grace period, switchMap will cancel the previous delay and start with the newer delay.
I´m using retryWhen when a external http request to one of my external services fails.
The problem is that I´m using
RxHelper.toObservable(httpClient.request(method, url))
To get my observable response, and becuase vertx internally use ReadStreamAdapter I cannot use the retryWhen because it´s complain
java java.lang.IllegalStateException: Request already complete
Here a code example:
RxHelper.toObservable(httpClient.request(method, url))
.retryWhen(new ServiceExceptionRetry())
.subscribe(f -> replySuccess(eventMsg, event, f), t -> handleError(t, eventMsg, event));
Any idea how to achieve this?
You can use defer to create an Observable from method and client every time like this:
Observable.defer(() -> RxHelper.toObservable(httpClient.request(method, url)))
.retryWhen(new ServiceExceptionRetry())
.subscribe(f -> replySuccess(eventMsg, event, f), t -> handleError(t, eventMsg, event));