How to Interrupt/Abort Rx Java Observable Chain Subscription? - java

I have a requirement to abort the Rx Java Observable chain, Take Until helps partially but does a graceful completion of Job. I need a handle onAborted on all the the observables part of this chain. Dispose/CompositeDispose none of them are helping to delegate the onAborted handle upward in observable.
Note: The RxJava is not being used here for UI Operations, rather to process/transform shorter data streams in memory.
dataObservable.map(row -> convertRow())
//.takeUntil(getAbortPredicate()/*return true when abort set to true*/)
.collect(Collectors.toList())
.doFinally(getResetAbortAction()/*resets abort flag*/)
.blockingGet()
The dataObservable itself here is chain of observables. I want to interrupt all such observables involved & get a handle on abort for each of them.
public class A {
private Observable<Row> observable;
private Boolean aborted;
private SomeUtilityWithResources utilty
public A(Observable<Row> observable){
this.observable = observable;
this.aborted = false;
this.utilty = SomeUtilityWithResources.Init();
}
public A mergeOp1(A aRight){
Observable<Row> merged = observable.flatMap(lr -> aRight.observable.map(rr -> utilty.merge1(lr.toList(), rr.toList())));
//When abruptly aborted I want to call utilty.resources.abort().... something like joined.OnAborted(utility.resources.abort())
//Tried OnDispose()/OnError doesn't get invoked though
//utilty.merge1 also returns an Observable
return new A(merged)
}
public A mergeOp2(A aRight){
Observable<Row> merged = observable.flatMap(lr -> aRight.observable.map(rr -> utilty.merge2(lr.toList(), rr.toList())));
//When abruptly aborted I want to call utilty.resources.abort().... something like joined.OnAborted(utility.resources.abort())
//Tried OnDispose()/OnError doesn't get invoked though
//utilty.merge2 also returns an Observable
return new A(merged)
}
public A mergeOp3(A aRight){
Observable<Row> merged = observable.flatMap(lr -> aRight.observable.map(rr -> utilty.merge3(lr.toList(), rr.toList())));
//When abruptly aborted I want to call utilty.resources.abort().... something like joined.OnAborted(utility.resources.abort())
//Tried OnDispose()/OnError doesn't get invoked though
//utilty.merge3 also returns an Observable
return new A(merged)
}
public List<Row> execWay1(){
observable.map(row -> convertRowWay1())
//.takeUntil(getAbortPredicate()/*return true when abort set to true*/)
.collect(Collectors.toList())
.doFinally(getResetAbortAction()/*resets abort flag*/)
.blockingGet()
}
public Long count(){
observable
//.takeUntil(getAbortPredicate()/*return true when abort set to true*/)
.count()
.doFinally(getResetAbortAction(action.toString()))
.blockingGet()
}
public void abort(){
this.aborted = true;
}
private Action getResetAbortAction(String action) {
return new Action() {
#Override
public void run() throws Throwable {
if (aborted.get()) {
aborted.set(false);
//LOGGER.error(action + " execution aborted successfully!");
//throw new Exception("Aborted")
}
}
};
}
private Predicate<T> getAbortPredicate() {
return new Predicate<T>() {
#Override
public boolean test(T t) throws Exception {
return aborted.get();
}
};
}
}
/* Executor would be calling:
A a = SomeUtilityWithResources.getDefaultObservableFromSomeSource();
a = a.mergeOp1.mergeOp2.mergedOp3
a.count()
//if taking longer.... a.abort() on seperate thread;
a.execWay1();

Related

Mockito Junit Test - “Wanted but not invoked; However there were other interactions with this mock” error

I have to runnable function in Completable Future with timeout. The runnable function should be invoked only when the original method takes more than the given timeout. The unit keeps giving=
Wanted but not invoked: However, there were exactly 3 interactions with this mock.
All I am trying to do is, I am trying to add timeout for a method execution (getResponseWithTimeoutFunction(itemRequest)) and if the method takes more time, then terminate it and publish count(to understand the timed out response rate) as metric.
#Test
public void testTimeoutFunction() throws Exception {
Response response = getResponseForTest();
when(requestAdapter.transform(itemRequest)).thenReturn(Request);
when(dataProvider
.provide(any(Request.class)))
.thenAnswer((Answer<Response>) invocation -> {
Thread.sleep(1000000);
return response;
});
processor = spy(getProcessor());
when(itemRequest.getRequestContext()).thenReturn(itemRequestContext);
when(itemRequestContext.getMetadata()).thenReturn(requestContextMetadata);
List<Item> output = processor.getItemist(ITEM_ID, itemRequest);
assertTrue(output.isEmpty());
verify(processor, times(1)).processRequest(Request);
verify(processor, times(1)).responseTimedOutCount();
}
This is method which I am testing for:
public class Process {
#VisibleForTesting
void responseTimedOutCount() {
//log metrics
}
private CompletableFuture<Response> getResponseAsync(final ScheduledExecutorService delayer,
final ItemRequest itemRequest) {
return timeoutWithTimeoutFunction(delayer, EXECUTION_TIMEOUT, TimeUnit.MILLISECONDS,
CompletableFuture.supplyAsync(() -> getResponseWithTimeoutFunction(itemRequest), executorService),
Response.emptyResponse(), () -> responseTimedOutCount());
}
private Response getResponseWithTimeoutFunction(final ItemRequest itemRequest) {
//do something and return response
}
public List<Item> getItemList(final String id, final ItemRequest itemRequest) throws Exception {
final ScheduledExecutorService delayer = Executors.newScheduledThreadPool(1);
Response response;
if(validateItemId(id){
try {
response = getResponseAsync(delayer, itemRequest).get();
} catch (final Throwable t) {
response = Response.emptyResponse();
} finally {
delayer.shutdown();
}
return transform(response, id).getItems();
} else {
return null;
}
}
}
And the timeout function use =
public static <T> CompletableFuture<T> timeoutWithTimeoutFunction(final ScheduledExecutorService es,
final long timeout,
final TimeUnit unit,
final CompletableFuture<T> f,
final T defaultValue,
final Runnable r) {
final Runnable timeoutFunction = () -> {
boolean timedOut = f.complete(defaultValue);
if (timedOut && r != null) {
r.run();
}
};
es.schedule(timeoutFunction, timeout, unit);
return f;
}
Exception from Junit :
Wanted but not invoked: process.responseTimedOutCount(); -> at processTest.testTimeoutFunction(processTest.java:377)
However, there were exactly 3 interactions with this mock:
process.getItemList( ITEM_ID, itemRequest ); -> at processTest.testTimeoutFunction(processTest.java:373)
process.validateItemId( ITEM_ID ); -> at process.getItemList(process.java:133)
process.processRequest( request ); -> at process.getResponseWithTimeoutFunction(process.java:170)
To test timeouts you probably want to mock the call you want the timeout tested for. Relative to the duration of the test it should take forever.
when(dataProvider
.provide(any(Request.class)))
.thenAnswer((Answer<Response>) invocation -> {
Thread.sleep(FOREVER);
return response;
});
The verify should have a timeout for threading handling. When the timeout is long, you probably should make sure it is configurable to allow a fast test. Something like verify(mock, timeout(LONGER_THAN_REAL_TIMEOUT)).someCall()
Make sure to put a timeout on the total test duration to make sure that current or future failures will not slow down your builds.
For testing asynchronous invocation, you can use:
verify(processor, timeout(1000)).processRequest(request);

Creating a Flowable that emits items at a limited rate to avoid the need to buffer events

I've got a data access object that passes each item in a data source to a consumer:
public interface Dao<T> {
void forEachItem(Consumer<T> item);
}
This always produces items in a single threaded way - I can't currently change this interface.
I wanted to create a Flowable from this interface:
private static Flowable<String> flowable(final Dao dao) {
return Flowable.create(emitter -> {
dao.forEachItem(item ->
emitter.onNext(item));
emitter.onComplete();
}, ERROR);
}
If I use this Flowable in a situation where the processing takes longer than the rate at which items are emitted then I understandably get a missing back pressure exception as I am using ERROR mode:
Dao<String> exampleDao =
itemConsumer ->
IntStream.range(0, 1_000).forEach(i ->
itemConsumer.accept(String.valueOf(i)));
flowable(exampleDao)
.map(v -> {
Thread.sleep(100);
return "id:" + v;
})
.blockingSubscribe(System.out::println);
I don't wish to buffer items - seems like this could lead to exhausting memory on very large data sets - if the operation is significantly slower than the producer.
I was hoping there would be a backpressure mode that would allow the emitter to block when passed next/completion events when it detects back pressure but that does not seem to be the case?
In my case as I know that the dao produces items in a single threaded way I thought I would be able to do something like:
dao.forEachItem(item -> {
while (emitter.requested() == 0) {
waitABit();
}
emitter.onNext(item)
});
but this seems to hang forever.
How wrong is my approach? :-) Is there a way of producing items in a way that respects downstream back pressure given my (relatively restrictive) set of circumstances?
I know I could do this with a separate process writing to a queue and then write a Flowable based on consuming from that queue- would that be the preferred approach instead?
Check the part of the Flowable, especially the part with Supscription.request(long). I hope that gets you on the right way.
The TestProducerfrom this example produces Integerobjects in a given range and pushes them to its Subscriber. It extends the Flowable<Integer> class. For a new subscriber, it creates a Subscription object whose request(long) method is used to create and publish the Integer values.
It is important for the Subscription that is passed to the subscriber that the request() method which calls onNext()on the subscriber can be recursively called from within this onNext() call. To prevent a stack overflow, the shown implementation uses the outStandingRequests counter and the isProducing flag.
class TestProducer extends Flowable<Integer> {
static final Logger logger = LoggerFactory.getLogger(TestProducer.class);
final int from, to;
public TestProducer(int from, int to) {
this.from = from;
this.to = to;
}
#Override
protected void subscribeActual(Subscriber<? super Integer> subscriber) {
subscriber.onSubscribe(new Subscription() {
/** the next value. */
public int next = from;
/** cancellation flag. */
private volatile boolean cancelled = false;
private volatile boolean isProducing = false;
private AtomicLong outStandingRequests = new AtomicLong(0);
#Override
public void request(long n) {
if (!cancelled) {
outStandingRequests.addAndGet(n);
// check if already fulfilling request to prevent call between request() an subscriber .onNext()
if (isProducing) {
return;
}
// start producing
isProducing = true;
while (outStandingRequests.get() > 0) {
if (next > to) {
logger.info("producer finished");
subscriber.onComplete();
break;
}
subscriber.onNext(next++);
outStandingRequests.decrementAndGet();
}
isProducing = false;
}
}
#Override
public void cancel() {
cancelled = true;
}
});
}
}
The Consumer in this example extends DefaultSubscriber<Integer> and on start and after consuming an Integer requests the next one. On consuming the Integer values, there is a little delay, so the backpressure will be built up for the producer.
class TestConsumer extends DefaultSubscriber<Integer> {
private static final Logger logger = LoggerFactory.getLogger(TestConsumer.class);
#Override
protected void onStart() {
request(1);
}
#Override
public void onNext(Integer i) {
logger.info("consuming {}", i);
if (0 == (i % 5)) {
try {
Thread.sleep(500);
} catch (InterruptedException ignored) {
// can be ignored, just used for pausing
}
}
request(1);
}
#Override
public void onError(Throwable throwable) {
logger.error("error received", throwable);
}
#Override
public void onComplete() {
logger.info("consumer finished");
}
}
in the following main method of a test class the producer and consumer are created and wired up:
public static void main(String[] args) {
try {
final TestProducer testProducer = new TestProducer(1, 1_000);
final TestConsumer testConsumer = new TestConsumer();
testProducer
.subscribeOn(Schedulers.computation())
.observeOn(Schedulers.single())
.blockingSubscribe(testConsumer);
} catch (Throwable t) {
t.printStackTrace();
}
}
When running the example, the logfile shows that the consumer runs continuously, while the producer only gets active when the internal Flowable buffer of rxjava2 needs to be refilled.

Cancel task on timeout in RxJava

I'm experimenting with RxJava and Java 8's CompletableFuture class
and do not quite get how to handle timeout conditions.
import static net.javacrumbs.futureconverter.java8rx.FutureConverter.toObservable;
// ...
Observable<String> doSomethingSlowly() {
CompletableFuture<PaymentResult> task = CompletableFuture.supplyAsync(() -> {
// this call may be very slow - if it takes too long,
// we want to time out and cancel it.
return processor.slowExternalCall();
});
return toObservable(task);
}
// ...
doSomethingSlowly()
.single()
.timeout(3, TimeUnit.SECONDS, Observable.just("timeout"));
This basically works (if the timeout of three seconds is reached, "timeout" is published). I would however additionally want to cancel the future task that I've wrapped in an Observable - is that possible with an RxJava-centric approach?
I know that one option would be to handle the timeout myself, using task.get(3, TimeUnit.SECONDS), but I wonder if it's possible to do all task handling stuff in RxJava.
Yes, you can do this. You would add a Subscription to the Subscriber.
This allows you to listen in on unsubscriptions, which will happen if you explicitly call subscribe().unsubscribe() or if the Observable completes successfully or with an error.
If you see an unsubscription before the future has completed, you can assume it's because of either an explicit unsubscribe or a timeout.
public class FutureTest {
public static void main(String[] args) throws IOException {
doSomethingSlowly()
.timeout(1, TimeUnit.SECONDS, Observable.just("timeout"))
.subscribe(System.out::println);
System.in.read(); // keep process alive
}
private static Observable<String> doSomethingSlowly() {
CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
}
return "Something";
});
return toObservable(future);
}
private static <T> Observable<T> toObservable(CompletableFuture<T> future) {
return Observable.create(subscriber -> {
subscriber.add(new Subscription() {
private boolean unsubscribed = false;
#Override
public void unsubscribe() {
if (!future.isDone()){
future.cancel(true);
}
unsubscribed = true;
}
#Override
public boolean isUnsubscribed() {
return unsubscribed;
}
});
future.thenAccept(value -> {
if (!subscriber.isUnsubscribed()){
subscriber.onNext(value);
subscriber.onCompleted();
}
}).exceptionally(throwable -> {
if (!subscriber.isUnsubscribed()) {
subscriber.onError(throwable);
}
return null;
});
});
}
}

Is there an observable that just propagates the error without terminating itself?

I am using PublishSubject in the class that is responsible for synchronization. When the synchronization is done all the subscribers will be notified. The same happens in case of an error.
I've noticed that the next time I subscribe after an error has occured, it is immediately return to the subscriber.
So the class may look like this:
public class Synchronizer {
private final PublishSubject<Result> mSyncHeadObservable = PublishSubject.create();
private final ThreadPoolExecutor mExecutor = new ThreadPoolExecutor(1, 1,
10, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(true),
new ThreadPoolExecutor.DiscardPolicy());
public Observable<Result> syncHead(final int chunkSize) {
mExecutor.execute(new Runnable() {
#Override
public void run() {
try {
//Do some work which either returns a result or throws an error
//...
mSyncHeadObservable.onNext(Notification.createOnNext(/*some result*/));
} catch (Throwable error) {
mSyncHeadObservable.onError(Notification.<Result>createOnError(error));
}
}
});
Is there an observable which can just serve as an proxy? May be some other Rx approach?
UPDATE:
I've followed #akarnokd approach and emit the events wrapped into the RxJava Notification. Then unwrap them via flatMap(). So the clients of Synchronizer class won't need to do it.
//...
private PublishSubject<Notification<Result>> mSyncHeadObservable = PublishSubject.create();
public Observable<Result> syncHead(final int chunkSize) {
return mSyncHeadObservable.flatMap(new Func1<Notification<Result>, Observable<Result>>() {
#Override
public Observable<Result> call(Notification<Result> result) {
if (result.isOnError()) {
return Observable.error(result.getThrowable());
}
return Observable.just(result.getValue());
}
}).doOnSubscribe(
new Action0() {
#Override
public void call() {
startHeadSync(chunkSize);
}
});
}
private void startHeadSync(final int chunkSize) {
mExecutor.execute(new Runnable() {
#Override
public void run() {
try {
//Do some work which either returns a result or throws an error
//...
mSyncHeadObservable.onNext(Notification.createOnNext(/*some result*/));
} catch (Throwable error) {
mSyncHeadObservable.onError(Notification.<Result>createOnError(error));
}
}
});
}
//...
I'm not sure what your want to achieve with this setup, but generally, in order to avoid a terminal condition with PublishSubject, you should wrap your value and error into a common structure and always emit those, never any onError and onCompleted. One option is to use RxJava's own event wrapper, Notification, and your Subscribers should unwrap the value.
When a error occurred, the observable reached an terminal state.
If you want to continue to observe it, you should resubscribe to you observable with retry operator or use another error handling operators

rxjava: Can I use retry() but with delay?

I am using rxjava in my Android app to handle network requests asynchronously. Now I would like to retry a failed network request only after a certain time has passed.
Is there any way to use retry() on an Observable but to retry only after a certain delay?
Is there a way to let the Observable know that is is currently being retried (as opposed to tried for the first time)?
I had a look at debounce()/throttleWithTimeout() but they seem to be doing something different.
Edit:
I think I found one way to do it, but I'd be interested in either confirmation that this is the correct way to do it or for other, better ways.
What I am doing is this: In the call() method of my Observable.OnSubscribe, before I call the Subscribers onError() method, I simply let the Thread sleep for the desired amount of time. So, to retry every 1000 milliseconds, I do something like this:
#Override
public void call(Subscriber<? super List<ProductNode>> subscriber) {
try {
Log.d(TAG, "trying to load all products with pid: " + pid);
subscriber.onNext(productClient.getProductNodesForParentId(pid));
subscriber.onCompleted();
} catch (Exception e) {
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
e.printStackTrace();
}
subscriber.onError(e);
}
}
Since this method is running on an IO thread anyway it does not block the UI. The only problem I can see is that even the first error is reported with delay so the delay is there even if there's no retry(). I'd like it better if the delay wasn't applied after an error but instead before a retry (but not before the first try, obviously).
You can use the retryWhen() operator to add retry logic to any Observable.
The following class contains the retry logic:
RxJava 2.x
public class RetryWithDelay implements Function<Observable<? extends Throwable>, Observable<?>> {
private final int maxRetries;
private final int retryDelayMillis;
private int retryCount;
public RetryWithDelay(final int maxRetries, final int retryDelayMillis) {
this.maxRetries = maxRetries;
this.retryDelayMillis = retryDelayMillis;
this.retryCount = 0;
}
#Override
public Observable<?> apply(final Observable<? extends Throwable> attempts) {
return attempts
.flatMap(new Function<Throwable, Observable<?>>() {
#Override
public Observable<?> apply(final Throwable throwable) {
if (++retryCount < maxRetries) {
// When this Observable calls onNext, the original
// Observable will be retried (i.e. re-subscribed).
return Observable.timer(retryDelayMillis,
TimeUnit.MILLISECONDS);
}
// Max retries hit. Just pass the error along.
return Observable.error(throwable);
}
});
}
}
RxJava 1.x
public class RetryWithDelay implements
Func1<Observable<? extends Throwable>, Observable<?>> {
private final int maxRetries;
private final int retryDelayMillis;
private int retryCount;
public RetryWithDelay(final int maxRetries, final int retryDelayMillis) {
this.maxRetries = maxRetries;
this.retryDelayMillis = retryDelayMillis;
this.retryCount = 0;
}
#Override
public Observable<?> call(Observable<? extends Throwable> attempts) {
return attempts
.flatMap(new Func1<Throwable, Observable<?>>() {
#Override
public Observable<?> call(Throwable throwable) {
if (++retryCount < maxRetries) {
// When this Observable calls onNext, the original
// Observable will be retried (i.e. re-subscribed).
return Observable.timer(retryDelayMillis,
TimeUnit.MILLISECONDS);
}
// Max retries hit. Just pass the error along.
return Observable.error(throwable);
}
});
}
}
Usage:
// Add retry logic to existing observable.
// Retry max of 3 times with a delay of 2 seconds.
observable
.retryWhen(new RetryWithDelay(3, 2000));
Inspired by Paul's answer, and if you are not concerned with retryWhen problems stated by Abhijit Sarkar, the simplest way to delay resubscription with rxJava2 unconditionnaly is :
source.retryWhen(throwables -> throwables.delay(1, TimeUnit.SECONDS))
You may want to see more samples and explanations on retryWhen and repeatWhen.
This example works with jxjava 2.2.2:
Retry without delay:
Single.just(somePaylodData)
.map(data -> someConnection.send(data))
.retry(5)
.doOnSuccess(status -> log.info("Yay! {}", status);
Retry with delay:
Single.just(somePaylodData)
.map(data -> someConnection.send(data))
.retryWhen((Flowable<Throwable> f) -> f.take(5).delay(300, TimeUnit.MILLISECONDS))
.doOnSuccess(status -> log.info("Yay! {}", status)
.doOnError((Throwable error)
-> log.error("I tried five times with a 300ms break"
+ " delay in between. But it was in vain."));
Our source single fails if someConnection.send() fails.
When that happens, the observable of failures inside retryWhen emits the error.
We delay that emission by 300ms and send it back to signal a retry.
take(5) guarantees that our signaling observable will terminate after we receive five errors.
retryWhen sees the termination and doesn't retry after the fifth failure.
This is a solution based on Ben Christensen's snippets I saw, RetryWhen Example, and RetryWhenTestsConditional (I had to change n.getThrowable() to n for it to work). I used evant/gradle-retrolambda to make the lambda notation work on Android, but you don't have to use lambdas (although it's highly recommended). For the delay I implemented exponential back-off, but you can plug in what ever backoff logic you want there. For completeness I added the subscribeOn and observeOn operators. I'm using ReactiveX/RxAndroid for the AndroidSchedulers.mainThread().
int ATTEMPT_COUNT = 10;
public class Tuple<X, Y> {
public final X x;
public final Y y;
public Tuple(X x, Y y) {
this.x = x;
this.y = y;
}
}
observable
.subscribeOn(Schedulers.io())
.retryWhen(
attempts -> {
return attempts.zipWith(Observable.range(1, ATTEMPT_COUNT + 1), (n, i) -> new Tuple<Throwable, Integer>(n, i))
.flatMap(
ni -> {
if (ni.y > ATTEMPT_COUNT)
return Observable.error(ni.x);
return Observable.timer((long) Math.pow(2, ni.y), TimeUnit.SECONDS);
});
})
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
instead of using MyRequestObservable.retry I use a wrapper function retryObservable(MyRequestObservable, retrycount, seconds) which return a new Observable that handle the indirection for the delay so I can do
retryObservable(restApi.getObservableStuff(), 3, 30)
.subscribe(new Action1<BonusIndividualList>(){
#Override
public void call(BonusIndividualList arg0)
{
//success!
}
},
new Action1<Throwable>(){
#Override
public void call(Throwable arg0) {
// failed after the 3 retries !
}});
// wrapper code
private static <T> Observable<T> retryObservable(
final Observable<T> requestObservable, final int nbRetry,
final long seconds) {
return Observable.create(new Observable.OnSubscribe<T>() {
#Override
public void call(final Subscriber<? super T> subscriber) {
requestObservable.subscribe(new Action1<T>() {
#Override
public void call(T arg0) {
subscriber.onNext(arg0);
subscriber.onCompleted();
}
},
new Action1<Throwable>() {
#Override
public void call(Throwable error) {
if (nbRetry > 0) {
Observable.just(requestObservable)
.delay(seconds, TimeUnit.SECONDS)
.observeOn(mainThread())
.subscribe(new Action1<Observable<T>>(){
#Override
public void call(Observable<T> observable){
retryObservable(observable,
nbRetry - 1, seconds)
.subscribe(subscriber);
}
});
} else {
// still fail after retries
subscriber.onError(error);
}
}
});
}
});
}
Based on kjones answer here is Kotlin version of RxJava 2.x retry with a delay as an extension. Replace Observable to create the same extension for Flowable.
fun <T> Observable<T>.retryWithDelay(maxRetries: Int, retryDelayMillis: Int): Observable<T> {
var retryCount = 0
return retryWhen { thObservable ->
thObservable.flatMap { throwable ->
if (++retryCount < maxRetries) {
Observable.timer(retryDelayMillis.toLong(), TimeUnit.MILLISECONDS)
} else {
Observable.error(throwable)
}
}
}
}
Then just use it on observable observable.retryWithDelay(3, 1000)
retryWhen is a complicated, perhaps even buggy, operator. The official doc and at least one answer here use range operator, which will fail if there are no retries to be made. See my discussion with ReactiveX member David Karnok.
I improved upon kjones' answer by changing flatMap to concatMap and by adding a RetryDelayStrategy class. flatMap doesn't preserve order of emission while concatMap does, which is important for delays with back-off. The RetryDelayStrategy, as the name indicates, let's the user choose from various modes of generating retry delays, including back-off.
The code is available on my GitHub complete with the following test cases:
Succeeds on 1st attempt (no retries)
Fails after 1 retry
Attempts to retry 3 times but succeeds on 2nd hence doesn't retry 3rd time
Succeeds on 3rd retry
See setRandomJokes method.
Same answer as from kjones but updated to latest version
For RxJava 2.x version: ('io.reactivex.rxjava2:rxjava:2.1.3')
public class RetryWithDelay implements Function<Flowable<Throwable>, Publisher<?>> {
private final int maxRetries;
private final long retryDelayMillis;
private int retryCount;
public RetryWithDelay(final int maxRetries, final int retryDelayMillis) {
this.maxRetries = maxRetries;
this.retryDelayMillis = retryDelayMillis;
this.retryCount = 0;
}
#Override
public Publisher<?> apply(Flowable<Throwable> throwableFlowable) throws Exception {
return throwableFlowable.flatMap(new Function<Throwable, Publisher<?>>() {
#Override
public Publisher<?> apply(Throwable throwable) throws Exception {
if (++retryCount < maxRetries) {
// When this Observable calls onNext, the original
// Observable will be retried (i.e. re-subscribed).
return Flowable.timer(retryDelayMillis,
TimeUnit.MILLISECONDS);
}
// Max retries hit. Just pass the error along.
return Flowable.error(throwable);
}
});
}
}
Usage:
// Add retry logic to existing observable.
// Retry max of 3 times with a delay of 2 seconds.
observable
.retryWhen(new RetryWithDelay(3, 2000));
Now with RxJava version 1.0+ you can use zipWith to achieve retry with delay.
Adding modifications to kjones answer.
Modified
public class RetryWithDelay implements
Func1<Observable<? extends Throwable>, Observable<?>> {
private final int MAX_RETRIES;
private final int DELAY_DURATION;
private final int START_RETRY;
/**
* Provide number of retries and seconds to be delayed between retry.
*
* #param maxRetries Number of retries.
* #param delayDurationInSeconds Seconds to be delays in each retry.
*/
public RetryWithDelay(int maxRetries, int delayDurationInSeconds) {
MAX_RETRIES = maxRetries;
DELAY_DURATION = delayDurationInSeconds;
START_RETRY = 1;
}
#Override
public Observable<?> call(Observable<? extends Throwable> observable) {
return observable
.delay(DELAY_DURATION, TimeUnit.SECONDS)
.zipWith(Observable.range(START_RETRY, MAX_RETRIES),
new Func2<Throwable, Integer, Integer>() {
#Override
public Integer call(Throwable throwable, Integer attempt) {
return attempt;
}
});
}
}
You can add a delay in the Observable returned in the retryWhen Operator
/**
* Here we can see how onErrorResumeNext works and emit an item in case that an error occur in the pipeline and an exception is propagated
*/
#Test
public void observableOnErrorResumeNext() {
Subscription subscription = Observable.just(null)
.map(Object::toString)
.doOnError(failure -> System.out.println("Error:" + failure.getCause()))
.retryWhen(errors -> errors.doOnNext(o -> count++)
.flatMap(t -> count > 3 ? Observable.error(t) : Observable.just(null).delay(100, TimeUnit.MILLISECONDS)),
Schedulers.newThread())
.onErrorResumeNext(t -> {
System.out.println("Error after all retries:" + t.getCause());
return Observable.just("I save the world for extinction!");
})
.subscribe(s -> System.out.println(s));
new TestSubscriber((Observer) subscription).awaitTerminalEvent(500, TimeUnit.MILLISECONDS);
}
You can see more examples here. https://github.com/politrons/reactive
Worked from me with
//retry with retryCount time after 1 sec of delay
observable.retryWhen(throwableFlowable -> {
return throwableFlowable.take(retryCount).delay(1, TimeUnit.SECONDS);
});
Simply do it like this:
Observable.just("")
.delay(2, TimeUnit.SECONDS) //delay
.flatMap(new Func1<String, Observable<File>>() {
#Override
public Observable<File> call(String s) {
L.from(TAG).d("postAvatar=");
File file = PhotoPickUtil.getTempFile();
if (file.length() <= 0) {
throw new NullPointerException();
}
return Observable.just(file);
}
})
.retry(6)
.subscribe(new Action1<File>() {
#Override
public void call(File file) {
postAvatar(file);
}
}, new Action1<Throwable>() {
#Override
public void call(Throwable throwable) {
}
});
For Kotlin & RxJava1 version
class RetryWithDelay(private val MAX_RETRIES: Int, private val DELAY_DURATION_IN_SECONDS: Long)
: Function1<Observable<out Throwable>, Observable<*>> {
private val START_RETRY: Int = 1
override fun invoke(observable: Observable<out Throwable>): Observable<*> {
return observable.delay(DELAY_DURATION_IN_SECONDS, TimeUnit.SECONDS)
.zipWith(Observable.range(START_RETRY, MAX_RETRIES),
object : Function2<Throwable, Int, Int> {
override fun invoke(throwable: Throwable, attempt: Int): Int {
return attempt
}
})
}
}
(Kotlin) I little bit improved code with exponential backoff and applied defense emitting of Observable.range():
fun testOnRetryWithDelayExponentialBackoff() {
val interval = 1
val maxCount = 3
val ai = AtomicInteger(1);
val source = Observable.create<Unit> { emitter ->
val attempt = ai.getAndIncrement()
println("Subscribe ${attempt}")
if (attempt >= maxCount) {
emitter.onNext(Unit)
emitter.onComplete()
}
emitter.onError(RuntimeException("Test $attempt"))
}
// Below implementation of "retryWhen" function, remove all "println()" for real code.
val sourceWithRetry: Observable<Unit> = source.retryWhen { throwableRx ->
throwableRx.doOnNext({ println("Error: $it") })
.zipWith(Observable.range(1, maxCount)
.concatMap { Observable.just(it).delay(0, TimeUnit.MILLISECONDS) },
BiFunction { t1: Throwable, t2: Int -> t1 to t2 }
)
.flatMap { pair ->
if (pair.second >= maxCount) {
Observable.error(pair.first)
} else {
val delay = interval * 2F.pow(pair.second)
println("retry delay: $delay")
Observable.timer(delay.toLong(), TimeUnit.SECONDS)
}
}
}
//Code to print the result in terminal.
sourceWithRetry
.doOnComplete { println("Complete") }
.doOnError({ println("Final Error: $it") })
.blockingForEach { println("$it") }
}
in the event when you need to print out the retry count,
you can use the example provided in Rxjava's wiki page https://github.com/ReactiveX/RxJava/wiki/Error-Handling-Operators
observable.retryWhen(errors ->
// Count and increment the number of errors.
errors.map(error -> 1).scan((i, j) -> i + j)
.doOnNext(errorCount -> System.out.println(" -> query errors #: " + errorCount))
// Limit the maximum number of retries.
.takeWhile(errorCount -> errorCount < retryCounts)
// Signal resubscribe event after some delay.
.flatMapSingle(errorCount -> Single.timer(errorCount, TimeUnit.SECONDS));
Use retryWhen
/**
* Retry Handler Support
* #param errors
* #param predicate filter error
* #param maxTry
* #param periodStrategy
* #param timeUnit
* #return
*/
private Flowable<?> retrySupport(Flowable<Throwable> errors, Predicate<? super Throwable> predicate , Integer maxTry , Function<Long, Long> periodStrategy , TimeUnit timeUnit )
{
LongAdder errorCount = new LongAdder();
return errors
.doOnNext(e -> {
errorCount.increment();
long currentCount = errorCount.longValue();
boolean tryContinue = predicate.test(e) && currentCount < maxTry;
Logger.i("No. of errors: %d , %s", currentCount,
tryContinue ? String.format("please wait %d %s.", periodStrategy.apply(currentCount), timeUnit.name()) : "skip and throw");
if(!tryContinue)
throw e;
} )
.flatMapSingle(e -> Single.timer( periodStrategy.apply(errorCount.longValue()), timeUnit));
}
Sample
private Single<DeviceInfo> getErrorToken( String device)
{
return Single.error( new IOException( "network is disconnect!" ) );
}
//only retry when emit IOExpcetion
//delay 1s,2s,4s,8s,16s
this.getErrorToken( this.deviceCode )
.retryWhen( error -> retrySupport( error,
e-> e instanceof IOException,
5 ,
count-> (long)Math.pow(2,count-1),TimeUnit.SECONDS ) )
.subscribe( deviceInfo1 -> Logger.i( "----Get Device Info---" ) ,
e -> Logger.e( e, "On Error" ) ,
() -> Logger.i("<<<<<no more>>>>>"));
I'm a bit too late for this one, but just in case this could still be useful for someone, I created a Kotlin extension function for RxJava 2 that will retry with an exponential backoff strategy:
private fun <T> Observable<T>.retryWithExponentialBackoff(): Observable<T> {
val retriesSubject = BehaviorSubject.createDefault(0)
return doOnNext { retriesSubject.onNext(0) }
.retryWhen {
it.withLatestFrom(retriesSubject) { _, retryCount ->
retriesSubject.onNext(retryCount + 1)
retryCount
}.flatMap { retryCount ->
when (retryCount) {
MAX_RETRY_COUNT -> Observable.error(RuntimeException("Max number of retries reached"))
else -> Observable.timer(2.0.pow(retryCount).toLong(), SECONDS)
}
}
}
}

Categories