We Keep Coding

How to ensure that rxjava methods execute in parallel, and finish?

Although I've been writing Java code for many years, I've barely done any work with RxJava, and I need to understand how to map it to expected results. I have a lot of existing code in services I work with, but I'm not convinced they are using RxJava properly.
Note that we're using an old version of RxJava, 2.1.10. I can't upgrade at this moment.
The following is a common pattern I see in our codebase:
Single<ResultType> result1 = Single.<ResultType>create(source -> {
source.onSuccess(method1(parameters));
}).subscribeOn(Schedulers.io());
Single<ReturnType> result2 = Single.<ResultType>create(source -> {
source.onSuccess(method2(parameters));
}).subscribeOn(Schedulers.io());
if (null != result1 && null != result2) {
The intent of this is that the execution of "method1" and "method2" run in parallel, and that the check for "null != result1 && null != result2" happens after both methods have finished executing. I'm thinking it's possible that neither of these intentions are being fulfilled here, but I need confirmation of that, and also how to achieve those goals properly.

Depending on how your sources are setup, you can use combineLatest() to wait for the result from both sources. A sample proof-of-concept code might look like this:
public static void main(String[] args) throws Exception {
Callable<Integer> c1 = new Callable<Integer>() {
#Override
public Integer call() throws Exception {
System.out.println(System.currentTimeMillis()+"|Starting first");
Thread.sleep(1111);
System.out.println(System.currentTimeMillis()+"|finished first");
return 42;
}};
Single<Integer> singleFirst = Single.fromCallable(c1).subscribeOn(Schedulers.newThread());
Callable<Integer> c2 = new Callable<Integer>() {
#Override
public Integer call() throws Exception {
System.out.println(System.currentTimeMillis()+"|Starting second");
Thread.sleep(5555);
System.out.println(System.currentTimeMillis()+"|finished second");
return 12;
}};
Single<Integer> singleSecond = Single.fromCallable(c2).subscribeOn(Schedulers.newThread());
BiFunction<Integer, Integer, Integer> func = (a,b) -> a+b;
ObservableSource<Integer> source1 = singleFirst.toObservable();
ObservableSource<Integer> source2 = singleSecond.toObservable();
Observable<Integer> resultSource = Observable.combineLatest(source1, source2, func);
System.out.println(System.currentTimeMillis()+"|All setup, wait for completion");
resultSource.blockingSubscribe(r -> {
System.out.println(System.currentTimeMillis()+"|Result is: "+r);
});
}
This might generate the following output:
1589229378890|All setup, wait for completion
1589229378895|Starting second
1589229378895|Starting first
1589229380007|finished first
1589229384451|finished second
1589229384452|Result is: 54
As you see the Single subscriptions run in parallel and their values are "collected" in a combineLatest() call at the end.

Execute code based on result of two parallel completable futures

I have a scenario where in I have to issue two REST calls that return a value each based on the current system state, and based on those two values, have to trigger a final clean up task asynchronously - the flow of control being more like a 'Y' scenario . I have looked through the CompletableFuture interface, and is unable to find a way to accomplish this in a non-blocking fashion
I have tried this, and cant seem to find a way to get it working
// Verify task status
CompletableFuture<AuditResult> checkOneFuture =
CompletableFuture.supplyAsync(() -> dummyService.fetchSystemState(var1, var2),
executorService);
CompletableFuture<AuditResult> checkTwoFuture =
CompletableFuture.supplyAsync(() -> dummyService.fetchSystemState(var1, var3),
executorService);
CompletableFuture<CompletableFuture<Boolean>> cleanUpFuture =
checkOneFuture.thenCombineAsync(checkTwoFuture, (check1, check2) -> {
if (check1.getSuccess() && check2.getSuccess()){
CompletableFuture<Boolean> cleanUpFutuer = CompletableFuture.supplyAsync(() -> cleanUp(check1.id), executorService);
return syncFuture;
} else {
return CompletableFuture.completedFuture(false);
}
}, executorService);
cleanUpFuture.join();
The cleanUpFuture is obviously syntactically not correct, and I am trying to figure ways to get this scenario working. Please help

As Slaw says in his comment, why not just return boolean?
CompletableFuture<Boolean> cleanUpFuture =
checkOneFuture.thenCombineAsync(checkTwoFuture, (check1, check2) -> {
if (check1.getSuccess() && check2.getSuccess()) {
return cleanUp(check1.id); // will be scheduled due to combineAsync
} else {
return false;
}
}, executorService);
Note: for a shorter version, you can do
(check1, check2) -> check1.getSuccess() && check2.getSuccess() && cleanUp(check1.id);

You can acheive this by a ForkJoinPool. Subdivizing your call in subtask by calling fork() and reassemble the whole with a join()
For this you have maybe to implements a RecursiveTask
EDIT : Using CompletableFuture
If your purpose is to run two async processings in parallel and trigger another on completion of the later then allOf() is the best method.
Here is an example :
public CompletableFuture<String> findSomeValue() {
return CompletableFuture.supplyAsync(() -> {
sleep(1);
return "Niraj";
});
}
#Test
public void completableFutureAllof() {
List<CompletableFuture<String>> list = new ArrayList<>();
IntStream.range(0, 5).forEach(num -> {
list.add(findSomeValue());
});
CompletableFuture<Void> allfuture = CompletableFuture.allOf(list.toArray(new CompletableFuture[list.size()]));//Created All of object
CompletableFuture<List<String>> allFutureList = allfuture.thenApply(val -> {
return list.stream().map(f -> f.join()).collect(Collectors.toList());
});
CompletableFuture<String> futureHavingAllValues = allFutureList.thenApply(fn -> {
System.out.println("I am here");
return fn.stream().collect(Collectors.joining());});
String concatenateString = futureHavingAllValues.join();
assertEquals("NirajNirajNirajNirajNiraj", concatenateString);
}
This is example and more explanations are provided in this article

Rxjava retryWhen called instantly

I'm having a very specific problem or misunderstanding with rxjava that someone hopefully can help with.
I'm running rxjava 2.1.5 and have the following code snippet:
public static void main(String[] args) {
final Observable<Object> observable = Observable.create(emitter -> {
// Code ...
});
observable.subscribeOn(Schedulers.io())
.retryWhen(error -> {
System.out.println("retryWhen");
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"));
}
After executing this, the program prints:
retryWhen
Process finished with exit code 0
My question, and what I don't understand is: Why is retryWhen called instantly upon subscribing to an Observable? The observable does nothing.
What I want is retryWhen to be called when onError is called on the emitter. Am I misunderstanding how rx works?
Thanks!
Adding new snippet:
public static void main(String[] args) throws InterruptedException {
final Observable<Object> observable = Observable.create(emitter -> {
emitter.onNext("next");
emitter.onComplete();
});
final CountDownLatch latch = new CountDownLatch(1);
observable.subscribeOn(Schedulers.io())
.doOnError(error -> System.out.println("doOnError: " + error.getMessage()))
.retryWhen(error -> {
System.out.println("retryWhen: " + error.toString());
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> latch.countDown());
latch.await();
}
Emitter onNext and complete is called. DoOnError is never called. Output is:
retryWhen: io.reactivex.subjects.SerializedSubject#35fb3008
subscribeNext
Process finished with exit code 0

retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
Observable.error(new IOException())
.retryWhen(e -> {
System.out.println("Setting up retryWhen");
int[] count = { 0 };
return e
.takeWhile(v -> ++count[0] < 3)
.doOnNext(v -> { System.out.println("Retrying"); });
})
.subscribe(System.out::println, Throwable::printStackTrace);
Since the e -> { } function body is executed for each individual subscriber, you can have a per subscriber state such as retry counter safely.
Using e -> e.retry() has no effect because the input error flow never gets its onError called.

One issue is, that you don't receive any more results because you'r creating a Thread using retryWhen() but your app seems to finish. To see that behaviour you may want to have a while loop to keep your app running.
That actually means that you need to add something like that to the end of your code:
while (true) {}
Another issue is that you dont emit any error in your sample. You need to emit at least one value to call onNext() else it wont repeat because it's waiting for it.
Here's a working example which a value, then it emits an error and repeat. you can use
.retryWhen(errors -> errors)
which is the same as
.retryWhen(errors -> errors.retry())
Working sample:
public static void main(String[] args) {
Observable
.create(e -> {
e.onNext("test");
e.onError(new Throwable("test"));
})
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> System.out.println("onCompleted")
);
while (true) {
}
}
The reason why you need to emit a result is, that Observable needs to emit a value, else it wait until it receives a new one.
This is because onError can only be called onec (in subscribe), but onNext emits 1..* values.
You can check this behaviour by using doOnError() which provides you the error everytime it retrys the Observable.
Observable
.create(e -> e.onError(new Exception("empty")))
.doOnError(e -> System.out.println("error received " + e))
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
nextOrSuccess -> System.out.println("nextOrSuccess " + nextOrSuccess),
error -> System.out.println("subscribeError")
);

retryWhen operator never retries

I'm implementing a DB update approach with retrials.. Following the common pattern for retryWhen() operator as explained here: Using Rx Java retryWhen() ..
..But my retry logic never executes. I'm debugging it and can see the breakpoint hitting at place 3 shown below but it never goes back to retry logic at place 2. After place 3, its always going to place 4 which is the onComplete handler.
(Code is using Java 8 lambdas)
I've applied a workaround by removing the retryWhen() block altogether
and now invoking the updateWithRetrials() recursively from subscribe's > onError() block. That is working but I don't like that approach.
Please can anyone suggest what is incorrect when I use retryWhen() operator ?
private void updateWithRetrials(some input x)
{
AtomicBoolean retryingUpdate = new AtomicBoolean(false);
...
// 1- Start from here
Observable.<JsonDocument> just(x).map(x1 -> {
if (retryingUpdate.get())
{
//2. retry logic
}
//doing sth with x1 here
...
return <some observable>;
})
.retryWhen(attempts -> attempts.flatMap(n -> {
Throwable cause = n.getThrowable();
if (cause instanceof <errors of interest>)
{
// 3 - break-point hits here
// retry update in 1 sec again
retryingUpdate.set(true);
return Observable.timer(1, TimeUnit.SECONDS);
}
// fail in all other cases...
return Observable.error(n.getThrowable());
}))
.subscribe(
doc -> {
//.. update was successful
},
onError -> {
//for unhandled errors in retryWhen() block
},
{
// 4. onComplete block
Sysout("Update() call completed.");
}
); //subscribe ends here
}

Your problem is due to some performance optimisation with Observable.just().
This Operator, after emmiting the item, does not check if the subscribtion is not cancelled and sends onComplete on all cases.
Observable.retryWhen (and retry) resubscribes on Error, but terminates when source sends onComplete.
Thus, even if the retry operator resubscribes, it gets onComplete from previous subscription and stops.
You may see, that code below fails (as yours):
#Test
public void testJustAndRetry() throws Exception {
AtomicBoolean throwException = new AtomicBoolean(true);
int value = Observable.just(1).map(v->{
if( throwException.compareAndSet(true, false) ){
throw new RuntimeException();
}
return v;
}).retry(1).toBlocking().single();
}
But if you "don't forget" to check subscription, it Works!:
#Test
public void testCustomJust() throws Exception {
AtomicBoolean throwException = new AtomicBoolean(true);
int value = Observable.create((Subscriber<? super Integer> s) -> {
s.onNext(1);
if (!s.isUnsubscribed()) {
s.onCompleted();
}
}
).map(v -> {
if (throwException.compareAndSet(true, false)) {
throw new RuntimeException();
}
return v;
}).retry(1).toBlocking().single();
Assert.assertEquals(1, value);
}

I suppose the error occurs inside map because it cannot occur in just. This is not how retryWhen works.
Implement your observable using create and make sure no errors occur in map. If any error will be thrown in the create block the retryWhen will be called and the unit of work retried depending on your retry logic.
Observable.create(subscriber -> {
// code that may throw exceptions
}).map(item -> {
// code that will not throw any exceptions
}).retryWhen(...)
...

RxJava. Initial onNext when subscription starts?

I'm trying to implement a class that emits changes using an Observable.
When a subscription is done to this observable I want to send an starting/initialization event. Then I want to send the usual events.
For example. Lets say I have two different subscribers A and B. A and B starts subscribing at different times. If MyClass.getChanges() emits event no. 1,2,3,4 and 5.
If A starts it subscription between event 1,2 then it should receive the following events:
InitialEvent, 2, 3, 4, 5.
If B starts it subscription between event 4 and 5, then B should receive the following events:
InitialEvent, 5.
How to do this using RxJava?
Thanks!
Edit 1
I think I need to explain that the "InitialEvent" is different each time it's emitted. It's calculated by MyClass each time a new subscriber starts to subscribe from getChanged().
My scenario is that MyClass contains a list. The "initialEvent" contains the list at the moment when the subscription is done. Then each change to this list is emitted from getChanges().

Sorry to post this 2 years later, but I had the same need and found this question unanswered.
What I did is the following:
public Observable<Event> observe() {
return Observable.defer(() ->
subject.startWith(createInitialEvent())
);
}
The idea is the following:
defer() executes the passed-in lambda expression when an observer subscribes to the Observable returned by the method observe(). So basically, it executes subject.startWith(...), which returns an Observable that is the actual source of event for the subscriber.
subject.startWith(...) emits an initial event (specified by startWith(...)) followed by those emitted by the subject.
So, if I come back to the original post:
if an observer starts it subscription between event 1,2 then it should receive the following events: InitialEvent, 2, 3, 4, 5.

What you're looking for is PublishSubject. Subjects are hot Observables, in that they do not wait for Observers to subscribe to them before beginning to emit their items. Here's a bit of info on Subjects.
Here's a quick demo of your use-case
PublishSubject<String> subject = PublishSubject.create();
Observable<String> InitEvent = Observable.just("init");
Observable<String> A = subject.asObservable();
Observable<String> B = subject.asObservable();
subject.onNext("1");
A.startWith(InitEvent)
.subscribe(s -> System.out.println("A: " + s));
subject.onNext("2");
subject.onNext("3");
subject.onNext("4");
B.startWith(InitEvent)
.subscribe(s -> System.out.println("B: " + s));
subject.onNext("5");

Possibly not really elegant way how about just using a flag? It looks like you just want to replace the first emitted event.
e.g. for one subscription the following logic:
boolean firstTimeA = true;
myCustomObservable.subscribe(s -> {
System.out.println(firstTimeA ? "initEvent" : s.toString());
if(firstTimeA) firstTimeA = false;
});
And since you want to have a second subscription just create a firstTimeB and update it your B subscription.

If I understand what you are asking something like this should work for you
int last = 0;
Observable obs;
List<Integer> list = new ArrayList<>();
public SimpleListObservable() {
obs = Observable.create(new Observable.OnSubscribe<Integer>() {
#Override
public void call(Subscriber<? super Integer> subscriber) {
while(last < 30) {
last++;
list.add(last);
subscriber.onNext(last);
}
subscriber.onCompleted();
}
});
}
public Observable<Integer> process() {
return Observable.from(list).concatWith(obs);
}
As the source observable collects values they are added to the List (you can transform the items as you see fit, filter them out, etc) and then when ObserverB subscribes it will get a replay of the items already collected in the List before continuing with the source observable output.
This simple test should demonstrate the outcome
public void testSequenceNext() {
final SimpleListObservable obs = new SimpleListObservable();
final Observer<Integer> ob2 = Mockito.mock(Observer.class);
obs.process().subscribe(new Observer<Integer>() {
#Override
public void onCompleted() {
ob1Complete = true;
}
#Override
public void onError(Throwable e) {
e.printStackTrace();
}
#Override
public void onNext(Integer integer) {
System.out.println("ob1: " + integer);
if (integer == 20) {
obs.process().subscribe(ob2);
}
}
});
ArgumentCaptor<Integer> captor = ArgumentCaptor.forClass(Integer.class);
Mockito.verify(ob2, Mockito.times(30)).onNext(captor.capture());
for (Integer value : captor.getAllValues()) {
System.out.println(value);
}
}

What do you think of this, I've made part of my API of course as I'm on a phone :
public class StreamOfSomething {
new StreamOfSomething() {
// source of events like
events = Observable.range(0, 1_000_000_000)
.doOnNext(set::add) // some operation there
.map(Event::change)
.publish()
.refCount();
}
public Observable<Event> observeChanges() {
return events.startWith(
Observable.just(Event.snapshot(set))); // start stream with generated event
}
}
And the client can do something like :
Observable.timer(2, 4, TimeUnit.SECONDS)
.limit(2)
.flatMap(t -> theSourceToWatch.observeChanges().limit(10))
.subscribe(System.out::println);
Note however if you are in a multithreaded environment you may have to synchronize when you are subscribing to block any modification, otherwise the list may change before it get's emitted. Or rework this class completely around observables, I don't know yet how to achieve this though.