We Keep Coding

How to get all elements immediately in other threads in Subscriber in RxJava?

If i execute such a code:
StopWatch stopWatch = new StopWatch();
stopWatch.start();
Flowable.fromIterable(Lists.newArrayList(1, 2, 3, 4, 5, 6))
.subscribeOn(Schedulers.io())
.subscribe(new Consumer<Integer>() {
#Override
public void accept(Integer integer) throws Throwable {
System.out.println(stopWatch + " value:" + integer);
Thread.sleep(1000);
}
});
Thread.sleep(100000);
In output, i get each element just after sleep time, like this:
00:00:00.027 value:1
00:00:01.030 value:2
00:00:02.030 value:3
00:00:03.031 value:4
00:00:04.031 value:5
00:00:05.031 value:6
but as i understand, if i use Schedulers.io(), than i must get all the values parallel, and i expected, that i will get all values immediately, than i will wait 1000 mills just once, and thats all
Like this:
00:00:00.027 value:1
00:00:00.030 value:2
00:00:00.030 value:3
00:00:00.031 value:4
00:00:00.031 value:5
00:00:00.031 value:6
How can i get them all in other threads, NOT one by one?
I dont wanna them to wait each other
I try Schedulers.computation() and other, but they still arrive one by one
How to get them all immediately?
P.S.
There is some text for better search in google. I get it from browser history.
rxjava how to many subscriber, rxjava how to post all elements synchronously, only one thread active rxjava, flowable from don't work in many threads

if i use Schedulers.io(), than i must get all the values parallel
No. RxJava flows are sequential by default, which means items are delivered one after the other. If your consumer blocks or sleeps, subsequent items are delayed every time.
How can i get them all in other threads, NOT one by one?
Use parallel:
StopWatch stopWatch = new StopWatch();
stopWatch.start();
Flowable.fromIterable(Lists.newArrayList(1, 2, 3, 4, 5, 6))
.parallel()
.runOn(Schedulers.io())
.doOnNext(new Consumer<Integer>() {
#Override
public void accept(Integer integer) throws Throwable {
System.out.println(stopWatch + " value:" + integer);
Thread.sleep(1000);
}
})
.sequential()
.subscribe();
Thread.sleep(100000);
Recommended reading: https://github.com/ReactiveX/RxJava#parallel-processing

Testing internal Flux.interval

I want to provide some data using Reactor's Flux. Since it may take a lot of time to provide this data, I decided to introduce a ping mechanism (e.g. to keep tcp connection alive and not get timeouts). Here is my simplified solution:
public class Example {
private final DataProvider dataProvider;
public Example(DataProvider dataProvider) {
this.dataProvider = dataProvider;
}
public Flux<String> getData() {
AtomicBoolean inProgress = new AtomicBoolean(true);
Flux<String> dataFlux = dataProvider.provide()
.doFinally(ignoreIt -> inProgress.set(false));
return dataFlux.mergeWith(ping(inProgress::get));
}
private Publisher<String> ping(Supplier<Boolean> inProgress) {
return Flux.interval(Duration.ofSeconds(1), Duration.ofSeconds(1))
.map((tick) -> "ping " + tick)
.takeWhile(ignoreIt -> inProgress.get());
}
interface DataProvider {
Flux<String> provide();
}
public static void main(String[] args) {
Callable<String> dataProviderLogic = () -> {
Thread.sleep(3500);
return "REAL DATA - SHOULD TERMINATE PING";
};
// wrapping synchronous call
DataProvider dataProvider = () -> Mono.fromCallable(dataProviderLogic)
.subscribeOn(Schedulers.boundedElastic())
.flux();
new Example(dataProvider).getData()
.doOnNext(data -> System.out.println("GOT: " + data))
.blockLast();
}
}
Above code prints on console:
GOT: ping 0
GOT: ping 1
GOT: ping 2
GOT: REAL DATA - SHOULD TERMINATE PING
So it works as expected.
The question is: how can I test this ping mechanism in a Junit5 test, so it won't take a lot of time (e.g. several seconds)?
In an ideal world I would like to write a test which imitates a delay for the data provision, check if expected number of pings was generated and verify if complete signal was emitted (to make sure that ping flux terminates as expected). Of course I would like to have a unit test, which can be run in ms.
I tried this, but with no luck:
#Test
void test() {
TestPublisher<String> publisher = TestPublisher.create();
Flux<String> data = new Example(publisher::flux).getData();
StepVerifier.withVirtualTime(() -> data)
.thenAwait(Duration.ofMillis(3500))
.then(() -> publisher.emit("REAL DATA - SHOULD TERMINATE PING"))
.then(publisher::complete)
.expectNextCount(4)
.verifyComplete();
}
Above test ends up with this error:
java.lang.AssertionError: expectation "expectNextCount(4)" failed (expected: count = 4; actual: counted = 1; signal: onComplete())
Is it possible at all to use virtual time for internally created Flux.interval?
Any ideas for an alternative ping solution will be appreciated.

Despite of the fact that above ping mechanism is not the best one (I suggest to use Sink instead of AtomicBoolean and use takeUntilOther instead of takeWhile), in my case the problem was probably related to the situation where not all flux instructions were wrapped with withVirtualTime. This code works as expected in the above case:
#Test
void test() {
StepVerifier.withVirtualTime(() -> {
Flux<String> data = Flux.just("REAL DATA - SHOULD TERMINATE PING").delayElements(Duration.ofMillis(3200));
return new Example(() -> data).getData();
})
.thenAwait(Duration.ofMillis(3500))
.expectNextCount(4)
.thenAwait(Duration.ofMillis(1000))
.verifyComplete();
}

Is Mono.toFuture() blocking?

From the Official Documentation of Mono#block() it is said that:
Subscribe to this Mono and block indefinitely until a next signal is received. Returns that value, or null if the Mono completes empty. In case the Mono errors, the original exception is thrown (wrapped in a RuntimeException if it was a checked exception).
So it is sure that block() method is blocking and it will not execute the next line untill block() resolved.
But my confusion is while I was using toFuture() expecting it will be non-blocking but it is behaving exactly like block method. And in the Documentation of Mono#toFuture() it is stated:
Transform this Mono into a CompletableFuture completing on onNext or onComplete and failing on onError.
Not much clear. Nowhere in this doc said Mono#toFuture() is blocking.
Please confirm me if toFuture() method blocking or non-blocking?
Also If it is non-blocking then, which thread will responsible to execute the code inside CompletableFuture?
Update: added code snippet
using Mono.block() method:
long time = System.currentTimeMillis();
String block = Mono.fromCallable(() -> {
logger.debug("inside in fromCallable() block()");
//Upstream httpcall with apache httpClient().
// which takes atleast 1sec to complete.
return "Http response as string";
}).block();
logger.info("total time needed {}", (System.currentTimeMillis()-time));
return CompletableFuture.completedFuture(block);
Using Mono.ToFuture() method:
long time = System.currentTimeMillis();
CompletableFuture<String> toFuture = Mono.fromCallable(() -> {
logger.debug("inside in fromCallable() block()");
//Upstream httpcall with apache httpClient().
// which takes atleast 1sec to complete.
return "Http response as string";
}).toFuture();
logger.info("total time needed {}", (System.currentTimeMillis()-time));
return toFuture;
these two code snippets behaves exactly same.

-- EDIT: I was wrong. mono.toFuture() doesn't block --
mono.toFuture() isn't blocking. Look at this test:
#Test
void testMonoToFuture() throws ExecutionException, InterruptedException {
System.out.println(LocalTime.now() + ": start");
Mono<String> mono = Mono.just("hello StackOverflow")
.delayElement(Duration.ofMillis(500))
.doOnNext((s) -> System.out.println(LocalTime.now() + ": mono completed"));
Future<String> future = mono.toFuture();
System.out.println(LocalTime.now() + ": future created");
String result = future.get();
System.out.println(LocalTime.now() + ": future completed");
assertThat(result).isEqualTo("hello StackOverflow");
}
This is the result:
20:18:49.557: start
20:18:49.575: future created
20:18:50.088: mono completed
20:18:50.088: future completed
The future is created almost immediately. Half a second later, the mono completes and immediately after that, the future completes. This is exactly what I would expect to happen.
So why does the mono seem blocking in the example provided in the question? It's because of the way mono.fromCallable() works. When and where does that callable actually run? mono.fromCallable() doesn't spawn an extra thread to do the work. From my tests it seems that the callable runs when you first call subscribe() or block() or something similar on the mono, and it will run in the thread that does that.
Here is a test that shows that if you create a mono with fromCallable(), subscribe will cause the callable to be executed in the main thread and even the subscribe() method will seem blocking.
#Test
void testMonoToFuture() throws ExecutionException, InterruptedException {
System.out.println(LocalTime.now() + ": start");
System.out.println("main thread: " + Thread.currentThread().getName());
Mono<String> mono = Mono.fromCallable(() -> {
System.out.println("callabel running in thread: " + Thread.currentThread().getName());
Thread.sleep(1000);
return "Hello StackOverflow";
})
.doOnNext((s) -> System.out.println(LocalTime.now() + ": mono completed"));
System.out.println("before subscribe");
mono.subscribe(System.out::println);
System.out.println(LocalTime.now() + ": after subscribe");
}
result:
20:53:37.071: start
main thread: main
before subscribe
callabel running in thread: main
20:53:38.099: mono completed
Hello StackOverflow
20:53:38.100: after subscribe
Conclusion: mono.toFuture() isn't any more blocking than mono.subscribe(). If you want to execute some piece of code asynchronously, you shouldn't be using Mono.fromCallable(). You could consider using Executors.newSingleThreadExecutor().submit(someCallable)
For reference, here is my original (wrong) answer where I belittle the mono.block() method that was assuredly written by people who know a lot more about Java and coding than I do. A personal lesson in humility, I guess.
EVERYTHING BELOW THIS IS NONSENSE
I wanted to verify exactly how this works so I wrote some tests. Unfortunately, it turns out that mono.toFuture() is indeed blocking and the result is evaluated synchronously. I honestly don't know why you would ever use this feature. The whole point of a Future is to hold the result of an asynchronous evaluation.
Here is my test:
#Test
void testMonoToFuture() throws ExecutionException, InterruptedException {
Mono<Integer> mono = Mono.fromCallable(() -> {
System.out.println("start mono");
Thread.sleep(1000);
System.out.println("mono completed");
return 0;
});
Future<Integer> future = mono.toFuture();
System.out.println("future created");
future.get();
System.out.println("future completed");
}
Result:
start mono
mono completed
future created
future completed
Here is an implementation of monoToFuture() that works the way that I would expect it to:
#Test
void testMonoToFuture() throws ExecutionException, InterruptedException {
Mono<Integer> mono = Mono.fromCallable(() -> {
System.out.println("start mono");
Thread.sleep(1000);
System.out.println("mono completed");
return 0;
});
Future<Integer> future = monoToFuture(mono, Executors.newSingleThreadExecutor());
System.out.println("future created");
future.get();
System.out.println("future completed");
}
private <T> Future<T> monoToFuture(Mono<T> mono, ExecutorService executorService){
return executorService.submit((Callable<T>) mono::block);
}
Result:
future created
start mono
mono completed
future completed

TL;DR
Mono.toFuture() is not blocking but Mono.toFuture().get() is blocking. block() is technically the same as toFuture().get() and both are blocking.
Mono.toFuture() just transforms Mono into a CompletableFuture by subscribing to it and resolving immediately. But it doesn't mean that you can access result (in your case String) of the corresponding Mono after this. CompletableFuture is still async and you can use methods like thenApply(), thenCompose(), thenCombine(), ... to continue async processing.
CompletableFuture<Double> result = getUserDetail(userId)
.toFuture()
.thenCompose(user -> getCreditRating(user));
where getUserDetail is defined as
Mono<User> getUserDetail(String userId);
Mono.toFuture is useful when you need to combine different async APIs. For example, AWS Java v2 API is async but based on CompletableFuture but we can combine APIs using Mono.toFuture or Mono.fromFuture.

Rxjava retryWhen called instantly

I'm having a very specific problem or misunderstanding with rxjava that someone hopefully can help with.
I'm running rxjava 2.1.5 and have the following code snippet:
public static void main(String[] args) {
final Observable<Object> observable = Observable.create(emitter -> {
// Code ...
});
observable.subscribeOn(Schedulers.io())
.retryWhen(error -> {
System.out.println("retryWhen");
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"));
}
After executing this, the program prints:
retryWhen
Process finished with exit code 0
My question, and what I don't understand is: Why is retryWhen called instantly upon subscribing to an Observable? The observable does nothing.
What I want is retryWhen to be called when onError is called on the emitter. Am I misunderstanding how rx works?
Thanks!
Adding new snippet:
public static void main(String[] args) throws InterruptedException {
final Observable<Object> observable = Observable.create(emitter -> {
emitter.onNext("next");
emitter.onComplete();
});
final CountDownLatch latch = new CountDownLatch(1);
observable.subscribeOn(Schedulers.io())
.doOnError(error -> System.out.println("doOnError: " + error.getMessage()))
.retryWhen(error -> {
System.out.println("retryWhen: " + error.toString());
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> latch.countDown());
latch.await();
}
Emitter onNext and complete is called. DoOnError is never called. Output is:
retryWhen: io.reactivex.subjects.SerializedSubject#35fb3008
subscribeNext
Process finished with exit code 0

retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
Observable.error(new IOException())
.retryWhen(e -> {
System.out.println("Setting up retryWhen");
int[] count = { 0 };
return e
.takeWhile(v -> ++count[0] < 3)
.doOnNext(v -> { System.out.println("Retrying"); });
})
.subscribe(System.out::println, Throwable::printStackTrace);
Since the e -> { } function body is executed for each individual subscriber, you can have a per subscriber state such as retry counter safely.
Using e -> e.retry() has no effect because the input error flow never gets its onError called.

One issue is, that you don't receive any more results because you'r creating a Thread using retryWhen() but your app seems to finish. To see that behaviour you may want to have a while loop to keep your app running.
That actually means that you need to add something like that to the end of your code:
while (true) {}
Another issue is that you dont emit any error in your sample. You need to emit at least one value to call onNext() else it wont repeat because it's waiting for it.
Here's a working example which a value, then it emits an error and repeat. you can use
.retryWhen(errors -> errors)
which is the same as
.retryWhen(errors -> errors.retry())
Working sample:
public static void main(String[] args) {
Observable
.create(e -> {
e.onNext("test");
e.onError(new Throwable("test"));
})
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> System.out.println("onCompleted")
);
while (true) {
}
}
The reason why you need to emit a result is, that Observable needs to emit a value, else it wait until it receives a new one.
This is because onError can only be called onec (in subscribe), but onNext emits 1..* values.
You can check this behaviour by using doOnError() which provides you the error everytime it retrys the Observable.
Observable
.create(e -> e.onError(new Exception("empty")))
.doOnError(e -> System.out.println("error received " + e))
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
nextOrSuccess -> System.out.println("nextOrSuccess " + nextOrSuccess),
error -> System.out.println("subscribeError")
);

RxJava: Why retryWhen/repeatWhen doesn't work?

I've been stuck with this for a day. Inspired in Dan Lew great post, I tried to make a simple testcase for repeatWhen() and retryWhen():
public class ObsTest {
private static final Logger LOG = LoggerFactory.getLogger(ObsTest.class);
#Test
public void test1() throws InterruptedException {
Observable<Integer> obs = rx.Observable.<Integer> create(observer -> {
LOG.info("onSubscribe");
Integer data = RandomUtils.nextInt(0, 1000);
if (data % 2 != 0) {
observer.onError(new RuntimeException("Odd number " + data));
} else {
observer.onNext(data);
}
observer.onCompleted();
}, BackpressureMode.BUFFER);
obs.repeatWhen(completed -> completed.delay(1, TimeUnit.MILLISECONDS))
.retryWhen(error -> error.delay(1, TimeUnit.MILLISECONDS))
.subscribe(i -> LOG.info("value={}", i), e -> LOG.info("Exception = {}", e.getMessage()));
}
My idea is this should run forever, emitting even numbers as "correct" results, and odd numbers as "errors".
Instead, this runs for one or two loops and then stops. And that is when the delay is 1 millisecond, for longer periods of time (ie. 1 second), it runs a single time, emitting just a single odd or even number.
I'm sure I'm doing something wrong, but I can't find what it is.

When you call delay which uses Schedulers.computation() by default you are introducing asynchrony. Once activity starts occurring in a background thread your test will finish and presumably your process is exited. You need to use a blockingSubscribe or put a longish Thread.sleep at the end.

As Dave Moten mentioned, delay uses Schedulers.computation() by default, but you can pass scheduler of your choice instead - for tests purposes you may use TestScheduler and "take control over time". Code below shows how can it be used - as you can see this subscription won't terminal for another 30 days, what is basically forever ;)
public class ObsTest {
#Test
public void test1() {
Observable<Integer> obs = rx.Observable.create(observer -> {
Integer data = RandomUtils.nextInt(0, 1000);
if (data % 2 != 0) {
observer.onError(new RuntimeException("Odd number " + data));
} else {
observer.onNext(data);
}
observer.onCompleted();
}, Emitter.BackpressureMode.BUFFER);
TestScheduler scheduler = Schedulers.<Integer>test();
AssertableSubscriber subscriber = obs.repeatWhen(completed -> completed.delay(1, TimeUnit.MILLISECONDS, scheduler))
.retryWhen(error -> error.delay(1, TimeUnit.MILLISECONDS, scheduler))
.subscribeOn(scheduler)
.test();
subscriber.assertNoValues();
scheduler.advanceTimeBy(30, TimeUnit.SECONDS);
subscriber.assertNoTerminalEvent();
scheduler.advanceTimeBy(30, TimeUnit.DAYS);
subscriber.assertNoTerminalEvent();
}
}