I was trying to find a flux operator which returns the original publisher after the new one finishes, and .delayUntil() was the key.
how to avoid .flatMap(x-> reactiveAction(x).thenReturn(x))
However, when it comes to flux, unlike flatMap, delayUntil seemed not to work concurrently.
public class BasicTest {
// flatMap
#Test
public void testFlatMap() {
Flux.range(1, 10)
.flatMap(num -> save(num).thenReturn(num))
.doOnNext(System.out::println)
.as(StepVerifier::create)
.thenConsumeWhile(p -> true)
.verifyComplete();
}
// delayUntil
#Test
public void testDelayUntil() {
Flux.range(1, 10)
.delayUntil(this::save)
.doOnNext(System.out::println)
.as(StepVerifier::create)
.thenConsumeWhile(p -> true)
.verifyComplete();
}
private Mono<Integer> save(Integer num) {
return Mono.just(num).log().delayElement(Duration.ofMillis(100));
}
}
I can confirm you that testDelayUntil takes twice more time than testFlatMap.
And I looked through the inner implementation of delayUntil, and it used .concatMap() inside.
I want delayUntil using flatMap, so that it does not wait for the new publisher made from the former signal to be completed, but keep subscribing to new publishers when the formal signals arrive.
Is there anything similar as .flatMap(v -> reactiveOperation(v).thenReturn(v)) instead of .delayUntil()? or should I request for new feature?
Thank you!
Related
The VertX example for when you need to query multiple asynchronous resources and use them all in a single operation is:
Future<HttpServer> httpServerFuture = Future.future();
httpServer.listen(httpServerFuture.completer());
Future<NetServer> netServerFuture = Future.future();
netServer.listen(netServerFuture.completer());
CompositeFuture.all(httpServerFuture, netServerFuture).setHandler(ar -> {
if (ar.succeeded()) {
// All servers started
} else {
// At least one server failed
}
});
We need to query two different databases and then use the results in business logic, but the flow is equivalent.
What's the VertX/RxJava equivalent?
Currently people are doing this by nesting a new .flatMap() call every time they need a new variable. I'm left feeling there must be a better way...
We don't actually need the queries to be concurrent but we need to cache both results and pass them to the business logic at the same time some how.
there are many ways to do this, but i've tried to pick an approach that tacks closely to your sample:
#Override
public void start(Future<Void> startFuture) throws Exception {
final HttpServer httpServer = vertx.createHttpServer();
final Completable initializeHttpServer = httpServer.rxListen().toCompletable();
final NetServer netServer = vertx.createNetServer();
final Completable initializeNetServer = netServer.rxListen().toCompletable();
initializeHttpServer.andThen(initializeNetServer)
.subscribe(
() -> { /* All servers started */ },
error -> { /* At least one server failed */ }
);
}
the rxListen() invocations are converted into Completable instances, which are then run serially upon subscription.
the subscriber's onComplete callback will be invoked when both servers are done binding to their respective ports, or...
the onError callback will be invoked if an exception occurs
(also, fwiw, "nesting" flatMap operations for something as trivial as this shouldn't be necessary. "chaining" such operations, however, would be idiomatic usage).
hope that helps!
--UPDATE--
having read the question more carefully, i now see that you were actually asking about how to handle the results of two discrete asynchronous operations.
an alternative to flatMap'ing your way to combining the results would be to use the zip operator, like so:
#Override
public void start(Future<Void> startFuture) throws Exception {
final Single<String> dbQuery1 = Single.fromCallable(() -> { return "db-query-result-1"; });
final Single<String> dbQuery2 = Single.fromCallable(() -> { return "db-query-result-2"; });
Single.zip(dbQuery1, dbQuery2, (result1, result2) -> {
// handle the results from both db queries
// (with Pair being a standard tuple-like class)
return new Pair(result1, result2);
})
.subscribe(
pair -> {
// handle the results
},
error -> {
// something went wrong
}
);
}
per the docs, zip allows you to specify a series of reactive types (Single, Observable, etc) along with a function to transform all the results at once, with the central idea being that it will not emit anything until all the sources have emitted once (or more, depending on the reactive type).
I have method which in async way calls connector.runSomeService(data) and handles the response in method handleServiceResponse(res, node).
public void runServiceOnAllNodes(Collection<Node> nodes, Object data) {
nodes.parallelStream().forEach(node -> {
CompletableFuture<ResponseEntity> response = CompletableFuture
.supplyAsync(()-> connector.runSomeService(data));
response.exceptionally(ex -> {
log.error("OMG...OMG!!!")
return null;
})
.thenAcceptAsync(res -> handleServiceResponse(res, node));
});
}
private void handleServiceResponse(ResponseEntity res, Node node) {
if (res.isOK) {
node.setOKStatus();
} else {
node.setFailStatus();
}
dbService.saveNode(node);
}
Try to create unit test but when I try to verify if response is properly handled, the result of UT is non deterministic.
#Test
public void testRunServiceOnAllNodes() {
// given
List<Collector> nodes = Arrays.asList(node1, node2, node3);
when(connector.runSomeService(eq(node1), eq(data))).thenReturn(ResponseEntity.ok().body("{message:OK}"));
when(connector.runSomeService(eq(node2), eq(data))).thenReturn(ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(""));
when(connector.runSomeService(eq(node3), eq(data))).thenThrow(new ResourceAccessException(""));
// when
engine.runServiceOnAllNodes(data, collectors);
// then
verify(connector, times(1)).runSomeService(eq(node1), eq(data));
verify(connector, times(1)).runSomeService(eq(node2), eq(data));
verify(connector, times(1)).runSomeService(eq(node3), eq(data));
verifyNoMoreInteractions(connector);
assertEquals(node1.getStatus(), "OK");
assertEquals(node2.getStatus(), "Fail");
}
It can end with a few different results eg.
Wanted but not invoked:
connector.runSomeService(node2);
However, there were other interactions with this mock:
connector.runSomeService(node1);
or
Argument(s) are different! Wanted:
connector.runSomeService(node1);
Actual invocation has different arguments:
connector.deployFileset(node2);
or sometimes it ends with success.
It is clear that the time of execution connector.runSomeService() and the time of the verification can interlace. The order of this two actions is not deterministic.
Using sleep sucks. Tried to gather all responses and calling future.get()
// when
engine.runServiceOnAllNodes(data, collectors);
for (CompletableFuture future : engine.getResponses()) {
future.get();
}
but I'm getting some exception but I still have the feeling that this way also sucks, isn't it?
I would suggest changing the runServiceOnAllNodes method to return a Future so your test, and, as a bonus, normal clients as well, can explicitly wait for the async behavior to finish.
public Future<Void> runServiceOnAllNodes(Collection<Node> nodes, Object data) {
return nodes.parallelStream().map(node -> {
CompletableFuture<ResponseEntity> response = CompletableFuture
.supplyAsync(()-> connector.runSomeService(data));
return response.exceptionally(ex -> {
LOGGER.error("OMG...OMG!!!");
return null;
})
.thenAcceptAsync(res -> handleServiceResponse(res, node));
})
.reduce(CompletableFuture::allOf).orElseGet(() -> CompletableFuture.completedFuture(null));
}
In your test, it is then simply a matter of calling get() on the future prior to making assertions and verifications.
I'm playing around with implementing my own observables or porting them from other languages for fun and profit.
The problem I've run into is that there's very little info on how to properly test observables or async code in general.
Consider the following test code:
// Create a stream of values emitted every 100 milliseconds
// `interval` uses Timer internally
final Stream<Number> stream =
Streams.interval(100).map(number -> number.intValue() * 10);
ArrayList<Number> expected = new ArrayList<>();
expected.add(0);
expected.add(10);
expected.add(20);
IObserver<Number> observer = new IObserver<Number>() {
public void next(Number x) {
assertEquals(x, expected.get(0));
expected.remove(0);
if(expected.size() == 0) {
stream.unsubscribe(this);
}
}
public void error(Exception e) {}
public void complete() {}
};
stream.subscribe(observer);
As soon as the stream is subscribed to, it emits the first value. onNext is called... And then the test exits successfully.
In JavaScript most test frameworks nowadays provide an optional Promise to the test case that you can call asynchronously on success/failure. Is anything similar available for Java?
Since the execution is asyncronious, you have to wait until is finish. You can just wait for some time in an old fashion way
your_code
wait(1000)
check results.
Or if you use Observables you can use TestSubscriber
In this example you can see how having an async operation we wait until the observer consume all items.
#Test
public void testObservableAsync() throws InterruptedException {
Subscription subscription = Observable.from(numbers)
.doOnNext(increaseTotalItemsEmitted())
.subscribeOn(Schedulers.newThread())
.subscribe(number -> System.out.println("Items emitted:" + total));
System.out.println("I finish before the observable finish. Items emitted:" + total);
new TestSubscriber((Observer) subscription)
.awaitTerminalEvent(100, TimeUnit.MILLISECONDS);
}
You can see more Asynchronous examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/scheduler/ObservableAsynchronous.java
I have an observable that emits values. Based on these values I need to subscribe/unsubscribe to/from another Observable.
Is there a handy way of doing so? A convenient way instead creating a field for the subscription and handling it manually?
Example:
Observable A emits Booleans. If it emits true then a subscription should be made to Observable B - if false this subscription should be unsubscribed.
I'm not sure if we're 100% on the same page but I think you're missing one point. Maybe you'll think I'm nitpicking, but I think it will be good to get our terms straight.
Observable starts emitting values when a Subscriber subscribes to it. So unless you're thinking about two separate Subscribers you can't react to an emitted value with a subscription because the Observer won't emit anything.
That said... what (I think) you wanna do could be done this way:
Observable<Boolean> observableA = /* observable A initialization */;
final Observable<SomeObject> observableB = /* observable B initialization */;
observableA
.flatMap(new Func1<Boolean, Observable<SomeObject>>() {
#Override
public Observable<SomeObject> call(Boolean aBoolean) {
if (!aBoolean) {
throw new IllegalStateException("A dummy exception that is here just to cause the subscription to finish with error.");
}
return observableB;
}
})
.subscribe(
new Action1<SomeObject>() {
#Override
public void call(SomeObject someObject) {
// THIS IS A PART OF THE SUBSCRIBER TO OBSERVABLE B.
// THIS METHOD WILL BE CALLED ONLY IF THE OBSERVABLE A RETURNED TRUE
}
},
new Action1<Throwable>() {
#Override
public void call(Throwable throwable) {
// A dummy Action1 so the subscription does not crash on the Exception
}
});
If all of observables has the same type or you can combine whatever you want based on values.
Observable.from(new int[]{1,2,3,4,5})
.filter(i -> i < 5) // filter out something
.flatMap(i -> {
if (i < 2) { // subscribe on some observable, based on item value
return Observable.just(i);
} else {
return Observable.just(3);
}
})
I'm new to RxJava and I'm trying to understand the best/recommended way to perform long running tasks asynchronously (e.g. network requests). I've read through a lot of examples online but would appreciate some feedback.
The following code works (it prints 'one', 'two', then 'User: x' ... etc) but should I really be creating/managing Threads manually?
Thanks in advance!
public void start() throws Exception {
System.out.println("one");
observeUsers()
.flatMap(users -> Observable.from(users))
.subscribe(user -> System.out.println(String.format("User: %s", user.toString()));
System.out.println("two");
}
Observable<List<User>> observeUsers() {
return Observable.<List<User>>create(s -> {
Thread thread = new Thread(() -> getUsers(s));
thread.start();
});
}
void getUsers(final Subscriber s) {
s.onNext(userService.getUsers());
s.onCompleted();
}
// userService.getUsers() fetches users from a web service.
Instead of managing your own thread try using the defer() operator. Meaning replace observeUsers() with Observable.defer(() -> Observable.just(userService.getUsers())). Then you can use the RxJava Schedulers to control what threads are used during subscription and observation. Here's your code modified with the above suggestions.
Observable.defer(() -> Observable.just(userService.getUsers()))
.flatMap(users -> Observable.from(users))
.subscribeOn(Schedulers.newThread())
.observeOn(Schedulers.trampoline())
.subscribe(user -> System.out.println(String.format("User: %s", user.toString()));