Make simultaneous WebClient calls and get only the first completed - java

I'm quite newbie in webflux and I want to do the following thing:
I want to make parallel http requests to the same url with different parameter values and to stop when I get the first non null (and non exceptional) result.
I'm following the example from here https://www.baeldung.com/spring-webclient-simultaneous-calls
but I have no idea how to stop when I got the result. Can anybody help me?
Currently I have something like this:
RetrySpec retrySpec = Retry.max(3);
return webClient.get().uri("/getMyObject/{id}", id)
.retrieve()
.bodyToMono(MyObject.class)
.retryWhen(retrySpec);
}
public Flux<> getMyObjs(List<String> ids) {
return Flux.fromIterable(ids)
.parallel(Runtime.getRuntime().availableProcessors())
.runOn()
.flatMap(this::getMyObject)
.;//// Stop when I get first non exceptional value
}

Try the next() operator in Flux.
public Mono<MyObject> getMyObjs(List<String> ids) {
return Flux.fromIterable(ids)
.parallel(Runtime.getRuntime().availableProcessors())
.runOn()
.flatMap(this::getMyObject)
.next();// Emit only the first item emitted by this Flux, into a new Mono. If called on an empty Flux, emits an empty Mono.
}
Reference: https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#next--
However check the firstWithSignal & firstWithValue operator as well.
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#firstWithSignal-java.lang.Iterable-
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#firstWithValue-java.lang.Iterable-
When I get a problem like this, normally I check the documentation to find a proper operator from Flux API.

Related

Using Optional's ifPresentOrElse method but return promise instead of void

I'm trying to get my around a current issue I'm facing.
I have a function that returns an Optional type (an object with a few properties)
One of the properties is an url that might be present or not. I extract that url in order to make an HTTP request
injectedClass.method(tenant.clientKey()).flatMap(optionalProperty ->
optionalProperty.ifPresentOrElse(fi -> {
Blocking.get(() -> httpClientProvider.withHttpClient((HttpClient httpClient) ->
httpClient.request(URI.create(optionalProperty.webTriggerUrl()), (RequestSpec spec) -> {
LogstashMarker markers = append("webTriggerUrl", fi.webTriggerUrl()).and(append("method", "Post").and(append("behaviour", objectMapper.writeValueAsString(baseDTO))));
logger.debug(markers, "Executed a Post request to something webTriggerUrl");
spec.method(HttpMethod.POST);
spec.getBody().type(HttpHeaderValues.APPLICATION_JSON).text(objectMapper.writeValueAsString(baseDTO), CharsetUtil.UTF_8);
final MutableHeaders headers = spec.getHeaders();
headers
.set(HttpHeaderNames.USER_AGENT, userAgent);
headers.set(CorrelationId.HEADER_NAME, correlationId.id());
})
)).then(resp -> logger.info("ok"));
}, () -> logger.error("something"))
Blocking.get brings back a Promise and I get an error in my code basically saying that the expected return type of ifPresentOrElse should be void and not Promise
Is there a functional and better way to achieve this?
Yes there are ways, but you also have to decide what to do if the Optional is empty. Currently you want to return a Promise if the optional is present, and return nothing ("void") if it is empty. This doesn't work, the types for both branches need to be the same.
You can just use optionalProperty.map() to map your original Optional to a Optional<Promise>, and then use ifPresentOrElse, to do something with either the Promise or with the empty Optional, e.g. logging as you seem to be doing in your case.
But you also have a higher-level flatMap which I'm unclear from which type it is. Does this flatmap a Promise? Then you must return a Promise also from the other branch of the optional, and you could use optionalProperty.map(...).orElse( <create empty Promise here> ).
Also check out orElseGet() instead of orElse(), if you want to create the empty branch lazily (via Supplier).
ifPresentOrElse returns void. What you probably want is a combination of map and orElseGet:
optionalProperty.map(/* code to return a Promise */)
.orElseGet(() -> /* code to return a Promise that is immediately resolved */);
Inside the supplier to orElseGet() you can put your logger.error statement.

Applying a Single to an ObservableSource and not over-reading

I'm pretty new to RX in general, and rxjava in particular, pardon mistakes.
This operation depends on a two async operations.
The first uses a filter function to attempt to get a single entity from a list returned by an async Observable.
The second is an async operation that communicates with a device and produces an Observable of status updates.
I want to take the Single that is created from the filter function, apply that to pairReader(...), and subscribe to its Observable for updates. I can get this to work as shown, but only if I include the take(1) commented, otherwise I get an exception because the chain tries to pull another value from the Single.
Observable<DeviceCredential> getCredentials() {
return deviceCredentialService()
.getCredentials()
.flatMapIterable(event -> event.getData());
}
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
// A device is logically constrained to only have a single cred per org
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.take(1) // Without this I get an exception
.singleOrError();
}
Function<Organization, Observable<Reader.EnrollmentState>> pairReader(String name) {
return org -> readerService().pair(name, org);
}
getOrgFromCreds(orgid)
.flatMapObservable(pairReader(readerid))
.subscribe(state -> {
switch(state) {
case BEGUN:
LOG.d(TAG, "Pairing begun");
break;
case PAIRED:
LOG.d(TAG, "Pairing success");
callback.success();
break;
case NOTIFIED_SERVER:
LOG.d(TAG, "Pairing server notified");
break;
}},
error -> {
Crashlytics.logException(error);
callback.error(error.getLocalizedMessage());
});
If the source stream emits more than one item, singleOrError() is supposed to emit an error. Doc
For your case, use either first() or firstOrError() instead.
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.firstOrError();
}
If I got you right, you need to make some action using previously retrieved async data. So, you could use .zip() operator.
Here is an example:
Observable.zip(
getOrgFromCreds().toObservable(),
getCredentials(),
(first, second) -> /*create output object here*/
)
.subscribe(
(n) -> /*do onNext*/,
(e) -> /*do onError*/
);
Note, that .zip() operator will wait for both emission from two streams, and then it will create outer emission using the function you provided in "create output object here".
If you don't want to wait for both items - you can use .combineLatest().
The problem here turned out to be that the API was designed in an odd way (and unfortunately has extremely poor documentation). I couldn't figure out why I was getting duplicates, and thought I was using flatMapIterable incorrectly.
What the deviceCredentialService.getCredentials() call actually creates is an observable that emits DataEvent objects which are simple wrappers over a list of results, and with a flag of where the results came from.
The API designer wanted to allow the user to use locally cached data to fill the UI immediately while a longer request to a REST API executes. The DataEvent.from property is an enum that flags the source, either from the local device cache or from the remote API call.
The way I solved this was to simply ignore the results coming from local cache and only emit results from the API:
Observable<DeviceCredential> getCredentials() {
return deviceCredentialService()
.getCredentials()
// Only get creds from network
.filter(e -> e.getFrom() == SyncedDataSourceObservableFactory.From.SOURCE)
.flatMapIterable(e -> e.getData());
}
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
// A device is logically constrained to only have a single cred per org
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.singleOrError();
}
The plan then is to use memoization to cache entities in a way that gives the implementing app access to cache invalidation. Since the provided interface doesn't allow squelching the API call, there is no way to work only with cache if the app feels its is fresh.

Flux endpoint from infinite java stream

I have an issue while processing a flux that is built from a Stream.generate construct.
The Java stream is fetching some data from a remote source, hence I implemented a custom supplier that has the data fetching logic embedded, and then used it to populate the Stream.
Stream.generate(new SearchSupplier(...))
My idea is to detect an empty list and use the Java9 feature of takeWhile ->
Stream.generate(new SearchSupplier(this, queryBody))
.takeWhile(either -> either.isRight() && either.get().nonEmpty())
(using Vavr's Either construct)
The repositoroy layer flux will then do:
return Flux.fromStream (
this.searchStream(...) //this is where the stream gets generated
)
.map(Either::get)
.flatMap(Flux::fromIterable);
The "service" layer is composed of some transformation steps on the flux, but the method signature is something like Flux<JsonObject> search(...).
Finally, the controller layer has a GetMapping:
#GetMapping(produces = "application/stream+json")
public Flux search(...) {
return searchService.search(...) //this is the Flux<JsonObject> parth
.subscriberContext(...) //stuff I need available during processing
.doOnComplete(() -> log.debug("DONE"));
}
My problem is that the Flux seems to never terminate.
Doing a call from Postman for example just shot the 'Loading...' part in the response section. When I terminate the process from my IDE the results are then flushed to postman and I see what I'm expecting. Also the doOnComplete lambda never gets called
What I noticed is that if I change the source of a Flux:
Flux.fromArray(...) //harcoded array of lists of jsons
the doOnComplete lambda is called and also the http connection closes, and results are displayed in postman.
Any idea of what might be the issue?
Thanks.
You could create the Flux directly using code that looks like this. Note that I'm adding some assumed methods which you would need to implement based on your how your SearchSupplier works:
Flux<SearchResultType> flux = Flux.generate(
() -> new SearchSupplier(this, queryBody),
(supplier, sink) -> {
SearchResultType current = supplier.next();
if (isNotLast(current)) {
sink.next(current);
} else {
sink.complete();
}
return supplier;
},
supplier -> anyCleanupOperations(supplier)
);

Code running on main thread even with subscribeOn specified

I'm in the process of migrating an AsyncTaskLoader to RxJava, trying to understand all the details about the RxJava approach to concurrency. Simple things were running ok, however I'm struggling with the following code:
This is the top level method that gets executed:
mCompositeDisposable.add(mDataRepository
.getStuff()
.subscribeOn(mSchedulerProvider.io())
.subscribeWith(...)
mDataRepository.getStuff() looks like this:
public Observable<StuffResult> getStuff() {
return mDataManager
.listStuff()
.flatMap(stuff -> Observable.just(new StuffResult(stuff)))
.onErrorReturn(throwable -> new StuffResult(null));
And the final layer:
public Observable<Stuff> listStuff() {
Log.d(TAG, ".listStuff() - "+Thread.currentThread().getName());
String sql = <...>;
return mBriteDatabase.createQuery(Stuff.TABLE_NAME, sql).mapToList(mStuffMapper);
}
So with the code above, the log will print out .listStuff() - main, which is not exactly what I'm looking for. And I'm not really sure why. I was under impression that by setting subscribeOn, every event pulled from the chain will be processed on the thread specified in the subscribeOn method.
What I think is happening, is that the source-aka-final-layer code, before reaching mBriteDatabase, is not from the RxJava world and therefore is not an event until createQuery is called. So I probably need some sort of a wrapper? I've tried applying .fromCallable, however that's a wrapper for non Rx code, and my database layer returns an observable...
Your Log.d call happens
immediately when listStuff gets called
which is immediately after getStuff gets called
which is the first thing happening in the top level code fragment you show us.
If you need to do it when the subscription happens, you need to be explicit:
public Observable<Stuff> listStuff() {
String sql = <...>;
return mBriteDatabase.createQuery(Stuff.TABLE_NAME, sql)
.mapToList(mStuffMapper)
.doOnsubscribe(() -> Log.d(TAG, ".listStuff() - "+Thread.currentThread().getName()));
}

Vertx Future does not wait

Since I´m using Vertx 3.1 in my stack, I was thinking to use the Future feature that the tools brings, but after read the API seems pretty limited to me. I cannot even find the way to make the the future wait for an Observable.
Here my code
public Observable<CommitToOrderCommand> validateProductRestrictions(CommitToOrderCommand cmd) {
Future<Observable<CommitToOrderCommand>> future = Future.future();
orderRepository.getOrder(cmd, cmd.orderId)
.flatMap(order -> validateOrderProducts(cmd, order))
.subscribe(map -> checkMapValues(map, future, cmd));
Observable<CommitToOrderCommand> result = future.result();
if(errorFound){
throw MAX_QUANTITY_PRODUCT_EXCEED.create("Fail"/*restrictions.getBulkBuyLimit().getDescription())*/);
}
return result;
}
private void checkMapValues(Multimap<String, BigDecimal> totalUnitByRestrictions, Future<Observable<CommitToOrderCommand>> future,
CommitToOrderCommand cmd) {
for (String restrictionName : totalUnitByRestrictions.keySet()) {
Restrictions restrictions = Restrictions.valueOf(restrictionName);
if (totalUnitByRestrictions.get(restrictionName)
.stream()
.reduce(BigDecimal.ZERO, BigDecimal::add)
.compareTo(restrictions.getBulkBuyLimit()
.getMaxQuantity()) == 1) {
errorFound = true;
}
}
future.complete(Observable.just(cmd));
}
In the onComplete of my first Observable I´m checking the results, and after finish is when I finish the future to unblock the operation.
But I´m looking that future.result is not block until future.complete is invoke as I was expecting. Instead is just returning null.
Any idea what´s wrong here?
Regards.
The vertx future doesn't block but rather work with a handler that is invoked when a result has been injected (see setHandler and isComplete).
If the outer layer of code requires an Observable, you don't need to wrap it in a Future, just return Observable<T>. Future<Observable<T>> doesn't make much sense, you're mixing two ways of doing async results.
Note that there are ways to collapse an Observable into a Future, but the difficulty is that an Observable may emit several items whereas a Future can hold only a single item. You already took care of that by collecting your results into a single emission of map.
Since this Observable only ever emits one item, if you want a Future out of it you should subscribe to it and call future.complete(yourMap) in the onNext method. Also define a onError handler that will call future.fail.

Categories