We Keep Coding

Multiple asynchronous calls using result of first call using spring webflux

I need to make an asynchronous call and use some values present in it to make multiple calls to same service. Combine the response on these calls with the first one and return.
For example When I make the first call I get below JSON, which has a list of ids. Now I have to make multiple calls to a service with these ids and make a list of their response and send it to downstream by appending it in same JSON.
{“id”: 145,
“object”:[{“id”:111}]
}
I have tried using zipWhen and
Flux.fromIterable(userIds).parallel().runOn(Schedulers.elastic()).flatMap()
But resulting list always comes as empty or null. How can we achieve this? Am I missing something here?
Edit1:
Resolved it by using Flux.fromIterable. Read more about it and finally understood the use and resolved it. The below method takes up one item from list and will call the inner method which will call multiple APIs:
return Flux.fromIterable(somelist).flatMap(listItem -> {
return someMethodToCallAnotherAPIUsingZipWith(listItem);
}).collectList();
Inner Method:
It calls 1st API, passed its result to zipWith and using this result we can call another API or we can simply use it with its response.
private Mono<Object> someMethodToCallAnotherAPIUsingZipWith(String listItem) {
return authService.getAccessToken().flatMap(accessToken ->
webClient.get().uri(builder -> builder.path("/path").build(listItem))
.header(HttpHeaders.AUTHORIZATION, accessToken)
.retrieve()
.toEntity(Entity.class).log()
.flatMap(entity -> {
//manipulate your response or create new object using it
return Mono.just(entity);
}).zipWhen(consent -> webClient.get().uri(builder -> builder.path("/otherpath").build(listItem))
.header(HttpHeaders.AUTHORIZATION, accessToken)
.retrieve().bodyToMono(Entity.class).log()
.flatMap(entity -> {
//example
listItem = entity.getString();
return Mono.just(listItem);
}), (string1, string2) -> string1 + string2));
}

private Mono<Object> someMethodToCallAnotherAPIUsingZipWith(String listItem) {
return authService.getAccessToken().flatMap(accessToken ->
webClient.get().uri(builder -> builder.path("/path").build(listItem))
.header(HttpHeaders.AUTHORIZATION, accessToken)
.retrieve()
.toEntity(Entity.class).log()
.flatMap(entity -> {
//manipulate your response or create new object using it
return Mono.just(entity);
}).zipWhen(consent -> webClient.get().uri(builder -> builder.path("/otherpath").build(listItem))
.header(HttpHeaders.AUTHORIZATION, accessToken)
.retrieve().bodyToMono(Entity.class).log()
.flatMap(entity -> {
//example
listItem = entity.getString();
return Mono.just(listItem);
}), (string1, string2) -> string1 + string2));
}

Spring-WebFlux Flux fails with Context

I want to use Context in my Flux pipe to bypass filtering.
Here's what I have:
public Flux<Bar> realtime(Flux<OHLCIntf> ohlcIntfFlux) {
return Flux.zip(
ohlcIntfFlux,
ohlcIntfFlux.skip(1),
Mono.subscriberContext().map(c -> c.getOrDefault("isRealtime", false))
)
.filter(l ->
l.getT3() ||
(!l.getT2().getEndTimeStr().equals(l.getT1().getEndTimeStr())))
.map(Tuple2::getT1)
.log()
.map(this::
}
which is input to this this:
public void setRealtime(Flux<Bar> input) {
Flux.zip(input, Mono.subscriberContext())
.doOnComplete(() -> {
...
})
.doOnNext(t -> {
...
})
.subscribe()
}
I can tell my code in ... is not failing, I can even access the Context map, but when the first iteration completes, I get:
onContextUpdate(Context1{reactor.onNextError.localStrategy=reactor.core.publisher.OnNextFailureStrategy$ResumeStrategy#35d5ac51})
and subscriber disconnects.
So my question is whether I am using it right and what can be an issue here?
EDIT:
I have tried to repeat() the Mono.subscriberContext() when I'm using value out of it:
return Flux.zip(
ohlcIntfFlux,
ohlcIntfFlux.skip(1),
Mono.subscriberContext()
.map(c -> c.getOrDefault("isRealtime", new AtomicBoolean())).repeat()
)
.filter(l ->
l.getT3().get() ||
(!l.getT2().getEndTime().isEqual(l.getT1().getEndTime())))
.map(Tuple2::getT1)
and set the AtomicBoolean to the context on the subscriber end and just change the value inside this variable reference, when I need the signal on the upstream, but it doesn't change at all:
input
.onErrorContinue((throwable, o) -> throwable.getMessage())
.doOnComplete(() -> {
System.out.println("Number of trades for the strategy: " + tradingRecord.getTradeCount());
// Analysis
System.out.println("Total profit for the strategy: " + new TotalProfitCriterion().calculate(timeSeries, tradingRecord));
})
.doOnNext(this::defaultRealtimeEvaluator)
.subscriberContext(Context.of("isRealtime", isRealtimeAtomic))
.subscribe();
at least with repeat the Flux doesn't disconnect but the value I'm getting out of it is not being updated. No other clues I have.
Spring-webflux: 2.1.3.RELEASE

this works:
input
.onErrorContinue((throwable, o) -> throwable.getMessage())
.doOnComplete(() -> { ... }
.flatMap(bar -> Mono.subscriberContext()
.map(c -> Tuples.of(bar, c)))
.doOnNext(this::defaultRealtimeEvaluator)
.subscriberContext(Context.of("isRealtime", new AtomicBoolean()))
.subscribe();
so the point is to set AtomicBoolean in my case as the cotnext and then extract this variable out of the context if you want to change it's value. the same on the upstream flux.

Chain two answers Single

I need to link two RX Single responses - Retrofit to get an ArrayList with the two responses that return a List
I tried to process both answers with Map, Flatmap, but I have not achieved what I expected
final ArrayList <List<Consent>> listAllConsents = new ArrayList<>();
Single<List<Consent>> responseDspConsent = subscriptionCenterRemoteDataSource.getConsents(Globals.getAuthorizationTokenUser());
Single<List<Consent>> responseDspConsentByApp = subscriptionCenterRemoteDataSource.getConsentsByApp(Globals.getAuthorizationTokenUser());
responseDspConsentByApp.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
responseDspConsent.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.flatMap(consentData -> {
List<Consent> consentList = consentData;
listAllConsents.add(consentList);
return responseDspConsentByApp.map(consentDataByApp -> {
List<Consent> consentListByApp = consentDataByApp;
listAllConsents.add(consentListByApp);
return listAllConsents;
});
})
.subscribe(consentData -> {
Log.v("Entramoss", "Valor: " + listAllConsents.get(0).get(0).getTitle());
paintAllConsents(listAllConsents);
});
I need to have all the objects of the two responses in the arrayList so I can paint them later.

You have 2 ways to do this.
1.You can use Observable.concat(Obs 1, Obs 2). The concat operator concatenates the observables and returns a single observable which first emits the items from the first observable then the second one. Source: http://reactivex.io/documentation/operators/concat.html
Single<List<Consent>> responseDspConsent = subscriptionCenterRemoteDataSource
.getConsents(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Single<List<Consent>> responseDspConsentByApp = subscriptionCenterRemoteDataSource
.getConsentsByApp(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable.concat(responseDspConsent.toObservable(),responseDspConsentByApp.toObservable())
.toList()
.doOnSuccess((list) -> {
paintAllConsents(list);
})
.subscribe();
2.You can use the .concatWith operator which does the same thing as the concat operator but now it concats an observable to another without creating a new observable.
Single<List<Consent>> responseDspConsent = subscriptionCenterRemoteDataSource
.getConsents(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Single<List<Consent>> responseDspConsentByApp = subscriptionCenterRemoteDataSource
.getConsentsByApp(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
responseDspConsent.concatWith(responseDspConsentByApp)
.toList()
.doOnSuccess((list) -> {
paintAllConsents(list);
})
.subscribe();

if your order do meter i suggest you to do .concat as #ebasha response but if your order doesn't meter i suggest you to use .merge becase is much more faster than .concat because concat subscribe streams one by one and merge subscribe streams immediately
Single<List<Consent>> responseDspConsent = subscriptionCenterRemoteDataSource
.getConsents(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Single<List<Consent>> responseDspConsentByApp = subscriptionCenterRemoteDataSource
.getConsentsByApp(Globals.getAuthorizationTokenUser())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable.merge(responseDspConsent.toObservable(),responseDspConsentByApp.toObservable())
.toList()
.doOnSuccess((list) -> {
paintAllConsents(list);
})
.subscribe();

How to use the result of a thenCompose multiple times in Java 8?

I have a series of thenCompose calls, similar to
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenCompose(id -> getProfile(id))
.thenCompose(userProfile -> updateSomething(userProfile))
.thenCompose(__ -> notifyUser(id))
.thenAccept(__ -> doSomething())
.exceptionally(t -> doSomethingElse());
The getUserID call returns a CompletionStage<String> which I use in the next call for getProfile. I need the same id again for the notifyUser call. How to make it available there? The IDE is showing
Cannot resolve symbol id.

The issue with your current code is that by the time you reach .thenCompose(__ -> notifyUser(id)), the variable id is not in scope anymore.
A simple solution in this case would be to invoke multiple thenCompose directly on the CompletionStage returned by getProfile:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenCompose(id ->
getProfile(id)
.thenCompose(userProfile -> updateSomething(userProfile))
.thenCompose(__ -> notifyUser(id))
)
// rest of chain calls

I think, your code becomes simpler, if you don’t insist on using thenCompose for every step:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenAccept(id -> {
updateSomething(getProfile(id).join());
notifyUser(id);
})
.thenRun(() -> doSomething())
.exceptionally(t -> doSomethingElse());
If having each step effectively sequential is your requirement, you can simply use join:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenAccept(id -> {
updateSomething(getProfile(id).join()).join();
notifyUser(id).join();
})
.thenRun(() -> doSomething())
.exceptionally(t -> doSomethingElse());
Considering that the whole chain is effectively sequential, you may just write it straight-forward:
myObject.updateDB(payload)
.thenRun(() -> {
YourUserIDType id = getUserID(payload.ID()).join();
updateSomething(getProfile(id).join()).join();
notifyUser(id).join();
doSomething();
})
.exceptionally(t -> doSomethingElse());

RxJava collect() & takeUntil()

I have a list of users of unknown size. What i want is to query first 30 and update UI. Then i want to query all others by offset with step of 100 until i get last pack of users - should i use takeUntil here?) and when i get - i update UI by adding remaining users (combined with reduce() i belive).
This is my code:
final int INITIAL_OFFSET = 0;
final int INITIAL_LIMIT = 30;
// Loading first 30 users to immediately update UI (better UX)
getServerApi().getAllFriends(userId, "photo_50", INITIAL_OFFSET, INITIAL_LIMIT)
// Loading remaining users 100 by 100 and updating UI after all users been loaded
.flatMap(users -> {
AtomicInteger newOffset = new AtomicInteger(INITIAL_LIMIT);
return Observable.just(users)
.flatMap(users1 -> getServerApi().getAllFriends(userId, "photo_50", newOffset.get(), Config.DEFAULT_FRIEND_REQUEST_COUNT))
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.collect(() -> new ArrayList<User>(), (b, s) -> {
b.addAll(s);
newOffset.set(newOffset.get() + Config.DEFAULT_FRIEND_REQUEST_COUNT);
})
.repeat()
.takeUntil(friends -> friends.size() == 0);
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(users -> getView().appendAllFriends(users),
throwable -> getView().setError(processFail(throwable, ServerApi.Action.GET_ALL_FRIENDS), false));
But seems i do something wrong because onNext is called each time the retrofit call is made.

Answering my own question. Adels answer is good, but i needed to have a single subscription (i'm using Nucleus MVP library) and i wanted to use collect() and takeUntil() instead of while loop (which requires blocking retrofit interface method).
Spent some hours and finally got it:
final int INITIAL_LIMIT = 30;
// Loading first 30 users to immediately update UI (better UX)
getServerApi().getAllFriends(userId, "photo_50", null, INITIAL_LIMIT)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
// Updating UI 1st time or show error
.doOnNext(users -> getView().appendAllFriends(users))
.doOnError(throwable -> getView().setError(processFail(throwable, ServerApi.Action.GET_ALL_FRIENDS), false))
// Loading remaining users 100 by 100 and updating UI after all users been loaded
.flatMap(users -> {
AtomicInteger newOffset = new AtomicInteger(INITIAL_LIMIT);
ArrayList<User> remainingUsers = new ArrayList<>();
AtomicBoolean hasMore = new AtomicBoolean(true);
return Observable.just(users)
.flatMap(users1 -> getServerApi().getAllFriends(userId, "photo_50", newOffset.get(), Config.DEFAULT_FRIEND_REQUEST_COUNT))
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.collect(() -> remainingUsers, (b, s) -> {
// Needed for takeUntil
hasMore.set(b.addAll(s));
newOffset.set(newOffset.get() + Config.DEFAULT_FRIEND_REQUEST_COUNT);
})
.repeat()
.takeUntil(friends -> !hasMore.get())
// Grab all items emitted by collect()
.last()
// Updating UI last time
.doOnNext(users2 -> getView().appendAllFriends(users2));
})
.subscribe();
Maybe it will be useful for other people which are also using Nucleus.

// cache() will ensure that we load the first pack only once
Observable<Users> firstPack = firstPack().cache();
// this subscription is for updating the UI on the first batch
firstPack
.observeOn(AndroidSchedulers.mainThread())
.subscribe(x -> draw(x), e -> whoops(e));
// this subscription is for collecting all the stuff
// do whatever tricks you need to do with your backend API to get the full list of stuff
firstPack
.flatMap(fp -> rest(fp))
.observeOn(AndroidSchedulers.mainThread())
.subscribe(x -> allUsers(x), e -> whoops(e));
// I would do this in a simple while loop
Observable<List<User>> rest(List<User> firstPack) {
return Observable.create(sub -> {
final List<User> total = firstPack;
try {
while (!sub.isUnsubscribed()) {
final List<User> friends = api.getFriendsBlocking(total.size());
if (friends.isEmpty()) {
sub.onNext(total);
sub.onCompleted();
} else {
total.addAll(friends);
}
}
} catch(IOException e) {
sub.onError(e);
}
})
}