We Keep Coding

Java Reactive. How to wait for all data in flux and then process them

I'm getting data from mongo reactive repository and updating it. Then I have to collect all data in one collection and path it to another service to get more info. Then I should map them in one Flux. My code is:
Flux<Views> views = someRepository.findAllByUsersIn(userId).doOnNext(v -> {
v.setInterlocutor(v.getUsers().stream().filter(u -> !userId.equals(u)).findFirst().orElse(null));
});
return Flux.zip(views.map(view -> conversionService.convert(view, ResponseViewDto.class)), getUserInfo(views).flux())
.flatMap(fZip -> {
ResponseViewDto dto = fZip.getT1();
dto.setInterlocutor(fZip.getT2().get(dto.getInter()));
return Flux.just(dto);
});
getUserInfo does collecting usersId and sends them to another service and returns expanded info.
I found that getting from DB calls 2 times and I can understand why, but is there any solution to do it once and still be not blocking.

Thanks to Adhika Setya Pramudita for help. The way to do what I need is just to use cache() method

Difference between Mono.then and Mono.flatMap/map

Say I want to call a webservice1 and then call webservice2 if the first was successful.
I can do the following (just indicative psuedo code) :-
Mono.just(reqObj)
.flatMap(r -> callServiceA())
.then(() -> callServiceB())
or
Mono.just(reqObj)
.flatMap(r -> callServiceA())
.flatMap(f -> callServiceB())
What is the difference between the two, when using the mono.just() for a single element?

flatMap should be used for non-blocking operations, or in short anything which returns back Mono, Flux.
map should be used when you want to do the transformation of an object /data in fixed time. The operations which are done synchronously.
For ex:
return Mono.just(Person("name", "age:12"))
.map { person ->
EnhancedPerson(person, "id-set", "savedInDb")
}.flatMap { person ->
reactiveMongoDb.save(person)
}
then should be used when you want to ignore element from previous Mono and want the stream to be finised

Here's a detailed explanation from #MuratOzkan
Copy pasting the TL DR answer:
If you care about the result of the previous computation, you can use map(), flatMap() or other map variant. Otherwise, if you just want the previous stream finished, use then().
In your example, looks like your service calls do not require the input of the upstream, then you could use this instead:
Mono.just(reqObj)
.then(() -> callServiceA())
.then(() -> callServiceB())

What is the top first use case you think of, when you see the 'flatMap' method in someone else's code?

Sorry for some kind of theoretical question, but I'd like to find a way of quick reading someone else's functional code, building chain of methods use templates.
For example:
Case 1.
When I see use of .peek method or .wireTap from Spring Integration, I primarily expect logging, triggering monitoring or just transitional running external action, for instance:
.peek(params ->
log.info("creating cache configuration {} for key class \"{}\" and value class \"{}\"",
params.getName(), params.getKeyClass(), params.getValueClass()))
or
.peek(p ->
Try.run(() -> cacheService.cacheProfile(p))
.onFailure(ex ->
log.warn("Unable to cache profile: {}", ex.toString())))
or
.wireTap(sf -> sf.handle(msg -> {
monitoring.profileRequestsReceived();
log.trace("Client info request(s) received: {}", msg);
Case 2.
When I see use of .map method or .transform from Spring Integration, I understand that I'm up to get result of someFunction(input), for instance:
.map(e -> GenerateTokenRs.builder().token(e.getKey()).phoneNum(e.getValue()).build())
or
.transform(Message.class, msg -> {
ErrorResponse response = (ErrorResponse) msg.getPayload();
MessageBuilder builder = some tranforming;
return builder.build();
})
Current case.
But I don't have such a common view to .flatMap method.
Would you give me your opinion about this, please?
Add 1:
To Turamarth: I know the difference between .map and .flatMap methods. I actively use both .map, and .flatMap in my code.
But I ask community for theirs experience and coding templates.

It always helps to study the signature/javadoc of the streamish methods to understand them:
The flatMap() operation has the effect of applying a one-to-many transformation to the elements of the stream, and then flattening the resulting elements into a new stream.
So, typical code I expect, or wrote myself:
return someMap.values().stream().flatMap(Collection::stream)
The values of that map are sets, and I want to pull the entries of all these sets into a single stream for further processing here.
In other words: it is about "pulling out things", and getting them into a stream/collection for further processing.

I've found one more use template for .flatMap.
Let's have a look at the following code:
String s = valuesFromDb
.map(v -> v.get(k))
.getOrElse("0");
where Option<Map<String, String>> valuesFromDb = Option.of(.....).
If there's an entry k=null in the map, then we'll get null as a result of code above.
But we'd like to have "0" in this case as well.
So let's add .flatMap:
String s = valuesFromDb
.map(v -> v.get(k))
.flatMap(Option::of)
.getOrElse("0");
Regardless of having null as map's value we will get "0".

Applying a Single to an ObservableSource and not over-reading

I'm pretty new to RX in general, and rxjava in particular, pardon mistakes.
This operation depends on a two async operations.
The first uses a filter function to attempt to get a single entity from a list returned by an async Observable.
The second is an async operation that communicates with a device and produces an Observable of status updates.
I want to take the Single that is created from the filter function, apply that to pairReader(...), and subscribe to its Observable for updates. I can get this to work as shown, but only if I include the take(1) commented, otherwise I get an exception because the chain tries to pull another value from the Single.
Observable<DeviceCredential> getCredentials() {
return deviceCredentialService()
.getCredentials()
.flatMapIterable(event -> event.getData());
}
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
// A device is logically constrained to only have a single cred per org
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.take(1) // Without this I get an exception
.singleOrError();
}
Function<Organization, Observable<Reader.EnrollmentState>> pairReader(String name) {
return org -> readerService().pair(name, org);
}
getOrgFromCreds(orgid)
.flatMapObservable(pairReader(readerid))
.subscribe(state -> {
switch(state) {
case BEGUN:
LOG.d(TAG, "Pairing begun");
break;
case PAIRED:
LOG.d(TAG, "Pairing success");
callback.success();
break;
case NOTIFIED_SERVER:
LOG.d(TAG, "Pairing server notified");
break;
}},
error -> {
Crashlytics.logException(error);
callback.error(error.getLocalizedMessage());
});

If the source stream emits more than one item, singleOrError() is supposed to emit an error. Doc
For your case, use either first() or firstOrError() instead.
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.firstOrError();
}

If I got you right, you need to make some action using previously retrieved async data. So, you could use .zip() operator.
Here is an example:
Observable.zip(
getOrgFromCreds().toObservable(),
getCredentials(),
(first, second) -> /*create output object here*/
)
.subscribe(
(n) -> /*do onNext*/,
(e) -> /*do onError*/
);
Note, that .zip() operator will wait for both emission from two streams, and then it will create outer emission using the function you provided in "create output object here".
If you don't want to wait for both items - you can use .combineLatest().

The problem here turned out to be that the API was designed in an odd way (and unfortunately has extremely poor documentation). I couldn't figure out why I was getting duplicates, and thought I was using flatMapIterable incorrectly.
What the deviceCredentialService.getCredentials() call actually creates is an observable that emits DataEvent objects which are simple wrappers over a list of results, and with a flag of where the results came from.
The API designer wanted to allow the user to use locally cached data to fill the UI immediately while a longer request to a REST API executes. The DataEvent.from property is an enum that flags the source, either from the local device cache or from the remote API call.
The way I solved this was to simply ignore the results coming from local cache and only emit results from the API:
Observable<DeviceCredential> getCredentials() {
return deviceCredentialService()
.getCredentials()
// Only get creds from network
.filter(e -> e.getFrom() == SyncedDataSourceObservableFactory.From.SOURCE)
.flatMapIterable(e -> e.getData());
}
Single<Organization> getOrgFromCreds(String orgid) {
return getCredentials()
// A device is logically constrained to only have a single cred per org
.map(DeviceCredential::getOrganization)
.filter(org -> org.getId().equals(orgid))
.singleOrError();
}
The plan then is to use memoization to cache entities in a way that gives the implementing app access to cache invalidation. Since the provided interface doesn't allow squelching the API call, there is no way to work only with cache if the app feels its is fresh.

Flux endpoint from infinite java stream

I have an issue while processing a flux that is built from a Stream.generate construct.
The Java stream is fetching some data from a remote source, hence I implemented a custom supplier that has the data fetching logic embedded, and then used it to populate the Stream.
Stream.generate(new SearchSupplier(...))
My idea is to detect an empty list and use the Java9 feature of takeWhile ->
Stream.generate(new SearchSupplier(this, queryBody))
.takeWhile(either -> either.isRight() && either.get().nonEmpty())
(using Vavr's Either construct)
The repositoroy layer flux will then do:
return Flux.fromStream (
this.searchStream(...) //this is where the stream gets generated
)
.map(Either::get)
.flatMap(Flux::fromIterable);
The "service" layer is composed of some transformation steps on the flux, but the method signature is something like Flux<JsonObject> search(...).
Finally, the controller layer has a GetMapping:
#GetMapping(produces = "application/stream+json")
public Flux search(...) {
return searchService.search(...) //this is the Flux<JsonObject> parth
.subscriberContext(...) //stuff I need available during processing
.doOnComplete(() -> log.debug("DONE"));
}
My problem is that the Flux seems to never terminate.
Doing a call from Postman for example just shot the 'Loading...' part in the response section. When I terminate the process from my IDE the results are then flushed to postman and I see what I'm expecting. Also the doOnComplete lambda never gets called
What I noticed is that if I change the source of a Flux:
Flux.fromArray(...) //harcoded array of lists of jsons
the doOnComplete lambda is called and also the http connection closes, and results are displayed in postman.
Any idea of what might be the issue?
Thanks.

You could create the Flux directly using code that looks like this. Note that I'm adding some assumed methods which you would need to implement based on your how your SearchSupplier works:
Flux<SearchResultType> flux = Flux.generate(
() -> new SearchSupplier(this, queryBody),
(supplier, sink) -> {
SearchResultType current = supplier.next();
if (isNotLast(current)) {
sink.next(current);
} else {
sink.complete();
}
return supplier;
},
supplier -> anyCleanupOperations(supplier)
);