Wrote a class as below:
class KeyFetcher {
String key = "";
Mono getKeys() {
try {
key = keyService.getKeys();
if(key == null) {
throw new RuntimeException("key value is null");
} else {
this.key = key;
}
} catch(Exception e) {
return Mono.error(e);
}
return Mono.just(key)
}
#Scheduled // spring scheduler
void fetchDataFromExternalService() {
client.fetchData(key) // returns Mono
.retry(3)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
}
}
I want to modify the fetchDataFromExternalService method to include the getKeys() also before calling external service, and retry three times on both methods. Retry will be like:
get keys - try 3 times on failure - then go to handleError and do not call external service.
if key are fetched within 3 retry boundary, then now fetch data from external service with max 3 retries.
if error encountered in client.fetchData(), then go to handleError but not retry getKeys9) again.
I tried below:
Mono.defer(()-> getKeys())
.flatMap(obj -> client.fetchData(key))
.retry(3)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
But the problem is that when getKey() returns response in say 2nd retry, it enters fetchData(), but if this methods throws exception, it enters handleError() before again re-entering getKey() which is not desirable. All I want is once getKey() retry is exhausted, control should enter handleError() and close the Mono. Same for fetchData(). I think I am doing something wrong above, but need suggestion as I am new to this. Thanks.
You are well describing your issue, and if you apply strictly your description to your code, you will get what you need.
In your code sample, you create a pipeline that fetch a key, then fetch data. Therefore, the retry is applied to the entire chain, because it receive a publisher that already chain both actions.
You want:
to fetch a key with 3 max retries:
Mono<Key> getKeyWithRetry = Mono.defer(() -> getKeys()).retry(3);
Fetch data from an input key, with retry:
Function<Key, Mono<Data>> fetchDataWithRetry = key -> fetchData(key).retry(3)
And finally, assemble all that with error handling:
getKeyWithRetry()
.flatMap(fetchDataWithRetry)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
Related
I am using Java 11 and project Reactor (from Spring). I need to make a http call to a rest api (I can only make it once in the whole flow).
With the response I need to compute two things:
Check if a document exists in the database (mongodb). If it does not exists then create it and return it. Otherwise just return it.
Compute some logic on the response and we are done.
In pseudo code it is something like this:
public void computeData(String id) {
httpClient.getData(id) // Returns a Mono<Data>
.flatMap(data -> getDocument(data.getDocumenId()))
// Issue here is we need access to the data object consumed in the previous flatMap but at the same time we also need the document object we get from the previous flatMap
.flatMap(document -> calculateValue(document, data))
.subscribe();
}
public Mono<Document> getDocument(String id) {
// Check if document exists
// If not create document
return document;
}
public Mono<Value> calculateValue(Document doc, Data data) {
// Do something...
return value;
}
The issue is that calculateValue needs the return value from http.getData but this was already consumed on the first flatMap but we also need the document object we get from the previous flatMap.
I tried to solve this issue using Mono.zip like below:
public void computeData(String id) {
final Mono<Data> dataMono = httpClient.getData(id);
Mono.zip(
new Mono<Mono<Document>>() {
#Override
public void subscribe(CoreSubscriber<? super Mono<Document>> actual) {
final Mono<Document> documentMono = dataMono.flatMap(data -> getDocument(data.getDocumentId()))
actual.onNext(documentMono);
}
},
new Mono<Mono<Value>>() {
#Override
public void subscribe(CoreSubscriber<? super Mono<Value>> actual) {
actual.onNext(dataMono);
}
}
)
.flatMap(objects -> {
final Mono<Document> documentMono = objects.getT1();
final Mono<Data> dataMono = objects.getT2();
return Mono.zip(documentMono, dataMono, (document, data) -> calculateValue(document, data))
})
}
But this is executing the httpClient.getData(id) twice which goes against my constrain of only calling it once. I understand why it is being executed twice (I subscribe to it twice).
Maybe my solution design can be improved somewhere but I do not see where. To me this sounds like a "normal" issue when designing reactive code but I could not find a suitable solution to it so far.
My question is, how can accomplish this flow in a reactive and non blocking way and only making one call to the rest api?
PS; I could add all the logic inside one single map but that would force me to subscribe to one of the Mono inside the map which is not recommended and I want to avoid following this approach.
EDIT regarding #caco3 comment
I need to subscribe inside the map because both getDocument and calculateValue methods return a Mono.
So, if I wanted to put all the logic inside one single map it would be something like:
public void computeData(String id) {
httpClient.getData(id)
.map(data -> getDocument(data).subscribe(s -> calculateValue(s, data)))
.subscribe();
}
You do not have to subscribe inside map, just continue building the reactive chain inside the flatMap:
getData(id) // Mono<Data>
.flatMap(data -> getDocument(data.getDocumentId()) // Mono<Document>
.switchIfEmpty(createDocument(data.getDocumentId())) // Mono<Document>
.flatMap(document -> calculateValue(document, data)) // Mono<Value>
)
.subscribe()
Boiling it down, your problem is analogous to:
Mono.just(1)
.flatMap(original -> process(original))
.flatMap(processed -> I need access to the original value and the processed value!
System.out.println(original); //Won't work
);
private static Mono<String> process(int in) {
return Mono.just(in + " is an integer").delayElement(Duration.ofSeconds(2));
}
(Silly example, I know.)
The problem is that map() (and by extension, flatMap()) are transformations - you get access to the new value, and the old one goes away. So in your second flatMap() call, you've got access to 1 is an integer, but not the original value (1.)
The solution here is to, instead of mapping to the new value, map to some kind of merged result that contains both the original and new values. Reactor provides a built in type for that - a Tuple. So editing our original example, we'd have:
Mono.just(1)
.flatMap(original -> operation(original))
.flatMap(processed -> //Help - I need access to the original value and the processed value!
System.out.println(processed.getT1()); //Original
System.out.println(processed.getT2()); //Processed
///etc.
);
private static Mono<Tuple2<Integer, String>> operation(int in) {
return Mono.just(in + " is an integer").delayElement(Duration.ofSeconds(2))
.map(newValue -> Tuples.of(in, newValue));
}
You can use the same strategy to "hold on" to both document and data - no need for inner subscribes or anything of the sort :-)
So I have a method which returns an Vavr Try:
public Try<Result> request() {...}
request comes from a source which I cannot modify. Currently, I flatmap over the result from request and depending if the Result has an error return a Try with an exception or a success with the data from the Result:
public Try<Data> fetchData() {
return request().flatMap(result -> {
if (result.hasError()) {
return Try.failure(new FailedRequestException());
} else {
return Try.success(result.data());
}
});
}
What I want is in some places where fetchData is used first do something with the data if the Try is a success and if it is a failure, log an error if the error is a FailedRequestException, else, do something else with the exception, something like the following:
fetchData().andThen(data -> ...).onFailure(ex -> {
if (ex instanceOf FailedRequestException) {
log.error("Could not fetch data: " + ex.getMessage());
} else {
// Do something with the exception
...
}
});
My problem with this approach is that fetchData returns a Try so the caller cannot know that a FailedRequestException is part of the possible failures. I can let fetchData return a Try<Either<FailedRequestException, Data>> but this doesn't feel right either. Is there any way to do the above in a more elegant way? I also tried using the Match and Case but the Case expects a Function as handler and not a Consumer.
To sum up: you actually have 3 scenarios (success, failure with FailedRequestException, any other failure). This sounds like a job for pattern matching! Let's make the code as visible and expressive as the business requirement :)
Match(fetchData()).of(
Case($Success($()), data -> doStuff(data)),
Case($Failure($(instanceOf(FailedRequestException.class))), fre -> logFreAndReturnValue(fre)),
Case($Failure($()), e -> doSomethingWithOtherException(e))
);
FWIW, you can rewrite your fetchData implementation as such:
Try(request())
.mapFailure(Case($(), ignored -> new FailedRequestException()))
.map(Result::data);
As a rule of thumb, try to stick to using flatMap when the context (Success or Failure) may change. In your current fetchData implementation a success remains a success, a failure remains a failure, so it is a mapping between the input and the output, hence use map family of functions.
Cheers!
Even though I am using Supplier for my streams and using Supplier.Get() each time I want to retrieve my strem and perform a terminal operation on it, I am still getting the "stream has already been operated upon or closed" exception. Could anyone please look at my code and suggest what I am doing wrong?
Method where exception is being thrown:
private static void printMyDetails(Supplier<Stream<Result>> mySupplier, String myStatus) {
checkNotNull(mySupplier);
checkArgument(isNotEmpty(myStatus), "Invalid My status");
if (mySupplier.get().noneMatch(result -> true)) { //<-This is where the exception is thrown
if (log.isInfoEnabled()) {
log.info("Number of My with status '{}': {}", My, 0);
}
} else {
log.info("Number of My with status '{}': {}", myStatus, mySupplier.get().count());
log.info("Details of My(s) with status: '{}'", myStatus);
mySupplier.get().sorted().forEach(Utils::printMyNameAndDetails);
}
}
Place which is calling the above method:
rb.keySet().stream().filter(key -> containsIgnoreCase(key, "status")).map(rb::getString)
.filter(StringUtils::isNotEmpty).forEach(status -> {
var resultsWithExpectedStatusSupplier = requireNonNull(getResultsWithExpectedStatusSupplier(results, status));
resultsWithExpectedStatusSupplier.ifPresentOrElse(streamSupplier -> printMyDetails(streamSupplier, status), () -> {
if (log.isInfoEnabled())
log.info("0 My with status: {}", status);
});
});
The stream supplier:
private static Optional<Supplier<Stream<Result>>> getResultsWithExpectedStatusSupplier(
#NotNull List<Result> results, #NotNull String expectedStatus) {
checkArgument(!results.isEmpty(), "Results list is empty");
checkArgument(isNotEmpty(expectedStatus), "Invalid expected status");
var resultStreamWithExpectedStatus = requireNonNull(results.stream().filter(result -> ofNullable(result).map(Result::getStatus)
.allMatch(s -> isNotEmpty(s) && s.equalsIgnoreCase(expectedStatus))));
return resultStreamWithExpectedStatus.count() == 0 ? Optional.empty() : Optional.of(() -> resultStreamWithExpectedStatus);
}
The general problem is as Christian Ullenboom said: The stream has already been consumed. The exact location in your code is the call to resultStreamWithExpectedStatus.count() in the method getResultsWithExpectedStatusSupplier, as Stream.count is a reduction/terminal operation which consumes the stream.
As stated in e.g. this answer, you cannot get the streams size without consuming it. Fix it by storing the filtered items in a collection (e.g. Collectors.toList), querying the size there and returning the collection itself rather than the stream?
On a side note, I think you misuse Optional a bit too much. The code could be simpler passing empty streams (or even better: pass an empty, filtered collection).
You can consume a Stream just once. It looks like the Supplier is always giving the same Stream again and again. After the first terminal operation the Stream is drained; the Stream from the Supplier has to be a new Stream all the time.
For example I have two network request:
Single<FirstResponse> firstRequest = firstCall()
Single<SecondResponse> secondRequest = secondCall()
Now, the Second should only be called when the first is successful, so I do something like:
firstRequest().flatmap( firstResponse -> secondRequest ).subscribe()
It works well when if both calls gets completed successfully but how about I dont want the first call to get called when it already returned success?
So what I want to achieve is that, when the firstRequest successfully completed and the second failed, I only want the first to be skipped and only call the second.
Currently the only thing I could think of doing is something like:
public firstResponse = null;
public Single<FirstResponse> getFirstRequest() {
if (firstResponse!=null && firstResponse.isSuccess()) {
return Single.just(firstResponse);
} else {
return firstRequest;
}
}
public void doRequest() {
getFirstRequest()
.doOnSuccess( firstResponse -> this.firstReponse = firstResponse )
.flatMap( firstResponse -> secondRequest)
.subscribe()
}
I wonder if there is a better way to do this.
Thanks in advance.
I am using ReactiveCassandraRepository and I can create new record as below.
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.save(toBeSaved).map(saved -> saved.getId());
}
But I could not imagine how to update a specific field in a DB record since 2 reactive operations (findById & save) are involved.
I wrote a code as below to create or update status if exists, but seems to be not working.
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.findById(toBeSaved.getId())
.map(current -> abcRepository.save(transform(toBeSaved, current)).map(saved -> saved.getId()))
.flatMap(id -> id);
}
private Abc transform(Abc toBeSaved, Abc current) {
if(current == null) {
return toBeSaved;
} else {
current.setStatus(toBeSaved.getStatus());
return current;
}
}
Can someone please assist on that?
I expect your abcRepository's method to look something like this:
interface AbcRepository {
Mono<Abc> findById(String id);
Mono<Abc> save(Abc abc);
}
I guess from your code, for a given Abc you want to
read an Abc from repository with the same id,
map the data from the given Abc to the found one,
or just use the given Abc if the repository did not find any,
asynchronously save this Abc
and return the id of the saved element as Mono
I would do it like this:
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.findById(toBeSaved.getId()) // (1)
.map(abc -> transform(toBeSaved, abc)) // (2)
.defaultIfEmpty(toBeSaved) // (3)
.flatMap(abcRepository::save) // (4)
.map(Abc::getId); // (5)
}
private Abc transform(Abc toBeSaved, Abc current) {
current.setStatus(toBeSaved.getStatus());
return current;
}
A Mono can only receive one or no element, so when using Mono:map (2) you don't need to handle null values. The Mono returned by the abcRepository will receive the found Abc in which case the transformation call (2) is done or it will just emit a complete signal in which case the map does nothing and defaultIfEmpty (3) emits toBeSaved as fallback.
If you have a transformation that is asynchronous itself and thus results in another Mono use flatMap (4), else your intermediate result would be an Mono<Mono<Abc>>.
And always remember: nothing happens until subscribe is called.
saveAbc(myNewAbc).subscribe(id -> System.out.println("Saved Abc with id: " + id));
In the above example I expected your repository to just emit a complete signal when findById doesn't find any matching Abc that will complete the Mono as empty (which is the case when using ReactiveCassandraRepository!). If instead the repository emits an exception in this case you can for example use
.onErrorResume(t -> Mono.just(toBeSaved))
instead of defaultIfEmpty (3).