I want to retrieve all pages from a third-party resource. To do that I wrote this:
final WebClient webClient = WebClient.builder()
.baseUrl("http://resource.com")
.build();
Flux.fromStream(IntStream.iterate(0, i -> i + 1).boxed())
.flatMap(i -> webClient.get()
.uri("/data?page={page}", i)
.retrieve()
.bodyToMono(Page.class))
.takeWhile(Page::isMoreAvailable)
.flatMapIterable(Page::getData)
but it doesn't work properly. flatMap is called multiple times and it performs multiple requests with different pages before takeWhile retrieves the first response.
If I change it to this:
.map(i -> webClient.get()
.uri("/data?page={page}", i)
.retrieve()
.bodyToMono(Page.class)
.block())
it works well.
So, how can I achieve this with a flatMap?
ok, so flatmap processing concurrent
.flatMap(i -> webClient.get()
.uri("/data?page={page}", i)
.retrieve()
.bodyToMono(Page.class), 1)
did the trick.
Related
A little background
I would like to call service's APIs, while doing retries on 5xx errors. Also, I would like to get an access to every failed request (for logging purposes).
Code
getClient()
.get()
.uri("http://example.com")
.retrieve()
.onStatus(HttpStatus::is5xxServerError, rsp -> Mono.error(new ApiServerException("Server error", rsp.rawStatusCode())))
.bodyToMono(ReportListResponse.class)
.retryWhen(
Retry
.backoff(3, Duration.ofSeconds(1))
.filter(throwable -> throwable instanceof ApiServerException)
)
.block();
Issue
How can I achieve the goal of being able to access the response of every failed request? I was trying to retrieve the body while using rsp.bodyToMono(String.class) in onStatus method. Unfortunately it didn't give me an expected output.
You would need to use response.bodyToMono in the onStatus to get response body. The following example shows how to deserialize body into String but you could define POJO as well.
getClient()
.get()
.uri("http://example.com")
.retrieve()
.onStatus(HttpStatus::isError, response ->
response.bodyToMono(String.class)
.doOnNext(responseBody ->
log.error("Error response from server: {}", responseBody)
)
// throw original error
.then(response.createException())
)
.bodyToMono(ReportListResponse.class)
}
The case is the following:
There is an API endpoint that returns as response "pending" or "pompleted".
I want to repeatedly call this, let's say every 10 seconds, until I get a "Completed" response. If it keeps responding "Pending" for 5 minutes, I want a timeout.
I've read about repeat, repeatWhen and repeatWhenEmpty, but I can't get it done.
An example of my code is the following :
String getStatusResponse = webClient
.get()
.uri(getStatusUri)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(String.class)
.filter(response -> equals("completed"))
.repeatWhenEmpty(Repeat.onlyIf(r -> false)
.fixedBackoff(Duration.ofSeconds(10))
.timeout(Duration.ofMinutes(5)))
.block();
Edit after #Michael McFadyen's comment :
My code is now the following :
GenerationApiGetStatus getStatusResponse = webClient
.get()
.uri(getStatusUri)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(GenerationApiGetStatus.class)
.filter(response -> response.getStatus().equals("completed"))
.repeatWhenEmpty(Repeat.onlyIf(r -> true)
.fixedBackoff(Duration.ofSeconds(10))
.timeout(Duration.ofMinutes(5)))
.block();
GenerationApiGetStatus has a property "status" for JSON unmarshalling.
The problem is that I keep receiving null. If a change the code to just get once the status and go on, I correctly receive "Pending".
I believe something is wrong with the filtering :
.filter(response -> response.getStatus().equals("completed"))
Any ideas?
From the Api Docs of Retry.onlyIf
Repeat function that repeats only if the predicate returns true.
In your code sample you are always returning false
Repeat.onlyIf(r -> false)
As a result, the repeat will never happen.
You can alter your Retry to the code below to get your desired behaviour.
Repeat.onlyIf(r -> true)
.fixedBackoff(Duration.ofSeconds(10))
.timeout(Duration.ofMinutes(5))
I have this post method:
webClientBuilder
.build()
.get()
.uri("uri")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::isError, response -> Mono.error(new CustomException("Response is in status: ".concat(response.statusCode().toString()))))
.bodyToMono(GetResponse.class)
.log()
.flatMap(response ->
Mono.fromSupplier(() -> updateMember(entity.getId(), getScore(response)))
.subscribeOn(Schedulers.boundedElastic()))
.block();
I should create a short poll on it, since the api I called could delay answering me, I should be able to call it several times, maybe every 3/5 seconds. The external service always gives me an answer, but I have to verify that a specific field in the answer is not null. I have to repeat it at most 5 times, if after 5 times it returns null, I will pass null to my method (updateMember).
I am trying something like this:
webClientBuilder
.build()
.get()
.uri("uri")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::isError, response -> Mono.error(new CustomException("response is in status: ".concat(response.statusCode().toString()))))
.bodyToMono(GetResponse.class)
.filter(response -> Objects.nonNull(response.getAnag().getSummary()))
.repeatWhenEmpty(Repeat.onlyIf(repeatContext -> true)
.exponentialBackoff(Duration.ofSeconds(5), Duration.ofSeconds(10)).timeout(Duration.ofSeconds(30)))
.delaySubscription(Duration.ofSeconds(10))
.flatMap(response -> Mono.fromSupplier(() -> updateMember(entity.getId(), response.getAnag().getSummary()))
.subscribeOn(Schedulers.boundedElastic()))
.block();
I would like to repeat at most 5 times, only if that value in the answer is null otherwise I pass the value directly.
Can you help me?
If you get a timeout exception because of the delay, you can add one of the '.retry()'s operators to the chain. Perhaps this will be helpful:
.retryWhen(Retry.fixedDelay(3, Duration.ofSeconds(3)))
Example from reactor reference documentation
I have been looking into Reactive Programming and recently tried to build a POC with Spring WebFlux. I want to start simple and just use the WebClient to download an image; specifically https://greatatmosphere.files.wordpress.com/2013/02/great-atmosphere-149-tenaya-lake-yosemite-national-park-2.jpg
I have tried the following code
String block = WebClient.create("https://greatatmosphere.files.wordpress.com/2013/02/great-atmosphere-149-tenaya-lake-yosemite-national-park-2.jpg")
.get()
.accept(MediaType.IMAGE_JPEG)
.retrieve()
.bodyToMono(String.class)
.doOnError(WebClientResponseException.class,
ex -> System.out.println(ex.getStatusCode() + ": " + ex.getResponseBodyAsString()))
.log()
.block();
System.out.println("output:" + block);
but it does not work as expected. It seems that the data is continually be streamed and the get request does not terminate.
I am certain that I am missing something simple but I cannot seem to figure it out. I have tried a variety of parameters but the results are the same.
How do I use the WebClient to download the image and then terminate?
Catch image as byte[] not String
byte[] image = WebClient.create("https://greatatmosphere.files.wordpress.com/2013/02/great-atmosphere-149-tenaya-lake-yosemite-national-park-2.jpg")
.get()
.accept(MediaType.IMAGE_JPEG)
.retrieve()
.bodyToMono(byte[].class)
.block();
I'm a java 7 developer (finally) taking his first steps in java 8. A lot of these things are still new to me. I'm trying to use the spring 5 WebClient since the documentation states RestTemplate will be moved away from in favor of WebClient.
webClient
.method(HttpMethod.POST)
.uri(uriBuilder -> uriBuilder.pathSegment("api", "payments").build())
.body(BodyInserters.fromObject(createPostRequest(paymentConfirmationData)))
.accept(MediaType.APPLICATION_JSON)
.exchange()
.doAfterSuccessOrError((clientResponse, throwable) -> {
if (clientResponse.statusCode().is5xxServerError()
|| clientResponse.statusCode().is4xxClientError()) {
logger.error("POST request naar orchestration layer mislukt, status: [{}]", clientResponse.bodyToMono(String.class));
Mono.error(throwable);
} else {
logger.error("POST request naar orchestration layer gelukt");
}
})
.block();
I'm trying to throw an exception in the .doAfterSuccesOrError. However I can't use throw throwable cause then it just adds a try catch around it. I read a few articles and this is my last attempt by adding Mono.error(throwable) but since there is no return I'm pretty sure this is the reason there is no effect.
This is a POST call that returns a 204 No Content on success. At the moment I'm getting a 422 although that shouldn't be important in this particular issue.
Can someone teach me how to throw exceptions back to the calling environment ?
There is a special method for handling the status codes. More here
Your code should look like
webClient.method(HttpMethod.POST)
.uri(uriBuilder -> uriBuilder.pathSegment("api", "payments").build())
.body(BodyInserters.fromObject(createPostRequest(paymentConfirmationData)))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::is4xxServerError, response -> ...)
.onStatus(HttpStatus::is5xxServerError, response -> ...)
...
.block();
Remember that when onStatus is used, if the response is expected to have content, then the onStatus callback should consume it. If not, the content will be automatically drained to ensure resources are released.
I ended up with the following code eventually
webClient
.method(HttpMethod.POST)
.uri(uriBuilder -> uriBuilder.pathSegment("api", "payments").build())
.body(BodyInserters.fromObject(createPostRequest(paymentConfirmationData)))
.accept(MediaType.APPLICATION_JSON)
.exchange()
.doOnSuccess((clientResponse) -> {
if (clientResponse.statusCode().is5xxServerError()
|| clientResponse.statusCode().is4xxClientError()) {
logger.error("POST request naar orchestration layer mislukt, status: [{}]", clientResponse.statusCode());
throw new RuntimeException("POST request naar orchestration layer mislukt");
} else {
logger.error("POST request naar orchestration layer gelukt");
}
})
.doOnError((throwable) -> {
logger.error("POST request naar orchestration layer mislukt");
throw new RuntimeException("POST request naar orchestration layer mislukt", throwable);
})
.block();
For those searching on how to approach exception and error handling. Have a look at this reference document on the Reactor Project: https://projectreactor.io/docs/core/release/reference/index.html#_error_handling_operators