We Keep Coding

Throws java.lang.IllegalStateException: block()/blockFirst()/blockLast() when retrying invalid token

I am trying to update the token when I get a response with 401 status code.
In order to do that, I used web client. I know that this mainly used to do reactive development but since resttemplate will soon be deprecated I went for this option.
The issue I am facing is that when it does call the api endpoint to get the new token, it throws a 'java.lang.IllegalStateException: block()/blockFirst()/blockLast() '. And make sense as it stated in the exception message It is not supported in thread reactor-http-nio-3.
I saw that there is a map and flatmap option, but I couldn't figure out how to use it inside the doBeforeRetry() to make it process in a different stream.
I need to have that new token before retrying.
So the question is : How can I get the token via another call and then still do the retry ?
I was able to make it work by using a try catch but I would like to find the solution how to use it inside that retry method.
I also try to block the token request by replacing the token response by a Mono and block it by using myMono.toFuture().get() as stated here block()/blockFirst()/blockLast() are blocking error when calling bodyToMono AFTER exchange()
Here is the code :
Method responsible for the call :
public String getValueFromApi(HashMap<String, Object> filter) {
String response = "";
response = webclient
.post()
.uri(endpoint)
.header("token", token.getToken())
.bodyValue(filter)
.retrieve()
.bodyToMono(String.class)
.retryWhen(Retry.max(3).doBeforeRetry(
retrySignal -> tokenService.getTokenFromApi(env)
).filter(InvalidTokenException.class::isInstance))
.block();
return response;
}
Method that retrieve the token :
public void getTokenFromApi(Environment env) {
HashMap<String, String> requestBody = new HashMap<>();
requestBody.put("name", "name");
requestBody.put("password", "password");
String response = WebClient
.builder()
.baseUrl(BASE_PATH)
.defaultHeader(HttpHeaders.CONTENT_TYPE, "application/json")
.build()
.post()
.uri(tokenUri)
.body(BodyInserters.fromValue(requestBody))
.retrieve()
.bodyToMono(String.class)
.block();
getTokenFromResponse(response);
}
private void getTokenFromResponse(String reponse) {
JsonObject tokenObject = new Gson().fromJson(reponse, JsonObject.class);
setToken(tokenObject.get("token").getAsString());
}
WebClient Builder :
#Bean
public WebClient webClientForApi(WebClient.Builder webClientBuilder) {
return webClientBuilder
.clientConnector(new ReactorClientHttpConnector(httpClient))
.filter(errorHandler())
.filter(logRequest())
.clone()
.baseUrl(BASE_PATH)
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, "application/json")
.build();
}
public ExchangeFilterFunction errorHandler() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
if (clientResponse.statusCode().equals(HttpStatus.UNAUTHORIZED)) {
return Mono.error(InvalidTokenException::new);
} else if (clientResponse.statusCode() == HttpStatus.INTERNAL_SERVER_ERROR) {
return Mono.error(ApiInternalServerException::new);
} else {
return Mono.just(clientResponse);
}
});
}
private ExchangeFilterFunction logRequest() {
return ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
logger.info("Request: {} {}", clientRequest.method(), clientRequest.url());
clientRequest.headers().forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value)));
return Mono.just(clientRequest);
});
}

Trying to send multiple lists of Strings in post request using webClient in spring

I am trying to send 3 different lists via webClient but I am getting a validation exception from API which expects interface java.util.List.
This is how controller endpoint looks like:
#PostMapping(value = "/uploadFiles", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseDto<List<UploadFileDto>> uploadFiles(#RequestParam("A") List<String> a,
#RequestParam("B") List<Short> b,
#RequestParam("C") List<MultipartFile> c)
And this is how i am trying to create a request:
private void callUploadFileEndpoint() throws IOException{
var uploadFileRequest = filesGenerator.generateRequest();
List<byte[]> byteArray = new ArrayList<>();
for(MultipartFile multipartFile : uploadFileRequest.getC()) {
var bytes = multipartFile.getBytes();
byteArray.add(bytes);
}
MultipartBodyBuilder multipartBodyBuilder = new MultipartBodyBuilder();
multipartBodyBuilder.part("A", uploadFileRequest.getA());
multipartBodyBuilder.part("B", uploadFileRequest.getB());
multipartBodyBuilder.part("C", byteArray);
try {
Tuple2<Long, String> responseTuple = webClient.post().uri("/v1/uploadFiles")
.headers(h -> h.setBearerAuth(bearerAuth))
.body(BodyInserters.fromMultipartData(multipartBodyBuilder.build()))
.retrieve()
.bodyToMono(String.class)
.elapsed()
.block();
System.out.printf("Response time: %d ms%n", responseTuple.getT1());
} catch (WebClientResponseException exception) {
System.err.println(exception.getResponseBodyAsString());
throw exception;
}
}
and UploadFileRequestClass :
#Data
#Builder
public class UploadFileRequest {
List<String> a;
List<Short> b;
List<MultipartFile> c;
}
Any idea how to build a proper body with multiple lists for the request? Is it even possible to build such complex request with webClient? Maybe I should use different one?

After some time I've realised that I wrongly understood the concept of the body as whole. I wanted to pass 3 different lists in a single body. Passing generic lists as queryParam string and multipart list as body resolved my issue.
for (int i = 0; i < uploadFileRequest.getC().size(); i++) {
multipartBodyBuilder.part("C", uploadFileRequest.getC().get(i).getResource());
}
try {
Tuple2<Long, String> responseTuple = webClient.post().uri(uriBuilder -> uriBuilder
.path("/v1/uploadFiles")
.queryParam("A", aSb.toString())
.queryParam("B", bSb.toString())
.build())
.headers(h -> h.setBearerAuth(bearerAuth))
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(BodyInserters.fromMultipartData(multipartBodyBuilder.build()))
.retrieve()
.bodyToMono(String.class)
.elapsed()
.block();
System.out.printf("Response time: %d ms%n", responseTuple.getT1());
System.out.println(responseTuple.getT2());
} catch (WebClientResponseException exception) {
System.err.println(exception.getResponseBodyAsString());
throw exception;
}

Spring WebClient - how to retry with delay based on response header

A little background
I've been learning Spring Webflux and reactive programming and have gotten stuck on a problem I'm trying to solve around retry logic using Spring Webclient. I've created a client and made successful calls to an external web-service GET endpoint that returns some JSON data.
Problem
When the external service responds with a 503 - Service Unavailable status, the response includes a Retry-After header with a value that indicates how long I should wait before retrying the request. I want to find a way within Spring Webflux/Reactor to tell the webClient to retry it's request after X period, where X is the difference between now and the DateTime that I parse out of the response header.
Simple WebClient GET request
public <T> Mono<T> get(final String url, Class<T> clazz) {
return webClient
.get().uri(url)
.retrieve()
.bodyToMono(clazz);
}
WebClient Builder
I use a builder to create the webClient variable used in the above method, and it's stored as an instance variable in the class.
webClientBuilder = WebClient.builder();
webClientBuilder.codecs(clientCodecConfigurer -> {
clientCodecConfigurer.defaultCodecs();
clientCodecConfigurer.customCodecs().register(new Jackson2JsonDecoder());
clientCodecConfigurer.customCodecs().register(new Jackson2JsonEncoder());
});
webClient = webClientBuilder.build();
Retry When
I've tried to understand and use the retryWhen method with the Retry class, but can't figure out if I can access or pass through the response header value there.
public <T> Mono<T> get(final String url, Class<T> clazz) {
return webClient
.get().uri(url)
.retrieve()
.bodyToMono(clazz);
.retryWhen(new Retry() {
#Override
public Publisher<?> generateCompanion(final Flux<RetrySignal> retrySignals) {
// Can I use retrySignals or retryContext to find the response header somehow?
// If I can find the response header, how to return a "yes-retry" response?
}
})
}
Filter(s) with Extra Logic and DB Interaction
I've also tried to do some extra logic and use filters with the WebClient.Builder, but that only gets me to a point of halting a new request (call to #get) until a previously established Retry-After value has elapsed.
webClientBuilder = WebClient.builder();
webClientBuilder.codecs(clientCodecConfigurer -> {
clientCodecConfigurer.defaultCodecs();
clientCodecConfigurer.customCodecs().register(new Jackson2JsonDecoder());
clientCodecConfigurer.customCodecs().register(new Jackson2JsonEncoder());
});
webClientBuilder.filter(ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
final Clock clock = Clock.systemUTC();
final int id = (int) clientRequest.attribute("id"); // id is saved as an attribute for the request, pull it out here
final long retryAfterEpochMillis = // get epoch millisecond from DB for id
if(epoch is in the past) {
return Mono.just(clientRequest);
} else { // have to wait until epoch passes to send request
return Mono.just(clientRequest).delayElement(Duration.between(clock.instant(), Instant.ofEpochMilli(retryAfterEpochMillis)));
}
})
);
webClient = webClientBuilder.build();
.onStatus(HttpStatus::isError, response -> {
final List<String> retryAfterHeaders = response.headers().header("Retry-After");
if(retryAfterHeaders.size() > 0) {
final long retryAfterEpochMillis = // parse millisecond epoch time from header
// Save millisecond time to DB associated to specific id
}
return response.bodyToMono(String.class).flatMap(body ->
Mono.error(new RuntimeException(
String.format("Request url {%s} failed with status {%s} and reason {%s}",
url,
response.rawStatusCode(),
body))));
})
Any help is appreciated, and if I can provide more contextual data to help, I will.

1. Retrieve header in retry builder
public class WebClientStatefulRetry3 {
public static void main(String[] args) {
WebClient webClient = WebClient.create();
call(webClient)
.retryWhen(Retry.indefinitely()
.filter(ex -> ex instanceof WebClientResponseException.ServiceUnavailable)
.doBeforeRetryAsync(signal -> Mono.delay(calculateDelay(signal.failure())).then()))
.block();
}
private static Mono<String> call(WebClient webClient) {
return webClient.get()
.uri("http://mockbin.org/bin/b2a26614-0219-4018-9446-c03bc1868ebf")
.retrieve()
.bodyToMono(String.class);
}
private static Duration calculateDelay(Throwable failure) {
String headerValue = ((WebClientResponseException.ServiceUnavailable) failure).getHeaders().get("Retry-After").get(0);
return // calculate delay here from header and current time;
}
}
2. Use expand operator to access the previous response and generate the next one
public class WebClientRetryWithExpand {
public static void main(String[] args) {
WebClient webClient = WebClient.create();
call(webClient)
.expand(prevResponse -> {
List<String> header = prevResponse.headers.header("Retry-After");
if (header.isEmpty()) {
return Mono.empty();
}
long delayInMillis = // calculate delay from header and current time
return Mono.delay(Duration.ofMillis(delayInMillis))
.then(call(webClient));
})
.last()
.block();
}
private static Mono<ResponseWithHeaders> call(WebClient webClient) {
return webClient.get()
.uri("https://example.com")
.exchangeToMono(response -> response.bodyToMono(String.class)
.map(rawResponse -> new ResponseWithHeaders(rawResponse, response.headers())));
}
#Data
static class ResponseWithHeaders {
private final String rawResponse;
private final ClientResponse.Headers headers;
}
}

How do I log out the body of a failed response to a Spring WebFlux WebClient request while returning the response to the caller?

I'm very new to reactive programming and I have a REST service that takes a request and then calls to another API using the WebFlux WebClient. When the API responds with a 4xx or 5xx response, I want to log the response body in my service, and then pass on the response to the caller. I've found a number of ways to handle logging the response, but they generally return Mono.error to the caller, which is not what I want to do. I have this almost working, but when I make the request to my service, while I get back the 4xx code that the API returned, my client just hangs waiting for the body of the response, and the service never seems to complete processing the stream. I'm using Spring Boot version 2.2.4.RELEASE.
Here's what I've got:
Controller:
#PostMapping(path = "create-order")
public Mono<ResponseEntity<OrderResponse>> createOrder(#Valid #RequestBody CreateOrderRequest createOrderRequest) {
return orderService.createOrder(createOrderRequest);
}
Service:
public Mono<ResponseEntity<OrderResponse>> createOrder(CreateOrderRequest createOrderRequest) {
return this.webClient
.mutate()
.filter(OrderService.errorHandlingFilter(ORDERS_URI, createOrderRequest))
.build()
.post()
.uri(ORDERS_URI)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(createOrderRequest)
.exchange()
.flatMap(response -> response.toEntity(OrderResponse.class));
}
public static ExchangeFilterFunction errorHandlingFilter(String uri, CreateOrderRequest request) {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
if (clientResponse.statusCode() != null && (clientResponse.statusCode().is5xxServerError() || clientResponse.statusCode().is4xxClientError())) {
return clientResponse.bodyToMono(String.class)
.flatMap(errorBody -> OrderService.logResponseError(clientResponse, uri, request, errorBody));
} else {
return Mono.just(clientResponse);
}
});
}
static Mono<ClientResponse> logResponseError(ClientResponse response, String attemptedUri, CreateOrderRequest orderRequest, String responseBody) {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
try {
log.error("Response code {} received when attempting to hit {}, request:{}, response:{}",
response.rawStatusCode(), attemptedUri, objectMapper.writeValueAsString(orderRequest),
responseBody);
} catch (JsonProcessingException e) {
log.error("Error attempting to serialize request object when reporting on error for request to {}, with code:{} and response:{}",
attemptedUri, response.rawStatusCode(), responseBody);
}
return Mono.just(response);
}
As you can see, I'm simply trying to return a Mono of the original response from the logResponseError method. For my testing, I'm submitting a body with a bad element which results in a 422 Unprocessable Entity response from the ORDERS_URI endpoint in the API I'm calling. But for some reason, while the client that called the create-order endpoint receives the 422, it never receives the body. If I change the return in the logResponseError method to be
return Mono.error(new Exception("Some error"));
I receive a 500 at the client, and the request completes. If anyone knows why it won't complete when I try to send back the response itself, I would love to know what I'm doing wrong.

Can't have your cake and eat it too!
The issue here is that you are trying to consume the body of the response twice, which is not allowed. Normally you would get an error for doing so.
Once in
return clientResponse.bodyToMono(String.class)
but also in
response.toEntity(OrderResponse.class)
which actually runs
#Override
public <T> Mono<ResponseEntity<T>> toEntity(Class<T> bodyType) {
return WebClientUtils.toEntity(this, bodyToMono(bodyType));
}
So one solution would be to process the ResponseEntity instead of the ClientResponse as follows since you don't actually want to do any reactive stuff with the body
public Mono<ResponseEntity<OrderResponse>> createOrder(CreateOrderRequest createOrderRequest) {
return this.webClient
//no need for mutate unless you already have things specified in
//base webclient?
.post()
.uri(ORDERS_URI)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(createOrderRequest)
.exchange()
//Here you map the response to an entity first
.flatMap(response -> response.toEntity(OrderResponse.class))
//Then run the errorHandler to do whatever
//Use doOnNext since there isn't any reason to return anything
.doOnNext(response ->
errorHandler(ORDERS_URI,createOrderRequest,response));
}
//Void doesn't need to return
public static void errorHandler(String uri, CreateOrderRequest request,ResponseEntity<?> response) {
if( response.getStatusCode().is5xxServerError()
|| response.getStatusCode().is4xxClientError())
//run log method if 500 or 400
OrderService.logResponseError(response, uri, request);
}
//No need for redundant final param as already in response
static void logResponseError(ResponseEntity<?> response, String attemptedUri, CreateOrderRequest orderRequest) {
//Do the log stuff
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
try {
log.error("Response code {} received when attempting to hit {}, request:{}, response:{}",
response.getStatusCodeValue(), attemptedUri, objectMapper.writeValueAsString(orderRequest),
response.getBody());
} catch (JsonProcessingException e) {
log.error("Error attempting to serialize request object when reporting on error for request to {}, with code:{} and response:{}",
attemptedUri, response.getStatusCodeValue(), response.getBody());
}
}
Note that there isn't really a reason to use the ExchangeFilter since you aren't actually doing any filtering, just performing an action based off the response

Get http response code and all available body

I want ot implement WebFlux example client which can make request with http params and get the response body and http response code. I tried this:
public ClientResponse execute(NotificationMessage nm)
Mono<String> transactionMono = Mono.just(convertedString);
return client.post().uri(builder -> builder.build())
.header(HttpHeaders.USER_AGENT, "agent")
.body(transactionMono, String.class).exchange().block();
}
private static String convert(Map<String, String> map) throws UnsupportedEncodingException {
String result = map.entrySet().stream().map(e -> encode(e.getKey()) + "=" + encode(e.getValue()))
.collect(Collectors.joining("&"));
return result;
}
private static String encode(String s) {
try {
return URLEncoder.encode(s, "UTF-8");
} catch (UnsupportedEncodingException e) {
throw new IllegalStateException(e);
}
}
Can you give me some advice after .exchange() how I can get the http status code and all available body.

From the ClientResponse object returned by exchange you can use response.statusCode() to get the status and use response.bodyToMono() or bodyToFlux() to get the actual body. You should avoid using .block() in reactive programming and use .subscribe() or .flatMap() or other operators to get the data from Mono or Flux objects. Read more about reactive programming and Project reactor (used by spring webflux) here.
For eg:
public Mono<Data> execute(NotificationMessage nm)
return client.post().uri(builder -> builder.build())
.header(HttpHeaders.USER_AGENT, "agent")
.body(transactionMono, String.class).exchange()
.flatMap(response -> {
HttpStatus code = response.statusCode();
Data data = response.bodyToMono(Data.class);
return data;
});
}