We Keep Coding

HttpClient asyncRequest and exponential backoff

I need to implement an exponential backoff on a request that might fail. However, it's implemented as an async request. Had this been done synchronously, I'd have a better idea on where to put the delay. Roughly, I'm thinking it'd work something like this:
// These would be configurable in application.yml
currentAttempt = 0;
maxAttempts = 3;
timeoutGrowth = 2;
currentDelayTime = 5ms;
repeatNeeded = false;
while(repeatNeeded && currentAttempt < maxAttempts) {
httpStatusCode = makeRequest(someService)
if(httpStatusCode == 503) {
repeatNeeded=true;
currentAttempt++;
currentDelayTime*=timeoutGrowthRate;
sleep(currentDelayTime)
}
}
However, with an async call, the caller to the function is given the time back to do something else until the Future is has something. Do I code the backoff within the getObservations() method below, or do I code this in the caller of that getObservations() method? Below is the call as it currently is:
public CompletableFuture getObservations(String text, Map<String, Object> bodyParams) throws URISyntaxException {
URI uri = getUri(text);
HttpRequest request = getRequest(uri, text, bodyParams);
Map<String, String> contextMap = Optional.ofNullable(MDC.getCopyOfContextMap()).orElse(Collections.emptyMap());
Instant startTime = Instant.now();
return httpClient.sendAsync(request, BodyHandlers.ofString())
.exceptionally(ex -> {
throw new ExternalToolException(externalServiceConfig.getName(), ex);
})
.thenApply(response -> {
long toolRequestDurationMillis = ChronoUnit.MILLIS.between(startTime, Instant.now());
if (HttpStatus.valueOf(response.statusCode()).is2xxSuccessful()) {
ToolResponse toolResponse = processResponse(response, toolRequestDurationMillis);
logToolResponse(toolResponse);
return toolResponse;
}
log.error("{} returned non-200 response code: {}", externalServiceConfig.getName(), response.statusCode());
throw new ExternalToolException(externalServiceConfig.getName(), response.statusCode());
});
}

If you could consider using reactive java that has very powerful API including retries. For example,
request()
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
there are more options like retries for the specific exceptions only or defining max backoff
request()
.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
.maxBackoff(5)
.filter(throwable -> isRetryableError(throwable))
You could use WebClient that is a non-blocking client exposing a fluent, reactive API over underlying HTTP client libraries such as Reactor Netty
webClient.get()
.uri("/api")
.retrieve()
.bodyToMono(String.class)
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
if for some reason, you still want to use HttpClient you can wrap CompletableFuture
Mono.fromFuture(httpClient.sendAsync(request, BodyHandlers.ofString()))
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

Testing onErrorResume() Spring Webflux

I have a service layer using Spring Webflux and reactor and I am writing unit test for this. I was able to test the good response scenario but not sure how to test onErrorResume() using StepVerifier. Also please let me know if there is a better way of handling exceptions in my controller(e.g: using switchIfEmpty())
Here is my controller method
public Mono<SomeType> getInfo(Integer id) {
return webClient
.get()
.uri(uriBuilder -> uriBuilder.path())
.header("", "")
.header("", "")
.header("", "")
.header(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.retrieve()
.bodyToMono(POJO.class)
.onErrorResume(ex -> {
if (ex instanceof WebFaultException)
return Mono.error(ex);
return Mono.error(new WebFaultException(ex.getMessage(), "Error on API Call", HttpStatus.INTERNAL_SERVER_ERROR));
});
}
}

You can mock the webclient and use Mockito.doThrow when webclientMock.get() is called.
YourWebClient webclientMock = mock(YourWebClient.class);
doThrow(RuntimeException.class)
.when(webclientMock)
.get();
// Call your method here
Exception exception = assertThrows(RuntimeException.class, () -> {
YourController.getInfo(someIntValue);
});
// If you chose to raise WebFaultException, addittionaly assert that the return values ( message, status) are the one you expected

An alternate way to test your WebClient code without having to mock the WebClient class itself, as that can quickly become very messy, is to build your WebClient with an ExchangeFunction that returns whatever response or error you expect. I've found this to be a happy medium between mocking out the WebClient and spinning up Wiremock for unit tests.
#Test
void ourTest() {
ExchangeFunction exchangeFunction = mock(ExchangeFunction.class);
// this can be altered to return either happy or unhappy responses
given(exchangeFunction.exchange(any(ClientRequest.class))).willReturn(Mono.error(new RuntimeException()));
WebClient webClient = WebClient.builder()
.exchangeFunction(exchangeFunction)
.build();
// code under test - this should live in your service
Mono<String> mono = webClient.get()
.uri("http://someUrl.com")
.retrieve()
.bodyToMono(POJO.class)
.onErrorResume(ex -> {
if (ex instanceof WebFaultException)
return Mono.error(ex);
return Mono.error(new WebFaultException(ex.getMessage(), "Error on API Call", HttpStatus.INTERNAL_SERVER_ERROR));
});
StepVerifier.create(mono)
.expectError(RuntimeException.class)
.verify();
}

CompletableFuture - skip errors and take only validate data

I have a specific code which is working fine if webclient response is OK. If any error, then the get() method throwing error and the thread blocked forever.
#SneakyThrows
public List<ResponseData> validateExpression(List<RequestData> RequestDataList, Data data) {
System.out.println(Instant.now());
final List<Mono<ResponseData>> monoList = new ArrayList<>();
RequestDataList.parallelStream().forEach(requestData -> {
try {
ObjectMapper mapper = new ObjectMapper();
log.info("Diversity API request data:");
log.info(mapper.writeValueAsString(requestData));
Mono<ResponseData> monoResponse = webClient
.post()
.uri("http://...")
.contentType(MediaType.APPLICATION_JSON)
.header(API_KEY_HEADER, config.getApiKey())
.body(Mono.just(requestData), RequestData.class)
.retrieve()
.bodyToMono(ResponseData.class);
System.out.println("create mono response lazy initialization");
monoList.add(monoResponse);
} catch (Exception e) {
log.info(e.getMessage());
}
});
System.out.println(Instant.now());
CompletableFuture<List<ResponseData>> futureCount = new CompletableFuture<>();
List<ResponseData> responseDataList = new ArrayList<>();
Mono.zip(monoList, Arrays::asList)
.flatMapIterable(objects -> objects)
.doOnComplete(() -> {
futureCount.complete(responseDataList);
}).subscribe(responseData -> {
responseDataList.add((ResponseData) responseData);
});
return futureCount.get();
}
It is working fine with successful case. If there is any error from the webclient it is throwing error and thread blocked forever.
How to skip the errors and get only validate response data ?
How to avoid deadLock on this case?

You should look at CompletableFuture::get method's doc:
It throws three different checked exceptions. Lombok's #SneakyThrows annotation hides them so they aren't managed by your method. You should probably add a try/catch block to manage these exceptions and skip the errors if you want so.

Spring 5 Webclient Handle 500 error and modify the response

I have a Spring app acting as a passthrough from one app to another, making a http request and returning a result to the caller using WebClient. If the client http call returns a 500 internal server error I want to be able to catch this error and update the object that gets returned rather than re-throwing the error or blowing up the app.
This is my Client:
final TcpClient tcpClient = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeout)
.doOnConnected(connection -> connection.addHandlerLast(new ReadTimeoutHandler(readTimeout))
.addHandlerLast(new WriteTimeoutHandler(writeTimeout)));
this.webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.filter(logRequest())
.filter(logResponse())
.filter(errorHandler())
.build();
And this is my error handler. I've commented where I want to modify the result, where ResponseDto is the custom object that is returned from the client call happy path
public static ExchangeFilterFunction errorHandler(){
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
ResponseDto resp = new ResponseDto();
if(nonNull(clientResponse.statusCode()) && clientResponse.statusCode().is5xxServerError()){
//this is where I want to modify the response
resp.setError("This is the error");
}
//not necessarily the correct return type here
return Mono.just(clientResponse);
});
}
How can I achieve this? I can't find any tutorials or any information in the docs to help explain it.
Disclaimer, I'm new to webflux. We're only starting to look at reactive programming

Non-blocking servlets

So,
Lately I started to play with non-blocking, async servlets (Servlet 3.1). As far as I understood,
by using async servlets there is a possibility to release a working thread while a non-blocking
operation is being done, so basically none of the threads is blocked. So i started doing some
load tests using Gatling stress tol, I used Spring RestController to create two simple controllers -
one doing a blocking HTTP get using RestTemplate (Spring's HTTP client), and one doing non-blocking
requests using AsyncRestTemplate and after processing a message with a callback, so those 2 follow:
#RequestMapping("/non-blocking")
public DeferredResult<String> nonBlocking() throws IOException {
DeferredResult<String> deferredResult = new DeferredResult<String>();
AsyncRestTemplate restTemplate = new AsyncRestTemplate();
String URL ="http://localhost:8080/static.html";
HttpMethod method = HttpMethod.GET;
Class<String> responseType = String.class;
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.TEXT_PLAIN);
HttpEntity<String> requestEntity = new HttpEntity<String>("params", headers);
ListenableFuture<ResponseEntity<String>> future = restTemplate.exchange(URL, method, requestEntity, responseType);
future.addCallback(new ListenableFutureCallback<ResponseEntity<String>>() {
#Override
public void onSuccess(ResponseEntity<String> result) {
deferredResult.setResult(result.getBody());
}
#Override
public void onFailure(Throwable ex) {
System.out.println("Failure while processing");
}
});
return deferredResult;
}
#RequestMapping("/blocking")
public String blockingRouter() {
HttpStatus status = HttpStatus.INTERNAL_SERVER_ERROR;
LOG.logStartBlocking();
try {
ResponseEntity<String> result = restTemplate.getForEntity("http://localhost:8080/static.html");
status = result.getStatusCode();
return result.getBody();
} finally {
System.out.println("Processing done");
}
}
So, as you can see non-blocking is returning DefferedResult (Spring's Future abstraction) and if I
get it right, after sending a HTTP request, thread should be freed and some (probably) other thread
would return a response when it's available. Well, after doing some load tests with 200 concurrent users, test went terribly wrong for a non-blocking version. Threads were fine, but upon reaching 400 req/s, controller just stopped returning responses - it got stuck, probably because of too high response time. Blocking one responded to all of the requests with cca 1000 req/s. So the question is did I do a good testing or missed something because results were totally unexpected for me. Server used was Tomcat (+NIO connector) with 50 threads - hoped to prove that non-blocking won't use many threads and reach it till the end and that blocking will fail due to thread exhaustion but it didn't happen. Any suggestions?
Thanks in advance.