We Keep Coding

Spring WebClient - How to handle error scenarios

We're using org.springframework.web.reactive.function.client.WebClient with
reactor.netty.http.client.HttpClient as part of Spring 5.1.9 to make requests using the exchange() method. The documentation for this method highlights the following:
... when using exchange(), it is the responsibility of the application
to consume any response content regardless of the scenario (success,
error, unexpected data, etc). Not doing so can cause a memory leak.
Our use of exchange() is rather basic, but the documentation for error scenarios is unclear to me and I want to be certain that we are correctly releasing resources for all outcomes. In essence, we have a blocking implementation which makes a request and returns the ResponseEntity regardless of the response code:
try {
...
ClientResponse resp = client.method(method).uri(uri).syncBody(body).exchange().block();
ResponseEntity<String> entity = resp.toEntity(String.class).block();
return entity;
} catch (Exception e) {
// log error details, return internal server error
}
If I understand the implementation, exchange() will always give us a response if the request was successfully dispatched, regardless of response code (e.g. 4xx, 5xx). In that scenario, we just need to invoke toEntity() to consume the response. My concern is for error scenarios (e.g. no response, low-level connection errors, etc). Will the above exception handling catch all other scenarios and will any of them have a response that needs to be consumed?
Note: ClientResponse.releaseBody() was only introduced in 5.2

The response have to be consumed when the request was made, but if you can't do the request probably an exception was be throwed before, and you will no have problems with response.
In the documentation says:
NOTE: When using a ClientResponse through the WebClient exchange() method, you have to make sure that the body is consumed or released by using one of the following methods:
body(BodyExtractor)
bodyToMono(Class) or bodyToMono(ParameterizedTypeReference)
bodyToFlux(Class) or bodyToFlux(ParameterizedTypeReference)
toEntity(Class) or toEntity(ParameterizedTypeReference)
toEntityList(Class) or toEntityList(ParameterizedTypeReference)
toBodilessEntity()
releaseBody()
You can also use bodyToMono(Void.class) if no response content is expected. However keep in mind the connection will be closed, instead of being placed back in the pool, if any content does arrive. This is in contrast to releaseBody() which does consume the full body and releases any content received.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/reactive/function/client/ClientResponse.html
You can try to use .retrieve() instead .exchange() and handle errors as your preference.
public Mono<String> someMethod() {
return webClient.method(method)
.uri(uri)
.retrieve()
.onStatus(
(HttpStatus::isError), // or the code that you want
(it -> handleError(it.statusCode().getReasonPhrase())) //handling error request
)
.bodyToMono(String.class);
}
private Mono<? extends Throwable> handleError(String message) {
log.error(message);
return Mono.error(Exception::new);
}
In this example I used Exception but you can create some exception more specific and then use some exception handler to return the http status that you want.
Is not recommended to use block, a better way is pass the stream forward.

create some exception classes
Autowired ObjectMapper
Create a method that returns Throwable
Create a custom class for Error.
return webClient
.get()
.uri(endpoint)
.retrieve()
.bodyToMono(Model.class)
.onErrorMap(WebClientException.class, this::handleHttpClientException);
private Throwable handleHttpClientException(Throwable ex) {
if (!(ex instanceof WebClientResponseException)) {
LOG.warn("Got an unexpected error: {}, will rethrow it", ex.toString());
return ex;
}
WebClientResponseException wcre = (WebClientResponseException)ex;
switch (wcre.getStatusCode()) {
case NOT_FOUND -> throw new NotFoundException(getErrorMessage(wcre));
case BAD_REQUEST -> throw new BadRequestException(getErrorMessage(wcre));
default -> {
LOG.warn("Got a unexpected HTTP error: {}, will rethrow it", wcre.getStatusCode());
LOG.warn("Error body: {}", wcre.getResponseBodyAsString());
return ex;
}
}
}
private String getErrorMessage(WebClientResponseException ex) {
try {
return mapper.readValue(ex.getResponseBodyAsString(), HttpErrorInfo.class).getMessage();
} catch (IOException ioe) {
return ex.getMessage();
}
}

Cannot find a handler for POST with boundary

I'm in the midst of testing my application which is using an HTTP-server. Instead of mocking I decided to go with a HTTP server fixture. Meaning that I do not have to mock any productional code. To accomplish this goal I currently chose for a free to use 3rd party library fixd.
I was able to successfully create several unit tests - which are working by means of a GET request. Most are quite simple, i.e.:
#Test
public void verifyConnectionTest()
{
try
{
final String body = FileUtils.readFileToString(RESOURCE);
final String path = "/";
this.server.handle(Method.GET, path).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
But I now have to send a POST request with multipart/form-data. Which does not make much of a difference other than changing the method and content-type:
#Test
public void executeStepTest()
{
try
{
final String body = FileUtils.readFileToString(SERVICE_RESPONSE);
final String path = "/";
this.server.handle(Method.POST, path, "multipart/form-data").with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
However I get the following error: [ERROR] could not find a handler for POST - / - multipart/form-data; boundary=bqCBI7t-VW1xaJW7BADmTiGMg9w_YM2sHH8ukJYx and my guess is that fixd doesn't recognize the boundary-party. Since the documentation does not show an example I'm quite stuck on this part.
I tried using some wildcards such as '*', no succes. Thus; I need a way to either tell fixd to accept that boundary or use some wildcards I didn't yet discover. Any help would be great, thanks!

I've been making some debug and it seems to be that the problem is in the fixd core.
Basically, fixd indexes every RequestHandlerImpl by a HandlerKey (which includes ContentType as part of the key) in the map handlerMap. See method org.bigtesting.fixd.core.FixtureContainer#resolve.
...
HandlerKey key = new HandlerKey(method, route, contentType);
RequestHandlerImpl handler = handlerMap.get(key);
if (handler == null) {
// Error
}
...
Problem: When the request is multipart/form-data, boundary data (which it's generated dinamically every request) is part of the content type. So, any handler is found in handlerMap because the key changes with every running.
I've made a little test only to check that this is the cause of the problem, passing the contentType to fixd server.handle after the creation of the multipart request, and it works fine.
See the test below:
#Test
public void verifyConnectionTest_multipart() {
try {
// 1. Create multipart request (example with http-commons 3.1)
PostMethod filePost = new PostMethod(url);
Part[] parts = { new StringPart("param", "value") };
MultipartRequestEntity request = new MultipartRequestEntity(parts, filePost.getParams());
filePost.setRequestEntity(request);
// 2. fixd server handle (passing the request content type)
this.server.handle(Method.POST, "/", request.getContentType()).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response) {
response.setStatusCode(200);
response.setContentType("text/xml");
}
});
// 3. Execute multipart request
HttpClient client = new HttpClient();
int status = client.executeMethod(filePost);
// 4. Assertions
Assert.assertEquals(200, status);
} catch (Exception e) {
Assert.fail(e.getMessage());
}
}
Hope it helps you to clarify the problem. Cheers

This was a bug in fixd, and has been fixed in version 1.0.3. Your original code should work using this new version of fixd.

Architecture for http client

I am developing application, with http client, and I wonder to make some elegant issue.
This is standard java http client whose work in background thread, and passing data by event's (witch realized by override methods). I have special class for background requests, that implements method sendRequest()
protected void sendRequest(final String url) {
Thread t = new Thread(new Runnable() {
#Override
public void run() {
URI website = null;
try {
website = new URI(url);
} catch (URISyntaxException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
HttpGet request = new HttpGet();
request.setURI(website);
HttpResponse response = null;
try {
response = client.execute(request, httpContext);
} catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
if (response != null)
{
HttpEntity entity = response.getEntity();
try {
InputStream is = entity.getContent();
if (Debug.isDebuggerConnected()==true)
{
String data = convertStreamToString(is);
int code = response.getStatusLine().getStatusCode();
if (httpEvent!=null)
httpEvent.HttpResponseArrived(data, code);
}
else
httpEvent.HttpResponseArrived(convertStreamToString(is),response.getStatusLine().getStatusCode());
}
catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
}
}
});
t.start();
}
And also child class, for API to web server, wich have methods like that:
public void getSomeData(SomeParams param)
{
sendRequest("http://xxx.yy"+gson.toJson(param));
httpEvent = new HttpHandler()
{
#Override
public void HttpResponseArrived(String data, int code)
{
switch (code)
{
case 200:
//some code
break;
case 401:
//some code
break;
}
}
};
}
And my question: how elegant to handle server errors, for example 401? I need to do this in one place, in method that sending requests - sendRequest(). At first sight it is very easy: just handle 401, and if it's because expired cookie - call method Login() (in my design, it's look like getSomeData). But I want, not just login again, I need to request data, that I failed to get because the error. Of course, I can implement calling Login() method in every switch, like this:
case 401:
{
Login(CurrentContext.AuthData.Login, CurrentContext.AuthData.Password);
break;
}
But the login event implemented in Login() method;
Also, I can just write sendRequest(string authdata), subscrube for HttpHandler and by recursion call method thats implements this code. But I thind, it's not very good decision.
I really hope, that somebody already solve this problem, and there is the way, to turn it's in beautiful code!
Thanks, if you could to read this to the end:)

As for answer not comment.
Try to use http client framework - I prefer Apache HTTPClient. It provides wide controll over request and responses. Moreover it supports most common methods like GET POST etc. Cookie management, redirection handling and SSL support is also provided. Don't invent something that is already invented.
HttpClient - use v4.x

java.io.IOException: Attempted read from closed stream

I am making a HTTPPost call using Apache HTTP Client and then I am trying to create an object from the response using Jackson.
Here is my code:
private static final Logger log = Logger.getLogger(ReportingAPICall.class);
ObjectMapper mapper = new ObjectMapper();
public void makePublisherApiCall(String jsonRequest)
{
String url = ReaderUtility.readPropertyFile().getProperty("hosturl");
DefaultHttpClient client = new DefaultHttpClient();
try {
HttpPost postRequest = new HttpPost(url);
StringEntity entity = new StringEntity(jsonRequest);
postRequest.addHeader("content-type", "application/json");
log.info("pub id :"+ExcelReader.publisherId);
postRequest.addHeader("accountId", ExcelReader.publisherId);
postRequest.setEntity(entity);
HttpResponse postResponse = client.execute(postRequest);
log.info(EntityUtils.toString(postResponse.getEntity()));
// Response<PublisherReportResponse> response = mapper.readValue(postResponse.getEntity().getContent(), Response.class);
// log.info("Reponse "+response.toString());
} catch (UnsupportedEncodingException ex) {
log.error(ex.getMessage());
log.error(ex);
Assert.assertTrue(false, "Exception : UnsupportedEncodingException");
} catch (ClientProtocolException ex) {
log.error(ex.getMessage());
log.error(ex);
Assert.assertTrue(false, "Exception : ClientProtocolException");
} catch (IOException ex) {
log.error(ex.getMessage());
log.error(ex);
Assert.assertTrue(false, "Exception : IOException");
}
Method makePublisherApiCall() will be called in a loop which runs for say 100 times.
Basically problem occurs when I uncomment the line:
// Response<PublisherReportResponse> response = mapper.readValue(postResponse.getEntity().getContent(), Response.class);
// log.info("Reponse "+response.toString());
After uncommenting I am getting exception:
Attempted read from closed stream.
17:26:59,384 ERROR com.inmobi.reporting.automation.reportingmanager.ReportingAPICall - java.io.IOException: Attempted read from closed stream.
Otherwise it works fine.
Could someone please let me know what I am doing wrong.

What does EntityUtils.toString(postResponse.getEntity()) do with the response entity? I would suspect, that it is consuming the entity's content stream. The HttpClient javadoc states, that only entities, which are repeatable can be consumed more than once. Therefore if the entity is not repeatable you cannot feed the content stream to the mapper again. To avoid this you should only let the mapper consume the stream - if logging of content is required, log the parsed Response object.

I had the same problem. Make sure you aren't consuming the entity's content stream in the "watch" or "inspect" section of your IDE. It's closed after it's consumed (read).
And sorry for my english.

I found an answer for similar issue with Spring RestTemplate here : https://www.baeldung.com/spring-rest-template-interceptor
if we want our interceptor to function as a request/response logger, then we need to read it twice – the first time by the interceptor and the second time by the client.
The default implementation allows us to read the response stream only once. To cater such specific scenarios, Spring provides a special class called BufferingClientHttpRequestFactory. As the name suggests, this class will buffer the request/response in JVM memory for multiple usage.
Here's how the RestTemplate object is initialized using BufferingClientHttpRequestFactory to enable the request/response stream caching:
RestTemplate restTemplate = new RestTemplate( new BufferingClientHttpRequestFactory( new SimpleClientHttpRequestFactory() ) );

I had the same problem.
The idea is that if you consume the postResponse, then you should put it in a variable in order to use it again in different places.
Else, the connection is closed and you can no longer consume the same response again.
I used to log it (for debug purposes) and always fails.

In my case this issue was related to another reason,
I am getting this issue because I haven't use
closeableresponce=client.Getmethod(FinalUrl);
In my first Test1, it was mention but when I missed out in Test2, I forget to put this code that's why stream closed message shows...
public void getapiwithheader() throws ParseException, IOException
{
client = new RestClient();
closeableresponce=client.Getmethod(FinalUrl);
HashMap<String, String> headermap = new HashMap<>();
headermap.put("content-type", "application/json");
//****************************************************************************
//Status code
//****************************************************************************
int statuscode =closeableresponce.getStatusLine().getStatusCode();
System.out.println("Status code "+statuscode);
Assert.assertEquals(statuscode,RESPONSE_STATUS_CODE_200, "Status code is not 200");
//****************************************************************************
// Json String
//****************************************************************************
String responsestring= EntityUtils.toString(closeableresponce.getEntity(),"UTF-8");
JSONObject respjsonobj = new JSONObject(responsestring);
System.out.println("Respose Json API"+respjsonobj);
//****************************************************************************
// Verify the value of the JSON
//****************************************************************************
String Email =TestUtil.getValueByJPath(respjsonobj,"/email");
System.out.println("******************************************");
System.out.println("Print the value of Email "+Email);
Assert.assertEquals(Email, "johndoe#google.com");
}

I had the same problem.
In my case, I need to get the response content via AOP/

How to mock Jersey REST client to throw HTTP 500 responses?

I am writing a Java class that uses Jersey under the hood to send an HTTP request to a RESTful API (3rd party).
I would also like to write a JUnit test that mocks the API sending back HTTP 500 responses. Being new to Jersey, it is tough for me to see what I have to do to mock these HTTP 500 responses.
So far here is my best attempt:
// The main class-under-test
public class MyJerseyAdaptor {
public void send() {
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
String uri = UriBuilder.fromUri("http://example.com/whatever").build();
WebResource service = client.resource(uri);
// I *believe* this is where Jersey actually makes the API call...
service.path("rest").path("somePath")
.accept(MediaType.TEXT_HTML).get(String.class);
}
}
#Test
public void sendThrowsOnHttp500() {
// GIVEN
MyJerseyAdaptor adaptor = new MyJerseyAdaptor();
// WHEN
try {
adaptor.send();
// THEN - we should never get here since we have mocked the server to
// return an HTTP 500
org.junit.Assert.fail();
}
catch(RuntimeException rte) {
;
}
}
I am familiar with Mockito but have no preference in mocking library. Basically if someone could just tell me which classes/methods need to be mocked to throw a HTTP 500 response I can figure out how to actually implement the mocks.

Try this:
WebResource service = client.resource(uri);
WebResource serviceSpy = Mockito.spy(service);
Mockito.doThrow(new RuntimeException("500!")).when(serviceSpy).get(Mockito.any(String.class));
serviceSpy.path("rest").path("somePath")
.accept(MediaType.TEXT_HTML).get(String.class);
I don't know jersey, but from my understanding, I think the actual call is done when get() method is invoked.
So you can just use a real WebResource object and replace the behavior of the get(String) method to throw the exception instead of actually execute the http call.

I'm writing a Jersey web application... and we throw WebApplicationException for HTTP error responses. You can simply pass the response code as the constructor-parameter. For example,
throw new WebApplicationException(500);
When this exception is thrown server-side, it shows up in my browser as a 500 HTTP response.
Not sure if this is what you want... but I thought the input might help! Best of luck.

I was able to simulate a 500 response with the following code:
#RunWith(MockitoJUnitRunner.class)
public class JerseyTest {
#Mock
private Client client;
#Mock
private WebResource resource;
#Mock
private WebResource.Builder resourceBuilder;
#InjectMocks
private Service service;
#Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
#Test
public void jerseyWith500() throws Exception {
// Mock the client to return expected resource
when(client.resource(anyString())).thenReturn(resource);
// Mock the builder
when(resource.accept(MediaType.APPLICATION_JSON)).thenReturn(resourceBuilder);
// Mock the response object to throw an error that simulates a 500 response
ClientResponse c = new ClientResponse(500, null, null, null);
// The buffered response needs to be false or else we get an NPE
// when it tries to read the null entity above.
UniformInterfaceException uie = new UniformInterfaceException(c, false);
when(resourceBuilder.get(String.class)).thenThrow(uie);
try {
service.get("/my/test/path");
} catch (Exception e) {
// Your assert logic for what should happen here.
}
}
}