Currently I’m having an issue with new Spring 5 WebClient and I need some help to sort it out.
The issue is:
I request some url that returns json response with content type text/html;charset=utf-8.
But unfortunately I’m still getting an exception:
org.springframework.web.reactive.function.UnsupportedMediaTypeException:
Content type 'text/html;charset=utf-8' not supported. So I can’t
convert response to DTO.
For request I use following code:
Flux<SomeDTO> response = WebClient.create("https://someUrl")
.get()
.uri("/someUri").accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToFlux(SomeDTO.class);
response.subscribe(System.out::println);
Btw, it really doesn’t matter which type I point in accept header, always returning text/html. So how could I get my response converted eventually?
As mentioned in previous answer, you can use exchangeStrategies method,
example:
Flux<SomeDTO> response = WebClient.builder()
.baseUrl(url)
.exchangeStrategies(ExchangeStrategies.builder().codecs(this::acceptedCodecs).build())
.build()
.get()
.uri(builder.toUriString(), 1L)
.retrieve()
.bodyToFlux( // .. business logic
private void acceptedCodecs(ClientCodecConfigurer clientCodecConfigurer) {
clientCodecConfigurer.customCodecs().encoder(new Jackson2JsonEncoder(new ObjectMapper(), TEXT_HTML));
clientCodecConfigurer.customCodecs().decoder(new Jackson2JsonDecoder(new ObjectMapper(), TEXT_HTML));
}
If you need to set the maxInMemorySize along with text/html response use:
WebClient invoicesWebClient() {
return WebClient.builder()
.exchangeStrategies(ExchangeStrategies.builder().codecs(this::acceptedCodecs).build())
.build();
}
private void acceptedCodecs(ClientCodecConfigurer clientCodecConfigurer) {
clientCodecConfigurer.defaultCodecs().maxInMemorySize(BUFFER_SIZE_16MB);
clientCodecConfigurer.customCodecs().registerWithDefaultConfig(new Jackson2JsonDecoder(new ObjectMapper(), TEXT_HTML));
clientCodecConfigurer.customCodecs().registerWithDefaultConfig(new Jackson2JsonEncoder(new ObjectMapper(), TEXT_HTML));
}
Having a service send JSON with a "text/html" Content-Type is rather unusual.
There are two ways to deal with this:
configure the Jackson decoder to decode "text/html" content as well; look into the WebClient.builder().exchangeStrategies(ExchangeStrategies) setup method
change the "Content-Type" response header on the fly
Here's a proposal for the second solution:
WebClient client = WebClient.builder().filter((request, next) -> next.exchange(request)
.map(response -> {
MyClientHttpResponseDecorator decorated = new
MyClientHttpResponseDecorator(response);
return decorated;
})).build();
class MyClientHttpResponseDecorator extends ClientHttpResponseDecorator {
private final HttpHeaders httpHeaders;
public MyClientHttpResponseDecorator(ClientHttpResponse delegate) {
super(delegate);
this.httpHeaders = new HttpHeaders(this.getDelegate().getHeaders());
// mutate the content-type header when necessary
}
#Override
public HttpHeaders getHeaders() {
return this.httpHeaders;
}
}
Note that you should only use that client in that context (for this host).
I'd strongly suggest to try and fix that strange content-type returned by the server, if you can.
Related
I have a webservice endpoint that should just proxy the received payload from another internal endpoint.
My goal is to neither having to read input body I receive, nor the response the I want to return. I just want to proxy it.
The following works, but it's probably suboptimal converting the response to a Mono<String>. But how could I do better?
#RestController
public class ProxyController {
#PostMapping("/proxy")
public Mono<Object> proxy(InputStream payload) {
return webClient.post().uri(url).bodyValue(payload).retrieve().bodyToMono(String.class);
}
}
This is what I used to do using rest template
#RequestMapping("/pass-to-service/**")
fun passThroughPostRequest(request: HttpServletRequest, #RequestBody body: Any?): ResponseEntity<String> {
val method = HttpMethod.resolve(request.method)!!
val requestEntity = RequestEntity(body, method, URI.create(myServiceUrl))
val responseEntity = restTemplate.exchange(requestEntity, String::class.java)
// response entity might have crazy headers, so add some decent/needed and ship back
val httpHeaders = HttpHeaders()
httpHeaders.contentType = MediaType.APPLICATION_JSON
return ResponseEntity(responseEntity.body, httpHeaders, responseEntity.statusCode)
}
In above example, I avoided as much of serialisation and deserialisation that I could. Kept it a passthrough from servlet.
Similarly, I am trying something like this using webclient:
#PostMapping("/v1/cars/{carId}/details")
fun ingestCarInfo(
#PathVariable("carId") carId: UUID,
request: HttpServletRequest, response: HttpServletResponse, #RequestBody body: Mono<CarDetailsReqDto>
) {
/** Step 1: I wanted to do some activity here */
/** Step 2: return a success response immediately, as my client do not care about the data processed.
* What I am unsure here? Does this return immediately without getting to next step
* */
response.setStatus(HttpStatus.OK.value())
/** Step 3: Fire & Forget request */
val uri = UriComponentsBuilder
.fromUriString("http://localhost:8080")
.path("/v3/cars/{carId}/details")
.build().encode().toUri()
webClient.method(HttpMethod.POST).uri(uri)
.body(BodyInserters.fromValue(body))
.header("OnewayRequest", "true")
.retrieve()
.toBodilessEntity()
.block()
}
Here, I haven't made everything generic, my body still has a shape defined but reactive. If it is a wildcard pass-through I would type it Mono<Any?>
NOTE: Still I am working on this. Will update once I find a better solution, also I need to check the performance and speed in realtime.
How can I intercept WebFilter requests after they have been converted from DTO to JSON String?
Of course I could add a ExchangeFilterFunction, but the clientRequest.body() only shows my input object, not the converted json string:
WebClient.builder().defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE).filter(logRequest()).build();
private ExchangeFilterFunction logRequest() {
return ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
LOGGER.info(clientRequest.body()); //body is MyRequest.class
return Mono.just(clientRequest);
});
}
MyRequestDto dto;
client.post().uri(url).syncBody(dto).retrieve().bodyToMono(MyResponseDto.class).block();
So I'm looking for a way to intercept the outgoing requests after they have been converted from dto to json string.
And the other way around, intercept responses before they are converted to dto from json.
Data is written as a String but rather as a bytes in DataBuffer by JacksonEncoder. One way to intercept request is to override JSON encoder, and access DataBuffer directly or interpret it as a String.
Consumer<ClientCodecConfigurer> consumer = configurer ->
configurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder() {
#Override
public Flux<DataBuffer> encode(Publisher<?> inputStream, DataBufferFactory bufferFactory, ResolvableType elementType, #Nullable MimeType mimeType, #Nullable Map<String, Object> hints) {
return super.encode(inputStream, bufferFactory, elementType, mimeType, hints)
.doOnNext(dataBuffer -> System.out.println(StandardCharsets.UTF_8.decode(dataBuffer.asByteBuffer()).toString()));
}
});
WebClient webClient = WebClient.builder()
.exchangeStrategies(ExchangeStrategies.builder().codecs(consumer).build())
.build();
If the purpose is just to log outgoing message body, see this question: how to log Spring 5 WebClient call
WebClient.builder().baseUrl("/").filter(contentTypeInterceptor()).build();
How can I modify the Content-Type of the received response (because I'm receiving a response from a webserver that emits the wrong content type. As I'm not in control of the external server, I'd like to correct the content type for further correct processing (eg with jackson library etc).
private ExchangeFilterFunction contentTypeInterceptor() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
org.springframework.web.reactive.function.client.ClientResponse.Headers headers = clientResponse.headers();
//TODO how to headers.setContentType("myval) or headers.set("Content-Type", "myval");
//headers.asHttpHeaders(); cannot be used as it is readonly
});
}
The question could be answered in general how to override any http header.
The root cause in my case is that I receive text/html, but the response body is actually a application/xml. And jackson rejects parsing that response due to:
org.springframework.web.reactive.function.UnsupportedMediaTypeException: Content type 'text/html' not supported for bodyType=MyResponse
I had similar issue and the accepted answer didn't work with me.
I done this instead, in order to override an invalid content-type that i was receiving.
/**
* webclient interceptor that overrides the response headers ...
* */
private ExchangeFilterFunction contentTypeInterceptor() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse ->
Mono.just(
ClientResponse
.from(clientResponse) //clientResponse is immutable, so,we create a clone. but from() only clones headers and status code
.headers(headers -> headers.remove(HttpHeaders.CONTENT_TYPE)) //override the content type
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE)
.body(clientResponse.body(BodyExtractors.toDataBuffers()) ) // copy the body as bytes with no processing
.build()));
}
Ahmed's response is technically correct. However, I believe that at the time of my posting this, that ClientResponse.from() is deprecated, and you should use the .mutate() method to create a new Builder.
private ExchangeFilterFunction contentTypeInterceptor() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse ->
Mono.just(clientResponse.mutate()
.headers(headers -> headers.remove(HttpHeaders.CONTENT_TYPE))
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_XML_VALUE)
.build()));
}
maybe something like this?
private ExchangeFilterFunction contentTypeInterceptor() {
return ExchangeFilterFunction.ofRequestProcessor(clientRequest ->
Mono.just(ClientRequest.from(clientRequest)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_XML_VALUE)
.build()));
}
I have a Spring app acting as a passthrough from one app to another, making a http request and returning a result to the caller using WebClient. If the client http call returns a 500 internal server error I want to be able to catch this error and update the object that gets returned rather than re-throwing the error or blowing up the app.
This is my Client:
final TcpClient tcpClient = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeout)
.doOnConnected(connection -> connection.addHandlerLast(new ReadTimeoutHandler(readTimeout))
.addHandlerLast(new WriteTimeoutHandler(writeTimeout)));
this.webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.filter(logRequest())
.filter(logResponse())
.filter(errorHandler())
.build();
And this is my error handler. I've commented where I want to modify the result, where ResponseDto is the custom object that is returned from the client call happy path
public static ExchangeFilterFunction errorHandler(){
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
ResponseDto resp = new ResponseDto();
if(nonNull(clientResponse.statusCode()) && clientResponse.statusCode().is5xxServerError()){
//this is where I want to modify the response
resp.setError("This is the error");
}
//not necessarily the correct return type here
return Mono.just(clientResponse);
});
}
How can I achieve this? I can't find any tutorials or any information in the docs to help explain it.
Disclaimer, I'm new to webflux. We're only starting to look at reactive programming
I am working on part of an API, which requires making a call to another external API to retrieve data for one of its functions. The call was returning an HTTP 500 error, with description "Content type 'application/octet-stream' not supported." The call is expected to return a type of 'application/json."
I found that this is because the response received doesn't explicitly specify a content type in its header, even though its content is formatted as JSON, so my API defaulted to assuming it was an octet stream.
The problem is, I'm not sure how to adjust for this. How would I get my API to treat the data it receives from the other API as an application/json even if the other API doesn't specify a content type? Changing the other API to include a contenttype attribute in its response is infeasible.
Code:
The API class:
#RestController
#RequestMapping(path={Constants.API_DISPATCH_PROFILE_CONTEXT_PATH},produces = {MediaType.APPLICATION_JSON_VALUE})
public class GetProfileApi {
#Autowired
private GetProfile GetProfile;
#GetMapping(path = {"/{id}"})
public Mono<GetProfileResponse> getProfile(#Valid #PathVariable String id){
return GetProfile.getDispatchProfile(id);
}
The service calling the external API:
#Autowired
private RestClient restClient;
#Value("${dispatch.api.get_profile}")
private String getDispatchProfileUrl;
#Override
public Mono<GetProfileResponse> getDispatchProfile(String id) {
return Mono.just(id)
.flatMap(aLong -> {
MultiValueMap<String, String> headers = new HttpHeaders();
headers.add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
return restClient.get(getDispatchProfileUrl, headers);
}).flatMap(clientResponse -> {
HttpStatus status = clientResponse.statusCode();
log.info("HTTP Status : {}", status.value());
return clientResponse.bodyToMono(GetProfileClientResponse.class);
// the code does not get past the above line before returning the error
}).map(GetProfileClientResponse -> {
log.debug("Response : {}",GetProfileClientResponse);
String id = GetProfileClientResponse.getId();
log.info("SubscriberResponse Code : {}",id);
return GetProfileResponse.builder()
// builder call to be completed later
.build();
});
}
The GET method for the RestClient:
public <T> Mono<ClientResponse> get(String baseURL, MultiValueMap<String,String> headers){
log.info("Executing REST GET method for URL : {}",baseURL);
WebClient client = WebClient.builder()
.baseUrl(baseURL)
.defaultHeaders(httpHeaders -> httpHeaders.addAll(headers))
.build();
return client.get()
.exchange();
}
One solution I had attempted was setting produces= {MediaType.APPLICATION_JSON_VALUE} in the #RequestMapping of the API to produces= {MediaType.APPLICATION_OCTET_STREAM_VALUE}, but this caused a different error, HTTP 406 Not Acceptable. I found that the server could not give the client the data in a representation that was requested, but I could not figure out how to correct it.
How would I be able to treat the response as JSON successfully even though it does not come with a content type?
Hopefully I have framed my question well enough, I've kinda been thrust into this and I'm still trying to figure out what's going on.
Are u using jackson library or jaxb library for marshalling/unmarshalling?
Try annotating Mono entity class with #XmlRootElement and see what happens.