I'm trying to write an interceptor that compresses a request body using Gzip.
My server does not support compressed requests, so I'll be using an application/octet-stream instead of Content-Type: gzip and compress the request body manually, it will be decompressed manually at backend.
public class GzipRequestInterceptor implements Interceptor {
final String CONTENT_TYPE = "application/octet-stream";
#Override
public Response intercept(Chain chain) throws IOException {
Request originalRequest = chain.request();
if (originalRequest.body() == null || CONTENT_TYPE.equals(originalRequest.header("Content-Type"))) {
return chain.proceed(originalRequest);
}
Request compressedRequest = originalRequest.newBuilder()
.header("Content-Type", CONTENT_TYPE)
.method(originalRequest.method(), gzip(originalRequest.body()))
.build();
return chain.proceed(compressedRequest);
}
private RequestBody gzip(final RequestBody body) throws IOException {
final Buffer inputBuffer = new Buffer();
body.writeTo(inputBuffer);
final Buffer outputBuffer = new Buffer();
GZIPOutputStream gos = new GZIPOutputStream(outputBuffer.outputStream());
gos.write(inputBuffer.readByteArray());
inputBuffer.close();
gos.close();
return new RequestBody() {
#Override
public MediaType contentType() {
return body.contentType();
}
#Override
public long contentLength() {
return outputBuffer.size();
}
#Override
public void writeTo(BufferedSink sink) throws IOException {
ByteString snapshot = outputBuffer.snapshot();
sink.write(snapshot);
}
};
}
}
It doesn't work - 30 seconds after request is fired, a 500 Server Error is received. On the server there's a timeout exception.
My guess is that I've done wrong with input/output on the gzip method... any ideas?
Update if I stop the app, the request goes through successfully, does this indicate that the app is still waiting for data from outputBuffer?
Is this what you are looking for Interceptors, check the Rewriting Requests -chapter.
Related
I am using feign client to connect to downstream service.
I got a requirement that when one of the downstream service endpoint returns 400 ( it's partial success scenario ) our service need this to be converted to 200 success with the response value.
I am looking for a best way of doing this.
We are using error decoder to handle the errors and the above conversion is applicable for only one endpoint not for all the downstream endpoints and noticed that decode() method should returns exception back.
You will need to create a customized Client to intercept the Response early enough to change the response status and not invoke the ErrorDecoder. The simplest approach is to create a wrapper on an existing client and create a new Response with a 200 status. Here is an example when using Feign's ApacheHttpClient:
public class ClientWrapper extends ApacheHttpClient {
private ApacheHttpClient delegate;
public ClientWrapper(ApacheHttpClient client) {
this.client = client;
}
#Override
public Response execute(Request request, Request.Options options) throws IOException {
/* execute the request on the delegate */
Response response = this.client.execute(request, options);
/* check the response code and change */
if (response.status() == 400) {
response = Response.builder(response).status(200).build();
}
return response;
}
}
This customized client can be used on any Feign client you need.
Another way of doing is by throwing custom exception at error decoder and convert this custom exception to success at spring global exception handler (using #RestControllerAdvice )
public class CustomErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
if (response.status() == 400 && response.request().url().contains("/wanttocovert400to200/clientendpoints") {
ResponseData responseData;
ObjectMapper mapper = new ObjectMapper();
try {
responseData = mapper.readValue(response.body().asInputStream(), ResponseData.class);
} catch (Exception e) {
responseData = new ResponseData();
}
return new PartialSuccessException(responseData);
}
return FeignException.errorStatus(methodKey, response);
}}
And the Exception handler as below
#RestControllerAdvice
public class GlobalControllerExceptionHandler {
#ResponseStatus(HttpStatus.OK)
#ExceptionHandler(PartialSuccessException.class)
public ResponseData handlePartialSuccessException(
PartialSuccessException ex) {
return ex.getResponseData();
}
}
Change the microservice response:
public class CustomFeignClient extends Client.Default {
public CustomFeignClient(
final SSLSocketFactory sslContextFactory, final HostnameVerifier
hostnameVerifier) {
super(sslContextFactory, hostnameVerifier);
}
#Override
public Response execute(final Request request, final Request.Options
options) throws IOException {
Response response = super.execute(request, options);
if (HttpStatus.SC_OK != response.status()) {
response =
Response.builder()
.status(HttpStatus.SC_OK)
.body(InputStream.nullInputStream(), 0)
.headers(response.headers())
.request(response.request())
.build();
}
return response;
}
}
Add a Feign Client Config:
#Configuration
public class FeignClientConfig {
#Bean
public Client client() {
return new CustomFeignClient(null, null);
}
}
My Goal is to receive some token from downstream server response headers by using ServerHttpResponseDecorator without this I am not able to get response headers in GlobalFilter. based on token I am planning to alter downstream response by raising a custom exception and handled in ErrorWebExceptionHandler.
The problem is once I have read the response headers from downstream service even exception also not able to stop the flow I am getting an original response whatever is coming from downstream service but if I raised an exception before headers reading It is working as expected.
GlobalFilter Sample code
#Component
public class CustomFilter implements GlobalFilter, Ordered {
#Override
public int getOrder() {
return -2;
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponse originalResponse = exchange.getResponse();
ServerHttpResponseDecorator decoratedResponse = new ServerHttpResponseDecorator(originalResponse) {
#Override
public HttpHeaders getHeaders() {
String tokenFromHeader = super.getHeaders().getFirst("TOKEN");
String regIdFromHeader = super.getHeaders().getFirst("regId");
if (false) { // if (true) { It is hadled by exception handler as expected
// I have some Buginese logic here
throw new RuntimeException();
}
if (tokenFromHeader != null && regIdFromHeader != null) {
if (true) {
//I have some Buginese logic here
// No use I am getting original response from down streams
throw new RuntimeException();
}
}
return getDelegate().getHeaders();
}
};
return chain.filter(exchange.mutate().response(decoratedResponse).build());
}
}
Exception Handler
public class MyWebExceptionHandler implements ErrorWebExceptionHandler {
#Override
public Mono<Void> handle(ServerWebExchange exchange, Throwable ex) {
byte[] bytes = ( "Some custom text").getBytes(StandardCharsets.UTF_8);
DataBuffer buffer = exchange.getResponse().bufferFactory().wrap(bytes);
exchange.getResponse().getHeaders().add("Content-Type", "application/json;charset=UTF-8");
return exchange.getResponse().writeWith(Flux.just(buffer));
}
}
Expected out put is
Some custom text
But I am getting an original response
I am using okhttp client 3.5.0 in java to cache the response and server response has below cache-control header:
Cache-Control: private, max-age=0, must-revalidate
First I am trying to make actual network call to get the server response, then in second request I want to get cached response.
Here is my java code:
HttpUrl.Builder httpUrlBuilder = HttpUrl.parse(serverUrl).newBuilder();
HttpUrl httpUrl = httpUrlBuilder.build();
Request request = new Request.Builder()
.url(httpUrl)
.cacheControl(new CacheControl.Builder().maxAge(1, TimeUnit.DAYS).build())
.build();
Call call = okHttpClient.newCall(request);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
}
#Override
public void onResponse(Call call, final Response response) throws IOException {
System.out.println("***************Response 1**************");
System.out.println("isSucess:"+response.isSuccessful());
Response text = response.cacheResponse();
System.out.println("Cached Response:" + text);
Response networkResponse = response.networkResponse();
System.out.println("Network Response ::" +networkResponse);
}
});
Thread.sleep(5000);
Request request2 = new Request.Builder()
.cacheControl(new CacheControl.Builder().onlyIfCached().build())
.url(httpUrlBuilder.build())
.build();
call = okHttpClient.newCall(request2);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
}
#Override
public void onResponse(Call call, final Response response) throws IOException {
System.out.println("***********Response 2************");
System.out.println("isSucess:" +response.isSuccessful());
final Response text = response.cacheResponse();
System.out.println("Cached Response:" + text);
Response networkResponse = response.networkResponse();
System.out.println("Network Response:" +networkResponse);
}
});
And below is the second request output:(partial output)
***********Response 2************
isSucess:false
Cached Response:null
Network Response:null
Whether I am missing anything?
OkHttp won’t cache a response unless you read and close the response body. This is what triggers those bytes to be downloaded and saved.
try {
if (response.cacheResponse() == null) {
body.source().skip(Long.MAX_VALUE); // Exhaust response body.
}
} finally {
body.close();
}
Our application makes various usages of apache HttpAsyncClient:
CloseableHttpAsyncClient client= ...
HttpGet get = new HttpGet(...);
Future<HttpResponse> f = client.execute(get,null);
HttpResponse resp=f.get()
I'm looking for some hook to capture the response just before it's passed on to the business code that invoked 'f.get()' . Inside this hook, I'll perform auditing and security sanitation. BTW Responses are short texts, so there's no problem with buffering.
Would anyone please happen to know of such hooks?
I tried HttpRequestInterceptor, but it seems to work only for synchronous client:
// hook to audit & sanitize *synchronous* client response:
HttpClients.custom().addInterceptorLast(new HttpRequestInterceptor(){
public void process(HttpRequest req, HttpContext ctx) {
HttpEntityEnclosingRequest enclosing=(HttpEntityEnclosingRequest)req;
String body=EntityUtils.toString(enclosing.getEntity());
// ... audit 'body'
// ... sanitize 'body'
enclosing.setEntity(new StringEntity(sanitizedBody))
Unfortunately it doesn't work for async client - I suspect the interceptor runs before response is ready; I'm looking for a hook that runs when async response is ready.
Thanks
Consider using a custom HttpAsyncResponseConsumer. This should give you a complete control over the response message processing.
CloseableHttpAsyncClient client = HttpAsyncClients.createDefault();
HttpAsyncResponseConsumer responseConsumer = new BasicAsyncResponseConsumer() {
#Override
protected void onResponseReceived(final HttpResponse response) throws IOException {
super.onResponseReceived(response);
}
#Override
protected void onEntityEnclosed(final HttpEntity entity, final ContentType contentType) throws IOException {
super.onEntityEnclosed(entity, contentType);
}
#Override
protected void onContentReceived(final ContentDecoder decoder, final IOControl ioctrl) throws IOException {
super.onContentReceived(decoder, ioctrl);
}
#Override
protected HttpResponse buildResult(HttpContext context) {
return super.buildResult(context);
}
#Override
protected void releaseResources() {
super.releaseResources();
}
};
client.execute(HttpAsyncMethods.createGet("http://target/"), consumer, null);
PS: one can have access to message content stream from inside a protocol interceptor with blocking HttpClient but not with HttpAsyncClient
I have multiple jax-rs services built using cxf/spring. I want to control the output payload response size of all services. For simplicity sake, let's say none of api's in any of the services should ever return a JSON response payload more than 500 characters and I want to control this in one place instead of relying on individual services to adhere to this requirement. (We already have other features built into the custom framework/base component that all services depend on).
I have tried implementing this using JAX-RS's WriterInterceptor, ContainerResponseFilter and CXF's Phase Interceptor, but none of the approaches seem to be completely satisfy my requirement. More details on what I've done so far:
Option 1: (WriterInteceptor) In the overridden method, I get the ouputstream and set the max size of the cache to 500. When I invoke an api that returns more than 500 characters in the response payload, I get an HTTP 400 Bad Request status, but the response body contains the entire JSON payload.
#Provider
public class ResponsePayloadInterceptor implements WriterInterceptor {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
#Override
public void aroundWriteTo(WriterInterceptorContext context) throws IOException, WebApplicationException {
final OutputStream outputStream = context.getOutputStream();
CacheAndWriteOutputStream cacheAndWriteOutputStream = new CacheAndWriteOutputStream(outputStream);
cacheAndWriteOutputStream.setMaxSize(500);
context.setOutputStream(cacheAndWriteOutputStream);
context.proceed();
}
}
Option 2a: (CXF Phase Inteceptor) In the overridden method, I get the response as String from the ouputstream and check it's size. If it's greater than 500, I create a new Response object with only the data Too much data and set it in the message. Even if the response is > 500 characters, I get an HTTP 200 OK status with the entire JSON. Only when I use the phase as POST_MARSHAL or a later phase, I'm able to get hold of the JSON response and check it's length, but by that time the response has already been streamed to the client.
#Provider
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.POST_MARSHAL);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
...
CachedOutputStream cachedOutputStream = (CachedOutputStream) outputStream;
String responseBody = IOUtils.toString(cachedOutputStream.getInputStream(), "UTF-8");
...
LOGGER.info("handleMessage() - Response: {}", responseBody);
LOGGER.info("handleMessage() - Response Length: {}", responseBody.length());
if (responseBody.length() > 500) {
Response response = Response.status(Response.Status.BAD_REQUEST)
.entity("Too much data").build();
message.getExchange().put(Response.class, response);
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error");
e.printStackTrace();
}
}
}
Option 2b: (CXF Phase Inteceptor) Same as above, but only the contents of if block is changed. If response length is greater than 500, I create a new output stream with the string Too much data and set it in message. But if the response payload is > 500 characters, I still get an HTTP 200 OK status with an invalid JSON response (entire JSON + additional text) i.e., the response looks like this: [{"data":"", ...}, {...}]Too much data (the text 'Too much data' is appended to the JSON)
if (responseBody.length() > 500) {
InputStream inputStream = new ByteArrayInputStream("Too much data".getBytes("UTF-8"));
outputStream.flush();
IOUtils.copy(inputStream, outputStream);
OutputStream out = new CachedOutputStream();
out.write("Too much data".getBytes("UTF-8"));
message.setContent(OutputStream.class, out);
}
Option 3: (ContainerResponseFilter) Using the ContainerResponseFilter, I added a Content-Length response header with value as 500. If response length is > 500, I get an HTTP 200 OK status with an invalid JSON response (truncated to 500 characters). If the response length is < 500, still get an HTTP 200 OK status, but the client waits for more data to be returned by the server (as expected) and times out, which isn't a desirable solution.
#Provider
public class ResponsePayloadFilter implements ContainerResponseFilter {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadFilter.class);
#Override
public void filter(ContainerRequestContext requestContext, ContainerResponseContext responseContext) throws IOException {
LOGGER.info("filter() - Response intercepted");
CachedOutputStream cos = (CachedOutputStream) responseContext.getEntityStream();
StringBuilder responsePayload = new StringBuilder();
ByteArrayOutputStream out = new ByteArrayOutputStream();
if (cos.getInputStream().available() > 0) {
IOUtils.copy(cos.getInputStream(), out);
byte[] responseEntity = out.toByteArray();
responsePayload.append(new String(responseEntity));
}
LOGGER.info("filter() - Content: {}", responsePayload.toString());
responseContext.getHeaders().add("Content-Length", "500");
}
}
Any suggestions on how I can tweak the above approaches to get what I want or any other different pointers?
I resolved this partially using help from this answer. I say partially because I'm successfully able to control the payload, but the not the response status code. Ideally, if the response length is greater than 500 and I modify the message content, I would like to send a different response status code (other than 200 OK). But this is a good enough solution for me to proceed at this point. If I figure out how to update the status code as well, I'll come back and update this answer.
import org.apache.commons.io.IOUtils;
import org.apache.cxf.interceptor.Fault;
import org.apache.cxf.io.CachedOutputStream;
import org.apache.cxf.message.Message;
import org.apache.cxf.phase.AbstractPhaseInterceptor;
import org.apache.cxf.phase.Phase;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.PRE_STREAM);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
CachedOutputStream cachedOutputStream = new CachedOutputStream();
message.setContent(OutputStream.class, cachedOutputStream);
message.getInterceptorChain().doIntercept(message);
cachedOutputStream.flush();
cachedOutputStream.close();
CachedOutputStream newCachedOutputStream = (CachedOutputStream) message.getContent(OutputStream.class);
String currentResponse = IOUtils.toString(newCachedOutputStream.getInputStream(), "UTF-8");
newCachedOutputStream.flush();
newCachedOutputStream.close();
if (currentResponse != null) {
LOGGER.info("handleMessage() - Response: {}", currentResponse);
LOGGER.info("handleMessage() - Response Length: {}", currentResponse.length());
if (currentResponse.length() > 500) {
InputStream replaceInputStream = IOUtils.toInputStream("{\"message\":\"Too much data\"}", "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
} else {
InputStream replaceInputStream = IOUtils.toInputStream(currentResponse, "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
}
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error", e);
throw new RuntimeException(e);
}
}