We Keep Coding

HttpServletRequest body lose after read it once [duplicate]

In spring I have a controller with an endpoint like so:
#RequestMapping(method = RequestMethod.POST)
#ResponseStatus(HttpStatus.CREATED)
#ResponseBody
public OutputStuff createStuff(#RequestBody Stuff stuff) {
//my logic here
}
This way if doing a POST on this endpoint, the JSON in request body will be automatically deserialized to my model (Stuff). The problem is, I just got a requirement to log the raw JSON as it is coming in! I tried different approaches.
Inject HttpServletRequest into createStuff, read the body there and log:
Code:
#RequestMapping(method = RequestMethod.POST)
#ResponseStatus(HttpStatus.CREATED)
#ResponseBody
public OutputStuff createStuff(#RequestBody Stuff stuff, HttpServletRequest req) {
StringBuilder sb = new StringBuilder();
req.getReader().getLines().forEach(line -> {
sb.append(line);
});
//log sb.toString();
//my logic here
}
The problem with this is that by the time I execute this, the reader's InputStream would have already been executed to deserialize JSON into Stuff. So I will get an error because I can't read the same input stream twice.
Use custom HandlerInterceptorAdapter that would log raw JSON before the actual handler is called.
Code (part of it):
public class RawRequestLoggerInterceptor extends HandlerInterceptorAdapter {
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
StringBuilder sb = new StringBuilder();
req.getReader().getLines().forEach(line -> {
sb.append(line);
});
//log sb.toString();
return true;
}
}
The problem with this tho is, that by the time the deserialization to stuff happens, the InputStream from the request would have been read already! So I would get an exception again.
Another option I considered, but not implemented yet, would be somehow forcing Spring to use my custom implementation of HttpServletRequest that would cache the input stream and allow multiple read of it. I have no idea if this is doable tho and I can't find any documentation or examples of that!
Yet another option would be not to read Stuff on my endpoint, but rather read the request body as String, log it and then deserialize it to Stuff using ObjectMapper or something like that. I do not like this idea either tho.
Are there better solutions, that I did not mention and/or am not aware of? I would appreciate help. I am using the latest release of SpringBoot.

To read the request body multiple times, we must cache the initial payload. Because once the original InputStream is consumed we can't read it again.
Firstly, Spring MVC provides the ContentCachingRequestWrapper class which stores the original content. So we can retrieve the body multiple times calling the getContentAsByteArray() method.
So in your case, you can make use of this class in a Filter:
#Component
public class CachingRequestBodyFilter extends GenericFilterBean {
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain chain)
throws IOException, ServletException {
HttpServletRequest currentRequest = (HttpServletRequest) servletRequest;
ContentCachingRequestWrapper wrappedRequest = new ContentCachingRequestWrapper(currentRequest);
// Other details
chain.doFilter(wrappedRequest, servletResponse);
}
}
Alternatively, you can register CommonsRequestLoggingFilter in your application. This filter uses ContentCachingRequestWrapper behind the scenes and is designed for logging the requests.

As referenced in this post: How to Log HttpRequest and HttpResponse in a file?, spring provides the AbstractRequestLoggingFilter you can use to log the request.
AbstractRequestLoggingFilter API Docs, found here

I also tried to do that in Spring but i could not find way to pass my custom http request to chain so what did was,i have written traditional j2ee filter in that i have passed my custom http request to chain that is it then onward i can read http request more than once
Check this example http://www.myjavarecipes.com/how-to-read-post-request-data-twice-in-spring/

How can i achieve req id for tracing using springboot microservices?

The ask is:
Whenever a client calls API's, i want to tag it with a unique identifier or use one supplied by the client (usually in a query param) and pass it across components until that request is fulfilled sucessfully or fails. The goal is to get a holistic picture of how a request was handled by different components and what happened at each component and quickly identify issues.
How can i achieve this using springboot microservices. please help me.

Spring Cloud Sleuth is what you are looking for: https://cloud.spring.io/spring-cloud-sleuth/reference/html/
Spring Cloud Sleuth’s solution is to inject span and trace IDs into log entries. A trace ID is a unique identifier that an entire request flow will share. IA span is more local and is defined for each request received for each request sent event. They define particular interaction points.
The initial span, or root span, is generated when a client request is received from outside the distributed system. This request lacks trace and span information. The root span becomes the trace ID for the rest of the request flow through the system / systems.
The diagram below shows how Sleuth span and trace generation would work through a hypothetical service network.
All you need to do in your code is to add the dependency spring-cloud-starter-sleuth and Spring will automatically instrument the following communication channels:
requests over messaging technologies like Apache Kafka or RabbitMQ
HTTP headers received at Spring MVC controllers
requests made with the RestTemplate

If you want to start simple you could define a filter (e.g. by extending OncePerRequestFilter) that generates/extracts a request ID. You can also put it into Logback's MDC so that it is included in every logging statement that is issued from the thread executing the request (if configured):
#Component
public class RequestIdFilter extends OncePerRequestFilter {
private final ThreadLocal<String> requestId = new ThreadLocal<>();
public Optional<String> getCurrentRequestId() {
return Optional.ofNullable(requestId.get());
}
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) throws ServletException, IOException {
try {
requestId.set(UUID.randomUUID().toString()); // or extract from request
MDC.put("requestId", requestId.get());
chain.doFilter(request, response);
} finally {
requestId.remove();
}
}
}

How do I enable a filter for url that includes a certain word?

I'm trying to log requests to certain URLs using a filter.
These certain URLs I want are the ones that include the word api so anything from "/aaaa/api" to "/api/items/3".
I had URL patterns set in Filter Config from before I was told to change it to match any URL with "api" in it but I deleted that and used regex to filter URLs in my ApiLogFilter instead as shown below.
My question is: Is there a way to achieve this by editing the pattern of addUrlPatterns in Filter Config? Also, what is the best practice?
#Configuration
public class FilterConfig {
#Bean
public FilterRegistrationBean<ApiLogFilter>
filterRegistrationBean() {
FilterRegistrationBean<ApiLogFilter> registrationBean = new
FilterRegistrationBean();
registrationBean.setFilter(new ApiLogFilter());
registrationBean.addUrlPatterns("/api/items/*");
return registrationBean;
}
#Override
public final void doFilterInternal(
HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain)
throws ServletException, IOException {
// 処理の開始時間を記録
long startTime = System.currentTimeMillis();
// フィルターチェーンの次のフィルターにリクエストとレスポンスを渡す
filterChain.doFilter(request, response);
// 処理が戻ってきた時間から処理時間を記録
long processingTime = System.currentTimeMillis() - startTime;
// logs only if the request URL follows this pattern
if (request.getRequestURI().matches(".*/api/.*")) {
// ログ出力
logger.info(
"{} \"{} {}\" {} {}(ms)", request.getRemoteHost(),
request.getMethod(),
request.getRequestURI(), response.getStatus(),
processingTime);
}
}

In Spring you have two options for handling HTTP request/response. These are using of servlet filter (as you do) or interceptor (link).
Filter can change request/response or even stop HTTP workflow at all. If you got unhandled exception in filter, your request stops working.
Interceptor can't change request/response. It can just listen. Unlike filter if you got unhendled exception in interceptor request doesn't stop working (you just get message in console or log).
Concerning URL pattern: it has very simple syntax. In fact almost all you can do is specifying asterisk at the start or end of the string. *.ext - means files with ext extension. /api/* - means everything starts with /api/. I guess this simplicity made in purpose of performance efficiency. But it doesn't fit your requirements.
There is nothing wrong you use regexp in your filter. It won't affect performance significantly. Everything is OK except of one remark. Regular expression processing consist of two parts: compiling of regexp and matching a string. getRequestURI().matches() does the both parts every time you call it. It would be better to compile regexp just once. Add to your filter:
private static Pattern pattern = Pattern.compile(".*/api/.*");
Then you can use precompiled pattern in filter's method:
#Override
public final void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) {
...
if (pattern.matcher(request.getRequestURI()).matches()) {
//process query
}
}
This approach allows to avoid recompilation of pattern every time you use it.

Limit size of http application/json request body in Spring, tomcat

I want to limit the size of accepted application/json http request bodys. So that it is not possible to send multiple megabytes of json to my application, which are then processed and make my app run for a long time.
I've read here that there is no out of the box solution to do this.
Spring boot Embedded Tomcat "application/json" post request restriction to 10KB
Is there another solution beside implementing something myself.
For me this seems like a very common use-case and I can't believe that there is no general solution for this, because this is a very easily exploitable security issue.

there is no out of the box solution, but simplest way would be to write a filter to check request length, something like this
#Component
static class ApplicationJsonRequestSizeLimitFilter extends OncePerRequestFilter {
#Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
if (isApplicationJson(request) && request.getContentLengthLong() > 10000) {
throw new IOException("Request content exceeded limit of 10000 bytes");
}
filterChain.doFilter(request, response);
}
private boolean isApplicationJson(HttpServletRequest httpRequest) {
return (MediaType.APPLICATION_JSON.isCompatibleWith(MediaType
.parseMediaType(httpRequest.getHeader(HttpHeaders.CONTENT_TYPE))));
}
}

you can try this for post requests in spring boot :
server.tomcat.max-http-form-post-size

Java Spring change the response before handle it

Ok, I have been trying to implement a system in which after check the parameters on the request, like the Path, i will actually modify the response to answer with different data. Idea is to create backend demo functionality, in this way client can Demo (NOT TEST) the application without actually making DB requests.
So, my first try was using servlet filters, a very good answer for this can be found here, and also some good document called The Essentials of Filters. But I could not make it work, i think because I'm using spring with #Controller and #ResponseBody, even I follow exactly same sample I would get a null as wrapperResponse.
Then I tried the Interceptors, here there is good example, and a good actually answer here. But the problem with this is that normally people will use the postHandle to modify the body, and I really do not want the handle to even trigger, because this means that the DB calls will be also triggered. And if I use the preHandler as here it will just make a new servlet, and I don't want that.
Finally I try #ControllerAdvice which basically allows you to re-write the body before is sent, but again, the handler gets processed and all DB calls with it.
MY goal, is that I do not have to put repeated code in each handler, I could make a preHandler insert some extra header and check that header in the #ControllerAdvice, but this means that i have to make some IF/ELSE in the handler so it doesn't get processed and I have to repeat that on the 100s of #Controllers that i have in the system, i want to be DRY.
I'm pretty sure the solution is on the filter in the way of this answer
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
System.out.println("BEFORE filter");
PrintWriter out = response.getWriter();
CharResponseWrapper responseWrapper = new CharResponseWrapper(
(HttpServletResponse) response);
chain.doFilter(request, responseWrapper);
String servletResponse = new String(responseWrapper.toString());
out.write(servletResponse + " filtered"); // Here you can change the response
System.out.println("AFTER filter, original response: "
+ servletResponse);
}
But I can't make it work with spring and #ResponseBody calls. And true be told, this doesn't answer my question.

This is the way I manage to do this.
First I created an interceptor, which actually filter the request to pass just the want we want to demo. In the pre handler instead of trying to create a response there using the Response outstream I just used the RequestDispatcher to forward the request to a new controller, which I called Demo controller.
#Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler) throws Exception {
// TODO Auto-generated method stub
Pattern pattern = Pattern.compile("someregex");
Matcher matcher = pattern.matcher(request.getPathInfo());
if (matcher.find())
{
if (matcher.group(0).equals("SOMETHING"))
{
HandlerMethod handlerMethod = ((HandlerMethod)handler);
request.setAttribute("methodName", handlerMethod.getBeanType().getSimpleName());
request.getRequestDispatcher("/demo").forward(request, response);
return false;
}
return true;
}
else
{
return true;
}
}
In the Demo controller then you can create a proper response you want to demo. The good thing here is that in the new demo forwarded request will have an attribute for the original request javax.servlet.forward.request_uri, and that you can insert data, as the controllerName on the request before forward. All this data can be extracted in the Demo controller in order to generate the required data.