I am trying to log (just to console write now for simplicity sake) the final rendered HTML that will be returned by the HttpServletResponse. (i.e. the body) To this end, I am using the HandlerInterceptorAdapter from Spring MVC like so:
public class VxmlResponseInterceptor extends HandlerInterceptorAdapter {
#Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
System.out.println(response.toString());
}
}
This works as expected and I see the HTTP response headers in the console. My question is if there is a relatively simple way to log the entire response body (i.e. final rendered HTML) to the console without having to resort to doing jumping jacks with PrintWriters, OutputStream's and the like.
Thanks in advance.
This would be better done using a Servlet Filter rather than a Spring HandlerInterceptor, for the reason that a Filter is allowed to substitute the request and/or response objects, and you could use this mechanism to substitute the response with a wrapper which logs the response output.
This would involve writing a subclass of HttpServletResponseWrapper, overriding getOutputStream (and possibly also getWriter()). These methods would return OutputStream/PrintWriter implementations that siphon off the response stream into a log, in addition to sending to its original destination. An easy way to do this is using TeeOutputStream from Apache Commons IO, but it's not hard to implement yourself.
Here's an example of the sort of thing you could do, making use of Spring's GenericFilterBean and DelegatingServletResponseStream, as well as TeeOutputStream, to make things easier:
public class ResponseLoggingFilter extends GenericFilterBean {
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain) throws IOException, ServletException {
HttpServletResponse responseWrapper = loggingResponseWrapper((HttpServletResponse) response);
filterChain.doFilter(request, responseWrapper);
}
private HttpServletResponse loggingResponseWrapper(HttpServletResponse response) {
return new HttpServletResponseWrapper(response) {
#Override
public ServletOutputStream getOutputStream() throws IOException {
return new DelegatingServletOutputStream(
new TeeOutputStream(super.getOutputStream(), loggingOutputStream())
);
}
};
}
private OutputStream loggingOutputStream() {
return System.out;
}
}
This logs everything to STDOUT. If you want to log to a file, it'll get a big more complex, what with making sure the streams get closed and so on, but the principle remains the same.
If you're using (or considering) logback as your logging framework, there is a nice servlet filter already available that does exactly that. Checkout the TeeFilter chapter in the documentation.
I've been looking for a way to log full HTTP Request/Response for a while and discovered it has been solved for me in the Tomcat 7 RequestDumperFilter. It works as advertised from a Tomcat 7 container. If you want to use it in Jetty, the class works fine stand-alone or, as I did, copied and adapted to the specific needs of my environment.
I made a small library spring-mvc-logger available via maven central.
Add to pom.xml:
<dependency>
<groupId>com.github.isrsal</groupId>
<artifactId>spring-mvc-logger</artifactId>
<version>0.2</version>
</dependency>
Add to web.xml:
<filter>
<filter-name>loggingFilter</filter-name>
<filter-class>com.github.isrsal.logging.LoggingFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>loggingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
Add to log4j.xml:
<logger name="com.github.isrsal.logging.LoggingFilter">
<level value="DEBUG"/>
</logger>
the code pasted below works with my tests and can be downloaded from my github project, sharing after applying a solution based on that on a production project
#Configuration
public class LoggingFilter extends GenericFilterBean {
/**
* It's important that you actually register your filter this way rather then just annotating it
* as #Component as you need to be able to set for which "DispatcherType"s to enable the filter
* (see point *1*)
*
* #return
*/
#Bean
public FilterRegistrationBean<LoggingFilter> initFilter() {
FilterRegistrationBean<LoggingFilter> registrationBean = new FilterRegistrationBean<>();
registrationBean.setFilter(new LoggingFilter());
// *1* make sure you sett all dispatcher types if you want the filter to log upon
registrationBean.setDispatcherTypes(EnumSet.allOf(DispatcherType.class));
// *2* this should put your filter above any other filter
registrationBean.setOrder(Ordered.HIGHEST_PRECEDENCE);
return registrationBean;
}
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
ContentCachingRequestWrapper wreq =
new ContentCachingRequestWrapper(
(HttpServletRequest) request);
ContentCachingResponseWrapper wres =
new ContentCachingResponseWrapper(
(HttpServletResponse) response);
try {
// let it be ...
chain.doFilter(wreq, wres);
// makes sure that the input is read (e.g. in 404 it may not be)
while (wreq.getInputStream().read() >= 0);
System.out.printf("=== REQUEST%n%s%n=== end request%n",
new String(wreq.getContentAsByteArray()));
// Do whatever logging you wish here, in this case I'm writing request
// and response to system out which is probably not what you wish to do
System.out.printf("=== RESPONSE%n%s%n=== end response%n",
new String(wres.getContentAsByteArray()));
// this is specific of the "ContentCachingResponseWrapper" we are relying on,
// make sure you call it after you read the content from the response
wres.copyBodyToResponse();
// One more point, in case of redirect this will be called twice! beware to handle that
// somewhat
} catch (Throwable t) {
// Do whatever logging you whish here, too
// here you should also be logging the error!!!
throw t;
}
}
}
Related
In spring I have a controller with an endpoint like so:
#RequestMapping(method = RequestMethod.POST)
#ResponseStatus(HttpStatus.CREATED)
#ResponseBody
public OutputStuff createStuff(#RequestBody Stuff stuff) {
//my logic here
}
This way if doing a POST on this endpoint, the JSON in request body will be automatically deserialized to my model (Stuff). The problem is, I just got a requirement to log the raw JSON as it is coming in! I tried different approaches.
Inject HttpServletRequest into createStuff, read the body there and log:
Code:
#RequestMapping(method = RequestMethod.POST)
#ResponseStatus(HttpStatus.CREATED)
#ResponseBody
public OutputStuff createStuff(#RequestBody Stuff stuff, HttpServletRequest req) {
StringBuilder sb = new StringBuilder();
req.getReader().getLines().forEach(line -> {
sb.append(line);
});
//log sb.toString();
//my logic here
}
The problem with this is that by the time I execute this, the reader's InputStream would have already been executed to deserialize JSON into Stuff. So I will get an error because I can't read the same input stream twice.
Use custom HandlerInterceptorAdapter that would log raw JSON before the actual handler is called.
Code (part of it):
public class RawRequestLoggerInterceptor extends HandlerInterceptorAdapter {
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
StringBuilder sb = new StringBuilder();
req.getReader().getLines().forEach(line -> {
sb.append(line);
});
//log sb.toString();
return true;
}
}
The problem with this tho is, that by the time the deserialization to stuff happens, the InputStream from the request would have been read already! So I would get an exception again.
Another option I considered, but not implemented yet, would be somehow forcing Spring to use my custom implementation of HttpServletRequest that would cache the input stream and allow multiple read of it. I have no idea if this is doable tho and I can't find any documentation or examples of that!
Yet another option would be not to read Stuff on my endpoint, but rather read the request body as String, log it and then deserialize it to Stuff using ObjectMapper or something like that. I do not like this idea either tho.
Are there better solutions, that I did not mention and/or am not aware of? I would appreciate help. I am using the latest release of SpringBoot.
To read the request body multiple times, we must cache the initial payload. Because once the original InputStream is consumed we can't read it again.
Firstly, Spring MVC provides the ContentCachingRequestWrapper class which stores the original content. So we can retrieve the body multiple times calling the getContentAsByteArray() method.
So in your case, you can make use of this class in a Filter:
#Component
public class CachingRequestBodyFilter extends GenericFilterBean {
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain chain)
throws IOException, ServletException {
HttpServletRequest currentRequest = (HttpServletRequest) servletRequest;
ContentCachingRequestWrapper wrappedRequest = new ContentCachingRequestWrapper(currentRequest);
// Other details
chain.doFilter(wrappedRequest, servletResponse);
}
}
Alternatively, you can register CommonsRequestLoggingFilter in your application. This filter uses ContentCachingRequestWrapper behind the scenes and is designed for logging the requests.
As referenced in this post: How to Log HttpRequest and HttpResponse in a file?, spring provides the AbstractRequestLoggingFilter you can use to log the request.
AbstractRequestLoggingFilter API Docs, found here
I also tried to do that in Spring but i could not find way to pass my custom http request to chain so what did was,i have written traditional j2ee filter in that i have passed my custom http request to chain that is it then onward i can read http request more than once
Check this example http://www.myjavarecipes.com/how-to-read-post-request-data-twice-in-spring/
I'm trying to log requests to certain URLs using a filter.
These certain URLs I want are the ones that include the word api so anything from "/aaaa/api" to "/api/items/3".
I had URL patterns set in Filter Config from before I was told to change it to match any URL with "api" in it but I deleted that and used regex to filter URLs in my ApiLogFilter instead as shown below.
My question is: Is there a way to achieve this by editing the pattern of addUrlPatterns in Filter Config? Also, what is the best practice?
#Configuration
public class FilterConfig {
#Bean
public FilterRegistrationBean<ApiLogFilter>
filterRegistrationBean() {
FilterRegistrationBean<ApiLogFilter> registrationBean = new
FilterRegistrationBean();
registrationBean.setFilter(new ApiLogFilter());
registrationBean.addUrlPatterns("/api/items/*");
return registrationBean;
}
#Override
public final void doFilterInternal(
HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain)
throws ServletException, IOException {
// 処理の開始時間を記録
long startTime = System.currentTimeMillis();
// フィルターチェーンの次のフィルターにリクエストとレスポンスを渡す
filterChain.doFilter(request, response);
// 処理が戻ってきた時間から処理時間を記録
long processingTime = System.currentTimeMillis() - startTime;
// logs only if the request URL follows this pattern
if (request.getRequestURI().matches(".*/api/.*")) {
// ログ出力
logger.info(
"{} \"{} {}\" {} {}(ms)", request.getRemoteHost(),
request.getMethod(),
request.getRequestURI(), response.getStatus(),
processingTime);
}
}
In Spring you have two options for handling HTTP request/response. These are using of servlet filter (as you do) or interceptor (link).
Filter can change request/response or even stop HTTP workflow at all. If you got unhandled exception in filter, your request stops working.
Interceptor can't change request/response. It can just listen. Unlike filter if you got unhendled exception in interceptor request doesn't stop working (you just get message in console or log).
Concerning URL pattern: it has very simple syntax. In fact almost all you can do is specifying asterisk at the start or end of the string. *.ext - means files with ext extension. /api/* - means everything starts with /api/. I guess this simplicity made in purpose of performance efficiency. But it doesn't fit your requirements.
There is nothing wrong you use regexp in your filter. It won't affect performance significantly. Everything is OK except of one remark. Regular expression processing consist of two parts: compiling of regexp and matching a string. getRequestURI().matches() does the both parts every time you call it. It would be better to compile regexp just once. Add to your filter:
private static Pattern pattern = Pattern.compile(".*/api/.*");
Then you can use precompiled pattern in filter's method:
#Override
public final void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) {
...
if (pattern.matcher(request.getRequestURI()).matches()) {
//process query
}
}
This approach allows to avoid recompilation of pattern every time you use it.
Ok, I have been trying to implement a system in which after check the parameters on the request, like the Path, i will actually modify the response to answer with different data. Idea is to create backend demo functionality, in this way client can Demo (NOT TEST) the application without actually making DB requests.
So, my first try was using servlet filters, a very good answer for this can be found here, and also some good document called The Essentials of Filters. But I could not make it work, i think because I'm using spring with #Controller and #ResponseBody, even I follow exactly same sample I would get a null as wrapperResponse.
Then I tried the Interceptors, here there is good example, and a good actually answer here. But the problem with this is that normally people will use the postHandle to modify the body, and I really do not want the handle to even trigger, because this means that the DB calls will be also triggered. And if I use the preHandler as here it will just make a new servlet, and I don't want that.
Finally I try #ControllerAdvice which basically allows you to re-write the body before is sent, but again, the handler gets processed and all DB calls with it.
MY goal, is that I do not have to put repeated code in each handler, I could make a preHandler insert some extra header and check that header in the #ControllerAdvice, but this means that i have to make some IF/ELSE in the handler so it doesn't get processed and I have to repeat that on the 100s of #Controllers that i have in the system, i want to be DRY.
I'm pretty sure the solution is on the filter in the way of this answer
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
System.out.println("BEFORE filter");
PrintWriter out = response.getWriter();
CharResponseWrapper responseWrapper = new CharResponseWrapper(
(HttpServletResponse) response);
chain.doFilter(request, responseWrapper);
String servletResponse = new String(responseWrapper.toString());
out.write(servletResponse + " filtered"); // Here you can change the response
System.out.println("AFTER filter, original response: "
+ servletResponse);
}
But I can't make it work with spring and #ResponseBody calls. And true be told, this doesn't answer my question.
This is the way I manage to do this.
First I created an interceptor, which actually filter the request to pass just the want we want to demo. In the pre handler instead of trying to create a response there using the Response outstream I just used the RequestDispatcher to forward the request to a new controller, which I called Demo controller.
#Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler) throws Exception {
// TODO Auto-generated method stub
Pattern pattern = Pattern.compile("someregex");
Matcher matcher = pattern.matcher(request.getPathInfo());
if (matcher.find())
{
if (matcher.group(0).equals("SOMETHING"))
{
HandlerMethod handlerMethod = ((HandlerMethod)handler);
request.setAttribute("methodName", handlerMethod.getBeanType().getSimpleName());
request.getRequestDispatcher("/demo").forward(request, response);
return false;
}
return true;
}
else
{
return true;
}
}
In the Demo controller then you can create a proper response you want to demo. The good thing here is that in the new demo forwarded request will have an attribute for the original request javax.servlet.forward.request_uri, and that you can insert data, as the controllerName on the request before forward. All this data can be extracted in the Demo controller in order to generate the required data.
For example, suppose I have this filter
#WebFilter("/api/*")
public class MyFilter implements Filter {
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
chain.doFilter(request, response);
}
}
and an interceptor
#Interceptor
public class HeaderLoggerInterceptor {
#AroundInvoke
public Object validateHeaders(InvocationContext ctx) throws Exception {
// pre-action
Object result = ctx.proceed();
// post-action
return result;
}
}
which object will be invoked first when a service endpoint is hit?
if I want to call setAttribute to request so some additional values can be written to local_access log. It doesn't work in Interceptor, only seems to work with Filter.
is it a bad idea to have both in the service?
which one is better to validate header?
which one is better to write response log before sending back to the client?
I really can't find document on this topic. thanks
I will answer my own question after my experiment
However, I discover another strange problem.
I am using an ExceptionMapper, which allows me to centralize the exception handling in one place;
When there is a missing header, I throw out a custom MissingHeaderException;
The exception will be caught by the Mapper, and we format it and provide a nice XML response back to the client
This logic works only with Interceptor not Filter. Any exception thrown from a Filter is not handled by the Mapper.
Server errors result in HTTP 500-responses to the client with a generic error message ("The server encountered an error..."). Is there any way to intercept this message and write a custom one?
I'm would like to have a way to uniquely identify a server error from the client. If I could for instance generate a GUID which I logged server-side upon a server error and then send that ID to the client, that would make it easy to search for that particular exception in the log at any point later in time.
I do realize that server errors are generated by exceptions in the code, so I'm looking for some kind of catch all exception hook in the app engine API. Of course, if such a hook exists, and the code which executes here generates a second exception, it would have to default to the general 500-error again.
I'm using the Java API for GAE
For GAE generated errors you can configure a custom error page. For errors generated by your code you should use a catch-all wrapper inside a first servlet filter.
I ended up coding a Servlet Filter by following the answer in this SO question. The filter wraps the doFilter() call in a general try-catch block and creates a reference number for the client while logging it at the server. I think this little snippet might be useful for others out there:
public class ExceptionFilter implements Filter {
private FilterConfig filterConfig;
public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain)
throws IOException, ServletException {
try {
filterChain.doFilter(request, response);
}
catch (Exception ex) {
String errorId = UUID.randomUUID().toString();
Mylog.e("Server error " + errorId); // Use whatever logging mechanizm you prefer
String clientResponse = "Server error. Reference no: " + errorId;
((HttpServletResponse) response).setStatus(500);
response.getWriter().write(clientResponse);
}
}
public FilterConfig getFilterConfig() {
return filterConfig;
}
public void init(FilterConfig filterConfig) {
this.filterConfig = filterConfig;
}
public void destroy() {}
}
You also need to configure web.xml like this (goes somewhere under the <web app> tag):
<filter>
<filter-name>ExceptionFilter</filter-name>
<filter-class>your.package.ExceptionFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ExceptionFilter</filter-name>
<servlet-name>Your Servlet Name As Defined In servlet-mapping</servlet-name>
</filter-mapping>
You didn't mention if your using python or java. Python error display https://developers.google.com/appengine/docs/python/config/appconfig#Custom_Error_Responses
Note these are just static pages that are shown in the event of any uncaught errors.
You can try and catch these errors in your main handler (I am talking about python), but you can't always. For instance you maybe able to catch a DeadlineExceededError some times, and you may have a tiny bit of time to emit a log, or a redirect (maybe to the same page to try again or to your own static page, with an arg with the GUID you mentioned, then have javascript render it some useful way) but often that won't work. So it very much depends on the nature of the error.