We Keep Coding

How to save requests and responses to database in spring boot

I would like to write an aspect or something like that and whenever a request comes to the controller it saves the request and the response to the database.
First question is what type I should use in my entity for request and response ( string, blob, etc)
Second question, how to get request,response and its controller name to create the entity to save to database ?
Lastly, is it possible to calculate response time (time spent in the controller) of the controller ?

First question is what type I should use in my entity for request and
response ( string, blob, etc)
It mainly depends on the database vendor and request/response length.
String may be limited for some vendors and blob is so required.
On the other hand, matching on blob is slower.
Another alternative is using a nosql format such as JSON.
Second question, how to get request,response and its controller name
to create the entity to save to database ?
There are really several ways.
You could take advantage of built-in Spring Boot http tracing features but it has a limitation : posted/received of request/responses are not available.
5.8. HTTP Tracing
HTTP Tracing can be enabled by providing a bean of type
HttpTraceRepository in your application’s configuration. For
convenience, Spring Boot offers an InMemoryHttpTraceRepository that
stores traces for the last 100 request-response exchanges, by default.
InMemoryHttpTraceRepository is limited compared to other tracing
solutions and we recommend using it only for development environments.
For production environments, use of a production-ready tracing or
observability solution, such as Zipkin or Spring Cloud Sleuth, is
recommended. Alternatively, create your own HttpTraceRepository that
meets your needs.
The httptrace endpoint can be used to obtain information about the
request-response exchanges that are stored in the HttpTraceRepository.
5.8.1. Custom HTTP tracing
To customize the items that are included in each trace, use the
management.trace.http.include configuration property. For advanced
customization, consider registering your own HttpExchangeTracer
implementation.
Alternatives are implementing a filter for requests/responses and log in it.
For example :
#Component
public class RequestResponseStoringFilter implements Filter {
#Override
public void doFilter(ServletRequest req, ServletResponse res,
FilterChain chain) throws IOException, ServletException {
HttpServletRequest request = (HttpServletRequest) req;
HttpServletResponse response = (HttpServletResponse) res;
long start = System.currentTimeMillis();
try {
chain.doFilter(req, resp);
} finally {
// Measure elapsed time
long elapsed = System.currentTimeMillis() - start;
// store request data and store response data in a DB
.....
}
}
#Override
public void destroy() {}
#Override
public void init(FilterConfig arg0) throws ServletException {}
}
Lastly, is it possible to calculate response time (time spent in the
controller) of the controller ?
The way implement a Filter can do that as shown above.
The httptrace endpoint way provides that with the timeTaken field.
FIY, here is the content of a HttpTrace instance :
HttpTrace.Principal getPrincipal()
HttpTrace.Request getRequest()
HttpTrace.Response getResponse()
HttpTrace.Session getSession()
Instant getTimestamp()
Long getTimeTaken()

This kinda expands on the other answers, but I think It warrants a separate answer.
If your pulling in spring-boot-starter-web, then you're already pulling in spring-aop. If your going to go down the point cut route though, I'd highly recommend just using the Micrometer #Timed annotation which comes with spring-boot-starter-actuator. I've written my own metric pointcuts a many times, but if your just after timings and counts of successes and failures, #Timed works great.
I'd also highly recommend looking into using a time series database (e.g influx) for storing things like response times and other performance metrics. Keep your raw payloads and other possible auditing concerns in a separate DB. There are some very powerful things you can do with influx and running Grafana or Chronograf on top of it. Without a doubt one of the best things my current company has done is years is adopting Influx/Chronograf.
With regards to the request/response capture, I had a weird edge case in my work flow once where the http trace just wasn't working for some hard requirements. You can capture the contents directly in a chain filter yourself with a ContentCachingRequestWrapper
Then you can access them with:
#Component
class MyPayloadCapturingFilter extends OncePerRequestFilter {
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
ContentCachingRequestWrapper requestWrapper = new ContentCachingRequestWrapper(request)
ContentCachingResponseWrapper responseWrapper = new ContentCachingResponseWrapper(response)
filterChain.doFilter(requestWrapper, responseWrapper)
def requestBody = new String(requestWrapper.contentAsByteArray)
def responseBody = new String(responseWrapper.contentAsByteArray)
//..do something with them
}
}
note the OncePerRequestFilter, I found times when my Filter was firing multiple times for the same request. This prevents that.

Add this dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
then, create an Around aspect for your controllers' methods' execution:
#Around("within(path.to.your.controller.*)")
public Object pointcutWithin(ProceedingJoinPoint joinPoint) throws Throwable {
logger.info(" ###### pointcutWithin() before");
long start = System.nanoTime();
Object result = joinPoint.proceed();
logger.info(" ###### pointcutWithin() after");
long end = System.nanoTime();
long timeElapsedInMillis = (end - start) / 1000000;
logger.info(" ###### elapsed time in millis: "+timeElapsedInMillis);
return result;
}
As for persisting: first, get the req and resp like so:
MyRequest req = (MyRequest) joinPoint.getArgs()[0];
MyResponse resp = (MyResponse) result;
then it's up to you what you actually want t o persist. For classes with simple fields I'd go with a varchar, just remember to override their toString methods.

Logging a request header before Spring Security filter chain

I want to log the contents of a given incoming request header as early as possible.
I know about approaches like CommonsRequestLoggingFilter or a logging HandlerInterceptor, however these seem to only log after Spring has executed a lot of other code, such as the Spring Security filter chain.
I want to log before Spring has done any of that, as early as possible based on a single requirement: the log message needs to be able to extract a header from the HTTP request.
Is there a way to do this?

I have found a way to do this using the embedded Tomcat. Since this receives the request before Spring does, you can capture the entire dispatched request from here.
public class CustomLoggerValve extends ValveBase {
private static final Logger logger = LoggerFactory.getLogger(CustomLoggerValve.class);
#Override
public void invoke(Request request, Response response) throws IOException, ServletException {
try {
MDC.put("requestId", UUID.randomUUID().toString());
logger.info("Received request");
getNext().invoke(request, response);
} finally {
MDC.remove("requestId");
}
}
}
Since I'm using Spring without Spring Boot, I can just add this to my Tomcat directly:
Tomcat tomcat = // ... your tomcat setup
tomcat.getService().getContainer().getPipeline().addValve(new CustomLoggerValve());
I haven't tried, but it looks like you could add this quite easily in Spring Boot too.
Presumably a similar approach would work with embedded Jetty/other JVM web servers.

Springboot: Modify incoming Web service response

I'm working on a springboot service currently and it needs to have the ability to modify the incoming response body received from various web service calls made by itself.
I googled around a lot and could find info about servlet filters, spring interceptors etc. But all of them sit between this service and its calling clients.
But I'm looking for a component which can sit between this service and the other services that it calls. The closest one I could find was spring's ClientHttpRequestInterceptor, but it doesn't seems to have the ability to modify response body.
Client apps ---> 2. My Springboot service. ---> 3. Other web services
I need to have a component between 2 and 3 here.
Can someone please shed some light on this? Thank you.
P.S: Also I know jaxrs ClientRequestFilter does the trick, but I need a solution for spring RestTemplate based service calls and not for jaxrs based.

In Spring RestTemplate allows us to add interceptors that implement ClientHttpRequestInterceptor interface .
The intercept(HttpRequest, byte[], ClientHttpRequestExecution) method of this interface will intercept the given request and return the response by giving us access to the request,
ClientHttpRequestExecution argument to do the actual execution, and pass on the request to the subsequent process chain
public class BodyInterceptor
implements ClientHttpRequestInterceptor {
#Override
public ClientHttpResponse intercept(
HttpRequest request,
byte[] body,
ClientHttpRequestExecution execution) throws IOException {
ClientHttpResponse response = execution.execute(request, body);
response.getHeaders().add("Iphone_version", "proX");
return response;
}
}

Spring AOP can help in your scenario. It can act as a component before invoking another controller or component.

Restricting access to localhost for Java Servlet endpoint

In short - I would like to add such service endpoints to my servlet that can only be called from localhost. The restriction should be coded in the servlet itself, i.e it should not depend on Tomcat/Apache to be configured in a certain way. At the same time, there are many other, existing endpoints that should be reachable externally.
Longer description - I am creating an HTTP API that 3rd parties can implement to integrate with my application. I am also supplying a default implementation, bundled together with my app, that customers with simple requirements can use, without having to implement anything.
The endpoints of my default implementation should be reachable only for my app, which happens to be the same servlet as the one supplying the implementation, i.e it runs on the same host. So for security reasons (the API is security related), I want my implementation to be usable only for my app, which in the first round means restricting access to localhost for a set of HTTP endpoints.
At the same time, I don't want to rely on customers setting up their container/proxy properly, but do the restriction in my servlet, so that there are no changes required for existing installations.
So far the only idea I had was to check the requestor's IP addess in a servlet filter - so I am wondering if there is a better, more sophisticated way.

I think you should add Web Filter to your application and check your url in doFilter method. Check request.getRemoteAddr() and endpoint link you can put in urlPattern.
Like this:
#WebFilter(urlPatterns = "/*")
public class RequestDefaultFilter implements Filter {
#Override
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
if (isForbidden(request, response))
return;
else
chain.doFilter(request, response);
}
}
isForbidden implementation is up to you. In response you just send 403 error code for example.
You can check make same check in servlet and send in response 403 error.

REST API for registration

We want to implement a public RESTful API integrated in our software (written in java) that might be used by various clients to build small e-commerce apps (e.g. for Android or iPhone). This API includes getting a list of products, categories, shopping cart support, etc.
We need to provide an API that will allow user registration and couple of other sensitive functions. How should we protect this API against spam and bruteforcing? In the standard product we use reCAPTCHA. Any alternative for the REST counterpart?

First, think of separation of concerns. What is the purpose of REST API?
A REST API should do offer a service to the client. Client sends a request via REST protocol, and gets a response for its request. In code, this looks something like:
#GET
public Response getClientInfo(#QueryParam("clientId") Integer clientId) {
ClientDTO clientDTO = database.getClientInfo(clientId);
return ResponseWrapper.wrap(clientDTO);
}
Now, you want your REST method doing ONLY this and nothing else. Otherwise, you would put block-bruteforce-and-spam-logic in your REST method and you would get a mess of the code that is not extensible, hard to version, etc. If you want to change your, e.g. blacklisting policy you would have to change each and every REST method, and it's bulky. If you want to check the calls before the make it to REST methods, then take a look at Filters. Every request and response pass through a chain of filters and could be check for misuse of the server.
I don't know what is your technology stack is, but I would suggest looking into these:
JBoss AS7.
DeltaSpike (enables you powerful Interceptors that will check user rights and execution rights before the execution of the REST method).
for example:
#LoggedInUser
#GET
public Response getClientInfo(...) {
...
}
This security annotation #LoggedInUser (which, by the way, you define) will give sign to an Interceptor to check this security constraint, e.g.
#Secures (built in annotation)
#LoggedInUser
public boolean hasRight(Identity identity) {
return identity.isLoggedIn(); //or if he is in certain group of users
}
Context and Dependency Injection context (used in DeltaSpike).
JBoss Filters (a filter chain where you can create your own filter that, for example, checks if some IP is trying to send multiple calls within a very short period ~ 10 lines of code).
An example of the Filter
#Startup
#ApplicationScoped
#Filter(around= "org.jboss.seam.web.ajax4jsfFilter")
public class IPTrackerFilter extends AbstractFilter {
//IPTracker is your #ApplicationScoped bean that remembers all IP addresses accessing the application.
#Inject
private IPTracker fIPTracker;
#Override
public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
if (!(req instanceof HttpServletRequest)) {
chain.doFilter(req, res);
return;
}
final String ipAddress= ((HttpServletRequest)req).getRemoteAddr();
if (fIPTracker.isBlackListed(ipAddress)) {
//implement error message here
sendErrorMessage(response);
return;
} else {
//all good, continue
chain.doFilter(req, res);
}
}
}
PS. I gave you the link for DeltaSpike, for others is really easy to find. Also, if you find DeltaSpike to obscure, try with JBoss Seam Security Framework.