We Keep Coding

asyc http client - hook to capture response?

Our application makes various usages of apache HttpAsyncClient:
CloseableHttpAsyncClient client= ...
HttpGet get = new HttpGet(...);
Future<HttpResponse> f = client.execute(get,null);
HttpResponse resp=f.get()
I'm looking for some hook to capture the response just before it's passed on to the business code that invoked 'f.get()' . Inside this hook, I'll perform auditing and security sanitation. BTW Responses are short texts, so there's no problem with buffering.
Would anyone please happen to know of such hooks?
I tried HttpRequestInterceptor, but it seems to work only for synchronous client:
// hook to audit & sanitize *synchronous* client response:
HttpClients.custom().addInterceptorLast(new HttpRequestInterceptor(){
public void process(HttpRequest req, HttpContext ctx) {
HttpEntityEnclosingRequest enclosing=(HttpEntityEnclosingRequest)req;
String body=EntityUtils.toString(enclosing.getEntity());
// ... audit 'body'
// ... sanitize 'body'
enclosing.setEntity(new StringEntity(sanitizedBody))
Unfortunately it doesn't work for async client - I suspect the interceptor runs before response is ready; I'm looking for a hook that runs when async response is ready.
Thanks

Consider using a custom HttpAsyncResponseConsumer. This should give you a complete control over the response message processing.
CloseableHttpAsyncClient client = HttpAsyncClients.createDefault();
HttpAsyncResponseConsumer responseConsumer = new BasicAsyncResponseConsumer() {
#Override
protected void onResponseReceived(final HttpResponse response) throws IOException {
super.onResponseReceived(response);
}
#Override
protected void onEntityEnclosed(final HttpEntity entity, final ContentType contentType) throws IOException {
super.onEntityEnclosed(entity, contentType);
}
#Override
protected void onContentReceived(final ContentDecoder decoder, final IOControl ioctrl) throws IOException {
super.onContentReceived(decoder, ioctrl);
}
#Override
protected HttpResponse buildResult(HttpContext context) {
return super.buildResult(context);
}
#Override
protected void releaseResources() {
super.releaseResources();
}
};
client.execute(HttpAsyncMethods.createGet("http://target/"), consumer, null);
PS: one can have access to message content stream from inside a protocol interceptor with blocking HttpClient but not with HttpAsyncClient

Response authorization with Jersey

I have a Jersey 2 application containing resources that consume and produce json. My requirement is to add a signature to an Authorization response header generated from a combination of various piece of response data (similar to the Amazon Webservices request signature). One of these pieces of data is the response body but I cant see that there are any filter or interception points that will allow me access to the json content. I imagine this is mainly because the response outputstream is for writing not reading.
Any ideas as to how I can read the response body - or alternative approaches ?
Thank you.

My understanding is that when your application is responding to a request, you want to modify the Authorization header by adding a signature to it's value.
If that's the case, you want to implement a ContainerResponseFilter:
public class MyContainerResponseFilter implements ContainerResponseFilter {
#Override
public void filter(ContainerRequestContext containerRequestContext, ContainerResponseContext containerResponseContext) throws IOException {
// You can get the body of the response from the ContainerResponseContext
Object entity = containerResponseContext.getEntity();
// You'll need to know what kind of Object the entity is in order to do something useful though
// You can get some data using these functions
Class<?> entityClass = containerResponseContext.getEntityClass();
Type entityType = containerResponseContext.getEntityType();
// And/or by looking at the ContainerRequestContext and knowing what the response entity will be
String method = containerRequestContext.getMethod();
UriInfo uriInfo = containerRequestContext.getUriInfo();
// Then you can modify your Authorization header in some way
String authorizationHeaderValue = containerResponseContext.getHeaderString(HttpHeaders.AUTHORIZATION);
authorizationHeaderValue = authorizationHeaderValue + " a signature you calculated";
containerResponseContext.getHeaders().putSingle(HttpHeaders.AUTHORIZATION, authorizationHeaderValue);
}
}
Be warned that the filter function will be called for all requests to your application, even when Jersey couldn't find a matching resource for the request path, so you may have to do some extra checking.

You can implement ContainerRequestFilter in order to access the content, and once you are finished with your interception logic, forward it to the request. E.g.
import java.io.*;
import com.sun.jersey.api.container.ContainerException;
import com.sun.jersey.core.util.ReaderWriter;
import com.sun.jersey.spi.container.ContainerRequest;
import com.sun.jersey.spi.container.ContainerRequestFilter;
public class ExampleFilter implements ContainerRequestFilter {
#Override
public ContainerRequest filter(ContainerRequest req) {
try(InputStream in = req.getEntityInputStream(); ByteArrayOutputStream out = new ByteArrayOutputStream();) {
if (in.available() > 0) {
StringBuilder content = new StringBuilder();
ReaderWriter.writeTo(in, out);
byte[] entity = out.toByteArray();
if (entity.length > 0) {
content.append(new String(entity)).append("\n");
System.out.println(content);
}
req.setEntityInputStream(new ByteArrayInputStream(entity));
}
} catch (IOException ex) {
//handle exception
}
return req;
}
}

Architecture for http client

I am developing application, with http client, and I wonder to make some elegant issue.
This is standard java http client whose work in background thread, and passing data by event's (witch realized by override methods). I have special class for background requests, that implements method sendRequest()
protected void sendRequest(final String url) {
Thread t = new Thread(new Runnable() {
#Override
public void run() {
URI website = null;
try {
website = new URI(url);
} catch (URISyntaxException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
HttpGet request = new HttpGet();
request.setURI(website);
HttpResponse response = null;
try {
response = client.execute(request, httpContext);
} catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
if (response != null)
{
HttpEntity entity = response.getEntity();
try {
InputStream is = entity.getContent();
if (Debug.isDebuggerConnected()==true)
{
String data = convertStreamToString(is);
int code = response.getStatusLine().getStatusCode();
if (httpEvent!=null)
httpEvent.HttpResponseArrived(data, code);
}
else
httpEvent.HttpResponseArrived(convertStreamToString(is),response.getStatusLine().getStatusCode());
}
catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
}
}
});
t.start();
}
And also child class, for API to web server, wich have methods like that:
public void getSomeData(SomeParams param)
{
sendRequest("http://xxx.yy"+gson.toJson(param));
httpEvent = new HttpHandler()
{
#Override
public void HttpResponseArrived(String data, int code)
{
switch (code)
{
case 200:
//some code
break;
case 401:
//some code
break;
}
}
};
}
And my question: how elegant to handle server errors, for example 401? I need to do this in one place, in method that sending requests - sendRequest(). At first sight it is very easy: just handle 401, and if it's because expired cookie - call method Login() (in my design, it's look like getSomeData). But I want, not just login again, I need to request data, that I failed to get because the error. Of course, I can implement calling Login() method in every switch, like this:
case 401:
{
Login(CurrentContext.AuthData.Login, CurrentContext.AuthData.Password);
break;
}
But the login event implemented in Login() method;
Also, I can just write sendRequest(string authdata), subscrube for HttpHandler and by recursion call method thats implements this code. But I thind, it's not very good decision.
I really hope, that somebody already solve this problem, and there is the way, to turn it's in beautiful code!
Thanks, if you could to read this to the end:)

As for answer not comment.
Try to use http client framework - I prefer Apache HTTPClient. It provides wide controll over request and responses. Moreover it supports most common methods like GET POST etc. Cookie management, redirection handling and SSL support is also provided. Don't invent something that is already invented.
HttpClient - use v4.x

Getting redirected URL in Apache HttpComponents

I'm using Apache HttpComponents to GET some web pages for some crawled URLs. Many of those URLs actually redirect to different URLs (e.g. because they have been processed with a URL shortener). Additionally to downloading the content, I would like to resolve the final URLs (i.e. the URL which provided the downloaded content), or even better, all URLs in the redirect chain.
I have been looking through the API docs, but got no clue, where I could hook. Any hints would be greatly appreciated.

One way is to turn off automatic redirect handling by setting the relevant parameter, and do it yourself by checking for 3xx responses, and manually extracting the redirect location from the responses "Location" header.

Here's a full demo of how to do it using Apache HttpComponents.
Important Details
You'll need to extend DefaultRedirectStrategy like so:
class SpyStrategy extends DefaultRedirectStrategy {
public final Deque<URI> history = new LinkedList<>();
public SpyStrategy(URI uri) {
history.push(uri);
}
#Override
public HttpUriRequest getRedirect(
HttpRequest request,
HttpResponse response,
HttpContext context) throws ProtocolException {
HttpUriRequest redirect = super.getRedirect(request, response, context);
history.push(redirect.getURI());
return redirect;
}
}
expand method sends a HEAD request which causes client to collect URIs in spy.history deque as it follows redirects automatically:
public static Deque<URI> expand(String uri) {
try {
HttpHead head = new HttpHead(uri);
SpyStrategy spy = new SpyStrategy(head.getURI());
DefaultHttpClient client = new DefaultHttpClient();
client.setRedirectStrategy(spy);
// FIXME: the following completely ignores HTTP errors:
client.execute(head);
return spy.history;
}
catch (IOException e) {
throw new RuntimeException(e);
}
}
You may want to set maximum number of redirects followed to something reasonable (instead of the default of 100) like so:
BasicHttpParams params = new BasicHttpParams();
params.setIntParameter(ClientPNames.MAX_REDIRECTS, 5);
DefaultHttpClient client = new DefaultHttpClient(params);

Proxy servlet hangs on HttpClient.execute()

I need to write a servlet that basically just proxies each incoming request to the same URL path on a different host. Here's what I came up with using Apache Commons Http Client 4.1.3:
#WebServlet("/data/*")
public class ProxyServlet extends HttpServlet {
protected void doGet (HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
HttpClient client = new DefaultHttpClient();
try {
String url = getMappedServiceUrlFromRequest(request);
HttpGet get = new HttpGet(url);
copyRequestHeaders(request, get);
HttpResponse getResp = client.execute(get);
response.setStatus(getResp.getStatusLine().getStatusCode());
copyResponseHeaders(getResp, response);
HttpEntity entity = getResp.getEntity();
if (entity != null) {
OutputStream os = response.getOutputStream();
try {
entity.writeTo(os);
} finally {
try { os.close(); } catch (Exception ignored) { }
}
}
} catch (Exception e) {
throw new ServletException(e);
} finally {
client.getConnectionManager().shutdown();
}
}
private void getMappedServiceUrlFromRequest (...)
private void copyResponseHeaders (...)
private void copyRequestHeaders (...)
}
This works just fine the first time the servlet is called. However, after the first time, the servlet hangs on the line client.execute(get).
There are plenty of Google hits for "HttpClient execute hangs", most of which suggest using an instance of ThreadSafeClientConnManager. Tried that, sadly didn't help.
I've spent several hours googling for the problem, but I haven't found anything that fixes it yet. I'd seriously appreciate any pointers as to what I am doing wrong here.

I suggest you are doing this the hard way. Just write a Filter that does the redirect.
Or even just a TCP server that listens at the port and just copies bytes back and forth. You don't really need to engage in the HTTP protocol at all in a proxy, unless you are implementing the CONNECT command, in which case that's the only piece of HTTP you need to understand, and its reply is the only HTTP response you need to know about. Everything else is just bytes.