We Keep Coding

Trouble with Sending an asynchronous call using HttpAsyncClient

I want to send an asynchronous post request to a servlet in java. I am using apache HttpAsyncClient as shown in the following method. When I debug, I see that the process that calls this method waits until the called servlet finishes its processing. In other words, the call seems to be synchronous and not asynchronous.
Do you know what is the part that I’m doing wrong?
Thank you!
public void sendPostRequestUsingHttpAsyncClient( String params) {
try (CloseableHttpAsyncClient client = HttpAsyncClients.createDefault()) {
client.start();
HttpPost request = new HttpPost(URL);
StringEntity entity = new StringEntity(params, ContentType.create("application/json", Consts.UTF_8));
request.setEntity(entity);
Future<HttpResponse> future = client.execute(request, null);
try {
System.out.println(future.get().getStatusLine().getStatusCode());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
} catch (IOException e) {
e.printStackTrace();
}
}

Your code is blocked until successful execution of HTTP request when you do future.get()
It seems that your expectation of asynchrony is a bit far from what really happening in your code. To get the benefit of HttpAsyncClient, you can execute multiple requests in the beginning and then wait for all of them to complete using some synchronization primitive.

Java: Exception and Return Value handling in REST service consume

I'm calling a REST Service that returns a JSON String. It works, but I'm not sure how to handle the exceptions and return values. Here are my two methods I wrote:
public static String callRestService(String id) {
try {
URL url = new URL("http://"localhost:8080/rest/api/2/issue/" + id);
String basicAuth = ConnectionHelper.getServerAuthentication(serverConfig.get("authenticationType"),
serverConfig.get("username"), serverConfig.get("password"));
HttpURLConnection connection = ConnectionHelper.getHttpURLConnection(url, "GET", "Accept", basicAuth);
if (connection != null) {
InputStream responseStream = connection.getInputStream();
String response = StringHelper.convertInputStreamToString(responseStream);
connection.disconnect();
return response;
}
return "";
} catch (Exception e) {
return "";
}
}
public static HttpURLConnection getHttpURLConnection(URL url, String requestMethod, String requestProperty,
String authentication) {
try {
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
if (authentication != null && !authentication.isEmpty()) {
connection.addRequestProperty("Authorization", authentication);
}
connection.setRequestMethod(requestMethod);
connection.addRequestProperty(requestProperty, "application/json");
return connection;
} catch (Exception e) {
return null;
}
}
Is my return value and exception handling ok? Or is there a better way to do this?

For better client side handling you should have an Enum with return cases
for example if we are building a registration module your enum should be like the following :
public enum RestResponseEnum{
DONE(1,"done"),DUPLICATE_RECORD(2,"Sorry this is a duplicate record"),ERROR(3,"There is an error happened")
//Getter & Setter
private int code;
//Getter & Setter
private String msg;
private(int code,String msg){
this.code=code;
this.msg=msg;
}
public static String getAsJson(RestResponseEnum restResponseEnum){
JSONObject jsonObject=new JSONObject();
jsonObject.put("code", restResponseEnum.getCode());
jsonObject.put("message", restResponseEnum.getMsg());
return jsonObject.toString();
}
}
Use it like this :
{
// Your function code
if(registeredEmailIsFoundInDatabase){
return RestResponseEnum.getAsJson(RestResponseEnum.DUPLICATE_RECORD);
}
}
You should always faclitate and clearify the response to the client
you can see this methodology from dealing with most of apis like this one from github : https://api.github.com/users/any/any

If it is a proper REST service it will add additional information about a call in the http response code. So if it doesnt start with 2, there is no point in parsing the response body at all (in case there is no contract to return error details in the body).
How to handle your exception much depends on your current application. General rules of thumb are:
Log exceptions
Handle them on an appropriate level
Sometimes you need to ensure encapsulation and handle them where they occur, sometimes it's okay to rethrow them an catch them globally. E.g. you are using a framework like JSF, user has triggered an external service call, log the exception, rethrow it, catch it and inform the user about it without sharing too much technical details. Like:
Error: YOUR_ERROR_CODE has occured. Please contact technical support
if this keeps happening.
Example:
if (connection.getResponseCode().startsWith("2") {
// do stuff
// if any checked exception occurs here, add it to throws clause and let the caller catch it
}
else if connection.getResponseCode().equals("404") {
throw new EntityNotFoundRuntimeException(...);
}
...
But whether or not this solution is good for your case depends on your architecture.

Cannot find a handler for POST with boundary

I'm in the midst of testing my application which is using an HTTP-server. Instead of mocking I decided to go with a HTTP server fixture. Meaning that I do not have to mock any productional code. To accomplish this goal I currently chose for a free to use 3rd party library fixd.
I was able to successfully create several unit tests - which are working by means of a GET request. Most are quite simple, i.e.:
#Test
public void verifyConnectionTest()
{
try
{
final String body = FileUtils.readFileToString(RESOURCE);
final String path = "/";
this.server.handle(Method.GET, path).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
But I now have to send a POST request with multipart/form-data. Which does not make much of a difference other than changing the method and content-type:
#Test
public void executeStepTest()
{
try
{
final String body = FileUtils.readFileToString(SERVICE_RESPONSE);
final String path = "/";
this.server.handle(Method.POST, path, "multipart/form-data").with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
However I get the following error: [ERROR] could not find a handler for POST - / - multipart/form-data; boundary=bqCBI7t-VW1xaJW7BADmTiGMg9w_YM2sHH8ukJYx and my guess is that fixd doesn't recognize the boundary-party. Since the documentation does not show an example I'm quite stuck on this part.
I tried using some wildcards such as '*', no succes. Thus; I need a way to either tell fixd to accept that boundary or use some wildcards I didn't yet discover. Any help would be great, thanks!

I've been making some debug and it seems to be that the problem is in the fixd core.
Basically, fixd indexes every RequestHandlerImpl by a HandlerKey (which includes ContentType as part of the key) in the map handlerMap. See method org.bigtesting.fixd.core.FixtureContainer#resolve.
...
HandlerKey key = new HandlerKey(method, route, contentType);
RequestHandlerImpl handler = handlerMap.get(key);
if (handler == null) {
// Error
}
...
Problem: When the request is multipart/form-data, boundary data (which it's generated dinamically every request) is part of the content type. So, any handler is found in handlerMap because the key changes with every running.
I've made a little test only to check that this is the cause of the problem, passing the contentType to fixd server.handle after the creation of the multipart request, and it works fine.
See the test below:
#Test
public void verifyConnectionTest_multipart() {
try {
// 1. Create multipart request (example with http-commons 3.1)
PostMethod filePost = new PostMethod(url);
Part[] parts = { new StringPart("param", "value") };
MultipartRequestEntity request = new MultipartRequestEntity(parts, filePost.getParams());
filePost.setRequestEntity(request);
// 2. fixd server handle (passing the request content type)
this.server.handle(Method.POST, "/", request.getContentType()).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response) {
response.setStatusCode(200);
response.setContentType("text/xml");
}
});
// 3. Execute multipart request
HttpClient client = new HttpClient();
int status = client.executeMethod(filePost);
// 4. Assertions
Assert.assertEquals(200, status);
} catch (Exception e) {
Assert.fail(e.getMessage());
}
}
Hope it helps you to clarify the problem. Cheers

This was a bug in fixd, and has been fixed in version 1.0.3. Your original code should work using this new version of fixd.

java Non-blocking HTTP client

I have a high volume java application in which I have to send http posts to another server.
Currently I'm using org.apache.commons.httpclient library:
private static void sendData(String data) {
HttpClient httpclient = new HttpClient();
StringRequestEntity requestEntity;
try {
requestEntity = new StringRequestEntity(data, "application/json", "UTF-8");
String address = "http://<my host>/events/"
PostMethod postMethod = new PostMethod(address);
postMethod.setRequestEntity(requestEntity);
httpclient.executeMethod(postMethod);
} catch (Exception e) {
LOG.error("Failed to send data ", e);
}
}
This means I'm sending my http requests synchronously, which doesn't fit my multithreaded high volume app. So I would like to change those calls to asynchronous non-blocking http calls.
I was going through number of options such as apache async client and xsocket but was not able to make it work.
Tried ning:
private static void sendEventToGrpahiteAsync(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) {
BoundRequestBuilder post = asyncHttpClient.preparePost();
post.addHeader("Content-Type", "application/json");
post.setBodyEncoding("UTF-8");
post.setBody(event);
post.execute(new HttpRequestCompletionHandler());
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I tried Apache HttpAsyncClient:
private static void sendEventToGrpahiteAsync(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault()) {
httpclient.start();
HttpPost request = new HttpPost(addr);
StringEntity entity = new StringEntity(event, ContentType.create("application/json", Consts.UTF_8));
request.setEntity(entity);
httpclient.execute(request, null);
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I tried xsocket:
private static void sendEventToGrpahiteAsync2(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (INonBlockingConnection con = new NonBlockingConnection(<SERVER_IP>, 80);
IHttpClientEndpoint httpClientConnection = new HttpClientConnection(con)) {
IHttpResponseHandler responseHandler = new MyResponseHandler();
IHttpRequest request = new PostRequest(url_address, "application/json", Consts.UTF_8.toString(), event);
request.setTransferEncoding(Consts.UTF_8.toString());
httpClientConnection.send(request, responseHandler);
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I get no exceptions but the post doesn't get to the target as well.
To be clear, the target is a graphite server so once a post arrives it is clearly seen in a graph. The synchronous posts works well, I can see the result on the graph, but none of the asynchronous posts shows on my destination graph.
What am I missing?
Thanks

Got it.
All the libraries I'n using are implemented using an extra IO thread, so my process probably ends before a full handshake.
Once I added Thread.sleep(2000) after the http calls things worked just fine.
So for a web app (which is my case) my suggested implementations are just fine (but for a java process you might consider NickJ's answer).

You could use the Java Executor framework:
First, create a Callable to do your work:
public class MyCallable implements Callable<MyResult> {
#Override
public MyResult call() throws Exception {
//do stuff
return result;
}
}
Get an Exectutor which will run your Callable. There are various way to get one, here's one example:
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
Finally, run it:
MyCallable callable = new MyCallable();
Future<MyResult> futureResult = executor.submit(callable);
Getting the result:
boolean resultReady = futureResult.isDone(); //is the result ready yet?
Result r = futureResult.get(); //wait for result and return it
try {
Result r = futureResult.get(10, TimeUnit.SECONDS); //wait max. 10 seconds for result
} catch (TimeOutException e) {
//result still not ready after waiting 10 seconds
}

Proxy servlet hangs on HttpClient.execute()

I need to write a servlet that basically just proxies each incoming request to the same URL path on a different host. Here's what I came up with using Apache Commons Http Client 4.1.3:
#WebServlet("/data/*")
public class ProxyServlet extends HttpServlet {
protected void doGet (HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
HttpClient client = new DefaultHttpClient();
try {
String url = getMappedServiceUrlFromRequest(request);
HttpGet get = new HttpGet(url);
copyRequestHeaders(request, get);
HttpResponse getResp = client.execute(get);
response.setStatus(getResp.getStatusLine().getStatusCode());
copyResponseHeaders(getResp, response);
HttpEntity entity = getResp.getEntity();
if (entity != null) {
OutputStream os = response.getOutputStream();
try {
entity.writeTo(os);
} finally {
try { os.close(); } catch (Exception ignored) { }
}
}
} catch (Exception e) {
throw new ServletException(e);
} finally {
client.getConnectionManager().shutdown();
}
}
private void getMappedServiceUrlFromRequest (...)
private void copyResponseHeaders (...)
private void copyRequestHeaders (...)
}
This works just fine the first time the servlet is called. However, after the first time, the servlet hangs on the line client.execute(get).
There are plenty of Google hits for "HttpClient execute hangs", most of which suggest using an instance of ThreadSafeClientConnManager. Tried that, sadly didn't help.
I've spent several hours googling for the problem, but I haven't found anything that fixes it yet. I'd seriously appreciate any pointers as to what I am doing wrong here.

I suggest you are doing this the hard way. Just write a Filter that does the redirect.
Or even just a TCP server that listens at the port and just copies bytes back and forth. You don't really need to engage in the HTTP protocol at all in a proxy, unless you are implementing the CONNECT command, in which case that's the only piece of HTTP you need to understand, and its reply is the only HTTP response you need to know about. Everything else is just bytes.