We Keep Coding

How to apply proper error handling of TimeOut for httpclient

In my Xpages application I am calling an external service to collect data.
Users are complaining that they sometimes get a timeout error message:
Connect to customerbank.acme.se:20543 [customerbank.acme.se/127.17.27.172] failed: Connection timed out: connect
I assumed the timeout would result in an IOException but apparently not. How can I catch this error?
Below is part of my code. The logic of handling the response I have left out.
private CloseableHttpClient httpclient;
try{
HttpClientBuilder cb = HttpClientBuilder.create();
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30 * 1000)
.setConnectTimeout(30 * 1000)
.setConnectionRequestTimeout(30 * 1000)
.build();
cb.setDefaultRequestConfig(requestConfig);
httpclient = cb.build();
HttpPost httpPost = new HttpPost(urlFromConfiguration);
httpPost.setHeader("Content-Type", "application/json");
HttpEntity entity;
entity = new ByteArrayEntity(JSONobj.toString().getBytes("UTF-8"));
httpPost.setEntity(entity);
CloseableHttpResponse response = httpclient.execute(httpPost);
if (200 == response.getStatusLine().getStatusCode()){//response received
//perform some logic with the response...
}
} catch (IOException e) {
OpenLogUtil.logError(e);
FacesContext.getCurrentInstance().addMessage(null, new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, "some IO exception occurred", ""));
} catch (Exception e) {
OpenLogUtil.logError(e);
FacesContext.getCurrentInstance().addMessage(null, new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, "some general error has occured" , ""));
}

I think this Baeldung page can help you:
"Note that the connection timeout will result in an
org.apache.http.conn.ConnectTimeoutException being thrown, while
socket timeout will result in a java.net.SocketTimeoutException."

Apache Http client that you are using is a great utility. But it could be a bit heavy and cumbersome for a relatively simple task that you are running. There is a much simpler Http client provided in MgntUtils Open source library (written by me). It may be not as comprehensive as Apache one, but is much simpler in use. It does throw IOException upon connection or time-out error. In your case it could be an alternative to use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc). All in all your code may look like this:
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setContentType("application/json");
String content = null;
try {
content = client.sendHttpRequest("http://yourUrl.com", HttpMethod.POST, JSONobj.toString());
//content holds the response. Do your logic here
} catch (IOException e) {
//Error Handling is here
content = TextUtils.getStacktrace(e, false);
}
}

Apache httpClient retrying POST request while ignoring timeouts

I'm currently trying to build an OSGi service that sends a POST request to a defined API. This API is used to virus-scan a file which is contained in the request body (JSON) as Base64 string.
For this, I am using Apache HttpClient contained in Adobe AEM uberjar v6.4.0
My current implementation works fine for smaller files (<2 MB), but as filesize gets bigger, the behaviour gets strange:
When I upload a 9 MB file, the request executes for ~1 minute, then gets a HTTP400 as response and afterwards retrys the request 7 times.
I tried to use a timeout with the request. If the timeout is below 60.000ms, a TimeoutException is thrown, if it's greater than 60.000ms, I get a HTTP400 Bad Request. I guess the latter is the APIs fault which I need to clarify.
However, in both cases after the exception is thrown, httpClient retries the request and I have not been able to prevent that since. I'm struggeling with many deprecated "HowTo's" on the web and now I'm here.
I have shortened the code a bit, as it's somehow big (mostly removing debug messages and some "if... return false" at the beginning). My Code:
public boolean isAttachmentClean(InputStream inputStream) throws IOException, JSONException, ServiceUnavailableException {
//prevent httpClient from retrying in case of an IOException
final HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(0, false);
HttpClient httpClient = HttpClients.custom().setRetryHandler(retryHandler).build();
HttpPost httpPost = new HttpPost(serviceUrl);
httpPost.setHeader("accept", "application/json");
//set some more headers...
//set timeout for POST from OSGi Config
RequestConfig timeoutConfig = RequestConfig.custom()
.setConnectionRequestTimeout(serviceTimeout)
.setConnectTimeout(serviceTimeout)
.setSocketTimeout(serviceTimeout)
.build();
httpPost.setConfig(timeoutConfig);
//create request body data
String requestBody;
try {
requestBody = buildDataJson(inputStream);
} finally {
inputStream.close();
}
HttpEntity requestBodyEntity = new ByteArrayEntity(requestBody.getBytes("UTF-8"));
httpPost.setEntity(requestBodyEntity);
//Execute and get the response.
HttpResponse response = httpClient.execute(httpPost);
if (response.getStatusLine().getStatusCode() != HttpServletResponse.SC_OK){
httpPost.abort();
throw new ServiceUnavailableException("API not available, Response Code was "+ response.getStatusLine().getStatusCode());
}
HttpEntity entity = response.getEntity();
boolean result = false;
if (entity != null) {
InputStream apiResult = entity.getContent();
try {
// check the response from the API (Virus yes or no)
result = evaluateResponse(apiResult);
} finally {
apiResult.close();
}
}
return result;
}
"buildDataJson()" simply reads the InputStream and creates a JSON needed for the API call.
"evaluateResponse()" also reads the InputStream, transforms it into a JSON and checks for a property named "Status:" "Clean".
I'd appreciate any tipps on why this request is retried over and over again.
/edit: So far, I found that Apache httpClient has some default mechanism that retries a request in case of an IOException - which is what I get here. Still, I have not found a solution on how to deactivate these retries.

Java HTTP connection.getInputStream() get response from exception

Hello all in my doe I have a try catch and I am catching the exception from a webservice
However If I run my web service in Firefox Poster add-on I get a response as well as a stastus exception
This obviously is not ALL the code but basically the exception is happening at getInputStream()
How can I get the response from the exception?
try{
//Get Response
stream = connection.getInputStream();
} catch (Exception e) {
throw new CustomException("Exception - " + e.getMessage());

HttpURLConnection connection = (HttpURLConnection) url.openConnection();
int code = connection.getResponseCode();
String msg = connection.getResponseMessage();
These methods will still throw IOException if you can't reach the server. But if the server responds, even with an error, these methods give you access to the response.

Using Apache HttpClient how to set the TIMEOUT on a request and response

I need to set time out for the Http Request we make to a service (not a web service). We are using Apache HTTP Client. I have added these 2 lines of code to set the time out on request and response to the service.
HttpConnectionParams.setConnectionTimeout(params, 10000);
HttpConnectionParams.setSoTimeout(params, 10000);
1) Currently I have set 10 seconds as the timeout since I see the response coming from the service almost instantaneously. Should I increase or decrease the timing?
2) What will happen when response is takes more than 10 seconds? Will it throw exception and what exception will it be? Is there any thing else I need to add to set the time out in the below code.
public HashMap<String, Object> getJSONData(String url) throw Exception{
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 10000);
HttpConnectionParams.setSoTimeout(params, 10000);
HttpHost proxy = new HttpHost(getProxy(), getProxyPort());
ConnRouteParams.setDefaultProxy(params, proxy);
URI uri;
InputStream data = null;
try {
uri = new URI(url);
HttpGet method = new HttpGet(uri);
HttpResponse response = httpClient.execute(method);
data = response.getEntity().getContent();
}
catch (Exception e) {
e.printStackTrace();
}
Reader r = new InputStreamReader(data);
HashMap<String, Object> jsonObj = (HashMap<String, Object>) GenericJSONUtil.fromJson(r);
return jsonObj;
}

I am guessing many people come here because of the title and because the HttpConnectionParams API is deprecated.
Using a recent version of Apache HTTP Client, you can set these timeouts using the request params:
HttpPost request = new HttpPost(url);
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(TIMEOUT_MILLIS)
.setConnectTimeout(TIMEOUT_MILLIS)
.setConnectionRequestTimeout(TIMEOUT_MILLIS)
.build();
request.setConfig(requestConfig);
Alternatively, you can also set this when you create your HTTP Client, using the builder API for the HTTP client, but you'll also need to build a custom connection manager with a custom socket config.
The configuration example file is an excellent resource to find out about how to configure Apache HTTP Client.

The exceptions you'll see will be ConnectTimeoutException and SocketTimeoutException. The actual timeout values you use should be the maximum time your application is willing to wait. One important note about the read timeout is that it corresponds to the timeout on a socket read. So it's not the time allowed for the full response to arrive, but rather the time given to a single socket read. So if there are 4 socket reads, each taking 9 seconds, your total read time is 9 * 4 = 36 seconds.
If you want to specify a total time for the response to arrive (including connect and total read time), you can wrap the call in a thread and use a thread timeout for that. For example, I usually do something like this:
Future<T> future = null;
future = pool.submit(new Callable<T>() {
public T call() {
return executeImpl(url);
}
});
try {
return future.get(10, TimeUnit.SECONDS);
}
catch (InterruptedException e) {
log.warn("task interrupted", name);
}
catch (ExecutionException e) {
log.error(name + " execution exception", e);
}
catch (TimeoutException e) {
log.debug("future timed out", name);
}
Some assumptions made in the code above are: 1) this is in a function with a url parameter, 2) it's in a class with a name variable, 3) log is a log4j instance, and 4) pool is a some thread pool executor. Note that even if you use a thread timeout, you should also specify a connect and socket timeout on the HttpClient, so that slow requests don't eat up the resources in the thread pool. Also note that I use a thread pool because typically I use this in a web service so the thread pool is shared across a bunch of tomcat threads. You're environment may be different, and you may prefer to simply spawn a new thread for each call.
Also, I've usually see the timeouts set via member functions of the params, like this:
params.setConnectionTimeout(10000);
params.setSoTimeout(10000);
But perhaps your syntax works as well (not sure).

HttpURLConnection timeout on long polling (comet) - android

I'm trying to retrieve a json string from a comet URL link.
Here is the API link: http://www.plurk.com/API#realtime
Here is the description:
You'll get an URL from /APP/Realtime/getUserChannel and you do GET requests to this URL to get new data. Your request will sleep for about 50 seconds before returning a response if there is no new data added to your channel. You won't get notifications on responses that the logged in user adds, but you will get notifications for new plurks.
I was able to obtain the comet_server url and paste that to firefox and get the result manually. However, when I tried to get these json string in android, I only got timeout error.
01:48:51.698 com.net.xerothermic.plurk INFO PLURK http://comet58.plurk.com:80/comet?channel=...&offset=0
01:53:43.680 com.net.xerothermic.plurk ERROR PLURK HTTP con. get
response error:Connection timed out
Here is the code I used to retrieve the data.
URL url = new URL(urlString);
HttpURLConnection conn = null;
try
{
conn = (HttpURLConnection) url.openConnection();
}
catch (IOException ex)
{
Log.e("PLURK", "HTTP con. open error:" + ex.getMessage());
return "";
}
try
{
conn.setRequestMethod("GET");
}
catch (ProtocolException ex)
{
Log.e("PLURK", "HTTP con. set method error:" + ex.getMessage());
}
try
{
return conn.getResponseMessage();
}
catch (IOException ex)
{
Log.e("PLURK", "HTTP con. get response error:" + ex.getMessage());
return "";
}
Any suggestion is much appreciated!
EDIT: here is the output from a browser. Did I miss to set some properties?

Even though the timeout value was set to 0 by default (meaning wait infinitely), I found I still need to explicit set the timeout value in order to not raise IOException.
setConnectTimeout(70000);
setReadTimeout(70000);
This is only needed on android but not Windows...