We Keep Coding

How to apply proper error handling of TimeOut for httpclient

In my Xpages application I am calling an external service to collect data.
Users are complaining that they sometimes get a timeout error message:
Connect to customerbank.acme.se:20543 [customerbank.acme.se/127.17.27.172] failed: Connection timed out: connect
I assumed the timeout would result in an IOException but apparently not. How can I catch this error?
Below is part of my code. The logic of handling the response I have left out.
private CloseableHttpClient httpclient;
try{
HttpClientBuilder cb = HttpClientBuilder.create();
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30 * 1000)
.setConnectTimeout(30 * 1000)
.setConnectionRequestTimeout(30 * 1000)
.build();
cb.setDefaultRequestConfig(requestConfig);
httpclient = cb.build();
HttpPost httpPost = new HttpPost(urlFromConfiguration);
httpPost.setHeader("Content-Type", "application/json");
HttpEntity entity;
entity = new ByteArrayEntity(JSONobj.toString().getBytes("UTF-8"));
httpPost.setEntity(entity);
CloseableHttpResponse response = httpclient.execute(httpPost);
if (200 == response.getStatusLine().getStatusCode()){//response received
//perform some logic with the response...
}
} catch (IOException e) {
OpenLogUtil.logError(e);
FacesContext.getCurrentInstance().addMessage(null, new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, "some IO exception occurred", ""));
} catch (Exception e) {
OpenLogUtil.logError(e);
FacesContext.getCurrentInstance().addMessage(null, new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, "some general error has occured" , ""));
}

I think this Baeldung page can help you:
"Note that the connection timeout will result in an
org.apache.http.conn.ConnectTimeoutException being thrown, while
socket timeout will result in a java.net.SocketTimeoutException."

Apache Http client that you are using is a great utility. But it could be a bit heavy and cumbersome for a relatively simple task that you are running. There is a much simpler Http client provided in MgntUtils Open source library (written by me). It may be not as comprehensive as Apache one, but is much simpler in use. It does throw IOException upon connection or time-out error. In your case it could be an alternative to use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc). All in all your code may look like this:
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setContentType("application/json");
String content = null;
try {
content = client.sendHttpRequest("http://yourUrl.com", HttpMethod.POST, JSONobj.toString());
//content holds the response. Do your logic here
} catch (IOException e) {
//Error Handling is here
content = TextUtils.getStacktrace(e, false);
}
}

Right way to close CloseableHttpResponse/CloseableHttpClient [duplicate]

This question already has answers here:
What is the difference between CloseableHttpClient and HttpClient in Apache HttpClient API?
(8 answers)
Closed 2 years ago.
I'm using CloseableHttpResponse (from apache-httpclient-4.5.3) and I'm not sure I'm using it right, I saw an answer with no votes to use EntityUtils.consume on finally:
CloseableHttpResponse response1 = httpclient.execute(httpGet);
try {
System.out.println(response1.getStatusLine());
} finally {
EntityUtils.consume(response1.getEntity());
CloseableHttpClient is abstract and has no close method to call although in this answer it's used:
CloseableHttpResponse response = httpclient.execute(httpget);
try {
//do something
} finally {
response.close();
}
Currently I'm using try with resources for CloseableHttpClient and CloseableHttpResponse inside of send method.
Am I not missing any resource open or using it in a wrong way?
private CloseableHttpResponse send()
throws URISyntaxException, UnsupportedEncodingException, IOException, ClientProtocolException {
URIBuilder uriBuilder = new URIBuilder(BASE_URL);
HttpHost target = new HttpHost(uriBuilder.getHost(), uriBuilder.getPort(), uriBuilder.getScheme());
HttpPost post = new HttpPost(uriBuilder.build());
try (CloseableHttpClient httpClient = HttpClients.custom().build(); CloseableHttpResponse response = httpClient.execute(target, post)) {
return response;
}

It has been explained in detail in the docs here.
Quoting the pseudo code from the docs here's a typical way to allocate/deallocate an instance of CloseableHttpClient:
try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
<...>
}
The same applies to CloseableHttpResponse :
try (CloseableHttpResponse response = httpclient.execute(httpget)) {
<...>
}
Now, about the close method in CloseableHttpClient. CloseableHttpClient is an abstract class that implements Closeable interface. That is, although it doesn't have a close method itself the classes that extend it are required to implement the close method. One class is InternalHttpClient. You can check the source code for the details.
Before Java7, explicit close would be required:
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
<...>
} finally {
httpclient.close();
}
CloseableHttpResponse response = httpclient.execute(httpget);
try {
<...>
} finally {
response.close();
}

You can avoid the finally by using the try(resource)
try (CloseableHttpResponse response = httpclient.execute(httpGet)) {
... }

How to force Apache HttpClient / HttpConnection to absolutely close tcp connection?

I use lot of http connection with Apache HttpClient (org.apache.httpcomponents 4.3.6) to test servers and I cannot force connections to close, even when I schedule HttpConnectionManager to httpClient.getConnectionManager().shutdown(); after 10 seconds in another thread.
httpClient.close() also doesn't help.
Connections can last for minutes or even hours.
I have tried custom SocketConfig and this also not helps:
SocketConfig socketConfig = SocketConfig.custom()
.setSoKeepAlive(false)
.setSoLinger(5)
.setSoReuseAddress(true)
.setSoTimeout(5000)
.setTcpNoDelay(true).build();
The way I am fetching the content:
HttpUriRequest request = new HttpGet(url);
HttpResponse response = httpClient.execute(request);
try (InputStream in = response.getEntity().getContent()) {
String result = IOUtils.toString(in, StandardCharsets.UTF_8);
httpClient.close();
return result;
}
The way I am building HTTP Client:
SocketConfig socketConfig = SocketConfig.custom()
.setSoKeepAlive(false)
.setSoLinger(configuration.getInt("proxy.list.socket.so.linger"))
.setSoReuseAddress(true)
.setSoTimeout(configuration.getInt("proxy.list.socket.so.timeout"))
.setTcpNoDelay(true).build();
HttpClientBuilder builder = HttpClientBuilder.create();
builder.disableAutomaticRetries();
builder.disableContentCompression();
builder.disableCookieManagement();
builder.disableRedirectHandling();
builder.setConnectionReuseStrategy(new NoConnectionReuseStrategy());
builder.setDefaultSocketConfig(socketConfig);
One of my prototypes of doing shutdown:
shutdownExecutor.schedule(() -> {
httpClient.getConnectionManager().closeExpiredConnections();
httpClient.getConnectionManager().closeIdleConnections(1, TimeUnit.SECONDS);
httpClient.getConnectionManager().shutdown();
httpClient.notifyAll();
try {
httpClient.close();
} catch (IOException ex) {
}
}, configuration.getInt("proxy.test.forced.timeout.seconds"), TimeUnit.SECONDS);
String content = HttpContentFetcher.getAndCloseClient(url, httpClient);

RequestConfig has helped. Now it looks like all the connections are discarded in specified limits.
RequestConfig config= RequestConfig.custom()
.setCircularRedirectsAllowed(false)
.setConnectionRequestTimeout(4000)
.setConnectTimeout(4000)
.setMaxRedirects(0)
.setRedirectsEnabled(false)
.setSocketTimeout(4000)
.setStaleConnectionCheckEnabled(true).build();
request.setConfig(config);

How to send HttpPost every 5 secs

In a Java, I want to send HttpPost every 5 secs without waiting for the response. How can I do that?
I use the following code:
HttpClient httpClient = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
StringEntity params = new StringEntity(json.toString() + "\n");
post.addHeader("content-type", "application/json");
post.setEntity(params);
httpClient.execute(post);
Thread.sleep(5000);
httpClient.execute(post);
but it does not work.
Even though I lose the previous connection and set up a new connection to send the second, the second execute function is always blocked.

Your question leaves a bunch of questions, but the basic point of it can be achieved by:
while(true){ //process executes infinitely. Replace with your own condition
Thread.sleep(5000); // wait five seconds
httpClient.execute(post); //execute your request
}

I tried your code and I got the exception :
java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.
This exception is already logged in HttpClient 4.0.1 - how to release connection?
I was able to release the connection by consuming the response with the following code:
public void sendMultipleRequests() throws ClientProtocolException, IOException, InterruptedException {
HttpClient httpClient = new DefaultHttpClient();
HttpPost post = new HttpPost("http://www.google.com");
HttpResponse response = httpClient.execute(post);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thread.sleep(5000);
response = httpClient.execute(post);
entity = response.getEntity();
EntityUtils.consume(entity);
}

Using DefaultHttpClient is synchronous which means that program is blocked waiting for the response. Instead of that you could use async-http-client library to perform asynchronous requests (you can download jar files from search.maven.org if you're not familiar with Maven). Sample code may look like:
import com.ning.http.client.*; //imports
try {
AsyncHttpClient asyncHttpClient = new AsyncHttpClient();
while(true) {
asyncHttpClient
.preparePost("http://your.url/")
.addParameter("postVariableName", "postVariableValue")
.execute(); // just execute request and ignore response
System.out.println("Request sent");
Thread.sleep(5000);
}
} catch (Exception e) {
System.out.println("oops..." + e);
}

java - multiple http requests at same time

Is there a better way to send thousands of http GET requests at the same time? My code sends requests one after other. Have looked at other answers but could not figure it out. Thanks.
for (int j=0; j<4; j++) {
DefaultHttpClient httpclient = new DefaultHttpClient();
CookieStore cookieStore = httpclient.getCookieStore();
HttpGet httpget = new HttpGet("");
try {
HttpResponse response = httpclient.execute(httpget);
List<Cookie> cookies = cookieStore.getCookies();
} catch (Exception e) {}
httpclient.getConnectionManager().shutdown();
}

you should create multiple threads and each of them should perform an HTTP Request
the below link may help
http://hc.apache.org/httpclient-3.x/threading.html