Please suggest a way to close httpclient connection, it causes hung thread generated in server.
org.apache.commons.httpclient.HttpClient
HttpClientManager httpClientManager;
HttpClient httpclient = httpClientManager.getHttpClient();
PostMethod p = new PostMethod(url);
try{
httpclient.executeMethod(p);
}
catch(HttpException e){}
finally{
httpclient.close(); //not available in HttpClient
}
Using old 3.1 version of jar where there is no method in HttpClient class to close httpclient connection.
Due to connection is not closing in the end , it causes hung thread in server.
Related
I am using httpClient version 4.5 to connect with our external vendor site. We do not need any connection pool or persistent connection, so I am using the BasicHttpClientConnectionManager to create the HttpClient.
This works fine for minimal number of requests, but if I test this for 1TPS for 1 hour, by the end of the test, we start seeing intermittent connection resets. (guessing request count > 100)
I/O exception (java.net.SocketException) caught when processing request to {s}->https://apiURL:443: Connection reset
Please find below the code snippet for making connection.
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), new X509TrustManager[] { new DefaultTrustManager() }, new SecureRandom());
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, new String[] { "TLSv1.2" }, null,SSLConnectionSocketFactory.getDefaultHostnameVerifier());
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslsf).register("http", new PlainConnectionSocketFactory()).build();
HttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager(socketFactoryRegistry);
HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(1, false);
RequestConfig defaultRequestConfig = RequestConfig.custom().setSocketTimeout(Integer.parseInt(30000)).setConnectTimeout(Integer.parseInt(30000)).setConnectionRequestTimeout(30000).setCookieSpec(CookieSpecs.STANDARD).build();
CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(connectionManager).setDefaultRequestConfig(defaultRequestConfig).setRetryHandler(retryHandler).evictExpiredConnections().build();
HttpPost httpPost = new HttpPost(<endpoint>);
httpPost.setEntity(new UrlEncodedFormEntity(requestData));
httpResponse = httpClient.execute(httpPost);
I saw that fix for a similar issue reported is already available with version 4.5. (Ref: https://issues.apache.org/jira/browse/HTTPCLIENT-1655) provided by Oleg
If thats the case, not sure why I am still facing this issue. Could someone please help on this.
Thanks!
Hi Oleg,
I had been using the httpclient version 4.5.3 when I was still seeing the connection reset errors as posted above.
Later noticed that the fix for reset issue was committed to the version 4.5.1 (https://issues.apache.org/jira/browse/HTTPCLIENT-1655). So, just tried updating that particular version, ran a test and not seeing the connection reset errors any more.
I had expected that this fix should also be available in the higher versions starting from 4.5.1. But, I guess its somehow missed out in the higher versions, have verified that its still an issue with version 4.5.3 for sure.
So, conclusion is that the connection reset error was fixed using httpclient 4.5.1 jar.
Thanks!
public HttpResponseBean get(String url, Map<String, String> headers) throws Exception {
logger.debug("Sending get request...");
HttpClient httpClient = null;
try {
int timeout = 30 * 1000; // 30 seconds
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(timeout)
.setConnectionRequestTimeout(timeout)
.setSocketTimeout(timeout).build();
httpClient = HttpClients.custom()
.setDefaultRequestConfig(requestConfig).build();
HttpGet httpGetRequest = new HttpGet(url);
if (headers != null) {
for (Entry<String, String> entry: headers.entrySet()) {
httpGetRequest.addHeader(new BasicHeader(entry.getKey(), entry.getValue()));
}
}
HttpResponse response = httpClient.execute(httpGetRequest);
HttpResponseBean hrb = new HttpResponseBean(response);
logger.debug("Get response: Response: " + hrb.toString());
return hrb;
} finally {
closeConnection(httpClient);
}
}
This works well most of the time...but once in a while it gets stuck on handshake and will take forever until the server(tomcat) is restarted.
As per this link looks like a bug -> apache httpclient 4.3 not timing out
Is there a way out of this? I am using httpclient 4.4.1
Is there a way out of this? I am using httpclient 4.4.1
Here's the associated bug on the Apache site. It looks like people have had problems with the 4.4.1 version:
I had this issue with version 4.4.1. At first I ignored this thread because it was flagged as resolved in 4.3.
But this seemed to have been resolved in the 4.5.1 version. I'd encourage you to upgrade if possible.
I encountered this issue on version 4.3.4. I upgraded to 4.5.1 and the issue was fixed there.
Here's additional information:
In version 4.3.4, http:// worked fine and timed out after 1 second. With https, the request hangs beyond 1 seconds and hangs until the [...] server closes the TCP connection.
In version 4.5.1, the [correct] http behaviour was identical, and on https I got the following exception: [...] org.apache.http.conn.ConnectTimeoutException: Connect to localhost:6171 [localhost/127.0.0.1] failed: Read timed out
Hope this helps.
I have a code in my java application that connects to a url with a post request and get response.
The code worked for a long time, and it still does for most people. I have one person that the code works for him most of the times, but sometimes I am getting timeout exception.
The code:
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost("http://example.com/link-to-api");
post.addHeader( "Data1", "my_data" );
HttpResponse response = null;
BufferedReader rd = null;
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("post_key1","post_data1"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
response = client.execute(post);
} catch (IOException e) {
System.out.println("error: "+ e);
}
The exception that I get is:
org.apache.http.conn.HttpHostConnectException: Connect to example.com:80 [example.com/__IP__] failed: Connection timed out: connect
I sniffed the network, and I can see the packets when the connection is ok, but every few minutes, when I get the above exception, I don't even see the attempt connection in the sniffer.
I did ping and traceroute to the server while the timeout is occuring, and both are ok (I get response from the server)
So it seems to me that the problem is in the java application, not even attempting the connection.
I am using httpclient 4.4.1, if it is of any importance.
Why I am getting these timeouts?
I solved it . At least in my case, all I did was reset the router on my computer, and the problem disappeared.
I have read on the internet many suggestions to change server's tcp/ip parameters. So if this happens to you, before you do anything, I suggest that you check your hardware as well.
This is scenario I am testing. The steps are as follows
Create an instance of HttpClient(org.apache.commons.httpclient.HttpClient)3.1
Use the instance to get response from a url (http://www.example.com/submitData)
Bring down the server hosting the url mentioned in step 2.
Bring up the url
Wait till the url is up
Use the instance of HttpClient created in step 1 to get response from the url (http://www.example.com/getData)
I get an error - Connection Reset.
Can anyone help me understand.
I have used the following params in the HttpClient
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
httpClient.getParams().setParameter("http.connection.stalecheck", new Boolean(true));
Edited after comments from Peter
This is how I create the HttpClient
HttpClient httpClient = new HttpClient();
AuthScope authScope = new AuthScope("www.example.com", 80, AuthScope.ANY_REALM);
Credentials defaultcreds = new UsernamePasswordCredentials("username", "pwd");
httpClient.getState().setCredentials(authScope, defaultcreds);
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
httpClient.getParams().setParameter("http.connection.stalecheck", new Boolean(true));
To establish connection, I use the following
GetMethod getMethod = new GetMethod("http://"+httpClient.getHostConfiguration().getHost()+"/getData");
int statusCode = httpClient.executeMethod(getMethod);
I want the httpclient instance to preserve the hostname and port and credentials.
This is exact execption message that I get - "Connection reset".
More info - I am executing the test from eclipse. If I run the test case in debug mode, I dont get this exception.
I think the behaviour is reasonable because when the server goes down the client socket on the server goes away too. The client still holds old socket but this socket is gone on the server. You should reconnect from client.
AFAIK the stale parameter allows the client to close the connection the clean way (without Exception) but not prevent from disconnection.
This looks suspicious
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
Having a timeout set to 0ms would likely terminate the request prematurely?
I have code similar to the following:
try {
HttpPost post = new HttpPost(httpsUrl);
setHeaders(post);
HttpEntity entity = new StringEntity(request, "UTF-8");
post.setEntity(entity);
HttpResponse response = httpclient.execute(post);
String result = EntityReader.readContent(response.getEntity());
checkAnswer(result);
return result;
} catch (Exception e) {
throw new ZapException("Error executing the http post request: "+e.getMessage(), e);
}
It sends the content of request to a server via POST using a httpclient instance that might have already been used before (it has persistent connections turned on, since we're sending quite some requests to the same server...).
This sometimes fails with a SocketTimeoutException with "Read timed out" as the message.
It's not clear to us, why it only fails at some times, when most times it doesn't. What gives?
In the following, I assume you are using Apache Commons HttpClient (org.apache.commons.httpclient.HttpClient).
Maybe you get thrown a SocketTimeoutException simply because, occasionally, the host your HttpClient instance is communicating with takes too long to respond, triggering HttpClient's cancellation routine.
You can increase the connection timeout and the socket timeout with the following
HttpConnectionParams params = httpclient.getHttpConnectionManager().getParams();
params.setConnectionTimeout(20000);
params.setSoTimeout(15000);
Aditionally, if you still face timeouts despite increasing the timeout limits, it is a good practice to handle the SocketTimeoutException gracefully - for example by retrying the connection a second and third time.