public HttpResponseBean get(String url, Map<String, String> headers) throws Exception {
logger.debug("Sending get request...");
HttpClient httpClient = null;
try {
int timeout = 30 * 1000; // 30 seconds
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(timeout)
.setConnectionRequestTimeout(timeout)
.setSocketTimeout(timeout).build();
httpClient = HttpClients.custom()
.setDefaultRequestConfig(requestConfig).build();
HttpGet httpGetRequest = new HttpGet(url);
if (headers != null) {
for (Entry<String, String> entry: headers.entrySet()) {
httpGetRequest.addHeader(new BasicHeader(entry.getKey(), entry.getValue()));
}
}
HttpResponse response = httpClient.execute(httpGetRequest);
HttpResponseBean hrb = new HttpResponseBean(response);
logger.debug("Get response: Response: " + hrb.toString());
return hrb;
} finally {
closeConnection(httpClient);
}
}
This works well most of the time...but once in a while it gets stuck on handshake and will take forever until the server(tomcat) is restarted.
As per this link looks like a bug -> apache httpclient 4.3 not timing out
Is there a way out of this? I am using httpclient 4.4.1
Is there a way out of this? I am using httpclient 4.4.1
Here's the associated bug on the Apache site. It looks like people have had problems with the 4.4.1 version:
I had this issue with version 4.4.1. At first I ignored this thread because it was flagged as resolved in 4.3.
But this seemed to have been resolved in the 4.5.1 version. I'd encourage you to upgrade if possible.
I encountered this issue on version 4.3.4. I upgraded to 4.5.1 and the issue was fixed there.
Here's additional information:
In version 4.3.4, http:// worked fine and timed out after 1 second. With https, the request hangs beyond 1 seconds and hangs until the [...] server closes the TCP connection.
In version 4.5.1, the [correct] http behaviour was identical, and on https I got the following exception: [...] org.apache.http.conn.ConnectTimeoutException: Connect to localhost:6171 [localhost/127.0.0.1] failed: Read timed out
Hope this helps.
Related
Please suggest a way to close httpclient connection, it causes hung thread generated in server.
org.apache.commons.httpclient.HttpClient
HttpClientManager httpClientManager;
HttpClient httpclient = httpClientManager.getHttpClient();
PostMethod p = new PostMethod(url);
try{
httpclient.executeMethod(p);
}
catch(HttpException e){}
finally{
httpclient.close(); //not available in HttpClient
}
Using old 3.1 version of jar where there is no method in HttpClient class to close httpclient connection.
Due to connection is not closing in the end , it causes hung thread in server.
I am using httpClient version 4.5 to connect with our external vendor site. We do not need any connection pool or persistent connection, so I am using the BasicHttpClientConnectionManager to create the HttpClient.
This works fine for minimal number of requests, but if I test this for 1TPS for 1 hour, by the end of the test, we start seeing intermittent connection resets. (guessing request count > 100)
I/O exception (java.net.SocketException) caught when processing request to {s}->https://apiURL:443: Connection reset
Please find below the code snippet for making connection.
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), new X509TrustManager[] { new DefaultTrustManager() }, new SecureRandom());
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, new String[] { "TLSv1.2" }, null,SSLConnectionSocketFactory.getDefaultHostnameVerifier());
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslsf).register("http", new PlainConnectionSocketFactory()).build();
HttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager(socketFactoryRegistry);
HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(1, false);
RequestConfig defaultRequestConfig = RequestConfig.custom().setSocketTimeout(Integer.parseInt(30000)).setConnectTimeout(Integer.parseInt(30000)).setConnectionRequestTimeout(30000).setCookieSpec(CookieSpecs.STANDARD).build();
CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(connectionManager).setDefaultRequestConfig(defaultRequestConfig).setRetryHandler(retryHandler).evictExpiredConnections().build();
HttpPost httpPost = new HttpPost(<endpoint>);
httpPost.setEntity(new UrlEncodedFormEntity(requestData));
httpResponse = httpClient.execute(httpPost);
I saw that fix for a similar issue reported is already available with version 4.5. (Ref: https://issues.apache.org/jira/browse/HTTPCLIENT-1655) provided by Oleg
If thats the case, not sure why I am still facing this issue. Could someone please help on this.
Thanks!
Hi Oleg,
I had been using the httpclient version 4.5.3 when I was still seeing the connection reset errors as posted above.
Later noticed that the fix for reset issue was committed to the version 4.5.1 (https://issues.apache.org/jira/browse/HTTPCLIENT-1655). So, just tried updating that particular version, ran a test and not seeing the connection reset errors any more.
I had expected that this fix should also be available in the higher versions starting from 4.5.1. But, I guess its somehow missed out in the higher versions, have verified that its still an issue with version 4.5.3 for sure.
So, conclusion is that the connection reset error was fixed using httpclient 4.5.1 jar.
Thanks!
I have a code in my java application that connects to a url with a post request and get response.
The code worked for a long time, and it still does for most people. I have one person that the code works for him most of the times, but sometimes I am getting timeout exception.
The code:
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost("http://example.com/link-to-api");
post.addHeader( "Data1", "my_data" );
HttpResponse response = null;
BufferedReader rd = null;
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("post_key1","post_data1"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
response = client.execute(post);
} catch (IOException e) {
System.out.println("error: "+ e);
}
The exception that I get is:
org.apache.http.conn.HttpHostConnectException: Connect to example.com:80 [example.com/__IP__] failed: Connection timed out: connect
I sniffed the network, and I can see the packets when the connection is ok, but every few minutes, when I get the above exception, I don't even see the attempt connection in the sniffer.
I did ping and traceroute to the server while the timeout is occuring, and both are ok (I get response from the server)
So it seems to me that the problem is in the java application, not even attempting the connection.
I am using httpclient 4.4.1, if it is of any importance.
Why I am getting these timeouts?
I solved it . At least in my case, all I did was reset the router on my computer, and the problem disappeared.
I have read on the internet many suggestions to change server's tcp/ip parameters. So if this happens to you, before you do anything, I suggest that you check your hardware as well.
This problem has blocked our whole team half a day!
We use apache httpclient 4.3.x to post and get data from an storage server which provides http api. In order to improve performance, we used PoolingHttpClientConnectionManager:
public HttpClient createHttpClient() {
Registry registry = RegistryBuilder.create()....build();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
connectionManager.setMaxTotal(50);
connectionManager.setDefaultMaxPerRoute(50);
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.build();
return httpClient;
}
Then we hold an instance of the httpClient in our program, reuse it with every http request:
Global httpClient:
HttpClient httpClient = createHttpClient();
Post some data:
HttpPost httpPut = new HttpPost("...");
HttpResponse response = httpClient.execute(httpPut);
// Notice we get the response content here!
String content = EntityUtils.toString(response.getEntity());
System.out.println(content);
httpPut.releaseConnection();
response.close();
Then get:
HttpGet httpGet = new HttpGet("...");
// Blocked at this line !!!!
HttpResponse response = httpClient.execute(httpGet);
String content = EntityUtils.toString(response.getEntity());
System.out.println(content);
httpPut.releaseConnection();
response.close();
Please notice the line: // Blocked at this line !!!!
The program has blocked at that line and never go to next line. In debugging mode, I can see it has been blocked at:
SocketInputStream.socketRead0()
I've searched for a lot of questions and documents, but no lucky.
My colleage just fix it by setting NoConnectionReuseStrategy.INSTANCE:
HttpClients.custom()
.setConnectionManager(connectionManager)
// Following line fixed the problem, but why?
.setConnectionReuseStrategy(NoConnectionReuseStrategy.INSTANCE)
.build();
Now it doens't blocked, but why?
What does "reuse connection" mean? And is there performance issue by using NoConnectionReuseStrategy?
Thank you, guys~
I tried to reproduce the blocking http-get (also as an exercise for myself) but even without closing responses I could not get it to block. The ONLY time I managed to make the http-get block is by doing a response.getEntity().getContent() without reading from the returned InputStream and without closing the returned InputStream.
For my tests I used Tomcat 7.0.47 with two very simple servlets (one responding "OK" to a get, the other echoing a post) as a server. The client started 50 threads with each thread performing 30 alternating http-get and http-post request (total of 1500 requests). The client did not use the RegistryBuilder, instead the default one is used (created by the PoolingHttpClientConnectionManager itself).
About the NoConnectionReuseStrategy: by default (HttpClient created with HttpClients.createDefault(), I used org.apache.httpcomponents:httpclient:4.3.1) a connection pool is used with a maximum of 2 connections to 1 server. E.g. even if 5 threads are doing all kinds of requests at the same time to 1 server, the connection pool opens only 2 connections, re-uses them for all requests and ensures that 1 connection is used by 1 thread at any given time. This can have a very positive impact on client performance and significantly reduces load on the server. The only thing you must make sure is to call response.close() in a finally-block (this ensures the connection is returned to the connection pool). By using the NoConnectionReuseStrategy you basically disable the connection pool: for each request a new connection will be created. I recommend you enable debug-logging for category org.apache.http.impl.conn.PoolingHttpClientConnectionManager, it is very informative.
A note about httpPut.releaseConnection(): this does not actually release a connection, it only ensures that you can re-use the "httpPut" object in a next request (see the apidocs, follow the shown link). Also note that in your code for the "httpGet", you call releaseConnection() on "httpPut" instead of "httpGet".
Ran into this problem just a while back. In case someone else comes across this problem, this post might be useful.
I am using a Java Servlet to service my requests. When I wrote to the response stream using the PrintWriter instance my client blocked. Tried writing to the OutputStream directlyresponse.getOutputStream.write("myresponse") and it worked.
trying to access http://forum.worldoftanks.eu/index.php?app=members using apache HttpClient but keep getting 403. Can anyone help out?
Been fiddling with this piece as a starting point:
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpRequestBase method = new HttpGet(theUrl);
String s = httpClient.execute(method, new BasicResponseHandler());
System.out.println(s);
httpClient.getConnectionManager().shutdown();
I don't think this is related to HttpClient. I tried this
$ wget http://forum.worldoftanks.eu/index.php?app=members
--2011-08-08 23:17:52-- http://forum.worldoftanks.eu/index.php?app=members
Resolving forum.worldoftanks.eu (forum.worldoftanks.eu)... 213.252.177.21, 213.2
52.177.20
Connecting to forum.worldoftanks.eu (forum.worldoftanks.eu)|213.252.177.21|:80..
. connected.
HTTP request sent, awaiting response... 403 Forbidden
2011-08-08 23:17:56 ERROR 403: Forbidden.
with no luck.
Yet I can hit it in the browser. It might be that there is some server logic returning 403s when an appropriate browser headers aren't sent. My next step would be to use FireBug and try to replicate the request as your browser makes it.
Also, try catching the exceptino
} catch (HttpResponseException e) {
System.err.println(e.response.parseAsString());
}