I am trying to write a simple Http client application in Java and am a bit confused by the seemingly different ways to establish Http client connections, and efficiently reuse the objects.
Current I am using the following steps (I have left out exception handling for simplicity)
Iterator<URI> uriIterator = someURIs();
HttpClient client = new DefaultHttpClient();
while (uriIterator.hasNext()) {
URI uri = uriIterator.next();
HttpGet request = new HttpGet(uri);
HttpResponse response = client.execute(request);
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
processStream (content );
content.close();
}
In regard to the code above, my questions is:
Assuming all URI's are pointing to the same host (but different resources on that host). What is the recommended way to use a single http connection for all requests?
And how do you close the connection after the last request?
--edit:
I am confused at why the above steps never use HttpURLConnection, I would assume client.execute() creates one, but since I never see it I am not sure how to close it or reuse it.
To make use of persistent connection efficiently, you need to use the pooled connection manager,
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(
new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
ClientConnectionManager cm = new ThreadSafeClientConnManager(schemeRegistry);
HttpClient httpClient = new DefaultHttpClient(cm);
My biggest problem with HttpURLConnection is its support for persistent connection (keep-alive) is very buggy.
Related
There is a file that will be downloaded when I make a get request to particular URL. I am able to get InputStream from both ways.
Method 1
Using URL class in java.net package.
java.net.URL url = new URL(downloadFileUrl);
InputStream inputStream = url.openStream();
Method 2
Using Apache's HttpClient class.
org.apache.http.impl.client.CloseableHttpClient httpclient = new CloseableHttpClient();
HttpGet request = new HttpGet(url);
CloseableHttpResponse response = httpclient.execute((HttpUriRequest)request);
InputStream inputStream = response.getEntity().getContent();
Are these methods the same? If not how? Which method is preferred generally or in a specific situation?
The examples I provided are simplistic. Assume I did the neccessary
congifurations with the URL and HttpClient objects to get successful response.
Both methods returns the input stream to read from the connection. There isn't difference between these methods. Since HttpClient is third party library, you need to keep a check for any vulnerabilities and keep updating the library.
Only difference is HttpClient supports only HTTP(s) protocol, whereas URLConnection can be used for other protocols too like FTP
In terms of functionalities, Apache HttpClient has more fine tuning options than URLConnection
Looking at the quick start guide it gives the following code example:
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://targethost/homepage");
CloseableHttpResponse response1 = httpclient.execute(httpGet);
// The underlying HTTP connection is still held by the response object
// to allow the response content to be streamed directly from the network socket.
// In order to ensure correct deallocation of system resources
// the user MUST call CloseableHttpResponse#close() from a finally clause.
// Please note that if response content is not fully consumed the underlying
// connection cannot be safely re-used and will be shut down and discarded
// by the connection manager.
try {
System.out.println(response1.getStatusLine());
HttpEntity entity1 = response1.getEntity();
// do something useful with the response body
// and ensure it is fully consumed
EntityUtils.consume(entity1);
} finally {
response1.close();
}
The two comments in the code above say that we must close the response object for
"correct deallocation of system resources"
and
"if response content is not fully consumed the underlying connection cannot be safely re-used and will be shut down and discarded by the connection manager".
Now Apache have very kindly implementend a CloseableHttpResponse for us, which means we can use a try-with-resources block. But the close method only closes the response object, why doesn't it also consume the entity?
Because it is hard to say at that point whether or not the caller intends to re-use the underlying connection. In some cases one may want to read just a small chunk from a large response body and immediately terminate the connection.
In other words, the same thing happens over and over again: there is no one way that can make everyone happy.
The code snippet will do ensure proper de-allocation of resources while trying to keep the underlying connection alive.
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://targethost/homepage");
CloseableHttpResponse response1 = httpclient.execute(httpGet);
try {
System.out.println(response1.getStatusLine());
} finally {
EntityUtils.consume(response1.getEntity());
}
This is scenario I am testing. The steps are as follows
Create an instance of HttpClient(org.apache.commons.httpclient.HttpClient)3.1
Use the instance to get response from a url (http://www.example.com/submitData)
Bring down the server hosting the url mentioned in step 2.
Bring up the url
Wait till the url is up
Use the instance of HttpClient created in step 1 to get response from the url (http://www.example.com/getData)
I get an error - Connection Reset.
Can anyone help me understand.
I have used the following params in the HttpClient
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
httpClient.getParams().setParameter("http.connection.stalecheck", new Boolean(true));
Edited after comments from Peter
This is how I create the HttpClient
HttpClient httpClient = new HttpClient();
AuthScope authScope = new AuthScope("www.example.com", 80, AuthScope.ANY_REALM);
Credentials defaultcreds = new UsernamePasswordCredentials("username", "pwd");
httpClient.getState().setCredentials(authScope, defaultcreds);
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
httpClient.getParams().setParameter("http.connection.stalecheck", new Boolean(true));
To establish connection, I use the following
GetMethod getMethod = new GetMethod("http://"+httpClient.getHostConfiguration().getHost()+"/getData");
int statusCode = httpClient.executeMethod(getMethod);
I want the httpclient instance to preserve the hostname and port and credentials.
This is exact execption message that I get - "Connection reset".
More info - I am executing the test from eclipse. If I run the test case in debug mode, I dont get this exception.
I think the behaviour is reasonable because when the server goes down the client socket on the server goes away too. The client still holds old socket but this socket is gone on the server. You should reconnect from client.
AFAIK the stale parameter allows the client to close the connection the clean way (without Exception) but not prevent from disconnection.
This looks suspicious
httpClient.getParams().setParameter("http.socket.timeout", new Integer(0));
Having a timeout set to 0ms would likely terminate the request prematurely?
I am working on a Spring Web MVC project and I have a requirement where I need to hit a URL and retrieve the CSV from it.
When I manually hit the URL, it prompts me to download the CSV to a file. I've been told this is just because of the browser.
I've also been advised to use Apache HTTP Components...
So I have this right now
DefaultHttpClient client = new DefaultHttpClient();
client.getCredentialsProvider().setCredentials(
new AuthScope(URL, PORTNUM),
new UsernamePasswordCredentials(username, password));
AuthCache authCache = new BasicAuthCache();
BasicScheme basicAuth = new BasicScheme();
authCache.put( new HttpHost( URL, PORTNUM), basicAuth);
BasicHttpContext localContext = new BasicHttpContext();
localContext.setAttribute( ClientContext.AUTH_CACHE, authCache);
HttpResponse response = client.execute(get);
HttpEntity entity = response.getEntity();
I've also been trying to follow another project my company has done to authenticate against the page, but it is slightly different.
This isn't providing me the results I need (of course...) since it is incomplete.
I need to return the response into an InputStream...
Maybe I am completely wrong about this.
HttpEntity provides the method getContent() which returns an InputStream.
You can use
InputStream is = entity.getContent()
and continue from there.
ByteArrayInputStream bais = new ByteArrayInputStream(EntityUtils.toByteArray(entity));
Using EntityUtils you can transform the entity to byte array and then create ByteArrayInputStream which inherits InputStream.
Can you not just add
InputStream inputStream = entity.getContent();
to what you have?
I am developing one application which is connecting to server to get some data.
In this I want to check first if application is connected to server or not. And then, if server is on or off? Based on the result I want to do my further manipulations.
So how do I get the result of server status?
Here is the code which I am using:
Code:
try {
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(
"http://192.168.1.23/sip_chat_api/getcountry.php");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
is = entity.getContent();
} catch (Exception e) {
}
Maintaining session cookies is best choice here, please see how to use session cookie: How do I make an http request using cookies on Android?
here, before sending request to server, check for session cookie. If it exists, proceed for the communication.
Update:
The Java equivalent -- which I believe works on Android -- should be:
InetAddress.getByName(host).isReachable(timeOut)
Check getStatusLine() method of HttpResponse
any status code other than 200 means there is a problem , and each status codes points to different problems happened.
http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpResponse.html?is-external=true
http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/StatusLine.html#getStatusCode()