If I create an HTTP java.net.URL and then call openConnection() on it, does it necessarily imply that an HTTP post is going to happen? I know that openStream() implies a GET. If so, how do you perform one of the other HTTP verbs without having to work with the raw socket layer?
If you retrieve the URLConnection object using openConnection() it doesn't actually start communicating with the server. That doesn't happen until you get the stream from the URLConnection(). When you first get the connection you can add/change headers and other connection properties before actually opening it.
URLConnection's life cycle is a bit odd. It doesn't send the headers to the server until you've gotten one of the streams. If you just get the input stream then I believe it does a GET, sends the headers, then lets you read the output. If you get the output stream then I believe it sends it as a POST, as it assumes you'll be writing data to it (You may need to call setDoOutput(true) for the output stream to work). As soon as you get the input stream the output stream is closed and it waits for the response from the server.
For example, this should do a POST:
URL myURL = new URL("http://example.com/my/path");
URLConnection conn = myURL.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
OutputStream os = conn.getOutputStream();
os.write("Hi there!");
os.close();
InputStream is = conn.getInputStream();
// read stuff here
While this would do a GET:
URL myURL = new URL("http://example.com/my/path");
URLConnection conn = myURL.openConnection();
conn.setDoOutput(false);
conn.setDoInput(true);
InputStream is = conn.getInputStream();
// read stuff here
URLConnection will also do other weird things. If the server specifies a content length then URLConnection will keep the underlying input stream open until it receives that much data, even if you explicitly close it. This caused a lot of problems for us as it made shutting our client down cleanly a bit hard, as the URLConnection would keep the network connection open. This probably probably exists even if you just use getStream() though.
No it does not. But if the protocol of the URL is HTTP, you'll get a HttpURLConnection as a return object. This class has a setRequestMethod method to specify which HTTP method you want to use.
If you want to do more sophisticated stuff you're probably better off using a library like Jakarta HttpClient.
Related
I have the following code:
HttpURLConnection conn = null;
BufferedReader in = null;
StringBuilder sb = null;
InputStream is = null;
conn = (HttpURLConnection) url.openConnection();
// Break-point A
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setRequestMethod("POST");
// Break-point B
conn.setRequestProperty("X-TP-APP", Constants.X_TP_APP);
conn.setRequestProperty("X-TP-DEVICE", Constants.X_TP_DEVICE);
conn.setRequestProperty("X-TP-LOCALE", Constants.X_TP_LOCALE);
conn.setRequestProperty("Content-Type", contentType);
conn.setRequestProperty("Accept", accept);
conn.setRequestProperty("Authorization", SystemApi.TOKEN_STR);
conn.setUseCaches(false);
conn.setConnectTimeout(30000);
conn.getOutputStream().write(req.getBytes("UTF-8"));
conn.getOutputStream().flush();
conn.getOutputStream().close();
is = conn.getInputStream();
in = new BufferedReader(new InputStreamReader(is));
int statusCode = conn.getResponseCode();
// Break-point C
The code is running fine without problem (when breakpoint(A,B) is disabled)
I tried to find out when does HttpURLConnection really call the request and place breakpoint(A) after conn = getConnection(strURL);
and continue the code, but then at the end, at breakpoint(C), server would return me 401 - Unauthorized, which mean my Authorization header is not in the request.
It seem like that we are trying to open a connection first, and then set the header as fast as we can. If we are not fast enough, then the request is called anyway, which doesn't seem right.
My question and concern:
When does HttpURLConnection really call the request?
Is this what is actually happening? Is this the correct way to do so?
Is there a better way to make sure the header is set before calling the request?
Per the docs, the actual connection is made when the connect() method is invoked on the [Http]UrlConnection. That may be done manually, or it may be done implicitly by certain other methods. The Javadocs for UrlConnection.connect() say, in part:
URLConnection objects go through two phases: first they are created, then they are connected. After being created, and before being connected, various options can be specified (e.g., doInput and UseCaches). After connecting, it is an error to try to set them. Operations that depend on being connected, like getContentLength, will implicitly perform the connection, if necessary.
Note in particular the last sentence. I don't see anything in your code that would require the connection to be established until the first conn.getOutputStream(), and I read the docs as saying that the connection object will not enter the "connected" state until some method is invoked on it that requires that. Until such a time, it is ok to set connection properties.
Moreover, the docs definitely state that methods that set properties on the connection (and setRequestProperty() in particular) will throw an IllegalStateException if invoked when the connection object is already connected.
It is possible that your Java library is buggy in the manner you describe, but that would certainly be in conflict with the API specification. I think it's more likely that the explanation for the behavior you observe is different, and I recommend you capture and analyze the actual HTTP traffic to determine what's really going on.
Actually what really happened is, in the debug mode, I used conn.getResponseCode() in the expressions, which force the conn.getResponseCode() to run.
When it is not connected yet, getResponseCode() would calls connect() before the request is prepared.
Hence it would return me 401.
Since Android using the same HttpURLConnection, I did some capture the packet exchange to see what is happening under the hood.
I detailed my experiment in this post Can you explain the HttpURLConnection connection process?
To outline the network activity for your program.
At Breakpoint A No physical connection is made to the remote server. You get a logical handle to a local connection object.
At Breakpoint B You just configure the local connection object, nothing more.
conn.getOutputStream() Network connection starts here, but no payload is transferred to the server.
conn.getInputStream() Payload (http headers, content) are sent to the server, and you get the response (buffered into input stream, and also the response code etc.)
To Answer your question
When does HttpURLConnection really call the request?
getInputStream() triggers network layer to send out application payload and got responses.
Is this what is actually happening? Is this the correct way to do so?
No. openConnection() does not initiate network activity. You are getting back a local handle for future connection, not an active connection.
Is there a better way to make sure the header is set before calling the request?
You don't need to make sure header is set. The header payload isn't sent to the server until you ask for response (such as getting the response code, or opening a inputStream )
I have a task to download & upload a file using HTTP protocol in Android (Java platform).
I am using following code for uploading a file:
HttpURLConnection httpURLConnection = (HttpURLConnection) serverUrl.openConnection();
....
httpURLConnection.connect();
OutputStream os = httpURLConnection.getOutputStream();
And Using following code for downloading a file:
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
...
urlConnection.connect();
DataInputStream stream = new DataInputStream(urlConnection.getInputStream());
As per my observation connect() for both the case takes time because it is communicating with network at this point. And for file upload, getOutputStream() gets execute very fast so does it means it is not communicating to network?
Whereas getInputStream() (in file download) takes some time (around 200 to 2500 mili sec) to execute. Does it mean it is communicating with network at this point? If yes then why so?
Experts, Please provide your comments on this & correct me if I am wrong anywhere.
HTTP is a request/response protocol. You need a TCP connection. The connect() method creates that. Then you need to send a request. You call getOutputStream() for that, and you write it.
At this point nothing has been written to the network (in normal transfer mode), because the content-length header has to be set, and Java doesn't know when you've finished writing. So when you call getInputStream() (or getResponseCode()), Java sets the content-length header, writes the request, waits for the server to start generating a response, reads all the response headers, and then gives you an input stream positioned at the beginning of the body of the response. All those steps take time.
You must limit buffering by specifying the streaming mode either by giving the final length of the uploaded information via setFixedLengthStreamingMode method, or setting mode to streaming if final length is not known via setChunkedStreamingMode method:
// For best performance, you should call either setFixedLengthStreamingMode(int) when the body length is known in advance,
// or setChunkedStreamingMode(int) when it is not. Otherwise HttpURLConnection will be forced to buffer the complete request body in memory
// before it is transmitted, wasting (and possibly exhausting) heap and increasing latency.
//
// see: https://developer.android.com/reference/java/net/HttpURLConnection.html
_connection.setChunkedStreamingMode(1024);
If you don't, the real transfer will occur when you call getInputStream().
See https://developer.android.com/reference/java/net/HttpURLConnection.html
I want to see the exact headers my android app is sending while making a web request so I thought I'd simply create a simple server app in java on my local machine and have my android app make a call to it. Then simply dump the request to the console so I could see what the app is sending. However when I tried to connect, the app hangs and stops responding.
I created a simple server the only accepts a connection and sysouts the data it gets. The server runs fine and if I hit it from a web browser on my computer will print the headers from the web browsers request. So I know the server works fine.
Here's the code from my app:
URL url = new URL("http://192.168.1.11:9000");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.connect();
PrintWriter writer = new PrintWriter(connection.getOuputStream(), true);
writer.write("hi");
writer.close();
Simple. I only want the headers after all. Now I started without a post and using:
URL url = new URL("http://192.168.1.11:9000");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
in.close();
but that doesn't work. The app stops responding on the getInputStream() request. It just stops and won't continue. The server gets no connection request either.
So in all, the app is blocking on the url connection's getInputStream and I can't figure out why.
Now I've searched for awhile and found these:
Android app communicating with server via sockets
socket exception socket not connected android
Android embedded browser cant connect to server on LAN
Using java.net.URLConnection to fire and handle HTTP requests
Client Socket cannot connect to running Socket server
But nothing helps. I'm not using the localhost like everyone with this problem seems to be and I've tried using the androids 10.0.0.2 but that doesnt work either.
I'm not on a network that restricts anything (I'm home) and I've tried using the first set of code shown in order to send a message to my server but not even that works (it runs fine but the server never gets a client. Hows that work?).
I tried using both URLConnection and HttpURLConnection, they both have the same problem.
I'm also using the internet permission in my app, so it does have the permission needed.
I'm at a loss at this point. Why can't I make a simple call to my server?
EDIT
I used the exact code from androids documentation:
private String downloadUrl(String myurl) throws IOException {
InputStream is = null;
// Only display the first 500 characters of the retrieved
// web page content.
int len = 500;
try {
URL url = new URL("http://10.0.2.2:9000");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int response = conn.getResponseCode();
is = conn.getInputStream();
// Convert the InputStream into a string
String contentAsString = readIt(is, len);
return contentAsString;
// Makes sure that the InputStream is closed after the app is
// finished using it.
} finally {
if (is != null) {
is.close();
}
}
}
but even that doesn't work. It still hangs. Only now it hangs on the getResponseCode(). Then throws a timeout exception. The server never gets a request though.
Your address must start with 'http://", try again!
I think the root of your issue is that Android is FCing your app before the connection completes, because I assume you haven't wrapped this in a Loader, AsyncTask or Thread. I suggest you follow the training guide Google provides, wrapping your call in an AsyncTask and seeing if that corrects the issue.
I have a Java class I use for making HTTP GET requests, I'm guessing its near identical to the android code your using so below I've dumped the relevant part of the code. I've used this class many times in Java applications (not on Android).
currentUrl = new URL(getUrl);
conn = (HttpURLConnection)currentUrl.openConnection();
conn.setRequestProperty("Cookie", getCookies(currentUrl.getHost()));
conn.setRequestProperty("User-Agent", "robadob.org/crawler");
if(referrer!=null){conn.setRequestProperty("Referrer",referrer);}
conn.setRequestMethod("GET");
conn.connect();
//Get response
String returnPage = "";
String line;
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
returnPage+=line+"\n";
}
rd.close();
I can't see anything obvious that would be causing your code to fail, but hopefully you can spot something from this. The setRequestProperty is me setting headers, so you shouldn't need those.
If that fails, flood your code with System.out's so you can see which statement its stalling at.
I am sending commands to a server using http, and I currently need to parse a response that the server sends back (I am sending the command via the command line, and the servers response appears in my browser).
There are a lot of resources such as this: Saving a web page to a file in Java, that clearly illustrate how to scrape a page such as cnn.com. However, since this is a response page that is only generated when the camera receives a specific command, my attempts to use the method described by Mike Deck (in the link above) have met with failure. (Specifically, when my program requests the page again the server returns a 401 error.)
The response from the server opens a new tab in my browser. Essentially, I need to know how to save the current web page using java, since reading in a file is probably the most simple way to approach this. Do any of you know how to do this?
TL;DR How do you save the current webpage to a webpage.html or webpage.txt file using java?
EDIT: I used Base64 from the Apache commons codec, which solved my 401 authentication issue. However, I am still getting a 400 error when I attempt to connect my InputStream (see below). Does this mean a connection isn't being established in the first place?
URL url = new URL ("http://"+ipAddress+"/axis-cgi/record/record.cgi?diskid=SD_DISK");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
EDIT 2: Changing the request to a GET resolved the issue.
So while scrutinizing my code above, I decided to change
connection.setRequestMethod("POST");
to
connection.setRequestMethod("GET");
This solved my problem. In hindsight, I think the server was not recognizing the HTTP because it is not set up to handle the various trappings that come along with post.
I recently was experimenting with java networking and I found a bit odd thing, suppose you have
URL url = new URL("http://www.google.com");
URLConnection con = url.openConnection();
then i can call methods, like con.getContentLength() and so on and they will give me correct values, even despite I didn't envoke con.connect(). How can that be? I mean, where from/how does URLConnection gets those headers, I didn't invoke con.connect() yet, so no requests were sent and so no headers should be available at that moment.
The actual TCP connect happens implicitly when you call any method that requires the response, such as getContentLength(), getInputStream(), getResponseCode(). It doesn't happen at openConnection(). The request is sent at that point.
Unless you are using one of the streaming modes and you're doing a PUT or POST with request content, in which case the connection is opened when you start writing the request.