We Keep Coding

How to test if Android HttpURLConnection is valid?

My app regularly downloads files from a server using HttpUrlConnection(). Brief code example below:
HttpURLConnection conn;
url = new URL( "http://google.com/index.html" );
conn = url.openConnection(); // returns non-null
int rcode = conn.getResponseCode(): // returns 200
// get input stream, read from it and process bytes read
This works just fine. But if I substitute a bogus URL, e.g.
url=new URL("http://BOGUS-SITE.com/index.html";
it still connects just fine, and getResponseCode() returns response code 200. The input stream reader returns -1 bytes read. OK, so be it. (Interestingly, if the filename part of the URL is bogus, I do get a File-Not-Found exception).
But how can I detect a bad connection (e.g., to non-existent host) before I actually try to read it? Maybe that's not possible?
I suppose I could parse the URL and try to resolve the site name or ping it, but that seems like a hack.
More complete code excerpt:
// Download a file by its URL
public static int
doDownload(
String fileurl) // file url, e.g. "http:google.com/index.html"
{
// NOTE: 'log()' is a wrapper for 'Log.i()'
int BUFSIZE=10000;
HttpURLConnection hconn;
int rcode,nr,nrtot=0;
InputStream is;
BufferedInputStream bis;
byte[] buf;
URL url;
try {
url = new URL( fileurl ); // form URL
hconn = (HttpURLConnection)url.openConnection(); // open connection
log( "Opened connection to \""+fileurl+"\"" );
rcode = hconn.getResponseCode(); // get response code
log("Read HTTP response code: " + rcode);
is = hconn.getInputStream();
bis = new BufferedInputStream( is ); // get buffered stream to read
buf = new byte[BUFSIZE];
while( true ) { // read loop
nr = bis.read( buf, 0, BUFSIZE ); // read some bytes
if( nr <= 0 ) break; // break read loop on EOF
nrtot += nr; // update total read count
}
}
catch( Exception e ) {
return( -1 ); // rtn ERROR
}
return( nrtot ); // return num bytes read
}
Update
I have done some further investigation. I found why sometimes response code 200 is returned and sometimes UnknownHostException occurs: It depends on the particular "bad" url host specified. For example if I specify
"http://google.com/index.html", code 200 is returned and the download succeeds.
If I specify ""http://google.comXXX/index.html", 200 is still returned but the download fails (no bytes read). In this case, a different IP address is reported, which nslookup reports as belonging to "akamaitechnologies.com" (???).
In other cases, UnknownHostException occurs.
I'm now trying other "bad" URLs to try to see a pattern.

I found the problem, it was a DNS issue. I discovered my AT&T phone was using DNS service by "sbcglobal.net" (AT&T's default DNS server). That DNS server returns an IP address even for a non-existent name. In particular, it returns an address belonging to "akamaitechnologies.com" (whatever that is). Since that is an existing site, http connects and getResponseCode returns 200. Since it cannot serve my requested file, the download fails. I think this is a marketing gimmick to generate traffic for akamaitechnologies.
When I set my phone to use a DNS of "dns.google" (8.8.8.8), everything works as expected.
This type of DNS spoofing is a Bad Thing because many apps depend on an Unknown-Host-Exception to detect an incorrectly entered domain name, e.g. in an email address.

HttpURLConnection FileNotFoundException on large request properties

I'm using HttpURLConnection to send JSON data from an Android Application to my Tomcat Server.
The POST works fine with small sized JSONs. On bigger data sets it fails with a FileNotFoundException.
What can it be?
Here's the code:
try {
URL url = new URL(urlIn);
strOut = "";
huc = (HttpURLConnection) url.openConnection();
huc.setRequestProperty("Connection", "Close");
huc.setRequestMethod("POST");
huc.setRequestProperty("User", userId);
huc.setRequestProperty("Action", action);
huc.setRequestProperty("JSON", jsonData);
huc.setConnectTimeout(10000);
in = new BufferedReader(new InputStreamReader(huc.getInputStream()));
while ((inputLine = in.readLine()) != null){
if (strOut.equalsIgnoreCase("")){
strOut = inputLine;
} else {
strOut = strOut + inputLine;
}
}
} catch (Exception e) {
strOut = "";
e.printStackTrace();
}
When jsonData get to a certain size (arround 10000 chars), the POST fails with the error mentioned. The content of the JSON does not have any special character.
Thanks in advance.
Best regards, Federico.

HTTPUrlConnection throws a FileNotFoundException if the server responds with a 404 response code, so the reason why this happens seems to be located on the server side rather than the client side. Most likely the server is configured to accept request headers up to a particular length and will return an error if that size is exceeded. A short Google-search brought up a couple of results, sizes of 16 KB are mentioned but shorter values are also reasonable.
As I mentioned in my comment to your question, you should change your process to receive the JSON-data (and the other values for User and Action as well BTW) as part of the request body, e.g. as url-encoded query string or as multipart formdata. Both ways are supported by HTTP client libraries you can use or are easily built manually.

After lots of reading and trying I gave up with configuring Tomcat to accept larger headers.
So I convinced the team in charge of the Tomcat app to make a servlet that is able to receive this data in the body, just as Lothar suggested.
Thanks!

How to Pass a File through an HttpURLConnection

I'm trying to get an image hosting on our server available to be displayed on a client. As per the specs of the project:
"When a Client receives such a URL, it must download the
contents (i.e., bytes) of the file referenced by the URL.
Before the Client can display the image to the user, it must first retrieve (i.e., download) the bytes of the
image file from the Server. Similarly, if the Client receives the URL of a known data file or a field help file
from the Server, it must download the content of those files before it can use them."
I'm pretty sure we have the server side stuff down, because if I put the url into a browser it retrieves and displays just fine. So it must be something with the ClientCommunicator class; can you take a look at my code and tell me what the problem is? I've spent hours on this.
Here is the code:
Where I actually call the function to get and display the file: (This part is working properly insofar as it is passing the right information to the server)
JFrame f = new JFrame();
JButton b = (JButton)e.getSource();
ImageIcon image = new ImageIcon(ClientCommunicator.DownloadFile(HOST, PORT, b.getLabel()));
JLabel l = new JLabel(image);
f.add(l);
f.pack();
f.setVisible(true);
From the ClientCommunicator class:
public static byte[] DownloadFile(String hostname, String port, String url){
String image = HttpClientHelper.doGetRequest("http://"+hostname+":"+port+"/"+url, null);
return image.getBytes();
}
The pertinent httpHelper:
public static String doGetRequest(String urlString,Map<String,String> headers){
URL url;
HttpURLConnection connection = null;
try {
//Create connection
url = new URL(urlString);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Language", "en-US");
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
if(connection.getResponseCode() == 500){
return "failed";
}
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer response = new StringBuffer();
while((line = rd.readLine()) != null) {
response.append(line);
}
rd.close();
return response.toString();
} catch (Exception e) {
e.printStackTrace();
return null;
} finally {
if(connection != null) {
connection.disconnect();
}
}
}
After that, it jumps into the server stuff, which as I stated I believe is working correctly because clients such as Chrome can retrieve the file and display it properly. The problem has to be somewhere in here.
I believe that it has to do with the way the bytes are converted into a string and then back, but I do not know how to solve this problem. I've looked at similar problems on StackOverflow and have been unable to apply them to my situation. Any pointers in the right direction would be greatly appreciated.

If your server is sending binary data, you do not want to use an InputStreamReader, or in fact a Reader of any sort. As the Java API indicates, Readers are for reading streams of characters (not bytes) http://docs.oracle.com/javase/7/docs/api/java/io/Reader.html, which means you will run into all sorts of encoding issues.
See this other stack overflow answer for how to read bytes from a stream:
Convert InputStream to byte array in Java

Do your homework.
Isolate the issue. Modify the server side to send only 256 all possible bytes. Do a binary search and reduce it to small set of bytes.
Use http proxy tools to monitor the bytes as they are transmitted. Fiddler in windows world. Find other ones for the *nix environments.
Then see where the problem is happening and google/bing the suspicions or share the result.

Proper way to test if server is up in Java?

What would be the proper way to simply see if a connection to a website/server can be made? I want this for an application I am coding that will just alert me if my website goes offline.
Thanks!

You can use an HttpURLConnection to send a request and check the response body for text that is unique to that page (rather than just checking to see if there's a response at all, just in case an error or maintenance page or something is being served).
Apache Commons has a library that removes a lot of the boiler plate of making Http requests in Java.
I've never done anything like this specifically on Android, but I'd be surprised if it's any different.
Here's a quick example:
URL url = new URL(URL_TO_APPLICATION);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
InputStream stream = connection.getInputStream();
Scanner scanner = new Scanner(stream); // You can read the stream however you want. Scanner was just an easy example
boolean found = false;
while(scanner.hasNext()) {
String next = scanner.next();
if(TOKEN.equals(next)) {
found = true;
break;
}
}
if(found) {
doSomethingAwesome();
} else {
throw aFit();
}

You want to also set the connection timeout using setConnectTimeout(int timeout) and setReadTimeout(int timeout). Otherwise the code might hang for a long time waiting for a non-responding server to reply.

HttpsURLConnection Connection Problems

I'm a problem with a HttpsURLConnection that I can't seem to solve. Basically, I'm sending up some info to a server and if some of that data is wrong, the server sends me a 500 response code. However, it also sends a message in the response telling me which bit of data was wrong. The problem is that the message is always empty when I read it in. I think this is because a filenotfound exception always gets thrown before the stream can be read. Am I right? I tried reading the errorstream as well but this is always empty. Here's a snippet:
conn = (HttpsURLConnection) connectURL.openConnection();
conn.setDoOutput(true);
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Length",
Integer.toString(outString.getBytes().length));
DataOutputStream wr = new DataOutputStream(conn
.getOutputStream());
wr.write(outString.getBytes());
wr.flush();
wr.close();
if(conn.getResponseCode>400{
String response = getErrorResponse(conn);
public String getErrorResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
//is = conn.getInputStream();
is = conn.getErrorStream();
// scoop up the reply from the server
int ch;
StringBuffer sb = new StringBuffer();
while ((ch = is.read()) != -1) {
sb.append((char) ch);
}
//System.out.println(sb.toString());
return sb.toString();
// return conferenceId;
}
catch (Exception e){
e.printStackTrace();
}
}

So just to follow up on this, here is how I solved it:
public static String getResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
if(conn.getResponseCode()>=400){
is = conn.getErrorStream();
}
else{
is=conn.getInputStream();
}
...read stream...
}
It seems that calling them like this produced an error stream with a message. Thanks for the suggestions!

Try setting content-type request property to "application/x-www-form-urlencoded"
The same is mentioned on this link:
http://developers.sun.com/mobility/midp/ttips/HTTPPost/
The Content-Length and Content-Type headers are critical because they tell the web server how many bytes of data to expect, and what kind, identified by a MIME type.
In MIDP clients the two most popular MIME types are application/octet-stream, to send raw binary data, and application/x-www-form-urlencoded, to send name-value pairs

Are you in control of the server? In other words, did you write the process that runs on the server and listens to the port you're trying to access?
If you did, then you should also be able to debug it and see why your process returns 404.
If you didn't, then describe your architecture (HTTP server, the component it invokes to respond to your HTTP(S) request, etc) and we'll take it from there.
In the very simplest case, of an HTTP server being an Apache server yielding control to some PHP script, it means that Apache couldn't assign your request to anything. Most likely a Web server misconfiguration. Provide some more details and we'll help you out.