We Keep Coding

HTTP GET request in java returns meaningless data from App Engine

I'm requesting a json file from an App Engine URL
http://1-1-26a.wordbuzzweb.appspot.com/json/level-images.json
The file encoding is UTF-8 without a BOM. If I look at this file on my local disk it's size is 12414 bytes. If I get the file in Chrome is reads it perfectly well. If I then save it it's 12414 bytes. However, if I try and download the file with a GET request in java I only get 780 bytes returned and the returned data would appear to be meaningless.
I've tried several different types of get request, both of the methods below I have used elsewhere perfectly effectively. The response code on the GET requests is 200. Interestingly, if I do a POST with no content instead of a GET, then I get the valid response.
If I download the file from this URL on Google Drive instead, then the GET methods below work perfectly.
edit This code is now working, however, this is a recurring issue that comes and goes. If anyone has any ideas what might be causing it then please say so!
This doesn't work
public static String doGetSync(String urlToRead) throws IOException {
URL url = new URL(urlToRead);
InputStream is = url.openStream();
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[BUFFER_SIZE];
while ((nRead = is.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
byte[] bytes = buffer.toByteArray();
return new String(bytes, "UTF-8");
}
Neither does this
public static String doGetSync2(String urlToRead) throws IOException {
final String charset = "UTF-8";
// Create the connection
HttpURLConnection connection = (HttpURLConnection) new URL(urlToRead).openConnection();
// Check the error stream first, if this is null then there have been no issues with the request
InputStream inputStream = connection.getErrorStream();
if (inputStream == null)
inputStream = connection.getInputStream();
// Read everything from our stream
BufferedReader responseReader = new BufferedReader(new InputStreamReader(inputStream, charset));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = responseReader.readLine()) != null) {
response.append(inputLine);
}
responseReader.close();
return response.toString();
}
This code works
public static String doPostSync(final String url, final String content) throws IOException {
final String charset = "UTF-8";
// Create the connection
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
// setDoOutput(true) implicitly set's the request type to POST
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-type", "application/json");
// Write to the connection
OutputStream output = connection.getOutputStream();
output.write(content.getBytes(charset));
output.close();
// Check the error stream first, if this is null then there have been no issues with the request
InputStream inputStream = connection.getErrorStream();
if (inputStream == null)
inputStream = connection.getInputStream();
// Read everything from our stream
BufferedReader responseReader = new BufferedReader(new InputStreamReader(inputStream, charset));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = responseReader.readLine()) != null) {
response.append(inputLine);
}
responseReader.close();
return response.toString();
}

Android Webview app not downloading successful

My Android app has a Webview to access to my website. I noticed in the server that when a file is downloaded by the app the bandwidth used is less than when is downloaded by another device or browser.
In method onDownloadStart I call to an AsyncTask class:
protected String doInBackground(String... sUrl) {
try {
URL url = new URL(sUrl[0]);
//Getting directory to store the file
//Connection handler
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoOutput(true);
connection.connect();
//Obtaining filename
File outputFile = new File(directory, filename);
InputStream input = new BufferedInputStream(connection.getInputStream());
OutputStream output = new FileOutputStream(outputFile);
byte buffer[] = new byte[1024];
int bufferLength = 0;
int total = 0;
while ((bufferLength=input.read(buffer))!=-1) {
total += bufferLength;
output.write(buffer, 0, bufferLength);
}
connection.disconnect();
output.flush();
output.close();
input.close();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
Files downloaded are empty altough their filename and format are correct and I receive HTTP 200 message from the server; also execution does not enter into the while loop. I have tried to change buffer size and the problem is not solved.

Android Partially Downloaded Files Report Wrong File Size

I'm putting together some code to download files from an HTTP address in Android. I'd like to support download resumption if the download fails mid way.
The output I get when starting the download, then killing the wifi connection and restarting again several times is the following:
Start size 0
Stop size 12333416
Start size 12333416
Stop size 16058200
Start size 3724784
I cannot understand why after the first resumption, subsequent file size readings of the partially downloaded file do not match.
Thanks in advance!
public void download(String source, String target) throws IOException {
BufferedOutputStream outputStream = null;
InputStream inputStream = null;
try {
File targetFile = new File(target);
currentBytes = targetFile.length();
Log.i(TAG, "Start size " + String.valueOf(currentBytes));
outputStream = new BufferedOutputStream(new FileOutputStream(targetFile));
// create the input stream
URLConnection connection = (new URL(source)).openConnection();
connection.setConnectTimeout(mCoTimeout);
connection.setReadTimeout(mSoTimeout);
inputStream = connection.getInputStream();
inputStream.skip(currentBytes);
// calculate the total bytes
totalBytes = connection.getContentLength();
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) > 0) {
// write the bytes to file
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
currentBytes += bytesRead;
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (outputStream != null) {
// close the output stream
outputStream.flush();
outputStream.close();
}
if (inputStream != null) {
// close the input stream
inputStream.close();
}
Log.i(TAG, "Stop size " + String.valueOf(currentBytes));
}
}

There are two things you are doing wrong:
To resume download to file you should append, not rewrite the file. Use special constructor for output stream:
FileOutputStream(targetFile, true)
To request part of file from server you should use HTTP 1.1 property "Range". You can do it like this:
HttpURLConnection httpConnection = (HttpURLConnection) connection;
connection.setRequestProperty("Range", "bytes=" + currentBytes + "-");

Java - Download zip file from url

I have a problem with downloading a zip file from an url.
It works well with firefox but with my app I have a 404.
Here is my code
URL url = new URL(reportInfo.getURI().toString());
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
// Check for errors
int responseCode = con.getResponseCode();
InputStream inputStream;
if (responseCode == HttpURLConnection.HTTP_OK) {
inputStream = con.getInputStream();
} else {
inputStream = con.getErrorStream();
}
OutputStream output = new FileOutputStream("test.zip");
// Process the response
BufferedReader reader;
String line = null;
reader = new BufferedReader(new InputStreamReader(inputStream));
while ((line = reader.readLine()) != null) {
output.write(line.getBytes());
}
output.close();
inputStream.close();
Any idea ?

In Java 7, the easiest way to save a URL to a file is:
try (InputStream stream = con.getInputStream()) {
Files.copy(stream, Paths.get("test.zip"));
}

As for why you're getting a 404 - that hard to tell. You should check the value of url, which as greedybuddha says, you should get via URI.getURL(). But it's also possible that the server is using a user agent check or something similar to determine whether or not to give you the resource. You could try with something like cURL to fetch in programmatic way but without having to write any code yourself.
However, there another problem looming. It's a zip file. That's binary data. But you're using InputStreamReader, which is designed for text content. Don't do that. You should never use a Reader for binary data. Just use the InputStream:
byte[] buffer = new byte[8 * 1024]; // Or whatever
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) > 0) {
output.write(buffer, 0, bytesRead);
}
Note that you should close the streams in finally blocks, or use the try-with-resources statement if you're using Java 7.

java.net.sockettimeout exception: Read Timed Out

I am reading the "interests" of a user using Facebook API.
When I am trying to read the URL using the following code, I am getting java.net.sockettimeoutexception:
public static String readURL(URL url) throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
URLConnection con = url.openConnection();
con.setConnectTimeout(20000);
con.setReadTimeout(20000);
InputStream is = con.getInputStream();
int r;
while ((r = is.read()) != -1) {
baos.write(r);
}
return new String(baos.toByteArray());
}
How can I handle this now? What should I do?

Increase the values of:
con.setConnectTimeout(20000);
con.setReadTimeout(20000);
and add a catch for that exception.