Implementing resume for download files via internet - java

my below code for downloading file work fine when don't implementing resume for that, after read more solution to implementing that and resolve problem i know i must check Last-Modified header and set it for connection,
but i can't do that because of i get error such as android Cannot set request property after connection is made or i get null for httpURLConnection,
i'm using this reference
getHeaderField heaser return:
{
null=[HTTP/1.1 200 OK],
Cache-Control=[public],
Connection=[keep-alive],
Content-Length=[8037404],
Content-Md5=[VEqXHCc/Off7a6D0gRFpiQ==],
Content-Type=[image/jpeg],
Date=[Tue, 19 Jan 2016 07:24:36 GMT],
Etag=["544a971c273f39f7fb6ba0f481116989"],
Expires=[Sat, 29 Jul 2017 10:07:00 GMT],
Last-Modified=[Thu, 18 Dec 2014 08:44:34 GMT],
Server=[bws],
X-Android-Received-Millis=[1501063623576],
X-Android-Response-Source=[NETWORK 200],
X-Android-Selected-Protocol=[http/1.1],
X-Android-Sent-Millis=[1501063623532]
}
now how can i set that to have resume for download files?
GitHub Link
public void run() {
final URL url;
HttpURLConnection httpURLConnection = null;
try {
try {
url = new URL(mUrl);
String lastModified = httpURLConnection.getHeaderField("Last-Modified");
if (!lastModified.isEmpty()) {
httpURLConnection.setRequestProperty("If-Range", lastModified);
}
httpURLConnection = (HttpURLConnection) url.openConnection();
if (mFile.exists()) {
downloadedLength = mFile.length();
Log.e("downloadedLength ", downloadedLength + "");
httpURLConnection.setRequestProperty("Range", "bytes=" + downloadedLength + "-");
fileOutputStream = new FileOutputStream(mFile, true);
} else {
fileOutputStream = new FileOutputStream(mFile);
}
httpURLConnection.setConnectTimeout(30000);
httpURLConnection.setReadTimeout(30000);
httpURLConnection.setRequestMethod("GET");
} catch (IOException e) {
}
final int responseCode;
final int total;
try {
responseCode = httpURLConnection.getResponseCode();
total = httpURLConnection.getContentLength();
} catch (IOException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
if (responseCode == 200) {
try {
inputStream = httpURLConnection.getInputStream();
} catch (IOException e) {
e.printStackTrace();
Log.e("IOException ", e.getMessage());
}
final byte[] buffer = new byte[4 * 1024];
int length = -1;
int finished = 0;
long start = System.currentTimeMillis();
try {
while ((length = inputStream.read(buffer)) != -1) {
if (!isDownloading()) {
throw new CanceledException("canceled");
}
fileOutputStream.write(buffer, 0, length);
finished += length;
if (System.currentTimeMillis() - start > 1000) {
onDownloadProgressing(finished, total);
start = System.currentTimeMillis();
}
}
onDownloadCompleted();
} catch (IOException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
} else {
Log.e("responseCode ", responseCode + "");
}
} catch (DownloadException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
} catch (CanceledException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
}
and also i get 206 response code instead of 200

1- You are getting null for httpURLConnection because you try to invoke it before being initialized,
i.e, this line
httpURLConnection = (HttpURLConnection) url.openConnection();
should come before this line:
String lastModified = httpURLConnection.getHeaderField("Last-Modified");
2- you can set the header before calling connect() on httpURLConnection
so you need to set whatever you want, then connect(). this way you should not get the error (android Cannot set request property after connection is made)
3- The 206 is perfectly right, it's what you should expect when using Range and it means Partial Content Success and that's what you are doing, you are getting part of the content, if you are getting full content you would get 200.
so to sum this up, your code can look like this:
Note: follow the //*** to see changes required.
EDIT: it all came to this line
httpURLConnection.setRequestProperty("If-Range", lastModified);
the error get thrown when you set that Property,
Anyways, when you look at this, it's meaningless, you are asking if last-modified is equal to the value that you just got from the connection!,
if you want to do this you need to store lastModified in your system,
then compare it with the one you got from the URLConn, and compare it to your file length (already downloaded)
then proceed with full download or resume download.
find new code below:
public void run() {
myLastModified = getLastModified(mFile.getName()); // get last stored value for this file (use file name or other key)
int total =0;
final URL url;
HttpURLConnection httpURLConnection = null;
try {
try {
url = new URL(mUrl);
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setDoInput(true);
httpURLConnection.setConnectTimeout(30000);
httpURLConnection.setReadTimeout(30000);
httpURLConnection.setRequestMethod("GET");
//*** new way to handle download process
total = httpURLConnection.getContentLength();
if(mFile.exists()){
if(mFile.length() == total){
//we are done, return.
return;
}else{
//file was not completly donwloaded, now check lastModified:
long lastModified = httpURLConnection.getLastModified();//this gets the header "Last-Modified" and convert to long
if (lastModified == myLastModified) { //myLastModified should be retrived on each download and stored locally on ur system
downloadedLength = mFile.length();
Log.e("downloadedLength ", downloadedLength + "");
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setDoInput(true);
httpURLConnection.setConnectTimeout(30000);
httpURLConnection.setReadTimeout(30000);
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setRequestProperty("Range", "bytes=" + downloadedLength + "-"+ total); //add + total (TO)
//append mode
fileOutputStream = new FileOutputStream(mFile, true);
}else{
//file was modified after 1st uncompleted-download:
storeLastModified(lastModified, mFile.getName()); // use file name as key. can store in db or file ...
//don't set ant Range ... we want full download, with a fresh file
fileOutputStream = new FileOutputStream(mFile);
}//last mod IF
}//Length check
}else{
//file not exist at all, create new file, set no Range we want full download...
mFile.createNewFile();
fileOutputStream = new FileOutputStream(mFile);
}//file exists.
} catch (IOException e) {
e.printStackTrace();
}
final int responseCode;
try {
responseCode = httpURLConnection.getResponseCode();
} catch (IOException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
//*****
if (responseCode == 200 || responseCode == 206) {
try {
inputStream = httpURLConnection.getInputStream();
} catch (IOException e) {
e.printStackTrace();
Log.e("IOException ", e.getMessage());
}
final byte[] buffer = new byte[4 * 1024];
int length = -1;
int finished = 0;
long start = System.currentTimeMillis();
try {
while ((length = inputStream.read(buffer)) != -1) {
if (!isDownloading()) {
throw new CanceledException("canceled");
}
fileOutputStream.write(buffer, 0, length);
finished += length;
if (System.currentTimeMillis() - start > 1000) {
onDownloadProgressing(finished, total);
start = System.currentTimeMillis();
}
}
onDownloadCompleted();
} catch (IOException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
} else {
Log.e("responseCode ", responseCode + "");
}
} catch (DownloadException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
} catch (CanceledException e) {
e.printStackTrace();
Log.e("ER UPDATE ", e.getMessage());
}
}

Take a look at this answer of POMATu.
But anyway, if You downloading files over the HTTP protocol, You can use DownloadManager - a system service (since API level 9) for long-running downloads in the background. It handles HTTP connections, connectivity changes, reboots, and ensures each download completes successfully. And it already supports resuming and also progress notifications.
You can find many tutorials like this or examples like that, and many solutions on stackoverflow by android-download-manager tag.

Related

Server sends html but no images to the browser HTTP

This code loads my html but it will not load my jpgs in the html, when I run the code it seems like it is working and all the bytes are being sent for each image but they do not show up in my browser. Im supposed to use the DataOutputStream to send the bytes but maybe thats the problem? im really lost and brand new to network programming.
public void run(){
System.out.println("CLIENT THREAD STARTING");
String req = "";
if(clientIn.hasNextLine()){
req = clientIn.nextLine();
System.out.println("Header: " + request);
}
PrintWriter toClient = null;
try {
toClient = new PrintWriter(sock.getOutputStream());
} catch (IOException e2) {
e2.printStackTrace();
}
//only sends files if client requests them
if(request.contains("GET")){
req = req.substring(request.indexOf("/") + 1);
String name = req.substring(0, req.indexOf(" "));
String type = "";
if(name.contains(".jpg")){
type = "text/html";
}
else if(name.contains(".html")){
type = "image/jpeg";
}
File theFile = new File(fileName);
FileInputStream fileIn = null;
try{
fileIn = new FileInputStream(fileName);
}catch(FileNotFoundException e){
toClientText.println("404");
toClientText.flush();
System.exit(-1);
}
System.out.println("File name " + name);
toClientText.println("HTTP/1.1 200 OK");
toClientText.println("Connection: closed");
toClientText.println("Content-Length: " + theFile.length());
toClientText.println("Content-Type: " + type);
toClientText.println("\n");
toClientText.flush();
DataOutputStream dataStream = null;
try {
dataStream = new DataOutputStream(sock.getOutputStream());
} catch (IOException e1) {
e1.printStackTrace();
}
byte[] send = new byte[1024] ;
try {
while ((fin.read(send)) != -1 ) {
toClientData.write(send);
toClientData.flush();
}
toClientData.close();
fin.close();
} catch (IOException e) {
e.printStackTrace();
}
}
else{
toClientText.println("Bad Request 400");
toClientText.flush();
}
}
toClientText.println("Content-Type: " + fileType);
toClientText.println("\n");
This will create 3 line breaks between your header and the file data, so your browser will think the third line break is part of the image, which will break the image.
Also, even though all browsers can handle \n linebreaks, be aware that the standard demands \r\n linebreaks, not \n.
This should work:
toClientText.println("Content-Type: " + fileType);
toClientText.println();

InputStream doens't read the whole answer of an URL connection

I'm struggling against the uncompleted download of a file.
For example, I upload some data on github :https://gist.githubusercontent.com/rdanniau/3b7f26bb1101b28400bf24f2f9664828/raw/980d6ff511404bf14d3efc56be3dfb081541991f/LEDirium.hex
and on pasteBin : http://pastebin.com/raw/FcVfLf5b
I want to retrieve them and save them into a file "filename".
I've watch a lot of example on internet and it must be working.
Here is the code :
private void download(final URL myUrl){
new Thread(new Runnable() {
//InputStream is = null;
//FileOutputStream fos = null;
public void run() {
try {
URLConnection connection = myURLL.openConnection();
//HttpURLConnection connection = (HttpURLConnection) myUrl.openConnection();
//connection.setRequestMethod("GET");
connection.setReadTimeout(5000);
connection.setConnectTimeout(10000);
connection.connect();
//is = myUrl.openStream();
is = connection.getInputStream();
File file = new File(context.getFilesDir(),fileName);
file.delete();
file = new File(context.getFilesDir(),fileName);
fos = new FileOutputStream(file);
byte data[] = new byte[1024];
String str ="";
int count = 0;
while ((count = is.read(data)) != -1) {
fos.write(data, 0, count);
}
is.close();
fos.close();
}
catch (Exception e) {
downloadedFileCallback.onError(e);
Log.e("DownloadedFile", "Unable to download : " + e.getMessage() + " cause :" + e.getCause());
return;
}
downloadedFileCallback.onDownloadedFinished();
readFile(context);
}
}).start();
}
public void readFile(Context context){
// read
try {
BufferedReader br = new BufferedReader(new FileReader(new File(context.getFilesDir(),fileName)));
String line;
while ((line = br.readLine()) != null) {
Log.v("DL", line);
}
br.close();
}
catch (Exception e) {
Log.e("DownloadedFile", "Unable to read : " + e.getMessage() + " cause :" + e.getCause());
}
//Log.v("DownloadedFile", text.toString());
}
where myURL are called like
URL myURL = new URL("http://pastebin.com/raw/FcVfLf5b");
In the Log.v, I can see that I downloaded only a part of the file which is never the same (it could be the entire file, the half, the quarter, we don' know...)
It's probably the inputStream connection which is closed too fast. But why ?
Last question, instead of using Log.v to check if the file is correctly downloaded. Where can I found it on my phone ? I searched in many folders but I never seen my File.
Thanks a lot for any advice
EDIT : It seems to be the same here InputStream returns -1 before end of file but no one answered.

Saving an XML Feed in an InputStream into a local file in Android?

I'm reading an XML feed from a web site, but, for performance reasons, I'd like to be able to read it into a local file on Android, so I can process it several times without incurring network delays and costs.
I'm reading the web file using the following code:
private InputStream downloadUrl(String urlString) throws IOException {
URL url = new URL(urlString);
try {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
InputStream instream = conn.getInputStream();
return instream;
} catch (Exception e) {
Log.i("downloadUrl", "4 - Exception: " + e.getMessage());
return null;
}
}
However, I'm getting a "4 - Exception: null" message from downloadUrl(), which is screwing up my file saving code below:
public Boolean copyStream(String feed, String file) {
InputStream input_data = null;
FileOutputStream output_data = null;
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
int buffer_count = 0;
try {
input_data = downloadUrl(feed);
output_data = openFileOutput(file, Context.MODE_PRIVATE);
while ((len = input_data.read(buffer)) != -1) {
output_data.write(buffer, 0, len);
buffer_count++;
}
input_data.close();
output_data.close();
} catch (Exception e) {
Log.e("copyStream", "Error=" + e.getMessage() + ", feed=" + feed
+ ", file=" + file + ", buffer_count=" + buffer_count);
return false;
}
return true;
}
If I can resolve the InputStream problem, will copyStream() send the XML text to a local file without permission errors?
Thanks in advance............Phil
greenapps,
This is the logcat, but it says little more than I'd already provided (replace 192.168.1.213 with 83.104.132.69 for an external URL):
06-19 08:05:04.414: I/downloadUrl(1108): 4 - Exception: null
06-19 08:05:04.414: E/copyStream(1108): Error=/summary.xml: open failed: EROFS (Read-only file system), feed=http://192.168.1.213/cgi-bin/beerxml_a.cgi?beer_display=Summary&beer_country=*All*&beer_brewer=*All*&beer_type=*All*, file=summary.xml

How to get VIEWSTATE before posting using HttpURLConnection (android/java)?

I am writing an Android application, and have been looking for a way to get the _VIEWSTATE from the server I want to post to so I can put it in my Http post content. A few people recommended regex, but then some other pros were strongly opposed to parsing HTML with regex. So, how to parse the _VIEWSTATE ? I am using HttpURLConnection/HttpsURLConnection in an AsyncTask. Also, don't I need to put the InputStream reader first, to get the _VIEWSTATE first? All the android examples put the input stream after the output stream. Here is what my code looks like so far (posting to one site that has three pages that have to be "clicked through"):
In my Activity, I call the Async task like this:
//execute AsyncTask for all three reports
submit_report.execute(report1, report2, report3);
My Async task doInBackground method:
class UploadReportTask extends AsyncTask<HashMap<String,String>, ProgressBar, Void> {
//this is called on task.execute
protected Void doInBackground(HashMap<String,String>...maps) {
System.out.println("Report is being uploaded");
URL url = null;
try {
url = new URL(getString(R.string.url_dnc));
} catch (MalformedURLException e) {
e.printStackTrace();
}
HttpURLConnection urlConnection = null;
try {
urlConnection = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
try {
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Accept-Charset", utf);
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + utf);
urlConnection.setChunkedStreamingMode(0);
//For each map in maps, encode the map,
//get the headers, add the headers to the map, convert to bytes,
//then post the bytes,
//get response.
for (HashMap<String,String> map : maps){
byte[] payload = makePayload(map);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
//urlConn.connect //I think this happens here
out.write(payload);
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
int length = in.read();
String result, line = reader.readLine();
result = line;
while (length != -1){
result+=line;
}
System.out.println(result);
out.flush();
out.close();
}
} catch (IOException e) {
e.printStackTrace();
}
finally{
urlConnection.disconnect();
}
return null;
}
protected String parseViewstate(String response){
int i = 0;
String viewstate = "";
while (true){
int found = response.indexOf("\"__VIEWSTATE\"", i);
if (found == -1) break;
int start = found + 38; //check numbers from start of "__V"
int end = (response.indexOf("/>", start)) -2;
viewstate = response.substring(start, end);
i = end + 1;
}return viewstate;
}

HTTP POST not working.Getting Not Found error

I am trying to Post a .txt file to a local tomcat webserver that i have on my system.
But when i try to do a post then i get a Error: Not Found.
The source file is present but even after that i get this error.
Can you please let me know what i am doing wrong here. i have pasted my code below.
File file = new File("C:\\xyz\\test.txt");
URL url = new URL("http://localhost:8080/process/files");
urlconnection = (HttpURLConnection) url.openConnection();
urlconnection.setDoOutput(true);
urlconnection.setDoInput(true);
urlconnection.setRequestMethod("POST");
BufferedOutputStream bos = new BufferedOutputStream(urlconnection.getOutputStream());
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
int i; // read byte by byte until end of stream
while ((i = bis.read()) >0) {
bos.write(i);
}
bos.close();
System.out.println(((HttpURLConnection)urlconnection).getResponseMessage());
} catch(Exception ae)
{
ae.printStackTrace();
}
try {
InputStream inputStream;
int responseCode=((HttpURLConnection)urlconnection).getResponseCode();
if ((responseCode>= 200) &&(responseCode<=202) ) {
inputStream = ((HttpURLConnection)urlconnection).getInputStream();
int j;
while ((j = inputStream.read()) >0) {
System.out.println("------ TESTING ------");
}
} else {
inputStream = ((HttpURLConnection)urlconnection).getErrorStream();
}
((HttpURLConnection)urlconnection).disconnect();
} catch (IOException e) { // TODO Auto-generated catch block
e.printStackTrace();
}
}
Can you please let me know what is going wrong here.
I am scratching my head on this for a long time now.
Thanks
Vikeng
The URL you are POSTing to needs to point to a servlet or something similar. You cannot upload a file to a directory just by sending a POST request--the POST request has to be handled by something.

Categories