I'm trying to perform an AsyncTask class in my Android application that analyzes the network connection speed in for downloading and uploading. I'm working on the download portion now, but I'm not getting results I expect. I'm testing on a Wifi network that gets 15Mbps down/up speeds consistently, however, the results I'm getting from my application are more around barely 1Mbps. When I run the speed test apk on the device I'm testing on that gets around 3.5Mbps. The function works, just seems to be half the speed it should be. Should the following code produce accurate results?
try {
String DownloadUrl = "http://ipv4.download.thinkbroadband.com:8080/5MB.zip";
String fileName = "testfile.bin";
File dir = new File (context.getFilesDir() + "/temp/");
if(dir.exists()==false) {
dir.mkdirs();
}
URL url = new URL(DownloadUrl); //you can write here any link
File file = new File(context.getFilesDir() + "/temp/" + fileName);
long startTime = System.currentTimeMillis();
Log.d("DownloadManager", "download begining: " + startTime);
Log.d("DownloadManager", "download url:" + url);
Log.d("DownloadManager", "downloaded file name:" + fileName);
/* Open a connection to that URL. */
URLConnection ucon = url.openConnection();
//Define InputStreams to read from the URLConnection.
InputStream is = ucon.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
//Read bytes to the Buffer until there is nothing more to read(-1).
ByteArrayBuffer baf = new ByteArrayBuffer(1024);
int current = 0;
while ((current = bis.read()) != -1) {
baf.append((byte) current);
}
long endTime = System.currentTimeMillis(); //maybe
/* Convert the Bytes read to a String. */
FileOutputStream fos = new FileOutputStream(file);
fos.write(baf.toByteArray());
fos.flush();
fos.close();
File done = new File(context.getFilesDir() + "/temp/" + fileName);
Log.d("DownloadManager", "Location being searched: "+ context.getFilesDir() + "/temp/" + fileName);
double size = done.length();
if(done.exists()) {
done.delete();
}
Log.d("DownloadManager", "download ended: " + ((endTime - startTime) / 1000) + " secs");
double rate = (((size / 1024) / ((endTime - startTime) / 1000)) * 8);
rate = Math.round( rate * 100.0 ) / 100.0;
String ratevalue;
if(rate > 1000)
ratevalue = String.valueOf(rate / 1024).concat(" Mbps");
else
ratevalue = String.valueOf(rate).concat(" Kbps");
Log.d("DownloadManager", "download speed: "+ratevalue);
} catch (IOException e) {
Log.d("DownloadManager", "Error: " + e);
}
Example output
10-08 15:09:52.658: D/DownloadManager(13714): download ended: 70 secs
10-08 15:09:52.662: D/DownloadManager(13714): download speed: 585.14 Kbps
Thanks in advance for the help. If there is a better method, please let me know.
Following on my comments, here is an example of how to read several bytes from the stream
//Define InputStreams to read from the URLConnection.
InputStream is = ucon.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
//I usually use a ByteArrayOutputStream, as it is more common.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int red = 0;
// This size can be changed
byte[] buf = new byte[1024];
while ((red = bis.read(buf)) != -1) {
baos.write(buf, 0, red);
}
What this does is it reads into a byte[] buffer, and return the amount of read bytes. This is in turn written to the OutputStream, specifying the amount of bytes to write.
ByteArrayOutputStream also have a toByteArray that behaves similarly.
Alternatively, you can also write directly to the file, if you consider that the write to file operation is significantly faster than the read function :
// Simply start by defining the fileoutputstream
FileOutputStream fos = new FileOutputStream(file);
int red = 0;
// This size can be changed
byte[] buf = new byte[1024];
while ((red = bis.read(buf)) != -1) {
// And directly write to it.
fos.write(buf, 0, red);
}
long endTime = System.currentTimeMillis(); //maybe
// Flush after, as this may trigger a commit to disk.
fos.flush();
fos.close();
Moreover, if you really only care about the download speed, it is not mandatory to write to the file, or to anywhere, this would be sufficient :
long size = 0;
byte[] buf = new byte[1024];
while ((red = bis.read(buf)) != -1) {
size += red;
}
Related
I just tried to download a file from a Website, i checked the URL and everything seems to be fine there, but when i try to do it like described it prints out that my update size is -1 Bytes, don't know why this is happening.
If anyone has a solution, i would be glad to here it.
My Code:
private void downloadFile(String link) throws MalformedURLException, IOException
{
URL url = new URL(link);
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
long max = conn.getContentLength();
outText.setText(outText.getText()+"\n"+"Downloding file...\nUpdate Size(compressed): "+max+" Bytes");
System.out.println("Update Size --> " + max + " Bytes");
BufferedOutputStream fOut = new BufferedOutputStream(new FileOutputStream(new File("update.zip")));
byte[] buffer = new byte[32 * 1024];
int bytesRead = 0;
int in = 0;
while ((bytesRead = is.read(buffer)) != -1) {
in += bytesRead;
fOut.write(buffer, 0, bytesRead);
}
fOut.flush();
fOut.close();
is.close();
outText.setText(outText.getText()+"\nDownload Complete!");
}
Big Thanks already :)
The javadoc is your friend:
Returns:
the content length of the resource that this connection's URL references, -1 if the content length is not known, or if the content length is greater than Integer.MAX_VALUE.
(emphasis mine)
I have two files: a chat server and a chat client. The chat client is supposed to say that it wants to upload a file to the server. And then it uploads. However, right now, all of the messages are being sent / received properly, but when I try to get the file transfer, the only thing I get is a file with 0 bytes (which is at the path I specify inside of the server class.
Broken part of the chatclient class:
/**
* Sends a broadcast to the server
*/
public static void broadcast() throws IOException {
if (UserInput.getText() == "/upload") {
File myFile = new File (FILE_TO_SEND);
byte [] mybytearray = new byte [(int)myFile.length()];
fis = new FileInputStream(myFile);
bis = new BufferedInputStream(fis);
bis.read(mybytearray,0,mybytearray.length);
os = Socket.getOutputStream();
System.out.println("Sending " + FILE_TO_SEND + "(" + mybytearray.length + " bytes)");
os.write(mybytearray,0,mybytearray.length);
os.flush();
System.out.println("Done.");
}
System.out.println("" + UserInput.getText());
outputStream.println(UserInput.getText());
outputStream.flush();
}
Broken part of the server class:
if (input.contains("/upload")) {
byte [] mybytearray = new byte [FILE_SIZE];
InputStream is = csocket.getInputStream();
fos = new FileOutputStream(FILE_TO_RECEIVED);
bos = new BufferedOutputStream(fos);
bytesRead = is.read(mybytearray,0,mybytearray.length);
current = bytesRead;
do {
bytesRead = is.read(mybytearray, current, (mybytearray.length-current));
if (bytesRead >= 0) current += bytesRead;
}
while(bytesRead > -1);
bos.write(mybytearray, 0 , current);
bos.flush();
System.out.println("File " + FILE_TO_RECEIVED + " downloaded (" + current + " bytes read)");
}
Your copy loop is nonsense. The canonical way to copy a stream in Java is as follows:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
where 'count' is an int, and 'buffer' is a byte[] array of length > 0. I usually use 8192.
You should try surrounding the broken code with try-catch block and print out the error message from the stack. this would give you a better idea of what is not working. It's not a solution, I know, but it's easier to find a solution if you know the exact problem.
I am trying to download a large (11MB) JSON file from a web service using this code:
public static void downloadBigFile(final String serverUrl,
final String fileName) throws MalformedURLException, IOException {
System.out.println("Downloading " + serverUrl + " (" + fileName + ")");
URL url = new URL(serverUrl);
URLConnection con = url.openConnection();
con.setConnectTimeout(10000);
con.setReadTimeout(2 * 60 * 1000);
int totalFileSize = con.getContentLength();
System.out.println("Total file size: " + totalFileSize);
InputStream inputStream = con.getInputStream();
FileOutputStream outputStream = new FileOutputStream(fileName);
// Used only for knowing the amount of bytes downloaded.
int downloaded = 0;
final byte[] buffer = new byte[1024 * 8];
int bytesRead;
bytesRead = inputStream.read(buffer);
while (bytesRead != -1) {
downloaded += bytesRead;
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
System.out.println(String.format("%d/%d (%.2f%%)", downloaded,
totalFileSize,
(downloaded * 1.0 / totalFileSize * 1.0) * 100));
}
System.out
.println(fileName + " downloaded! (" + downloaded + " bytes)");
inputStream.close();
outputStream.close();
}
However the call to con.getContentLength() blocks the thread for several minutes, while it downloads the whole file I presume.
The problem is I need a quick way to discover file size before the download starts so I can notify the user accordingly.
Note: already tried to call con.connect() and con.getHeaderField("Content-Length").
If the server does not specify the Content-Length header, the only way to get the content length is to download the whole file and see how big it is.
I'm trying to transfer files between client-server if the file is bigger than 12MB than it sends by blocks otherwise it sends normally. my main problem is that everytime I transfer something it gets more bytes than the original one, and I need to use digest messages after I complete this so it wont work the way it is, and the other one is when I try to send a file by network the client reads the file to send faster than the server writes it so client closes the program closing the connection, corrupting the file. my transfer code is below:
this is the client transfer code:
if(fSize>maxfileSize){
totbLidos = 0;
byte[] fBytes = new byte[fBsize];
while(totbLidos < fSize){
int bRemain = (int) f.length() - totbLidos;
if(bRemain < fBsize){
fBsize = bRemain;
}
int bRead = tFile.read(fBytes, 0, fBsize);
tServidor.write(fBytes, 0, fBsize);
tServidor.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Total Bytes Lidos: " + totbLidos);
}
tFile.close();
System.out.println("Ficheiro enviado");
cliente.close();
}
else{
totbLidos = 0;
byte[] fBytes = new byte[fSize];
while(totbLidos < fSize){
int bRead = tFile.read(fBytes,0,fSize);
if(bRead>0){
totbLidos+=bRead;
}
tServidor.write(fBytes, 0, fSize);
System.out.println("Total Bytes Lidos: " + totbLidos);
tServidor.flush();
}
tFile.close();
System.out.println("Ficheiro enviado");
cliente.close();
}
}
server transfer code:
if(fSize > maxfileSize){
totbLidos = 0;
DataInputStream tFile = new DataInputStream(cliente.getInputStream());
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(fName));
byte[] fBytes = new byte[fBsize];
while(totbLidos < fSize){
int bRemain = size - totbLidos;
if(bRemain < fBsize){
fBsize = bRemain;
}
int bRead = tFile.read(fBytes, 0, fBsize);
fos.write(fBytes);
fos.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Bytes lidos: " + bRead);
System.out.println("Total Bytes Escritos: " + totbLidos);
}
System.out.println("Ficheiro recebido");
fos.close();
tFile.close();
cliente.close();
servidor.close();
}
else if(fSize < maxfileSize){
totbLidos = 0;
DataInputStream tFile = new DataInputStream(cliente.getInputStream());
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(fName));
byte[] fBytes = new byte[fSize];
while(totbLidos < fSize){
int bRead = tFile.read(fBytes,0,fSize);
fos.write(fBytes);
fos.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Total Bytes Escritos: " + totbLidos);
}
System.out.println("Ficheiro recebido");
fos.close();
tFile.close();
cliente.close();
servidor.close();
}
}
You are not writing the same number of bytes you read here.
fos.write(fBytes);
try using
fos.write(fBytes, 0, bRead);
In addition to #Peter Lawrey answer I suggest you to reuse Apache Commons IO API to avoid this kind of bugs in future:
IOUtils.copyLarge(InputStream input, OutputStream output, long inputOffset, long length)
URL url = new URL("http://download.thinkbroadband.com/20MB.zip");
URLConnection connection = url.openConnection();
File fileThatExists = new File(path);
OutputStream output = new FileOutputStream(path, true);
connection.setRequestProperty("Range", "bytes=" + fileThatExists.length() + "-");
connection.connect();
int lenghtOfFile = connection.getContentLength();
InputStream input = new BufferedInputStream(url.openStream());
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
output.write(data, 0 , count);
}
in this code I try to resume download. Target file is 20MB. But when I stop download on 10mb, then contunue, I get file with filesize 30MB. It seems that it continue writing to file, but cant partly download from server. Wget -c works great with this file. How can I resume file download?
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if(ISSUE_DOWNLOAD_STATUS.intValue()==ECMConstant.ECM_DOWNLOADING){
File file=new File(DESTINATION_PATH);
if(file.exists()){
downloaded = (int) file.length();
connection.setRequestProperty("Range", "bytes="+(file.length())+"-");
}
}else{
connection.setRequestProperty("Range", "bytes=" + downloaded + "-");
}
connection.setDoInput(true);
connection.setDoOutput(true);
progressBar.setMax(connection.getContentLength());
in = new BufferedInputStream(connection.getInputStream());
fos=(downloaded==0)? new FileOutputStream(DESTINATION_PATH): new FileOutputStream(DESTINATION_PATH,true);
bout = new BufferedOutputStream(fos, 1024);
byte[] data = new byte[1024];
int x = 0;
while ((x = in.read(data, 0, 1024)) >= 0) {
bout.write(data, 0, x);
downloaded += x;
progressBar.setProgress(downloaded);
}
This is not my code, but it works.
I guess the problem you are facing is calling url.openStream() after url.openConnection().
url.openStream() is equivalent to url.openConnection().getInputStream(). Hence, you are requesting the url twice. Particularly the second time, it is not specifying the range property. Therefore download always starts at the beginning.
You should replace url.openStream() with connection.getInputStream().
This is what I am using to download the file in chunk Updating the UI with progress.
/*
* #param callback = To update the UI with appropriate action
* #param fileName = Name of the file by which downloaded file will be saved.
* #param downloadURL = File downloading URL
* #param filePath = Path where file will be saved
* #param object = Any object you want in return after download is completed to do certain operations like insert in DB or show toast
*/
public void startDownload(final IDownloadCallback callback, String fileName, String downloadURL, String filePath, Object object) {
callback.onPreExecute(); // Callback to tell that the downloading is going to start
int count = 0;
File outputFile = null; // Path where file will be downloaded
try {
File file = new File(filePath);
file.mkdirs();
long range = 0;
outputFile = new File(file, fileName);
/**
* Check whether the file exists or not
* If file doesn't exists then create the new file and range will be zero.
* But if file exists then get the length of file which will be the starting range,
* from where the file will be downloaded
*/
if (!outputFile.exists()) {
outputFile.createNewFile();
range = 0;
} else {
range = outputFile.length();
}
//Open the Connection
URL url = new URL(downloadURL);
URLConnection con = url.openConnection();
// Set the range parameter in header and give the range from where you want to start the downloading
con.setRequestProperty("Range", "bytes=" + range + "-");
/**
* The total length of file will be the total content length given by the server + range.
* Example: Suppose you have a file whose size is 1MB and you had already downloaded 500KB of it.
* Then you will pass in Header as "Range":"bytes=500000".
* Now the con.getContentLength() will be 500KB and range will be 500KB.
* So by adding the two you will get the total length of file which will be 1 MB
*/
final long lenghtOfFile = (int) con.getContentLength() + range;
FileOutputStream fileOutputStream = new FileOutputStream(outputFile, true);
InputStream inputStream = con.getInputStream();
byte[] buffer = new byte[1024];
long total = range;
/**
* Download the save the content into file
*/
while ((count = inputStream.read(buffer)) != -1) {
total += count;
int progress = (int) (total * 100 / lenghtOfFile);
EntityDownloadProgress entityDownloadProgress = new EntityDownloadProgress();
entityDownloadProgress.setProgress(progress);
entityDownloadProgress.setDownloadedSize(total);
entityDownloadProgress.setFileSize(lenghtOfFile);
callback.showProgress(entityDownloadProgress);
fileOutputStream.write(buffer, 0, count);
}
//Close the outputstream
fileOutputStream.close();
// Disconnect the Connection
if (con instanceof HttpsURLConnection) {
((HttpsURLConnection) con).disconnect();
} else if (con instanceof HttpURLConnection) {
((HttpURLConnection) con).disconnect();
}
inputStream.close();
/**
* If file size is equal then return callback as success with downlaoded filepath and the object
* else return failure
*/
if (lenghtOfFile == outputFile.length()) {
callback.onSuccess(outputFile.getAbsolutePath(), object);
} else {
callback.onFailure(object);
}
} catch (Exception e) {
e.printStackTrace();
callback.onFailure(object);
}
}
interface IDownloadCallback {
void onPreExecute(); // Callback to tell that the downloading is going to start
void onFailure(Object o); // Failed to download file
void onSuccess(String path, Object o); // Downloaded file successfully with downloaded path
void showProgress(EntityDownloadProgress entityDownloadProgress); // Show progress
}
public class EntityDownloadProgress {
int progress; // range from 1-100
long fileSize;// Total size of file to be downlaoded
long downloadedSize; // Size of the downlaoded file
public void setProgress(int progress) {this.progress = progress;}
public void setFileSize(long fileSize) {this.fileSize = fileSize;}
public void setDownloadedSize(long downloadedSize) {this.downloadedSize = downloadedSize;}
}
Check out this thread which has a problem similar to yours. If wget is working, then the server clearly supports resuming downloads. It looks like you're not setting the If-Range header as mentioned in the accepted answer of the above link. ie. add:
// Initial download.
String lastModified = connection.getHeaderField("Last-Modified");
// ...
// Resume download.
connection.setRequestProperty("If-Range", lastModified);
How about this?
public static void download(DownloadObject object) throws IOException{
String downloadUrl = object.getDownloadUrl();
String downloadPath = object.getDownloadPath();
long downloadedLength = 0;
File file = new File(downloadPath);
URL url = new URL(downloadUrl);
BufferedInputStream inputStream = null;
BufferedOutputStream outputStream = null;
URLConnection connection = url.openConnection();
if(file.exists()){
downloadedLength = file.length();
connection.setRequestProperty("Range", "bytes=" + downloadedLength + "-");
outputStream = new BufferedOutputStream(new FileOutputStream(file, true));
}else{
outputStream = new BufferedOutputStream(new FileOutputStream(file));
}
connection.connect();
inputStream = new BufferedInputStream(connection.getInputStream());
byte[] buffer = new byte[1024*8];
int byteCount;
while ((byteCount = inputStream.read(buffer)) != -1){
outputStream.write(buffer, 0, byteCount);
break;
}
inputStream.close();
outputStream.flush();
outputStream.close();
}
Used break; to test the code.. ;)
I have a way for your code to work.
First, check if the file exits or not
If the file exists, set the connection:
connection.setRequestProperty("Range", "bytes=" + bytedownloaded + "-");
If file does not exist, do the same download in a new file.
Since the question is tagged with Android:
Have you tried using DownloadManager.
It handles all this stuff nicely for you.