java.lang.OutOfMemoryError while downloading file - java

I'm trying to download file at my android device. This is my code:
InputStream in = null;
ByteArrayOutputStream out = null;
FileOutputStream fos = null;
try {
URL link = new URL(fileURL);
in = new BufferedInputStream(link.openStream());
out = new ByteArrayOutputStream();
byte[] buf = new byte[8192];
int n, bytesBuffered = 0;
while (-1 != (n = in.read(buf))) {
bytesBuffered += n;
out.write(buf, 0, n);
if (bytesBuffered >1048576) {
bytesBuffered = 0;
out.flush();
}
}
byte[] response = out.toByteArray();
fos = new FileOutputStream(filePath);
fos.write(response);
return true;
} catch (Throwable e) {
e.printStackTrace();
return false;
} finally {
//closing streams
}
It fails at out.write(buf, 0, n); with out of memory error. What is wrong? I've read I'm able to set bigger heap size at manifest, but I don't find it a good solution. What is wrong?

if your goal is to write the content on a file, there is no need to read the content in memory before writing it.
fos = new FileOutputStream(filePath);
while (-1 != (n = in.read(buf))) {
bytesBuffered += n;
fos.write(buf, 0, n);
}
fos.flush();
and I would also add a finally clause to the try block to close fos. The finally block is call either if you return successfully or if an error occurs. This way you will not leak the FileOutputStream

You are doing it wrong!
Open your FileOutputStream and just read from the input and copy it to the output stream:
private static final int BUFSIZE = 8096;
//
final byte[] buf = new byte[BUFSIZE];
int nrBytes;
while ((nrBytes = in.read(buf)) != -1)
out.write(buf, 0, nrBytes);
(note: closing descriptors not handled here, left as an exercise; but sicne this is Android and you therefore don't have try-with-resource or even JSR 203, I suggest you use Guava and its Closer)

You need a do instead of a while loop :
do {
n = in.read(buf);
bytesBuffered += n;
out.write(buf, 0, n);
if (bytesBuffered >1048576) {
bytesBuffered = 0;
out.flush();
}
} while (n != -1)
and also add a finally block as pointed out by tony

Related

Getting the full InputStream

I am working on a Web Server, I am stuck on the HTTP method PUT ... I currently can only bet 10 bytes of data from the client when he tries to upload a file, bellow is what I have so far.
InputStream stream = connection.getInputStream();
OutputStream fos = Files.newOutputStream(path);
int count = 0;
while (count < 10) {
int b = stream.read();
if (b == -1) break;
fos.write(b);
++count;
}
fos.close();
Please tell me how I can get the as much input the client writes.
You limited it to 10 by your while loop using 10. Since stream.read() returns -1 at the end of the stream, use that in the while loop as the control:
int count = 0;
int b = 0;
while ((b=stream.read()) !=-1)
{
fos.write(b);
count++;
}
public void receiveFile(InputStream is){
//Set a really big filesize
int filesize = 6022386;
int bytesRead;
int current = 0;
byte[] mybytearray = new byte[filesize];
try(FileOutputStream fos = new FileOutputStream("fileReceived.txt");
BufferedOutputStream bos = new BufferedOutputStream(fos)){
//Read till you get a -1 returned by is.read(....)
bytesRead = is.read(mybytearray, 0, mybytearray.length);
current = bytesRead;
do {
bytesRead = is.read(mybytearray, current,
(mybytearray.length - current));
if (bytesRead >= 0)
current += bytesRead;
} while (bytesRead > -1);
bos.write(mybytearray, 0, current);
bos.flush();
bos.close();
}
catch (FileNotFoundException fnfe){
System.err.println("File not found.");
}
catch (SecurityException se){
System.err.println("A Security Issue Occurred.");
}
}
Based on this one: FTP client server model for file transfer in Java

Packet loss in socket programming java

I am trying to send a file from client to server. Below is the code i have tried. But at times, there is a packet loss during the transfer. I am not sure where i am wrong.
SERVER SIDE CODE:
public static void ReadAndWrite(byte[] aByte, Socket clientSocket,
InputStream inputStream, String fileOutput)
throws FileNotFoundException, IOException {
int bytesRead;
FileOutputStream fileOutputStream = null;
BufferedOutputStream bufferedOutputStream = null;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try
{
fileOutputStream = new FileOutputStream( fileOutput );
bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
bytesRead = inputStream.read(aByte, 0, aByte.length);
System.out.println("The length is "+bytesRead);
int count = 0;
do {
count++;
byteArrayOutputStream.write(aByte);
bytesRead = inputStream.read(aByte);
} while (bytesRead != -1);
System.out.println("The count is "+count);
System.out.println("The length is "+byteArrayOutputStream.size());
bufferedOutputStream.write(byteArrayOutputStream.toByteArray());
bufferedOutputStream.flush();
bufferedOutputStream.close();
clientSocket.close();
}
catch(Exception ex)
{
Logger.writeLog(ex,Listen.class.getName(), LogType.EXCEPTION);
throw ex;
}
CLIENT SIDE CODE:
public void readByteArrayAndWriteToClientSocket(
Socket connectionSocket, BufferedOutputStream outToClient, String fileToSend ) throws Exception
{
try{
if (outToClient != null)
{
File myFile = new File(fileToSend);
System.out.println(myFile.length());
byte[] byteArray = new byte[(int) myFile.length()];
FileInputStream fileInputStream = null;
try {
fileInputStream = new FileInputStream(myFile);
} catch (IOException ex) {
Logger.writeLog(ex, FileUtility.class.getName(), LogType.EXCEPTION);
throw ex;
}
BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
try {
bufferedInputStream.read(byteArray, 0, byteArray.length);
outToClient.write(byteArray, 0, byteArray.length);
outToClient.flush();
outToClient.close();
connectionSocket.close();
return;
} catch (IOException ex) {
Logger.writeLog(ex, FileUtility.class.getName(), LogType.EXCEPTION);
throw ex;
}
}
}catch (Exception e) {
Logger.writeLog(e, getClass().getName(), LogType.EXCEPTION);
throw e;
}
}
There is no 'packet loss', just bugs in your code.
The canonical way to copy a stream in Java is as follows:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
If you know the number of bytes in advance and the sender must keep the connection open after the transfer, it becomes:
while (total < expected && (count = in.read(buffer, 0, expected-total > buffer.length ? buffer.length : (int)(expected-total))) > 0)
{
out.write(buffer, 0, count);
total += count;
}
Forget all the ByteArrayInput/OutputStreams and the extra copies. Just read from the file and send to the socket, or read from the socket and write to the file.
The sockets read method will return when its has obtained all the bytes you asked for, OR, when it stops receiving data from the network.
As transmission is often interrupted in any real network you need to keep issuing read calls until you have the number of bytes you want.
You need code something like this:
char [] buffer = new char[1024];
int expect = 1000;
int sofar = 0;
int chars_read;
try
{
while((chars_read = from_server.read(buffer[sofar])) != -1)
{
sofar = sofar + chars_read;
if (sofar >= expected) break;
}
}
catch(IOException e)
{
to_user.println(e);
}

File is not transferred completely (Android)

I am developing an Android App to send a file via bluetooth to a java server using the BlueCove library version 2.1.0 based on this snippet. At the beginning everything looks fine, but the file will not transfered completly. Only about 7KB of 35KB.
Android
private void sendFileViaBluetooth(byte[] data){
OutputStream outStream = null;
BluetoothDevice device = btAdapter.getRemoteDevice(address);
btSocket = device.createRfcommSocketToServiceRecord(MY_UUID);
btSocket.connect();
try {
outStream = btSocket.getOutputStream();
outStream.write( data );
outStream.write("end of file".getBytes());
outStream.flush();
} catch (IOException e) {
} finally{
try {
outStream.close();
btSocket.close();
device = null;
} catch (IOException e) {
}
}
}
PC Server
InputStream inStream = connection.openInputStream();
byte[] buffer = new byte[1024];
File f = new File("d:\\temp.jpg");
FileOutputStream fos = new FileOutputStream (f);
InputStream bis = new BufferedInputStream(inStream);
int bytes = 0;
boolean eof = false;
while (!eof) {
bytes = bis.read(buffer);
if (bytes > 0){
int offset = bytes - 11;
byte[] eofByte = new byte[11];
eofByte = Arrays.copyOfRange(buffer, offset, bytes);
String message = new String(eofByte, 0, 11);
if(message.equals("end of file")) {
eof = true;
} else {
fos.write (buffer, 0, bytes);
}
}
}
fos.close();
connection.close();
I tried already to split the byte array before writing:
public static byte[][] divideArray(byte[] source, int chunksize) {
byte[][] ret = new byte[(int)Math.ceil(source.length / (double)chunksize)][chunksize];
int start = 0;
for(int i = 0; i < ret.length; i++) {
ret[i] = Arrays.copyOfRange(source,start, start + chunksize);
start += chunksize ;
}
return ret;
}
private void sendFileViaBluetooth(byte[] data){
[...]
byte[][] chunks = divideArray(data, 1024);
for (int i = 0; i < (int)Math.ceil(data.length / 1024.0); i += 1) {
outStream.write( chunks[i][1024] );
}
outStream.write("end of file".getBytes());
outStream.flush();
[...]
}
Every help or ideas are appreciated.
You don't need any of this. The canonical way to copy a stream in Java is this:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
out.close();
Same at both ends. TCP/IP will do all the chunking for you. All you need to do is cope correctly with varying size reads, which this code does.

Java Updating Program questions

I have a program that updates files on the computer using information sent by a server, using sockets. The way I had it worked, but i wanted to make it more intuitive, simpler, more reliable, etc. here is the previous code:
int filesize = 6022386; // filesize temporary hardcoded
int bytesRead;
int current = 0;
/**
* receive file
*/
try {
byte[] byteArray = new byte[filesize];
java.io.InputStream inStream = socket.getInputStream();
bytesRead = inStream.read(byteArray, 0, byteArray.length);
FileOutputStream fileOutStream = new FileOutputStream(
"C:\\Program Files\\AVTECH\\NPS\\Files\\bin\\NPS Game.txt");
BufferedOutputStream buffOutStream = new BufferedOutputStream(
fileOutStream);
current = bytesRead;
do {
bytesRead = inStream.read(byteArray, current,
(byteArray.length - current));
if (bytesRead >= 0)
current += bytesRead;
} while (bytesRead > -1);
buffOutStream.write(byteArray, 0, current);
buffOutStream.flush();
buffOutStream.close();
inStream.close();
socket.close();
} catch (Exception e) {
e.printStackTrace();
socket.close();
}
as you can see, in the do, while loop, it is using the input stream to get the data. now that i've updated my program, i have the stream sending an object called UpdateObject, which holds the byte[] array along with the file directory. here is that code:
int filesize = 6022386; // filesize temporary hardcoded
int bytesRead;
int current = 0;
try {
byte[] byteArray = o.getFile();
java.io.InputStream inStream = socket.getInputStream();
bytesRead = inStream.read(byteArray, 0, byteArray.length);
FileOutputStream fileOutStream = new FileOutputStream(o.getPath());
BufferedOutputStream buffOutStream = new BufferedOutputStream(
fileOutStream);
current = bytesRead;
do {
bytesRead = inStream.read(byteArray, current,
(byteArray.length - current));
if (bytesRead >= 0)
current += bytesRead;
} while (bytesRead > -1);
buffOutStream.write(byteArray, 0, current);
buffOutStream.flush();
buffOutStream.close();
inStream.close();
socket.close();
} catch (Exception e) {
e.printStackTrace();
}
now my question is this: how do i change it so instead of using the instream, to use just a byte[] object in the UpdateObject sent over the socket? i've done some google searching, but i dont feel like i know the right question to ask. any help would be great! thanks in advance!!!
By replacing most of your code inside the try catch block with:
FileOutputStream fileOutStream = new FileOutputStream(
UpdateObject.getDirectory()+"\\NPS Game.txt");
fileOutStream.write(UpdateObject.getBytes()); //this is the byte[] array
fileOutStream.close();
Hope this helps.

How to write files to an SD Card from the internet?

An example would be a simple image.
I have tried so many things and it just refuses to work despite making a whole lot of sense.
What I've done so far is I'm able to grab 25 pictures and add them to
/sdcard/app name/sub/dir/filename.jpg
They all appear there according to the DDMS but they always have a filesize of 0.
I'm guessing it's probably because of my input stream?
Here's my function that handles the downloading and saving.
public void DownloadPages()
{
for (int fileC = 0; fileC < pageAmount; fileC++)
{
URL url;
String path = "/sdcard/Appname/sub/dir/";
File file = new File(path, fileC + ".jpg");
int size=0;
byte[] buffer=null;
try{
url = new URL("http://images.bluegartr.com/bucket/gallery/56ca6f9f2ef43ab7349c0e6511edb6d6.png");
InputStream in = url.openStream();
size = in.available();
buffer = new byte[size];
in.read(buffer);
in.close();
}catch(Exception e){
}
if (!new File(path).exists())
new File(path).mkdirs();
FileOutputStream out;
try{
out = new FileOutputStream(file);
out.write(buffer);
out.flush();
out.close();
}catch(Exception e){
}
}
}
It just keeps giving me 25 files in that directory but all of their file sizes are zero. I have no idea why. This is practically the same code I've used in a java program.
PS...
If you're gonna give me a solution... I've already tried code like this. It doesn't work.
try{
url = new URL(urlString);
in = new BufferedInputStream(url.openStream());
fout = new FileOutputStream(filename);
byte data[] = new byte[1024];
int count;
System.out.println("Now downloading File: " + filename.substring(0, filename.lastIndexOf(".")));
while ((count = in.read(data, 0, 1024)) != -1){
fout.write(data, 0, count);
}
}finally{
System.out.println("Download complete.");
if (in != null)
in.close();
if (fout != null)
fout.close();
}
}
Here's an image of what my directories look like
http://oi48.tinypic.com/2cpcprm.jpg
A bit change to your second option, try it as following way,
byte data[] = new byte[1024];
long total = 0;
int count;
while ( ( count = input.read(data)) != -1 )
{
total += count;
output.write( data,0,count );
}
This one is different in while statement while ((count = in.read(data, 0, 1024)) != -1)
Using Guava something like this should work:
String fileUrl = "xxx";
File file = null;
InputStream in;
FileOutputStream out;
try {
Uri url = new URI(fileUrl);
in = url.openStream();
out = new FileOutputStream(file)
ByteStreams.copy(in, out);
}
catch (IOException e) {
System.out.println(e.toString());
}
finally {
in.close();
out.flush();
out.close();
}

Categories