In my app, I'm uploading files to the google drive using GD API. It works fine for small file sizes, but when file size is large (ex: 200MB), it throws java.lang.OutOfMemoryError: exception. I know why it crashes it loads the whole data into the memory, can anyone suggest how can I fix this problem?
This is my code:
OutputStream outputStream = result.getDriveContents().getOutputStream();
FileInputStream fis;
try {
fis = new FileInputStream(file.getPath());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[8192];
int n;
while (-1 != (n = fis.read(buf)))
baos.write(buf, 0, n);
byte[] photoBytes = baos.toByteArray();
outputStream.write(photoBytes);
outputStream.close();
outputStream = null;
fis.close();
fis = null;
} catch (FileNotFoundException e) {
}
This line would allocate 200 MB of RAM and can definitely cause OutOfMemoryError exception:
byte[] photoBytes = baos.toByteArray();
Why are you not writing directly to your outputStream:
while (-1 != (n = fis.read(buf)))
outputStream.write(buf, 0, n);
Related
I have a file in the server, I want to create three java APIs, which will do the below three operations in dependently.
Get the file size
Move a file with different file name to a different server location
Zip the file
In my existing code we are executing Linux commands to perform those operations, unfortunately, Linux commands are not getting executed, this is due to some server/set up issue, so I am forced to use Java commands.(We use JDK 1.6)
I am not a Java developer. I have gone through some of the previously answered questions, but they are not explaining about file in server path. Any help is much appreciated.
To get the file size in bytes:
File file = new File("filename.txt");
long fileSize = file.length();
To move a file you must first copy it and then delete the original:
InputStream inStream = null;
OutputStream outStream = null;
try {
File fromFile = new File("startfolder\\filename.txt");
File toFile = new File("endfolder\\filename.txt");
inStream = new FileInputStream(fromFile);
outStream = new FileOutputStream(toFile);
byte[] buffer = new byte[1024];
int length;
while ((length = inStream.read(buffer)) > 0){
outStream.write(buffer, 0, length);
}
inStream.close();
outStream.close();
fromFile.delete();
} catch(IOException e) {
e.printStackTrace();
}
To zip a file:
byte[] buffer = new byte[1024];
try {
FileInputStream fileToZip = new FileInputStream("filename.txt");
FileOutputStream fileOutputStream = new FileOutputStream("filename.zip");
ZipOutputStream zipOutputStream = new ZipOutputStream(fileOutputStream);
ZipEntry zipEntry= new ZipEntry("filename.txt");
zipOutputStream.putNextEntry(zipEntry);
int len;
while ((len = fileToZip.read(buffer)) > 0) {
zipOutputStream.write(buffer, 0, len);
}
fileToZip.close();
zipOutputStream.closeEntry();
zipOutputStream.close();
} catch(IOException e) {
e.printStackTrace();
}
I have my client server chat
Client sends files and server receives them. But, the problem is that, i don't think that files are received properly because when i check the size of the files i see the difference is halfed for some reasons!
I am using GUI to browse for files in the client side, and then i'm sending a command to the server to know that the client is sending a file. But it is not working
Here is the client and server
public void sendFiles(String file) {
try {
BufferedOutputStream outToClient = null;
outToClient = new BufferedOutputStream(sock.getOutputStream());
System.out.println("Sending file...");
if (outToClient != null) {
File myFile = new File( file );
byte[] mybytearray = new byte[(int) myFile.length()];
FileInputStream fis = null;
fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
this.out.println("SF");
bis.read(mybytearray, 0, mybytearray.length);
outToClient.write(mybytearray, 0, mybytearray.length);
this.out.flush();
outToClient.flush();
outToClient.close();
System.out.println("File sent!");
return;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Server
public void recvFile() {
try {
byte[] aByte = new byte[1];
int bytesRead;
InputStream is = null;
is = sock.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
if (is != null) {
FileOutputStream fos = null;
BufferedOutputStream bos = null;
try {
fos = new FileOutputStream("/Users/Documents/Received.png");
bos = new BufferedOutputStream(fos);
bytesRead = is.read(aByte, 0, aByte.length);
do {
baos.write(aByte);
bytesRead = is.read(aByte);
} while (bytesRead != -1);
bos.write(baos.toByteArray());
bos.flush();
bos.close();
// clientSocket.close();
} catch (IOException ex) {
// Do exception handling
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
Can someone help me with this issue? As i don't know how to properly send and receive files
Thank you
You are using two copy techniques, and they are both wrong.
First:
byte[] mybytearray = new byte[(int) myFile.length()];
bis.read(mybytearray, 0, mybytearray.length);
outToClient.write(mybytearray, 0, mybytearray.length);
Here you are assuming:
That the file fits into memory.
That the file length fits into an int.
That read() fills the buffer.
None of these assumptions is valid.
Second:
byte[] aByte = new byte[1];
bytesRead = is.read(aByte, 0, aByte.length);
do {
baos.write(aByte);
bytesRead = is.read(aByte);
} while (bytesRead != -1);
Here you are:
Using a ridiculously small buffer of one byte.
Writing an extra byte if the file length is zero.
Using a do/while where the situation naturally calls for a while (as 99.99% of situations do), and therefore:
Using two read() calls, and only correctly checking the result of one of them.
Pointlessly using a ByteArrayOutputStream, which, as above, assumes the file fits into memory and that its size fits into an int. It also pointlessly adds latency.
Throw them both away and use this, at both ends:
byte[] buffer = new byte[8192];
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
where:
in is a FileInputStream in the case of sending the file, or the socket input stream in the case of receiving the file.
out is a FileOutputStream in the case of receiving the file, or the socket output stream in the case of sending the file
I already tried to download the image like this:
File file4 = new File("C:\\Users\\" + user + "\\AppData\\Roaming"
+ "\\.MINECRAFT2D\\Recources\\"
+ "tileset_texture_new_now.png");
try {
Image image = null;
URL url = new URL("http://www.mediafire.com/view/"
+ "htgmcgtg7yo5swy/tileset_texture_new_now.png");
InputStream in = new BufferedInputStream(url.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n = 0;
while (-1 != (n = in.read(buf))) {
out.write(buf, 0, n);
}
out.close();
in.close();
byte[] response = out.toByteArray();
FileOutputStream fos = new FileOutputStream(file4);
fos.write(response);
fos.close();
} catch (Exception e) {}
but it leaves an un-viewable image in the location. The image will say: "Photo Gallery can't open this photo or video. The file may be unsupported, damaged or corrupted." Is there a way to fix it?
try like this:
byte[] response = out.toByteArray();
Close the stream once you made the byte array
while (-1 != (n = in.read(buf))) {
out.write(buf, 0, n);
}
byte[] response = out.toByteArray();
out.close();
in.close();
Your url is pointing to 'http://www.mediafire.com/view/htgmcgtg7yo5swy/tileset_texture_new_now.png'. It does not resolve to an image/png. I believe that this is the reason of the corrupted image.
Take a look at FileUtils.copyURLToFile(URL, File), from Apache IO Commons. It might help you downloading files.
I am using following code for reading image file from socket. It reads all the bytes from server because size of file on server and android machine are same. When i open this file it does not open the file and generate error that is the file is corrupted or too large.
public Bitmap fileReceived(InputStream is)
throws FileNotFoundException, IOException {
Bitmap bitmap = null;
String baseDir = Environment.getExternalStorageDirectory().getAbsolutePath();
String fileName = "a.png";
String imageInSD = baseDir + File.separator + fileName;
System.out.println(imageInSD);
if (is!= null) {
FileOutputStream fos = null;
BufferedOutputStream bos = null;
try {
fos = new FileOutputStream(imageInSD);
bos = new BufferedOutputStream(fos);
byte[] aByte = new byte[1024];
int bytesRead;
while ( true ) {
bytesRead = is.read(aByte);
bos.write(aByte, 0, bytesRead);
if ( is.available()==0)
break;
}
bos.flush();
bos.close();
// is.reset();
// here it give error i.e --- SkImageDecoder::Factory returned null
bitmap = BitmapFactory.decodeFile(imageInSD);
} catch (IOException ex) {
// Do exception handling
Log.i("IMSERVICE", "exception ");
}
}
return bitmap;
}
Don't use available() for this, it won't work reliably!
The docs state:
[ available() ] Returns an estimate of the number of bytes that can be read [...] It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.
Do it like:
while ( (bytesRead = is.read(aByte)) > 0 ) {
bos.write(aByte, 0, bytesRead);
}
This is a piece of code to output a PDF file to browser, could it be faster?
This is implemented in a Java servlet.
private ByteArrayOutputStream getByteArrayOutputStreamFromFile(File file) throws Exception {
BufferedInputStream bis = null;
ByteArrayOutputStream bos = null;
try {
bis = new BufferedInputStream(new FileInputStream(file));
bos = new ByteArrayOutputStream();
byte[] byteContent = new byte[1024 * 1024];
int len = 0;
while ((len = bis.read(byteContent)) != -1) {
bos.write(byteContent, 0, len);
}
return bos;
} catch (Exception ex) {
throw ex;
} finally {
if (bis != null) {
bis.close();
}
if (bos != null) {
bos.close();
}
}
}
Using Google Guava you can summarize this in one line:
import com.google.common.io.Files;
private OutputStream getOutputStream(File file) throws IOException {
return Files.newOutputStreamSupplier(file).getOutput();
}
response.setContentType("application/pdf");
ServletContext ctx = getServletContext();
InputStream is = ctx.getResourceAsStream("/erp.pdf");
int read =0;
byte[] bytes = new byte[1024];
OutputStream os = response.getOutputStream();
while((read = is.read(bytes)) != -1)
{
os.write(bytes, 0, read);
}
os.flush();
os.close();
A suggestion:
always look to libraries such as Apache Commons FileUtils. They provide simple and easy to use methods.
You can also leave out the BufferedOutputStream as you're already using a buffer. But that's not going to make a big difference. Try using the nio instead of the streams. This might make some difference.
Also look at this: How to download and save a file from Internet using Java? might help you some way.