Extream usage of RAM memory by ByteArrayOutputStream - java

So my problem sounds like this. I need to make a base64 encoded string of a file and for this, I use this method:
public String getStringFile(File f) {
InputStream inputStream = null;
String encodedFile= "", lastVal;
try {
inputStream = new FileInputStream(f.getAbsolutePath());
byte[] buffer = new byte[10240];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
while ((bytesRead = inputStream.read(buffer)) != -1) {
output64.write(buffer, 0, bytesRead);
}
output64.close();
encodedFile = output.toString();
} catch (Exception e) {
e.printStackTrace();
}
lastVal = encodedFile;
return lastVal;
}
and the thing is when I try to encode file something around 20 Mb (exact file size is 19,35 Mb) I get an OutOfMemoryException.
Before:
After:
What am I doing wrong and how can I fix this issue? Thanks in advance.

What am I doing wrong
You are attempting to encode a ~20MB file using base64 into a string. You will not have adequate heap space on many Android devices to have a single memory allocation that large.
how can I fix this issue?
If "this issue" is "create a ~26MB string of base64-encoded data", there is no reliable way to do this. You would have to find some other solution to whatever problem you are trying to solve by creating such a string.

ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
If you upload the base64 yourself with HttpUrlConnection you can do way with the ByteArrayOutputStream and replace above lines -while directly uploading- with
OutputStream output = con.getOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
Untested.
You could also directly base64 encode to a FileOutputStream of course.
OutputStream output = new FileOutputStream(......);
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
and then upload that file with POJO.

Related

send images through java android sockets

I want to send images through sockets but I have not been able to do it in android, could someone help me?
System.out.println("iniciooooo");
//converting image to bytes with base64
Bitmap b = BitmapFactory.decodeFile("/sdcard/ajeffer.jpg");
ByteArrayOutputStream byte2= new ByteArrayOutputStream();
b.compress(Bitmap.CompressFormat.JPEG,70,byte2);
byte[] enbytes = byte2.toByteArray();
String bb = Base64.encodeToString(enbytes,Base64.DEFAULT);
System.out.println(Base64.encodeToString(enbytes,Base64.DEFAULT));
data.writeUTF(bb);
FileOutputStream file;
//receiving the image in bytes to convert it into an image
DataInputStream dain = new DataInputStream(s.getInputStream());
msg = dain.readUTF();
File ff = new File("/sdcard/a2jeffer.jpg");
byte[] deco = Base64.decode(dain.readUTF(),Base64.DEFAULT);
Bitmap bit = BitmapFactory.decodeByteArray(deco,0,deco.length);
file = new FileOutputStream(ff);
bit.compress(Bitmap.CompressFormat.JPEG,70,file);
//the image is not created
I realized that my code did not work because I had to put this android: requestLegacyExternalStorage =" true " in the manifest, also I see that you are right about writeUTF () since in order to send images I must drastically lower the quality but it works If you have an idea on how to improve this, let me know, thank you very much.
You were right, this works great for sending and receiving any file.
Send file
OutputStream outputStream = socket.getOutputStream();
InputStream inputStream = new FileInputStream(file);
byte[] datita = new byte[16*1024];
int count;
while((count = inputStream.read(datita))>0){
outputStream.write(datita,0,count);
}
outputStream.close();
inputStream.close();
Receive file
OutputStream outputStream = new FileOutputStream(file);
InputStream inputStream = s.getInputStream();
byte[] datita = new byte[16*1024];
int count;
while((count = inputStream.read(datita))>0){
outputStream.write(datita,0,count);
}
outputStream.close();
inputStream.close();

Upload large files to the Google Drive

In my app, I'm uploading files to the google drive using GD API. It works fine for small file sizes, but when file size is large (ex: 200MB), it throws java.lang.OutOfMemoryError: exception. I know why it crashes it loads the whole data into the memory, can anyone suggest how can I fix this problem?
This is my code:
OutputStream outputStream = result.getDriveContents().getOutputStream();
FileInputStream fis;
try {
fis = new FileInputStream(file.getPath());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[8192];
int n;
while (-1 != (n = fis.read(buf)))
baos.write(buf, 0, n);
byte[] photoBytes = baos.toByteArray();
outputStream.write(photoBytes);
outputStream.close();
outputStream = null;
fis.close();
fis = null;
} catch (FileNotFoundException e) {
}
This line would allocate 200 MB of RAM and can definitely cause OutOfMemoryError exception:
byte[] photoBytes = baos.toByteArray();
Why are you not writing directly to your outputStream:
while (-1 != (n = fis.read(buf)))
outputStream.write(buf, 0, n);

Load image in a String and save after change image

I'm trying load an image in a string and after do something with this String, save the image.
The problem appear when i try to asignate the value of the FileInputStream to the String targetFileStr. If i don't to this, and I save the image, everything it's ok, but when i save it on the String, the image change, no matter if I try save the image from the String or from the FileInputStream.
FileInputStream fis = null;
File file = new File("image.png");
fis = new FileInputStream(file);
String targetFileStr = IOUtils.toString(fis, "UTF-8");
*InputStream inputStream = IOUtils.toInputStream(targetFileStr, "UTF-8");
*InputStream inputStream = fis;
// no matter which one i use, both ways fail
OutputStream outputStream = null;
try {
outputStream = new FileOutputStream(new File("image2.png"));
int read = 0;
byte[] bytes = new byte[1024];
while ((read = inputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, read);
}
} catch (Exception e) {
e.printStackTrace();
}
You may want to consider converting the image into a String via Base64 encoding/decoding. This is an example of encoding.
After encoding, you can modify the String (actually you create new strings, you cannot modify the existing one), but be sure to produce valide base64-encoded outputs, otherwise you won't be able to decode.

How can I put a downloadable file into the HttpServletResponse?

I have the following problem: I have an HttpServlet that create a file and return it to the user that have to receive it as a download
byte[] byteArray = allegato.getFile();
InputStream is = new ByteArrayInputStream(byteArray);
Base64InputStream base64InputStream = new Base64InputStream(is);
int chunk = 1024;
byte[] buffer = new byte[chunk];
int bytesRead = -1;
OutputStream out = new ByteArrayOutputStream();
while ((bytesRead = base64InputStream.read(buffer)) != -1) {
out.write(buffer, 0, bytesRead);
}
As you can see I have a byteArray object that is an array of bytes (byte[] byteArray) and I convert it into a file in this way:
First I convert it into an InputStream object.
Then I convert the InputStream object into a Base64InputStream.
Finally I write this Base64InputStream on a ByteArrayOutputStream object (the OutputStream out object).
I think that up to here it should be ok (is it ok or am I missing something in the file creation?)
Now my servlet have to return this file as a dowload (so the user have to receive the download into the browser).
So what have I to do to obtain this behavior? I think that I have to put this OutputStream object into the Servlet response, something like:
ServletOutputStream stream = res.getOutputStream();
But I have no idea about how exactly do it? Have I also to set a specific MIME type for the file?
It's pretty easy to do.
byte[] byteArray = //your byte array
response.setContentType("YOUR CONTENT TYPE HERE");
response.setHeader("Content-Disposition", "filename=\"THE FILE NAME\"");
response.setContentLength(byteArray.length);
OutputStream os = response.getOutputStream();
try {
os.write(byteArray , 0, byteArray.length);
} catch (Exception excp) {
//handle error
} finally {
os.close();
}
EDIT:
I've noticed that you are first decoding your data from base64, the you should do the following:
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[chunk];
int bytesRead = -1;
while ((bytesRead = base64InputStream.read(buffer)) != -1) {
os.write(buffer, 0, bytesRead);
}
You do not need the intermediate ByteArrayOutputStream
With org.apache.commons.compress.utils.IOUtils you can just "copy" from one file or stream (e.g. your base64InputStream) to the output stream:
response.setContentType([your file mime type]);
IOUtils.copy(base64InputStream, response.getOutputStream());
response.setStatus(HttpServletResponse.SC_OK);
You'll find that class here https://mvnrepository.com/artifact/org.apache.commons/commons-compress
A similar class (also named IOUtils) is also in Apache Commons IO (https://mvnrepository.com/artifact/commons-io/commons-io).

How do you read from an InputStream in Java and convert to byte array?

I am currently trying to read in data from a server response. I am using a Socket to connect to a server, creating a http GET request, then am using a Buffered Reader to read in data. Here is what the code looks like compacted:
Socket conn = new Socket(server, 80);
//Request made here
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String response;
while((response = inFromServer.readLine()) != null){
System.out.println(response);
}
I would like to read in the data, instead of as a String, as a byte array, and write it to a file. How is this possible? Any help is greatly appreciated, thank you.
You need to use a ByteArrayOutputStream, do something like the below code:
Socket conn = new Socket(server, 80);
//Request made here
InputStream is = conn.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int readBytes = -1;
while((readBytes = is.read(buffer)) > 1){
baos.write(buffer,0,readBytes);
}
byte[] responseArray = baos.toByteArray();
One way is to use Apache commons-io IOUtils
byte[] bytes = IOUtils.toByteArray(inputstream);
With plain java:
ByteArrayOutputStream output = new ByteArrayOutputStream();
try(InputStream stream = new FileInputStream("myFile")) {
byte[] buffer = new byte[2048];
int numRead;
while((numRead = stream.read(buffer)) != -1) {
output.write(buffer, 0, numRead);
}
} catch(IOException e) {
e.printStackTrace();
}
// and here your bytes
byte[] myDesiredBytes = output.toByteArray();
If you are not using Apache commons-io library in your project,I have pretty simple method to do the same without using it..
/*
* Read bytes from inputStream and writes to OutputStream,
* later converts OutputStream to byte array in Java.
*/
public static byte[] toByteArrayUsingJava(InputStream is)
throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int reads = is.read();
while(reads != -1){
baos.write(reads);
reads = is.read();
}
return baos.toByteArray();
}

Categories