We Keep Coding

How to read InputStream twice if I am using ReadableByteChannel and BufferedReader?

How to read an InputStream twice if I am using ReadableByteChannel and BufferedReader?
Here is my code:
ReadableByteChannel inputChannel = Channels.newChannel(input);
WritableByteChannel outputChannel = Channels.newChannel(output);
InputStream ind = Channels.newInputStream(inputChannel);
ReadableByteChannel inputChannel1 = Channels.newChannel(ind);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
org.apache.commons.io.IOUtils.copy(ind, baos);
ByteBuffer buffer = ByteBuffer.allocateDirect(10240);
long size = 0;
while (inputChannel1.read(buffer) != -1) {
buffer.flip();
size += outputChannel.write(buffer);
buffer.clear();
}
byte[] bytes = baos.toByteArray();
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
BufferedReader in = new BufferedReader(new InputStreamReader(bais));
String inputLine;
StringBuffer bufferResponse = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
bufferResponse.append(inputLine);
}
JSONObject jsonResponse = new JSONObject(bufferResponse.toString());

You've written a lot of code to copy input to two destinations: output and jsonResponse. As you have made an in-memory copy of input => bytes there is no need to scan input twice, and you don't need to use IOUtils for a simple copy to byte[] which you can re-use to send to the two destinations:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
input.transferTo(baos);
byte[] bytes = baos.toByteArray();
output.write(bytes);
Then do as #g00se suggests - if the char encoding is platform default:
String s = new String(bytes /*, or insert another charset here */);
JSONObject jsonResponse = new JSONObject(s);
You should also deal with closing the input/output streams, best done with try-with-resources block.

Extream usage of RAM memory by ByteArrayOutputStream

So my problem sounds like this. I need to make a base64 encoded string of a file and for this, I use this method:
public String getStringFile(File f) {
InputStream inputStream = null;
String encodedFile= "", lastVal;
try {
inputStream = new FileInputStream(f.getAbsolutePath());
byte[] buffer = new byte[10240];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
while ((bytesRead = inputStream.read(buffer)) != -1) {
output64.write(buffer, 0, bytesRead);
}
output64.close();
encodedFile = output.toString();
} catch (Exception e) {
e.printStackTrace();
}
lastVal = encodedFile;
return lastVal;
}
and the thing is when I try to encode file something around 20 Mb (exact file size is 19,35 Mb) I get an OutOfMemoryException.
Before:
After:
What am I doing wrong and how can I fix this issue? Thanks in advance.

What am I doing wrong
You are attempting to encode a ~20MB file using base64 into a string. You will not have adequate heap space on many Android devices to have a single memory allocation that large.
how can I fix this issue?
If "this issue" is "create a ~26MB string of base64-encoded data", there is no reliable way to do this. You would have to find some other solution to whatever problem you are trying to solve by creating such a string.

ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
If you upload the base64 yourself with HttpUrlConnection you can do way with the ByteArrayOutputStream and replace above lines -while directly uploading- with
OutputStream output = con.getOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
Untested.
You could also directly base64 encode to a FileOutputStream of course.
OutputStream output = new FileOutputStream(......);
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
and then upload that file with POJO.

Can not convert Base64 String and unGzip it properly

I have Base64 String. I am trying to decode it, then decompress it.
String textToDecode = "H4sIAAAAAAAAAAEgAN//0JTQtdGC0LDQu9C40LfQuNGA0L7QstCw0L3QvdGL0LmRCuyiIAAAAA==\n";
byte[] data = Base64.decode(textToDecode, Base64.DEFAULT);
String result = GzipUtil.decompress(data);
Code that I am using for decompression:
public static String decompress(byte[] compressed) throws IOException {
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
I should get this String:
Детализированный
Insteam of it, I am getting this String with question mark symbols:
Детализирован��ый
What is my mistake? And how to solve it?

One problem is that when converting from bytes to String (internally Unicode)
the encoding is not given. And for a multi-byte encoding like UTF-8 one cannot take a fixed number of bytes (like 32) and then at the end have a valid sequence.
You experienced the loss of evidently a half sequence. Hence the encoding probably is UTF-8.
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
baos.write(data, 0, bytesRead);
}
gis.close();
return baos.toString("UTF-8"); // Or "Windows-1251" ...
The above does away with buffer boundary problems, and specifies the encoding, so the same code runs on different computers.
And mind:
new String(bytes, encoding)
string.getBytes(encoding)

It is possible that the problem is here:
string.append(new String(data, 0, bytesRead))
You are using the default character encoding to decode bytes into a Java String. If the (current) default encoding is different to the encoding used when encoding the original characters to bytes (prior to compression, etc), then you could get bytes that don't decode correctly. The decoder will then replace them with the decoder's replacement character; i.e. '\uFFFD' by default.
If this is the problem, then the solution is to find out what the correct character encoding is and use String(byte[], int, int, Charset) to create the String.

If you work only with streams you can avoid encoding problems, this few line of code should do the job well
public static String decompress(byte[] compressed) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
try (GZIPInputStream gis = new GZIPInputStream(
new ByteArrayInputStream(compressed))) {
org.apache.commons.compress.utils.IOUtils.copy(gis, bos);
}
return bos.toString();
}
}

How do you read from an InputStream in Java and convert to byte array?

I am currently trying to read in data from a server response. I am using a Socket to connect to a server, creating a http GET request, then am using a Buffered Reader to read in data. Here is what the code looks like compacted:
Socket conn = new Socket(server, 80);
//Request made here
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String response;
while((response = inFromServer.readLine()) != null){
System.out.println(response);
}
I would like to read in the data, instead of as a String, as a byte array, and write it to a file. How is this possible? Any help is greatly appreciated, thank you.

You need to use a ByteArrayOutputStream, do something like the below code:
Socket conn = new Socket(server, 80);
//Request made here
InputStream is = conn.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int readBytes = -1;
while((readBytes = is.read(buffer)) > 1){
baos.write(buffer,0,readBytes);
}
byte[] responseArray = baos.toByteArray();

One way is to use Apache commons-io IOUtils
byte[] bytes = IOUtils.toByteArray(inputstream);

With plain java:
ByteArrayOutputStream output = new ByteArrayOutputStream();
try(InputStream stream = new FileInputStream("myFile")) {
byte[] buffer = new byte[2048];
int numRead;
while((numRead = stream.read(buffer)) != -1) {
output.write(buffer, 0, numRead);
}
} catch(IOException e) {
e.printStackTrace();
}
// and here your bytes
byte[] myDesiredBytes = output.toByteArray();

If you are not using Apache commons-io library in your project,I have pretty simple method to do the same without using it..
/*
* Read bytes from inputStream and writes to OutputStream,
* later converts OutputStream to byte array in Java.
*/
public static byte[] toByteArrayUsingJava(InputStream is)
throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int reads = is.read();
while(reads != -1){
baos.write(reads);
reads = is.read();
}
return baos.toByteArray();
}

Receive byte[] using ByteArrayInputStream from a socket

Here is the code but got error:
bin = new ByteArrayInputStream(socket.getInputStream());
Is it possible to receive byte[] using ByteArrayInputStream from a socket?

No. You use ByteArrayInputStream when you have an array of bytes, and you want to read from the array as if it were a file. If you just want to read arrays of bytes from the socket, do this:
InputStream stream = socket.getInputStream();
byte[] data = new byte[100];
int count = stream.read(data);
The variable count will contain the number of bytes actually read, and the data will of course be in the array data.

You can't get an instance of ByteArrayInputStream by reading directly from socket.
You require to read first and find byte content.
Then use it to create an instance of ByteArrayInputStream.
InputStream inputStream = socket.getInputStream();
// read from the stream
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] content = new byte[ 2048 ];
int bytesRead = -1;
while( ( bytesRead = inputStream.read( content ) ) != -1 ) {
baos.write( content, 0, bytesRead );
} // while
Now, as you have baos in hand, I don't think you still need a bais instance.
But, to make it complete,
you can generate byte array input stream as below
ByteArrayInputStream bais = new ByteArrayInputStream( baos.toByteArray() );