We Keep Coding

java.io.StreamCorruptedException: invalid stream header: 00000000 when reading from ObjectInputStream

I'm writing some networking code that writes basic packet constructs and sends them over a SocketChannel. I have the following relevant code:
in IOPacket (abstract implementation of the Packet interface, main thing getting sent over the channel):
//returns a bytebuffer representation of the object
#Override
public ByteBuffer write() throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(this);
oos.flush();
oos.close();
ByteBuffer buf = ByteBuffer.wrap(baos.toByteArray());
buf.position(0);
return buf;
}
//returns a packet read from the given bytebuffer
public static Packet read(ByteBuffer buffer) throws IOException {
buffer.position(0);
System.out.println("[SQNetEvents] Remaining: " + buffer.remaining());
ByteArrayInputStream bais = new ByteArrayInputStream(buffer.array(), buffer.arrayOffset(), buffer.remaining());
ObjectInputStream ois = new ObjectInputStream(bais);
Packet packet = null;
try {
packet = (Packet) ois.readObject();
}
catch(Exception e) {
throw new IOException("Read object is not an implementation of Packet");
}
ois.close();
return packet;
}
in Connection (wrapper for a SocketChannel and all the behaviors needed to read, write, connect, etc.)
//sends a packet to the associated address
public void sendPacket(Packet packet) throws IOException {
System.out.println("In sendPacket()");
int length = getChannel().write(packet.write());
System.out.println("[SQNetEvents] Sent packet of length " + length + " to " + getChannel().getRemoteAddress().toString());
}
private class ListenThread extends Thread {
...
byte[] bytes = new byte[4096];
ByteBuffer buffer = ByteBuffer.wrap(bytes);
getChannel().read(buffer);
System.out.println("In Connection$ListenThread.run()");
Packet packet = IOPacket.read(buffer);
...
}
I get the StreamCorruptedException on the IOPacket.read(ByteBuffer) call, from the ObjectInputStream constructor call. I've been able to successfully send a string literal ("String1") converted to a byte[] and then wrapped in a ByteBuffer over the SocketChannel and read it without dealing with the ByteArray or Object streams, so I know that the issue must be somewhere in the stream code. The code on both ends is identical, and blocking is configured true for the channels. Any help is appreciated, thanks.

Can not convert Base64 String and unGzip it properly

I have Base64 String. I am trying to decode it, then decompress it.
String textToDecode = "H4sIAAAAAAAAAAEgAN//0JTQtdGC0LDQu9C40LfQuNGA0L7QstCw0L3QvdGL0LmRCuyiIAAAAA==\n";
byte[] data = Base64.decode(textToDecode, Base64.DEFAULT);
String result = GzipUtil.decompress(data);
Code that I am using for decompression:
public static String decompress(byte[] compressed) throws IOException {
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
I should get this String:
Детализированный
Insteam of it, I am getting this String with question mark symbols:
Детализирован��ый
What is my mistake? And how to solve it?

One problem is that when converting from bytes to String (internally Unicode)
the encoding is not given. And for a multi-byte encoding like UTF-8 one cannot take a fixed number of bytes (like 32) and then at the end have a valid sequence.
You experienced the loss of evidently a half sequence. Hence the encoding probably is UTF-8.
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
baos.write(data, 0, bytesRead);
}
gis.close();
return baos.toString("UTF-8"); // Or "Windows-1251" ...
The above does away with buffer boundary problems, and specifies the encoding, so the same code runs on different computers.
And mind:
new String(bytes, encoding)
string.getBytes(encoding)

It is possible that the problem is here:
string.append(new String(data, 0, bytesRead))
You are using the default character encoding to decode bytes into a Java String. If the (current) default encoding is different to the encoding used when encoding the original characters to bytes (prior to compression, etc), then you could get bytes that don't decode correctly. The decoder will then replace them with the decoder's replacement character; i.e. '\uFFFD' by default.
If this is the problem, then the solution is to find out what the correct character encoding is and use String(byte[], int, int, Charset) to create the String.

If you work only with streams you can avoid encoding problems, this few line of code should do the job well
public static String decompress(byte[] compressed) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
try (GZIPInputStream gis = new GZIPInputStream(
new ByteArrayInputStream(compressed))) {
org.apache.commons.compress.utils.IOUtils.copy(gis, bos);
}
return bos.toString();
}
}

How do you read from an InputStream in Java and convert to byte array?

I am currently trying to read in data from a server response. I am using a Socket to connect to a server, creating a http GET request, then am using a Buffered Reader to read in data. Here is what the code looks like compacted:
Socket conn = new Socket(server, 80);
//Request made here
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String response;
while((response = inFromServer.readLine()) != null){
System.out.println(response);
}
I would like to read in the data, instead of as a String, as a byte array, and write it to a file. How is this possible? Any help is greatly appreciated, thank you.

You need to use a ByteArrayOutputStream, do something like the below code:
Socket conn = new Socket(server, 80);
//Request made here
InputStream is = conn.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int readBytes = -1;
while((readBytes = is.read(buffer)) > 1){
baos.write(buffer,0,readBytes);
}
byte[] responseArray = baos.toByteArray();

One way is to use Apache commons-io IOUtils
byte[] bytes = IOUtils.toByteArray(inputstream);

With plain java:
ByteArrayOutputStream output = new ByteArrayOutputStream();
try(InputStream stream = new FileInputStream("myFile")) {
byte[] buffer = new byte[2048];
int numRead;
while((numRead = stream.read(buffer)) != -1) {
output.write(buffer, 0, numRead);
}
} catch(IOException e) {
e.printStackTrace();
}
// and here your bytes
byte[] myDesiredBytes = output.toByteArray();

If you are not using Apache commons-io library in your project,I have pretty simple method to do the same without using it..
/*
* Read bytes from inputStream and writes to OutputStream,
* later converts OutputStream to byte array in Java.
*/
public static byte[] toByteArrayUsingJava(InputStream is)
throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int reads = is.read();
while(reads != -1){
baos.write(reads);
reads = is.read();
}
return baos.toByteArray();
}

DataInputStream and OutputStream write/read string with length

How to read String with DataInputStream, which was stored with this code:
DataOutputStream dataOut = new DataOutputStream (out); // Some other stream
String title = processed.getTitle();
dataOut.writeInt(title.length());
dataOut.writeBytes(title);

You can read like this.
DataInputStream dataIn = new DataInputStream (input);
int length = dataIn.readInt();
byte[] array = new byte[length];
dataIn.read(array);

You can use ByteArrayOutputStream and ByteArrayInputStream and a byte array as intermediate buffer..
ByteArrayOutputStream out = new ByteArrayOutputStream();
// Some other streams
DataOutputStream dataOut = new DataOutputStream (out);
String title = processed.getTitle();
dataOut.writeInt(title.length());
dataOut.writeBytes(title);
ByteArrayInputStream in = new ByteArrayInputStream(out.toByteArray());
DataInputStream dataIn = new DataInputStream(in);

dataOut.writeUTF(title);
// ...
String title = dataIn.readUTF();
... provided the title doesn't need more than 65533 bytes when written in this format: see the Javadoc.

converting DataHandler to byte[]

I need a code snippt for converting DataHandler to byte[].
This data handler contains Image.

It can be done by using below code without much effort using apache IO Commons.
final InputStream in = dataHandler.getInputStream();
byte[] byteArray=org.apache.commons.io.IOUtils.toByteArray(in);

You can do it like this:
public static byte[] toBytes(DataHandler handler) throws IOException {
ByteArrayOutputStream output = new ByteArrayOutputStream();
handler.writeTo(output);
return output.toByteArray();
}

private static final int INITIAL_SIZE = 1024 * 1024;
private static final int BUFFER_SIZE = 1024;
public static byte[] toBytes(DataHandler dh) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream(INITIAL_SIZE);
InputStream in = dh.getInputStream();
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
while ( (bytesRead = in.read(buffer)) >= 0 ) {
bos.write(buffer, 0, bytesRead);
}
return bos.toByteArray();
}
Beware that ByteArrayOutputStream.toByteArray() creates a copy of the internal byte array.

I use this code:
public static byte[] getContentAsByteArray(DataHandler handler) throws IOException {
byte[] bytes = null;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
handler.writeTo(bos);
bos.flush();
bos.close();
bytes = bos.toByteArray();
return bytes;
}

Is something like this what you are looking for?
public static byte[] getBytesFromDataHandler(final DataHandler data) throws IOException {
final InputStream in = data.getInputStream();
byte out[] = new byte[0];
if(in != null) {
out = new byte[in.available()];
in.read(out);
}
return out;
}
UPDATE:
Based on dkarp's comment this is incorrect. According to the docs for InputStream:
Returns the number of bytes that can be read (or skipped over) from this input stream without blocking by the next caller of a method for this input stream. The next caller might be the same thread or or another thread.
It looks like Costi has the correct answer here.