Java newbie here. Are there any helper functions to serialize data in and out of byte arrays? I am writing a Java package that implements a network protocol. So I have to write some typical variables like a version (1byte), sequence Number (long) and binary data (bytes) in a loop. How do I do this in Java? Coming from C I am thinking of creating a byte array of the required size and then since there is no memcpy() I am converting the long into a temporary byte array and then copying it into the actual byte array. It seems so inefficient and also really error prone. Is there a class I could use to marshall and unmarshall parameters to a byte array?
Also why does all the Socket classes only deals with char[] and not byte[]? A socket by definition has to deal with binary data also. How is this done in Java?
I am sure what I am missing is the Java mindset. Appreciate it if some one can point it to me.
EDIT: I did look at DataOutputStream and DataInputStream but I cannot convert the bytes to a String not to a byte[] which means the information might be lost in the conversion to write to a socket.
Pav
Have a look at DataInputStream, DataOutputStream, ObjectInputStream and ObjectOutputStream. Check first if the layout of the data is acceptable to you. Also, Serialization.
Sockets neither deal with char[] nor with byte[] but with InputStream and OutputStream which are used to read and write bytes.
If you are sending the data over a socket, then you don't need a temporary byte array at all; you can wrap the socket's OutputStream with DataOutputStream or ObjectOutputStream and just write what you want to write.
There might be an aspect I've missed that means you do actually need temporary byte arrays. If so, look at ByteArrayOutputStream. Also, there's no memcpy(), sure, but there is System.arraycopy.
As above, DataInputStream and DataOutputStream are exactly what you are looking for. Re your comment about String, if you're planning to use Java Strings over the wire, you're not designing a network protocol, youre designing a Java protocol. There are readUTF() and writeUTF() if you're sure the other end is Java or if you can code the other end to understand these formats. Or you can send as bytes along with the appropriate charset, or predefine the charset for the entire protocol if that makes sense.
Related
I have a ruby program that writes data to a socket with sock.write, and I'm reading the data with ObjectInputStream in a java file. I'm getting an invalid header error that translate to the first few characters of my stream.
I've read that if you use ObjectInputStream you must write with ObjectOutputStream, but since the writing file is in ruby im not sure how to accomplish this.
As you say, ObjectInputStream assumes that the bytes it's receiving have been formatted by an ObjectOutputStream. That is, it is expecting the incoming bytes to be a specific representation of a Java primitive or object.
Your Ruby code is unlikely to format bytes in such a way.
You need to define exactly the byte format of the message passing from the Ruby to the Java process. You could tell us more about that message format, but it's likely you will need to use Java's ByteArrayInputStream (https://docs.oracle.com/javase/7/docs/api/java/io/ByteArrayInputStream.html). The data will come into the Java program as a raw array of bytes, and you will need to parse/unpack/process these bytes into whatever objects are appropriate.
Unless performance is critical, you'd probably be best off using JSON or YAML as the intermediate format. They would make it simple to send simple objects such as strings, arrays, and hashes (maps).
I have a Java client connected via socket to a C++ server.
The C++ server sends back to the client serialized objects.
However serialization works differently for Java and C++, so I cannot read the objects in that way:
objectInputStream.readObject();
This forces me to read each single value of the object manually:
byte[] buffer = read(FOUR_BYTES);
int flag = convertBufferToInt(buffer);
byte[] buffer = read(FOUR_BYTES);
float price = convertBufferToFloat(buffer);
// More stuff
myObject.setFlag(flag);
myObject.setPrice(price);
// More stuff
That's very hard to maintain. Isn't there an easier way to fill in my object with data?
To solve this in general you would need to write a C++ parser for objects serialized in Java. This is no small task.
Rather, I would recommend that you find some serialization format that is easy to parse and share between your Java and C++ programs. Preferably a format where there exists Java as well as C++ libraries for the serialization/deserialization. JSON or Google Protocol Buffers are obvious candidates.
Yes there is (are). You have 2 options using only the standard library:
Using the DataInputStream class
Check out the DataInputStream class. It has methods to read values of primitive types like readByte(), readInt(), readLong(), readFloat(), readChar(), readUTF() (for reading UTF-8 encoded String) etc.
So your code becomes as simple as:
// Obtain InputStream from Socket:
InputStream is = ...;
// Create DataInputStream:
DataInputStream dis = new DataInputStream(is);
myObject.setFlag(dis.readInt());
myObject.setPrice(dis.readFloat());
Using the ByteBuffer class
For this you have to read first the whole data into a byte array. Once you've done that, you can create a ByteBuffer using the ByteBuffer.wrap(byte[] array) method. The ByteBuffer class also supports reading primitive types just like the DataInputStream class.
The good thing about ByteBuffer that it supports changing the byte order (the order how the low and high bytes of a multi-byte value like int are read/written): ByteBuffer.order(ByteOrder bo). This is very useful if you're communicating with systems which use a differnet byte order (which might apply in your case).
Example using ByteBuffer:
// Read all your input data:
byte[] data = ...;
// Create ByteBuffer:
ByteBuffer bb = ByteBuffer.wrap(data);
myObject.setFlag(bb.getInt());
myObject.setPrice(bb.getFloat());
I have a BufferedReader object and a PrintWriter object. So I can work passing String objects made by json-io of any type (e.g.: List, Map, MyOwnClass)
My class have a byte[] attribute, this byte[] will keep a file bytes, such as an image.
The json generated of my class is very very big, obviously... Then i started to think that must have a better way to transfer files.
Should I change all the mechanism to transfer only byte[] instead of String? Does someone know what is the mechanism used by chat programs? Should I reserve the first 20 bytes of the array for the message identification?
I would write it to the socket in binary:
Assuming a class with one String and one byte[].
The String
The length of the String is written with DataOutputStream.writeInt(int) (or methods for smaller integers) and then OutputStream.write(byte[]) on the return value of String.getBytes(String) with the charset explicitly specified.
The byte[]
The length is written with DataOutputStream.writeInt(int) (or methods for smaller integers) and then OutputStream.write(byte[]) for the byte[] to transfer.
On the other side you would do the exact opposite of this procedure.
I chose this binary approach over JSON because even though you could transmit the byte[] with JSON almost as efficiently as in binary, it would defeat the very purpose of JSON: being human-readable.
I am using Java UDP datagrampacket to receive a python packet which contains a Struct.pack string. How could I unpack it in Java?
If you have the packet as byte[] array in Java, you can use java.io.java.io.ByteArrayInputStream to create an InputStream from it which can be wrapped by java.io.DataInputStream which provides methods to read several simple datatypes.
Be aware that DataInputStream works with big endian. If you use little endian some byte juggling will be necessary for multi-byte integer types.
theres a guy in my job that connects to a socket thru C using structs of bytes. How can i do the same in Android/Java. I know i can add more info, but i think i need a few answers to develop more into the right point.
Thanks in advance
Read up on the java.net.Socket class.
Upd: getOutputStream() gives you an OutputStream instance. You can write byte arrays directly into it. To simulate C datatypes, you'll have to decompose your variables into bytes using shift and mask operators. For example, to write an int (assuming it's little-endian on the wire), use:
int n;
MyOutStream.write(new byte[]{n&0xff, (n>>8)&0xff, (n>>16)&0xff, (n>>24)&0xff});
For strings, use toByteArray() with the appropriate encoding ("US-ASCII" is not sure).
There's also the DataOutputStream class, which can do the same for you, but IIRC it writes a small header in front of the data, so the wire data will be incompatible with the existing protocol.