I am using Java UDP datagrampacket to receive a python packet which contains a Struct.pack string. How could I unpack it in Java?
If you have the packet as byte[] array in Java, you can use java.io.java.io.ByteArrayInputStream to create an InputStream from it which can be wrapped by java.io.DataInputStream which provides methods to read several simple datatypes.
Be aware that DataInputStream works with big endian. If you use little endian some byte juggling will be necessary for multi-byte integer types.
Related
I am creating a simple TCP server in java which accepts clients. Each client that connects to the server sends a message, and sends an answer based on the message.
In order to accept clients I am using ServerSocket class. In order to read the client's message and write to him I am allowed to use only in the Socket, DataOutputStream and DataInputStream classes and in StandardCharsets.UTF_8. In addition, every message that the server gets and sends must be in UTF-8 encoding.
However, I am not sure how to read and write messages in UTF-8 encoding using those classes. The size of the messages is unbounded. I tried to read about the read function of DataInputStream class, but I couldn't understand how to use it (if this indeed the function I need to use).
DataOutputStream has a writeUTF() method, and DataInputStream has a readUTF() method. Just note that these methods deal in modified UTF-8 rather than standard UTF-8, and they limit the UTF-8 data to 65535 bytes max.
To provide interoperability with non-Java clients, and/or handle longer strings, would be better to just handle the UTF-8 yourself manually.
The sender can use String.getBytes() to encode a String to a byte[] array using StandardCharsets.UTF_8, then send the array's length using DataOutputStream.writeInt(), and then finally send the actual bytes using DataOutputStream.write(byte[]).
The receiver can then read the array length using DataInputStream.readInt(), then allocate a byte[] array and read it using DataInputStream.readFully(), and then finally use the String constructor to decode the bytes using StandardCharsets.UTF_8.
I have a ruby program that writes data to a socket with sock.write, and I'm reading the data with ObjectInputStream in a java file. I'm getting an invalid header error that translate to the first few characters of my stream.
I've read that if you use ObjectInputStream you must write with ObjectOutputStream, but since the writing file is in ruby im not sure how to accomplish this.
As you say, ObjectInputStream assumes that the bytes it's receiving have been formatted by an ObjectOutputStream. That is, it is expecting the incoming bytes to be a specific representation of a Java primitive or object.
Your Ruby code is unlikely to format bytes in such a way.
You need to define exactly the byte format of the message passing from the Ruby to the Java process. You could tell us more about that message format, but it's likely you will need to use Java's ByteArrayInputStream (https://docs.oracle.com/javase/7/docs/api/java/io/ByteArrayInputStream.html). The data will come into the Java program as a raw array of bytes, and you will need to parse/unpack/process these bytes into whatever objects are appropriate.
Unless performance is critical, you'd probably be best off using JSON or YAML as the intermediate format. They would make it simple to send simple objects such as strings, arrays, and hashes (maps).
I have a Java client connected via socket to a C++ server.
The C++ server sends back to the client serialized objects.
However serialization works differently for Java and C++, so I cannot read the objects in that way:
objectInputStream.readObject();
This forces me to read each single value of the object manually:
byte[] buffer = read(FOUR_BYTES);
int flag = convertBufferToInt(buffer);
byte[] buffer = read(FOUR_BYTES);
float price = convertBufferToFloat(buffer);
// More stuff
myObject.setFlag(flag);
myObject.setPrice(price);
// More stuff
That's very hard to maintain. Isn't there an easier way to fill in my object with data?
To solve this in general you would need to write a C++ parser for objects serialized in Java. This is no small task.
Rather, I would recommend that you find some serialization format that is easy to parse and share between your Java and C++ programs. Preferably a format where there exists Java as well as C++ libraries for the serialization/deserialization. JSON or Google Protocol Buffers are obvious candidates.
Yes there is (are). You have 2 options using only the standard library:
Using the DataInputStream class
Check out the DataInputStream class. It has methods to read values of primitive types like readByte(), readInt(), readLong(), readFloat(), readChar(), readUTF() (for reading UTF-8 encoded String) etc.
So your code becomes as simple as:
// Obtain InputStream from Socket:
InputStream is = ...;
// Create DataInputStream:
DataInputStream dis = new DataInputStream(is);
myObject.setFlag(dis.readInt());
myObject.setPrice(dis.readFloat());
Using the ByteBuffer class
For this you have to read first the whole data into a byte array. Once you've done that, you can create a ByteBuffer using the ByteBuffer.wrap(byte[] array) method. The ByteBuffer class also supports reading primitive types just like the DataInputStream class.
The good thing about ByteBuffer that it supports changing the byte order (the order how the low and high bytes of a multi-byte value like int are read/written): ByteBuffer.order(ByteOrder bo). This is very useful if you're communicating with systems which use a differnet byte order (which might apply in your case).
Example using ByteBuffer:
// Read all your input data:
byte[] data = ...;
// Create ByteBuffer:
ByteBuffer bb = ByteBuffer.wrap(data);
myObject.setFlag(bb.getInt());
myObject.setPrice(bb.getFloat());
theres a guy in my job that connects to a socket thru C using structs of bytes. How can i do the same in Android/Java. I know i can add more info, but i think i need a few answers to develop more into the right point.
Thanks in advance
Read up on the java.net.Socket class.
Upd: getOutputStream() gives you an OutputStream instance. You can write byte arrays directly into it. To simulate C datatypes, you'll have to decompose your variables into bytes using shift and mask operators. For example, to write an int (assuming it's little-endian on the wire), use:
int n;
MyOutStream.write(new byte[]{n&0xff, (n>>8)&0xff, (n>>16)&0xff, (n>>24)&0xff});
For strings, use toByteArray() with the appropriate encoding ("US-ASCII" is not sure).
There's also the DataOutputStream class, which can do the same for you, but IIRC it writes a small header in front of the data, so the wire data will be incompatible with the existing protocol.
Java newbie here. Are there any helper functions to serialize data in and out of byte arrays? I am writing a Java package that implements a network protocol. So I have to write some typical variables like a version (1byte), sequence Number (long) and binary data (bytes) in a loop. How do I do this in Java? Coming from C I am thinking of creating a byte array of the required size and then since there is no memcpy() I am converting the long into a temporary byte array and then copying it into the actual byte array. It seems so inefficient and also really error prone. Is there a class I could use to marshall and unmarshall parameters to a byte array?
Also why does all the Socket classes only deals with char[] and not byte[]? A socket by definition has to deal with binary data also. How is this done in Java?
I am sure what I am missing is the Java mindset. Appreciate it if some one can point it to me.
EDIT: I did look at DataOutputStream and DataInputStream but I cannot convert the bytes to a String not to a byte[] which means the information might be lost in the conversion to write to a socket.
Pav
Have a look at DataInputStream, DataOutputStream, ObjectInputStream and ObjectOutputStream. Check first if the layout of the data is acceptable to you. Also, Serialization.
Sockets neither deal with char[] nor with byte[] but with InputStream and OutputStream which are used to read and write bytes.
If you are sending the data over a socket, then you don't need a temporary byte array at all; you can wrap the socket's OutputStream with DataOutputStream or ObjectOutputStream and just write what you want to write.
There might be an aspect I've missed that means you do actually need temporary byte arrays. If so, look at ByteArrayOutputStream. Also, there's no memcpy(), sure, but there is System.arraycopy.
As above, DataInputStream and DataOutputStream are exactly what you are looking for. Re your comment about String, if you're planning to use Java Strings over the wire, you're not designing a network protocol, youre designing a Java protocol. There are readUTF() and writeUTF() if you're sure the other end is Java or if you can code the other end to understand these formats. Or you can send as bytes along with the appropriate charset, or predefine the charset for the entire protocol if that makes sense.