I am coding some sort of packet which has different fields with different length in bytes.
So field1 is 2 byte long field2 is 3 bytes long field3 is 6 bytes long and when I addup these fields, I should be getting 11 bytes in length.
But I have no idea how I can declare something with this byte long.
Use an array:
byte[] byteArray = new byte[11];
How's about:
byte[] arr = new byte[11];
You could use a class to represent your packet:
public class Packet
{
public byte[] Field1, Field2, Field3;
public Packet(byte[] packetBytes)
{
ByteBuffer packet = ByteBuffer.wrap(packetBytes);
Field1 = new byte[2];
Field2 = new byte[3];
Field3 = new byte[6];
packet.get(Field1, 0, 2);
packet.get(Field2, 2, 3);
packet.get(Field3, 5, 6);
}
}
ByteBuffer is good for byte-manipulation.
I have found that java.nio.ByteBuffer is typically better for this sort of thing. It has nice methods for dealing with interpreting the bytes in the buffer. The docs are here.
import java.nio.ByteBuffer;
ByteBuffer buffer = ByteBuffer.allocate(11);
Check out the docs and look at the nice methods such as getInt() and getChar().
Java has a limited collection of primitive types, which all have a fixed size. You can see a list of them here. That means you can't decide how many bytes your variable will consist of.
Of course, as others have already mentioned, you can always create a new byte[11]. Note that Java's byte is signed, however. It goes from -128 to 127, not from 0 to 255.
I recommend the utility classes in Javolution for dealing with binary protocol streams such as this. They've come in handy for me several times when dealing with low-level binary streams.
You should probably design your code to separate the message you want to manipulate in java from the wire level format you need to read/write.
e.g. If you have a ScreenResolution concept, you could represent it in java with a ScreenResolution class:
public class ScreenResolution {
public int height;
public int width;
}
This class is easy to work with in Java. Transforming this to a packet that can be transmitted over a network/saved to a file, etc. according to some file format or protocol is another concern.
Say the height and width is to be laid out in 3 bytes each, with some ID and length for the "wire format", you make something like
public byte[] marshalScreenResolution(ScreenResolution obj) {
byte[] buf = new byte[9];
//length of this packet, 2 bytes
buf[0] = 0;
buf[1] = 9;
buf[2] = SCREENRESOLUTION_OPCODE;
//marshal the height/width , 3 least significant bytes.
buf[3] = (obj.height&0xff0000) >> 16;
buf[4] = (obj.height&0x00ff00) >> 8;
buf[5] = (obj.height&0x0000ff) ;
buf[6] = (obj.width&0xff0000) >> 16;
buf[7] = (obj.width&0x00ff00) >> 8;
buf[8] = (obj.width&0x0000ff) ;
return buf;
}
And you make a demarshalScreenResolution function for going from a packet to a ScreenResolution object. The point is you decouple the representation in java from the external representation, and you assemble the fields in the external representation using bytes + some basic bit fiddling.
Related
The problem:
I need to implement a network input reader from an electronic device which has fixed definitions of a packet format. It has a header, footer and data body. The fields in the packet are defined in term of bits.
Example:
The field sequence_number variable is the bits numbered 8 to 31 and ack_status is bit 32. How do I create a custom serialization function so that I can convert a java object to and from a byte[] read from a packet?
In a 32 Bit Packet
StartOfPacketHeader 4bits
CommandCode 4bits
SequenceNumber 23bits
AckStatus 1bit
How do I make a Java wrapper class for this object with a byte[] constructor and a toBytes() method.
Using the Serializable Interface has no control on individual data sizes. The data sizes may be odd values like a 3-bit or 18-bit variable. I cannot specify the length of the variables in terms of bits.
What I have done so far:
The source actually sends unsigned integers, meaning Java int type cannot handle its entire range. I need to define all data types as long to handle all cases. I have used the BitSet class to carry out byte[] to Long conversion but it's just too tedious and confusing as bitset reverses the bit order to Little Endian by default.
Summary:
How do I convert to my object from the bytes[] I read from the network and vice versa with custom definitions for each field in the most efficient way?
You use DataInputStream and DataOutputStream for this, with a BufferedInputStream or BufferedOutputStream under them respectively for efficiency purposes. The bit fields you will have to program yourself, but these classes provide all the Java primitives in network byte order.
For the packet you posted, you need something like this:
class Packet
{
private byte header;
private byte commandCode;
private int sequenceNumber;
private boolean ackStatus;
void write(DataOutput out) throws IOException
{
int wirePacket = header|(commandCode << 4)|(sequenceNumber << 8);
if (ackStatus)
{
wirePacket |= 0x80000000;
}
out.writeInt(wirePacket);
}
static Packet read(DataInput in) throws IOException
{
Packet packet = new Packet();
int wirePacket = in.readInt();
packet.header = (byte)(wirePacket & 0x0f);
packet.commandCode = (byte)((wirePacket >>> 4) & 0x0f);
packet.sequenceNumber = (wirePacket >>> 8) & 0x7FFFFF;
packet.ackStatus = (wirePacket & 0x80000000) != 0;
return packet;
}
// getters and setters. The setters must ensure that the values don't go out of range.
}
I need to construct a Java byte array out of mixed data types, but I don't know how to do this. These are my types:
byte version = 1; // at offset 0
short message_length = // the size of the byte[] message I am constructing here, at offset 1
short sub_version = 15346; // at offset 3
byte message_id = 2; // at offset 5
int flag1 = 10; // at offset 6
int flag2 = 0; // at offset 10
int flag3 = 0; // at offset 14
int flag4 = 0; // at offset 18
String message = "the quick brown fox jumps over the lazy dog"; // at offset 22
I know for the String, I can use
message.getBytes("US_ASCII");
I know for the int values, I can use
Integer.byteValue();
I know for the short values, I can use
Short.byteValue();
And the byte values are already bytes, I am just not sure of how to combine all of these into a single byte array. I have read about
System.arraycopy();
Is this the correct process, I just convert all the data to bytes, and start "concatenating" the byte array with arraycopy?
I am communicating with some distant server I have no control over, and this is the message process they require.
Wrap a DataOutputStream around a ByteArrayOutputStream. This way you can write all the primitive types like int and short directly to the DataOutputStream, which converts them to bytes and forwards them to the ByteArrayOutputStream, from which you can then retrieve the whole thing as one byte array:
ByteArrayOutputStream bOut = new ByteArrayOutputStream();
DataOutputStream dOut = new DataOutputStream(bOut);
dOut.writeByte(version);
dOut.writeShort(message_length);
dOut.writeShort(sub_version);
dOut.writeByte(message_id);
dOut.writeInt(flag1);
dOut.writeInt(flag2);
dOut.writeInt(flag3);
dOut.writeInt(flag4);
dOut.write(message.getBytes(), 0, message.length());
dOut.flush();
byte[] result = bOut.toByteArray();
The best thing about this is that you can do the exact opposite (extracting values from a byte array) with DataInputStream and ByteArrayInputStream completely analoguously to the above code.
If by a 'mixed type' you mean a class with different member field types, then one approach is to make your class serializable, and use ApacheUtils
byte[] data = SerializationUtils.serialize(yourObject);
All, I wanted to post my own solution to my problem here. I did a quick Google search on how to insert a short into java byte array. One of the results talked about a Java ByteBuffer. After some reading, I determined this was the best and quickest way for me to get the results I needed. One section in the Java API that really made me interested in the ByteBuffer was this:
Methods in this class that do not otherwise have a value to return are specified to return the buffer upon which they are invoked. This allows method invocations to be chained. The sequence of statements
bb.putInt(0xCAFEBABE);
bb.putShort(3);
bb.putShort(45);
can, for example, be replaced by the single statement
bb.putInt(0xCAFEBABE).putShort(3).putShort(45);
So, that is what I did:
byte version = 1;
short message_length = 72;
short sub_version = 15346;
byte message_id = 2;
int flag1 = 10;
int flag2 = 0;
int flag3 = 0;
int flag4 = 0;
String message = "the quick brown fox jumps over the lazy dog";
ByteBuffer messageBuffer = ByteBuffer.allocate(message_length);
messageBuffer.put(version).putShort(message_length).putShort(sub_version).put(message_id).putInt(flag1).putInt(flag2).putInt(flag3).putInt(flag4).put(message.getBytes());
byte[] myArray = messageBuffer.array();
That was fast and easy, and just what I needed. Thank you all who took the time to read and reply.
Certainly you can concatenate these values with arrayCopy, as you've suggested.
You can also append your bytes onto a ByteArrayOutputStream.
The key is to understand exactly what the receiving system is expecting. How does it know where one field ends and the next begins? How does it know what type it's reading at a given position in the stream? There are lots of ways they could have chosen to do that - with length headers in the protocol; with type headers; with null-termination of strings; with a set order of fields and their lengths; and so on.
Whatever method you choose, write unit tests that check for edge cases like negative numbers, very large numbers, non-ASCII text and so on. It's easy to get stung when everything has been working fine, then suddenly the server chokes on a Unicode character or a negative number that it interprets as a very large number.
One other option -- perhaps slight overkill for your needs, but flexible and with high performance -- is Google's protocol buffers library.
I am trying to parse a DatagramPacket that I will receive at a socket. I know the format of the packet I will receive, which is a DHCPREQUEST packet, but I don't think that really matters. For simplicity's sake, let's just consider the first six fields:
First field is the "opcode", which is 1 byte.
Second field is the "hardware type" which is 1 byte.
Third, "hardware address length", 1 byte.
Fourth, "hops", 1 byte.
Fifth, "transaction identifier xid", 4 bytes.
Sixth, "seconds", 2 bytes.
After I receive the packet, my approach is to convert it to a byte array.
DatagramPacket request = new DatagramPacket(new byte[1024], 1024);
socket.receive(request);
byte[] buf = request.getData();
At this point, the packet is stored in the byte array buf as a series of bytes. Since I know what the structure of this byte sequence is, how can I parse it? The one-byte fields are simple enough, but how about the multiple-bit fields? For example, how can I extract bytes 4 to 7, and store them in a variable named xid?
I could manually put each byte into an array:
byte[] xid = new byte[4];
xid[0] = buf[4];
xid[1] = buf[5];
xid[2] = buf[6];
xid[3] = buf[7];
But that's just tedious, and impractical for fields that are hundreds of bytes in length. The String class can parse substrings given an offset and length; is there a similar method for byte arrays in Java?
Or am I somehow making things difficult for myself?
Wrap the byte array in a ByteArrayOutputStream; wrap a DataInputStream around that; then use the methods of DataInputStream.
The cleanest way to do something like this is probably to use the utility method Arrays.copyOfRange.
What you do is write yourself some helper methods to extract 2 byte, 4 byte, etc values from the packet, reading the bytes and assembling them into Java short, int or whatever values.
For example
public short getShort(byte[] buffer, int offset) {
return (short) ((buffer[offset] << 8) | buffer[offset + 1]);
}
Then you use these helper methods as often as you need to. (If you want to be fancy, you could have the methods update an attribute that holds the current position, so that you don't have to pass an offset argument.)
Alternatively, if you were not worried by the overheads, you could wrap the byte array in ByteArrayInputStream and a DataInputStream, and use the latter's API to read bytes, shorts, ints, and so on. IIRC, DataInputStream assumes that numbers are represented in the stream in "network byte order" ... which is almost certainly what the DHCP spec mandates.
I'm a bit late to this, but there's a ByteBuffer class:
ByteBuffer b = ByteBuffer.wrap(request.getData());
byte opcode = b.get();
byte hwtype = b.get();
byte hw_addr_len = b.get();
byte hops = b.get();
int xid = b.getInt();
short seconds = b.getShort();
Or, if you only need a single field:
ByteBuffer b = ByteBuffer.wrap(request.getData());
int xid = b.getInt(4);
Given a integer value from a string, I want to convert it to 2 byte signed integer.
BigInteger does the job, but I don't know how to grant 2 bytes...
public void handleThisStringValue(String x, String y){
BigInteger bi_x = new BigInteger(x, 10);
BigInteger bi_y = new BigInteger(y, 10);
byte[] byteX = bi_x.toByteArray();
byte[] byteY = bi_y.toByteArray();
}
I noticed that BigInteger.toByteArray() handles negative values which is suitable for me.
Then I need to read those values (negative and positive ones), or saying convert byte[2] to signed int. Any suggestion?
Well, your questions still lacks certain information.
First, Java integers are 32-bit long, so they will not fit into a 2-byte array, you need a 4-byte array, otherwise you are actually dealing with short data type, which is 16-bit long.
Also, not sure if you need to deal with any kind of byte ordering (little endian, big endian).
At any rate, assuming that you are using integers that only fit in 16-bits and big endian byte ordering, you could do something as follows to create the byte array:
public static byte[] toByteArray(String number){
ByteBuffer buffer = ByteBuffer.allocate(4);
buffer.putInt(Integer.parseInt(number));
return Arrays.copyOfRange(buffer.array(), 2, 4); //asumming big endian
}
And as follows to convert it back:
public static int toInteger(byte[] payload){
byte[] data = new byte[4];
System.arraycopy(payload, 0, data, 2, 2);
return ByteBuffer.wrap(data).getInt();
}
You can also change the byte order of the ByteBuffer with the ByteBuffer.order method.
I used it as follows:
byte[] payload = toByteArray("255");
int number = toInteger(payload);
System.out.println(number);
Output is 255
int x = bs[0] | ((int)bs[1] << 8);
if (x >= 0x8000) x -= 0x10000;
// Reverse
bs[0] = (byte)(x & 0xFF);
bs[1] = (byte)((x >> 8) & 0xFF);
You can make the inverse:
new BigInteger(byteX);
new BigInteger(byteY);
It's exactly what you want, and then you can use .intvalue() to get it as an int
The solution is simple, based in posts I found here (thank you for all):
Remember that I wanted a 2 byte integer... so it is a Short!
String val= "-32";
short x = Short.parseShort(val);
byte[] byteX = ByteBuffer.allocate(2).putShort(x).array();
... and it works!
Then, I'm using BigInteger to read it back!
int x1 = new BigInteger(byteX).intValue();
or
short x2 = new BigInteger(x).shortValue();
I'm trying to send and receive a byte stream in which certain ranges of bytes represent different pieces of data. I've found ways to convert single primitive datatypes into bytes, but I'm wondering if there's a straightforward way to place certain pieces of data into specified byte regions.
For example, I might need to produce or read something like the following:
byte 1 - int
byte 2-5 - int
byte 6-13 - double
byte 14-21 - double
byte 25 - int
byte 26-45 - string
Any suggestions would be appreciated.
Try DataOutputStream/DataInputStream or, for arrays, the ByteBuffer class.
For storing the integer in X bytes, you may use the following method. If you think it is badly named, you may use the much less descriptive i2os name which is used in several (crypto) algorithm descriptions. Note that the returned octet string uses Big Endian encoding of unsigned ints, which you should specify for your protocol.
public static byte[] possitiveIntegerToOctetString(
final long value, final int octets) {
if (value < 0) {
throw new IllegalArgumentException("Cannot encode negative values");
}
if (octets < 1) {
throw new IllegalArgumentException("Cannot encode a number in negative or zero octets");
}
final int longSizeBytes = Long.SIZE / Byte.SIZE;
final int byteBufferSize = Math.max(octets, longSizeBytes);
final ByteBuffer buf = ByteBuffer.allocate(byteBufferSize);
for (int i = 0; i < byteBufferSize - longSizeBytes; i++) {
buf.put((byte) 0x00);
}
buf.mark();
buf.putLong(value);
// more bytes than long encoding
if (octets >= longSizeBytes) {
return buf.array();
}
// less bytes than long encoding (reset to mark first)
buf.reset();
for (int i = 0; i < longSizeBytes - octets; i++) {
if (buf.get() != 0x00) {
throw new IllegalArgumentException("Value does not fit in " + octets + " octet(s)");
}
}
final byte[] result = new byte[octets];
buf.get(result);
return result;
}
EDIT before storing the string, think of a padding mechanism (spaces would be most used), and character-encoding e.g. String.getBytes(Charset.forName("ASCII")) or "Latin-1". Those are the most common encodings with a single byte per character. Calculating the size of "UTF-8" is slightly more difficult (encode first, add 0x20 valued bytes at the end using ByteBuffer).
You may want to consider having a constant size for each data type. For example, the 32-bit Java int will take up 4 bytes a long will take 8, etc. In fact, if you use Java's DataInputStream and DataOutputStreams, you'll basically be doing that anyway. They have really nice methods like read/writeInt, etc.