I do some learning of using voip over udp in a small network. I know there are bundles of libraries ready to do and overdo everything I ever need with a few method calls, but as I said I am learning, so need to reinvent the wheel to see how it works.
I am currently investigating the DatagramPacket class and I've noticed that there is no method that would set header information(ie packet order sequence number which I need to know to do interleaving) in DatagramPacket class.
A little code to reflect the environment:
byte[] block;
DatagramPacket packet; // UDP packet
/* x Bytes per block , y blocks per second,
z ms time block playback duration */
block = recorder.getBlock(); // assume I have class that handles audio
// recording and returns speech in a
// uncompressed form of bytes
packet = new DatagramPacket(block, block.length, clientIP, PORT);
Firstly, I assume that because it is UDP, the sender doesnt really care anything whatsoever besides the simple fact that he throws packets somewhere. So that is why there is no such method inside.
Secondly, I assume that I need to do it myself - add extra bytes to the byte block to be sent , which would contain a sequence number of a packet? However am also concerned that if I do that, then how do I recognize if bytes are header bytes not audio bytes? I can make assumption that first byte represents a number, however we know that byte can only represent 258 numbers. I've never really worked on byte level before. Or there maybe other techniques?
Shortly saying, to do interleaving I need to know how to set up packet sequence number as I can't order unordered packets :-)
Thank You,
You'll need to serialize/deserialize data types your program uses onto a byte array.
Lets assume you're talking about RTP, and you'd want to send a packet with these fields - look at chapter 5 in the RTP specs:
Version = 2
padding = 0
extension = 0
CSRC count = 1
marker = 0
payload type = 8 (G711 alaw)
sequence number = 1234
timestamp = 1
one CSRC = 4321
Lets put these into some variables, using integers for ease, or long when we need to deal with an unsigned 32 bit value:
int version = 2;
int padding = 0;
int extension = 0;
int csrcCount = 1;
int marker = 0;
int payloadType = 8;
int sequenceNumber = 1234;
long timestamp = 1;
long ourCsrc = 4321;
byte buf[] = ...; //allocate this big enough to hold the RTP header + audio data
//assemble the first bytes according to the RTP spec (note, the spec marks version as bit 0 and 1, but
//this is really the high bits of the first byte ...
buf[0] = (byte) ((version & 0x3) << 6 | (padding & 0x1) << 5 | (extension & 0x1) << 4 | (csrcCount & 0xf));
//2.byte
buf[1] = (byte)((marker & 0x1) << 7 | payloadType & 0x7f);
//squence number, 2 bytes, in big endian format. So the MSB first, then the LSB.
buf[2] = (byte)((sequenceNumber & 0xff00) >> 8);
buf[3] = (byte)(sequenceNumber & 0x00ff);
//packet timestamp , 4 bytes in big endian format
buf[4] = (byte)((timestamp & 0xff000000) >> 24);
buf[5] = (byte)((timestamp & 0x00ff0000) >> 16);
buf[6] = (byte)((timestamp & 0x0000ff00) >> 8);
buf[7] = (byte) (timestamp & 0x000000ff);
//our CSRC , 4 bytes in big endian format
buf[ 8] = (byte)((sequenceNumber & 0xff000000) >> 24);
buf[ 9] = (byte)((sequenceNumber & 0x00ff0000) >> 16);
buf[10] = (byte)((sequenceNumber & 0x0000ff00) >> 8);
buf[11] = (byte) (sequenceNumber & 0x000000ff);
That's the header, now you can copy the audio bytes into buf, starting at buf[12] and send buf as one packet.
Now, the above is ofcourse just to show the principles, an actual serializer for a RTP packet would have to deal with much more, in accordance to the RTP specificaion (e.g. you might need some extension headers, you might need more than one CSRC, you need the correct payload type according to the format of the audio data you have, you need to packetize and schedule those audio data correctly - e.g. for G.711Alaw you'll should fill each RTP packet with 160 bytes of audio data and send one packet every 20 milisecond.
Related
I have an application in which I'm trying to send UDP messages using TSLv5. The protocol is somewhat complicated, but to do it I need to write 16 bit values as little-endian and then send that via UDP.
Here's my code doing just that:
buffer.order(ByteOrder.LITTLE_ENDIAN);
buffer.putShort(SCREEN_POS, screen);
buffer.putShort(INDEX_POS, index);
ByteBuffer text = ByteBuffer.wrap(value.getBytes());
short textLength = (short) value.getBytes().length;
buffer.putShort(LENGTH_POS, textLength);
ByteBuffer combined = ByteBufferUtils.concat(buffer, text);
short control = 0x00;
control |= rhTally << 0;
control |= textTally << 2;
control |= lhTally << 4;
control |= brightness << 6;
combined.putShort(CONTROL_POS, control);
short msgLength = (short) (combined.array().length - 2);
combined.putShort(PBC_POS, msgLength);
return new DatagramPacket(combined.array(), combined.array().length, ip, 9000);
This mostly works, but the problem is when I have values that are greater than 127.
For example, my index is 148 and when all is said and done, my control comes out to be 193. When I write those values to the ByteBuffer they become -108 and -63, respectively.
I know why this happens, a ByteBuffer is an array of bytes and bytes can't be greater than 127. What I don't know is how I can achieve this? The protocol does not work if I send signed values, it has to be the exact number.
I can assure that a signed java byte will be read correctly in the two bytes of a short. I have simplified the code, writing the fields one after the other in linear fashion, with message fields in front. Also just used one ByteBuffer.
(Maybe there is some small error like a wrong offset.)
Also I send the text bytes as being in UTF-8. You used the implicit platform encoding, which may differ on every computer.
byte[] text = value.getBytes(StandardCharsets.UTF_8);
int textLength = text.length;
int length = 2 + 2 + 2 + 2 + 2 + textLength;
ByteBuffer buffer = ByteBuffer.allocate(length)
.order(ByteOrder.LITTLE_ENDIAN);
short control = 0x00;
control |= rhTally << 0;
control |= textTally << 2;
control |= lhTally << 4;
control |= brightness << 6;
buffer.putShort(/*CONTROL_POS,*/ control);
short msgLength = (short) (length - 2);
buffer.putShort(/*PBC_POS,*/ msgLength);
buffer.putShort(/*SCREEN_POS,*/ screen);
buffer.putShort(/*INDEX_POS,*/ index);
buffer.putShort(/*LENGTH_POS,*/ (short)textLength);
buffer.put(text, 0, textLength);
return new DatagramPacket(buffer.array(), length, ip, 9000);
I'm trying to create a UDP client server. I am already able to read the messages by doing the following:
My incoming message is BIG_ENDIAN and the structure is this:
UINT8 type;
UINT8 flags;
UINT16 len;
UINT32 sequenceN;
UINT16 startIdx;
UINT16 endIdx;
The corresponding Java objects are:
short type;
short flags;
int len;
long sequenceN;
int startIdx;
int endIdx;
To convert from UDP to Java I use the following:
typeArray = Arrays.copyOfRange(msg, 0, 1);
type = Util.reassembleShort(typeArray);
flagsArray = Arrays.copyOfRange(msg, 1, 2);
flags = Util.reassembleShort(flagsArray);
lenArray = Arrays.copyOfRange(msg, 2, 4);
len = Util.reassembleInt(lenArray);
seqArray = Arrays.copyOfRange(msg, 4, 8);
sequenceN = Util.reassembleLong(seqArray);
startArray = Arrays.copyOfRange(msg, 8, 10);
startIdx = Util.reassembleInt(startArray);
endArray = Arrays.copyOfRange(msg, 10, 12);
endIdx = Util.reassembleInt(endArray);
To reassemble byte array portions into Java objects I use the following (AKA the calls to Util.reassemble* above):
Short
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
short result = ((short) (buffer.get() & 0xff));
Long
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
long result = ((long) buffer.getInt() & 0xffffffffL);
Int
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
int result = (buffer.getShort() & 0xffff);
String
String result = new String(removeStringGarbage(input), Charset.forName("US-ASCII"));
This works great. My question is... how do I do the reverse and get the objects into a correctly sized bytebuffer to send back on UDP?
In case you must follow a pre-existing on-the-wire binary format precisely, give "Kaitai Struct" (http://kaitai.io/) a try.
If you can afford to change you serialization format (i.e. you are in control of it), take a look at the "Protocol Buffers": https://developers.google.com/protocol-buffers/docs/javatutorial
In both cases, using a pre-existing library, to describe you binary protocol declaratively, will save you a great deal of time and effort,
and will normally result in a more robust code (as code generators can handle broken data better and never make typos).
And with "Protocol Buffers" you also get extensibility, i.e. you can evolve your protocol while preserving backward compatibility.
If you still want to serialize/deserialize your data manually,
just use the ByteBuffer's putX methods as follows:
buffer.put((byte)(type & 0xFF));
buffer.put((byte)(flags & 0xFF));
buffer.putShort((short)(len & 0xFFFF));
buffer.putInt((int)(sequenceN & 0xFFFFFFFF));
buffer.putShort((short)(startIdx & 0xFFFF));
buffer.putShort((short)(endIdx & 0xFFFF));
Put operation type should match your binary field size (i.e. put() for UINT8, putShort for UINT16, putInt for UINT32...), and you must apply a proper mask to it (i.e. 0xFF for short, 0xFFFF for int e.t.c.)
EDITED & SOVLED (below)
I'm using Java for Android trying to send the byte 255 (0xFF in WriteSingleCoil function) to a ModBUS server device.
Device is not runnig, I don't know if because of not able to interpretate the signed byte -1 or because of I'm wrong calculating the CRC.
I don't know how to calculate CRC for negative bytes.
Summarizing: I don't know how to send function 05 Write Single Coil with 0xFF value for switch on the coil for from Java to ModBUS server.
Can anyone help me please?
SOLUTION:
" iIndex = ucCRCLo ^ b: operations like this must be written as iIndex
= (ucCRCLo ^ b)&0xff because the & will cast ucCRCLo, b and the result to int, which is 32 bits while short is 16 so you will have a lot of
extra bits set to 1 "
This answer helped me. Thanks a lot to TheDayOfcondor
But also my huge problem was the usual problem in Java with signed bytes. My CRC calculating function is doing it right for unsigned bytes, but it give errors if I pass inside signed bytes.
The trick for work with bytes for ModBUS comunication is work in the whole App with shorts as bytes, for have the range 0-255, even calculating trames and CRC. And only in the last step, when sending trame to ModBUS sever, cast them to bytes again. This is running.
Hope it will helps to someone in future.
EXPLAINING PROBLEM:
I'm trying to set ON a coil to ModBUS with function 05, this is explaining of function:
Request
I'm tryiing to set ON the coil on address 1:
This hex: 0A 05 00 01 ff 00 DC 81
This byte array: 10 5 0 1 255 0 220 129
10: The Slave Address (10 = 0A hex)
05: The Function Code (Force Single Coil)
0001: The Data Address of the coil. (coil# 1 = 01 hex)
FF00: The status to write ( FF00 = ON, 0000 = OFF )
DC81: The CRC (cyclic redundancy check) for error checking.
The thing is that Java is using signed bytes, so I can't put 255 on my byte array.
I understand I should put -1, but then I can't calculate CRC correctly, because of I have a couple of precalculated array of bytes for get the CRC but the function send a negative index.
So: I don't know if I'm doing right trying to send -1, if I have an alternative for sending 255, neither how to calculate CRC for -1.
This is function for calculate CRC:
public short[] GenerateCRC (byte[] pMsg) {
short ucCRCHi = 0xFF;
short ucCRCLo = 0xFF;
int iIndex;
for (byte b : pMsg)
{
iIndex = ucCRCLo ^ b;
try {
ucCRCLo = (short)(ucCRCHi ^ aucCRCHi[ ( iIndex ) ]);
ucCRCHi = aucCRCLo[ ( iIndex ) ];
} catch (Exception e) {
Log.e(LOGTAG, "GenerateCRC: " + e.toString(), e);
e.printStackTrace();
}
}
short[]result= new short[2];
result0]= ucCRCHi;
result1]= ucCRCLo;
return result;
}
The question is not very clear - however, the most common problem dealing with bytes is the fact Java does not have unsigned bytes, and boolean operation are always between int
The best way to deal with bytes is to use integers, and-ing every operation with 0xff. Also use >>> for the shift right (it is the unsigned version)
Example:
byte b= (byte)(255 & 0xff) // gives you the "unsigned byte"
byte b= (byte) ((b<<2)0xff ) // shift left must be truncated
If you post your code to calculate the CRC I can have a look into it
The best way to define a byte array without using negative numbers is like this:
byte[]={ (byte)0xff, (byte)0xff, (byte)0xff };
I send a int in a NSData like this:
NSData* dataLength = [[NSData alloc] initWithBytes:&theInt length:sizeof(theInt)];
then in java side, I get a int like this:
int theInt = aInputStreamOfSocket.readInt();
but the value changed! In my case, I send 1225516 and get 749933056
what's the problem?
Your trouble is a difference in endianness. Intel based processors use little-endian byte order while network based transports are almost always big-endian. Java thus expects big-endian for readInt(). Ideally you find a way to send the int as big-endian to conform to expected behavior. I however don't have that code offhand, so here's how to read little-endian on the Java side:
int ch1 = aInputStreamOfSocket.read();
int ch2 = aInputStreamOfSocket.read();
int ch3 = aInputStreamOfSocket.read();
int ch4 = aInputStreamOfSocket.read();
if ((ch1 | ch2 | ch3 | ch4) < 0) {
throw new EOFException();
}
int theInt = ch1 + (ch2 << 8) + (ch3 << 16) + (ch4 << 24);
Let's look at the hex for both of those numbers
1225516 = 0x0012B32C
749933056 = 0x2CB31200
You can see that the byte order (a.k.a. endianness) is reversed.
Generally, if you're sending data over a socket, you convert from the local byte order to network byte order with the functions htonl, htons, etc. On the receiving end, you convert from network byte order back to the local byte order. In java, you can do this by setting the byte order on the buffer with ByteBuffer#order(ByteOrder)
See this question also.
I am trying to find the value of the first 2 bytes in a UDP packet which corresponds to the length of the remaining payload. What is the best method to find this value in Java given that I know the first 2 bytes? Would java.nio.ByteBuffer be of any use?
Thanks
I usually use something like this:
static public int buildShort(byte high, byte low)
{
return ((0xFF & (int) high) * 256) + ((0xFF & (int) low));
}
Then you take first two bytes of your DatagramPacket:
int length = buildShort(packet.getData()[0], packet.getData()[1]);
Mind that I used length as an int because also short data type (as everyone) is signed in Java, so you need a larger space.
Using a ByteBuffer is convenient, just don't get tripped up by Java signed 16-bit values:
byte[] data = new byte[MAX_LEN];
ByteBuffer buf = ByteBuffer.wrap(data);
DatagramPacket pkt = new DatagramPacket(data, data.length);
⋮
while (connected) {
socket.receive(pkt);
int len = buf.getShort() & 0xFFFF;
⋮
}
If you don't want to use ByteBuffer, the conversion is still fairly easy. The equivalent multiplication and addition can be used, but I see bit operators used more frequently:
int len = (data[0] & 0xFF) << 8 | data[1] & 0xFF;
You can indeed make use of java.nio.ByteBuffer. Here's a kickoff example:
ByteBuffer buffer = ByteBuffer.allocate(2);
buffer.order(ByteOrder.LITTLE_ENDIAN);
buffer.put(byte1);
buffer.put(byte2);
int length = buffer.getShort(0) & 0xFFFF; // Get rid of sign.
Using ByteBuffer would only be of value if you are reading the UDP packets (using nio). You can create a utility method:
static final int getLength(DatagramPacket packet) {
byte data[] = DatagramPacket.getData();
return (int)((0xFF & (int)data[0]) << 8) | (0xFF & (int)data[1]));
}