We Keep Coding

Java object to UDP byte[]

I'm trying to create a UDP client server. I am already able to read the messages by doing the following:
My incoming message is BIG_ENDIAN and the structure is this:
UINT8 type;
UINT8 flags;
UINT16 len;
UINT32 sequenceN;
UINT16 startIdx;
UINT16 endIdx;
The corresponding Java objects are:
short type;
short flags;
int len;
long sequenceN;
int startIdx;
int endIdx;
To convert from UDP to Java I use the following:
typeArray = Arrays.copyOfRange(msg, 0, 1);
type = Util.reassembleShort(typeArray);
flagsArray = Arrays.copyOfRange(msg, 1, 2);
flags = Util.reassembleShort(flagsArray);
lenArray = Arrays.copyOfRange(msg, 2, 4);
len = Util.reassembleInt(lenArray);
seqArray = Arrays.copyOfRange(msg, 4, 8);
sequenceN = Util.reassembleLong(seqArray);
startArray = Arrays.copyOfRange(msg, 8, 10);
startIdx = Util.reassembleInt(startArray);
endArray = Arrays.copyOfRange(msg, 10, 12);
endIdx = Util.reassembleInt(endArray);
To reassemble byte array portions into Java objects I use the following (AKA the calls to Util.reassemble* above):
Short
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
short result = ((short) (buffer.get() & 0xff));
Long
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
long result = ((long) buffer.getInt() & 0xffffffffL);
Int
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
int result = (buffer.getShort() & 0xffff);
String
String result = new String(removeStringGarbage(input), Charset.forName("US-ASCII"));
This works great. My question is... how do I do the reverse and get the objects into a correctly sized bytebuffer to send back on UDP?

In case you must follow a pre-existing on-the-wire binary format precisely, give "Kaitai Struct" (http://kaitai.io/) a try.
If you can afford to change you serialization format (i.e. you are in control of it), take a look at the "Protocol Buffers": https://developers.google.com/protocol-buffers/docs/javatutorial
In both cases, using a pre-existing library, to describe you binary protocol declaratively, will save you a great deal of time and effort,
and will normally result in a more robust code (as code generators can handle broken data better and never make typos).
And with "Protocol Buffers" you also get extensibility, i.e. you can evolve your protocol while preserving backward compatibility.
If you still want to serialize/deserialize your data manually,
just use the ByteBuffer's putX methods as follows:
buffer.put((byte)(type & 0xFF));
buffer.put((byte)(flags & 0xFF));
buffer.putShort((short)(len & 0xFFFF));
buffer.putInt((int)(sequenceN & 0xFFFFFFFF));
buffer.putShort((short)(startIdx & 0xFFFF));
buffer.putShort((short)(endIdx & 0xFFFF));
Put operation type should match your binary field size (i.e. put() for UINT8, putShort for UINT16, putInt for UINT32...), and you must apply a proper mask to it (i.e. 0xFF for short, 0xFFFF for int e.t.c.)

Operating on byte[] and int[], the same memory

I am looking to speed up my program. Currently I have a function that does this:
public void updateBitmap(byte[] buf, int thisPacketLength, int standardOffset, int thisPacketOffset) {
int pixelCoord = thisPacketOffset / 3 - 1;
for (int bufCoord = standardOffset; bufCoord < thisPacketLength; bufCoord += 3) {
pixelCoord++;
pixelData[pixelCoord] = 0xFF << 24 | (buf[bufCoord + 2] << 16) & 0xFFFFFF | (buf[bufCoord + 1] << 8) & 0xFFFF | buf[bufCoord] & 0xFF;
}
}
I basically need to copy ints in byte[] form into an int[] array. I realized that if I can treat the int[] array as a byte array then I can simply modify the bytes directly, instead of doing all this shifting, which I imagine would be faster. However, I can't figure out how to do that.
What I want is to have int[] pixelData and byte[] pixelDataBytes both point to the memory. Just be two different "views" of the same bits in memory if you understand what I mean. Then I can update the individual bytes as they come in without shifting them, while still maintaining the int[] representation I need for other parts of the code. It seems like this should be possible, but I haven't figured out how to do it yet.

You should use a byte buffer instead. You can then access its backing as an int buffer, and read/write byte by byte or int by int,
Create it with:
ByteBuffer bb = ByteBuffer.wrap(buf);
and get an IntBuffer:
IntBuffer ib = bb.asIntBuffer();
You can set values in this buffer by writing to an index:
ib.put(2, 400);
were 2 is the index and 400 is the value. Any changes to ib will be backed by bb and buf.

Java - from short to byte[2] using LITTLE_ENDIAN

I have some problems trying yo convert short value to byte[2]. I'm using this to make some transformations on some audio data buffer(applying gain to buffer). First I load the audio buffer like this:
mRecorder.read(buffer, 0, buffer.length);
where buffer is
private byte[] buffer;
Than, I get the sample (the recording is in 16bit sample size), like this:
short sample = getShort(buffer[i*2], buffer[i*2+1]);
The getShort is define like this:
/*
*
* Converts a byte[2] to a short, in LITTLE_ENDIAN format
*
*/
private short getShort(byte argB1, byte argB2)
{
return (short)(argB1 | (argB2 << 8));
}
Then I apply gain to the sample:
sample *= rGain;
After this, I try to get back the byte array from the multiplied sample:
byte[] a = getByteFromShort(sample);
But this fails, because the sound has a lot of noise even if the gain is 1.
Below is the getByteFromShort method definion:
private byte[] getByteFromShort(short x){
//variant 1 - noise
byte[] a = new byte[2];
a[0] = (byte)(x & 0xff);
a[1] = (byte)((x >> 8) & 0xff);
//variant 2 - noise and almost broke my ears - very loud
// ByteBuffer buffer = ByteBuffer.allocate(2);
// buffer.putShort(x);
// buffer.flip();
return a;
}
So the problem is when converting the short value to byte[2]. When the gain was 1.0, the sound was fill with noise.
Below is the full gain applying method:
for (int i=0; i<buffer.length/2; i++)
{ // 16bit sample size
short curSample = getShort(buffer[i*2], buffer[i*2+1]);
if(rGain != 1){
//apply gain
curSample *= rGain;
//convert back from short sample that was "gained" to byte data
byte[] a = getByteFromShort(curSample);
//modify buffer to contain the gained sample
buffer[i*2] = a[0];
buffer[i*2 + 1] = a[1];
}
}
Could you guys please take a look over getByteFromShort method and tell me where I'm wrong?
Thanks.

getByteFromShort() seems OK.
getShort(byte argB1, byte argB2) is wrong. It produces incorrect result when argB1 is negative.
It should be
return (short)((argB1 & 0xff) | (argB2 << 8));

Use the following code:
ret[0] = (byte)(x & 0xff);
ret[1] = (byte)((x >> 8) & 0xff);

I would use ByteBuffer
ByteBuffer buffer = ByteBuffer.allocate(8*1024);
mRecorder.read(buffer.array(), 0, buffer.capacity());
// using NIO
mRecorder.read(buffer);
while(buffer.remaining() > 1) {
short s = bb.getShort(x);
// do something with s
}

ByteBuffer and its cohorts in java.nio can help with this. Basically, you will create a ByteBuffer backed by an array with your data ByteBuffer.wrap(array). You can then set the endianness of the buffer with ByteBuffer.order() and use functions like get/put Int/Short/byte... to manipulate data in the underlying array.

Convert long to byte array and add it to another array

I want to change a values in byte array to put a long timestamp value in in the MSBs. Can someone tell me whats the best way to do it. I do not want to insert values bit-by-bit which I believe is very inefficient.
long time = System.currentTimeMillis();
Long timeStamp = new Long(time);
byte[] bArray = new byte[128];
What I want is something like:
byte[0-63] = timeStamp.byteValue();
Is something like this possible . What is the best way to edit/insert values in this byte array. since byte is a primitive I dont think there are some direct implementations I can make use of?
Edit:
It seems that System.currentTimeMillis() is faster than Calendar.getTimeInMillis(), so replacing the above code by it.Please correct me if wrong.

There are multiple ways to do it:
Use a ByteBuffer (best option - concise and easy to read):
byte[] bytes = ByteBuffer.allocate(Long.SIZE / Byte.SIZE).putLong(someLong).array();
You can also use DataOutputStream (more verbose):
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
dos.writeLong(someLong);
dos.close();
byte[] longBytes = baos.toByteArray();
Finally, you can do this manually (taken from the LongSerializer in Hector's code) (harder to read):
byte[] b = new byte[8];
for (int i = 0; i < size; ++i) {
b[i] = (byte) (l >> (size - i - 1 << 3));
}
Then you can append these bytes to your existing array by a simple loop:
// change this, if you want your long to start from
// a different position in the array
int start = 0;
for (int i = 0; i < longBytes.length; i ++) {
bytes[start + i] = longBytes[i];
}

If you want to really get under the hood...
public byte[] longToByteArray(long value) {
return new byte[] {
(byte) (value >> 56),
(byte) (value >> 48),
(byte) (value >> 40),
(byte) (value >> 32),
(byte) (value >> 24),
(byte) (value >> 16),
(byte) (value >> 8),
(byte) value
};
}

For me ByteBuffer and other utils are expensive from time perspective. Here are 2 methods that you can use:
// first method (that is using the second method), it return the array allocated and fulfilled
public byte[] longToByteArray(long value)
{
byte[] array = new byte[8];
longToByteArray(value,array,0);
return array;
}
// this method is useful if you have already allocated the buffer and you want to write the long a specific location in the array.
public void longToByteArray(long value, byte[] array, int startFrom)
{
for (int i=7; i>=0; i--)
{
array[startFrom+7-i] = (byte) (value >> i*8);
}
}

It doesn't look like you can slice a byte array to insert something into a subset without doing it byte by byte. Look at Grab a segment of an array in Java without creating a new array on heap . Basically what I would do is set create a 64 byte array and set the time to it then append a blank 64 byte array to it. Or just do it byte by byte.

I am updating this post because I have just announced a pre-release version of a library that will convert longs to byte arrays (and back again). The library is very small and will convert any java primitive to a byte array.
http://rschilling.wordpress.com/2013/09/26/pre-release-announcement-pend-oreille/
http://code.google.com/p/pend-oreille/
If you use it you can do things like convert long arrays to byte arrays:
Double[] doubles = new Double[1000];
for (int i = 2; i < 1002; i++) {
doubles[i - 2] = (double) i;
}
byte[] resultBytes1 = (byte[]) new PrimitiveHelper(PrimitiveUtil.unbox(doubles))
.asType(byte[].class);
You can also convert a single long value as well.
byte[] resultBytes1 = (byte[]) new PrimitiveHelper(1000l)
.asType(byte[].class);
Feel free to provide some feedback.
Update on October 4, 2013:
I've now released the production of the library http://rschilling.wordpress.com/2013/10/04/pend-oreille-official-1-0-release/

Finding value of 2 bytes in Java

I am trying to find the value of the first 2 bytes in a UDP packet which corresponds to the length of the remaining payload. What is the best method to find this value in Java given that I know the first 2 bytes? Would java.nio.ByteBuffer be of any use?
Thanks

I usually use something like this:
static public int buildShort(byte high, byte low)
{
return ((0xFF & (int) high) * 256) + ((0xFF & (int) low));
}
Then you take first two bytes of your DatagramPacket:
int length = buildShort(packet.getData()[0], packet.getData()[1]);
Mind that I used length as an int because also short data type (as everyone) is signed in Java, so you need a larger space.

Using a ByteBuffer is convenient, just don't get tripped up by Java signed 16-bit values:
byte[] data = new byte[MAX_LEN];
ByteBuffer buf = ByteBuffer.wrap(data);
DatagramPacket pkt = new DatagramPacket(data, data.length);
⋮
while (connected) {
socket.receive(pkt);
int len = buf.getShort() & 0xFFFF;
⋮
}
If you don't want to use ByteBuffer, the conversion is still fairly easy. The equivalent multiplication and addition can be used, but I see bit operators used more frequently:
int len = (data[0] & 0xFF) << 8 | data[1] & 0xFF;

You can indeed make use of java.nio.ByteBuffer. Here's a kickoff example:
ByteBuffer buffer = ByteBuffer.allocate(2);
buffer.order(ByteOrder.LITTLE_ENDIAN);
buffer.put(byte1);
buffer.put(byte2);
int length = buffer.getShort(0) & 0xFFFF; // Get rid of sign.

Using ByteBuffer would only be of value if you are reading the UDP packets (using nio). You can create a utility method:
static final int getLength(DatagramPacket packet) {
byte data[] = DatagramPacket.getData();
return (int)((0xFF & (int)data[0]) << 8) | (0xFF & (int)data[1]));
}