We Keep Coding

Writing shorts greater than 127 to a ByteBuffer in Java

I have an application in which I'm trying to send UDP messages using TSLv5. The protocol is somewhat complicated, but to do it I need to write 16 bit values as little-endian and then send that via UDP.
Here's my code doing just that:
buffer.order(ByteOrder.LITTLE_ENDIAN);
buffer.putShort(SCREEN_POS, screen);
buffer.putShort(INDEX_POS, index);
ByteBuffer text = ByteBuffer.wrap(value.getBytes());
short textLength = (short) value.getBytes().length;
buffer.putShort(LENGTH_POS, textLength);
ByteBuffer combined = ByteBufferUtils.concat(buffer, text);
short control = 0x00;
control |= rhTally << 0;
control |= textTally << 2;
control |= lhTally << 4;
control |= brightness << 6;
combined.putShort(CONTROL_POS, control);
short msgLength = (short) (combined.array().length - 2);
combined.putShort(PBC_POS, msgLength);
return new DatagramPacket(combined.array(), combined.array().length, ip, 9000);
This mostly works, but the problem is when I have values that are greater than 127.
For example, my index is 148 and when all is said and done, my control comes out to be 193. When I write those values to the ByteBuffer they become -108 and -63, respectively.
I know why this happens, a ByteBuffer is an array of bytes and bytes can't be greater than 127. What I don't know is how I can achieve this? The protocol does not work if I send signed values, it has to be the exact number.

I can assure that a signed java byte will be read correctly in the two bytes of a short. I have simplified the code, writing the fields one after the other in linear fashion, with message fields in front. Also just used one ByteBuffer.
(Maybe there is some small error like a wrong offset.)
Also I send the text bytes as being in UTF-8. You used the implicit platform encoding, which may differ on every computer.
byte[] text = value.getBytes(StandardCharsets.UTF_8);
int textLength = text.length;
int length = 2 + 2 + 2 + 2 + 2 + textLength;
ByteBuffer buffer = ByteBuffer.allocate(length)
.order(ByteOrder.LITTLE_ENDIAN);
short control = 0x00;
control |= rhTally << 0;
control |= textTally << 2;
control |= lhTally << 4;
control |= brightness << 6;
buffer.putShort(/*CONTROL_POS,*/ control);
short msgLength = (short) (length - 2);
buffer.putShort(/*PBC_POS,*/ msgLength);
buffer.putShort(/*SCREEN_POS,*/ screen);
buffer.putShort(/*INDEX_POS,*/ index);
buffer.putShort(/*LENGTH_POS,*/ (short)textLength);
buffer.put(text, 0, textLength);
return new DatagramPacket(buffer.array(), length, ip, 9000);

Java object to UDP byte[]

I'm trying to create a UDP client server. I am already able to read the messages by doing the following:
My incoming message is BIG_ENDIAN and the structure is this:
UINT8 type;
UINT8 flags;
UINT16 len;
UINT32 sequenceN;
UINT16 startIdx;
UINT16 endIdx;
The corresponding Java objects are:
short type;
short flags;
int len;
long sequenceN;
int startIdx;
int endIdx;
To convert from UDP to Java I use the following:
typeArray = Arrays.copyOfRange(msg, 0, 1);
type = Util.reassembleShort(typeArray);
flagsArray = Arrays.copyOfRange(msg, 1, 2);
flags = Util.reassembleShort(flagsArray);
lenArray = Arrays.copyOfRange(msg, 2, 4);
len = Util.reassembleInt(lenArray);
seqArray = Arrays.copyOfRange(msg, 4, 8);
sequenceN = Util.reassembleLong(seqArray);
startArray = Arrays.copyOfRange(msg, 8, 10);
startIdx = Util.reassembleInt(startArray);
endArray = Arrays.copyOfRange(msg, 10, 12);
endIdx = Util.reassembleInt(endArray);
To reassemble byte array portions into Java objects I use the following (AKA the calls to Util.reassemble* above):
Short
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
short result = ((short) (buffer.get() & 0xff));
Long
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
long result = ((long) buffer.getInt() & 0xffffffffL);
Int
ByteBuffer buffer = ByteBuffer.wrap(input);
buffer.order(ByteOrder.BIG_ENDIAN);
int result = (buffer.getShort() & 0xffff);
String
String result = new String(removeStringGarbage(input), Charset.forName("US-ASCII"));
This works great. My question is... how do I do the reverse and get the objects into a correctly sized bytebuffer to send back on UDP?

In case you must follow a pre-existing on-the-wire binary format precisely, give "Kaitai Struct" (http://kaitai.io/) a try.
If you can afford to change you serialization format (i.e. you are in control of it), take a look at the "Protocol Buffers": https://developers.google.com/protocol-buffers/docs/javatutorial
In both cases, using a pre-existing library, to describe you binary protocol declaratively, will save you a great deal of time and effort,
and will normally result in a more robust code (as code generators can handle broken data better and never make typos).
And with "Protocol Buffers" you also get extensibility, i.e. you can evolve your protocol while preserving backward compatibility.
If you still want to serialize/deserialize your data manually,
just use the ByteBuffer's putX methods as follows:
buffer.put((byte)(type & 0xFF));
buffer.put((byte)(flags & 0xFF));
buffer.putShort((short)(len & 0xFFFF));
buffer.putInt((int)(sequenceN & 0xFFFFFFFF));
buffer.putShort((short)(startIdx & 0xFFFF));
buffer.putShort((short)(endIdx & 0xFFFF));
Put operation type should match your binary field size (i.e. put() for UINT8, putShort for UINT16, putInt for UINT32...), and you must apply a proper mask to it (i.e. 0xFF for short, 0xFFFF for int e.t.c.)

Operating on byte[] and int[], the same memory

I am looking to speed up my program. Currently I have a function that does this:
public void updateBitmap(byte[] buf, int thisPacketLength, int standardOffset, int thisPacketOffset) {
int pixelCoord = thisPacketOffset / 3 - 1;
for (int bufCoord = standardOffset; bufCoord < thisPacketLength; bufCoord += 3) {
pixelCoord++;
pixelData[pixelCoord] = 0xFF << 24 | (buf[bufCoord + 2] << 16) & 0xFFFFFF | (buf[bufCoord + 1] << 8) & 0xFFFF | buf[bufCoord] & 0xFF;
}
}
I basically need to copy ints in byte[] form into an int[] array. I realized that if I can treat the int[] array as a byte array then I can simply modify the bytes directly, instead of doing all this shifting, which I imagine would be faster. However, I can't figure out how to do that.
What I want is to have int[] pixelData and byte[] pixelDataBytes both point to the memory. Just be two different "views" of the same bits in memory if you understand what I mean. Then I can update the individual bytes as they come in without shifting them, while still maintaining the int[] representation I need for other parts of the code. It seems like this should be possible, but I haven't figured out how to do it yet.

You should use a byte buffer instead. You can then access its backing as an int buffer, and read/write byte by byte or int by int,
Create it with:
ByteBuffer bb = ByteBuffer.wrap(buf);
and get an IntBuffer:
IntBuffer ib = bb.asIntBuffer();
You can set values in this buffer by writing to an index:
ib.put(2, 400);
were 2 is the index and 400 is the value. Any changes to ib will be backed by bb and buf.

Java - from short to byte[2] using LITTLE_ENDIAN

I have some problems trying yo convert short value to byte[2]. I'm using this to make some transformations on some audio data buffer(applying gain to buffer). First I load the audio buffer like this:
mRecorder.read(buffer, 0, buffer.length);
where buffer is
private byte[] buffer;
Than, I get the sample (the recording is in 16bit sample size), like this:
short sample = getShort(buffer[i*2], buffer[i*2+1]);
The getShort is define like this:
/*
*
* Converts a byte[2] to a short, in LITTLE_ENDIAN format
*
*/
private short getShort(byte argB1, byte argB2)
{
return (short)(argB1 | (argB2 << 8));
}
Then I apply gain to the sample:
sample *= rGain;
After this, I try to get back the byte array from the multiplied sample:
byte[] a = getByteFromShort(sample);
But this fails, because the sound has a lot of noise even if the gain is 1.
Below is the getByteFromShort method definion:
private byte[] getByteFromShort(short x){
//variant 1 - noise
byte[] a = new byte[2];
a[0] = (byte)(x & 0xff);
a[1] = (byte)((x >> 8) & 0xff);
//variant 2 - noise and almost broke my ears - very loud
// ByteBuffer buffer = ByteBuffer.allocate(2);
// buffer.putShort(x);
// buffer.flip();
return a;
}
So the problem is when converting the short value to byte[2]. When the gain was 1.0, the sound was fill with noise.
Below is the full gain applying method:
for (int i=0; i<buffer.length/2; i++)
{ // 16bit sample size
short curSample = getShort(buffer[i*2], buffer[i*2+1]);
if(rGain != 1){
//apply gain
curSample *= rGain;
//convert back from short sample that was "gained" to byte data
byte[] a = getByteFromShort(curSample);
//modify buffer to contain the gained sample
buffer[i*2] = a[0];
buffer[i*2 + 1] = a[1];
}
}
Could you guys please take a look over getByteFromShort method and tell me where I'm wrong?
Thanks.

getByteFromShort() seems OK.
getShort(byte argB1, byte argB2) is wrong. It produces incorrect result when argB1 is negative.
It should be
return (short)((argB1 & 0xff) | (argB2 << 8));

Use the following code:
ret[0] = (byte)(x & 0xff);
ret[1] = (byte)((x >> 8) & 0xff);

I would use ByteBuffer
ByteBuffer buffer = ByteBuffer.allocate(8*1024);
mRecorder.read(buffer.array(), 0, buffer.capacity());
// using NIO
mRecorder.read(buffer);
while(buffer.remaining() > 1) {
short s = bb.getShort(x);
// do something with s
}

ByteBuffer and its cohorts in java.nio can help with this. Basically, you will create a ByteBuffer backed by an array with your data ByteBuffer.wrap(array). You can then set the endianness of the buffer with ByteBuffer.order() and use functions like get/put Int/Short/byte... to manipulate data in the underlying array.

send a int from objective-c to java by socket,but the value changed in java side

I send a int in a NSData like this:
NSData* dataLength = [[NSData alloc] initWithBytes:&theInt length:sizeof(theInt)];
then in java side, I get a int like this:
int theInt = aInputStreamOfSocket.readInt();
but the value changed! In my case, I send 1225516 and get 749933056
what's the problem?

Your trouble is a difference in endianness. Intel based processors use little-endian byte order while network based transports are almost always big-endian. Java thus expects big-endian for readInt(). Ideally you find a way to send the int as big-endian to conform to expected behavior. I however don't have that code offhand, so here's how to read little-endian on the Java side:
int ch1 = aInputStreamOfSocket.read();
int ch2 = aInputStreamOfSocket.read();
int ch3 = aInputStreamOfSocket.read();
int ch4 = aInputStreamOfSocket.read();
if ((ch1 | ch2 | ch3 | ch4) < 0) {
throw new EOFException();
}
int theInt = ch1 + (ch2 << 8) + (ch3 << 16) + (ch4 << 24);

Let's look at the hex for both of those numbers
1225516 = 0x0012B32C
749933056 = 0x2CB31200
You can see that the byte order (a.k.a. endianness) is reversed.
Generally, if you're sending data over a socket, you convert from the local byte order to network byte order with the functions htonl, htons, etc. On the receiving end, you convert from network byte order back to the local byte order. In java, you can do this by setting the byte order on the buffer with ByteBuffer#order(ByteOrder)
See this question also.