We Keep Coding

Get Bit values from Byte Array

I have a byte array from which I need to read specific bits and convert to int (see the byte array structure below). Even though the bits information I want to read is in 3 bytes, I tried reading 4 bytes (6-9) as integer and then read the bits from that integer value with bits or bitsValue method but somehow I am not able to see the right values from the bit manipulation. And with my expertise in bits I am pretty sure I am doing something wrong.
Can someone please suggest I am doing it correctly and why its not working. Thanks in Advance!!
Byte array is in Little Endian format.
0th byte - Some Value
1st Byte - Some Value
2nd - 5th Byte - Some Value
6th - 9th Byte - first 18 bits represent some value
- Next 5 bits represent some value
- Next 1 bit represent some value
- Last 8 bits represent some value
public class Test {
public static void main(String... dataProvider) {
String s = "46 00 ef 30 e9 08 cc a5 03 43";
byte[] bytes = new byte[s.length()];
bytes = hexStringToByteArray(s);
int bytePointer = 0;
int msgType = getIntFromSingleByte(bytes[bytePointer]); // 0th byte
int version = getIntFromSingleByte(bytes[++bytePointer]); // 1st byte
int tickInMS = getIntValue(bytes, ++bytePointer); // 2nd-5th bytes
bytePointer = bytePointer + 4;
int headr = getIntValue(bytes, bytePointer); // 6th-9th bytes
int utcTime = bits(headr, 0, 18); // 6th-9th bytes - 18 bits
int reserved = bits(headr, 18, 5); // 6th-9th bytes- 5 bits
int reportOrEvent = bits(headr, 23, 1); // 6th-9th bytes - 1 bits
int reportId = bitsValue(headr, 24, 32); // 6th-9th- 8 bits
}
public static int getIntFromSingleByte(byte data) {
return (data & 0xFF);
}
public static int getIntValue(byte[] bytes, int startPosition) {
byte[] dest = new byte[4];
System.arraycopy(bytes, startPosition, dest, 0, dest.length);
return toInt(dest);
}
// took from Stack overflow
static int bits(int n, int offset, int length) {
// shift the bits rightward, so that the desired chunk is at the right end
n = n >> (31 - offset - length);
// prepare a mask where only the rightmost `length` bits are 1's
int mask = ~(-1 << length);
// zero out all bits but the right chunk
return n & mask;
}
public static int bitsValue(int intNum, int startBitPos, int endBitPos) {
// parameters checking ignored for now
int tempValue = intNum << endBitPos;
return tempValue >> (startBitPos + endBitPos);
}
public static byte[] hexStringToByteArray(final String s) {
String[] splits = s.split(" ");
final byte[] data = new byte[splits.length];
for (int i = 0; i < splits.length; i++) {
char first = splits[i].length() < 2 ? '0' : splits[i].charAt(0);
char second = splits[i].length() < 2 ? splits[i].charAt(0) : splits[i].charAt(1);
data[i] = (byte) ((Character.digit(first, 16) << 4) + Character.digit(second, 16));
}
return data;
}
public static int toInt(byte[] data) {
if (data == null || data.length != 4)
return 0x0;
return (int) ((0xff & data[0]) << 24 | (0xff & data[1]) << 16 | (0xff & data[2]) << 8
| (0xff & data[3]) << 0);
}
}

Wrapping your input data in a ByteBuffer will simplify parsing and allow you to adjust endianness as necessary.
Your bits method is wrong. The constant 31 should be 32. Also, the method uses MSB 0 bit numbering, which is odd for little-endian data. You should confirm that your input is documented as using this bit numbering scheme.
Your bitsValue method is wrong too. May as well just use bits after fixing it.
This code is simpler and extracts the bit fields correctly:
public static void main(String... args) {
String s = "46 0 79 37 a8 3 9f 37 1 43 eb 7a f 3 3 fe c4 1 c5 4 c5 5e";
byte[] input = hexStringToByteArray(s);
// Wrap the input in a ByteBuffer for parsing. Adjust endianness if necessary.
ByteBuffer buffer = ByteBuffer.wrap(input).order(ByteOrder.BIG_ENDIAN);
int msgType = buffer.get() & 0xff;
int version = buffer.get() & 0xff;
int tickInMS = buffer.getInt();
int header = buffer.getInt();
int utcTime = bits(header, 0, 18); // 6th-9th bytes - 18 bits
int reserved = bits(header, 18, 5); // 6th-9th bytes - 5 bits
int reportOrEvent = bits(header, 23, 1); // 6th-9th bytes - 1 bit
int reportId = bits(header, 24, 8); // 6th-9th bytes - 8 bits
System.out.printf("utc: %d, report? %d, id: %d\n", utcTime, reportOrEvent, reportId);
}
/**
* Extract a bit field from an int. Bit numbering is MSB 0.
*/
public static int bits(int n, int offset, int length) {
return n >> (32 - offset - length) & ~(-1 << length);
}

Instead of error prone bit mangeling you should rather use the binary string representation of the numbers and do your "bit picking" as string operations:
String s = "46 00 ef 30 e9 08 cc a5 03 43";
String[] hexNumbers = s.split(" ");
for(String hexNumber : hexNumbers) {
String binaryNumber = String.format("%8s", new BigInteger(hexNumber,16).toString(2)).replace(' ','0');
System.out.print(String.format("value in hex : %s, value in binary: %s", hexNumber,binaryNumber));
}

How to change 37 bits in a byte array to all complement of existing value

I have a byte array myByteArray[82]
I want to change exactly 37 bits in this array to the complement value. ie. if bit0 has a '1' I want to change it to '0'. I need to change first 37 such bits to introduce error in this byte array.
Please suggest how to do this

It's not entirely clear what you are trying to do. My best understanding is that you have an array of 82 bytes and you want to invert the lowest 37 bits of the array. Since a byte is 8 bits, so you can do this:
byte[] myByteArray = new byte[82];
// invert lowest 32 bits, 8 at a time
for (int i = 0; i < 4; ++i) {
myByteArray[i] = (byte)(~myByteArray[i]);
}
// invert next five bits
myByteArray[4] = (byte) (
(myByteArray[4] & 0xE0) // top 3 bits unchanged
|
((~myByteArray[4)) & 0x1F) // bottom 5 bits inverted
);

try
byte[] a82 = ...
Set<Integer> set = new HashSet<Integer>();
while (set.size() < 37) {
set.add((int) (Math.random() * 82));
}
for (int i : set) {
int ibyte = i / 8;
int ibit = i % 8;
int m = 1 << ibit;
a[ibyte] ^= m;
}

This works:
int nBits = 37;
int i = 0;
for (; i<nBits / 8; ++i)
myByteArray[i] = (byte)((byte) myByteArray[i] ^ 0xFF);
myByteArray[i] = (byte)(myByteArray[i] ^ ((0xFF >>> 5) ^ 0xFF));
Whether or not you do the final XOR with the 0xFF on the last line depends on whether you consider the most significant bit to be the first bit (then use it) or the last bit (then omit it);

To invert a specific 37 bits, randomly chosen:
// array of 82 bytes with 37 selected bits set to 1, all the rest zero
// you could generate this programmatically as well if you need a different
// set of bits each time, but your question implies you don't
byte[] mask = { 0x00, 0x01, 0x02, 0x80, .... 0x00 };
for (int i=0; i<myByteArray.length; i++)
{
myByteArray[i] ^= mask[i];
}
Use the bit-exclusive-or operator ^. The truth table for xor is
M a s k
| 0 | 1
D -+---+---
a 0| 0 | 1
t -+---+---
a 1| 1 | 0
Wherever there's a 1 in the mask the corresponding bit in the data will be flipped.

Switching hexadecimal symbols in a byte

I want to switch the two hexadecimals symbols in a byte, for example if
input = 0xEA
then
output = 0xAE
It has to be in java.
I already have this method I made, but it only works in some cases:
public static final byte convert(byte in){
byte hex1 = (byte) (in << 4);
byte hex2 = (byte) (in >>> 4);
return (byte) (hex1 | hex2);
}
A working example is:
input: 0x3A
hex1: 0xA0
hex2: 0x03
output: 0xA3
A not working example is:
input: 0xEA
hex1: 0xA0
hex2: 0xFE
output: 0xFE
Anyone can shed some lights on why this is not working?

I suspect the problem is the sign extension. Specifically, you probably need to do
byte hex2 = (byte) ((in >>> 4) & 0xF);

try
byte hex1 = (byte) (in << 4);
byte hex2 = (byte) ( in >>> 4);
return (byte) (hex1 | hex2 & 0x0F);
this is like in a known puzzle
byte x = (byte)0xFF;
x = (byte) (x >>> 1);
System.out.println(x);
prints -1 because before unsigned shift 0xFF is promoted to int -> 0xFFFFFFFF; after shift it is 0x7FFFFFFF; cast to byte -> 0xFF
but
byte x = (byte)0xFF;
x = (byte) ((x & 0xFF) >>> 1);
System.out.println(x);
prints 127 because we truncated 0xFFFFFFFF -> 0x000000FF, now shift produces 0x0000007F, cast to byte -> 0x7F
Actually, this promotion is done at compile time. JVM works only with 4 or 8 bytes operands (local variables on stack). Even boolean in bytecode is 0 or 1 int.

Java stripping zeros from function

I'm trying to flip some bytes around in Java and the function I have is working correctly for some bytes and failing for others.
The function I am using is this:
public static int foldInByte(int m, int pos, byte b) {
int tempInt = (b << (pos * 8));
tempInt = tempInt & (0x000000ff << (pos * 8));
m = m | tempInt;
return m;
}
And the code that implements this is:
byte[] bitMaskArray = new byte[]{
byteBuffer.get(inputIndex),
byteBuffer.get(inputIndex + 1),
byteBuffer.get(inputIndex + 2),
byteBuffer.get(inputIndex + 3)};
int tempInt = 0;
tempInt = foldInByte(0, 3, bitMaskArray[3]);
tempInt = foldInByte(tempInt, 2, bitMaskArray[2]);
tempInt = foldInByte(tempInt, 1, bitMaskArray[1]);
tempInt = foldInByte(tempInt, 0, bitMaskArray[0]);
bitMask = tempInt;
The bytes are being read from a ByteBuffer with the byteOrder being Little Endian.
For example, the bytes 00 01 B6 02 set the bitMask to: 2B60100 - which works perfectly in my program.
However, if the bytes are A0 01 30 00, the bitMask is set to: 3001A0 - which has stipped the last zero from the bitmask.
Is there any way I can stop Java from stipping off trailing zeros?
I hope that makes sense.
Thanks
Tony

The zeros are not being stripped -- both examples cited are correct.
00 01 B6 02 is the 4-byte little-endian for 2B60100
A0 01 30 00 is the 4-byte little-endian for 3001A0
The zeros are there, but probably just not being printed. The System.out.print family of calls will not print leading zero digits.
I might mention that your method is needlessly complex. Here is a single method that computes the same value:
static int extractLittleEndian4(byte[] buf, int index)
{
int a = buf[index+0]&0xff, b = buf[index+1]&0xff, c = buf[index+2]&0xff, d = buf[index+3]&0xff;
return a | (b << 8) | (c << 16) | (d << 24);
}

It looks like you have a ByteBuffer filled with your bytes already. Why don't you let the ByteBuffer reverse the bytes for you? Just add the bytes to the buffer (BIG_ENDIAN is the default if you want to add an integer instead of bytes) and then change the order before reading the integer.
byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
int output = byteBuffer.getInt(0);
If all you're doing is reversing the byte order, let the library do the work for you. If you happened to start with an integer value, you can even just do this:
int input = ...;
int output = Integer.reverseBytes(input);

How to convert a byte array to its numeric value (Java)?

I have an 8 byte array and I want to convert it to its corresponding numeric value.
e.g.
byte[] by = new byte[8]; // the byte array is stored in 'by'
// CONVERSION OPERATION
// return the numeric value
I want a method that will perform the above conversion operation.

One could use the Buffers that are provided as part of the java.nio package to perform the conversion.
Here, the source byte[] array has a of length 8, which is the size that corresponds with a long value.
First, the byte[] array is wrapped in a ByteBuffer, and then the ByteBuffer.getLong method is called to obtain the long value:
ByteBuffer bb = ByteBuffer.wrap(new byte[] {0, 0, 0, 0, 0, 0, 0, 4});
long l = bb.getLong();
System.out.println(l);
Result
4
I'd like to thank dfa for pointing out the ByteBuffer.getLong method in the comments.
Although it may not be applicable in this situation, the beauty of the Buffers come with looking at an array with multiple values.
For example, if we had a 8 byte array, and we wanted to view it as two int values, we could wrap the byte[] array in an ByteBuffer, which is viewed as a IntBuffer and obtain the values by IntBuffer.get:
ByteBuffer bb = ByteBuffer.wrap(new byte[] {0, 0, 0, 1, 0, 0, 0, 4});
IntBuffer ib = bb.asIntBuffer();
int i0 = ib.get(0);
int i1 = ib.get(1);
System.out.println(i0);
System.out.println(i1);
Result:
1
4

Assuming the first byte is the least significant byte:
long value = 0;
for (int i = 0; i < by.length; i++)
{
value += ((long) by[i] & 0xffL) << (8 * i);
}
Is the first byte the most significant, then it is a little bit different:
long value = 0;
for (int i = 0; i < by.length; i++)
{
value = (value << 8) + (by[i] & 0xff);
}
Replace long with BigInteger, if you have more than 8 bytes.
Thanks to Aaron Digulla for the correction of my errors.

If this is an 8-bytes numeric value, you can try:
BigInteger n = new BigInteger(byteArray);
If this is an UTF-8 character buffer, then you can try:
BigInteger n = new BigInteger(new String(byteArray, "UTF-8"));

Simply, you could use or refer to guava lib provided by google, which offers utiliy methods for conversion between long and byte array. My client code:
long content = 212000607777l;
byte[] numberByte = Longs.toByteArray(content);
logger.info(Longs.fromByteArray(numberByte));

You can also use BigInteger for variable length bytes. You can convert it to Long, Integer or Short, whichever suits your needs.
new BigInteger(bytes).intValue();
or to denote polarity:
new BigInteger(1, bytes).intValue();

Complete java converter code for all primitive types to/from arrays
http://www.daniweb.com/code/snippet216874.html

Each cell in the array is treated as unsigned int:
private int unsignedIntFromByteArray(byte[] bytes) {
int res = 0;
if (bytes == null)
return res;
for (int i=0;i<bytes.length;i++){
res = res | ((bytes[i] & 0xff) << i*8);
}
return res;
}

public static long byteArrayToLong(byte[] bytes) {
return ((long) (bytes[0]) << 56)
+ (((long) bytes[1] & 0xFF) << 48)
+ ((long) (bytes[2] & 0xFF) << 40)
+ ((long) (bytes[3] & 0xFF) << 32)
+ ((long) (bytes[4] & 0xFF) << 24)
+ ((bytes[5] & 0xFF) << 16)
+ ((bytes[6] & 0xFF) << 8)
+ (bytes[7] & 0xFF);
}
convert bytes array (long is 8 bytes) to long

You can try use the code from this answer: https://stackoverflow.com/a/68393576/7918717
It parses bytes as a signed number of arbitrary length. A few examples:
bytesToSignedNumber(false, 0xF1, 0x01, 0x04) returns 15794436 (3 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x01, 0x04) returns -251592444 (4 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x04) returns -1080581331768770303 (8 of 9 bytes as long)