We Keep Coding

Java - Assign Value to Fixed Byte Array

I'm am attempting to assign an integer value to an abnormal fixed size byte array (3). I saw about ByteBuffers allocate feature, however putInt attempts to put in 4 bytes, which then breaks due to overflow
For Example:
byte[] messageLength = ByteBuffer.allocate(3).putInt(Integer.parseUnsignedInt("300")).array();
Results in the following exception
Exception in thread "main" java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:527)
at java.nio.HeapByteBuffer.putInt(HeapByteBuffer.java:372)
Obviously, 300 can fit into 3 bytes since in binary it is 0001 0010 1100.
What can I do to put a perfectly legal sized integer value into a non 4 byte array?

You need 4 bytes to allocate an Integer inside a byte buffer.
You can read from the byte buffer manually if you need a special unpacking rule.
Here's an example:
public static byte[] convertInts(int[] source) {
ByteBuffer buffer = ByteBuffer.allocate(4 * source.length);
for (int data : source) {
buffer.putInt(data);
}
buffer.flip();
byte[] destination = new byte[3 * source.length];
for (int i = 0; i < source.length; i++) {
buffer.get();
destination[i * 3] = buffer.get();
destination[i * 3 + 1] = buffer.get();
destination[i * 3 + 2] = buffer.get();
}
return destination;
}
Example usage:
int[] source = {
Integer.parseUnsignedInt("30"),
Integer.parseUnsignedInt("300"),
Integer.parseUnsignedInt("3000"),
Integer.parseUnsignedInt("300000"),
};
byte[] data = convertInts(source);

A simple solution is to convert the Integer value into a byte[] that contains only the necessary bits. The following code works with integers that fit in 1, 2, 3 and 4 bytes:
private static byte[] compressInteger(int value) {
if (value >= Byte.MIN_VALUE && value <= Byte.MAX_VALUE) {
return new byte[] { (byte) value };
} else if (value >= Short.MIN_VALUE && value <= Short.MAX_VALUE) {
return new byte[] { (byte) (value >>> 8), (byte) (value) };
} else if ((byte)(value >>> 24) == 0) {
return new byte[] { (byte) (value >>> 16), (byte) (value >>> 8), (byte) (value) };
} else {
return new byte[] { (byte) (value >>> 24), (byte) (value >>> 16), (byte) (value >>> 8), (byte) (value) };
}
}
If you want to convert the byte[] back to an integer value, you can do:
private static int decompressInteger(byte[] bytes) {
int value = 0;
for (int i = bytes.length - 1; i >= 0; i--) {
for (int bit = 0; bit <= 7; bit++) {
boolean isSet = ((bytes[i] >>> bit) & 1) == 1;
if (isSet) {
int shift = 8 * (bytes.length - 1 - i) + bit;
int mask = 1 << shift;
value |= mask;
}
}
}
return value;
}

Bit shift operations on a byte array in Java

How do I shift a byte array n positions to the right? For instance shifting a 16 byte array right 29 positions? I read somewhere it can be done using a long? Would using a long work like this:
Long k1 = byte array from 0 to 7
Long k2 = byte array from 8 to 15
Then right rotating these two longs using Long.rotateRight(Long x, number of rotations).How would the two longs be joined back into a byte array?

I believe you can do this using java.math.BigInteger which supports shifts on arbitrarily large numbers. This has advantage of simplicity, but disadvantage of not padding into original byte array size, i.e. input could be 16 bytes but output might only be 10 etc, requiring additional logic.
BigInteger approach
byte [] array = new byte[]{0x7F,0x11,0x22,0x33,0x44,0x55,0x66,0x77};
// create from array
BigInteger bigInt = new BigInteger(array);
// shift
BigInteger shiftInt = bigInt.shiftRight(4);
// back to array
byte [] shifted = shiftInt.toByteArray();
// print it as hex
for (byte b : shifted) {
System.out.print(String.format("%x", b));
}
Output
7f1122334455667 <== shifted 4 to the right. Looks OK
Long manipulation
I don't know why you'd want to do this as rotateRight() as this makes life more difficult, you have to blank at the bits that appear at the left hand side in K1 etc. You'd be better with using shift IMO as describe below. I've used a shift of 20 as divisible by 4 so easier to see the nibbles move in the output.
1) Use ByteBuffer to form two longs from 16 byte array
byte[] array = { 0x00, 0x00, 0x11, 0x11, 0x22, 0x22, 0x33, 0x33, 0x44, 0x44, 0x55, 0x55, 0x66, 0x66, 0x77, 0x77 };
ByteBuffer buffer = ByteBuffer.wrap(array);
long k1 = buffer.getLong();
long k2 = buffer.getLong();
2) Shift each long n bits to the right
int n = 20;
long k1Shift = k1 >> n;
long k2Shift = k2 >> n;
System.out.println(String.format("%016x => %016x", k1, k1Shift));
System.out.println(String.format("%016x => %016x", k2, k2Shift));
0000111122223333 => 0000000001111222
4444555566667777 => 0000044445555666
Determine bits from k1 that "got pushed off the edge"
long k1CarryBits = (k1 << (64 - n));
System.out.println(String.format("%016x => %016x", k1, k1CarryBits));
0000111122223333 => 2333300000000000
Join the K1 carry bits onto K2 on right hand side
long k2WithCarray = k2Shift | k1CarryBits;
System.out.println(String.format("%016x => %016x", k2Shift, k2WithCarray));
0000044445555666 => 2333344445555666
Write the two longs back into a ByteBuffer and extract as a byte array
buffer.position(0);
buffer.putLong(k1Shift);
buffer.putLong(k2WithCarray);
for (byte each : buffer.array()) {
System.out.print(Long.toHexString(each));
}
000011112222333344445555666

Here is what I came up with to shift a byte array by some arbitrary number of bits left:
/**
* Shifts input byte array len bits left.This method will alter the input byte array.
*/
public static byte[] shiftLeft(byte[] data, int len) {
int word_size = (len / 8) + 1;
int shift = len % 8;
byte carry_mask = (byte) ((1 << shift) - 1);
int offset = word_size - 1;
for (int i = 0; i < data.length; i++) {
int src_index = i+offset;
if (src_index >= data.length) {
data[i] = 0;
} else {
byte src = data[src_index];
byte dst = (byte) (src << shift);
if (src_index+1 < data.length) {
dst |= data[src_index+1] >>> (8-shift) & carry_mask;
}
data[i] = dst;
}
}
return data;
}

1. Manually implemented
Here are left and right shift implementation without using BigInteger (ie. without creating a copy of the input array) and with unsigned right shift (BigInteger only supports arithmetic shifts of course)
Left Shift <<
/**
* Left shift of whole byte array by shiftBitCount bits.
* This method will alter the input byte array.
*/
static byte[] shiftLeft(byte[] byteArray, int shiftBitCount) {
final int shiftMod = shiftBitCount % 8;
final byte carryMask = (byte) ((1 << shiftMod) - 1);
final int offsetBytes = (shiftBitCount / 8);
int sourceIndex;
for (int i = 0; i < byteArray.length; i++) {
sourceIndex = i + offsetBytes;
if (sourceIndex >= byteArray.length) {
byteArray[i] = 0;
} else {
byte src = byteArray[sourceIndex];
byte dst = (byte) (src << shiftMod);
if (sourceIndex + 1 < byteArray.length) {
dst |= byteArray[sourceIndex + 1] >>> (8 - shiftMod) & carryMask;
}
byteArray[i] = dst;
}
}
return byteArray;
}
Unsigned Right Shift >>>
/**
* Unsigned/logical right shift of whole byte array by shiftBitCount bits.
* This method will alter the input byte array.
*/
static byte[] shiftRight(byte[] byteArray, int shiftBitCount) {
final int shiftMod = shiftBitCount % 8;
final byte carryMask = (byte) (0xFF << (8 - shiftMod));
final int offsetBytes = (shiftBitCount / 8);
int sourceIndex;
for (int i = byteArray.length - 1; i >= 0; i--) {
sourceIndex = i - offsetBytes;
if (sourceIndex < 0) {
byteArray[i] = 0;
} else {
byte src = byteArray[sourceIndex];
byte dst = (byte) ((0xff & src) >>> shiftMod);
if (sourceIndex - 1 >= 0) {
dst |= byteArray[sourceIndex - 1] << (8 - shiftMod) & carryMask;
}
byteArray[i] = dst;
}
}
return byteArray;
}
Used in this class by this Project.
2. Using BigInteger
Be aware that BigInteger internally converts the byte array into an int[] array so this may not be the most optimized solution:
Arithmetic Left Shift <<:
byte[] result = new BigInteger(byteArray).shiftLeft(3).toByteArray();
Arithmetic Right Shift >>:
byte[] result = new BigInteger(byteArray).shiftRight(2).toByteArray();
3. External Library
Using the Bytes java library*:
Add to pom.xml:
<dependency>
<groupId>at.favre.lib</groupId>
<artifactId>bytes</artifactId>
<version>{latest-version}</version>
</dependency>
Code example:
Bytes b = Bytes.wrap(someByteArray);
b.leftShift(3);
b.rightShift(3);
byte[] result = b.array();
*Full Disclaimer: I am the developer.

The is an old post, but I want to update Adam's answer.
The long solution works with a few tweak.
In order to rotate, use >>> instead of >>, because >> will pad with significant bit, changing the original value.
second, the printbyte function seems to miss leading 00 when it prints.
use this instead.
private String getHexString(byte[] b) {
StringBuilder result = new StringBuilder();
for (int i = 0; i < b.length; i++)
result.append(Integer.toString((b[i] & 0xff) + 0x100, 16)
.substring(1));
return result.toString();
}

Algorithm to convert a String of decimal digits to BCD

I am looking a way to convert a string to BCD equivalent. I use Java, but it is not a question of the language indeed. I am trying to understand step by step how to convert a string to BCD.
For example, suppose I have the following string;
"0200" (This string has four ASCII characters, if we were in java this string had been contained in a byte[4] where byte[0] = 48, byte[1] = 50, byte[2] = 48 and byte[3] = 48)
In BCD (according this page: http://es.wikipedia.org/wiki/Decimal_codificado_en_binario):
0 = 0000
2 = 0010
0 = 0000
0 = 0000
Ok, I think the conversion is correct but I have to save this in a byte[2]. What Should I have to do? After, I have to read the BCD and convert it to the original string "0200" but first I have to resolve String to BCD.

Find a utility class to do this for you. Surely someone out there has written a BCD conversion utility for Java.
Here you go. I Googled "BCD Java" and got this as the first result. Copying code here for future reference.
public class BCD {
/*
* long number to bcd byte array e.g. 123 --> (0000) 0001 0010 0011
* e.g. 12 ---> 0001 0010
*/
public static byte[] DecToBCDArray(long num) {
int digits = 0;
long temp = num;
while (temp != 0) {
digits++;
temp /= 10;
}
int byteLen = digits % 2 == 0 ? digits / 2 : (digits + 1) / 2;
boolean isOdd = digits % 2 != 0;
byte bcd[] = new byte[byteLen];
for (int i = 0; i < digits; i++) {
byte tmp = (byte) (num % 10);
if (i == digits - 1 && isOdd)
bcd[i / 2] = tmp;
else if (i % 2 == 0)
bcd[i / 2] = tmp;
else {
byte foo = (byte) (tmp << 4);
bcd[i / 2] |= foo;
}
num /= 10;
}
for (int i = 0; i < byteLen / 2; i++) {
byte tmp = bcd[i];
bcd[i] = bcd[byteLen - i - 1];
bcd[byteLen - i - 1] = tmp;
}
return bcd;
}
public static String BCDtoString(byte bcd) {
StringBuffer sb = new StringBuffer();
byte high = (byte) (bcd & 0xf0);
high >>>= (byte) 4;
high = (byte) (high & 0x0f);
byte low = (byte) (bcd & 0x0f);
sb.append(high);
sb.append(low);
return sb.toString();
}
public static String BCDtoString(byte[] bcd) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < bcd.length; i++) {
sb.append(BCDtoString(bcd[i]));
}
return sb.toString();
}
}
There's also this question: Java code or lib to decode a binary-coded decimal (BCD) from a String.

The first step would be to parse the string into an int so that you have the numeric value of it. Then, get the individual digits using division and modulus, and pack each pair of digits into a byte using shift and add (or shift and or).
Alternatively, you could parse each character of the string into an int individually, and avoid using division and modulus to get the numbers, but I would prefer to parse the entire string up front so that you discover right away if the string is invalid. (If you get a NumberFormatException, or if the value is less than 0 or greater than 9999 then it is invalid.)
Finally, once you have assembled the two individual bytes, you can put them into the byte[2].

You can use following:
//Convert BCD String to byte array
public static byte[] String2Bcd(java.lang.String bcdString) {
byte[] binBcd = new byte[bcdString.length() / 2];
for (int i = 0; i < binBcd.length; i++) {
String sByte = bcdString.substring(i*2, i*2+2);
binBcd[i] = Byte.parseByte(sByte, 16);
}
return binBcd;
}

You can try the following code:
public static byte[] hex2Bytes(String str) {
byte[] b = new byte[str.length() / 2];
int j = 0;
for (int i = 0; i < b.length; i++) {
char c0 = str.charAt(j++);
char c1 = str.charAt(j++);
b[i] = ((byte) (parse(c0) << 4 | parse(c1)));
}
return b;
}

Java: reading a variable number of bits and converting to an integer

I need to write a method which reads a variable number of bits from a ByteBuffer and converts those bytes into a primitive int or long (if more than 32 bits). I am not very proficient at working with bits, but I did look at some code examples and answers to similar(ish) questions and here is the best that I could come up with (bb refers to an internal ByteBuffer):
public int readBits(int number) {
int initialPosition = bb.position();
int value = 0;
int remaining = number;
int position = bb.position();
while (remaining > 0) {
int index = (position >> 3);
int bit = (position & 7);
int bitsLeft = Math.min(8 - bit, remaining);
int nibble = (bb.get(index) >> (bit & BIT_MASKS[bitsLeft]));
value |= nibble << (number - remaining);
position += bitsLeft;
remaining -= bitsLeft;
}
bb.position(initialPosition + ((position - initialPosition) >> 3));
return value;
}
The BIT_MASKS array is defined as following:
private static final int[] BIT_MASKS = new int[] { 0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff };
I tested this method using this test code:
public static void main(String[] args) {
int nm = 4;
BitSet bs = new BitSet(nm);
bs.set(2);
bs.set(3);
BitBuffer bb = new BitBuffer(ByteBuffer.wrap(bs.toByteArray()));
System.out.println(bb.readBits(nm));
}
I expected a value of 6, but it prints 12 and I'm not sure why.
I would appreciate some help with this. Thanks.

adler32 java bitwise and of input

I found the code for adler32 here http://developer.classpath.org/doc/java/util/zip/Adler32-source.html
however my update code looks like following
private int a = 1, b = 0;
public void update(byte[] buf, int offset, int len)
{
for (int i = offset; i < len; i++) {
a = (a + buf[i]) % MOD_ADLER;
b = (b + a) % MOD_ADLER;
}
}
as oppose code on the link
public void update(byte[] buf, int offset, int len)
{
for (int i = offset; i < len; i++) {
a = (a + (buf[i] & 0xff)) % MOD_ADLER; // <<=== Why & 0xff ?
b = (b + a) % MOD_ADLER;
}
}
I do not understand need to AND with 0xff as buf[i] is already 8 bytes, I understand it will be promoted to int as a is int, still the type promotion shouldn't change value in byte. May be I'm missing some more details as it doesnt work without & 0xff, I tested it with the values computed by java.util.zip.Adler32
Thanks for the answer, however it will only matter for values that result in negative numbers, for example in my test
byte a = -4;
int n1 = a & 0xff;
int n2 = a;
out.printf(" a %4d %s\n", a,Integer.toBinaryString(a));
out.printf("n1 %4d %s\n",n1,Integer.toBinaryString(n1));
out.printf("n2 %4d %s\n",n2,Integer.toBinaryString(n2));
prints
a -4 11111111111111111111111111111100
n1 252 11111100
n2 -4 11111111111111111111111111111100

The problem is that byte is signed in Java. Hence, the automatic type promotion byte -> int would always give a integer in the range (-128,127) instead of (0,255), as we want. The & 0xff operation fixes that.