We Keep Coding

Java - Assign Value to Fixed Byte Array

I'm am attempting to assign an integer value to an abnormal fixed size byte array (3). I saw about ByteBuffers allocate feature, however putInt attempts to put in 4 bytes, which then breaks due to overflow
For Example:
byte[] messageLength = ByteBuffer.allocate(3).putInt(Integer.parseUnsignedInt("300")).array();
Results in the following exception
Exception in thread "main" java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:527)
at java.nio.HeapByteBuffer.putInt(HeapByteBuffer.java:372)
Obviously, 300 can fit into 3 bytes since in binary it is 0001 0010 1100.
What can I do to put a perfectly legal sized integer value into a non 4 byte array?

You need 4 bytes to allocate an Integer inside a byte buffer.
You can read from the byte buffer manually if you need a special unpacking rule.
Here's an example:
public static byte[] convertInts(int[] source) {
ByteBuffer buffer = ByteBuffer.allocate(4 * source.length);
for (int data : source) {
buffer.putInt(data);
}
buffer.flip();
byte[] destination = new byte[3 * source.length];
for (int i = 0; i < source.length; i++) {
buffer.get();
destination[i * 3] = buffer.get();
destination[i * 3 + 1] = buffer.get();
destination[i * 3 + 2] = buffer.get();
}
return destination;
}
Example usage:
int[] source = {
Integer.parseUnsignedInt("30"),
Integer.parseUnsignedInt("300"),
Integer.parseUnsignedInt("3000"),
Integer.parseUnsignedInt("300000"),
};
byte[] data = convertInts(source);

A simple solution is to convert the Integer value into a byte[] that contains only the necessary bits. The following code works with integers that fit in 1, 2, 3 and 4 bytes:
private static byte[] compressInteger(int value) {
if (value >= Byte.MIN_VALUE && value <= Byte.MAX_VALUE) {
return new byte[] { (byte) value };
} else if (value >= Short.MIN_VALUE && value <= Short.MAX_VALUE) {
return new byte[] { (byte) (value >>> 8), (byte) (value) };
} else if ((byte)(value >>> 24) == 0) {
return new byte[] { (byte) (value >>> 16), (byte) (value >>> 8), (byte) (value) };
} else {
return new byte[] { (byte) (value >>> 24), (byte) (value >>> 16), (byte) (value >>> 8), (byte) (value) };
}
}
If you want to convert the byte[] back to an integer value, you can do:
private static int decompressInteger(byte[] bytes) {
int value = 0;
for (int i = bytes.length - 1; i >= 0; i--) {
for (int bit = 0; bit <= 7; bit++) {
boolean isSet = ((bytes[i] >>> bit) & 1) == 1;
if (isSet) {
int shift = 8 * (bytes.length - 1 - i) + bit;
int mask = 1 << shift;
value |= mask;
}
}
}
return value;
}

Java. Extracting integers from bits in a byte array not fitting the byte boundary

I have the following array of bytes:
01010110 01110100 00100101 01001011
These bytes are broken into two groups to encode seven integers. I know that the first group consists of 3 values 4 bits each (0101 0110 0111) that represent numbers 5,6,7. The second group consists of 4 values 5 bits each (01000 01001 01010 01011), which represent integers 8,9,10, and 11.
To extract the integers, I am currently using the following approach. Convert the array into a binary string:
public static String byteArrayToBinaryString(byte[] byteArray)
{
String[] arrayOfStrings = new String[byteArray.length];
for(int i=0; i<byteArray.length; i++)
{
arrayOfStrings[i] = byteToBinaryString(byteArray[i]);
}
String bitsetString = "";
for(String testArrayStringElement : arrayOfStrings)
{
bitsetString += testArrayStringElement;
}
return bitsetString;
}
// Taken from here: http://helpdesk.objects.com.au/java/converting-large-byte-array-to-binary-string
public static String byteToBinaryString(byte byteIn)
{
StringBuilder sb = new StringBuilder("00000000");
for (int bit = 0; bit < 8; bit++)
{
if (((byteIn >> bit) & 1) > 0)
{
sb.setCharAt(7 - bit, '1');
}
}
return sb.toString();
}
Then, I split the binary string into 2 substrings: 12 characters and 20 characters. Then I split each substring into new substrings, each of which has length that equals the number of bits. Then I convert each sub-substring into an integer.
It works but a byte array representing thousands of integers takes 30 seconds to a minute to extract.
I am a bit at a loss here. How do I do this using bitwise operators?
Thanks a lot!

I assume you have an understanding of the basic bit operations and how to express them in Java.
Use a pencil to draw a synthetic picture of the problem
byte 0 byte 1 byte 2 byte 3
01010110 01110100 00100101 01001011
\__/\__/ \__/\______/\___/\______/\___/
a b c d e f g
To extract a, b and c we need to do the following
a b c
byte 0 byte 0 byte 1
01010110 01010110 01110100
\. \. |||||||| \. \.
'\ '\ XXXX|||| '\ '\
0.. 0101 0.. 0110 0.. 0111
Shift And Shift
In Java
int a = byteArray[0] >>> 4, b = byteArray[0] & 0xf, c = byteArray[1] >>> 4;
The other values d, e, f and g are computed similarly but some of them require to read two bytes from the array (d and f actually).
d e
byte 1 byte 2 byte 2
01110100 00100101 00100101
||||\\\\ | |\\\\\
XXXX \\\\ | X \\\\\
\\\\| \\\\\
0.. 01000 01001
To compute d we need to isolate the least four bits of byte 1 with byteArray[1] & 0xf then make space for the bit from byte 2 with (byteArray[1] & 0xf) << 1, extract that bit with byteArray[1] >>> 7 and finally merge together the result.
int d = (byteArray[1] & 0xf) << 1 | byteArray[2] >>> 7;
int e = (byteArray[2] & 0x7c) >>> 2;
int f = (byteArray[2] & 0x3) << 3 | byteArray[3] >>> 5;
int g = byteArray[3] & 0x1f;
When you are comfortable with handling bits operations you may consider generalizing the function that extract the integers.
I made function int extract(byte[] bits, int[] sizes, int[] res), that given an array of bytes bits, an array of sizes sizes, where the even indices hold the size of the integers to extract in bits and the odd indices the number of integers to extract, and an output array res large enough to hold all the integers in output, extracts from bits all the integers expressed by sizes.
It returns the number of integers extracted.
For example the original problem can be solved as
int res[] = new int[8];
byte bits[] = new byte[]{0x56, 0x74, 0x25, 0x4b};
//Extract 3 integers of 4 bits and 4 integers of 5 bits
int ints = BitsExtractor.extract(bits, new int[]{4, 3, 5, 4}, res);
public class BitsExtractor
{
public static int extract(byte[] bits, int[] sizes, int[] res)
{
int currentByte = 0; //Index into the bits array
int intProduced = 0; //Number of ints produced so far
int bitsLeftInByte = 8; //How many bits left in the current byte
int howManyInts = 0; //Number of integers to extract
//Scan the sizes array two items at a time
for (int currentSize = 0; currentSize < sizes.length - 1; currentSize += 2)
{
//Size, in bits, of the integers to extract
int intSize = sizes[currentSize];
howManyInts += sizes[currentSize+1];
int temp = 0; //Temporary value of an integer
int sizeLeft = intSize; //How many bits left to extract
//Do until we have enough integer or we exhaust the bits array
while (intProduced < howManyInts && currentByte <= bits.length)
{
//How many bit we can extract from the current byte
int bitSize = Math.min(sizeLeft, bitsLeftInByte); //sizeLeft <= bitsLeftInByte ? sizeLeft : bitsLeftInByte;
//The value to mask out the number of bit extracted from
//The current byte (e.g. for 3 it is 7)
int byteMask = (1 << bitSize) - 1;
//Extract the new bits (Note that we extract starting from the
//RIGHT so we need to consider the bits left in the byte)
int newBits = (bits[currentByte] >>> (bitsLeftInByte - bitSize)) & byteMask;
//Create the new temporary value of the current integer by
//inserting the bits in the lowest positions
temp = temp << bitSize | newBits;
//"Remove" the bits processed from the byte
bitsLeftInByte -= bitSize;
//Is the byte has been exhausted, move to the next
if (bitsLeftInByte == 0)
{
bitsLeftInByte = 8;
currentByte++;
}
//"Remove" the bits processed from the size
sizeLeft -= bitSize;
//If we have extracted all the bits, save the integer
if (sizeLeft == 0)
{
res[intProduced++] = temp;
temp = 0;
sizeLeft = intSize;
}
}
}
return intProduced;
}
}

Well I did the first group , the second can be done in similar fashion
public static void main(String args[]) {
//an example 32 bits like your example
byte[] bytes = new byte[4];
bytes[0] = 31;//0001 1111
bytes[1] = 54;//0011 0110
bytes[2] = 67;
bytes[3] = 19;
//System.out.println(bytes[0]);
int x = 0;
int j = -1; // the byte number
int k = 0; // the bit number in that byte
int n = 0; // the place of the bit in the integer we are trying to read
for (int i = 0; i < 32; i++) {
if (i < 12) { //first group
if (i % 8 == 0) {
j++;
k = 0;
}
if (i % 4 == 0) {
x = 0;
n = 0;
}
byte bit = (byte) ((bytes[j] & (1 << (7 - k))) >> (7 - k));
System.out.println("j is :" + j + " k is :" + k + " " + bit);
x = x | bit << (3 - n);
if ((i + 1) % 4 == 0) {
System.out.println(x);
}
k++;
n++;
} else {
}
}
}
It's a bit tricky because you are trying to encode an integer on less than what java allocates (8 bits). So I had to take each bit and "construct" the int from them
To get each bit
byte bit = (byte) ((bytes[j] & (1 << (7 - k))) >> (7 - k));
this takes the byte we are at and does And operation. For example I want the 3rd bit of the 1st byte, I do
bytes[0] & 1 << (7 - 3)
but this gives me an integer encoded over 8 bits, so I still have to shift it to get that single bit with >> (7 - 3)
Then I just Or it with x (the int we are trying to decode). All while putting it at the right position with << (3 - n) . 3 because your integer is encoded over 4 bits
Try running the code and reading the output.
I am honestly not sure if this is the best way, but I believe it's at least faster than dealing with Strings

Reading bytes in a file :java.lang.NegativeArraySizeException

I have gone through the similar posts but none of them really answered my question. Hence I post my question in a separate thread.
I need to skip some bytes in the file which I am reading in the byte array.I am trying to achieve this through code below
1. byte [] readBytesToSKip = null;
2. readBytesToSKip = new byte[(int)bytesToSkip];
3. bytesReadToSkip = System.in.read(readBytesToSKip) ;
4. if(bytesReadToSkip > 0)
5. {
6. baos_.write(readBytesToSKip, 0, bytesReadToSkip);
7. }
But I get a NegativeArraySizeException at line 2 where the size exceeds Integer.MAX_VALUE. I am not sure how to achieve this otherwise.
bytesToSkip is long as I calculate in function below:
public static long bytesToLong1(byte[] bytes) {
long value = 0;
for (int i = 0; i < bytes.length; i++)
{
value = (value << 8) + (bytes[i] & 0xff);
}
return value;
}

Starting from your posted bytes->long function
public static long bytesToLong1(byte[] bytes) {
long value = 0;
for (int i = 0; i < bytes.length; i++)
{
value = (value << 8) + (bytes[i] & 0xff);
}
return value;
}
Whenever bytes.length==8 and the first byte is >= 128 in absolute value (in Java specific: it's a negative byte value), you'll end having a negative value for your long.
Example:
byte vals[]={0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
System.out.println(bytesToLong1(vals));
will result it -1 being printed. Which will convert quite fine to an int with a -1 value, but will still be improper for the length of an array.

Sending Java int to C over TCP

I'm trying to send Java's signed integers over TCP to a C client.
At the Java side, I write the integers to the outputstream like so:
static ByteBuffer wrapped = ByteBuffer.allocateDirect(4); // big-endian by default
public static void putInt(OutputStream out, int nr) throws IOException {
wrapped.rewind();
wrapped.putInt(nr);
wrapped.rewind();
for (int i = 0; i < 4; i++)
out.write(wrapped.get());
}
At the C side, I read the integers like so:
int cnt = 0;
char buf[1];
char sizebuf[4];
while(cnt < 4) {
iResult = recv(ConnectSocket, buf, 1, 0);
if (iResult <= 0) continue;
sizebuf[cnt] = buf[0];
cnt++;
}
However, how do I convert the char array to an integer in C?
Edit
I have tried the following (and the reverse):
int charsToInt(char* array) {
return (array[3] << 24) | (array[2] << 16) | (array[1] << 8) | array[0];
}
Edited again, because I forgot the tags.
Data
For example of what happens currently:
I receive:
char 0
char 0
char 12
char -64
the int becomes 2448
and use the function for creating the int from the char array:
int charsToInt(char* array) {
return ntohl(*((int*) array));
}
I expect the signed integer: 3264
Update
I will investigate more after some sleep..
Update
I have a Java client which interprets the integers correctly and receives the exact same bytes:
0
0
12
-64

That depends on endianness, but you want either:
int x = sizebuf[0] +
(sizebuf[1] << 8) +
(sizebuf[2] << 16) +
(sizebuf[3] << 24);
or:
int x = sizebuf[3] +
(sizebuf[2] << 8) +
(sizebuf[1] << 16) +
(sizebuf[0] << 24);
Note that sizebuf needs to have an unsigned type for this to work correctly. Otherwise you need to mask off any sign-extended values you don't want:
int x = (sizebuf[3] & 0x000000ff) +
((sizebuf[2] << 8) & 0x0000ff00) +
((sizebuf[1] << 16) & 0x00ff0000) +
((sizebuf[0] << 24) & 0xff000000);

The classical C library has the method you want already, and it is independent from the machine endianness: ntohl!
// buf is a char */uint8_t *
uint32_t from_network = *((uint32_t *) buf);
uint32_t ret = ntohl(from_network);
This, and htonl for the reverse etc expect that the "network order" is big endian.
(the code above presupposes that buf has at least 4 bytes; the return type, and argument type, of ntohl and htonl are uint32_t; the JLS defines an int as 4 bytes so you are guaranteed the result)

To convert you char array, one possibility is to cast it to int* and to store the result :
int result = *((int*) sizebuf)
This is valid and one line. Other possibility is to compute integer from chars.
for (i = 0 ; i < 4; i++)
result = result << sizeof(char) + buf[0]
Choose the one that you prefer.
Alexis.
Edit :
sizeof(char) is 1 because sizeof return a Byte result. So the right line is :
result = result << (sizeof(char) * 8) + buf[0]

Algorithm to convert a String of decimal digits to BCD

I am looking a way to convert a string to BCD equivalent. I use Java, but it is not a question of the language indeed. I am trying to understand step by step how to convert a string to BCD.
For example, suppose I have the following string;
"0200" (This string has four ASCII characters, if we were in java this string had been contained in a byte[4] where byte[0] = 48, byte[1] = 50, byte[2] = 48 and byte[3] = 48)
In BCD (according this page: http://es.wikipedia.org/wiki/Decimal_codificado_en_binario):
0 = 0000
2 = 0010
0 = 0000
0 = 0000
Ok, I think the conversion is correct but I have to save this in a byte[2]. What Should I have to do? After, I have to read the BCD and convert it to the original string "0200" but first I have to resolve String to BCD.

Find a utility class to do this for you. Surely someone out there has written a BCD conversion utility for Java.
Here you go. I Googled "BCD Java" and got this as the first result. Copying code here for future reference.
public class BCD {
/*
* long number to bcd byte array e.g. 123 --> (0000) 0001 0010 0011
* e.g. 12 ---> 0001 0010
*/
public static byte[] DecToBCDArray(long num) {
int digits = 0;
long temp = num;
while (temp != 0) {
digits++;
temp /= 10;
}
int byteLen = digits % 2 == 0 ? digits / 2 : (digits + 1) / 2;
boolean isOdd = digits % 2 != 0;
byte bcd[] = new byte[byteLen];
for (int i = 0; i < digits; i++) {
byte tmp = (byte) (num % 10);
if (i == digits - 1 && isOdd)
bcd[i / 2] = tmp;
else if (i % 2 == 0)
bcd[i / 2] = tmp;
else {
byte foo = (byte) (tmp << 4);
bcd[i / 2] |= foo;
}
num /= 10;
}
for (int i = 0; i < byteLen / 2; i++) {
byte tmp = bcd[i];
bcd[i] = bcd[byteLen - i - 1];
bcd[byteLen - i - 1] = tmp;
}
return bcd;
}
public static String BCDtoString(byte bcd) {
StringBuffer sb = new StringBuffer();
byte high = (byte) (bcd & 0xf0);
high >>>= (byte) 4;
high = (byte) (high & 0x0f);
byte low = (byte) (bcd & 0x0f);
sb.append(high);
sb.append(low);
return sb.toString();
}
public static String BCDtoString(byte[] bcd) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < bcd.length; i++) {
sb.append(BCDtoString(bcd[i]));
}
return sb.toString();
}
}
There's also this question: Java code or lib to decode a binary-coded decimal (BCD) from a String.

The first step would be to parse the string into an int so that you have the numeric value of it. Then, get the individual digits using division and modulus, and pack each pair of digits into a byte using shift and add (or shift and or).
Alternatively, you could parse each character of the string into an int individually, and avoid using division and modulus to get the numbers, but I would prefer to parse the entire string up front so that you discover right away if the string is invalid. (If you get a NumberFormatException, or if the value is less than 0 or greater than 9999 then it is invalid.)
Finally, once you have assembled the two individual bytes, you can put them into the byte[2].

You can use following:
//Convert BCD String to byte array
public static byte[] String2Bcd(java.lang.String bcdString) {
byte[] binBcd = new byte[bcdString.length() / 2];
for (int i = 0; i < binBcd.length; i++) {
String sByte = bcdString.substring(i*2, i*2+2);
binBcd[i] = Byte.parseByte(sByte, 16);
}
return binBcd;
}

You can try the following code:
public static byte[] hex2Bytes(String str) {
byte[] b = new byte[str.length() / 2];
int j = 0;
for (int i = 0; i < b.length; i++) {
char c0 = str.charAt(j++);
char c1 = str.charAt(j++);
b[i] = ((byte) (parse(c0) << 4 | parse(c1)));
}
return b;
}