So say I have a byte array, and I have a function that checks whether the n-th least significant bit index of the byte array is a 1 or a 0. The function returns true if the bit is a 1 and false if the bit is a 0. The least significant bit of the byte array is defined as the last significant bit in the 0th index of the byte array, and the most significant bit of the byte array is defined as the most significant bit in the (byte array.length - 1)th index of the byte array.
For instance,
byte[] myArray = new byte[2];
byte[0] = 0b01111111;
byte[1] = 0b00001010;
Calling:
myFunction(0) = true;
myFunction(1) = true;
myFunction(7) = false;
myFunction(8) = false;
myFunction(9) = true;
myFunction(10) = false;
myFunction(11) = true;
What is the best way to do this?
Thanks!
You can use this method:
public boolean isSet(byte[] arr, int bit) {
int index = bit / 8; // Get the index of the array for the byte with this bit
int bitPosition = bit % 8; // Position of this bit in a byte
return (arr[index] >> bitPosition & 1) == 1;
}
bit % 8 is the bit position relative to a byte.
arr[index] >> bit % 8 moves the bit at index to bit 0 position.
Related
I never write here, but as almost everyone I often read questions and answers... now it's my time to ask :)
So, I was reading a long piece of code that works on a stream of bytes to decode tags and values of ePassports (ICAO Doc 9303 defines the International Technical Specifications)... it wasn't difficult until I found this part (I placed comments before and after the cryptic part)
int len = s.read(); // s is an InputStream
readPos++;
if ((len > 0x80)) {
int lenlen = len - 0x80;
len = 0;
for (int i = 0; (i < lenlen); i++) {
if ((readPos == length)) {
throw new ParseException();
}
// wtf begin
len = (len << 8) | ((byte) (s.read()));
// wtf end
readPos++;
}
}
size = readPos + len;
I understand that it reads from an InputStream, byte by byte, and increases the position in the byte stream, but why does it perform the shift and save the result in the variable len? If I'm not wrong, len gets overwritten over and over, so how could this variable have a meaningful value after the for loop?
It is processing lenlen number of bytes, and making out of those bytes a big endian number:
case len was <= 128
if is not entered, and len represents a single byte number
case len == 128+1, +2, +3, +4
Respectively 1, 2, 3, 4 bytes are used: every byte is placed at the least
significant position and where the older bytes are shifted a byte position to the left (<< 8). So 129 for a 1 byte length (for numbers between 128 and 255),
and 130 for a 2 bytes number.
at the end: len bytes are skipped.
Data integrity would require to check for the original len >= 0, and len <= 128 + 4 and for the calculated len being >= 0 too.
The writing process The inverse algorithm you might encounter:
if (len <= 128) {
out.write((byte)len);
} else {
int bytes = 4 - Integer.numberOfLeadingZeroes(len) / 8;
out.write(128 + bytes);
for (int i = 0; i < bytes; ++i) {
int b = len >>>(bytes - 1);
out.write((byte)b); // Truncates the int to just 1 byte.
}
}
(Assuming len is positive.)
I was going through a piece of code in the Apache commons library and was wondering what these conditions do exactly.
public static byte[] decodeHex(final char[] data) throws DecoderException {
final int len = data.length;
if ((len & 0x01) != 0) { // what does this condition do
throw new DecoderException("Odd number of characters.");
}
final byte[] out = new byte[len >> 1];
// two characters form the hex value.
for (int i = 0, j = 0; j < len; i++) {
int f = toDigit(data[j], j) << 4;
j++;
f = f | toDigit(data[j], j);
j++;
out[i] = (byte) (f & 0xFF); // what is happening here.
}
return out;
}
thanks in advance.
This checks if the last digit in the binary writing of len is a 1.
xxxxxxxy
& 00000001
gives 1 if y is 1, 0 if y is 0, ignoring the other digits.
If y is 1, the length of the char array is odd, which shouldn't happen in this hex writing, hence the exception.
Another solution would have been
if (len%2 != 0) {
which would have been clearer in my opinion. I doubt the slight performance increase just before a loop really matters.
It's a 1337 (high performance) way of coding:
if (len % 2 == 1)
i.e. is len odd. It works because the binary representation of every odd integer has its least significant (ie last) bit set. Performaning a bitwise AND with 1 masks all other bits, leaving a result of either 1 if it's odd or 0 if even.
It's a carryover from C, where you can code simply:
if (len & 1)
This line checks if len is an odd number or not.
If len isn't odd, len & 1 will be equal to 0. (1 and 0x01 are the same value, 0x01 is just the hexadecimal notation)
I have the following array of bytes:
01010110 01110100 00100101 01001011
These bytes are broken into two groups to encode seven integers. I know that the first group consists of 3 values 4 bits each (0101 0110 0111) that represent numbers 5,6,7. The second group consists of 4 values 5 bits each (01000 01001 01010 01011), which represent integers 8,9,10, and 11.
To extract the integers, I am currently using the following approach. Convert the array into a binary string:
public static String byteArrayToBinaryString(byte[] byteArray)
{
String[] arrayOfStrings = new String[byteArray.length];
for(int i=0; i<byteArray.length; i++)
{
arrayOfStrings[i] = byteToBinaryString(byteArray[i]);
}
String bitsetString = "";
for(String testArrayStringElement : arrayOfStrings)
{
bitsetString += testArrayStringElement;
}
return bitsetString;
}
// Taken from here: http://helpdesk.objects.com.au/java/converting-large-byte-array-to-binary-string
public static String byteToBinaryString(byte byteIn)
{
StringBuilder sb = new StringBuilder("00000000");
for (int bit = 0; bit < 8; bit++)
{
if (((byteIn >> bit) & 1) > 0)
{
sb.setCharAt(7 - bit, '1');
}
}
return sb.toString();
}
Then, I split the binary string into 2 substrings: 12 characters and 20 characters. Then I split each substring into new substrings, each of which has length that equals the number of bits. Then I convert each sub-substring into an integer.
It works but a byte array representing thousands of integers takes 30 seconds to a minute to extract.
I am a bit at a loss here. How do I do this using bitwise operators?
Thanks a lot!
I assume you have an understanding of the basic bit operations and how to express them in Java.
Use a pencil to draw a synthetic picture of the problem
byte 0 byte 1 byte 2 byte 3
01010110 01110100 00100101 01001011
\__/\__/ \__/\______/\___/\______/\___/
a b c d e f g
To extract a, b and c we need to do the following
a b c
byte 0 byte 0 byte 1
01010110 01010110 01110100
\. \. |||||||| \. \.
'\ '\ XXXX|||| '\ '\
0.. 0101 0.. 0110 0.. 0111
Shift And Shift
In Java
int a = byteArray[0] >>> 4, b = byteArray[0] & 0xf, c = byteArray[1] >>> 4;
The other values d, e, f and g are computed similarly but some of them require to read two bytes from the array (d and f actually).
d e
byte 1 byte 2 byte 2
01110100 00100101 00100101
||||\\\\ | |\\\\\
XXXX \\\\ | X \\\\\
\\\\| \\\\\
0.. 01000 01001
To compute d we need to isolate the least four bits of byte 1 with byteArray[1] & 0xf then make space for the bit from byte 2 with (byteArray[1] & 0xf) << 1, extract that bit with byteArray[1] >>> 7 and finally merge together the result.
int d = (byteArray[1] & 0xf) << 1 | byteArray[2] >>> 7;
int e = (byteArray[2] & 0x7c) >>> 2;
int f = (byteArray[2] & 0x3) << 3 | byteArray[3] >>> 5;
int g = byteArray[3] & 0x1f;
When you are comfortable with handling bits operations you may consider generalizing the function that extract the integers.
I made function int extract(byte[] bits, int[] sizes, int[] res), that given an array of bytes bits, an array of sizes sizes, where the even indices hold the size of the integers to extract in bits and the odd indices the number of integers to extract, and an output array res large enough to hold all the integers in output, extracts from bits all the integers expressed by sizes.
It returns the number of integers extracted.
For example the original problem can be solved as
int res[] = new int[8];
byte bits[] = new byte[]{0x56, 0x74, 0x25, 0x4b};
//Extract 3 integers of 4 bits and 4 integers of 5 bits
int ints = BitsExtractor.extract(bits, new int[]{4, 3, 5, 4}, res);
public class BitsExtractor
{
public static int extract(byte[] bits, int[] sizes, int[] res)
{
int currentByte = 0; //Index into the bits array
int intProduced = 0; //Number of ints produced so far
int bitsLeftInByte = 8; //How many bits left in the current byte
int howManyInts = 0; //Number of integers to extract
//Scan the sizes array two items at a time
for (int currentSize = 0; currentSize < sizes.length - 1; currentSize += 2)
{
//Size, in bits, of the integers to extract
int intSize = sizes[currentSize];
howManyInts += sizes[currentSize+1];
int temp = 0; //Temporary value of an integer
int sizeLeft = intSize; //How many bits left to extract
//Do until we have enough integer or we exhaust the bits array
while (intProduced < howManyInts && currentByte <= bits.length)
{
//How many bit we can extract from the current byte
int bitSize = Math.min(sizeLeft, bitsLeftInByte); //sizeLeft <= bitsLeftInByte ? sizeLeft : bitsLeftInByte;
//The value to mask out the number of bit extracted from
//The current byte (e.g. for 3 it is 7)
int byteMask = (1 << bitSize) - 1;
//Extract the new bits (Note that we extract starting from the
//RIGHT so we need to consider the bits left in the byte)
int newBits = (bits[currentByte] >>> (bitsLeftInByte - bitSize)) & byteMask;
//Create the new temporary value of the current integer by
//inserting the bits in the lowest positions
temp = temp << bitSize | newBits;
//"Remove" the bits processed from the byte
bitsLeftInByte -= bitSize;
//Is the byte has been exhausted, move to the next
if (bitsLeftInByte == 0)
{
bitsLeftInByte = 8;
currentByte++;
}
//"Remove" the bits processed from the size
sizeLeft -= bitSize;
//If we have extracted all the bits, save the integer
if (sizeLeft == 0)
{
res[intProduced++] = temp;
temp = 0;
sizeLeft = intSize;
}
}
}
return intProduced;
}
}
Well I did the first group , the second can be done in similar fashion
public static void main(String args[]) {
//an example 32 bits like your example
byte[] bytes = new byte[4];
bytes[0] = 31;//0001 1111
bytes[1] = 54;//0011 0110
bytes[2] = 67;
bytes[3] = 19;
//System.out.println(bytes[0]);
int x = 0;
int j = -1; // the byte number
int k = 0; // the bit number in that byte
int n = 0; // the place of the bit in the integer we are trying to read
for (int i = 0; i < 32; i++) {
if (i < 12) { //first group
if (i % 8 == 0) {
j++;
k = 0;
}
if (i % 4 == 0) {
x = 0;
n = 0;
}
byte bit = (byte) ((bytes[j] & (1 << (7 - k))) >> (7 - k));
System.out.println("j is :" + j + " k is :" + k + " " + bit);
x = x | bit << (3 - n);
if ((i + 1) % 4 == 0) {
System.out.println(x);
}
k++;
n++;
} else {
}
}
}
It's a bit tricky because you are trying to encode an integer on less than what java allocates (8 bits). So I had to take each bit and "construct" the int from them
To get each bit
byte bit = (byte) ((bytes[j] & (1 << (7 - k))) >> (7 - k));
this takes the byte we are at and does And operation. For example I want the 3rd bit of the 1st byte, I do
bytes[0] & 1 << (7 - 3)
but this gives me an integer encoded over 8 bits, so I still have to shift it to get that single bit with >> (7 - 3)
Then I just Or it with x (the int we are trying to decode). All while putting it at the right position with << (3 - n) . 3 because your integer is encoded over 4 bits
Try running the code and reading the output.
I am honestly not sure if this is the best way, but I believe it's at least faster than dealing with Strings
As homework, I'm implementing Karatsuba's algorithm and benchmarking it against a primary-school-style O(n^2) multiplication algorithm on large integers.
I guessed my only choice here was to bring the numbers to their byte array representations and then work them from there.
Well, I'm stuck here... when using the * operator, I don't know how would I detect/correct if the number overflows a byte multiplication or adds a carry. Any ideas?
public static BigInteger simpleMultiply(BigInteger x, BigInteger y){
//BigInteger result = x.multiply(y);
byte [] xByteArray = x.toByteArray();
byte [] yByteArray = y.toByteArray();
int resultSize = xByteArray.length*yByteArray.length;
byte [][] rowsAndColumns = new byte[resultSize][resultSize];
for (int i =0; i<xByteArray.length;i++)
for (int j=0; j<yByteArray.length;j++){
rowsAndColumns[i][j] = (byte )(xByteArray[i] * yByteArray[j]);
// how would I detect/handle carry or overflow here?
}
return null;
}
The result of a byte multiplication is 2 bytes. You have to use the low order byte as the result and the high order byte as the carry (overflow).
I would also advise you to be careful of the sign of your bytes. Since bytes in Java are signed, you'll have to either use only the low 7 bits of them or convert them to ints and correct the sign before multiplying them.
You'll want a loop like:
for (int i =0; i<xByteArray.length;i++)
for (int j=0; j<yByteArray.length;j++){
// convert bytes to ints
int xDigit = xByteArray[i], yDigit = yByteArray[j];
// convert signed to unsigned
if (xDigit < 0)
xDigit += 256;
if (yDigit < 0)
yDigit += 256;
// compute result of multiplication
int result = xDigit * yDigit;
// capture low order byte
rowsAndColumns[i][j] = (byte)(result & 0xFF);
// get overflow (high order byte)
int overflow = result >> 8;
// handle overflow here
// ...
}
The best way to avoid overflow is not to have it happen in the first place. Make all your calculations with a higher width numbers to avoid problems.
For example, lets say we have base 256 numbers and each digit is stored as a single unsigned byte.
d1 = (int) digits[i] //convert to a higher-width number
d2 = (int) digits[j]
product = d1*d2 //ints can handle up to around 2^32. Shouldn't overflow w/ 256*256
result = product % 256
carry = product / 256
You could be fancy and convert the divisions by powers of two into bit operations, but it isn't really necessary.
I am calculating the int equivalent of a given set of bits and storing that in memory. From there, I would like to determine all 1 value bits from the original bitmask. Example:
33 --> [1,6]
97 --> [1,6,7]
Ideas for an implementation in Java?
On BitSet
Use java.util.BitSet to store, well, a set of bits.
Here's how you can convert from an int to a BitSet, based on which bits in the int is set:
static BitSet fromInt(int num) {
BitSet bs = new BitSet();
for (int k = 0; k < Integer.SIZE; k++) {
if (((num >> k) & 1) == 1) {
bs.set(k);
}
}
return bs;
}
So now you can do the following:
System.out.println(fromInt(33)); // prints "{0, 5}"
System.out.println(fromInt(97)); // prints "{0, 5, 6}"
And just for completeness, here's the reverse transformation:
static int toInt(BitSet bs) {
int num = 0;
for (int k = -1; (k = bs.nextSetBit(k + 1)) != -1; ) {
num |= (1 << k);
}
return num;
}
So composing both together, we always get back the original number:
System.out.println(toInt(fromInt(33))); // prints "33"
System.out.println(toInt(fromInt(97))); // prints "97"
On 0-based indexing
Note that this uses 0-based indexing, which is the more commonly used indexing for bits (and most everything else in Java). This is also more correct. In the following, ^ denotes exponentiation:
33 = 2^0 + 2^5 = 1 + 32 97 = 2^0 + 2^5 + 2^6 = 1 + 32 + 64
33 -> {0, 5} 97 -> {0, 5, 6}
If you insist on using 1-based indexing, however, you can use bs.set(k+1); and (1 << (k-1)) in the above snippets. I would advise strongly against this recommendation, however.
Related questions
What does the ^ operator do in Java? -- it's actually not exponentiation
For bit fiddling, java.lang.Integer has some very helpful static methods. Try this code as a starting base for your problem:
public int[] extractBitNumbers(int value) {
// determine how many ones are in value
int bitCount = Integer.bitCount(value);
// allocate storage
int[] oneBits = new int[bitCount];
int putIndex = 0;
// loop until no more bits are set
while (value != 0) {
// find the number of the lowest set bit
int bitNo = Integer.numberOfTrailingZeros(value);
// store the bit number in array
oneBits[putIndex++] = bitNo+1;
// clear the bit we just processed from the value
value &= ~(1 << bitNo);
}
return oneBits;
}
I can show you C# implementation, Java should be very similar.
int value = 33;
int index = 1;
while (value > 0)
{
if ((value % 2) == 1)
Console.WriteLine(index);
index++;
value /= 2;
}
If you want to get an array like that you'll likely need to loop the number of bits you want to check & the integer with a bit shifted 1 for each step.
Something like (pseudo):
Init array
mask = 1
for (0 to BitCount):
if Integer & mask
array[] = pos
mask << 1
A bit-crunching variation would be something like:
int[] getBits(int value) {
int bitValue = 1;
int index = 1;
int[] bits = new int[33];
while (value >= bitValue)
{
bits[index++] = (value & bitValue);
bitValue << 1; // or: bitValue *= 2;
}
return bits;
}
Note that since the bits are indexed from 1 as you requested, bits[0] is left unused.