We Keep Coding

this is code for Rearrange array in alternating positive & negative items with O(1) extra space can you please explain what is & 0x01 elaborately? [duplicate]

I was going through a piece of code in the Apache commons library and was wondering what these conditions do exactly.
public static byte[] decodeHex(final char[] data) throws DecoderException {
final int len = data.length;
if ((len & 0x01) != 0) { // what does this condition do
throw new DecoderException("Odd number of characters.");
}
final byte[] out = new byte[len >> 1];
// two characters form the hex value.
for (int i = 0, j = 0; j < len; i++) {
int f = toDigit(data[j], j) << 4;
j++;
f = f | toDigit(data[j], j);
j++;
out[i] = (byte) (f & 0xFF); // what is happening here.
}
return out;
}
thanks in advance.

This checks if the last digit in the binary writing of len is a 1.
xxxxxxxy
& 00000001
gives 1 if y is 1, 0 if y is 0, ignoring the other digits.
If y is 1, the length of the char array is odd, which shouldn't happen in this hex writing, hence the exception.
Another solution would have been
if (len%2 != 0) {
which would have been clearer in my opinion. I doubt the slight performance increase just before a loop really matters.

It's a 1337 (high performance) way of coding:
if (len % 2 == 1)
i.e. is len odd. It works because the binary representation of every odd integer has its least significant (ie last) bit set. Performaning a bitwise AND with 1 masks all other bits, leaving a result of either 1 if it's odd or 0 if even.
It's a carryover from C, where you can code simply:
if (len & 1)

This line checks if len is an odd number or not.
If len isn't odd, len & 1 will be equal to 0. (1 and 0x01 are the same value, 0x01 is just the hexadecimal notation)

Reading bytes in a file :java.lang.NegativeArraySizeException

I have gone through the similar posts but none of them really answered my question. Hence I post my question in a separate thread.
I need to skip some bytes in the file which I am reading in the byte array.I am trying to achieve this through code below
1. byte [] readBytesToSKip = null;
2. readBytesToSKip = new byte[(int)bytesToSkip];
3. bytesReadToSkip = System.in.read(readBytesToSKip) ;
4. if(bytesReadToSkip > 0)
5. {
6. baos_.write(readBytesToSKip, 0, bytesReadToSkip);
7. }
But I get a NegativeArraySizeException at line 2 where the size exceeds Integer.MAX_VALUE. I am not sure how to achieve this otherwise.
bytesToSkip is long as I calculate in function below:
public static long bytesToLong1(byte[] bytes) {
long value = 0;
for (int i = 0; i < bytes.length; i++)
{
value = (value << 8) + (bytes[i] & 0xff);
}
return value;
}

Starting from your posted bytes->long function
public static long bytesToLong1(byte[] bytes) {
long value = 0;
for (int i = 0; i < bytes.length; i++)
{
value = (value << 8) + (bytes[i] & 0xff);
}
return value;
}
Whenever bytes.length==8 and the first byte is >= 128 in absolute value (in Java specific: it's a negative byte value), you'll end having a negative value for your long.
Example:
byte vals[]={0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
System.out.println(bytesToLong1(vals));
will result it -1 being printed. Which will convert quite fine to an int with a -1 value, but will still be improper for the length of an array.

Bit shift operations on a byte array in Java

How do I shift a byte array n positions to the right? For instance shifting a 16 byte array right 29 positions? I read somewhere it can be done using a long? Would using a long work like this:
Long k1 = byte array from 0 to 7
Long k2 = byte array from 8 to 15
Then right rotating these two longs using Long.rotateRight(Long x, number of rotations).How would the two longs be joined back into a byte array?

I believe you can do this using java.math.BigInteger which supports shifts on arbitrarily large numbers. This has advantage of simplicity, but disadvantage of not padding into original byte array size, i.e. input could be 16 bytes but output might only be 10 etc, requiring additional logic.
BigInteger approach
byte [] array = new byte[]{0x7F,0x11,0x22,0x33,0x44,0x55,0x66,0x77};
// create from array
BigInteger bigInt = new BigInteger(array);
// shift
BigInteger shiftInt = bigInt.shiftRight(4);
// back to array
byte [] shifted = shiftInt.toByteArray();
// print it as hex
for (byte b : shifted) {
System.out.print(String.format("%x", b));
}
Output
7f1122334455667 <== shifted 4 to the right. Looks OK
Long manipulation
I don't know why you'd want to do this as rotateRight() as this makes life more difficult, you have to blank at the bits that appear at the left hand side in K1 etc. You'd be better with using shift IMO as describe below. I've used a shift of 20 as divisible by 4 so easier to see the nibbles move in the output.
1) Use ByteBuffer to form two longs from 16 byte array
byte[] array = { 0x00, 0x00, 0x11, 0x11, 0x22, 0x22, 0x33, 0x33, 0x44, 0x44, 0x55, 0x55, 0x66, 0x66, 0x77, 0x77 };
ByteBuffer buffer = ByteBuffer.wrap(array);
long k1 = buffer.getLong();
long k2 = buffer.getLong();
2) Shift each long n bits to the right
int n = 20;
long k1Shift = k1 >> n;
long k2Shift = k2 >> n;
System.out.println(String.format("%016x => %016x", k1, k1Shift));
System.out.println(String.format("%016x => %016x", k2, k2Shift));
0000111122223333 => 0000000001111222
4444555566667777 => 0000044445555666
Determine bits from k1 that "got pushed off the edge"
long k1CarryBits = (k1 << (64 - n));
System.out.println(String.format("%016x => %016x", k1, k1CarryBits));
0000111122223333 => 2333300000000000
Join the K1 carry bits onto K2 on right hand side
long k2WithCarray = k2Shift | k1CarryBits;
System.out.println(String.format("%016x => %016x", k2Shift, k2WithCarray));
0000044445555666 => 2333344445555666
Write the two longs back into a ByteBuffer and extract as a byte array
buffer.position(0);
buffer.putLong(k1Shift);
buffer.putLong(k2WithCarray);
for (byte each : buffer.array()) {
System.out.print(Long.toHexString(each));
}
000011112222333344445555666

Here is what I came up with to shift a byte array by some arbitrary number of bits left:
/**
* Shifts input byte array len bits left.This method will alter the input byte array.
*/
public static byte[] shiftLeft(byte[] data, int len) {
int word_size = (len / 8) + 1;
int shift = len % 8;
byte carry_mask = (byte) ((1 << shift) - 1);
int offset = word_size - 1;
for (int i = 0; i < data.length; i++) {
int src_index = i+offset;
if (src_index >= data.length) {
data[i] = 0;
} else {
byte src = data[src_index];
byte dst = (byte) (src << shift);
if (src_index+1 < data.length) {
dst |= data[src_index+1] >>> (8-shift) & carry_mask;
}
data[i] = dst;
}
}
return data;
}

1. Manually implemented
Here are left and right shift implementation without using BigInteger (ie. without creating a copy of the input array) and with unsigned right shift (BigInteger only supports arithmetic shifts of course)
Left Shift <<
/**
* Left shift of whole byte array by shiftBitCount bits.
* This method will alter the input byte array.
*/
static byte[] shiftLeft(byte[] byteArray, int shiftBitCount) {
final int shiftMod = shiftBitCount % 8;
final byte carryMask = (byte) ((1 << shiftMod) - 1);
final int offsetBytes = (shiftBitCount / 8);
int sourceIndex;
for (int i = 0; i < byteArray.length; i++) {
sourceIndex = i + offsetBytes;
if (sourceIndex >= byteArray.length) {
byteArray[i] = 0;
} else {
byte src = byteArray[sourceIndex];
byte dst = (byte) (src << shiftMod);
if (sourceIndex + 1 < byteArray.length) {
dst |= byteArray[sourceIndex + 1] >>> (8 - shiftMod) & carryMask;
}
byteArray[i] = dst;
}
}
return byteArray;
}
Unsigned Right Shift >>>
/**
* Unsigned/logical right shift of whole byte array by shiftBitCount bits.
* This method will alter the input byte array.
*/
static byte[] shiftRight(byte[] byteArray, int shiftBitCount) {
final int shiftMod = shiftBitCount % 8;
final byte carryMask = (byte) (0xFF << (8 - shiftMod));
final int offsetBytes = (shiftBitCount / 8);
int sourceIndex;
for (int i = byteArray.length - 1; i >= 0; i--) {
sourceIndex = i - offsetBytes;
if (sourceIndex < 0) {
byteArray[i] = 0;
} else {
byte src = byteArray[sourceIndex];
byte dst = (byte) ((0xff & src) >>> shiftMod);
if (sourceIndex - 1 >= 0) {
dst |= byteArray[sourceIndex - 1] << (8 - shiftMod) & carryMask;
}
byteArray[i] = dst;
}
}
return byteArray;
}
Used in this class by this Project.
2. Using BigInteger
Be aware that BigInteger internally converts the byte array into an int[] array so this may not be the most optimized solution:
Arithmetic Left Shift <<:
byte[] result = new BigInteger(byteArray).shiftLeft(3).toByteArray();
Arithmetic Right Shift >>:
byte[] result = new BigInteger(byteArray).shiftRight(2).toByteArray();
3. External Library
Using the Bytes java library*:
Add to pom.xml:
<dependency>
<groupId>at.favre.lib</groupId>
<artifactId>bytes</artifactId>
<version>{latest-version}</version>
</dependency>
Code example:
Bytes b = Bytes.wrap(someByteArray);
b.leftShift(3);
b.rightShift(3);
byte[] result = b.array();
*Full Disclaimer: I am the developer.

The is an old post, but I want to update Adam's answer.
The long solution works with a few tweak.
In order to rotate, use >>> instead of >>, because >> will pad with significant bit, changing the original value.
second, the printbyte function seems to miss leading 00 when it prints.
use this instead.
private String getHexString(byte[] b) {
StringBuilder result = new StringBuilder();
for (int i = 0; i < b.length; i++)
result.append(Integer.toString((b[i] & 0xff) + 0x100, 16)
.substring(1));
return result.toString();
}

Algorithm to convert a String of decimal digits to BCD

I am looking a way to convert a string to BCD equivalent. I use Java, but it is not a question of the language indeed. I am trying to understand step by step how to convert a string to BCD.
For example, suppose I have the following string;
"0200" (This string has four ASCII characters, if we were in java this string had been contained in a byte[4] where byte[0] = 48, byte[1] = 50, byte[2] = 48 and byte[3] = 48)
In BCD (according this page: http://es.wikipedia.org/wiki/Decimal_codificado_en_binario):
0 = 0000
2 = 0010
0 = 0000
0 = 0000
Ok, I think the conversion is correct but I have to save this in a byte[2]. What Should I have to do? After, I have to read the BCD and convert it to the original string "0200" but first I have to resolve String to BCD.

Find a utility class to do this for you. Surely someone out there has written a BCD conversion utility for Java.
Here you go. I Googled "BCD Java" and got this as the first result. Copying code here for future reference.
public class BCD {
/*
* long number to bcd byte array e.g. 123 --> (0000) 0001 0010 0011
* e.g. 12 ---> 0001 0010
*/
public static byte[] DecToBCDArray(long num) {
int digits = 0;
long temp = num;
while (temp != 0) {
digits++;
temp /= 10;
}
int byteLen = digits % 2 == 0 ? digits / 2 : (digits + 1) / 2;
boolean isOdd = digits % 2 != 0;
byte bcd[] = new byte[byteLen];
for (int i = 0; i < digits; i++) {
byte tmp = (byte) (num % 10);
if (i == digits - 1 && isOdd)
bcd[i / 2] = tmp;
else if (i % 2 == 0)
bcd[i / 2] = tmp;
else {
byte foo = (byte) (tmp << 4);
bcd[i / 2] |= foo;
}
num /= 10;
}
for (int i = 0; i < byteLen / 2; i++) {
byte tmp = bcd[i];
bcd[i] = bcd[byteLen - i - 1];
bcd[byteLen - i - 1] = tmp;
}
return bcd;
}
public static String BCDtoString(byte bcd) {
StringBuffer sb = new StringBuffer();
byte high = (byte) (bcd & 0xf0);
high >>>= (byte) 4;
high = (byte) (high & 0x0f);
byte low = (byte) (bcd & 0x0f);
sb.append(high);
sb.append(low);
return sb.toString();
}
public static String BCDtoString(byte[] bcd) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < bcd.length; i++) {
sb.append(BCDtoString(bcd[i]));
}
return sb.toString();
}
}
There's also this question: Java code or lib to decode a binary-coded decimal (BCD) from a String.

The first step would be to parse the string into an int so that you have the numeric value of it. Then, get the individual digits using division and modulus, and pack each pair of digits into a byte using shift and add (or shift and or).
Alternatively, you could parse each character of the string into an int individually, and avoid using division and modulus to get the numbers, but I would prefer to parse the entire string up front so that you discover right away if the string is invalid. (If you get a NumberFormatException, or if the value is less than 0 or greater than 9999 then it is invalid.)
Finally, once you have assembled the two individual bytes, you can put them into the byte[2].

You can use following:
//Convert BCD String to byte array
public static byte[] String2Bcd(java.lang.String bcdString) {
byte[] binBcd = new byte[bcdString.length() / 2];
for (int i = 0; i < binBcd.length; i++) {
String sByte = bcdString.substring(i*2, i*2+2);
binBcd[i] = Byte.parseByte(sByte, 16);
}
return binBcd;
}

You can try the following code:
public static byte[] hex2Bytes(String str) {
byte[] b = new byte[str.length() / 2];
int j = 0;
for (int i = 0; i < b.length; i++) {
char c0 = str.charAt(j++);
char c1 = str.charAt(j++);
b[i] = ((byte) (parse(c0) << 4 | parse(c1)));
}
return b;
}

Convert each character of a string in bits

String message= "10";
byte[] bytes = message.getBytes();
for (int n = 0; n < bytes.length; n++) {
byte b = bytes[n];
for (int i = 0; i < 8; i++) {//do something for each bit in my byte
boolean bit = ((b >> (7 - i) & 1) == 1);
}
}
My problem here is that it takes 1 and 0 as their ASCII values, 49 and 48, instead of 1 and 0 as binary(00000001 and 00000000). How can I make my program treat each character from my string as a binary sequence of 8 bits?
Basicly, I want to treat each bit of my number as a byte. I do that like this byte b = bytes[n]; but the program treats it as the ASCII value.
I could assign the number to an int, but then, I can't assign the bits to a byte.

It's a bit messy, but the first thing that comes to mind is to first, split your message up into char values, using the toCharArray() method. Next, use the Character.getNumericValue() method to return the int, and finally Integer.toBinaryString.
Example
String message = "123456";
for(char c : message.toCharArray())
{
int numVal = Character.getNumericValue(c);
String binaryString = Integer.toBinaryString(numVal);
for(char bit : binaryString)
{
// Do something with your bits.
}
}

String msg = "1234";
for(int i=0 ; i<msg.length() ; i++ ){
String bits = Integer.toBinaryString(Integer.parseInt(msg.substring(i, i+1)));
for(int j=0;j<8-bits.length();j++)
bits = "0"+bits;
}
Now bits is a string of length 8.
1
00000001
10
00000010
11
00000011
100
00000100
You can use getBytes() on the String

Use Java's parseInt(String s, int radix):
String message= "10";
int myInt = Integer.parseInt(message, 2); //because we are parsing it as base 2
At that point you have the correct sequence of bits, and you can do your bit-shifting.
boolean[] bits = new boolean[message.length()];
System.out.println("Parsed bits: ");
for (int i = message.length()-1; i >=0 ; i--) {
bits[i] = (myInt & (1 << i)) != 0;
System.out.print(bits[i] ? "1":"0");
}
System.out.println();
You could make it bytes if you really want to, but booleans are a better representation of bits...