We Keep Coding

Bitwise operator in java

When I wrote
int x = 11<<12+12>>11;
and print x complier shows 45056 which makes sense as it is like 11<<24>>11 then solving first from left to right it is 11 * 2^24 / 2^11.
Similarly for
int x = 1<<12+1>>12;
it shows 2^23. But when I wrote
int x = 1<<22+22>>1;
it is showing 2048 which is 2^11 rather than 2^41. Please correct me where I am wrong.

All the following are equal
int x = 1<<22+22>> 1;
int x = (1<<(22+22))>>1
int x = 1<<44>>1
int x = 1<<12>>1
int x = 4096 >> 1
int x = 2048
When you shift an int by N you are shifting by the low order 5 bits of N.
When you shift a long by N you are shifting by the low order 6 bits of N
As explained in the Java Language Specification - bitwise operator
Section 15.19
If the promoted type of the left-hand operand is int, then only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.

Weird behaviour of bit-shifting with byte in Java

As I was using bit-shifting on byte, I notice I was getting weird results when using unsigned right shift (>>>). With int, both right shift (signed:>> and unsigned:>>>) behave as expected:
int min1 = Integer.MIN_VALUE>>31; //min1 = -1
int min2 = Integer.MIN_VALUE>>>31; //min2 = 1
But when I do the same with byte, strange things happen with unsigned right shift:
byte b1 = Byte.MIN_VALUE; //b1 = -128
b1 >>= 7; //b1 = -1
byte b2 = Byte.MIN_VALUE; //b2 = -128
b2 >>>= 7; //b2 = -1; NOT 1!
b2 >>>= 8; //b2 = -1; NOT 0!
I figured that it could be that the compiler is converting the byte to int internally, but does not seem quite sufficient to explain that behaviour.
Why is bit-shifting behaving that way with byte in Java?

This happens exactly because byte is promoted to int prior performing bitwise operations. int -128 is presented as:
11111111 11111111 11111111 10000000
Thus, shifting right to 7 or 8 bits still leaves 7-th bit 1, so result is narrowed to negative byte value.
Compare:
System.out.println((byte) (b >>> 7)); // -1
System.out.println((byte) ((b & 0xFF) >>> 7)); // 1
By b & 0xFF, all highest bits are cleared prior shift, so result is produced as expected.

Shift operators for byte, short and char are always done on int.
Therefore, the value really being shifted is the int value -128, which looks like this
int b = 0b11111111_11111111_11111111_10000000;
When you do b2 >>= 7; what you are really doing is shifting the above value 7 places to the right, then casting back to a byte by only considering the last 8 bits.
After shifting 7 places to the right we get
0b11111111_11111111_11111111_11111111;
When we convert this back to a byte, we get just 11111111, which is -1, because the byte type is signed.
If you want to get the answer 1 you could shift 31 places without sign extension.
byte b2 = Byte.MIN_VALUE; //b2 = -128
b2 >>>= 31;
System.out.println(b2); // 1

Refer to JLS 15.19 Shift Operators:
Unary numeric promotion (§5.6.1) is performed on each operand separately.
and in 5.6.1 Unary Numeric Promotion :
if the operand is of compile-time type byte, short, or char, it is promoted to a value of type int by a widening primitive conversion
So, your byte operands are promoted to int before shifting. The value -128 is 11111111111111111111111110000000 .
After the shifting 7 or 8 times, the lowest 8 bits are all 1s, which when assigning to a byte, a narrowing primitive conversion occurs. Refer to JLS 5.1.3 Narrowing Primitive Conversion :
A narrowing conversion of a signed integer to an integral type T simply discards all but the n lowest order bits, where n is the number of bits used to represent type T.

Understanding unsigned right shift

The JLS 15.19 describes the formula for >>> operator.
The value of n >>> s is n right-shifted s bit positions with
zero-extension, where:
If n is positive, then the result is the same as that of n >> s.
If n is negative and the type of the left-hand operand is int, then
the result is equal to that of the expression (n >> s) + (2 << ~s).
If n is negative and the type of the left-hand operand is long, then
the result is equal to that of the expression (n >> s) + (2L << ~s).
Why does n >>> s = (n >> s) + (2 << ~s), where ~s = 31 - s for int and ~s = 63 - s for long?

If n is negative it means that the sign bit is set.
>>> s means shift s places to the right introducing zeros into the vacated slots.
>> s means shift s places to the right introducing copies of the sign bit into the vacated slots.
E.g.
10111110000011111000001111100000 >>> 3 == 00010111110000011111000001111100
10111110000011111000001111100000 >> 3 == 11110111110000011111000001111100
Obviously if n is not negative, n >> s and n >>> s are the same. If n is negative, the difference will consist of s ones at the left followed by all zeros.
In other words:
(n >>> s) + X == n >> s (*)
where X consists of s ones followed by 32 - s zeros.
Because there are 32 - s zeros in X, the right-most one in X occurs in the position of the one in 1 << (32 - s), which is equal to 2 << (31 - s), which is the same as 2 << ~s (because ~s == -1 - s and shift amounts work modulo 32 for ints).
Now what happens when you add 2 << ~s to X? You get zero! Let's demonstrate this in the case s == 7. Notice that the the carry disappears off the left.
11111110000000000000000000000000
+ 00000010000000000000000000000000
________________________________
00000000000000000000000000000000
It follows that -X == 2 << ~s. Therefore adding -X to both sides of (*) we get
n >>> s == (n >> s) + (2 << ~s)
For long it's exactly the same, except that shift amounts are done modulo 64, because longs have 64 bits.

Here is some additional context that will help you understand pbabcdefp's answer if you don't already know the basics he assumes:
To understand bitwise operators you must think about numbers as strings of binary digits, eg. 20 = 00010100 and -4 = 11111100 (For the sake of clarity and not having to write so many digits I will be writing all binary numbers as bytes; ints are the same but four times as long). If you are unfamiliar with binary and binary operations, you can read more here. Note how the first digit is special: It makes numbers negative, as if it had a place value (remember elementary math, the ones/tens/hundreds places?) of the most negative number possible, so Byte.MIN_VALUE = -128 = 1000000, and setting any other bit to 1 always increases the number. To easily read a negative number such as 11110011, know that -1 = 11111111, then read the 0s as if they were 1s in a positive number, then that number is how far away you are from -1. So 11110011 = -1 - 00001100 = -1 - 12 = -13.
Also understand that ~s is bitwise NOT: It takes all the digits and flips them, this is actually equivalent to ~s = -1 - s. Eg ~5 (00000101) is -6 (11111010). Observe how my suggested method for reading negative binary numbers is simply a trick to be able to read the bitwise NOT of the number rather than the number itself, which is easier for negative numbers close to zero because those numbers have fewer 0s than 1s.

Bitwise shift to get max unsigned value

I need to generate max value for unsigned 64 bit long in java, using bitwise shift. So here is my function:
public static final boolean isMaxLimitMiss(int bitsCount, BigDecimal value, int signum) {
if (signum == 0) {
BigDecimal val = new BigDecimal(Long.valueOf((1 << (bitsCount)) - 1));
return Long.compareUnsigned(value.longValue(), val.longValue()) > 0 ? true : false;
}
return value.compareTo(new BigDecimal((1L << (bitsCount - 1)) - 1)) > 0 ? true : false;
}
It just checks that value is in range for max value that bitsCount provides.
It works fine for signed data.
In case 64 bit, and for unsigned (Yes, there are no unsigned long in java. I trying to simulate it) - it is incorrect. ((1 << (bitsCount)) - 1 evaluates to 0
How can I pass value = 2^64 - 1 into BigDecimal constructor?

One first thing:
(1 << (bitsCount)) - 1
This is a mistake because you are only performing integer arithmetic. You needed:
(1L << (bitsCount)) - 1
But anyway I assume bitsCount is 64. In Java you cannot shift by a number larger than 63 for a long and 31 for an int (JLS 15.19):
If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.
This is why you get 0, because the expression is reduced to (1 << 0) - 1.
You need to compute your max value a different way, for example:
-1L >>> (64 - bitsCount)
or:
BigInteger.ONE.shiftLeft(bitsCount).subtract(BigInteger.ONE)
If you are testing a BigDecimal it would be safer to not use long at all here.

Why does Java mask shift operands with 0x1F?

In Java:
(0xFFFFFFFF << 1) = 0xFFFFFFFE = 0b1111111111111110
: : :
(0xFFFFFFFF << 30) = 0xE0000000 = 0b1110000000000000
(0xFFFFFFFF << 30) = 0xC0000000 = 0b1100000000000000
(0xFFFFFFFF << 31) = 0x80000000 = 0b1000000000000000
However:
(0xFFFFFFFF << 32) = 0xFFFFFFFF = 0b1111111111111111
Logically this makes no sense, but what I believe to be happening is Java performing an operation similar to:
a << (b % Integer.SIZE) [edit, apparently:] a << (b & 0x1F)
This applies to >> and >>>, too.
Obviously shifting by >= 32 (in the case of an Integer) removes all data from the data-type, but there are times when this is useful. For example:
int value = 0x3F43F466; // any value
int shift = 17; // any value >= 0
int carry = value & (-1 << (Integer.SIZE - shift));
if (carry > 0)
; // code...
Of course this can be fixed, but finding these bugs can be quite time consuming (I just spent hours tracking a similar one down).
So, my question: Is there reason for not returning the logical value when shifting all bits out?
UPDATE:
I tried this in C99, using the following:
#include<stdio.h>
main()
{
int i, val;
for (i = 0; i <=36; i++) {
val = (-1 << i);
printf("%d :\t%d\n", i, val);
}
}
I found that it behaves the same as Java, masking i & 0x1F, whereas it provides a warning at compilation when given a constant value:
warning: left shift count >= width of type

Sure, there is: it's how most processors (specifically including x86) implement bit shifting, and to do what you want -- to check if the shift is greater than 32, and if so, return zero -- requires a branch, which can be expensive on modern CPUs. It's not just "annoying," it can slow things down by orders of magnitude.
In short, doing what you want would add significant overhead to an operation that is expected to be blazing fast by high-performance code.
For reference, the logic isn't exactly the same as %, it's a mask. See JLS 15.19 for details:
If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.

JLS 15.19 If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive
to put it simply 0xFFFFFFFF << 32 is equivalnt to 0xFFFFFFFF << (32 & 0x1f) is equivalent to 0xFFFFFFFF << 0