We Keep Coding

Which real use cases exist for arithmetic right bit shifting?

I stumbled upon a question that asks whether you ever had to use bit shifting in real projects. I have used bit shifts quite extensively in many projects, however, I never had to use arithmetic bit shifting, i.e., bit shifting where the left operand could be negative and the sign bit should be shifted in instead of zeros. For example, in Java, you would do arithmetic bit shifting with the >> operator (while >>> would perform a logical shift). After thinking a lot, I came to the conclusion that I have never used the >> with a possibly negative left operand.
As stated in this answer arithmetic shifting is even implementation defined in C++, so – in contrast to Java – there is not even a standardized operator in C++ for performing arithmetic shifting. The answer also states an interesting problem with shifting negative numbers that I was not even aware of:
+63 >> 1 = +31 (integral part of quotient E1/2E2)
00111111 >> 1 = 00011111
-63 >> 1 = -32
11000001 >> 1 = 11100000
So -63>>1 yields -32 which is obvious when looking at the bits, but maybe not what most programmers would anticipate on first sight. Even more surprising (but again obvious when looking at the bits) is that -1>>1 is -1, not 0.
So, what are concrete use cases for arithmetic right shifting of possibly negative values?

Perhaps the best known is the branchless absolute value:
int m = x >> 31;
int abs = x + m ^ m;
Which uses an arithmetic shift to copy the signbit to all bits. Most uses of arithmetic shift that I've encountered were of that form. Of course an arithmetic shift is not required for this, you could replace all occurrences of x >> 31 (where x is an int) by -(x >>> 31).
The value 31 comes from the size of int in bits, which is 32 by definition in Java. So shifting right by 31 shifts out all bits except the signbit, which (since it's an arithmetic shift) is copied to those 31 bits, leaving a copy of the signbit in every position.

It has come in handy for me before, in the creation of masks that were then used in '&' or '|' operators when manipulating bit fields, either for bitwise data packing or bitwise graphics.
I don't have a handy code sample, but I do recall using that technique many years ago in black-and-white graphics to zoom in (by extending a bit, either 1 or 0). For a 3x zoom, '0' would become '000' and '1' would become '111' without having to know the initial value of the bit. The bit to be expanded would be placed in the high order position, then an arithmetic right shift would extend it, regardless of whether it was 0 or 1. A logical shift, either left or right, always brings in zeros to fill vacated bit positions. In this case the sign bit was the key to the solution.

Here's an example of a function that will find the least power of two greater than or equal to the input. There are other solutions to this problem that are probably faster, namly any hardware oriented solution or just a series of right shifts and ORs. This solution uses arithmetic shift to perform a binary search.
unsigned ClosestPowerOfTwo(unsigned num) {
int mask = 0xFFFF0000;
mask = (num & mask) ? (mask << 8) : (mask >> 8);
mask = (num & mask) ? (mask << 4) : (mask >> 4);
mask = (num & mask) ? (mask << 2) : (mask >> 2);
mask = (num & mask) ? (mask << 1) : (mask >> 1);
mask = (num & mask) ? mask : (mask >> 1);
return (num & mask) ? -mask : -(mask << 1);
}

Indeed logical right shift is much more commonly used. However there are many operations that require an arithmetic shift (or are solved much more elegantly with an arithmetic shift)
Sign extension:
Most of the time you only deal with the available types in C and the compiler will automatically sign extend when casting/promoting a narrower type to a wider one (like short to int) so you may not notice it, but under the hood a left-then-right shift is used if the architecture doesn't have an instruction for sign extension. For "odd" number of bits you'll have to do the sign extension manually so this would be much more common. For example if a 10-bit pixel or ADC value is read into the top bits of a 16-bit register: value >> 6 will move the bits to the lower 10 bit positions and sign extend to preserve the value. If they're read into the low 10 bits with the top 6 bits being zero you'll use value << 6 >> 6 to sign extend the value to work with it
You also need signed extension when working with signed bit fields
struct bitfield {
int x: 15;
int y: 12;
int z: 5;
};
int f(bitfield b) {
return (b.x/8 + b.y/5) * b.z;
}
Demo on Godbolt. The shifts are generated by the compiler but usually you don't use bitfields (as they're not portable) and operate on raw integer values instead so you'll need to do arithmetic shifts yourself to extract the fields
Another example: sign-extend a pointer to make a canonical address in x86-64. This is used to store additional data in the pointer: char* pointer = (char*)((intptr_t)address << 16 >> 16). You can think of this as a 48-bit bitfield at the bottom
V8 engine's SMI optimization stores the value in the top 31 bits so it needs a right shift to restore the signed integer
Round signed division properly when converting to a multiplication, for example x/12 will be optimized to x*43691 >> 19 with some additional rounding. Of course you'll never do this in normal scalar code because the compiler already does this for you but sometimes you may need to vectorize the code or make some related libraries then you'll need to calculate the rounding yourself with arithmetic shift. You can see how compilers round the division results in the output assembly for bitfield above
Saturated shift or shifts larger than bit width, i.e. the value becomes zero when the shift count >= bit width
uint32_t lsh_saturated(uint32_t x, int32_t n) // returns 0 if n == 32
{
return (x << (n & 0x1F)) & ((n-32) >> 5);
}
uint32_t lsh(uint32_t x, int32_t n) // returns 0 if n >= 32
{
return (x << (n & 0x1F)) & ((n-32) >> 31);
}
Bit mask, useful in various cases like branchless selection (i.e. muxer). You can see lots of ways to conditionally do something on the famous bithacks page. Most of them are done by generating a mask of all ones or all zeros. The mask is usually calculated by propagating the sign bit of a subtraction like this (x - y) >> 31 (for 32-bit ints). Of course it can be changed to -(unsigned(x - y) >> 31) but that requires 2's complement and needs more operations. Here's the way to get the min and max of two integers without branching:
min = y + ((x - y) & ((x - y) >> (sizeof(int) * CHAR_BIT - 1)));
max = x - ((x - y) & ((x - y) >> (sizeof(int) * CHAR_BIT - 1)));
Another example is m = m & -((signed)(m - d) >> s); in Compute modulus division by (1 << s) - 1 in parallel without a division operator

I am not too sure what you mean. BUt i'm going to speculate that you want to use the bit shift as an arithmetic function.
One interesting thing i have seen is this property of binary numbers.
int n = 4;
int k = 1;
n = n << k; // is the same as n = n * 2^k
//now n = (4 * 2) i.e. 8
n = n >> k; // is the same as n = n / 2^k
//now n = (8 / 2) i.e. 4
hope that helps.
But yes you want to be careful of negative numbers
i would mask and then turn it back accordingly

In C when writing device drivers, bit shift operators are used extensively since bits are used as switches that need to be turned on and off. Bit shift allow one to easily and correctly target the right switch.
Many hashing and cryptographic functions make use of bit shift. Take a look at Mercenne Twister.
Lastly, it is sometimes useful to use bitfields to contain state information. Bit manipulation functions including bit shift are useful for these things.

Is java bit shifting circular?

I have this behavior using Java:
int b=16;
System.out.println(b<<30);
System.out.println(b<<31);
System.out.println(b<<32);
System.out.println(b<<33);
output:
0
0
16
32
Is java bit shift circular? IF not, why I get 0 when b<<30 and 16 when b<<32?

Bit shifting is not circular; for bit-shifting ints, Java only uses the 5 least-significant bits, so that (b << 0) is equivalent to (b << 32) (is equivalent to (b << 64), etc.). You can simply take the bit-shifting amount and take the remainder when dividing by 32.
Something similar occurs for bit-shifting longs, where Java only uses the 6 least-significant bits, so that (aLong << 0) is equivalent to (aLong << 64).
Section 15.19 of the JLS talks about this:
If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.
(emphasis mine)
(You can't bit-shift floats or doubles, and attempting to bit-shift a short or a byte would be subject the value to unary numeric promotion to an int anyway.)
You get 0 from 16 << 30, because the 1-bit from 16
00000000 00000000 00000000 00010000
gets shifted off the end of the int and gets discarded.
// Discarded - Result-----------------------------
(00000100) 00000000 00000000 00000000 00000000

No, it's not circular shift. It's normal left-shift. It's just that, for int type left side operand, Java uses just 5 lower order bits of the right operand for shifting. This is as per JLS §15.9:
If the promoted type of the left-hand operand is int, only the five
lowest-order bits of the right-hand operand are used as the shift
distance. It is as if the right-hand operand were subjected to a
bitwise logical AND operator & (§15.22.1) with the mask value 0x1f
(0b11111). The shift distance actually used is therefore always in the
range 0 to 31, inclusive
So, for 16 << 32, considering only 5 lower order bits of 32, the expression is equivalent to:
16 << 32 & 0x1f
which is equal to 16.

Shifting by Negative Numbers

I am pretty confused with this expression here. I am a Java programmer but I am not very well versed with bit manipulation.
I think I understand the below correctly:
Input : 1 << 10
Output: 0000000000000000000010000000000
For positive numbers, I think it is you move 1 by 10 bits.
The confusion is when I have the below:
int val = -10 (binary representation : 1111111111111111111111111110110 )
Input : 1 << val
Output: 0000000010000000000000000000000
That would be really great if someone can explain me the meaning of left shifting or right shifting by negative number.

<< (and other shift operators) only takes 5 least significant bits of its right operand for int, and 6 for long, because it makes no sense to shift int by more than 31.
In your case it's 0b10110 = 22.
Therefore 1 << (-10) is equivalent to 1 << 22.

From the JLS, section 15.19:
If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
In other words,
1 << -10
is equivalent to:
1 << (-10 & 0x1f)
... which is
1 << 22

What does this bit shift operation means in Java RegularEnumSet implementation?

In RegularEnumSet implementation, there is a code:
elements = -1L >>> -universe.lengh
It uses a Long type integer to implement efficient EnumSet.
What is notable is that the right-hand operand of >>> is a negative figure.
I have tested and found that
int i = -1;
i >>> -3
has equivalent effect with
int i = 1;
i << 3;
So why bother to write in this ambiguous form?

From the "Java Language Specification", section 15.19, about shifting "If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-and operand were subjected to a bitwise logical AND operator & with the mask value 0x3f. The shift distance actually used is therefore awlays 0 to 64, inclusive."
So, >>> -3 has equivalent effect as >>> 61, and -1L >>> -n equals -1L >>> (0x3f&-n)
Actually this seems to be the most effective way of producing a consecutive 1s in the lowest n-bits of a long integer.

First, note that the code you quoted uses -1L, not -1. Longs are unsigned, so -1L is equivalent to setting all the bits on the long, or getting the highest possible long number.
>>> -3 has equivalent effect with << 3
It doesn't. The unsigned right shift operator (>>>) shifts zeroes into the sign bit, while >> and << preserve the sign of the value being shifted, shifting 1 or 0 accordingly. That's why -1L >>> -3 is 7 (0111b) and 1L << 3 is 8 (1000b).

What is reduced modulo 32 or 64?

In the Core Java Volume1 book there is a caution that say:
CAUTION: The right-hand side argument of the shift operators is reduced modulo 32
(unless the left-hand side is a long, in which case the right-hand side is reduced modulo 64).
For example, the value of 1 << 35 is the same as 1 << 3 or 8.
What exactly does this mean? also why 1 become 8, instead of being 0 after 35 left shifting?
many thanks

Reduced modulo 32 means (at its base level) you keep subtracting 32 until you have a number between 0 and 31 inclusive.
In other words:
actualValue = givenValue % 32;
The reason it does this is because it makes little sense to shift a 32-bit value 32 bits to the left (or right) since it will always be zero (because you're shifting bits out on one side and shifting zeros in on the other side - doing that 32 times to a 32-bit value is going to result in zero no matter what you started with).
So for Java integers (32-bit), 31 is the sensible limit. For longs (64-bit), 63 is the sensible limit.
In the example you give, 1 << 35 has the shift value reduced from 35 to 3 (since 35 % 32 == 3) and 1 << 3 is 8:
Binary
0000 0001 (1 << 0) == 1
0000 0010 (1 << 1) == 2
0000 0100 (1 << 2) == 4
0000 1000 (1 << 3) == 8
||||
|||+--- 1
||+---- 2
|+----- 4
+------ 8

In many programming languages, shifting by more than the size of a numeric data type (32 bits for an int, 64 bits for a long) is undefined. On the other hand, Java defines it such that (n << d) is equivalent to (n << (d % 32)) where n is an int, and (n << d) is equivalent to (n << (d % 64)) where n is a long.
So, 1 << 35 is equivalent to 1 << (35 % 32), which equals 1 << 3 = 8.

More succinctly
a << b
is the same as
a << (b & 31)
for int types.
The difference is that -1 % 32 is -1 whereas -1 & 31 is 31 and 1 << -1 == 0x80000000
This behaviour is defined in JLS 15.19
If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & (§15.22.1) with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.