We Keep Coding

Don't understand how these bitwise operators operate on bytes and integers

I am working with some code that takes in a binary file as input. However, I am having trouble understanding the for loop in the code, as I don't understand what the bitwise operators do to IFD_Address, such as the |=, <<, and & 0xff. I think IFD_Address refers to a pointer in the binary file, but I am not sure. What is this piece of code trying to achieve?
byte[] IFD_Address_tmp = Arrays.copyOfRange(bytes, 4, 8);
int IFD_Address = 0;
int i = 0;
int shiftBy = 0;
for (shiftBy = 0; shiftBy < 32; shiftBy += 8) {
IFD_Address |= ((long) (IFD_Address_tmp[i] & 0xff)) << shiftBy;
i++;
}

This behavior is best understood in terms of moving bits around, not numbers. Bytes comprise eight bits, integers, 32 bits. The loop basically takes each byte in the array and places the corresponding bits in the integer IFD_Address in 8-bit chunks, from right (least significant) to left (most significant), like this:
About the bitwise operations:
& 0xff is required to capture the 8 bits into an integer;
<< shifts the bits to the left to select the appropriate place in IFD_Address;
|= sets the bits in IFD_Address.
See this tutorial for details.

Bit operations converting to an integer

I have some binary operations that are not working like I expect.
I have byte array with the first 2 bytes having these values : 0x5, and 0xE0.
I want to combine them into an integer value that should be 0x5E0.
I tried doing :
int val = (b[i]) << 8 | b[i+1];
but the value is coming out 0xFFFFFFEE0 and the first byte 0x5 is getting lost
I thought this would be easy? What am I doing wrong?

Try: int val = ((b[i] & 0xff) << 8) | (b[i + 1] & 0xff). Bytes are (unfortunately) signed in Java, so if the high bit is set, it gets sign-extended when converted to an integer.

The problem is that byte data type is signed. Therefore, b[i+1] gets sign-extended before performing the operation, becoming 0xFFFFFFE0. When it gets OR-ed with 0x0500 from b[i]<<8, the 0x0500 gets lost.
You can fix this by AND-ing with 0xFF before performing the operation:
public static int toInt16(byte high, byte low) {
int res = (high << 8);
res |= (low & 0xFF);
return res & 0xFFFF;
}
Demo.

Get n Least Significant Bits from an Int

This seems fairly straightforward, but I cant find an answer. If I have an int X, what is the best way to get N least significant bits from this int, in Java?

This should work for all non-negative N < 33 32:
x & ((1 << N) - 1)
It's worth elaborating on how this works for N == 31 and N == 32. For N == 31, we get 1 << N == Integer.MIN_VALUE. When you subtract 1 from that, Java silently wraps around to Integer.MAX_VALUE, which is exactly what you need. For N == 32, the 1 bit is shifted completely out, so 1 << N == 0; then (1 << N) - 1 == -1, which is all 32 bits set.
For N == 32, this unfortunately doesn't work because (thanks, #zstring!) the << operator only shifts by the right side mod 32. Instead, if you want to avoid testing for that case specially, you could use:
x & ((int)(1L << N) - 1)
By shifting a long, you get the full 32-bit shift, which, after casting back to an int, gets you 0. Subtracting 1 gives you -1 and x & -1 is just x for any int value x (and x is the value of the lower 32 bits of x).

Ted's approach is likely to be faster but here is another approach
x << -N >>> -N
This shift all the bit up and then down to chop off the top bits.
int i = -1;
System.out.println(Integer.toBinaryString(i));
i = i << -5 >>> -5;
System.out.println(Integer.toBinaryString(i));
prints
11111111111111111111111111111111
11111

You can also use a mask. If you use the & bitwise operator you can then remove whatever bit you would want to remove (say the highest x bits);
int mask = 0x7FFFFFFF //Example mask where you will remove the
// most significant bit
// (0x7 = 0111b and 0xF = 1111b).
int result = numberToProcess & mask; //And apply the mask with the &bitwise op.
The disadvantage to this is that you will need to make a mask for each bit, so perhaps this is better seen as another method of approach in general.

Is there a way to represent int value in byte over 127 without doing bitwise operations?

I am really short on time for doing the learning of bitwise operations.
I want to convert large integer(>127) values without doing '<<' or anything similar.
I need byte representation of integer values used to identify sequence numbers of packets in header sent across UDP. If there is no solution I will introduce two bytes..
Something like: 1, 1 ; 1,2 ; 1,3 ; packet lost ; 1,4 ; packet lost; 2,1 ,2,2
and then reset it upon reaching 127; 127
I can introduce third, but this is rather ugly.
It would be really useful to have black box that is part of java api doing all that byte conversion for me. Is there?
Thanks,

To pack an unsigned 8-bit value into a byte:
static byte toByte(int i) {
if ((i < 0) || (i > 255))
throw new IllegalArgumentException(String.valueOf(i));
return (byte) i;
}
To convert back:
static int toInt(byte b) {
return (b < 0) ? (b + 256) : b;
}
After reading your comments on other answers, it sounds like you might want something like this:
byte[] b = BigInteger.valueOf(counter).toByteArray();
and
long counter = new BigInteger(b).longValue();
Since the length of the array would vary as the counter grows, you'd need some way to indicate its length or delimit it. But this technique will convert any integer value to an array of bytes.

Is the problem that you want unsigned bytes, as in, numbers between 128 and 255 inclusive?
That's...tricky. The Java language won't let you directly treat bytes as unsigned...but with library support it gets a little easier. Guava provides an UnsignedBytes utility class for some of these needs. Addition, multiplication, and subtraction are all exactly the same on signed and unsigned bytes.
EDIT: Judging from your additional comments, you might be interested in Ints.toByteArray(int) and the like, which work on types between byte and BigInteger.

According to my understanding, you want to separate an int into 4 bytes. If so, then just copy paste this code:
int i = /* your int */
int[] b = { (i >> 24) & 0xff, (i >> 16) & 0xff, (i >> 8) & 0xff, i & 0xff };
Indices 0-3 are each of the 4 bytes in the int.

Converting Little Endian to Big Endian

All,
I have been practicing coding problems online. Currently I am working on a problem statement Problems where we need to convert Big Endian <-> little endian. But I am not able to jot down the steps considering the example given as:
123456789 converts to 365779719
The logic I am considering is :
1 > Get the integer value (Since I am on Windows x86, the input is Little endian)
2 > Generate the hex representation of the same.
3 > Reverse the representation and generate the big endian integer value
But I am obviously missing something here.
Can anyone please guide me. I am coding in Java 1.5

Since a great part of writing software is about reusing existing solutions, the first thing should always be a look into the documentation for your language/library.
reverse = Integer.reverseBytes(x);
I don't know how efficient this function is, but for toggling lots of numbers, a ByteBuffer should offer decent performance.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
...
int[] myArray = aFountOfIntegers();
ByteBuffer buffer = ByteBuffer.allocate(myArray.length*Integer.BYTES);
buffer.order(ByteOrder.LITTLE_ENDIAN);
for (int x:myArray) buffer.putInt(x);
buffer.order(ByteOrder.BIG_ENDIAN);
buffer.rewind();
int i=0;
for (int x:myArray) myArray[i++] = buffer.getInt(x);
As eversor pointed out in the comments, ByteBuffer.putInt() is an optional method, and may not be available on all Java implementations.
The DIY Approach
Stacker's answer is pretty neat, but it is possible to improve upon it.
reversed = (i&0xff)<<24 | (i&0xff00)<<8 | (i&0xff0000)>>8 | (i>>24)&0xff;
We can get rid of the parentheses by adapting the bitmasks. E. g., (a & 0xFF)<<8 is equivalent to a<<8 & 0xFF00. The rightmost parentheses were not necessary anyway.
reversed = i<<24 & 0xff000000 | i<<8 & 0xff0000 | i>>8 & 0xff00 | i>>24 & 0xff;
Since the left shift shifts in zero bits, the first mask is redundant. We can get rid of the rightmost mask by using the logical shift operator, which shifts in only zero bits.
reversed = i<<24 | i>>8 & 0xff00 | i<<8 & 0xff0000 | i>>>24;
Operator precedence here, the gritty details on shift operators are in the Java Language Specification

Check this out
int little2big(int i) {
return (i&0xff)<<24 | (i&0xff00)<<8 | (i&0xff0000)>>8 | (i>>24)&0xff;
}

The thing you need to realize is that endian swaps deal with the bytes that represent the integer. So the 4 byte number 27 looks like 0x0000001B. To convert that number, it needs to go to 0x1B000000... With your example, the hex representation of 123456789 is 0x075BCD15 which needs to go to 0x15CD5B07 or in decimal form 365779719.
The function Stacker posted is moving those bytes around by bit shifting them; more specifically, the statement i&0xff takes the lowest byte from i, the << 24 then moves it up 24 bits, so from positions 1-8 to 25-32. So on through each part of the expression.
For example code, take a look at this utility.

Java primitive wrapper classes support byte reversing since 1.5 using reverseBytes method.
Short.reverseBytes(short i)
Integer.reverseBytes(int i)
Long.reverseBytes(long i)
Just a contribution for those who are looking for this answer in 2018.

I think this can also help:
int littleToBig(int i)
{
int b0,b1,b2,b3;
b0 = (i&0x000000ff)>>0;
b1 = (i&0x0000ff00)>>8;
b2 = (i&0x00ff0000)>>16;
b3 = (i&0xff000000)>>24;
return ((b0<<24)|(b1<<16)|(b2<<8)|(b3<<0));
}

Just use the static function (reverseBytes(int i)) in java which is under Integer Wrapper class
Integer i=Integer.reverseBytes(123456789);
System.out.println(i);
output:
365779719

the following method reverses the order of bits in a byte value:
public static byte reverseBitOrder(byte b) {
int converted = 0x00;
converted ^= (b & 0b1000_0000) >> 7;
converted ^= (b & 0b0100_0000) >> 5;
converted ^= (b & 0b0010_0000) >> 3;
converted ^= (b & 0b0001_0000) >> 1;
converted ^= (b & 0b0000_1000) << 1;
converted ^= (b & 0b0000_0100) << 3;
converted ^= (b & 0b0000_0010) << 5;
converted ^= (b & 0b0000_0001) << 7;
return (byte) (converted & 0xFF);
}