I've got a long variable and I need to reverse its byte order. For example: B1, B2, ... , B8 I should return a long that consists of B8, B7, ..., B1. How can I do it by using bitwise operations?
you can use Long.reverseBytes(long)
Or for more methods which include bitwise operations, you can refer to this stack overflow question
Heres another method you may like, I'd still recommend the above but it's better than bitwise where you can easily make mistakes.
Bytebuffer
byte[] bytes = ByteBuffer.allocate(8).putLong(someLong).array();
for (int left = 0, right = bytes.length - 1; left < right; ++left, --right) {
byte temp = bytes[left];
bytes[left] = bytes[right];
bytes[right] = temp;
}
I am trying to steer you away from bitwise solutions because they are cumbersome and very easy to mess up if you do not know what you are doing... But bitwise would look like this:
byte[] bytes = new byte[8];
// set the byte array from smallest to largest byte
for(int i = 0; i < 8; ++i) {
byte[i] = (your_long >> i*8) & 0xFF;
}
// build the new long from largest to smallest byte (reversed)
long l = ((buf[0] & 0xFFL) << 56) |
((buf[1] & 0xFFL) << 48) |
((buf[2] & 0xFFL) << 40) |
((buf[3] & 0xFFL) << 32) |
((buf[4] & 0xFFL) << 24) |
((buf[5] & 0xFFL) << 16) |
((buf[6] & 0xFFL) << 8) |
((buf[7] & 0xFFL) << 0) ;
You might want to use Long.reverseBytes instead of using bitwise operations. See the Java Reference for details.
Otherwise, you could have a look at the JDK sources (src.zip in your JDK folder) in Long.java but mind the copyright by Oracle.
Here's an old trick that you can use to endian swap a register:
static long swapblock(long a, long mask, int shift) {
long b1 = a & mask; // extract block
long b2 = a ^ b1; // extract remaining bits
return (b1 << shift) |
((b2 >> shift) & mask); // mask again to clear sign extension
}
static long endianswap(long a) {
a = swapblock(a, 0x00000000ffffffffL, 32);
a = swapblock(a, 0x0000ffff0000ffffL, 16);
a = swapblock(a, 0x00ff00ff00ff00ffL, 8);
return a;
}
The idea is to progressively swap sub blocks until you reach the desired level you want to stop at. By adding swaps of sizes 4, 2, and 1, you can change this into a bit mirror function.
There is only one tricky bit due to lack of unsigned types in java. You need to mask out high order bits when shifting right, because the sign bit is replicated by the shift amount, filling the high order bits with ones (0x8000000000000000 >> 8 is 0xFF80000000000000).
long reverse(long x){
x = (x >> 32) | (x << 32); // step 1
x = ((x & 0xffff0000ffff0000 ) >> 16)
| ((x & 0x0000ffff0000ffff ) << 16); // step 2
x = ((x & 0xff00ff00ff00ff00 ) >> 8)
| ((x & 0x00ff00ff00ff00ff ) << 8); // step 3
return x;
}
If we assume that bitwise operator works in O(1) time, reverse function works in O(lg(number of bits) ) time.
Explanation
Step 0 : B1 B2 B3 B4 B5 B6 B7 B8
Step 1 : B5 B6 B7 B8 B1 B2 B3 B4
Step 2 : B7 B8 B5 B6 B3 B4 B1 B2
Step 3 : B8 B7 B6 B5 B4 B3 B2 B1
Plain answer with loops:
public static long byteReverse(long a) {
long result = 0;
for(int i = 0; i < 8; i++){
// grab the byte in the ith place
long x = (a >> (i*8)) & (0b11111111);
result <<= 8;
result |= x;
}
return result;
}
bitwise only:
public static long byteReverse(long a) {
a = (a << 32) | (a >>> 32);
a = ((a & 0xffff0000ffff0000L) >>> 16) | ((a & 0x0000ffff0000ffffL) << 16);
a = ((a & 0x00ff00ff00ff00ffL) << 8) | ((a & 0xff00ff00ff00ff00L) >>> 8);
return a;
}
Related
I'm using Kryo to deserialize a class originally serialized in Spark. Kryo writes all of its primitives in BigEndian format, but when I try to deserialize the values on another machine, the value is being returned as if it were LittleEndian.
Underlying method in Kryo:
public int readInt () throws KryoException {
require(4); // Does a basic positionality check that passes in this case
byte[] buffer = this.buffer;
int p = this.position;
this.position = p + 4;
return buffer[p] & 0xFF //
| (buffer[p + 1] & 0xFF) << 8 //
| (buffer[p + 2] & 0xFF) << 16 //
| (buffer[p + 3] & 0xFF) << 24;
}
This returns the value 0x70000000. But when my program (in Scala) uses Kryo's readByte method:
public byte readByte () throws KryoException {
if (position == limit) require(1);
return buffer[position++];
}
and reads the bytes individually, like this:
val a = input.readByte()
val b = input.readByte()
val c = input.readByte()
val d = input.readByte()
val x = (a & 0xFF) << 24 | (b & 0xFF) << 16 | (c & 0xFF) << 8 | d & 0xFF
Then I get 0x70 for x. I don't understand what's happening here. Is it some kind of conversion issue between Scala and Java, or something to do with Kryo and the underling byte array?
The code you wrote:
val a = input.readByte()
val b = input.readByte()
val c = input.readByte()
val d = input.readByte()
val x = (a & 0xFF) << 24 | (b & 0xFF) << 16 | (c & 0xFF) << 8 | d & 0xFF
is converting bytes to int in the wrong way. If you closely inspect the readInt() method you'll see that you've switched the order.
val x = (a & 0xFF) | (b & 0xFF) << 8 | (c & 0xFF) << 16 | d & 0xFF << 24;
would be the correct way to write this.
I know there are many similar questions in here, but I have some weird case. What I want to do is to convert a byte[4] to an int.
Here is the conversion from int to byte:
int data24bit = 51;
for (int i = 0; i < 3; i++) {
data8bit = (byte)(data24bit & 0x0000FF);
data24bit = data24bit >> 8;
byte_file.put(data8bit);
}
So far is clear enough. After that I want to read this 4 bytes to get the 51 back. I tried to do it by different ways:
Reading 4 bytes:
byte[] bytes = new byte[4];
for (int i = 0; i < 3; i++) {
byte b = dis.readByte();
bytes[i] = b;
}
// bytes[3] = (byte)(0x000000);
Convert bytes to int:
int value = 0;
value = ((0xFF & bytes[0]) << 24) | ((0xFF & bytes[1]) << 16) |
((0xFF & bytes[2]) << 8) | (0xFF & bytes[3]);
or
value = ByteBuffer.wrap(bytes).getInt();
or
value = new BigInteger(bytes).intValue();
I always get 855638016 as result where 51 is expected.
When I debug the code and look into the byte array I can see the following content: [51, 0, 0, 0].
What am I doing wrong?
The problem is that you're writing the bytes in little-endian (least significant byte first), but read it back assuming big-endian.
After writing it out, your byte array looks like this:
[51, 0, 0, 0]
Then you're trying to convert that back into an integer, like in this example from your post:
value = ((0xFF & bytes[0]) << 24)
| ((0xFF & bytes[1]) << 16)
| ((0xFF & bytes[2]) << 8)
| (0xFF & bytes[3]);
If you fill in the actual values, that calculation is basically this:
value = 51 * 256 * 256 * 256
+ 0 * 256 * 256
+ 0 * 256
+ 0
= 855638016
While what you actually want is this:
value = 0 * 256 * 256 * 256
+ 0 * 256 * 256
+ 0 * 256
+ 51
= 51
The fixed calculation would thus be this:
value = ((0xFF & bytes[3]) << 24)
| ((0xFF & bytes[2]) << 16)
| ((0xFF & bytes[1]) << 8)
| (0xFF & bytes[0]);
Ok stupid enough but I just didn't preserve the byte order.
[51, 0, 0, 0] -> is 855638016
[0, 0, 0, 51] -> is 51
What is the difference between:
int n = (b1 & 0xff) << 24 | (b2 & 0xff) << 16 | (b3 & 0xff) << 8 | b4 & 0xff
and
int n = b1 << 24 | b2 << 16 | b3 << 8 | b4
? Why should we AND each byte with 255?
First you need to understand that b1 <<< 24 expects the a operand to be of type int or long. So, if b1 is a byte, Java will convert it to an int before the shift takes place.
That conversion happens by mapping each byte value into the corresponding int value. The problem is that byte is a signed type, which means that the possible values are -128 through to +127. Therefore, when you convert a byte with a negative value, you will get an int with a negative value; e.g. the byte 11111111 (binary) which is -1 becomes 11111111111111111111111111111111 (binary) which is also -1.
Those leading ones are the result of sign extension, and we do NOT want them if we are about to shift them and combine them in a bitwise fashion.
Hence, you will often see a byte being masked with 0xff to remove the (unwanted) results of sign extension after the conversion. For example
11111111111111111111111111101110 &
00000000000000000000000011111111
gives
00000000000000000000000011101110
Let me assume types of b1, b2, b3 and b4 are byte.
In this case, their values will be sign extended and if they are negative, the result will be wrong if AND isn't used.
Demo:
class Test {
public static void main(String[] args) {
byte a = (byte)128;
int x1 = a << 8;
int x2 = (a & 0xff) << 8;
System.out.println(x1);
System.out.println(x2);
}
}
Output:
-32768
32768
So I'm trying to understand base64 encoding better and I came across this implementation on wikipedia
private static String base64Encode(byte[] in) {
StringBuffer out = new StringBuffer((in.length * 4) / 3);
int b;
for (int i = 0; i < in.length; i += 3) {
b = (in[i] & 0xFC) >> 2;
out.append(codes.charAt(b));
b = (in[i] & 0x03) << 4;
if (i + 1 < in.length) {
b |= (in[i + 1] & 0xF0) >> 4;
out.append(codes.charAt(b));
b = (in[i + 1] & 0x0F) << 2;
if (i + 2 < in.length) {
b |= (in[i + 2] & 0xC0) >> 6;
out.append(codes.charAt(b));
b = in[i + 2] & 0x3F;
out.append(codes.charAt(b));
} else {
out.append(codes.charAt(b));
out.append('=');
}
} else {
out.append(codes.charAt(b));
out.append("==");
}
}
return out.toString();
}
And I'm following along and I get to the line:
b = (in[i] & 0xFC) >> 2;
and I don't get it...why would you bitwise and 252 to a number then shift it right 2...wouldn't it be the same if you just shifted the byte itself without doing the bitwise operation? example:
b = in[i] >> 2;
Say my in[i] was the letter e...represented as 101 or in binary 01100101. If I shift that 2 to the right I get 011001 or 25. If I bitwise & it I get
01100101
11111100
--------
01100100
but then the shift is going to chop off the last 2 anyway...so why bother doing it?
Can somebody clarify for me please. Thanks.
IN in[i] >> 2, in[i] is converted to an int first. If it was a negative byte (with the high bit set) it will be converted to a negative int (with the now-highest 24 bits set as well).
In (in[i] & 0xFC) >> 2, in[i] is converted to an int as above, and then & 0xFC makes sure the extra bits are all reset to 0.
You're partially right, in that (in[i] & 0xFF) >> 2 would give the same result. & 0xFF is a common way to convert a byte to a non-negative int in the range 0 to 255.
The only way to know for sure why the original developer used 0xFC, and not 0xFF, is to ask them - but I speculate that it's to make it more obvious which bits are being used.
alright, so my code to read bytes into a int is like so:
int offset = (byte << 16) | (byte2 << 8) | byte3;
And it's reading the bytes "00 00 be" as -66.
How do I read it as the 190 it's meant to be?
byte b = -66;
int i = b & 0xff;
byte b = -66;
int i = b < 0 ? b + 256 : b;
It might be useful declare helper function for this.