I am trying to shift a 7 byte array to the right by 7 bits.
To do this, I am using BigInteger's shiftright method. However, when shifting right for negative BigIntegers, padding with 1s are added or sometimes the leading bit is removed.
Here is the following bit of code doing the shifting:
byte[] vcwManD = decryptedVCW;
BigInteger bigIntD = new BigInteger(vcwManD); // create big int array for shift
BigInteger shiftIntD= bigIntD.shiftRight(7); // shift right 7 bits
vcwManD = shiftIntD.toByteArray();
For a byte array E865037A9C6424 in binary:
11101000011001010000001101111010100111000110010000100100
When shifted I get D0CA06F538C8 in binary:
110100001100101000000110111101010011100011001000
As you can see, it has shifted 7 bits to the right, however the leading bit has been stripped.
Another issue is the 1 padding. For byte array 90998951A37908 in binary
10010000100110011000100101010001101000110111100100001000
produces FF213312A346F2 in binary:
11111111001000010011001100010010101000110100011011110010
This time around 7 1's have been padded at the start.
Does anyone know how to solve this issue?
Many thanks
Shiv
If you are doing bit shifts you are likely working with a long unsigned value rather than signed value. So when you create your BigInteger use this constructor:
new BigInteger(1, vcwManD);
That way you are guaranteed to have a positive number and you should be able to shift it around with no consequence.
Negative numbers are stored in memory using two's complement. This means that if the number is negative, the first bit will be 1. When you bit shift a negative number to the right, it must remain negative, so the newly introduced values will be 1's.
Related
I have the following value:
int x = -51232;
Java integers are 32 bits, so in binary this should be the following:
10000000000000001100100000100000
The sign bit on the left is set to 1 since x is negative.
Then I do the operation
x = (x & Integer.MAX_VALUE);
Integer.MAX_VALUE is 2147483647 and in binary that would be:
01111111111111111111111111111111
0 on the left because the value is positive.
So why does x & Integer.MAX_VALUE yield 2147432416? The AND operator should only retrieve bits that x and Integer.MAX_VALUE have in common, which should be equivalent to -x (since they do not share the same sign bit).
What's going on here?
Your misunderstanding is caused by lack of knowledge regarding how negative integers are represented in binary in Java. You should read about 2's complement.
10000000000000001100100000100000 is not the binary representation of -51232.
11111111111111110011011111100000 is.
And when you run bitwise AND, you get:
11111111111111110011011111100000 (-51232)
01111111111111111111111111111111 (Integer.MAX_VALUE)
--------------------------------
01111111111111110011011111100000 (2147432416)
Here's the binary representation of -51232 next to the binary representation of 51232. You can see that their sum is 232. that's always the case with 2's complement, for any pair of ints x and -x.
00000000000000001100100000100000 (-51232)
11111111111111110011011111100000 (51232)
Integers are stored in two complement: https://en.m.wikipedia.org/wiki/Two%27s_complement.
Thus while the leftmost bit indicate negative numbers, it is not really a sign bit as it would be for floating point numbers in common representation.
The main reason of this representation is that it make it easy to do addition and substraction among other at the hardware level.
One should not that with two complement notation, -1 is represented by all bit set to 1.
I've 2 integer values stored in a bytebuffer, in little-endian format. These integers are actually the 32-bit pieces of a long. I've to store them as a class' member variables, loBits and hiBits.
This is what I did:
long loBits = buffer.getInt(offset);
long hiBits = buffer.getInt(offset + Integer.BYTES);
I want to know why directly assigning signed int to long is wrong. I kind of know what's going on, but would really appreciate an explanation.
The int I read from the buffer is signed (because Java). If it is negative then directly assigning it to a long value (or casting it like (long)) would change all the higher order bits in the long to the signed bit value.
For e.g. Hex representation of an int, -1684168480 is 9b9da0e0. If I assign this int to a long, all higher order 32 bits would become F.
int negativeIntValue = -1684168480;
long val1 = negativeIntValue;
long val2 = (long) negativeIntValue;
Hex representation of:
negativeIntValue is 0x9b9da0e0
val1 is 0xffffffff9b9da0e0
val2 is 0xffffffff9b9da0e0
However, if I mask the negativeIntValue with 0x00000000FFFFFFFFL, I get a long which has the same hex representation as negativeIntValue and a positive long value of 2610798816.
So my questions are:
Is my understanding correct?
Why does this happen?
Yes, your understanding is correct (at least if I understood your understanding correctly).
The reason this happens is because (most) computers use 2's complement to store signed values. So when assigning a smaller datatype to a larger one, the value is sign extended meaning that the excess part of the datatype is filled with 0 or 1 bits depending on whether the original value was positive or negative.
Also related is the difference between >> and >>> operators in Java. The first one performs sign extending (keeping negative values negative) the second one does not (shifting a negative value makes it positive).
The reason for this is that negative values are stored as two's complement.
Why do we use two's complement?
In a fixed width numbering system what happens, if you substract 1 from 0?
0000b - 0001b -> 1111b
and what is the next lesser number to 0? It is -1.
Therfore we thread a binary number with all bits set (for a signed datatype) as -1
The big advantage is that the CPU does not need to do any special operation when changing from positive to negative numbers. It handles 5 - 3 the same as 3 - 5
Question:
This code in Java:
BigInteger mod = new BigInteger("86f71688cdd2612ca117d1f54bdae029", 16);
produces (in java) the number
179399505810976971998364784462504058921
However, when I use C#,
BigInteger mod = BigInteger.Parse("86f71688cdd2612ca117d1f54bdae029", System.Globalization.NumberStyles.HexNumber); // base 16
i don't get the same number, I get:
-160882861109961491465009822969264152535
However, when I create the number directly from decimal, it works
BigInteger mod = BigInteger.Parse("179399505810976971998364784462504058921");
I tried converting the hex string in a byte array and reversing it, and creating a biginteger from the reversed array, just in case it's a byte array with different endianness, but that didn't help...
I also encountered the following problem when converting Java-Code to C#:
Java
BigInteger k0 = new BigInteger(byte[]);
to get the same number in C#, I must reverse the array because of different Endianness in the biginteger implementation
C# equivalent:
BigInteger k0 = new BigInteger(byte[].Reverse().ToArray());
Here's what MSDN says about BigInteger.Parse:
If value is a hexadecimal string, the Parse(String, NumberStyles) method interprets value as a negative number stored by using two's complement representation if its first two hexadecimal digits are greater than or equal to 0x80. In other words, the method interprets the highest-order bit of the first byte in value as the sign bit. To make sure that a hexadecimal string is correctly interpreted as a positive number, the first digit in value must have a value of zero. For example, the method interprets 0x80 as a negative value, but it interprets either 0x080 or 0x0080 as a positive value.
So, add a 0 in front of the parsed hexadecimal number to force an unsigned interpretation.
As for round-tripping a big integer represented by a byte array between Java and C#, I'd advise against that, unless you really have to. But both implementations happen to use a compatible two's complement representation, if you fix the endianness issue.
MSDN says:
The individual bytes in the array returned by this method appear in little-endian order. That is, the lower-order bytes of the value precede the higher-order bytes. The first byte of the array reflects the first eight bits of the BigInteger value, the second byte reflects the next eight bits, and so on.
Java docs say:
Returns a byte array containing the two's-complement representation of this BigInteger. The byte array will be in big-endian byte-order: the most significant byte is in the zeroth element.
I have made a variable in java, byte a = 0xA6; //10100110
then I made this :
System.out.println(Integer.toHexString( ((short)a<<8)&0xFFFF ));
The result is 0xA600. This is the right result. But when i tried
System.out.println(Integer.toHexString( ((short)a<<3)&0xFFFF ));
The expected result should be : 0x530 (10100110000)
but I got 0xFD30(1111110100110000) Emm... Can somebody explain how I got that wrong result...??
thanks... :-)
The byte value A6 represents a negative number (bytes are signed in Java). When you cast to a short it gets sign extended to FFA6. Moreover the shift operation is executed with integer values so it is again sign extended to FFFFFFA6. Shift left by three bits gives FFFFFD30 and taking the lower 16 bits gives 0000FD30.
This does not matter if you shift by 8 bits because you shift out and mask the additional 1 bits.
When you declare initialize byte variable you have to downcast it from integer:
byte a = (byte) 0xA6;
So, instead of 10100110 you've got 11111111111111111111111110100110.
And, beacuse of this left shift works in that way:
((short)a<<8)&0xFFFF
returns 1010011000000000
((short)a<<3)&0xFFFF
returns 1111110100110000
Need a solution on how to perform the following: receive a decimal value, convert it to 32-bit Hex, then separate that 32-bit hex and get high 16-bit and low 16-bit values. I have been digging around net and cannot find much info.
Not sure why you are converting to hex but to turn a 32-bit value straight into two 16-bit values.
int x = ...
short high = x >> 16;
short low = x & 0xFFFF;
I expect this is a homework problem. Accordingly, I will give you information that can help you solve it rather than a solution.
Convert the number to hexadecimal. This can be done with the Integer's toHexString() method.
Add enough zeroes to the left to make it eight characters long (8 hexadecimal characters represent 32 bits). You can do this by adding zeroes one by one in a loop until it's 8 characters long, or (better approach) just add 7 zeroes to the left and only deal with the rightmost 8 characters.
Take the the rightmost 4 characters as the lower 16 bits and the 4 characters immediately to the left of that as the higher 16 bits. This can be done with the String's substring() method along with length() and some simple subtraction.
Some APIs you might find useful:
http://download.oracle.com/javase/6/docs/api/java/io/DataInputStream.html
http://download.oracle.com/javase/6/docs/api/java/lang/Integer.html#parseInt(java.lang.String, int)
http://download.oracle.com/javase/6/docs/api/java/lang/Integer.html#toHexString(int)
http://commons.apache.org/codec/apidocs/org/apache/commons/codec/binary/Hex.html