Java vs. C#: BigInteger hex string yields different result? - java

Question:
This code in Java:
BigInteger mod = new BigInteger("86f71688cdd2612ca117d1f54bdae029", 16);
produces (in java) the number
179399505810976971998364784462504058921
However, when I use C#,
BigInteger mod = BigInteger.Parse("86f71688cdd2612ca117d1f54bdae029", System.Globalization.NumberStyles.HexNumber); // base 16
i don't get the same number, I get:
-160882861109961491465009822969264152535
However, when I create the number directly from decimal, it works
BigInteger mod = BigInteger.Parse("179399505810976971998364784462504058921");
I tried converting the hex string in a byte array and reversing it, and creating a biginteger from the reversed array, just in case it's a byte array with different endianness, but that didn't help...
I also encountered the following problem when converting Java-Code to C#:
Java
BigInteger k0 = new BigInteger(byte[]);
to get the same number in C#, I must reverse the array because of different Endianness in the biginteger implementation
C# equivalent:
BigInteger k0 = new BigInteger(byte[].Reverse().ToArray());

Here's what MSDN says about BigInteger.Parse:
If value is a hexadecimal string, the Parse(String, NumberStyles) method interprets value as a negative number stored by using two's complement representation if its first two hexadecimal digits are greater than or equal to 0x80. In other words, the method interprets the highest-order bit of the first byte in value as the sign bit. To make sure that a hexadecimal string is correctly interpreted as a positive number, the first digit in value must have a value of zero. For example, the method interprets 0x80 as a negative value, but it interprets either 0x080 or 0x0080 as a positive value.
So, add a 0 in front of the parsed hexadecimal number to force an unsigned interpretation.
As for round-tripping a big integer represented by a byte array between Java and C#, I'd advise against that, unless you really have to. But both implementations happen to use a compatible two's complement representation, if you fix the endianness issue.
MSDN says:
The individual bytes in the array returned by this method appear in little-endian order. That is, the lower-order bytes of the value precede the higher-order bytes. The first byte of the array reflects the first eight bits of the BigInteger value, the second byte reflects the next eight bits, and so on.
Java docs say:
Returns a byte array containing the two's-complement representation of this BigInteger. The byte array will be in big-endian byte-order: the most significant byte is in the zeroth element.

Related

Error parsing 16 bit short

short: The short data type is a 16-bit signed two's complement integer. It has a minimum value of -32,768 and a maximum value of 32,767 (inclusive)
Why does the following
System.out.println(Short.parseShort("1111111111111111", 2));
Return
java.lang.NumberFormatException: Value out of range.
An exception of type NumberFormatException is thrown if any of the following situations occurs:
The first argument is null or is a string of length zero.
The radix is either smaller than Character.MIN_RADIX or larger than Character.MAX_RADIX.
Any character of the string is not a digit of the specified radix, except that the first character may be a minus sign '-' ('\u002D') or plus sign '+' ('\u002B') provided that the string is longer than length 1.
The value represented by the string is not a value of type short.
I assume the error is from the last bullet, but
I thought 16 '1' bits is equivalent to -1 when using Short. Thus, it should be valid?
1111111111111111 should be converted to 65535, which is greater than the maximum value(32,767) short can represent. Try some smaller numbers.
The javadoc that you quoted states that Short.parseShort parses numbers as signed numbers.
1111111111111111 (16 1 digits) when read as a signed number means 216 - 1 or 65535. That is too large to be represented as a short.
Alternatives:
If there was an alternative to parseShort that parsed unsigned values, that would work. (But there isn't ...)
You could use Integer.parseInt, do a range check on the int result, and then cast it to a short.
I thought 16 '1' bits is equivalent to -1 when using Short. Thus, it should be valid?
Unfortunately, no. The parseInt method parses signed values.
Thought experiment: what if the user entered 1111111111111111 with the intention that it really meant a positive signed number; i.e. +65535? How would that mesh with your idea that the parse method treats signed and unsigned as interchangeable?

Rounding error from BigDecimal to byte array in Java

How should one properly handle a conversion rounding error with BigDecimal in Java:
BigDecimal -> byte[] -> BigDecimal
I have a custom datatype 32 bytes in length (yep, 32 bytes not 32 bits) and I need to decode the fractional part of BigDecimal into byte[].
I understand that I will lose some accuracy. Are there any established techniques to implement such a conversion?
NOTE:
It is fixed point datatype of form MxN, where M % 8 == N % 8 == 0
Your fixed-point fractional part can be interpreted as the numerator, n, of a fraction n/(2256). I suggest, therefore, computing the BigDecimal value representing 1/(2256) (this is exactly representable as a BigDecimal) and storing a reference to it in a final static field.
To convert to a byte[], then, use the two-arg version of BigDecimal.divideToIntegralValue() to divide the fractional part of your starting number by 1/(2256), using the MathContext argument to specify the rounding mode you want. Presumably you want either RoundingMode.HALF_EVEN or RoundingMode.HALF_UP. Then get the BigInteger unscaled value of the result (which should be numerically equal to the scaled value, since an integral value should have scale 0) via BigDecimal.unscaledValue(). BigInteger.toByteArray() will then give you a byte[] closely related to what you're after.*
To go the other way, you can pretty much reverse the process. BigDecimal has a constructor that accepts a byte[] that, again, is very closely related to your representation. Using that constructor, convert your byte[] to a BigInteger, and thence to BigDecimal via the appropriate constructor. Multiply by that stored 1/(2256) value to get the fractional part you want.
* The biggest trick here may involve twiddling signs appropriately. If your BigDecimals may be negative, then you probably want to first obtain their absolute values before converting to byte[]. More importantly, the byte[]s produced and consumed by BigInteger use a two's complement representation (i.e. with a sign bit), whereas I suppose you'll want an unsigned, pure binary representation. That mainly means that you'll need to allow for an extra bit -- and therefore a whole extra byte -- when you convert. Be also aware of byte order; check the docs of BigInteger for the byte order it uses, and adjust as appropriate.

Why am I getting a value back from Long.parseUnsignedLong when I shouldn't be

When I do
Long.parseUnsignedLong("FBD626CC4961A4FC", 16)
I get back -300009666327239428
Which seems wrong, since the meaning of unsigned long according to this answer https://stackoverflow.com/a/2550367/1754020 is that the range is always positive.
To get the correct number from this HEX value I do
BigInteger value = new BigInteger("FBD626CC4961A4FC", 16);
When I print value it prints the correct value. but if I do value.longValue()
again I get the same -300009666327239428 is this of the number being too big and overflowing ?
Java 8 does (somewhat) support unsigned longs, however, you can't just print them directly. Doing so will give you the result that you saw.
If you have an unsigned long
Long number = Long.parseUnsignedLong("FBD626CC4961A4FC", 16);
you can get the correct string representation with the function
String numberToPrint = Long.toUnsignedString(number);
If you now print numberToPrint you get
18146734407382312188
To be more exact, your number is still going to be a regular signed long which is why it shows overflow if printed directly. However, there are new static functions that will treat the value as if it was unsigned, such as this Long.toUnsignedString(long x) or Long.compareUnsigned(long x, long y).
The hexadecimal number "FBD626CC4961A4FC", converted to decimal, is exactly 18146734407382312188. That number is indeed larger than the maximum possible long, defined as Long.MAX_VALUE and which is equal to 263-1, or 9223372036854775807:
System.out.println(new BigInteger("FBD626CC4961A4FC", 16)); // 18146734407382312188
System.out.println(Long.MAX_VALUE); // 9223372036854775807
As such, it's normal that you get back a negative number.
You do not have an exception, as it is exactly the purpose of those new *Unsigned* methods added in Java 8, to give the ability to handle unsigned longs (like compareUnsigned or divideUnsigned). Since the type long in Java is still unsigned, those methods work by understanding negative values as values greater than MAX_VALUE: it simulates an unsigned long. parseUnsignedLong says:
An unsigned integer maps the values usually associated with negative numbers to positive numbers larger than MAX_VALUE.
If you print a long that was the result of parseUnsignedLong, and it is negative, all it means is that the value is greater than the max long value as defined by the language, but that methods taking unsigned longs as parameter will correctly interpret those values, as if they were greater than the max value. As such, instead of printing it directly, if you pass that number to toUnsignedString, you'll get the right output, like shown in this other answer. Not all of these methods are new to Java 8, for example toHexString also interprets the given long as an unsigned long in base 16, and printing Long.toHexString(Long.parseUnsignedLong("FBD626CC4961A4FC", 16)) will give you back the right hex String.
parseUnsignedLong will throw an exception only when the value cannot be represented as an unsigned long, i.e. not a number at all, or greater than 264-1 (and not 263-1 which is the maximum value for a signed long).
Yes, it overflows when you are trying to print it, as it is converted to Java long type. To understand why let's take log2 of your dec value.
First thing, original value is 18146734407382312188. It's log2 is ~63.9763437545.
Second, look into documentation: in java long type represents values of -2^63 and a maximum value of 2^63-1.
So, your value is obviously greater then 2^63-1, hence it overflows:
-2^63 + (18146734407382312188 - 2^63 + 1) = -300009666327239428
But as #Keiwan brilliantly mentioned, you still can print proper value using Long.toUnsignedString(number);
Internally unsigned and signed numbers are represented in the same way, i.e. as 8 bytes in case of a long. The difference only how the "sign" bit interpreted, i.e. if you'd do the same in a C/C++ program and store your value into an uint64_t then cast/map it to a asigned int64_t you should get the same result.
Since the maximum value 8 bytes or 64 bits can hold is 2^64-1 that's the hard constraint for such numbers. Also Java doesn't directly support unsigned numbers and thus the only way to store an unsigned long in a long is to allow for a value that's higher than the signed Long.MAX_VALUE. In fact Java doesn't know whether the string/hexcode you're reading is meant to represent a signed or unsigned long so it's up to you to provide that interpretation, either by converting back to a string or using a larger datatype such as BigInteger.

How can i convert an integer to byte? [duplicate]

This question already has answers here:
Odd behavior when Java converts int to byte?
(12 answers)
Closed 9 years ago.
I want to know how can i convert an integer to byte theoretically.
I mean i don't want to use a predefined program but i want just to know how can i implement it.
What i know that from -128 to 127 an integer is the same as byte but the problemes is from 128 to 128 to +infinite and from -129 to -infinite.
For example given the following code:
Integer a = 140;//10001100 this is his binary conversion
Byte zz = (byte) a.byteValue();
System.out.println(zz);// result is -116
How that conversion works in java?
Thanks in advance
That value is out of range of the Byte and hence overflows.
Refer JLS 4.2.1:
The values of the integral types are integers in the following ranges:
For byte, from -128 to 127, inclusive
A byte is 8 bits , the most significant bit specifies the sign of the number and are are encoded in two's complement.
Read this wonderful SO answer for more.
when you are narrowing a primitive, you must explicitly make a cast - so you acknowledge a possible loss of data.
There is no loss if value is within the -128...127 byte value range
Byte value binary representation will not change, it will still be 10001100 but it will be interpreted differently, since byte is a signed type in two's complement representation http://en.wikipedia.org/wiki/Twos_complement and since bit 7 is set it means that now it's a negative number -116
You can not convert value grate than 127 into byte as it do not have place to store it. As 140 in binary is 10001100 it mean that you need a type that store at lest 8 bits. A byte store 8 bits but one of then is reserved for sign. So you can not fit it into it. You can use short witch store 16 bits (15 and 1 for sign).
The zero in different types expressed in binary;
byte b = 0b000_0000;
short s = 0b000_0000_0000_0000;
int i = 0b000_0000_0000_0000_0000_0000_0000_0000;
If you try to declare something like:
byte b = 0b1000_1100;
The compiler will tall you Type mismatch: cannot convert from int to byte.

Wrapper's parseXXX() for signed binary misunderstanding

Let's take Byte.parseByte() as an example as one of the wrappers' parseXXX().
From parseByte(String s, int radix)'s JavaDoc:
Parses the string argument as a signed byte in the radix specified by
the second argument.
But that's not quite true if radix = 2. In other words, the binary literal of -127 is 10000000:
byte b = (byte) 0b10000000;
So the following should be true:
byte b = Byte.parseByte("10000000", 2);
but unfortunately, it throws NumberFormatException, and instead I have to do it as follows:
byte b = Byte.parseByte("-111111", 2);
where parseByte() parses the binary string as a sign-magnitude (the sign and the magnitude), where it should parse as a signed binary (2's complement, i.e. MSB is the sign-bit).
Am I wrong about this?
Am I wrong about this?
Yes. The Javadoc says nothing about 2's-complement. Indeed, it explicitly states how it recognises negative values (i.e. a - prefix, so effectively "human-readable" sign-magnitude).
Think about it another way. If parseByte interpreted radix-2 as 2's-complement, what would you want it to do for radix-10 (or indeed, any other radix)? For consistency, it would have to be 10's-complement, which would be inconvenient, I can assure you!
This is because for parseByte "10000000" is a positive value (128) which does not fit into byte values range -128 to 128. But we can parse two's comlement binary string representation with BigInteger:
byte b = new BigInteger("10000000", 2).byteValue()
this gives expected -128 result

Categories