I try to compare 2 byte arrays.
Byte array 1 is an array with the last 3 bytes of a sha1 hash:
private static byte[] sha1SsidGetBytes(byte[] sha1)
{
return new byte[] {sha1[17], sha1[18], sha1[19]};
}
Byte array 2 is an array that I fill with 3 bytes coming from an hexadecimal string:
private static byte[] ssidGetBytes(String ssid)
{
BigInteger ssidBigInt = new BigInteger(ssid, 16);
return ssidBigInt.toByteArray();
}
How is it possible that this comparison:
if (Arrays.equals(ssidBytes, sha1SsidGetBytes(snSha1)))
{
}
works most of the times but sometimes not. Byte Order?
e.g. for "6451E6" (hex string) it works fine, for "ABED74" it does not...
The problem is pretty obvious if you try this:
BigInteger b1 = new BigInteger("6451E6", 16);
BigInteger b2 = new BigInteger("ABED74", 16);
System.out.println(b1.toByteArray().length);
System.out.println(b2.toByteArray().length);
Specifically, ABED74 creates a BigInteger whose byte array is 4 bytes long--so of course it's not going to be equal to any three byte array.
The straightforward fix is to change the return statement in ssidGetBytes from
return ssidBigInt.toByteArray();
to
byte[] ba = ssidBigInt.toByteArray();
return new byte[] { ba[ba.length - 3], ba[ba.length - 2], ba[ba.length - 1] };
Your approach of parsing a hex string via BigInteger is flawed, basically. For example, new BigInteger("ABED74").toByteArray() returns an array of 4 bytes, not three. While you could hack around this, you're fundamentally not trying to do anything involving BigInteger values... you're just trying to parse hex.
I suggest you use the Apache Codec library to do the parsing:
byte[] array = (byte[]) new Hex().decode(text);
(The API for Apache Codec leaves something to be desired, but it does work.)
From the javadoc's (emphasis mine):
http://download.oracle.com/javase/1.5.0/docs/api/java/math/BigInteger.html#toByteArray%28%29
Returns a byte array containing the
two's-complement representation of
this BigInteger. The byte array will
be in big-endian byte-order: the most
significant byte is in the zeroth
element. The array will contain the
minimum number of bytes required to
represent this BigInteger, including
at least one sign bit, which is
(ceil((this.bitLength() + 1)/8)).
(This representation is compatible
with the (byte[]) constructor.)
There is a lot of computations going on inside the ByteInteger(String,radix) constructor that you are using, which does not guarantee the constructed BigInteger will produce a byte array (via its toByteArray() method) comparable to the result of a String's getBytes() encoding.
The output of toByteArray() is intended to be used (mostly) as input to the (byte[]) constructor of BigInteger. It makes no guarantee for uses other than those.
Look at it like this: the output of toByteArray() is the byte representation of the BigInteger object and everything in it including internal attributes like magnitude. Those attributes do not exist in the input String, but are computed during construction of the BitInteger object.
That will be incompatible to the byte representation of the input String which only carries the initial numeric value with which to create a BigInteger.
Related
I have a function for hashing passwords, that returns a byte[] with entries using the full range of the byte datatype from -128 to 127. I have tried to convert the byte[] to a String using new String(byte_array, StandardCharsets.UTF_8);. This does return a String - however it can not properly encode negative numbers - hence it encodes them to a "�" character. When comparing two of those characters using: new String(new byte[]{-1}, StandardCharsets.UTF_8).equals(new String(new byte[]{-2}, StandardCharsets.UTF_8)) it turns out the String representation for all negative numbers is equal as the expression above returns true. While this doesn't fully ruin my hashing functionality as the hash of the same expression will still always yield the same result, this is obviously not what I want as it increases the chance of two different inputs yielding the same output drastically.
Is there some easy fix for this or any alternative idea how to convert the byte[] to a String? For context I want to use the String to later write it to a file to store it in a file and later read it again to compare it to other hashes.
Edit: After a bit of trying around with the tips from the comments my solution is to convert the byte[] to a char[] and add 128 to every value. The char array can then easily be converted to a String or be written to a file directly (byteHash is the byte[]):
char[] charHash = new char[byteHash.length];
for(int i = 0; i < byteHash.length; i++){
charHash[i] = (char) (byteHash[i]+128);
}
return new String(charHash);
I do not really like the solution but it works.
The appropriate solution to this is to use an encoding like hexadecimal (https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/HexFormat.html) or Base64 (https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html) to convert an arbitrary byte sequence to a string reversibly.
I am trying to hash strings using MD5. I need the hashed value as a 128 bit unsigned integer in Java.
MessageDigest md = MessageDigest.getInstance("MD5");
String toHash = "HashThis";
md.update(toHash.getBytes());
byte[] isHashed = md.digest();
How do I convert isHashed to an integer?
Java does not have a 128 bit int type. But it has a BigInteger class, which have a constructor that takes a signum and a magnitude expressed as a byte[], as you require.
BigInteger value=new BigInteger(1,isHashed);
Use BigInteger.
BigInteger value = new BigInteger(isHashed);
To ensure that the resulting value is positive (as the byte array is expected to be in 2's complement), make sure the most significant bit is zero. This can be easily achieved by making the array 1 element bigger, with the first byte being 0.
The question is about the correct way of creating a hash in Java:
Lets assume I have a positive BigInteger value that I would like to create a hash from. Lets assume that below instance of the messageDigest is a valid instance of (SHA-256)
public static final BigInteger B = new BigInteger("BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58", 16);
byte[] byteArrayBBigInt = B.toByteArray();
this.printArray(byteArrayBBigInt);
messageDigest.reset();
messageDigest.update(byteArrayBBigInt);
byte[] outputBBigInt = messageDigest.digest();
Now I only assume that the code below is correct, as according to the test the hashes I produce match with the one produced by:
http://www.fileformat.info/tool/hash.htm?hex=BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58
However I am not sure why we are doing the step below i.e.
because the returned byte array after the digest() call is signed and in this case it is a negative, I suspect that we do need to convert it to a positive number i.e. we can use a function like that.
public static String byteArrayToHexString(byte[] b) {
String result = "";
for (int i=0; i < b.length; i++) {
result += Integer.toString((b[i] & 0xff) + 0x100, 16).substring(1);
}
return result;
}
thus:
String hex = byteArrayToHexString(outputBBigInt)
BigInteger unsignedBigInteger = new BigInteger(hex, 16);
When I construct a BigInteger from the new hex string and convert it back to byte array then I see that the sign bit, that is most significant bit i.e. the leftmost bit, is set to 0 which means that the number is positive, moreover the whole byte is constructed from zeros ( 00000000 ).
My question is: Is there any RFC that describes why do we need to convert the hash always to a "positive" unsigned byte array. I mean even if the number produced after the digest call is negative it is still a valid hash, right? thus why do we need that additional procedure. Basically, I am looking for a paper: standard or rfc describing that we need to do so.
A hash consists of an octet string (called a byte array in Java). How you convert it to or from a large number (a BigInteger in Java) is completely out of the scope for cryptographic hash algorithms. So no, there is no RFC to describe it as there is (usually) no reason to treat a hash as a number. In that sense a cryptographic hash is rather different from Object.hashCode().
That you can only treat hexadecimals as unsigned is a bit of an issue, but if you really want to then you can first convert it back to a byte array, and then perform new BigInteger(result). That constructor does threat the encoding within result as signed. Note that in protocols it is often not needed to convert back and forth to hexadecimals; hexadecimals are mainly for human consumption, a computer is fine with bytes.
I have a Java class
public class MsgLayout{
int field1;
String field2;
long field3;
}
I have to write this object as a byte array in a Socket output stream. The three fields (instance variables) have a layout. i.e. field1 must occupy 1 byte, field2 must occupy 4 bytes and field3 must occupy 8 bytes.
ByteBuffer bbf = ByteBuffer.allocate(TOTAL_SIZE);
bbf.put(Integer.toString(this.getField1()).getBytes(), 0, FIELD1_SIZE);
bbf.position(FIELD2_OFFSET);
bbf.put(Long.toString(this.getField2()).getBytes(), 0, FIELD2_SIZE);
bbf.position(FIELD3_OFFSET);
bbf.put(Long.toString(this.getField3()).getBytes(), 0, FIELD3_SIZE);
byte[] msg = bbf.array();
Using the above code, I am trying to fit each field in the byte array according to its desired size. But I am getting IndexOutOfBoundException
In short, the problem is about how to fit the fields in the layout-defined size. For Example FIELD1_OFFSET = 0, FIELD1_SIZE=1, FIELD2_OFFSET=1, FIELD2_SIZE=4, FIELD3_OFFSET=5, FIELD3_SIZE=8.
Now when I convert field1 into String, it does not fit into 1 byte when converted into byte[]. If I do not convert to String, and use putInt(int) it writes 4 bytes into the resulting byte array.
What your code is currently doing is encoding your numeric fields as strings and then outputting the bytes of those characters.
I would suggest using the DataOutputStream class to wrap your SocketOutput stream and write your binary data as so:
DataOutput output = new DataOutputStream(socketOutputStream);
int field1 = 1;
String field2 = "Hello";
long field3 = 5000000000L;
output.writeByte(field1);
output.writeBytes(field2.substring(0, 3));
output.writeLong(field3);
There are a couple assumptions in this code. First I'm assuming for field 2 you want 4 characters serialized as a single byte each. If you want to do any multibyte encoding using something like UTF-8, then you need to do something a little differently. Second, I'm assuming that field 2 will always have at least 4 characters.
field1 may only have one byte of data, but its string representation will be one or more characters (e.g. "0", "63", "127"). Each character in the String is in fact a char (a two byte value). So I would expect one byte of data to inflate to two to six bytes of data when it goes through a byte->String->byte[] conversion.
I converted an int to a byte array using ByteBuffer's putInt() method. How do I do the opposite? So convert those bytes to an int?
Furthermore, I converted a string to an array of bytes using the String's getBytes() method. How do I convert it the other way round? The bytesArray.getString() does not return a readable string. I get things like BF#DDAD
You can use the ByteBuffer.getInt method, specifying the offset at which the integer occurs, to convert a series of bytes into an integer. Alternatively, if you happen to know the byte ordering, you can use bitwise operators to explicitly reconstruct the 32-bit integer from its 8-bit octets.
To convert an array of bytes into a String, you can use the String(byte[]) constructor to construct a new String out of the byte array. For example:
byte[] bytes = /* ... get array of bytes ... */
String fromBytes = new String(bytes);