How would I go about doing that? I tried using SHA-1 and MD5 but the output is too long for my requirements and truncation would not make it unique.
Input : String containing numbers e.g. (0302160123456789)
Received output : 30f2bddc3e2fba9c05d97d04f8da4449
Desired Output: Unique number within range (0000000000000000 - FFFFFFFFFFFFFFFF) and 16 characters long
Any help/ pointers are greatly appreciated.
How big is your input domain? If it is bigger than your output domain, then the Pigeon Hole principle applies and you can't get unique output by definition.
If the input domain is smaller or equal to the output domain, then you can easily accomplish this with a Pseudo-Random Permutation (PRP) that block ciphers provide.
The output of 16 hexits is equivalent to 8 bytes and equivalent to 64 bit. DES (and Triple DES) is a block cipher that has this block size.
Parse the input string to a byte array in a compact fashion. If the input always consists of numerical digits, you can use Ebbe M. Pedersen's approach with
byte[] plaintext = new BigInteger("0302160123456789").toByteArray();
Then you can generate some random, but fixed key of 24 bytes for Triple DES and instantiate the cipher with:
Cipher c = Cipher.getInstance("DESede/ECB/PKCS5Padding");
c.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(key, "DESede"));
byte[] ciphertext = c.doFinal(plaintext);
Use some kind of Hex converter to get the representation you want.
You can "hash" numbers up to 36028797018963968 with this. If you want larger numbers (up to 9223372036854775808), then you need to use "DESede/ECB/NoPadding" and pad yourself with some padding bytes.
are you going to receive more than FFFFFFFFFFFFFFFF different strings?
if not then it's a simple problem of generating integers: the first string will get 0 the next 1 etc; you just keep a list of the strings and check if something the same appears.
You could just convert your number to hex using BigInteger like this:
String id = new BigInteger("0302160123456789").toString(16);
System.out.println(id);
That gives:
112d022d2ed15
Related
I have String hash in hex form ("e6fb06210fafc02fd7479ddbed2d042cc3a5155e") and I would like to compare it to crypt.digest().
One way, which works fine, is to convert crypt.digest() to hex, but I would like to avoid multiple conversions and rather convert hash from hex form (above) to byte array.
What I tried was:
byte[] hashBytes = new BigInteger(hash, 16).toByteArray();
but it does not match with crypt.digest(). When I convert hashBytes back to hex I get "00e6fb06210fafc02fd7479ddbed2d042cc3a5155e".
The leading zeros seem to be the reason why I fail to match byte arrays. Why do they occur? How can I get the same result using crypt.digest() and toByteArray?
The reason for the extra 00 is that e6 has it high (sign) bit set.
A redundant byte 00 makes it an unsigned value for BigInteger.
String hash = "e6fb06210fafc02fd7479ddbed2d042cc3a5155e";
byte[] hashBytes = new BigInteger(hash, 16).toByteArray();
hashBytes = hashBytes.length > 1 && hashBytes[0] == 0
? Arrays.copyOfRange(hashBytes, 1, hashBytes.length) : hashBytes;
System.out.println(Arrays.toString(hashBytes));
The question arises, what if the hash actually starts with a 00?
Then you need the hash length, or do a lenient comparison.
The answer can be found in the following answer from a thread about the highly related question Convert a string representation of a hex dump to a byte array using Java?:
The issue with BigInteger is that there must be a "sign bit". If the leading byte has the high bit set then the resulting byte array has an extra 0 in the 1st position. But still +1.
– Gray Oct 28 '11 at 16:20
Since the first bit has a special meaning (indicating the sign, 0 for positive, 1 for negative), BigInteger will prefix the data with an additional 0 in case your data started with a 1 on the high bit. Otherwise it would be interpreted as negative although it was not negative to begin with.
I.e. data like
101110
is turned into
0101110
You could easily undo this manually by using Arrays.copyOfRange(data, 1, data.length) if it happens.
However, instead of fixing that code, I would suggest using one of the other solutions posted in the linked thread. They are cleaner and easier to read and maintain.
I have a byte array, which is the hash of a file. This is made with messageDigest, so there is a padding. Then I make a shorthash, which is just the two first bytes of the hash, like this:
byte[] shorthash = new byte[2];
System.arraycopy(hash, 0, shortHash, 0, 2);
To make it readable for the user and to save it in a DB, I'm converting it to String with a Base64 Encoder:
Base64.getUrlEncoder().encodeToString(hash); //Same for shorthash
What I don't understand is:
Why is the String representing my shorthash four characters long? I thought a char was one or two bytes, so since I'm copying only two bytes, I shouldn't have more than two chars, right?
Why isn't my shorthash String the same as the start of the hash String?
For example, I'll have :
Hash: LE5D8vCsMp3Lcf-RBwBRbO1v4soGq7BBZ9kB_2SJnGY=
Shorthash: Rak=
You can see the = at the end of each; it certainly comes from the MessageDigest padding, so it is normal for the hash, but why for the shorthash? It should be the two FIRST bytes, and the = is at the end!
Moreover: since I wanted to get rid of this Padding, I decided to do that:
String finalHash = Base64.getUrlEncoder().withoutPadding().encodeToString(hash);
byte[] shorthash = new byte[2];
System.arraycopy(hash.getBytes(), 0, shortHash, 0, 2);
String finalShorthash = Base64.getUrlEncoder().encodeToString(shorthash);
I didn't wanted to copy directly the String, since, I'm not really sure what would be two bytes in a string.
Then, the = is gone for my hash, but not for my shorthash. I guess I need to add the "withoutPadding" option to my shorthash, but I don't understand why, since it's a copy of my hash who shouldn't have padding anymore. Except if the padding is gone only on the String representation and not in the Byte behind it?
Can someone explain this behavior? Does it comes from the conversion between byte[] and String?
"Why is the String representing my shorthash four characters long?"
Because you base64 encoded it. Each base64 digit represents exactly 6 bits of data. You have 16 bits. 2 digits is not enough (just 12 bits), so you need 3 digits to represent those bits. The 4th digit is padding, because base64 usually gets normalized to be a multiple of 4 digits.
I have written a method to convert a plain text into it's hashcode using MD5 algorithm. Please find the code below which I used.
public static String convertToMD5Hash(final String plainText){
MessageDigest messageDigest = null;
try {
messageDigest = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
LOGGER.warn("For some wierd reason the MD5 algorithm was not found.", e);
}
messageDigest.reset();
messageDigest.update(plainText.getBytes());
final byte[] digest = messageDigest.digest();
final BigInteger bigInt = new BigInteger(1, digest);
String hashtext = bigInt.toString(8);
return hashtext;
}
This method works perfectly but it returns a lengthy hash. I need to limit this hash text to 8 characters. Is there any possibilities to set the length of the hashcodes in Java?
Yes and No. You can use a substring of the original hash if you always cut the original hash-string similary (ie. 8 last/first characters). What are you going to do with that "semi-hash" is another thing.
Whatever it is you're going to do, be sure it has nothing to do with security.
Here's why: MD5 is 128-bit hash, so there's 2^128 = ~340,000,000,000,000,000,000,000,000,000,000,000,000 possible permutations. The quite astronomical amount of permutations is the thing that makes bruteforcing this kind of string virtually impossible. By cutting down to 8 characters, you'll end up with 32-bit hash. This is because a single hex-value takes 4 bits to represent (thus, also 128-bit / 4 bit = 32 hex-values). With 32-bit hash there's only 2^32 = 4,294,967,296 combinations. That's about 79,228,162,514,264,337,593,543,950,336 times less secure than original 128-bit hash and can be broken in matter of seconds with any old computer that has processing power of an 80's calculator.
No. MD5 is defined to return 128 bit values. You could use Base64 to encode them to ASCII and truncate it using String#substring(0, 8).
In Java 8 (not officially released yet), you can encode a byte[] to Base64 as follows:
String base64 = Base64.getEncoder().encodeToString(digest);
For earlier Java versions see Decode Base64 data in Java
all hash algorithms should randomly change bits in whole hash whenever any part of data has changed. so you can just choose 8 chars from your hash. just don't pick them randomly - it must be reproducible
Firstly as everyone has mentioned, the 64 bit hash is not secure enough. Ultimately it depends on what you exactly plan to do with the hash.
If you still need to convert this to 8 characters, I suggest downcasting the BigInteger to a Long value using BigIteger.longValue()
It will ensure that the long value it produces is consistent with the hash that was produced.
I am not sure if taking most significant 64 bits from the 128 bit hash is good idea. I would rather take least significant 64 bits. What this ensures is that
when hash(128, a) = hash(128, b) then hash(64, a) = hash(64, b) will always be true.
But we have to live with collision in case of 64 bits i.e. when hash(64, a) = hash(64, b) then hash(128, a) = hash(128, b) is not always true.
In a nutshell, we ensure that we do not have a case where 128 bit hashes of 2 texts are different, but their 64 bit hashes are same. It depends on what you really use the hash for, but I personally feel this approach is more correct.
I want to encrypt an integer with DES, the resultant cipher text should also be an integer.
Decryption function should follow this notion as well.
I am trying to modifying the code at Encrypting a String with DES, by converting the byte array to integer, instead of using Base64 encoding. However the decryption function throws an exception of improper padding, as the conversion of integer to byte[] results in a 4 byte array.
Is there any other encryption algorithm that I can use to achieve this.
I am not concerned about the weakness of the cipher text.
If you are running an Integer value through DES to produce another Integer value, and you don't care about the cipher text weakness, then you are merely doing a very expensive hashing operation. You'd be better off generating a random integer as the key and bitwise xor-ing the exponent and the random integer. This would take nanoseconds to compute and have exactly the same security.
DES has a 64 bit blocksize, so in general the output from the encryption of a 32 bit int will be a 64 bit block. It will be easier to encrypt a 64 bit long to another 64 bit long. Use ECB mode so that padding is not an issue, or at least you are adding zero bits to the front of your int to extend it to 64 bits.
If you merely want to smush up your int then Jim's suggestion is excellent.
You want to look at Format Perserving Encryption. There are a couple of techniques for it, but in general all of them will generate a value in the same domain as your input ( i.e. integers, credit card numbers,etc)
I got the following method body to generate a key for encryption:
new BigInteger(500, new SecureRandom()).toString(64);
Could anyone explain what is the size of generated key?
It's a secure random number with a length of 500 bit in your case. Have a look at the javadoc of the BigInteger(int numBits, Random rnd) constructor.
Your line of code creates a 500 bit integer and apparently tries to convert it to a String in Base64 - that's the toString() call. But that won't work, because BigInteger.toString() only works up to base 36 and defaults to decimal otherwise. If you need a Base64 representation, you have to use a third-party class, as there is AFAIK no Base64 encoder in the standard API.
Normally you would want your encryption key to be a power of 2. So perhaps you mean 512 bits?
First, as other suggested, you will get IllegalArgumentException because BigInteger doesn't support radix 64.
Even if you use a valid radix, the number of characters generated varies because BigInteger strips leading 0s and you might also get minus sign in the string.
To get random keys, just use random bytes directly. Say you want 128-bit (16 bytes) AES key, just do this,
byte[] keyBytes = new byte[16];
new SecureRandom().nextBytes(keyBytes);
SecretKey aesKey = new SecretKeySpec(keyBytes, "AES");