I have been experimenting with using UUIDs as database keys. I want to take up the least amount of bytes as possible, while still keeping the UUID representation human readable.
I think that I have gotten it down to 22 bytes using base64 and removing some trailing "==" that seem to be unnecessary to store for my purposes. Are there any flaws with this approach?
Basically my test code does a bunch of conversions to get the UUID down to a 22 byte String, then converts it back into a UUID.
import java.io.IOException;
import java.util.UUID;
public class UUIDTest {
public static void main(String[] args){
UUID uuid = UUID.randomUUID();
System.out.println("UUID String: " + uuid.toString());
System.out.println("Number of Bytes: " + uuid.toString().getBytes().length);
System.out.println();
byte[] uuidArr = asByteArray(uuid);
System.out.print("UUID Byte Array: ");
for(byte b: uuidArr){
System.out.print(b +" ");
}
System.out.println();
System.out.println("Number of Bytes: " + uuidArr.length);
System.out.println();
try {
// Convert a byte array to base64 string
String s = new sun.misc.BASE64Encoder().encode(uuidArr);
System.out.println("UUID Base64 String: " +s);
System.out.println("Number of Bytes: " + s.getBytes().length);
System.out.println();
String trimmed = s.split("=")[0];
System.out.println("UUID Base64 String Trimmed: " +trimmed);
System.out.println("Number of Bytes: " + trimmed.getBytes().length);
System.out.println();
// Convert base64 string to a byte array
byte[] backArr = new sun.misc.BASE64Decoder().decodeBuffer(trimmed);
System.out.print("Back to UUID Byte Array: ");
for(byte b: backArr){
System.out.print(b +" ");
}
System.out.println();
System.out.println("Number of Bytes: " + backArr.length);
byte[] fixedArr = new byte[16];
for(int i= 0; i<16; i++){
fixedArr[i] = backArr[i];
}
System.out.println();
System.out.print("Fixed UUID Byte Array: ");
for(byte b: fixedArr){
System.out.print(b +" ");
}
System.out.println();
System.out.println("Number of Bytes: " + fixedArr.length);
System.out.println();
UUID newUUID = toUUID(fixedArr);
System.out.println("UUID String: " + newUUID.toString());
System.out.println("Number of Bytes: " + newUUID.toString().getBytes().length);
System.out.println();
System.out.println("Equal to Start UUID? "+newUUID.equals(uuid));
if(!newUUID.equals(uuid)){
System.exit(0);
}
} catch (IOException e) {
}
}
public static byte[] asByteArray(UUID uuid) {
long msb = uuid.getMostSignificantBits();
long lsb = uuid.getLeastSignificantBits();
byte[] buffer = new byte[16];
for (int i = 0; i < 8; i++) {
buffer[i] = (byte) (msb >>> 8 * (7 - i));
}
for (int i = 8; i < 16; i++) {
buffer[i] = (byte) (lsb >>> 8 * (7 - i));
}
return buffer;
}
public static UUID toUUID(byte[] byteArray) {
long msb = 0;
long lsb = 0;
for (int i = 0; i < 8; i++)
msb = (msb << 8) | (byteArray[i] & 0xff);
for (int i = 8; i < 16; i++)
lsb = (lsb << 8) | (byteArray[i] & 0xff);
UUID result = new UUID(msb, lsb);
return result;
}
}
output:
UUID String: cdaed56d-8712-414d-b346-01905d0026fe
Number of Bytes: 36
UUID Byte Array: -51 -82 -43 109 -121 18 65 77 -77 70 1 -112 93 0 38 -2
Number of Bytes: 16
UUID Base64 String: za7VbYcSQU2zRgGQXQAm/g==
Number of Bytes: 24
UUID Base64 String Trimmed: za7VbYcSQU2zRgGQXQAm/g
Number of Bytes: 22
Back to UUID Byte Array: -51 -82 -43 109 -121 18 65 77 -77 70 1 -112 93 0 38 -2 0 38
Number of Bytes: 18
Fixed UUID Byte Array: -51 -82 -43 109 -121 18 65 77 -77 70 1 -112 93 0 38 -2
Number of Bytes: 16
UUID String: cdaed56d-8712-414d-b346-01905d0026fe
Number of Bytes: 36
Equal to Start UUID? true
I was also trying to do something similar. I am working with a Java application which uses UUIDs of the form 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8 (which are generated with the standard UUID lib in Java). In my case I needed to be able to get this UUID down to 30 characters or less. I used Base64 and these are my convenience functions. Hopefully they will be helpful for someone as the solution was not obvious to me right away.
Usage:
String uuid_str = "6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8";
String uuid_as_64 = uuidToBase64(uuid_str);
System.out.println("as base64: "+uuid_as_64);
System.out.println("as uuid: "+uuidFromBase64(uuid_as_64));
Output:
as base64: b8tRS7h4TJ2Vt43Dp85v2A
as uuid : 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
Functions:
import org.apache.commons.codec.binary.Base64;
private static String uuidToBase64(String str) {
Base64 base64 = new Base64();
UUID uuid = UUID.fromString(str);
ByteBuffer bb = ByteBuffer.wrap(new byte[16]);
bb.putLong(uuid.getMostSignificantBits());
bb.putLong(uuid.getLeastSignificantBits());
return base64.encodeBase64URLSafeString(bb.array());
}
private static String uuidFromBase64(String str) {
Base64 base64 = new Base64();
byte[] bytes = base64.decodeBase64(str);
ByteBuffer bb = ByteBuffer.wrap(bytes);
UUID uuid = new UUID(bb.getLong(), bb.getLong());
return uuid.toString();
}
You can safely drop the padding "==" in this application. If you were to decode the base-64 text back to bytes, some libraries would expect it to be there, but since you are just using the resulting string as a key, it's not a problem.
I'd use Base-64 because its encoding characters can be URL-safe, and it looks less like gibberish. But there's also Base-85. It uses more symbols and codes 4 bytes as 5 characters, so you could get your text down to 20 characters.
Here's my code, it uses org.apache.commons.codec.binary.Base64 to produce url-safe unique strings that are 22 characters in length (and that have the same uniqueness as UUID).
private static Base64 BASE64 = new Base64(true);
public static String generateKey(){
UUID uuid = UUID.randomUUID();
byte[] uuidArray = KeyGenerator.toByteArray(uuid);
byte[] encodedArray = BASE64.encode(uuidArray);
String returnValue = new String(encodedArray);
returnValue = StringUtils.removeEnd(returnValue, "\r\n");
return returnValue;
}
public static UUID convertKey(String key){
UUID returnValue = null;
if(StringUtils.isNotBlank(key)){
// Convert base64 string to a byte array
byte[] decodedArray = BASE64.decode(key);
returnValue = KeyGenerator.fromByteArray(decodedArray);
}
return returnValue;
}
private static byte[] toByteArray(UUID uuid) {
byte[] byteArray = new byte[(Long.SIZE / Byte.SIZE) * 2];
ByteBuffer buffer = ByteBuffer.wrap(byteArray);
LongBuffer longBuffer = buffer.asLongBuffer();
longBuffer.put(new long[] { uuid.getMostSignificantBits(), uuid.getLeastSignificantBits() });
return byteArray;
}
private static UUID fromByteArray(byte[] bytes) {
ByteBuffer buffer = ByteBuffer.wrap(bytes);
LongBuffer longBuffer = buffer.asLongBuffer();
return new UUID(longBuffer.get(0), longBuffer.get(1));
}
I have an application where I'm doing almost exactly this. 22 char encoded UUID. It works fine. However, the main reason I'm doing it this way is that the IDs are exposed in the web app's URIs, and 36 characters is really quite big for something that appears in a URI. 22 characters is still kinda long, but we make do.
Here's the Ruby code for this:
# Make an array of 64 URL-safe characters
CHARS64 = ("a".."z").to_a + ("A".."Z").to_a + ("0".."9").to_a + ["-", "_"]
# Return a 22 byte URL-safe string, encoded six bits at a time using 64 characters
def to_s22
integer = self.to_i # UUID as a raw integer
rval = ""
22.times do
c = (integer & 0x3F)
rval += CHARS64[c]
integer = integer >> 6
end
return rval.reverse
end
It's not exactly the same as base64 encoding because base64 uses characters that would have to be escaped if they appeared in a URI path component. The Java implementation is likely to be quite different since you're more likely to have an array of raw bytes instead of a really big integer.
Here is an example with java.util.Base64 introduced in JDK8:
import java.nio.ByteBuffer;
import java.util.Base64;
import java.util.Base64.Encoder;
import java.util.UUID;
public class Uuid64 {
private static final Encoder BASE64_URL_ENCODER = Base64.getUrlEncoder().withoutPadding();
public static void main(String[] args) {
// String uuidStr = UUID.randomUUID().toString();
String uuidStr = "eb55c9cc-1fc1-43da-9adb-d9c66bb259ad";
String uuid64 = uuidHexToUuid64(uuidStr);
System.out.println(uuid64); //=> 61XJzB_BQ9qa29nGa7JZrQ
System.out.println(uuid64.length()); //=> 22
String uuidHex = uuid64ToUuidHex(uuid64);
System.out.println(uuidHex); //=> eb55c9cc-1fc1-43da-9adb-d9c66bb259ad
}
public static String uuidHexToUuid64(String uuidStr) {
UUID uuid = UUID.fromString(uuidStr);
byte[] bytes = uuidToBytes(uuid);
return BASE64_URL_ENCODER.encodeToString(bytes);
}
public static String uuid64ToUuidHex(String uuid64) {
byte[] decoded = Base64.getUrlDecoder().decode(uuid64);
UUID uuid = uuidFromBytes(decoded);
return uuid.toString();
}
public static byte[] uuidToBytes(UUID uuid) {
ByteBuffer bb = ByteBuffer.wrap(new byte[16]);
bb.putLong(uuid.getMostSignificantBits());
bb.putLong(uuid.getLeastSignificantBits());
return bb.array();
}
public static UUID uuidFromBytes(byte[] decoded) {
ByteBuffer bb = ByteBuffer.wrap(decoded);
long mostSigBits = bb.getLong();
long leastSigBits = bb.getLong();
return new UUID(mostSigBits, leastSigBits);
}
}
The UUID encoded in Base64 is URL safe and without padding.
You don't say what DBMS you're using, but it seems that RAW would be the best approach if you're concerned about saving space. You just need to remember to convert for all queries, or you'll risk a huge performance drop.
But I have to ask: are bytes really that expensive where you live?
This is not exactly what you asked for (it isn't Base64), but worth looking at, because of added flexibility: there is a Clojure library that implements a compact 26-char URL-safe representation of UUIDs (https://github.com/tonsky/compact-uuids).
Some highlights:
Produces strings that are 30% smaller (26 chars vs traditional 36 chars)
Supports full UUID range (128 bits)
Encoding-safe (uses only readable characters from ASCII)
URL/file-name safe
Lowercase/uppercase safe
Avoids ambiguous characters (i/I/l/L/1/O/o/0)
Alphabetical sort on encoded 26-char strings matches default UUID sort order
These are rather nice properties. I've been using this encoding in my applications both for database keys and for user-visible identifiers, and it works very well.
The codecs Base64Codec and Base64UrlCodec can encode UUIDs efficiently to base-64 and base-64-url.
// Returns a base-64 string
// input:: 01234567-89AB-4DEF-A123-456789ABCDEF
// output: ASNFZ4mrTe+hI0VniavN7w
String string = Base64Codec.INSTANCE.encode(uuid);
// Returns a base-64-url string
// input:: 01234567-89AB-4DEF-A123-456789ABCDEF
// output: ASNFZ4mrTe-hI0VniavN7w
String string = Base64UrlCodec.INSTANCE.encode(uuid);
There are codecs for other encodings in the same package of uuid-creator.
Below is what I use for a UUID (Comb style). It includes code for converting a uuid string or uuid type to base64. I do it per 64 bits, so I don't deal with any equal signs:
JAVA
import java.util.Calendar;
import java.util.UUID;
import org.apache.commons.codec.binary.Base64;
public class UUIDUtil{
public static UUID combUUID(){
private UUID srcUUID = UUID.randomUUID();
private java.sql.Timestamp ts = new java.sql.Timestamp(Calendar.getInstance().getTime().getTime());
long upper16OfLowerUUID = this.zeroLower48BitsOfLong( srcUUID.getLeastSignificantBits() );
long lower48Time = UUIDUtil.zeroUpper16BitsOfLong( ts );
long lowerLongForNewUUID = upper16OfLowerUUID | lower48Time;
return new UUID( srcUUID.getMostSignificantBits(), lowerLongForNewUUID );
}
public static base64URLSafeOfUUIDObject( UUID uuid ){
byte[] bytes = ByteBuffer.allocate(16).putLong(0, uuid.getLeastSignificantBits()).putLong(8, uuid.getMostSignificantBits()).array();
return Base64.encodeBase64URLSafeString( bytes );
}
public static base64URLSafeOfUUIDString( String uuidString ){
UUID uuid = UUID.fromString( uuidString );
return UUIDUtil.base64URLSafeOfUUIDObject( uuid );
}
private static long zeroLower48BitsOfLong( long longVar ){
long upper16BitMask = -281474976710656L;
return longVar & upper16BitMask;
}
private static void zeroUpper16BitsOfLong( long longVar ){
long lower48BitMask = 281474976710656L-1L;
return longVar & lower48BitMask;
}
}
Surprised no one mentioned uuidToByteArray(…) from commons-lang3.
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.12.0</version>
</dependency>
And then the code will be
import org.apache.commons.lang3.Conversion;
import java.util.*;
public static byte[] asByteArray(UUID uuid) {
return Conversion.uuidToByteArray(uuid, new byte[16], 0, 16);
}
Here's my approach in kotlin:
val uuid: UUID = UUID.randomUUID()
val uid = BaseEncoding.base64Url().encode(
ByteBuffer.allocate(16)
.putLong(uuid.mostSignificantBits)
.putLong(uuid.leastSignificantBits)
.array()
).trimEnd('=')
Related
I need to implement 256 bit AES encryption for cash flow i have c# answer but the answer is not the same,for a newbie, I am not sure if my direction is correct.
this is my code
public static void main(String[] args) {
String key = "12345678901234567890123456789012";
String hashIv = "1234567890123456";
String value = "MerchantID=3430112RespondType=JSONTimeStamp=1485232229Version=1.4MerchantOrderNo=S_1485232229Amt=40ItemDesc=UnitTest";
String result = encrypt(key, hashIv, value);
System.out.println(result);
System.out.println();
String sha256 = encrySha256("HashKey=" + key + "&" + result + "&HashIV=" + hashIv);
System.out.println(sha256.trim());
}
public static String encrypt(String hashKey, String hashIv, String text) {
try {
SecretKeySpec skeySpec = new SecretKeySpec(hashKey.getBytes("UTF-8"), "AES");
IvParameterSpec ivParameterSpec = new IvParameterSpec(hashIv.getBytes("UTF-8"));
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec, ivParameterSpec);
byte[] encrypted = cipher.doFinal((text.getBytes("UTF-8")));
String test = bytesToHex(encrypted);
return test.toLowerCase();
} catch (Exception e) {
System.out.println(e.getMessage());
}
return null;
}
public static String bytesToHex(byte[] bytes) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < bytes.length; i++) {
String hex = Integer.toHexString(bytes[i] & 0xFF);
if (hex.length() == 1) {
hex = '0' + hex;
}
sb.append(hex.toUpperCase());
}
return sb.toString();
}
public static String encrySha256(String value) {
try {
MessageDigest messageDigest = MessageDigest.getInstance("SHA-256");
messageDigest.update(value.getBytes());
byte byteBuffer[] = messageDigest.digest();
StringBuffer strHexString = new StringBuffer();
for (int i = 0; i < byteBuffer.length; i++) {
String hex = Integer.toHexString(0xff & byteBuffer[i]);
if (hex.length() == 1) {
strHexString.append('0');
}
strHexString.append(hex);
}
return strHexString.toString().toUpperCase();
} catch (Exception e) {
}
return null;
}
sample encrypt answer :
ff91c8aa01379e4de621a44e5f11f72e4d25bdb1a18242db6cef9ef07d80b0165e476fd1d
9acaa53170272c82d122961e1a0700a7427cfa1cf90db7f6d6593bbc93102a4d4b9b66d9
974c13c31a7ab4bba1d4e0790f0cbbbd7ad64c6d3c8012a601ceaa808bff70f94a8efa5a4f
984b9d41304ffd879612177c622f75f4214fa
encryptSha256 answer : EA0A6CC37F40C1EA5692E7CBB8AE097653DF3E91365E6A9CD7E91312413C7BB8
this is c# code and this is sample data
[MerchantID] => 3430112 [RespondType] => JSON [TimeStamp] => 1485232229
[Version] => 1.4 [MerchantOrderNo] => S_1485232229 [Amt] => 40 [ItemDesc] =>
UnitTest
public string EncryptAES256(string source)//加密
{
string sSecretKey = "12345678901234567890123456789012";
string iv = "1234567890123456";
byte[] sourceBytes =
AddPKCS7Padding(Encoding.UTF8.GetBytes(source), 32);
var aes = new RijndaelManaged();
aes.Key = Encoding.UTF8.GetBytes(sSecretKey);
aes.IV = Encoding.UTF8.GetBytes(iv);
aes.Mode = CipherMode.CBC;
aes.Padding = PaddingMode.None;
ICryptoTransform transform = aes.CreateEncryptor();
return ByteArrayToHex(transform.TransformFinalBlock(sourceBytes, 0,
sourceBytes.Length)).ToLower();
}
private static byte[] AddPKCS7Padding(byte[] data, int iBlockSize)
{
int iLength = data.Length;
byte cPadding = (byte)(iBlockSize - (iLength % iBlockSize));
var output = new byte[iLength + cPadding];
Buffer.BlockCopy(data, 0, output, 0, iLength);
for (var i = iLength; i < output.Length; i++)
output[i] = (byte)cPadding;
return output;
}
private static string ByteArrayToHex(byte[] barray)
{
char[] c = new char[barray.Length * 2];
byte b;
for (int i = 0; i < barray.Length; ++i)
{
b = ((byte)(barray[i] >> 4));
c[i * 2] = (char)(b > 9 ? b + 0x37 : b + 0x30);
b = ((byte)(barray[i] & 0xF));
c[i * 2 + 1] = (char)(b > 9 ? b + 0x37 : b + 0x30);
}
return new string(c);
}
The reason for the different encrypted data is that you compare different plain texts. In your Java code you encrypt the plain text
MerchantID=3430112RespondType=JSONTimeStamp=1485232229Version=1.4MerchantOrderNo=S_1485232229Amt=40ItemDesc=UnitTest
and you compare the encrypted data with your reference data
ff91c8aa01379e4de621a44e5f11f72e4d25bdb1a18242db6cef9ef07d80b0165e476fd1d9acaa53170272c82d122961e1a0700a7427cfa1cf90db7f6d6593bbc93102a4d4b9b66d9974c13c31a7ab4bba1d4e0790f0cbbbd7ad64c6d3c8012a601ceaa808bff70f94a8efa5a4f984b9d41304ffd879612177c622f75f4214fa
However, these reference data correspond to a different plain text. The latter you can easily derive by decrypting the reference data with the C# DecryptAES256-method which provides
MerchantID=3430112&RespondType=JSON&TimeStamp=1485232229&Version=1.4&MerchantOrderNo=S_1485232229&Amt=40&ItemDesc=UnitTest
Here, in contrast to the plain text in your Java code, a &-delimiter is used.
If you use the same plain text the Java encrypt- and the C# EncryptAES256-method provide the same encrypted data (same key, IV and padding supposed; for the latter see EDIT-section).
In the following testcase the plain text from your Java code is used:
encrypt("12345678901234567890123456789012", "1234567890123456", "MerchantID=3430112RespondType=JSONTimeStamp=1485232229Version=1.4MerchantOrderNo=S_1485232229Amt=40ItemDesc=UnitTest")
and
EncryptAES256("MerchantID=3430112RespondType=JSONTimeStamp=1485232229Version=1.4MerchantOrderNo=S_1485232229Amt=40ItemDesc=UnitTest")
both provide the encrypted data:
ff91c8aa01379e4de621a44e5f11f72ef45b7b9f9663d386da51af13f7f3b8f2b1ed4a3b7ac6b7783402193ea1d766e3046b6acf612d62568ccdbc475e5a14d114273735b069464dcc8281f4e5bf8486eb97d31602c3fe79cfe7140d2848413edad9d96fabf54d103f3d7a9b401c83fa5e4f17b10a280df10b3d61f23e69bbb8
which (as expected) differs from your reference data (with the exception of the first block).
EDIT
There is a second issue concerning the padding: Your C# EncryptAES256-method uses a custom padding provided by the C# AddPKCS7Padding-method which pads to a multiple of 32 bytes.
In contrast, your Java encrypt-method uses PKCS5Padding which pads to a multiple of 16 bytes.
Thus, the encrypted data of the Java encrypt- and the C# EncryptAES256-method differ if the length of the plain text is between 16 * n byte and 16 * (n + 1) - 1 byte with even n (0,2,4,...).
For odd n (1,3,5,...) the encrypted data are identical. In the example above the byte array of the plain text has 116 bytes that is n = 7 (112 <= 116 <= 127) and therefore the encrypted data are the same.
If the Java encrypt-method should use the same padding as the C# EncryptAES256-method you additionally have to implement an analogous Java-method e.g.:
private static byte[] addPKCS7Padding(byte[] data, int iBlockSize)
{
int iLength = data.length;
byte cPadding = (byte)(iBlockSize - (iLength % iBlockSize));
byte[] output = new byte[iLength + cPadding];
System.arraycopy(data, 0, output, 0, iLength);
for (int i = iLength; i < output.length; i++)
output[i] = (byte)cPadding;
return output;
}
and in the Java encrypt-method you have to replace:
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
with
Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
and also
byte[] encrypted = cipher.doFinal(text.getBytes("UTF-8"));
with
byte[] encrypted = cipher.doFinal(addPKCS7Padding(text.getBytes("UTF-8"), 32));
before:
MerchantID=3430112RespondType=JSONTimeStamp=1485232229Version=1.4MerchantOrderNo=S_1485232229Amt=40ItemDesc=UnitTest
After:
MerchantID=3430112&RespondType=JSON&TimeStamp=1485232229&Version=1.4&MerchantOrderNo=S_1485232229&Amt=40&ItemDesc=UnitTest
I only add "&" with each parameter, it's work!!!!
try it!!!
(我只加了&就成功拉，你的代碼沒問題，只是參數要加&而已)
I'm trying to convert a 32-byte string of hex characters (b8aa30d8f1d398883f0eeb5079777c42) into a 16-byte string of hex values (x'b8aa30d8f1d398883f0eeb5079777c42')
I've tried some of the suggestions I've found here.
StringBuffer sb = new StringBuffer();
for (int i = 0; i < input.length(); i++) {
String s = input.substring(i, i + 2);
byte value = Byte.valueOf(s, 16);
sb.append(value);
}
this resulted in this Exception: Value out of range. Value:"b8" Radix:16
String bin = "";
String binFragment = "";
int iHex;
hex = hex.trim();
hex = hex.replaceFirst("0x", "");
for(int i = 0; i < hex.length(); i++){
iHex = Integer.parseInt(""+hex.charAt(i),16);
binFragment = Integer.toBinaryString(iHex);
while(binFragment.length() < 4){
binFragment = "0" + binFragment;
}
bin += binFragment;
}
this doesn't handle leading 0's correctly
does anyone have a way to do this conversion?
I also tried this:
public static void convert() {
String s = "0c871eea3af7526f9b2e7054f6277984";
byte[] bytes = DatatypeConverter.parseHexBinary(s);
StringBuilder outputStringBuilder = new StringBuilder();
for (byte currentByte : bytes) {
outputStringBuilder.append((char) currentByte);
}
System.out.println(String.format("%s, %s", bytes, bytes.length));
String out = bytes.toString();
System.out.println(String.format("%s, %s", out, out.length()));
}
and got these results:
[B#f01a1e, 16
[B#f01a1e, 9
There are 16 bytes in the byte array before creating the output string, but only 9 after.
If I understand you correctly, you are trying to convert a sequence of bytes (given in hexadecimal notation as a String) to the corresponding characters.
You can use the DatatypeConverter class to convert the hex-string into an array of bytes ...
byte[] bytes = DatatypeConverter.parseHexBinary(inputString);
... and convert each byte to the character it represents:
StringBuilder outputStringBuilder = new StringBuilder();
for (byte currentByte : bytes) {
outputStringBuilder.append((char) currentByte);
}
Well, the problem with your second attempt is that you read two hex characters at a time (which is correct, since each pair of hex characters is a single byte), but your loop condition and increment are wrong.
Try this:
StringBuilder sb = new StringBuilder ();
for (int i = 0; i < input.length()/2; i+=2) {
String s = input.substring(i, i + 2);
byte value = Byte.valueOf(s, 16);
sb.append(value);
}
In addition you should also check that the input has an even number of hex characters.
as the title says, how do I do it? Its easy to convert from string -> byte -> string binary, But how do I convert back? Below is a example.
The output is :
'f' to binary: 01100110
294984
I read somewhere that I could use the Integer.parseInt but clearly that is not the case :( Or am I doing something wrong?
Thanks,
:)
public class main{
public static void main(String[] args) {
String s = "f";
byte[] bytes = s.getBytes();
StringBuilder binary = new StringBuilder();
for (byte b : bytes)
{
int val = b;
for (int i = 0; i < 8; i++)
{
binary.append((val & 128) == 0 ? 0 : 1);
val <<= 1;
}
binary.append(' ');
}
System.out.println("'" + s + "' to binary: " + binary);
System.out.println(Integer.parseInt("01100110", 2));
}
}
You can use Byte.parseByte() with a radix of 2:
byte b = Byte.parseByte(str, 2);
Using your example:
System.out.println(Byte.parseByte("01100110", 2));
102
You can parse it to an integer in base 2, and convert to a byte array.
In your example you've got 16 bits you can also use short.
short a = Short.parseShort(b, 2);
ByteBuffer bytes = ByteBuffer.allocate(2).putShort(a);
byte[] array = bytes.array();
Just in case if you need it for a Very Big String.
String b = "0110100001101001";
byte[] bval = new BigInteger(b, 2).toByteArray();
I made like this, converted a string s -> byte[] and then used Integer.toBinaryString to get binaryStringRep. I converted bianryStringRep by using Byte.parseByte to get the bianryStringRep into byte and the String(newByte[]) to get the byte[] into a String! Hope it helps others then me aswell! ^^
public class main{
public static void main(String[] args) throws UnsupportedEncodingException {
String s = "foo";
byte[] bytes = s.getBytes();
byte[] newBytes = new byte[s.getBytes().length];
for(int i = 0; i < bytes.length; i++){
String binaryStringRep = String.format("%8s", Integer.toBinaryString(bytes[i] & 0xFF)).replace(' ', '0');
byte newByte = Byte.parseByte(binaryStringRep, 2);
newBytes[i] = newByte;
}
String str = new String(newBytes, "UTF-8");
System.out.println(str);
}
}
I have many hexcodes here and I want to get them into Java without appending 0x to every entity. Like:
0102FFAB and I have to do the following:
byte[] test = {0x01, 0x02, 0xFF, 0xAB};
And I have many hexcodes which are pretty long. Is there any way to make this automatically?
You could try and put the hex codes into a string and then iterate over the string, similar to this:
String input = "0102FFAB";
byte[] bytes = new byte[input.length() / 2];
for( int i = 0; i < input.length(); i+=2)
{
bytes[i/2] = Integer.decode( "0x" + input.substring( i, i + 2 ) ).byteValue();
}
Note that this requires even length strings and it is quite a quick and dirty solution. However, it should still get you started.
You can use BigInteger to load a long hex string.
public static void main(String[] args) {
String hex = "c33b2cfca154c3a3362acfbde34782af31afb606f6806313cc0df40928662edd3ef1d630ab1b75639154d71ed490a36e5f51f6c9d270c4062e8266ad1608bdc496a70f6696fa6e7cd7078c6674188e8a49ecba71fad049a3d483ccac45d27aedfbb31d82adb8135238b858143492b1cbda2e854e735909256365a270095fc";
byte[] bytes2 = hexToBytes(hex);
for(byte b: bytes2)
System.out.printf("%02x", b & 0xFF);
}
public static byte[] hexToBytes(String hex) {
// add a 10 to the start to avoid sign issues, or an odd number of characters.
BigInteger bi2 = new BigInteger("10" +hex, 16);
byte[] bytes2 = bi2.toByteArray();
byte[] bytes = new byte[bytes2.length-1];
System.arraycopy(bytes2, 1, bytes, 0, bytes.length);
return bytes;
}
prints
0c33b2cfca154c3a3362acfbde34782af31afb606f6806313cc0df40928662edd3ef1d630ab1b75639154d71ed490a36e5f51f6c9d270c4062e8266ad1608bdc496a70f6696fa6e7cd7078c6674188e8a49ecba71fad049a3d483ccac45d27aedfbb31d82adb8135238b858143492b1cbda2e854e735909256365a270095fc
note: it handles the possibility that there is one hex value short at the start.
In Java, is there a library anywhere that will, given a byte sequence (preferable expressed as hex), translate to another byte sequence given an InputStream? For example:
InputStream input = new FileInputStream(new File(...));
OutputStream output = new FileOutputStream(new File(...));
String fromHex = "C3BEAB";
String toHex = "EAF6"
MyMagicLibrary.translate(fromHex, toHex, input, output)
So if the input file (in hex looked like)
00 00 12 18 33 C3 BE AB 00 23 C3 BE AB 00
after translation, the result would be
00 00 12 18 33 EA F6 00 23 EA F6 00
Once I did something like this (for trivially patching exe-files) using regexes. I read the whole input into a byte[] and converted into String using latin1, then did the substitution and converted back. It wasn't efficient but it didn't matter at all. You don't need regexes, simple String.replace would do.
But in your case it can be done quite simply and very efficiently:
int count = 0;
while (true) {
int n = input.read();
if (n == (fromAsByteArray[count] & 255)) {
++count;
if (count==fromAsByteArray.length) { // match found
output.write(toAsByteArray);
count = 0;
}
} else { // mismatch
output.write(fromAsByteArray, 0, count); // flush matching chars so far
count = 0;
if (n == -1) break;
output.write(n);
}
}
}
If u mean that u want to use a class whch translate from hex and to hex
here's two methods I usualluy use, u can put them inside a class and reuse it any where u want
public static String toHex(byte buf[]) {
StringBuffer strbuf = new StringBuffer(buf.length * 2);
int i;
for (i = 0; i < buf.length; i++) {
if (((int) buf[i] & 0xff) < 0x10) {
strbuf.append("0");
}
strbuf.append(Long.toString((int) buf[i] & 0xff, 16));
}
return strbuf.toString();
}
public static byte[] fromHexString(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i + 1), 16));
}
return data;
}
Actually I don't understand each line in the code, but I usually reuse them.
Since your input can have spaces then you first need to scrub your input to remove the spaces. After reading a pair of characters just use Byte.parseByte(twoCharString, 16) then use String.format to convert back to a String.
Doing it byte by byte would most likely be VERY inefficient, though easy to test. Once you get the result you want, you can tweak it by reading and parsing a whole buffer and spitting out more than one resulting byte a time, maybe 16 "byte" characters per line for formatting. It is all up to you at that point.
One way to implement this is to use IOUtils and String replace method.
public static void translate(byte[] fromHex, byte[] toHex, InputStream input, OutputStream output) throws IOException {
IOUtils.write(translate(fromHex, toHex, IOUtils.toByteArray(input)), output);
}
public static byte[] translate(byte[] fromHex, byte[] toHex, byte[] inBytes) throws UnsupportedEncodingException {
String inputText = new String(inBytes, "ISO-8859-1");
String outputText = inputText.replace(new String(fromHex, "ISO-8859-1"), new String(toHex, "ISO-8859-1"));
return outputText.getBytes("ISO-8859-1");
}