Convert byte[] to string to byte[] [duplicate] - java
I have to convert a byte array to string in Android, but my byte array contains negative values.
If I convert that string again to byte array, values I am getting are different from original byte array values.
What can I do to get proper conversion? Code I am using to do the conversion is as follows:
// Code to convert byte arr to str:
byte[] by_original = {0,1,-2,3,-4,-5,6};
String str1 = new String(by_original);
System.out.println("str1 >> "+str1);
// Code to convert str to byte arr:
byte[] by_new = str1.getBytes();
for(int i=0;i<by_new.length;i++)
System.out.println("by1["+i+"] >> "+str1);
I am stuck in this problem.
Your byte array must have some encoding. The encoding cannot be ASCII if you've got negative values. Once you figure that out, you can convert a set of bytes to a String using:
byte[] bytes = {...}
String str = new String(bytes, StandardCharsets.UTF_8); // for UTF-8 encoding
There are a bunch of encodings you can use, look at the supported encodings in the Oracle javadocs.
The "proper conversion" between byte[] and String is to explicitly state the encoding you want to use. If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to.
If you really must use a String to hold binary data then the safest way is to use Base64 encoding.
The root problem is (I think) that you are unwittingly using a character set for which:
bytes != encode(decode(bytes))
in some cases. UTF-8 is an example of such a character set. Specifically, certain sequences of bytes are not valid encodings in UTF-8. If the UTF-8 decoder encounters one of these sequences, it is liable to discard the offending bytes or decode them as the Unicode codepoint for "no such character". Naturally, when you then try to encode the characters as bytes the result will be different.
The solution is:
Be explicit about the character encoding you are using; i.e. use a String constructor and String.toByteArray method with an explicit charset.
Use the right character set for your byte data ... or alternatively one (such as "Latin-1" where all byte sequences map to valid Unicode characters.
If your bytes are (really) binary data and you want to be able to transmit / receive them over a "text based" channel, use something like Base64 encoding ... which is designed for this purpose.
For Java, the most common character sets are in java.nio.charset.StandardCharsets. If you are encoding a string that can contain any Unicode character value then UTF-8 encoding (UTF_8) is recommended.
If you want a 1:1 mapping in Java then you can use ISO Latin Alphabet No. 1 - more commonly just called "Latin 1" or simply "Latin" (ISO_8859_1). Note that Latin-1 in Java is the IANA version of Latin-1 which assigns characters to all possible 256 values including control blocks C0 and C1. These are not printable: you won't see them in any output.
From Java 8 onwards Java contains java.util.Base64 for Base64 encoding / decoding. For URL-safe encoding you may want to to use Base64.getUrlEncoder instead of the standard encoder. This class is also present in Android since Android Oreo (8), API level 26.
We just need to construct a new String with the array: http://www.mkyong.com/java/how-do-convert-byte-array-to-string-in-java/
String s = new String(bytes);
The bytes of the resulting string differs depending on what charset you use. new String(bytes) and new String(bytes, Charset.forName("utf-8")) and new String(bytes, Charset.forName("utf-16")) will all have different byte arrays when you call String#getBytes() (depending on the default charset)
Using new String(byOriginal) and converting back to byte[] using getBytes() doesn't guarantee two byte[] with equal values. This is due to a call to StringCoding.encode(..) which will encode the String to Charset.defaultCharset(). During this encoding, the encoder might choose to replace unknown characters and do other changes. Hence, using String.getBytes() might not return an equal array as you've originally passed to the constructor.
Why was the problem: As someone already specified:
If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to.
I was observing this problem when I was trying to create byte[] from a pdf file and then converting it to String and then taking the String as input and converting back to file.
So make sure your encoding and decoding logic is same as I did. I explicitly encoded the byte[] to Base64 and decoded it to create the file again.
Use-case:
Due to some limitation I was trying to sent byte[] in request(POST) and the process was as follows:
PDF File >> Base64.encodeBase64(byte[]) >> String >> Send in request(POST) >> receive String >> Base64.decodeBase64(byte[]) >> create binary
Try this and this worked for me..
File file = new File("filePath");
byte[] byteArray = new byte[(int) file.length()];
try {
FileInputStream fileInputStream = new FileInputStream(file);
fileInputStream.read(byteArray);
String byteArrayStr= new String(Base64.encodeBase64(byteArray));
FileOutputStream fos = new FileOutputStream("newFilePath");
fos.write(Base64.decodeBase64(byteArrayStr.getBytes()));
fos.close();
}
catch (FileNotFoundException e) {
System.out.println("File Not Found.");
e.printStackTrace();
}
catch (IOException e1) {
System.out.println("Error Reading The File.");
e1.printStackTrace();
}
Even though
new String(bytes, "UTF-8")
is correct it throws a UnsupportedEncodingException which forces you to deal with a checked exception. You can use as an alternative another constructor since Java 1.6 to convert a byte array into a String:
new String(bytes, StandardCharsets.UTF_8)
This one does not throw any exception.
Converting back should be also done with StandardCharsets.UTF_8:
"test".getBytes(StandardCharsets.UTF_8)
Again you avoid having to deal with checked exceptions.
private static String toHexadecimal(byte[] digest){
String hash = "";
for(byte aux : digest) {
int b = aux & 0xff;
if (Integer.toHexString(b).length() == 1) hash += "0";
hash += Integer.toHexString(b);
}
return hash;
}
I did notice something that is not in any of the answers. You can cast each of the bytes in the byte array to characters, and put them in a char array. Then the string is new String(cbuf) where cbuf is the char array. To convert back, loop through the string casting each of the chars to bytes to put into a byte array, and this byte array will be the same as the first.
public class StringByteArrTest {
public static void main(String[] args) {
// put whatever byte array here
byte[] arr = new byte[] {-12, -100, -49, 100, -63, 0, -90};
for (byte b: arr) System.out.println(b);
// put data into this char array
char[] cbuf = new char[arr.length];
for (int i = 0; i < arr.length; i++) {
cbuf[i] = (char) arr[i];
}
// this is the string
String s = new String(cbuf);
System.out.println(s);
// converting back
byte[] out = new byte[s.length()];
for (int i = 0; i < s.length(); i++) {
out[i] = (byte) s.charAt(i);
}
for (byte b: out) System.out.println(b);
}
}
This works fine for me:
String cd = "Holding some value";
Converting from string to byte[]:
byte[] cookie = new sun.misc.BASE64Decoder().decodeBuffer(cd);
Converting from byte[] to string:
cd = new sun.misc.BASE64Encoder().encode(cookie);
Following is the sample code safely converts byte array to String and String to byte array back.
byte bytesArray[] = { 1, -2, 4, -5, 10};
String encoded = java.util.Base64.getEncoder().encodeToString(bytesArray);
byte[] decoded = java.util.Base64.getDecoder().decode(encoded);
System.out.println("input: "+Arrays.toString(bytesArray));
System.out.println("encoded: "+encoded);
System.out.println("decoded: "+Arrays.toString(decoded));
Output:
input: [1, -2, 4, -5, 10]
encoded: Af4E+wo=
decoded: [1, -2, 4, -5, 10]
javax.xml.bind.DatatypeConverter should do it:
byte [] b = javax.xml.bind.DatatypeConverter.parseHexBinary("E62DB");
String s = javax.xml.bind.DatatypeConverter.printHexBinary(b);
byte[] bytes = "Techie Delight".getBytes();
// System.out.println(Arrays.toString(bytes));
// Create a string from the byte array without specifying
// character encoding
String string = new String(bytes);
System.out.println(string);
Heres a few methods that convert an array of bytes to a string. I've tested them they work well.
public String getStringFromByteArray(byte[] settingsData) {
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(settingsData);
Reader reader = new BufferedReader(new InputStreamReader(byteArrayInputStream));
StringBuilder sb = new StringBuilder();
int byteChar;
try {
while((byteChar = reader.read()) != -1) {
sb.append((char) byteChar);
}
}
catch(IOException e) {
e.printStackTrace();
}
return sb.toString();
}
public String getStringFromByteArray(byte[] settingsData) {
StringBuilder sb = new StringBuilder();
for(byte willBeChar: settingsData) {
sb.append((char) willBeChar);
}
return sb.toString();
}
While base64 encoding is safe and one could argue "the right answer", I arrived here looking for a way to convert a Java byte array to/from a Java String as-is. That is, where each member of the byte array remains intact in its String counterpart, with no extra space required for encoding/transport.
This answer describing 8bit transparent encodings was very helpful for me. I used ISO-8859-1 on terabytes of binary data to convert back and forth successfully (binary <-> String) without the inflated space requirements needed for a base64 encoding, so is safe for my use-case - YMMV.
This was also helpful in explaining when/if you should experiment.
Using Kotlin on Android I found out it is handy to create some simple extension functions for that purpose.
Solution based on Base64 encoding/decoding to be able to pass via JSON, XML, etc:
import android.util.Base64
fun ByteArray.encodeToString() = String(Base64.encode(this, Base64.NO_WRAP), Charsets.UTF_8)
fun String.decodeToBytes(): ByteArray = Base64.decode(toByteArray(Charsets.UTF_8), Base64.NO_WRAP)
So you can use it
val byteArray = byteArrayOf(0, 1, 2, -1, -2, -3)
val string = byteArray.encodeToString()
val restoredArray = string.decodeToBytes()
import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;
private static String base64Encode(byte[] bytes)
{
return new BASE64Encoder().encode(bytes);
}
private static byte[] base64Decode(String s) throws IOException
{
return new BASE64Decoder().decodeBuffer(s);
}
I succeeded converting byte array to a string with this method:
public static String byteArrayToString(byte[] data){
String response = Arrays.toString(data);
String[] byteValues = response.substring(1, response.length() - 1).split(",");
byte[] bytes = new byte[byteValues.length];
for (int i=0, len=bytes.length; i<len; i++) {
bytes[i] = Byte.parseByte(byteValues[i].trim());
}
String str = new String(bytes);
return str.toLowerCase();
}
This one works for me up to android Q:
You can use the following method to convert o hex string to string
public static String hexToString(String hex) {
StringBuilder sb = new StringBuilder();
char[] hexData = hex.toCharArray();
for (int count = 0; count < hexData.length - 1; count += 2) {
int firstDigit = Character.digit(hexData[count], 16);
int lastDigit = Character.digit(hexData[count + 1], 16);
int decimal = firstDigit * 16 + lastDigit;
sb.append((char)decimal);
}
return sb.toString();
}
with the following to convert a byte array to a hex string
public static String bytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}
Here the working code.
// Encode byte array into string . TemplateBuffer1 is my bytearry variable.
String finger_buffer = Base64.encodeToString(templateBuffer1, Base64.DEFAULT);
Log.d(TAG, "Captured biometric device->" + finger_buffer);
// Decode String into Byte Array. decodedString is my bytearray[]
decodedString = Base64.decode(finger_buffer, Base64.DEFAULT);
You can use simple for loop for conversion:
public void byteArrToString(){
byte[] b = {'a','b','$'};
String str = "";
for(int i=0; i<b.length; i++){
char c = (char) b[i];
str+=c;
}
System.out.println(str);
}
byte[] image = {...};
String imageString = Base64.encodeToString(image, Base64.NO_WRAP);
Try to specify an 8-bit charset in both conversions. ISO-8859-1 for instance.
Read the bytes from String using ByteArrayInputStream and wrap it with BufferedReader which is Char Stream instead of Byte Stream which converts the byte data to String.
package com.cs.sajal;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
public class TestCls {
public static void main(String[] args) {
String s=new String("Sajal is a good boy");
try
{
ByteArrayInputStream bis;
bis=new ByteArrayInputStream(s.getBytes("UTF-8"));
BufferedReader br=new BufferedReader(new InputStreamReader(bis));
System.out.println(br.readLine());
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
Output is:
Sajal is a good boy
You can do the following to convert byte array to string and then convert that string to byte array:
// 1. convert byte array to string and then string to byte array
// convert byte array to string
byte[] by_original = {0, 1, -2, 3, -4, -5, 6};
String str1 = Arrays.toString(by_original);
System.out.println(str1); // output: [0, 1, -2, 3, -4, -5, 6]
// convert string to byte array
String newString = str1.substring(1, str1.length()-1);
String[] stringArray = newString.split(", ");
byte[] by_new = new byte[stringArray.length];
for(int i=0; i<stringArray.length; i++) {
by_new[i] = (byte) Integer.parseInt(stringArray[i]);
}
System.out.println(Arrays.toString(by_new)); // output: [0, 1, -2, 3, -4, -5, 6]
But to convert the string to byte array and then convert that byte array to string, below approach can be used:
// 2. convert string to byte array and then byte array to string
// convert string to byte array
String str2 = "[0, 1, -2, 3, -4, -5, 6]";
byte[] byteStr2 = str2.getBytes(StandardCharsets.UTF_8);
// Now byteStr2 is [91, 48, 44, 32, 49, 44, 32, 45, 50, 44, 32, 51, 44, 32, 45, 52, 44, 32, 45, 53, 44, 32, 54, 93]
// convert byte array to string
System.out.println(new String(byteStr2, StandardCharsets.UTF_8)); // output: [0, 1, -2, 3, -4, -5, 6]
A string is a collection of char's (16bit unsigned). So if you are going to convert negative numbers into a string, they'll be lost in translation.
public class byteString {
/**
* #param args
*/
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
String msg = "Hello";
byte[] buff = new byte[1024];
buff = msg.getBytes("UTF-8");
System.out.println(buff);
String m = new String(buff);
System.out.println(m);
}
}
Related
Sending files as byteArray to kafka using spring kafka [duplicate]
I have to convert a byte array to string in Android, but my byte array contains negative values. If I convert that string again to byte array, values I am getting are different from original byte array values. What can I do to get proper conversion? Code I am using to do the conversion is as follows: // Code to convert byte arr to str: byte[] by_original = {0,1,-2,3,-4,-5,6}; String str1 = new String(by_original); System.out.println("str1 >> "+str1); // Code to convert str to byte arr: byte[] by_new = str1.getBytes(); for(int i=0;i<by_new.length;i++) System.out.println("by1["+i+"] >> "+str1); I am stuck in this problem.
Your byte array must have some encoding. The encoding cannot be ASCII if you've got negative values. Once you figure that out, you can convert a set of bytes to a String using: byte[] bytes = {...} String str = new String(bytes, StandardCharsets.UTF_8); // for UTF-8 encoding There are a bunch of encodings you can use, look at the supported encodings in the Oracle javadocs.
The "proper conversion" between byte[] and String is to explicitly state the encoding you want to use. If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to. If you really must use a String to hold binary data then the safest way is to use Base64 encoding.
The root problem is (I think) that you are unwittingly using a character set for which: bytes != encode(decode(bytes)) in some cases. UTF-8 is an example of such a character set. Specifically, certain sequences of bytes are not valid encodings in UTF-8. If the UTF-8 decoder encounters one of these sequences, it is liable to discard the offending bytes or decode them as the Unicode codepoint for "no such character". Naturally, when you then try to encode the characters as bytes the result will be different. The solution is: Be explicit about the character encoding you are using; i.e. use a String constructor and String.toByteArray method with an explicit charset. Use the right character set for your byte data ... or alternatively one (such as "Latin-1" where all byte sequences map to valid Unicode characters. If your bytes are (really) binary data and you want to be able to transmit / receive them over a "text based" channel, use something like Base64 encoding ... which is designed for this purpose. For Java, the most common character sets are in java.nio.charset.StandardCharsets. If you are encoding a string that can contain any Unicode character value then UTF-8 encoding (UTF_8) is recommended. If you want a 1:1 mapping in Java then you can use ISO Latin Alphabet No. 1 - more commonly just called "Latin 1" or simply "Latin" (ISO_8859_1). Note that Latin-1 in Java is the IANA version of Latin-1 which assigns characters to all possible 256 values including control blocks C0 and C1. These are not printable: you won't see them in any output. From Java 8 onwards Java contains java.util.Base64 for Base64 encoding / decoding. For URL-safe encoding you may want to to use Base64.getUrlEncoder instead of the standard encoder. This class is also present in Android since Android Oreo (8), API level 26.
We just need to construct a new String with the array: http://www.mkyong.com/java/how-do-convert-byte-array-to-string-in-java/ String s = new String(bytes); The bytes of the resulting string differs depending on what charset you use. new String(bytes) and new String(bytes, Charset.forName("utf-8")) and new String(bytes, Charset.forName("utf-16")) will all have different byte arrays when you call String#getBytes() (depending on the default charset)
Using new String(byOriginal) and converting back to byte[] using getBytes() doesn't guarantee two byte[] with equal values. This is due to a call to StringCoding.encode(..) which will encode the String to Charset.defaultCharset(). During this encoding, the encoder might choose to replace unknown characters and do other changes. Hence, using String.getBytes() might not return an equal array as you've originally passed to the constructor.
Why was the problem: As someone already specified: If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to. I was observing this problem when I was trying to create byte[] from a pdf file and then converting it to String and then taking the String as input and converting back to file. So make sure your encoding and decoding logic is same as I did. I explicitly encoded the byte[] to Base64 and decoded it to create the file again. Use-case: Due to some limitation I was trying to sent byte[] in request(POST) and the process was as follows: PDF File >> Base64.encodeBase64(byte[]) >> String >> Send in request(POST) >> receive String >> Base64.decodeBase64(byte[]) >> create binary Try this and this worked for me.. File file = new File("filePath"); byte[] byteArray = new byte[(int) file.length()]; try { FileInputStream fileInputStream = new FileInputStream(file); fileInputStream.read(byteArray); String byteArrayStr= new String(Base64.encodeBase64(byteArray)); FileOutputStream fos = new FileOutputStream("newFilePath"); fos.write(Base64.decodeBase64(byteArrayStr.getBytes())); fos.close(); } catch (FileNotFoundException e) { System.out.println("File Not Found."); e.printStackTrace(); } catch (IOException e1) { System.out.println("Error Reading The File."); e1.printStackTrace(); }
Even though new String(bytes, "UTF-8") is correct it throws a UnsupportedEncodingException which forces you to deal with a checked exception. You can use as an alternative another constructor since Java 1.6 to convert a byte array into a String: new String(bytes, StandardCharsets.UTF_8) This one does not throw any exception. Converting back should be also done with StandardCharsets.UTF_8: "test".getBytes(StandardCharsets.UTF_8) Again you avoid having to deal with checked exceptions.
private static String toHexadecimal(byte[] digest){ String hash = ""; for(byte aux : digest) { int b = aux & 0xff; if (Integer.toHexString(b).length() == 1) hash += "0"; hash += Integer.toHexString(b); } return hash; }
I did notice something that is not in any of the answers. You can cast each of the bytes in the byte array to characters, and put them in a char array. Then the string is new String(cbuf) where cbuf is the char array. To convert back, loop through the string casting each of the chars to bytes to put into a byte array, and this byte array will be the same as the first. public class StringByteArrTest { public static void main(String[] args) { // put whatever byte array here byte[] arr = new byte[] {-12, -100, -49, 100, -63, 0, -90}; for (byte b: arr) System.out.println(b); // put data into this char array char[] cbuf = new char[arr.length]; for (int i = 0; i < arr.length; i++) { cbuf[i] = (char) arr[i]; } // this is the string String s = new String(cbuf); System.out.println(s); // converting back byte[] out = new byte[s.length()]; for (int i = 0; i < s.length(); i++) { out[i] = (byte) s.charAt(i); } for (byte b: out) System.out.println(b); } }
This works fine for me: String cd = "Holding some value"; Converting from string to byte[]: byte[] cookie = new sun.misc.BASE64Decoder().decodeBuffer(cd); Converting from byte[] to string: cd = new sun.misc.BASE64Encoder().encode(cookie);
Following is the sample code safely converts byte array to String and String to byte array back. byte bytesArray[] = { 1, -2, 4, -5, 10}; String encoded = java.util.Base64.getEncoder().encodeToString(bytesArray); byte[] decoded = java.util.Base64.getDecoder().decode(encoded); System.out.println("input: "+Arrays.toString(bytesArray)); System.out.println("encoded: "+encoded); System.out.println("decoded: "+Arrays.toString(decoded)); Output: input: [1, -2, 4, -5, 10] encoded: Af4E+wo= decoded: [1, -2, 4, -5, 10]
javax.xml.bind.DatatypeConverter should do it: byte [] b = javax.xml.bind.DatatypeConverter.parseHexBinary("E62DB"); String s = javax.xml.bind.DatatypeConverter.printHexBinary(b);
byte[] bytes = "Techie Delight".getBytes(); // System.out.println(Arrays.toString(bytes)); // Create a string from the byte array without specifying // character encoding String string = new String(bytes); System.out.println(string);
Heres a few methods that convert an array of bytes to a string. I've tested them they work well. public String getStringFromByteArray(byte[] settingsData) { ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(settingsData); Reader reader = new BufferedReader(new InputStreamReader(byteArrayInputStream)); StringBuilder sb = new StringBuilder(); int byteChar; try { while((byteChar = reader.read()) != -1) { sb.append((char) byteChar); } } catch(IOException e) { e.printStackTrace(); } return sb.toString(); } public String getStringFromByteArray(byte[] settingsData) { StringBuilder sb = new StringBuilder(); for(byte willBeChar: settingsData) { sb.append((char) willBeChar); } return sb.toString(); }
While base64 encoding is safe and one could argue "the right answer", I arrived here looking for a way to convert a Java byte array to/from a Java String as-is. That is, where each member of the byte array remains intact in its String counterpart, with no extra space required for encoding/transport. This answer describing 8bit transparent encodings was very helpful for me. I used ISO-8859-1 on terabytes of binary data to convert back and forth successfully (binary <-> String) without the inflated space requirements needed for a base64 encoding, so is safe for my use-case - YMMV. This was also helpful in explaining when/if you should experiment.
Using Kotlin on Android I found out it is handy to create some simple extension functions for that purpose. Solution based on Base64 encoding/decoding to be able to pass via JSON, XML, etc: import android.util.Base64 fun ByteArray.encodeToString() = String(Base64.encode(this, Base64.NO_WRAP), Charsets.UTF_8) fun String.decodeToBytes(): ByteArray = Base64.decode(toByteArray(Charsets.UTF_8), Base64.NO_WRAP) So you can use it val byteArray = byteArrayOf(0, 1, 2, -1, -2, -3) val string = byteArray.encodeToString() val restoredArray = string.decodeToBytes()
import sun.misc.BASE64Decoder; import sun.misc.BASE64Encoder; private static String base64Encode(byte[] bytes) { return new BASE64Encoder().encode(bytes); } private static byte[] base64Decode(String s) throws IOException { return new BASE64Decoder().decodeBuffer(s); }
I succeeded converting byte array to a string with this method: public static String byteArrayToString(byte[] data){ String response = Arrays.toString(data); String[] byteValues = response.substring(1, response.length() - 1).split(","); byte[] bytes = new byte[byteValues.length]; for (int i=0, len=bytes.length; i<len; i++) { bytes[i] = Byte.parseByte(byteValues[i].trim()); } String str = new String(bytes); return str.toLowerCase(); }
This one works for me up to android Q: You can use the following method to convert o hex string to string public static String hexToString(String hex) { StringBuilder sb = new StringBuilder(); char[] hexData = hex.toCharArray(); for (int count = 0; count < hexData.length - 1; count += 2) { int firstDigit = Character.digit(hexData[count], 16); int lastDigit = Character.digit(hexData[count + 1], 16); int decimal = firstDigit * 16 + lastDigit; sb.append((char)decimal); } return sb.toString(); } with the following to convert a byte array to a hex string public static String bytesToHex(byte[] bytes) { char[] hexChars = new char[bytes.length * 2]; for (int j = 0; j < bytes.length; j++) { int v = bytes[j] & 0xFF; hexChars[j * 2] = hexArray[v >>> 4]; hexChars[j * 2 + 1] = hexArray[v & 0x0F]; } return new String(hexChars); }
Here the working code. // Encode byte array into string . TemplateBuffer1 is my bytearry variable. String finger_buffer = Base64.encodeToString(templateBuffer1, Base64.DEFAULT); Log.d(TAG, "Captured biometric device->" + finger_buffer); // Decode String into Byte Array. decodedString is my bytearray[] decodedString = Base64.decode(finger_buffer, Base64.DEFAULT);
You can use simple for loop for conversion: public void byteArrToString(){ byte[] b = {'a','b','$'}; String str = ""; for(int i=0; i<b.length; i++){ char c = (char) b[i]; str+=c; } System.out.println(str); }
byte[] image = {...}; String imageString = Base64.encodeToString(image, Base64.NO_WRAP);
Try to specify an 8-bit charset in both conversions. ISO-8859-1 for instance.
Read the bytes from String using ByteArrayInputStream and wrap it with BufferedReader which is Char Stream instead of Byte Stream which converts the byte data to String. package com.cs.sajal; import java.io.BufferedReader; import java.io.ByteArrayInputStream; import java.io.InputStreamReader; import java.io.UnsupportedEncodingException; public class TestCls { public static void main(String[] args) { String s=new String("Sajal is a good boy"); try { ByteArrayInputStream bis; bis=new ByteArrayInputStream(s.getBytes("UTF-8")); BufferedReader br=new BufferedReader(new InputStreamReader(bis)); System.out.println(br.readLine()); } catch(Exception e) { e.printStackTrace(); } } } Output is: Sajal is a good boy
You can do the following to convert byte array to string and then convert that string to byte array: // 1. convert byte array to string and then string to byte array // convert byte array to string byte[] by_original = {0, 1, -2, 3, -4, -5, 6}; String str1 = Arrays.toString(by_original); System.out.println(str1); // output: [0, 1, -2, 3, -4, -5, 6] // convert string to byte array String newString = str1.substring(1, str1.length()-1); String[] stringArray = newString.split(", "); byte[] by_new = new byte[stringArray.length]; for(int i=0; i<stringArray.length; i++) { by_new[i] = (byte) Integer.parseInt(stringArray[i]); } System.out.println(Arrays.toString(by_new)); // output: [0, 1, -2, 3, -4, -5, 6] But to convert the string to byte array and then convert that byte array to string, below approach can be used: // 2. convert string to byte array and then byte array to string // convert string to byte array String str2 = "[0, 1, -2, 3, -4, -5, 6]"; byte[] byteStr2 = str2.getBytes(StandardCharsets.UTF_8); // Now byteStr2 is [91, 48, 44, 32, 49, 44, 32, 45, 50, 44, 32, 51, 44, 32, 45, 52, 44, 32, 45, 53, 44, 32, 54, 93] // convert byte array to string System.out.println(new String(byteStr2, StandardCharsets.UTF_8)); // output: [0, 1, -2, 3, -4, -5, 6]
A string is a collection of char's (16bit unsigned). So if you are going to convert negative numbers into a string, they'll be lost in translation.
public class byteString { /** * #param args */ public static void main(String[] args) throws Exception { // TODO Auto-generated method stub String msg = "Hello"; byte[] buff = new byte[1024]; buff = msg.getBytes("UTF-8"); System.out.println(buff); String m = new String(buff); System.out.println(m); } }
Java byte[] to/from String conversion
Why does this junit test fail? import org.junit.Assert; import org.junit.Test; import java.io.UnsupportedEncodingException; public class TestBytes { #Test public void testBytes() throws UnsupportedEncodingException { byte[] bytes = new byte[]{0, -121, -80, 116, -62}; String string = new String(bytes, "UTF-8"); byte[] bytes2 = string.getBytes("UTF-8"); System.out.print("bytes2: ["); for (byte b : bytes2) System.out.print(b + ", "); System.out.print("]\n"); Assert.assertArrayEquals(bytes, bytes2); } } I would assume that the incoming byte array equaled the outcome, but somehow, probably due to the fact that UTF-8 characters take two bytes, the outcome array differs from the incoming array in both content and length. Please enlighten me.
The reason is 0, -121, -80, 116, -62 is not a valid UTF-8 byte sequence. new String(bytes, "UTF-8") does not throw any exception in such situations but the result is difficult to predict. Read http://en.wikipedia.org/wiki/UTF-8 Invalid byte sequences section.
The array bytes contains negative noted vales, these have the 8th bit (bit7) set and are converted into UTF-8 as multibyte sequences. bytes2 will be identical to bytes if you use only bytes with values in range 0..127. To make a copy of bytes as given one may use for example the arraycopy method: byte[] bytes3 = new byte[bytes.length]; System.arraycopy(bytes, 0, bytes3, 0, bytes.length);
Problems converting byte array to string and back to byte array
There are a lot of questions with this topic, the same solution, but this doesn't work for me. I have a simple test with an encryption. The encryption/decryption itself works (as long as I handle this test with the byte array itself and not as Strings). The problem is that don't want to handle it as byte array but as String, but when I encode the byte array to string and back, the resulting byte array differs from the original byte array, so the decryption doesn't work anymore. I tried the following parameters in the corresponding string methods: UTF-8, UTF8, UTF-16, UTF8. None of them work. The resulting byte array differs from the original. Any ideas why this is so? Encrypter: public class NewEncrypter { private String algorithm = "DESede"; private Key key = null; private Cipher cipher = null; public NewEncrypter() throws NoSuchAlgorithmException, NoSuchPaddingException { key = KeyGenerator.getInstance(algorithm).generateKey(); cipher = Cipher.getInstance(algorithm); } public byte[] encrypt(String input) throws Exception { cipher.init(Cipher.ENCRYPT_MODE, key); byte[] inputBytes = input.getBytes("UTF-16"); return cipher.doFinal(inputBytes); } public String decrypt(byte[] encryptionBytes) throws Exception { cipher.init(Cipher.DECRYPT_MODE, key); byte[] recoveredBytes = cipher.doFinal(encryptionBytes); String recovered = new String(recoveredBytes, "UTF-16"); return recovered; } } This is the test where I try it: public class NewEncrypterTest { #Test public void canEncryptAndDecrypt() throws Exception { String toEncrypt = "FOOBAR"; NewEncrypter encrypter = new NewEncrypter(); byte[] encryptedByteArray = encrypter.encrypt(toEncrypt); System.out.println("encryptedByteArray:" + encryptedByteArray); String decoded = new String(encryptedByteArray, "UTF-16"); System.out.println("decoded:" + decoded); byte[] encoded = decoded.getBytes("UTF-16"); System.out.println("encoded:" + encoded); String decryptedText = encrypter.decrypt(encoded); //Exception here System.out.println("decryptedText:" + decryptedText); assertEquals(toEncrypt, decryptedText); } }
It is not a good idea to store encrypted data in Strings because they are for human-readable text, not for arbitrary binary data. For binary data it's best to use byte[]. However, if you must do it you should use an encoding that has a 1-to-1 mapping between bytes and characters, that is, where every byte sequence can be mapped to a unique sequence of characters, and back. One such encoding is ISO-8859-1, that is: String decoded = new String(encryptedByteArray, "ISO-8859-1"); System.out.println("decoded:" + decoded); byte[] encoded = decoded.getBytes("ISO-8859-1"); System.out.println("encoded:" + java.util.Arrays.toString(encoded)); String decryptedText = encrypter.decrypt(encoded); Other common encodings that don't lose data are hexadecimal and base64, but sadly you need a helper library for them. The standard API doesn't define classes for them. With UTF-16 the program would fail for two reasons: String.getBytes("UTF-16") adds a byte-order-marker character to the output to identify the order of the bytes. You should use UTF-16LE or UTF-16BE for this to not happen. Not all sequences of bytes can be mapped to characters in UTF-16. First, text encoded in UTF-16 must have an even number of bytes. Second, UTF-16 has a mechanism for encoding unicode characters beyond U+FFFF. This means that e.g. there are sequences of 4 bytes that map to only one unicode character. For this to be possible the first 2 bytes of the 4 don't encode any character in UTF-16.
Accepted solution will not work if your String has some non-typical charcaters such as š, ž, ć, Ō, ō, Ū, etc. Following code worked nicely for me. byte[] myBytes = Something.getMyBytes(); String encodedString = Base64.encodeToString(bytes, Base64.NO_WRAP); byte[] decodedBytes = Base64.decode(encodedString, Base64.NO_WRAP);
Now, I found another solution too... public class NewEncrypterTest { #Test public void canEncryptAndDecrypt() throws Exception { String toEncrypt = "FOOBAR"; NewEncrypter encrypter = new NewEncrypter(); byte[] encryptedByteArray = encrypter.encrypt(toEncrypt); String encoded = String.valueOf(Hex.encodeHex(encryptedByteArray)); byte[] byteArrayToDecrypt = Hex.decodeHex(encoded.toCharArray()); String decryptedText = encrypter.decrypt(byteArrayToDecrypt); System.out.println("decryptedText:" + decryptedText); assertEquals(toEncrypt, decryptedText); } }
Your problem is that you cannot build a UTF-16 (or any other encoding) String from an arbitrary byte array (see UTF-16 on Wikipedia). It is up to you, however, to serialize and deserialize the encrypted byte array without any loss, in order to, say, persist it, and make use of it later. Here's the modified client code that should give you some insight of what's actually happening with the byte arrays: public static void main(String[] args) throws Exception { String toEncrypt = "FOOBAR"; NewEncrypter encrypter = new NewEncrypter(); byte[] encryptedByteArray = encrypter.encrypt(toEncrypt); System.out.println("encryptedByteArray:" + Arrays.toString(encryptedByteArray)); String decoded = new String(encryptedByteArray, "UTF-16"); System.out.println("decoded:" + decoded); byte[] encoded = decoded.getBytes("UTF-16"); System.out.println("encoded:" + Arrays.toString(encoded)); String decryptedText = encrypter.decrypt(encryptedByteArray); // NOT the "encoded" value! System.out.println("decryptedText:" + decryptedText); } This is the output: encryptedByteArray:[90, -40, -39, -56, -90, 51, 96, 95, -65, -54, -61, 51, 6, 15, -114, 88] decoded:<some garbage> encoded:[-2, -1, 90, -40, -1, -3, 96, 95, -65, -54, -61, 51, 6, 15, -114, 88] decryptedText:FOOBAR The decryptedText is correct, when restored from the original encryptedByteArray. Please note that the encoded value is not the same as encryptedByteArray, due to the data loss during the byte[] -> String("UTF-16")->byte[] conversion.
Convert byte[] to String using binary encoding
I want to translate each byte from a byte[] into a char, then put those chars on a String. This is the so-called "binary" encoding of some databases. So far, the best I could find is this huge boilerplate: byte[] bytes = ...; char[] chars = new char[bytes.length]; for (int i = 0; i < bytes.length; ++i) { chars[i] = (char) (bytes[i] & 0xFF); } String s = new String(chars); Is there another option from Java SE or perhaps from Apache Commons? I wish I could have something like this: final Charset BINARY_CS = Charset.forName("BINARY"); String s = new String(bytes, BINARY_CS); But I'm not willing to write a Charset and their codecs (yet). Is there such a ready binary Charset in JRE or in Apache Commons?
You could use the ASCII encoding for 7-bit characters String s = "Hello World!"; byte[] b = s.getBytes("ASCII"); System.out.println(new String(b, "ASCII")); or 8-bit ascii String s = "Hello World! \u00ff"; byte[] b = s.getBytes("ISO-8859-1"); System.out.println(new String(b, "ISO-8859-1")); EDIT System.out.println("ASCII => " + Charset.forName("ASCII")); System.out.println("US-ASCII => " + Charset.forName("US-ASCII")); System.out.println("ISO-8859-1 => " + Charset.forName("ISO-8859-1")); prints ASCII => US-ASCII US-ASCII => US-ASCII ISO-8859-1 => ISO-8859-1
You could skip the step of a char array and putting in String and even use a StringBuilder (or StringBuffer if you are worried about multi-threading). My example shows StringBuilder. byte[] bytes = ...; StringBuilder sb = new StringBuilder(bytes.length); for (int i = 0; i < bytes.length; i++) { sb.append((char) (bytes[i] & 0xFF)); } return sb.toString(); I know it doesn't answer your other question. Just seeking to help with simplifying the "boilerplate" code.
There is a String constructor that takes an array of bytes and a string specifying the format of the bytes: String s = new String(bytes, "UTF-8"); // if the charset is UTF-8 String s = new String(bytes, "ASCII"); // if the charset is ASCII
You can use base64 encoding. There is an implementation done by apache http://commons.apache.org/codec/ Base 64 http://commons.apache.org/codec/apidocs/org/apache/commons/codec/binary/Base64.html
String to byte array and then to MD5 in Java
in the last 5 hours im trying to do something that should be very simple and did it in like 10 minutes in C#, but no luck with Java. I got a 32 UpperCase and Numeric String (A-Z0-9), I need to convert this String to Dec, and then md5 it. My problem is that I dont have unsgined bytes so I cant md5 my array :\ Here is what I need to do in python: salt = words[1].decode("hex") passwordHash = generatePasswordHash(salt, pw) generatePasswordHash(salt, password): m = md5.new() m.update(salt) m.update(password) return m.digest() and here it is in C# : public static string GeneratePasswordHash(byte[] a_bSalt, string strData) { MD5 md5Hasher = MD5.Create(); byte[] a_bCombined = new byte[a_bSalt.Length + strData.Length]; a_bSalt.CopyTo(a_bCombined, 0); Encoding.Default.GetBytes(strData).CopyTo(a_bCombined, a_bSalt.Length); byte[] a_bHash = md5Hasher.ComputeHash(a_bCombined); StringBuilder sbStringifyHash = new StringBuilder(); for (int i = 0; i < a_bHash.Length; i++) { sbStringifyHash.Append(a_bHash[i].ToString("X2")); } return sbStringifyHash.ToString(); } protected byte[] HashToByteArray(string strHexString) { byte[] a_bReturn = new byte[strHexString.Length / 2]; for (int i = 0; i < a_bReturn.Length; i++) { a_bReturn[i] = Convert.ToByte(strHexString.Substring(i * 2, 2), 16); } return a_bReturn; } I will be very happy to get a help with this :)
To parse a hex string into a byte : (byte) Integer.parseInt(s, 16). To transform your password string into a byte array, using the default encoding (which I suggest not to do : always specify a specific encoding) : password.getBytes() (or password.getBytes(encoding) for a specific encoding). To hash a byte array : MessageDigest.getInstance("MD5").digest(byte[]). To transform a byte to a hex String : See In Java, how do I convert a byte array to a string of hex digits while keeping leading zeros?
I believe that something like the following will work: // convert your hex string to bytes BigInteger bigInt = new BigInteger(salt, 16); byte[] bytes = bigInt.toByteArray(); // get the MD5 digest library MessageDigest md5Digest = null; try { md5Digest = MessageDigest.getInstance("MD5"); } catch (NoSuchAlgorithmException e) { // error handling here... } // by default big integer outputs a 0 sign byte if the first bit is set if (bigInt.testBit(0)) { md5Digest.update(bytes, 1, bytes.length - 1); } else { md5Digest.update(bytes); } // get the digest bytes byte[] digestBytes = md5Digest.digest(); Here's more ideas for converting a hex string to a byte[] array: Convert a string representation of a hex dump to a byte array using Java?
You can use unsigned numbers in java with applying bit masks. Take a look details here.