We Keep Coding

Convert byte[] to string to byte[] [duplicate]

I have to convert a byte array to string in Android, but my byte array contains negative values.
If I convert that string again to byte array, values I am getting are different from original byte array values.
What can I do to get proper conversion? Code I am using to do the conversion is as follows:
// Code to convert byte arr to str:
byte[] by_original = {0,1,-2,3,-4,-5,6};
String str1 = new String(by_original);
System.out.println("str1 >> "+str1);
// Code to convert str to byte arr:
byte[] by_new = str1.getBytes();
for(int i=0;i<by_new.length;i++)
System.out.println("by1["+i+"] >> "+str1);
I am stuck in this problem.

Your byte array must have some encoding. The encoding cannot be ASCII if you've got negative values. Once you figure that out, you can convert a set of bytes to a String using:
byte[] bytes = {...}
String str = new String(bytes, StandardCharsets.UTF_8); // for UTF-8 encoding
There are a bunch of encodings you can use, look at the supported encodings in the Oracle javadocs.

The "proper conversion" between byte[] and String is to explicitly state the encoding you want to use. If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to.
If you really must use a String to hold binary data then the safest way is to use Base64 encoding.

The root problem is (I think) that you are unwittingly using a character set for which:
bytes != encode(decode(bytes))
in some cases. UTF-8 is an example of such a character set. Specifically, certain sequences of bytes are not valid encodings in UTF-8. If the UTF-8 decoder encounters one of these sequences, it is liable to discard the offending bytes or decode them as the Unicode codepoint for "no such character". Naturally, when you then try to encode the characters as bytes the result will be different.
The solution is:
Be explicit about the character encoding you are using; i.e. use a String constructor and String.toByteArray method with an explicit charset.
Use the right character set for your byte data ... or alternatively one (such as "Latin-1" where all byte sequences map to valid Unicode characters.
If your bytes are (really) binary data and you want to be able to transmit / receive them over a "text based" channel, use something like Base64 encoding ... which is designed for this purpose.
For Java, the most common character sets are in java.nio.charset.StandardCharsets. If you are encoding a string that can contain any Unicode character value then UTF-8 encoding (UTF_8) is recommended.
If you want a 1:1 mapping in Java then you can use ISO Latin Alphabet No. 1 - more commonly just called "Latin 1" or simply "Latin" (ISO_8859_1). Note that Latin-1 in Java is the IANA version of Latin-1 which assigns characters to all possible 256 values including control blocks C0 and C1. These are not printable: you won't see them in any output.
From Java 8 onwards Java contains java.util.Base64 for Base64 encoding / decoding. For URL-safe encoding you may want to to use Base64.getUrlEncoder instead of the standard encoder. This class is also present in Android since Android Oreo (8), API level 26.

We just need to construct a new String with the array: http://www.mkyong.com/java/how-do-convert-byte-array-to-string-in-java/
String s = new String(bytes);
The bytes of the resulting string differs depending on what charset you use. new String(bytes) and new String(bytes, Charset.forName("utf-8")) and new String(bytes, Charset.forName("utf-16")) will all have different byte arrays when you call String#getBytes() (depending on the default charset)

Using new String(byOriginal) and converting back to byte[] using getBytes() doesn't guarantee two byte[] with equal values. This is due to a call to StringCoding.encode(..) which will encode the String to Charset.defaultCharset(). During this encoding, the encoder might choose to replace unknown characters and do other changes. Hence, using String.getBytes() might not return an equal array as you've originally passed to the constructor.

Why was the problem: As someone already specified:
If you start with a byte[] and it does not in fact contain text data, there is no "proper conversion". Strings are for text, byte[] is for binary data, and the only really sensible thing to do is to avoid converting between them unless you absolutely have to.
I was observing this problem when I was trying to create byte[] from a pdf file and then converting it to String and then taking the String as input and converting back to file.
So make sure your encoding and decoding logic is same as I did. I explicitly encoded the byte[] to Base64 and decoded it to create the file again.
Use-case:
Due to some limitation I was trying to sent byte[] in request(POST) and the process was as follows:
PDF File >> Base64.encodeBase64(byte[]) >> String >> Send in request(POST) >> receive String >> Base64.decodeBase64(byte[]) >> create binary
Try this and this worked for me..
File file = new File("filePath");
byte[] byteArray = new byte[(int) file.length()];
try {
FileInputStream fileInputStream = new FileInputStream(file);
fileInputStream.read(byteArray);
String byteArrayStr= new String(Base64.encodeBase64(byteArray));
FileOutputStream fos = new FileOutputStream("newFilePath");
fos.write(Base64.decodeBase64(byteArrayStr.getBytes()));
fos.close();
}
catch (FileNotFoundException e) {
System.out.println("File Not Found.");
e.printStackTrace();
}
catch (IOException e1) {
System.out.println("Error Reading The File.");
e1.printStackTrace();
}

Even though
new String(bytes, "UTF-8")
is correct it throws a UnsupportedEncodingException which forces you to deal with a checked exception. You can use as an alternative another constructor since Java 1.6 to convert a byte array into a String:
new String(bytes, StandardCharsets.UTF_8)
This one does not throw any exception.
Converting back should be also done with StandardCharsets.UTF_8:
"test".getBytes(StandardCharsets.UTF_8)
Again you avoid having to deal with checked exceptions.

private static String toHexadecimal(byte[] digest){
String hash = "";
for(byte aux : digest) {
int b = aux & 0xff;
if (Integer.toHexString(b).length() == 1) hash += "0";
hash += Integer.toHexString(b);
}
return hash;
}

I did notice something that is not in any of the answers. You can cast each of the bytes in the byte array to characters, and put them in a char array. Then the string is new String(cbuf) where cbuf is the char array. To convert back, loop through the string casting each of the chars to bytes to put into a byte array, and this byte array will be the same as the first.
public class StringByteArrTest {
public static void main(String[] args) {
// put whatever byte array here
byte[] arr = new byte[] {-12, -100, -49, 100, -63, 0, -90};
for (byte b: arr) System.out.println(b);
// put data into this char array
char[] cbuf = new char[arr.length];
for (int i = 0; i < arr.length; i++) {
cbuf[i] = (char) arr[i];
}
// this is the string
String s = new String(cbuf);
System.out.println(s);
// converting back
byte[] out = new byte[s.length()];
for (int i = 0; i < s.length(); i++) {
out[i] = (byte) s.charAt(i);
}
for (byte b: out) System.out.println(b);
}
}

This works fine for me:
String cd = "Holding some value";
Converting from string to byte[]:
byte[] cookie = new sun.misc.BASE64Decoder().decodeBuffer(cd);
Converting from byte[] to string:
cd = new sun.misc.BASE64Encoder().encode(cookie);

Following is the sample code safely converts byte array to String and String to byte array back.
byte bytesArray[] = { 1, -2, 4, -5, 10};
String encoded = java.util.Base64.getEncoder().encodeToString(bytesArray);
byte[] decoded = java.util.Base64.getDecoder().decode(encoded);
System.out.println("input: "+Arrays.toString(bytesArray));
System.out.println("encoded: "+encoded);
System.out.println("decoded: "+Arrays.toString(decoded));
Output:
input: [1, -2, 4, -5, 10]
encoded: Af4E+wo=
decoded: [1, -2, 4, -5, 10]

javax.xml.bind.DatatypeConverter should do it:
byte [] b = javax.xml.bind.DatatypeConverter.parseHexBinary("E62DB");
String s = javax.xml.bind.DatatypeConverter.printHexBinary(b);

byte[] bytes = "Techie Delight".getBytes();
// System.out.println(Arrays.toString(bytes));
// Create a string from the byte array without specifying
// character encoding
String string = new String(bytes);
System.out.println(string);

Heres a few methods that convert an array of bytes to a string. I've tested them they work well.
public String getStringFromByteArray(byte[] settingsData) {
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(settingsData);
Reader reader = new BufferedReader(new InputStreamReader(byteArrayInputStream));
StringBuilder sb = new StringBuilder();
int byteChar;
try {
while((byteChar = reader.read()) != -1) {
sb.append((char) byteChar);
}
}
catch(IOException e) {
e.printStackTrace();
}
return sb.toString();
}
public String getStringFromByteArray(byte[] settingsData) {
StringBuilder sb = new StringBuilder();
for(byte willBeChar: settingsData) {
sb.append((char) willBeChar);
}
return sb.toString();
}

While base64 encoding is safe and one could argue "the right answer", I arrived here looking for a way to convert a Java byte array to/from a Java String as-is. That is, where each member of the byte array remains intact in its String counterpart, with no extra space required for encoding/transport.
This answer describing 8bit transparent encodings was very helpful for me. I used ISO-8859-1 on terabytes of binary data to convert back and forth successfully (binary <-> String) without the inflated space requirements needed for a base64 encoding, so is safe for my use-case - YMMV.
This was also helpful in explaining when/if you should experiment.

Using Kotlin on Android I found out it is handy to create some simple extension functions for that purpose.
Solution based on Base64 encoding/decoding to be able to pass via JSON, XML, etc:
import android.util.Base64
fun ByteArray.encodeToString() = String(Base64.encode(this, Base64.NO_WRAP), Charsets.UTF_8)
fun String.decodeToBytes(): ByteArray = Base64.decode(toByteArray(Charsets.UTF_8), Base64.NO_WRAP)
So you can use it
val byteArray = byteArrayOf(0, 1, 2, -1, -2, -3)
val string = byteArray.encodeToString()
val restoredArray = string.decodeToBytes()

import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;
private static String base64Encode(byte[] bytes)
{
return new BASE64Encoder().encode(bytes);
}
private static byte[] base64Decode(String s) throws IOException
{
return new BASE64Decoder().decodeBuffer(s);
}

I succeeded converting byte array to a string with this method:
public static String byteArrayToString(byte[] data){
String response = Arrays.toString(data);
String[] byteValues = response.substring(1, response.length() - 1).split(",");
byte[] bytes = new byte[byteValues.length];
for (int i=0, len=bytes.length; i<len; i++) {
bytes[i] = Byte.parseByte(byteValues[i].trim());
}
String str = new String(bytes);
return str.toLowerCase();
}

This one works for me up to android Q:
You can use the following method to convert o hex string to string
public static String hexToString(String hex) {
StringBuilder sb = new StringBuilder();
char[] hexData = hex.toCharArray();
for (int count = 0; count < hexData.length - 1; count += 2) {
int firstDigit = Character.digit(hexData[count], 16);
int lastDigit = Character.digit(hexData[count + 1], 16);
int decimal = firstDigit * 16 + lastDigit;
sb.append((char)decimal);
}
return sb.toString();
}
with the following to convert a byte array to a hex string
public static String bytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}

Here the working code.
// Encode byte array into string . TemplateBuffer1 is my bytearry variable.
String finger_buffer = Base64.encodeToString(templateBuffer1, Base64.DEFAULT);
Log.d(TAG, "Captured biometric device->" + finger_buffer);
// Decode String into Byte Array. decodedString is my bytearray[]
decodedString = Base64.decode(finger_buffer, Base64.DEFAULT);

You can use simple for loop for conversion:
public void byteArrToString(){
byte[] b = {'a','b','$'};
String str = "";
for(int i=0; i<b.length; i++){
char c = (char) b[i];
str+=c;
}
System.out.println(str);
}

byte[] image = {...};
String imageString = Base64.encodeToString(image, Base64.NO_WRAP);

Try to specify an 8-bit charset in both conversions. ISO-8859-1 for instance.

Read the bytes from String using ByteArrayInputStream and wrap it with BufferedReader which is Char Stream instead of Byte Stream which converts the byte data to String.
package com.cs.sajal;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
public class TestCls {
public static void main(String[] args) {
String s=new String("Sajal is a good boy");
try
{
ByteArrayInputStream bis;
bis=new ByteArrayInputStream(s.getBytes("UTF-8"));
BufferedReader br=new BufferedReader(new InputStreamReader(bis));
System.out.println(br.readLine());
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
Output is:
Sajal is a good boy

You can do the following to convert byte array to string and then convert that string to byte array:
// 1. convert byte array to string and then string to byte array
// convert byte array to string
byte[] by_original = {0, 1, -2, 3, -4, -5, 6};
String str1 = Arrays.toString(by_original);
System.out.println(str1); // output: [0, 1, -2, 3, -4, -5, 6]
// convert string to byte array
String newString = str1.substring(1, str1.length()-1);
String[] stringArray = newString.split(", ");
byte[] by_new = new byte[stringArray.length];
for(int i=0; i<stringArray.length; i++) {
by_new[i] = (byte) Integer.parseInt(stringArray[i]);
}
System.out.println(Arrays.toString(by_new)); // output: [0, 1, -2, 3, -4, -5, 6]
But to convert the string to byte array and then convert that byte array to string, below approach can be used:
// 2. convert string to byte array and then byte array to string
// convert string to byte array
String str2 = "[0, 1, -2, 3, -4, -5, 6]";
byte[] byteStr2 = str2.getBytes(StandardCharsets.UTF_8);
// Now byteStr2 is [91, 48, 44, 32, 49, 44, 32, 45, 50, 44, 32, 51, 44, 32, 45, 52, 44, 32, 45, 53, 44, 32, 54, 93]
// convert byte array to string
System.out.println(new String(byteStr2, StandardCharsets.UTF_8)); // output: [0, 1, -2, 3, -4, -5, 6]

A string is a collection of char's (16bit unsigned). So if you are going to convert negative numbers into a string, they'll be lost in translation.

public class byteString {
/**
* #param args
*/
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
String msg = "Hello";
byte[] buff = new byte[1024];
buff = msg.getBytes("UTF-8");
System.out.println(buff);
String m = new String(buff);
System.out.println(m);
}
}

What kind of hashing / encryption is this?

private String getString(byte[] bytes)
{
StringBuffer sb = new StringBuffer();
for (int i = 0; i < bytes.length; i++)
{
byte b = bytes[i];
sb.append(0xFF & b);
}
return sb.toString();
}
public String encrypt(String source)
{
try
{
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] bytes = md.digest(source.getBytes());
return getString(bytes);
}
catch (Exception e)
{
e.printStackTrace(); }
return null;
}
If my text = "test"
The First Part toString()) produces a value of "Encryption$2#6966b26b"
And the second part then gets that and produces a value of "91431072057033211115202222781313839180246"
But why is the md5 a number and not 31f521a06d5060d1f38159c74a1f7cf2 or something similar?

The function "encrypt()" returns a MD5 hash. You should rename it to "hash", because hashing != encrypting.
If you want to encrypt a string, you can look here: https://gist.github.com/bricef/2436364

It's clearly stated in code yuou are using MD5 hashing algorithm
Now your question is why:
But why is the md5 a number and not 31f521a06d5060d1f38159c74a1f7cf2 or something similar?
your answer is simple, look at code which generates you string from your byte array.
byte b = bytes[i];
sb.append(0xFF & b);
you take byte, ie 0x20 then you perform logical and operation with integer 0x255 and then you add decimal representation of result yo your StringBuilder.
What you want to do is more like
sb.append(Integer.toHexString(0xff&b));

I would say MD5 hash, because the code says MessageDigest.getInstance("MD5") :D

30 char md5 digest

I use the following code for generating md5 for blob data in database.
md5Checksum.update(byte[] --- read from database);
String result = new BigInteger(1,md5Checksum.digest()).toString(16);
The checksum i obtain is varying in length(30-32) for different byte arrays.
For 31 char length checksum, as I understood can be an effect of the removal of leading zeros. (I handled it by adding leading zeros)
Can any one tell me why I am getting a 30 char hash in some cases?
Thanks,
Mithun

Do not convert a digest to a number!
Use something like:
byte[] b = md5Checksum.digest();
// now convert these bytes to chars
There are many different methods to convert byte[] to HexStrings:
public class HexConverter {
// thanks to http://www.rgagnon.com/javadetails/java-0596.html
static private final String HEXES = "0123456789ABCDEF";
public String toHex(byte[] raw) {
if (raw == null) {
return null;
}
final StringBuilder hex = new StringBuilder(2 * raw.length);
for (final byte b : raw) {
hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F)));
}
return hex.toString();
}
}
or from Getting a File's MD5 Checksum in Java
(this link also shows how to se DigestUtils from Apache Commons Codec)
public static String getMD5Checksum(String filename) throws Exception {
byte[] b = createChecksum(filename);
String result = "";
for (int i=0; i < b.length; i++) {
result += Integer.toString( ( b[i] & 0xff ) + 0x100, 16).substring( 1 );
}
return result;
}

There is a chance that the high n bits could be zero. As you convert the byte[] to a number.If n=4,then you could lose one '0' char at the beginning.
If n=8, then you lose '00' at the beginning of your md5 hex code.

How do I apply SHA-1 to a large hex number byte by byte?

I'm trying to create a program that will enable me to convert a MEID (a hex number of length 14) to pseudo ESN (a hex number of length 8). The way to obtain a pESN from MEID is fairly simple in theory. For example, given MEID 0xA0000000002329, to make a pESN, SHA-1 needs to be applied to the MEID. SHA-1 on A0000000002329 gives e3be267a2cd5c861f3c7ea4224df829a3551f1ab. Take the last 6 hex numbers of this result, and append it to 0x80 - the result is 0x8051F1AB.
Now here is the code I have so far:
public void sha1() throws NoSuchAlgorithmException {
String hexMEID = "A0000000002329";
MessageDigest mDigest = MessageDigest.getInstance("SHA1");
byte[] b = new BigInteger(hexMEID,16).toByteArray();
byte[] result = mDigest.digest(b);
StringBuilder sb = new StringBuilder();
for (int i = 0; i < result.length; i++) {
sb.append(Integer.toString((result[i] & 0xff) + 0x100, 16).substring(1));
}
System.out.println(sb.toString());
}
The problem is that using this method, SHA-1 on A0000000002329 gives 6ad447f040941bf43c0693d2b391c6c79fa58320 instead of e3be267a2cd5c861f3c7ea4224df829a3551f1ab. What am I doing wrong here??
Someone gave me a hint that
The trick is to apply SHA-1 to the number representing the MEID, not
the string representing the MEID. You'll need to process it
byte-by-byte, so you must give it two hex numbers at a time (since two
hex numbers make a byte) and make sure they are interpreted as numbers
and not ASCII characters.
If these instructions are true, then how do I apply SHA-1 to my hex number byte by byte??

You have a tiny little issue which is the consequence of using BigInteger to get your byte array. Since the MEID is only 7 bytes long, when you pump it through the BigInteger, you will get a byte array of length 8 because the BigInteger outputs the exta byte which holds the sign. This extra byte causes the SHA-1 hash of your input to be completely different, of course. You need to strip it off.
So here is what the HEX MEID to ESN code will look like:
String hexMEID = "A0000000002329";
MessageDigest mDigest = MessageDigest.getInstance( "SHA1" );
byte[] input = new byte[ 7 ]; // MEIDs are only 7 bytes
// Now copy the bytes from BigInteger skipping the extra byte added by it
System.arraycopy( new BigInteger( hexMEID, 16 ).toByteArray(), 1, input, 0, 7 );
// Get the SHA-1 bytes
byte[] result = mDigest.digest( input );
// Build the SHA-1 String
StringBuilder sb = new StringBuilder();
for ( int i = 0; i < result.length; i++ )
{
String hex = Integer.toHexString( 0xFF & result[ i ] );
if ( hex.length() == 1 )
{
sb.append( '0' );
}
sb.append( hex );
}
String sha1 = sb.toString();
// Grab the last 6 characters of the SHA-1 hash
String lastSix = sha1.substring( sha1.length() - 6 );
// And prepend '80', now you have the ESN
System.out.println( "80" + lastSix );
// Will print 8051f1ab which is exactly what you want

Strelok has found the problem about BigInteger adding an extra byte in the returned array. This simpler version also gives the expected result :
String hexMEID = "A0000000002329";
MessageDigest mDigest = MessageDigest.getInstance("SHA1");
byte[] b = new BigInteger(hexMEID,16).toByteArray();
// skip the first byte set by BigInteger and retain only 7 bytes (length of MEID)
byte[] result = mDigest.digest(Arrays.copyOfRange(b, 1, 8));
StringBuilder sb = new StringBuilder("80");
// need only the last 3 bytes
for (int i=17; i<20; i++) {
sb.append(Integer.toHexString((result[i] & 0xff) | 0x100).substring(1));
}
String pESN = sb.toString();
System.out.println(pESN);
// -> 8051f1ab

Convert MD5 array to String java

I know that there is a lot of similar topics, but still... can someone provide me a working example of method which generates MD5 String.
I'm currently using MessageDigest, and I'm doing the following to get a string
sun.misc.BASE64Encoder().encode(messageDigest.digest())
I guess there is some better way to do that.
Thanks in advance!

I'd use commons-codec
Base64 - Base64.encodeBase64(digestBytes)
Hex-string - Hex.encodeHex(digestBytes)

MessageDigest md = MessageDigest.getInstance("MD5");
byte[] arr = md.digest(bytesOfMessage);
return Base64.getEncoder().encodeToString(arr);
note: md5 is not considered as good hash algorithm anymore, consider choosing SHAs

// Convert to hex string
StringBuffer sb = new StringBuffer();
for (int i = 0; i < messageDigest.length; i++) {
if ((0xff & messageDigest[i]) < 0x10) {
sb.append('0');
}
sb.append(Integer.toHexString(0xff & messageDigest[i]));
}
String md5 = sb.toString();
This assumes you actually want your MD5 printed as an hex string, not BASE64-encoded. That's the way it is normally represented.

I've seen next solution:
byte[] digest = md.digest(someDataByteArray);
StringBuilder hex = new StringBuilder();
for (byte b : digest) {
hex.append(String.format("%02x", b));
}

import javax.xml.bind.DatatypeConverter;
import java.security.MessageDigest;
...
String input = "westerngun";
MessageDigest digest = MessageDigest.getInstance("MD5"); // not thread-safe, create instance for each thread
byte[] result = digest.digest(input.getBytes()); // get MD5 hash array, could contain negative
String hex = DatatypeConverter.printHexBinary(result).toLowerCase(); // convert byte array to hex string
If you want a number:
Integer number = Integer.parseInt(hex, 16); // parse hex number to integer. If overflowed, use Long.parseLong()