We Keep Coding

Convert a byte[] array into a string of characters INCLUDING escape characters in java

I searched google about this information but the answers I found do not apply to my case.
I have an HEX string like the following:
hexString = '7d940ef9790c31334ac6f116814148b9abe73f32'
Python can convert this string to a binary value using the following function:
unhexlify('7d940ef9790c31334ac6f116814148b9abe73f32')
whose result is:
binString = '}\x94\x0e\xf9y\x0c13J\xc6\xf1\x16\x81AH\xb9\xab\xe7?2'
That is, a string containing the binary information of the original hex string.
I tried to use the .getBytes("encoding") method in java, but I am not able to reproduce this result, and unfortunately this result is critical for my application (I need exactly the same result).
I'm not an encodings pro, so it could easily be me overlooking something.
I need to convert to the same kind of string as "binString" a byte[] array resulting from e.g. a md5 digest, so any insight on how to convert a byte[] to such a string would be most appreciated.

It's not a solution, but it could be helpful:
import java.nio.charset.Charset;
import javax.xml.bind.DatatypeConverter;
public class Main {
public static void main(String args[])
{
String hexString = "7d940ef9790c31334ac6f116814148b9abe73f32";
byte[] out = toByteArray(hexString);
String result = new String(out,Charset.forName("UTF-8"));
System.out.println(result);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
}
The output in my machine is:
}??y13J???AH????2
It can print all the ascii chracters, but there are problems with the escape characters like \x.

Trying to translate this line of Python to Java- getting different answers

Original Python code:
import hashlib
return int(hashlib.md5("string").hexdigest(), 16) % 100
My attempt to translate into Java:
import java.security.*;
import java.math.*;
String s = "string";
MessageDigest m = MessageDigest.getInstance("MD5");
m.update(s.getBytes(), 0, s.length());
BigInteger i = BigInteger(1,m.digest());
return i % 100;
What am I doing wrong?

I see you haven't even tried to compile this code, since it's not working. BigInteger isn't a primitive type, therefore % operator doesn't work. You should use .mod method instead.
Your code, fixed:
String s = "string";
MessageDigest m = MessageDigest.getInstance("MD5");
m.update(s.getBytes(), 0, s.length());
BigInteger i = new BigInteger(1, m.digest());
return i.mod(BigInteger.valueOf(100));
And since you're updating 0 - s.length(), you can just pass all the bytes to m.digest, make it one line shorter:
String s = "string";
MessageDigest m = MessageDigest.getInstance("MD5");
BigInteger i = new BigInteger(1, m.digest(s.getBytes()));
return i.mod(BigInteger.valueOf(100));
And if you really want an one-liner... Warning, highly unreadable:
return new BigInteger(1, MessageDigest.getInstance("MD5").digest( s.getBytes()) ).mod(BigInteger.valueOf(100));

Suggestion
It's always recommended to specify the encoding of the input string while converting to byte stream.
Without specifying an encoding scheme, programming language implementations are free to choose their default format. This can vary across programming languages and even across different versions of the same programming language.
Python
An advantage of python is, the ability to handle arbitrarily large integer values
import hashlib
data: str = "string"
print(int(hashlib.md5(data.encode('utf-8')).hexdigest(), 16) % 100)
Java
BigInteger is needed to deal with long values outside the Long.MIN_VALUE and Long.MAX_VALUE boundaries.
MessageDigest instances are not thread safe, so use with care(new instances or ThreadLocal) in multithreaded use cases.
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.math.BigInteger;
public class TestMD5 {
private static final String DIGEST_ALGO = "MD5";
public static void main(String[] args) throws Exception {
String data = "string";
byte[] digest = MessageDigest.getInstance(DIGEST_ALGO).digest(data.getBytes(StandardCharsets.UTF_8));
int value = new BigInteger(1, digest).mod(BigInteger.valueOf(100)).intValueExact();
System.out.println(value);
}
}
Both these solutions will print 81 for the given input string
Note
The implementation can print the same value for the given input string even without adding the encoding. It should not relied upon as this string is within the ascii range.
Even for unicode character set, it is recommended to specify encoding as the same code point can be encoded in different encoding schemes like utf-8, utf-16
while converting a byte stream to string, always specify the (de)encoding scheme

How do I decode a string from base 32 to a BigInteger

I tried the following code:
import java.math.BigInteger;
import org.apache.commons.codec.binary.Base32;
import org.junit.Test;
public class Sandbox
{
#Test
public void testSomething() {
String sInput = "GIYTINZUHAZTMNBX";
BigInteger bb = new BigInteger(new Base32().decode(sInput));
System.out.println("number = " + bb);
}
}
and heres the output:
number = 237025977136523702055991
using this website to convert between base 32 I get a different result than the actual output. Heres the result I expect to see based on what I got from the website:
expected output = 2147483647
Any idea why this is happening?
Edit:
Forgive me for making it confusing by purposefully attempting to convert 2^31-1.
Using the conversion website I linked to earlier, I changed the input:
String sInput = "GE4DE===";
Expected output:
number = 182
Actual output:
number = 3225650

What you're doing is correct... assuming that the Base32 string comes from Base32-encoding a byte array you get from calling BigInteger.toByteArray().
BigInteger(byte[] val) does not really take an array of arbitrary bytes. It takes the byte[] representation of a BigInteger. Also, it assumes the most-significant byte is in val[0]).

If it's base-32 the X, Y, and Z shouldn't be there. Are you sure it isn't base-36?

Blowfish decryption in python of string encoded by javax.crypto

Using the code found at https://raw.github.com/usefulfor/usefulfor/master/security/JBoss.java, I did thefollowing:
bash-3.2$ java -cp . JBoss -e testpython
-27038292d345798947e2852756afcf0a
bash-3.2$ java -cp . JBoss -d -27038292d345798947e2852756afcf0a
testpython
However, I can't for the life of me, figure out how to decrypt the string '27038292d345798947e2852756afcf0a' using pycrypto in python. My understanding is that the Java code is using Blowfish, and the phrase 'jaas is the way' as the key for the cipher. But I can't make any sense of how to do this in python. The following results in mostly unprintable garbage:
import Crypto
from Crypto.Cipher import Blowfish
from base64 import b64encode, b64decode
bs = Blowfish.block_size
key = 'jaas is the way'
plaintext = b'27038292d345798947e2852756afcf0a'
iv = '\0' * 8
c1 = Blowfish.new(key, Blowfish.MODE_ECB)
c2 = Blowfish.new(key, Blowfish.MODE_CBC, iv)
c3 = Blowfish.new(key, Blowfish.MODE_CFB, iv)
c4 = Blowfish.new(key, Blowfish.MODE_OFB, iv)
msg1 = c1.decrypt(plaintext)
msg2 = c2.decrypt(plaintext)
msg3 = c3.decrypt(plaintext)
msg4 = c4.decrypt(plaintext)
print "msg1 = %s\n" % msg1
print "msg2 = %s\n" % msg2
print "msg3 = %s\n" % msg3
print "msg4 = %s\n" % msg4
What am I missing ?
thanks.

First of all, that Java example code is very bad. It outputs the ciphertext as an integer, whereas ciphertext should remain a binary string. The reason is that an integer can be represented in an infinite number of binary encodings. For instance, the number one can be '0x01' (1 byte), '0x0001' (2 bytes), and so forth. When you are dealing with cryptographic functions, you must be extremely precise with the representation.
Additionally, the example uses the default values of the javax.crypto API, which are not described anywhere. So it is really trial and error.
For the solution, you must know how to convert negative integers to hex strings in Python. In this case, you don't need a hex string, but its byte representation. The concept is the same though. I use PyCrypto's long_to_bytes to convert a positive integer (of arbitrary length) to a byte string.
from Crypto.Cipher import Blowfish
from Crypto.Util.number import long_to_bytes
def tobytestring(val, nbits):
"""Convert an integer (val, even negative) to its byte string representation.
Parameter nbits is the length of the desired byte string (in bits).
"""
return long_to_bytes((val + (1 << nbits)) % (1 << nbits), nbits/8)
key = b'jaas is the way'
c1 = Blowfish.new(key, Blowfish.MODE_ECB)
fromjava = b"-27038292d345798947e2852756afcf0a"
# We don't know the real length of the ciphertext, assume it is 16 bytes
ciphertext = tobytestring(int(fromjava, 16), 16*8)
print c1.decrypt(ciphertext)
The output is:
'testpython\x06\x06\x06\x06\x06\x06'
From that you see that javax.crypto also adds PKCS#5 padding, which you need to remove by yourself. That is trivial to do though.
However, the real solution to your problem is to do Java encryption in a better way. The Python code will be greatly simplified.

This helps me
private byte[] encrypt(String key, String plainText) throws GeneralSecurityException {
SecretKey secret_key = new SecretKeySpec(key.getBytes(), ALGORITM);
Cipher cipher = Cipher.getInstance(ALGORITM);
cipher.init(Cipher.ENCRYPT_MODE, secret_key);
return cipher.doFinal(plainText.getBytes());
}
hope this will be useful for you, more http://dexxtr.com/post/57145943236/blowfish-encrypt-and-decrypt-in-java-android

Java SHA256 outputs different hash to PHP SHA256?

PHP code:
echo hash('sha256', 'jake');
PHP output:
cdf30c6b345276278bedc7bcedd9d5582f5b8e0c1dd858f46ef4ea231f92731d
Java code:
String s = "jake";
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(s.getBytes(Charset.forName("UTF-8")));
byte[] hashed = md.digest();
String s2 = "";
for (byte b : hashed) {
s2 += b;
}
System.out.println(s2);
Java output:
-51-1312107528211839-117-19-57-68-19-39-43884791-1141229-4088-12110-12-223531-11011529
I had expected the two to return the same result. Obviously, this is not the case. How can I get the two to match up or is it impossible?
EDIT: I had made a mistake, think I have the answer to the question now anyway.

Well, the very first thing you need to do is use a consistent string encoding. I've no idea what PHP will do, but "jake".getBytes() will use whatever your platform default encoding is for Java. That's a really bad idea. Using UTF-8 would probably be a good start, assuming that PHP copes with Unicode strings to start with. (If it doesn't, you'll need to work out what it is doing and try to make the two consistent.) In Java, use the overload of String.getBytes() which takes a Charset or the one which takes the name of a Charset. (Personally I like to use Guava's Charsets.UTF_8.)
Then persuade PHP to use UTF-8 as well.
Then output the Java result in hex. I very much doubt that the code you've given is the actual code you're running, as otherwise I'd expect output such as "[B#e48e1b". Whatever you're doing to convert the byte array into a string, change it to use hex.

They are printing the same .. convert your byte[] to a hex string, then you'll see CDF30C6B345276278BEDC7BCEDD9D5582F5B8E0C1DD858F46EF4EA231F92731D as Java output, too:
public void testSomething() throws Exception {
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update("jake".getBytes());
System.out.println(getHex(md.digest()));
}
static final String HEXES = "0123456789ABCDEF";
public static String getHex( byte [] raw ) {
if ( raw == null ) {
return null;
}
final StringBuilder hex = new StringBuilder( 2 * raw.length );
for ( final byte b : raw ) {
hex.append(HEXES.charAt((b & 0xF0) >> 4))
.append(HEXES.charAt((b & 0x0F)));
}
return hex.toString();
}

You need to convert the digest to a HEX string before printing it out. Example code can be found here.