We Keep Coding

How to use MySQL AES_DECRYPT to decrypt data encrypted with BouncyCastles PBEWITHSHA256AND256BITAES-CBC-BC Algorithm

We use BouncyCastle PBEWITHSHA256AND256BITAES-CBC-BC to encrypt data with our java application and store the encrypted result in a MySql Database.
Example Code:
StandardPBEStringEncryptor configurationEncryptor = new StandardPBEStringEncryptor();
configurationEncryptor.setAlgorithm("PBEWITHSHA256AND256BITAES-CBC-BC");
configurationEncryptor.setProviderName("BC");
configurationEncryptor.setSaltGenerator(new RandomSaltGenerator());
configurationEncryptor.setKeyObtentionIterations(1000);
configurationEncryptor.setPassword("aTestPassword");
String input = "A Test String!";
String cypherText = configurationEncryptor.encrypt(input);
String plainText = configurationEncryptor.decrypt(cypherText);
System.out.println("Input:" + input + " cypher:" + cypherText + " plain:" + plainText);
Output:
Input:A Test String! cypher:DhCSPbCWcZ76TUD/dDeGczlHbI9dQJyB2lKAiL7dDEk= plain:A Test String!
The cypher string above is a base64 encoded string which we store in our database.
I would now like to attempt to decrypt the cypher string stored in our database using the AES utilities provided by MySql.
I am trying to understand how the BC provider concatenates the encrypted data so that I can split it up and recreate the required parameters to enable me to decrypt the data with other tools - in this case MySql's AES_DECRYPT function.
Inspecting the code I can see that the first 16bytes of the cypher text (when base 64 decoded) is the salt, I am unsure where the init vector (IV) is stored in the remainder of the cypher text data.
If we can parse out the IV, salt and encrypted value from the string, then it should be possible to use external tools to decrypt the data.
A sample Mysql AES usage is as follows:
SET block_encryption_mode = 'aes-256-cbc';
SET #key_str = SHA2('aTestPassword',256);
SET #init_vector = RANDOM_BYTES(16);
SET #crypt_str = AES_ENCRYPT('A Test String!',#key_str,#init_vector);
SELECT AES_DECRYPT(#crypt_str,#key_str,#init_vector);
Output:
A Test String!
I would like to know how to parse the BouncyCastle cypher text to obtain its component parts, and also how to use the salt to generate the correct key hash with the number of iterations specified for use by Mysql to decrypt the data.
Any help much appreciated!

This answer is not a solution in code but will help you in finding the code.
First: you are NOT using Bouncy Castle to en-/decrypt directly - of course the cipher is used as provider for the en-/decryption.
The library that does the complete en-/decryption is JASYPT and here we can find answers for your question.
Base for my research is the GitHub https://github.com/jboss-fuse/jasypt/tree/master/jasypt/src/main/java/org/jasypt/encryption/pbe and I'm starting with "StandardPBEStringEncryptor.java":
As we are trying to understand the encryption in use I found
// The StandardPBEByteEncryptor that will be internally used.
private final StandardPBEByteEncryptor byteEncryptor;
and later the encrypt-method:
...
// The StandardPBEByteEncryptor does its job.
byte[] encryptedMessage = this.byteEncryptor.encrypt(messageBytes);
...
if (this.stringOutputTypeBase64) {
encryptedMessage = this.base64.encode(encryptedMessage);
result = new String(encryptedMessage,ENCRYPTED_MESSAGE_CHARSET);
} else {
result = CommonUtils.toHexadecimal(encryptedMessage);
}
As you get a Base64-encoded string this class just returns the encryptedMessage in Base64-encding.
Let's see the base class "StandardPBEByteEncryptor.java":
Searching for ivInUse:
// Initialization Vector to be used for encryption and decryption.
private byte[] ivInUse = null;
...
// Initialize Initialization Vector
this.ivInUse = new byte[algorithmBlockSize];
That means we do have a static IV of 16 bytes length (blocklength for AES) filled with "x00".
salt:
The DefaultSaltLength is set to 8 but when using a block cipher the salt length equals to the cipher block size (for AES 16):
// The salt size for the chosen algorithm is set to be equal
// to the algorithm's block size (if it is a block algorithm).
final int algorithmBlockSize = this.encryptCipher.getBlockSize();
if (algorithmBlockSize > 0) {
this.saltSizeBytes = algorithmBlockSize;
}
The salt is generated with the saltGenerator and after encryption it is concatenated with the ciphertext in the form salt|encryptedMessage:
encrypt:
...
// Finally we build an array containing both the unencrypted salt
// and the result of the encryption. This is done only
// if the salt generator we are using specifies to do so.
if (this.saltGenerator.includePlainSaltInEncryptionResults()) {
// Insert unhashed salt before the encryption result
return CommonUtils.appendArrays(salt, encryptedMessage);
}
The number of iterations is given by initialization (1000).
Last part to solve is the algorithm for the cipher-init and when using OpenJava 11 I find:
PBEWithHmacSHA256AndAES_256
that (hopefully) works in CBC-mode.

'javax.crypto.BadPaddingException' while using cipherInputStream

I'm writing a program to encrypt and decrypt data.
for encrypting,
I created a symmetric key using keyGenerator.
I transferred the key to the cipher, and created a string version of the key:
String keyString = Base64.getEncoder().encodeToString(symmetricKey.getEncoded());
in order to store it in a configuration file (so I can retrieve the key in the decrypt function).
Now, in the decrypt function I need to get that string back to key format, so I can send it as a parameter to the cipher in dercypt mode.
I convert it back to key this way:
byte[] keyBytes = key.getBytes(Charset.forName("UTF-8"));
Key newkey = new SecretKeySpec(keyBytes,0,keyBytes.length, "AES");
And I transffer it to the cipher and write the output (the decrypted data) using CipherInputStream:
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, newkey, newiv, SecureRandom.getInstance("SHA1PRNG"));
CipherInputStream cipherInputStream = new CipherInputStream(
new ByteArrayInputStream(encryptedBytes), cipher);
ArrayList<Byte> decryptedVal = new ArrayList<>();
int nextByte;
while ((nextByte = cipherInputStream.read()) != -1) {
decryptedVal.add((byte) nextByte);
}
byte[] bytes = new byte[decryptedVal.size()];
for (int i = 0; i < bytes.length; i++) {
bytes[i] = decryptedVal.get(i);
}
String decryptedData = new String(bytes);
cipherInputStream.close();
System.out.println("decryptedData: " + decryptedData);
I get this error:
Exception in thread "main" java.io.IOException: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
So I suspect that there might be a problem with the way I treat the key.
Any suggestions? help would be appreciated!

I think you have not sent IV to decryption function. For decryption in CBC mode, you must provide an IV which is used in encryption process.
Update:
IV will affect only first block in CBC decryption mode. So my answer may affect the unpadding if your data is less than 1 block. It will just change the decrypted plaintext of the first block otherwise.

Of course you get this error: first you apply base 64 encoding:
String keyString = Base64.getEncoder().encodeToString(symmetricKey.getEncoded());
and then you use character-encoding to turn it back into bytes:
byte[] keyBytes = key.getBytes(Charset.forName("UTF-8"));
which just keeps be base64 encoding, probably expanding the key size from 16 bytes to 24 bytes which corresponds with a 192 bit key instead of a 128 bit key. Or 24 bytes key to a 32 bytes key of course - both seem to work.
To solve this you need to use Base64.getDecoder() and decode the key.
Currently you get a key with a different size and value. That means that each block of plaintext, including the last one containing the padding, will decrypt to random plaintext. As random plaintext is unlikely to contain valid padding, you will be greeted with a BadPaddingException.
Reminder:
encoding, e.g. base 64 or hex: encoding bytes to a text string
character-encoding, e.g. UTF-8 or ASCII: encoding a text string into bytes
They are not opposites, that would be decoding and character-decoding respectively.
Remarks:
yes, listen to Ashfin; you need to use a random IV during encryption and then use it during decryption, for instance by prefixing it to the ciphertext (unencrypted);
don't use ArrayList<Byte>; that stores a reference to each separate byte (!) - use ByteArrayOutputStream or any other OutputStream instead;
you can better use a byte buffer and use that to read / write to the streams (note that the read function may not fill the buffer, even if at the start or in the middle of the stream) - reading a single byte at the time is not performant;
lookup try-with-resources for Java;
using a KeyStore may be better than storing in a config file;
GCM mode (AES/GCM/NoPadding) also authenticates data and should be preferred over CBC mode.

How to replicate Java encryption in PHP?

Some background of what I'm trying to accomplish.
Part 1.
PHP server communicates with a Java-based device. PHP uses OpenSSL to generate a public/private keypair, then sends the public key to the device which in turn gives back an encrypted macKey (generated using the public key), encoded in base64. PHP now needs to base64-decode and decrypt the macKey using the private key.
What is the equivalent of the below Java code snippet in PHP?
String base64EncodedMacKey = "LkvTT9LFj5lcxRRB8KrwwN906fSIDDcJvQK3E7a5PbR+Ox9WnslOs32jSCC9FkE8ouvr2MfWwtppuZmoPjaxwg3yAQI4UN3T1loISuF2VwKWfJ45fywbK9bNnD5Cw7336mjoGctv77Tg3JXPrsRwgMGIlBsNwdt1B0wgT4MMMAjl32TnBI3iwQ94VTMHffrK+QToddTahRHHoVsr3FVrETdiqKXdkiX1jES53im5lrXYIsY89UFkGzPo+3u4ijKIQWSLvYnA5wXI128gFHKxKYS82MbJDUn9i1RVFsGaP6T3nQRSX5SZNpSe5yGFWwMgYOx0KXMgET82FeaL2hfWuw==";
byte[] base64DecodedMacKey = DatatypeConverter.parseBase64Binary(base64EncodedMacKey);
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
cipher.init(Cipher.DECRYPT_MODE, keypair.getPrivate());
byte[] macKey = cipher.doFinal(base64DecodedMacKey);
Here's what I attempted in PHP, however I'm confused about using byte array versus string when decrypting the macKey
$macKey = 'LkvTT9LFj5lcxRRB8KrwwN906fSIDDcJvQK3E7a5PbR+Ox9WnslOs32jSCC9FkE8ouvr2MfWwtppuZmoPjaxwg3yAQI4UN3T1loISuF2VwKWfJ45fywbK9bNnD5Cw7336mjoGctv77Tg3JXPrsRwgMGIlBsNwdt1B0wgT4MMMAjl32TnBI3iwQ94VTMHffrK+QToddTahRHHoVsr3FVrETdiqKXdkiX1jES53im5lrXYIsY89UFkGzPo+3u4ijKIQWSLvYnA5wXI128gFHKxKYS82MbJDUn9i1RVFsGaP6T3nQRSX5SZNpSe5yGFWwMgYOx0KXMgET82FeaL2hfWuw==';
$base64DecodedMacKey = base64_decode($macKey);
openssl_private_decrypt($base64DecodedMacKey, $decrypted, $privateKey);
The $decrypted above holds some binary data as it appears, so I'm unsure whether I need to convert it into a byte array or treat it as a string...
Part 2.
Each request has a counter. The macKey in Java code above is used to create a MAC value out of the counter.
What is the equivalent of the below Java code snippet in PHP?
int counter = 0;
String nextCounter = String.valueOf(++counter);
SecretKeySpec signingKey = new SecretKeySpec(macKey, "AES");
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(signingKey);
byte[] counterMac = mac.doFinal(nextCounter.getBytes("UTF-8"));
String base64EncodedMac = DatatypeConverter.printBase64Binary(counterMac);
The base64EncodedMac above is finally sent to the device to validate communication.
I've tried googling different solutions, however I've not been successful in generating a valid base64EncodedMac string in PHP for the device to approve it.

Found the solution myself. For Part 1, I chose to use phpseclib to generate the public/private keys and to specify the encryption algorithm. Decrypting macKey:
$rsa = new Crypt_RSA();
$keys = $rsa->createKey(2048);
// [...]
$macKey = base64_decode($base64EncodedMacKey);
$rsa->setEncryptionMode(CRYPT_RSA_ENCRYPTION_PKCS1);
$rsa->loadKey($keys['privatekey']);
$decryptedMac = $rsa->decrypt($macKey);
Followed by Part 2:
$counter = 0;
$hmac = hash_hmac('sha256', ++$counter, $decryptedMac, true);
$counterMac = base64_encode($hmac);
The main confusing part was that in Java, HMAC was done out of byte array, while in PHP the hash_hmac function expects a String as its 2nd parameter, so using unpack() was not sufficient. However, it seems to have worked with passing the $counter directly. It was also important to use the 4th parameter as TRUE to return raw data.

Cannot output correct hash in Java. What is wrong?

In my Android app I have a SHA256 hash which I must further hash with the RIPEMD160 message digest algorithm.
I can output the correct sha256 and ripemd160 hash of any string, but when I try to hash the sha256 hash with ripemd160 I get a hash which is incorrect.
According to online hash calculators, the SHA256 value of the string 'test'(all lowercase) is:
9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
And the RIPEMD160 value of the string 'test' is:
5e52fee47e6b070565f74372468cdc699de89107
The value from hashing the resulting sha256 hash with ripemd160 according to online calcs is:
4efc1c36d3349189fb3486d2914f56e05d3e66f8
And the one my app gives me is:
cebaa98c19807134434d107b0d3e5692a516ea66
which is obviously wrong.
Here is my code:
public static String toRIPEMD160(String in)
{
byte[] addr = in.getBytes();
byte[] out = new byte[20];
RIPEMD160Digest digest = new RIPEMD160Digest();
byte[] sha256 = sha256(addr);
digest.update(sha256,0,sha256.length);
digest.doFinal(out,0);
return getHexString(out);
}
public static byte[] sha256(byte[] data)
{
byte[] sha256 = new byte[32];
try
{
sha256 = MessageDigest.getInstance("SHA-256").digest(data);
}
catch(NoSuchAlgorithmException e)
{}
return sha256;
}
For the ripemd160 algorithm, you need bouncycastle and java.security.MessageDigest for sha256.

Your "online calculator" result is the result of hashing the bytes of the string "test" with SHA-256, converting the result of that hash to a hex string, then taking the bytes corresponding to the ASCII characters of that hex string and hashing those a second time. This is very different from your Java code, which passes the bytes that come out of the first hash directly to the second one, without printing them as hex and turning those characters back into bytes in between. The single byte with value 254 (decimal) becomes "fe" in hex, which becomes the two-byte sequence [0x66, 0x65] when converted back to bytes.

Your hash is working fine. The problem is that the online calculators that you're using are treating your input:
9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
as a string instead of an array of bytes. In other words, it's treating each character as a byte instead of parsing character pairs as bytes in hexadecimal. If I give this as a string to online calculators, I indeed get exactly what you got:
4efc1c36d3349189fb3486d2914f56e05d3e66f8
However, you're treating the output as an array of bytes instead of a String and that's giving you different results. You should encode your raw SHA256 hash as a string, then pass the encoded string to the hash function. I see you have a getHexString method, so we'll just use that.
public static String toRIPEMD160(String in) {
try {
byte[] addr = in.getBytes();
byte[] out = new byte[20];
RIPEMD160Digest digest = new RIPEMD160Digest();
// These are the lines that changed
byte[] rawSha256 = sha256(addr);
String encodedSha256 = getHexString(rawSha256);
byte[] strBytes = encodedSha256.getBytes("UTF-8");
digest.update(strBytes, 0, strBytes.length);
digest.doFinal(out, 0);
return getHexString(out);
} catch (UnsupportedEncodingException ex) {
// Never happens, everything supports UTF-8
return null;
}
}
If you want to know it's working, take the value of encodedSha256 and put that into an online hash calculator. As long as the calculator uses UTF-8 encoding to turn the string into a byte array, it will match your output.

To get printable version of byte[] digest use this code:
StringBuffer hexString = new StringBuffer();
for (int i=0;i<out.length;i++) {
hexString.append( String.format("%02x", 0xFF & out[i]) );
}
and then call hexString.toString();

Decrypt Rijndael 256 (from PhP) encoded text in java with little information

I have some data from a external party which is encrypted according to them in: 'Rijndeal 256 with the private key'
Alongside these records there are a public and private key certificate which look like RSA certificates.
From what i've learned so far it seems the common way to use encryption with certifcates is to generate a 'secret key' or some kind in initialization vector and use this to encrypted text. So i'm thinking this is probably what they have done (the data was encrypted by a PHP application)
I'm trying to decrypt this text with javax.crypto.Cipher but i think i problably need more information on the specific encryption, but i dont really know what information to ask for, and think its likely the 'default options' will probably work. (Communication with the supplying party is difficult and slow).
i'm currently Using the following code to get the private key:
InputStreamReader ir = new InputStreamReader(the_inputstream_for_the_private_key_record);
Security.addProvider(new BouncyCastleProvider());
pemr = new PEMReader(ir);
Object o = pemr.readObject();
keyPair kp = (KeyPair) o;
return kp.getPrivate();
This seems to work as i get a instantiated PrivateKey object without errors the toString looks like:
RSA Private CRT Key
modulus: c98faa50ba69<trimmed>
public exponent: 10001
private exponent: bb889fbe5cb2a6763f...<trimmed>
primeP: eb73e85dc636f5751b...<trimmed>
primeQ: db269bd603a2b81fc9...<trimmed>
primeExponentP: 85b9f111c190595cc8...<trimmed>
primeExponentQ: a66d59a75bb77530de...<trimmed>
crtCoefficient: 79415b078c4c229746...<trimmed>
For each record i also have a entry like the following:
{
"decryptedLength":128389,
"symKeyLength":32,
"symKey":"SImE8VnSZaAu1Ve...<trimmed (this is always 685 chars long) >...ayaJcnpSeOqAGM7q="
}
Basically this is where i'm a bit stuck.
My guess would be that that 'symkey' value is encrypted with RSA which in turn when decrypted would yield the secretKey for the AES part, but if i try:
Cipher rsaCipher = Cipher.getInstance("RSA");
rsaCipher.init(Cipher.DECRYPT_MODE, key);
byte[] b = rsaCipher.doFinal('symkey'.getbytes());
this gets me "javax.crypto.IllegalBlockSizeException: Data must not be longer than 512 bytes", which seems logical since this string is 685characters long
I'm probably missing something very obvious here...
Any suggestions are appreciated.

Just guessing, but I think the value
"symKey":"SImE8VnSZaAu1Ve...<trimmed (this is always 685 chars long) >...ayaJcnpSeOqAGM7q="
is the base64 encoded output from RSA encryption using a 4096-bit public key. You need to first base64 decode the value into a byte[] array, then decrypt it with the private key, the result of which will be a 256-bit key. Note that "Rijndael 256" is ambiguous, since Rijndael supports both a 256 bit blocksize and also a 256 bit keysize.

with GregS's answer i finaly got this to work.
(adding an answer in case someone else needs to decrypt similar php encoded stuff).
The first part was to decrypt de symmetricKey ("symkey") from the metaData string
This was as Greg notes a Base64 encoded, RSA encrypted key which was decoded like so:
Cipher rsaCipher = Cipher.getInstance("RSA");
rsaCipher.init(Cipher.DECRYPT_MODE, key);
byte[] encryptedRijndaelKey = Base64.decodeBase64(base64EncodedSymetricKey); //from the metaData
byte[] rijndaelKeyBytes = rsaCipher.doFinal(encryptedRijndaelKey);
This Rijndael key was then used to decrypt de actual encrypted data like so:
RijndaelEngine rijndaelEngine = new RijndaelEngine(256); // *1 *2
KeyParameter keyParam = new KeyParameter(rijndaelKeyBytes)
rijndaelEngine.init(false, keyParam); //false == decrypt
PaddedBufferedBlockCipher bbc = new PaddedBufferedBlockCipher(rijndaelEngine, new ZeroBytePadding()); // *3
byte[] decryptedBytes = new byte[decryptedLenght]; //from the storageOptions string
int processed = bbc.processBytes(inputBytes, 0, inputBytes.length, decryptedBytes, 0);
bbc.doFinal(decryptedBytes, processed);
*1 because the Sun JCA only supports common AES which has a 128bits keysize i had to use a different provider (BouncyCastle).
*2 apparently the blocksize was also 256 bits (trail & error)
*3 apparently there was no padding used, thus the ZeroPadding for padding (again trail & error).

The symKey value is Base64 encoded and must be decoded before you can do the private key decryption on it. Also, the symmetric encryption sounds like it is AES-256. (AES is based on the Rijndael cipher).