We Keep Coding

Encoding PrivateKey in PKCS#1 format, then restoring to PrivateKey

I have a 2048-bit RSA PublicKey that I wish to convert into a PKCS#1 formatted ByteArray, and then restore the PublicKey object from that ByteArray. I'm converting to PKCS#1 with BouncyCastle using Kotlin on the JVM with the following code:
fun privKeyToPkcs1Bytes(privKey: PrivateKey): ByteArray {
val privKeyInfo = PrivateKeyInfo.getInstance(privKey.encoded)
val privKeyAsn1Encodable: ASN1Encodable = privKeyInfo.parsePrivateKey()
val privKeyAsn1Primitive: ASN1Primitive = privKeyAsn1Encodable.toASN1Primitive()
val privKeyPkcs1Bytes: ByteArray = privKeyAsn1Primitive.getEncoded()
return privKeyPkcs1Bytes
}
and then I'm trying to restore the returned ByteArray to a PrivateKey object using the following code:
fun privKeyFromPkcs1Bytes(privKeyPkcs1Bytes: ByteArray): PrivateKey {
val rsaPrivKey: RSAPrivateKey = RSAPrivateKey.getInstance(ASN1Sequence.fromByteArray(privKeyPkcs1Bytes))
val privKeySpec: RSAPrivateKeySpec = RSAPrivateKeySpec(rsaPrivKey.modulus, rsaPrivKey.privateExponent)
val privKey: PrivateKey = KeyFactory.getInstance("RSA").generatePrivate(privKeySpec)
return privKey
}
The code runs without error, but the PKCS#8 representations of the original PrivateKey and restored PrivateKey do not match. I have thoroughly searched for a resolution to this issue, and haven't found a solution that works. What am I doing wrong here?

how to use (RSA/ECB/PKCS1Padding) in android

About a year ago, I wrote an application for Android and used a class in it RSA In this class, there was the following code snippet and the application worked
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding")
But when I re-entered the application code, I did not open the new encrypted information to change the private key until I changed the above code line to the following code line.
Cipher cipher = Cipher.getInstance(keyFactory.getAlgorithm());
The problem is that if I replace the above code snippet in class RSA it is no longer possible to open previously encrypted information (with the same keys as before).
And I see the following error
javax.crypto.BadPaddingException: error:04000084:RSA routines:OPENSSL_internal:PKCS_DECODING_ERROR
RSA decryption
public static byte[] decryptByPrivateKey(byte[] data, String key)
throws Exception {
byte[] keyBytes = decryptBASE64(key);
PKCS8EncodedKeySpec pkcs8KeySpec = new PKCS8EncodedKeySpec(keyBytes);
KeyFactory keyFactory = KeyFactory.getInstance(KEY_ALGORITHM);
Key privateKey = keyFactory.generatePrivate(pkcs8KeySpec);
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
// Cipher cipher = Cipher.getInstance(keyFactory.getAlgorithm());
cipher.init(Cipher.DECRYPT_MODE, privateKey);
return cipher.doFinal(data);
}

RSA key pairs can be used within different RSA based schemes, such as PKCS#1 and OAEP padding for encryption, and PKCS#1 and PSS padding for signing. However, there is only one key pair generation possible, which is simply denoted "RSA".
If only "RSA" is used as input string it will use the defaults set for the specific cryptography provider, which is - in this case - the first provider that implements RSA using keys in software. Apparently that's different on Android from PKCS#1 padding (assuming that you still use the original list of providers, of course). One stupid thing in Java is that you cannot programmatically find out which defaults are used; getAlgorithm() ususally just returns the string you've provided earlier. The only thing you can do is to get the provider using getProvider() and then lookup the defaults...
I would never go for any defaults (except for SecureRandom defaults) as it is unspecified which defaults will be used for Java. Always specify the algorithm in full; your earlier string was fine.

My function
private fun getEncryptCodeWord(publicKey:String, codeWord:String):String{
try{
val publicBytes = Base64.decode(publicKey, Base64.NO_WRAP)
val keySpec = X509EncodedKeySpec(publicBytes)
val keyFactory = KeyFactory.getInstance("RSA")
val pubKey = keyFactory.generatePublic(keySpec)
val encryptCodeWord = Cipher.getInstance("RSA/ECB/PKCS1Padding")
.apply { init(Cipher.ENCRYPT_MODE, pubKey) }
.doFinal(codeWord.toByteArray())
return Base64.encodeToString(encryptCodeWord, Base64.NO_WRAP)
}
catch (ex:Exception){
Crash.recordException(ex)
Crash.setKey("error_get_encrypt_code_word",ex.message)
}
return codeWord
}
and for RSA/ECB/OAEPWithSHA-256AndMGF1Padding
private fun getEncryptCodeWord(publicKey:String,codeWord:String):String{
try{
val publicBytes = Base64.decode(publicKey, Base64.NO_WRAP)
val keySpec = X509EncodedKeySpec(publicBytes)
val keyFactory = KeyFactory.getInstance("RSA")
val pubKey = keyFactory.generatePublic(keySpec)
val sp = OAEPParameterSpec("SHA-256", "MGF1", MGF1ParameterSpec("SHA-1"), PSource.PSpecified.DEFAULT)
val encrypt = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding")
encrypt.init(Cipher.ENCRYPT_MODE, pubKey, sp)
val encryptCodeWord = encrypt.doFinal(codeWord.toByteArray())
return Base64.encodeToString(encryptCodeWord, Base64.NO_WRAP)
}
catch (ex:Exception){
Crash.recordException(ex)
Crash.setKey("error_get_encrypt_code_word",ex.message)
}
return codeWord
}

Spark Scala : Got error javax.crypto.AEADBadTagException: Tag mismatch! from AES GCM binary file decryption

Hi I'm trying to decrypt the zip file using sc.BinaryFiles
Scala version 2.11.12
Spark version 2.3
Here is my code
val input_bin = spark.sparkContext.binaryFiles("file.csv.gz.enc")
val input_utf = input_bin.map{f => f._2}.collect()(0).open().readUTF()
val input_base64 = Base64.decodeBase64(input_utf)
val GCM_NONCE_LENGTH = 12
val GCM_TAG_LENGTH = 16
val key = keyToSpec("<key>", 32, "AES", "UTF-8", Some("SHA-256"))
val cipher = Cipher.getInstance("AES/GCM/NoPadding")
val nonce = input_base64.slice(0, GCM_NONCE_LENGTH)
val spec = new GCMParameterSpec(128, nonce)
cipher.init(Cipher.DECRYPT_MODE, key, spec)
cipher.doFinal(input_base64)
def keyToSpec(key: String, keyLengthByte: Int, encryptAlgorithm: String,
keyEncode:String = "UTF-8", digestMethod: Option[String] = Some("SHA-1")): SecretKeySpec = {
//prepare key for encrypt/decrypt
var keyBytes: Array[Byte] = key.getBytes(keyEncode)
if (digestMethod.isDefined) {
val sha: MessageDigest = MessageDigest.getInstance(digestMethod.get)
keyBytes = sha.digest(keyBytes)
keyBytes = util.Arrays.copyOf(keyBytes, keyLengthByte)
}
new SecretKeySpec(keyBytes, encryptAlgorithm)
}
And I found error
javax.crypto.AEADBadTagException: Tag mismatch!
at com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:571)
at com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1046)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:983)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:845)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
... 50 elided
The input file is quite big (around 2GB).
I'm not sure, I read file incorrectly or the decryption has something wrong.
I also have the Python version and this code is work fine for me
real_key = key
hash_key = SHA256.new()
hash_key.update(real_key)
for filename in os.listdir(kwargs['enc_path']):
if (re.search("\d{12}.csv.gz.enc$", filename)) and (dec_date in filename):
with open(os.path.join(kwargs['enc_path'], filename)) as f:
content = f.read()
f.close()
ct = base64.b64decode(content)
nonce, tag = ct[:12], ct[-16:]
cipher = AES.new(hash_key.digest(), AES.MODE_GCM, nonce)
dec = cipher.decrypt_and_verify(ct[12:-16], tag)
decrypted_data = gzip.decompress(dec).decode('utf-8')
Any suggestion?
Thank you
Update #1
I was able to resolved the problem, by change the file reading method from Spark to be local file reading (scala.io) and add the AAD for decryption also apply the answers from #Topaco and #blackbishop
cipher.init(Cipher.DECRYPT_MODE, key, spec)
cipher.updateAAD(Array[Byte]()) // AAD can be changed to be any value but must be exact the same value when file is encrypted
cipher.doFinal(input_base64.drop(12)) // drop nonce before decrypt
I'm still finding out, why Spark doesn't work

You are not using the same decryption method as in Python. In the scala code, you are ignoring the tag and when you call doFinal you're passing all the cipher.
Try with these changes :
// specify the tag length when creating GCMParameterSpec
val spec = new GCMParameterSpec(GCM_TAG_LENGTH, nonce)
// remove the nonce part from the cipher before calling doFinal
val dec = cipher.doFinal(input_base64.drop(GCM_NONCE_LENGTH))

How to extract the enc encrypted data using AES 256 CBC in JAVA or SCALA?

I need help on AES 256 Decryption using JAVA or SCALA. I am able to extract using openssl.
Base 64 encoded Key: 5UX8IBWruBk1QmMZlZ1ESYmZRiC9w1DsrPpLIP9QF+Q=
Base 64 encoded payload:
U2FsdGVkX19XO99r3f7LgNbPTW8tKexv26mCPUYTMTTiwSSayFvB/QraYJkfzKjEB+tisqzzrU9aZu/tQ5CIVrHHwkUxpyjKEjx3N5q+Ba3weNK/NthpcCsNw5GQxl3NWGoDPe2IFXHMpvpy9xb2mbMnPtwr3m4nF3JzRD6Ft34Q7bHmmTCDkh5kEF9Hx+nbeiLURqLJ1S5YeGq7xhZqalimuQPwT7cr3MPkWPGyZVtNtrKJfIRStoMqP9F2qvm6.
OpenSSL decryption command:
# extract the json from payload
$ openssl enc -d -aes-256-cbc -salt -in data.payload -out output.json -pass file:./key.otkey
input key.otkey mentioned above and data.payload file contain base64 decoded string mentioned above.
I am able to decode using openssl and output :
{"fields":["NSN","store_busn_dt","all_net_sales_amt","all_net_sales_qty","dt_net_sale_qty","brfst_net_sale_qty"],"data":[[38099,"2018-04-01",7675.900000000001,998,752,262]]}
But i can't able to extract using Scala code:
import java.nio.charset.StandardCharsets
import java.util.Base64
import javax.crypto.Cipher
import javax.crypto.spec.SecretKeySpec
import javax.crypto.spec.IvParameterSpec
import javax.xml.bind.DatatypeConverter
object AesDecryption extends App {
val key: String = "5UX8IBWruBk1QmMZlZ1ESYmZRiC9w1DsrPpLIP9QF+Q="
val keyOut = Base64.getDecoder.decode(key)
val otKey: SecretKeySpec = new SecretKeySpec(keyOut, "AES")
val payload: String = "U2FsdGVkX19XO99r3f7LgNbPTW8tKexv26mCPUYTMTTiwSSayFvB/QraYJkfzKjEB+tisqzzrU9aZu/tQ5CIVrHHwkUxpyjKEjx3N5q+Ba3weNK/NthpcCsNw5GQxl3NWGoDPe2IFXHMpvpy9xb2mbMnPtwr3m4nF3JzRD6Ft34Q7bHmmTCDkh5kEF9Hx+nbeiLURqLJ1S5YeGq7xhZqalimuQPwT7cr3MPkWPGyZVtNtrKJfIRStoMqP9F2qvm6"
val encryptedData: Array[Char] = payload.toCharArray
//Base64.getDecoder.decode(payload)
val encData: Array[Byte] = DatatypeConverter.parseBase64Binary(new String(encryptedData))
println(new String(encData, StandardCharsets.ISO_8859_1))
val cipher = Cipher.getInstance("AES/CBC/PKCS5Padding")
// Generating IV.// Generating IV.
val ivSize = 16
val iv = new Array[Byte](ivSize)
// Extract IV.// Extract IV.
System.arraycopy(encData, 0, iv, 0, iv.length)
val ivParameterSpec = new IvParameterSpec(iv)
// extract data
cipher.init(Cipher.DECRYPT_MODE, otKey, ivParameterSpec)
val output: Array[Byte] = cipher.doFinal(encData)
// print result
println(new String(output, StandardCharsets.ISO_8859_1))
}
Output :
Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:989)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:845)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
Excepted output above mentioned json. Thanks in advance.!

I think you're using your initialization vector (IV) wrong.
In your code you derive your ivParameterSpec (IV) from encData (in turn derived from payload i.e. encrypted data)
You don't specify it explicitly during encryption and based on openssl docs:
If not explicitly given it will be derived from the password. See key derivation for details.
So, my advice here is to either:
get IV you need to use from key file you use for encryption with openssl
specify IV explicitly during encryption (openssl -iv <IV here>) and reuse it during decryption
UPDATE: Also, check out this answer for code example (includes using proper IV).

Finally i am able to decrypt the data for give encrypted file and -pass key using AES 256 bit CBC decoder.
$ echo -n "U2FsdGVkX19XO99r3f7LgNbPTW8tKexv26mCPUYTMTTiwSSayFvB/QraYJkfzKjE
B+tisqzzrU9aZu/tQ5CIVrHHwkUxpyjKEjx3N5q+Ba3weNK/NthpcCsNw5GQxl3N
WGoDPe2IFXHMpvpy9xb2mbMnPtwr3m4nF3JzRD6Ft34Q7bHmmTCDkh5kEF9Hx+nb
eiLURqLJ1S5YeGq7xhZqalimuQPwT7cr3MPkWPGyZVtNtrKJfIRStoMqP9F2qvm6
" | openssl enc -aes-256-cbc -salt -base64 -pass pass:5UX8IBWruBk1QmMZlZ1ESYmZRiC9w1DsrPpLIP9QF+Q= -d
Scala Code:
def aesDecryption(secret: String, data: String): String = {
val decoded: Array[Byte] = Base64.getDecoder.decode(data)
val salt = decoded.slice(8,16)
val encryptedData = decoded.slice(16, decoded.length)
val (key, iv) = getKeyIv(secret.getBytes(), salt)
val sKeySpec = new SecretKeySpec(key, "AES")
val ivSpec = new IvParameterSpec(iv)
val cipher = Cipher.getInstance("AES/CBC/PKCS5Padding")
cipher.init(Cipher.DECRYPT_MODE, sKeySpec, ivSpec)
new String(cipher.doFinal(encryptedData)) }
private def getKeyIv(password: Array[Byte], salt: Array[Byte]): (Array[Byte], Array[Byte]) = {
val md5: MessageDigest = MessageDigest.getInstance("MD5")
// first digest
md5.update(password)
md5.update(salt)
val hash1 = md5.digest()
// second digest
md5.reset()
md5.update(hash1)
md5.update(password)
md5.update(salt)
val hash2 = md5.digest()
// third digest
md5.reset()
md5.update(hash2)
md5.update(password)
md5.update(salt)
val hash3 = md5.digest()
val key = hash1.take(16) ++ hash2.take(16)
val iv = hash3.take(16)
(key, iv)
}
aesDecryption(secret, encryptedData) using this functional call, we can able to decrypt the data.
Reference: https://github.com/chmduquesne/minibackup/blob/master/samples/OpensslAES.java

Using public private key combination for symmetric encryption with Java 8

I'm trying to use an asymmetric private and public key combination to generate a symmetric key for encrypting and decrypting some text, but, I'm stuck unable to use the generated key as it is 128bytes in size and this is unacceptable for the AES encryption. I'd like to solve this problem using just the JRE (no external libraries). Do you have a solution?
I've included my example code below, there's a comment indicating the line I get the exception thrown.
(encryptCipher.init(Cipher.ENCRYPT_MODE, tomSecretKeySpec, iv);)
I read about KDF hashing, but Java doesn't seem to have an obvious way of invoking this on my 128byte key. Also, Im not sure this is the right answer since my understanding is that the longer the key, the more secure the encryption (for a given algorithm). Perhaps I need to switch from using AES/CBC/PKCS5Padding, but none of the other algorithms included with the JDK as standard seem to support the 128byte key either.
public void demoSymmetricEncryption() throws NoSuchAlgorithmException, InvalidKeyException, NoSuchPaddingException, InvalidAlgorithmParameterException, UnsupportedEncodingException, IllegalBlockSizeException, BadPaddingException {
String keyAlgorithm = "DiffieHellman";
String keyAgreementAlgorithm = "DiffieHellman";
String keySpecAlgorithm = "AES";
String cipherAlgorithm = "AES/CBC/PKCS5Padding";
KeyPairGenerator keyGenerator = KeyPairGenerator.getInstance(keyAlgorithm);
keyGenerator.initialize(1024, new SecureRandom());
KeyPair tomKeyPair = keyGenerator.generateKeyPair();
PrivateKey tomPrivateKey = tomKeyPair.getPrivate();
PublicKey tomPublicKey = tomKeyPair.getPublic();
KeyPair steveKeyPair = keyGenerator.generateKeyPair();
PrivateKey stevePrivateKey = steveKeyPair.getPrivate();
PublicKey stevePublicKey = steveKeyPair.getPublic();
int maxKeyLen = Cipher.getMaxAllowedKeyLength("AES");
System.out.println("Limited encryption policy files installed : " + (maxKeyLen == 128)); // returns false
KeyAgreement tomKeyAgreement = KeyAgreement.getInstance(keyAgreementAlgorithm);
keyGenerator.initialize(1024, new SecureRandom());
tomKeyAgreement.init(tomPrivateKey);
tomKeyAgreement.doPhase(stevePublicKey, true);
byte[] tomSecret = tomKeyAgreement.generateSecret();
SecretKeySpec tomSecretKeySpec = new SecretKeySpec(tomSecret, keySpecAlgorithm);
KeyAgreement steveKeyAgreement = KeyAgreement.getInstance(keyAgreementAlgorithm);
steveKeyAgreement.init(stevePrivateKey);
steveKeyAgreement.doPhase(tomPublicKey, true);
byte[] steveSecret = steveKeyAgreement.generateSecret();
SecretKeySpec steveSecretKeySpec = new SecretKeySpec(steveSecret, keySpecAlgorithm);
System.out.println("Secret Keys are identical : " + steveSecretKeySpec.equals(tomSecretKeySpec)); // returns true
String initVector = "RandomInitVector";
Cipher encryptCipher = Cipher.getInstance(cipherAlgorithm);
IvParameterSpec iv = new IvParameterSpec(initVector.getBytes("UTF-8"));
// fails because AES key is 128 bytes not 128 bits in length - think I need to use KDF hash to shrink it appropriately.
encryptCipher.init(Cipher.ENCRYPT_MODE, tomSecretKeySpec, iv);
// Attempt to use the cipher
byte[] encryptedData = encryptCipher.doFinal("Hello".getBytes());
Cipher decryptCipher = Cipher.getInstance(cipherAlgorithm);
iv = new IvParameterSpec(initVector.getBytes("UTF-8"));
decryptCipher.init(Cipher.DECRYPT_MODE, steveSecretKeySpec, iv);
byte[] decryptedData = decryptCipher.doFinal(encryptedData);
System.out.println("Decrypted Data : " + new String(decryptedData));
}
The output from the program is as follows:
Limited encryption policy files installed : false
Secret Keys are identical : true
Exception in thread "main" java.security.InvalidKeyException: Invalid AES key length: 128 bytes
at com.sun.crypto.provider.AESCrypt.init(AESCrypt.java:87)
at com.sun.crypto.provider.CipherBlockChaining.init(CipherBlockChaining.java:91)
at com.sun.crypto.provider.CipherCore.init(CipherCore.java:582)
at com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:339)
at javax.crypto.Cipher.implInit(Cipher.java:806)
at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
at javax.crypto.Cipher.init(Cipher.java:1396)
at javax.crypto.Cipher.init(Cipher.java:1327)
at crypto.SymetricEncryptionTest.demoSymmetricEncryption(SymetricEncryptionTest.java:76)
at crypto.SymetricEncryptionTest.main(SymetricEncryptionTest.java:29)

The error is: * Invalid AES key length: 128 bytes*
Valid AES key sizes are 128-bits, 192-bits and 256-bits or in bytes: 16-bytes, 24-bytes and 32-bytes.
Use an AES key size that is valid.
The general method of generation a symmetric key is just to get the bytes from a cryptographic PRNG. For Java see Class SecureRandom.
For key derivation use PBKDF2, see Class SecretKeyFactory and Java Cryptography Architecture Standard Algorithm Name Documentation "PBKDF2WithHmacSHA1" (Constructs secret keys using the Password-Based Key Derivation Function function).
For an example see OWASP Hashing Java but use "PBKDF2WithHmacSHA1" as the algorithm.

The reason the code wasn't working was that I was using incompatible algorithms. The corrections are as follows:
Replace lines:
String keyAlgorithm = "DiffieHellman";
String keyAgreementAlgorithm = "DiffieHellman";
with
String keyAlgorithm = "EC";
String keyAgreementAlgorithm = "ECDH";
int keySize = 128;
and replace lines
keyGenerator.initialize(1024, new SecureRandom());
with
keyGenerator.initialize(keySize, new SecureRandom());
Program now produces output:
Limited encryption policy files installed : false
Secret Keys are identical : true
Decrypted Data : Hello
Technically, you probably also want to Base64 encode the encrypted output and then decode it again prior to the decode as below:
String encryptedData = Base64.encode(encryptCipher.doFinal("Hello".getBytes()));
byte[] decryptedData = decryptCipher.doFinal(Base64.decode(encryptedData));