We Keep Coding

Test a REST Endpoint with AES Encrypted data

Environment: Java 8 / SpringBoot v2.0.2
I'm trying to fix a security issue in the code by changing Cipher instance from just "AES" to "AES/GCM/NoPadding". However, the existing test for the REST endpoint which has an encrypted path parameter value which uses this encrypt method fails.
Here is my encrypt method,
public String encrypt(final Long transactionId) {
Assert.notNull(transactionId, "Transaction Id Should Not Be null");
String encryptedText = "";
try
{ final byte[] encodedSecretKey = Base64.decodeBase64(encryptKey);
final SecretKeySpec secretKey = new SecretKeySpec(encodedSecretKey, "AES");
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(1, secretKey);
final byte[] contentAsBytes = transactionId.toString().getBytes();
byte[] contentAsByteCypherText = cipher.doFinal(contentAsBytes);
byte[] iv = cipher.getIV();
byte[] message = new byte[NUMBER_OF_IV_BYTES + contentAsByteCypherText.length];
System.arraycopy(iv, SRC_POSITION, message, DEST_POSITION, NUMBER_OF_IV_BYTES);
System.arraycopy(contentAsByteCypherText, SRC_POSITION, message, NUMBER_OF_IV_BYTES, contentAsByteCypherText.length);
encryptedText = Base64.encodeBase64URLSafeString(message);
} catch (NoSuchAlgorithmException | NoSuchPaddingException | InvalidKeyException | IllegalBlockSizeException | BadPaddingException e) {
LOGGER.error("Failed to encrypt Transaction ID ", e);
throw new BlahException("Failed to encrypt transaction id", e);
}
return encryptedText;
}
My Test
public void testJourney(final Long transactionId) throws Exception {
final String request = loadExpectedContent(transactionId);
mockRestServiceServer.reset();
mockRestServiceServer.expect(requestTo("/spring/rest/transaction/" + webClient.encrypt(transactionId)))
.andExpect(method(HttpMethod.GET))
.andRespond(withSuccess().contentType(MediaType.APPLICATION_JSON).body(request));
final CompletableFuture<String> completable = webClient.getWebJourney(transactionId);
mockRestServiceServer.verify();
final String response = completable.get();
Assert.assertNotNull(response);
LOGGER.info(LoggingUtils.format(LoggingUtils.keyValue("Request", transactionId),
LoggingUtils.keyValue("Response", LoggingUtils.parse(response))));
}
In this test the encrypt method is hit twice.
when trying to create the expect
inside webClient.getWebJourney(transactionId)
The difference is when Cipher.getInstance("AES/GCM/NoPadding"); is used those two times it returns two different values failing the test. But when Cipher.getInstance("AES"); is used (without IV) it returns same value both times.
My question is how can I test this REST endpoint with "AES/GCM/NoPadding"?

To test it you don't want to get the same ciphertext, you want to get the same cleartext after decryption. With GCM you want a test that encrypts something twice and doesn't get the same result.
You should not re-use IV values (with GCM the same IV and cleartext results in the same ciphertext).
So this:
byte[] iv = cipher.getIV();
Will return a secure random generated array of 12 bytes, hence the ciphertext is different each time.
Calling Cipher.getInstance("AES") without IV is definitely not what you want to do:
it's likely defaulting to ECB, which is old and badly broken (have a look at the penguin in the "encrypted" image.
not using an IV is why you get the same result - the idea of the IV is to start with some randomness to avoid repeating patterns that can be used to find the key.
I would change the test to perform encryption, then decrypt (in the test) and ensure the ciphertext is the same.

Sending IV with cipher text

I am encrypting my message with a constant secret key and random IV using AES/CTR/NoPadding in Java.
I plan to prepend the IV to the cipher text and then Base64 encode it.
Now, there is one problem that is bothering me. There can be multiple IV + Cipher text combinations that can result in my original message. Isn't that a problem? Also, is it safe to send IV as such (i.e. prepend/append to the cipher text) or there is some procedure that I should follow?
I am relatively new to cryptography, so pardon me if it's very simple question. I couldn't find any satisfying answer to this.
EDIT:
public static String encrypt(String message) {
try {
Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
byte[] iv = generateRandomIV();
cipher.init(Cipher.ENCRYPT_MODE, SECRET_KEY, new IvParameterSpec(iv));
byte[] cipherText = cipher.doFinal(message.getBytes("utf-8"));
return DatatypeConverter.printBase64Binary(concat(iv, cipherText));
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}
public static String decrypt(String encryptedMessage) {
try {
byte[] bytes = DatatypeConverter.parseBase64Binary(encryptedMessage);
byte[] iv = getIV(bytes);
byte[] cipherText = getCipherText(bytes);
Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, SECRET_KEY, new IvParameterSpec(iv));
return new String(cipher.doFinal(cipherText));
} catch (Exception ex) {
ex.printStackTrace();
return encryptedMessage;
}
}
private static byte[] getIV(byte[] bytes) {
return Arrays.copyOfRange(bytes, 0, 16);
}
private static byte[] getCipherText(byte[] bytes) {
return Arrays.copyOfRange(bytes, 16, bytes.length);
}
and then
public static void main(String[] args) {
System.out.println(decrypt("wVroKV1UnL2NXiImS83hLKpLLJKk"));
System.out.println(decrypt("Q0tWAMZDhqMo0LbtEY7lF9D8Dkor"));
}
Both these produce same output -- "goody"

There can be multiple IV + Cipher text combinations that can result in my original message. Isn't that a problem?
The reason that you're using different IV's is that you can send the same message twice using the same key, with different ciphertext.
If it would generate the same ciphertext an adversary would know that the same message was send, leaking information about the message. So the idea of the IV is that it generates a different ciphertext and in most cases that is beneficial rather than a problem.
If it is a problem depends on your protocol. Note that the ciphertext length may still show information about the plaintext (i.e. "Affirmative, Sergeant" will of course be different than the encryption of "No").
You'll need an authentication tag / MAC value to protect against change of the message. Furthermore, you may want to include something like a message sequence number to make sure that replay attacks don't happen.
However, the deeper you go the more complex encryption becomes. If you require secure transport then in the end it is infinitely easier to use TLS or SSH (or any other applicable transport layer security).
Also, is it safe to send IV as such (i.e. prepend/append to the cipher text) or there is some procedure that I should follow?
Prepending it ("prepend" is not a word in many dictionaries, you could use "prefix" as well) is a common way of handling the IV, yes. The IV may be sent anyway and it doesn't need to be kept confidential. In the case of CBC however it must be random, not a sequence number.

symmetric encryption in c# resembles JAVA

private static byte[] encryptData(ByteArrayOutputStream data, byte[] symmetricKey) throws EncryptionException {
try {
SecretKey secKey = new SecretKeySpec(symmetricKey, "AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secKey);
return cipher.doFinal(data.toByteArray());
} catch (NoSuchAlgorithmException | NoSuchPaddingException | IllegalBlockSizeException |
InvalidKeyException |
BadPaddingException e) {
throw new EncryptionException(e);
}
}
I have a situation where I need to encrypt data using .NET and decrypt the same data using JAVA. Essentially, I need to rewrite the above encryption method in .NET.
public byte[] Encrypt(byte[] key, byte[] plainText)
{
using (AesCryptoServiceProvider aesProvider = new AesCryptoServiceProvider())
{
using (ICryptoTransform encryptor = aesProvider.CreateEncryptor(key, magicIV))
{
using (MemoryStream ms = new MemoryStream())
{
using (CryptoStream cs = new CryptoStream(ms, encryptor, CryptoStreamMode.Write))
{
cs.Write(plainText, 0, plainText.Length);
}
byte[] cipherText = ms.ToArray();
return cipherText;
}
}
}
}
The above code I used somewhere mandates IV which JAVA is not asking for. What is the IV used in JAVA code?
I tried many links which didn't work.
Symmetric Encryption between .NET and Java
Please help

If your current Java decryption code also does not ask for an IV (and your decryption returns the same data you encrypted) then Cipher.getInstance("AES") is returning an object using the ECB block mode.
.NET symmetric algorithms default to the CBC block mode, which requires an IV.
You have a couple of options:
Set aesProvider.Mode = CipherMode.ECB before calling CreateEncryptor.
You probably don't want this, see the penguin at https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Common_modes.
Pass aesProvider.IV to the IV parameter of CreateEncryptor. The IV property will make a cryptographically random value on the first read if it's not set.
You will need to pass this data to the decryption routine, which should then use "AES/CBC/PKCS5Padding", and set the IV however one does that in Java.
One common method of transport is to simply prepend the data to the ciphertext, then just pick off the first 16 bytes at decryption time.
DO NOT use a fixed value for an IV, because it's then almost the same as ECB.

Is there any difference, if I init AES cipher, with and without IvParameterSpec

I was wondering, is there any difference, if I init AES cipher, with and without IvParameterSpec?
With IvParameterSpec
SecretKeySpec skeySpec = new SecretKeySpec(key, "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec, new IvParameterSpec(new byte[16]));
Without IvParameterSpec
SecretKeySpec skeySpec = new SecretKeySpec(key, "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec);
I tested with some sample test data, their encryption and decryption result yield the same.
However, since I'm not the security expert, I don't want to miss out anything, and create a potential security loop hole. I was wondering, which is the correct way?

A bit of background (I'm sorry if you already know this, it's just worth making sure we're using the same terminology):
AES is a block cipher, an encryption algorithm that operates on 128-bit blocks.
CBC is a block cipher mode, a way of using a block cipher to encrypt large amounts of data.
Block cipher modes need an initialisation vector (IV), which is a block of initialisation data, usually the same size as the block size of the underlying cipher.
(The Wikipedia on block cipher modes - http://en.wikipedia.org/wiki/Block_cipher_mode - is really good, and makes it clear why you need an IV.)
Different block modes impose different requirements on the IV selection process, but they all have one thing in common:
You must never encrypt two different messages with the same IV and key.
If you do, an attacker can usually get your plaintext, and sometimes your key (or equivalently useful data).
CBC imposes an additional constraint, which is that the IV must be unpredictable to an attacker - so artjom-b's suggestion of using a SecureRandom to generate it is a good one.
Additionally, as artjob-b points out, CBC only gives you confidentiality. What that means in practice is that your data is kept secret, but there's no guarantee that it arrives in one piece. Ideally, you should use an authenticated mode, such as GCM, CCM, or EAX.
Using one of these modes is a really, really good idea. Encrypt-then-MAC is unwieldy even for the experts; avoid it if you can. (If you have to do it, remember that you must use different keys for encryption and MAC.)

By default when you encrypt - your cipher will generate a random IV. You must use exactly that specific IV when you decrypt that data.
The good news is that IV is not a secret thing - you can store it in public. The main idea is to keep it different for every encrypt-decrypt operation.
Most of the times you will need to encrypt-decrypt various data and storing each IV for each piece of data is a pain.
That's why IV is often stored along with the encrypted data in a single string, as a fixed size prefix.
So that when you decrypt your string - you definitely know that first 16 bytes (in my case) are your IV, the rest of the bytes - are the encrypted data and you need to decrypt it.
Your payload (to store or send) will have the following structure:
[{IV fixed length not encrypted}{encrypted data with secret key}]
Let me share my encrypt and decrypt methods, I'm using AES, 256 bit secret key, 16 bit IV, CBC MODE and PKCS7Padding.
As Justin King-Lacroix stated above you better use GCM, CCM, or EAX block modes. Do not use ECB!
Result of encrypt() method is safe & ready to store in DB or send anywhere.
Note a comment where you can use custom IV - just replace new SecureRandom() with new IvParameterSpec(getIV()) (you can input there your static IV but this is strongly NOT recommended)
private Key secretAes256Key is a class field with a secret key, it is initialized in the constructor.
private static final String AES_TRANSFORMATION_MODE = "AES/CBC/PKCS7Padding"
the encrypt() method:
public String encrypt(String data) {
String encryptedText = "";
if (data == null || secretAes256Key == null)
return encryptedText;
}
try {
Cipher encryptCipher = Cipher.getInstance(AES_TRANSFORMATION_MODE);
encryptCipher.init(Cipher.ENCRYPT_MODE, secretAes256Key, new SecureRandom());//new IvParameterSpec(getIV()) - if you want custom IV
//encrypted data:
byte[] encryptedBytes = encryptCipher.doFinal(data.getBytes("UTF-8"));
//take IV from this cipher
byte[] iv = encryptCipher.getIV();
//append Initiation Vector as a prefix to use it during decryption:
byte[] combinedPayload = new byte[iv.length + encryptedBytes.length];
//populate payload with prefix IV and encrypted data
System.arraycopy(iv, 0, combinedPayload, 0, iv.length);
System.arraycopy(encryptedBytes, 0, combinedPayload, iv.length, encryptedBytes.length);
encryptedText = Base64.encodeToString(combinedPayload, Base64.DEFAULT);
} catch (NoSuchAlgorithmException | BadPaddingException | NoSuchPaddingException | IllegalBlockSizeException | UnsupportedEncodingException | InvalidKeyException e) {
e.printStackTrace();
}
return encryptedText;
}
And here is the decrypt() method:
public String decrypt(String encryptedString) {
String decryptedText = "";
if (encryptedString == null || secretAes256Key == null)
return decryptedText;
}
try {
//separate prefix with IV from the rest of encrypted data
byte[] encryptedPayload = Base64.decode(encryptedString, Base64.DEFAULT);
byte[] iv = new byte[16];
byte[] encryptedBytes = new byte[encryptedPayload.length - iv.length];
//populate iv with bytes:
System.arraycopy(encryptedPayload, 0, iv, 0, 16);
//populate encryptedBytes with bytes:
System.arraycopy(encryptedPayload, iv.length, encryptedBytes, 0, encryptedBytes.length);
Cipher decryptCipher = Cipher.getInstance(AES_TRANSFORMATION_MODE);
decryptCipher.init(Cipher.DECRYPT_MODE, secretAes256Key, new IvParameterSpec(iv));
byte[] decryptedBytes = decryptCipher.doFinal(encryptedBytes);
decryptedText = new String(decryptedBytes);
} catch (NoSuchAlgorithmException | BadPaddingException | NoSuchPaddingException | IllegalBlockSizeException | InvalidAlgorithmParameterException | InvalidKeyException e) {
e.printStackTrace();
}
return decryptedText;
}
Hope this helps.

When no IvParameterSpec is provided then the Cipher should initialize a random IV itself, but it seems that in your case, it doesn't do this (new byte[16] is an array filled with 0x00 bytes). It seems the Cipher implementation is broken. In that case you should always provide a new random IV (necessary for semantic security).
This is usually done this way:
SecureRandom r = new SecureRandom(); // should be the best PRNG
byte[] iv = new byte[16];
r.nextBytes(iv);
cipher.init(Cipher.ENCRYPT_MODE, skeySpec, new IvParameterSpec(iv));
When you then send or store the ciphertext, you should prepend the IV to it. During decryption you only need to slice the IV off the front of the ciphertext to use it. It doesn't need to be kept secret, but it should be unique.
Note that CBC mode alone only gives you confidentiality. If any type of manipulation of ciphertexts (malicious or non-malicious) is possible then you should use an authenticated mode like GCM or EAX. Those will also give you integrity in addition to confidentiality. If you don't have access to those (SpongyCastle has them), you could use a message authentication code (MAC) in an encrypt-then-MAC scheme, but it is much harder to implement correctly.

Java public private key decryption issue

I am trying to encrypt and decrypt a message as mentioned in the below code. Basically I want to encrypt a message with a public key and convert that encrypted message from byte array to String. And decrypt this string into original text. Here are the both methods. Here encryption works fine but decryption fails (error is "Data must start with zero"). I think this is causing because I convert encrypted byte array into String.
How do I solve this? (I want to have encrypted byte array as string and use it for decryption) Is there any other approach (with public and private keys)
public static String getEncryptedMessage(String publicKeyFilePath,
String plainMessage) {
byte[] encryptedBytes;
try {
Cipher cipher = Cipher.getInstance("RSA");
byte[] publicKeyContentsAsByteArray = getBytesFromFile(publicKeyFilePath);
PublicKey publicKey = getPublicKey(publicKeyContentsAsByteArray);
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
encryptedBytes = cipher.doFinal(plainMessage.getBytes());
return new String(encryptedBytes);
} catch (Throwable t) {
}
}
public static String getDecryptedMessage(
String privateKeyFilePath, String encryptedMessage)
{
byte[] decryptedMessage;
try {
Cipher cipher = Cipher.getInstance("RSA");
byte[] privateKeyContentsAsByteArray = getBytesFromFile(privateKeyFilePath);
PrivateKey privateKey = getPrivateKey(privateKeyContentsAsByteArray);
cipher.init(Cipher.DECRYPT_MODE, privateKey);
decryptedMessage = cipher.doFinal(encryptedMessage.getBytes());
return new String(decryptedMessage);
} catch (Throwable t) {
}

If you look at this page (http://www.wikijava.org/wiki/Secret_Key_Cryptography_Tutorial) you will need to do base-64 encoding to turn the bytes into a string, then to decrypt it you would just decode it then decrypt.
Base-64 encoding uses the first 7 bits of a byte, to make something that is printable or emailable, for example.
UPDATE:
I made a mistake, there are 64 characters that it would be encoded in, again, in order to make it easier to use as something printable.

Why don't you treat the message as byte array from encryption to decryption? Why changing it to String in the middle? (I know it seems like a question, but it's actually an answer...)

Using RSA directly on unformatted data may leave your application vulnerable to an adaptive chosen ciphertext attack. For details please see Chapter 8, pages 288-289, of the Handbook of Applied Cryptography, a freely-available book from CRC Press. (It's well worth buying the bound edition, if you're really interested in cryptography -- you'll be stunned at the quality for the price.)
Because of this attack, most protocols that integrate RSA use RSA for encrypting randomly-generated session keys or signing hash functions with outputs that ought to be indistinguishable from random, OR using very carefully formatted messages that will fail to be correctly interpreted. (See Note 8.63 in HAC for details.)