My goal is to create my own TLS handshake server, so I can understand how encryption between server-client works. I am following this reference. In the Server Key Exchange Generation section from the reference, the private key length are said to be 32 bytes. But the example length is 64 bytes. Also, ssh-keygen output 1612 chars for private.key and 567 chars for private.pub.
My expected result: The private.pub should be used as is in Server Hello section of the reference which sends the public key with length 32 bytes to the client.
My actual result: The ssh-keygen or even keytool (I can't read the file with IntelliJ IDEA, so right now, I am avoiding it) output more than 32 bytes.
The cmd used
ssh-keygen
keytool -genkeypair -keyalg RSA -validity 7 -keystore keystore
test\experiment\tls\tls\TLSHandshake.java
class TLSHandshakeTest {
#Test
void doHandshake() {
String key = "MC4CAQAwBQYDK2VuBCIEIJCRkpOUlZaXmJmam5ydnp+goaKjpKWmp6ipqqusra6v";
byte[] bytes = key.getBytes();
System.out.println(Arrays.toString(bytes));
// output: 64 instead of 32, does this mean that the `private.pub` file must be processed with an encoder?
System.out.println(bytes.length);
}
}
The 32 byte (128 bit) secret key produced in the "server key exchange generation" step is not the RSA private key. It is actually the key for the symmetric encryption algorithm that will be used to protect application data.
What happens is that the RSA asymmetric algorithm is used to establish identities, and to protect the negotiation. During this process the symmetric algorithm is agreed and 128 bit secret keys are generated and exchanged (securely). Once the session is established, the 128 bit keys are use to encrypt and decrypt the data sent over the connection.
Why don't they use RSA for everything?
Basically, it is too computationally expensive to use RSA (or any other asymmetric algorithm) for bulk data encryption / decryption. The symmetric (block cypher) algorithms are much more efficient. (So you could view the initial negotiation as a "key distribution mechanism" for the session private keys.)
As for the size of the private and public key files generated by ssh-keygen:
The files contain more than just one key:
The public key file contains the value of the modulus and the public exponent e
The private key file usually contains the values of the modulus and both e and the private exponent d and several more values (p, q, dp, dq, qinvp). This is part of why the private key file is bigger than the public key file!
The file contents are encoded in ASN.1 which adds some extra type headers.
The (binary) ASN.1 encoded data is then base64 encoded which uses roughly one ASCII character for each 6 bits of data (plus some padding).
Some additional stuff is added to that:
The private key has line breaks added, along with "PEM" header and trailer lines.
The public key has the name of the algorithm (e.g. "ssh-rsa") and a "comment" added which serves to identify the person the key belongs to. (The comment is largely ignored by SSH ... but it is useful when you are manually adding or removing keys from an "authorized keys" file.)
For later versions of OpenSSH, OpenSSH key formats may be used instead of OpenSSL key formats. These use XDR encoding in place of ASN.1. This may be relevant to you if you are trying to write your own SSH implementation1.
As we can see, the "private key" file generated by ssh-keygen actually contains both the public and private keys for the keypair. Indeed it is possible to recover the keypair's public key from a private key file; e.g.
ssh-keygen -f ~/.ssh/id_rsa -y > ~/.ssh/id_rsa.pub
Refer to Create a public ssh key from the private key for more details.
1 - To be honest, I can't see the point of doing that. It is a lot of effort ... just to learn a bunch of details that you most likely don't need to know.
Related
My code using RSA on the front end：
const rsa = new JSEncrypt();
rsa.setPublicKey(k);
const resultText = rsa.encrypt("violet");
console.log(resultText);
My code using RSA in the backend：
byte[] inputByte = org.apache.commons.codec.binary.Base64.decodeBase64(str.getBytes("UTF-8"));
byte[] decoded = org.apache.commons.codec.binary.Base64.decodeBase64(privateKey);
PrivateKey priKey = KeyFactory.getInstance("RSA").generatePrivate(new
PKCS8EncodedKeySpec(decoded));
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.DECRYPT_MODE,priKey);
String outStr=new String(cipher.doFinal(inputByte));
return outStr;
PublicKey like this:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA13gYdCmOjR9yqQD7ldzG
ZXabSon6SiLceCK6vRXf4NMbF+EQke0vRpqU3IZ/S1pFdvoQswQabsA4zf0WACVT
iaGIhWDlPu3mecri8rYtmOSfd8GCE0vEgFNvSD6IXRLPeLCB+i7WENBa4fCEtW8W
Hzdas96CLiESbjSAruRasQXP2OLqEA2GU83/069vh8uRKzui+yw0aAXZFyFyFRFa
lxYltFadVpz3+kBplvpzuj82t4fc3yCRbrpeRyTyX1sz0ULSxx/k3/p1OuJtIq9Y
9uN0G4gxhcDFJ4L41uXOln5CPapk7tlsYobhhvxYHw1rrweY+06hrQ7r0Hblv2nH
GQIDAQAB
-----END PUBLIC KEY-----
PrivateKey like this:
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEA13gYdCmOjR9yqQD7ldzGZXabSon6SiLceCK6vRXf4NMbF+EQ
ke0vRpqU3IZ/S1pFdvoQswQabsA4zf0WACVTiaGIhWDlPu3mecri8rYtmOSfd8GC
E0vEgFNvSD6IXRLPeLCB+i7WENBa4fCEtW8WHzdas96CLiESbjSAruRasQXP2OLq
EA2GU83/069vh8uRKzui+yw0aAXZFyFyFRFalxYltFadVpz3+kBplvpzuj82t4fc
3yCRbrpeRyTyX1sz0ULSxx/k3/p1OuJtIq9Y9uN0G4gxhcDFJ4L41uXOln5CPapk
7tlsYobhhvxYHw1rrweY+06hrQ7r0Hblv2nHGQIDAQABAoIBAAyqFmXde294BblB
QYhRbafRDNeYvIlW+zZkdC1g98OzJMiGhf7NvhWcSFud3CWFrMeNcyXSe+s+iRgy
Y/SmPP6969RLGa5VNVK7RhOV+aAe7/COAyM3NNmGDehlJIaz8FXbqggWcKaUWIMn
K+WuHdK/4ijoTyZ+8yJfG6Er8tisryLQ9io9+op9g/ZWzaUKgu934/cDxUt70bfm
x+ZEPi6YfkJ1uOpXnnadDyw2RUDcvCM3fK3KF5fqM7SJAXY9b1pmLr+Ccn1qkT9G
I+QHidEsGfJciX5AoHnlIMLPMVIPKBbq4GwC/Ngb41LprNJWlPR38N2ySjky/Jyt
159XWHECgYEA9lx2KfFmyLyVjnkIF3JI50mSZAw4YPBBqB27UInacvxXbjfVURht
xK60GB9OkDbFdeNh89x86Gfwvm5bTq4W8YSH4Obd5Fg8XjTuaicTi03CSfF5SdJn
JLLOUmlqP75gkbEPNUoOfqhqq6IbyJVB3egyL90cd2/wCdJOVLEUly8CgYEA3+Y4
lNdl0xedkDNkWsCyyA4iPSUzVxsuch9hW/VGBwzga8rtiNllpKifVvctQOEu/KUe
vVQRF78ojZaMGT33l6TivnUL54z9Lo9uWghoG8TqMfnG34pFPe3R+zvGP87Hrozw
1EUhiMT198SlB/YHrgGGGlJbG+rlm5GIx3lEdDcCgYA4RSw0LlA0v2ZFkX14pYDj
WxmVwnjKI3ZLqObU4XfE1cA+i4AssrC3wNOfwt7V77ywTYxc/9qD1uHVDS3LzdWt
uoCyrOi3tDOtrNdb5asAIXWkIAR9CRLH/hNEHZHIF3rFLDT2DgE7iso6g59m9DiE
L/nulsleunGQPLnpfDzgvwKBgDRV5Q3tl3CTSZJGYQPRnTikDR7LzkdjJCUq7qAH
IhpNyTuJEKL3ZgnqHGzAlERhHpGRqzDIMMKjPUBzW0YfNPuuYA3y4Bh83UV/42SK
KIOtMK0D3JeuA2dparbWDw4lMIm9iiGkEyWcHH6Q6Z6FxN/InWcTrxZEfu0xRI6T
6wtbAoGAfl5dW9LNoaNfQbgIq+PJKZm9f1frza55mFTJgo3ravdb3GmzWVHN8xRf
nLKyKyNLqbkT35IGc39NkALJLxT5RibkAZLiUiwqdMF63sgODbA9AGTmhe+JHS+V
hBmFnCyp6UiN9E4ZAWcZQILa0rRMftMFngAJ3El0ZP+HziRnNzs=
-----END RSA PRIVATE KEY-----
Bu-t, when i do the java code decryption, it reported such an error：
java.security.InvalidKeyException: IOException : DerInputStream.getLength(): lengthTag=111, too big.
How can i solve this problem ?
1. You are decoding wrong. PEM format has a dash-BEGIN line identifying the type of data, a block of base64 encoding the data, and a dash-END line. The BEGIN and END lines are part of the format, but they do not contain base64-encoded data; only the lines in between contain the base64-encoded data. You are apparently passing the whole thing, including the BEGIN and END lines, to commons.codec.Base64, which results in decoding a bunch of garbage before and after the actual data. That garbage isn't valid ASN.1 DER, so when Java tries to parse it as DER it fails.
2. Plus your data is not a PKCS8-clear privatekey. The PEM type 'RSA PRIVATE KEY' is an OpenSSL-defined format that contains a 'traditional' or 'legacy' format, namely the PKCS1 representation of the private key. This is not PKCS8, which is the only key format Java supports natively; that's why the spec class is named PKCS8EncodedKeySpec, because it is a key spec encoded as PKCS8 and more specifically PKCS8-clear. If you fix the above problem by removing the BEGIN and END lines before base64-decoding, Java can parse the result as DER, but not as a PKCS8-clear key; you get a different exception about 'algid parse error, not a sequence'. To fix this there are 5 approaches:
change whatever process you use to initially generate the keypair so it generates PKCS8, not OpenSSL-legacy PKCS1. Especially since you need anyway to replace the keypair you compromised by publishing it, as 207421 said. You give no clue what that process is or was, so I can't give any details.
convert your generated privatekey, or a copy, to PKCS8-clear. This is not programming or development and offtopic, but if you have or get OpenSSL (on the same or any accessible and secure system), you can do
openssl pkey -in oldfile -out newfile # 1.0.0 up only, but older is now rare
# or
openssl pkcs8 -topk8 -nocrypt -in oldfile -out newfile # even ancient versions
Once you have a PKCS8-clear file, just remove the BEGIN and END lines and base64-decode what is left, and pass that to KeyFactory as PKCS8EncodedKeySpec as you already do.
use https://www.bouncycastle.org . The 'bcpkix' jar has (Java) code to read a large range of OpenSSL-supported PEM formats, including the RSA-PKCS1 private key format you have. There are lots of existing Qs about this; just search for PEMParser and JcaPEMKeyConverter.
convert it yourself. Decode the body of the file you have, after removing the BEGIN and END lines, to get the PKCS1 key, then build the PKCS8 format for that key, and then pass it to KeyFactory as PKCS8EncodedKeySpec. See answers by Noa Resare and Jean-Alexis Aufauvre on Getting RSA private key from PEM BASE64 Encoded private key file or mine in Java: Convert DKIM private key from RSA to DER for JavaMail .
do it entirely yourself. Decode the file you have without BEGIN/END to get PCKS1, parse that as DER following e.g. RFC8447, and build RSAPrivateCrtKeySpec. Some other As on the Q I linked just above do this. However, this requires either: using undocumented internal sun.* classes, which used to work in Java (hence the existing answers) but which 'modular' Java versions (9 up) since 2017 have steadily made more difficult or impossible; using BouncyCastle which has documented (and good) support for ASN.1 -- but then it's easier to use bcpkix for the whole job as above; or writing your own ASN.1 parsing, which is a good deal of work.
PS: encrypting text with RSA is usually a bad design; it's not suited for that. But that's not really a programming issue and doesn't belong here.
So I have a very basic openssl command that was provided to me openssl smime -encrypt -binary -aes-256-cbc -in $inPath -out $encryptedPath -outform DER $pubCert, this command also works correctly and outputs an encrypted file. I need to use the equivalent of this command in a java application, preferably without invoking process and using openssl itself (only because I feel like that is probably bad practice).
I have researched quite a lot and there does not seem to be any equivalent out there that I can find.. I have tried several things and most of them do not seem to work. The weird thing is... I am able to get a simple "Hello World" string to encrypt using the code I wrote (although I don't believe it was encrypting it correctly because I had the cipher set to "RSA" not "AES") but when the byte array was coming from a file, it silently failed and just wrote 0 bytes. Right now this is what my code looks like.
Cipher aes = Cipher.getInstance("RSA");
CertificateFactory certF = CertificateFactory.getInstance("X.509");
File public_cert = new File( getClass().getClassLoader().getResource("public.crt").getFile());
FileInputStream certIS = new FileInputStream(public_cert);
X509Certificate cert = (X509Certificate) certF.generateCertificate(certIS);
certIS.close();
aes.init(Cipher.ENCRYPT_MODE, cert);
File tarGz = new File("C:\\volatile\\generic.tar.gz");
FileInputStream fis = new FileInputStream(tarGz);
byte[] tarGzBytes = FileUtils.readFileToByteArray(tarGz);
tarGzBytes = "Hello World".getBytes();
ByteArrayInputStream bais = new ByteArrayInputStream("Hello World".getBytes());
File encFile = new File("C:\\volatile\\generic.tar.gz.enc");
FileOutputStream enc = new FileOutputStream(encFile);
CipherOutputStream cos = new CipherOutputStream(enc, aes);
cos.write(tarGzBytes);
//IOUtils.copy(fis, cos);
//IOUtils.copy(bais, cos);
cos.flush();
cos.close();
So this works, and encrypts a little file with Hello World encrypted in it. I don't believe this is AES-256-CBC though, and it does not work when I use the FileUtils.readFileToByteArray(tarGz), although the resulting byte array in a debugger is correctly sized at about 94MB. Which seems really odd to me, that it works with "Hello World".toByteArray() and not FileUtils.readAllBytes(tarGz). Also as a side note, the ByteArrayInputStream using IOUtils.copy works, whereas the FileInputStream version writes 0 bytes as well.
Also, when I set the cipher mode to AES/CBC/PKCS5Padding (because I found something online suggesting to set it to that and it looks more like what I want) I get the following error message:
java.security.InvalidKeyException: No installed provider supports this key: sun.security.rsa.RSAPublicKeyImpl
at javax.crypto.Cipher.chooseProvider(Cipher.java:892)
at javax.crypto.Cipher.init(Cipher.java:1724)
~~~~
If anyone has any suggestions, or if I need to provide more information please let me know. I am fairly stuck right now and I am at this point debating writing a script to simply run the openssl command and run that script from java...
Conclusion
After reading through #dave-thompson-085's answer I realized that there was a really good reason why I could not find what I was wanting to do. So therefore I decided to go ahead and just call the openssl process from java using a process builder. I was able to recreate the openssl command from above as a Process in java, start it and run it with the following code:
File cert = new File(getClass().getClassLoader().getResource("public.crt").getFile());
ProcessBuilder openSslBuilder = new ProcessBuilder("openssl", "smime", "-encrypt", "-binary",
"-aes-256-cbc", "-in", "C:\\volatile\\generic.tar.gz", "-out",
"C:\\volatile\\generic.tar.gz.enc", "-outform", "DER", cert.getPath());
Process openssl = openSslBuilder.start();
openssl.waitFor();
System.out.println(openssl.exitValue());
openssl.destroy();
Hopefully this helps someone else who is looking to attempt this as well and maybe save someone a bunch of time!
First, to be clear: the openssl smime command actually handles both S/MIME and CMS (aka PKCS7) formats; these are related but different standards that basically use different file formats for essentially the same cryptographic operations. With -outform DER you are actually doing CMS/PKCS7.
Second and more fundamental: CMS/PKCS7, and S/MIME, and most other common cryptographic schemes like PGP, actually does hybrid encryption. Your data is not actually encrypted with RSA; instead your data is encrypted with a symmetric algorithm (here AES-256-CBC, since you selected that) using a randomly generated key called the DEK (data encryption key) and the DEK is encrypted with RSA using the recipient's publickey (obtained from their certificate), and both of those results plus a good deal of metadata is arranged into a fairly complicated data structure. The recipient can parse the message to extract these pieces, then use RSA with their privatekey to decrypt the DEK, then AES-decrypt the data with the DEK. Note you always use RSA keys for RSA, and AES keys for AES; symmetric keys are pretty much all just bits and only vary in size, but public-key cryptographic keys including RSA (also DH, DSA, ECC and more) are much more complicated and cannot be intermixed.
Trying to encrypt data directly with RSA as you did, in addition to being wrong, won't work in general because RSA can only encrypt limited amounts of data, depending on the key size used, typically about 100-200 bytes. Symmetric encryption also has some limits, but they are generally much larger; AES-CBC is good for about 250,000,000,000,000,000 bytes.
If you want to implement this yourself, you need to read the standard for CMS particularly the section on EnvelopedData using KeyTransRecipientInfo (for RSA), combined with the rules for ASN.1 BER/DER encoding. This is not a simple job, although it can be done if you want to put the effort in.
If you can use a third-party library in Java, the 'bcpkix' jar from https://www.bouncycastle.org has routines that support CMS, among several other things. This is usually easy if you are writing a program to run yourself, or in your department. If this is to be delivered to outside users or customers who may not like having to manage a dependency, maybe not.
That said, running another program to do something isn't necessarily bad practice in my book, and can be done directly from java (no script). Unless you (need to) do it very often, such as 100 times a second.
How can I convert this RSA public key:
109120132967399429278860960508995541528237502902798129123468757937266291492576446330739696001110 6039072308886100726558188253585034290 57592827629436413108566029093628 2126359538366865626758497206207862794310902180176810615217550567108238764764442605581471797071 19674283982419152118103759076030616683978566631413
to *.pem file?
A "pem" file is not really a format, and different kinds of objects may be be stored in things called "pem" files. Most often these files contain either a base64-encoded X509 certificate or else a base64-encoded private key object.
Assuming you want an X509 certificate, you should next realize that a certificate consists of many fields, only one of which is the public key. So would need to decide on the values of the other fields. Finally, a certificate must be signed, with a private key.
PS. An RSA public key consists of a modulus and a public exponent. What is your public exponent?
If your hex string is just a simple convert from base64 to hex. Then you can reverse it. Here is the script I did (in PHP) to solve my case:
<?php
$list = array_slice($argv, 1);
foreach ($list as $file) {
$hex = str_replace("\n", "", file_get_contents($file));
$pem = str_replace(".hex", ".pem", $file);
$b64 = base64_encode(hex2bin($hex));
$fd = fopen($pem, 'w');
fprintf($fd, "-----BEGIN PUBLIC KEY-----\n%s\n-----END PUBLIC KEY----\n", implode("\n", str_split($b64, 64)));
fclose($fd);
}
Given a list of .hex files, it convert back to ".pem". You can run it like this:
php script.php *.hex
I would use OpenSSL. Assuming that you have the key in DER format, you can use this command to convert from DER to PEM.
openssl x509 -inform der -in input.der -out output.pem
If you are not sure whether you have the correctly formatted DER (ASN.1) encoded key, try this tool to parse your input file.
First of all President James K. Polk is right. A complete RSA Key is formed by a modulus and a public exponent.
In Java, the interface RSAPublicKey from the package java.security.interfaces can hold well-formed RSA Keys. Assuming you have such object, the process of converting to a PEM format would involve encoding it, as follows:
String pemFormatKey = Base64.getEncoder().encodeToString(rsaPublicKey.getEncoded());
use putty programs, PuTTYgen does this conversion
I'm trying to use a Google's OAuth 2.0 for Server to Server Applications and I ran into a roadblock.
It states:
"Sign the UTF-8 representation of the input using SHA256withRSA (also known as RSASSA-PKCS1-V1_5-SIGN with the SHA-256 hash function) with the private key obtained from the API console. The output will be a byte array."
So I got most of down using Java libraries but how do I use a String as a private key?
I guess you need a fixed size key. So you can get the String,
hash the String and the result is your key for RSA.
Maybe this also helps:
bytes[] values = myString.getBytes(); //get byte[] from String
i would like to know how to convert byte array into key.
i am doing an AES encryption/decryption. instead of generating a key, i would like to use my generated byte array.
byte[] clientCK = Milenage.f3(sharedSecret16, RANDbytes, opc);
let say i have a byte array called clientCK, stated above. i want to use it in AES encryption as shown below.
Cipher c = Cipher.getInstance("AES");
c.init(Cipher.ENCRYPT_MODE, key);
byte[] encValue = c.doFinal(valueToEnc.getBytes());
String encryptedValue = new BASE64Encoder().encode(encValue);
therefore, i need to convert that byte array clientCK into key format. please help.
You may want to use a SecretKeySpec:
public SecretKeySpec(byte[] key, String algorithm)
Constructs a secret key from the given byte array.
U will require policy jars names below in required places, which give higher key sizes for certain algorithms/and set higher algorithms for specific countries (as per US export crypto policy) Eg JDK directory on my PC.
C:\Program Files\Java\jdk1.5.0_17\jre\lib\security
U also require java.security file setup with proper crypto provider as given in shown in your java.security file as below (in directory above)
Example : security.provider.7=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.6=com.sun.crypto.provider.SunJCE
security.provider.5=cryptix.provider.Cryptix
The security provider numbering as seen above tells the order in which the underlying security providers will be used by respective JVM. In above example 3 security providers are set for use in underlying JVM They are BouncyCastle, SunJCE, and Cryptix.
local_policy.jar and US_export_policy.jar files to be put in your JVM lib\security directory
You will need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files listed above.If you don't, the keysize is limited due to US export laws.(The above error could also be because of really wrong keysize given in application eg AES keysize given as 20 bytes instead of 32 bytes in application)