This question already has answers here:
Get java.security.PrivateKey from private key file generated by hyperledger ca
(1 answer)
How to read .pem file to get private and public key
(10 answers)
Closed 2 months ago.
I am trying to get a bearer token using oAuth2 for connecting to a service account on GCP, I am following this documentation. To sign the JWT, the docs tell me to use "the private key obtained from the Google API Console".
Sign the UTF-8 representation of the input using SHA256withRSA (also known as RSASSA-PKCS1-V1_5-SIGN with the SHA-256 hash function) with the private key obtained from the Google API Console.
I have the below code in Java to read that private key from a file
File privKeyFile = new File(keyPath);
BufferedInputStream bis = new BufferedInputStream(new FileInputStream('myprivatekey.pem'));
byte[] privKeyBytes = new byte[8192]
bis.read(privKeyBytes);
bis.close();
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
KeySpec ks = new PKCS8EncodedKeySpec(privKeyBytes);
RSAPrivateKey privateKey = (RSAPrivateKey) keyFactory.generatePrivate(ks);
But this code is currently giving this exception
java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException:
invalid key format at java_security_KeyFactory$generatePrivate$0.call(Unknown Source)
I tried converting private Key to PKCS#8 format as per this post but when I run this command
openssl pkcs8 -topk8 -inform PEM -outform DER -in myprivatekey.pem -nocrypt > pkcs8_key
I get this error
unable to load key
140735932699592:error:0906D06C:PEM routines:PEM_read_bio:no start line:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.50.3/libressl/crypto/pem/pem_lib.c:704:Expecting: ANY PRIVATE KEY
Was that not a valid RSA key which Google console provided? How can I read that key into my code so I can use it to sign the JWT?
The service account private key is PEM encoded RSA PKCS #8. The PKCS8EncodedKeySpec wants ASN.1 encoding without newlines, header, and tail.
The private key in your post appears valid and in the correct format.
WARNING: if that private key is from Google Cloud, immediately delete the key.
Use code similar to this:
public RSAPrivateKey readPrivateKey(File file) throws Exception {
String key = new String(Files.readAllBytes(file.toPath()), Charset.defaultCharset());
String privateKeyPEM = key
.replace("-----BEGIN PRIVATE KEY-----", "")
.replaceAll(System.lineSeparator(), "")
.replace("-----END PRIVATE KEY-----", "");
byte[] encoded = Base64.decodeBase64(privateKeyPEM);
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(encoded);
return (RSAPrivateKey) keyFactory.generatePrivate(keySpec);
}
I'm looking at reading one or more private keys from PEM-encoded DER files and I'm wondering if it can be done robustly without just trying all possible algorithms (which is ugly and brute-forcey) or writing an ASN.1 parser?
I'm trying to do this without any dependencies other than the standard Java library. I'm sure BouncyCastle can do this, but I'm interested in what is possible with out-of-the-box Java.
A PEM-encoded DER file is the somewhat familiar file that looks like this:
-----BEGIN EC PRIVATE KEY-----
[base64-encoded DER data]
-----END EC PRIVATE KEY-----
In the example above, the text indicates the type of key: it's an elliptic-curve key, so we can ask Java for an EC key factory:
byte[] keyBytes = // get DER bytes
KeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
KeyFactory kf = KeyFactory.getInstance("EC");
PrivateKey key = kf.generateKey(spec);
But it's also possible that the file does not advertise its contents, like this:
-----BEGIN PRIVATE KEY-----
[base64 encoded DER data]
-----END PRIVATE KEY-----
In this case, I see only two obvious possibilities given the API:
Loop through all supported algorithms, trying them all and ignoring certain errors
Implement an ASN.1 parser to determine the actual type of the key, then use the right KeyFactory
Looping seems straightforward, but IMO distasteful:
byte[] keyBytes = // get DER bytes
KeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
PrivateKey key = null;
for(String algorithm : new String[] { "EC", "RSA", "DSA" }) {
try {
KeyFactory kf = KeyFactory.getInstance(algorithm);
key = kf.generatePrivateKey(keyBytes);
} catch (....) {
// Ignore "wrong algorithm" exceptions
}
}
// "key" is either null (couldn't read/interpret) or non-null and OK
This is also potentially time-consuming, though typically key setup is a "rare" operation, so performance isn't strictly important.
The only other option I can think of is inspecting the data to find out exactly what the data represent, and then use the correct type:
byte[] keyBytes = // get DER bytes
KeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
ASN1Parser a1p = new ASN1Parser();
String algorithm = a1p.parse(keyBytes).getAlgorithm();
KeyFactory kf = KeyFactory.getInstance(algorithm);
PrivateKey key = fk.generatePrivateKey(spec);
Contrast this with reading an X.509 Certificate, where you don't have to know anything about the algorithm to load the certificate:
CertificateFactory cf = CertificateFactory.getInstance("X.509");
cf.generateCertificate(new ByteArrayInputStream(certBytes));
Am I missing something in the API that would allow this kind of "just do it for me" mechanics, or is it strictly necessary to know the algorithm before loading the key material?
Writing an ASN.1 parser isn't trivial and though they exist in third-party libraries, I'd like to limit the dependencies as much as possible.
Good day,
There is another third party that need my web application to send them some data in encrypt format. Thus they send me some guide to do so, however, I am not familiar with it, I am trying to google around but looks like I am google wrong way.
The guide is something as follow:
Run openssl command to generate a privatekey:
openssl ecparam -name prime256v1 -genkey -out myprivate.pem
After run this command, I output a priv.pem file, and I saw inside got some key end with '==', which is as follow:
-----BEGIN EC PARAMETERS-----
BggqhkjOPQMBBw==
-----END EC PARAMETERS-----
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEILefWfeuZOgnbDlxpwo3uQ2xQXfhXHUPTS+vKzvVZdCToAoGCCqGSM49
AwEHoUQDQgAE4MeQspGRJ1qdpweBfiaT5P84alZdga1f7mSpa5HqXTH58u0ZWJUQ
J7ToU/bUOPITh4FX07AV6wrgFCmwtUenDQ==
-----END EC PRIVATE KEY-----
Second one is run openssl command to generate the public key, and then send them:
openssl ec -in myprivate.pem -pubout -out mypublic.pem
Convert the private key to pkcs8 format:
openssl pkcs8 -topk8 -nocrypt -in myprivate.pem -out mypkcs8.pem
The third party will give me a public key in string format, then ask me to generate a secret key, and provide me some java code as follow:
first is to generate secret key and second one is encrypt:
public static SecretKey generateSharedSecret(PrivateKey privateKey,
PublicKey publicKey) {
try {
KeyAgreement keyAgreement = KeyAgreement.getInstance( "ECDH" );
keyAgreement.init( privateKey );
keyAgreement.doPhase( publicKey, true );
SecretKeySpec key = new SecretKeySpec(
keyAgreement.generateSecret( ), "AES" );
return key;
} catch ( Exception e ) {
// TODO Auto-generated catch block
e.printStackTrace( );
return null;
}
}
public static String encryptString(SecretKey key, String plainText) {
try {
String myIv = "Testing # IV!";
byte[] iv = myIv.getBytes( "UTF-8" );
IvParameterSpec ivSpec = new IvParameterSpec( iv );
Cipher cipher = Cipher.getInstance( "AES / CBC / PKCS5Padding" );
byte[] plainTextBytes = plainText.getBytes( "UTF-8" );
byte[] cipherText;
cipher.init( Cipher.ENCRYPT_MODE, key, ivSpec );
cipherText = new byte[cipher.getOutputSize( plainTextBytes.length )];
int encryptLength = cipher.update( plainTextBytes, 0,
plainTextBytes.length, cipherText, 0 );
encryptLength += cipher.doFinal( cipherText, encryptLength );
return bytesToHex( cipherText );
} catch ( Exception e ) {
e.printStackTrace( );
return null;
}
}
and also the bytes to hex string method:
public static String bytesToHex(byte[] byteArray) {
StringBuffer hexStringBuffer = new StringBuffer( );
for ( int i = 0; i < byteArray.length; i++ ) {
hexStringBuffer.append( String.format( "%02X", byteArray[ i ] ) );
}
return hexStringBuffer.toString( );
}
I have self gen a private key and also a public key by using openssl command, but the 4th step telling me that they will give me a public key as well, thus I am not understand, which public key should I use.
And also, how can I convert a String into java PrivateKey and PublicKey object?
* add on *
I try to convert the der file to java PublicKey object, it looks work. Before this, I convert the pem to der using openssl command:
openssl pkey -pubin -in ecpubkey.pem -outform der -out ecpubkey.der
Here is the java code:
File f = new File("/home/my/Desktop/key/ecpubkey.der");
FileInputStream fis = new FileInputStream(f);
DataInputStream dis = new DataInputStream(fis);
byte[] keyBytes = new byte[(int) f.length()];
dis.readFully(keyBytes);
dis.close();
KeyFactory fact = KeyFactory.getInstance("EC");
PublicKey theirpub = fact.generatePublic(new X509EncodedKeySpec(keyBytes));
However, I am hitting java.security.spec.InvalidKeySpecException: java.io.IOException: insufficient data when I try to convert der file to java PrivateKey object, the following is what I did:
openssl ecparam -name prime256v1 -genkey -out priv.pem
openssl pkcs8 -topk8 -nocrypt -in priv.pem -outform der -out priv.der
And the following is my java code:
File f2 = new File("/home/my/Desktop/key/priv.der");
FileInputStream fis2 = new FileInputStream(f2);
DataInputStream dis2 = new DataInputStream(fis2);
byte[] keyBytes2 = new byte[(int) f.length()];
dis2.readFully(keyBytes2);
dis2.close();
KeyFactory fact2 = KeyFactory.getInstance("EC");
PrivateKey pKey = fact2.generatePrivate( new PKCS8EncodedKeySpec(keyBytes2) ); // this line hit insufficient data
Diffie-Hellman is well-explained in wikipedia -- and probably some of the hundreds of Qs here, and crypto.SX and security.SX, about it, but I can't easily find which. In brief:
you generate a keypair, keep your privatekey, and provide your publickey to the other party
the other party does the same thing (or its reflection): generate a keypair, keep their privatekey, and provide their publickey to you
you use your privatekey and their publickey to compute the 'agreement' value
they similarly use their privatekey and your publickey to compute the same 'agreement' value. This is also called a shared secret, because you and the other party know it, but anyone eavesdropping on your traffic does not.
The 'provide' in that synopsis omits a lot of very important details. It is vital that when you provide your publickey to the other party they actually get your publickey and not a value altered or replaced by an adversary, and similarly when they provide their publickey to you it is vital you get the real one and not a modified or fake one. This is where actual DH systems mostly break down, and the fact you mention none of the protections or complications needed here suggests your scheme will be insecure and easily broken -- if used for anything worth stealing.
Note you should NEVER disclose or 'send' your privatekey to anyone, and they should similarly not disclose theirs. That's the main basis for public-key (or 'asymmetric') cryptography to be of any value or use at all.
There are numerous ways that keys can be represented, but only some are relevant to you.
Public keys are often represented either in
the ASN.1 structure SubjectPublicKeyInfo defined in X.509 and more conveniently in PKIX, primarily in rfc5280 #4.1 and #4.1.2.7 and rfc3279 2.3, encoded in DER, which has the limitation that many of the bytes used in this encoding are not valid characters and cannot be correctly displayed or otherwise manipulated and sometimes not transmitted or even stored; or
that same ASN.1 DER structure 'wrapped' in 'PEM' format, which converts the troublesome binary data to all displayable characters in an easily manipulable form. PEM format was originally created for a secure-email scheme call Privacy Enhanced Mail which has fallen by the wayside, replaced by other schemes and technologies, but the format it defined is still used. The publickey PEM format was recently re-standardized by rfc7468 #13 (which as you see referenced rfc5280).
OpenSSL supports both of these, but the commandline utility which you are using mostly defaults to PEM -- and since you need to convey your key to 'them', and they need to convey their key to you, PEM may well be the most reliable and/or convenient way of doing so. (Although other formats are possible, if you and they agree -- and if they require something else you'll have to agree for this scheme to work at all.)
Java directly supports only DER, thus assuming you receive their publickey in SPKI PEM, to use it in Java you need to convert it to DER. You can either do this in OpenSSL
openssl pkey -pubin -in theirpub.pem -outform der -out theirpub.der
and then read the DER into a Java crypto KeyFactory:
byte[] theirpubder = Files.readAllBytes(Paths.get(whatever));
KeyFactory fact = KeyFactory.getInstance("EC");
PublicKey theirpub = fact.generatePublic(new X509EncodedKeySpec(theirpubder));
// can downcast to ECPublicKey if you want to be more specific
Alternatively you can have Java convert the PEM which isn't too hard; there are several variations but I like:
String theirpubpem = new String(Files.readAllBytes(Paths.get(whatever)));
// IN GENERAL letting new String(byte[]) default the charset is dangerous, but PEM is OK
byte[] theirpubder = Base64.getMIMEDecoder().decode(theirpubpem.replaceAll("-----[^\\n]*\\n","") );
// continue as for DER
For private keys
there are significantly more representations, but only one (or two-ish) that Java shares with OpenSSL. Since you only need to store the private key locally and not 'send' it, PEM may not be needed; if so you can just add -outform der to your pkcs8 -topk8 -nocrypt command, adjusting the name appropriately, and read the result directly in a Java KeyFactory in the same fashion as above except with PKCS8EncodedKeySpec and generatePrivate and [EC]PrivateKey. If you do want to store it in (PKCS8-clear) PEM, you can also combine the above.
Using the DH agreement value directly as a symmetric cipher (e.g. AES) key is nonstandard and generally not considered good practice, although for ECDH with prime256v1 (aka secp256r1 or P-256) it is technically possible. AFAIK all good standards use a key-derivation step (aka Key Derivation Function or KDF) in between. Since you haven't shown us their 'guide' I can't say if this is correct -- for at least small values of correct.
To be sure you know, using CBC with a fixed IV more than once for the same key (which in this case is the same DH result) is insecure. I assume 'Testing' means you plan to replace it with something better.
Also FYI you don't need to use the full complication of the Cipher.init,update,doFinal API. When the data is small enough to fit in memory, as here, you can just do:
cipher.init(ENCRYPT_MODE, key, parms);
byte[] encrypted = cipher.doFinal (plainbytes);
// or since you want to hexify it
... bytesToHex (cipher.doFinal (plainbytes)) ...
Finally because Java byte is signed, your bytesToHex will output almost exactly half of all bytes with FFFFFF prefixed. This is very unusual, and phenomenally ugly, but again I don't know if it is 'correct' for you.
Base on dave_thompson_085 explanation and code, I manage to create my java PublicKey and Privatekey with following:
public static PublicKey getPublicKey(String filename) throws IOException, GeneralSecurityException {
String publicKeyPEM = getKey(filename);
return getPublicKeyFromString(publicKeyPEM);
}
private static String getKey(String filename) throws IOException {
// Read key from file
String strKeyPEM = "";
BufferedReader br = new BufferedReader(new FileReader(filename));
String line;
while ((line = br.readLine()) != null) {
strKeyPEM += line + "\n";
}
br.close();
return strKeyPEM;
}
public static PublicKey getPublicKeyFromString(String key) throws IOException, GeneralSecurityException {
String publicKeyPEM = key;
publicKeyPEM = publicKeyPEM.replace("-----BEGIN PUBLIC KEY-----\n", "");
publicKeyPEM = publicKeyPEM.replace("-----END PUBLIC KEY-----", "");
BASE64Decoder b = new BASE64Decoder();
byte[] encoded = b.decodeBuffer(publicKeyPEM);
KeyFactory kf = KeyFactory.getInstance("EC");
PublicKey pubKey = (PublicKey) kf.generatePublic(new X509EncodedKeySpec(encoded));
return pubKey;
}
and this is for private key
public static PrivateKey getPrivateKey(String filename) throws IOException, GeneralSecurityException {
String privateKeyPEM = getKey(filename);
return getPrivateKeyFromString(privateKeyPEM);
}
public static PrivateKey getPrivateKeyFromString(String key) throws IOException, GeneralSecurityException {
String privateKeyPEM = key;
privateKeyPEM = privateKeyPEM.replace("-----BEGIN PRIVATE KEY-----\n", "");
privateKeyPEM = privateKeyPEM.replace("-----END PRIVATE KEY-----", "");
BASE64Decoder b = new BASE64Decoder();
byte[] encoded = b.decodeBuffer(privateKeyPEM);
KeyFactory kf = KeyFactory.getInstance("EC");
PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(encoded);
PrivateKey privKey = (PrivateKey) kf.generatePrivate(keySpec);
return privKey;
}
Many thanks to #dave_thompson_085 explanation.
Trying to read a PKCS8 private key in PEM format with the following:
private static PrivateKey loadPrivateKey()
throws IOException, GeneralSecurityException, OperatorCreationException, PKCSException {
FileReader fileReader = new FileReader(certsRoot + "/pep-client-key.pem");
PEMParser keyReader = new PEMParser(fileReader);
JcaPEMKeyConverter converter = new JcaPEMKeyConverter();
InputDecryptorProvider decryptionProv = new JceOpenSSLPKCS8DecryptorProviderBuilder().build("mypassword".toCharArray());
Object keyPair = keyReader.readObject();
PrivateKeyInfo keyInfo;
if (keyPair instanceof PKCS8EncryptedPrivateKeyInfo) {
keyInfo = ((PKCS8EncryptedPrivateKeyInfo) keyPair).decryptPrivateKeyInfo(decryptionProv); // Exception thrown from here
keyReader.close();
return converter.getPrivateKey(keyInfo);
}
return null;
}
generates this error:
org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: Cannot find any provider supporting 1.2.840.113549.3.7
at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source)
I've checked with OpenSSL that the file can be processed as PKCS8 PEM, with the password provided.
Any idea? I don't mind if there is a solution not involving BouncyCastle's libraries.
1.2.840.113549.3.7 is the OID for DES-EDE3-CBC-Pad (in PBES2) in PKCS5 = rfc2898 sec B.2.2. (1.2.840.113549.1.5.13 is the 'outer' OID for all PBES2 variants.)
The Sun-now-Oracle (default) providers do support the DES-EDE3 algorithm (aka TripleDES or TDEA keying option 1) with CBC and PKCS5/7 padding but do not have this OID mapping for it. The BouncyCastle provider does have the mapping, so if you use the BC provider for this operation it should work. This can be done
for all JVMs by configuring security.provider.<i> in JRE/lib/security/java.security (update: in j9+ JRE/conf/security/java.security) or
for a JVM by java.lang.security.Provider.addProvider (new BouncyCastleProvider()) or
for this operation by adding .setProvider() with the name of or object for the BC provider to your JceOpenSSLPKCS8DecryptorProviderBuilder invocation
Note BC for TripleDES seems to require the 'unlimited strength policy' on Oracle Java below j8u151; see Cannot open PKCS12 store because of password and InvalidKeyException Illegal key size and many other dupes.
The BouncyCastle cryptography APIs allow for creating and verifying digital signatures using the regular java.security package objects, such as java.security.PublicKey, java.security.PrivateKey and their container java.security.KeyPair.
Suppose I use OpenSSL to create a .pem (or, if easier, a .der file) containing the elliptic curve private key I want to use in my application. For example, it looks like this:
-----BEGIN EC PARAMETERS-----
BgUrgQQACg==
-----END EC PARAMETERS-----
-----BEGIN EC PRIVATE KEY-----
MHQCAQEEIDzESrZFmTaOozu2NyiS8LMZGqkHfpSOoI/qA9Lw+d4NoAcGBSuBBAAK
oUQDQgAE7kIqoSQzC/UUXdFdQ9Xvu1Lri7pFfd7xDbQWhSqHaDtj+XY36Z1Cznun
GDxlA0AavdVDuoGXxNQPIed3FxPE3Q==
-----END EC PRIVATE KEY-----
How do I use the BouncyCastle APIs to obtain a java.security.KeyPair containing both this private key and a corresponding public key?
Please note I want to use the APIs available in BouncyCastle 1.50 (which is current at the time of writing) and no deprecated APIs. This unfortunately excludes the PEMReader class used in other SO answers. Furthermore, this question is specific to the format of elliptic curves; they contain additional parameters when compared RSA or DSA key files.
In addition to the standard JCE approach shown by divanov as long as you give it the correct input (see my comment thereto), or just using JCE in the first place like your selfanswer, BouncyCastle 1.48 up DOES still contain the old PEMReader functionality just organized a bit differently and for this case you can use something like:
static void SO22963581BCPEMPrivateEC () throws Exception {
Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
Reader rdr = new StringReader ("-----BEGIN EC PRIVATE KEY-----\n"
+"MHQCAQEEIDzESrZFmTaOozu2NyiS8LMZGqkHfpSOoI/qA9Lw+d4NoAcGBSuBBAAK\n"
+"oUQDQgAE7kIqoSQzC/UUXdFdQ9Xvu1Lri7pFfd7xDbQWhSqHaDtj+XY36Z1Cznun\n"
+"GDxlA0AavdVDuoGXxNQPIed3FxPE3Q==\n"+"-----END EC PRIVATE KEY-----\n");
Object parsed = new org.bouncycastle.openssl.PEMParser(rdr).readObject();
KeyPair pair = new org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter().getKeyPair((org.bouncycastle.openssl.PEMKeyPair)parsed);
System.out.println (pair.getPrivate().getAlgorithm());
}
In Java this will be pretty much the same code. After striping guarding strings away and decoding Base64 data give it to this utility method:
public static PrivateKey keyToValue(byte[] pkcs8key)
throws GeneralSecurityException {
PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(pkcs8key);
KeyFactory factory = KeyFactory.getInstance("ECDSA");
PrivateKey privateKey = factory.generatePrivate(spec);
return privateKey;
}
Since I only need this for a quick and dirty demo, I solved it in the following way (in Scala). First, I generate a public private key pair in the REPL and print out its data:
Security.addProvider(new BouncyCastleProvider)
val SignatureScheme = "some signature scheme, eg ECDSA"
val RandomAlgorithm = "some random algorithm, eg SHA1PRNG"
val keygen = KeyPairGenerator.getInstance(SignatureScheme)
val rng = SecureRandom.getInstance(RandomAlgorithm)
rng.setSeed(seed)
keygen.initialize(KeySize, rng)
val kp = keygen.generateKeyPair()
println(kp.getPublic.getEncoded.toSeq) // toSeq so that Scala actually prints it
println(kp.getPrivate.getEncoded.toSeq)
Then using the generated data,
val hardcodedPublic = Array[Byte]( /* data */ )
val hardcodedPrivate = Array[Byte]( /* data */ )
val factory = KeyFactory.getInstance(SignatureScheme)
val publicSpec = new X509EncodedKeySpec(hardcodedPublic)
val publicKey = factory.generatePublic(publicSpec)
val privateSpec = new PKCS8EncodedKeySpec(hardcodedPrivate)
val privateKey = factory.generatePrivate(privateSpec)
The key thing you need to know here is that by default public key data uses X509 encoding and private key data uses PKCS8 encoding. It should be possible to get OpenSSL to output these formats and parse them manually, but I did not check how.
I used information from this blog post about SpongyCastle (which is Android's BouncyCastle alias) quite helpful. It is unfortunate that documentation is fragmented like this, and that BouncyCastle's wiki was down at the time of this question.
Update: the BouncyCastle wiki is up, and you can find the documentation here.