I'm trying to use the pyjks module to grab keys from a keystore, however loading the keystore fails with the following error:
ValueError: Hash mismatch; incorrect password or data corrupted
If I try using keytool to load the keystore, I have no issues. I was wondering if anyone has ever used pyjks to do this and done so successfully. Here's my python code snippet:
ks = jks.KeyStore.load("/tmp/keystore.jceks", "changeit")
Disclaimer: I wrote the initial JCEKS support for pyjks.
This might be due to the lack of support for SecretKey entries at the time. The parsing routine tracks the current position in the file as it reads through it, and at the end expects the next N bytes to be the correct signature. Because SecretKeys were not yet implemented, they did not advance the current position, thus causing a bad hash check.
I'm responding because I recently added the missing SecretKey support to pyjks. So if your situation is still relevant, feel free to grab the latest source from https://github.com/doublereedkurt/pyjks and try it out.
Related
I have a java program that has a method:
static void verifyAndDecrypt(PGPPublicKey publicKeyIn, PGPSecretKeyRingCollection privateKey, String password, InputStream toBeDecrypted, OutputStream outputStream)
It uses bouncy castle to decrypt and verify a file. The output is written to outputStream.
In production I have run into the scenario of receiving files with signatures that the java code is able to decrypt, but not do signature verification on.
Lets call such a production file prodFile.txt.gpg.
That file is thus encrypted with our public key, and signed with the senders private key.
(I have yet to figure out why it fails, these files are able to be verified using GnuPG)
I have added the option of being lenient and just log a warning when signature fails.
My question is, how do I write unit tests for this?
I could (and have locally on my machine) write a test that takes prodFile.txt.gpg and call verifyAndDecrypt() on it, given that I pass in our secret key and password, and asserts that the file is decrypted, and when the signature verification fails logs a warning.
But I can't check in this code, since it would leave password and private keys in the repo.
So my question is,
How do I create a file testFile.txt.gpg that replicates the above scenario?
I could create key pairs for testing, but I don't know how to force the failing of the signature in the same way as with prodFile.txt.gpg.
I am also wondering if it's possible to "unpack" the content of prodFile.txt.gpg so that I have the cleartext file: prodFile.txt and the corresponding signature that I have problem with. And from there make a modification to prodfile.txt so that the signature doesn't match, then encrypt it with the public key of a keypair for testing purposes while using the "bad" signature. But I can't seem to find a way to do this. I suspect the gpg command won't let me do something like this.
So I have a very basic openssl command that was provided to me openssl smime -encrypt -binary -aes-256-cbc -in $inPath -out $encryptedPath -outform DER $pubCert, this command also works correctly and outputs an encrypted file. I need to use the equivalent of this command in a java application, preferably without invoking process and using openssl itself (only because I feel like that is probably bad practice).
I have researched quite a lot and there does not seem to be any equivalent out there that I can find.. I have tried several things and most of them do not seem to work. The weird thing is... I am able to get a simple "Hello World" string to encrypt using the code I wrote (although I don't believe it was encrypting it correctly because I had the cipher set to "RSA" not "AES") but when the byte array was coming from a file, it silently failed and just wrote 0 bytes. Right now this is what my code looks like.
Cipher aes = Cipher.getInstance("RSA");
CertificateFactory certF = CertificateFactory.getInstance("X.509");
File public_cert = new File( getClass().getClassLoader().getResource("public.crt").getFile());
FileInputStream certIS = new FileInputStream(public_cert);
X509Certificate cert = (X509Certificate) certF.generateCertificate(certIS);
certIS.close();
aes.init(Cipher.ENCRYPT_MODE, cert);
File tarGz = new File("C:\\volatile\\generic.tar.gz");
FileInputStream fis = new FileInputStream(tarGz);
byte[] tarGzBytes = FileUtils.readFileToByteArray(tarGz);
tarGzBytes = "Hello World".getBytes();
ByteArrayInputStream bais = new ByteArrayInputStream("Hello World".getBytes());
File encFile = new File("C:\\volatile\\generic.tar.gz.enc");
FileOutputStream enc = new FileOutputStream(encFile);
CipherOutputStream cos = new CipherOutputStream(enc, aes);
cos.write(tarGzBytes);
//IOUtils.copy(fis, cos);
//IOUtils.copy(bais, cos);
cos.flush();
cos.close();
So this works, and encrypts a little file with Hello World encrypted in it. I don't believe this is AES-256-CBC though, and it does not work when I use the FileUtils.readFileToByteArray(tarGz), although the resulting byte array in a debugger is correctly sized at about 94MB. Which seems really odd to me, that it works with "Hello World".toByteArray() and not FileUtils.readAllBytes(tarGz). Also as a side note, the ByteArrayInputStream using IOUtils.copy works, whereas the FileInputStream version writes 0 bytes as well.
Also, when I set the cipher mode to AES/CBC/PKCS5Padding (because I found something online suggesting to set it to that and it looks more like what I want) I get the following error message:
java.security.InvalidKeyException: No installed provider supports this key: sun.security.rsa.RSAPublicKeyImpl
at javax.crypto.Cipher.chooseProvider(Cipher.java:892)
at javax.crypto.Cipher.init(Cipher.java:1724)
~~~~
If anyone has any suggestions, or if I need to provide more information please let me know. I am fairly stuck right now and I am at this point debating writing a script to simply run the openssl command and run that script from java...
Conclusion
After reading through #dave-thompson-085's answer I realized that there was a really good reason why I could not find what I was wanting to do. So therefore I decided to go ahead and just call the openssl process from java using a process builder. I was able to recreate the openssl command from above as a Process in java, start it and run it with the following code:
File cert = new File(getClass().getClassLoader().getResource("public.crt").getFile());
ProcessBuilder openSslBuilder = new ProcessBuilder("openssl", "smime", "-encrypt", "-binary",
"-aes-256-cbc", "-in", "C:\\volatile\\generic.tar.gz", "-out",
"C:\\volatile\\generic.tar.gz.enc", "-outform", "DER", cert.getPath());
Process openssl = openSslBuilder.start();
openssl.waitFor();
System.out.println(openssl.exitValue());
openssl.destroy();
Hopefully this helps someone else who is looking to attempt this as well and maybe save someone a bunch of time!
First, to be clear: the openssl smime command actually handles both S/MIME and CMS (aka PKCS7) formats; these are related but different standards that basically use different file formats for essentially the same cryptographic operations. With -outform DER you are actually doing CMS/PKCS7.
Second and more fundamental: CMS/PKCS7, and S/MIME, and most other common cryptographic schemes like PGP, actually does hybrid encryption. Your data is not actually encrypted with RSA; instead your data is encrypted with a symmetric algorithm (here AES-256-CBC, since you selected that) using a randomly generated key called the DEK (data encryption key) and the DEK is encrypted with RSA using the recipient's publickey (obtained from their certificate), and both of those results plus a good deal of metadata is arranged into a fairly complicated data structure. The recipient can parse the message to extract these pieces, then use RSA with their privatekey to decrypt the DEK, then AES-decrypt the data with the DEK. Note you always use RSA keys for RSA, and AES keys for AES; symmetric keys are pretty much all just bits and only vary in size, but public-key cryptographic keys including RSA (also DH, DSA, ECC and more) are much more complicated and cannot be intermixed.
Trying to encrypt data directly with RSA as you did, in addition to being wrong, won't work in general because RSA can only encrypt limited amounts of data, depending on the key size used, typically about 100-200 bytes. Symmetric encryption also has some limits, but they are generally much larger; AES-CBC is good for about 250,000,000,000,000,000 bytes.
If you want to implement this yourself, you need to read the standard for CMS particularly the section on EnvelopedData using KeyTransRecipientInfo (for RSA), combined with the rules for ASN.1 BER/DER encoding. This is not a simple job, although it can be done if you want to put the effort in.
If you can use a third-party library in Java, the 'bcpkix' jar from https://www.bouncycastle.org has routines that support CMS, among several other things. This is usually easy if you are writing a program to run yourself, or in your department. If this is to be delivered to outside users or customers who may not like having to manage a dependency, maybe not.
That said, running another program to do something isn't necessarily bad practice in my book, and can be done directly from java (no script). Unless you (need to) do it very often, such as 100 times a second.
Whenever I install WebLogicServer, it doesn't create DemoIdentity.jks under base_domain. As such, I can't use SSL 7002.
I can't create a a new .jks file using instructions here : create new jks
It gives me error: java.security.invalidkeyexception : exponent is larget than modulus
Make sure the CA files are in DER format.
Which they are.
Is there a reason for this happening? Any solutions to it?
In order to bypass this issue, use the "-noskid" parameter when running utils.CertGen .
I am getting certificates from the keystore based on alias name using the below code,
KeyStore keyStore = KeyStore.getInstance("Windows-MY");
Enumeration aliasesEnum = keyStore.aliases();
while(aliasesEnum.hasMoreElements())
{
aAliasName = (String)aliasesEnum.nextElement();
X509Certificate certificate = (X509Certificate)keyStore.getCertificate(aAliasName);
}
Is there any way to get aliases from the current token instead of getting from keystore?.
Thanks in advance.
It is not possible to get a particular certificate other than knowing the alias.
The best practice would be to have unique alias in your use case, so you will know which certificate you are getting.
If you are storing the Certificate through the KeyStore api, you can do a containsAlias(alias) to see if the alias already exists before you save.
Normally alias names are unique. This is a problem specific to MSCAPI keystore. There are several bug reports (some of them very old) regarding this issue:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=2162058
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6483657
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8058544
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6672015
Follow the last link, it contains a workaround. You basically have to modify the alias names via reflection API into something unique (see code in link). Not exactly a clean solution, but unfortunately the only way.
I have a java application that signs a string using a certificate. It works encrypting the string it with SHA1. I am trying to translate the code to Delphi 2010, but I have no idea how to get it working the same way the java app does (using sha1). So far, I have found this:
Delphi 7 access Windows X509 Certificate Store
It does work, but it does not use sha1 and I get different results when I run the java app.
Java code
char[] pass = (char[]) null;
PrivateKey key = (PrivateKey) getKeyStore().getKey(alias, pass);
Certificate[] chain = getKeyStore().getCertificateChain(alias);
CertStore certsAndCRLs = CertStore.getInstance("Collection", new CollectionCertStoreParameters(Arrays.asList(chain)), "BC");
X509Certificate cert = (X509Certificate) chain[0];
CMSSignedDataGenerator gen = new CMSSignedDataGenerator();
gen.addSigner(key, cert, CMSSignedDataGenerator.DIGEST_SHA1);
gen.addCertificatesAndCRLs(certsAndCRLs);
CMSProcessable data = new CMSProcessableByteArray(conteudoParaAssinar);
CMSSignedData signed = gen.generate(data, true, "SunMSCAPI");
byte[] envHex = signed.getEncoded();
CertInfo certInfo = new CertInfo();
certInfo.Hash = new BigInteger(envHex).toString(16);
return certInfo;
Delphi Code
var
lSigner: TSigner;
lSignedData: TSignedData;
fs: TFileStream;
qt: integer;
ch: PChar;
msg : WideString;
content : string;
cert: TCertificate;
begin
cert := Self.GetCert;
content := 'test';
lSigner := TSigner.Create(self);
lSigner.Certificate := cert.DefaultInterface;
lSignedData := TSignedData.Create(self);
lSignedData.content := content;
msg := lSignedData.Sign(lSigner.DefaultInterface, false, CAPICOM_ENCODE_BASE64);
lSignedData.Free;
lSigner.Free;
EDIT
Based on the java code, should I get the cert info in binary format, apply sha1 on it and them convert it to hex? Is this the right order and the same thing the java code does? I can see some SHA1 constants in the capicom tlb as well as a hash class, maybe I should use those classes, but I dont know how.
We use DCPCrypt in some delphi apps that interface with our Java Tomcat App and are able to get SHA-256 compatible hashes. I suspect SHA1 is also easy.
Here's an example
function Sha256FileStreamHash(fs : TFileStream): String;
var
Hash: TDCP_sha256;
Digest: array[0..31] of byte; // RipeMD-160 produces a 160bit digest (20bytes)
i: integer;
s: string;
begin
if fs <> nil then
begin
fs.Seek(0, soFromBeginning);
Hash:= TDCP_sha256.Create(nil); // create the hash
try
Hash.Init; // initialize it
Hash.UpdateStream(fs,fs.Size); // hash the stream contents
Hash.Final(Digest); // produce the digest
s:= '';
for i:= 0 to 31 do
s:= s + IntToHex(Digest[i],2);
Result:= s; // display the digest
finally
Hash.Free;
end;
end;
end;
First, what makes you think you're not using SHA-1 ? I'm asking because CAPICOM's sign function only works with SHA-1 signature.
Second, how do you know that you're getting a different result ? Have you tried to validate the answer ? If yes, using what ?
Third, there is something that you MUST know about CAPICOM: the "content" property is a widestring. This has various implication, including the fact that all content will be padded to 16-bits. If your input data is of different size, you'll get a different result.
Based on the java code, should I get the cert info in binary format, apply sha1 on it and them convert it to hex?
No. You get an interface to an instance of a ICertificate object (or, more likely, ICertificate2) and you just use that directly. If you have the B64 encoded version of the certificate, you can create a new ICertificate instance and then call the ICertificate.Import method. The hash of the certificate itself is only used by the signing authority to sign that specific cert.
The hash algorythm is actually used during the data signature process: the library reads the data, creates a hash of that data (using SHA-1 in case of CAPICOM) and then digitally sign that hash value. This reduction is necessary because signing the whole data block would be far too slow and because, that way, you only have to carry the hash if you're using a hardware crypto system.
Is this the right order and the same thing the java code does?
Yes and no. The Java code does all the necessary steps in explicit details, something you don't have (and actually cannot) do with CAPICOM. It should result in compatible result, though.
It also has an additional step not related to the signature itself: I'm not sure what it does because it seems to create a dummy certificate information data and store the SHA-1 hash value of the signed CMS message and return the resulting instance. I suppose that it's a way the Java dev has found to pass the hash value back to the caller.
I can see some SHA1 constants in the capicom tlb as well as a hash class, maybe I should use those classes, but I dont know how.
The HashedData class is used to (surprise) hash data. It has the same limitation as Signeddata i.e. it only works on widestrings so compatibility with other frameworks is dodgy at best.
Final note: Windows offers access to much more comprehensive cryptographic functions through the CAPI group of functions. CAPICOM is only an interface to that library that is used (mostly) in script language (JavaScript on web pages, VB, etc). You should do yourself a favor and try using it instead of CAPICOM because there is a good chance you'll encounter something that you simply cannot do properly using CAPICOM. At that stage, you will have to rewrite part for all of your application using CAPI (or another library). So save time now and skip CAPICOM if you don't have a requirement to use it.