We Keep Coding

Two-steps signing of PDF with itextpdf for Java using existing certificate, intermediary file and remote created signature

The use-case is as following: a local application must sign a PDF document using a full PKCS#7 detached signature generated on a remote server which requires a one-time-password sent by e-mail/SMS; the remote server generates the detached signature based on the associated document hash and the resulting signature is an “external
signature”, as defined in the CMS RFC (RFC5652), section 5.2, such as the resulting signature file is a separate file.
I'm struggling for days now to implement a working solution for this use-case (using itextpdf 5.5.13.1), but the final signed file has the signature in error with the message "Certification by is invalid" -- see the "signed_with_error" file attached.
The conceptual "two-steps signing" implemented process is:
1st step: from the original document file -- see the "original" attached file -- I create an intermediary file -- see the "intermediary" file attached -- in which I've inserted an empty signature based on which the associated document hash is created. In this step, the signature digest is saved to be used in the next step
2nd step: remote server is called, the detached signature file is received and I'm creating the signed file by inserting content in the signature from the pre-signed file using the signature digest from previous step and detached signature content received from the remote server.
Original, intermediary and signed with error sample files are:
original
intermediary
signed_with_error
Anyone has any idea of what could be wrong with my code beloow?
Relevant code parts are as following:
1st step:
ByteArrayOutputStream preSignedDocument = new ByteArrayOutputStream();
Path customerPathInDataStorage = storageService.resolveCustomerPathInDataStorage(customerExternalId);
PdfReader pdfReader = new PdfReader(originalDocumentContent);
PdfStamper stamper = PdfStamper.createSignature(pdfReader, preSignedDocument, '\0', customerPathInDataStorage.toFile(), true);
// create certificate chain using certificate received from remote server system
byte[] certificateContent = certificateInfo.getData(); // this is the customer certificate received one time from the remote server and used for every document signing initialization
X509Certificate certificate = SigningUtils.buildCertificateFromCertificateContent(certificateContent);
java.security.cert.Certificate[] certificatesChain = CertificateFactory.getInstance(CERTIFICATE_TYPE).generateCertPath(
Collections.singletonList(certificate)).getCertificates().toArray(new java.security.cert.Certificate[0]);
// create empty digital signature inside pre-signed document
PdfSignatureAppearance signatureAppearance = stamper.getSignatureAppearance();
signatureAppearance.setVisibleSignature(new Rectangle(72, 750, 400, 770), 1, "Signature_" + customerExternalId);
signatureAppearance.setCertificationLevel(PdfSignatureAppearance.CERTIFIED_NO_CHANGES_ALLOWED);
signatureAppearance.setCertificate(certificate);
CustomPreSignExternalSignature externalSignatureContainer =
new CustomPreSignExternalSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
MakeSignature.signExternalContainer(signatureAppearance, externalSignatureContainer, 8192);
ExternalDigest digest = new SignExternalDigest();
PdfPKCS7 pdfPKCS7 = new PdfPKCS7(null, certificatesChain, DOCUMENT_HASHING_ALGORITHM, null, digest, false);
byte[] signatureDigest = externalSignatureContainer.getSignatureDigest();
byte[] authAttributes = pdfPKCS7.getAuthenticatedAttributeBytes(signatureDigest, null, null,
MakeSignature.CryptoStandard.CMS);
pdfReader.close();
documentDetails.setPreSignedContent(preSignedDocument.toByteArray()); // this is the intermediary document content used in 2nd step in the line with the comment ***PRESIGNED_CONTENT****
documentDetails.setSignatureDigest(signatureDigest); // this is the signature digest used in 2nd step in the line with comment ****SIGNATURE_DIGEST****
byte[] hashForSigning = DigestAlgorithms.digest(new ByteArrayInputStream(authAttributes),
digest.getMessageDigest(DOCUMENT_HASHING_ALGORITHM));
documentDetails.setSigningHash(hashForSigning); // this is the hash sent to remote server for signing
2nd step:
// create certificate chain from detached signature
byte[] detachedSignatureContent = documentDetachedSignature.getSignature(); // this is the detached signature file content received from the remote server which contains also customer the certificate
X509Certificate certificate = SigningUtils.extractCertificateFromDetachedSignatureContent(detachedSignatureContent);
java.security.cert.Certificate[] certificatesChain = CertificateFactory.getInstance(CERTIFICATE_TYPE).generateCertPath(
Collections.singletonList(certificate)).getCertificates().toArray(new java.security.cert.Certificate[0]);
// create digital signature from detached signature
ExternalDigest digest = new SignExternalDigest();
PdfPKCS7 pdfPKCS7 = new PdfPKCS7(null, certificatesChain, DOCUMENT_HASHING_ALGORITHM, null, digest, false);
pdfPKCS7.setExternalDigest(detachedSignatureContent, null, SIGNATURE_ENCRYPTION_ALGORITHM);
byte[] signatureDigest = documentVersion.getSignatureDigest(); // this is the value from 1st step for ****SIGNATURE_DIGEST****
byte[] encodedSignature = pdfPKCS7.getEncodedPKCS7(signatureDigest, null, null, null, MakeSignature.CryptoStandard.CMS);
ExternalSignatureContainer externalSignatureContainer = new CustomExternalSignature(encodedSignature);
// add signature content to existing signature container of the intermediary PDF document
PdfReader pdfReader = new PdfReader(preSignedDocumentContent);// this is the value from 1st step for ***PRESIGNED_CONTENT****
ByteArrayOutputStream signedPdfOutput = new ByteArrayOutputStream();
MakeSignature.signDeferred(pdfReader, "Signature_" + customerExternalId, signedPdfOutput, externalSignatureContainer);
return signedPdfOutput.toByteArray();
dependencies:
public static final String DOCUMENT_HASHING_ALGORITHM = "SHA256";
public static final String CERTIFICATE_TYPE = "X.509";
public static final String SIGNATURE_ENCRYPTION_ALGORITHM = "RSA";
public class CustomPreSignExternalSignature implements ExternalSignatureContainer {
private static final Logger logger = LoggerFactory.getLogger(CustomPreSignExternalSignature.class);
private PdfDictionary dictionary;
private byte[] signatureDigest;
public CustomPreSignExternalSignature(PdfName filter, PdfName subFilter) {
dictionary = new PdfDictionary();
dictionary.put(PdfName.FILTER, filter);
dictionary.put(PdfName.SUBFILTER, subFilter);
}
#Override
public byte[] sign(InputStream data) throws GeneralSecurityException {
try {
ExternalDigest digest = new SignExternalDigest();
signatureDigest = DigestAlgorithms.digest(data, digest.getMessageDigest(DOCUMENT_HASHING_ALGORITHM));
} catch (IOException e) {
logger.error("CustomSignExternalSignature - can not create hash to be signed", e);
throw new GeneralSecurityException("CustomPreSignExternalSignature - can not create hash to be signed", e);
}
return new byte[0];
}
#Override
public void modifySigningDictionary(PdfDictionary pdfDictionary) {
pdfDictionary.putAll(dictionary);
}
public byte[] getSignatureDigest() {
return signatureDigest;
}
}
public class CustomExternalSignature implements ExternalSignatureContainer {
private byte[] signatureContent;
public CustomExternalSignature(byte[] signatureContent) {
this.signatureContent = signatureContent;
}
#Override
public byte[] sign(InputStream data) throws GeneralSecurityException {
return signatureContent;
}
#Override
public void modifySigningDictionary(PdfDictionary pdfDictionary) {
}
}
public class SignExternalDigest implements ExternalDigest {
#Override
public MessageDigest getMessageDigest(String hashAlgorithm) throws GeneralSecurityException {
return DigestAlgorithms.getMessageDigest(hashAlgorithm.toUpperCase(), null);
}
}
Later comment:
Even if it is not correct, I've observed that if, in 2nd step, instead of line:
byte[] signatureDigest = documentVersion.getSignatureDigest(); // this is the value from 1st step for ****SIGNATURE_DIGEST****
I'll use:
byte[] signatureDigest = new byte[0];
than the signed file will be generated with a visible certificate which can be validate but the PDF document is invalid with error "Document certification is INVALID". --- "The document has been altered or corrupted since Certification was applied." - see attached file signed_certification_invalid. It looks legit to be invalid, but for me is strange why in this case certification is displayed in document but it is broken when using - what I consider - "the right" signatureDigest value.

You say
the remote server generates the detached signature based on the associated document hash and the resulting signature is an “external signature”, as defined in the CMS RFC (RFC5652), section 5.2
thus, what you retrieve here in step 2:
// create certificate chain from detached signature
byte[] detachedSignatureContent = documentDetachedSignature.getSignature(); // this is the detached signature file content received from the remote server which contains also customer the certificate
already is a CMS signature container. Consequentially you must not insert this into a PKCS#7 / CMS signature container anymore as you do in the following lines but can immediately inject it into the PDF.
So your second step should just be
// create certificate chain from detached signature
byte[] detachedSignatureContent = documentDetachedSignature.getSignature(); // this is the detached signature file content received from the remote server which contains also customer the certificate
ExternalSignatureContainer externalSignatureContainer = new CustomExternalSignature(detachedSignatureContent);
// add signature content to existing signature container of the intermediary PDF document
PdfReader pdfReader = new PdfReader(preSignedDocumentContent);// this is the value from 1st step for ***PRESIGNED_CONTENT****
ByteArrayOutputStream signedPdfOutput = new ByteArrayOutputStream();
MakeSignature.signDeferred(pdfReader, "Signature_" + customerExternalId, signedPdfOutput, externalSignatureContainer);
return signedPdfOutput.toByteArray();
Furthermore, the wrong hash gets signed. Indeed, in step 1 you should not play around with the PdfPKCS7 class either but simply use
documentDetails.setSigningHash(externalSignatureContainer.getSignatureDigest()); // this is the hash sent to remote server for signing

MD5 with RSA in php

I'm trying to implement digital signature in php as in java sample code below:
Signature rsaSig = Signature.getInstance("MD5withRSA");
RSAPrivateKey clientPrivateKey = readPrivateKeyFromFile(fileName);
rsaSig.initSign(clientPrivateKey);
String source = msg;
byte temp[] = source.getBytes();
rsaSig.update(temp);
byte sig[] = rsaSig.sign();
BASE64Encoder encoder = new BASE64Encoder();
return encoder.encode(sig);
My php code :
$rsa = new Crypt_RSA();
$rsa->loadKey('...'); // in xml format
$plaintext = '...';
$rsa->setSignatureMode(CRYPT_RSA_SIGNATURE_PKCS1);
$signature = $rsa->sign($plaintext);
But looks like some thing is missing. We should get same signature as java code returns.Can anybody guide me in this?

By default phpseclib uses sha1 as the hash. You probably need to do $rsa->setHash('md5').

Convert java.security "NONEwithRSA" signature into BouncyCastle lightweight API

I need to convert Java app into C# and therefore need to migrate from java.security API into BouncyCastle lightweight API.
My working code (java.security) looks like this:
private byte[] computeSignature(byte[] message, PrivateKey key) {
Signature signature = Signature.getInstance("NONEwithRSA");
signature.initSign(privateKey);
signature.update(message);
return signature.sign();
}
This is my verification:
private void verifySignature(byte[] signature, byte[] message, PublicKey publicKey) {
Signature signature = Signature.getInstance("NONEwithRSA");
signature.initVerify(publicKey);
signature.update(message);
System.out.println(signer.verify(result) ? "OK" : "FAIL");
}
Now I am trying to migrate it to BC like this:
problem with NONEwithRSA algorithm which doesn't exist (not sure how to add it)
private byte[] computeSignature(byte[] message, AsymmetricKeyParameter key) {
AlgorithmIdentifier sigAlgId = new DefaultSignatureAlgorithmIdentifierFinder().find("NONEwithRSA");
AlgorithmIdentifier digAlgId = new DefaultDigestAlgorithmIdentifierFinder().find(sigAlgId);
ContentSigner signer = new BcRSAContentSignerBuilder(sigAlgId, digAlgId).build(key);
signer.getOutputStream().write(Arrays.copyOf(message, message.length), 0, message.length);
byte[] signature = signer.getSignature();
}
doesn't provide good signature
private byte[] computeSignature(byte[] message, AsymmetricKeyParameter privateKey) {
Signer signer = new GenericSigner(new RSAEngine(), new NullDigest());
signer.init(true, privateKey);
signer.update(message, 0, message.length);
return signer.generateSignature();
}
Do you have any suggestions? Or is it even possible to migrate the NONEwithRSA algorithm into BC LW API? I assume that I need to write my own Signer, but as a newb to BC and with the BC documentation I can't handle this on my own.

Try this:
RSABlindedEngine engine = new RSABlindedEngine();
PKCS1Encoding paddedEngine = new PKCS1Encoding(engine);
paddedEngine.init(true, privateKey);
return paddedEngine.processBlock(message, 0, message.length);

Calculating ECDSA signature in Java as per an RFC test vector

I am writing a test harness in java for a program relating to the ikev2 protocol. As part of this i need to be able to calculate an ECDSA signature (specifically using the NIST P-256 curve).
RFC 4754 Describes the the use of ECDSA in IKEv2 and helpfully provides a set of test vectors (Including for the p256 curve that i need).
I am trying to run the ECDSA-256 Test Vector values (Section 8.1 in the RFC) through java's ECDSA signature implementation using the following code:
//"abc" for the input
byte[] input = { 0x61, 0x62, 0x63 };
//Ugly way of getting the ECParameterSpec for the P-256 curve by name as opposed to specifying all the parameters manually.
KeyPairGenerator kpg = KeyPairGenerator.getInstance("EC");
ECGenParameterSpec kpgparams = new ECGenParameterSpec("secp256r1");
kpg.initialize(kpgparams);
ECParameterSpec params = ((ECPublicKey) kpg.generateKeyPair().getPublic()).getParams();
//Create the static private key W from the Test Vector
ECPrivateKeySpec static_privates = new ECPrivateKeySpec(new BigInteger("DC51D3866A15BACDE33D96F992FCA99DA7E6EF0934E7097559C27F1614C88A7F", 16), params);
KeyFactory kf = KeyFactory.getInstance("EC");
ECPrivateKey spriv = (ECPrivateKey) kf.generatePrivate(static_privates);
//Perfrom ECDSA signature of the data with SHA-256 as the hash algorithm
Signature dsa = Signature.getInstance("SHA256withECDSA");
dsa.initSign(spriv);
dsa.update(input);
byte[] output = dsa.sign();
System.out.println("Result: " + new BigInteger(1, output).toString(16));
The result should be:
CB28E099 9B9C7715 FD0A80D8 E47A7707 9716CBBF 917DD72E
97566EA1 C066957C 86FA3BB4 E26CAD5B F90B7F81 899256CE 7594BB1E A0C89212
748BFF3B 3D5B0315
Instead I get:
30460221 00dd9131 edeb5efd c5e718df c8a7ab2d 5532b85b 7d4c012a e5a4e90c 3b824ab5 d7022100 9a8a2b12 9e10a2ff 7066ff79 89aa73d5 ba37c868 5ec36517 216e2e43 ffa876d7
I know that the length difference is due to Java ASN.1 Encoding the signature. However, the rest of it is completely wrong and I'm stumped as to why.
Any help or advice would be greatly appreciated!
P.S I am not a ECDSA or Java crypto expert so it is probably a stupid mistake I am making

I'm guessing that each time you run your program, you get a different signature value for the same plaintext (to-be-signed) input.
ECDSA specifies that a random ephemeral ECDSA private key be generated per signature. To that end, Signature.getInstance("SHA256withECDSA") doesn't let you specify an ephemeral key (this is a good thing, to prevent many a self shot in the foot!). Instead, it gets its own SecureRandom instance that will make your output nondeterministic.
This probably means you can't use JCE (Signature.getInstance()) for test vector validation.
What you could do is extend SecureRandom in a way that it returns deterministic data. Obviously you shouldn't use this in a real deployment:
public class FixedSecureRandom extends SecureRandom {
private static boolean debug = false;
private static final long serialVersionUID = 1L;
public FixedSecureRandom() { }
private int nextBytesIndex = 0;
private byte[] nextBytesValues = null;
public void setBytes(byte[] values) {
this.nextBytesValues = values;
}
public void nextBytes(byte[] b) {
if (nextBytesValues==null) {
super.nextBytes(b);
} else if (nextBytesValues.length==0) {
super.nextBytes(b);
} else {
for (int i=0; i<b.length; i++) {
b[i] = nextBytesValues[nextBytesIndex];
nextBytesIndex = (nextBytesIndex + 1) % nextBytesValues.length;
}
}
}
}
Phew. Ok now you have a SecureRandom class that returns you some number of known bytes, then falls back to a real SecureRandom after that. I'll say it again (excuse the shouting) - DO NOT USE THIS IN PRODUCTION!
Next you'll need to use a ECDSA implementation that lets you specify your own SecureRandom. You can use BouncyCastle's ECDSASigner for this purpose. Except here you're going to give it your own bootlegged FixedSecureRandom, so that when it calls secureRandom.getBytes(), it gets the bytes you want it to. This lets you control the ephemeral key to match that specified in the test vectors. You may need to massage the actual bytes (eg. add zero pre-padding) to match what ECDSASigner is going to request.
ECPrivateKeyParameters ecPriv = ...; // this is the user's EC private key (not ephemeral)
FixedSecureRandom fsr_k = new FixedSecureRandom();
fsr_k.setBytes(tempKeyK);
ECDSASigner signer = new ECDSASigner();
ParametersWithRandom ecdsaprivrand = new ParametersWithRandom(ecPriv, fsr_k);
signer.init(true, ecdsaprivrand);
Note that BC's ECDSASigner implements only the EC signature part, not the hashing. You'll still need to do your own hashing (assuming your input data is in data):
Digest md = new SHA256Digest()
md.reset();
md.update(data, 0, data.length);
byte[] hash = new byte[md.getDigestSize()];
md.doFinal(hash, 0);
before you create the ECDSA signature:
BigInteger[] sig = signer.generateSignature(hash);
Finally, this BigInteger[] (should be length==2) are the (r,s) values. You'll need to ASN.1 DER-encode it, which should give you the droids bytes you're looking for.

here's my complete test following tsechin's solution using BouncyCastle but sticking to good old JCA API:
byte[] input = { 0x61, 0x62, 0x63 };
//Create the static private key W from the Test Vector
ECNamedCurveParameterSpec parameterSpec = ECNamedCurveTable.getParameterSpec("secp256r1");
org.bouncycastle.jce.spec.ECPrivateKeySpec privateKeySpec = new org.bouncycastle.jce.spec.ECPrivateKeySpec(new BigInteger("DC51D3866A15BACDE33D96F992FCA99DA7E6EF0934E7097559C27F1614C88A7F", 16), parameterSpec);
KeyFactory kf = KeyFactory.getInstance("EC");
ECPrivateKey spriv = (ECPrivateKey) kf.generatePrivate(privateKeySpec);
//Perfrom ECDSA signature of the data with SHA-256 as the hash algorithm
Signature dsa = Signature.getInstance("SHA256withECDSA", "BC");
FixedSecureRandom random = new FixedSecureRandom();
random.setBytes(Hex.decode("9E56F509196784D963D1C0A401510EE7ADA3DCC5DEE04B154BF61AF1D5A6DECE"));
dsa.initSign(spriv, random);
dsa.update(input);
byte[] output = dsa.sign();
// compare the signature with the expected reference values
ASN1Sequence sequence = ASN1Sequence.getInstance(output);
DERInteger r = (DERInteger) sequence.getObjectAt(0);
DERInteger s = (DERInteger) sequence.getObjectAt(1);
Assert.assertEquals(r.getValue(), new BigInteger("CB28E0999B9C7715FD0A80D8E47A77079716CBBF917DD72E97566EA1C066957C", 16));
Assert.assertEquals(s.getValue(), new BigInteger("86FA3BB4E26CAD5BF90B7F81899256CE7594BB1EA0C89212748BFF3B3D5B0315", 16));

sign the message and verify with recover message by Bouncy Castle

I write the following code for signing the message and then verify it, in java by Bouncy Castle.
signing work properly but verifying not work. the result of code print:
signature tampered
can not recover
and return null.
why eng.hasFullMessage() function return false and why the following code doesn't work?thanks all.
public static String sigVer(PublicKey pu, PrivateKey pr, String original) throws Exception{
//sign
BigInteger big = ((RSAKey) pu).getModulus();
byte[] text = original.getBytes();
RSAKeyParameters rsaPriv = new RSAKeyParameters(true, big,((RSAPrivateKey) pr).getPrivateExponent());
RSAKeyParameters rsaPublic = new RSAKeyParameters(false, big,((RSAPublicKey) pu).getPublicExponent());
RSAEngine rsa = new RSAEngine();
byte[] data;
Digest dig = new SHA1Digest();
ISO9796d2Signer eng = new ISO9796d2Signer(rsa, dig, true);
eng.init(true, rsaPriv);
eng.update(text[0]);
eng.update(text, 1, text.length - 1);
data = eng.generateSignature();
String signature = data.toString();
//verify
eng = new ISO9796d2Signer(rsa, dig, true);
eng.init(false, rsaPublic);
text = signature.getBytes();
if (!eng.verifySignature(text)) {
System.out.println("signature tampered");
}
try{
if (eng.hasFullMessage()) {
eng.updateWithRecoveredMessage(signature.getBytes());
}
byte[] message = eng.getRecoveredMessage();
String ss = message.toString();
return ss;
}
catch (Exception e) {
System.out.println("can not recover");
return null;
}
}

Actually to verify large messages you have to provide the input for verification which is the original message and not only the signature which works only for full recovery
//verify
eng = new ISO9796d2Signer(rsa, dig, true);
eng.init(false, rsaPublic);
// when verifying there has to be also the original plain text
eng.update(text,0,text.length);
signBytes = signature.getBytes();
if (!eng.verifySignature(signBytes)) {
System.out.println("signature tampered");
}

I played around with this method and also receive the errors. In addition the "update" method mentioned does not work and the "updateWithRecoveredMessage" does not exist.
After some testing I figured that:
I have to sue getBytes("UTF-8") and String s = new String(message, "UTF-8")
I can use up to 234 bytes in the signature, with 235 it will break. (guess it would be 256 bytes but some are used for padding/markers)
If I understand correctly the problem is that the signer only allows a certain amount of bytes to be included in the signature.
If you use a longer payload you MUST include the payload in the verification step as well:
eng.init(false, rsaPublic);
byte[] message = payload.getBytes("UTF-8");
eng.update(message, 0, message.length);
then verification works just fine and you get the first part of the payload with eng.getRecoveredMessage().
For me this defeated the purpose, so I went another way.
The workaround I use is to two a two-step approach:
1) I generate a short key which is stored in the signature
2) use a symmetric (i.e. AES) to encrypt the large payload. I also generate a hash for the payload.
3) the key and hash for the payload is the one included in the signature.

Most of the times we don't have the original message to update and verifiy by the signature. On the other hand the length is limited to 234 bytes.