We Keep Coding

force-sending client certificate

I'm using a Java SE Jersey client to connect to a HTTPS resource which uses two-way SSL.
My SSLContext looks like this:
private SSLContext getSSLContext() {
SslConfigurator sslConfig = SslConfigurator.newInstance()
.keyStoreFile("src/main/certificates/testcert.p12")
.keyPassword("mypassword");
return sslConfig.createSSLContext();
}
The problem is that the client certificate is never sent.
I get error "Warning: no suitable certificate found - continuing without client authentication" and I've tracked the reason to the fact that the client certificate isn't issued to one of the Cert Authorities listed in the server's CertificateRequest message to the client. I know from testing with cURL that the server will accept the certificate regardless. The endpoint is a public test system.
My question: How do I force-send my client certificate? (i.e. my Java SE client should ignore the fact that the testcert.p12 certificate's issuer is not the list of issuers that the server has said it would accept)
Please, don't point me to answers that are about disabling check of the server's certificate or about using self-signed certificates. The server's certificate is just fine.
UPDATE
It turned out my problem was another one. I debug by setting system property javax.net.debug=all. After examining the resulting output it looked to me as if the keystore was empty, even after doing the above. So no wonder why "no suitable certificate found".
Jersey has this 'clever' SslConfigurator class which is there to help you set up an SSLContext. Perhaps just too clever for me, because I couldn't make it work with the above code. So instead, I now configure my SSLContext like below:
KeyStore ks = KeyStore.getInstance("PKCS12");
FileInputStream fis = new FileInputStream("src/main/certificates/testcert.p12");
ks.load(fis, "mypassword".toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, "mypassword".toCharArray());
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(kmf.getKeyManagers(), null, null);
// now use 'sc' in Jersey
This works for me where Jersey's helper class didn't. I fully sympathize with Jersey's idea of a helper class for SSLContext, because JSSE seems overly complex here for such a simple use case. Well, well.

You can't. It would be a TLS protocol violation, and therefore there is no API to support it. The various TLS APIs will only send a client certificate if:
It was requested, and
A client certificate can be found that conforms to what is specified in the CertificateRequest message.
You will have to arrange for your server to trust your client certificate, either by getting it signed by a CA or by exporting it to the server's trusted certificate store, whatever form that takes at the server.

Partial chain verification on Java SSL TrustManager

I have a requirement where one can provide an intermediate CA to trust but not the CA(s) that have signed it. And using that as the trust store, I'd like to be able to trust an SSL server that has a certificate that's signed by this intermediate CA. The default implementation is to expect to build the whole chain until a trusted self-signed root CA is found. I believe that's how the whole X509 platform is based on. But for certain reasons, I can only provide the intermediate CA.
The code is the usual SSLContext creation:
// keystore part is pseudocode to make a point
KeyStore keyStore = someWayToGenerateKeyStore;
keyStore.add(intermediateCa);
//keyStore.add(rootCaThatSignedTheIntermediateCaAbove); it will work if I add this. But I don't want to for reasons.
TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
tmf.init(keyStore);
SSLContext ctx = SSLContext.getInstance("TLSv1.2");
ctx.init(new KeyManager[], tmf.getTrustManagers(), new SecureRandom());
SslContextFactory sslContextFactory = new SslContextFactory();
sslContextFactory.setSslContext(ctx);
client = new WebSocketClient(sslContextFactory);
OpenSSL has a parameter for this that seems to work. So I guess it's not a completely unorthodox approach.
openssl verify -CApath /dev/null -partial_chain -trusted g1 g0
There are two reasons for the requirement to only have the intermediate CA in the truststore.
It makes it easier to pass around one trusted certificate around the system that currently depends on a single self-signed CA certificate. The communication and setup of many components expect a single certificate at the moment and changing that would require significant refactoring.
We want to make sure that we only trust the certificate signed by the given intermediate CA certificate. If we add the other CA certificates in the chain into the trust store, the SSL implementation will trust any certificate signed by the other CAs also which we want to avoid. There are probably other ways to do it like checking the issuer_dn but I'd like to explore different approaches.

The default implementation is to expect to build the whole chain until a trusted self-signed root CA is found
No it isn't. It is to verify the whole chain until a trusted signing certificate is found in your truststore. 'Self-signed' has nothing to do with it, and 'trusted ... root CA' means nothing more than that it is present in your truststore.
So all you need to do is to add that certificate to your truststore. You don't need to write any code at all.
But why you want to trust an intermediate signer without trusting the root signer is a mystery.

Java TLS socket : No trusted certificate found

Let me explain quickly what I'm trying to do. I'm trying to build my own Apple's Push Notification service in java (for testing purposes). This service works thanks to TLS socket.
I have a java client to create a TLS socket to send push notifications to the APNs. I changed the host url to redirect the socket to localhost:2195. Now I'm trying to write a java socket server to get the notification request.
However, I get an exception during the handshake and can't find how to fix it.
Note : I'm using the same certificate on both sides, it's a standard .p12 file that works to send push notifications to the APNs.
Here is the client (simplified) :
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream(certificatePath), password.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("sunx509");
kmf.init(ks, password.toCharArray());
TrustManagerFactory tmf = TrustManagerFactory.getInstance("sunx509");
tmf.init((KeyStore)null);
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
SSLSocketFactory ssf = sc.getSocketFactory();
SSLSocket socket = (SSLSocket) ssf.createSocket(InetAddress.getLocalHost(), 2195);
socket.startHandshake();
Here is the server :
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream(certificatePath), password.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("sunx509");
kmf.init(ks, password.toCharArray());
SSLContext context = SSLContext.getInstance("TLS");
context.init(kmf.getKeyManagers(), null, null);
SSLServerSocketFactory ssf = context.getServerSocketFactory();
serverSocket = (SSLServerSocket) ssf.createServerSocket(2195);
And here is the exception :
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found
I guess the client isn't trusting the server's certificate. I tryed to set the client's TrustManager to accept the server's p12 and it worked, however I need this to work without editing the client (since it's working that way with the real APNs).
What kind of certificate needs the server to be trusted by the client ?
Thanks in advance.

EDIT: I WAS WRONG! tmf.init(null) DOES use the default keystore just like sslctx.init(,null,) !
That default is normally the cacerts file in JRE/lib/security which DOES trust many established CAs
so now I think we can be confident the real server is using a cert under an established CA (and so
is trusted by your client) while the cert in your p12 apparently does not;
but there are two possibilities here:
it is selfsigned, or issued by an unknown, obscure, or unproven CA
it is issued by a 'real' CA under an 'intermediate' CA that needs a chain cert (or several)
and you do not have the chain cert(s) in your p12. Note this could still work for client auth
to the real server, because the real server can easily have the chain cert(s) 'preloaded'
in its truststore even though they aren't in Java's.
To distinguish these, look at keytool -keystore file -storetype pkcs12 -list -v
and see what cert or sequence of certs you have there.
Then there may be several approaches to solution:
if you are only missing chain cert(s) for an established CA get them and add them.
keytool only allows you to replace the whole chain so you must get all needed certs;
openssl (if you have or get it) can break out the key and cert(s) from a pkcs12,
replace or add individual certs, and join them back together.
create a different store and key for the server and get it a cert (chain) from an established CA.
Usually costs some money and requires you prove control of the server's domain name.
(Your client can and should still use this p12. The two sides needn't be the same.)
locate the trust anchor (from the p12, or from somewhere else like the CA) and have it
in a truststore the client explicitly loads. You effectively tried this by using the
p12 as the truststore and say you don't want that.
put the trust anchor in the client's default truststore, so the client continues
using the default. If you don't mind modifying your JRE (and no other user or application
on your system is bothered) just add to JRE/lib/security/cacerts. Or, assuming you can set
system properties, put the anchor in a store or just leave it in the p12
and set javax.net.ssl.trustStore{,Password,Type} to point to that store.
(If you copy you should take only the cert; a p12 is a KEY AND cert not just a cert.
Don't just -importkeystore; -importcert a cert file, created with -exportcert if necessary.)
(You can System.setProperty in your code, but that's changing your code. If you run from
commandline you can use 'java -Dname=value...'. For other cases YMMV.)
There is one possible 'type' issue: if the cert was issued with ExtendedKeyUsage extension
and that value specifies only TLSclient and not TLSserver (which the CA can choose to do)
then using it for server probably won't work -- it appears JSSE enforces EKU restrictions.
But if that is a problem you'll get a very different Exception.
And you can see this also in the keytool -list -v above.
Since you (rightly) want to use this p12 for your client, your server logic similarly needs
to trust it. (Using it for outgoing auth does NOT automatically make it trusted for incoming auth.)
But only if/when clientAuth is actually done, which is not the default; does your server code
.setNeedClientAuth(true) on the SSLServerSocket before accepting the connection?
Possible approaches are equivalent to the above except skipping #2 as inapplicable.
If both client and server use the same JRE, that makes the cacerts way a little easier.
Finally, yes TrustManager 'PKIX' is newer and generally more featureful than 'SunX509'.
But for the basic test 'is the trust anchor in our truststore' they are equivalent.
Sorry again for the mislead.

Error when opening https URL: keyCertSign bit is not set

I am calling a remote https URL with the following code:
def inputStream = new URL("https://somewebsite.com").openStream()
This works great on my local machine, but when I deploy to the server, I get the following Exception:
java.security.cert.CertPathValidatorException: CA key usage check failed: keyCertSign bit is not set
What is the cause of this error, and what could account for it working on one machine and not another?
UPDATE
I'm running an Ubuntu server in production and developing on a Mac locally. The site I'm trying to access (let's call it peopleware.com) has the following certificate info:
AddTrust External CA Root
UTN-USERFirst-Hardware
peopleware.com
I have tried saving the .cer files from my browser and installing them into the keystore at /etc/ssl/certs/java/castore

I'm assuming that you're talking about this certificate from UTN-USERFirst-Hardware:
-----BEGIN CERTIFICATE-----
MIIEdDCCA1ygAwIBAgIQRL4Mi1AAJLQR0zYq/mUK/TANBgkqhkiG9w0BAQUFADCB
lzELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYDVQQHEw5TYWx0IExha2Ug
Q2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYDVQQLExho
dHRwOi8vd3d3LnVzZXJ0cnVzdC5jb20xHzAdBgNVBAMTFlVUTi1VU0VSRmlyc3Qt
SGFyZHdhcmUwHhcNOTkwNzA5MTgxMDQyWhcNMTkwNzA5MTgxOTIyWjCBlzELMAkG
A1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYDVQQHEw5TYWx0IExha2UgQ2l0eTEe
MBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYDVQQLExhodHRwOi8v
d3d3LnVzZXJ0cnVzdC5jb20xHzAdBgNVBAMTFlVUTi1VU0VSRmlyc3QtSGFyZHdh
cmUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCx98M4P7Sof885glFn
0G2f0v9Y8+efK+wNiVSZuTiZFvfgIXlIwrthdBKWHTxqctU8EGc6Oe0rE81m65UJ
M6Rsl7HoxuzBdXmcRl6Nq9Bq/bkqVRcQVLMZ8Jr28bFdtqdt++BxF2uiiPsA3/4a
MXcMmgF6sTLjKwEHOG7DpV4jvEWbe1DByTCP2+UretNb+zNAHqDVmBe8i4fDidNd
oI6yqqr2jmmIBsX6iSHzCJ1pLgkzmykNRg+MzEk0sGlRvfkGzWitZky8PqxhvQqI
DsjfPe58BEydCl5rkdbux+0ojatNh4lz0G6k0B4WixThdkQDf2Os5M1JnMWS9Ksy
oUhbAgMBAAGjgbkwgbYwCwYDVR0PBAQDAgHGMA8GA1UdEwEB/wQFMAMBAf8wHQYD
VR0OBBYEFKFyXyYbKJhDlV0HN9WFlp1L0sNFMEQGA1UdHwQ9MDswOaA3oDWGM2h0
dHA6Ly9jcmwudXNlcnRydXN0LmNvbS9VVE4tVVNFUkZpcnN0LUhhcmR3YXJlLmNy
bDAxBgNVHSUEKjAoBggrBgEFBQcDAQYIKwYBBQUHAwUGCCsGAQUFBwMGBggrBgEF
BQcDBzANBgkqhkiG9w0BAQUFAAOCAQEARxkP3nTGmZev/K0oXnWO6y1n7k57K9cM
//bey1WiCuFMVGWTYGufEpytXoMs61quwOQt9ABjHbjAbPLPSbtNk28Gpgoiskli
CE7/yMgUsogWXecB5BKV5UU0s4tpvc+0hY91UZ59Ojg6FEgSxvunOxqNDYJAB+gE
CJChicsZUN/KHAG8HQQZexB2lzvukJDKxA4fFm517zP4029bHpbj4HR3dHuKom4t
3XbWOTCC8KucUvIqx69JXn7HaOWCgchqJ/kniCrVWFCVH/A7HFe7fRQ5YiuayZSS
KqMiDP+JJn1fIytH1xUdqWqeUQ0qUZ6B+dQ7XnASfxAynB67nfhmqA==
-----END CERTIFICATE-----
In a human-readable version:
Version: 3 (0x2)
Serial Number:
44:be:0c:8b:50:00:24:b4:11:d3:36:2a:fe:65:0a:fd
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, ST=UT, L=Salt Lake City, O=The USERTRUST Network, OU=http://www.usertrust.com, CN=UTN-USERFirst-Hardware
Validity
Not Before: Jul 9 18:10:42 1999 GMT
Not After : Jul 9 18:19:22 2019 GMT
Subject: C=US, ST=UT, L=Salt Lake City, O=The USERTRUST Network, OU=http://www.usertrust.com, CN=UTN-USERFirst-Hardware
Subject Public Key Info:
[...]
X509v3 extensions:
X509v3 Key Usage:
Digital Signature, Non Repudiation, Certificate Sign, CRL Sign
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Subject Key Identifier:
A1:72:5F:26:1B:28:98:43:95:5D:07:37:D5:85:96:9D:4B:D2:C3:45
X509v3 CRL Distribution Points:
Full Name:
URI:http://crl.usertrust.com/UTN-USERFirst-Hardware.crl
X509v3 Extended Key Usage:
TLS Web Server Authentication, IPSec End System, IPSec Tunnel, IPSec User
Essentially, we have here a CA certificate with both X509v3 Key Usage and X509v3 Extended Key Usage.
However, RFC 3280 says the following about the extended key usage extension:
In general, this extension will appear only in end entity
certificates.
This doesn't start too well for a CA cert, but later on, the same section also says this:
If a certificate contains both a key usage extension and an extended
key usage extension, then both extensions MUST be processed
independently and the certificate MUST only be used for a purpose
consistent with both extensions. If there is no purpose consistent
with both extensions, then the certificate MUST NOT be used for any
purpose.
The only extended key usage extension in this cert that is in this RFC is TLS Web Server Authentication:
id-kp-serverAuth OBJECT IDENTIFIER ::= { id-kp 1 }
-- TLS WWW server authentication
-- Key usage bits that may be consistent: digitalSignature,
-- keyEncipherment or keyAgreement
Of course, this is not consistent with keyCertSign, which, according to RFC 3280 (and RFC 5280). (I also doubt any of the IPSec extensions are compatible with keyCertSign). This makes this certificate useless to issue certificates (not very useful for a CA certificate).
I would contact the website using this certificate to ask them to contact their CA (UTN-USERFirst-Hardware, apparently Comodo) and ask them to fix this. I must say it doesn't look good coming from people who make their money on the back of these RFCs.
Of course, this could take a while and wouldn't solve your immediate problem.
I think I've seen this Subject DN (UTN-USERFirst-Hardware) in other intermediate CA certificates, so the one above might not be the one you're using.
What you might be able to do (provided that you're able to verify the server certificate itself manually despite these problems) is to use an SSLContext and a TrustManager specifically limited to use that very certificate, for this connection. This could prevent the certification path algorithm to try to find the issuer certificate and fall into this problem.
EDIT:
Here are more details on this workaround (which should still keep your connection secure).
Connect with Firefox to this website.
Click on the blue/green bar and choose 'More information...'
Security -> View Certificate -> Details
Choose the server certificate from the list at the top and choose 'Export...'
Same the PEM file somewhere.
Use keytool to create a new keystore (choose to trust that certificate and choose a sensible password):
keytool -importcert -keystore example.jks -file example.pem
Then, use this Java code, which shouldn't be too hard to port to Groovy:
TrustManagerFactory tmf = TrustManagerFactory
.getInstance(TrustManagerFactory.getDefaultAlgorithm());
KeyStore ks = KeyStore.getInstance("JKS");
FileInputStream fis = new FileInputStream("/.../example.jks");
ks.load(fis, null);
// or ks.load(fis, "thepassword".toCharArray());
fis.close();
tmf.init(ks);
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, tmf.getTrustManagers(), null);
URL url = new URL("https://somewebsite.com");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setSSLSocketFactory(sslContext.getSocketFactory());
InputStream is = conn.getInputStream();

How to make 'simple SSL' thru Web Services?

I know how to secure Web Services using certificates. that's my client code:
SSLContext ssl = SSLContext.getInstance("SSLv3");
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore store = KeyStore.getInstance(KeyStore.getDefaultType());
String password = Configuration.getConfig("keyStorePassword");
store.load(new FileInputStream(new File(Configuration.getConfig("keyStore"))), password.toCharArray());
kmf.init(store, password.toCharArray());
KeyManager[] keyManagers = new KeyManager[1];
keyManagers = kmf.getKeyManagers();
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(store);
TrustManager[] trustManagers = tmf.getTrustManagers();
ssl.init(keyManagers, trustManagers, new SecureRandom());
HttpsConfigurator configurator = new HttpsConfigurator(ssl);
Integer port = Integer.parseInt(Configuration.getConfig("port"));
HttpsServer httpsServer = HttpsServer.create(new InetSocketAddress(Configuration.getConfig("host"), port), 0);
httpsServer.setHttpsConfigurator(configurator);
Implementor implementor = new Implementor(); // class with #WebService etc.
HttpContext context = (HttpContext) httpsServer.createContext("/EventWebService");
Endpoint endpoint = Endpoint.create( implementor );
endpoint.publish(context);
Now, how to make 'simple SSL' ? How to make SSL connection without storing certificate on the client side. (Like connecting thru HTTPS in browser)

Java Runtime Environment does come with a lots (most widely used) Certificate Authorities in cacerts file. If the certificate you used to secure your service is signed by one of those root CAs, then you need not worry about sharing any certificate with clients.
However if you used self-signed certificate, and you don't want to pass/import certificate in truststore then you can implement custom X509TrustManager and create custom SSLContext for your connections. More details in this blog.
Self-signed certificate are useful for development and test environments but you really should consider getting your server certificate signed from a recognized Certificate Authority like Verisign, Thwate etc.

If I understand you correctly, then you want to have only server-side authentication much in the same way as if you connected to an https site in your browser, without requiring your clients to manage any certificates.
Your clients would connect as usual, simply replacing an http for an https in the connection URL. Java manages its own set of "default trusted root CA authorities" in the form of cacerts, a JKS keystore file located in $JRE HOME/lib/security. If you buy a certificate from any CA whose issuing certificate roots in one of the certificates contained in cacerts, then the client's certificate validation will automagically succeed. Google for "SSL/TLS server certificate" and you will find suitable vendors.
If you would use a self-issued certificate on the other hand, then there's no way to make certificate validation succeed on the client other than importing your self-made certificate in the client's certificate trust store. But that's why a "real" SSL/TLS certificate costs money and your self-issued certificate doesn't - anyone can generate their home-grown certificates, but trusting them is an entirely different story.

You can control if the https server requires client certificates in this way:
HttpsConfigurator cfg = new HttpsConfigurator(sslCtx){
public void configure(HttpsParameters params) {
SSLParameters sslparams = getSSLContext().getDefaultSSLParameters();
// Modify the default params:
// Using this, server will require client certs
//sslparams.setNeedClientAuth(true);
// Using this, server will request client certs. But if not available,
// it will continue anyway.
sslparams.setWantClientAuth(true);
params.setSSLParameters(sslparams);
}
};
HttpsServer httpsS = HttpsServer.create(new InetSocketAddress(8081), 50);
httpsS.setHttpsConfigurator(cfg);
If client certs are not required, clients can connect without client certificate, so simple calling https will work.
In my blog you can see example of client for how to bypass the server certificate and hostname validation (although not recommended, useful e.g. for testing)
http://jakubneubauer.wordpress.com/2011/09/06/java-webservice-over-ssl/

Just make the connection with HTTPS. As long as the client is using standard trusted certs it will work just fine. If they have a self signed cert you will need to to import the cert into the java keystore.

HTTPS in browsers works because there is a truststore containing SSL certificates on the client. In other words: There are certificates stored on the client side.
If you want HTTPS without any certificate stored on the client side, I think you should have a look at this article, which explains how to turn off the default certificate validation on HTTPS connection.