We Keep Coding

Jboss EAP 7 - Dynamic SSL authentication for REST API clients

I'm trying to get my head around some questions about SSL certificates and their signing.
My need : I need to be able to add (generate?) client certificate to allow them to use my API.
My problem : It seems that Wildfly needs to be reload if the java truststore is changed in order for the modifications to be available.
What I understood : This is a wrong approach of the problem and what should be done is the following:
Get a CA certificate from a trusted authority
Add this certificate to my keystore
Use that certificate to sign the clients certificates
This way only my CA certificate needs to be on my java keystores and through the CA validation chain, my client would be allowed on my service.
Questions :
Did I got this right ?
If I'm supposed to sign my clients certificates, what are they supposed to send me for the signing to be possible ?
Can this be done without calling keytool utility ? I'd like as possible can be to use java to do this and not rely on the OS.
If my CA certificate comes from a trusted authority and I use it to sign my clients certificates, those this still count as self-signed certificate ?
Thanks for all the pointers that you could provide.

Ok,
Since it may help others I'm going to put what my various testing helped me to figured out. :
Did I got this right ?
As far as I can tell : Yes
Why can't I be sure about it ? Because I'm encountering some testing limitations due to my development environment and the fact that I can't have a real valid certificate with a valid domain to test it.
If I'm supposed to sign my clients certificates, what are they supposed to send me for the signing to be possible ?
So what they need to send for the signing to be possible is a CSR (Certificate Signing Request) and their public key.
Can this be done without calling keytool utility ? I'd like as possible can be to use java to do this and not rely on the OS.
This should be doable using the Bouncycastle library. I'm using should because I did not go into implementing it, more on tha later.
If my CA certificate comes from a trusted authority and I use it to sign my clients certificates, those this still count as self-signed certificate ?
I'm still not clear on that, but it seems that there is some kind of scope on certificate signing that prevents anyone from faking the actual CA. Anyone that can put more sure knowledge on this, is welcome.
What I actually ended up doing :
Being our own CA would be too much hurdle, so what we're actually going to do is ask our clients to get their certificates from known CA and add those CA certs to our truststore. This is the same principle as before but instead of us playing at being CA, we'll leverage actual CA sources.
Hope this helps anyone that would some kind of equivalent requirement.

Can a signed certificate be used without importing explicitly?

I've read a lot of articles regarding the import of a cert, but I am still unclear on a couple things.
When connecting to an SSL site from a Java application [in this case, a JBOSS web app], does the client cert need to be explicitly installed on the application server prior?
I can install a client cert manually, but there is an expiration date. So I'll need to manage the expiration dates of all client installed certs on our application server, and take an outage to update each one.
It feels like there should be a better way.
Shouldn't the application automatically accept a valid signed cert? [In this case, it is signed by VeriSign]
We are getting an exception currently when trying to access an https url from the application without explicitly installing the cert.
The API proxy library is swallowing the internal exception, so I dont know the details.
If the cert should be accepted automatically, then there may be a different issue here...

Can a signed certificate be used without importing explicitly?
Yes, it does not need to be installed prior to use. In fact, if you know in advance of what to expect, then you can include that information into the application. That has an added benefit of improving the application's security posture.
To avoid importing the certificate, use a custom X509TrustManager and override checkServerTrusted. In checkServerTrusted, ensure the server's public key is expected (i.e., pin the server's certificate or public key); or verify the server's certificate is valid (i.e., is within validity and forms a chain to your trusted root).
When connecting to an SSL site from a Java application [in this case, a JBOSS web app], does the client cert need to be explicitly installed on the application server prior?
In the case of client certificates, the server advertises the issuer whom it relies upon to issue client certificates. So the server will need to know the trust point for issuing client certifcates for authenticating clients.
In this case, it is signed by VeriSign
This could be really bad. In this case, you will trust all of your clients signed under the Verisign PKI, and all of Verisign's other clients signed under the Verisign PKI.
In this case, it would probably be better to avoid public CAs and run your own PKI (i.e., be your own CA). In this case, pick up a copy of Network Security with OpenSSL. The book will show you how to accomplish the customary tasks using both the openssl command and programmatically.

How to connect with CA to sign ssl certificate?

I have never done https so far, but trying to get grips with it. I understand how to do self-signed certificates which is from what I understood is completely pointless as browsers will fire up straight away the warning. "As one cannot say trust me because I am honest."
From SO posts I understood that Tomcat or any other server environment is responsible for configuration. All clear and good.
The confusion is how these certificate signed when not self-signed. In other words, how do I point my keystore to CA provider.
I am following guide:
http://consultingblogs.emc.com/richardtiffin/archive/2010/10/15/applying-ssl-to-a-spring-web-application-on-tomcat.aspx
The problem is that I don't know what happens after I buy certificate from CA. Do I get a file which I need to point my keystore to or I import keystore itself remotely?
For self-signed we do:
"
Generating a keystore file (Self Certification)
The keystore file is the one which would store the details of the certificates necessary to make the protocol secured, to-do this we'll use the keytool provided as part of JDK 1.6, the following should create the keystore :-
From your Java installation directory %JAVA_HOME%/bin
keytool -genkey -alias emc -keypass password -keystore emc_tomcat.bin -storepass password"
This implies that for not self-signed the details must come from CA provider? So need to add something to the line(URL path?)? Or is it the client responsibility to verify whether my certificate is valid...then how do I link my certificate to the one I purchased. I am completely confused.
Rephrasing other way: how to install CA from official CA providers?
Or I am telling complete rubbish and missing something fundamental. :-)
My apologies if there is duplicate, I've spent quite good bit of time of researching SO and reading wikipedia articles, but the internal mechanics are very difficult for novice users. I've marked bold the actual questions for people short on time. Thanks.

All the information you are asking for is readily provided by any reputable ("trusted") CA. See for example http://www.digicert.com/ssl-certificate-installation.htm
Quoting:
After you create your CSR, purchase your certificate, and the SSL
Certificate validation and processing are complete, you are ready to
install your SSL Certificate(s).
Your certificate will be provided via email or will be available to
download in your DigiCert Management Console. The SSL Certificate is a
text file with encrypted data that your server will use once the
certificate is installed
And following down the Tomcat link:
Tomcat SSL installations can be a bit tricky, but don't worry. Our
knowledgeable support staff is familiar with both the keytool utility
as well as common Tomcat installation methods. We will be happy to
help you generate your Certificate Signing Request (or CSR), install
the certificate to your Java keystore, and configure your server to
use it via either the Tomcat Admintool utility, or by editing the
server.xml configuration file directly.
It seems that this answers all your questions... "buy it and you will see how easy it is".

A straight forward way to open self-signed HTTPS SSL URLs in java?

I'm building an application that needs to open self-signed HTTPS SSL URLs in java. I've learned that you can bypass the SSL problems by adding a call to HttpsURLConnection.setDefaultHostnameVerifier() where you say what hostnames are allowed.
However, I have a second problem where my servers are running self-signed certs. So even with the hostname bypass I'm getting exceptions like:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I've looked around and the only solutions I've found is to add the certificate to the java key store, but this is a problem for me because I have no control over when or how they update java, and I've read that the keystore isn't preserved between updates, and I have no access to the system outside of the JVM.
My application will only make calls to a single server so if there was a way to bypass the self-signed restrictions, and never consult keystores, it wouldn't be a security problem, but is there a way to do this?

I'm building an application that needs to open self-signed HTTPS SSL
URLs in java. I've learned that you can bypass the SSL problems by
adding a call to HttpsURLConnection.setDefaultHostnameVerifier() where
you say what hostnames are allowed.
There are some misconceptions from your question:
Hostname verification is unrelated to whether the certificate is self-signed or not.
It is a verification that matches the domain you are trying to access with the certificate info (CN or Subject Alt Name).
You would need to configure the verifier if you needed to accept a certificate that did not match the url (not recommended at all!!!)
Concerning the self-signed.
This is irrelevant.
You can configure your application to load your custom truststore which would include the certificate your application will trust. In fact this is the best approach (than using Java's cacerts).
All you have to do is import the certificate in a keystore (JKS or PKCS12) and load it in your custom TrustManagers in your application.
Just google arround, plenty of examples E.g. self-signed-ssl

As much as I hate to say this, sometimes it's better to just go with the flow.
Java is attempting to make applications more secure through the use of proper SSL verification practices. In this case, it is succeeding: had you been able to tell the program "it's okay, accept the untrusted self-signed certificate", your program would have been vulnerable to a man-in-the-middle attack where Mallory puts his server (with its own self-signed certificate, just as valid as yours!) in between the host and the target it's attempting to communicate with. Then he proceeds to read all the traffic you thought was nice and safe, and you don't even notice.
So, your assertion that telling Java to "trust any self-signed certificate when connecting to this host" is secure is, regrettably, not correct.
You can get free, totally valid SSL certificates from StartSSL. They're good folks.

That's how PKI is supposed to work - you must have complete chain of trust from some trusted certificate stored in you keystore to your certificate. So you can
set you certificate as trusted
ask somebody already trusted (i.e. with trusted certificate in the keystore) to sign you certificate
Trying to bypass that is not good bad idea. You can install you certificate in some Java post-install hook, you may have some cron job checking it periodically or do it in exception handling. You can even download this way a certificate from the server and install it everytime it changes (extracting the cert with openssl is easy). But for god's sake, if you decide to do such thing, at least write some audit log about it to some third machine a make sure somebody reads it.
You can also write "hacker-friendly" on you doors. :)

(Note that when you're talking about "keystore" in your question, you're in fact talking about the trust store (which is a keystore). More details on this unfortunately confusing Java terminology are in this answer.)
My application will only make calls to a single server so if there was
a way to bypass the self-signed restrictions, and never consult
keystores, it wouldn't be a security problem, but is there a way to do
this?
Actually, it would be a security problem. Your application may indeed be intended to call only a single server, but the trust store is precisely there to help make sure you're connecting to the machine you intended to connect to. Without it, you wouldn't be able to know whether you're connecting to that server or a MITM server.
If you want the security provided by SSL/TLS, don't bypass these rules. (In particular, don't use a trust manager that will accept any certificate.)
I've looked around and the only solutions I've found is to add the
certificate to the java key store, but this is a problem for me
because I have no control over when or how they update java, and I've
read that the keystore isn't preserved between updates, and I have no
access to the system outside of the JVM.
Quoting myself from this answer (to a more specific question):
The right way is to import this self-signed certificate into the client's trust store, using keytool for example:
keytool -import -file server-cert.pem -alias myserver -keystore mytruststore.jks
You can either do it directly in the JRE's trust store (lib/security/cacerts), which may lack some flexibility, or do it in your own copy of this file, which you then set as the trust store (the default password is changeit or changeme on OSX). You configure this truststore globally for your application using the usual javax.net.ssl.trustStore* system properties (e.g. -Djavax.net.ssl.trustStore=mytruststore system property (and -Djavax.net.ssl.trustStorePassword`). [...]
You don't actually have to use the trust store provided by the JRE (and which may be updated regularly). You could import your self-signed cert into a new empty keystore, which you'll use as a trust store within your application (don't import the private key with it). How to discuss trust store management was in fact discussed in this question: nothing prevents you from using a different trust store specifically for your application or part of it (and in fact Sun/Oracle make no guarantee as to the suitability of the default trust store).
I'm building an application that needs to open self-signed HTTPS SSL
URLs in java. I've learned that you can bypass the SSL problems by
adding a call to HttpsURLConnection.setDefaultHostnameVerifier() where
you say what hostnames are allowed.
While it may indeed by slightly less of a problem if you only have a single self-certificate in your trust store, host name verification is also an essential component of the security provided by SSL/TLS. Don't bypass it, verify that the certificate you're connecting to matches the name you're trying to connect to. (To make an analogy, if you want to check someone's identity, you not only want to check the their passport was emitted by a country whose passports you trust, but you'll also want to check they have the right name inside, otherwise you could be in front of anyone from that country.)
Making the CN= RDN of the Subject Distinguished Name of your self-signed certificate be the host name of the server should be enough, although it's the legacy way of doing it. You may also want to add the Subject Alternative Name extension. More on this in this answer.
Generally speaking do not bypass the SSL "problems". These mechanisms are precisely in place to make the usage of SSL/TLS secure.

Can I create self-signed certificate in Java which will be automatically trusted by web browsers?

I've generated a self-signed certificate for my Java app using keytool. However, when I go to the site in a browser it always pops up with a warning - saying this site does not own the certificate - is there a way to self-sign/doctor a certificate so I won't get these warnings in a browser? Both server and browser are located on the same host and I navigate to the site using "http://localhost/". I do not want to add an exception to the browser because I have tests which run on a big build farm so it is excessive to add an exception to all browsers on all build machines.

No, you can't. You might as well ask "How can I make a fake certificate for hsbc.com?"
There are two ways to get a browser to accept a certificate:
Buy a certificate for a domain from a trusted authority (which means proving to that authority that you own that domain) and then use that domain as the name of your test servers
Install your signing certificate into the browsers, so that you effectively become a trusted authority for that browser.
Without touching the browsers, there's no other way to do it - how could there be, if the internet is to remain secure?

You could also setup a self-signed Certificate Authority (CA) using OpenSSL or possibly your Java tool. You can then use that CA to sign a number of server certs.
You are still going to need to manually trust your self-signed CA on all clients that access your test servers, but at least you only have to trust one root CA, rather than a bunch of individual self-signed server certs.
Another option is to check out CAcert.

Is the certificate you created for localhost or for test.textbox.com? If you create a certificate for the FQDN test.textbox.com, that's how you need to be reaching the server to not get those errors, as long as the certificate is properly signed. You can't generate a certificate for the FQDN and then use an IP or an alias (localhost) to access it without being warned that things aren't matching up properly. Or am I misunderstanding your problem?

Make the certificate for "localhost" instead. It needs to match the hostname you have in the URL.
You will still be bothered as the certificate is not trusted, but that is another issue.