We Keep Coding

Why I need to import server certificates in java

We have developed an console application which will bring data from ESB (https://esb.mkcl.org/) which is on HTTPS. When I hit to this web site I get javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated exception. I changed URL to https://netbanking.hdfcbank.com/netbanking/ which is also on HTTPS and after giving hit I have NOT received any exception.
After some searching I come to know that I have to import public certificate in java's keystore i.e cacerts (C:\Program Files\Java\jre6\lib\security). So I exported certificate of https://esb.mkcl.org/ from browser and imported in java using keytool and then executed console application and it works!! no exception is occurred.
So the question is why I need to import certificate for that particulate URL (where as other HTTPS URLs are working without importing any certificate in java)?

The root certificate used by that server's certificate (COMODO RSA Certification Authority) was added to Java in Java 8 Update 51, so that means the java version you're using (java6 from your jre path) is too old to already include it.
To have that certificate trusted by default, update to a never java version.
And by the way, the ssl configuration for that server is pretty insecure.

A client-side SSL implementation relies on a set of known "trusted root certificates". These are certificates for SSL Certification Authorities that are known to be trusted / trustworthy.
The "problem" is that the set of trusted certs in the keystore that is distributed in your Java JDK / JRE is (typically) smaller than the set in a typical web browser.
There are reasons for this. For example:
A "server" Java installation probably needs to be more conservative on who to trust by default ... for security reasons.
You may be running an older (i.e. out of maintenance) version of Java. Obviously, Oracle will not be refreshing the keystore with new trusted certs.
Some JREs can make use of either the host OSes default keystore, or a browser keystore.
Access to the Mac OS X keystore was introduced in Java 7u4.
Browser plugins can use the browser keystore.

SMTP TLS certificate

I'm having some problems understanding how TLS/SSL is working for email.
I have some questions.
In my development machine if I debug the following code fails the first time arround on the "sslSocket.startHandshake()" line, but if I try it again straight away it is working fine.
The error message that I'm getting is: "Remote host closed connection during handshake".
When I deploy the same code to our staging environment and send an email the code is working fine first time.
Both the development and staging server are in the same network and both have no anti virus programs runnning.
The only thing that I can think of as to why it is not working the first time around in the development environment is because I'm stepping through the code with the debugger and it's slower because of this.
Do you have any knowledge as to why I am receiving this error?
The code underneath is creating an SSL Socket. I'm curious to know if this code is enough for the connection with the mail server to be secure. Are these SSLSocketFactory classes dealing with certificates themselves?
2a) Or do I still need to specify a certificate somehow?
2b) Or is this code getting the certificate from the server and using the certificate to encrypt the data and send the encrypted data back and forth to the email server?
I know that it should work like it is described here:
RFC 3207 defines how SMTP connections can make use of encryption. Once a connection is established, the client issues a STARTTLS command. If the server accepts this, the client and the server negotiate an encryption mechanism. If the negotiation succeeds, the data that subsequently passes between them is encrypted.
2c) Is the code underneath doing this?
socket.setKeepAlive(true);
SSLSocket sslSocket = (SSLSocket) ((SSLSocketFactory) SSLSocketFactory.getDefault()).createSocket(
socket,
socket.getInetAddress().getHostAddress(),
socket.getPort(),
true);
sslSocket.setUseClientMode(true);
sslSocket.setEnableSessionCreation(true);
sslSocket.setEnabledProtocols(new String[]{"SSLv3", "TLSv1"});
sslSocket.setKeepAlive(true);
// Force handshake. This can throw!
sslSocket.startHandshake();
socket = sslSocket;
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));

In my development machine if I debug the following code fails the first time arround on the "sslSocket.startHandshake()" line, but if I try it again straight away it is working fine.
The error message that I'm getting is: "Remote host closed connection during handshake". []
The only thing that I can think of as to why it is not working the first time around in the development environment is because I'm stepping through the code with the debugger and it's slower because of this.
If you just do startHandshake() again with the underlying socket closed it should never work. If you go back to doing the TCP connection (e.g. new Socket(host,port)) and the initial SMTP exchange and STARTTLS, then yes I would expect it to avoid whatever problem affected the previous connection.
Yes, the server timing out because of the delay while you were debugging is quite possible, but to be certain you need to check logs on the server(s).
The code underneath is creating an SSL Socket. I'm curious to know if this code is enough for the connection with the mail server to be secure. Are these SSLSocketFactory classes dealing with certificates themselves?
Indirectly, yes. SSLSocketFactory creates an SSLSocket linked to an SSLContext which includes a TrustManager which is normally loaded from a truststore file. Your code defaults to the default SSLContext which has a TrustManager loaded from the default truststore, which is the file jssecacerts if present and otherwise cacerts in the lib/security directory in the JRE you are running. If your JRE hasn't been modified (by you or anyone else authorized on your system), depending on your variant or packaging of Java the installed JRE usually has no jssecacerts and contains or links to a cacerts file that (initially) contains root certs for about a hundred 'well-known' or established certificate authorities like Symantec, GoDaddy, Comodo, etc.
2a) Or do I still need to specify a certificate somehow?
Since when the handshake is done it is successful, obviously not.
2b) Or is this code getting the certificate from the server and using the certificate to encrypt the data and send the encrypted data back and forth to the email server?
Kind of/sort of/not quite. With some exceptions not applicable here, in an SSL/TLS handshake the server always provides its own certificate and usually intermediate or 'chain' certificates that link its cert to a trusted root cert (such as the abovementioned Symantec etc). The server cert is always used to authenticate the server, and sometimes alone but often combined with other mechanisms (particularly Diffie-Hellman ephemeral DHE or its elliptic-curve variant ECDHE) used to establish a set of several symmetric key values which are then used to encrypt and authenticate the data in both directions. For a more complete explanation see the canonical question and (multi-part!) answer in security.SX https://security.stackexchange.com/questions/20803/how-does-ssl-work/
2c) Is the code underneath doing this?
It is starting an SSLv3 or TLSv1 client-side session on an existing socket. I'm not sure what other question you have here.
You might be better off leaving out the setEnabledProtocols(). Sun/Oracle Java version 8, which is the only one now supported, supports TLS 1.0, 1.1 and 1.2 by default. 1.1 and especially 1.2 are definitely better than 1.0, and should definitely be offered so that if the server supports them they get used. (Sun/Oracle 7 is more problematic; it implements 1.1 and 1.2, but does not enable them client side by default. There I would look at .getSupportedProtocols and if 1.1 and 1.2 are supported but not enabled I would add enable them. But if possible I would just upgrade to 8. Other versions of Java, notably IBM, differ significantly in crypto details.)
SSLv3 should not be offered unless absolutely necessary; it is now badly broken by POODLE (search on security.SX for dozens of Qs about POODLE). I would try without it, and only if the server insists on it re-enable it temporarily, _along with TLS 1.0 through 1.2 whenever possible, and simultaneously urge the server to upgrade so I can remove it again.

java 1.6 TLS1.2 support using proxy nginx/ squid solution issues

I have a legacy java web application that makes calls to an external webservice. The provider of that service is turning off TLS1.0 support. So, I am trying to see how the application can continue to work with the service.
The options I have seen are a) use BouncyCastle JCE instead of Java JCE http://boredwookie.net/index.php/blog/how-to-use-bouncy-castle-lightweight-api-s-tlsclient/, which I guess requires code change/ recompile (we don't have the luxury of doing it) or 2) use proxy servers https://www.reddit.com/r/sysadmin/comments/48gzbi/proxy_solution_to_bump_tls_10_connection_to_tls_12/
I have tried nginx proxy - it doesn't seem to handle the switch between TLS1.0 incoming and TLS1.2 that the end server expects.
server { listen 443 ssl; server_name proxy.mydomain.com;
ssl_certificate D:/apps/openssl/proxy.mydomain.com.cert;
ssl_certificate_key D:/apps/openssl/proxy.mydomain.com.private;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:SEED:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!RSAPSK:!aDH:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!SRP;
ssl_prefer_server_ciphers on;
location / {
proxy_pass https://fancyssl.hboeck.de/;
}
This fails with a 502/ bad gateway error since https://fancyssl.hboeck.de only support TLS1.2 but works with https://www.google.com that supports TLS1.0.
I am doing this on Windows.

It's not TLSv1.2, it's lack of SNI leading to renegotiation.
First, I set up nginx (1.8.1/Windows) with a config like yours except using my own key&cert and proxying to my own test server. It worked fine, connecting from Java6 requester with TLSv1.0 and to server with TLSv1.2 (and even ECDHE-RSA-AES256GCM-SHA384, one of the 'best' ciphersuites) and returned pages just fine. I tried fancyssl.hboek.de and got 502 like you.
With wireshark I saw that nginx does not send SNI (by default) and at least using the IPv4 address 46.4.40.249 (I don't have IPv6) that server apparently hosts more than one domain because without SNI it provides a different (and expired!) certificate, for *.schokokeks.org, and after the first application data (the request) it sends an encrypted handshake (a renegotiation request -- which nginx does not honor). Testing with openssl s_client confirms that the server with SNI immediately sends the page but without it renegotiates first; repointing nginx to openssl s_server confirms that if the server requests renegotiation, receives no response, and closes nginx treats that as 502.
I would guess that Apache is renegotiating because it realizes the requested Host is not covered by the certificate -- except that it again uses the 'wrong' certificate. I haven't tried to track that part down.
Google does support TLSv1.2 (and ECDHE-RSA-AESGCM) when I connect, but even without SNI doesn't renegotiate, presumably because it's such high volume nothing else runs on www.google.com servers and there's no ambiguity. My test server doesn't have vhosts so didn't need SNI.
The nginx documentation reveals a directive proxy_ssl_server_name which can be set on to enable SNI, and then proxying to this server works.
FYI: several of the statements on that webpage are wrong, although its conclusion (if possible use TLSv1.2 with ECDHE or DHE and AES-GCM) is good.
Also, most of your ssl_ciphers string is useless, but you didn't ask about that.
ssl_ciphers HIGH:SEED:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!RSAPSK:!aDH:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!SRP
HIGH is an excellent start.
SEED is useless in a server used (only) by Java/JSSE client, because it's not implemented on the Java side. Even outside of Java it was pretty much used only in South Korea, where it was created as an alternative to DES or IDEA, and even there it is mostly obsoleted by ARIA which is an alternative to AES -- but is not implemented by OpenSSL and hence nginx.
aNULL is probably unneeded because JSSE disables 'anonymous' suites by default, but here it's worth it as defense in depth.
!eNULL does nothing; no eNULL suites are in HIGH, or DEFAULT, or even ALL. You can only get them explicitly or with the bizarre COMPLEMENTOFALL -- which you shouldn't.
!EXPORT !DES !RC4 do nothing; none of them are in HIGH. If instead you started from DEFAULT on older versions of OpenSSL, or from ALL, then they would be good.
!PSK is unneeded; nginx doesn't appear to configure for PSK and JSSE doesn't implement it anyway.
!RSAPSK is ignored because OpenSSL doesn't implement that keyexchange, and if it did those suites are already covered as above.
!aDH !aECDH are covered by !aNULL and thus do nothing.
!EDH-DSS-DES-CBC3-SHA is silly; there's no reason to exclude this one suite when you keep other DHE_DSS and 3DES suites.
!KRB5-DES-CBC3-SHA is ignored because OpenSSL doesn't implement Kerberos, and if it did nginx wouldn't be configured for it plus again it would be silly to exclude one suite while keeping similars.
!SRP is unneeded; like PSK nginx apparently doesn't configure and JSSE doesn't implement.
So: HIGH:!aNULL is all you need.

Client Certificate using Weblogic

Background:
I am working on a project that involves the client to authenticate itself (via client certificate). We are trying to launch a third party app which first authenticates the client cert and then launches its app.
We are using Java and SSL hanshake, and our client server is Weblogic 10.x. The client certificate is in PKCS12 format (cannot change this)
This is the piece of code for presenting the certificate:
System.setProperty("javax.net.ssl.keyStore","MyPath/cert.pfx");
System.setProperty("javax.net.ssl.keyStorePassword","MyPwd");
System.setProperty("javax.net.ssl.keyStoreType","PKCS12");
url.openConnection();
Now all this works pretty well from a standalone. The problem starts when we put this on weblogic. Sometimes it works, sometimes 'Client certificate not present' (http 403.7) is returned. I have tried various combinations (including configuring custome keystore on weblogic console) but nothing seems to work. Any idea why would weblogic altogether ignore the SSL settings (it doesnt throw an error even if I set keystore to a non-existent file)? System.setProperty for SSL just appears useless on weblogic!

In Weblogic 12c you also need add the parameter -DUseSunHttpHandler=true in order to tell the weblogic server to use the Sun Http Handlers instead of its own. Therefore you need the following Java VM parameters:
-Djavax.net.ssl.keyStore=MyPath/cert.pfx
-Djavax.net.ssl.keyStoreType=PKCS12
-Djavax.net.ssl.keyStorePassword=MyPwd
-DUseSunHttpHandler=true

Finally found the solution! Weblogic seems to over-ride the keystore configurations if it is done through Java as above. The thing that worked for me was to configure the SSL settings on JVM level, at web-logic start-up.
To make it work, I just added these Java options in the start-script for weblogic:
-Djavax.net.ssl.keyStore=MyPath/cert.pfx
-Djavax.net.ssl.keyStoreType=MyPwd
-Djavax.net.ssl.keyStorePassword=PKCS12
Restarted web-logic and SSL has been working great since then :)

What would cause SSL negotiations to succeed under .NET but fail under Java?

We have to create a web service client using Apache CXF in Java. The thing is I cannot seem to get the SSL session to properly engage. Either it fails altogether, the server fails to decipher what is sent to it once the application data is transmitted or I fail to read the responses from the server.
However when trying the same transaction using a simple soap test client built in .NET everything runs smoothly.
Server is using double authentication.
Everything is certificate based (x509) stored in the windows certificate store (windows-MY and windows-ROOT)
edit
yes, double authentication is indeed client AND server authentication.
Thus far using the bountyCastle provider instead of SunMSCAPI seems to get further but still cannot get the client authentication to work.
PLatform of client CXF 2.2.9, Sun JDK 1.6_21
server IIS 6 ASP.NET unfortunately is all I could gather, I have no control over the server and must use it as-is.
update
I am using a JKS keystore now but still am getting the problem. It seems the client is not sending his certificate to the server as part of the authentication process. As a result I get a 403.7 error from the server.
Funny thing is that I receive this error message as an HTML page that must first be decrypted before it is readable !

Presumably, by double authentication, you mean you're using client-certificate authentication in addition to server-certificate authentication (which is more common).
It would be useful to know which versions of the platforms are used on either side, and which patches have been applied.
It's possible that some of the problem come from the re-negotiation fix to CVE-2009-3555 (or lack of fix).
The problem is a flaw in the initial design of the re-negotiation in TLS, which is what was used to re-negotiate a client-certificate. There are two ways of getting a client-certificate: either the server asks for it during the initial TLS handshake, or it asks for it during a subsequent handshake (for example, once it has figured out what the request was aimed for and/or when trying to access a certain restricted area). The second method is the re-negotiation. Unfortunately, there was a security flaw in the design of the TLS protocol in that respect, which has since been fixed thanks to a TLS extension described in RFC 5746.
When the flaw was initially disclosed (around November 2009), some platforms and libraries such as Sun Java or OpenSSL rolled out a quick fix which simply disallowed any re-negotiation (so only initial negotiation of the client-certificate would work). Later on, once RFC 5746 was written, these libraries started to roll out implementations supporting this extension.
As far as I'm aware, Microsoft's default in IIS and its web framework was to use re-negotiation and not initial negotiation. In addition, it didn't roll out the initial fix to disable re-negotiation (effectively keeping the known vulnerability). It only rolled out a patch (still tolerant to old implementations by default) quite recently: Microsoft Security Bulletin MS10-049 - Critical.
There is also an explanation of the problem on this Microsoft security blog:
http://blogs.technet.com/b/srd/archive/2010/08/10/ms10-049-an-inside-look-at-cve-2009-3555-the-tls-renegotiation-vulnerability.aspx
Essentially, if you're trying to talk to a server that only supports the old negotiation style from a stack that only has the new re-negotiation style or no renegotiation at all, it's not going to work.
If your server is running using IIS or similar environment, you might be able to turn on initial client-certificate negotiation using netsh and its clientcertnegotiation=enable option.

Java doesn't rely on the OS certificate store and needs to use its own.
This will import your self-signed certificates.
cd JAVA_HOME/jre/lib/security
keytool -import -file server_cert.cer -keystore cacerts

I post this as an answer though I realize now the question was not formulated properly as I got thrown in a loop because the .NET example I had was actually performing a hack to get around the problem.
The proper question should have been
How to get Java to perform Client side Authentication on a server that does not ask for Ask for certificates ?
the answer is actually under our very noses, however to get to the answer one needs the correct question !!
Great thanks to Bruno who provided some very helpful information.
the solution can pretty much be summed up in these two questions :
Java HTTPS client certificate authentication
Client SSL authentication causing 403.7 error from IIS
Although the client is "not supposed" to send a certificate if not asked I found that by tweaking the client certificate in the keystore to contain the following :
Client certificate with all extensions
Client Private key
A concatenation of the client's complete certification chain.
push all this in the same certificate store and use it as keystore. Then load again the certification chain as a trust store. From there it should just work. This being said there is still a possibility for failure. the safest way to solve this particular issue is to have the server actively ask for a authentication certificate from the client by providing a list of accepted CA.
Hope this helps anyone else that can be stuck in the same problem, sure tooke me for a spin for a while before I reach the root of evil.