We Keep Coding

How to use HttpClient with ANY ssl cert, no matter how "bad" it is

I'm using Apache HttpClient in a web crawler that is only for crawling public data.
I'd like it to be able to crawl sites with invalid certificates, no matter how invalid.
My crawler won't be passing in any usernames, passwords, etc and no sensitive data is being sent or received.
For this use case, I'd crawl the http version of a site if it exists, but sometimes it doesn't of course.
How can this be done with Apache's HttpClient?
I tried a few suggestions like this one, but they still fail for some invalid certs, for example:
failed for url:https://dh480.badssl.com/, reason:java.lang.RuntimeException: Could not generate DH keypair
failed for url:https://null.badssl.com/, reason:Received fatal alert: handshake_failure
failed for url:https://rc4-md5.badssl.com/, reason:Received fatal alert: handshake_failure
failed for url:https://rc4.badssl.com/, reason:Received fatal alert: handshake_failure
failed for url:https://superfish.badssl.com/, reason:Connection reset
Note that I've tried this with my $JAVA_HOME/jre/lib/security/java.security file's jdk.tls.disabledAlgorithms set to nothing, to ensure this wasn't an issue, and I still get failures like the above.

The short answer to your question, which is to specifically trust all certs, would be to use the TrustAllStrategy and do something like this:
SSLContextBuilder sslContextBuilder = new SSLContextBuilder();
sslContextBuilder.loadTrustMaterial(null, new TrustAllStrategy());
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(
sslContextBuilder.build());
CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(
socketFactory).build();
However... an invalid cert may not be your main issue. A handshake_failure can occur for a number of reasons but in my experience it's usually due to a SSL/TLS version mismatch or cipher suite negotiation failure. This doesn't mean the ssl cert is "bad", it's just a mismatch between the server and client. You can see exactly where the handshake is failing using a tool like Wireshark (more on that)
While Wireshark can be great to see where it's failing, it won't help you come up with a solution. Whenever I've gone about debugging handshake_failures in the past I've found this tool particularly helpful: https://testssl.sh/
You can point that script at any of your failing websites to learn more about what protocols are available on that target and what your client needs to support in order to establish a successful handshake. It will also print information about the certificate.
For example (showing only two sections of the output of testssl.sh):
./testssl.sh www.google.com
....
Testing protocols (via sockets except TLS 1.2, SPDY+HTTP2)
SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 offered
TLS 1.1 offered
TLS 1.2 offered (OK)
....
Server Certificate #1
Signature Algorithm SHA256 with RSA
Server key size RSA 2048 bits
Common Name (CN) "www.google.com"
subjectAltName (SAN) "www.google.com"
Issuer "Google Internet Authority G3" ("Google Trust Services" from "US")
Trust (hostname) Ok via SAN and CN (works w/o SNI)
Chain of trust "/etc/*.pem" cannot be found / not readable
Certificate Expiration expires < 60 days (58) (2018-10-30 06:14 --> 2019-01-22 06:14 -0700)
....
Testing all 102 locally available ciphers against the server, ordered by encryption strength
(Your /usr/bin/openssl cannot show DH/ECDH bits)
Hexcode Cipher Suite Name (OpenSSL) KeyExch. Encryption Bits
------------------------------------------------------------------------
xc030 ECDHE-RSA-AES256-GCM-SHA384 ECDH AESGCM 256
xc02c ECDHE-ECDSA-AES256-GCM-SHA384 ECDH AESGCM 256
xc014 ECDHE-RSA-AES256-SHA ECDH AES 256
xc00a ECDHE-ECDSA-AES256-SHA ECDH AES 256
x9d AES256-GCM-SHA384 RSA AESGCM 256
x35 AES256-SHA RSA AES 256
xc02f ECDHE-RSA-AES128-GCM-SHA256 ECDH AESGCM 128
xc02b ECDHE-ECDSA-AES128-GCM-SHA256 ECDH AESGCM 128
xc013 ECDHE-RSA-AES128-SHA ECDH AES 128
xc009 ECDHE-ECDSA-AES128-SHA ECDH AES 128
x9c AES128-GCM-SHA256 RSA AESGCM 128
x2f AES128-SHA RSA AES 128
x0a DES-CBC3-SHA RSA 3DES 168
So using this output we can see that if your client only supported SSLv3, the handshake would fail because that protocol isn't supported by the server. The protocol offering is unlikely the problem but you can double check what your java client supports by getting the list of enabled protocols. You can provide an overridden implementation of the SSLConnectionSocketFactory from above code snippet to get the list of enabled/supported protocols and cipher suites as follows (SSLSocket):
class MySSLConnectionSocketFactory extends SSLConnectionSocketFactory {
#Override
protected void prepareSocket(SSLSocket socket) throws IOException {
System.out.println("Supported Ciphers" + Arrays.toString(socket.getSupportedCipherSuites()));
System.out.println("Supported Protocols" + Arrays.toString(socket.getSupportedProtocols()));
System.out.println("Enabled Ciphers" + Arrays.toString(socket.getEnabledCipherSuites()));
System.out.println("Enabled Protocols" + Arrays.toString(socket.getEnabledProtocols()));
}
}
I often encounter handshake_failure when there is a cipher suite negotiation failure. To avoid this error, your client's list of supported cipher suites must contain at least one match to a cipher suite from the server's list of supported cipher suites.
If the server requires AES256 based cipher suites you probably need the java cryptographic extensions (JCE). These libraries are nation restricted so they may not be available to someone outside the US.
More on cryptography restrictions, if you're interested: https://crypto.stackexchange.com/questions/20524/why-there-are-limitations-on-using-encryption-with-keys-beyond-certain-length

I think that the post you are referring is very close to what it needs to be done. Have you tried something like:
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
SSLContextBuilder sslContextBuilder = SSLContextBuilder.create();
sslContextBuilder.setSecureRandom(new java.security.SecureRandom());
try {
sslContextBuilder.loadTrustMaterial(new TrustStrategy() {
#Override
public boolean isTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {
return true;
}
});
clientBuilder.setSSLContext(sslContextBuilder.build());
} catch (Throwable t) {
Logger.getLogger(getClass().getName()).log(Level.SEVERE, "Can't set ssl context", t);
}
CloseableHttpClient apacheHttpClient = clientBuilder.build();
I have not tried this code but hopefully it could work.
Cheers

If you are fine to use other open source libraries like netty then worth trying below:
SslProvider provider = SslProvider.JDK; // If you are not concerned about http2 / http1.1 then JDK provider will be enough
SSLContext sslCtx = SslContextBuilder.forClient()
.sslProvider(provider)
.trustManager(InsecureTrustManagerFactory.INSTANCE) // This will trust all certs
... // Any other required parameters used for ssl context.e.g. protocols , ciphers etc.
.build();
I have used below version of netty for trusting any certificates with above code:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.29.Final</version>
</dependency>

I think #nmorenor answer is pretty close to the mark. What I would have done in addition is explicitly enabling SSLv3 (HttpClient automatically disables it by default due to security concerns) and disabling host name verification.
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial((chain, authType) -> true)
.build();
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext,
new String[]{"SSLv3", "TLSv1", "TLSv1.1", "TLSv1.2"},
null,
NoopHostnameVerifier.INSTANCE))
.build();

You can do it with core jdk too, but iirc, httpclient also allows you to set the SSL Socket Factory too.
The factory defines and uses a ssl context that you construst with a trust manager. That manager would simply not verify the cert chain, as shown in above post.
You also need a hostnameverifier instance that would also choose to ignore the potential mismatch of cert hostname with the url's host (or ip). Otherwise, it would still fail even if the cert signer is blindly trusted.
I used to convert many client stack to 'accept self-signed' and it's quite easy in most stack. The worse cases is when the 3rd party lib doesn't allow choosing a ssl socket factory instance but only its clasname. In that case, I use a ThreadLocalSSLSocketFactory which doesn't own any actual factory but simply looks up the threadlocal to find one that the upper stackframes (that you can control) would have prepared. This only works if the 3rd party lib is not doing the work on distinct thread of course. I know http client can be told to use a specific ssl socket factory so this is easy.
Also take the time to read the JSSE doc, it is totally worth the time it takes to read.

Java8 Secure WebSocket Handshake Problems (Tyrus and Jetty)

I'm attempting to implement a WebSocket Client in an application that supports secure transmissions through SSL. The application already supports standard SSL connections over HTTP, by implementing custom Key and Trust managers (these custom implementations are in place to prompt the user for a certificate when needed).
I'm having trouble getting a secure connection to our remote WebSocket endpoint. The failure is occurring during the handshake. I've tried two different implementations of the WebSocket API (both Tyrus and Jetty), and both fail in the same way, which, of course, leads me to point to our SSL implementation.
As I mentioned, the failure is occurring during the handshake. It seems that the connection cannot figure out that there are client certificates that are signed by the supported authorities returned from the server. I'm stumped to figure out if I haven't supplied the client certificates to the WebSocket API correctly, or if our custom Key/Trust managers are even getting used.
Here's a dump of the SSL Debug logs:
*** CertificateRequest
Cert Types: RSA, DSS
Cert Authorities:
(list of about 15 cert authorities supported by the server)
*** ServerHelloDone
Warning: no suitable certificate found - continuing without client authentication
*** CertificateChain
<empty>
***
I've set breakpoints in our TrustManager implementation, to determine if they are ever getting called, and it seems that they are not being called at this point.
I've been attempting to debug this for a few days now, and am running out of things to try.
Any suggestions would be greatly appreciated!
Here's a snippet of the Jetty Code:
SSLContext context = SSLContext.getInstance("TLS");
// getKeyManagers / getTrustManagers retrieves an
// array containing the custom key and trust manager
// instances:
KeyManager[] km = getKeyManagers();
TrustManager[] tm = getTrustManagers();
context.init(km, tm, null);
SslContextFactory contextFactory = new SslContextFactory();
contextFactory.setContext(context);
WebSocketClient client = new WebSocketClient(contextFactory);
SimpleEchoClient echoClient = new SimpleEchoClient();
try {
client.start();
ClientUpgradeRequest request = new ClientUpgradeRequest();
Future<Session> connection = client.connect(echoClient, uri, request);
Session session = connection.get();
// if everything works, do stuff here
session.close();
client.destroy();
} catch (Exception e) {
LOG.error(e);
}

can you try with rejectUnAuthorized:false so that your certificates for which your browser is unable to authorize will skip the authorization.
var ws = new WebSocket('wss://localhost:xxxx', {
protocolVersion: 8,
origin: 'https://localhost:xxxx',
rejectUnauthorized: false
});

JAVA 6 does not support SNI, Any Alternative way to verify and accept SSL Certificate over TLS?

I know the reason why browsers and java7 are not affected is because they send Server Name Indication-SNI as part of the SSL information. So, apache knows what virtual host to use before starting SSL handshake and returns the proper certificate. Java 6 doesn't support SNI. So My question is, how do I verify and allow certificate to be accepted in java 6.
I made a spring client to consume webservice, Here is my code
public class classname1 {
static {
System.setProperty("javax.net.ssl.trustStore", "E://Workspace//keystore");
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
System.setProperty("https.protocols", "SSLv3");
System.setProperty("https.protocols", "TLSv1");
}
static {
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier()
{
public boolean verify(String hostname, SSLSession session)
{
if (hostname.equals("192.168.10.22"))
return true;
return false;
}
});
}
private static void main(String[] args) {
try {
SOAPConnectionFactory soapConnectionFactory = SOAPConnectionFactory.newInstance();
SOAPConnection soapConnection = soapConnectionFactory.createConnection();
String url = "https://192.168.10.22/getInformationSearch.asmx?wsdl";
SOAPMessage soapResponse = soapConnection.call(createSOAPRequest(), url);
printSOAPResponse(soapResponse);
soapConnection.close();
} catch (Exception e) {
e.printStackTrace();
}
}
private static SOAPMessage createSOAPRequest() throws Exception {
// ... Code for request, which will be hitted to url
}
private static void printSOAPResponse(SOAPMessage soapResponse) throws Exception {
// ... Code for response, which will fetch information from webservice
// URL
}
}
As you can see code, I made two methods, 1. for Request createSOAPRequest() 2. for Response printSOAPResponse(). (Url name is changed in above snippet)
In main() method, below line will generate request and send that request to given url SOAPMessage soapResponse = soapConnection.call(createSOAPRequest(), url); after that it goes to static block as shown above HttpsURLConnection.setDefaultHostnameVerifier() method.
At that time,debugger says: ssl handshake failed and SSL peer shut down incorrectly. It happens only in JAVA 6 but these code working proper with java 7/8.
Instead of that static block, I have tried below code in java 6
HostnameVerifier hv = new HostnameVerifier() {
public boolean verify(String urlHostName,
SSLSession session) {
return true;
}
};
But it won't work anyhow!!!
I am using following jars xercesImpl.jar,saaj-api.jar,saaj-impl.jar,spring-ws-core-1.5.6.jar,spring-ws-security-1.5.6.jar and Certificate for this SSL domain has been imported into keystore and It works in java7/8, So there is no issue in truststore right? (I made 2 certificate using keytool of java 6 and 7 as well, both works fine in java7/8 but not in 6)
I followed this thread but it wont work. So is there any alternative way to verify certificate and get response in java 6 environment, or should I change any JAR ?

You can't if the server isn't configured to allow you to connect without SNI.
Without SNI, the server cannot tell which virtual host you want until after the SSL connection has been established.
To see the difference, if you have OpenSSL, you can use the s_client option to extract the different certificates presented, or perhaps note a failure to connect when you don't use SNI:
SNI:
echo | openssl s_client -servername www.virtualhost.com -connect 192.168.1.55:443 | openssl x509 -text
Replace 192.168.1.55 with the actual IP address of the server, and www.virtualhost.com with the host name of the virtual host you want to connect to. The echo command prepended to the pipeline prevents openssl from hanging in a wait for input.
Without SNI:
echo | openssl s_client -connect 192.168.1.55:443 | openssl x509 -text
Without SNI, you may see a completely different certificate, or you may get a connection error.

As you already mentioned Java 6 does not support SNI. But, the problem of missing SNI is not only with verifying the certificate (because the server sends the wrong certificate) but also that depending on the servers configuration the server will cause a handshake error if the SNI extension is not present and does not point to a hostname available at the server. And this is obviously the case here:
... ssl handshake failed and SSL peer shut down incorrectly. It happens only in JAVA 6 but these code working proper with java 7/8.
Trying to change the validation of the certificate as you've tried will not help at all in this case because the error occurs before the client even has a certificate to verify. This means that there is no way around it: you need to support SNI in your client if you want to communicate with this specific server.

C client with OpenSSL + Java server : javax.net.ssl.SSLHandshakeException: no cipher suites in common

I have Java SSL socket and c client with OpenSSL (java clients works ok with this Java server). Handshake fails and i'm getting Java exception:
javax.net.ssl.SSLHandshakeException: no cipher suites in common
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:269)
at sun.security.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:901)
at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:629)
at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:167)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:837)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:889)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at EchoServer.main(EchoServer.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Here is how server SSL socket is created:
public class EchoServer {
public static void main(String[] arstring) {
try {
final KeyStore keyStore = KeyStore.getInstance("JKS");
final InputStream is = new FileInputStream("/Path/mySrvKeystore.jks");
keyStore.load(is, "123456".toCharArray());
final KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory .getDefaultAlgorithm());
kmf.init(keyStore, "123456".toCharArray());
final TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory .getDefaultAlgorithm());
tmf.init(keyStore);
SSLContext sc = SSLContext.getInstance("TLSv1.2");
sc.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new java.security.SecureRandom());
SSLServerSocketFactory sslserversocketfactory = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
SSLServerSocket sslserversocket = (SSLServerSocket) sslserversocketfactory.createServerSocket(9997);
SSLSocket sslsocket = (SSLSocket) sslserversocket.accept();
sslsocket.setEnabledCipherSuites(sc.getServerSocketFactory().getSupportedCipherSuites());
InputStream inputstream = sslsocket.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String string = null;
while ((string = bufferedreader.readLine()) != null) {
System.out.println(string);
System.out.flush();
}
} catch (Exception exception) {
exception.printStackTrace();
}
}
}
C client:
BIO *certbio = NULL;
BIO *outbio = NULL;
SSL_METHOD *ssl_method;
SSL_CTX *ssl_ctx;
SSL *ssl;
int sd;
// These function calls initialize openssl for correct work.
OpenSSL_add_all_algorithms();
ERR_load_BIO_strings();
ERR_load_crypto_strings();
SSL_load_error_strings();
// Create the Input/Output BIO's.
certbio = BIO_new(BIO_s_file());
outbio = BIO_new_fp(stdout, BIO_NOCLOSE);
// initialize SSL library and register algorithms
if(SSL_library_init() < 0)
BIO_printf(outbio, "Could not initialize the OpenSSL library !\n");
ssl_method = (SSL_METHOD*)TLSv1_2_method();
// Try to create a new SSL context
if ( (ssl_ctx = SSL_CTX_new(ssl_method)) == NULL)
BIO_printf(outbio, "Unable to create a new SSL context structure.\n");
// flags
SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION | SSL_OP_CIPHER_SERVER_PREFERENCE);
// Create new SSL connection state object
ssl = SSL_new(ssl_ctx);
// Make the underlying TCP socket connection
struct sockaddr_in address;
memset(&address, 0, sizeof(address));
address.sin_family = AF_INET;
address.sin_port = htons(port);
const char *dest_url = this->host.c_str();
address.sin_addr.s_addr = inet_addr(dest_url);
address.sin_port = htons(port);
sd = socket(AF_INET, SOCK_STREAM, 0);
int connect_result = ::connect(sd, (struct sockaddr*)&address, sizeof(address));
if (connect_result != 0) {
BIO_printf(outbio, "Failed to connect over TCP with error %i\n", connect_result);
throw IOException("Connection refused");
} else {
BIO_printf(outbio, "Successfully made the TCP connection to: %s:%i\n", dest_url, port);
}
// Attach the SSL session to the socket descriptor
SSL_set_fd(ssl, sd);
// Try to SSL-connect here, returns 1 for success
int ssl_connect_result = SSL_connect(ssl);
if (ssl_connect_result != 1)
BIO_printf(outbio, "Error: Could not build a SSL session to: %s:%i with error %i\n", dest_url, port, ssl_connect_result);
else
BIO_printf(outbio, "Successfully enabled SSL/TLS session to: %s\n", dest_url);
Here is output on client side:
Error: Could not build a SSL session to: 127.0.0.1:9997 with error -1
Update 1
int ssl_connect_result = SSL_connect(ssl);
if (ssl_connect_result != 1) {
int error_code = SSL_get_error(ssl, ssl_connect_result); // =1
BIO_printf(outbio, "Error: Could not build a SSL session to: %s:%i with error %i (%i)\n", dest_url, port, ssl_connect_result, error_code);
} else {
BIO_printf(outbio, "Successfully enabled SSL/TLS session to: %s\n", dest_url);
}
And the output is:
Error: Could not build a SSL session to: 127.0.0.1:9997 with error -1 (1)
Update 2
I forgot to note that I'm using self-signed certificate, generated by keytool from JDK.
Update 3
I've noted i missed some lines and I've added:
OpenSSL_add_all_ciphers();
OpenSSL_add_all_digests();
but still no luck - getting the same error -1.
Update 4
Here is Java client which is accepted by the server code above:
SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket(ip, port);
sslsocket.setEnabledCipherSuites(sslsocketfactory.getSupportedCipherSuites());
InputStream inputstream = System.in;
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
OutputStream outputstream = sslsocket.getOutputStream();
OutputStreamWriter outputstreamwriter = new OutputStreamWriter(outputstream);
String string = null;
outputstreamwriter.write("hello");
outputstreamwriter.flush();
while ((string = bufferedreader.readLine()) != null) {
outputstreamwriter.write(string);
outputstreamwriter.flush();
}
sslsocket.close();
I've checked that I can't see plain data in packets intercepted in the network so it does perform some data encryption.

I don't believe that Java server accepts a Java client either, unless the Java client similarly does .setEnabledCipherSuites (all-supported) -- if so it is using an anonymous (unauthenticated) ciphersuite that is not secure against active attack, and although many people are still stuck in the passive-only threat model of about 1980, today active attacks are common. That is why JSSE's default cipherlist excludes anonymous ciphers -- unless you override it. And why OpenSSL's default cipherlist also excludes them -- which you didn't override.
(add) To explain in smaller words, anonymous ciphersuites ARE encrypted (with a few exceptions not relevant here because they are never preferred) but NOT authenticated. The word "unauthenticated" means "not authenticated"; it does not mean "not encrypted". The word "unencrypted" is used to mean "not encrypted". "Not encrypted" means something that just looks at the channel, like Wireshark, can see the plaintext. "Not authenticated" means an attacker who intercepts (possibly diverts) your traffic can cause you to establish your "secure" session with the attacker in the middle, and they can decrypt your data, copy and/or change it as they wish, re-encrypt it, and send it on, and you will think it is correct and private when it isn't. Google or search here (I think mostly security.SE and superuser) for things like "man in the middle attack", "ARP spoof", "MAC spoof", "DNS poisoning", "BGP attack", etc.
The immediate problem is you aren't using the keystore. You create an SSLContext with key and trust managers from it, but then you create the socket from SSLServerSocketFactory.getDefault() which doesn't use the context. Use sc.getServerSocketFactory() instead.
(add) why? Every SSLSocket (and SSLServerSocket) is linked to an SSLContext which among other things controls the privatekey(s) and certificate(s) or chain(s) used and certificates trusted. (SSL/TLS connections normally authenticate the server only, so in practice the server only needs a key-and-chain, and the client only needs the root cert, but Java uses the same keystore file format for both and it's easy to just code both.) Since your code has set the particular SSLContext sc to contain a suitable key-and-cert, sc.getServerSocketFactory() creates a factory which creates an SSLServerSocket which in turn creates an SSLSocket (for each connection if more than one) which uses that key-and-(as long as the client's supported cipher list allows it, and here it does).
(add) SSLServerSocketFactory.getDefault() creates a factory, and thus sockets, using the default SSL context, which by default contains NO key-and-chain, although you can change this with system properties as described in the documentation cleverly hidden at http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html . As a result it cannot negotiate a ciphersuite that is authenticated. Since both Java and OpenSSL by default disable unauthenticated ciphersuites, this is why you get "no cipher suites in common" unless you .setEnabled to include the unauthenticated and insecure ciphersuites in Java, and still get it for OpenSSL client since you didn't do anything to enable unauthenticated and insecure ciphersuites there.
(add) If you look carefully at your Wireshark trace, you will see in the ServerHello that the selected ciphersuite uses DH_anon or ECDH_anon key exchange -- "anon" is an abbreviation for "anonymous" which means "not authenticated" as explained above -- and there is no Certificate message from the server, and (less obvious unless you know it) the ServerKeyExchange data is not signed. Also I predict if you have your Java client check sslsocket.getSession().getCipherSuite() and/or sslsocket.getSession().getPeerCertificates() after the handshake is done, which since you don't do it explicitly will be on the first socket-level I/O which will be the outputstreamwriter.flush(), you will see the anonymous ciphersuite, and no peer cert (it throws PeerNotAuthenticated).
Other points:
(1) In general whenever you get SSL_ERROR from SSL_get_error(), or any error return from lower level routines like EVP_* and BIO_*, you should use the ERR_* routines to get details of the error and log/display them; see https://www.openssl.org/docs/faq.html#PROG6 and https://www.openssl.org/docs/manmaster/crypto/ERR_print_errors.html et amici. Especially since you HAVE loaded the error strings, thus avoiding https://www.openssl.org/docs/faq.html#PROG7 . In this case however you already know enough from the server side, so client side details aren't needed.
(2) You don't need _add_all_ciphers and _add_all_digests, they are included in _add_all_algorithms.
(3) OP_NO_SSLv2 and 3 have no effect on TLSv1_2_method and OP_SERVER_CIPHER_PREFERENCE has no effect on a client. (They do no harm, they are just useless and possibly confusing.)
(4) Once you get past the cipher negotiation, the OpenSSL client will need the root cert for the server; since you intend to use a self-signed cert (once you fix the server to use the keystore at all) that cert is its own root. In 1.0.2 (not earlier) you could also use a non-root trust anchor, but not by default and you don't have one anyway. I assume certbio was intended for this, but you never open it on an actual file or do anything else with it, and anyway the SSL library cannot use a BIO for its truststore. You have three choices:
put the cert(s) in a file, or a directory using special hash names, and pass the file and/or directory name(s) to SSL_CTX_load_verify_locations. If you only want one root (your own) using the CAfile option is easier.
put or add the cert(s) to the default file or hashed directory determined by your OpenSSL library compilation and call SSL_CTX_set_default_verify_paths; this is commonly something like /etc/pki or /var/ssl. If you want to use the same cert(s) for multiple programs or for commandline openssl this shared location is usually easier.
use the BIO and/or other means to (open and) read the cert(s) into memory, build your own X509_STORE containing them, and put that in your SSL_CTX. This is more complicated, so I won't expand on it unless you want to.
(5) Your dest_url is (at least in this case?) an address string, not a URL; those are different though related things and thinking they are the same will cause you lots more problems. For most programs it is better to handle a host name string with classic gethostbyname and fall back to inet_addr, or better the "new" (since 1990s) getaddrinfo which can handle both name and address strings and both IPv4 and v6 (also new since 1990s but finally gaining traction). At the very least you should check for inet_addr returning INADDR_NONE indicating it failed.

SSLContext sc = SSLContext.getInstance("TLSv1.2");
Java used to do two thing with code similar to this:
enable SSLv3
disable TLS 1.1 and 1.2
... Even though you called out TLS. Effectively, all you could get was a SSLContext with SSLv3 and TLS 1.0 (unless you were willing to do more work). It may not be the case anymore, but it would explain the error you are seeing (especially if your are using Java 7 or 8).
You need to do more work in Java to get "TLS 1.0 or above" and "just TLS 1.2". For that, see Which Cipher Suites to enable for SSL Socket?. It shows you how to enable/disable both protocols and cipher suites.
SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION | SSL_OP_CIPHER_SERVER_PREFERENCE);
You should also set a cipher suite list since OpenSSL includes broken/weak/wounded ciphers by default. Something like:
const char PREFERRED_CIPHERS[] = "HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4";
long res = SSL_CTX_set_cipher_list(ssl_ctx, PREFERRED_CIPHERS);
ASSERT(res == 1);
I think your final step is to ensure the self-signed certificate is trusted.
For a Java client, see How to properly import a selfsigned certificate into Java keystore that is available to all Java applications by default?. For an OpenSSL client, just load it up using SSL_CTX_load_verify_locations, see SSL_CTX_load_verify_locations Fails with SSL_ERROR_NONE.
Note well: OpenSSL prior to 1.1.0 did not perform hostname validation. You will have to do that yourself. If you need to perform hostname validation, then lift the code from cURL's url.c. I seem to recall looking at Daniel's code, and its very solid. You can also lift the code from Google's CT project. I never audited it, so I don't know what its like.
There are two other checks you have to make; they are discussed at SSL/TLS Client on the OpenSSL wiki.

How to create own trust manager in java for apache httpclient library to ignore certificate in apache httpclient 4.2?

Can somebody tell me how can i ignore the ssl certificate during web service call.
I am calling https weburl to get api response but getting peer not authenticated error.
Old examples are not working as some of methods are deprecated so can somebody tell me/ provide some sample code so that i will not get this error.
I just came to know that the problem is coming due to Certificate.
I am using 3rd party API for db calls & they have ssl certificate for their domain
i.e. www.dbprovider.com (SSL certificate is *.dbprovider.com)
& they created subdomain for us which look like myapp.dbprovider.com
So now the problem is no peer certificate is available when i try to hit through command
openssl s_client -ssl3 -showcerts -connect myapp.dbprovider.com:443
openssl s_client -tls1 -showcerts -connect myapp.dbprovider.com:443
Can somebody tell me what i should now do with it. Is there any control on dbprovider site so that they can provide me some configuration or i have to write code to ignore their certificate (but for ignoring certificate we are not getting their peer certificate)

Use a custom SSLSocketFactory as described here: http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e512. One such factory that ignores self-signed certs is EasySSLProtocolSocketFactory.
ProtocolSocketFactory factory = new EasySSLProtocolSocketFactory();
try {
URI uri = new URI(config.getBaseUrl());
int port = uri.getPort();
if (port == -1) {
port = 443;
}
Protocol easyHttps = new Protocol(uri.getScheme(), factory, port);
hostConfiguration.setHost(uri.getHost(), port, easyHttps);
} catch (URISyntaxException e) {
throw new IOException("could not parse URI " + config.getBaseUrl(), e);
}
Source: http://frightanic.com/software-development/self-signed-certificates-in-apache-httpclient/