We Keep Coding

How to connect 'spring boot application (2.3.4.RELEASE) with elasticsearch 7.9.2' using HTTPS

I'm new to spring-boot & Elasticsearch technology stack and I want to establish secure HTTPS connection
between my spring-boot app & elastic search server which runs locally.
These are the configurations that I have done in elasticsearch.yml
Giving credintials for elasticsearch server
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
For secure inter nodes connection inside elasticsearch cluster
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
For secure Https connection with clients and elasticsearch clustrer
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12
xpack.security.http.ssl.truststore.path: elastic-certificates.p12
xpack.security.http.ssl.client_authentication: optional
Enabling PKI authentication
xpack.security.authc.realms.pki.pki1.order: 1
I have generated CA and client certificate which signed by generated CA according to this link
https://www.elastic.co/blog/elasticsearch-security-configure-tls-ssl-pki-authentication
And I have added CA to my java keystore.
This is the java code i'm using to establish connectivity with elasticsearch server.
#Configuration
public class RestClientConfig extends AbstractElasticsearchConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(RestClientConfig.class);
private static final String CERT_FILE = "client.p12";
private static final String CERT_PASSWORD = "";
private static final String USER_NAME = "elastic";
private static final String USER_PASS = "pwd";
#Override
#Bean
public RestHighLevelClient elasticsearchClient() {
final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
.connectedTo("localhost:9200") // set the address of the Elasticsearch cluster
.usingSsl(createSSLContext()) // use the SSLContext with the client cert
.withBasicAuth(USER_NAME, USER_PASS) // use the headers for authentication
.build();
return RestClients.create(clientConfiguration).rest();
}
private SSLContext createSSLContext() {
try {
SSLContext sslContext = SSLContext.getInstance("TLS");
KeyManager[] keyManagers = getKeyManagers();
sslContext.init(keyManagers, null, null);
return sslContext;
} catch (Exception e) {
LOG.error("cannot create SSLContext", e);
}
return null;
}
private KeyManager[] getKeyManagers()
throws KeyStoreException, NoSuchAlgorithmException, IOException, CertificateException, UnrecoverableKeyException {
try (InputStream inputStream = getClass().getClassLoader().getResourceAsStream(CERT_FILE)) {
KeyStore clientKeyStore = KeyStore.getInstance("PKCS12");
clientKeyStore.load(inputStream, CERT_PASSWORD.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(clientKeyStore, CERT_PASSWORD.toCharArray());
return kmf.getKeyManagers();
}
}
And my client certificate called "client.p12" included in resources folder in the spring-boot application. In elasticsearch side everything seems to be fine.
But when I run the spring-boot app it gives this warning & exception
Cannot create index: Host name 'localhost' does not match the certificate subject provided by the peer (CN=instance); nested exception is java.lang.RuntimeException:
Host name 'localhost' does not match the certificate subject provided by the peer (CN=instance)
java.io.IOException: Host name 'localhost' does not match the certificate subject provided by the peer (CN=instance)
Caused by: javax.net.ssl.SSLPeerUnverifiedException: Host name 'localhost' does not match the certificate subject provided by the peer (CN=instance)
I'm doing this because I have an idea to migrate elastic search server to a another VPS hosting later.
this is the command I used for generate client certificate
bin/elasticsearch-certutil cert --ca
config/elastic-stack-ca.p12
-name "CN=something,OU=Consulting Team,DC=mydomain,DC=com"
ENTER
client.p12 ENTER
ENTER
searching this error in internet I figured out there is a issue with my client certificate. It has something to do with SAN names with elastic nodes. But I have very little knowledge
about these certificate stuff. So it's really confusion for me. If Some one can give me in-detailed explanation why this is occurring & solution to this I'm really appreciating it & grateful. A Proper link will also be helpful. This question
already asked in
How to connect 'spring boot 2.1 with elasticsearch 6.6' with cluster node 'https'?
link
But no luck for me.

Your issue looks similar to another issue, see here: Certificate for <localhost> doesn't match any of the subject alternative names
So I would assume that if you add the SAN extension localhost as DNS and the ip address of localhost to the elasticsearch certificate it should work. So adding the following additional parameters: --dns localhost --ip 127.0. 0.1. Can you give the command below a try and share your results here?
bin/elasticsearch-certutil cert --ca config/elastic-stack-ca.p12 -name "CN=something,OU=Consulting Team,DC=mydomain,DC=com" --dns localhost --ip 127.0. 0.1
By the way you have configured Elasticsearch to optionally request the client certificate, but not marked as required see: xpack.security.http.ssl.client_authentication: optional. So it is not required and therefor it does not make sense to create a KeyManager and construct a sslcontext from it. But I think you need to export the public certificate of the certificate of Elasticsearch and supply that to your http client by transforming it to a TrustManager and creating a sslcontext from it. I have a working example here: ElasticSearch RestHighLevelClient SSL example.

Java8 Secure WebSocket Handshake Problems (Tyrus and Jetty)

I'm attempting to implement a WebSocket Client in an application that supports secure transmissions through SSL. The application already supports standard SSL connections over HTTP, by implementing custom Key and Trust managers (these custom implementations are in place to prompt the user for a certificate when needed).
I'm having trouble getting a secure connection to our remote WebSocket endpoint. The failure is occurring during the handshake. I've tried two different implementations of the WebSocket API (both Tyrus and Jetty), and both fail in the same way, which, of course, leads me to point to our SSL implementation.
As I mentioned, the failure is occurring during the handshake. It seems that the connection cannot figure out that there are client certificates that are signed by the supported authorities returned from the server. I'm stumped to figure out if I haven't supplied the client certificates to the WebSocket API correctly, or if our custom Key/Trust managers are even getting used.
Here's a dump of the SSL Debug logs:
*** CertificateRequest
Cert Types: RSA, DSS
Cert Authorities:
(list of about 15 cert authorities supported by the server)
*** ServerHelloDone
Warning: no suitable certificate found - continuing without client authentication
*** CertificateChain
<empty>
***
I've set breakpoints in our TrustManager implementation, to determine if they are ever getting called, and it seems that they are not being called at this point.
I've been attempting to debug this for a few days now, and am running out of things to try.
Any suggestions would be greatly appreciated!
Here's a snippet of the Jetty Code:
SSLContext context = SSLContext.getInstance("TLS");
// getKeyManagers / getTrustManagers retrieves an
// array containing the custom key and trust manager
// instances:
KeyManager[] km = getKeyManagers();
TrustManager[] tm = getTrustManagers();
context.init(km, tm, null);
SslContextFactory contextFactory = new SslContextFactory();
contextFactory.setContext(context);
WebSocketClient client = new WebSocketClient(contextFactory);
SimpleEchoClient echoClient = new SimpleEchoClient();
try {
client.start();
ClientUpgradeRequest request = new ClientUpgradeRequest();
Future<Session> connection = client.connect(echoClient, uri, request);
Session session = connection.get();
// if everything works, do stuff here
session.close();
client.destroy();
} catch (Exception e) {
LOG.error(e);
}

can you try with rejectUnAuthorized:false so that your certificates for which your browser is unable to authorize will skip the authorization.
var ws = new WebSocket('wss://localhost:xxxx', {
protocolVersion: 8,
origin: 'https://localhost:xxxx',
rejectUnauthorized: false
});

SunMSCAPI not recognizing private keys for some keystore entries

I apologize that this is so long. If you are familiar with doing client-auth in Java you can probably skim/skip to the bottom. I took me a long time to find all the relevant bits of information from disparate sources. Maybe this will help someone else.
I'm working on a proof-of-concept to get client-side authentication working from a Java client using the Windows keystore. I have a servlet that I have created that can request or require client certificates. It simply returns an HTML response containing the certificate information including subject DN and the serial number. I've worked through a number of experiments with browsers (mainly Chrome and IE) to verify it is working. I've gotten successful client authentication working with both certs I've generated with SSL and also using certs issued by my companies internal CA.
My next step is to have a Java client that works with the MSCAPI keystore in Java. The reason I went this route is that the certificates we want to use are issued by our internal CA and are automatically added to the Personal keystore in Windows and are marked as non-exportable. I know there are ways to get the private key out of the keystore with some free tools if you are an admin on your workstation. This isn't a viable option in this case for various reasons. Here's the code I ended up with:
private static SSLSocketFactory getFactory() throws NoSuchAlgorithmException, KeyStoreException, CertificateException, IOException, UnrecoverableKeyException, KeyManagementException
{
KeyStore roots = KeyStore.getInstance("Windows-ROOT", new SunMSCAPI());
KeyStore personal = KeyStore.getInstance("Windows-MY", new SunMSCAPI());
roots.load(null,null);
personal.load(null,null);
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(roots);
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(personal, "".toCharArray());
SSLContext context = SSLContext.getInstance("TLS");
context.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());
return context.getSocketFactory();
}
This works but when I connect, the Java client is authenticating with the client-certs I had previously generated. I don't see an obvious way to force the selection of a specific key so I figured it was just picking the first key it came across that the host trusts. I then removed these certs that I had created from the windows keystore and it would no longer authenticate. I triple-checked that the server-side was still works with the browsers and it does. I then readded these certs to the personal keystore and removed the trust of my pseudo-CA from the server-side. Still works in the browser and not in the client.
I added -Djavax.net.debug=ssl:handshake to my VM parameters and started looking through the output. One thing I see is are lines "found key for : [alias]" for each of the certs that I have added but not for the ones added through the 'self-enrollment' feature of certmgr.msc. Initially I thought it was because they were exportable but I removed them and added one back as non-exportable and java can still use it. I'm at a loss as to why SunMSCAPI won't use these certs. Both the ones created by me and internal CA are sha256RSA. Both are enabled for client authentication. When I add the following code, I see that isKeyEntry() is false for all the key-pairs I want to use:
Enumeration<String> aliases = personal.aliases();
while (aliases.hasMoreElements())
{
String alias = aliases.nextElement();
System.out.println(alias + " " + personal.isKeyEntry(alias));
}
I found someone else with a similar issue here but no definitive answer.

Is it possible to add an additional subdomain to a existing SSL certificate for HttpClient 4.4?

We are using HttpClient 4.4 to communicate with some external servers (server1.company.com, server2.company.com, server3.company.com). They recently added an additional environment (server4.company.com) which uses the same certificate as the other 3. The certificate in question has the first 3 servers listed as "Certificate Subject Alternative Names", but server4 is not mentioned.
Is it possible for me to tell keytool that the certificate is valid for additional SANs? Or is there any other way to tell HttpClient to 'trust' this one certificate for some extra domains? Are there any other options or must I go back to company.com and ask them to get a new certificate?

There are two aspects of certificate verifications (in general):
Verifying that the certificate is genuine and issued by someone you trust (that's the PKI aspect).
Verifying that it belongs to the host name you want to connect to (that's the host name verification).
(Perhaps this question, about libcurl might be of interest if you need analogies.)
According to what you are saying, that particular certificate is trusted and valid for other host names. Hence, it will pass the PKI verification (what the TrustStrategy implements).
What you need is to make build an exceptional case, only for that particular certificate, for the host name verification aspect.
I can't remember off the top of my head how it is used with Apache HTTP Client 4.4, but you should use your own verifier instead of the DefaultHostnameVerifier.
The methods to implement are verify(String hostYouAreAfter, SSLSession sessionYouActuallyGet) and verify(String hostYouAreAfter, X509Certificate certYouActuallyGet).
You can provide your own implementation along these lines:
public verify(String hostYouAreAfter, X509Certificate certYouActuallyGet) {
if (certYouActuallyGet.equals(referenceCertificate)) {
if ("server4.company.com".equalsIgnoreCase(hostYouAreAfter)) {
// All good, don't fail and throw an exception.
} else {
super.verify(hostYouAreAfter, certYouActuallyGet);
}
} else {
super.verify(hostYouAreAfter, certYouActuallyGet);
}
}
You can do the same with verify(String,SSLSession) and get the X509Certificate from the SSLSession's peer chain (position 0). The logic is the same, but you need to return true/false instead of throwing exceptions.
Here, I'm assuming that you've loaded referenceCertificate from a place of reference where have that particular certificate. You could for example load it from a known keystore, or load it with a CertificateFactory from a reference PEM file configured in your application.
There are two key differences with a TrustStrategy where you'd implement isTrusted(final X509Certificate[] chain, final String authType) as return "nice guy".equalsIgnoreCase(issuerDN.getName());:
You're actually making this exceptional case only for that very certificate, not for any other certificate that would also happen to be issue with the name you're after.
It only affects the connections where you expect to connect to that particular host (not other hosts). You indeed have access to the first String parameter of HostnameVerifier.verify(...), which is the host name you intend to connect to. At least you have it to use for comparison with the certificate you get, which is something you don't get with a TrustStrategy.

One can trust certain select certificates by using a custom TrustStrategy
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new TrustStrategy() {
#Override
public boolean isTrusted(final X509Certificate[] chain, final String authType)
throws CertificateException {
X509Certificate x509Certificate = chain[0];
Principal issuerDN = x509Certificate.getIssuerDN();
return "nice guy".equalsIgnoreCase(issuerDN.getName());
}
}).build();
CloseableHttpClient client = HttpClients.custom()
.setSslcontext(sslcontext)
.build();

Java SSL connect, add server cert to keystore programmatically

I am connecting an SSL client to my SSL server.
When the client fails to verify a certificate due to the root not existing in the client's key store, I need the option to add that certificate to the local key store in code and continue.
There are examples for always accepting all certificates, but I want the user to verify the cert and add it to local key store without leaving the application.
SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket("localhost", 23467);
try{
sslsocket.startHandshake();
} catch (IOException e) {
//here I want to get the peer's certificate, conditionally add to local key store, then reauthenticate successfully
}
There is a whole lot of stuff about custom SocketFactory, TrustManager, SSLContext, etc and I don't really understand how they all fit together or which would be the shortest path to my goal.

You could implement this using a X509TrustManager.
Obtain an SSLContext with
SSLContext ctx = SSLContext.getInstance("TLS");
Then initialize it with your custom X509TrustManager by using SSLContext#init. The SecureRandom and the KeyManager[] may be null. The latter is only useful if you perform client authentication, if in your scenario only the server needs to authenticate you don't need to set it.
From this SSLContext, get your SSLSocketFactory using SSLContext#getSocketFactory and proceed as planned.
As concerns your X509TrustManager implementation, it could look like this:
TrustManager tm = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] chain,
String authType)
throws CertificateException {
//do nothing, you're the client
}
public X509Certificate[] getAcceptedIssuers() {
//also only relevant for servers
}
public void checkServerTrusted(X509Certificate[] chain,
String authType)
throws CertificateException {
/* chain[chain.length -1] is the candidate for the
* root certificate.
* Look it up to see whether it's in your list.
* If not, ask the user for permission to add it.
* If not granted, reject.
* Validate the chain using CertPathValidator and
* your list of trusted roots.
*/
}
};
Edit:
Ryan was right, I forgot to explain how to add the new root to the existing ones. Let's assume your current KeyStore of trusted roots was derived from cacerts (the 'Java default trust store' that comes with your JDK, located under jre/lib/security). I assume you loaded that key store (it's in JKS format) with KeyStore#load(InputStream, char[]).
KeyStore ks = KeyStore.getInstance("JKS");
FileInputStream in = new FileInputStream("<path to cacerts"");
ks.load(in, "changeit".toCharArray);
The default password to cacerts is "changeit" if you haven't, well, changed it.
Then you may add addtional trusted roots using KeyStore#setEntry. You can omit the ProtectionParameter (i.e. null), the KeyStore.Entry would be a TrustedCertificateEntry that takes the new root as parameter to its constructor.
KeyStore.Entry newEntry = new KeyStore.TrustedCertificateEntry(newRoot);
ks.setEntry("someAlias", newEntry, null);
If you'd like to persist the altered trust store at some point, you may achieve this with KeyStore#store(OutputStream, char[].

In the JSSE API (the part that takes care of SSL/TLS), checking whether a certificate is trusted doesn't necessarily involve a KeyStore. This will be the case in the vast majority of cases, but assuming there will always be one is incorrect. This is done via a TrustManager.
This TrustManager is likely to use the default trust store KeyStore or the one specified via the javax.net.ssl.trustStore system property, but that's not necessarily the case, and this file isn't necessarily $JAVA_HOME/jre/lib/security/cacerts (all this depends on the JRE security settings).
In the general case, you can't get hold of the KeyStore that's used by the trust manager you're using from the application, more so if the default trust store is used without any system property settings.
Even if you were able to find out what that KeyStore is, you would still be facing two possible problems (at least):
You might not be able to write to that file (which is not necessarily a problem if you're not saving the changes permanently).
This KeyStore might not even be file-based (e.g. it could be the Keychain on OSX) and you might not have write access to it either.
What I would suggest is to write a wrapper around the default TrustManager (more specifically, X509TrustManager) which performs the checks against the default trust manager and, if this initial check fails, performs a callback to the user-interface to check whether to add it to a "local" trust store.
This can be done as shown in this example (with a brief unit test), if you want to use something like jSSLutils.
Preparing your SSLContext would be done like this:
KeyStore keyStore = // ... Create and/or load a keystore from a file
// if you want it to persist, null otherwise.
// In ServerCallbackWrappingTrustManager.CheckServerTrustedCallback
CheckServerTrustedCallback callback = new CheckServerTrustedCallback {
public boolean checkServerTrusted(X509Certificate[] chain,
String authType) {
return true; // only if the user wants to accept it.
}
}
// Without arguments, uses the default key managers and trust managers.
PKIXSSLContextFactory sslContextFactory = new PKIXSSLContextFactory();
sslContextFactory
.setTrustManagerWrapper(new ServerCallbackWrappingTrustManager.Wrapper(
callback, keyStore));
SSLContext sslContext = sslContextFactory.buildSSLContext();
SSLSocketFactory sslSocketFactory = sslContext.getSslSocketFactory();
// ...
// Use keyStore.store(...) if you want to save the resulting keystore for later use.
(Of course, you don't need to use this library and its SSLContextFactory, but implement your own X509TrustManager, "wrapping" or not the default one, as you prefer.)
Another factor you have to take into account is the user interaction with this callback. By the time the user has made the decision to click on accept or reject (for example), the handshake may have timed out, so you may have to make a second attempt to connect when the user has accepted the certificate.
Another point to take into account in the design of the callback is that the trust manager doesn't know which socket is being used (unlike its X509KeyManager counterpart), so there should be as little ambiguity as to what user action caused that pop-up (or however you want to implement the callback). If multiple connections are made, you wouldn't want to validate the wrong one.
It seems possible to solve this by using a distinct callback, SSLContext and SSLSocketFactory per SSLSocket that's supposed to make a new connection, some way of tying up the SSLSocket and the callback to the action taken by the user to trigger that connection attempt in the first place.