We Keep Coding

SOAP client is not able to authenticate with mutual tls

I have an Apache CXF client that is connecting a SOAP service, and authenticating with mutual TLS. The client fails during the TLS Handshake because the service sends an empty list of client certificates to the server. I am testing this with self-signed certs, and I can prove that my server works with a curl request and with postman. I am pretty sure that the certificates are setup correctly, and I am sure that I am missing a config step in the CXF client.
Here is how I have my client setup
// setting up certs & keystores
String keystore = "client-keystore.jks";
String keystorePassword = "changeit"; // local self-signed certs
String trustStore = "truststore.jks";
String trustStorePassword = "changeit"; // local self-signed certs
// client keystore
KeyStore ks = KeyStore.getInstance("JKS");
ks.load(new FileInputStream(keystore), keystorePassword.toCharArray());
// ca truststore
KeyStore ts = KeyStore.getInstance("JKS");
ts.load(new FileInputStream(trustStore), trustStorePassword.toCharArray());
// key managers
var kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(ks, keystorePassword.toCharArray());
KeyManager[] kms = kmf.getKeyManagers();
// trust managers
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(ts);
TrustManager[] tms = tmf.getTrustManagers();
TLSClientParameters param = new TLSClientParameters();
param.setSecureSocketProtocol("TLSv1.2");
param.setDisableCNCheck(false);
param.setTrustManagers(tms);
param.setKeyManagers(kms);
// Get the client & setup the tls parameters
BindingProvider bp = (BindingProvider) port;
var client = ClientProxy.getClient(bp);
HTTPConduit https = (HTTPConduit)client.getConduit();
https.setTlsClientParameters(param);
Here is how I created the certificates. My java version is azul zulu openjdk 11.
# Create the CA Authority that both the client and server can trust
openssl req -new -x509 -nodes -days 365 -subj '/CN=my-ca' -keyout ca.key -out ca.crt
# Create the server's key, certificate signing request, and certificate
openssl genrsa -out server.key 2048
openssl req -new -key server.key -subj '/CN=localhost' -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 365 -out server.crt
# Create the client's key, certificate signing request, and certificate
openssl genrsa -out client.key 2048
openssl req -new -key client.key -subj '/CN=my-client' -out client.csr
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 365 -out client.crt
openssl x509 --in client.crt -text --noout
# Create the root truststore
keytool -import -alias my-ca -file ca.crt -keystore truststore.jks
# Create pkcs12 file for key and cert chain
openssl pkcs12 -export -name server-tls -in server.crt -inkey server.key -out server.p12
# Create JKS for server
keytool -importkeystore -destkeystore server-keystore.jks -srckeystore server.p12 -srcstoretype pkcs12 -alias server-tls
# Create pkcs12 file for key and cert chain
openssl pkcs12 -export -name client-tls -in client.crt -inkey client.key -out client.p12
# Create JKS for client
keytool -importkeystore -destkeystore client-keystore.jks -srckeystore client.p12 -srcstoretype pkcs12 -alias client-tls
I set debugging on with -Djavax.net.debug=ssl,handshake,data for both the server & the client.
When I use the CXF client to issue a request to the server, it initiates the mutual tls handshake, but the server fails with Fatal (BAD_CERTIFICATE): Empty server certificate chain, and the client fails with Fatal (HANDSHAKE_FAILURE): Couldn't kickstart handshaking...readHandshakeRecord, because it does indeed send an empty certificate list right before hand.
Produced client Certificate handshake message (
"Certificates": <empty list>
)
I have tried a number of different things, but I cannot seem to get the client to work.
Update
Out of curiosity, I ran the ws-security sample from the CXF repo, and used my ca certificate, client, and server certificates in the sample. That worked, and it is configured through an xml bean. I tried the same thing with my local, and it still fails.
The difference between the demo and my client is that when it looks for a x.509 RSA certificate, it fails for my client, but succeeds in the demo app. I have it configured mostly the same.
javax.net.ssl|ALL|01|main|2021-07-02 14:17:32.039 EDT|X509Authentication.java:213|No X.509 cert selected for EC
javax.net.ssl|WARNING|01|main|2021-07-02 14:17:32.040 EDT|CertificateMessage.java:1066|Unavailable authentication scheme: ecdsa_secp256r1_sha256
javax.net.ssl|ALL|01|main|2021-07-02 14:17:32.040 EDT|X509Authentication.java:213|No X.509 cert selected for EC
javax.net.ssl|WARNING|01|main|2021-07-02 14:17:32.040 EDT|CertificateMessage.java:1066|Unavailable authentication scheme: ecdsa_secp384r1_sha384
javax.net.ssl|ALL|01|main|2021-07-02 14:17:32.040 EDT|X509Authentication.java:213|No X.509 cert selected for EC
javax.net.ssl|WARNING|01|main|2021-07-02 14:17:32.040 EDT|CertificateMessage.java:1066|Unavailable authentication scheme: ecdsa_secp521r1_sha512
javax.net.ssl|ALL|01|main|2021-07-02 14:17:32.040 EDT|X509Authentication.java:213|No X.509 cert selected for RSA
javax.net.ssl|WARNING|01|main|2021-07-02 14:17:32.040 EDT|CertificateMessage.java:1066|Unavailable authentication scheme: rsa_pss_rsae_sha256
That last error is not present when using the demo app and instead, it returns back the certificate.

For anyone who stumbles upon this question, here's how I resolved it.
Once I started playing with the CXF demo code, I was able to simplify it to just its bare minimum set of dependencies and configurations. From there I was able to sort out that it was a matter of a missing dependency in my project.
For starters, we use dropwizard for the server, and we have a dependency on dropwizard-jaxws which brings in the cxf dependencies. I found by whittling away all of the layers, that the demo app only works if cxf-rt-transports-http-jetty is in the list of dependencies.
The transitive dependencies that dropwizard-jaxws include are:
cxf-rt-frontend-jaxws
cxf-rt-transports-http
I also had a dependency on all of dropwizard-core in my client which may have implemented some SPI interface that cxf-rt-transports-http-jetty implements (conjecture). Once I simplified the dependencies and included the one missing dependency, I have a repeatable, working solution.

Client server authentication with self signed certificate - Curl is working but Apache HttpClient throws exception

I'm trying to implement client and server authentication with Apache HTTPClient,
and to test it with a self signed certificate.
I've tried to follow several tutorials and answers to similar questions here but
with no success. I've tried to detail as much as possible all the steps that I've
been doing, hopefully someone can point out what I'm doing wrong:
Created a file req.conf for configuration
[req]
prompt=no
distinguished_name = req_distinguished_name
[ req_distinguished_name ]
O=selfSignedO
CN=selfSignedCn
DC=selfSignedDc
Generated server private key and the self-signed certificate
openssl req \
-config req.conf \
-x509 \
-newkey rsa:4096 \
-keyout server/server-private-key.pem \
-out server/server.crt \
-days 3650 \
-nodes
Created PKCS12 keystore containing the private key and certificate created in the previous step
openssl pkcs12 \
-export \
-out server/server-key-store.p12 \
-inkey server/server-private-key.pem \
-in server/server.crt
let's say the password I used was 123456
Generated a client private key and a certificate signing request
openssl req \
-config req.conf \
-new \
-newkey rsa:4096 \
-out client/client-request.csr \
-keyout client/client-private-key.pem \
-nodes
Signed the client's certificate signing request with the server's private key
and certificate
openssl x509 \
-req \
-days 360 \
-in client/client-request.csr \
-CA server/server.crt \
-CAkey server/server-private-key.pem \
-CAcreateserial \
-out client/client-signed-cert.crt \
-sha256
Created a PKCS12 keystore containing the client's private key and certificate certificate created in the previous step.
openssl pkcs12 \
-export \
-out client/client-keystore.p12 \
-inkey client/client-private-key.pem \
-in client/client-signed-cert.crt \
-certfile server/server.crt
we used 123456 as password again.
Generated server trust store containing the client signed certificate
keytool \
-import \
-trustcacerts \
-alias root \
-file client/client-signed-cert.crt \
-keystore server/server-trust-store.jks
password? 123456
Curl is working, but only with -k
curl -k \
--cert client/client-signed-cert.crt \
--key client/client-private-key.pem \
https://localhost:443:/my/endpoint
without the -k I get the error:
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.
Configured Apache HTTPClient:
private HttpClient createClient() throws Exception {
String keyPassword = "123456";
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(resourceAsStream("/client/client-key-store.p12"), keyPassword.toCharArray());
SSLContext sslContext = new SSLContextBuilder()
.setProtocol("TLSv1.2")
.loadKeyMaterial(ks, keyPassword.toCharArray())
.loadTrustMaterial(null, new TrustSelfSignedStrategy())
.build()
return HttpClients.custom()
.setSSLContext(sslContext)
.setSSLHostnameVerifier(new NoopHostnameVerifier())
.build();
}
(The construction is done via multiple methods that I squeezed here to one, so if something is weird or missing please let me know, perhaps I miscopy-pasted something.)
but when trying to send the same request as with Curl I'm getting:
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2038)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1135)
at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1779)
at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:124)
at sun.security.ssl.Handshaker.sendChangeCipherSpec(Handshaker.java:1156)
at sun.security.ssl.ClientHandshaker.sendChangeCipherAndFinish(ClientHandshaker.java:1266)
at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:1178)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:348)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

Grpc Java : Set up SSLContext on Server

I want to set-up SSL on my GRPC server.
For that I need certificate chain and a pkcs8 private key.
I have done the following:
Generate CA key:
openssl genrsa -des3 -out ca.key 4096
Generate CA certificate:
openssl req -new -x509 -days 365 -key ca.key -out ca.crt
Generate server key:
openssl genrsa -des3 -out server.key 4096
Generate server signing request:
openssl req -new -key server.key -out server.csr
Self-sign server certificate:
openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
Remove passphrase from the server key:
openssl rsa -in server.key -out server.key
Conver to pkcs8
openssl pkcs8 -topk8 -nocrypt -in server.key -out pkcs8_key.pem
Now that I have my server.cert and pkcs8_key.pem files, I've created the server as such:
InputStream certChain = MyServer.class.getResourceAsStream("/server.crt");
InputStream privateKey = MyServer.class.getResourceAsStream("/pkcs8_key.pem");
SslContext sslContext = GrpcSslContexts.forServer(certChain, privateKey, "password").build();
Server server = NettyServerBuilder.forPort(8080)
.sslContext(sslContext)
.addService(new ChatService())
.addService(new HelloWorldService())
.useTransportSecurity(certChain, privateKey)
.build();
The classpath is configured properly.
The error stack I'm getting:
Exception in thread "main" java.lang.IllegalArgumentException: Input stream does not contain valid private key.
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:296)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:104)
at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:162)
at server.MyServer.main(MyServer.java:20)
Caused by: java.io.IOException: overrun, bytes = 2353
at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:92)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContext.generateKeySpec(SslContext.java:978)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1034)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1024)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:294)
... 3 more

The exception is being caused by this line:
SslContext sslContext = GrpcSslContexts
.forServer(certChain, privateKey, "password").build();
Since your pkcs8 key has no password, you should not be passing a password and instead use the two-argument method:
SslContext sslContext = GrpcSslContexts
.forServer(certChain, privateKey).build();
Note that calling useTransportSecurity() will overwrite your call to sslContext(), so you shouldn't call both. Calling both would break in the current code because forServer() consumes and close the provided InputStreams, so you'd be passing closed streams to useTransportSecurity().

Setup to avoid PKIX path building failed error

Sorry for opening another question with the same topic, but i think this sub-question would bloat the other one into oblivion.
I run into the mentioned error message, which is quite unspecific (at least for me). The debug output shows the certificates are loaded and then only the mentioned error. I generated the test certificate with its own CA chain:
CA -> SubCA -> ClientCert
I try to connect a client and a server on the same machine (to test a two way protocol) with SSL.
I generate my ca certificates using these commands:
openssl req -batch -x509 -config ${ROOTCA_CONFIG} -newkey rsa:2048 -sha1 -nodes -out ${ROOTCA_CERT} -outform PEM -days 7300
openssl req -batch -config ${SUBCA_CONFIG} -newkey rsa:2048 -sha1 -nodes -out ${SUBCA_CSR} -outform PEM
openssl ca -batch -config ${ROOTCA_CONFIG} -policy signing_policy -extensions signing_req_CA -out ${SUBCA_CERT} -infiles ${SUBCA_CSR}
They seem to be fine. The only thing that puzzles me is: If concatenate both certificates into a single file and verify them with that chain, it is fine. If it try to verify with subCA or the root CA only, verification fails.
Then i create my client/server cert:
openssl req -batch -config ${CLIENT_CONFIG} -newkey rsa:2048 -sha256 -nodes -out ${CLIENT_CSR} -outform PEM -keyout $1.key
openssl ca -batch -config ${SUBCA_CONFIG} -policy signing_policy -extensions signing_req -out ${CLIENT_CERT} -infiles ${CLIENT_CSR}
With this i create a PKCS12 file to use in my keystore:
openssl pkcs12 -export -inkey ${CONNECTOR_KEY} -in ${CONNECTOR_CERT} -out ${CONNECTOR_P12}
I do this by calling my script twice, once for the server and once for the client. Let's call them client.cert and server.cert, even if client/server is confusing since they both are local protocol endpoints.
I then use these commands to generate the truststore and keystore for client and server:
keytool -keystore $2-truststore.jks -importcert -alias ca -file test_ca_certs/rootca.cert
keytool -keystore $2-truststore.jks -importcert -alias subca -file test_ca_certs/subca.cert
keytool -v -importkeystore -srckeystore $1 -srcstoretype PKCS12 -destkeystore $2-keystore.jks -deststoretype JKS
Let $2 be client and server each (server-truststore etc.) and $1 be the same as ${CONNECTOR_P12} before (somefile.p12)
So now i have a truststore with CA and SubCA and a keystore with the PKCS12 Token. Truststore is the same on client and server side, Token is pretty much the same, but has different keypairs, since they are generated each time.
The ssl debug output indicates the certs are loaded:
keystore (...) has type [jks], and contains aliases [1].
***
found key for : 1
chain [0] = [
[
Version: V3
Subject: CN=cnname, OU=ouname, O=oname, L=location, ST=bavaria, C=DE
Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
Key: Sun RSA public key, 2048 bits
modulus: 2999...
public exponent: 65537
...
...
keystore has type [jks], and contains aliases [ca, subca].
adding as trusted cert:
Subject: CN=my Root CA 2016, O=organization, C=DE
Issuer: CN=my Root CA 2016, O=organization, C=DE
Algorithm: RSA; Serial number: 0xfc8239c0355555c1
Valid from Wed Oct 19 10:14:36 CEST 2016 until Tue Oct 14 10:14:36 CEST 2036
adding as trusted cert:
Subject: CN=my SubCA 2016, O=Fraunhofer, C=DE
Issuer: CN=my Root CA 2016, O=Fraunhofer, C=DE
Algorithm: RSA; Serial number: 0x1
Valid from Wed Oct 19 10:14:38 CEST 2016 until Thu Oct 17 10:14:38 CEST 2024
Is there some general flaw in my understanding? Again, sorry for posting two questions but i start to believe i do something wrong in a more fundamental fashion. Thanks!

I finally found the solution. I only set debugging to SSL. This was my mistake. I would have needed to set the debug output to "all". Then i can see this error message:
Caused by: sun.security.validator.ValidatorException: Extended key
usage does not permit use for TLS server authentication
This is much more specific. To fix that, indeed i needed to change my extended key usage to this:
keyUsage = digitalSignature, keyEncipherment, nonRepudiation
extendedKeyUsage = clientAuth, serverAuth
Thank you very much!

Connection Reset while Transfering files over FTPS using java

I am using Java(Zehon) to transfer files over FTPS. This is my code snippet.
try {
FTPsClient ftpClient = new FTPsClient(host, port,username ,password ,false,keyStorePath,keyStorePass);
ftpClient.sendFile(absFilePath, ftpsFolder);
}catch (FileTransferException e) {e.printStackTrace();}
I have telnet the host ip and i am getting connected. I am quite sure that the credentials i am passing is correct.The exception am getting is com.zehon.exception.FileTransferException: java.net.SocketException: Connection reset
Any suggestions as to what else i may need to add while connecting to the host because the javadoc for FTPsClient does not show any more methods to connect to the host.

The Problem was with configuring the keystore file. This is how you actually need to make it:
Download OPENSSL and type this command
openssl pkcs12 -export -in /path/to/YourVeriSignSSLCert.crt -inkey /path/to/YourPrivateKey.key -out mycert.p12 -name tomcat -CAfile /path/to/YourIntermediateCertificate.cer -caname root
YourVeriSignSSLCert.crt is your current openssl certificate
YourPrivateKey.key is your current private key
YourIntermediateCertificate.cer is the VeriSign Intermediate CA
The exported keystore will be in 'mycert.p12'
Now the keystore file is of pkcs12 format i am converting this into jks format :
keytool -v -importkeystore -trustcacerts -srckeystore mycert.p12 -srcstoretype PKCS12 -destkeystore md_keystore.jks -deststoretype JKS
Now this is the keystore file that needs to be passed to the program.