We Keep Coding

How to connect to SQL DB from Java JDBC with mutual TLS/SSL encryption

Cloud databases are very prevalent today and sometimes we need to connect to one from a laptop or similar. Usually there is an option for an unencrypted connection but that's not very secure. So how do I connect to a mutual TLS mysql server (like google cloud SQL)?
From mysql cli it's rather straight-forward after downloading the server ca cert, client cert and key from google cloud console:
mysql -u <user-name> -h <server-ip> -p --ssl-ca=server-ca.pem --ssl-cert=client-cert.pem --ssl-key=client-key.pem
Note: for Google cloud you can also use the Cloud SQL Proxy but I want to connect without that

While not is not exactly an answer to the question I will add this example of how to make this work with logstash jdbc plugin deployed on kubernetes since it was not completely straight-forward and it might be useful for similar systems.
This solution is based on the first solution but the truststore must be based on the full cacerts of the jvm.
In your input definition add the parameters to the jdbc URL:
input {
jdbc {
...
jdbc_connection_string => "jdbc:mysql://<server-ip>:3306/<db>?verifyServerCertificate=true&useSSL=true&requireSSL=true"
jdbc_password => "${MYSQL_PASSWORD}"
...
}
}
Then create a secret from the trust- and keystore and for mysql password:
kubectl create secret generic java-tls-creds --from-file=./keystore.jks --from-file=./truststore.jks
kubectl create secret generic mysql-password --from-literal='password=<password>'
Then we modify the deployment yaml to mount the credentials and add the LS_JAVA_OPTS to point to them:
apiVersion: apps/v1
kind: Deployment
...
env:
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-password
key: password
- name: LS_JAVA_OPTS
value: -Djavax.net.ssl.keyStore=/java-tls-creds/keystore.jks -Djavax.net.ssl.keyStorePassword=<password> -Djavax.net.ssl.trustStore=/java-tls-creds/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit
...
volumeMounts:
- name: java-tls-creds
mountPath: "/java-tls-creds"
readOnly: true
...
volumes:
- name: java-tls-creds
secret:
secretName: java-tls-creds

Logstash 7 and jvm 9+ below.
OK, I finally figured it out piecing together information from different sources. This solution is tested with google cloud SQL and hibernate but should work in other setups as well.
The solution requires using java keystore (client cert/key) and truststore (server-ca cert) and passing some jdbc URL params and JVM options.
First off, lets create a truststore from the server-ca.pem file:
keytool -importcert -alias gcpRoot -file server-ca.pem -keystore truststore.jks -storepass <chose a password and remember>
EDIT: Or add CA to jvms existing cacerts file (to ensure other https calls will work), copy cacerts it rename to truststore.jks and run:
OR: keytool -importcert -alias gcpRoot -file server-ca.pem -keystore truststore.jks -storepass changeit
Secondly, we need to import the client cert and key to a keystore file in two steps (I used my SQL username as alias but I don't think it matters)
openssl pkcs12 -export -in client-cert.pem -inkey client-key.pem -out keystore.p12 -name "<keystore-alias>" -passout pass:<chose a password and remember>
keytool -importkeystore -alias <keystore-alias> -srckeystore keystore.p12 -srcstoretype pkcs12 -destkeystore keystore.jks -srcstorepass <insert same password here> -deststoretype JKS -deststorepass <password-for-output-jks-keystore>
Next, we need to modify the jdbc connection URL (NOTE! Some IDEs - like intelliJ - requires & to be escaped and be replaced with &
jdbc:mysql://<server-ip>/<db-name>?verifyServerCertificate=true&useSSL=true&requireSSL=true
Finally we need to provide location and passwords for keystore and truststore as JVM parameters:
-Djavax.net.ssl.trustStore="truststore.jks"
-Djavax.net.ssl.trustStorePassword="<password>"
-Djavax.net.ssl.keyStore="keystore.jks"
-Djavax.net.ssl.keyStorePassword="<password>"
Update:
If you are running on a jvm version 9 or above (like logstash docker images for 7+) there are some class-loading issues to be worked around to get the jdbc driver to even load and you need a recent version of the mysql driver for TLS to work.
First you apparently have to put the .jar file of the jdbc driver in <logstash-dir>/logstash-core/lib/jars/mysql-connector-java-8.0.17.jar (this will cause the jar to be loaded automatically). And we also need to add the following to the input config:
jdbc_driver_library => ""
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
For some reason I still get the warning Loading class com.mysql.jdbc.Driver. This is deprecated. The new driver class iscom.mysql.cj.jdbc.Driver'` but it works despite this warning.

Crash shell unable to connect by ssh

I have a standalone Tomcat application with CRaSH shell library embedded.
I can successfully connect to the application via telnet but I am struggling with ssh.
Here is my crash.properties config :
# Key authentication
crash.auth=key
crash.auth.key.path=/usr/share/tomcat8/conf/id_rsa.pem
# SSH configuration
crash.ssh.port=2000
I have generated a ssh keypair via ssh-keygen, then I created a pem file via openssl rsa -in id_rsa -outform pem -pubout > id_rsa.pem command.
Whatever the ssh key used, I have the error when I try to connect via ssh -i /usr/share/tomcat8/conf/id_rsa -p 2000 myserver.example.com command :
Permission denied (publickey).
I have tried the solution described here but without success. I have also tried to put the id_rsa.pem key in my war /WEB-INF/sshd folder. Same result.
Any clue ?

After debugging, I notice that org.crsh.auth.KeyAuthenticationPlugin was comparing the authorized key and the key given at login, and they were identical but the method authenticate was always returning false (bad authentication). After fixing the bug, I realized that it was already declared here and a pull request was available the same way.

Spring boot after https: The Tomcat connector configured to listen on port 8444 failed to start.

I followed a guide to enable https in Spring Boot. The application was beforehand working on https://localhost:8080
I've created a keystore.jks which is in the same directory as my application.properties, which now looks like:
# Define a custom port instead of the default 8080
server.port = 8444
# Tell Spring Security (if used) to require requests over HTTPS
security.require-ssl=true
# The format used for the keystore
server.ssl.key-store-type:PKCS12
# The path to the keystore containing the certificate
server.ssl.key-store=keystore.p12
# The password used to generate the certificate
server.ssl.key-store-password=<somepassword>
# The alias mapped to the certificate
server.ssl.key-alias=tomcat
Now, if I run the main method to start the spring boot app, it throws:
Description:
The Tomcat connector configured to listen on port 8444 failed to start. The port may already be in use or the connector may be misconfigured.
Action:
Verify the connector's configuration, identify and stop any process that's listening on port 8444, or configure this application to listen on another port.
The port isn't in use, so it must be misconfiguration?
I'm unsure of what to change. It's a simple SPA app, Spring just serves an index.html and has a single REST endpoint. How should tomcat/spring be configured to accept https in this case, and start up without errors?

I too had the same problem and was able to fix it. My problem was generating the keystore.p12 file.
If you have a certificate file and private key file, you can generatekeystore.p12 file using following command.
openssl pkcs12 -export -in <mycert.crt> -inkey <mykey.key> -out keystore.p12 -name <alias>
You will be prompted for a password,there you can enter a password you like.
Once the keystore file is generated copy it to the directory where your .jar file exist.
Following is a working example configuration.
server.port=8443
security.require-ssl=true
server.ssl.key-store-type=PKCS12
server.ssl.key-store=file:keystore.p12
server.ssl.key-store-password=<password>
server.ssl.key-alias=<alias>
Note the key store file path file:keystore.p12 if it is going to reside in the same directory as the executable .jar file.

I solved the same issue by using the following configuration
# Define a custom port instead of the default 8080
server.port=8443
# Tell Spring Security (if used) to require requests over HTTPS
security.require-ssl=true
# The format used for the keystore
server.ssl.key-store-type=PKCS12
# The path to the keystore containing the certificate
server.ssl.key-store=src/main/resources/keystore.p12
# The password used to generate the certificate
server.ssl.key-store-password=root0
I removed alias name and it worked perfectly.
"You probably won't need a key alias, since there will only be one key entry" referred from
TOMCAT SSL Error: Alias name does not identify a key entry

From Spring Boot 2.0 and higher, you can ignore this property.
security.require-ssl=true
To enable SSL, use the below configuration in your application.properties
The format used for the keystore
server.ssl.key-store-type=JKS
The path to the keystore containing the certificate
server.ssl.key-store=classpath:somecert.jks
The password used to generate the certificate
server.ssl.key-store-password=password
The alias mapped to the certificate
server.ssl.key-alias=alias_name
Note : server.ssl.key-store refers to the keystore location. Use
classpath prefix, if it is present in src/main/resources. Otherwise use,
file:/some/location.

I had the same issue as well but in my case the file path (in application.properties) for keystore file was incorrect on Linux and causing this error message.

I had same problem. for me server.ssl.key-alias was set to a wrong key. So, it sounds that some server mis-configurations in application.properties can cause this error message to appear.

Spring Boot - Enable SSL (HTTPS) with CA certificate

Maybe I will find help here. I want to enable SSL on Spring Boot application. I have a configuration like:
server: port: 8999 ssl:
enabled: true
key-store: classpath:keystore.jks
key-store-password: mypass
key-password: mypass
The problem is my keystore. I've imported *crt file into keystore with alias 'tomcat':
keytool -importcert -file certificate.crt -keystore native.jks -alias tomcat
However, I still can't properly access my rest api. It vomits with error in firefox:
SSL_ERROR_RX_RECORD_TOO_LONG
it doesn't work. How can I make proper keystore to make it work? I'm issuing with CA certificate, NOT self-signed. I have files like:
certificate.crt, certificate.p12, certificate.pk, privkey.pem and
three files root_ca_1.ca-bundle, root_ca_2.ca-bundle,
root_ca_3.ca-bundle
. That's all I have. I'm very fresh with ssl topic, just read some tutorials and tried few keytool commands to make it work. I'd be grateful for help. Thank You in advance.

I just spent the afternoon figuring out this exact problem, so I'll share my process here.
Each of the references below provides instructions on how to generate and use a self-signed certificate. That's not exactly what I'm trying to do, but these each contain some useful background information.
REFERENCES:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://mkyong.com/spring-boot/spring-boot-ssl-https-examples/
https://dzone.com/articles/how-to-enable-the-https-into-spring-boot-applicati
I have already purchased a real CA-issued SSL certificate for the *.jimtough.org 'wildcard' domain. I purchased the certificate from http://www.namecheap.com/, but the actual Certificate Authority (CA) is Comodo.
As part of the purchase/activation procedure with the CA, I needed to follow these instructions:
https://www.namecheap.com/support/knowledgebase/article.aspx/9592/14/generating-a-csr-on-amazon-web-services-aws/
Note that I opted to follow their AWS instructions because I am an AWS user and have an EC2 server readily available with OpenSSL and Java already installed on the server. There are lots of other alternatives to do the same procedure, so search further to find the 'generating a CSR' instructions that are right for you.
At the end of this step, I have the following two files:
csr.pem - This is used as part of the SSL cert request/activation process
private.key - This is the private key part of my SSL cert, which I will need later to install the cert on my servers. Keep it secret. Keep it safe.
After I completed the purchase and verification procedure for my SSL cert, the CA sent me back a .zip file that contained a .p7b, .crt, and .ca-bundle file.
One of the reference links below explains the difference between these certificate file types:
.p7b - This type should be compatible with Java-based applications (PKCS#7 format)
.crt - This type should be compatible with most everything else - the link above suggests this is PEM format
.ca-bundle - Not sure when to use this - the link above suggests this is PEM format
REFERENCES:
https://www.ssls.com/knowledgebase/what-are-certificate-formats-and-what-is-the-difference-between-them/
Next I need to figure out how to use the files that I listed above to configure my Spring Boot application for HTTPS.
I will follow the relevant parts of the below tutorials to get what I need:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://www.baeldung.com/x-509-authentication-in-spring-security
NOTE: In both tutorials, I will NOT follow their sections on creating a self-signed certificates, since I already possess a real certificate that was issued by a real CA.
The first relevant step in their instructions is to create a new Java keystore. The requirements are:
Must have Java installed, so I have access to the 'keytool' utility
Must have the 'openssl' utility installed, so I can create a .p12 file using my .key and .p7b files as input
I will use my AWS EC2 Linux server to do this. My server already has the Java/keytool and OpenSSL utilities installed.
First I need to use the OpenSSL utility to create a .p12 file that (if I understand correctly) will contain both my
private key, and the CA-issue certificate. Second, I need to create a new Java keystore that will contain an imported
copy of the .p12 file.
openssl pkcs12 -export -out jimtough-dot-org.p12 -name "jimtough-dot-org" -inkey private.key -in __jimtough_org.crt
IMPORTANT: You need to provide a password at the 'export password' prompt, otherwise the keytool import in the next step will fail
keytool -importkeystore -srckeystore jimtough-dot-org.p12 -srcstoretype PKCS12 -destkeystore jimtough-dot-org-keystore.jks -deststoretype JKS
You will need to provide the password you used in the 'openssl' command
You will also need to provide a new password for the keystore that you are creating
(OPTIONAL) keytool -importkeystore -srckeystore jimtough-dot-org-keystore.jks -destkeystore jimtough-dot-org-keystore.pkcs12 -deststoretype pkcs12
I received a warning from 'keytool' about JKS being a proprietary format, and a suggestion that I convert my keystore to PKCS12 format, so I did so with this optional command
Finally, I need to package my newly created Java keystore with my Spring Boot application and configure the application to use it.
REFERENCES:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://www.baeldung.com/x-509-authentication-in-spring-security
I referred back to the two Baeldung tutorials above, and was able to get the details I needed to make my Spring Boot (with Spring Security enabled) to work.
I created a new 'keystore' folder under the existing 'src/main/resources', and copied both of my newly created keystore files there (I kept both formats).
I added the block below to my Spring Boot application's application.properties file.
#--------------------------------------------------------------------------------------------------
# SSL CONFIGURATION
# The format used for the keystore. It could be set to JKS in case it is a JKS file
#server.ssl.key-store-type=JKS
server.ssl.key-store-type=PKCS12
# The path to the keystore containing the certificate
#server.ssl.key-store=classpath:keystore/jimtough-dot-org-keystore.jks
server.ssl.key-store=classpath:keystore/jimtough-dot-org-keystore.pkcs12
server.ssl.key-store-password=mykeystorepassword
server.ssl.key-alias=jimtough-dot-org
server.ssl.enabled=true
server.port=8443
#--------------------------------------------------------------------------------------------------
As expected, I get a bunch of warnings from my browser when I attempt to connect to a locally running instance of the Spring Boot application using https://localhost:8443/ as the URL. This happens because the browser correctly identifies the mismatch between 'localhost' and the expected 'jimtough.org' domain that my SSL certificate was created to run on. There shouldn't be any warnings when I deploy the application to a server whose hostname is 'anything.jimtough.org' (or just www.jimtough.org).
That's it! Happy HTTPS'ing!

Don't use spring-boot for SSL-connections. Let a reverse proxy handle the SSL-termination.
So it is possible, that a tool can automatic renew your certificates (like the certbot).
And you don't need to put a (sercret) private-key to your application.
It is simpler to share your application. A server-admin needs only to setup reverse proxy (or have something familar already in cloud-systems like kubernetes) and can scale up your application.
Benefits
Scalable application
Simpler spring-setup
No secrets in your application
Simpler use of application (#cloudSystems)
Most reverseProxies are compatible with certbot

Well, you require few more properties to be added in ssl configuration
server:
ssl:
key-store: classpath:config/tls/keystore.p12
key-store-password: password ##this will be your certificate password
key-store-type: PKCS12
ciphers: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
enabled-protocols: TLSv1.2
http2:
enabled: true
Make sure that the p12 keystore file is present under classpath (src/main/resources) in config/tls
But to generate keystore file use the below openssl command where you
will use .crt and .key file to generate .p12 file.
.crt and .key are present under Folder CA and file p12 file will be
generated under CA
Please note that you will asked for the certificate password after
running the below command
openssl pkcs12 -export -in CA/certificate.crt -inkey CA/certificate.key -name certificate -out CA/certificate.p12
If you want to add this certificate to your local cacert then use the
below command
under jre\lib\security
keytool -importcert -noprompt -trustcacerts -alias ca.certificate -file CA/certificate.crt -keystore cacerts -storepass changeit

Trouble authenticating against MongoDB v2.6 with x509 certificates using mongo-java-driver

I am having difficulty connecting to mongo v2.6 using x509 certificates for authentication from Java/Groovy. I have built mongo with ssl and feel I have it configured correctly.
Our project has two applications that connect to Mongo - one written in NodeJS and one written in Java/Groovy. The NodeJS project is successfully able to authenticate using an X509 Certificate and query. Additionally, I am able to shell into mongo by specifying ssl and providing the PEM file on the command line. However, I am unable to connect to mongo using the Java Mongo Driver. I would greatly appreciate some assistance and here are the steps I have taken thus far:
Building MongoDB 2.6 to run with ssl using SCONS:
I have downloaded the 2.6 release of MongoDB from the mongodb github page
Then, built it with scons to include ssl
scons --64 --dd --ssl all
scons --ssl --prefix=/opt/mongo install
Generating Keys for Mongo::
I generated the following keys/certs
openssl req -new -newkey rsa:1024 -nodes -out myMongo.req -keyout myMongo.key -subj "/C=US/ST=myState/CN=myMongo/OU=myUnit/L=myLocation" -days 36500
openssl x509 -CA myCA.pem -CAkey myCa.key -CAserial myCa.srl -req -in myMongo.req -out myMongo.pem -days 36500
This gives me the following files:
myMongo.key ( Containing a private key )
myMongo.pem ( Containing a certificate )
myMongo.req ( Containing a certificate request ) (I'm not sure if I need this one, and haven't used it yet )
In compliance with the mongo instructions, I concatenated the key and certificate together into 1 file
cat myMongo.key myMongo.pem > combined.pem
Following the mongo instructions for x509 configuration:
http://docs.mongodb.org/manual/tutorial/configure-x509/
openssl x509 -in combined.pem -inform PEM -subject -nameopt RFC2253
which gives me the subject: "C=US,ST=myState,CN=myMongo,OU=myUnit,L=myLocation"
I then add that subject to the users on the database
db.getSiblingDB("$external").runCommand(
{
createUser: "C=US,ST=myState,CN=myMongo,OU=myUnit,L=myLocation",
roles: [
{ role: 'readWrite', db: 'mydbName' },
{ role: 'userAdminAnyDatabase', db: 'admin' }
],
writeConcern: { w: "majority" , wtimeout: 5000 }
}
)
In my mongod.conf file, I specify the following options:
sslMode = requireSSL
sslPEMKeyFile = /path/to/my/combined.pem
sslCAFile = /path/to/myCA.pem
At this point, I can launch
mongod --config /path/to/my/mongod.conf
after launching, I can only access the shell by specifying
mongo --ssl -- sslPEMKeyFile /path/to/my/combined.pem
Connecting to MongoDB from Java-Mongo-Driver version 2.12.0
I'm trying to follow the example given on the java-mongo-driver github page:
"https://github.com/mongodb/mongo-java-driver/blob/master/src/examples/example/X509CredentialsExample.java"
The code fails with an exception which we will offer up below, but it seems pretty strange that you would attempt to connect by just providing the DN in string format rather than actually attaching a certificate to the request. Doesn't that defeat the entire purpose of X509 to begin with? We looked at the Ruby example and they attach a pem file. My team understands java keystores and truststores, and will happily send that pem file along its merry way if that's what we're supposed to be doing. Additionally, when using the node driver we're absolutely specifying the location of the PEM file.
The final result is this. We can instantiate the MongoClient just fine, and we can even get a collection using the db object. When we attempt to query any collection in any way, we get back an exception whose message is "Unable to connect to any server."
It seems logical that the MongoClientOptions.Builder would allow us to add this information as that seems to be how it's done elsewhere. But we did an introspection of the class just in case it was missing from the documentation, and there's nothing in there that relates to x509 as far as we can tell.
I appreciate your help.

In the example that you linked to, note that it's just using SSLSocketFactory.getDefault(), which relies on the JDK-defined system properties for specifying keystores and truststores, e.g
javax.net.ssl.keyStoreType=pkcs12
javax.net.ssl.keyStore=/path/to/pkc
javax.net.ssl.keyStorePassword=
javax.net.ssl.trustStoreType=jks
javax.net.ssl.trustStore=/path/to/truststore
javax.net.ssl.trustStorePassword=
You have to add your client certificate to the keystore and the signing authority's certificate for the MongoDB server to the truststore.