We Keep Coding

How to connect to SQL DB from Java JDBC with mutual TLS/SSL encryption

Cloud databases are very prevalent today and sometimes we need to connect to one from a laptop or similar. Usually there is an option for an unencrypted connection but that's not very secure. So how do I connect to a mutual TLS mysql server (like google cloud SQL)?
From mysql cli it's rather straight-forward after downloading the server ca cert, client cert and key from google cloud console:
mysql -u <user-name> -h <server-ip> -p --ssl-ca=server-ca.pem --ssl-cert=client-cert.pem --ssl-key=client-key.pem
Note: for Google cloud you can also use the Cloud SQL Proxy but I want to connect without that

While not is not exactly an answer to the question I will add this example of how to make this work with logstash jdbc plugin deployed on kubernetes since it was not completely straight-forward and it might be useful for similar systems.
This solution is based on the first solution but the truststore must be based on the full cacerts of the jvm.
In your input definition add the parameters to the jdbc URL:
input {
jdbc {
...
jdbc_connection_string => "jdbc:mysql://<server-ip>:3306/<db>?verifyServerCertificate=true&useSSL=true&requireSSL=true"
jdbc_password => "${MYSQL_PASSWORD}"
...
}
}
Then create a secret from the trust- and keystore and for mysql password:
kubectl create secret generic java-tls-creds --from-file=./keystore.jks --from-file=./truststore.jks
kubectl create secret generic mysql-password --from-literal='password=<password>'
Then we modify the deployment yaml to mount the credentials and add the LS_JAVA_OPTS to point to them:
apiVersion: apps/v1
kind: Deployment
...
env:
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-password
key: password
- name: LS_JAVA_OPTS
value: -Djavax.net.ssl.keyStore=/java-tls-creds/keystore.jks -Djavax.net.ssl.keyStorePassword=<password> -Djavax.net.ssl.trustStore=/java-tls-creds/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit
...
volumeMounts:
- name: java-tls-creds
mountPath: "/java-tls-creds"
readOnly: true
...
volumes:
- name: java-tls-creds
secret:
secretName: java-tls-creds

Logstash 7 and jvm 9+ below.
OK, I finally figured it out piecing together information from different sources. This solution is tested with google cloud SQL and hibernate but should work in other setups as well.
The solution requires using java keystore (client cert/key) and truststore (server-ca cert) and passing some jdbc URL params and JVM options.
First off, lets create a truststore from the server-ca.pem file:
keytool -importcert -alias gcpRoot -file server-ca.pem -keystore truststore.jks -storepass <chose a password and remember>
EDIT: Or add CA to jvms existing cacerts file (to ensure other https calls will work), copy cacerts it rename to truststore.jks and run:
OR: keytool -importcert -alias gcpRoot -file server-ca.pem -keystore truststore.jks -storepass changeit
Secondly, we need to import the client cert and key to a keystore file in two steps (I used my SQL username as alias but I don't think it matters)
openssl pkcs12 -export -in client-cert.pem -inkey client-key.pem -out keystore.p12 -name "<keystore-alias>" -passout pass:<chose a password and remember>
keytool -importkeystore -alias <keystore-alias> -srckeystore keystore.p12 -srcstoretype pkcs12 -destkeystore keystore.jks -srcstorepass <insert same password here> -deststoretype JKS -deststorepass <password-for-output-jks-keystore>
Next, we need to modify the jdbc connection URL (NOTE! Some IDEs - like intelliJ - requires & to be escaped and be replaced with &
jdbc:mysql://<server-ip>/<db-name>?verifyServerCertificate=true&useSSL=true&requireSSL=true
Finally we need to provide location and passwords for keystore and truststore as JVM parameters:
-Djavax.net.ssl.trustStore="truststore.jks"
-Djavax.net.ssl.trustStorePassword="<password>"
-Djavax.net.ssl.keyStore="keystore.jks"
-Djavax.net.ssl.keyStorePassword="<password>"
Update:
If you are running on a jvm version 9 or above (like logstash docker images for 7+) there are some class-loading issues to be worked around to get the jdbc driver to even load and you need a recent version of the mysql driver for TLS to work.
First you apparently have to put the .jar file of the jdbc driver in <logstash-dir>/logstash-core/lib/jars/mysql-connector-java-8.0.17.jar (this will cause the jar to be loaded automatically). And we also need to add the following to the input config:
jdbc_driver_library => ""
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
For some reason I still get the warning Loading class com.mysql.jdbc.Driver. This is deprecated. The new driver class iscom.mysql.cj.jdbc.Driver'` but it works despite this warning.

Java Keystore : missing decryptionKey and signingKey aliases

I would need help about this exception that I don't understand...
and I don't even find anything on google about that.
*java.security.KeyStoreException] : [The supplied keystore is not configured correctly, it must contain the 'decryptionKey' alias and the 'signingKey' alias at com.entrust.toolkit.x509.jsse.d.&lt
What are these aliases ? Do you have idea of the reason why they are missing?
This is the line of code throwing the exception
CLIENT_KEYSTORE_PATH = full path of the "client certificate" pfx file.
CLIENT_KEYSTORE_PASS = password of the pfx file
KeyStore cks = KeyStore.getInstance("PKCS12");
cks.load(new FileInputStream(CLIENT_KEYSTORE_PATH), CLIENT_KEYSTORE_PASS.toCharArray());
SSLContext sslcontext = SSLContexts.custom().loadKeyMaterial(cks, CLIENT_KEYSTORE_PASS.toCharArray()).build();
Thank you for your help

Checkout the keytool command that comes bundeld with Java binaries.
You can find how to use it googling on line.
Don't miss the key store type option to specify PKCS12
Here there is a link to one place for example.

Error: "The keystore does not contain a private key associated with this alias"
Problem
When importing a Code Signing Certificate for Sun Java or Symantec SSL Certificate into a keystore, you receive the following error:
Error: "The keystore does not contain a private key associated with this alias"
Cause
This error occurs when one of the following conditions are true:
The certificate is being imported into the incorrect keystore.
The certificate is being imported into the incorrect alias.
Solution:
To resolve this problem:
Locate the correct Keystore. The keystore must be the same keystore
that was used to originally generate the Certificate Signing Request
(CSR). If a new keystore was generated with the same name and alias,
the hash will be different. Therefore, the certificate will still not
match the private key in the keystore.
Locate the correct alias. To verify the alias, run the following
command:
keytool -list -v -keystore [keystore name]
The correct alias will state Entry type: keyEntry
Once you have located the correct keystore and alias, run the following command to import the certificate:
keytool -import -trustcacerts -keystore [keystore_filename] -alias [alias_name] -file [cert_file]
Resource Link:
https://knowledge.symantec.com/support/code-signing-support/index?page=content&id=SO6863&pmv=print&actp=PRINT&viewlocale=en_US

Spring Boot - Enable SSL (HTTPS) with CA certificate

Maybe I will find help here. I want to enable SSL on Spring Boot application. I have a configuration like:
server: port: 8999 ssl:
enabled: true
key-store: classpath:keystore.jks
key-store-password: mypass
key-password: mypass
The problem is my keystore. I've imported *crt file into keystore with alias 'tomcat':
keytool -importcert -file certificate.crt -keystore native.jks -alias tomcat
However, I still can't properly access my rest api. It vomits with error in firefox:
SSL_ERROR_RX_RECORD_TOO_LONG
it doesn't work. How can I make proper keystore to make it work? I'm issuing with CA certificate, NOT self-signed. I have files like:
certificate.crt, certificate.p12, certificate.pk, privkey.pem and
three files root_ca_1.ca-bundle, root_ca_2.ca-bundle,
root_ca_3.ca-bundle
. That's all I have. I'm very fresh with ssl topic, just read some tutorials and tried few keytool commands to make it work. I'd be grateful for help. Thank You in advance.

I just spent the afternoon figuring out this exact problem, so I'll share my process here.
Each of the references below provides instructions on how to generate and use a self-signed certificate. That's not exactly what I'm trying to do, but these each contain some useful background information.
REFERENCES:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://mkyong.com/spring-boot/spring-boot-ssl-https-examples/
https://dzone.com/articles/how-to-enable-the-https-into-spring-boot-applicati
I have already purchased a real CA-issued SSL certificate for the *.jimtough.org 'wildcard' domain. I purchased the certificate from http://www.namecheap.com/, but the actual Certificate Authority (CA) is Comodo.
As part of the purchase/activation procedure with the CA, I needed to follow these instructions:
https://www.namecheap.com/support/knowledgebase/article.aspx/9592/14/generating-a-csr-on-amazon-web-services-aws/
Note that I opted to follow their AWS instructions because I am an AWS user and have an EC2 server readily available with OpenSSL and Java already installed on the server. There are lots of other alternatives to do the same procedure, so search further to find the 'generating a CSR' instructions that are right for you.
At the end of this step, I have the following two files:
csr.pem - This is used as part of the SSL cert request/activation process
private.key - This is the private key part of my SSL cert, which I will need later to install the cert on my servers. Keep it secret. Keep it safe.
After I completed the purchase and verification procedure for my SSL cert, the CA sent me back a .zip file that contained a .p7b, .crt, and .ca-bundle file.
One of the reference links below explains the difference between these certificate file types:
.p7b - This type should be compatible with Java-based applications (PKCS#7 format)
.crt - This type should be compatible with most everything else - the link above suggests this is PEM format
.ca-bundle - Not sure when to use this - the link above suggests this is PEM format
REFERENCES:
https://www.ssls.com/knowledgebase/what-are-certificate-formats-and-what-is-the-difference-between-them/
Next I need to figure out how to use the files that I listed above to configure my Spring Boot application for HTTPS.
I will follow the relevant parts of the below tutorials to get what I need:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://www.baeldung.com/x-509-authentication-in-spring-security
NOTE: In both tutorials, I will NOT follow their sections on creating a self-signed certificates, since I already possess a real certificate that was issued by a real CA.
The first relevant step in their instructions is to create a new Java keystore. The requirements are:
Must have Java installed, so I have access to the 'keytool' utility
Must have the 'openssl' utility installed, so I can create a .p12 file using my .key and .p7b files as input
I will use my AWS EC2 Linux server to do this. My server already has the Java/keytool and OpenSSL utilities installed.
First I need to use the OpenSSL utility to create a .p12 file that (if I understand correctly) will contain both my
private key, and the CA-issue certificate. Second, I need to create a new Java keystore that will contain an imported
copy of the .p12 file.
openssl pkcs12 -export -out jimtough-dot-org.p12 -name "jimtough-dot-org" -inkey private.key -in __jimtough_org.crt
IMPORTANT: You need to provide a password at the 'export password' prompt, otherwise the keytool import in the next step will fail
keytool -importkeystore -srckeystore jimtough-dot-org.p12 -srcstoretype PKCS12 -destkeystore jimtough-dot-org-keystore.jks -deststoretype JKS
You will need to provide the password you used in the 'openssl' command
You will also need to provide a new password for the keystore that you are creating
(OPTIONAL) keytool -importkeystore -srckeystore jimtough-dot-org-keystore.jks -destkeystore jimtough-dot-org-keystore.pkcs12 -deststoretype pkcs12
I received a warning from 'keytool' about JKS being a proprietary format, and a suggestion that I convert my keystore to PKCS12 format, so I did so with this optional command
Finally, I need to package my newly created Java keystore with my Spring Boot application and configure the application to use it.
REFERENCES:
https://www.baeldung.com/spring-boot-https-self-signed-certificate
https://www.baeldung.com/x-509-authentication-in-spring-security
I referred back to the two Baeldung tutorials above, and was able to get the details I needed to make my Spring Boot (with Spring Security enabled) to work.
I created a new 'keystore' folder under the existing 'src/main/resources', and copied both of my newly created keystore files there (I kept both formats).
I added the block below to my Spring Boot application's application.properties file.
#--------------------------------------------------------------------------------------------------
# SSL CONFIGURATION
# The format used for the keystore. It could be set to JKS in case it is a JKS file
#server.ssl.key-store-type=JKS
server.ssl.key-store-type=PKCS12
# The path to the keystore containing the certificate
#server.ssl.key-store=classpath:keystore/jimtough-dot-org-keystore.jks
server.ssl.key-store=classpath:keystore/jimtough-dot-org-keystore.pkcs12
server.ssl.key-store-password=mykeystorepassword
server.ssl.key-alias=jimtough-dot-org
server.ssl.enabled=true
server.port=8443
#--------------------------------------------------------------------------------------------------
As expected, I get a bunch of warnings from my browser when I attempt to connect to a locally running instance of the Spring Boot application using https://localhost:8443/ as the URL. This happens because the browser correctly identifies the mismatch between 'localhost' and the expected 'jimtough.org' domain that my SSL certificate was created to run on. There shouldn't be any warnings when I deploy the application to a server whose hostname is 'anything.jimtough.org' (or just www.jimtough.org).
That's it! Happy HTTPS'ing!

Don't use spring-boot for SSL-connections. Let a reverse proxy handle the SSL-termination.
So it is possible, that a tool can automatic renew your certificates (like the certbot).
And you don't need to put a (sercret) private-key to your application.
It is simpler to share your application. A server-admin needs only to setup reverse proxy (or have something familar already in cloud-systems like kubernetes) and can scale up your application.
Benefits
Scalable application
Simpler spring-setup
No secrets in your application
Simpler use of application (#cloudSystems)
Most reverseProxies are compatible with certbot

Well, you require few more properties to be added in ssl configuration
server:
ssl:
key-store: classpath:config/tls/keystore.p12
key-store-password: password ##this will be your certificate password
key-store-type: PKCS12
ciphers: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
enabled-protocols: TLSv1.2
http2:
enabled: true
Make sure that the p12 keystore file is present under classpath (src/main/resources) in config/tls
But to generate keystore file use the below openssl command where you
will use .crt and .key file to generate .p12 file.
.crt and .key are present under Folder CA and file p12 file will be
generated under CA
Please note that you will asked for the certificate password after
running the below command
openssl pkcs12 -export -in CA/certificate.crt -inkey CA/certificate.key -name certificate -out CA/certificate.p12
If you want to add this certificate to your local cacert then use the
below command
under jre\lib\security
keytool -importcert -noprompt -trustcacerts -alias ca.certificate -file CA/certificate.crt -keystore cacerts -storepass changeit

JMeter - Setting javax.net.ssl.keyStore and javax.net.ssl.keyStorePassword based on application environment

We have 2 environments and for each we have a certificate PFX file.
I tried to import these 2 certs to make one JKS/P12 file and tried to access cert based on alias name. Though I can see 2 entries using the command
keytool -list -v -keystore "C:/yyyyy/xxxx.jks" -storepass "kdjfkdjfkdjf" > "C:/KeyStoreContents.txt", I am unable to access any of the cert and seeing the error 'alias not found'.
I there a way to set the values of javax.net.ssl.keyStore and javax.net.ssl.keyStorePassword based on application environment?
I tried to set them in beanshell pre-processor with:
System.setProperty("javax.net.ssl.keyStore","C:/yyyyy/" + pOneTwoName + ".p12");
System.setProperty("javax.net.ssl.keyStorePassword", pOneTwoPwd);
It didn't work as well.
I also tried ${__P(javax.net.ssl.keyStore, C:/yyyyyy/QA.p12)} and ${__setProperty(javax.net.ssl.keyStore, C:/yyyyyy/QA.p12)} and none of the above worked.
Is there a way to load these values dynamically before Keystore Configuration element gets executed?

I am able to solve this issue with the below steps:
Get all the certs(pfx) for the application environments
Optional: Change the extension of the files to p12
Set a meaningful alias names to these certs with the below command:
keytool -changealias -alias "ExistingAliasName" -destalias "AppName_Environment" -keystore "C:\JKS\AppName_environment.p12" -storepass "StorePassword"
Notes for step 3:
a. It might ask to enter keypass and we can enter the store pass
b. Even we provide upper case in alias name, the text will be converted to lower case and will be set as alias name
c. Here I followed this alias naming convention: application name_environment name as mentioned in JMeter
Import the certs into a JKS file and we can set any password to JKS and need not be same as cert. See the below command:
keytool -importkeystore -srckeystore "C:\ JKS\ AppName _ Environment.p12" -srcstoretype pkcs12 -srcstorepass "SourceStorePasword” -destkeystore "C:\HomesiteJKS\BungalowFinal.jks" -deststoretype JKS -storepass "JKSPassword"
Repeat the above step for all the remaining certs by changing the required parameter values
Once we have the final JKS, we can start changing password for each of the cert in JKS and make it same as JKS(keystore) password. See the below command:
keytool -keypasswd -alias " AppName_Environment" -keystore “C:\JKS\Final.jks"
Note for step 6: It will ask you to enter keystore password, existing cert password, new password and repeat new password
Repeat this step for all the cert entries of JKS file to have JKS password for all the certs
Now we can use this file and add below entries in system.properties of JMeter:
javax.net.ssl.keyStore=C:/JKS/Final.jks
javax.net.ssl.keyStorePassword=JKSPassword
Add the below entry either in user.properties or jmeter.properties of JMeter, which will allow us to use multiple certs from JKS file:
https.use.cached.ssl.context=false
Change Http Request implementation to HttpClient4
Add Keystore configuration element to JMeter test with the below values:
a. Preload: True
b. Variable name holding certificate alias: CertAlias
c. Alias Start index: 0
d. Alias End index: 10000
JMeter user defined variables must have CertAlias with the required alias name
All set and we should be able to run the script on different application environments by loading particular cert with alias name

I don't really understand why do you need to use 2 different keystores as Keystore Configuration provides possibility to use several certificates.
Quote from documentation:
To make JMeter use more than one certificate you need to ensure that:
https.use.cached.ssl.context=false is set in jmeter.properties or user.properties
You use either HTTPClient 4 (ADVISED) or HTTPClient 3.1 (DEPRECATED SINCE 3.0) implementations for HTTP Request
If for any reason you still need to use 2 keystores, not 2 certificates from one keystore, unfortunately there is no way to execute a Beanshell Test Element (or anything) prior to Keystore Configuration initialization.
However you should be able to force re-initialization from Beanshell script.
Example code:
import org.apache.jmeter.config.KeystoreConfig;
import org.apache.jmeter.engine.StandardJMeterEngine;
import org.apache.jmeter.testelement.TestPlan;
import org.apache.jorphan.collections.HashTree;
import org.apache.jorphan.collections.SearchByClass;
import java.lang.reflect.Field;
import java.util.Collection;
StandardJMeterEngine engine = ctx.getEngine();
Field test = engine.getClass().getDeclaredField("test");
test.setAccessible(true);
HashTree testPlanTree = (HashTree) test.get(engine);
SearchByClass keyStoreConfigs = new SearchByClass(KeystoreConfig.class);
testPlanTree.traverse(keyStoreConfigs);
Collection keystoreConfigResults = keyStoreConfigs.getSearchResults();
KeystoreConfig keystoreConfig = (KeystoreConfig)keystoreConfigResults.toArray()[0];
keystoreConfig.testStarted();
See How to Use BeanShell: JMeter's Favorite Built-in Component article for advanced information on Beanshell scripting in JMeter.

Problems with deploying HTTPRouter action with a keystore file

I am attempting to create a keystore and truststore file from a self-signed certificate, for use in a HTTPRouter action, deployed within a JBoss ESB. I used openssl to retrieve the certificate of interest and generated a keystore file and a truststore file with the following commands:
keytool -import -alias ejb-ssl -file cert.der -keystore cert.truststore
keytool -import -alias ejb-ssl -file cert.der -keystore cert.keystore -trustcacerts
Before generating the keystore and truststore files, I am converting the certificate to X509 format, otherwise the keytool utility does not work, returning an exception with the message 'input not an x.509 certificate' exception. To convert the certificate of interest, I am using the following command:
openssl x509 -in cert.cer -outform DER -out cert.der
I then copied these files into the my ESB's 'esbcontent/META-INF' folder. Below are the properties I am setting for the HTTPRouter action
#Configurators
configurators=HttpProtocol
#HttpProtocol Config...
protocol-socket-factory=org.jboss.soa.esb.http.protocol.SelfSignedSSLProtocolSocketFactoryBuilder
keystore=/META-INF/keystore/cert.keystore
keystore-passw=password
truststore=/META-INF/truststore/cert.truststore
truststore-passw=password
When I deploy the ESB I am getting the following error:
Caused by: org.jboss.soa.esb.ConfigurationException: Invalid 'keystore' config. Must be valid URL.
Looking at the certificate retrieved from the third party webservice, all URL's look OK. Does anyone have any idea why JBoss would not accept the URL in the generated keystore? I'm starting to tear my hair out on this one!
Also, I have been trying to use the org.apache.commons.httpclient.contrib.ssl.EasySSLProtocolSocketFactory for the protocol-socket-factory. When using this, the ESB deploys OK. However, the HTTPRouter does not seem to send the request to the third party web service. I've used SoapUI to establish the web service is ok, so I think it's a problem with my configuration of the HTTPRouter action.
Any help offered is greatly appreaciated!

This took me an awful long time to figure out, but the solution turned out to be very simple. The path to the keystore file must be absolute. It CANNOT BE RELATIVE! Therefore, replacing
'/META-INF/keystore/cert.keystore' path
with
'C:/dev/server/jboss/jboss-as/server/default/deploy/MyEsb.esb/META-INF/keystore/cert.keystore
Solved the problem!
Leaving the properties file with this absolute path isn't always suitable when wanting to deploy the ESB within various different environments (Windows and Ubuntu). I'm using gradle as my build tool, so I used the ReplaceTokens feature to replace the keystore token with the absolute path required. I suppose you could also copy the keystore file into the deploy directory so it's available for all ESB's who require it.
Hope this helps someone else who comes across this problem. Was a simple solution in the end, but there is no mention anywhere in the docs for the keystore file to be referenced with an absolute path. But, doing that fixed the issue for me.
Thanks