I am working on Java Springboot with MongoDB using Kubernetes. Currently I just hard coded the URI in application properties and I would like to know
how can I access to the MongoDB credentials on Kubernetes with Java?
The recommended way of passing credentials to Kubernetes pods is to use secrets and to expose them to the application either as environment variables, or as a volume. The link above describes in detail how each approach works.
If I properly understood the question, it is specifically about Java Spring Boot applications running on Kubernetes.
Few options come to my mind...some not that secure or exclusive to running on Kubernetes but still mentioned here:
Environment variables with values in the deployment/pod configuration. Everyone with access to the configuration will be able to see them.
Use ${<env-var>} / ${<end-var>:<default-value>} to access the environment variables in Spring Boot's application.properties/.yaml file. For example, if DB_USERNAME and DB_PASSWORD are two such environment variables:
spring.data.mongodb.username = ${DB_USERNAME}
spring.data.mongodb.password = ${DB_PASSWORD}
...or
spring.data.mongodb.uri = mongodb://${DB_USERNAME}:${DB_PASSWORD}#<host>:<port>/<dbname>
This will work regardless whether the application uses spring.data.mongodb.* properties or properties with custom names injected in a #Configuration class with #Value.
Based on how the Java application is started in the container, startup arguments can be defined in the deployment/pod configuration, similarly to the bullet point above.
Environment variables with values populated from secret(s). Access the environment variables from SpringBoot as above.
Secrets as files - the secrets will "appear" in a file dynamically added to the container at some location/directory; it would require you to define your own #Configuration class that loads the user name and password from the file using #PropertySource.
The whole application.properties could be put in a ConfigMap. Notice that the properties will be in clear text. Then populate a Volume with the ConfigMap so that application.properties will be added to the container at some location/directory. Point Spring Boot to that location using spring.config.location as env. var, system property, or program argument.
Spring Cloud Vault
Some other external vault-type of secure storage - an init container can fetch the db credentials and make them available to the Java application in a file on a shared volume in the same pod.
Spring Cloud Config...even though it is unlikely you'd want to put db credentials in its default implementation of the server storage backend - git.
Related
I have some java services which use environment variables for config values.
I'd like to migrate them to use Spring Cloud config instead of environment variables.
Currently, my config is all in application.yml files, as the following:
someKey: ${SOME_KEY_ENV_VAR}
If I were to migrate to using Spring Cloud Config, how would I modify the above line to load its value from the cloud config server, instead of environment variables? (Assuming I've separately setup the maven dependencies & other configuration, to hook them up)
All examples of cloud config clients only show java code, e.g:
#Value("${someKey}")
private String someKey
Is that enough, or will I also need to make any changes to the yaml?
What about things like datasource URLs which don't have a corresponding #Value but are only defined in yaml?
I'm curious on how the #Value works internally on Spring so that it can actually read value from ConfigMap of Kubernetes Cluster.
I know that:
#Value("${my.nested.variable}") were used to access variables
declared on application.properties or in OS' environment variable
(higher priority).
When creating new ConfigMap on kubernetes (for Spring project), you usually do kubectl create configmap my-config-name --from-file=application.properties, and it will magically connect those ConfigMap values with respective #Value() on Spring, of course we have to select my-config-name on deployment YAML file.
Notice above that we didnt expose/map those configmap to container's environment variable, already checked the inside container with printenv , can't find it.
However, Spring were still able to retrieve those value from ConfigMap to be used in java program.
How is it possible? anyone know how Spring's #Value works or how the ConfigMap actually works internally so those two can magically connected?
Thank You.
This is a community wiki answer. Feel free to expand it.
As already mentioned by David Maze in the comments, Spring Cloud Kubernetes is reading ConfigMaps by using Kubernetes API. The mechanisms behind it are described in the linked docs.
Due to some new security requirments the api I'm developing now is required to store several urls, azure account names etc. in the azure key vault, rather than in the application.yml config file.
The issue is that I'm having trouble authenticating / accessing the key vault client in a Local environment. I have very limited access to the azure functions / key vault itself so testing the new code I'm writing is near impossible at current:
public String getSecretFromKeyVault(String key) {
/**
* Breaks in the constructor call, as the system.env variables for MSI_ENDPOINT and MSI_SECRET are null.
**/
AppServiceMSICredentials credentials = new AppServiceMSICredentials(AzureEnvironment.AZURE);
KeyVaultClient client = new KeyVaultClient(credentials);
SecretBundle secret = client.getSecret("url-for-key-vault", key);
return secret.value();
}
I'm aware that the variables will be set in the cloud server, but my question is how can I best verify that the vault calls have been implemented properly(unit, integration, e2e local tests), and how would I manage to use key vault calls during local development / runtime?
The alternative to MSI would be to enter the client id and key manually, following authentication against the active directory. This could be a solution for local development, but Would still require the declaration of confidential information in the source code.
Ive also tried logging in to azure using az login before running the server but that didn't work either.
Does anyone have any suggestions on how I might resolve this issue, or what my best options are going forward?
Notes on application:
Java version: 8
Spring boot
Azure / vsts development and deployment environment
Since you're using spring-boot you may be better off using Microsoft's property source implementation that maps the keyvault properties into Spring properties and for local development and testing you set equivalent properties in property files.
Use Spring profiles. let's say you have azure and local profiles. In your application-azure.yml file configure your app to use keyvault:
# endpoint on the azure internal network for getting the identity access token
MSI_ENDPOINT: http://169.254.169.254/metadata/identity/oauth2/token
MSI_SECRET: unused
# this property triggers the use of keyvault for properties
azure.keyvault:
uri: https://<your-keyvault-name>.vault.azure.net/
Now you can inject secret properties from the spring context into your variables and they will be read from keyvault:
#Value("${superSecretValue}")
String secretValue;
To make this work locally for testing, in your application-local.yml file you can set the secret property to something appropriate:
superSecretValue: dummy-for-testing-locally
The Azure dependency you need to add to build.gradle is:
implementation "com.microsoft.azure:azure-keyvault-secrets-spring-boot-starter:2.1.6"
Run your spring-boot jar with azure as the active profile when deployed, and local when testing and developing away from azure. This is tested and working with azure java containers.
I have simple Spring boot application which I need to deploy on development and prod different namespaces on a Kubernetes cluster using Helm.
I was thinking about keeping multiple application.properties (application-dev.properties, application-prod.properties) files for each environment and then create configmaps from them through values.yaml files which also will be different for each environment and specified when I execute Helm upgrade.
Now the question is how do I consume values from config.maps as I understand I can either mount the properties file inside container for example /deployment/application.properties
Or expose each property as an environment variable inside container.
But how do I consume them from Java application?
Also at the moment when I create container image it has current application .properties inside /resources/ files embedded and this is what application is using from default so I need to overwrite this behaviour when application is running inside container as opposite to then when its just build and run manually on developer desktop.
Springboot can automatically infer variables from environment variables. In your application.properties or application.yaml, just use ${MY_ENVIRONMENT_VARIABLE:my-defaultvalue}.
Use helm to populate your configmap.
Use configmap as environment variables into your deployment manifest.
This way you do not need to have multiple application.properties for dev, int ,prod inside your image. Keeping it intact across deployment.
And then in your helm chart, you can have multiple values.yaml example values-dev.yaml or values-int.yaml. you can also dynamically set helm values from command line, overriding the yaml file.
I have a demo app in github https://github.com/balchua/demo, which uses this pattern.
You could certainly use environment variables as Bal Chua suggests. If you do that you can override particular values at install time using --set or if you've a lot of config you can use the '-- values' flag and pass in a custom values.yaml file.
Another approach is to load a whole file using .Files.Glob (example in github) and load the file as part of the chart. You can then mount the file to /config to consume it in your spring boot application. Then your config file would be in the same form as a Spring boot config file, rather than a helm values.yaml. Though in many cases there needn't be much difference.
There's a discussion of how you could do similar for secrets (presumably you'll want to put your passwords in secrets) and use it for CI/CD in https://dzone.com/articles/hunting-treasure-with-kubernetes-configmaps-and-se (which is the article accompanying the github example). Basically you would use .Files.Glob with .AsSecrets instead of .AsConfig so as to encode the content. Many helm charts have the option to generate a random password if not specified but I'd guess you probably don't need that.
I'd recommend mounting the files (application.properties or application.yml) inside the ConfigMap onto somewhere on the file system that Spring Boot can automatically detect - then your app stays nice and simple
I have a small API running on PCF using Spring JPA. Of course, within the code, I could use a JDBC connection running prepared statements to access a bound MySQL instance. Doing this requires a username and password, as per normal standards when connecting to a database via Java.
However, with Spring JPA, I don't have to do any of this. I simply bind the MySQL instance and can perform my queries using the JPA API.
For lack of a better question, what is this magic?
Cloudfoundry with Spring Cloud follows 12-factor app patterns through out.
For configuration also it uses the config pattern suggested by 12-factor app patterns.
According to this pattern we should be storing properties outside the code in the environment as environment variables. So that application bundle can be deployed to any environment once it's built without any modifications. Since it picks up configuration from the environment variables, different environments have to define same environment variables with the different values.
Whenever you add a service to your application using cf bind-service Cloudfoundry sets predefined environment variables related to that service in the virtual machine (or container or whatever it has).
You can check these environment variables using cf env app-name.(Command Refeference)
Sample output of cf env app-name
{
"VCAP_APPLICATION": {
"application_id": "fa05c1a9-0fc1-4fbd-bae1-139850dec7a3",
"application_name": "my-app",
"application_uris": [
"my-app.10.244.0.34.xip.io"
],
"application_version": "fb8fbcc6-8d58-479e-bcc7-3b4ce5a7f0ca",
"limits": {
"disk": 1024,
"fds": 16384,
"mem": 256
},
"name": "my-app",
"space_id": "06450c72-4669-4dc6-8096-45f9777db68a",
"space_name": "my-space",
"uris": [
"my-app.10.244.0.34.xip.io"
],
"users": null,
"version": "fb8fbcc6-8d58-479e-bcc7-3b4ce5a7f0ca"
}
Using the spring actuator endpoints you can inspect all environment variables using /env endpoint. It lists more properties than cf env.
When spring detects that
cloud profile is active (set by spring.profiles.active environment property, or spring.profile property in spring cloud)
Auto Configuration is enabled (enabled by #SpringBootApplication)
No in memory Datasource dependency is present on the classpath (though I assume it would give cloud datasource configuration preference, even if in memory dependency were present)
No data source has been explicitly configured
Spring creates the Datasource bean itself using environment variables if a datasource service (like Postgres) has been bound to application.
Below is the link for the environment properties that it uses for creating Datasource.
https://docs.cloudfoundry.org/buildpacks/java/spring-service-bindings.html
Here is a list of Datasource only properties.
cloud.services.<database-service-name>.connection.hostname
cloud.services.<database-service-name>.connection.name
cloud.services.<database-service-name>.connection.password
cloud.services.<database-service-name>.connection.port
cloud.services.<database-service-name>.connection.username
cloud.services.<database-service-name>.plan
cloud.services.<database-service-name>.type
database-service-name is defined in the Manifest.yml file in the env: block
In my experience if there's only one database service added to the application, there was no need to define the database service name in the environment variables section.
Note: By default spring would try to use the servlet container's poolable connection support, however most of the time we our self have to configure some properties that are only supported by connection pool providers like Apache DBCP. In these cases we have to create Datasource bean manually using environment properties (using System.getProperty() or spring Environment.getProperty()).