I have some java services which use environment variables for config values.
I'd like to migrate them to use Spring Cloud config instead of environment variables.
Currently, my config is all in application.yml files, as the following:
someKey: ${SOME_KEY_ENV_VAR}
If I were to migrate to using Spring Cloud Config, how would I modify the above line to load its value from the cloud config server, instead of environment variables? (Assuming I've separately setup the maven dependencies & other configuration, to hook them up)
All examples of cloud config clients only show java code, e.g:
#Value("${someKey}")
private String someKey
Is that enough, or will I also need to make any changes to the yaml?
What about things like datasource URLs which don't have a corresponding #Value but are only defined in yaml?
Related
I am using spring boot 2.4 and application related properties are stored in spring cloud config server. It works fine and I am able to read all properties in the application. Below properties have been configured in application.properties for this purpose.
spring.application.name=app-prop-config
spring.cloud.config.label=61465
spring.cloud.config.enable=true
spring.config.import=configserver:https://vmcloud-configsvc.farm-dev.ab.com
The above properties translates to: https://vmcloud-configsvc.farm-dev.ab.com/61465/app-prop-config-dev.properties
Per my requirement, I need to read few more properties as well and these properties are already available in another spring cloud config server which can be accessed using:
spring.application.name=common-prop-config
spring.cloud.config.label=61468
spring.cloud.config.enable=true
spring.config.import=configserver:https://vmcloud-common-configsvc.farm.ab.com
The above properties translates to: https://vmcloud-common-configsvc.farm.ab.com/61468/common-prop-config-dev.properties
The above config server(https://vmcloud-common-configsvc.farm.ab.com) properties have been used by multiple applications and duplicating properties into my config server(https://vmcloud-configsvc.farm-dev.ab.com) would cause maintenance issue in future as any change in properties have to get updated in 2 servers.
Is it possible to use above 2 spring cloud config servers in spring boot app so that I don't have to copy required properties into my existing config server?
Hi it's working for me when i use bootstrap first config. You can put multiple config name that you want to load. For example if you want to retreive user-service.properties et data-rest.properties in the same app your bootstrap.properties should be like:
spring.cloud.config.uri= http://config-server-host
spring.cloud.config.name= user-service, data-rest
I am working on Java Springboot with MongoDB using Kubernetes. Currently I just hard coded the URI in application properties and I would like to know
how can I access to the MongoDB credentials on Kubernetes with Java?
The recommended way of passing credentials to Kubernetes pods is to use secrets and to expose them to the application either as environment variables, or as a volume. The link above describes in detail how each approach works.
If I properly understood the question, it is specifically about Java Spring Boot applications running on Kubernetes.
Few options come to my mind...some not that secure or exclusive to running on Kubernetes but still mentioned here:
Environment variables with values in the deployment/pod configuration. Everyone with access to the configuration will be able to see them.
Use ${<env-var>} / ${<end-var>:<default-value>} to access the environment variables in Spring Boot's application.properties/.yaml file. For example, if DB_USERNAME and DB_PASSWORD are two such environment variables:
spring.data.mongodb.username = ${DB_USERNAME}
spring.data.mongodb.password = ${DB_PASSWORD}
...or
spring.data.mongodb.uri = mongodb://${DB_USERNAME}:${DB_PASSWORD}#<host>:<port>/<dbname>
This will work regardless whether the application uses spring.data.mongodb.* properties or properties with custom names injected in a #Configuration class with #Value.
Based on how the Java application is started in the container, startup arguments can be defined in the deployment/pod configuration, similarly to the bullet point above.
Environment variables with values populated from secret(s). Access the environment variables from SpringBoot as above.
Secrets as files - the secrets will "appear" in a file dynamically added to the container at some location/directory; it would require you to define your own #Configuration class that loads the user name and password from the file using #PropertySource.
The whole application.properties could be put in a ConfigMap. Notice that the properties will be in clear text. Then populate a Volume with the ConfigMap so that application.properties will be added to the container at some location/directory. Point Spring Boot to that location using spring.config.location as env. var, system property, or program argument.
Spring Cloud Vault
Some other external vault-type of secure storage - an init container can fetch the db credentials and make them available to the Java application in a file on a shared volume in the same pod.
Spring Cloud Config...even though it is unlikely you'd want to put db credentials in its default implementation of the server storage backend - git.
I have multiple microservices which communicates with each other through REST calls.
I have used spring boot and spring rest and have configured the URLS of the rest end points in application.properties file.
Now the problems is if the URL for one end point changes then I to have to manually modify all the property files of the services which are calling that particular end point which has got changed.
Is there a workaround for this so that the URLS can be somehow placed in a centralized location so that any modification does not impacts the other services which are using it.
You can use spring-cloud to achieve this. Usual way used in spring-cloud is by configuring the required properties in a git repo. And then those properties can be accessed by any micro-service you want with minimal configurations. You can refer projects in this repo
limits-services acts as a client that needs certain properties those are configured in spring-cloud-config-server. Hope this helps.
In case with microservices you can use Spring Cloud Config (Spring Cloud Config, Spring Cloud Config Server). It's very usefull and you can update your configuration at runtime.
Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system. With the Config Server you have a central place to manage external properties for applications across all environments. The concepts on both client and server map identically to the Spring Environment and PropertySource abstractions, so they fit very well with Spring applications, but can be used with any application running in any language. As an application moves through the deployment pipeline from dev to test and into production you can manage the configuration between those environments and be certain that applications have everything they need to run when they migrate.
As others have mentioned you can use Spring Cloud Config Server to remotly load your application configuration. All you need is git repository containing your configuration.
Spring cloud configuration supporst Git, database as your store for configuration.
Idea is to create an spring-boot app that can provide configuration to other applications.
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
You can configurae port and provide your git repository using key spring.cloud.config.server
server.port: 8888
spring.cloud.config.server.git.uri: file://${user.home}/config-repo
At client side, if you have spring-config in your classpath, application will try to connect to an application runnign at port 8888 to retrieve configuration.
More information can be found here.
may put configuration inside a database.
after that need have one centralize cache service that used by other services, can be .jar service,
then the values can be load inside a cache class in this service,
then in the front end side need have update button for updating the cache after modify the URL value in the database, so then all impacted services can use new value.
and also to be easier may have stand alone UI for update those configuration rather than updating database directly.
You can use Microconfig.IO to manage your service configuration and it's placeholders functionality to reference configuration values of certain services from others. So in your case you configure your deploy url in your server and put placeholders on it in your clients. This allows you to edit value only in one place and then everyone who depend on it will get it automatically.
I have a small API running on PCF using Spring JPA. Of course, within the code, I could use a JDBC connection running prepared statements to access a bound MySQL instance. Doing this requires a username and password, as per normal standards when connecting to a database via Java.
However, with Spring JPA, I don't have to do any of this. I simply bind the MySQL instance and can perform my queries using the JPA API.
For lack of a better question, what is this magic?
Cloudfoundry with Spring Cloud follows 12-factor app patterns through out.
For configuration also it uses the config pattern suggested by 12-factor app patterns.
According to this pattern we should be storing properties outside the code in the environment as environment variables. So that application bundle can be deployed to any environment once it's built without any modifications. Since it picks up configuration from the environment variables, different environments have to define same environment variables with the different values.
Whenever you add a service to your application using cf bind-service Cloudfoundry sets predefined environment variables related to that service in the virtual machine (or container or whatever it has).
You can check these environment variables using cf env app-name.(Command Refeference)
Sample output of cf env app-name
{
"VCAP_APPLICATION": {
"application_id": "fa05c1a9-0fc1-4fbd-bae1-139850dec7a3",
"application_name": "my-app",
"application_uris": [
"my-app.10.244.0.34.xip.io"
],
"application_version": "fb8fbcc6-8d58-479e-bcc7-3b4ce5a7f0ca",
"limits": {
"disk": 1024,
"fds": 16384,
"mem": 256
},
"name": "my-app",
"space_id": "06450c72-4669-4dc6-8096-45f9777db68a",
"space_name": "my-space",
"uris": [
"my-app.10.244.0.34.xip.io"
],
"users": null,
"version": "fb8fbcc6-8d58-479e-bcc7-3b4ce5a7f0ca"
}
Using the spring actuator endpoints you can inspect all environment variables using /env endpoint. It lists more properties than cf env.
When spring detects that
cloud profile is active (set by spring.profiles.active environment property, or spring.profile property in spring cloud)
Auto Configuration is enabled (enabled by #SpringBootApplication)
No in memory Datasource dependency is present on the classpath (though I assume it would give cloud datasource configuration preference, even if in memory dependency were present)
No data source has been explicitly configured
Spring creates the Datasource bean itself using environment variables if a datasource service (like Postgres) has been bound to application.
Below is the link for the environment properties that it uses for creating Datasource.
https://docs.cloudfoundry.org/buildpacks/java/spring-service-bindings.html
Here is a list of Datasource only properties.
cloud.services.<database-service-name>.connection.hostname
cloud.services.<database-service-name>.connection.name
cloud.services.<database-service-name>.connection.password
cloud.services.<database-service-name>.connection.port
cloud.services.<database-service-name>.connection.username
cloud.services.<database-service-name>.plan
cloud.services.<database-service-name>.type
database-service-name is defined in the Manifest.yml file in the env: block
In my experience if there's only one database service added to the application, there was no need to define the database service name in the environment variables section.
Note: By default spring would try to use the servlet container's poolable connection support, however most of the time we our self have to configure some properties that are only supported by connection pool providers like Apache DBCP. In these cases we have to create Datasource bean manually using environment properties (using System.getProperty() or spring Environment.getProperty()).
I have a Spring application with xml configuration (v4.0.8) where I need to access properties dynamically rather than using #Value annotation. I tried using tho methods for this, one of them is using #ConfigurationProperties with a Map which gives me all properties in a map, and the other way is using Environment.getProperty.
Both methods are getting the properties from the propertySources of the Environment. Unfortunately that contains 5 property sources including system properties, etc, but not my properties files. Therefore I cannot access my properties.
I'm adding my properties using EncryptablePropertySourcesPlaceholderConfigurer from jasypt which is a simple implementation of PropertySourcesPlaceholderConfigurer that decrypts encrypted property values. PropertySourcesPlaceholderConfigurer does not add properties to Environment.propertySources and I couldn't figure out how can I extend it myself rather than using jasypt implementation and add them to property sources manually.
Two notes:
#Value annotation works fine, because it's not using Environment but goes through configurers during bean creation. There's not problem with that.
I have a spring boot application where I cold achieve my goal appending to Environment.propertySources by listening to ApplicationEnvironmentPreparedEvent of spring boot and adding my properties to Environment rather than implementing PropertySourcesPlaceholderConfigurer. But this is only applicable to spring boot applications and my legacy application is not a spring boot app.