I am developing a RESTful service with Dropwizard. Now I need deploy it to different environment(test, staging, prod) & data centers(cn, us, etc). As such, it needs different config for different environment/data-center.
Curious how is this usually handled? I could potentially have one config for each env/dc, but in that case, they will have a lot of duplication of common config.
I was expecting that I have a base config, say base.yaml, containing the common configs; And then each environment&DC will have one config, which extend/override the base yaml with their custom config. Is this possible within Dropwizard?
Thank you!
I don't think DropWizard has that feature, but you can always write a small script that composes a few YAML files and starts DropWizard with the result. You can use a simple template engine like Mustache for it.
config-template.yml:
# ...
server:
applicationConnectors:
- type: http
port: {{PORT}}
config-parms-us.yml:
---
PORT: "8080"
---
And then use:
mustache config-parms-us.yml config-template.yml > config.yml
java ... server config.yml
I think this is what you are looking for:
https://github.com/constretto/constretto-dropwizard
It allows you to have environment specific configuration like this:
database: # the JDBC URL
url: jdbc:oracle:thin:#//oracle-testing:1521/name
.staging.url: jdbc:oracle:thin:#//oracle-staging:1521/name
.production.url: jdbc:oracle:thin:#//oracle-production:1521/name
When you want to start your application with certain environment use:
-DCONSTRETTO_TAGS=$ENVIRONMENT
You can create your own configuration bundle which implements Bundle to add your own configuration settings.
So you also need to create your own ConfigurationSourceProvider, then override the open(String path) method, so here you can do find your configuration basename.
Finally you can implements EnvironmentAware which can help you resolve all your configuration files which is a list of url. And you can set your default name you want to loaded.
Related
Versions
Spring Parent: 2.7.4, Spring Cloud Version: 2021.0.4, Java Version: 11
Issue
My Spring service has been using Eureka to connect to the config server for a long time, but I want to upgrade to Spring 2.7.4. I understand that as of Spring 2.4, the bootstrap context has been deprecated (source) and I need to make some adjustments to the old bootstrap properties and move them over to application.properties.
The documentation for Spring Cloud specifies that in order for me to continue to use discovery-first config lookup, I need to define a spring.config.import property with an optional configserver entry (source). Since I'm also using Vault, I define the property as follows:
spring.config.import = optional:configserver:placeholder,vault://<my-generic-backend>/dev
Next, I need to define the following properties (source). These properties were already defined in my old bootstrap.properties, so all I need to do is copy and paste.
spring.cloud.config.discovery.enabled = true
spring.cloud.config.discovery.serviceId = config-server
eureka.client.serviceUrl.defaultZone = <my-eureka-url>
Unless I'm missing something, these are all the steps I need to take in order to upgrade to 2.7.4. However, when I run the Spring service, it complains that it can't find the config server (via Eureka, or via URL), then it registers successfully with Eureka, and then continues trying and failing to find the config server.
Here is some of the output of the program:
> Running with Spring Boot v2.7.4, Spring v5.3.23
> Could not locate configserver via discovery: No instances found of configserver (config-server)
> Could not locate PropertySource ([ConfigServerConfigDataResource#2aa6311a uris = array<String>['placeholder'], optional = true, profiles = list['local']]): Invalid URL: placeholder
...
> DiscoveryClient_<my-project-name>/local - registration status: 204
I understand why it's failing to find a config server at URL: placeholder since that's not a valid URL, but I don't understand how the service can successfully register with Eureka yet not be able to find the config server. I know the service is registered because the output of the program says it registered correctly (and I can see it in the registry), and I know that the config server has the correct entity ID (config-server) because it was copied and pasted from the old bootstrap (and I can see config-server in the registry).
Workaround with Hardcoded URL
When I hardcode the config server URL like this (and set spring.cloud.config.discovery.enabled to false), the config is loaded properly from the server:
spring.config.import=configserver:https://<my-hardcoded-config-url>.com,vault://<my-generic-backend>/dev
Workaround with Bootstrap
It's possible to return to using the bootstrap context and still use Spring 2.7.4 with discovery-first config lookup by adding the "spring-cloud-starter-bootstrap" dependency. So I added the dependency to my POM and moved these properties back to bootstrap.properties from application.properties.
spring.cloud.config.discovery.enabled=true
spring.cloud.config.discovery.service-id=config-server
I moved the Vault and Eureka properties back into bootstrap.properties as well. The new application.properties now contains no values relating to Eureka, Vault, and Cloud Config.
When I run the service, it does indeed find the address for the config server through Eureka, as expected (although it fails to connect because it's the internal address and I'm running locally).
Conclusion
While these are valid workarounds, it's frustrating to not be able to have a dynamic URL for the config server (as is the entire point of using Eureka). Right now, it looks like my choices are either to use a hard-coded URL and risk having to change every property file, or use a deprecated behavior that Spring documentation specifically disfavors (source).
I would appreciate any guidance you have on the issue, and I thank you in advance.
everyone!
this is going to be my first time pushing a newly developed Spring Boot App and I was wondering if there is a way to protect passwords and other sensitive information written in the application.properties file.
Assuming we have the following lines:
# PostgreSQL connection settings
spring.datasource.jdbc-url=jdbc:postgresql://localhost:5432/bdreminder
spring.datasource.username=username
spring.datasource.password=password
The source code is to be first stored on GitHub and having the credentials stored in plain text does not seem to be a good idea.
So, I could probably add the file to the .gitignore one; I could set some environment variables on the host but how would it populate the .properties file afterward? Also, this seems quite cumbersome in terms of the scaling later on.
So, basically, I am trying to see how it is done in a real-life :)
Please, help :)
Simplest option is to create a profile specific application.properties file and activate that profile. So for example create application-private.properties and activate profile private. Of course you have to watch out to not commit this file.
Alternatively, and probably a safer option, is to define a file outside your project and import it in your application.properties with following line:
spring.config.import=file:../path/to/your/external.properties
Spring Boot has extensive support for external configuration. The usual approach is to use one of environment variables, configuration provided by a platform such as Kubernetes, or a specialized configuration system through Spring Cloud Config; these all keep secrets (or just environment-specific information) entirely outside of the code. They also have the advantages of providing a common style of configuration for other applications that do not use Spring Boot.
I have lots of microservices and they work in a similar format. What my requirement is to make my confiuration.yml files work also for keycloak instead of working with standalone.xml (because of my requirements from our customers). I did not see any option that I can override when I check keycloak’s source code since the configuration is done before SPIs.
What I will go in worst scenario is to run some program to convert given configuration.yml to standalone.xml. I appreciate to hear any ideas about how to achieve this. Basically I want to work with YML instead of XML either natively importing something in keycloak or having some converters.
Maybe you can ask why you need this (even for customers), since all of our microservices works like;
server:
port: 80
we don’t want to force our clients to learn any different logic to edit our microservices. They will do always in same way to change some basic configurations like above and we can convert these values to proper standalone.xml way. So can I achieve this converter natively extending keycloak or what could be the best option?
It depends on what settings you want to extract to yml. Standalone.xml allows you to specify configuration values as properties like
<socket-binding name="http" port="${jboss.http.port:8080}"/>
so you can provide a properties file or prorety value during startup like
./standalone.sh --properties=foo.properties -Dfoo=bar
but in your case you could store all of this properties in yml and use some startup wrapper for keycloak that will parse your yml and expose all its settings as java properties like:
#!/bin/bash
./standalone.sh $(magictool foo.yml)
Apache Cayenne keeps a "...project.xml" file within the resources directory. This file contains the password to the database. This is a problem because the [deployment server] password should not visible to developers. Further, it would need to be a different user/password and connection to a different database during development.
What is the best practice to manage this "...project.xml" when using Cayenne? Any suggestions?
EDIT:
Instead of putting the database connection info (incl. password) into the XML file, is it possible to inject the info programatically into the DataSource object? If so, I can load the info from a config file when the app starts and then inject it.
Yes of course. There is a "cayenne.jdbc.password" property that can be used to define DataSource password in runtime. It can be applied in two alternative ways:
As a system property on command line:
java -Dcayenne.jdbc.password=xxxxx
Via injection:
ServerModule.contributeProperties(binder)
.put(Constants.JDBC_PASSWORD_PROPERTY, "xxxxx");
This and other config properties are documented here.
I am using spring cloud config server to save all the configuration in various environments,upto now, it worked great, but now, I suddenly met an issue, don't knwo how to change it.
By default, the url pattern is: http://xxx.xx.xx:8888/{appName}/{environemnt}
But now I need to deploy multiple service together, and I can no longer use path to the root, and I need to use this one:
http://xxx.xxx.xx/pathToConfig/{appName}/{env}
But I noticed that spring config server will consider as appName instead there.
May I ask if there are any configuration I can change to make sure spring config server be able to consider url only after some path after the root?
Thanks
spring.cloud.config.server.prefix: /<path> only changes the prefix for the config server api, not the whole application.
You can specify server.contextPath: /pathToConfig in your application.yml of other configuration.
Then it will prefix all the mappings with the specified path.