We Keep Coding

Reading custom objects from additional YAML in Spring Boot

I have a task of reading settings from a YAML file in a Spring Boot application. The requirement is that these settings are stored in a specific file, separate from application.yml. The file is called applicationFeatureToggles.yml and is supposed to have contents like these:
features:
- key: feature1
isEnabled: false
description: First feature
- key: feature2
isEnabled: true
description: Second feature
...
What I need to implement right now is to check in my components' code if a specific feature is enabled. To do this, I created a class for a single feature:
#NoArgsConstructor
#Getter
#Setter
public class Feature {
private String key;
private boolean isEnabled;
private String description;
}
then a configuration properties class to store all settings:
#Component
#ConfigurationProperties
#PropertySource(value = "classpath:applicationFeatureToggles.yml", factory = YamlPropertySourceFactory.class)
public class FeatureProperties {
private List<Feature> features;
// Constructor, getters and setters
}
and a service that uses it to check if a feature is enabled:
#Service
#EnableConfigurationProperties(FeatureProperties.class)
public class FeatureService {
#Autowired
private FeatureProperties featureProperties;
// logic that reads required info from featureProperties
}
The class YamlPropertySourceFactory used in FeatureProperties looks like this:
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
Properties properties = factory.getObject();
return new PropertiesPropertySource(resource.getResource().getFilename(), properties);
}
}
As I understand, this should result in FeatureService having access to FeatureProperties populated with data from applicationFeatureToggles.yml, but the data is missing. At the same time, I checked with a breakpoint that YamlPropertySourceFactory is invoked and reads the data, it's present in the properties object before exiting createPropertySource(). So all seems fine with reading properties from the file, but they don't get into the FeatureProperties object.
What else can my code need to populate FeatureProperties?
If it's not possible at all or can cause some other issues, I'd also be thankful for details, as it may help convince the architect to change the approach.
Spring Boot version used: 2.5.6

I have found the cause. One other thing I did in my code was calling FeatureService in an SpEL expression in a #Conditional for another bean. So it seems FeatureService tried to initialize before FeatureProperties. Without this condition, FeatureService gets proper FeatureProperties.
I think I will eventually find a workaround now, but would be thankful if someone knows how to conditionally initialize a bean depending on some ConfigurationProperties if it also involves some logic to process the configs.

Set #Value to use #ConfigurationProperties prefix

I'm using a properties file with the value:
com.abc.cpuUtilization.okThreshold = 0.5
I want to use the following configuration class:
#Component
#ConfigurationProperties(prefix="com.abc")
public class SystemConfiguration{
#Value("${cpuUtilization.okThreshold}")
private Double cpuUtilizationOkThreshold;
// getters and setters of cpuUtilizationOkThreshold
}
}
But I get an exception of Could not resolve placeholder 'cpuUtilization.okThreshold'
When setting #Value to be: "${com.abc.cpuUtilization.okThreshold}" it works, but it makes the code look ugly and cumbersome.
Is there a way to configure this class, so I will not have to write the whole prefix for the #Value annotation?

Spring configuration properties scanning works like package scanning.
The #Value annotation works with the full property name, or a raw string value. And so the value you set should be.
Assume to have the com.abc.cpu-utilization.okThreshold=0.5 property.
Solution 1: your SystemConfiguration modify the prefix and delete the #Value:
#Getter
#Setter
#Configuration
#ConfigurationProperties(prefix = "com.abc.cpu-utilization")
public class SystemConfiguration {
private Double okThreshold;
}
Solution 2: your SystemConfiguration could point to com.abc and contain an inner configuration for the cpu-utilization intermediate package:
#Getter
#Setter
#Configuration
#ConfigurationProperties(prefix = "com.abc")
public class SystemConfiguration {
private CpuUtilizationConfig cpuUtilization;
}
#Data
public class CpuUtilizationConfig {
private Double okThreshold;
}
Note that okThreshold and cpuUtilization directly reflect the property naming we have prior defined. Then, Spring will do the magic :-)
See:
Baeldung - Guide to #ConfigurationProperties in Spring Boot
Baeldung - A Quick Guide to Spring #Value

As thepaoloboi said, The #ConfigurationProperties(prefix="com.abc") annotation will allow you to bind fields via their name. If you specify a prefix of "com.abc" and you have a variable named "cpuUtilization", the value of the variable will be that of the "com.abc.cpuUtilization" property.
The #Value annotation fetches the property with the exact same name. #Value("${cpuUtilization.threshold}") will fetch the property with that exact name. It does not take the prefix into account.

For congiuation propertiestion you just need to add the values in proeprties file with the same key with the variable.
in application.properties.
com.abc.cpuUtilization.okThreshold=123
Your class should be:
#Component
#ConfigurationProperties(prefix="com.abc.cpuUtilization")
public class SystemConfiguration{
private Double okThreshold;
}
}

The #ConfigurationProperties(prefix="com.abc") annotation will allow you to bind fields via their name. As #Olgun YILDIZ stated, if you specify a prefix of "com.abc" and you have a variable named "cpuUtilization" , the value of the variable will be that of the "com.abc.cpuUtilization" property. (In fact, you could even name your variable "cpuutilization" , "cpu_utilization" , "cpu-utilization" or "CPU_UTILIZATION" because of Spring's relaxed rules for binding properties).
The #Value annotation fetches the property with the exact same name. #Value("${cpuUtilization.threshold}") will fetch the property with that exact name. It does not take the prefix into account.
Either you do as #Olgun suggested (prefix of "com.abc.cpuUtilization" and variable name "okThreshold" ) or you set the whole property name in the #Value.

#ConfigurationProperties works best with hierarchical properties that all have the same prefix therefore, you add a prefix of com.abc.cpuUtilization.Your POJO class Should be like
#Configuration
#ConfigurationProperties(prefix="com.abc.cpuUtilization")
public class SystemConfiguration{
private Double okThreshold;
public Double getOkThreshold() {
return okThreshold;
}
public void setOkThreshold(Double okThreshold) {
this.okThreshold = okThreshold;
}
}
}
If you don't use #Configuration in the POJO, then you need to add #EnableConfigurationProperties(ConfigProperties.class) in the main Spring application class to bind the properties into the POJO.
Then you can set value in application.properties
com.abc.cpuUtilization.okThreshold=0.5
Refer:https://www.baeldung.com/configuration-properties-in-spring-boot

First of all, change your "cpuUtilization.okThreshold" value to com.abc.cpuUtilization-okThreshold = 0.5
Then in the config class:
#Component
#ConfigurationProperties("com.abc")
public class SystemConfiguration{
private Double cpuUtilizationOkThreshold;
// getters and setters of cpuUtilizationOkThreshold
}
}
Try to configure without using "prefix" and without using the #Value annotation, it worked for me.

Does Spring follow any naming convention for application configuration?

Issue :
Configuration defined in application.properties is not overridden by environment variable.
I'm facing strange issue with spring configuration as configuration defined in application.properties is not overridden by environment variable when configuration is named in specific way. As mentioned in Externalized Configuration OS environment variables takes precedence over application.properties but this doesn't happen when configuration is defined as myExternal_url but it works when configuration is defined as my_external_url (in sample code below, we need to change configuration to my_external_url in ApplicationProperties.java and application.properties)
Sample Code -
#SpringBootApplication
public class ConfigApplication implements ApplicationRunner {
#Autowired private ApplicationProperties applicationProperties;
public static void main(String[] args) {
SpringApplication.run(ConfigApplication.class, args);
}
#Override
public void run(ApplicationArguments arg0) {
System.out.println("External URL = " + applicationProperties.getMyExternalUrl());
}
}
Application Bean configuration -
#Configuration
public class AppConfig {
#Bean
#ConfigurationProperties(prefix = "")
public ApplicationProperties applicationProperties() {
return new ApplicationProperties();
}
}
ApplicationProperties class -
public class ApplicationProperties {
#Value("${myExternal_url}")
private String myExternalUrl;
public String getMyExternalUrl() {
return this.myExternalUrl;
}
public void setMyExternalUrl(String myExternalUrl) {
this.myExternalUrl = myExternalUrl;
}
}
application.properties:
myExternal_url=external_url_env_application_properties
What could be reason for this ?
EDIT - adding gradle
Gradle configuration
plugins {
id 'org.springframework.boot' version '2.4.0-M1'
id 'io.spring.dependency-management' version '1.0.9.RELEASE'
id 'java'
}
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
compileOnly 'org.projectlombok:lombok:1.18.6'
annotationProcessor 'org.projectlombok:lombok:1.18.6'
}
test {
useJUnitPlatform()
}
EDIT 2
Trace log shows that myExternal_url is resolved correctly from Environment variable. Then Spring tries to resolve autowired dependencies 'applicationProperties' by calling AutowiredAnnotationBeanPostProcessor and then value is overridden by application.properties value (screen shot).
o.s.c.e.PropertySourcesPropertyResolver : Found key 'myExternal_url' in PropertySource 'systemEnvironment' with value of type String
o.s.c.e.PropertySourcesPropertyResolver : Found key 'myExternal_url' in PropertySource 'environmentProperties' with value of type String

TL;DR #Value has correct myExternal_Url from system variables injected, but its value is later set by #ConfigurationProperties.
The trace log is correct in that the ordering from Spring will place systemEnvironment before classpath:/application.properties in the propertySource list.
The issue you are having is because of a combination of using both #Value and #ConfigurationProperties to inject/bind your properties.
Say these are the values you provide:
system:
myExternal_Url: foo
my_external_url: bar
applications.properties:
myExternal_url: aaa
In your ApplicationProperties:
#Value("${myExternal_url}")
private String myExternalUrl; // injected with foo
myExternalUrl is injected correctly with value (foo) you defined in your Environment Variables. However, #ConfigurationProperties binds the values after by using the setter methods. Since it uses relaxed bindings, it checks for different variations of myExternalUrl, it first looks for what's in your system variables and finds that myExternal_url(both camel and Underscore) isn't in one of relaxed binding forms , but then my_external_url (underscore only) is. So my_external_url's value is provided to setter:
public void setMyExternalUrl(String myExternalUrl) { // bar
this.myExternalUrl = myExternalUrl; // myExternalUrl is reassigned from foo to bar
}
So it should be clear that your #Value would always be overridden since #ConfigurationProperties binds values after. Simply have:
public class ApplicationProperties {
private String myExternalUrl;
...
then have one of binding forms - MY_EXTERNAL_URL, my-external-url, or my_external_url(Maybe there is more) - defined in your system. Then have consistent case in your application.yml in case you don't want your system variables.
my-external-url=aaa
Side Note. It is recommended that you use the form MY_EXTERNAL_URL as system environment variables.

I cannot reproduce your issue in 2.3.1.RELEASE and is working as expected and is overridden by my environment variable.
However I don't see the spring document where it says, #Value("${myExternal_url}") use relaxed binding as well. All I see is #ConfigurationProperties will use relaxed binding.
https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-external-config-relaxed-binding
So I would update your class as follows (even through it works with and without in 2.3.1.RELEASE) #Value annotation removed
public class ApplicationProperties {
private String myExternalUrl;
public String getMyExternalUrl() {
return this.myExternalUrl;
}
public void setMyExternalUrl(String myExternalUrl) {
this.myExternalUrl = myExternalUrl;
}
}
Please note, both properties will be still present in the Environment but when it is binding, it seems to decide if one overrides the other. You can see this by printing. Here I am getting the environment from applicationContext but you can autowire Environment and test this
System.out.println(context.getEnvironment()
.getProperty("myExternal_url"));
System.out.println(context.getEnvironment()
.getProperty("my_external_url"));
To confirm the above point that only #ConfigurationProperties support relaxed binding but not #Value. I created the following class
#Component
public class ValueInjection {
// This prints application properties.
#Value("${myExternal_url}")
private String myExternal_url;
// This prints the environment variable
#Value("${my_external_url}")
private String my_external_url;
// If you uncomment this the application will not start saying
// there is no such property.
//
// #Value("${myExternalUrl}")
// private String myExternalUrl;
// ... getters and setters
}

Spring indeed supports some sort of naming convention in the form of relaxed binding (as pointed by #Kavithakaran Kanapathippillai). In case of #ConfigurationProperties and #Value annotation, Spring tries to resolve variable from Configuration Source in the order as defined in Externalized Configuration using relaxed binding.
As answered #夢のの夢 - spring correctly match property using #Value("${myExternal_url}") with environment variable but later overridden by #ConfigurationProperties(prefix = "").
As bean ApplicationProperties is defined as #ConfigurationProperties(prefix = ""), Spring tries to match variable with configuration source (using relaxed binding) and finds match in application.propertied variable myExternalUrl and overrides property resolved using #Value("${myExternal_url}"). Problem lies in using both #Value and #ConfigurationProperties(prefix = "").
Side note - #Value supports limited relaxed binding and Spring recommends #Value property names to be defined using kebab-case.
From documentation -
If you do want to use #Value, we recommend that you refer to property
names using their canonical form (kebab-case using only lowercase
letters). This will allow Spring Boot to use the same logic as it does
when relaxed binding #ConfigurationProperties. For example,
#Value("{demo.item-price}") will pick up demo.item-price and
demo.itemPrice forms from the application.properties file, as well as
DEMO_ITEMPRICE from the system environment. If you used
#Value("{demo.itemPrice}") instead, demo.item-price and DEMO_ITEMPRICE
would not be considered.

Read environment variable in SpringBoot

What is the best way to read environment variables in SpringBoot?
In Java I did it using:
String foo = System.getenv("bar");
Is it possible to do it using #Value annotation?

Quoting the documentation:
Spring Boot allows you to externalize your configuration so you can work with the same application code in different environments. You can use properties files, YAML files, environment variables and command-line arguments to externalize configuration. Property values can be injected directly into your beans using the #Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via #ConfigurationProperties.
So, since Spring boot allows you to use environment variables for configuration, and since Spring boot also allows you to use #Value to read a property from the configuration, the answer is yes.
For example, the following will give the same result:
#Component
public class TestRunner implements CommandLineRunner {
#Value("${bar}")
private String bar;
private final Logger logger = LoggerFactory.getLogger(getClass());
#Override
public void run(String... strings) throws Exception {
logger.info("Foo from #Value: {}", bar);
logger.info("Foo from System.getenv(): {}", System.getenv("bar")); // Same output as line above
}
}

You can do it with the #Value annotation:
#Value("${bar}")
private String myVariable;
You can also use colon to give a default value if not found:
#Value("${bar:default_value}")
private String myVariable;

Here are three "placeholder" syntaxes that work for accessing a system environment variable named MY_SECRET:
#Value("${MY_SECRET:aDefaultValue}")
private String s1;
#Value("#{environment.MY_SECRET}")
private String s2;
#Value("${myApp.mySecretIndirect:aDefaultValue}") // via application property
private String s3;
In the third case, the placeholder references an application property that has been initialized from the system environment in a properties file:
myApp.mySecretIndirect=${MY_SECRET:aDefaultValue}
For #Value to work, it must be used inside a live #Component (or similar). There are extra gochas if you want this to work during unit testing -- see my answer to Why is my Spring #Autowired field null?

Alternatively, you can use the org.springframework.core.env.Environment interface to access environment variables:
import org.springframework.core.env.Environment;
#Autowired
private Environment env;
//...
System.out.println(env.getProperty("bar"));
Read more...

Yes, you can. However, most answer didn't mention, the ordering is very important, please check this https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html
Your OS environment variables will overwrite the value come from Application properties packaged inside your jar (application.properties and YAML variants)., so basically, your environment variables has higher priority.

you can use it with The #Value annotation for the #Components and #service class
Some times it won't work if it is a normal class
Example:
#Component
public class Testclass{
#Value("${MY_SECRET:aDefaultValue}")
private String test1;
#Value("#{environment.MY_SECRET}")
private String test1;
#Value("${myApp.mySecretIndirect:aDefaultValue}")
private String test1;
//to get the properties list whih are in "," seperated
#Value("${my.list.of.strings}")
private List<String> myList;
}

You can place your environment variable in an application.yml/application.properties file and then you can fetch the value using the #Value annotation.
But in order to use #Value annotation your class should be a bean and should be annotated with #Component annnotation.
You can also provide a default value for the variable.
#Component
#NoArgsConstructor
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
public class MyClass {
#Value("${something.variable:<default-value>}")
private String myEnvVariable;
}

First, you have to define the relevant field information in the properties configuration file, and then use # value to obtain and use
example:
#Value("${abc}")
private String abc;

Spring properties that depend on other properties

I would like to have properties, that I can reference via #Value in spring beans, that can only be created dependend on other properties.
In particular I am having a property, that describes the file system location of a directory.
myDir=/path/to/mydir
And by convention, there is a file in that directory, that is always called myfile.txt.
Now i want to have access to both, the directory and the file, via #Value annotations inside my beans. And sometimes I want to access them as Strings, sometimes as java.io.Files and sometimes as org.springframework.core.io.FileSystemResource (which by the way works very well out of the box!). But because of that concatenating Strings on demand is not an option.
So what I of course could do is just declare both, but I would end up with
myDir=/path/to/mydir
myFile/path/to/mydir/myfile.txt
and I would like to avoid that.
So I came up with an #Configuration class, that takes the property and adds it as new PropertySource:
#Autowired
private ConfigurableEnvironment environment;
#Value("${myDir}")
private void addCompleteFilenameAsProperty(Path myDir) {
Path absoluteFilePath = myDir.resolve("myfile.txt");
Map<String, Object> props = new HashMap<>();
props.put("myFile, absoluteFilePath.toString());
environment.getPropertySources().addFirst(new MapPropertySource("additional", props));
}
As you can see, in my context I even created a PropertyEditor, that can convert to java.nio.file.Paths.
Now the problem is, that for some reason, this "works on my machine" (in my IDE), but does not run on the intended target environment. There I get
java.lang.IllegalArgumentException: Could not resolve placeholder 'myFile' in string value "${myFile}"

Spring can combine properties
myDir=/path/to/mydir
myFile=${myDir}/myfile.txt
You can also use a default value without defining your myFile in the properties at first:
Properties file
myDir=/path/to/mydir
In class:
#Value("#{myFile:${myDir}/myfile.txt}")
private String myFileName;

Spring expressions can be used to refer the properties.
In my example it was
query-parm=QueryParam1=
query-value=MyParamaterValue
Now while binding them in Spring Bean.
#Configuration
public class MyConfig {
#Value("${query-param}${query-value}")
private String queryString;
}
Above code will inject QueryParam1=MyParamaterValue to the variable queryString.