I have a task of reading settings from a YAML file in a Spring Boot application. The requirement is that these settings are stored in a specific file, separate from application.yml. The file is called applicationFeatureToggles.yml and is supposed to have contents like these:
features:
- key: feature1
isEnabled: false
description: First feature
- key: feature2
isEnabled: true
description: Second feature
...
What I need to implement right now is to check in my components' code if a specific feature is enabled. To do this, I created a class for a single feature:
#NoArgsConstructor
#Getter
#Setter
public class Feature {
private String key;
private boolean isEnabled;
private String description;
}
then a configuration properties class to store all settings:
#Component
#ConfigurationProperties
#PropertySource(value = "classpath:applicationFeatureToggles.yml", factory = YamlPropertySourceFactory.class)
public class FeatureProperties {
private List<Feature> features;
// Constructor, getters and setters
}
and a service that uses it to check if a feature is enabled:
#Service
#EnableConfigurationProperties(FeatureProperties.class)
public class FeatureService {
#Autowired
private FeatureProperties featureProperties;
// logic that reads required info from featureProperties
}
The class YamlPropertySourceFactory used in FeatureProperties looks like this:
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
Properties properties = factory.getObject();
return new PropertiesPropertySource(resource.getResource().getFilename(), properties);
}
}
As I understand, this should result in FeatureService having access to FeatureProperties populated with data from applicationFeatureToggles.yml, but the data is missing. At the same time, I checked with a breakpoint that YamlPropertySourceFactory is invoked and reads the data, it's present in the properties object before exiting createPropertySource(). So all seems fine with reading properties from the file, but they don't get into the FeatureProperties object.
What else can my code need to populate FeatureProperties?
If it's not possible at all or can cause some other issues, I'd also be thankful for details, as it may help convince the architect to change the approach.
Spring Boot version used: 2.5.6
I have found the cause. One other thing I did in my code was calling FeatureService in an SpEL expression in a #Conditional for another bean. So it seems FeatureService tried to initialize before FeatureProperties. Without this condition, FeatureService gets proper FeatureProperties.
I think I will eventually find a workaround now, but would be thankful if someone knows how to conditionally initialize a bean depending on some ConfigurationProperties if it also involves some logic to process the configs.
Related
We have a spring boot application with configuration being driven from application.yml file. In this configuration file we use the feature of defining a property by referring to another property inside the same application.yml file:
my-games-app:
base-property: foo
games:
- game-one:
game-name: ${my-games-app.base-property}one
game-location: ${my-games-app.base-property}/one
- game-two:
game-name: ${my-games-app.base-property}two
game-location: ${my-games-app.base-property}/two
And we have a #ConfigurationProperties bean loading games configuration:
#Configuration
#ConfigurationProperties(prefix = "my-games-app.games")
public class GamesConfig {
private Map<String, Game> games;
...
}
Useless to say the above is just an example, in reality it is a very complex setup with GamesConfig bean being used as a constructor argument for many other beans inside our application:
#Component
public class GamesRunner {
private final GamesConfig gamesConfig;
...
}
Everything works as expected. The problem we have is related to testing the beans where GamesConfig is injected; in the above example GamesRunner. At the moment we use #SpringBootTest to get hold of the beans we want to test. This again, works OK but the main inconvenient is that the whole application needs to be started in order to access the GamesConfig bean. This means setting up a lot of infrastructure such as a Database a JMS message broker and a Kafka broker. This takes time and makes our CI builds longer to run which started to become a bit of an inconvenient. Because the beans we want to test don't need any other setup than having the GamesConfig constructor argument provided we would prefer to have unit tests in place rather than integration tests as they are much faster to run.
In other words, we want to be able to recreate GamesConfig by hand by parsing our application.yml with a test helper method. To do this we use snakeyaml library:
public final class TestHelper {
public static GamesConfig getGamesConfig() {
var yaml = new Yaml();
var applicationYaml = (Map<String, Object>) yaml.load(readResourceAsString("application.yml");
return createGamesConfig(applicationYaml.get("games");
}
private static GamesConfig createGamesConfig(Object config) {
// The config Object passed here is a `Map<String, Map<String, String>>`
// as defeined in our `games` entry in our `application.yml`.
// The issue is that game name and game locations are loaded exactly like
// configured without property place holders being resolved
return gamesConfig;
}
}
We resolved the issue by manually parsing the property placeholders and looking up their values in the application.yml file. Even if our own property placeholder implementation is quite generic, my feeling is that this extra work is not needed as it should be a basic expectation the library would have some specific set up to do this out of the box. Being very new to snakeyaml I hope someone else hit the same problem and knows how to do it.
We use snakeyaml because it just happened to be in the class path as a transitive dependency, we are open to any suggestions that would achieve the same thing.
Thank you in advance.
To my knowledge, SnakeYAML only supports substitution of environment variables, which is why what you want is not possible as far as I know. What you can do instead, of course, is simply use Spring's classes without setting up a full ApplicationContext.
For example, assuming your game config from above, you could use:
final var loader = new YamlPropertySourceLoader();
final var sources = loader.load(
"games-config.yml",
new ClassPathResource("games-config.yml")
);
final var mutablePropertySources = new MutablePropertySources();
sources.forEach(mutablePropertySources::addFirst);
final var resolver = new PropertySourcesPropertyResolver(mutablePropertySources);
resolver.setIgnoreUnresolvableNestedPlaceholders(true);
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-location"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-location"));
which outputs:
fooone
foo/one
footwo
foo/two
If you are actually interested in how Spring does it, a good starting point is the source code of the PropertySourcesPlaceholderConfigurer class.
I know there are similar questions like this one: Spring boot / mongo wont create index with the index annotation
Also issues in Github like 'spring.data.mongodb.auto-index-creation=true' not working
And also I've tried this Baeldung guide: Spring Data MongoDB – Indexes, Annotations and Converters
So the problem is I'm trying to add an index to an existing collection using #Indexed anotation like this:
#CreatedDate
#Indexed(name="timestamp_index", expireAfterSeconds=100))
private Date timestamp;
The field is a timestamp which is created automatically when an element is inserted into the DB.
Also the class has #Document tag.
So, what have I tried?
Following other answers first thing I did is to add spring.data.mongodb.auto-index-creation: true in this way:
spring:
data:
mongodb:
uri: ${env.mongo-database.url}
auto-index-creation: true
This not works... but I've also read that the problem can be if I have a AbstractMongoClientConfiguration class.
Currently the project doesn't have that class, but exists a MongoConfiguration class with the #Configuration tag. I don't know if this can interfere or something.
The class is like this:
#Configuration
public class MongoConfiguration { /*creates some beans*/ }
This class create a #Bean named mongodb so I've tried to add manually auto-index to true here:
#Primary
#Bean(name = "mongodb")
#ConfigurationProperties(prefix = "spring.data.mongodb")
public MongoProperties getMongodbProperties() {
MongoProperties mongoProperties = new MongoProperties();
mongoProperties.setAutoIndexCreation(true);
return mongoProperties;
}
But this doesn't work either. The index is not created.
Also this class doesn't extends from AbstractMongoClientConfiguration so I can't override the method autoIndexCreation
Also I can create the index programantically with this code:
mongoTemplate
.indexOps(PagoDto.class)
.ensureIndex(new Index().on("timestamp", Sort.Direction.ASC).expire(100));
But for clearer implementation I'd like to do using only annotations in the model.
So the point is: Neither use auto-index-creation in application properties nor trying to add the value into properties works.
Maybe the configuration class doesn't allow to create automatically? It interferes in any way?
I'm not supposed to be authorized to modify the configuration class, if is a little change like call setAutoIndexCreation method there is no problem but I can't change the logic and extends from AbstractMongoClientConfiguration. So the ideal scenario is to get it to work with the annotation #Indexed.
Thanks in advance
Below is my project structure
SomeProject
-src/main/java
-src/main/resources
-src/test/java
-src/test/resources
application-test.yml
Below are the contents of my properties file
application-test.yml
pre:
someUrl: http://someurl
Below are the contents of the configuration class
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties(prefix = "pre")
public class SomeConfiguration {
private String someUrl;
public String getsomeUrl() {
return someUrl;
}
public void setsomeUrl(String someUrl) {
this.someUrl = someUrl;
}
}
Below is my test class
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes=SomeConfiguration.class)
#TestPropertySource(locations = {"classpath:application-test.yml"})
public class SomeServiceTest {
SomeObject someObject;
#Autowired
private SomeConfiguration someConfiguration;
#Test
public void somMethodTest() {
someObject = new SomeObject ();
someObject.setsomeUrl(someConfiguration.getsomeUrl());
}
}
The problem is I am getting null when I am trying to set someURL in someObject. I have seen similar questions on stackoverflow and accepted answers as well, but I am still getting null.
According to the #ConfigurationProperties documentation:
Getters and setters are usually mandatory, since binding is through
standard Java Beans property descriptors.
A setter for private String sameUrl is setSameUrl and NOT setsameUrl.
So spring may read that from properties file but it cannot inject that through the setter.
Unfortunately yml files are not supported with #TestPropertySource or #PropertySource.
I don't think the documentation for #TestPropertySource is clear on this fact, but the following JIRA has been closed. One of the comments says...
the locations attribute in #TestPropertySource already provides the following documentation:
Supported File Formats
Both traditional and XML-based properties file formats are supported — for example, "classpath:/com/example/test.properties" or "file:/path/to/file.xml".
The following from the spring docs spells it out for #PropertySource:
24.7.4 YAML Shortcomings
YAML files cannot be loaded by using the #PropertySource annotation.
So, in the case that you need to load values that way, you need to use
a properties file.
You can get #PropertySource to load yaml if you create a suitable factory, not sure if you can do it with #TestPropertySource.
What is the best way to read environment variables in SpringBoot?
In Java I did it using:
String foo = System.getenv("bar");
Is it possible to do it using #Value annotation?
Quoting the documentation:
Spring Boot allows you to externalize your configuration so you can work with the same application code in different environments. You can use properties files, YAML files, environment variables and command-line arguments to externalize configuration. Property values can be injected directly into your beans using the #Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via #ConfigurationProperties.
So, since Spring boot allows you to use environment variables for configuration, and since Spring boot also allows you to use #Value to read a property from the configuration, the answer is yes.
For example, the following will give the same result:
#Component
public class TestRunner implements CommandLineRunner {
#Value("${bar}")
private String bar;
private final Logger logger = LoggerFactory.getLogger(getClass());
#Override
public void run(String... strings) throws Exception {
logger.info("Foo from #Value: {}", bar);
logger.info("Foo from System.getenv(): {}", System.getenv("bar")); // Same output as line above
}
}
You can do it with the #Value annotation:
#Value("${bar}")
private String myVariable;
You can also use colon to give a default value if not found:
#Value("${bar:default_value}")
private String myVariable;
Here are three "placeholder" syntaxes that work for accessing a system environment variable named MY_SECRET:
#Value("${MY_SECRET:aDefaultValue}")
private String s1;
#Value("#{environment.MY_SECRET}")
private String s2;
#Value("${myApp.mySecretIndirect:aDefaultValue}") // via application property
private String s3;
In the third case, the placeholder references an application property that has been initialized from the system environment in a properties file:
myApp.mySecretIndirect=${MY_SECRET:aDefaultValue}
For #Value to work, it must be used inside a live #Component (or similar). There are extra gochas if you want this to work during unit testing -- see my answer to Why is my Spring #Autowired field null?
Alternatively, you can use the org.springframework.core.env.Environment interface to access environment variables:
import org.springframework.core.env.Environment;
#Autowired
private Environment env;
//...
System.out.println(env.getProperty("bar"));
Read more...
Yes, you can. However, most answer didn't mention, the ordering is very important, please check this https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html
Your OS environment variables will overwrite the value come from Application properties packaged inside your jar (application.properties and YAML variants)., so basically, your environment variables has higher priority.
you can use it with The #Value annotation for the #Components and #service class
Some times it won't work if it is a normal class
Example:
#Component
public class Testclass{
#Value("${MY_SECRET:aDefaultValue}")
private String test1;
#Value("#{environment.MY_SECRET}")
private String test1;
#Value("${myApp.mySecretIndirect:aDefaultValue}")
private String test1;
//to get the properties list whih are in "," seperated
#Value("${my.list.of.strings}")
private List<String> myList;
}
You can place your environment variable in an application.yml/application.properties file and then you can fetch the value using the #Value annotation.
But in order to use #Value annotation your class should be a bean and should be annotated with #Component annnotation.
You can also provide a default value for the variable.
#Component
#NoArgsConstructor
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
public class MyClass {
#Value("${something.variable:<default-value>}")
private String myEnvVariable;
}
First, you have to define the relevant field information in the properties configuration file, and then use # value to obtain and use
example:
#Value("${abc}")
private String abc;
We are actually using Spring Boot's #ConfigurationProperties as basically a configuration mapper : it provides us an easy shortcut to map properties on objects.
#ConfigurationProperties("my.service")
public class MyService {
private String filePrefix;
private Boolean coefficient;
private Date beginDate;
// getters/setters mandatory at the time of writing
public void doBusinessStuff() {
// ...
}
}
Although this was a nice productivity boost when we were prototyping the app, we came to question if this was right usage.
I mean, configuration properties have a different status in Spring Boot's context, they're exposed through actuator endpoints, they can be used to trigger conditional beans, and seem more oriented toward technical configuration properties.
Question : Is it "correct" to use this mechanism on any business property/value, or is it plain misuse ?
Any potential drawback we missed ?
Right now our only concern is that we cannot use #ConfigurationProperties on immutable classes, which is closely related to this issue on Spring Boot's tracker : Allow field based #ConfigurationProperties binding
If your property represents something that is configurable based on the environment/profile that is what the mechanism is there for. Though I'm a little unclear what you mean by
"map properities on objects".
I would not favor this style in general, especially if your bean has multiple properties to set. A more standard idiom is to have a class that encapsulates the properties/settings used to create your bean:
#ConfigurationProperties("my.service")
public class MyServiceProperties {
private String filePrefix;
private Boolean coefficient;
private Date beginDate;
// getters/setters mandatory at the time of writing
}
then your Service class would look like this:
#EnableConfigurationProperties(MyServiceProperties.class)
public class MyService {
#Autowired
private MyServiceProperties properties;
//do stuff with properties
public void doBusinessStuff() {
// ...
}
}
This would at least allow you to pass the properties easily into an immutable class through it's constructor (make copies of any mutable properties). Also having the properties bean can be reused if you find other parts of your app need some shared configuration.