We Keep Coding

How do I replace the values of a YAML file with definitions in the same file? [duplicate]

We have a spring boot application with configuration being driven from application.yml file. In this configuration file we use the feature of defining a property by referring to another property inside the same application.yml file:
my-games-app:
base-property: foo
games:
- game-one:
game-name: ${my-games-app.base-property}one
game-location: ${my-games-app.base-property}/one
- game-two:
game-name: ${my-games-app.base-property}two
game-location: ${my-games-app.base-property}/two
And we have a #ConfigurationProperties bean loading games configuration:
#Configuration
#ConfigurationProperties(prefix = "my-games-app.games")
public class GamesConfig {
private Map<String, Game> games;
...
}
Useless to say the above is just an example, in reality it is a very complex setup with GamesConfig bean being used as a constructor argument for many other beans inside our application:
#Component
public class GamesRunner {
private final GamesConfig gamesConfig;
...
}
Everything works as expected. The problem we have is related to testing the beans where GamesConfig is injected; in the above example GamesRunner. At the moment we use #SpringBootTest to get hold of the beans we want to test. This again, works OK but the main inconvenient is that the whole application needs to be started in order to access the GamesConfig bean. This means setting up a lot of infrastructure such as a Database a JMS message broker and a Kafka broker. This takes time and makes our CI builds longer to run which started to become a bit of an inconvenient. Because the beans we want to test don't need any other setup than having the GamesConfig constructor argument provided we would prefer to have unit tests in place rather than integration tests as they are much faster to run.
In other words, we want to be able to recreate GamesConfig by hand by parsing our application.yml with a test helper method. To do this we use snakeyaml library:
public final class TestHelper {
public static GamesConfig getGamesConfig() {
var yaml = new Yaml();
var applicationYaml = (Map<String, Object>) yaml.load(readResourceAsString("application.yml");
return createGamesConfig(applicationYaml.get("games");
}
private static GamesConfig createGamesConfig(Object config) {
// The config Object passed here is a `Map<String, Map<String, String>>`
// as defeined in our `games` entry in our `application.yml`.
// The issue is that game name and game locations are loaded exactly like
// configured without property place holders being resolved
return gamesConfig;
}
}
We resolved the issue by manually parsing the property placeholders and looking up their values in the application.yml file. Even if our own property placeholder implementation is quite generic, my feeling is that this extra work is not needed as it should be a basic expectation the library would have some specific set up to do this out of the box. Being very new to snakeyaml I hope someone else hit the same problem and knows how to do it.
We use snakeyaml because it just happened to be in the class path as a transitive dependency, we are open to any suggestions that would achieve the same thing.
Thank you in advance.

To my knowledge, SnakeYAML only supports substitution of environment variables, which is why what you want is not possible as far as I know. What you can do instead, of course, is simply use Spring's classes without setting up a full ApplicationContext.
For example, assuming your game config from above, you could use:
final var loader = new YamlPropertySourceLoader();
final var sources = loader.load(
"games-config.yml",
new ClassPathResource("games-config.yml")
);
final var mutablePropertySources = new MutablePropertySources();
sources.forEach(mutablePropertySources::addFirst);
final var resolver = new PropertySourcesPropertyResolver(mutablePropertySources);
resolver.setIgnoreUnresolvableNestedPlaceholders(true);
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-location"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-location"));
which outputs:
fooone
foo/one
footwo
foo/two
If you are actually interested in how Spring does it, a good starting point is the source code of the PropertySourcesPlaceholderConfigurer class.

SpringBoot 2.7.5 - #Value annotation not assigning any values from application.properties file

I am using Spring Boot 2.7.5
While using #Value annotation to inject values form application.properties file, getting null assignment to variables under #Configuration class.
Example:
#Value("${file.name}")
private String fileName;
Here I am getting seeing assignment to variable fileName
Ideally it should assign value '${file.name}' if key isn't matching. But null assignment means something is breaking in the project (at least I think so, I need experts comments on this).
Same thing is working in another project but not in this particular.
Let me know if my question is not elaborative enough and will try to explain in detail
My question is, why is it working in other project but not in this one. What configurations could go wrong which I need to check. Have gone through multiple stackoverflow solutions and verified all below:
application.properties file spell check
#Configuration annotation at top of class where #Value is being used
key value pair assignment and spell check of all keys
no issues with library imports
Tried to add #PropertySource("classpath:foo.properties")
import org.springframework.beans.factory.annotation.Value;
resources folder is correctly marked as 'Resources Root'
Temporary Alternative
private Properties props = new Properties();
props.load(VaultCredential.class.getResourceAsStream("/application.properties"));
props.getProperty("file.name");

Can you please try with below code
private String PathForImageGallery;
#Value("${file.name}")
public void PathForImageGallery(String PathForImageGallery) {
this.PathForImageGallery = PathForImageGallery;
}

Apologies everyone. I was calling a class where was trying to bind #Value with properties and calling it in main() method of #SpringBootApplication.
Totally missed that main() method will not load property values until spring application is up and running.
Working solution if some one want to load properties before even running a SpringBoot application -
private Properties props = new Properties();
props.load(VaultCredential.class.getResourceAsStream("/application.properties"));
props.getProperty("file.name");
PS - Thanks all for your efforts. Admins can close it down if considered as non-logical question.

Spring Configuration creating complex data structure with yaml

Using yaml in my Spring-boot application (with snakeyaml dependency 1.16) I am attempting to create a #ConfigurationProperties based off of my application.yml file. I want to create a data structure like the json below which is a Map with String Keys and Array values.
mapName: {
"key1": ["elem0","elem1"],
"key2": ["hello","world"]
}
Attempting to create a Spring configuration class as follows
#Component
#ConfigurationProperties(prefix = "channel-broker")
#EnableConfigurationProperties
public class BrokerConfiguration {
private Map<String, Set<String>> broker = new HashMap<>();
public Map<String, Set<String>> getBroker() {
return broker;
}
}
I have tried the following for my yaml
channel-broker:
broker: {message-delivery: ['all'], facebook: ['client1']}
Attempt two
channel-broker:
message-delivery: ['all']
facebook: ['client1']
Attempt three
channel-broker:
message-delivery:
- ['all']
facebook:
- ['client1']
I have also tried initializing the HashMap in the #ConfigurationProperties class as such ... new HashMap<String, Set<String>> this didn't work either
All attempts result in this error which makes me believe its an error when converting to the object not that there is anything wrong with the yaml syntax.
Caused by: org.springframework.beans.InvalidPropertyException: Invalid
property 'brokerTest[message-delivery][0]' of bean class
[my.classpackage.clasname]:
Property referenced in indexed property path
'brokerTest[message-delivery][0]' is neither an array nor a List nor a
Map; returned value was [all]
Is it possible to create such an object? How would I accomplish this
-UPDATE-
If I change the Set to an ArrayList (or List interface) this works but that isn't what I'm looking for. changed to this
private Map<String, ArrayList<String>> brokerTest = new HashMap<>();
but need this doesn't work with Set interface either:
private Map<String, HashSet<String>> brokerTest = new HashMap<>();

This issue was being caused by the format of the yaml file. The following structure allowed me to build my graph like data structure out of yaml
channel-broker:
broker:
message-delivery:
all
facebook:
client1,client2
The Set doesn't want anything extra surrounding the key. Note if your Set will contain multiple values you can add a comma to separate them. Just like Json the last element will not have a comma after.

What you are looking for is this :
channel-broker: {broker: {message-delivery:['all', ...], facebook:['client1', ...]}}
see Complete idiot's introduction to yaml

If you use [] then it's an array so arraylist works, for hashset/hashmap you need to use {} brackets.
channel-broker: {
broker: {
message-delivery:{'all', '123'},
facebook:{'client1', 'cleant2'}
}
}
will work for hashset.
(hashmap example)

is Spring provide Local storage?

I have to develop one desktop base application(it has no request/response) and it has many classes.
my Main/Controller class read (*.properties) file,
like
allExtensions = properties.getProperty("ReportFileExtension");
and i have mention some file extension in .properties file like ReportFileExtensionn = .pdf,.doc etc.
This key read from .properties file in Main class and i want use this key value in other class without passing argument in any method or constructor.
is it spring provide a local storage ? so i can use to store attribute and use it other class.
Thanks in Advc.

Your question is bit confusing but based on comments I think you are struggling to understand how to get keys from properties.
Check PropertiesLoaderUtils
Resource resource = new ClassPathResource("/my.properties");
Properties props = PropertiesLoaderUtils.loadProperties(resource);
Now iterate over the props object
Try this
public Set<Object> getAllKeys(){
Set<Object> keys = prop.keySet();
return keys;
}
Or this
public void printProperties() {
for(Entry<Object, Object> e : props.entrySet()) {
System.out.println(e);
}
}
Properties are available globally but if you want you can create a static map and cache the properties in it.

How to load all key/value pairs in properties file

I'm trying to load all key/value pairs in my properties file.
One approach is that I load all the properties using #Value manually, but for that I should know all the keys.
I cannot do this, since property file may be changed in future to include more set of key/value pairs and I may need to modify code again to accommodate them.
Second approach is that I should some how load the properties file and Iterate over it to load all the key/value pairs without knowing the keys.
Say I have following properties file sample.properties
property_set.name="Database MySQL"
db.name=
db.url=
db.user=
db.passwd=
property_set.name="Database Oracle"
db.name=
db.url=
db.user=
db.passwd=
Here is what I'm trying to do
#Configuration
#PropertySource(value="classpath:sample.properties")
public class AppConfig {
#Autowired
Environment env;
#Bean
public void loadConfig(){
//Can I some how iterate over the loaded sampe.properties and load all
//key/value pair in Map<String,Map<String, String>>
// say Map<"Database MySQL", Map<key,vale>>
// I cannot get individual properties like env.getProperty("key");
// since I may not know all the keys
}
}

Spring stores all properties in Environment.
Environment contains collection of PropertySource. Every PropertySource contains properties from specific source. There are system properties, and java environment properties and many other. Properties from you file will be there as well.
Any source has own name. In your case automatically generated name will be look like "class path resource [sample.properties]". As you see, the name is not so convenient. So lets set more convenient name:
#PropertySource(value="classpath:sample.properties", name="sample.props")
Now you can get source by this name:
AbstractEnvironment ae = (AbstractEnvironment)env;
org.springframework.core.env.PropertySource source =
ae.getPropertySources().get("sample.props");
Properties props = (Properties)source.getSource();
Note that I specified full name of PropertySource class, to avoid conflict with #PropertySource annotation class. After that, you can work with properties. For example output them to console:
for(Object key : props.keySet()){
System.out.println(props.get(key));
}

You can autowire in an EnumerablePropertySource which contains the method getPropertyNames()

you can look up the class Properties in the jdk api and use the method load(InputStream inStream)
InputStream in = new FileInputStream("your properties location");
Properties prop = new Properties();
prop.load(in);
ps: prop is subClass of HashTable and don't forget to close stream