We have a spring boot application with configuration being driven from application.yml file. In this configuration file we use the feature of defining a property by referring to another property inside the same application.yml file:
my-games-app:
base-property: foo
games:
- game-one:
game-name: ${my-games-app.base-property}one
game-location: ${my-games-app.base-property}/one
- game-two:
game-name: ${my-games-app.base-property}two
game-location: ${my-games-app.base-property}/two
And we have a #ConfigurationProperties bean loading games configuration:
#Configuration
#ConfigurationProperties(prefix = "my-games-app.games")
public class GamesConfig {
private Map<String, Game> games;
...
}
Useless to say the above is just an example, in reality it is a very complex setup with GamesConfig bean being used as a constructor argument for many other beans inside our application:
#Component
public class GamesRunner {
private final GamesConfig gamesConfig;
...
}
Everything works as expected. The problem we have is related to testing the beans where GamesConfig is injected; in the above example GamesRunner. At the moment we use #SpringBootTest to get hold of the beans we want to test. This again, works OK but the main inconvenient is that the whole application needs to be started in order to access the GamesConfig bean. This means setting up a lot of infrastructure such as a Database a JMS message broker and a Kafka broker. This takes time and makes our CI builds longer to run which started to become a bit of an inconvenient. Because the beans we want to test don't need any other setup than having the GamesConfig constructor argument provided we would prefer to have unit tests in place rather than integration tests as they are much faster to run.
In other words, we want to be able to recreate GamesConfig by hand by parsing our application.yml with a test helper method. To do this we use snakeyaml library:
public final class TestHelper {
public static GamesConfig getGamesConfig() {
var yaml = new Yaml();
var applicationYaml = (Map<String, Object>) yaml.load(readResourceAsString("application.yml");
return createGamesConfig(applicationYaml.get("games");
}
private static GamesConfig createGamesConfig(Object config) {
// The config Object passed here is a `Map<String, Map<String, String>>`
// as defeined in our `games` entry in our `application.yml`.
// The issue is that game name and game locations are loaded exactly like
// configured without property place holders being resolved
return gamesConfig;
}
}
We resolved the issue by manually parsing the property placeholders and looking up their values in the application.yml file. Even if our own property placeholder implementation is quite generic, my feeling is that this extra work is not needed as it should be a basic expectation the library would have some specific set up to do this out of the box. Being very new to snakeyaml I hope someone else hit the same problem and knows how to do it.
We use snakeyaml because it just happened to be in the class path as a transitive dependency, we are open to any suggestions that would achieve the same thing.
Thank you in advance.
To my knowledge, SnakeYAML only supports substitution of environment variables, which is why what you want is not possible as far as I know. What you can do instead, of course, is simply use Spring's classes without setting up a full ApplicationContext.
For example, assuming your game config from above, you could use:
final var loader = new YamlPropertySourceLoader();
final var sources = loader.load(
"games-config.yml",
new ClassPathResource("games-config.yml")
);
final var mutablePropertySources = new MutablePropertySources();
sources.forEach(mutablePropertySources::addFirst);
final var resolver = new PropertySourcesPropertyResolver(mutablePropertySources);
resolver.setIgnoreUnresolvableNestedPlaceholders(true);
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-location"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-location"));
which outputs:
fooone
foo/one
footwo
foo/two
If you are actually interested in how Spring does it, a good starting point is the source code of the PropertySourcesPlaceholderConfigurer class.
Does Spring put all properties into the same hashmap/list?
How can I get a list of all of the properties being managed by Spring
For example, if I have two properties files and they define overlapping properties which
// FILE: src/main/resources/file1.properties
prop1.color = red
prop1.fill = green
// FILE: src/main/resources/file2.properties
prop2.color = purple
prop1.fill = ORANGE
The configuration class used to tell Spring to read the two files is something like this:
#PropertySource("classpath:file1.properties")
#PropertySource("classpath:file2.properties")
#Configuration
public class MyConfig {
}
My question is Can I get an iterator to access all of the properties defined? If so how.
Searching for answer
So far I haven't found a solution to this question.
I did an Internet search for "stack overflow Spring properties" and did not found the answer.
Some SO questions I found are:
Spring boot get all properties from properties file and load into hashmap
Spring Boot application.properties
But they didn't answer my question
Spring provides a PropertySources interface which is implemented by the MutablePropertySources class. This class basically just holds a list of PropertySource instances (e.g. instances of the PropertiesPropertySource class - not to be confused with the #PropertySource annotation) and provides methods for iterating over these instances.
If you look at the implementation of MutablePropertySources you can see that there are multiple methods for adding PropertySource instances at different positions and relative to other PropertySources in that list. If you want to get the value of a property, you can call the get method of a MutablePropertySources instance with the name of that property. The get method then iterates over all PropertySources in its list and tries to find a property with the given name.
Then there is the Environment interface that is implemented by the abstract class AbstractEnvironment. You can use the latter to access the PropertySources instance that is used by that environment.
For example, for testing purposes, you can do the following to print out all properties and their values that are managed by MapPropertySources - a subtype of PropertySource.
#PropertySource("classpath:file1.properties")
#PropertySource("classpath:file2.properties")
#Configuration
public class ExampleConfig {
#Autowired
public ExampleConfig(Environment environment) {
AbstractEnvironment env = (AbstractEnvironment) environment;
for (org.springframework.core.env.PropertySource<?> source : env.getPropertySources()) {
if (source instanceof MapPropertySource mapPropertySource) {
System.out.println(mapPropertySource.getSource());
}
}
}
}
You can check out the org.springframework.core.env package which contains all of these classes.
So, to answer your first answer more explicitly: No, Spring does not put all properties into a single map. It puts properties from different locations into different PropertySources which are then added to a PropertySources instance.
I have a few classes that need some environment specific configuration. I turned to using properties files, which are loaded in the constructor of the class.
public class MyClass {
public MyClass() {
try {
ValidatedEnvironmentProperties props = new ValidatedEnvironmentProperties();
props.load(MyClass.class.getResourceAsStream("/myclass.properties"));
ValidatedEnvironmentProperties extends Properties. Basically, it uses a Java System Property to set a key prefix. I set the System property to, say, production, and in the properties files, I have staging.url=... and production.url=.... This allows me build/runtime selection of which configuration is used while not needing to change property file name.
myclass.properties is stored in src/main/resources.
That works fine, and I rather like the how it works. My problem is that I'm sort of stuck with TestNG. I want to test a bunch of other properties in TestNG unit tests. This led me to create src/test/resources/myclass.properties. Instead of "environment" keys, I use test names like bad_url_test.url=this_ain't_a_url.
I was thinking that src/test/resources would get a higher priority in the classpath/classloader (terminology?), causing the test-specific properties to load. Then, for my various tests, I just set the Java System property to bad_url_test, instantiate MyClass, test my assertions, and then set the System property to a new test, instantiate a new object, and repeat.
I believe the source of the problem is this line in MyClass's constructor:
props.load(MyClass.class.getResourceAsStream("/myclass.properties"));
In MyClassTest, I put these lines to try to understand what's happening:
File f = new File(MyClass.class.getProtectionDomain().getCodeSource().getLocation().getPath());
System.out.println("MyClass classpath: " + f.toString());
f = new File(MyClassTest.class.getProtectionDomain().getCodeSource().getLocation().getPath());
System.out.println("MyClassTest classpath: " + f.toString());
Output is:
MyClass classpath: /home/fandingo/code/project/build/classes/main
MyClassTest classpath: /home/fandingo/code/project/build/classes/test
MyClassTest is correct, but I need something accessible within MyClass's constructor that will automatically prefer /src/test/resources/ when running tests but /src/main/resources/ when running normally.
You need inversion of control. i.e. MyClass needs to be told where to get its properties from and not decide on its own.
There are lots of ways to do this but they all come down to the same idea: MyClass should not know at compile time where its properties come from.
e.g.
In MyClass.java
ValidatedEnvironmentProperties props = new ValidatedEnvironmentProperties();
String resourceSupplierClassName = System.getProperty("resource-supplier-class-name",
MyClass.class.getName());
Class<?> resourceSupplierClass = Class.forName(resourceSupplierClassName);
props.load(resourceSupplierClass.getResourceAsStream("/myclass.properties"));
In MyClassTest.java
System.setProperty("resource-supplier-class-name", MyClassTest.class.getName());
Again, there are many ways to do this. Instead of passing around system properties, etc. you can also use dependency injection (a form of inversion of control). You can change the MyClass constructor to take your props as an argument and then put the onus on the instantiators of MyClass to provide the props or you can use a dependency injection framework such as Spring or Guice to manage creating the appropriate props instance for main/test execution and provide it to objects that need it as needed.
Let us assume up to runtime we do not know what are the details of configuration(may user need to configure these parameters in config file before running the application.
I want to read those configuration details and need to reuse them wherever I need them in my application. For that I want to make them as global constants(public static final).
So, My doubt is, is there any performance implications if I read from config file directly from the required class? since,runtime values I can not directly put in separate Interface.
I am thinking it will impact performance.Please suggest me any better way to do this.
UPDATE: Can I use separate final class for configuration details?
putting all configuration details as constants in a separate public final class
(To read all configuration details at once from the configuration file and storing them as global constants for later use in application)
I am thinking it will impact performance.
I doubt that this will be true.
Assuming that the application reads the configuration file just once at startup, the time taken to read the file is probably irrelevant to your application's overall performance. Indeed, the longer the application runs, the less important startup time will be.
Standard advice is to only optimize for application performance when you have concrete evidence (i.e. measurements) to say that performance is a significant issue. Then, only optimize those parts of your code that profiling tells you are really a performance bottleneck.
Can I use separate final class for configuration details
Yes it is possible to do that. Nobody is going to stop you1.
However, it is a bad idea. Anything that means that you need to recompile your code to change configuration parameters is a bad idea. IMO.
To read all configuration details at once from the configuration file and storing them as global constants for later use in application.
Ah ... so you actually want to read the values of the "constants" instead of hard-wiring them.
Yes, that is possible. And it makes more sense than hard-wiring configuration parameters into the code. But it is still not a good idea (IMO).
Why? Well lets look at what the code has to look like:
public final class Config {
public static final int CONST_1;
public static final String CONST_2;
static {
int c1;
String c2;
try (Scanner s = new Scanner(new File("config.txt"))) {
c1 = s.nextInt();
c2 = s.next();
} catch (IOException ex) {
throw RuntimeException("Cannot load config properties", ex);
}
CONST_1 = c1;
CONST_2 = c2;
}
}
First observation is that makes no difference that the class is final. It is declaring the fields as final that makes them constant. (Declaring the class as final prevents subclassing, but that has no impact on the static fields. Static fields are not affected by inheritance.)
Next observation is that this code is fragile in a number of respects:
If something goes wrong in the static initializer block. the unchecked exception that is thrown by the block will get wrapped as an ExceptionInInitializerError (yes ... it is an Error!!), and the Config class will be marked as erroneous.
If that happens, there is no realistic hope of recovering, and it possibly even a bad idea to try and diagnose the Error.
The code above gets executed when the Config class is initialized, but determining when that happens can be tricky.
If the configuration filename is a parameter, then you have the problem of getting hold of the parameter value ... before the static initialization is triggered.
Next, the code is pretty messy compared with loading the state into a instance variables. And that messiness is largely a result of having to work within the constraints of static initializers. Here's what the code looks like if you use final instance variables instead.
public final class Config {
public final int CONST_1;
public final String CONST_2;
public Config(File file) throws IOException {
try (Scanner s = new Scanner(file)) {
CONST_1 = s.nextInt();
CONST_2 = s.next();
}
}
}
Finally, the performance benefits of static final fields over final fields are tiny:
probably one or two machine instructions each time you access one of the constants,
possibly nothing at all if the JIT compiler is smart, and you handle the singleton Config reference appropriately.
In either case, in the vast majority of cases the benefits will be insignificant.
1 - OK ... if your code is code-reviewed, then someone will probably stop you.
Have you ever heard of apache commons configuration http://commons.apache.org/proper/commons-configuration/ ?
It is the best configuration reader I have ever found and even am using it in my application which is running in production since 1 year. Never found any issues, very easy to understand and use, great performance. I know its a bit of dependency to your application but trust me you will like it.
All you need to do is
Configuration config = new ConfigSelector().getPropertiesConfiguration(configFilePath);
String value = config.getString("key");
int value1 = config.getInt("key1");
String[] value2 = config.getStringArray("key2");
List<Object> value3 = config.getList("key3");
And thats it. Your config object will hold all the config values and you can just pass that object to as many classes as you want. With so many available helpful methods you can extract whichever type of key you want.
It will be only one time cost if you are putting them in a property file and reading the file at the start of your application and initialize all the parameters as system parameters(System.setProperty) and then define constants in your code like
public static final String MY_CONST = System.getProperty("my.const");
But ensure the initialization at start of your application before any other class is loaded.
There are different types of configuration.
Usually some sort of bootstrapping configuration, for example to connect to a database or service, is needed to be able to start the application. The J2EE way to specify database connection parameters is via a 'datasource' specified in your container's JNDI registry (Glassfish, JBoss, Websphere, ...). This datasource is then looked up by name in your persistence.xml. In non-J2EE applications it is more common to specify these in a Spring context or even a .properties file. In any case, you usually need something to connect your application to some sort of data store.
After bootstrapping to a data store an option is to manage config values inside this datastore. For example if you have a database you can use a separate table (represented by e.g. a JPA Entity in your application) for configuration values. If you don't want/need this flexibility you can use simple .properties file for this instead. There is good support for .properties files in Java (ResourceBundle) and in frameworks like Spring. The vanilla ResourceBundle just loads the properties once, the Spring helper offers configurable caching and reloading (this helps with the performance aspect which you mentioned). Note: you can also use Properties backed by a data store instead of a file.
Often both approaches coexist in an application. Values that never change within a deployed application (like the application name) can be read from a properties file. Values that might need to be changed by an application maintainer at runtime without redeployment (e.g. the session timeout interval) might better be kept in a reloadable .properties file or in a database. Values that can be changed by users of the application should be kept in the application's data store and usually have an in-application screen to edit them.
So my advise is to separate your configuration settings into categories (e.g. bootstrap, deployment, runtime and application) and select an appropriate mechanism to manage them. This also depends on the scope of your application, i.e. is it a J2EE web app, a desktop app, command-line utility, a batch process?
What kind of configuration file do you have in mind? If it is a properties file, this might suit you:
public class Configuration {
// the configuration file is stored in the root of the class path as a .properties file
private static final String CONFIGURATION_FILE = "/configuration.properties";
private static final Properties properties;
// use static initializer to read the configuration file when the class is loaded
static {
properties = new Properties();
try (InputStream inputStream = Configuration.class.getResourceAsStream(CONFIGURATION_FILE)) {
properties.load(inputStream);
} catch (IOException e) {
throw new RuntimeException("Failed to read file " + CONFIGURATION_FILE, e);
}
}
public static Map<String, String> getConfiguration() {
// ugly workaround to get String as generics
Map temp = properties;
Map<String, String> map = new HashMap<String, String>(temp);
// prevent the returned configuration from being modified
return Collections.unmodifiableMap(map);
}
public static String getConfigurationValue(String key) {
return properties.getProperty(key);
}
// private constructor to prevent initialization
private Configuration() {
}
}
You could also return the Properties object immediately from the getConfiguration() method, but then it could potentially be modified by the code that access it. The Collections.unmodifiableMap() does not make the configuration constant (since the Properties instance gets its values by the load() method after it was created), however since it is wrapped in an unmodifiable map, the configuration cannot be changed by other classes.
Well this is a great problem which is faced in every one's life once in a will. Now coming to the problem, this can be solved by creating a singleton class which has instance variables same as in configuration file with default values. Secondly this class should have a method like getInstance() which reads the properties once and every times returns the same object if it exists. For reading file we can use Environmental variable to get path or something like System.getenv("Config_path");. Reading the properties (readProperties() method) should read each item from config file and set the value to the instance variables of singleton object. So now a single object contains all the configuration parameter's value and also if the parameter is empty than default value is considered.
One more way is to define a class and read the properties file in that class.
This class needs to be at the Application level and can be marked as Singleton.
Marking the class as Singleton will avoid multiple instances to be created.
Putting configuration keys directly to classes is bad: configuration keys will be scattered over the code. Best practice is separation of application code and configuration code. Usually dependency injection framework like spring is used. It loads a configuration file and constructs the objects using configuration values. If you need some configuration value in your class you should create a setter for this value. Spring will set this value during context initialization.
I recommend using JAXB or a similar binding framework that works with text based files. Since a JAXB implementation is part of the JRE, it's pretty easy to use. As Denis I advise against configuration keys.
Here is a simple example for an easy to use and still pretty mighty way to configure you application with XML and JAXB. When you use a DI framework you can just add a similar config object to the DI context.
#XmlRootElement
#XmlAccessorType(XmlAccessType.FIELD)
public class ApplicationConfig {
private static final JAXBContext CONTEXT;
public static final ApplicationConfig INSTANCE;
// configuration properties with defaults
private int number = 0;
private String text = "default";
#XmlElementWrapper
#XmlElement(name = "text")
private List<String> texts = new ArrayList<>(Arrays.asList("default1", "default2"));
ApplicationConfig() {
}
static {
try {
CONTEXT = JAXBContext.newInstance(ApplicationConfig.class);
} catch (JAXBException ex) {
throw new IllegalStateException("JAXB context for " + ApplicationConfig.class + " unavailable.", ex);
}
File applicationConfigFile = new File(System.getProperty("config", new File(System.getProperty("user.dir"), "config.xml").toString()));
if (applicationConfigFile.exists()) {
INSTANCE = loadConfig(applicationConfigFile);
} else {
INSTANCE = new ApplicationConfig();
}
}
public int getNumber() {
return number;
}
public String getText() {
return text;
}
public List<String> getTexts() {
return Collections.unmodifiableList(texts);
}
public static ApplicationConfig loadConfig(File file) {
try {
return (ApplicationConfig) CONTEXT.createUnmarshaller().unmarshal(file);
} catch (JAXBException ex) {
throw new IllegalArgumentException("Could not load configuration from " + file + ".", ex);
}
}
// usage
public static void main(String[] args) {
System.out.println(ApplicationConfig.INSTANCE.getNumber());
System.out.println(ApplicationConfig.INSTANCE.getText());
System.out.println(ApplicationConfig.INSTANCE.getTexts());
}
}
The configuration file looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<applicationConfig>
<number>12</number>
<text>Test</text>
<texts>
<text>Test 1</text>
<text>Test 2</text>
</texts>
</applicationConfig>
protected java.util.Properties loadParams() throws IOException {
// Loads a ResourceBundle and creates Properties from it
Properties prop = new Properties();
URL propertiesFileURL = this.getClass().getResource("/conf/config.properties");
prop.load(new FileInputStream(new File(propertiesFileURL.getPath())));
return prop;
}
Properties prop = loadParams();
String prop1=(String) prop.get("x.y.z");
Given the prevalence of YML to express configuration, I'd recommend creating a YML file with the configuration inside it and then loading that once, at startup, into a POJO, then accessing the fields of that POJO to get the configuration:
user: someuser
password: somepassword
url: jdbc://mysql:3306/MyDatabase
With Java Class
public class Config {
private String user;
private String password;
private String url;
// getters/setters
Jackson can be used to load YML as can SnakeYml directly.
On top of this, you could use the OS project I've been working on - https://github.com/webcompere/lightweight-config - which allows you to wrap this up, and even express placeholders in your file to interpolate environment variables:
user: ${USER}
password: ${PASSWORD}
url: jdbc://${DB_HOST}:3306/MyDatabase
then
Config config = ConfigLoader.loadYmlConfigFromResource("config.yml", Config.class);
The purpose of my code is to load some settings value from a *.properties file so that I later can use these values in some if-statements in my code. I want to load in some list-structure, but since that seems hard, an array will do. I have not really gotten that far, since I am stuck at the trivial matter of loading just a String from the properties file.
When I try to debug my code that is using some spring specific data. I get some interesting behaviour, pointing on the definition in the code right above the breakpoint gives me that the variable value is null.
#Value(value = "${ViewableReportFilter.allStates.verify}")
String verifyStringStates;
public ViewableReportFilter() {
viewStates = null;
log.debug("Read in properties for states: verify:" + verifyStringStates);
/*BREAKPOINT HERE*/
in my my.properties file:
ViewableReportFilter.allStates.verify=ONHOLD
And my config to use the properties-file:
<context:property-placeholder location="classpath:properties/my.properties" order="1" ignore-unresolvable="true" />
Spring can't set the fields of an object before that object is created. The first thing Spring does is use reflection to instantiate your class. It'll use either Class#newInstance() or use Constructor#newInstance() depending on the context. Only when the constructor has finished its work and returned can Spring, again using reflection, set the value of fields.
An alternative is to put a #Value annotated parameter in the constructor parameter list and set your field inside your constructor from the argument that's given to it by Spring.
public ViewableReportFilter(#Value String verify) {
this.verifyStringStates = verify;
...
Go through the Spring documentation for its IoC container. It explains all of this in much detail.
Updated the constructor, and added Autowire annotation. No changes in the properties file, no XML.
String arrayOfStrings;
#Autowired
public ViewableReportFilter(
#Value("${TMSViewableReportFilter.allStates.verify}") String[] verifyStringStates) {
arrayOfStrings = verifyStringStates;
public logViewableReportFilter() {
log.debug("Read in properties for states: verify:" + arrayOfString);
}
Try using this:
#Value(value = "${allStates.verify}")
And in your property my.properties:
allStates.verify=ONHOLD