We have a spring boot application with configuration being driven from application.yml file. In this configuration file we use the feature of defining a property by referring to another property inside the same application.yml file:
my-games-app:
base-property: foo
games:
- game-one:
game-name: ${my-games-app.base-property}one
game-location: ${my-games-app.base-property}/one
- game-two:
game-name: ${my-games-app.base-property}two
game-location: ${my-games-app.base-property}/two
And we have a #ConfigurationProperties bean loading games configuration:
#Configuration
#ConfigurationProperties(prefix = "my-games-app.games")
public class GamesConfig {
private Map<String, Game> games;
...
}
Useless to say the above is just an example, in reality it is a very complex setup with GamesConfig bean being used as a constructor argument for many other beans inside our application:
#Component
public class GamesRunner {
private final GamesConfig gamesConfig;
...
}
Everything works as expected. The problem we have is related to testing the beans where GamesConfig is injected; in the above example GamesRunner. At the moment we use #SpringBootTest to get hold of the beans we want to test. This again, works OK but the main inconvenient is that the whole application needs to be started in order to access the GamesConfig bean. This means setting up a lot of infrastructure such as a Database a JMS message broker and a Kafka broker. This takes time and makes our CI builds longer to run which started to become a bit of an inconvenient. Because the beans we want to test don't need any other setup than having the GamesConfig constructor argument provided we would prefer to have unit tests in place rather than integration tests as they are much faster to run.
In other words, we want to be able to recreate GamesConfig by hand by parsing our application.yml with a test helper method. To do this we use snakeyaml library:
public final class TestHelper {
public static GamesConfig getGamesConfig() {
var yaml = new Yaml();
var applicationYaml = (Map<String, Object>) yaml.load(readResourceAsString("application.yml");
return createGamesConfig(applicationYaml.get("games");
}
private static GamesConfig createGamesConfig(Object config) {
// The config Object passed here is a `Map<String, Map<String, String>>`
// as defeined in our `games` entry in our `application.yml`.
// The issue is that game name and game locations are loaded exactly like
// configured without property place holders being resolved
return gamesConfig;
}
}
We resolved the issue by manually parsing the property placeholders and looking up their values in the application.yml file. Even if our own property placeholder implementation is quite generic, my feeling is that this extra work is not needed as it should be a basic expectation the library would have some specific set up to do this out of the box. Being very new to snakeyaml I hope someone else hit the same problem and knows how to do it.
We use snakeyaml because it just happened to be in the class path as a transitive dependency, we are open to any suggestions that would achieve the same thing.
Thank you in advance.
To my knowledge, SnakeYAML only supports substitution of environment variables, which is why what you want is not possible as far as I know. What you can do instead, of course, is simply use Spring's classes without setting up a full ApplicationContext.
For example, assuming your game config from above, you could use:
final var loader = new YamlPropertySourceLoader();
final var sources = loader.load(
"games-config.yml",
new ClassPathResource("games-config.yml")
);
final var mutablePropertySources = new MutablePropertySources();
sources.forEach(mutablePropertySources::addFirst);
final var resolver = new PropertySourcesPropertyResolver(mutablePropertySources);
resolver.setIgnoreUnresolvableNestedPlaceholders(true);
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[0].game-one.game-location"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-name"));
System.out.println(resolver.getProperty("my-games-app.games[1].game-two.game-location"));
which outputs:
fooone
foo/one
footwo
foo/two
If you are actually interested in how Spring does it, a good starting point is the source code of the PropertySourcesPlaceholderConfigurer class.
Assume I have a configuration class accessible via the stock CDI that defines some application-wide parameters:
#ApplicationScoped
class AppConfig {
public double getMaxAllowedBrightness() { ... }
};
And I have a simple class for my data objects:
class LightSource {
double brightness;
...
boolean isValid() {
double maxAllowedBrightness = ...; // Somehow use AppConfig#getMaxAllowedBrightness() here
return brightness <= maxAllowedBrightness;
}
}
How can my data object access the single AppConfig instance?
Somehow I hate the idea of autowiring AppConfig into every single data object (there are lots of them). Is there any other way to get access to AppConfig in the above example from my data object?
What's the best pattern to use here?
The simplest example is a runtime lookup akin to:
import jakarta.enterprise.inject.spi.CDI;
CDI.current().select(cls).get();
With cls being the class that you're looking up. (Note the package name, this is the latest version of CDI 2.x in the new jakarta namespace, the original is in javax.)
It gets more detailed from there, but that's the gist of it.
Note, that semantically there's little difference between autowiring something and doing a runtime lookup, especially for something mostly static at the instance level. It's still a dependency. You still have to touch the code of the classes to pull it off.
A nice thing of relying on the autowiring is that you can disable it situationally, and the class reverts to a simple bean, that you can do with what you will. Coding in the lookup, it's a little bit more than that.
Dynamic lookup is more for special circumstances.
On my current project, our team has been doing this using the #Value annotation. In our case, we have all the properties in a properties bean, which I'll call mainAppConfiguration. The bean is populated from a properties file like main-app-config.properties (which was read into the bean with a Properties prop = new Properties().load(mainAppConfigFilePath) method.
Assuming you have something like that set up, then we inject the properties into the classes that need them using a little SpEL magic something like:
private Integer refreshRateSeconds;
#Value("#{ mainAppConfiguration.getProperties()['funny-property-base-name.refreshRateSeconds'] }")
public void setRefreshRateSeconds(Integer refreshRateSeconds) {
if (refreshRateSeconds == null) {
throw new IllegalArgumentException("Required config property 'funny-property-base-name.refreshRateSeconds' was not found"));
}
this.refreshRateSeconds = refreshRateSeconds;
}
Baeldung has examples (without defaults) and more with defaults.
I was exploring ways to do simple, plain-old file-based configuration in Java. I looked into Java's built-in Properties and the Apache Common Configuration library. For the latter, the distilled code is as follows:
Configurations configs = new Configurations();
Configuration config = null;
try
{
config = configs.properties(new File("config.properties"));
}
catch (ConfigurationException cex)
{
}
long loadQPS = config.getInt("loadQPS");
The issue I have with this is that I find myself inserting this in every single class, which is suboptimal for at least two reasons: 1) I'm reading the file once for every class, when I should only read it once. 2) code duplication.
One obvious solution would be to create a Singleton configuration class that I then access from every other class. But surely this is a desired feature in almost every use case, so shouldn't it be included with the configuration library itself (am I missing something)? I also thought of using Spring configuration, which can create a Singleton configuration class for me, but isn't there too much overhead just for file-based configuration? (Spring's strength is in DI, as I understand.)
What's a good solution, or best practice (if there is one)?
EDIT: A simple static solution suggested in the answer:
public class ConfigClass {
static Configuration config;
static {
Configurations configs = new Configurations();
Logger sysLogger = LoggerFactory.getLogger("sysLogger");
try
{
config = configs.properties(new File("config.properties"));
}
catch (ConfigurationException cex)
{
sysLogger.error("Config file read error");
}
}
}
Access in the package by ConfigClass.config.
So you have a couple options. One simple one would be to store and access the Configuration object statically.
Another one that I like when I want Dependency Injection without Spring, is to structure program in DI friendly way. You can emulate a DI container by transforming your main() function into a "configuration" of your program that ultimately launches it.
Consider a typical multi-tier web application: A DI friendly main() method might look like:
public class AddressBookApp {
public static void main(String[] args) {
Configuration conf = new Configuration(args[0]);
// Creates our Repository, this might do some internal JDBC initialization
AddressBookRepository repo = new AddressBookRepository(conf);
// Pass the Repository to our Service object so that it can persist data
AddressBookService service = new AddressBookService(repo);
// Pass the Service to the web controller so it can invoke business logic
AddressBookController controller = new AddressBookController(conf, service);
// Now launch it!
new WebApp(new Controller[] { controller }).start();
}
}
This main() serves as a central place to "wire up" your application so it's easy to pass your Configuration object to every component that needs it.
Let us assume up to runtime we do not know what are the details of configuration(may user need to configure these parameters in config file before running the application.
I want to read those configuration details and need to reuse them wherever I need them in my application. For that I want to make them as global constants(public static final).
So, My doubt is, is there any performance implications if I read from config file directly from the required class? since,runtime values I can not directly put in separate Interface.
I am thinking it will impact performance.Please suggest me any better way to do this.
UPDATE: Can I use separate final class for configuration details?
putting all configuration details as constants in a separate public final class
(To read all configuration details at once from the configuration file and storing them as global constants for later use in application)
I am thinking it will impact performance.
I doubt that this will be true.
Assuming that the application reads the configuration file just once at startup, the time taken to read the file is probably irrelevant to your application's overall performance. Indeed, the longer the application runs, the less important startup time will be.
Standard advice is to only optimize for application performance when you have concrete evidence (i.e. measurements) to say that performance is a significant issue. Then, only optimize those parts of your code that profiling tells you are really a performance bottleneck.
Can I use separate final class for configuration details
Yes it is possible to do that. Nobody is going to stop you1.
However, it is a bad idea. Anything that means that you need to recompile your code to change configuration parameters is a bad idea. IMO.
To read all configuration details at once from the configuration file and storing them as global constants for later use in application.
Ah ... so you actually want to read the values of the "constants" instead of hard-wiring them.
Yes, that is possible. And it makes more sense than hard-wiring configuration parameters into the code. But it is still not a good idea (IMO).
Why? Well lets look at what the code has to look like:
public final class Config {
public static final int CONST_1;
public static final String CONST_2;
static {
int c1;
String c2;
try (Scanner s = new Scanner(new File("config.txt"))) {
c1 = s.nextInt();
c2 = s.next();
} catch (IOException ex) {
throw RuntimeException("Cannot load config properties", ex);
}
CONST_1 = c1;
CONST_2 = c2;
}
}
First observation is that makes no difference that the class is final. It is declaring the fields as final that makes them constant. (Declaring the class as final prevents subclassing, but that has no impact on the static fields. Static fields are not affected by inheritance.)
Next observation is that this code is fragile in a number of respects:
If something goes wrong in the static initializer block. the unchecked exception that is thrown by the block will get wrapped as an ExceptionInInitializerError (yes ... it is an Error!!), and the Config class will be marked as erroneous.
If that happens, there is no realistic hope of recovering, and it possibly even a bad idea to try and diagnose the Error.
The code above gets executed when the Config class is initialized, but determining when that happens can be tricky.
If the configuration filename is a parameter, then you have the problem of getting hold of the parameter value ... before the static initialization is triggered.
Next, the code is pretty messy compared with loading the state into a instance variables. And that messiness is largely a result of having to work within the constraints of static initializers. Here's what the code looks like if you use final instance variables instead.
public final class Config {
public final int CONST_1;
public final String CONST_2;
public Config(File file) throws IOException {
try (Scanner s = new Scanner(file)) {
CONST_1 = s.nextInt();
CONST_2 = s.next();
}
}
}
Finally, the performance benefits of static final fields over final fields are tiny:
probably one or two machine instructions each time you access one of the constants,
possibly nothing at all if the JIT compiler is smart, and you handle the singleton Config reference appropriately.
In either case, in the vast majority of cases the benefits will be insignificant.
1 - OK ... if your code is code-reviewed, then someone will probably stop you.
Have you ever heard of apache commons configuration http://commons.apache.org/proper/commons-configuration/ ?
It is the best configuration reader I have ever found and even am using it in my application which is running in production since 1 year. Never found any issues, very easy to understand and use, great performance. I know its a bit of dependency to your application but trust me you will like it.
All you need to do is
Configuration config = new ConfigSelector().getPropertiesConfiguration(configFilePath);
String value = config.getString("key");
int value1 = config.getInt("key1");
String[] value2 = config.getStringArray("key2");
List<Object> value3 = config.getList("key3");
And thats it. Your config object will hold all the config values and you can just pass that object to as many classes as you want. With so many available helpful methods you can extract whichever type of key you want.
It will be only one time cost if you are putting them in a property file and reading the file at the start of your application and initialize all the parameters as system parameters(System.setProperty) and then define constants in your code like
public static final String MY_CONST = System.getProperty("my.const");
But ensure the initialization at start of your application before any other class is loaded.
There are different types of configuration.
Usually some sort of bootstrapping configuration, for example to connect to a database or service, is needed to be able to start the application. The J2EE way to specify database connection parameters is via a 'datasource' specified in your container's JNDI registry (Glassfish, JBoss, Websphere, ...). This datasource is then looked up by name in your persistence.xml. In non-J2EE applications it is more common to specify these in a Spring context or even a .properties file. In any case, you usually need something to connect your application to some sort of data store.
After bootstrapping to a data store an option is to manage config values inside this datastore. For example if you have a database you can use a separate table (represented by e.g. a JPA Entity in your application) for configuration values. If you don't want/need this flexibility you can use simple .properties file for this instead. There is good support for .properties files in Java (ResourceBundle) and in frameworks like Spring. The vanilla ResourceBundle just loads the properties once, the Spring helper offers configurable caching and reloading (this helps with the performance aspect which you mentioned). Note: you can also use Properties backed by a data store instead of a file.
Often both approaches coexist in an application. Values that never change within a deployed application (like the application name) can be read from a properties file. Values that might need to be changed by an application maintainer at runtime without redeployment (e.g. the session timeout interval) might better be kept in a reloadable .properties file or in a database. Values that can be changed by users of the application should be kept in the application's data store and usually have an in-application screen to edit them.
So my advise is to separate your configuration settings into categories (e.g. bootstrap, deployment, runtime and application) and select an appropriate mechanism to manage them. This also depends on the scope of your application, i.e. is it a J2EE web app, a desktop app, command-line utility, a batch process?
What kind of configuration file do you have in mind? If it is a properties file, this might suit you:
public class Configuration {
// the configuration file is stored in the root of the class path as a .properties file
private static final String CONFIGURATION_FILE = "/configuration.properties";
private static final Properties properties;
// use static initializer to read the configuration file when the class is loaded
static {
properties = new Properties();
try (InputStream inputStream = Configuration.class.getResourceAsStream(CONFIGURATION_FILE)) {
properties.load(inputStream);
} catch (IOException e) {
throw new RuntimeException("Failed to read file " + CONFIGURATION_FILE, e);
}
}
public static Map<String, String> getConfiguration() {
// ugly workaround to get String as generics
Map temp = properties;
Map<String, String> map = new HashMap<String, String>(temp);
// prevent the returned configuration from being modified
return Collections.unmodifiableMap(map);
}
public static String getConfigurationValue(String key) {
return properties.getProperty(key);
}
// private constructor to prevent initialization
private Configuration() {
}
}
You could also return the Properties object immediately from the getConfiguration() method, but then it could potentially be modified by the code that access it. The Collections.unmodifiableMap() does not make the configuration constant (since the Properties instance gets its values by the load() method after it was created), however since it is wrapped in an unmodifiable map, the configuration cannot be changed by other classes.
Well this is a great problem which is faced in every one's life once in a will. Now coming to the problem, this can be solved by creating a singleton class which has instance variables same as in configuration file with default values. Secondly this class should have a method like getInstance() which reads the properties once and every times returns the same object if it exists. For reading file we can use Environmental variable to get path or something like System.getenv("Config_path");. Reading the properties (readProperties() method) should read each item from config file and set the value to the instance variables of singleton object. So now a single object contains all the configuration parameter's value and also if the parameter is empty than default value is considered.
One more way is to define a class and read the properties file in that class.
This class needs to be at the Application level and can be marked as Singleton.
Marking the class as Singleton will avoid multiple instances to be created.
Putting configuration keys directly to classes is bad: configuration keys will be scattered over the code. Best practice is separation of application code and configuration code. Usually dependency injection framework like spring is used. It loads a configuration file and constructs the objects using configuration values. If you need some configuration value in your class you should create a setter for this value. Spring will set this value during context initialization.
I recommend using JAXB or a similar binding framework that works with text based files. Since a JAXB implementation is part of the JRE, it's pretty easy to use. As Denis I advise against configuration keys.
Here is a simple example for an easy to use and still pretty mighty way to configure you application with XML and JAXB. When you use a DI framework you can just add a similar config object to the DI context.
#XmlRootElement
#XmlAccessorType(XmlAccessType.FIELD)
public class ApplicationConfig {
private static final JAXBContext CONTEXT;
public static final ApplicationConfig INSTANCE;
// configuration properties with defaults
private int number = 0;
private String text = "default";
#XmlElementWrapper
#XmlElement(name = "text")
private List<String> texts = new ArrayList<>(Arrays.asList("default1", "default2"));
ApplicationConfig() {
}
static {
try {
CONTEXT = JAXBContext.newInstance(ApplicationConfig.class);
} catch (JAXBException ex) {
throw new IllegalStateException("JAXB context for " + ApplicationConfig.class + " unavailable.", ex);
}
File applicationConfigFile = new File(System.getProperty("config", new File(System.getProperty("user.dir"), "config.xml").toString()));
if (applicationConfigFile.exists()) {
INSTANCE = loadConfig(applicationConfigFile);
} else {
INSTANCE = new ApplicationConfig();
}
}
public int getNumber() {
return number;
}
public String getText() {
return text;
}
public List<String> getTexts() {
return Collections.unmodifiableList(texts);
}
public static ApplicationConfig loadConfig(File file) {
try {
return (ApplicationConfig) CONTEXT.createUnmarshaller().unmarshal(file);
} catch (JAXBException ex) {
throw new IllegalArgumentException("Could not load configuration from " + file + ".", ex);
}
}
// usage
public static void main(String[] args) {
System.out.println(ApplicationConfig.INSTANCE.getNumber());
System.out.println(ApplicationConfig.INSTANCE.getText());
System.out.println(ApplicationConfig.INSTANCE.getTexts());
}
}
The configuration file looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<applicationConfig>
<number>12</number>
<text>Test</text>
<texts>
<text>Test 1</text>
<text>Test 2</text>
</texts>
</applicationConfig>
protected java.util.Properties loadParams() throws IOException {
// Loads a ResourceBundle and creates Properties from it
Properties prop = new Properties();
URL propertiesFileURL = this.getClass().getResource("/conf/config.properties");
prop.load(new FileInputStream(new File(propertiesFileURL.getPath())));
return prop;
}
Properties prop = loadParams();
String prop1=(String) prop.get("x.y.z");
Given the prevalence of YML to express configuration, I'd recommend creating a YML file with the configuration inside it and then loading that once, at startup, into a POJO, then accessing the fields of that POJO to get the configuration:
user: someuser
password: somepassword
url: jdbc://mysql:3306/MyDatabase
With Java Class
public class Config {
private String user;
private String password;
private String url;
// getters/setters
Jackson can be used to load YML as can SnakeYml directly.
On top of this, you could use the OS project I've been working on - https://github.com/webcompere/lightweight-config - which allows you to wrap this up, and even express placeholders in your file to interpolate environment variables:
user: ${USER}
password: ${PASSWORD}
url: jdbc://${DB_HOST}:3306/MyDatabase
then
Config config = ConfigLoader.loadYmlConfigFromResource("config.yml", Config.class);
So I have a class like so:
public class HBaseUtil {
private final String fileName = "hbase.properties";
private Configuration config;
private HBaseUtil() {
try {
config = new PropertiesConfiguration(fileName);
} catch (ConfigurationException e) {
// some exception handling logging
}
}
// now some getters pulling data out of the config object
public static String getProperty(String fieldKeyName) {...}
public static String getColumnFamily(String fieldName) {...}
// ... some more getters
// NO setters (thus making this a read-only class)
}
Thus, basically I have for myself a Singleton class, that the very first time that it is put to use, sets up a configuration object, and then simply keeps listening for get calls. There are a number of problems with this class:
Unit testing the static methods within class HBaseUtil becomes difficult because of a tight-knit coupling between the Singleton and the configurations file.
What I really want is me being able to supply the filename/filename+path to the class so that it can go in there, read the configuration properties from that file and offer them to incoming read requests. One important note here though: I need this flexibility in specifying the properties file ONLY ONCE per JVM launch. So I certainly don't need to maintain state.
Here is what I was able to come up with:
Instead of a Singleton, I have a normal class with all static methods and no explicit constructor defined.
public class HBaseUtil {
// directly start with getters
public static String getProperty(Configuration config, String fieldKeyName) {...}
public static String getColumnFamily(Configuration config, String fieldKeyName) {...}
// ...and so on
}
And then, instead of using the class in my other code like such:
HBaseUtil.getProperty(String fieldKeyName)
I'd use it like so:
Configuration externalConfig = new PropertiesConfiguration("my-custom-hbase.properties");
HbaseUtil.getProperty(externalConfig, fieldKeyName)
My questions:
Am I even thinking in the right direction? My requirement is to have the flexibility in the class only ONCE per JVM. All that needs to be configurable in my project for this, is the location/contents of the HBase .properties file. I was thinking having a Singleton is overkill for this requirement.
What other better approaches are there for my requirement (stated in above point)?
Thanks!
Note: I've read this StackOverflow discussion, but now it's gotten me even more confused.
You should avoid all static methods and instead design a class which does not mandate its lifecycle: it can be a typical immutable POJO with a public constructor.
Then, when you need it as a singleton, use it as a singleton. For testing, use it in some other way.
Usually, dependency injection is the preferred avenue to solve these problems: instead of hard-coding a pulling mechanism for your configuration object, you have the object delivered to any class which needs it. Then you can decide late what bean you will deliver.
Since you are probably not using Spring (otherwise dependency injection would be your default), consider using Guice, which is a very lightweight and non-intrusive approach to dependency injection.