In a multi-tenant Spring Boot application, I'm trying to load configuration objects. Ideally, I'd like to load certain properties file into a configuration object programmatically. I'm looking for a simple way to load the configuration by passing a properties file and the final class to map it to. The following is just an example of what I'm trying to achieve.
Directory structure of the configurations:
config/
- common.properties
all_tenants_config/
- foo_tenant/
- database.properties
- api.properties
- bar_tenant/
- database.properties
- api.properties
Configuration POJOs:
class DatabaseProperties {
#Value("${database.url}")
private String url;
}
class APIProperties {
#Value("${api.endPoint}")
private String endPoint;
}
Configuration Provider:
#Singleton
class ConfigurationProvider {
private Map<String, DatabaseProperties> DB_PROPERTIES = new HashMap<>();
private Map<String, APIProperties> API_PROPERTIES = new HashMap<>();
public ConfigurationProvider(#Value(${"tenantsConfigPath"}) String tenantsConfigPath) {
for (File tenant : Path.of(tenantsConfigPath).toFile().listFiles()) {
String tenantName = tenant.getName();
for (File configFile : tenant.listFiles()) {
String configName = configFile.getName();
if ("database.properties".equals(configName)) {
// This is what I'm looking for. An easy way to load the configuration by passing a properties file and the final class to map it to.
DB_PROPERTIES.put(tenant, SPRING_CONFIG_LOADER.load(configFile, DatabaseProperties.class));
} else if ("api.properties".equals(configName)) {
API_PROPERTIES.put(tenant, SPRING_CONFIG_LOADER.load(configFile, API.class));
}
}
}
}
public currentTenantDBProperties() {
return DB_PROPERTIES.get(CURRENT_TENANT_ID);
}
public currentTenantAPIProperties() {
return API_PROPERTIES.get(CURRENT_TENANT_ID);
}
}
In short, is there a way in Spring that allows to map a properties file to an object without using the default Spring's configuration annotations.
Well, in this case you do not need any Spring's feature.
Spring is a bean container, but in this place you just new an object by yourself and put it on your map cache.
Step 1: decode property file to Java Properties Class Object
Step 2: turn your properties object to your target object, just use some utils like objectmapper
FileReader reader = new FileReader("db.properties"); // replace file name with your variable
Properties p = new Properties();
p.load(reader);
ObjectMapper mapper = new ObjectMapper();
DatabaseProperties databaseProperties = mapper.convertValue(p,
DatabaseProperties.class);
I've been trying to wrap my head around this for hours but it doesn't make much sense what I'm doing wrong. I am trying to create a Map object in Java from a .yml file. Reason being for a map, I don't know what/how many children will appear under "present", so I rather have a dynamic way of creating a map object out of that..
Below is my .yml file. I would like the key-value pair under "present":
present:
now: LOCAL TESTING
later: testing
Below is my config class (everything commented out is what I've tried - in different combinations):
//#Data
#Component
#EnableConfigurationProperties
#ConfigurationProperties(prefix = "present")
//#ConfigurationProperties
public class stat {
//#Getter
//#Data
#Value("${present}")
private Map<String, String> present;
//private Map<String, String> present = new HashMap<String, String>();
}
I tried looking at other SO posts and I feel like I understand it but my Spring Boot (v1.5.8) application isn't seeing that value. It keeps throwing an error for me or the map object is null (or not being populated).
I know that I can read values from this .yml file because if I try to obtain a single value with a code snippet below, it works:
#Data
#Value("${present.now}")
private String status; // String value "LOCAL TESTING"
Here are the other links that I've tried:
Spring Boot yaml configuration for a list of strings
how to convert yml file to java pojo
Am I missing something obvious? Thanks!
You can try to create a POJO to represent the yml structure you are trying to read from.
For example:
#Configuration
#ConfigurationProperties(prefix = "present")
#Data
public class Present {
private String now;
private String later;
}
So I figured it out (for those who have this problem later):
#Value isn't needed and the prefix param in the #ConfigurationProperties isn't needed.
Then you need to have a getter method for the fields that you want - I thought the Lombok library had autogenerated these but I was wrong (probably need to read more about it later - #Setter and #Data won't work properly).
So it should look something like this:
#Component
#EnableConfigurationProperties
#ConfigurationProperties
public class stat {
private Map<String, String[]> present = new HashMap<String, String[]>();
public Map<String, String[]> getPresent() {
return present;
}
}
Now let's give a more complex example (nested maps). Say that my .yml file looks like this:
parent:
present:
foo: dey, tok
bar: ar, jerbs
later:
foo: day, dok
mar: r, darbs
The POJO would look like this:
#Component
#EnableConfigurationProperties
#ConfigurationProperties
public class stat {
private Map<String, Map<String, String[]>> parent = new HashMap<String, Map<String, String[]>>();
public Map<String, Map<String, String[]>> getParent() {
return parent;
}
}
Another key thing to note is that the field that you are obtaining a value from must match the variable name - it may not matter if you use prefix but it still didn't work for me. Hope this helps.
Is it possible to use Spring #Value, to map values from properties file to the HashMap.
Currently I have something like this, and mapping one value is not a problem.
But I need to map custom values in HashMap expirations.
Is something like this possible?
#Service
#PropertySource(value = "classpath:my_service.properties")
public class SomeServiceImpl implements SomeService {
#Value("#{conf['service.cache']}")
private final boolean useCache = false;
#Value("#{conf['service.expiration.[<custom name>]']}")
private final HashMap<String, String> expirations = new HashMap<String, String>();
Property file: 'my_service.properties'
service.cache=true
service.expiration.name1=100
service.expiration.name2=20
Is it posible to map like this key:value set
name1 = 100
name2 = 20
You can use the SPEL json-like syntax to write a simple map or a map of list in property file.
simple.map={'KEY1': 'value1', 'KEY2': 'value3', 'KEY3': 'value5'}
map.of.list={\
'KEY1': {'value1','value2'}, \
'KEY2': {'value3','value4'}, \
'KEY3': {'value5'} \
}
I used \ for multiline property to enhance readability
Then, in Java, you can access and parse it automatically with #Value like this.
#Value("#{${simple.map}}")
Map<String, String> simpleMap;
#Value("#{${map.of.list}}")
Map<String, List<String>> mapOfList;
Here with ${simple.map}, #Value gets the following String from the property file:
"{'KEY1': 'value1', 'KEY2': 'value3', 'KEY3': 'value5'}"
Then, it is evaluated as if it was inlined
#Value("#{{'KEY1': 'value1', 'KEY2': 'value3', 'KEY3': 'value5'}}")
You can learn more in the official documentation
Is it possible to use Spring #Value, to map values from properties file to the HashMap?
Yes, it is. With a little help of code and Spel.
Firstly, consider this singleton Spring-bean (you should scan it):
#Component("PropertySplitter")
public class PropertySplitter {
/**
* Example: one.example.property = KEY1:VALUE1,KEY2:VALUE2
*/
public Map<String, String> map(String property) {
return this.map(property, ",");
}
/**
* Example: one.example.property = KEY1:VALUE1.1,VALUE1.2;KEY2:VALUE2.1,VALUE2.2
*/
public Map<String, List<String>> mapOfList(String property) {
Map<String, String> map = this.map(property, ";");
Map<String, List<String>> mapOfList = new HashMap<>();
for (Entry<String, String> entry : map.entrySet()) {
mapOfList.put(entry.getKey(), this.list(entry.getValue()));
}
return mapOfList;
}
/**
* Example: one.example.property = VALUE1,VALUE2,VALUE3,VALUE4
*/
public List<String> list(String property) {
return this.list(property, ",");
}
/**
* Example: one.example.property = VALUE1.1,VALUE1.2;VALUE2.1,VALUE2.2
*/
public List<List<String>> groupedList(String property) {
List<String> unGroupedList = this.list(property, ";");
List<List<String>> groupedList = new ArrayList<>();
for (String group : unGroupedList) {
groupedList.add(this.list(group));
}
return groupedList;
}
private List<String> list(String property, String splitter) {
return Splitter.on(splitter).omitEmptyStrings().trimResults().splitToList(property);
}
private Map<String, String> map(String property, String splitter) {
return Splitter.on(splitter).omitEmptyStrings().trimResults().withKeyValueSeparator(":").split(property);
}
}
Note: PropertySplitter class uses Splitter utility from Guava. Please refer to its documentation for further details.
Then, in some bean of yours:
#Component
public class MyBean {
#Value("#{PropertySplitter.map('${service.expiration}')}")
Map<String, String> propertyAsMap;
}
And finally, the property:
service.expiration = name1:100,name2:20
It's not exactly what you've asked, because this PropertySplitter works with one single property that is transformed into a Map, but I think you could either switch to this way of specifying properties, or modify the PropertySplitter code so that it matches the more hierarchical way you desire.
From Spring 4.1.x ( I can't remember specific version though ), you can do something like
#Value("#{${your.properties.key.name}}")
private Map<String, String> myMap;
where your.properties.key.name in your properties file should be something like
your.properties.key.name={\
name1 : 100, \
name2 : 200 \
}
Just make sure that you should create PropertySourcesPlaceholderConfigurer bean to make it work both in your app and if you are writing any unit test code to test your code, otherwise ${...} placeholder for the property value won't work as expected and you'll see some weird SpringEL errors.
The quickest Spring Boot based solution I can think of follows. In my particular example I am migrating data from one system to another. That is why I need a mapping for a field called priority.
First I've created the properties file (priority-migration.properties) like such:
my.prefix.priority.0:0
my.prefix.priority.10:1
my.prefix.priority.15:2
my.prefix.priority.20:2
another.prefix.foo:bar
and put it on the classpath.
Assuming you want to use the map in a spring managed bean/component, annotate your class with:
#Component
#PropertySource("classpath:/priority-migration.properties")
What you actually want in your map is of course only the key/value pairs which are prefixed with my.prefix, i.e. this part:
{
0:0
10:1
15:2
20:2
}
To achieve that you need to annotate your component with
#ConfigurationProperties("my.prefix")
and create a getter for the priority infix. The latter proved to be mandatory in my case (although the Sring Doc says it is enough to have a property priority and initialize it with a mutable value)
private final Map<Integer, Integer> priorityMap = new HashMap<>();
public Map<Integer, Integer> getPriority() {
return priorityMap;
}
In the End
It looks something like this:
#Component
#ConfigurationProperties("my.prefix")
#PropertySource("classpath:/priority-migration.properties")
class PriorityProcessor {
private final Map<Integer, Integer> priorityMap = new HashMap<>();
public Map<Integer, Integer> getPriority() {
return priorityMap;
}
public void process() {
Integer myPriority = priorityMap.get(10)
// use it here
}
}
I make one solution inspired by the previous post.
Register property file in the Spring configuration:
<util:properties id="myProp" location="classpath:my.properties"/>
And I create component:
#Component("PropertyMapper")
public class PropertyMapper {
#Autowired
ApplicationContext applicationContext;
public HashMap<String, Object> startWith(String qualifier, String startWith) {
return startWith(qualifier, startWith, false);
}
public HashMap<String, Object> startWith(String qualifier, String startWith, boolean removeStartWith) {
HashMap<String, Object> result = new HashMap<String, Object>();
Object obj = applicationContext.getBean(qualifier);
if (obj instanceof Properties) {
Properties mobileProperties = (Properties)obj;
if (mobileProperties != null) {
for (Entry<Object, Object> e : mobileProperties.entrySet()) {
Object oKey = e.getKey();
if (oKey instanceof String) {
String key = (String)oKey;
if (((String) oKey).startsWith(startWith)) {
if (removeStartWith)
key = key.substring(startWith.length());
result.put(key, e.getValue());
}
}
}
}
}
return result;
}
}
And when I want to map all properties that begin with specifix value to HashMap, with #Value annotation:
#Service
public class MyServiceImpl implements MyService {
#Value("#{PropertyMapper.startWith('myProp', 'service.expiration.', true)}")
private HashMap<String, Object> portalExpirations;
Solution for pulling Map using #Value from application.yml property coded as multiline
application.yml
other-prop: just for demo
my-map-property-name: "{\
key1: \"ANY String Value here\", \
key2: \"any number of items\" , \
key3: \"Note the Last item does not have comma\" \
}"
other-prop2: just for demo 2
Here the value for our map property "my-map-property-name" is stored in JSON format inside a string and we have achived multiline using \ at end of line
myJavaClass.java
import org.springframework.beans.factory.annotation.Value;
public class myJavaClass {
#Value("#{${my-map-property-name}}")
private Map<String,String> myMap;
public void someRandomMethod (){
if(myMap.containsKey("key1")) {
//todo...
} }
}
More explanation
\ in yaml it is Used to break string into multiline
\" is escape charater for "(quote) in yaml string
{key:value} JSON in yaml which will be converted to Map by #Value
#{ } it is SpEL expresion and can be used in #Value to convert json int Map or Array / list Reference
Tested in a spring boot project
Use the same variable name as the Yaml name
Eg:
private final HashMap<String, String> expiration
instead of
private final HashMap<String, String> expirations
Or something similar to this in properties file
org.code=0009,0008,0010
org.code.0009.channel=30,40
org.code.0008.channel=30,40
org.code.0010.channel=30,40
in Java, read org.code and then loop thru each org.code and build org.code..channel and put it into a map....
I'm using Spring Boot 1.4.3 #AutoConfiguration where I create beans automatically based on properties user specifies. User can specify an array of services, where name and version are required fields:
service[0].name=myServiceA
service[0].version=1.0
service[1].name=myServiceB
service[1].version=1.2
...
If the user forgets to specify a required field on even just one service, I want to back-off and not create any beans. Can I accomplish this with #ConditionalOnProperty? I want something like:
#Configuration
#ConditionalOnProperty({"service[i].name", "service[i].version"})
class AutoConfigureServices {
....
}
This is the custom Condition I created. It needs some polishing to be more generic (ie not hardcoding strings), but worked great for me.
To use, I annotated my Configuration class with #Conditional(RequiredRepeatablePropertiesCondition.class)
public class RequiredRepeatablePropertiesCondition extends SpringBootCondition {
private static final Logger LOGGER = LoggerFactory.getLogger(RequiredRepeatablePropertiesCondition.class.getName());
public static final String[] REQUIRED_KEYS = {
"my.services[i].version",
"my.services[i].name"
};
#Override
public ConditionOutcome getMatchOutcome(ConditionContext context, AnnotatedTypeMetadata metadata) {
List<String> missingProperties = new ArrayList<>();
RelaxedPropertyResolver resolver = new RelaxedPropertyResolver(context.getEnvironment());
Map<String, Object> services = resolver.getSubProperties("my.services");
if (services.size() == 0) {
missingProperties.addAll(Arrays.asList(REQUIRED_KEYS));
return getConditionOutcome(missingProperties);
}
//gather indexes to check: [0], [1], [3], etc
Pattern p = Pattern.compile("\\[(\\d+)\\]");
Set<String> uniqueIndexes = new HashSet<String>();
for (String key : services.keySet()) {
Matcher m = p.matcher(key);
if (m.find()) {
uniqueIndexes.add(m.group(1));
}
}
//loop each index and check required props
uniqueIndexes.forEach(index -> {
for (String genericKey : REQUIRED_KEYS) {
String multiServiceKey = genericKey.replace("[i]", "[" + index + "]");
if (!resolver.containsProperty(multiServiceKey)) {
missingProperties.add(multiServiceKey);
}
}
});
return getConditionOutcome(missingProperties);
}
private ConditionOutcome getConditionOutcome(List<String> missingProperties) {
if (missingProperties.isEmpty()) {
return ConditionOutcome.match(ConditionMessage.forCondition(RequiredRepeatablePropertiesCondition.class.getCanonicalName())
.found("property", "properties")
.items(Arrays.asList(REQUIRED_KEYS)));
}
return ConditionOutcome.noMatch(
ConditionMessage.forCondition(RequiredRepeatablePropertiesCondition.class.getCanonicalName())
.didNotFind("property", "properties")
.items(missingProperties)
);
}
}
Old question, but I hope my answer will help for Spring2.x:
Thanks to #Brian, I checked migration guide, where I was inspired by example code. This code works for me:
final List<String> services = Binder.get(context.getEnvironment()).bind("my.services", List.class).orElse(null);
I did try to get List of POJO (as AutoConfigureService) but my class differs from AutoConfigureServices. For that purpose, I used:
final Services services = Binder.get(context.getEnvironment()).bind("my.services", Services.class).orElse(null);
Well, keep playing :-D
Here's my take on this issue with the use of custom conditions in Spring autoconfiguration. Somewhat similar to what #Strumbels proposed but more reusable.
#Conditional annotations are executed very early in during the application startup. Properties sources are already loaded but ConfgurationProperties beans are not yet created. However we can work around that issue by binding properties to Java POJO ourselves.
First I introduce a functional interface which will enable us to define any custom logic checking if properties are in fact present or not. In your case this method will take care of checking if the property List is empty/null and if all items within are valid.
public interface OptionalProperties {
boolean isPresent();
}
Now let's create an annotation which will be metannotated with Spring #Conditional and allow us to define custom parameters. prefix represents the property namespace and targetClass represents the configuration properties model class to which properties should be mapped.
#Target({ElementType.TYPE, ElementType.METHOD})
#Retention(RetentionPolicy.RUNTIME)
#Documented
#Conditional(OnConfigurationPropertiesCondition.class)
public #interface ConditionalOnConfigurationProperties {
String prefix();
Class<? extends OptionalProperties> targetClass();
}
And now the main part. The custom condition implementation.
public class OnConfigurationPropertiesCondition extends SpringBootCondition {
#Override
public ConditionOutcome getMatchOutcome(ConditionContext context, AnnotatedTypeMetadata metadata) {
MergedAnnotation<ConditionalOnConfigurationProperties> mergedAnnotation = metadata.getAnnotations().get(ConditionalOnConfigurationProperties.class);
String prefix = mergedAnnotation.getString("prefix");
Class<?> targetClass = mergedAnnotation.getClass("targetClass");
// type precondition
if (!OptionalProperties.class.isAssignableFrom(targetClass)) {
return ConditionOutcome.noMatch("Target type does not implement the OptionalProperties interface.");
}
// the crux of this solution, binding properties to Java POJO
Object bean = Binder.get(context.getEnvironment()).bind(prefix, targetClass).orElse(null);
// if properties are not present at all return no match
if (bean == null) {
return ConditionOutcome.noMatch("Binding properties to target type resulted in null value.");
}
OptionalProperties props = (OptionalProperties) bean;
// execute method from OptionalProperties interface
// to check if condition should be matched or not
// can include any custom logic using property values in a type safe manner
if (props.isPresent()) {
return ConditionOutcome.match();
} else {
return ConditionOutcome.noMatch("Properties are not present.");
}
}
}
Now you should create your own configuration properties class implementing OptionalProperties interface.
#ConfigurationProperties("your.property.prefix")
#ConstructorBinding
public class YourConfigurationProperties implements OptionalProperties {
// Service is your POJO representing the name and version subproperties
private final List<Service> services;
#Override
public boolean isPresent() {
return services != null && services.stream().all(Service::isValid);
}
}
And then in Spring #Configuration class.
#Configuration
#ConditionalOnConfigurationProperties(prefix = "", targetClass = YourConfigurationProperties.class)
class AutoConfigureServices {
....
}
There are two downsides to this solution:
Property prefix must be specified in two locations: on #ConfigurationProperties annotation and on #ConditionalOnConfigurationProperties annotation. This can partially be alleviated by defining a public static final String PREFIX = "namespace" in your configuration properties POJO.
Property binding process is executed separately for each use of our custom conditional annotation and then once again to create the configuration properties bean itself. It happens only during app startup so it shouldn't be an issue but it still is an inefficiency.
You can leverage the org.springframework.boot.autoconfigure.condition.OnPropertyListCondition class. For example, given you want to check for the service property having at least one value:
class MyListCondition extends OnPropertyListCondition {
MyListCondition() {
super("service", () -> ConditionMessage.forCondition("service"));
}
}
#Configuration
#Condition(MyListCondition.class)
class AutoConfigureServices {
}
See the org.springframework.boot.autoconfigure.webservices.OnWsdlLocationsCondition used on org.springframework.boot.autoconfigure.webservices.WebServicesAutoConfiguration#wsdlDefinitionBeanFactoryPostProcessor for an example within Spring itself.
I have a Spring managed bean...
#Component("Foobean")
#Scope("prototype")
public class foobean {
private String bar1;
private String bar2;
public String getBar1() {
return bar1;
}
public void setBar1(String bar1) {
this.bar1 = bar1;
}
public String getBar2() {
return bar2;
}
public void setBar2(String bar2) {
this.bar2 = bar2;
}
}
...and because I am using Dojo Dgrid to display an ArrayList of this bean, I am returning it into the controller as a JSON string:
#Controller
#RequestMapping("/bo")
public class FooController {
#Autowired
private FooService fooService
#RequestMapping("action=getListOfFoos*")
#ResponseBody
public String clickDisplayFoos(
Map<String, Object> model) {
List<Foobean> foobeans = fooService.getFoobeans();
ObjectMapper objMapper = new ObjectMapper();
String FooJson = null;
try {
FooJson = objMapper.writeValueAsString(foobeans);
} catch (JsonGenerationException e) {
etc.
}
However, my grid needs an additional column which will contain a valid action for each Foo; that action is not really dependent on any data in individual Foos -- they'll all have the same valid action -- repeated on each line of the resulting DGrid -- but that value is actually dependent upon security roles on the session...which can't be sent to the front end in a Json. So, my solution is twofold:
First I need to add a "virtual" Json property to the bean... which I can do in the bean with #JsonProperty on a method...
#JsonProperty("validActions")
public String writeValidActions {
return "placeHolderForSerializerToChange";
}
...but it just generates a placeholder. To really generate a valid value,
I need to reference the security role of the session,
which I am very reluctant to code in the above method. (A service call in
the domain bean itself? Seems very wrong.) I
think I should create a custom serializer and put the logic -- and the reference
to the Session.Security role in there. Are my instincts right, not to
inject session info into a domain bean method? And if so, what would such a
custom serializer look like?
Yes, I wouldn't put Session Info in to the domain or access session directly in my domain.
Unless there is a specific reason, you could simply add the logic in your action class.
public String clickDisplayFoos(){
List<Foo> foos = service.getFoos();
for(iterate through foos){
foo.setValidAction(session.hasSecurityRole())
}
String json = objMapper.writeValueAsString(foobeans);
return json;
}
I don't like the idea of setting new values as part of the serialization process. I feel custom serializers are meant to transform the representation of a particular property rather than add new values to a property.