I tried to ovveride the property
kafka.servers=s101lbakafpep1:9092,s102lbakafpep2:9092,s101lbakafpep3:9092
defined in my src/main/resources/config/application-kafka.properties file
with this value
kafka.servers=localhost:9092
defined in my src/main/resources/application-dev.properties file
I tried every combination possible reading the spring boot doc changing in my application.properties the order of
spring.profiles.active=config,health,planete,dgfip,mapping,kafka,dev
spring.profiles.active=dev,config,health,planete,dgfip,mapping,kafka
using spring.config.use-legacy-processing to true or false or .include, it's always the kafka config that wins
It's not working since i changed spring boot version to 2.4
Thanks for the very helpful hint #gviczai, solved my problem loading and overriding configs from YAML files.
I completely missed the following sentence in the documentation which made my unit tests fail because values have not been overridden as it was the case with Spring Boot 2.3.
Imports can be considered as additional documents inserted just below the document that declares them. They follow the same top-down ordering as regular multi-document
files: An import will only be imported once, no matter how many times it is declared.
So if you want to override imported values a new document has to be started after the import (--- in yaml, #--- in properties).
# imported-config.yaml
my-key: my-value
# application.yaml
spring:
config:
import:
- classpath:imported-config.yaml
# before starting a new document the value can not be modified, it would still be "my-value"
my-key: here-overriding-does-not-work
---
# after the start of the new document the value can be modified
my-key: my-overridden-value
In Spring Boot 2.4, configuration file handling is completely rethought and rewritten.
Long story short: Forget the legacy profile-dependent documents. From now on, you have to use only one big application.properties file, but it can be divided into various profile-activated sections. These sections then can come from other files or even documents from URLs - see cloud-config.
And the main rule is: definitions BELOW always overwrite definitions ABOVE. So be careful with the order the sections (thus profiles) follow each other! ;)
You can separate the sections with "#---" and you can define which profile activates the section by providing "spring.config.activate.on-profile=<your_profile>"
So, in your case your application.properties should look like this:
my.property=anything
...
server.name=myserver
#in your 'default' section, you can activate any profile, so it will be active by default
spring.profiles.active=kafka
#---
spring.config.activate.on-profile=kafka
spring.config.import=application-kafka.properties
#---
spring.config.activate.on-profile=dev
spring.config.import=application-dev.properties
#---
spring.config.activate.on-profile=cloud
spring.config.import=optional:configserver:http://my.config.server:8080/cloud-config
Of course, you can use yaml file if you prefer. In this case the document separator is the standard "---".
Read more about this new paradigm of config file processing here: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4
(And I guess 'kafka' profile wins over 'dev' because 'k' is AFTER 'd' in the abc... BTW, I think it is better not to name the imported documents according to the legacy profile-dependent "application-<profile>.properties" naming convention, because it may interfere with the profile-handling code. Better to be safe than sorry.)
Tip: Note, that in the same 'document' (a section in the same file considered a document) even the spring.config.import can overwrite previous values. So if you need to import multiple sources within the same section, use a comma-separated list:
spring.config.import=classpath:config/kafka.properties,classpath:db/postgres.properties
they're not in the same folder and the run configuration probably indicates /config for the scan.
It's working again with spring-boot 2.5.6, so it was fixed in 2.5.x
Related
We are working on moving our application to use only Spring-Boot application.properties files. The old way we were doing was that each library/dependency would have their properties stored in a dedicated properties file like res/environment/some-library-override.properties. The values would then be retrieved in the library using #Value("$some-library-{PROPERTY_NAME}").
However, since moving all of these override properties to dedicated application.properties files, it is no longer resolving the properties and we get errors like java.lang.NumberFormatException: For input string: "$some-library-{PROPERTY_NAME}".
I assume this is because it is still expecting the property to be in that dedicated properties file.
Is there a solution to this that doesn't involve modifying the library/dependency? Is it possible to have it ignore the prefix and only look for the PROPERTY_NAME in the application.properties files?
if you have declared propertie var likeproperty.name=XXXX or added environment var like PROPERTY_NAME=XXXX.
you need to use this way
#Value("some-library-${property.name}")
// will inject value "some-library" + "XXXX"
I am using Inifinispan v12.1 with String Boot v2.5.2 via org.infinispan:infinispan-spring-boot-starter-embedded. In our application we are using custom classes which we would like to cache (very common case), however it turned out that starting from v10 these classes need to be listed in "allow list".
We are using infinispan.xml configuration passed via infinispan.embedded.config-xml property as advised by sample project.
Question: How is it possible to configure allow list globally for all caches by the means of XML configuration file?
I have considered the following options:
System property infinispan.deserialization.allowlist.regexps (from ClassAllowList) – not good choice as configuration will be spread between XML file and e.g. some other place. More over if the property is renamed in future Infinispan versions one would notice it only when application is run.
Defining the <cache-container><serialization><allow-list> as to documentation is not good option because will result several identical per-cache XML configuration blocks.
The corresponding Java Config for Spring Boot application would be:
#org.springframework.context.annotation.Configuration
public class InfinispanConfiguration {
#Bean
public InfinispanGlobalConfigurationCustomizer globalCustomizer() {
return builder -> builder.allowList().addRegexp("^org\\.mycompany\\.");
}
}
P.S. Javadoc in GlobalConfiguration assumes that there is <default> XML section the configuration can be read from, but in fact XML does not support it anymore.
P.P.S. Arguably the dots in the packages should be escaped in SpringEmbeddedModule and start with ^ because ClassAllowList uses Matcher#find() (boolean regexMatch = compiled.stream().anyMatch(p -> p.matcher(className).find());):
serializationAllowList.addRegexps("^java\\.util\\..*", "^org\\.springframework\\..*");
I need to implement client code in java for using spring cloud config server. The client side is not a spring app nor a web service.
We will have an application.properties file. We will have an app-specific file. And we will have environment-specific files for both of them. My understanding of spring config precedence rules is that the precedence, from lowest to highest should be:
application.properties (lowest)
specificApp.properties
application-dev.properties
specificApp-dev.properties (highest)
However, when I call the service with http://localhost:8888/specifcApp/dev, the property sources are returned in the opposite order from the above:
0: specificApp-dev.properties
1: application-dev.properties
2: specificApp.properties
3: application.properties
I am not certain, but I believe that the intention is that the order returned indicates the precedence from lowest to highest (the last one wins). If that is true, then this is indicating that application.properties has precedence over the others. That is the opposite of what I would expect.
I have yet to find spring documentation that covers this completely all in one location. There is confusing information scattered around. But in this document, it says, "...profile-specific files always [override] the non-specific ones, whether or not the profile-specific files are inside or outside your packaged jar.
What are the actual precedence rules, and what does the order of the property sources in the response indicate?
The second part of the question is, how can I iterate over the configService property sources in ANY order?
I found the sample code below. However, the problem is that CompositePropertySource.getPropertySources() returns a Collection. In other words, there is no way to get an ordered list from CompositePropertySource. (The sources are stored internally in CompositePropertySource in a Set.)
final CompositePropertySource compositePropertySource
= (CompositePropertySource)environment.getPropertySources().get("configService");
if (compositePropertySource != null && compositePropertySource.getPropertySources() != null) {
// compositePropertySource.getPropertySources() is not ordered!
for (final Object mapPropertySourceObject : compositePropertySource.getPropertySources()) {
final MapPropertySource propertySource = (MapPropertySource)mapPropertySourceObject;
if (propertySource != null) {
p.putAll(propertySource.getSource());
LOG.info("fetched remote properties");
}
}
}
To summarize:
What is the precedence rule supposed to be?
What does the order of property sources indicate in the Json raw server response?
How can I access the actual order of the property sources that make up the configService composite property source?
I am using Spring Boot 1.5.14.RELEASE and Spring Cloud Edgware.SR4.
Edit: I see now that the CompositePropertySource propertySources set is of type
LinkedHashSet, so it does have an order. And CompositePropertySource.getPropertyNames(), it's clear that properties from property sources later in the list will override those earlier in the list. If that's true, then application.properties takes precedence over the app- or profile-specific configurations. I don't see how that can be the expected behavior. It means that no property can be overridden, in effect.
Edit #2: I am testing this using a Git repo. All four of the files are in the same folder within the repo.
Edit #3: I realized some properties influence this. When I submitted this, I should have stated that I was launching the client with JVM options: -Dspring.profiles.active=dev -Dspring.profiles.default=dev and -Dspring.cloud.config.profile=dev. I also tried it with just -Dspring.cloud.config.profile=dev.
Update: I am closer to a solution.
I tried the following:
Rename application.properties to application-default.properties.
Rename specificApp.properties to specificApp-default.properties.
Then I ran with -Dspring.profiles.active=dev,default. I had no -Dspring.profiles.default property and no -Dspring.cloud.config.profile property.
In this case, they were returned in this order:
specificApp-default.properties
application-default.properties
specificApp-dev.properties
application-dev.properties
This is still not what I would expect. It means that the shared application properties will override (i.e. take precedence over) the appSpecific properties. I would expect the following order, but am getting closer to a solution, I think:
application-default.properties
specificApp-default.properties
application-dev.properties
specificApp-dev.properties
or even this order:
application-default.properties
application-dev.properties
specificApp-default.properties
specificApp-dev.properties
How do I add global variables to an embedded Gremlin server instance?
Also, I want to avoid loading the server configuration from a file, although I can load resources from the classpath.
getGlobalBindings() on GremlinExecutor is indeed deprecated, but the javadoc explains how you should proceed:
replaced by getScriptEngineManager() to add global scoped bindings
directly to that object.
That comes from the 3.2.5 javadoc when it was originally deprecated in preparation for pretty large changes in 3.3.0 when new interfaces were implement to better generalize the GremlinScriptEngine. While these new interfaces were defined for default use in 3.3.0, they are actually present in 3.2.x and may be used there. Note that the getGlobalBindings() method was actually removed completely in 3.3.0 so when you upgrade you will end up with compilation errors.
Where there may be some confusion with respect to that javadoc comment is that to use the getScriptEngineManager() you must also use what is the default 3.3.0 yaml configuration on the 3.2.x line of code...an example is shown here:
https://github.com/apache/tinkerpop/blob/3.3.0/gremlin-server/conf/gremlin-server-classic.yaml#L25
Note that under this new model, you have two other options for adding global bindings...you could also either:
Use the BindingsGremlinPlugin to add global bindings programmatically
Write your own GremlinPlugin instance to add your bindings
Looks like we can do it this way, although getGlobalBindings() is deprecated.
Graph graph = this.createGraph();
GraphTraversalSource g = graph.traversal();
this.server = new GremlinServer(getSettings());
this.server.getServerGremlinExecutor().getGraphManager().putGraph("graph", graph);
this.server.getServerGremlinExecutor().getGremlinExecutor().getGlobalBindings().put("graph", graph);
this.server.getServerGremlinExecutor().getGremlinExecutor().getGlobalBindings().put("g", g);
this.server.start();
I need to decide which configuration framework to use. At the moment I am thinking between using properties files and XML files. My configuration needs to have some primitive grouping, e.g. in XML format would be something like:
<configuration>
<group name="abc">
<param1>value1</param1>
<param2>value2</param2>
</group>
<group name="def">
<param3>value3</param3>
<param4>value4</param4>
</group>
</configuration>
or a properties file (something similar to log4j.properties):
group.abc.param1 = value1
group.abc.param2 = value2
group.def.param3 = value3
group.def.param4 = value4
I need bi-directional (read and write) configuration library/framework. Nice feature would be - that I could read out somehow different configuration groups as different objects, so I could later pass them to different places, e.g. - reading everything what belongs to group "abc" as one object and "def" as another. If that is not possible I can always split single configuration object into smaller ones myself in the application initialization part of course.
Which framework would best fit for me?
Since you are saying that it is possible to also store objects in the config, I would suggest this:
http://commons.apache.org/configuration/
The simplest way to do this would be to use Simple XML. It can bind XML to Java POJOs in a very simple manner. Also, it is much faster than other such XML binding frameworks.
http://simple.sourceforge.net
Only 270K with no dependencies.
Please take a look at this URL: http://issues.apache.org/jira/browse/CONFIGURATION-394
The Configuration framework which we're looking for it is something on top of Apache Commons Configuration and must support Concurrency Issues, JMX issues and most of stores(e.g .properties file, .xml files or PreferencesAPI).
What weblogic team provides on 'Administration Console' is intersting which through it you can have transactional(atomic) updates on configurations so that are registered listeners be notified.
The Apache guys insist that this project is out of scopes of Commons Configuration, maybe!
I've attached a simple configuration framework, take look please