TL;DR
How to have system properties on CLI or environment variables override properties that are provided by an Azure App Configuration Store? Or is this maybe not possible at all?
Longer story
Let us assume a property named app.prop. Let us further assume the following entry in application.yml or in application-<profile>.yml:
app:
prop: Default
Usually, you are able to start the Spring Boot application and provide a system property (e.g. -Dapp.prop=SYS) or an environment variable (e.g. export APP_PROP=ENV) with the effect that the latter overrides the value of the YML config files. If you - for example - provided the environment variable, your application has the value ENV for the property app.prop.
When reading the same property from an Azure App Configuration Store, you can provide a system property or an environment variable as you like. But the value is not overridden anymore; it is the value that is stored in the Azure App Configuration Store.
Some code
I am using Spring Boot version 2.5.7:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.7</version>
</parent>
Further, I am using the following library for accessing the Azure App Configuration Store:
<properties>
<azure-spring-cloud.version>2.7.0</azure-spring-cloud.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>azure-spring-cloud-dependencies</artifactId>
<version>${azure-spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>azure-spring-cloud-appconfiguration-config</artifactId>
</dependency>
</dependencies>
Additionally, for starting the application, I am providing the following property:
spring.cloud.azure.appconfiguration.stores[0].connection-string = ...
This all works very well. In the Azure App Configuration Store, I have the following property:
app.prop = Azure
If now starting the application with the following environment variable APP_PROP = ENV, the value of the property app.prop is still Azure, and not ENV.
Is there any setting missing, so that I can have the same behavior that I had without the above mentioned library?
Actually, I searched a lot, but did not find anything except for some statements regarding overriding values of remote properties in the Spring Cloud documentation, which is not really my case (I am using Azure App Configuration Store).
The whole point of using Azure App Configuration is to store your config in one place and easily manage it without redeploying / restarting the app. Therefore I don't think this is should be even possible.
I would recommend to use labels to load specific version of your prop based on labels data. Few cases:
If you need this property only locally, don't specify it in App Configuration.
If you need multiple versions of it, then just create same property with multiple labels and use your spring.profile (or other conf-property) to distinguish the version.
If you need to load multiple versions of it, load multiple labels:
As described here: https://learn.microsoft.com/en-us/java/api/overview/azure/spring-cloud-starter-appconfiguration-config-readme?view=azure-java-stable#load-from-multiple-labels
You can use this sample to see how it works:
https://github.com/Azure/azure-sdk-for-java/tree/azure-spring-boot_3.6.0/sdk/appconfiguration/spring-cloud-starter-azure-appconfiguration-config
Related
Small question on how to disable Spring Cloud Kubernetes in local mode please.
The project is a simple SpringBoot + SpringCloud project deployed in Kubernetes.
Hence, there is this dependency in the class path:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-fabric8</artifactId>
</dependency>
And when we deployed the app in a Kubernetes environment, everything is fine.
However, the same app run in local mode will yield this warning, but most of all, a 20 seconds increased start time.
o.s.c.k.f.Fabric8AutoConfiguration : No namespace has been detected. Please specify KUBERNETES_NAMESPACE env var, or use a later kubernetes version (1.3 or later)
In local, while removing the dependency entirely, things are "back to normal". The message disappears, and the start up time comes back down.
However, commenting and uncommenting the dependency based on the local environment might not be the best solution.
Is there a property to disable Spring Cloud Kubernetes entirely that I can configure in local please?
Thank you
As the documentation says, you can do that by adding:
spring.cloud.kubernetes.enabled=false
that, in turn, could be an environment property that you can enable/disable per environment.
What worked for me was adding the spring.cloud.kubernetes.enabled=false property in the boostrap.properties/yaml file and not in the application.properties/yaml file.
Create the file "bootstrap.properties" into the resources folder
Then add the following lines:
spring.cloud.kubernetes.enabled=false
spring.cloud.kubernetes.discovery.enabled=false
I have a Spring Boot project that works perfectly when run in IDE. I would like to run this via AWS CodeStar. Unfortunately, the default Spring template created by CodeStar uses Spring MVC.
I cannot just overwrite the default Spring MVC project with my Spring Boot project (it doesn't work). I can copy some of my resources to the MVC project, for example index.html and that works. But then features like Thymeleaf don't work. For this and other reasons, I would like to change the provided Spring MVC into the Spring Boot structure I already have.
I followed the instructions here: https://www.baeldung.com/spring-boot-migration
Unfortunately, this doesn't help. I can create Application Entry Point and add Spring Boot dependencies without the app breaking. But when I remove the default dependencies or the configuration associated with the MVC, the app breaks. When trying to reach the URL, I get a 404 error with description:
The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.
Debugging this error message (e.g. https://www.codejava.net/java-ee/servlet/solved-tomcat-error-http-status-404-not-found) didn't help.
The message seems like it's connected to the web resource. I have my web resources in folder resources as well as webapp/resources. And Spring Boot doesn't need any location configuration, right? It uses this location by default.
Can somebody tell me what things to remove and what to add to be able to use my existing Spring Boot project?
EDIT:
This is a link to a default template for AWS CodeStar Spring web application: https://github.com/JanHorcicka/AWS-codestar-template
And this is my Spring Boot project structure:
I realize that you indicated that previously you tried to use your Spring Boot project with some modifications without success, but I think it could be actually a possibility to successfully deploy your application on AWS CodeStar, and it will be my advice.
I also realized that in your screenshot you included several of the required artifacts and classes, but please, double check that you followed these steps when you deployed your application to AWS CodeStar.
Let's start with a pristine version of your Spring Boot project running locally, without any modification, and then, perform the following changes.
First, as indicated in the GitHub link you shared, be sure that you include the following files in your project. They are required for the deployment infrastructure of AWS:
appspec.yml
buildspec.yml
template.yml
template-configuration.json
The whole scripts directory
Please, adapt any necessary configuration to your specific needs, especially, template-configuration.json.
Then, perform the following modifications in your pom.xml. Some of them are required for Spring Boot to work as a traditional deployment and others are required by the deployment in AWS CodeStar.
Be sure that you indicate packaging as war:
<packaging>war</packaging>
To ensure that the embedded servlet container does not interfere with the Tomcat to which the war file is deployed, either mark the Tomcat dependency as being provided as suggested in the above-mentioned documentation:
<dependencies>
<!-- … -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<!-- … -->
</dependencies>
Or exclude the Tomcat dependency in your pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
If necessary, apply this exclusion using some kind of profile that allows you to boot Spring Boot locally and in an external servlet container at the same time.
Next, parameterize the maven war plugin to conform to the AWS CodeStar deployment needs:
<build>
<pluginManagement>
<plugins>
<!-- ... -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<warSourceDirectory>src/main/webapp</warSourceDirectory>
<warName>ROOT</warName>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<!-- ... -->
<plugins>
</pluginManagement>
</build>
I do not consider it necessary, but just to avoid any kind of problem, adjust the name of your final build:
<finalName>ROOT</finalName>
Lastly, as also indicated in the Spring documentation, be sure that your MyProjectApplication - I assume this class is your main entry point subclass SpringBootServletInitializer and override the configure accordingly, something like:
#SpringBootApplication
public class MyProjectApplication extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(MyProjectApplication.class);
}
public static void main(String[] args) {
SpringApplication.run(MyProjectApplication.class, args);
}
}
Please, feel free to adapt the class to your specific use case.
With this setup, try to deploy your application and see if it works: perhaps you can find some kind of library dependencies problem, but I think for the most part it should work fine.
At a first step, you can try to deploy locally the version of the application you will later deploy to AWS CodeStar following the instructions you provided in your project template, basically, once configured with the necessary changes described in the answer, by running:
mvn clean package
And deploying the generated war on your local tomcat environment. Please, be aware that probably the ROOT application already exists in a standard tomcat installation (you can verify it by inspecting the webapps folder): you can override that war file.
For local testing you can even choose a different application name (configuring build.finalName and the warName in your pom.xml file): the important thing is verify if locally the application runs successfully.
If you prefer to, you can choose to deploy the app directly to AWS CodeStar and inspect the logs later it necessary.
In any case, please, pay attention on two things: on one hand, if you have any absolute path configured in your application, it can be the cause of the 404 issue you mention in the comments. Be aware that your application will be deployed in Tomcat with context root '/'.
On the other hand, review how you configured your database access.
Probably you used application.properties and it is fine, but please, be aware that when employing the application the database must be reachable: perhaps Spring is unable to create the necessary datasources, and the persistence manager or related stuff associated with and, as a consequence, the application is not starting. Again, it may be the reason of the 404 error code.
To simplify database connectivity, for testing, at first glance, I recommend you to use simple properties for configuring your datasource, namely the driver class, connection string, username and password. If that setup works properly, you can later enable JNDI or what deemed necessary.
Remember that if you need to change your context name and/or define a datasource pool in Tomcat you can place a context.xml file under a META-INF directory in your web app root path.
This context.xml should look like something similar to:
<?xml version="1.0" encoding="UTF-8"?>
<Context path="/">
<Resource name="jdbc/myDS"
type="javax.sql.DataSource"
maxActive="100"
maxIdle="30"
maxWait="10000"
url="jdbc:mysql://localhost:3306/myds"
driverClassName="com.mysql.jdbc.Driver"
username="root"
password="secret"
/>
</Context>
My SpringBoot project has the dependency
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-configuration2</artifactId>
<version>2.4</version>
</dependency>
And my bootstrap.properties file has lines such as aws.s3.name=${env:S3_NAME}
According to documentation on https://commons.apache.org/proper/commons-configuration/userguide/howto_basicfeatures.html, it is supposed to work with this syntax.
However when I try to use it:
#Value("${aws.s3.name}")
private String bucketName;
inside my #Service class, it is initialized to "S3_NAME".
Why? What am I doing wrong?
EDIT: I forgot to add that I am starting the application in a docker container, passing -e S3_NAME=some_bucket_name along with my docker run command
Turned out we were not using the correct tool (or correctly) - the right way was to move the environment variables properties from bootstrap.properties to application.properties - and then it started working!
I dont know why there is difference in the way those two files function in Spring Boot.
Intent : I am working on a POC which intends to use AWS Parameter store as a property store.This would store the confidential application properties in AWS SSM's Parameter store.I am using Java 8 with spring boot/cloud version 2.
Resource : I followed this ref guide from spring docs
and
also a comprehensive article Integrating the AWS Parameter Store with Spring Cloud .Hence was trying to utilize
spring-cloud-starter-aws-parameter-store-config.jar
and hence added required dependency in the build file.
Expected output :
Actual output :
Here is snapshot from AWS console I am trying to access below shown parameters from AWS parameter store
Below are my spring property files:
application.yml
bootstrap.yml
I am using maven with below dependencies in POM.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
</dependency>
</dependencies>
Am I missing something here?? Please let me know if someone has already faced and resolved this issue.
I am able to put and get parameter from command line,its just not able to get this java lib working.
GitHub repo of the sample I am trying -
GitHub repo link
I checked your app, it didn't work as expected for me as I had ~.aws/config file which leads to misconfiguration of AWS credentials(cause by DefaultAWSCredentialsProviderChain, read more here ), so I removed it, and I tried again but it fails saying that spring can't find aws region in the env, so apparently those specified in application.yml won't be used until spring loads properties from AWS parameter store.
How I made it work
I added:
System.setProperty("aws.accessKeyId","My_Key");
System.setProperty("aws.secretKey","Secret");
System.setProperty("aws.region","us-east-1");//same region where all your params exist
before SpringApplication.run(DemoApplication.class, args); and then it worked.
when changing the aws.region to another one where there are no params value defined I got the exact same result as yours (empty values).
make sure there isn't any aws config on your machine or EC2 instance that will override those provided in your app.
I have a maven project, and in the pom.xml I set properties as such:
<project>
<modelVersion>4.0.0</modelVersion>
<artifactId>myArtifact</artifactId>
<name>SomeProject</name>
<packaging>jar</packaging>
<properties>
<some-system-property>1.9.9</some-system-property>
</properties>
<...>
</project>
I want to pull the some-system-property value from within the java code, similar to
String someSystemPropery = System.getProperty("some-system-property");
But, this always returns null. Looking over StackOverflow, most of the answers seem to revolve around enhanced maven plugins which modify the code - something that's a nonstarter in my environment.
Is there a way to just get a property value from a pom.xml within the codebase? Alternatively, can one get the version of a dependency as described in the pom.xml (the 1.9.9 value below):
<dependencies>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.9</version>
</dependency>
</dependencies>
from code? Either one would solve my needs
Those are Maven properties that apply during the build, not runtime system properties. One typical approach is to use Maven resource filtering to write the value into a properties file in the target directory.
Maven properties and not system properties.
Generally you should set the system property for a maven plugin that is triggering the execution:
surefire for unit tests,
exec for execution,
jetty or similar for starting a web container
There is also properties maven plugin than can set properties:
http://mojo.codehaus.org/properties-maven-plugin/set-system-properties-mojo.html
Property values are accessible anywhere within a POM by using the notation ${X}, where X is the property, not outside. All properties accessible via java.lang.System.getProperties() are available as POM properties, such as ${java.home}, but not the other way around. So for your java code, it will need to scan the pom.xml as a xml parsing use case, but not sure why you want to do it.