Due to some new security requirments the api I'm developing now is required to store several urls, azure account names etc. in the azure key vault, rather than in the application.yml config file.
The issue is that I'm having trouble authenticating / accessing the key vault client in a Local environment. I have very limited access to the azure functions / key vault itself so testing the new code I'm writing is near impossible at current:
public String getSecretFromKeyVault(String key) {
/**
* Breaks in the constructor call, as the system.env variables for MSI_ENDPOINT and MSI_SECRET are null.
**/
AppServiceMSICredentials credentials = new AppServiceMSICredentials(AzureEnvironment.AZURE);
KeyVaultClient client = new KeyVaultClient(credentials);
SecretBundle secret = client.getSecret("url-for-key-vault", key);
return secret.value();
}
I'm aware that the variables will be set in the cloud server, but my question is how can I best verify that the vault calls have been implemented properly(unit, integration, e2e local tests), and how would I manage to use key vault calls during local development / runtime?
The alternative to MSI would be to enter the client id and key manually, following authentication against the active directory. This could be a solution for local development, but Would still require the declaration of confidential information in the source code.
Ive also tried logging in to azure using az login before running the server but that didn't work either.
Does anyone have any suggestions on how I might resolve this issue, or what my best options are going forward?
Notes on application:
Java version: 8
Spring boot
Azure / vsts development and deployment environment
Since you're using spring-boot you may be better off using Microsoft's property source implementation that maps the keyvault properties into Spring properties and for local development and testing you set equivalent properties in property files.
Use Spring profiles. let's say you have azure and local profiles. In your application-azure.yml file configure your app to use keyvault:
# endpoint on the azure internal network for getting the identity access token
MSI_ENDPOINT: http://169.254.169.254/metadata/identity/oauth2/token
MSI_SECRET: unused
# this property triggers the use of keyvault for properties
azure.keyvault:
uri: https://<your-keyvault-name>.vault.azure.net/
Now you can inject secret properties from the spring context into your variables and they will be read from keyvault:
#Value("${superSecretValue}")
String secretValue;
To make this work locally for testing, in your application-local.yml file you can set the secret property to something appropriate:
superSecretValue: dummy-for-testing-locally
The Azure dependency you need to add to build.gradle is:
implementation "com.microsoft.azure:azure-keyvault-secrets-spring-boot-starter:2.1.6"
Run your spring-boot jar with azure as the active profile when deployed, and local when testing and developing away from azure. This is tested and working with azure java containers.
Related
Our application needs to connect to confluent kafka and thus we have the following setups inside application.yaml file
kafka:
properties:
sasl:
mechanism: PLAIN
jaas:
config: org.apache.kafka.common.security.plain.PlainLoginModule required username={userName} password={passWord};
The {userName} and {passWord} need to be replaced by value fetching from AWS secret manager. These are what I have done so far.
Step 1: Use the following maven dependency
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-secretsmanager</artifactId>
</dependency>
Step 2: Create a configuration class and create a method annotated with #Bean to init a AWSSecretsManager client object.And we can get some key value pairs by using AWSSecretsManager object.
// Create a Secrets Manager client
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(region)
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey)))
.build();
I have the following questions to ask:
How can we inject the value we get from secret manager and replace the placeholder in the application.yml file?
To access AWSSecretsManager we need to pass AWS accessKey and seretKey. What is a good practice to provide those two values?
Some more info:
our application will be running on AWS ECS
I wouldn't recommend doing this via Java code at all. I would totally remove the aws-java-sdk-secretsmanager dependency, and use the ECS support for injecting SecretsManager values as environment variables.
My answer here will focus on the Secrets Manager API part of your question
I recommend that you move from AWS SDK for Java V1 to AWS SDK for Java V2. You can find V2 Java Secret Manager examples here.
https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/secretsmanager
Here is the Service Client for V2.
SecretsManagerClient secretsClient = SecretsManagerClient.builder()
.region(region)
.credentialsProvider(ProfileCredentialsProvider.create())
.build();
In this example, I am using a ProfileCredentialsProvider that reads creds from .aws/Credentials. You can learn more about how V2 handles creds in the AWS Java V2 DEV Guide.
Using credentials
You cannot use ProfileCredentialsProvider in an app deployed to a container as this file structure not part of the container. So you can use Amazon ECS container credentials:
The SDK uses the ContainerCredentialsProvider class to load credentials from the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI system environment variable.
See point 5 in the above Doc.
I am working on Java Springboot with MongoDB using Kubernetes. Currently I just hard coded the URI in application properties and I would like to know
how can I access to the MongoDB credentials on Kubernetes with Java?
The recommended way of passing credentials to Kubernetes pods is to use secrets and to expose them to the application either as environment variables, or as a volume. The link above describes in detail how each approach works.
If I properly understood the question, it is specifically about Java Spring Boot applications running on Kubernetes.
Few options come to my mind...some not that secure or exclusive to running on Kubernetes but still mentioned here:
Environment variables with values in the deployment/pod configuration. Everyone with access to the configuration will be able to see them.
Use ${<env-var>} / ${<end-var>:<default-value>} to access the environment variables in Spring Boot's application.properties/.yaml file. For example, if DB_USERNAME and DB_PASSWORD are two such environment variables:
spring.data.mongodb.username = ${DB_USERNAME}
spring.data.mongodb.password = ${DB_PASSWORD}
...or
spring.data.mongodb.uri = mongodb://${DB_USERNAME}:${DB_PASSWORD}#<host>:<port>/<dbname>
This will work regardless whether the application uses spring.data.mongodb.* properties or properties with custom names injected in a #Configuration class with #Value.
Based on how the Java application is started in the container, startup arguments can be defined in the deployment/pod configuration, similarly to the bullet point above.
Environment variables with values populated from secret(s). Access the environment variables from SpringBoot as above.
Secrets as files - the secrets will "appear" in a file dynamically added to the container at some location/directory; it would require you to define your own #Configuration class that loads the user name and password from the file using #PropertySource.
The whole application.properties could be put in a ConfigMap. Notice that the properties will be in clear text. Then populate a Volume with the ConfigMap so that application.properties will be added to the container at some location/directory. Point Spring Boot to that location using spring.config.location as env. var, system property, or program argument.
Spring Cloud Vault
Some other external vault-type of secure storage - an init container can fetch the db credentials and make them available to the Java application in a file on a shared volume in the same pod.
Spring Cloud Config...even though it is unlikely you'd want to put db credentials in its default implementation of the server storage backend - git.
I have multiple microservices which communicates with each other through REST calls.
I have used spring boot and spring rest and have configured the URLS of the rest end points in application.properties file.
Now the problems is if the URL for one end point changes then I to have to manually modify all the property files of the services which are calling that particular end point which has got changed.
Is there a workaround for this so that the URLS can be somehow placed in a centralized location so that any modification does not impacts the other services which are using it.
You can use spring-cloud to achieve this. Usual way used in spring-cloud is by configuring the required properties in a git repo. And then those properties can be accessed by any micro-service you want with minimal configurations. You can refer projects in this repo
limits-services acts as a client that needs certain properties those are configured in spring-cloud-config-server. Hope this helps.
In case with microservices you can use Spring Cloud Config (Spring Cloud Config, Spring Cloud Config Server). It's very usefull and you can update your configuration at runtime.
Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system. With the Config Server you have a central place to manage external properties for applications across all environments. The concepts on both client and server map identically to the Spring Environment and PropertySource abstractions, so they fit very well with Spring applications, but can be used with any application running in any language. As an application moves through the deployment pipeline from dev to test and into production you can manage the configuration between those environments and be certain that applications have everything they need to run when they migrate.
As others have mentioned you can use Spring Cloud Config Server to remotly load your application configuration. All you need is git repository containing your configuration.
Spring cloud configuration supporst Git, database as your store for configuration.
Idea is to create an spring-boot app that can provide configuration to other applications.
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
You can configurae port and provide your git repository using key spring.cloud.config.server
server.port: 8888
spring.cloud.config.server.git.uri: file://${user.home}/config-repo
At client side, if you have spring-config in your classpath, application will try to connect to an application runnign at port 8888 to retrieve configuration.
More information can be found here.
may put configuration inside a database.
after that need have one centralize cache service that used by other services, can be .jar service,
then the values can be load inside a cache class in this service,
then in the front end side need have update button for updating the cache after modify the URL value in the database, so then all impacted services can use new value.
and also to be easier may have stand alone UI for update those configuration rather than updating database directly.
You can use Microconfig.IO to manage your service configuration and it's placeholders functionality to reference configuration values of certain services from others. So in your case you configure your deploy url in your server and put placeholders on it in your clients. This allows you to edit value only in one place and then everyone who depend on it will get it automatically.
I am using Xero's Java SDK to build my application. My application is now facing a requirement of having to work with several Xero private apps, therefore I need to manage and performing authentication (OAuth) via the key certificate file and appropriate consumer key and secret.
I was thinking to very simply store these details in a database table and retrieve them appropriately more or less as in the following:
// create a Xero config instance
Config config = JsonConfig.getInstance();
// build config file - details will be obtained from database
config.setConsumerKey("key");
config.setConsumerSecret("secret");
// this line will have me authenticate with the Xero service using the config file built
XeroClient client = new XeroClient(config);
The problem with this approach is that I am not pointing at the public_privatekey.pfx key file which is another essential element required to authenticate.
The reason why I am not doing so is that the SDK does not seem to support this using the Config instance as shown above - there is no option for me to select the appropriate public_private.pfx file (and neither an option for me to just load the contents of the file). It doesn't make sense to me that an SDK would be missing a feature, therefore questioning my approach; have I overlooked a detail or am I approaching the problem incorrectly?
Take a look at the read me under the heading Customize Request Signing
https://github.com/XeroAPI/Xero-Java/blob/master/README.md
You can provide your own signing mechanism by using the public XeroClient(Config config, SignerFactory signerFactory) constructor. Simply implement the SignerFactory interface with your implementation.
You can also provide a RsaSignerFactory using the public RsaSignerFactory(InputStream privateKeyInputStream, String privateKeyPassword) constructor to fetch keys from any InputStream.
I need to call a web service with a java client.
This service authenticates clients through certificates at the message level (Ws-Security, not SSL).
It should be possible since, I can generate web services with JAX-WS with mutual certificate security in this dialog.
But I don't manage to create a client. Does anyone has an idea ?
I did not tried it myself, but from http://download.oracle.com/docs/cd/E17802_01/webservices/webservices/docs/2.0/tutorial/doc/ :
Configuring Message Security Using XWSS
The Application Server contains all of the JAR files necessary to use XWS-Security for securing JAX-WS applications, however, in order to view the sample applications, you must download and install the standalone Java WSDP bundle. You can download the Java WSDP from http://java.sun.com/webservices/downloads/webservicespack.html.
To add message security to an existing JAX-WS application using XWSS, follow these steps on the client side:
Create a client security configuration. The client security configuration file specifies the order and type of message security operations that will be used for the client application. For example, a simple security configuration to perform a digital signature operation looks like this:
<xwss:Sign id="s" includeTimestamp="true">
<xwss:X509Token encodingType="http://docs.oasis-
open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary"
valueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-
x509-token-profile-1.0#X509SubjectKeyIdentifier"
certificateAlias="xws-security-client"
keyReferenceType="Identifier"/>
</xwss:Sign>
</xwss:SecurityConfiguration>
</xwss:Service>
<xwss:SecurityEnvironmentHandler>
simple.client.SecurityEnvironmentHandler
</xwss:SecurityEnvironmentHandler>
For more information on writing and understanding security configurations and setting up SecurityEnvironmentHandlers, please see the Java Web Services Developer Pack 1.6 Tutorial at http://java.sun.com/webservices/docs/1.6/tutorial/doc/index.html.
In your client code, create an XWSSecurityConfiguration object initialized with the security configuration generated. Here is an example of the code that you would use in your client file. For an example of a complete file that uses this code, look at the example client in the \jaxws2.0\simple-doclit\src\simple\client\ directory.
FileInputStream f = new FileInputStream("./etc/client_security_config.xml");
XWSSecurityConfiguration config = SecurityConfigurationFactory.newXWSSecurityConfiguration(f);
Set security configuration information on the RequestContext by using the XWSSecurityConfiguration.MESSAGE_SECURITY_CONFIGURATION property. For an example of a complete file that uses this code, look at the example client in the \jaxws2.0\simple-doclit\src\simple\client\ directory.
// put the security config info
((BindingProvider)stub).getRequestContext().put(
XWSSecurityConfiguration.MESSAGE_SECURITY_CONFIGURATION,
config);
Invoke the method on the stub as you would if you were writing the client without regard to adding XWS-Security. The example for the application from the \jaxws2.0\simple-doclit\src\simple\client\ directory is as shown below:
Holder<String> hold = new Holder("Hello !");
stub.ping(ticket, hold);