I'm currently working on a an application with a microservice architecture. I have two services for now: product service and configuration service.
The configuration service plays the role of spring cloud config server, while the product service plays the role of spring cloud config client.
To launch my application, the configuration service should be run first, and then product service, this way product service can ask its configuration (application.properties) file to set up its connection to database etc. What I want now is to encrypt some sensitive properties like database credentials. Why? 'cause if you reach the endpoint of the cloud config service http://localhost:8888/productservice/env you can find these informations as plain text, which is not good.
For the time being I thought to use symetric encryption so following the documentation if I set a variable environement in my OS as ENCRYPT_KEY, my spring cloud config server should encrypt and decrypt my data.
I'm using windows 10 and using the set command I can see that the ENCRYPT_KEY is set properly with the correct secret word.
However when I try to reach the /encrypt endpoint with a POST method of my cloud config server, it responds with a 404 not found:
{
"description": "No key was installed for encryption service",
"status": "NO_KEY"
}
I'm using Edgware.RELEASE version of spring-cloud.
Thank you
I think the problem is that you set the environment variable with the set command.
When you set a variable with the set command that variable is available only in that CMD session.
What you need is to make the variable visible in your App. To achieve that you can set the environment variable through the System Properties option (tap Windows key and type edit the system environment variables).
Also make sure your app can read the variable value by doing something like System.getenv("ENCRYPT_KEY") in your Application main method.
By the way, if you are using Eclipse you can set environment variables in Run, Run Configurations..., Environment tab.
Related
I have a Spring Boot application which runs in Azure Kubernetes. The application uses Liquibase (Liquibase-Core 4.4.3) to create and amend database tables for the application on start up. For security reasons the application has different DB credentials to the Liquibase user and the Liquibase user credentials are added via the application.properties file as per below using environment variables:
spring.liquibase.url=${env_db_connection_url}
spring.liquibase.user=${env_db_lb_username}
spring.liquibase.password=${env_db_lb_password}
We add the credentials using this method with environment variables as the credentials are different depending on the environment (e.g. UAT, production) the app is being deployed to and this allows the pipeline to handle the differences without any changes being necessary to the code.
Our security team want us to stop using environment variables and so we are moving to using csi drivers for any sensitive information. This effectively creates something similar to an external properties file which sits outside the application but within the same container.
However, I am now having trouble resolving the db credentials for Liquibase and I don't know the best method to achieve what I need. I need a solution where we can still tokenize the value within the code but it is able to retrieve the secret values (db password etc) from the external properties file.
As you can imagine hard-coding these values or even just encrypting them wouldn't solve the issue due to the changing values between environments. We are able to retrieve the values within the application however I feel this is done too late in the process as Liquibase will have already attempted to start (and therefore fail) before the application code is run.
Any help or suggestions would be appreciated.
I used a method like this for a similar problem.
I'm running things on Jenkins server. But it's not necessary
I do not keep database connection information in files. After I send the files to the server, I add the database connection information to these files with bash script. The DB connection information is on the server, not the version control system.
Thus, I hide some db connection information from unauthorized people (only those who have access to the server can access it).
A- File example; liquibase.prod.properties
there is no db data
changeLogFile=./master.xml
logFile: liquibase.prod.log
logLevel: INFO
liquibase.hub.mode=off
B- Example of file where I keep database connections on server
spring.liquibase.url = my_env_db_connection_url
spring.liquibase.user= my_env_db_lb_username
spring.liquibase.password= my_env_db_lb_password
C- I am adding lines to this file with Jenkins and cmd bash
type E:\file1.cfg >> file2.cfg
This was the solution we took in the end. Effectively we worked out that the spring properties file could look directly in an external properties file (the csi driver in this case) to get the value.
Added to Spring properties file spring.config.import=optional:configtree:${env_var_for_external_property_file_location:#{null}}/liquibase/
For clarity, below is the secret provider class yaml which stores the liquibase password. The object alias value matching what Liquibase config expects for db password.
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: name
spec:
provider: azure
parameters:
keyvaultName: ****
tenantId: ****
objects: |
array:
- |
objectName: lqb
objectAlias: liquibase/spring.liquibase.password
objectType: secret
I'm developing a POC website with Spring Boot to learn about GCP products, in specific Google Cloud Storage. The context is I am trying to save a profile picture when a user is registered.
From Google documentation, I can use
StorageOptions.getDefaultInstance().getService();
to validate my credentials if I have GOOGLE_APPLICATION_CREDENTIALS on my environment variables. The fact is, I do have GOOGLE_APPLICATION_CREDENTIALS as an environment variable (I'm using Linux Mint) pointing to my .json file (that is valid because using the path method it works), but it always returns 401 UNAUTHORIZED from GCP lib.
After spending HOURS searching, just to make sure I run this command on terminal:
gcloud auth application-default login
The response from this command is:
The environment variable [GOOGLE_APPLICATION_CREDENTIALS] is set to:
[/home/<myuser>/<some_folder>/<myapplication>-<id>.json]
Credentials will still be generated to the default location:
[/home/<myuser>/.config/gcloud/application_default_credentials.json]
To use these credentials, unset this environment variable before
running your application.
Do you want to continue (Y/n)?
After confirming and login with my browser, to my surprise it worked using StorageOptions.getDefaultInstance().getService();
So I'm assuming that GCP takes credentials from application_default_credentials file instead of the GOOGLE_APPLICATION_CREDENTIALS env var.
So on my local environment, it's working fine. The problem is: This website is deployed on Heroku and I can't use the path method because I don't want to put this credentials on GitHub.
I already put the environment variables at Heroku and didn't work.
Already tried to use these buildpacks too:
https://github.com/elishaterada/heroku-google-application-credentials-buildpack
https://github.com/gerywahyunugraha/heroku-google-application-credentials-buildpack
but I can't get it to work when on Heroku Cloud. Someone knows a way I can make this run using the 12factor rules, please?
In order to pass credentials you can do it via environment variables, that in fact is the one you are mentioning. To do it in Linux you should use the below command
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
Nevertheless, something to keep in mind is that this variable only applies to your current shell session, so if you open a new session, set the variable again.
On the other hand, another option for perform this task is passing credentials using code, and as you can see in the shared link you should use something like below.
GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(jsonPath))
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();
Finally, in case you do not want to point the credentials inside of your applications's code, you can use Cloud Key Management Service. I think that KMS is a good option for your case but at the end, you should make the decision.
I am developing a Google App Engine (Standard environment) application which uses Google Cloud Storage. I have used App Engine APIs for Cloud Storage until now, which provides a local emulation for the Cloud Storage using Datastore. As those APIs are getting obsolete now, I have decided to use the recommended APIs, however I am struggling with the credentials when running on the Local Server (I am already using the new Cloud Code plugin, not the old App Engine one).
I have created a service account and I have created and downloaded the key for it. If I would be running a normal Java app, I would be able to specify enviroment variables for the VM and I could provide the necessary -DGOOGLE_APPLICATION_CREDENTIALS=xxxxx.json parameters. The server provided by the Cloud Code does not seem to have any way how to provide environment variables, I can only provide VM options, therefore I do not know how can I provide the necessary environment to it, or how to pass the credentials to it in some other way. The only way I got it kind of working was using
gcloud auth application-default login
which has saved credentials in D:\Users\xxxx\AppData\Roaming\gcloud\application_default_credentials.json. This works, but any time I am debugging my application, I get following warning:
com.google.auth.oauth2.DefaultCredentialsProvider warnAboutProblematicCredentials
WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error.
I am not sure how serious this warning is, but it sure sounds scary to me.
In my application I use this code (Scala, Java would be very similar) to create the service with the credentials:
val credentials = GoogleCredentials.getApplicationDefault
val storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService
What is the proper way to pass service account credentials when running on a local Google App Engine server?
The issue with the big warning is that Google does not want you to use User Credentials in place of Service Account credentials. Google is locking down (restricting) what scopes/data third-party services (you) can request. My advice is to not use User Credentials anymore as they will eventually no longer work.
There are several methods to solve this.
Method 1: Setup the CLI to use a service account:
gcloud auth activate-service-account test#development.iam.gserviceaccount.com --key-file=test_google_account.json
Use the correct email address for the service account. This can be found in the Google Cloud console and also in the JSON file. Google libraries will find these credentials. On my website, I have written several articles on the details of services accounts, Application Default Credentials (ADC), etc.
Method 2: Specify the service account in your code
credentials = GoogleCredentials.fromStream(new FileInputStream(service_account_filename))
Create a flag or environment variable so that your code can if-else decide when running on your desktop to process credentials.
Method 3:
If the system (not the VM command line) environment variable GOOGLE_APPLICATION_CREDENTIALS is set, the libraries will use the filename that the variable points to for service account credentials. This file is a Google Cloud Service Account credentials file in JSON format.
Set this environment variable before you launch your IntelliJ.
My Document Links:
https://www.jhanley.com/google-cloud-application-default-credentials/
https://www.jhanley.com/google-cloud-setting-up-gcloud-with-service-account-credentials/
https://www.jhanley.com/google-cloud-creating-and-authorizing-service-account-credentials-with-the-cli/
Following this tutorial: https://cloud.google.com/solutions/using-firebase-real-time-events-app-engine#securedata
I've got it working except that when I authenticate the custom token with Firebase, I get the following error:
Error: The custom token format is incorrect. Please check the documentation.
I have isolated the problem to the App Identity token signing. This is because the code works fine deployed to app-engine. It's just an issue when I try on devserver/localhost.
I have following the AppEngine documentation about setting up the default Service Account locally.
Setting the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of the .json service account keys. I've triple checked the variable is being set.
I tried the alternative method using the gcloud sdk, logging in as my owner account: gcloud beta auth application-default login. Still no luck.
To check I actually changed account, I use this method to get: AppIdentityServiceFactory.getAppIdentityService().getServiceAccountName();. In both configurations I detailed about, this method returns appname#localhost. I was expecting this to be the service account ID given in the .json. ie appname#appspot.gserviceaccount.com.
So I'm a bit at a loss how to debug this problem further, I assume the check I'm doing above is actually proving the validity of the configured service account. Perhaps its something unique to my setup?
Using App Engine Java, latest SDK 1.9.54
Run configuration launched with Maven 3 + IntelliJ
ArchLinux environment
I have GAE Application. It uses Oauth 2.0 protocol. I use Java library at the moment.
At the moment, I manually switch the redirect URL.
When I'm testing on localhost I use localhost:8888/oauth2callback
And when I deploy app, I set example.appspot.com/oauth2callback.
So how can I detect, if the app is running on the localhost or is deployed ?
As per the documentation, you can do the following
You can check the value of the System property com.google.appengine.runtime.environment to determine if it is "Production" or "Development".
Sample code from docs:
if (SystemProperty.environment.value() ==
SystemProperty.Environment.Value.Production) {
// The app is running on App Engine...
}
and so on.
On the Python side of things, check out the following documentation snippet:
SERVER_SOFTWARE: In the development web server, this value is "Development/X.Y" where "X.Y" is the version of the runtime. When running on App Engine, this value is "Google App Engine/X.Y.Z".
You can use oAuthService methods to detect localhost environment
OAuth for Java Overview
[...]
On the local development server, oauth.getCurrentUser() always returns
a User object with email set to "example#example.com" and user ID set
to 0 regardless of whether or not a valid OAuth request was made.
[...]