I’m writing this post to ask for a assistance with AWS s3 sdk. I can’t find any example of how to approach with MFA and java sdk. In the web console I’m using security token for login then switching role. In the terminal using saml2aws for credentials for one hour. And Not sure how to approach?
The Java SDK for AWS supports temporary credentials obtained from the Security Token Service. They can be stored, for example, in your .aws/credentials file or as environment variables.
You can obtain temporary credentials using a long-term aws_access_key_id/aws_secret_access_key pair and the AWS CLI. For example:
My .aws/credentials file looks like this:
[long-term]
aws_access_key_id=<VALID_AWS_ACCESS_KEY_ID>
aws_secret_access_key=<VALID=_AWS_SECRET_ACCESS_KEY>
And I obtain a new set of temporary credentials by running:
$ aws sts get-session-token --profile long-term --serial-number <MFA_DEVICE_ARN> --token-code <MFA_TOKEN>
Related
I'm new to Google's Cloud services and to Java, but I'm trying to set up a Java function on AWS Lambda that makes a call to Google Cloud KMS. I have working Java code locally, but from what I can tell the only way to authenticate the Google client is to set an environment variable with the path to a JSON file containing your credentials. I can do that easily locally when triggering my Java function - I just set the environment variable pointing to a file on my computer when running the code. Can anyone give me any pointers for how to do that in Lambda where all I seem to be able to do is upload a single .jar file?
You will probably want to store the JSON file in AWS Secret Manager and then retrieve the JSON file at function boot by authenticating to the AWS Secret Manager. Then you should configure the Google client library to use that credential contents.
Alternatively (more complex but also more secure) would be to configure GCP as an OIDC provider for AWS and then create an AWS role with permission to call GCP KMS directly - no credential file required.
I am developing a Google App Engine (Standard environment) application which uses Google Cloud Storage. I have used App Engine APIs for Cloud Storage until now, which provides a local emulation for the Cloud Storage using Datastore. As those APIs are getting obsolete now, I have decided to use the recommended APIs, however I am struggling with the credentials when running on the Local Server (I am already using the new Cloud Code plugin, not the old App Engine one).
I have created a service account and I have created and downloaded the key for it. If I would be running a normal Java app, I would be able to specify enviroment variables for the VM and I could provide the necessary -DGOOGLE_APPLICATION_CREDENTIALS=xxxxx.json parameters. The server provided by the Cloud Code does not seem to have any way how to provide environment variables, I can only provide VM options, therefore I do not know how can I provide the necessary environment to it, or how to pass the credentials to it in some other way. The only way I got it kind of working was using
gcloud auth application-default login
which has saved credentials in D:\Users\xxxx\AppData\Roaming\gcloud\application_default_credentials.json. This works, but any time I am debugging my application, I get following warning:
com.google.auth.oauth2.DefaultCredentialsProvider warnAboutProblematicCredentials
WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error.
I am not sure how serious this warning is, but it sure sounds scary to me.
In my application I use this code (Scala, Java would be very similar) to create the service with the credentials:
val credentials = GoogleCredentials.getApplicationDefault
val storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService
What is the proper way to pass service account credentials when running on a local Google App Engine server?
The issue with the big warning is that Google does not want you to use User Credentials in place of Service Account credentials. Google is locking down (restricting) what scopes/data third-party services (you) can request. My advice is to not use User Credentials anymore as they will eventually no longer work.
There are several methods to solve this.
Method 1: Setup the CLI to use a service account:
gcloud auth activate-service-account test#development.iam.gserviceaccount.com --key-file=test_google_account.json
Use the correct email address for the service account. This can be found in the Google Cloud console and also in the JSON file. Google libraries will find these credentials. On my website, I have written several articles on the details of services accounts, Application Default Credentials (ADC), etc.
Method 2: Specify the service account in your code
credentials = GoogleCredentials.fromStream(new FileInputStream(service_account_filename))
Create a flag or environment variable so that your code can if-else decide when running on your desktop to process credentials.
Method 3:
If the system (not the VM command line) environment variable GOOGLE_APPLICATION_CREDENTIALS is set, the libraries will use the filename that the variable points to for service account credentials. This file is a Google Cloud Service Account credentials file in JSON format.
Set this environment variable before you launch your IntelliJ.
My Document Links:
https://www.jhanley.com/google-cloud-application-default-credentials/
https://www.jhanley.com/google-cloud-setting-up-gcloud-with-service-account-credentials/
https://www.jhanley.com/google-cloud-creating-and-authorizing-service-account-credentials-with-the-cli/
I'm working on an app that will run on the Google Cloud Platform, and it needs to authenticate to the Google Admin 'Directory' SDK. The recommended approach that everyone (including Google) seems to suggest, is to create a service account in the Google Cloud Platform, and use the private key credentials for the service account to authenticate within the app that's running on Google Cloud Platform. Here is Google's recommended approach:
Perform G Suite Domain-Wide Delegation of Authority
To summarise the approach, for a Java app it requires the following:
Upload the private key file to a file location available application (the private key file can be JSON instead of P12 as suggested in the above guide)
Programatically load the file in the application, and use its credentials to authenticate
My question is this - if the file is directly uploaded to the src/main/resources folder for the application that runs on the Google Cloud Platform, is this is a significant security risk? How easy would it be for a hacker to access the file? If it's a risk, then what is a secure alternative to this approach?
For this answer I will assume that you are running on Google Compute Engine. The same type of answer will apply to App Engine, Containers, Kubernetes and Cloud Functions.
My question is this - if the file is directly uploaded to the
src/main/resources folder for the application that runs on the Google
Cloud Platform, is this is a significant security risk?
This is a security risk and is a very poor security practice. Numerous prominent companies have been breached with credentials stored in source code.
How easy would it be for a hacker to access the file?
Unknown. There are many different types of breaches. If a hacker obtains shell access to your instance you are in serious trouble. If a hacker obtains root shell access, they can do almost anything they want.
If it's a risk, then what is a secure alternative to this approach?
Best practices for security regarding credentials: do not store credentials in your source code or on your computing resources.
Normally you would access your credentials from Compute Engine Metadata server. These credentials are created by Google Cloud when your instance starts up. You can control these permissions in the Google Console under "Cloud API access scopes" OR via a service account that you specify for the VM.
However, with G-Suite, you need to create delegated credentials. I do not recommend mixing the credentials that are used by your VM with the credentials used for G-Suite. This means that you still need access to credentials in Json format (or P12 for legacy applications).
Since best security practices are to not store these credentials in source code or on the instance, you need to store them securely someplace else that you can access securely. One option is Google Cloud Storage. Assign a Cloud Storage read-only scope to your instance so that you can read the credentials Json file from Cloud Storage. Read the credentials using your SDK directly into memory and do not involve disk operations or utilities such as gsutil. I recommend that you create a seperate bucket for privileged files such as credentials.
Note that you are using several credentials. 1) the credentials stored in Google's metadata server. 2) the credentials that you download from Cloud Storage. 3) the delegated credentials that you create to access G-Suite and other Google applications.
The first set of credentials (Application Default Credentials - ADC) are used for normal cloud access including reading the second set of credentials from Cloud Storage. The third set are created in your software.
The credentials that you store on Cloud Storage do not need any privileges except domain wide delegation. The privileges are added via scopes when you create the G-Suite credentials (credentials #3).
Note: Do not store the email address used for delegated credentials in your source code. Store this in another Json file on Cloud Storage.
I am trying to use the approach outlined in the following blog article to authenticate a cognito identity to S3 from Java:
https://aws.amazon.com/blogs/mobile/understanding-amazon-cognito-authentication-part-4-enhanced-flow/
I have successfully implemented the developer authentication provider and can retrieve a valid OpenId token for my cognito identity. That token works in both the iOS and Android SDKs when uploading/downloading files to S3. Unfortunately, I have not been able to successfully authenticate the same identity to S3 in my Java app. Here is the code I am using:
Map<String, String> logins = new HashMap();
logins.put("cognito-identity.amazonaws.com", cognitoOpenIdToken);
GetCredentialsForIdentityRequest getCredentialsRequest =
new GetCredentialsForIdentityRequest()
.withIdentityId(cognitoIdentityId)
.withLogins(logins);
AmazonCognitoIdentityClient cognitoIdentityClient =
new AmazonCognitoIdentityClient();
GetCredentialsForIdentityResult getCredentialsResult =
cognitoIdentityClient.getCredentialsForIdentity(getCredentialsRequest);
The call to getCredentialsForIdentity fails with the error "com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain". This method is documented as a public API that does not require authentication. However, I have noticed that the call succeeds if done from an environment where there are valid IAM credentials configured in the system variables. It fails in environments where that is not the case.
Am I missing something simple?
Have you tried initializing the AmazonCognitoIdentityClient in following manner?
new AmazonCognitoIdentityClient(new AnonymousAWSCredentials());
Otherwise, by design of AWS Java SDK, the service clients look for AWS credentials, because same client can be used for accessing Authenticated APIs as well.
maybe is not that you need to be as another IAM user... but that the user that is running you method has a "deny" on sts:* (or some other actions that you need)
I created IAM role associated with the EC2 instance on AMAZON and as I understood from the amazon docs I can retrieve temp AWS credentials and do some stuff with that.I read that the EC2 metadata api(which is used internally by InstanceProfileCredentialsProvider) is only available for calls from within the instance, not from the outside world.
What this means? How can I get secure communication with AWS when develop app on local tomcat server?
You should use the The default provider chain and EC2 instance profiles. In your case, since you've already added the role to your instance, and considering you are using the Java SDK, you need to call:
InstanceProfileCredentialsProvider mInstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider();
AWSCredentials credentials = mInstanceProfileCredentialsProvider.getCredentials();
Or, if you are using a specific service, such as AWS S3, you can directly call:
AmazonS3 s3Client = new AmazonS3Client(new DefaultAWSCredentialsProviderChain());
For more information: http://docs.aws.amazon.com/java-sdk/latest/developer-guide/java-dg-roles.html
And just a reminder: you should NEVER leave your Access Key and Key Secret in your code.
It appears that your situation is:
You have an application running on a computer that is not an Amazon EC2 instance
You wish to give that application the ability to make API calls to AWS services
In this situation, it is not appropriate to use an IAM role.
Instead, you will need to provide your application with a set of valid AWS credentials (Access Key + Secret Key). This can be done by creating an IAM User, copying the credentials provided and placing them in your application's configuration.
When making an API call from an application that uses an AWS SDK, the SDK will automatically look in various location for valid credentials. In the case of Java, the DefaultAWSCredentialsProviderChain that looks for credentials in this order:
Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (RECOMMENDED since they are recognized by all the AWS SDKs and CLI except for .NET), or AWS_ACCESS_KEY and AWS_SECRET_KEY (only recognized by Java SDK)
Java System Properties - aws.accessKeyId and aws.secretKey
Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable,
Instance profile credentials delivered through the Amazon EC2 metadata service
Therefore, store the credentials in one of the first three options.