The use case is to pull the file from the S3 bucket in real time. Here we are using Spring cloud streaming. So I have created an S3Client bean that fetches any new data using a file synchronizer. But the token gets expired after one hour. So I am getting an error after every hour. Is there any way to refresh the token after some specified time?
**This is my code : **
#Bean
public S3Client s3Client() {
AwsCredentialsProvider awsCredentialsProvider =
StaticCredentialsProvider.create(AwsBasicCredentials.create(ACCESS_KEY, SECRET_KEY));
StsClient client = StsClient
.builder()
.credentialsProvider(awsCredentialsProvider)
.region(Region.US_EAST_1)
.build();
StsAssumeRoleCredentialsProvider
stsAssumeRoleCredentialsProvider =
StsAssumeRoleCredentialsProvider.builder().stsClient(client).refreshRequest((req) -> {
req.roleArn("arn:aws:iam::{account_id}:role/svc.saas_usage_dev").roleSessionName("test-session")
.durationSeconds(900).build();
}).prefetchTime(Duration.ofSeconds(300L)).build();
AwsSessionCredentials myCreds = (AwsSessionCredentials) stsAssumeRoleCredentialsProvider.resolveCredentials();
return S3Client
.builder()
.region(Region.US_EAST_1)
.credentialsProvider(StaticCredentialsProvider.create(myCreds))
.build();
}
Related
I am connecting to SQS through my Java code, and the connection is required to be alive for more than 60 minutes, as I keep polling a queue for a long time. However the credentials I use seem to expire. How can I keep the credentials alive?
Below is how I create the credentials provider:
#Provides
public AwsCredentialsProvider getSTSCredentialProvider() {
final String credentialsPath = System.getenv("CREDENTIAL_PATH");
final AwsCredentialsProvider credentialsProvider = ProfileCredentialsProvider.builder().profileFile(
ProfileFile.builder().content(Paths.get(credentialsPath)).type(ProfileFile.Type.CREDENTIALS).build()
).profileName("default").build();
return credentialsProvider;
}
Below is how I create the SQS client:
#Provides
#Singleton
public SqsClient provideSqsClient(AwsCredentialsProvider awsCredentialsProvider) {
return SqsClient.builder().credentialsProvider(awsCredentialsProvider)
.region(REGION).build();
}
How can I allow the SQS client to refresh the credentials? Appreciate any help/guidance.
I've spring boot app with QueueMessagingTemplate as client to access Amazon SQS using temporary security credentials(STS). Getting temp token using STS-AssumeRole . Can you help me how to refresh/auto-refresh session token when it expires?
Error:
com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is expired
Here is the code:
#Configuration
#Slf4j
public class QueueConfig {
#Bean
public QueueMessagingTemplate queueMessagingTemplate(#Autowired BasicSessionCredentials sessionCredentials) {
log.info("queueMessagingTemplate refresh");
return new QueueMessagingTemplate(amazonSQSAsync(sessionCredentials));
}
#Bean
#Primary
public AmazonSQSAsync amazonSQSAsync(BasicSessionCredentials sessionCredentials) {
return AmazonSQSAsyncClientBuilder
.standard()
.withRegion(Regions.US_WEST_1)
.withCredentials(new AWSStaticCredentialsProvider(sessionCredentials))
.build();
}
}
Here is the code for AWS STS cred
#Configuration
#Slf4j
public class AwsRoleCredentials {
#Bean(name = "sessionCredentials")
public BasicSessionCredentials sessionCredentials(){
try {
String roleArn = "XXXX";
String roleSessionName = "XXX";
Region region = Region.US_WEST_1;
StsClient stsClient = StsClient.builder()
.region(region)
.build();
AssumeRoleRequest roleRequest = AssumeRoleRequest.builder()
.roleArn(roleArn)
.roleSessionName(roleSessionName)
.build();
AssumeRoleResponse roleResponse = stsClient.assumeRole(roleRequest);
Credentials myCreds = roleResponse.credentials();
BasicSessionCredentials sessionCred = new BasicSessionCredentials(
myCreds.accessKeyId(),
myCreds.secretAccessKey(),
myCreds.sessionToken());
return sessionCred;
} catch (StsException e) {
log.error("ERROR while get token:"+ e.getMessage());
}
return null;
}
}
I was just about to implement it myself and then i found that in version 2 of the sdk its already there, you can use StsAssumeRoleCredentialsProvider which takes care of refreshing the token when it is about to expire. I don't know if there is something equivalent in the old SDK.
But you can implement it pretty easily for the older SDK as well, just store the expiry and make another assumeRole request when it's about to expire
Edit- I was confused because you use the v1 sdk for SQS but you do use the V2 SDK for STS, so you can simply use StsAssumeRoleCredentialsProvider instead. Also, I suggest using either V1 or V2, but not both
I have WebClient in my Spring Boot application that connects to the external service via OAuth2, and the configuration of it looks like following:
#Configuration
#RequiredArgsConstructor
public class OAuth2ClientConfiguration {
private final OAuth2ClientProperties properties;
#Bean
ReactiveClientRegistrationRepository clientRegistration() {
ClientRegistration registration = ClientRegistration
.withRegistrationId(properties.getClientRegistrationId())
.tokenUri(properties.getTokenUri())
.clientId(properties.getClientId())
.clientSecret(properties.getClientSecret())
.authorizationGrantType(new AuthorizationGrantType(properties.getAuthorizationGrantType()))
.build();
return new InMemoryReactiveClientRegistrationRepository(registration);
}
#Bean
WebClient webClient(ReactiveClientRegistrationRepository clientRegistration) {
var clientService = new InMemoryReactiveOAuth2AuthorizedClientService(clientRegistration);
var authorizedClientManager = new AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(clientRegistration, clientService);
var oauth = new ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager);
oauth.setDefaultClientRegistrationId(properties.getClientRegistrationId());
return WebClient.builder()
.filter(oauth)
.build();
}
}
and here is an access token:
{
"access_token": "some_generated_access_token",
"token_type": "bearer",
"expires_in": 82822,
"scope": "api",
"jti": "6e1a8d7c-3909-4acf-9168-cf912fcd0c8a"
}
It is working and everything is Ok, but... it is not storing the access token after it gets it, it is getting new access token each time it is called. I figured it out when launching my integration tests and verifying Authorization Server calls. However in configuration shown above it should store in memory.
I found out in internet this kind of problem can occur with SpringBoot version up to 2.2.3 and "org.springframework.security:spring-security-oauth2-client:5.2.1.RELEASE"
But I am using newest version of the Spring Boot 2.4.9, and it uses org.springframework.security:spring-security-oauth2-client:5.4.7
How can this issue be solved?
Since I cannot keep client secret in application.yml , so it's kept in vault and from there it gets resolved. However, I can see that ClientRegistration is a final class , hence it's client secret can't be set later once the bean is already initialized.
In such case how can I set secret & use new object of ClientRegistration in all the referred beans.
Something like below I am trying to achieve but don't how to set enrichedClientRegistration in webclient or other referred places.
#Slf4j
#Configuration
public class WebClientConfig {
#Bean
WebClient authWebClient(ClientRegistrationRepository clientRegistrations,
OAuth2AuthorizedClientRepository authorizedClients,
PasswordResolver passwordResolver) {
var clientRegistration = clientRegistrations.findByRegistrationId("myApp");
log.info("Before client secret is {}",clientRegistration.getClientSecret());
var clientSecret = passwordResolver.resolve(clientRegistration.getClientSecret());
log.info("Resolved client secret is {}", clientSecret);
var enrichedClientRegistration=ClientRegistration.withClientRegistration(clientRegistration)
.clientSecret(clientSecret)
.build();
log.info("After client secret is {}",clientRegistrations.findByRegistrationId("myApp").getClientSecret());
var oauth = new ServletOAuth2AuthorizedClientExchangeFilterFunction(clientRegistrations, authorizedClients);
oauth.setDefaultClientRegistrationId("myApp");
return WebClient.builder()
.apply(oauth.oauth2Configuration())
.build();
}
}
Since ClientRegistration is a final class which in injected into ClientRegistrationRepository, so you need completely override ClientRegistrationRepository as per example given in spring documentation.
https://docs.spring.io/spring-security/site/docs/5.0.x/reference/html/jc.html#jc-oauth2login-completely-override-autoconfiguration
I trying to connect to s3 bucket to upload/download images.
My code to create s3 client as follows:
AmazonS3 s3 = AmazonS3ClientBuilder
.standard()
.withRegion("EU-WEST-2")
.build();
I getting exceptions as follows:
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 8574612863BD8DC2; S3 Extended Request ID: ueyZy/RLMerNtHeYaOTlRVAqD7w1CksWrjfNLuMgxPWXQbNGDF1Y04RUs4Gh9HeHMwLXxjBc+5o=), S3 Extended Request ID: ueyZy/RLMerNtHeYaOTlRVAqD7w1CksWrjfNLuMgxPWXQbNGDF1Y04RUs4Gh9HeHMwLXxjBc+5o=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1630)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1302)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4330)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4277)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1410)
at uk.nhs.digital.cid.pyi.services.paycasso.PaycassoService.registerDocument(PaycassoService.java:80)
at uk.nhs.digital.cid.pyi.harness.PaycassoClientTestHarness.testVeriSure(PaycassoClientTestHarness.java:61)
at uk.nhs.digital.cid.pyi.harness.PaycassoClientTestHarness.main(PaycassoClientTestHarness.java:36)
Try this, you need to change env.getProperty("amazon.accessKey") as per your access key and secret.
public AmazonS3 getAmazonS3Client() {
ClientConfiguration clientConfig = new ClientConfiguration();
clientConfig.setProtocol(Protocol.HTTP);
AmazonS3 s3client = new AmazonS3Client(getAmazonCredentials(), clientConfig);
s3client.setS3ClientOptions(S3ClientOptions
.builder()
.setPathStyleAccess(true)
.disableChunkedEncoding().build());
return s3client;
}
public AWSCredentials getAmazonCredentials() {
AWSCredentials credentials = new BasicAWSCredentials(
env.getProperty("amazon.accessKey"),
env.getProperty("amazon.secretKey")
);
return credentials;
}
To check bucket exists and upload file check this.
AmazonS3 s3client = amazonS3ClientService.getAmazonS3Client();
if (!s3client.doesBucketExistV2(env.getProperty("amazon.bucket"))) {
System.out.println("Bucket is not Exist.");
return RepeatStatus.FINISHED;
}
// Upload Dir
TransferManager transferManager = new TransferManager(amazonS3ClientService.getAmazonCredentials());
MultipleFileUpload upload =
transferManager.uploadDirectory(env.getProperty("amazon.bucket"), file.getName(), file,true);
if you want to upload a single file then try this,
s3client .putObject(bucket_name, key_name, new File(file_path));
You have two problems.
You are using a string for the region. You need to use .withRegion(Regions.EU_WEST_2).
From the comments to your question, I understand that you are not using credentials. Even if your bucket is public, you must use AWS credentials to use AWS APIs. Anonymous credentials are not supported.
If you want to use anonymous credentials (which means no credentials) use the normal HTTP URL: https://s3.amazonaws.com/bucket/object with a library such as HttpUrlConnection.
In some cases you are allowed to use a string for .withRegion(), but only if the region is not in the Regions enum.
For your IAM role provide Programmable access, Also in bucket policy give write permission
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"mybucketpolicy",
"Effect":"Allow",
"Principal": {"Service": "s3.amazonaws.com"},
"Action":["s3:PutObject"],
"Resource":["arn:aws:s3:::destination-bucket/*"],
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:::source-bucket"
},
"StringEquals": {
"aws:SourceAccount": "accid",
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
I have tried with this as well
`AWSCredentials credentials;
try {
credentials = new ProfileCredentialsProvider().getCredentials();
} catch (Exception e) {
throw new AmazonClientException("Cannot load the credentials from the credential profiles file. "
+ "Please make sure that your correct credentials file is at the correct "
+ "location (/Users/userid/.aws/credentials), and is in valid format.", e);
}
AWSSecurityTokenServiceClient stsClient = new AWSSecurityTokenServiceClient(credentials);
AssumeRoleRequest assumeRequest = new AssumeRoleRequest().withRoleArn(ROLE_ARN).withDurationSeconds(3600)
.withRoleSessionName("demo");
AssumeRoleResult assumeResult = stsClient.assumeRole(assumeRequest);
BasicSessionCredentials temporaryCredentials = new BasicSessionCredentials(
assumeResult.getCredentials().getAccessKeyId(), assumeResult.getCredentials().getSecretAccessKey(),
assumeResult.getCredentials().getSessionToken());
s3Client = new AmazonS3Client(temporaryCredentials).withRegion(Regions.EU_WEST_2)`