We Keep Coding

Refresh temporary tokens of S3Client bean

The use case is to pull the file from the S3 bucket in real time. Here we are using Spring cloud streaming. So I have created an S3Client bean that fetches any new data using a file synchronizer. But the token gets expired after one hour. So I am getting an error after every hour. Is there any way to refresh the token after some specified time?
**This is my code : **
#Bean
public S3Client s3Client() {
AwsCredentialsProvider awsCredentialsProvider =
StaticCredentialsProvider.create(AwsBasicCredentials.create(ACCESS_KEY, SECRET_KEY));
StsClient client = StsClient
.builder()
.credentialsProvider(awsCredentialsProvider)
.region(Region.US_EAST_1)
.build();
StsAssumeRoleCredentialsProvider
stsAssumeRoleCredentialsProvider =
StsAssumeRoleCredentialsProvider.builder().stsClient(client).refreshRequest((req) -> {
req.roleArn("arn:aws:iam::{account_id}:role/svc.saas_usage_dev").roleSessionName("test-session")
.durationSeconds(900).build();
}).prefetchTime(Duration.ofSeconds(300L)).build();
AwsSessionCredentials myCreds = (AwsSessionCredentials) stsAssumeRoleCredentialsProvider.resolveCredentials();
return S3Client
.builder()
.region(Region.US_EAST_1)
.credentialsProvider(StaticCredentialsProvider.create(myCreds))
.build();
}

Kinesis data stream will not load to s3 bucket

I'm trying to load an s3 bucket with my Kinesis data streams (at the closure of each stream) but I am receiving an error preventing me from doing this. Is there a way to do this with the java sdk?
Error -
Error: The website redirect location must have a prefix of 'http://' or 'https://' or '/'. (Service: Amazon S3; Status Code: 400; Error Code: InvalidRedirectLocation; Request ID: ***; S3 Extended Request ID: ***; Proxy: null)
Method to load s3 bucket with stream -
public void uploadStreamToS3Bucket() {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(String.valueOf(awsRegion))
.build();
try {
String fileName = connectionRequestRepository.findStream() +".json";
String bucketName = "downloadable-cases";
String locationData = "arn:aws-us-***-1:kinesis:***:stream/" + connectionRequestRepository.findStream();
s3Client.putObject(new PutObjectRequest(bucketName, fileName, locationData));
} catch (AmazonServiceException ex) {
System.out.println("Error: " + ex.getMessage());
}
}

how to auto-refresh AWS STS Temporary security credentials when it expires while accessing Amazon SQS?

I've spring boot app with QueueMessagingTemplate as client to access Amazon SQS using temporary security credentials(STS). Getting temp token using STS-AssumeRole . Can you help me how to refresh/auto-refresh session token when it expires?
Error:
com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is expired
Here is the code:
#Configuration
#Slf4j
public class QueueConfig {
#Bean
public QueueMessagingTemplate queueMessagingTemplate(#Autowired BasicSessionCredentials sessionCredentials) {
log.info("queueMessagingTemplate refresh");
return new QueueMessagingTemplate(amazonSQSAsync(sessionCredentials));
}
#Bean
#Primary
public AmazonSQSAsync amazonSQSAsync(BasicSessionCredentials sessionCredentials) {
return AmazonSQSAsyncClientBuilder
.standard()
.withRegion(Regions.US_WEST_1)
.withCredentials(new AWSStaticCredentialsProvider(sessionCredentials))
.build();
}
}
Here is the code for AWS STS cred
#Configuration
#Slf4j
public class AwsRoleCredentials {
#Bean(name = "sessionCredentials")
public BasicSessionCredentials sessionCredentials(){
try {
String roleArn = "XXXX";
String roleSessionName = "XXX";
Region region = Region.US_WEST_1;
StsClient stsClient = StsClient.builder()
.region(region)
.build();
AssumeRoleRequest roleRequest = AssumeRoleRequest.builder()
.roleArn(roleArn)
.roleSessionName(roleSessionName)
.build();
AssumeRoleResponse roleResponse = stsClient.assumeRole(roleRequest);
Credentials myCreds = roleResponse.credentials();
BasicSessionCredentials sessionCred = new BasicSessionCredentials(
myCreds.accessKeyId(),
myCreds.secretAccessKey(),
myCreds.sessionToken());
return sessionCred;
} catch (StsException e) {
log.error("ERROR while get token:"+ e.getMessage());
}
return null;
}
}

I was just about to implement it myself and then i found that in version 2 of the sdk its already there, you can use StsAssumeRoleCredentialsProvider which takes care of refreshing the token when it is about to expire. I don't know if there is something equivalent in the old SDK.
But you can implement it pretty easily for the older SDK as well, just store the expiry and make another assumeRole request when it's about to expire
Edit- I was confused because you use the v1 sdk for SQS but you do use the V2 SDK for STS, so you can simply use StsAssumeRoleCredentialsProvider instead. Also, I suggest using either V1 or V2, but not both

Gmail API, get new access token after revoke

I'm developing application that provides access to gmail mailbox. I created new project and logged in with few email accounts. When I revoked access token through this endpoint: https://oauth2.googleapis.com/revoke, the number of accounts attached to my project didn't change.
When i try to select account again, google redirects me to uri I specified, but code within url is corrupted and I can't get new access token using code from this url.
Response i get using code from url:
{
"error": "invalid_grant",
"error_description": "Malformed auth code."
}
How can i completely log out from my app?
Credential authorize=getCredentials(gmailMailBox, emailProperties.getGoogleClientSecrets());
authorize.refreshToken();
Unirest.post(emailProperties.getRevokeTokenUrl(authorize.getAccessToken()));
getCredentials method:
Credential authorize = new GoogleCredential.Builder()
.setTransport(GoogleNetHttpTransport.newTrustedTransport())
.setJsonFactory(JacksonFactory.getDefaultInstance())
.setClientSecrets(clientSecrets)
.build()
.setAccessToken(gmailMailBox.getAccessToken())
.setRefreshToken(gmailMailBox.getRefreshToken());
private Credential getCredentials(GmailMailBox gmailMailBox, GoogleClientSecrets clientSecrets)
throws GeneralSecurityException, IOException {
return new GoogleCredential.Builder().setTransport(GoogleNetHttpTransport.newTrustedTransport())
.setJsonFactory(JacksonFactory.getDefaultInstance())
.setClientSecrets(clientSecrets)
.build()
.setAccessToken(gmailMailBox.getAccessToken())
.setRefreshToken(gmailMailBox.getRefreshToken());
}
public class GmailMailBox {
private String accessToken;
private String refreshToken;
private LocalDateTime expiresIn;
}
public GoogleClientSecrets getGoogleClientSecrets() {
GoogleClientSecrets clientSecrets = new GoogleClientSecrets();
GoogleClientSecrets.Details clientSecretsDetails = new GoogleClientSecrets.Details();
clientSecretsDetails.set("client_id", client_id);
clientSecretsDetails.set("project_id", project_id);
clientSecretsDetails.set("auth_uri", auth_uri);
clientSecretsDetails.set("token_uri", token_uri);
clientSecretsDetails.set("auth_provider_x509_cert_url", auth_provider_x509_cert_url);
clientSecretsDetails.set("client_secret", client_secret);
clientSecrets.setWeb(clientSecretsDetails);
return clientSecrets;
}

why does putObject throw AmazonS3Exception when using PutObjectRequest, but works fine without PutObjectRequest?

I am writing a Java method that takes 3 string parameters: bucketName, objectKey, objectContent. The method then puts the object into the bucket. The following code works with no problems.
AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(REGION).build();
s3.putObject(bucketName, objectKey, content);
Now I want to set a content type for the objects, because I will be using the method to store e.g. "text/plain" or "text/xml" files. So I use the following code.
AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(REGION).build();
byte[] fileContentBytes = content.getBytes(StandardCharsets.UTF_8);
InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);
ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentType(contentType);
metaData.setContentLength(fileContentBytes.length);
PutObjectRequest putObjReq = new PutObjectRequest(bucketName, objectKey, content);
putObjReq.setMetadata(metaData);
s3.putObject(putObjReq);
When I run this code, I get an exception, as listed below. Why?
com.amazonaws.services.s3.model.AmazonS3Exception: The website redirect location must have a prefix of 'http://' or 'https://' or '/'. (Service: Amazon S3; Status Code: 400; Error Code: InvalidRedirectLocation; Request ID: F8032DFF52EBF6F2; S3 Extended Request ID: vZX1/oTjeWU0Fok6twiyB5mEi2d0GDXYWT+akeETrapXo9CUbG+DgcabAaiFVlGXOu072vGghD4=), S3 Extended Request ID: vZX1/oTjeWU0Fok6twiyB5mEi2d0GDXYWT+akeETrapXo9CUbG+DgcabAaiFVlGXOu072vGghD4=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4926)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4872)
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:390)
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5806)
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1794)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1754)
at util.DataUtils.saveContentToS3(DataUtils.java:155)
at builder.SEOGenerator.main(SEOGenerator.java:53)
I should note that I use this S3 bucket to host a static website. I use CloudFront in front of S3 and then Route 53 for my domain. My S3 bucket policy is as follows.
{
"Version": "2012-10-17",
"Id": "http referer policy - my-domain.com",
"Statement": [
{
"Sid": "Allow get requests originating from my domain",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-s3-bucket/*",
"Condition": {
"StringLike": {
"aws:Referer": [
"http://www.my-domain.com/*",
"http://my-domain.com/*",
"https://www.my-domain.com/*",
"https://my-domain.com/*"
]
}
}
}
]
}

There are 3 different ways to call the PutObjectRequest constructor. You're using this one:
PutObjectRequest(String bucketName, String key, String redirectLocation)
So, your 'content' is being treated as a redirect location, hence that error.
I think your intent is to use this one instead:
PutObjectRequest(String bucketName, String key, InputStream input, ObjectMetadata metadata)
Which means you'd have to do something like:
AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(REGION).build();
byte[] fileContentBytes = content.getBytes(StandardCharsets.UTF_8);
InputStream in = new ByteArrayInputStream(fileContentBytes);
ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentType(contentType);
metaData.setContentLength(fileContentBytes.length);
PutObjectRequest putObjReq = new PutObjectRequest(bucketName, objectKey, in, metaData);
s3.putObject(putObjReq);