We Keep Coding

Method Not allowed in S3

I am trying to create a bucket with ceph and s3 library and get the below exception 405. Any pointers to resolve this issue?
com.amazonaws.services.s3.model.AmazonS3Exception: null (Service:
Amazon S3; Status Code: 405; Error Code: MethodNotAllowed; Request ID:
tx00000000000000000000a-005d37c963-1009-
Code:
BasicAWSCredentials credentials = new BasicAWSCredentials("", "");
ClientConfiguration clientConfig = new ClientConfiguration();
clientConfig.setProtocol(Protocol.HTTP);
AmazonS3 conn = new AmazonS3Client(credentials, clientConfig);
conn.setEndpoint("localhost:8080");
Bucket bucket = conn.createBucket("my-new-bucket");

Try to add below code
conn.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));

I got stuck for ages on MethodNotAllowed trying to create a ceph bucket.
Firstly I'd note that you should be able to use the s3cmd command line tool to create a bucket with the same user (or you should be able to see the same MethodNotAllowed response) to verify whether it's a problem with your java code.
For me the answer turn out to be this: You're not allowed to name your bucket "documents"! (not sure what other reserved words there are)

Not able to put object in S3 bucket when it does not have public access

I'm tying to write object to s3 bucket in my aws account but it fails with below error.
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 34399CEF4B28B50D; S3 Extended Request ID:
I tried making the bucket public with full access and then I'm able to write to it.
Code I have written to write object to S3 bucket :
......
private final AmazonS3 amazonS3Client;
........
final PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, s3Key, stream,
metadata);
amazonS3Client.putObject(putObjectRequest);
final URL url = amazonS3Client.getUrl(bucketName, s3Key);
I am building my S3 client as :
#Configuration
public class AWSConfig {
#Bean
public AmazonS3 amazonS3Client() {
String awsRegion = System.getenv("AWS_REGION");
if (StringUtils.isBlank(awsRegion)) {
awsRegion = Regions.getCurrentRegion().getName();
}
return AmazonS3ClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.withRegion(awsRegion)
.build();
}
}
Please suggest me if I am missing anything and how can I fix the error mentioned above.

You are missing Access Keys (Access Key ID and Secret Access Key)
Right now it only works if you set the bucket to public, because you are not providing any Access Keys.
Access Keys are can be best compared with API keys, which provide a secure way of accessing private data in AWS.
You will need:
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(ACCESSKEY, SECRETKEY)));
See the official documentation on how to generate/obtain them.

Make sure below option is "off"
s3--> mybucket --> Permissions --> Block all public access -->> off

I have found the solution for this. Issue was that the java service from where I was trying to call put object request does not have access to s3 bucket. For resolving this, I have added permission for the instance where my service was running to access the s3 bucket which resolved the problem.

Amazon S3 Copy between two buckets with different Authentication

I have two buckets, each with a Private ACL.
I have an authenticated link to the source:
String source = "https://bucket-name.s3.region.amazonaws.com/key?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=...&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=...Signature=..."
and have been trying to use the Java SDK CopyObjectRequest to copy it into another bucket using:
AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey)
AWSCredentialsProvider provider = new AWSStaticCredentialsProvider(credentials)
AmazonS3 s3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(provider)
AmazonS3URI sourceURI = new AmazonS3URI(URI(source))
CopyObjectRequest request = new CopyObjectRequest(sourceURI.getBucket, sourceURI.getKey, destinationBucket, destinationKey);
s3Client.copyObject(request);
However I get AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied because My AWS credentials I've set the SDK up with do not have access to the source file.
Is there a way I can provide an authenticated source URL instead of just the bucket and key?

This isn't supported. The PUT+Copy service API, which is used by s3Client.copyObject(), uses an internal S3 mechanism to copy of the object, and the source object is passed as /bucket/key -- not as a full URL. There is no API functionality that can be used for fetching from a URL, S3 or otherwise.
With PUT+Copy, the user making the request to S3...
must have READ access to the source object and WRITE access to the destination bucket
https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html
The only alternative is download followed by upload.
Doing this from EC2... or a Lambda function running in the source region would be the most cost-effective, but if the object is larger than the Lambda temp space, you'll have to write hooks and handlers to read from the stream and juggle the chunks into a multipart upload... not impossible, but requires some mental gyrations in order to understand what you're actually trying to persuade your code to do.

Is it possible to upload data to read only bucket on amazon s3 using java sdk

The bucket is in this region: null. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: PermanentRedirect; Request ID: 437E211C5EDD014D)
This Exception occurs when i tried to upload objects to amazon s3 bucket using java sdk using IAM user account aacess key and secret key.
I have only read access to the bucket and one object inside the bucket named xyz.
All permissions i do have is full access inside the xyz object.
Please help!

Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: xxxxxxxxxxxxx)

I am trying to access my s3 bucket using a application deployed on my tomcat running on ec2.
I could see lots of posts related to this, but look like most of them complaint about not having proper access. I have proper access to all buckets, I am able to upload the file from another application using different application like jenkins s3 plugin without any issues. I am clueless why this should happen only for a java web application deployed on tomcat. I have confirmed below things.
The ec2 instance was created with an IAM role.
The IAM role has write access to bucket. The puppet scripts is able to write to bucket.
Tried with other application to check the IAM role and it is working fine with out any issues.
As per my understanding if I do not specify any credentials while creating the S3 bucket client(AmazonS3Client ),it will take the IAM role authentication as default.
This is a sample function which I wrote to test the permission.
public boolean checkWritePermission(String bucketName) {
AmazonS3Client amazonS3Client=new AmazonS3Client();
LOG.info("Checking bucket write permission.....");
boolean hasWritePermissions = false;
final ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(0);
// Create empty content
final InputStream emptyContent = new ByteArrayInputStream(new byte[0]);
// Create a PutObjectRequest with test object
final PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName,
"TestDummy.txt", emptyContent, metadata);
try {
if (amazonS3Client.putObject(putObjectRequest) != null) {
LOG.info("Permissions validated!");
// User has write permissions, TestPassed.
hasWritePermissions = true;
}
}
catch (AmazonClientException s3Ex) {
LOG.warn("Write permissions not available!", s3Ex.getMessage());
LOG.error("Write permissions not available!", s3Ex);
}
return hasWritePermissions;
}
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: xxxxxxxxxxxxxx).

Not sure if you have solved this issue yet; however, if you are using custom KMS keys on your bucket and the file you are trying to reach is encrypted with the custom key then this error will also be thrown.
This issue is sometimes hidden by the fact you can still list objects inside your S3 bucket. Make sure your IAM policy includes kms permissions to decrypt.