When using AWS CLI version aws-cli/1.15.50, Python/3.7.0, Darwin/16.7.0, botocore/1.10.49, the command
aws sqs list-queues
returns a list of the format
https://us-west-2.queue.amazonaws.com/<a number>/<queue-name>
When I call the equivalent from the Java SDK (SDK version 1.11.344
called from Scala version 2.12.6), I get a list of the format
https://sqs.us-west-2.amazonaws.com/<a number>/<queue-name>
PLEASE NOTE: the number is the same in both URLs as are the corresponding queue names.
The differences are:
The CLI begins with the region (us-west-2) while the SDK begins with sqs.
After the region, the CLI's domain name is .queue.amazonaws.com but the JDK has just .amazonaws.com. (The SDK does not have the token queue..)
I get the same results when using get-queue-url and getQueueUrl (either overload in the SDK).
Messages sent using aws sqs send-message with the URL returned by the CLI are not received by the Scala program using the URL returned by the AWS Java SDK.
What am I doing wrong?
The AWS SQS Queue url is provided in the following format (see this link):
https://{REGION_ENDPOINT}/queue.|api-domain|/{YOUR_ACCOUNT_NUMBER}/{YOUR_QUEUE_NAME}
If you had given us the code you used to retrieve the urls in java, we could have seen what the real root cause is. What you have got as the response for your JAVA SDK call is an AWS Endpoint. (see this link).
To reduce data latency in your applications, most Amazon Web Services
offer a regional endpoint to make your requests. An endpoint is a URL
that is the entry point for a web service. For example,
https://dynamodb.us-west-2.amazonaws.com is an entry point for the
Amazon DynamoDB service.
You can try out to retrieve the SQS URL using the JAVA SDK using the following code. This is for a single Queue URL given the Queue name. (See this doc)
AmazonSQS sqs = AmazonSQSClientBuilder.defaultClient();
String queue_url = sqs.getQueueUrl(QUEUE_NAME).getQueueUrl();
You can also use the listQueues method to retrieve the Queue URL list, as given in this doc.
If you are still getting the SQS Endpoint...
If you keep on getting the SQS Queue URL Endpoint, instead of the SQS actual Queue URL, you can use the endpoint to access and manipulate the queue as you require. Have a look at this example written in Java, which will help you understand how to use the endpoint and create a workaround with it.
Related
I use Java AWS S3 SDK to presign my requests. I have the following code:
var request = new GeneratePresignedUrlRequest(bucketName, filename)
.withMethod(method)
.withExpiration(expiration());
// do something with request
return s3Client.generatePresignedUrl(request);
What I need to write in place of comment to add custom conditions like content-length-range?
For browser-based POST uploads to S3, the AWS Java SDK doesn't provide a way to generate pre-signed URLs with conditions. There's an open feature request to add this to the v2 SDK. Note that the PHP, Node.js, and Python SDKs do all provide this feature.
For regular HTTP PUT pre-signed URLs, you can't apply content length restrictions to pre-signed URLs. You can place conditions using a custom policy but that only supports:
DateLessThan
DateGreaterThan
IpAddress
If you need to deal with objects outside of a given size range then you could potentially do that in AWS Lambda, after the object has been uploaded.
I am trying to access AWS elastic search (not the normal Elastic which could be hosted on some machine, but the AWS version of Elastic) using java. One thing I have identified is that we have to use REST TEMPLATE instead of the TransportClient method as AWS ES is hosted on port 80 and to get the data, we have to send a POST request with payload.
I am able to get simple data in this process but the request doesn't take wild card characters. It gives me below error:
{"type":"parse_exception","reason":"Failed to derive xcontent"}
Questions:
1. Is my understanding correct about Java hitting AWS ES on port 80? Does this mean we have to use POST instead of GET to send requests having attribute level filtering?
Is there a way to pass the attribute to the elastic search in the url itself?
I have tried below example which doesn't work
e.g. : http://helloworld.amazon.com/customer/_search?q=emailID:*abc#gmail.*
How do we pass wildcard character via java client to AWS ES to fetch the data?
I am fairly new to the IoT. I have created a thing on Aws and I have created a client .Everything works fine when I test the set up. I am unable to parse the message sent from the client to the Aws IoT.
Any Suggestion would be helpful
You can create a Rule and use SQL statements to parse or filter messages sent by your things.
.
For example: You can insert those messages into dynamodb or s3, or call a lambda function to process further.
Check this link for more information.
I'm using AWS signV4 to sign requests for calling AWS API Gateway.
Using Requests with headers are not suited for my use case hence the initiative to work with signed urls and query strings.
I had used AWS Sample Python sample that works great for IAM user creation.
the same Access Key and Secret are used in both cases (IAM Creation , API Gateway call) so its definitely not an issue of wrong access key / secret.
The main issue i see is that the signature i get is different than the one the API Gateway thinks i should send.
I've added all the documented query string but still get the below error.
What am i missing here?
Here is the request URL:
https://*******.execute-api.us-east-1.amazonaws.com/******?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=**********%2F20170424%2Fus-east-1%2Fexecute-api%2Faws4_request&X-Amz-Date=20170424T124521Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=**********821742cee7661ef3a0ab1e5c
While using it with the execute-api service i had received the following error :
{
"message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'GET\n/*******\nX-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=*********%2F20170424%2Fus-east-1%2Fexecute-api%2Faws4_request&X-Amz-Date=20170424T124521Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host\nhost:*******.execute-api.us-east-1.amazonaws.com\n\nhost\n*********'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20170424T124521Z\n20170424/us-east-1/execute-api/aws4_request\n**********'\n"
}
I've Checked also the java implementation using the samples from : AWS Signer Java
but the results were the same.
SigV4 signing and this error message isn't anything specific to API Gateway itself. The signing algorithm and the verification at the AWS end is common for all AWS services.
Can you compare the canonical string that is in the error message and the one that you are generating while sending the request? That should help you identify the issue.
Also, go through the step-by-step explanation to see if you are doing everything right. One of the common issues is case-sensitivity of header names and the sorting order of parameters.
I am trying to perform some S3 operations (like create bucket, get bucket, put object, list object etc) with V4 authentication using Query parameter as Authentication method.
I want to perform the operation using AWS JAVA SDKs
Question-
1. Does AWS JAVA SDK support Query parameter as Authentication method?If yes which package needs to be imported.
2. Any example of how to do that using AWS JAVA SDK will be greatly appreciated
You can generate pre-signed URL requests to access s3 resources using query parameter authentication. Here's how to do it with the Java SDK:
URL url = s3Client.generatePresignedUrl(bucketName, key, expirationDate);
I've only used it successfully with GET requests.