I am trying to access AWS elastic search (not the normal Elastic which could be hosted on some machine, but the AWS version of Elastic) using java. One thing I have identified is that we have to use REST TEMPLATE instead of the TransportClient method as AWS ES is hosted on port 80 and to get the data, we have to send a POST request with payload.
I am able to get simple data in this process but the request doesn't take wild card characters. It gives me below error:
{"type":"parse_exception","reason":"Failed to derive xcontent"}
Questions:
1. Is my understanding correct about Java hitting AWS ES on port 80? Does this mean we have to use POST instead of GET to send requests having attribute level filtering?
Is there a way to pass the attribute to the elastic search in the url itself?
I have tried below example which doesn't work
e.g. : http://helloworld.amazon.com/customer/_search?q=emailID:*abc#gmail.*
How do we pass wildcard character via java client to AWS ES to fetch the data?
Related
As per my API testing, I had to first login to two ssh hosts through terminal(mac machine) then we use curl command for requests. I am asked to automate this using RestAssured.
To login hosts, I have used jcraft.jsch.JSch java library which i am able to access to hosts successfully but after this when I use below rest assured code, I am getting error "Connection refused"
Response response = RestAssured.given().header(<header>).body(<payload>).relaxedHTTPSValidation().when().post(<url>);
Is there any way to connect ssh hosts through Rest Assured framework or any other solution ? Please help.
When using AWS CLI version aws-cli/1.15.50, Python/3.7.0, Darwin/16.7.0, botocore/1.10.49, the command
aws sqs list-queues
returns a list of the format
https://us-west-2.queue.amazonaws.com/<a number>/<queue-name>
When I call the equivalent from the Java SDK (SDK version 1.11.344
called from Scala version 2.12.6), I get a list of the format
https://sqs.us-west-2.amazonaws.com/<a number>/<queue-name>
PLEASE NOTE: the number is the same in both URLs as are the corresponding queue names.
The differences are:
The CLI begins with the region (us-west-2) while the SDK begins with sqs.
After the region, the CLI's domain name is .queue.amazonaws.com but the JDK has just .amazonaws.com. (The SDK does not have the token queue..)
I get the same results when using get-queue-url and getQueueUrl (either overload in the SDK).
Messages sent using aws sqs send-message with the URL returned by the CLI are not received by the Scala program using the URL returned by the AWS Java SDK.
What am I doing wrong?
The AWS SQS Queue url is provided in the following format (see this link):
https://{REGION_ENDPOINT}/queue.|api-domain|/{YOUR_ACCOUNT_NUMBER}/{YOUR_QUEUE_NAME}
If you had given us the code you used to retrieve the urls in java, we could have seen what the real root cause is. What you have got as the response for your JAVA SDK call is an AWS Endpoint. (see this link).
To reduce data latency in your applications, most Amazon Web Services
offer a regional endpoint to make your requests. An endpoint is a URL
that is the entry point for a web service. For example,
https://dynamodb.us-west-2.amazonaws.com is an entry point for the
Amazon DynamoDB service.
You can try out to retrieve the SQS URL using the JAVA SDK using the following code. This is for a single Queue URL given the Queue name. (See this doc)
AmazonSQS sqs = AmazonSQSClientBuilder.defaultClient();
String queue_url = sqs.getQueueUrl(QUEUE_NAME).getQueueUrl();
You can also use the listQueues method to retrieve the Queue URL list, as given in this doc.
If you are still getting the SQS Endpoint...
If you keep on getting the SQS Queue URL Endpoint, instead of the SQS actual Queue URL, you can use the endpoint to access and manipulate the queue as you require. Have a look at this example written in Java, which will help you understand how to use the endpoint and create a workaround with it.
Getting Nova Server From Metadata
Hi,
I'm using jclouds SDK with Java to retrieve OpenStack Nova Servers, i can retrieve the server through its id, but i didn't find any other way that i can get a Nova Server.
I saw in the OpenStack documentation that i can get a server using the API /servers/{server_id} or i can list all the servers, but assume that i have a case that i only need to get the servers that designated with certain data, such as i need to list all servers that are designated as delete-able which i can set in the metadata when i create the server
in this case, is there any way to use some sort of filtration to the metadata for the servers?
Thanks
I don't think you can filter directly by the server metadata, but you should be able to filter using any of the query parameters that are available when listing servers.
You can just call the ServerApi.list(options) by passing the query parameters you want. You can build the options object by using PaginationOptions.queryParameters method.
As part of current caching system, we are trying to fetch twitter share count for our site url using "https://cdn.api.twitter.com/1/urls/count.json?url=" . We call this API from Spring Java Rest Controller. This Api works fine for few attempts for the same url. But many time this is throwing timeout exception even after retrial.
We tried to run curl command on above API for same url from APP server and we see the same behavior. Do we know, why it is happening. Does twitter blocks server Ip address if many requests comes from same server. Is it the right API to get count for a url or do we need to use different API.
Any help in this regard will be great.
Thanks
Amit
I just found one solution to do that.
API URL : https://api.twitter.com/1.1/search/tweets.json?q=www.myABCwebsite.com
Look for the following field under "statuses" object.
"statuses_count": 6400,
I 'm getting 620 error response codes back from the google maps geocoding api if i send the request directly from my app engine servlet, so i have no choice but to use a proxy to receive a successful response. I set up a proxy server, and ive tested it from several computers. Now, all I want to do is make a url request from my GAE servlet through my proxy.
I've tried every possible solution out there and none of them work....
-java.net.Proxy isnt supported in the app engine runtime...
-setting properties as follows:
Properties props = System.getProperties();
props.put("http.proxyHost", "proxyhostname");
props.put("http.proxyPort", "proxyhostport");
didnt do anything.
What is the easiest way to send an http GET via a proxy in app engine?
It seems like this is not possible: Google's App Engine APIs don't support it. Using a third-party library (like Apache's HTTPCore/HTTPClient) or writing it yourself is not possible because essential network classes like java.net.Socket are not whitelisted.
Not sure why you can't access the Google Map API, but if that really does not work, your only choice is to write some application on your proxy server that responds to normal HTTP requests and then forwards them to Google Maps.
Update: Googled a bit, seems like a well-known problem: the Map API has a limit of 2500 requests per day and IP, and this is limit is reached quickly on GAE where you share your IP with many other applications. The only thing you can do is move the requests to the client, use some proxy with own IP, or use a different service.