I'm using the Java API for Amazon AWS. I successfully authenticate, then get all images and my images are not among them (my AMIs are private, but I suppose that I will still see them since I have been authenticated). Here is my source...
final AmazonEC2 client = new AmazonEC2Client(credentails);
for(Image image: client.describeImages().getImages()){
if(image.getOwnerId().equals("1234567890")){
//... do something usefull with the AMI
}
}
And my "OwnerId" is not among the received ones. What is the problem, I won't make my AMIs public, how can I get my AMIs?
ANSWER: I was in a wrong region, and I was getting only AMIs from that region, not mine.
The way to change region is:
client.setEndpoint("ec2.us-west-1.amazonaws.com");
FYI, if you're only interested in your own instances you can dramatically reduce the amount of bandwidth used in a DescribeInstances call using:
DescribeImagesRequest request = new DescribeImagesRequest();
request.withOwners("self");
Collection<Image> images = client.describeImages(request).getImages();
Related
I am creating an android app to record a user's activity using Google Maps SDK and the Google Play Services Location API. I am attempting to retrieve the user's elevation based on a given latitude and longitude. I originally used Location#getAltitude() but then realised that does not give the elevation above sea level.
I proceeded to use the open elevation API using the following query string:
String url = "https://api.open-elevation.com/api/v1/lookup?locations=" + latLng.latitude + "," + latLng.longitude;
However, that API appears to be much too slow in generating a response. I then found the Google Maps Elevation API which we can make a request using a URL also. However, we need to pass an API key and I do not want to pass this API key in the URL string and end up committing it to the remote repository.
In this repo (https://github.com/googlemaps/google-maps-services-java) I found the class:
/src/main/java/com/google/maps/ElevationApi.java which I thought I could use to avoid messing around with http requests.
In my gradle, I included this dependency:
implementation 'com.google.maps:google-maps-services:0.18.0'
At the moment, the code to retrieve the elevation is as follows:
ElevationApi.getByPoint(new GeoApiContext.Builder().apiKey(API_KEY).build(), latLng)
.setCallback(new PendingResult.Callback<ElevationResult>() {
#Override
public void onResult(ElevationResult result) {
consumer.doAction(result.elevation);
}
#Override
public void onFailure(Throwable e) {
e.printStackTrace();
}
});
What do I pass in for API_KEY here since I don't want to commit it to the repository? I have an api key defined in local.properties for maps, however, like so:
MAPS_API_KEY=<API_KEY_HERE>
Basically, my question is, can I define an API key in a properties file that is not committed to GitHub and then reference it in the code?
Thanks for any help.
Update:
I have managed to read the API key from local.properties using gradle but got an exception from the ElevationApi saying API 21+ expected, but was 30...strange. So I went back to the open-elevation API with the following Volley request:
/**
* Calculates elevation gain for the provided recording service
* #param recordingService the recording service to calculate elevation gain for
* #param response the handler to consume the elevation gain with
*/
public static void calculateElevationGain(RecordingService recordingService, ActionHandlerConsumer<Double> response) {
ArrayList<Location> locations = recordingService.getLocations();
JSONArray array = constructLocations(locations);
try {
if (array != null) {
RequestQueue requestQueue = Volley.newRequestQueue(recordingService);
String url = "https://api.open-elevation.com/api/v1/lookup";
JSONObject requestHeader = new JSONObject(); // TODO this seems very slow
requestHeader.put("locations", array);
JsonObjectRequest request = new JsonObjectRequest(Request.Method.POST, url, requestHeader,
response1 -> handleSuccessfulResponse(response1, response), RecordingUtils::handleErrorResponse);
request.setRetryPolicy(new DefaultRetryPolicy(500000, DefaultRetryPolicy.DEFAULT_MAX_RETRIES, DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));
requestQueue.add(request);
}
} catch (JSONException ex) {
ex.printStackTrace();
}
}
I had to set the timeout to a high number not sure how hight it should be because I was getting Volley timeout errors due to the slow response times.
Are there any other ways I can retrieve elevation about sea level?
Yeah, open-elevation.com has intermittent issues with timeouts and latency.
There are some alternatives listed on this GIS stack exchange question Seeking alternative to Google Maps Elevation API. I'm the developer of Open Topo Data which is the most-voted answer over there. You can host your own server with docker, and I also have a free public API which has pretty good latency and uptime.
There's also GPXZ as an alternative to the Google Elevation API with higher-quality data, but it requires an API key so would have the same issue as with Google Maps.
I advise a different direction: stay with the Google and the API key, but employ best practices regarding secrets and source repositories. Since you are dealing with an Android app and not a webapp your key can be somewhat safe inside your app binary (versus a key in a web deployed app is exposed).
Bets practices:
Do not commit the API key. The best to achieve this is to exclude the file which contains the key from the source control repo. That can simply be done with .gitignore. For example this Codelab has a file with the secret, but it has a dummy value and normally this file should be excluded from the source. It is only there because that is an educational code lab.
As a security measure take advantage of GitGuardian to scan your repos in case you'd accidentally push an API key. In such events you'd get a notification. As for me I forked that Geospatial API codelab and saw the key file was in the gitignore and I accidentally pushed a key.
In case you accidentally push a key in a commit it's not enough to reverse the commit and delete the file! Scavenger bots will still find the information in your git history. Rather immediately disable the key and generate another one.
If you are dealing with a webapp you can restrict the API key usage to your webapp's domain. Similarly you can restrict the key to specific Android app signatures (don't forget to add your developer environment's signature) too. This guarantees that even if someone steals the key they probably won't be able to use it.
I used StartApplicationRequest to create a sample request to start the application as given below:
StartApplicationRequest request = StartApplicationRequest.builder()
.applicationId("test-app-name")
.build();
Then, I used the ReactorCloudFoundryClient to start the application as shown below:
cloudFoundryClient.applicationsV3().start(request);
But my test application test-app-name is not getting started. I'm using latest Java CF client version (v4.5.0 RELEASE), but not seeing a way around to start the application.
Quite surprisingly, the outdated version seems to be working with the below code:
cfstatus = cfClient.startApplication("test-app-name"); //start app
cfstatus = cfClient.stopApplication("test-app-name"); //stop app
cfstatus = cfClient.restartApplication("test-app-name"); //stop app
I want to do the same with latest CF client library, but I don't see any useful reference. I referred to test cases written at CloudFoundry official Github repo. I derived to the below code after checking out a lot of docs:
StartApplicationRequest request = StartApplicationRequest.builder()
.applicationId("test-app-name")
.build();
cloudFoundryClient.applicationsV3().start(request);
Note that cloudFoundryClient is ReactorCloudFoundryClient instance as the latest library doesn't support the client class used with outdated code. I would like to do all operations (start/stop/restart) with latest library. The above code isn't working.
A couple things here...
Using the reactor based client, your call to cloudFoundryClient.applicationsV3().start(request) returns a Mono<StartApplicationResponse>. That's not the actual response, it's the possibility of one. You need to do something to get the response. See here for more details.
If you would like similar behavior to the original cf-java-client, you can call .block() on the Mono<StartApplicationResponse> and it will wait and turn into a response.
Ex:
client.applicationsV3()
.start(StartApplicationRequest.builder()
.applicationId("test-app-name")
.build())
.block()
The second thing is that it's .applicationId not applicationName. You need to pass in an application guid, not the name. As it is, you're going to get a 404 saying the application doesn't exist. You can use the client to fetch the guid, or you can use CloudFoundryOperations instead (see #3).
The CloudFoundryOperations interface is a higher-level API. It's easier to use, in general, and supports things like starting an app based on the name instead of the guid.
Ex:
ops.applications()
.start(StartApplicationRequest.builder()
.name("test-app-name").build())
.block();
I do not want to block threads in my application and so I am wondering are calls to the the Google Datastore async? For example the docs show something like this to retrieve an entity:
// Key employeeKey = ...;
LookupRequest request = LookupRequest.newBuilder().addKey(employeeKey).build();
LookupResponse response = datastore.lookup(request);
if (response.getMissingCount() == 1) {
throw new RuntimeException("entity not found");
}
Entity employee = response.getFound(0).getEntity();
This does not look like an async call to me, so it is possible to make aysnc calls to the database in Java? I noticed App engine has some libraries for async calls in its Java API, but I am not using appengine, I will be calling the datastore from my own instances. As well, if there is an async library can I test it on my local server (for example app engine's async library I could not find a way to set it up to use my local server for example I this library can't get my environment variables).
In your shoes, I'd give a try to Spotify's open-source Asynchronous Google Datastore Client -- I have not personally tried it, but it appears to meet all of your requirements, including being able to test on your local server. Please give it a try and let us all know how well it meets your needs, so we can all benefit and learn -- thanks!
Can someone help me with name of api which enables realtime prediction of a model. Please note that i am not requesting for RealtimeEndpointRequest object. i have gone through the entire documentation of AWS Machine Learning SDK but haven't found any thing.
Edit 1 :
This is the code that i have used -
CreateRealTimePrediction createRealTimePrediction ;
CreateRealtimeEndpointRequest createRealtimeEndPointReq;
CreateRealtimeEndpointResult createRealtimeEndPointRes;
PredictRequest predReq;
String mlModelId="ml-Lkqmcs8cM2W";
createRealtimeEndPointReq.setMLModelId(mlModelId);
PredictResult predRes = null;
Map<String,String> record=null;
// assume i have set a record in the Map.
createRealtimeEndPointRes = amlClient.createRealtimeEndpoint(createRealtimeEndPointReq);
String predictEndpoint=createRealtimeEndPointRes.getRealtimeEndpointInfo().getEndpointUrl();
predReq= new PredictRequest();
predReq.setMLModelId(mlModelId);
for (int i=0;i<recordKeys.length;i++){
record.put(recordKeys[i],recordValues[i]);
}
predReq.setRecord(record);
predReq.setPredictEndpoint(predictEndpoint);
predRes=amlClient.predict(predReq);
return predRes;
}
Now what is happening is - if i enable the real time prediction by using aws management console manually and then run this segment of code, then the results are generated as expected but when i the realtime prediction is disabled, then i get this error -
Exception in thread "main" com.amazonaws.services.machinelearning.model.PredictorNotMountedException: Either ML Model with id ml-Lkqmcs8
cM2W is not enabled for real-time predictions or the MLModelId is invalid. (Service: AmazonMachineLearning; Status Code: 400; Error Code
: PredictorNotMountedException; Request ID: 2dc70e58-07d0-11e5-a0c7-bb93f17d1b2e)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302)
at com.amazonaws.services.machinelearning.AmazonMachineLearningClient.invoke(AmazonMachineLearningClient.java:1995)
at com.amazonaws.services.machinelearning.AmazonMachineLearningClient.predict(AmazonMachineLearningClient.java:637)
at com.nrift.aml.prediction.realtime.CreateRealTimePrediction.createRealTimePrediction(CreateRealTimePrediction.java:61)
at RealTimePrediction.main(RealTimePrediction.java:53)
which effectively means that this segment of code is not enabling the real time prediction though i have used
CreateRealtimeEndpoint
api in it.
P.s- the code segment i have posted is a not complete, the complete code is working correctly so you can make assumptions about the correctness of code.
The API you are looking for is CreateRealtimeEndpoint. Creating a real-time endpoint is the mechanism for enabling the model to be used for real-time predictions. When you no longer need to use this model, you can destroy the endpoint with the DeleteRealtimeEndpoint API. The model always stays intact, so you can create/delete endpoints when needed.
I'm trying to start a Amazon EC2 cloud machine with [startInstance][2] method using aws-sdk in Java. My code is as follows.
public String startInstance(String instanceId) throws Exception {
List<String> instanceIds = new ArrayList<String>();
instanceIds.add(instanceId);
StartInstancesRequest startRequest = new StartInstancesRequest(
instanceIds);
startRequest.setRequestCredentials(getCredentials());
StartInstancesResult startResult = ec2.startInstances(startRequest);
List<InstanceStateChange> stateChangeList = startResult
.getStartingInstances();
log.trace("Starting instance '{}':", instanceId);
// Wait for the instance to be started
return waitForTransitionCompletion(stateChangeList, "running",
instanceId);
}
When I run the above code, i'm getting the following AWS error:
Status Code: 400, AWS Request ID: e1bd4795-a609-44d1-9e80-43611e80006b, AWS Erro
r Code: InvalidInstanceID.NotFound, AWS Error Message: The instance ID 'i-2b97ac
2f' does not exist
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpCli
ent.java:538)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.ja
va:283)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:168
)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.jav
a:5208)
at com.amazonaws.services.ec2.AmazonEC2Client.startInstances(AmazonEC2Cl
ient.java:2426)
AWS Error Message: The instance ID 'i-2b97ac2f' does not exist
You'll have to take the AWS response for granted here, i.e. the instance does not exist ;)
But seriously: Presumably you have already verified that you are actually running an instance with this ID in your account? Then this is most likely caused by targeting the wrong API endpoint, insofar an instance ID is only valid within a specific region (if not specified, the region defaults to 'us-east-1', see below).
In this case you need to specify the actual instance region via the setEndpoint() method of the AmazonEC2Client object within the apparently global ec2 variable before calling startInstances().
There are some examples regarding Using Regions with the AWS SDKs and all currently available AWS regional endpoint URLs are listed in Regions and Endpoints, specifically the Amazon Elastic Compute Cloud (EC2) defaults to 'us-east-1':
If you just specify the general endpoint (ec2.amazonaws.com), Amazon
EC2 directs your request to the us-east-1 endpoint.
We run a service (Qubole) that frequently spawns and then tags (and in some cases terminates) AWS instances immediately.
We have found that Amazon will, every once in a while, claim an instanceid as invalid - even though it has just created it. Retrying a few times with some sleep time thrown in usually solves the problem. Even a total retry interval of 15s proved insufficient in rare cases.
This experience comes from the useast region. We do not make api calls to different regions - so that is not an explanation. More likely - this is the infamous eventual consistency at work - where AWS is unable to provide read-after-write consistency for these api calls.
I am using the AWS ruby api and I noticed the same issue when creating an AMI image and its status is pending when I look in the AWS console but after a while the image is available for use.
Here is my script
image = ec2.images.create(:name => image_name, :instance_id => ami_id, :description => desc)
sleep 5 while image.state != :available
I sleep for about 5 sec for image to be in available but I get the error saying that the "AWS Error Message: InvalidInstanceID.NotFound". During my testing this is fine but most of the time this seems to be failing during continuous integration builds.
InvalidInstanceID.NotFound means the specified instance does not exist.
Ensure that you have indicated the region in which the instance is located, if it's not in the default region.
This error may occur because the ID of a recently created instance has not propagated through the system. For more information, see Eventual Consistency.