How do we get the latest log stream name from a given log group using java code.
I have tried many possibilities and still not able to retrieve it.
AWSLogs awsLogs = AWSLogsClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withRegion(Regions.EU_WEST_1)
.build();
DescribeLogStreamsRequest describeLogStreamsRequest = new DescribeLogStreamsRequest()
.withLogGroupName("/aws/lambda/a250348-ecndesc-conversion-lambda-reci-euw1-qe");
describeLogStreamsResult = awsLogs.describeLogStreams(describeLogStreamsRequest);
System.out.println(describeLogStreamsResult.getLogStreams());
for(int i =0; i<describeLogStreamsResult.getLogStreams().size(); i++) {
logStream = describeLogStreamsResult.getLogStreams().get(i).getLastEventTimestamp();
Date date = new Date(logStream);
dateList.add(date);
}
latestDate = Collections.max(dateList);
System.out.println(latestDate);
System.out.println(dateList.indexOf(latestDate));
logStream1 = describeLogStreamsResult.getLogStreams().get(dateList.indexOf(latestDate)).getLogStreamName();
System.out.println(logStream1);
Try to order your results:
DescribeLogStreamsRequest describeLogStreamsRequest = new DescribeLogStreamsRequest()
.withLogGroupName("/aws/lambda/a250348-ecndesc-conversion-lambda-reci-euw1-qe")
.withOrderBy(com.amazonaws.services.logs.model.OrderBy.LastEventTime)
.withDescending(true);
This will return first the log stream with the latest event time (and not necessarily the last created). This is how results are presented in AWS console.
Related
I am trying to search a large number of repositories with "searchRepository" method in https://github.com/eclipse/egit-github/tree/master/org.eclipse.egit.github.core
However, there is a limitation to get more than 1000 results
https://developer.github.com/v3/search/#about-the-search-api
Since it will throw an exception "Only the first 1000 search results are available (422)" (based on the code example below)
A solution is presented in github search limit results
My question is how can I split up the search into segments by the date (as mentioned in the thread), or is there a another way to do this with the Java GitHub API.
int countRepos = 0;
Map<String, String> searchQuery = new HashMap<String, String>();
searchQuery.put("language", "java");
List<SearchRepository> searchRes = null;
GitHubClient client = new GitHubClient();
client.setCredentials("xxx", "xxxxx");
RepositoryService service = new RepositoryService(client);
for(int page = 1 ; page <12 ; page++){
try {
searchRes = service.searchRepositories(searchQuery,page);
} catch (IOException e) {
e.printStackTrace();
}
for(SearchRepository repo : searchRes){
System.out.println("Repository"+countRepos+": "+repo.getOwner()+"/"+repo.getName());
countRepos++;
}
}
System.out.println("Total number of repositories are="+countRepos);
Thanks in advance.
Can any one help me on how to create shared link in BOX using java SDK. I am using below code:-
BoxFile file = new BoxFile(api, ID);
BoxSharedLink.Permissions permissions = new BoxSharedLink.Permissions();
permissions.setCanDownload(true);
permissions.setCanPreview(true);
Date unshareDate = new Date();
BoxSharedLink sharedLink = file.createSharedLink(
BoxSharedLink.Access.OPEN, unshareDate, permissions);
Getting error :-
The API returned the error code: 400
{"type":"error","status":400,"code":"bad_request","context_info":{"errors":[{"reason":"invalid_parameter","name":"unshared_at","message":"Invalid value '1471842735'."}]},"help_url":"http:\/\/developers.box.com\/docs\/#errors","message":"Bad Request","request_id":"208420399157ba89af5e170"}
private static BoxSharedLink createSharedLink(BoxAPIConnection api, String fileId) {
BoxFile file = new BoxFile(api, fileId);
BoxSharedLink.Permissions permissions = new BoxSharedLink.Permissions();
permissions.setCanDownload(true);
permissions.setCanPreview(true);
Date date = new Date();
Calendar unshareAt = Calendar.getInstance();
unshareAt.setTime(date);
unshareAt.add(Calendar.DATE, 14);
BoxSharedLink sharedLink = file.createSharedLink(BoxSharedLink.Access.COMPANY, unshareAt.getTime(), permissions);
logger.info("shared link: " + sharedLink.getURL());
return sharedLink;
}
I just passed "null" in place of unsharedDate..I am able to get a shared link.
BoxSharedLink sharedLink = file.createSharedLink(
BoxSharedLink.Access.OPEN, null, permissions);
I am not sure what null value means. I am guessing there is no unsharedDate set if you pass null. couldn't find any api documentation for this.
I am learning Amazon Cloud Search but I couldn't find any code in either C# or Java (though I am creating in C# but if I can get code in Java then I can try converting in C#).
This is just 1 code I found in C#: https://github.com/Sitefinity-SDK/amazon-cloud-search-sample/tree/master/SitefinityWebApp.
This is 1 method i found in this code:
public IResultSet Search(ISearchQuery query)
{
AmazonCloudSearchDomainConfig config = new AmazonCloudSearchDomainConfig();
config.ServiceURL = "http://search-index2-cdduimbipgk3rpnfgny6posyzy.eu-west-1.cloudsearch.amazonaws.com/";
AmazonCloudSearchDomainClient domainClient = new AmazonCloudSearchDomainClient("AKIAJ6MPIX37TLIXW7HQ", "DnrFrw9ZEr7g4Svh0rh6z+s3PxMaypl607eEUehQ", config);
SearchRequest searchRequest = new SearchRequest();
List<string> suggestions = new List<string>();
StringBuilder highlights = new StringBuilder();
highlights.Append("{\'");
if (query == null)
throw new ArgumentNullException("query");
foreach (var field in query.HighlightedFields)
{
if (highlights.Length > 2)
{
highlights.Append(", \'");
}
highlights.Append(field.ToUpperInvariant());
highlights.Append("\':{} ");
SuggestRequest suggestRequest = new SuggestRequest();
Suggester suggester = new Suggester();
suggester.SuggesterName = field.ToUpperInvariant() + "_suggester";
suggestRequest.Suggester = suggester.SuggesterName;
suggestRequest.Size = query.Take;
suggestRequest.Query = query.Text;
SuggestResponse suggestion = domainClient.Suggest(suggestRequest);
foreach (var suggest in suggestion.Suggest.Suggestions)
{
suggestions.Add(suggest.Suggestion);
}
}
highlights.Append("}");
if (query.Filter != null)
{
searchRequest.FilterQuery = this.BuildQueryFilter(query.Filter);
}
if (query.OrderBy != null)
{
searchRequest.Sort = string.Join(",", query.OrderBy);
}
if (query.Take > 0)
{
searchRequest.Size = query.Take;
}
if (query.Skip > 0)
{
searchRequest.Start = query.Skip;
}
searchRequest.Highlight = highlights.ToString();
searchRequest.Query = query.Text;
searchRequest.QueryParser = QueryParser.Simple;
var result = domainClient.Search(searchRequest).SearchResult;
//var result = domainClient.Search(searchRequest).SearchResult;
return new AmazonResultSet(result, suggestions);
}
I have already created domain in Amazon Cloud Search using AWS console and uploaded document using Amazon predefine configuration option that is movie Imdb json file provided by Amazon for demo.
But in this method I am not getting how to use this method, like if I want to search Director name then how do I pass in this method as because this method parameter is of type ISearchQuery?
I'd suggest using the official AWS CloudSearch .NET SDK. The library you were looking at seems fine (although I haven't look at it any detail) but the official version is more likely to expose new CloudSearch features as soon as they're released, will be supported if you need to talk to AWS support, etc, etc.
Specifically, take a look at the SearchRequest class -- all its params are strings so I think that obviates your question about ISearchQuery.
I wasn't able to find an example of a query in .NET but this shows someone uploading docs using the AWS .NET SDK. It's essentially the same procedure as querying: creating and configuring a Request object and passing it to the client.
EDIT:
Since you're still having a hard time, here's an example. Bear in mind that I am unfamiliar with C# and have not attempted to run or even compile this but I think it should at least be close to working. It's based off looking at the docs at http://docs.aws.amazon.com/sdkfornet/v3/apidocs/
// Configure the Client that you'll use to make search requests
string queryUrl = #"http://search-<domainname>-xxxxxxxxxxxxxxxxxxxxxxxxxx.us-east-1.cloudsearch.amazonaws.com";
AmazonCloudSearchDomainClient searchClient = new AmazonCloudSearchDomainClient(queryUrl);
// Configure a search request with your query
SearchRequest searchRequest = new SearchRequest();
searchRequest.Query = "potato";
// TODO Set your other params like parser, suggester, etc
// Submit your request via the client and get back a response containing search results
SearchResponse searchResponse = searchClient.Search(searchRequest);
I am using Java EMR API to run pig job on EMR cluster. I am using following code to add Steps in JobFLow:
String jobFlowId = "j-assdasd";
AmazonElasticMapReduceClient client = new AmazonElasticMapReduceClient(
credentials);
StepFactory stepFactory = new StepFactory();
StepConfig executePig = new StepConfig()
.withName("Execute Pig")
.withActionOnFailure(ActionOnFailure.CANCEL_AND_WAIT)
.withHadoopJarStep(
stepFactory
.newRunPigScriptStep("s3://bucket/script/load.pig"));
AddJobFlowStepsRequest pig = new AddJobFlowStepsRequest(jobFlowId)
.withSteps( executePig);
AddJobFlowStepsResult result = client.addJobFlowSteps(pig);
How can i get the status of the "Execute Pig" status? I want to make program wait till the step finishes on EMR.
I found a way to do it java:
List<String> id = result.getStepIds();
DescribeStepResult res = client.describeStep(new DescribeStepRequest().withStepId(id.get(0)));
StepStatus status = res.getStep().getStatus();
String stas = status.getState();
But, here we need to loop on status till its return completed.
As Ajay mentioned on his own answer, there is a need for a loop to constantly check the statuses of the cluster, bootstrap actions, and steps. This post shows how to create such loop to keep the program inside of it until a certain status is reached.
I want to view events over specific time range for a specific calendar, but am having trouble using the API, It is a generic API, and it reminds me of using the DOM. The problem is that it seems difficult to work with because much of the information is in generic base classes.
How do I get the events for a calendar using Groovy or Java?
Does anybody have an example of passing credentials using curl?
Example code would be appreciated.
This document has examples for most of the common use cases. For example, here's the code for retrieving events for a specific time range
URL feedUrl = new URL("http://www.google.com/calendar/feeds/default/private/full");
CalendarQuery myQuery = new CalendarQuery(feedUrl);
myQuery.setMinimumStartTime(DateTime.parseDateTime("2006-03-16T00:00:00"));
myQuery.setMaximumStartTime(DateTime.parseDateTime("2006-03-24T23:59:59"));
CalendarService myService = new CalendarService("exampleCo-exampleApp-1");
myService.setUserCredentials("jo#gmail.com", "mypassword");
// Send the request and receive the response:
CalendarEventFeed resultFeed = myService.query(myQuery, Feed.class);
You could make this a bit Groovier, using something like:
def myQuery = new CalendarQuery("http://www.google.com/calendar/feeds/default/private/full".toURL()).with {
minimumStartTime = DateTime.parseDateTime("2006-03-16T00:00:00");
maximumStartTime = DateTime.parseDateTime("2006-03-24T23:59:59");
it
}
def myService = new CalendarService("exampleCo-exampleApp-1");
myService.setUserCredentials("jo#gmail.com", "mypassword");
// Send the request and receive the response:
def resultFeed = myService.query(myQuery, Feed);
If you do not need to alter the calendar, you only need to get the calendars private feed url, and you can use something like this (taken from the http://eu.gr8conf.org/agenda page). It uses the ICal4J library.
def url = "http://www.google.com/calendar/ical/_SOME_URL_/basic.ics".toURL()
def cal = Calendars.load(url)
def result = cal.components.sort { it.startDate.date }.collect {
def e = new Expando()
e.startDate = it.startDate.date
e.endDate = it.endDate.date
e.title = it.summary.value
if (it.location) {
e.presentation = Presentation.findByName(it.location.value, [fetch:"join"])
}
e.toString = {
"$startDate: $title"
}
return e
}
result
Happy hacking.