Is there a limit to the number of entries returned in a DocumentListFeed? I'm getting 100 results, and some of the collections in my account are missing.
How can I make sure I get all of the collections in my account?
DocsService service = new DocsService(APP_NAME);
service.setHeader("Authorization", "Bearer " + accessToken);
String feedUrl = new URL("https://docs.google.com/feeds/default/private/full/-/folder?v=3&showfolders=true&showroot=true");
DocumentLisFeed feed = service.getFeed(feedUrl, DocumentListFeed.class);
List<DocumentListEntry> entries = feed.getEntries();
The size of entries is 100.
A single request to the Documents List feed by default returns 100 element, but you can configure that value by setting the ?max-results query parameter.
Regardless, in order to retrieve all documents and files you should always take into account sending multiple requests, one per page, as explained in the documentation:
https://developers.google.com/google-apps/documents-list/#getting_all_pages_of_documents_and_files
Please also note that it is now recommended to switch to the newer Google Drive API, which interacts with the same resources and has complete documentation and sample code in multiple languages, including Java:
https://developers.google.com/drive/
You can call
feed.getNextLink().getHref()
to get a URL that you an form another feed with. This can be done until the link is null, at which point all the entries have been fetched.
Related
Two things:
I am trying to set custom metadata on a GCS object signed URL.
I am trying to set a maximum file size on a GCS object signed URL.
Using the following code:
Map<String, String> headers = new HashMap<>();
headers.put("x-goog-meta-" + usernameKey, username);
if (StringUtils.hasText(purpose)) {
headers.put("x-goog-meta-" + purposeKey, purpose);
}
if (maxFileSizeMb != null) {
headers.put("x-goog-content-length-range", String.format("0,%d", maxFileSizeMb * 1048576));
}
List<Storage.SignUrlOption> options = new ArrayList<>();
options.add(Storage.SignUrlOption.httpMethod(HttpMethod.POST));
options.add(Storage.SignUrlOption.withExtHeaders(headers));
String documentId = documentIdGenerator.generateDocumentId().getFormatted();
StorageDocument storageDocument =
StorageDocument.builder().id(documentId).path(getPathByDocumentId(documentId)).build();
storageDocument.setFormattedName(documentId);
SignedUrlData.SignedUrlDataBuilder builder =
SignedUrlData.builder()
.signedUrl(storageInterface.signUrl(gcpStorageBucket, storageDocument, options))
.documentId(documentId)
.additionalHeaders(headers);
First of all the generated signed URL works and I can upload a document.
Now I am expecting to see the object metadata through the console view. There is no metadata set though. Also the content-length-range is not respected. I can upload a 1.3 MB file when the content-length-range is set to 0,1.
Something happens when I upload a bigger file (~ 5 MB), but within the content-length-range. I receive an error message: Metadata part is too large..
As you can see here content-length-range requires both a minimum and maximum size. The unit used for the range is bytes, as you can see in this example.
I also noticed that you used x-goog-content-length-range, I found this documentation for it, when using this header take into account:
Use a PUT request, otherwise it will be silently ignored.
If the size of the request's content is outside the specified range, the request fails and a 400 Bad Request code is returned in the response.
You have to set the minimum and maximum size in bytes.
In api automation, let's consider below code to hit the api and get response.
Response res= given().
formParam("email", "value").
formParam("password", "value").
formParam("action", "login").
header("token","value").
when().post("MyResource").then().assertThat().statusCode(200).extract().response();
Actually in the above code we're just building an api with formParams, header and resource with post request right!
so, there we have created 3 formParm manually and passed the values right? now i want that to added automatically based on no of parameters that we have from an excel sheet.
Thing is these parameters can be deleted or new parameters gets added in future, so that's why i want to add those dynamically from an excel sheet data.
how i can do that? any suggestions?
If i've understood you right, you want to have the parameters on the spreadsheet to use dynamically and on runtime.
To do that, you'll need to:
Read the spreadsheet file and get the parameters contained in a data structure such as a Map<String,String>, in which, the key is the field and the value is... the value :P
Iterate upon the data structure selected and set the formParam generically
final RequestSpecification given = given();
for(final Map.Entry<String,String> entry : map.entrySet()) {
given.formParam(entry.getKey(), entry.getValue());
}
final Response res = given.other_things() ...
I hope it helps, but keep in mind the use of Map is not necessary. Choose the data structure more appropriate to your situation!
I'm working on a custom Solr search component which takes into account the number of documents in the collection. Currently the number of documents is hard coded in my Solr configuration file, and that's bad because the number of documents is dynamic. Is it possible to get the number of documents (in the whole collection, not in a single core) from the response builder? So far I have found a way to get the cloud descriptor (rb.req.getCore().getCoreDescriptor().getCloudDescriptor()), but in contrast to my expectations I did not see a getNumDocs() method in there.
I used following code to get the NumberOfDocuments in my SOLR Cloud Collection.
HttpSolrServer httpSolrServer = new HttpSolrServer("http://localhost:8983/solr/collectionname/");
QueryResponse response = httpSolrServer.query(new SolrQuery(), METHOD.POST);
SolrDocumentList solrDocumentList = queryResponse.getResults();
solrDocumentList.getNumFound();
solrDocumentList.getStart();
Hope this Helps you!!!
I am trying to sort a Google Spreadsheet with the Java API but unfortunately it doesn't seem to work. The code I am using is really simple as shown in the API reference.
URL listFeedUrl = new URI(worksheet.getListFeedUrl().toString() + "?orderby=columnname").toURL();
However, this does not work. The feed returned is not sorted at all. Am I missing something? FYI the column I am trying to sort contains email addresses.
EDIT: I just realized that the problem only happens with the old version of Google Spreadsheet.
maybe this happens. The query is performed on the spreadsheet xml and xml tags are in lower case, for example the title of my column in my spreadseet is "Nombre" and the xml <gsx:nombre>is not working so instead of using [?orderby=Nombre], use [?orderby=nombre] with a lowercase "n"
The correct query for this is.
URL listFeedUrl = new URI(worksheet.getListFeedUrl().toString() + "?orderby=nombre").toURL();
OK so I am starting a Bing search, then retrieving a couple resulting urls and using those as starting points to traverse other pages, parsing links from them and adding them to a List.
The problem I'm having is, I don't want to visit the same domain twice. I can stop it from visiting the same URL but if a page has link to another part of the website (such as an about page) I can't.
Currently I've a LinkedList where I add a URL to every time I parse one from the document using Jsoup. And I have a HashMap for storing already visited URLs. So I have it set up in a basic "if" like this:
if(!urlsVisited.containsKey(url))
{
urlsToVisit.add(url);
urlsVisited.put(url, url);
}
This is in a for loop where I retrieve the links on each page (currently 4 threads handling 4 pages).
This stops it from adding the likes of "http://www.stackoverflow.com" twice but doesn't work if I were to come across "http://www.stackoverflow.com/questions/ask".
I would like to add one link from StackOverflow (for example) and then be done with that domain. Any ideas?
I'm using Jsoup api in Java to parse results.
You can use URI class to parse your URLs. I also recommend to use Set<String> to store visited domains:
Set<String> urlsVisited = new HashSet<String>();
...
String domain = new URI(url).getHost();
if(!urlsVisited.contains(domain))
{
urlsToVisit.add(url);
urlsVisited.add(domain);
}
Use the java.net.URL class to pull the host name, and use that as the key to your urlsVisited map.
http://docs.oracle.com/javase/6/docs/api/java/net/URL.html#getHost()