We Keep Coding

Loading all contacts using the Microsoft Graph API sometimes looses/skips pages

We have an application that loads all contacts stored in an account using the Microsoft Graph API. The initial call we issue is https://graph.microsoft.com/v1.0/users/{userPrincipalName}/contacts$count=true&$orderBy=displayName%20ASC&$top=100, but we use the Java JDK to do that. Then we iterate over all pages and store all loaded contacts in a Set (local cache).
We do this every 5 minutes using an account with over 3000 contacts and sometimes, the count of contacts we received due to using $count does not match the number of contacts we loaded and stored in the local cache.
Verifying the numbers manually we can say, that the count was always correct, but there are contacts missing.
We use the following code to achieve this.
public List<Contact> loadContacts() {
Set<Contact> contacts = new TreeSet<>((contact1, contact2) -> StringUtils.compare(contact1.id, contact2.id));
List<QueryOption> requestOptions = List.of(
new QueryOption("$count", true),
new QueryOption("$orderBy", "displayName ASC"),
new QueryOption("$top", 100)
);
ContactCollectionRequestBuilder pageRequestBuilder = null;
ContactCollectionRequest pageRequest;
boolean hasNextPage = true;
while (hasNextPage) {
// initialize page request
if (pageRequestBuilder == null) {
pageRequestBuilder = graphClient.users(userId).contacts();
pageRequest = pageRequestBuilder.buildRequest(requestOptions);
} else {
pageRequest = pageRequestBuilder.buildRequest();
}
// load
ContactCollectionPage contactsPage = pageRequest.get();
if (contactsPage == null) {
throw new IllegalStateException("request returned a null page");
} else {
contacts.addAll(contactsPage.getCurrentPage());
}
// handle next page
hasNextPage = contactsPage.getNextPage() != null;
if (hasNextPage) {
pageRequestBuilder = contactsPage.getNextPage();
} else if (contactsPage.getCount() != null && !Objects.equals(contactsPage.getCount(), (long) contacts.size())) {
throw new IllegalStateException(String.format("loaded %d contacts but response indicated %d contacts", contacts.size(), contactsPage.getCount()));
} else {
// done
}
}
log.info("{} contacts loaded using graph API", contacts.size());
return new ArrayList<>(contacts);
}
Initially, we did not put the loaded contacts in a Set by ID but just in a List. With the List we very often got more contacts than $count. My idea was, that there is some caching going on and some pages get fetched multiple times. Using the Set we can make sure, that we only have unique contacts in our local cache.
But using the Set, we sometimes have less contacts than $count, meaning some pages got skipped and we end up in the condition that throws the IllegalStateException.
Currently, we use microsoft-graph 5.8.0 and azure-identiy 1.4.2.
Have you experienced similar issues and can help us solve this problem?
Or do you have any idea what could be causing these inconsistent results?
Your help is very much appreciated!

View.getView() returns null

I'm trying to find my folder or view in my database. Which named Team Documents this folder has some filter option like By Date, By Category. But this returns me a null even the folder already exists.
String dbServer = "d23dbm95/23/A/IBM", dbFileName = "dbom\\farizan\\stsklb1.nsf";
public void runNotes()
{
Session session = null;
Database db = null;
View view = null;
Document doc = null;
try
{
NotesThread.sinitThread();
session = NotesFactory.createSession();
System.out.println("User = " + session.getUserName());
db = session.getDatabase(dbServer, dbFileName);
if(db.isOpen())
{
System.out.println("Title "+db.getTitle());
view = db.getView("Team Documents \\ By Date");
if(view == null)
{
System.out.println("still null");
}
}
}
catch(NotesException e)
{
e.printStackTrace();
}
}
I tried also to fill my getView() method like Team Documents. But still returns a null. Any approach to this problem?

While it would have been more helpful if you had included a link to a screenshot of your Domino Designer client's folder list, my best guess is that you have two folders, not one folder with "filter options". Also, my guess is that "Team Documents" is not actually a folder; it's just a prefix on the folder names that makes them appear to be nested in a parent folder.
If that's the case, you would need
iew = db.getView("Team Documents\\By Category");
Or
iew = db.getView("Team Documents\\By Date");
Note: No spaces before & after the backslashes.
If my assumptions above are not correct, then my suggestion would be to assign alias names to the folders in Domino Designer and use the aliases instead of the display names in your code. Frankly, that's always a good practice, because it allows your code to continue working even if you decide to change the display names.

Create new directory in share-point using SOAP in java

I have just started working on share-point using java my firm uses share-point 2010 version so I have to work using SOAP and I have done following functionality.
Download Document from share-point.
Upload Document to share-point.
But I am stuck at creating new directory, I have tried some code by googling but no luck.
Here is my code:
public void createFolder(ListsSoap ls, String filePathToCreate,
String fileName, LoginDO loginDO, HttpServletResponse response)
throws Exception {
try {
String query = "";
String folderName = "NewFolder";
// 1. Prepare Query, Query Options and field Options
if (CommonUtilities.isValidStr(folderName)) {
// Prepare Query & Query Options for child folders
query = "<Batch OnError=\"Continue\" PreCalc=\"TRUE\" ListVersion=\"0\" " +
"RootFolder=\"https://xxx/Shared%20Documents/FPL%20SOs%20-%20JD%20Documents\">"
+ "<Method ID=\"1\" Cmd=\"New\">"
+ "<Field Name=\"FSObjType\">1</Field>"
+"<Field Name=\"ID\">New</Field>"
+ "<Field Name=\"BaseName\">" + folderName + "</Field>"
+ "</Method></Batch>";
} else {
// Prepare Query & Query Options for Parent folders
query = "<Query><Where><Eq><FieldRef Name=\"FSObjType\" />"
+ "<Value Type=\"Lookup\">1</Value></Eq></Where></Query>";
}
UpdateListItems.Updates updates = null;
// 2. Prepare Query, QueryOptions & ViewFields object as per options
if (CommonUtilities.isValidStr(query)) {
updates = new UpdateListItems.Updates();
updates.getContent().add(
sharepointUtil.createSharePointCAMLNode(query));
}
// 3. Call Web service to get result for selected options
UpdateListItemsResult result = ls
.updateListItems(SHAREPOINT_FOLDER_NAME,updates);
/*
* CommonUtilities
* .getApplicationProperty(ApplicationConstants.SHAREPOINT_FOLDER_NAME
* ), "", msQuery, viewFields, "", msQueryOptions, "");
*/
// 4. Get elements from share point result
Element element = (Element) result.getContent().get(0);
NodeList nl = element.getElementsByTagName("z:row");
for (int i = 0; i < nl.getLength(); i++) {
Node node = nl.item(i);
System.out.println("Some.!");
}
} catch (Exception e) {
e.printStackTrace();
}
// logger.logMethodEnd();
}`
Anyhow updateListItems() method is executed without error but there is nothing in result.
Any help is appreciated.
Thank you :)

seems there was problem in
"RootFolder=\"https://xxx/Shared%20Documents/FPL%20SOs%20-%20JD%20Documents\">"
while passing root directory to create folder within.
found other way around by passing folderName with full directories hierarchy.
Thank you all for your inputs :)

NetSuite SOAP API (SuiteTalk) to dump General Ledger

Can anyone give me advice on how to read the general ledger using SuiteTalk, the SOAP API from NetSuite?
For example, if you look at an account or a transaction on the NetSuite UI, there is an option to select "GL Impact". This produces a list of relevant general ledger entries.
However, I couldn't figure out a way to get the same list using SuiteTalk. One initially promising SOAP operation I tried calling was getPostingTransactionSummary(), but that is just a summary and lacks detail such as transaction dates. Another way is to call search() passing a TransactionSearchBasic object. That returns too many types of transaction and I'm not sure which of those actually have an impact on the general ledger.
I'm using Java and Axis toolkit for the SOAP operations, but examples in any language whatsoever (or raw SOAP XML) would be appreciated.

you are on the right track with your transaction search.
You are looking for posting is true and where the line has an account.
However I'd set this up in the saved search editor at least until you've figured out how you are going to filter to manageable numbers of lines. Then use TransactionSearchAdvanced with savedSearchId to pull that info via SuiteTalk

I am able to search GL transaction with below code, this could help you.
public void GetTransactionData()
{
DataTable dtData = new DataTable();
string errorMsg = "";
LoginToService(ref errorMsg);
TransactionSearch objTransSearch = new TransactionSearch();
TransactionSearchBasic objTransSearchBasic = new TransactionSearchBasic();
SearchEnumMultiSelectField semsf = new SearchEnumMultiSelectField();
semsf.#operator = SearchEnumMultiSelectFieldOperator.anyOf;
semsf.operatorSpecified = true;
semsf.searchValue = new string[] { "Journal" };
objTransSearchBasic.type = semsf;
objTransSearchBasic.postingPeriod = new RecordRef() { internalId = "43" };
objTransSearch.basic = objTransSearchBasic;
//Set Search Preferences
SearchPreferences _searchPreferences = new SearchPreferences();
Preferences _prefs = new Preferences();
_serviceInstance.preferences = _prefs;
_serviceInstance.searchPreferences = _searchPreferences;
_searchPreferences.pageSize = 1000;
_searchPreferences.pageSizeSpecified = true;
_searchPreferences.bodyFieldsOnly = false;
//Set Search Preferences
try
{
SearchResult result = _serviceInstance.search(objTransSearch);
List<JournalEntry> lstJEntry = new List<JournalEntry>();
List<JournalEntryLine> lstLineItems = new List<JournalEntryLine>();
if (result.status.isSuccess)
{
for (int i = 0; i <= result.recordList.Length - 1; i += 1)
{
JournalEntry JEntry = (JournalEntry)result.recordList[i];
lstJEntry.Add((JournalEntry)result.recordList[i]);
if (JEntry.lineList != null)
{
foreach (JournalEntryLine line in JEntry.lineList.line)
{
lstLineItems.Add(line);
}
}
}
}
try
{
_serviceInstance.logout();
}
catch (Exception ex)
{
}
}
catch (Exception ex)
{
throw ex;
}
}

Cannot retrieve Storage Gateway snapshots using Java API

I'm trying to grab a list of snapshots from our Storage Gateway and put them into a JTable. However, when I use the AWS Java API to retrieve a list of snapshots, I only am able to retrieve what appears to be the public snapshots published by Amazon. When I set the DescribeSnapshotsRequest.setOwnerIds() to include "self", the list is empty.
Here is the offending code:
private void createTable() {
Object[][] data = null;
String[] columns = new String[]{"Snapshot ID", "Owner ID", "Snapshot Date"};
DescribeSnapshotsRequest req = new DescribeSnapshotsRequest();
req.setOwnerIds(Arrays.<String>asList("self"));
try {
snapshots = ec2Client.describeSnapshots(req).getSnapshots();
data = new Object[snapshots.size()][3];
int i = 0;
for(Snapshot item : snapshots) {
data[i][0] = item.getSnapshotId();
data[i][1] = item.getOwnerId();
data[i][2] = item.getStartTime();
i++;
}
} catch(Exception e) {
System.out.println("Invalid Credentials!");
}
table = new JTable(data, columns);
table.setAutoCreateRowSorter(true);
}
The List snapshots is empty unless I either remove the DescribeSnapshotsRequest, or set the owner ID to "amazon".
Long story short, why can't I access my private snapshots from the Storage Gateway?

Figured it out. Turns out you have to explicitly define the EC2 endpoint. Somehow I missed that step.
Here is the list of endpoints:
http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region
AmazonEC2Client.setEndpoint("<Endpoint URL>");