We Keep Coding

How to you implements mongoTemplate.findDistinct()?

Im trying to use the findDistinct function from mongoTemplate but i always retrieve an empty result list.
Can you help me out to spot the problem ? Or maybe you have a simpliest way to do it
NB:
I do have data in my collection
(on a basic find, i fetch more than 300 results in the list but all of this result are the same excepting on one key, i want all the distinct object from their NAME value for instance )
I tryied this :
List<DiffusionListImpl> list = new ArrayList<>();
try{
query = new Query(Criteria.where("CUSTOMERNUMBER").is(1));
list = mongoTemplate.findDistinct(query, KeyWhereIWantTheDistinct, collectionName,
KlassResultModel.class);
} catch (MongoException e) {
logger.error("MongoException: " + e);
} catch (Exception e) {
logger.error("Error: " + e);
}
return list;

My bad, i misread the documentation.
But i find it akward to have this kind of comportement of this function.
I have to make a call to the DB to fetch a list of distinct value and then make another Call of the same DB to retrieve the object.
Is there any way to do it in one call? (Performance issue)

It can be done in one DB call, use below code.
final List<DiffusionListImpl> result =
IteratorUtils.toList(this.mongoTemplate.getCollection("collectionName")
.distinct("fieldName", query.getQueryObject(), DiffusionListImpl.class)
.iterator());
for IteratorUtils you can use apache
import org.apache.commons.collections4.IteratorUtils;

How to catch and modify data over exception occurred during Stream usage

I am currently working over CSV exports. I am fetching header from properties file with below code -
String[] csvHeader = exportables.get(0).getCSVHeaderMap(currentUser).keySet().stream().
map(s ->messageSource.getMessage("report."+s, null, locale)).toArray(String[]::new);
The above code works well. But i need to find a way to handle exception and also fetch data from another file, if it is not found in above file. I am expecting to use somewhat below code -
try{
exportables.get(0).getCSVHeaderMap(currentUser).keySet().stream().
map(s ->messageSource.getMessage("report."+s, null, locale)).toArray(String[]::new);
}catch(NoSuchMessageException e){
// code to work over lacking properties
}
I want to catch the 's' element in catch block (or in other good way). So that i can fetch it from another file and also add its return to current csvHeader.

One way is to make for each element a try catch block like:
exportables.get(0).getCSVHeaderMap(currentUser).keySet().stream().
map(s -> {
String result;//Put the class to which you map
try{
result = messageSource.getMessage("report."+s, null, locale);
}catch(NoSuchMessageException e){
// code to work over lacking properties here you have access to s
}
return result;
}
).toArray(String[]::new);
Another solution will be to check for specific problems and then there is no need to catch exceptions. For example if s is null and then you want to get the data from another place:
exportables.get(0).getCSVHeaderMap(currentUser).keySet().stream().
map(s -> {
String result;//Put the class to which you map
if(null == s)// Some condition that you want to check.
{
//get info from another place
//result = ....
}
else
{
result = messageSource.getMessage("report."+s, null, locale);
}
return result;
}
).toArray(String[]::new);

Ldap search by email value can't find records

I have written a Java application that searches Active directory via LDAP for user information. I have a list of instances of custom Person class that is passed in. In it I have either DN or email defined. I am modifying the search criteria accordingly. Here is the code:
for (Person person : members) {
boolean ready = false;
String filter = getConfig().getUserSearchFilter();
// (&(|(objectclass=user)(objectclass=person)(objectclass=inetOrgPerson)(objectclass=organizationalPerson)))
String base = person.getDistinguishedName();
if (base != null && !base.isEmpty()) {
ready = true;
} else if (person.getEmail() != null) {
base = getConfig().getMemberSearchBase();
// ou=Users,ou=Managed,dc=division,dc=company,dc=com
String mail = person.getEmail();
StringBuilder filterBuilder = new StringBuilder(filter);
int pIdx = filterBuilder.lastIndexOf(")");
filterBuilder.insert(pIdx, "(|(mail=" + mail + ")(x-personalmail=" + mail + "))");
filter = filterBuilder.toString();
LOG.debug("New value of a filter = {}", filter);
ready = true;
}
if (ready) {
try {
NamingEnumeration<SearchResult> search = getContext().search(base, filter, searchControls);
...
} catch (NamingException nex) {
throw new IOException(nex);
}
} else {
LOG.error("Incorrect search criteria for user {} of group {}. Person skipped", person.getName(), this.group.getName());
}
}
Code is working without errors, but when DN is specified it does find a person, but when email is defined it finds nothing.
However, If I copy generated filter string and pass it to ldapsearch command in a form of:
ldapsearch -LLL -x -H ldaps://my.ldap.server.com -D 'svc-acct#corp-dev.company.com' -W -b "ou=Users,ou=Managed,dc=division,dc=company,dc=com" '(&(|(objectclass=user)(objectclass=person)(objectclass=inetOrgPerson)(objectclass=organizationalPerson))(|(mail=person#domain.com)(x-personalmail=person#domain.com)))'
It does find this person perfectly.
Did anyone faced similar problem? Do you see any flaws in my code?
Please, do help me.

I did find the cause of my problem.
In the search control I had scope defined as OBJECT_SCOPE.
It does work when you are specifying DN, but with the search per one of the fields it fails finding the object.
I changed the scope to SUBTREE_SCOPE and everything started working as expected.

document locking returns not expected value

I have a function in which I would like to check if a user can set a lock on document. If so I know the user can edit the document.
public boolean canWriteDocument(String docId, String userName) {
boolean canWrite = false;
Session session = null;
session = getCurrentSession();
try {
Database db = session.getDatabase("", this.activeDb);
if (db.isDocumentLockingEnabled()) {
//Document locking is enabled
Document doc = db.getDocumentByUNID(docId);
if (doc.lock(userName)) {
canWrite = true;
System.out.println("document can be locked by user");
doc.unlock();
} else {
System.out.println("document can NOT be locked by user");
}
} else {
//Document locking is NOT enabled
}
} catch (NotesException e) {
// fail silently
System.out.println("failure docLock");
//e.printStackTrace();
}
return canWrite;
}
I call the function as followed:
canWriteDocument("99DE330A432849AFC125803400313C73", "CN=John Doe/O=quintessens")
However there must be something wrong because when the user (with ACL Author access level) is not listed in the field of type Authors it returns true.
When I lower the ACL access level to Reader the returned value is still true.
Anyone can explain why this is happening?

It may be that the database access level is cached within the server. Try rebooting the server (if it's a development environment) or testing with a brand new user that has never been in the ACL before.
If Database.queryAccess() is also not correct, that would confirm caching is the issue.
(It's possible that the two ways of checking access work differently under the hood, so if queryAccess() returns the correct value, it may not categorically mean the database locking option should.)

Controlling the list of URL(s) to be crawled at runtime

In crawler4j we can override a function boolean shouldVisit(WebUrl url) and control whether that particular url should be allowed to be crawled by returning 'true' and 'false'.
But can we add URL(s) at runtime ? if yes , what are ways to do that ?
Currently I can add URL(s) at beginning of program using addSeed(String url) function before the start(BasicCrawler.class, numberOfCrawlers) in CrawlController class and if I try to add new url using addSeed(String url), it gives error. Here is error image .
Any help will be appreciative and please let me know if any more detail about project is required to answer the question .

You can do this.
Use public void schedule(WebURL url) to add URLs to the crawler frontier which is a member of the Frontier.java class. But for this you need to have your url of type WebURL. If you want to make a WebURL out of your string. Please have a look at the addSeed() (below code) which is in the CrawlController.java class to see how it has converted the string (url) into a WebURL.
Also use the existing frontier instance.
Hope this helps..
public void addSeed(String pageUrl, int docId) {
String canonicalUrl = URLCanonicalizer.getCanonicalURL(pageUrl);
if (canonicalUrl == null) {
logger.error("Invalid seed URL: " + pageUrl);
return;
}
if (docId < 0) {
docId = docIdServer.getDocId(canonicalUrl);
if (docId > 0) {
// This URL is already seen.
return;
}
docId = docIdServer.getNewDocID(canonicalUrl);
} else {
try {
docIdServer.addUrlAndDocId(canonicalUrl, docId);
} catch (Exception e) {
logger.error("Could not add seed: " + e.getMessage());
}
}
WebURL webUrl = new WebURL();
webUrl.setURL(canonicalUrl);
webUrl.setDocid(docId);
webUrl.setDepth((short) 0);
if (!robotstxtServer.allows(webUrl)) {
logger.info("Robots.txt does not allow this seed: " + pageUrl);
} else {
frontier.schedule(webUrl); //method that adds URL to the frontier at run time
}
}

Presumably you can implement this function however you like, and have it depend on a list of URLs that should not be crawled. The implementation of shouldVisit is then going to involve asking if a given URL is in your list of forbidden URLs (or permitted URLs), and returning true or false on that basis.