We Keep Coding

Get number of entries in properties file apache commons

I'm creating a list of IP address' to ping in which a user can add to the list which is then saved to a properties file in the form of site.name1 = ... site.name2 = ...
Currently I have a for loop with a fixed amount, is there a way to get the number of entries in a properties file so I can set this in the for loop rather than wait for a exception?
PropertiesConfiguration config = configs.properties(new File("IPs.properties"));
//initially check for how many values there are - set to max increments for loop
for (int i = 0; i < 3; i++) { //todo fix
siteName = config.getString("site.name" + i);
siteAddress = config.getString("site.address" + i);
SiteList.add(i, siteName);
IPList.add(i, siteAddress);
}
I've looked through the documentation and other questions but they seem to be unrelated.

It looks to me based on the documentation you should be able to use PropertiesConfiguration#getLayout#getKeys to get a Set of all keys as a String.
I had to modify the code a bit to use apache-commons-configuration-1.10
PropertiesConfiguration config = new PropertiesConfiguration("ips.properties");
PropertiesConfigurationLayout layout = config.getLayout();
String siteName = null;
String siteAddress = null;
for (String key : layout.getKeys()) {
String value = config.getString(key);
if (value == null) {
throw new IllegalStateException(String.format("No value found for key: %s", key));
}
if (key.equals("site.name")) {
siteName = value;
} else if (key.equals("site.address")) {
siteAddress = value;
} else {
throw new IllegalStateException(String.format("Unsupported key: %s", key));
}
}
System.out.println(String.format("name=%s, address=%s", siteName, siteAddress));

Why doesn't Active Directory return me a PagedResultsResponseControl?

I am attempting to enumerate all the groups in our Active Directory from Java. There are quite a lot, so I get a SizeLimitExceededException after 1000 results. I am attempting to use PagedResultsControl, with my code very closely modelled on all the examples out on the web, and it sort-of-works, in that it no longer throws the SizeLimitExceededException, and returns a number of results matching the specified page size (provided that isn't greater than 1000).
However, the next step is to get the cookie from the response and use it to get the next page, and my issue is that there is no PagedResultsResponseControl in the context after the call to search(). In fact getResponseControls() returns null.
I have searched extensively and can't seem to find anyone else reporting this issue, and I'm pretty much stuck here. So what am I doing wrong? Why don't I get a PagedResultsResponseControl?
Our domain is running on Windows Server 2016 and I have reduced my code down to the following test case:
public class PagingTest {
public static void main(String[] args) throws Exception {
Hashtable<String, String> env = new Hashtable<>();
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.SECURITY_PRINCIPAL, "username");
env.put(Context.SECURITY_CREDENTIALS, "password");
env.put(Context.PROVIDER_URL, "ldap://campus.uni.ac.uk/DC=campus,DC=uni,DC=ac,DC=uk");
LdapContext adContext = new InitialLdapContext(env, null);
// Set up search controls.
SearchControls searchControl = new SearchControls();
searchControl.setSearchScope(SearchControls.SUBTREE_SCOPE);
String[] attributesToFetch = {"cn"};
searchControl.setReturningAttributes(attributesToFetch);
// Set up a paged search.
final int pageSize = 500;
byte[] cookie = null;
adContext.setRequestControls(new Control[]{
new PagedResultsControl(pageSize, Control.CRITICAL)
});
// Do the search.
int count = 0;
boolean finished = false;
while (!finished) {
NamingEnumeration<SearchResult> records
= adContext.search("OU=Groups", "objectClass=group", searchControl);
// Examine the page's results control response and act accordingly.
Control[] controls = adContext.getResponseControls();
if (controls != null) {
for (int i = 0; i < controls.length; ++i) {
if (controls[i] instanceof PagedResultsResponseControl) {
PagedResultsResponseControl prrc =
(PagedResultsResponseControl) controls[i];
cookie = prrc.getCookie();
if (cookie == null) {
finished = true;
}
}
}
} else {
cookie = null;
finished = true;
}
// Process the page of results.
while (records != null && records.hasMore()) {
SearchResult sr = records.next();
Attributes attribs = sr.getAttributes();
BasicAttribute ba = (BasicAttribute) attribs.get("cn");
String cn = (String) ba.get();
System.out.println(cn);
++count;
}
// Re-activate paged results with the new cookie.
adContext.setRequestControls(new Control[]{
new PagedResultsControl(pageSize, cookie, Control.CRITICAL)
});
}
System.out.println("Found " + count + " groups");
}
}

Maybe your ldap server does not support paging query, you can use ldapsearch command like this:
ldapsearch -H ldap://xxxx:389 -x -D "uid=zhangsan,ou=employee,dc=test,dc=com" -W -b "" -s base -a always "(objectClass=*)" "supportedControl"
If the return values contain 1.2.840.113556.1.4.319, it shows your ldap server supports paging query.

Paginate on LDAP server which does not support PagedResultsControl

I'm trying to get all entries on an LDAP server using Spring LDAP (version 2.3.2). Within my code, I make use of PagedResultsDirContextProcessor to paginate through all the result. This works fine on the servers which support PagedResultsControl.
However, I now need to connect to an LDAP server which does not support PagedResultsControl. How can I get all entries without using PagedResultsControl?

You can use VirtualListView via JNDI. You have to retrieve and re-supply the 'contextID' to paginate, as follows:
static final int LIST_SIZE = 20; // Adjust to suit
#Test
public void TestVLV() throws NamingException, IOException
{
Hashtable<String,Object> env = new Hashtable<>();
env.put(Context.INITIAL_CONTEXT_FACTORY,
"com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.PROVIDER_URL, "ldap://localhost");
env.put(Context.SECURITY_AUTHENTICATION, "simple");
env.put(Context.SECURITY_PRINCIPAL, "cn=XXXXXXX");
env.put(Context.SECURITY_CREDENTIALS, "YYYYYYY");
try
{
/* Create initial context with no connection request controls */
LdapContext ctx = new InitialLdapContext(env, null);
/* Sort Control is required for VLV to work */
SortKey[] sortKeys =
{
// sort by cn
new SortKey("cn", true, "caseIgnoreOrderingMatch")
};
// Note: the constructors for SortControl that take String or String[]
// as the first argument produce 'no ordering rule' errors with OpenLDAP.
SortControl sctl = new SortControl(
// "cn",
// new String[]{"cn"},
sortKeys,
Control.CRITICAL);
/* VLV that returns the first 20 answers */
VirtualListViewControl vctl =
new VirtualListViewControl(1, 0, 0, LIST_SIZE-1, Control.CRITICAL);
/* Set context's request controls */
ctx.setRequestControls(new Control[]
{
sctl,
vctl
});
int count = 0;
SearchControls sc = new SearchControls(SearchControls.SUBTREE_SCOPE, 0, 0, null, false, false);
for (;;)
{
/* Perform search */
// System.out.println("namespace="+ctx.getNameInNamespace());
// System.out.println("count limit="+sc.getCountLimit());
// System.out.println("search scope="+sc.getSearchScope());
NamingEnumeration<SearchResult> ne =
ctx.search("ou=Users,dc=xxxx,dc=com", "(objectClass={0})", new String[]{"inetOrgPerson"}, sc);
/* Enumerate search results */
while (ne.hasMore())
{
count++;
SearchResult sr = ne.next();
// System.out.println(i+": "+sr.getName());
System.out.println(count+": "+sr.getNameInNamespace());
}
ne.close();
// Get the contextID.
Control[] controls = ctx.getResponseControls();
VirtualListViewResponseControl vlvrc = null;
byte[] contextID = null;
for (int j = 0; j < controls.length; j++)
{
if (controls[j] instanceof VirtualListViewResponseControl)
{
vlvrc = (VirtualListViewResponseControl)controls[j];
contextID = vlvrc.getContextID();
System.out.println("contextID=0x"+new BigInteger(1,contextID).toString(16));
if (contextID != null)
{
vctl = new VirtualListViewControl(vlvrc.getTargetOffset()+LIST_SIZE, 0, 0, LIST_SIZE-1, Control.CRITICAL);
vctl.setContextID(contextID);
ctx.setRequestControls(new Control[]
{
sctl,
vctl
});
}
break; // there should only be one VLV response control, and we're not interested in anything else.
}
}
if (vlvrc != null && contextID != null && count < vlvrc.getListSize())
{
System.out.println("Continuing");
}
else
{
System.out.println("Finished");
break;
}
}
ctx.close();
}
finally
{
}
}
Adjust the authentication and search root and filter to suit yourself, of course.
And to test whether it is supported (although an 'unsupported critical control' exception from the above code will tell you just as well):
/**
* Is VLV Control supported?
*
* Query the rootDSE object to find out if VLV Control is supported.
* #return true if it is supported.
*/
static boolean isVLVControlSupported(LdapContext ctx)
throws NamingException
{
String[] returningAttributes =
{
"supportedControl"
};
// Fetch the supportedControl attribute of the rootDSE object.
Attributes attrs = ctx.getAttributes("", returningAttributes);
Attribute attr = attrs.get("supportedControl");
System.out.println("supportedControls="+attr);
if (attr != null)
{
// Fast way to check. add() would have been just as good. Does no damage to the DIT.
return attr.remove(VLV_CONTROL_OID);
}
return false;
}
The VirtualListViewControl and VirtualListViewResponseControl are part of the Sun/Oracle LDAP Booster Pack, which you can obtain via Maven as:
<dependency>
<groupId>com.sun</groupId>
<artifactId>ldapbp</artifactId>
<version>1.0</version>
<type>jar</type>
</dependency>

Super frustrating..
I wouldn't really recommend it, but you could pull off something like this where you manually paginate by cn/sn..
List<String> alphabetRange = getAlphabetRange();
for (int i = 0; i < alphabetRange.size() - 1; i++) {
String filter = "(&(sn>=" + alphabetRange.get(i) + ")" + "(sn<=" + alphabetRange.get(i + 1) + " ))";
NamingEnumeration<SearchResult> searchResult = context.search(base_dn, filter, controls);
while (searchResult.hasMore()) {
// searchResult.next().getAttributes() and do something with it
}
}
private List<String> getAlphabetRange() {
List<String> result = new ArrayList<>();
for (char alph = 'A'; alph <= 'Z'; alph++) {
if (alph == 'S') {
result.add("S");
result.add("Sd");
} else {
result.add(String.valueOf(alph));
}
}
result.add("Zz");
return result;
}
as soon as you run over (i.e. 1.000) in one "page", you will face the dreaded javax.naming.SizeLimitExceededException. You might add more "pages", like in that example between [S-Sd][Sd-T]

What exacly servers are you using?
if server has no queue limits you cant try set unlimited search:
SearchControls controls = new SearchControls();
controls.setTimeLimit(0);
controls.setCountLimit(0);

LensKit: LensKit demo is not reading my data file

When I run the LensKit demo program I get this error:
[main] ERROR org.grouplens.lenskit.data.dao.DelimitedTextRatingCursor - C:\Users\sean\Desktop\ml-100k\u - Copy.data:4: invalid input, skipping line
I reworked the ML 100k data set so that it only holds this line although I dont see how this would effect it:
196 242 3 881250949
186 302 3 891717742
22 377 1 878887116
244
Here is the code I am using too:
public class HelloLenskit implements Runnable {
public static void main(String[] args) {
HelloLenskit hello = new HelloLenskit(args);
try {
hello.run();
} catch (RuntimeException e) {
System.err.println(e.getMessage());
System.exit(1);
}
}
private String delimiter = "\t";
private File inputFile = new File("C:\\Users\\sean\\Desktop\\ml-100k\\u - Copy.data");
private List<Long> users;
public HelloLenskit(String[] args) {
int nextArg = 0;
boolean done = false;
while (!done && nextArg < args.length) {
String arg = args[nextArg];
if (arg.equals("-e")) {
delimiter = args[nextArg + 1];
nextArg += 2;
} else if (arg.startsWith("-")) {
throw new RuntimeException("unknown option: " + arg);
} else {
inputFile = new File(arg);
nextArg += 1;
done = true;
}
}
users = new ArrayList<Long>(args.length - nextArg);
for (; nextArg < args.length; nextArg++) {
users.add(Long.parseLong(args[nextArg]));
}
}
public void run() {
// We first need to configure the data access.
// We will use a simple delimited file; you can use something else like
// a database (see JDBCRatingDAO).
EventDAO base = new SimpleFileRatingDAO(inputFile, "\t");
// Reading directly from CSV files is slow, so we'll cache it in memory.
// You can use SoftFactory here to allow ratings to be expunged and re-read
// as memory limits demand. If you're using a database, just use it directly.
EventDAO dao = new EventCollectionDAO(Cursors.makeList(base.streamEvents()));
// Second step is to create the LensKit configuration...
LenskitConfiguration config = new LenskitConfiguration();
// ... configure the data source
config.bind(EventDAO.class).to(dao);
// ... and configure the item scorer. The bind and set methods
// are what you use to do that. Here, we want an item-item scorer.
config.bind(ItemScorer.class)
.to(ItemItemScorer.class);
// let's use personalized mean rating as the baseline/fallback predictor.
// 2-step process:
// First, use the user mean rating as the baseline scorer
config.bind(BaselineScorer.class, ItemScorer.class)
.to(UserMeanItemScorer.class);
// Second, use the item mean rating as the base for user means
config.bind(UserMeanBaseline.class, ItemScorer.class)
.to(ItemMeanRatingItemScorer.class);
// and normalize ratings by baseline prior to computing similarities
config.bind(UserVectorNormalizer.class)
.to(BaselineSubtractingUserVectorNormalizer.class);
// There are more parameters, roles, and components that can be set. See the
// JavaDoc for each recommender algorithm for more information.
// Now that we have a factory, build a recommender from the configuration
// and data source. This will compute the similarity matrix and return a recommender
// that uses it.
Recommender rec = null;
try {
rec = LenskitRecommender.build(config);
} catch (RecommenderBuildException e) {
throw new RuntimeException("recommender build failed", e);
}
// we want to recommend items
ItemRecommender irec = rec.getItemRecommender();
assert irec != null; // not null because we configured one
// for users
for (long user: users) {
// get 10 recommendation for the user
List<ScoredId> recs = irec.recommend(user, 10);
System.out.format("Recommendations for %d:\n", user);
for (ScoredId item: recs) {
System.out.format("\t%d\n", item.getId());
}
}
}
}
I am really lost on this one and would appreciate any help. Thanks for your time.

The last line of your input file only contains one field. Each input file line needs to contain 3 or 4 fields.

Why does the NetBeans Java debugger never reach this code?

I'm trying to debug a method in Java using NetBeans.
That method is:
public Integer getNumberOfClamps(Locations paLocation) {
Integer ret = -1;
List list = new ArrayList();
String removeme = "ABC";
if (paLocation == null) {
return ret;
}
try {
IO io = new IO(this.getSchemaName());
Session session = io.getSession();
String sql = "select count(*) from assets a join assettypes at on (at.id = a.assettype_id) ";
sql += "where a.currentlocation_id = " + paLocation.getId() + " and at.clamp = 1 and at.active = 1;";
list = session.createQuery(sql).list();
// for some reason, list is empty yet MySQL reports 40 records
// and the following two lines are never reached!
ret = list.size();
removeme = "WHAT???";
} catch (Exception ex) {
ret = -1; // explicitly set return
} finally {
return ret;
}
}
Towards the middle of the method you will see list = session.createQuery(sql).list();
For some reason, this is returning an empty list even though when the SQL is run manually, I get 40 results.
But the odd part is that once the .list() is called, it jumps to the finally block and never reaches the rest! So for testing, 'removeme' should equal WHAT??? but the debugger reports it as still ABC.
What gives?

You are using the wrong method. 'createQuery' is expecting HQL syntax. Change your method to 'createSQLQuery'