We Keep Coding

Develop a web application based on triplestore database

I've recently developed a "classic" 3-tier web applications using Java EE.
I've used GlassFish as application server, MS SQL Server as DBMS and xhtml pages with primefaces components for the front end.
Now, for educational purposes, I want to substitute the relational db with a pure triplestore database but I'm not sure about the procedure to follow.
I've searched a lot on google and on this site but I didn't find what I was looking for, because every answer I found was more theoretical than practical.
If possible, I need a sort of tutorial or some practical tips.
I've read the documentation about Apache Jena but I'm not able to find a solid starting point.
In particoular:
- In order to use MS SQL Server with GlassFish I've used a JDBC Driver, created a datasource and a connection pool. Does it exist an equivalent procedure to set up a triple store database?
- To handle users authentication, I've used a Realm. What should I do now?
For the moment I've created "by hand" a RDF schema and using Jena Schemagen I've translated it into a Java Class. What should I do now?

After several attempts and other research on the net I finally achieved my goal.
I decided to develop a hybrid solution in which I manage users login and their navigation permits via MS SQL Server and JDBCRealm, while I use Jena TDB to save all the other data.
Starting with an RDF schema, I created a Java class that contains resources and properties to easily create my statements via code. Here's an example:
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns="http://www.stackoverflow.com/example#"
xml:base="http://www.stackoverflow.com/example">
<rdfs:Class rdf:ID="User"></rdfs:Class>
<rdfs:Class rdf:ID="Project"></rdfs:Class>
<rdf:Property rdf:ID="email"></rdf:Property>
<rdf:Property rdf:ID="name"></rdf:Property>
<rdf:Property rdf:ID="surname"></rdf:Property>
<rdf:Property rdf:ID="description"></rdf:Property>
<rdf:Property rdf:ID="customer"></rdf:Property>
<rdf:Property rdf:ID="insertProject">
<rdfs:domain rdf:resource="http://www.stackoverflow.com/example#User"/>
<rdfs:range rdf:resource="http://www.stackoverflow.com/example#Project"/>
</rdf:Property>
</rdf:RDF>
And this is the Java class:
public class MY_ONTOLOGY {
private static final OntModel M = ModelFactory.createOntologyModel(OntModelSpec.RDFS_MEM);
private static final String NS = "http://www.stackoverflow.com/example#";
private static final String BASE_URI = "http://www.stackoverflow.com/example/";
public static final OntClass USER = M.createClass(NS + "User");
public static final OntClass PROJECT = M.createClass(NS + "Project");
public static final OntProperty EMAIL = M.createOntProperty(NS + "hasEmail");
public static final OntProperty NAME = M.createOntProperty(NS + "hasName");
public static final OntProperty SURNAME = M.createOntProperty(NS + "hasSurname");
public static final OntProperty DESCRIPTION = M.createOntProperty(NS + "hasDescription");
public static final OntProperty CUSTOMER = M.createOntProperty(NS + "hasCustomer");
public static final OntProperty INSERTS_PROJECT = M.createOntProperty(NS + "insertsProject");
public static final String getBaseURI() {
return BASE_URI;
}
}
Then I've created a directory on my PC where I want to store the data, like C:\MyTDBdataset.
To store data inside it, I use the following code:
String directory = "C:\\MyTDBdataset";
Dataset dataset = TDBFactory.createDataset(directory);
dataset.begin(ReadWrite.WRITE);
try {
Model m = dataset.getDefaultModel();
Resource user = m.createResource(MY_ONTOLOGY.getBaseURI() + "Ronnie", MY_ONTOLOGY.USER);
user.addProperty(MY_ONTOLOGY.NAME, "Ronald");
user.addProperty(MY_ONTOLOGY.SURNNAME, "Red");
user.addProperty(MY_ONTOLOGY.EMAIL, "ronnie#myemail.com");
Resource project = m.createResource(MY_ONTOLOGY.getBaseURI() + "MyProject", MY_ONTOLOGY.PROJECT);
project.addProperty(MY_ONTOLOGY.DESCRIPTION, "This project is fantastic");
project.addProperty(MY_ONTOLOGY.CUSTOMER, "Customer & Co");
m.add(user, MY_ONTOLOGY.INSERTS_PROJECT, project);
dataset.commit();
} finally {
dataset.end();
}
If I want to read statements in my TDB, I can use something like this:
dataset.begin(ReadWrite.READ);
try {
Model m = dataset.getDefaultModel();
StmtIterator iter = m.listStatements();
while (iter.hasNext()) {
Statement stmt = iter.nextStatement();
Resource subject = stmt.getSubject();
Property predicate = stmt.getPredicate();
RDFNode object = stmt.getObject();
System.out.println(subject);
System.out.println("\t" + predicate);
System.out.println("\t\t" + object);
System.out.println("");
}
m.write(System.out, "RDF/XML"); //IF YOU WANT TO SEE AT CONSOLE YOUR DATA AS RDF/XML
} finally {
dataset.end();
}
If you want to navigate your model in different ways, look at this tutorial provided by Apache.
If you want to remove specific statements in your model, you can write something like this:
dataset.begin(ReadWrite.WRITE);
try {
Model m = dataset.getDefaultModel();
m.remove(m.createResource("http://http://www.stackoverflow.com/example/Ronnie"), MY_ONTOLOGY.NAME, m.createLiteral("Ronald"));
dataset.commit();
} finally {
dataset.end();
}
That's all! Bye!

RallyRestAPI querying ScopedAttributeDefinition types

Using the RallyRestAPI, is there a way to query ScopedAttributeDefinition types? This appears to define custom datatypes in Rally. I am trying to build a data dictionary of the custom types we use in Rally and associate these custom types to the object they are attributes of. (In case that doesn't make sense, here's an example: We have a custom field called Enabler Lead on Rally PortfolioItems. I'd like to query Rally for all custom fields for PortfolioItem and get the Enabler Field, and its metadata, from the Rally REST API.)
I'm using the Java client.

I filed a github issue to add support for the ScopedAttributeDefinition: https://github.com/RallyTools/RallyRestToolkitForJava/issues/19
In the meantime though you can work around it by using the underlying http client directly. This code queries for all projects in your workspace and then finds the type definition for Portfolio Item and then for each project grabs all the custom attribute defs via the scopedattributedefinition endpoint and prints out their hidden/required status.
QueryRequest projectQuery = new QueryRequest("project");
projectQuery.setLimit(Integer.MAX_VALUE);
QueryResponse projectResponse = restApi.query(projectQuery);
QueryRequest typeDefQuery = new QueryRequest("typedefinition");
typeDefQuery.setQueryFilter(new QueryFilter("Name", "=", "Portfolio Item"));
QueryResponse typeDefResponse = restApi.query(typeDefQuery);
JsonObject piTypeDef = typeDefResponse.getResults().get(0).getAsJsonObject();
for(JsonElement projectResult : projectResponse.getResults()) {
JsonObject project = projectResult.getAsJsonObject();
System.out.println("Project: " + project.get("Name").getAsString());
//Begin hackery (note we're not handling multiple pages-
// if you have more than 200 custom attributes you'll have to page
String scopedAttributeDefUrl = "/project/" + project.get("ObjectID").getAsLong() +
"/typedefinition/" + piTypeDef.get("ObjectID").getAsLong() + "/scopedattributedefinition" +
"?fetch=Hidden,Required,Name&query=" + URLEncoder.encode("(Custom = true)", "utf-8");
String attributes = restApi.getClient().doGet(scopedAttributeDefUrl);
QueryResponse attributeResponse = new QueryResponse(attributes);
//End hackery
for(JsonElement customAttributeResult : attributeResponse.getResults()) {
JsonObject customAttribute = customAttributeResult.getAsJsonObject();
System.out.print("\tAttribute: " + customAttribute.get("Name").getAsString());
System.out.print(", Hidden: " + customAttribute.get("Hidden").getAsBoolean());
System.out.println(", Required: " + customAttribute.get("Required").getAsBoolean());
}
System.out.println();
}
Any other info you'd like for each of those custom fields should be accessible just by querying the Attributes collection from the piTypeDef.

Using ELKI with Mongodb

Using test cases I was able to see how ELKI can be used directly from Java but now I want to read my data from MongoDB and then use ELKI to cluster geographic (long, lat) data.
I can only cluster data from a CSV file using ELKI. Is it possible to connect de.lmu.ifi.dbs.elki.database.Database with MongoDB? I can see from the java debugger that there is a databaseconnection field in de.lmu.ifi.dbs.elki.database.Database.
I query MongoDB creating POJO for each row and now I want to cluster these objects using ELKI.
It is possible to read data from MongoDB and write it in a CSV file then use ELKI to read that CSV file but I would like to know if there is a simpler solution.
---------FINDINGS_1:
From ELKI - Use List<String> of objects to populate the Database I found that I need to implement de.lmu.ifi.dbs.elki.datasource.DatabaseConnection and specifically override the loadData() method which returns an instance of MultiObjectsBundle.
So I think I should wrap a list of POJO with MultiObjectsBundle. Now i'm looking at the MultiObjectsBundle and it looks like the data should be held in columns. Why columns datatype is List> shouldnt it be List? just a list of items you want to cluster?
I'm a little confused. How is ELKI going to know that it should look at the long and lat for POJO? Where do I tell ELKI to do this? Using de.lmu.ifi.dbs.elki.data.type.SimpleTypeInformation?
---------FINDINGS_2:
I have tried to use ArrayAdapterDatabaseConnection and I have tried implementing DatabaseConnection. Sorry I need thing in very simple terms for me to understand.
This is my code for clustering:
int minPts=3;
double eps=0.08;
double[][] data1 = {{-0.197574246, 51.49960695}, {-0.084605692, 51.52128377}, {-0.120973687, 51.53005939}, {-0.156876, 51.49313},
{-0.144228881, 51.51811784}, {-0.1680743, 51.53430039}, {-0.170134484,51.52834133}, { -0.096440751, 51.5073853},
{-0.092754157, 51.50597426}, {-0.122502346, 51.52395143}, {-0.136039674, 51.51991453}, {-0.123616824, 51.52994371},
{-0.127854211, 51.51772703}, {-0.125979294, 51.52635795}, {-0.109006325, 51.5216612}, {-0.12221963, 51.51477076}, {-0.131161087, 51.52505093} };
// ArrayAdapterDatabaseConnection dbcon = new ArrayAdapterDatabaseConnection(data1);
DatabaseConnection dbcon = new MyDBConnection();
ListParameterization params = new ListParameterization();
params.addParameter(de.lmu.ifi.dbs.elki.algorithm.clustering.DBSCAN.Parameterizer.MINPTS_ID, minPts);
params.addParameter(de.lmu.ifi.dbs.elki.algorithm.clustering.DBSCAN.Parameterizer.EPSILON_ID, eps);
params.addParameter(DBSCAN.DISTANCE_FUNCTION_ID, EuclideanDistanceFunction.class);
params.addParameter(AbstractDatabase.Parameterizer.DATABASE_CONNECTION_ID, dbcon);
params.addParameter(AbstractDatabase.Parameterizer.INDEX_ID,
RStarTreeFactory.class);
params.addParameter(RStarTreeFactory.Parameterizer.BULK_SPLIT_ID,
SortTileRecursiveBulkSplit.class);
params.addParameter(AbstractPageFileFactory.Parameterizer.PAGE_SIZE_ID, 1000);
Database db = ClassGenericsUtil.parameterizeOrAbort(StaticArrayDatabase.class, params);
db.initialize();
GeneralizedDBSCAN dbscan = ClassGenericsUtil.parameterizeOrAbort(GeneralizedDBSCAN.class, params);
Relation<DoubleVector> rel = db.getRelation(TypeUtil.DOUBLE_VECTOR_FIELD);
Relation<ExternalID> relID = db.getRelation(TypeUtil.EXTERNALID);
DBIDRange ids = (DBIDRange) rel.getDBIDs();
Clustering<Model> result = dbscan.run(db);
int i =0;
for(Cluster<Model> clu : result.getAllClusters()) {
System.out.println("#" + i + ": " + clu.getNameAutomatic());
System.out.println("Size: " + clu.size());
System.out.print("Objects: ");
for(DBIDIter it = clu.getIDs().iter(); it.valid(); it.advance()) {
DoubleVector v = rel.get(it);
ExternalID exID = relID.get(it);
System.out.print("DoubleVec: ["+v+"]");
System.out.print("ExID: ["+exID+"]");
final int offset = ids.getOffset(it);
System.out.print(" " + offset);
}
System.out.println();
++i;
}
The ArrayAdapterDatabaseConnection produces two clusters, I just had to play around with the value of epsilon, when I set epsilon=0.008 dbscan started creating clusters. When i set epsilon=0.04 all the items were in 1 cluster.
I have also tried to implement DatabaseConnection:
#Override
public MultipleObjectsBundle loadData() {
MultipleObjectsBundle bundle = new MultipleObjectsBundle();
List<Station> stations = getStations();
List<DoubleVector> vecs = new ArrayList<DoubleVector>();
List<ExternalID> ids = new ArrayList<ExternalID>();
for (Station s : stations){
String strID = Integer.toString(s.getId());
ExternalID i = new ExternalID(strID);
ids.add(i);
double[] st = {s.getLongitude(), s.getLatitude()};
DoubleVector dv = new DoubleVector(st);
vecs.add(dv);
}
SimpleTypeInformation<DoubleVector> type = new VectorFieldTypeInformation<>(DoubleVector.FACTORY, 2, 2, DoubleVector.FACTORY.getDefaultSerializer());
bundle.appendColumn(type, vecs);
bundle.appendColumn(TypeUtil.EXTERNALID, ids);
return bundle;
}
These long/lat are associated with an ID and I need to link them back to this ID to the values. Is the only way to go that using the ID offset (in the code above)? I have tried to add ExternalID column but I don't know how to retrieve the ExternalID for a particular NumberVector?
Also after seeing Using ELKI's Distance Function I tried to use Elki's longLatDistance but it doesn't work and I could not find any examples to implement it.

The interface for data sources is called DatabaseConnection.
JavaDoc of DatabaseConnection
You can implement a MongoDB-based interface to get the data.
It is not complicated interface, it has a single method.

Error when retrieving from my xml document using xquery

I am trying to retrieve the authors from my XML documents but some of the authors have apostrophe's in their names so the results throw an error.
Input:
<dblp>
<book mdate="2002-01-03" key="books/aw/CeriF97">
<author>Stefano Ceri</author>
<author>Piero Fraternali</author>
<title>Designing Database Applications with Objects and Rules: The IDEA Methodology</title>
<publisher href="db/publishers/aw.html">Addison-Wesley</publisher>
<year>1997</year>
<isbn>0-201-40369-2</isbn>
</book>
</dblp>
Java/XQuery Code:
public ArrayList<String> getArrayListOfAuthors(){
String query = "for $x in fn:distinct-values(doc(\"" +xml_file_name+ "\")//author) " +
"order by $x "+
"return $x";
System.out.println("XQuery query:"+query);
ArrayList<String> myList = new ArrayList<String>();
try{
XQDataSource ds = new SaxonXQDataSource();
XQConnection conn = ds.getConnection();
XQExpression exp = conn.createExpression();
XQSequence seq = exp.executeQuery(query);
int i = 1;
while (seq.next()) {
i++;
//System.out.println(seq.getAtomicValue());
myList.add(seq.getAtomicValue());
}
//System.out.println("\n== Total number of authors is "+i+" ==");
seq.close();
} catch (XQException err) {
System.out.println("Failed as expected: " + err.getMessage());
}
return myList;
}
Error Message:
XPST0003 XQuery syntax error near #...e $y/author = 'Kieran O'Neill'#:
Unmatched quote in expression
Error on line 1 column 109

The error message strongly suggests that you are constructing a query by string concatenation, perhaps by processing the list of authors obtained from the query you have shown us. (Look for a query containing $y, which isn't the one in your sample).
Then change it so that instead of constructing a query using concatenation like this:
query = "//author[#name="' + name + "']"
you construct the query to contain a parameter:
query = "declare variable $name external; //author[#name=$name]"
and execute this supplying the value of $name as a run-time parameter. There are several benefits apart from avoiding the problem of names containing apostrophes: you avoid the security problems of injection attacks, and you get a performance benefit because you can compile the query once and use it repeatedly.

I cannot access an object property

I'm using Java and Jena API.
I have a class Marriage which have 3 Object Properties called "hasHusband", "Haswife" and "dateOfMarriage". The first two are associated with a class Person which has the datatypeproperties like hasFirstName, hasLastName, dateOfBirth....
I'd like to access the objects properties "Haswife" and "hasHusband" and then the wife's first name and the husband's first name.
Here is how that is represented in my rdf file:
(...)
<j.0:FAMmariage rdf:about=http://www.fam.com/FAM#BrunoCatherine>
<j.0:FAMaDateMariage>25/07/2011</j.0:FAMaDateMariage>
<j.0:FAMhasWife>
<rdf:Description rdf:about="http://www.fam.com/FAM#Catherine20/03/1982">
<j.0:FAMDateOfBirth>20/03/1980</j.0:FAMDateOfBirth>
<j.0:FAMHasName>Gomez</j.0:FAMHasName>
<j.0:FAMHasFirstName>Catherine</j.0:FAMHasFirstName>
</rdf:Description>
</j.0:FAMHasWife>
<j.0:FAMHusband>
<rdf:Description rdf:about="http://www.fam.com/FAM# Bruno15/06/1980 ">
<j.0:FAMaDateOfBirth>15/06/1980 </j.0:FAMDateOfBirth>
<j.0:FAMHasName>Jeandet </j.0:FAMHasName>
<j.0:FAMHasFirstName>Bruno</j.0:FAMHasFirstName>
</rdf:Description>
</j.0:FAMHusband>
</j.0:FAMmariage>
(...)
I tried like this but it still does not works:
StmtIterator iter = onto.model.listStatements();
while (iter.hasNext()) {
Statement stmt = iter.nextStatement();
Resource subject = stmt.getSubject();
Property predicate = stmt.getPredicate();
RDFNode object = stmt.getObject();
if (predicate.equals(onto.hasWife))
{
System.out.print(" " + object.toString() + " ");
}
}
Can you please tell me what's wrong?
Thanks
EDITED
More useful details:
(...)
person = model.createClass(uriBase+"person");
person.addSubClass(man);
person.addSubClass(woman);
marriage = model.createClass(uriBase+"marriage");
(...)
hasHusband = model.createObjectProperty(uriBase+"hasHusband");
hasHusband.setDomain(marriage);
hasHusband.setRange(man);
hasWife = model.createObjectProperty(uriBase+"hasWife");
hasWife.setDomain(marriage);
hasWife.setRange(woman);
hasFirstName = model.createDatatypeProperty(uriBase+"hasFirstName");
hasFirstName.setDomain(person);
hasFirstName.setRange(XSD.xstring);
(...)

does that work ?
ont.hasWife=model.createProperty("your namespace URI here","FAMhasWife");
StmtIterator iter = onto.model.listStatements(null,onto.hasWife,(RDFNode)null);
while (iter.hasNext()) {
Statement stmt = iter.nextStatement();
System.out.print(stmt.getObject().toString());
}

With the information you provide it's impossible to know what's the problem exactly. It is possible that you made a mistake in one of your URIs, either in your data or in your programme. Your question is inconcistent wrt naming terms.
May I suggest a few things? First, use the datatypes xsd:date or even xsd:dateTime for dates of marriage and dates of birth. Use the YYYY-MM-DD pattern for naming entities as this sorts them chronologically if sorted lexicographically. Use consistent naming convention, e.g., hasWife and hasHusband instead of hasWife and husband. "Marriage" has two "r" in English. Use a meaningful prefix when presenting your data to people. Better, use a human friendly syntax such as Turtle.
fam:BrunoCatherine a fam:Marriage;
fam:dateOfMarriage "2011-07-25"^^xsd:date;
fam:hasWife fam:Catherine1982-03-20;
fam:hasHusband fam:Bruno1980-06-15 .
fam:Catherine1982-03-20 fam:dateOfBirth "1982-03-20"^^xsd:date;
fam:hasLastName "Gomez";
fam:hasFirstName "Catherine" .
fam:Bruno1980-06-15 fam:dateOfBirth "1980-06-15"^^xsd:date;
fam:hasLastName "Jeandet";
fam:hasFirstName "Bruno" .
Doesn't it look nicer?