I need some help with a project I am planing to do. At this stage I am trying to learn using NoSQL Databases in Java.
I've got a few nested documents looking like this:
MongoDB nesting structure
Like you can see on the image, my inner attributes are "model" and "construction".
Now I need to iterate through all the documents in my collection, whose keynames are unknown, because they are generated in runtime, when a user enters some information.
At the end I need to list them in a TreeView, keeping the structure they have already in the database.
What I've tried is getting keySets from documents, but I cannot pass the second layer of the structure. I am able to print the whole Object in Json format, but I cannot access the specific attributes like "model" or "construction".
MongoCollection collection= mongoDatabase.getCollection("test");
MongoCursor<Document> cursor = collection.find().iterator();
for(String keys: document.keySet()) {
Document vehicles = (Document) document.getString(keys);
//System.out.println(keys);
//System.out.println(document.get(keys));
}
/Document cars = (Document) vehicle.get("cars");
Document types = (Document) cars.get("coupes");
Document brands = (Document) types.get("Ford");
Document model = (Document) brands.get("Mustang GT");
Here I tried to get some properties, by hardcoding the keynames of the documents, but I can't seem to get any value either. It keeps telling me that it could not read from vehicle, because it is null.
The most tutorials and posts in forums, somehow does not work for me. I don't know if they have any other version of MongoDB Driver. Mine is: mongodb driver 3.12.7. if this helps you in any way.
I am trying to get this working for days now and it is driving me crazy.
I hope there is anyone out there who is able to help me with this problem.
Here is a way you can try using the Document class's methods. You use the Document#getEmbedded method to navigate the embedded (or sub-document) document's path.
try (MongoCursor<Document> cursor = collection.find().iterator()) {
while (cursor.hasNext()) {
// Get a document
Document doc = (Document) cursor.next();
// Get the sub-document with the known key path "vehicles.cars.coupes"
Document coupes = doc.getEmbedded(
Arrays.asList("vehicles", "cars", "coupes"),
Document.class);
// For each of the sub-documents within the "coupes" get the
// dynamic keys and their values.
for (Map.Entry<String, Object> coupe : coupes.entrySet()) {
System.out.println(coupe.getKey()); // e.g., Mercedes
// The dynamic sub-document for the dynamic key (e.g., Mercedes):
// {"S-Class": {"model": "S-Class", "construction": "2011"}}
Document coupeSubDoc = (Document) coupe.getValue();
// Get the coupeSubDoc's keys and values
coupeSubDoc.keySet().forEach(k -> {
System.out.println("\t" + k); // e.g., S-Class
System.out.println("\t\t" + "model" + " : " +
coupeSubDoc.getEmbedded(Arrays.asList(k, "model"), String.class));
System.out.println("\t\t" + "construction" + " : " +
coupeSubDoc.getEmbedded(Arrays.asList(k, "construction"), String.class));
});
}
}
}
The above code prints to the console as:
Mercedes
S-Class
model : S-Class
construction : 2011
Ford
Mustang
model : Mustang GT
construction : 2015
I think it's not the complete answer to his question.
Here he says:
Now I need to iterate through all the documents in my collection, whose keynames are unknown, because they are generated in runtime, when a user enters some information.
Your answer #prasad_ just refers to his case with vehicles, cars and so on. He needs a way to handle unknown key/value pairs i guess. For example, in this case he only knows the keys:vehicle,cars,coupe,Mercedes/Ford and their subkeys. If another user inserts some new key/value paairs in the collection he will have problems because he can't navigate trough the new document without to have a look into the database.
I'm also interested in the solution because I never nested my key/value pairs and cant see the advantage of it. Am I wrong or does it make the programming more difficult?
Related
basically below is the java coding part, which the result will then be populated to a .csv file. However, I dont seem get the ordering part right (last line in below snippet).
Map<String, String> map = new HashMap<String, String>();
map.put("path", "/etc/crx-db/form-data/career");
map.put("type", "nt:unstructured");
map.put("p.limit", "-1");
map.put("daterange.property", "created");
map.put("daterange.lowerBound", from);
map.put("daterange.lowerOperation", ">=");
map.put("daterange.upperOperation", "<=");
map.put("daterange.upperBound", to);
map.put("orderby", "created"); //<--here
Providing that in crx repositry (/etc/crx-db/form-data/career), I have nodes: data1, data2, data3...
Then for each node, there is one property - Name: created | Type: Date | Value: 2014-01-28T23:21:15.029+08:00 (eg)
However my result in .csv is incorrect like (row 1 to 5):
2014-01-28T23:21:15.029+08:00
2014-01-28T23:48:12.219+08:00
2014-02-10T18:44:38.914+08:00 <-- unsorted
2014-02-10T18:43:32.426+08:00 <-- unsorted
2014-02-10T18:46:53.319+08:00
Pretty sure my code wasn't running. Any idea on how can I tweak my java code to make the sorting happen? As in returning sorted data1, data2, data3... based on the property created. Thanks.
You were almost there. It can be done as follows.
map.put("orderby", "#created");
map.put("orderby.sort", "desc"); // in case you want it descending
In case you need to check property within a child node, you can provide the relative path to that for the orderby value. For eg., if you are searching for dam:Asset and want to order them based on the jcr:lastModified property of its metadata, then your query would be something similar to this.
map.put("path", "/content/dam/geometrixx");
map.put("type", "dam:Asset");
map.put("orderby","#jcr:content/metadata/jcr:lastModified");
For further learning refer this
While searching using java api in elaticsearch, I would like to retrieve only one column.
Currently when I query using the Java API it returns the whole record like this: [{_id=123-456-7890, name=Wonder Woman, gender=FEMALE}, {_id=777-990-7890, name=Cat Woman, gender=FEMALE}]
The record above correctly matches the search condition shown in th . As shown in the code below:
List<Map<String, Object>> result = new ArrayList<Map<String, Object>>();
SearchRequestBuilder srb = client.prepareSearch("heros")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH);
MatchQueryBuilder mqb;
mqb = QueryBuilders.matchQuery("name", "Woman");
srb.setQuery(mqb);
SearchResponse response = srb.execute().actionGet();
long totalHitCount = response.getHits().getTotalHits();
System.out.println(response.getHits().getTotalHits());
for (SearchHit hit : response.getHits()) {
result.add(hit.getSource());
}
System.out.println(result);
I want only one column to be returned. If I search for name I just want the full names back in a list: "Wonder Woman", "Cat Woman" only not the whole json record for each of them. If you think I need to iterate over the result list of maps in java please propose an example of how to do that in this case.
You can specify the fields to be returned from a search, per documentation. This can be set via SearchRequestBuilder.addFields(String... fields), ie:
SearchRequestBuilder srb = client.prepareSearch("heros")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.addFields("name");
Better combine both:
use .addFields("name") to tell ES that it needs to return only this
column
use hit.field("name").getValue().toString() to get the result
It is important to use .addFields when you don't need the whole document, but the specific field/s as it will lower the overhead and the network traffic
I figured it out.
List<String> valuesList= new ArrayList<String>();
for (SearchHit hit : response.getHits()) {
result.add(hit.getSource());
valuesList.add(hit.getSource().get("name").toString());
}
The other solutions didn't work for me, hit.getSource() was returning null. Maybe they are deprecated? Not sure. But here was my solution, which FYI can speed things up considerably if you are only getting one field and you are getting lots of results.
Use addFields(Strings) on your SearchRequestBuilder as mentioned, but then when you are getting the values you need to use:
hit.getFields().get( fieldName ).getValue()
or
hit.getFields().get( fieldName ).getValues()
to get a single value or a list of values depending on the field.
Maybe I'm really missing something.
I have indexed a bunch of key/value pairs in Lucene (v4.1 if it matters). Say I have
key1=value1 and key2=value2, e.g. as read from a properties file.
They get indexed both as specific fields and into a catchall "ALL" field, e.g.
new Field("key1", "value1", aFieldTypeMimickingKeywords);
new Field("key2", "value2", aFieldTypeMimickingKeywords);
new Field("ALL", "key1=value1", aFieldTypeMimickingKeywords);
new Field("ALL", "key2=value2", aFieldTypeMimickingKeywords);
// then get added to the Document of course...
I can then do a wildcard search, using
new WildcardQuery(new Term("ALL", "*alue1"));
and it will find the hit.
But, it would be nice to get more info, like "what was complete value (e.g. "key1=value1") that goes with that hit?".
The best I can figure out it to get the Document, then get the list of IndexableFields, then loop over all of them and see if the field.stringValue().contains("alue1"). (I can look at the data structures in the debugger and all the info is there)
This seems completely insane cause isn't that what Lucene just did? Shouldn't the Hit information return some of the Fields?
Is Lucene missing what seems like "obvious" functionality? Google and starting at the APIs hasn't revealed anything straightforward, but I feel like I must be searching on the wrong stuff.
You might want to try with IndexSearcher.explain() method. Once you get the ID of the matching document, prepare a query for each field (using the same search keywords) and invoke Explanation.isMatch() for each query: the ones that yield true will give you the matched field. Example:
for (String field: fields){
Query query = new WildcardQuery(new Term(field, "*alue1"));
Explanation ex = searcher.explain(query, docID);
if (ex.isMatch()){
//Your query matched field
}
}
I've gone through the related questions on this site but haven't found a relevant solution.
When querying my Solr4 index using an HTTP request of the form
&facet=true&facet.field=country
The response contains all the different countries along with counts per country.
How can I get this information using SolrJ?
I have tried the following but it only returns total counts across all countries, not per country:
solrQuery.setFacet(true);
solrQuery.addFacetField("country");
The following does seem to work, but I do not want to have to explicitly set all the groupings beforehand:
solrQuery.addFacetQuery("country:usa");
solrQuery.addFacetQuery("country:canada");
Secondly, I'm not sure how to extract the facet data from the QueryResponse object.
So two questions:
1) Using SolrJ how can I facet on a field and return the groupings without explicitly specifying the groups?
2) Using SolrJ how can I extract the facet data from the QueryResponse object?
Thanks.
Update:
I also tried something similar to Sergey's response (below).
List<FacetField> ffList = resp.getFacetFields();
log.info("size of ffList:" + ffList.size());
for(FacetField ff : ffList){
String ffname = ff.getName();
int ffcount = ff.getValueCount();
log.info("ffname:" + ffname + "|ffcount:" + ffcount);
}
The above code shows ffList with size=1 and the loop goes through 1 iteration. In the output ffname="country" and ffcount is the total number of rows that match the original query.
There is no per-country breakdown here.
I should mention that on the same solrQuery object I am also calling addField and addFilterQuery. Not sure if this impacts faceting:
solrQuery.addField("user-name");
solrQuery.addField("user-bio");
solrQuery.addField("country");
solrQuery.addFilterQuery("user-bio:" + "(Apple OR Google OR Facebook)");
Update 2:
I think I got it, again based on what Sergey said below. I extracted the List object using FacetField.getValues().
List<FacetField> fflist = resp.getFacetFields();
for(FacetField ff : fflist){
String ffname = ff.getName();
int ffcount = ff.getValueCount();
List<Count> counts = ff.getValues();
for(Count c : counts){
String facetLabel = c.getName();
long facetCount = c.getCount();
}
}
In the above code the label variable matches each facet group and count is the corresponding count for that grouping.
Actually you need only to set facet field and facet will be activated (check SolrJ source code):
solrQuery.addFacetField("country");
Where did you look for facet information? It must be in QueryResponse.getFacetFields (getValues.getCount)
In the solr Response you should use QueryResponse.getFacetFields() to get List of FacetFields among which figure "country". so "country" is idenditfied by QueryResponse.getFacetFields().get(0)
you iterate then over it to get List of Count objects using
QueryResponse.getFacetFields().get(0).getValues().get(i)
and get value name of facet using QueryResponse.getFacetFields().get(0).getValues().get(i).getName()
and the corresponding weight using
QueryResponse.getFacetFields().get(0).getValues().get(i).getCount()
I have this entity - I'm trying to determine the type of its properties - in Google App Engine's internal data-types PREFERRED (as opposed to Java data types).
The below code is obviously simplified. In reality I do not know the entity's properties or anything else about it.
final DatastoreService dss = DatastoreServiceFactory.getDatastoreService();
final Query query = new Query("Person");
final PreparedQuery pq = dss.prepare(query);
for (Entity entity : pq.asIterable())
{
final Object property = entity.getProperty("some_property");
// Here I want to determine which data type 'property' represents - GAE-wise.
}
In App Engine's Java code I've found some hints:
DataTypeTranslator
DataTypeTranslator.typeMap (internal private member)
Property.Meaning.GD_PHONENUMBER
I'm unable to link those together into what I need - some sort of reflection.
I wish I was able to do something like this:
entity.getPropertyType("some_property");
Does anyone know better?
DataTypeTranslator source code here
Edit #1: <<
INGORE this one. It's me who put these postfixes (I was confused by the doc).
Here's more important info I've found.
I'm getting it in Eclipse' tool-tip mini-window when I point over an entity (one which I just fetched from the Datastore).
The Datastore seems to send it (this payload) as raw text which is nice, maybe I'll have to parse it (but, how do I get it from code LOL).
Pay attention to the types in here, it's written plain simple.
Here it is:
<Entity [Bird(9)]:
Int64Type:44rmna4kc2g23i9brlupps74ir#Int64Type = 1234567890
String:igt7qvk9p89nc3gjqn9s3jq69c = 7tns1l48vpttq5ff47i3jlq3f9
PhoneNumber:auih50aecl574ud23v9h4rfvt1#PhoneNumberType = 03-6491234
Date:k1qstkn9np0mpb6fp41cj6i3am = Wed Jul 20 23:03:13 UTC 2011
>
For example, property named String:igt7qvk9p89nc3gjqn9s3jq69c has the value of 7tns1l48vpttq5ff47i3jlq3f9 and it doesn't tell its type. Also property Date:k1qstkn9np0mpb6fp41cj6i3am.
Property named Int64Type:44rmna4kc2g23i9brlupps74ir has the value of "1234567890" and here it strictly mentions that the data type is of "Int64Type".
I'm searching for it too.
It's a bit of a hack, but at least my output includes the type (without needing a secret decoder ring). But my code is slightly different:
Query allusersentityquery = new Query();
allusersentityquery.setAncestor(userKey);
for (final Entity entity : datastore.prepare(allusersentityquery).asIterable()) {
Map<String, Object> properties = entity.getProperties();
String[] propertyNames = properties.keySet().toArray(
new String[properties.size()]);
for(final String propertyName : propertyNames) {
// propertyNames string contains
// "com.google.appengine.api.datastore.PostalAddress" if it is a Postal Address
}
}
There seems to be no documents about determining the Property Types here.