I have some data like this:
{id: 1, text: "This is a sentence about dogs", indices: ["sentence", "dogs"]}
{id: 2, text: "This sentence is about cats and dogs", indices: ["sentence", "cats", "dogs"]}
Where I have manually extracted key terms from the text and stored them as indices. I want to be able to do a search and order the results with the most matching indices. So for this example, I would like to be able to pass "cats" and "dogs" and get both objects returned, but id=2 should be first with score=2.
I first tried to use the DBCollection.group function
{public DBObject group(DBObject key,
DBObject cond,
DBObject initial,
String reduce,
String finalize)}
But I don't see a way to send parameters. I tried:
key: {id: true},
cond: {"indices" $in ['cats', 'dogs']},
initial: {score: 0}
reduce: function(doc, out){ out.score++; }
but obviously this will just return a count of 1 for each of the 2 objects.
I realised that I could send the keyword parameters as part of the initial config of the reduced object.
final List<String> targetTerms = Arrays.asList("dogs", "cats");
final Datastore ds = ….
final DBCollection coll = ds.getCollection(Example.class);
BasicDBObject key = new BasicDBObject("_id", true);
BasicDBObject cond = new BasicDBObject();
cond.append("indices", new BasicDBObject("$in", targetTerms));
BasicDBObject initial = new BasicDBObject();
initial.append("score", 0);
initial.append("targetTerms", targetTerms);
String reduce = "function (obj, prev) { " +
" for (i in prev.targetTerms) {" +
" targetTerm = prev.targetTerms[i];"+
" for (j in obj.indices) {" +
" var index = obj.indices[j];"+
" if (targetTerm === index) prev.score++;" +
" }" +
" }" +
"}";
String fn = null;
final BasicDBList group = (BasicDBList) coll.group(key, cond, initial, reduce, fn);
I get results like this:
{ "_id" : { "$oid" : "4dcfe16c05a063bb07ccbb7b"} , "score" : 1.0 , "targetTerms" : [ "virtual" , "library"]}
{ "_id" : { "$oid" : "4dcfe17d05a063bb07ccbb83"} , "score" : 2.0 , "targetTerms" : [ "virtual" , "library"]}
This got me the score values that I wanted, and I am able to narrow down the entries to be processed with more specific conditional rules.
So I have a few questions:
Is this a good way to send "parameters" to the group action's reduce function?
Is there a way to sort (and perhaps limit) the output inside mongodb before returning to the client?
Will this break on sharded Mongodb instances?
Related
I am trying to query a mongoDB collection sorting by an array value(in addition to other fields) and the results need to be paginated also. The sorting is not working for the array value, but it works for other fields. Here is the code:
sortBy = "programInstance.title.descriptions[0].value";
PageRequest pageRequest = = PageRequest.of(filter.getPageNumber(), filter.getPageSize(),
new Sort("DESC".equalsIgnoreCase(filter.getSortOrder()) ? Sort.Direction.DESC : Sort.Direction.ASC, sortBy));
Page<Offer> st = new PageImpl<>(mongoTemplate.find(query.with(pageRequest),Offer.class), pageRequest, pageCount);
Any help is very much appreciated..
It worked successfully with this expression
sortBy = "programInstance.title.descriptions.0.value";
//One or two records from your collection
const data = [{
name: 'abc',
place: 'US',
zone: 'Pacific'
}, {
name: 'xyz',
place: 'PK',
zone: 'Asia'
}, ]
const fields = Object.keys(data[0]);
console.log("Fields in collection: ",fields);
const sortBy = fields[0];
console.log("Sort By: ", sortBy);
//const sort = new Sort("DESC".equalsIgnoreCase(filter.getSortOrder()) ? //Sort.Direction.DESC : Sort.Direction.ASC, sortBy);
//PageRequest pageRequest = = PageRequest.of(filter.getPageNumber(), //filter.getPageSize(), sort);
//Page < Offer > st = new PageImpl < > //(mongoTemplate.find(query.with(pageRequest), Offer.class), pageRequest, //pageCount);
The sort parameter is a value (string type), on which the sort is performed, you can't pass a string which needs further evaluation (in your case) to get the value. Instead directly save your value into your sortBy variable and then pass sortBy to your query.
I am updating mongodb item in collection using the below command.
Now i want to take the set and filter value in java.
db.practice.update({"name" : "testmysqlupdate"},{$set:{"name" : "5"}})
I am able to take the value which need to be update i.e. name : 5, but not the conditional value i.e. "name" : "testmysqlupdate"
static ChangeStreamDocument<BasicDBObject> dbOb;
static MongoCursor<ChangeStreamDocument<BasicDBObject>> cursor;
cursor = collection.watch().fullDocument(FullDocument.UPDATE_LOOKUP).iterator();
System.out.println("value of update stream us " + dbOb.getUpdateDescription().getUpdatedFields().toString());
System.out.println("Id Value is " + dbOb.getDocumentKey().toString());
document = dbOb.getUpdateDescription().getUpdatedFields().toString();
BasicDBObject searchQuery = new BasicDBObject();
searchQuery.put("name", "testmysqlupdate");
BasicDBObject newDocument = new BasicDBObject();
newDocument.append("$set", new BasicDBObject().append("name", "5"));
collection.update(searchQuery, newDocument);
Try this to update a field in MongoDB.
I have tried to fetch data for the particular column value in the mongo document but its displaying whole data.
Following is the mongo document:
{
"_id" : ObjectId("59db2321811a592384865711"),
"User_ID" : "demo",
"Project_ID" : "demo-1",
"Project_Information" : {
"Project_Description" : "Sample",
"Primary_Building_Type" : "Office",
"State" : "AR",
"Analysis_Type" : "1",
"Project_Billing_Number" : "WY",
"Country" : "USA",
"Climate_Zone" : "3A",
"Zip_Code" : "71611"
"City" : "WA",
"Units" : "IP"
}
}
I want to fetch the following output:
[
{
"User_ID": "demo",
"Project_Description": "Sample"
}]
I have tried using dot: Project_Information.Project_Description.The code is as below:
public Object[] addDemo1(String User_ID) throws Exception {
DB db = ConnectToDB.getConnection();
Properties prop = new Properties();
InputStream input = null;
input = GetProjectStatus.class.getClassLoader().getResourceAsStream("config.properties");
prop.load(input);
String col = prop.getProperty("COLLECTION_PI");
System.out.println("data is.." + col);
DBCollection collection = db.getCollection(col);
BasicDBObject obj = new BasicDBObject();
BasicDBObject fields = new BasicDBObject();
BasicDBObject fields2 = new BasicDBObject();
List<DBObject> obj1 = null;
if (User_ID != null && !User_ID.equals("") && User_ID.length() > 0) {
obj.put("User_ID", User_ID);
fields.put("_id", 0);
fields.put("User_ID", 1);
fields.put("Project_ID", 1);
fields.append("Project_Information.Project_Description", "Project_Description");
BasicDBObject fields1 = new BasicDBObject();
fields1.put("User_ID", User_ID);
}
DBCursor cursor = collection.find(obj, fields);
System.out.println("count is:" + cursor.count());
obj1 = cursor.toArray();
System.out.println("" + obj1);
cursor.close();
db.getMongo().close();
return obj1.toArray();
}
But it displays the whole structure of Project_Information.
Please specify how to achieve this. Thanks for help.
Using a 2.x MongoDB Java Driver
Here's an example using the MongoDB 2.x Java driver:
DBCollection collection = mongoClient.getDB("stackoverflow").getCollection("demo");
BasicDBObject filter = new BasicDBObject();
BasicDBObject projection = new BasicDBObject();
// project on "Project_Information.Project_Description"
projection.put("Project_Information.Project_Description", 1);
DBCursor documents = collection.find(filter, projection);
for (DBObject document : documents) {
// the response contains a sub document under the key: "Project_Information"
DBObject projectInformation = (DBObject) document.get("Project_Information");
// the "Project_Description" is in this sub document
String projectDescription = (String) projectInformation.get("Project_Description");
// prints "Sample"
System.out.println(projectDescription);
// to return this single String value in an Object[] (as implied by your OP) just do create the Object[] like this and then return it ...
Object[] r = new Object[] {projectDescription};
// prints the entire projected document e.g.
// { "_id" : { "$oid" : "59db2321811a592384865711" }, "Project_Information" : { "Project_Description" : "Sample" } }
System.out.println(document.toString());
}
Using a 3.x MongoDB Java Driver
Here's an example using the MongoDB 3.x Java driver:
// this finds all documents in a given collection (note: no parameter supplied to the find() call)
// and for each document it projects on Project_Information.Project_Description
FindIterable<Document> documents =
mongoClient.getDatabase("...").getCollection("...")
.find()
// for each attrbute you want to project you must include its dot notation path and the value 1 ...
// this is the equivalent of specifying {'Project_Information.Project_Description': 1} in the MongoDB shell
.projection(new Document("Project_Information.Project_Description", 1));
for (Document document : documents) {
// the response contains a sub document under the key: "Project_Information"
Document projectInformation = (Document) document.get("Project_Information");
// the "Project_Description" is in this sub document
String projectDescription = projectInformation.getString("Project_Description");
// prints "Sample"
System.out.println(projectDescription);
// to return this single String value in an Object[] (as implied by your OP) just do create the Object[] like this and then return it ...
Object[] r = new Object[] {projectDescription};
// prints the entire projected document e.g. { "_id" : { "$oid" : "59db2321811a592384865711" }, "Project_Information" : { "Project_Description" : "Sample" } }
System.out.println(document.toJson());
}
Java libraries won't let you directly access using dot.
They have build in getter and setter methods.
You have not mentioned which package you are using.
Here's the query that you need:
db.mycol.find({},{User_ID:1,"Project_Information.Project_Description":1})
It will give:
{ "_id" : ObjectId("59db2321811a592384865711"),
"User_ID" : "demo",
"Project_Information" : { "Project_Description" : "Sample" }
}
You will have to convert the query in whatever format your package accepts.
Here's a tutorial:
https://www.mongodb.com/blog/post/getting-started-with-mongodb-and-java-part-i
I am using mongodb with Java 3.0 driver. I have a scenario where I have to perform logical and i.e, $and on my queries. For example, I have two documents already created and I am trying to do something like this:
iterable = mongoDatabase.getCollection("restaurants").find(
new Document("$and", asList(abc,
updatedDocumentTypeOne)));
where abc is one document and updatedDocumentTypeOne is another document. I found this in mongodb manual but I am getting error as first create asList Method.
Or how to replicate the following in Java:
db.inventory.find( {
$and : [
{ $or : [ { price : 0.99 }, { price : 1.99 } ] },
{ $or : [ { sale : true }, { qty : { $lt : 20 } } ] }
]
} )`
You can also try the code below that adds a filter for query replication in Java:
// Where db is the object reference of "inventory" collection's database
MongoCollection<Document> inventory = db.getCollection("inventory");
//List for result return
List<Document> result = new ArrayList<Document>();
//Query replication in Java and return result into the list
inventory.find(Filters.and(
Filters.or(Filters.eq("price", 0.99),Filters.eq("price", "1.99")),
Filters.or(Filters.eq("sale", true),Filters.lt("qty", 20))
)).into(result);
Change from asList() to Arrays.asList()
Instead of writing Arrays.asList(), you have specified as asList(). So compiler is searching for the method asList(), which is NOT available.
Check the below code :
iterable = mongoDatabase.getCollection("restaurants").find(
new Document("$and", Arrays.asList(abc,
updatedDocumentTypeOne)));
For your above query, You can code as below :
/* First OR condition */
Document price1 = new BasicDBObject("price",0.99);
Document price2 = new BasicDBObject("price",1.99);
BasicDBList or_first = new BasicDBList();
or_first.add(price1);
or_first.add(price2);
DBObject query = new BasicDBObject("$or", or_first);
/* Second OR condition */
boolean val = true;
Document sale = new BasicDBObject("sale",val);
Document qty = new BasicDBObject("qty", new BasicDocument("$lt",20));
BasicDBList or_second = new BasicDBList();
or_second.add(sale);
or_second.add(qty);
DBObject query = new BasicDBObject("$or", or_second);
/* AND condition logic */
BasicDBList and_op = new BasicDBList();
and_op.add(or_first);
and_op.add(or_second);
iterable = mongoDatabase.getCollection("restaurants").find( new Document("$and", and_op ));
I'm testing against a database with one document in it. With the following code I get the following result from the database
BasicDBObject keys = new BasicDBObject();
keys.put("symbol", 1 );
keys.put("price", 1 );
keys.put("Exchange", "SH");
keys.put("NumShares", new BasicDBObject("$gte", 0) );
BasicDBObject empty = new BasicDBObject();
DBCursor cursor_02 = coll_tmp.find( empty, keys );
System.out.println( "<2> " + cursor_02.count());
while(cursor_02.hasNext()) {
System.out.println( "<2> " + cursor_02.next());
}
This will print out the following data
<2> { <id & oid> , "symbol" : "ESLR" , "Exchange" : "SH"}
The document also has a "NumShares" field set to a value of 2.34508872. Wanting to try out the $gt, $gte, $lt, $lte, and $ne operators, I add in a line of code after the last 'keys.put()' line.
keys.put("Price", new BasicDBObject("$gte", 0) );
This code should continue to select the lone document, as it's NumShares field is indeed greater than or equal to zero. Instead I get the following error.
Can't canonicalize query: BadValue Unsupported projection option: NumShares: { $gte: 0 }
The goal ultimately is to have a MongoDB equivalent of the SQL query "SELECT price, symbol, exhange FROM stock WHERE NumShares >= 0, etc..."
I realize that mkyong doesn't cover this particular query structure in his tutorial, so I'm asking you all. Thanks
You want to put your criteria in the query, not in the list of keys you want to return:
BasicDBObject keys = new BasicDBObject(); // will specify the returned fields (SELECT)
keys.put("symbol", 1 );
keys.put("price", 1 );
keys.put("Exchange", 1);
BasicDBObject query = new BasicDBObject(); // will select the documents you want (WHERE)
query.put("numShares", new BasicDBObject("$gte", 0) );
DBCursor cursor_02 = coll_tmp.find( query, keys );
System.out.println( "<2> " + cursor_02.count());
while(cursor_02.hasNext()) {
System.out.println( "<2> " + cursor_02.next());
}
You need to put your result-restricting query arguments to your first Object of the query. The second Object is a Projection of the result. It can only specify the tags to be displayed (as true/1 or false/0) there are no other restrictions like "$lte" allowed. Putting it to the query object like in the following should work:
BasicDBObject keys = new BasicDBObject();
keys.put("symbol", 1 );
keys.put("price", 1 );
keys.put("Exchange", 1);
BasicDBObject search = new BasicDBObject();
search.put("Exchange", "SH");
search.put("numShares", new BasicDBObject("$gte", 0) );
DBCursor cursor_02 = coll_tmp.find(search, keys);
Also MongoDBs keys of documents are case-sensitive. So if your key is "NumShares" you cannot find it by looking for "numShares".