We Keep Coding

MongoDB get number of matches by $in command

Definition
I'm creating searching application and mongo db is used to store searching information. This is example dataset of collection "Resource".
{
_id:"5b3b84e02360a26f9a9ae96e",
name:"Advanced Java",
keywords:[
"java", "thread", "state", "public", "void"
]
},
{
_id:"5b3b84e02360a26f9a9ae96f",
name:"Java In Simple",
keywords:[
"java", "runnable", "thread", "sleep", "array"
]
}
This contains name of books and most frequent words (in keywords array) of each. I'm using spring framework with mongo template. If I run below code,
MongoOperations mongoOperations = new MongoTemplate(new MongoClient("127.0.0.1", 27017), "ResourceDB");
Query query = new Query(where("keywords").in("java", "thread", "sleep"));
List<Resource> resources = mongoOperations.find(query, Resource.class);
It results both "Advanced Java" and "Java In Simple" and its ok.
Problem
But in my case, I need them in order. Because "Java In Simple" match 3 words and "Advanced Java" matches only 2 words. So possibility of most relevant book should be "Java In Simple" and it should be in first.
Expecting Order
Java In Simple
Advanced Java
Is it possible to get result in matching order. Or is there any way to get number of matches for each item. For example If is search for ("java", "thread", "sleep"), I'm expecting output like below.
Advanced Java - 2 matches
Java in Simple - 3 matches
Any help appreciated.

$in doesn't match 3 or 2 items. It stops after first match. You need to use aggregation pipeline to calculate intersection of keywords and the array from the query and order by size of the result:
db.collection.aggregate([
{ $addFields: {
matchedTags: { $size: {
$setIntersection: [ "$keywords", [ "java", "thread", "sleep" ] ]
} }
} },
{ $match: { matchedTags: { $gt: 0 } } },
{ $sort: { matchedTags: -1 } }
])

This is for someone who looking to run #Alex Blex's query in java. It looks like mongo template does not have implementation for intersection. So I have done it using mongoDB java client.
List<String> keywords = Arrays.asList("java", "thread", "sleep");
BasicDBList intersectionList = new BasicDBList();
intersectionList.add("$keywords");
intersectionList.add(keywords);
AggregateIterable<Document> aggregate = new MongoClient("127.0.0.1", 27017).getDatabase("ResourceDB").getCollection("Resource").aggregate(
Arrays.asList(
new BasicDBObject("$addFields",
new BasicDBObject("matchedTags",
new BasicDBObject("$size",
new BasicDBObject("$setIntersection", intersectionList)))),
new BasicDBObject("$match",
new BasicDBObject("matchedTags",
new BasicDBObject("$gt", 0))),
new BasicDBObject("$sort",
new BasicDBObject("matchedTags", -1))
)
);
MongoCursor<Document> iterator = aggregate.iterator();
while (iterator.hasNext()){
Document document = iterator.next();
System.out.println(document.get("name")+" - "+document.get("matchedTags"));
}

Java API to insert a document with array of elements in mongodb collection

I want to insert the following JSON into the Mongodb collection using java API . Here bookmarks is an arrayList of bookmarks POJO .
{
"_id": 5,
"email": "xxx#gmail.com",
"bookmarks": [
{
"name": "chipotle",
"category": "restaurant",
"stats": "203 likes",
"tried": true
},
{
"name": "olivegarden",
"category": "restaurant",
"stats": "203 likes",
"tried": true
}
]
}
I used the following API . but it doesn't seem to work
BasicDBObject document = new BasicDBObject();
document.append("email", userList.get(i).getEmail());
document.append("bookmarks", userList.get(i).getBookmarksList() ) ;
WriteResult result = collection.insert(document);
This is the error I got when I ran the unit test .
java.lang.IllegalArgumentException: can't serialize class com.xxx.pojo.Bookmark
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:272)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:173)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:119)
at com.mongodb.DefaultDBEncoder.writeObject(DefaultDBEncoder.java:27)
at com.mongodb.OutMessage.putObject(OutMessage.java:289)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:239)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:204)
at com.mongodb.DBCollection.insert(DBCollection.java:76)
at com.mongodb.DBCollection.insert(DBCollection.java:60)
at com.mongodb.DBCollection.insert(DBCollection.java:105)
Even after making the Bookmark POJO serializable I got the same error again .So I guess am using the Java API for insert in a wrong way . How to map the POJO directly into the mongodb element ?

As the error suggests can't serialize class com.xxx.pojo.Bookmark , which means that the List containing Bookmark.class can't be directly inserted in BasicDBObject document.
You need to use : BasicDBList as follows:
BasicDBList bookmark_list = new BasicDBList();
List<Bookmark> bmk_list = userList.get(i).getBookmarksList();
for(int i=0;i<bmk_list.size();i++)
{
String name = bmk_list.get(i).getName();
String category = bmk_list.get(i).getCategory();
String stats = bmk_list.get(i).getStats();
boolean tried = bmk_list.get(i).getTried();
DBObject db_obj = new BasicDBObject();
db_obj.put("name",name);
db_obj.put("category",category);
db_obj.put("stats",stats);
db_obj.put("tried",tried);
bookmark_list.add(db_obj);
}
Now add this bookmark_list in your document as follows:
BasicDBObject document = new BasicDBObject();
document.append("email", userList.get(i).getEmail());
document.append("bookmarks", bookmark_list ) ;
WriteResult result = collection.insert(document);

Case insensitive sorting in MongoDB

How can I sort a MongoDB collection by a given field, case-insensitively? By default, I get A-Z before a-z.

Update:
As of now mongodb have case insensitive indexes:
Users.find({})
.collation({locale: "en" })
.sort({name: 1})
.exec()
.then(...)
shell:
db.getCollection('users')
.find({})
.collation({'locale':'en'})
.sort({'firstName':1})
Update: This answer is out of date, 3.4 will have case insensitive indexes. Look to the JIRA for more information https://jira.mongodb.org/browse/SERVER-90
Unfortunately MongoDB does not yet have case insensitive indexes: https://jira.mongodb.org/browse/SERVER-90 and the task has been pushed back.
This means the only way to sort case insensitive currently is to actually create a specific "lower cased" field, copying the value (lower cased of course) of the sort field in question and sorting on that instead.

Sorting does work like that in MongoDB but you can do this on the fly with aggregate:
Take the following data:
{ "field" : "BBB" }
{ "field" : "aaa" }
{ "field" : "AAA" }
So with the following statement:
db.collection.aggregate([
{ "$project": {
"field": 1,
"insensitive": { "$toLower": "$field" }
}},
{ "$sort": { "insensitive": 1 } }
])
Would produce results like:
{
"field" : "aaa",
"insensitive" : "aaa"
},
{
"field" : "AAA",
"insensitive" : "aaa"
},
{
"field" : "BBB",
"insensitive" : "bbb"
}
The actual order of insertion would be maintained for any values resulting in the same key when converted.

This has been an issue for quite a long time on MongoDB JIRA, but it is solved now. Take a look at this release notes for detailed documentation. You should use collation.
User.find()
.collation({locale: "en" }) //or whatever collation you want
.sort({name:1})
.exec(function(err, users) {
// use your case insensitive sorted results
});

Adding the code .collation({'locale':'en'}) helped to solve my issue.

As of now (mongodb 4), you can do the following:
mongo shell:
db.getCollection('users')
.find({})
.collation({'locale':'en'})
.sort({'firstName':1});
mongoose:
Users.find({})
.collation({locale: "en" })
.sort({name: 1})
.exec()
.then(...)
Here are supported languages and locales by mongodb.

In Mongoose:-
Customer.find()
.collation({locale: "en" })
.sort({comapany: 1})

Here it is in Java. I mixed no-args and first key-val variants of BasicDBObject just for variety
DBCollection coll = db.getCollection("foo");
List<DBObject> pipe = new ArrayList<DBObject>();
DBObject prjflds = new BasicDBObject();
prjflds.put("field", 1);
prjflds.put("insensitive", new BasicDBObject("$toLower", "$field"));
DBObject project = new BasicDBObject();
project.put("$project", prjflds);
pipe.add(project);
DBObject sort = new BasicDBObject();
sort.put("$sort", new BasicDBObject("insensitive", 1));
pipe.add(sort);
AggregationOutput agg = coll.aggregate(pipe);
for (DBObject result : agg.results()) {
System.out.println(result);
}

If you want to sort and return all data in a document, you can add document: "$$ROOT"
db.collection.aggregate([
{
$project: {
field: 1,
insensitive: { $toLower: "$field" },
document: "$$ROOT"
}
},
{ $sort: { insensitive: 1 } }
]).toArray()

Tried all the above and answers
Consolidating the result
Answer-1:
db.collection.aggregate([
{ "$project": {
"field": 1,
"insensitive": { "$toLower": "$field" }
}},
{ "$sort": { "insensitive": 1 } } ])
Aggregate query converts the field into lower, So performance is low for large data.
Answer-2:
db.collection.find({}).collation({locale: "en"}).sort({"name":1})
By default mongo follows uft-8 encoding(Z has high piriority then a) rules ,So overriding with language-specific rules.
Its fast compare to above query
Look into an official document to customize rules
https://docs.mongodb.com/manual/reference/collation/

We solve this problem with the help of .sort function in JavaScript array
Here is the code
function foo() {
let results = collections.find({
_id: _id
}, {
fields: {
'username': 1,
}
}).fetch();
results.sort((a, b)=>{
var nameA = a.username.toUpperCase();
var nameB = b.username.toUpperCase();
if (nameA nameB) {
return 1;
}
return 0;
});
return results;
}

mongo and java finding nested keys and values

my mongo collections contains following documents
{
"_id" : ObjectId("52d43cd29b85346a4aa6fe17"),
"windowsServer" : [
{
"topProcess" : [ ]
}]
},
{
"_id" : ObjectId("52d43cd29b85346a4aa6fe18"),
"windowsServer" : [
{
"topProcess" : [ {pid:1,name:"wininit"}]
}]
}
Now in my java code I want to used only topProcess in above case I want only second document which topProcess having some data. For this I write my java code as below
BasicDBObject criteria = new BasicDBObject();
BasicDBObject projections = new BasicDBObject();
criteria.put("windowsServer.topProcess", new BasicDBObject("$ne", "[]"));
projections.put("windowsServer.topProcess",1);
DBCursor cur = coll.find(criteria,projections);
while(cur.hasNext() && !isStopped()) {
String json = cur.next().toString();
}
when I execute above code and print json string it also contains the both topProcess. Can any one knows how should I get only second documents topProcess?

Try this one (and translate it to your java driver):
"windowsServer.topProcess": {$not: {$size: 0} }

In your code, you only have mistake in the following line.
criteria.put("windowsServer.topProcess", new BasicDBObject("$ne", "[]"));
You try to compare if an array is empty by using brackets as a String. You can use BasicDBList() for an empty array. Update above line with the following and it should work.
criteria.put("windowsServer.topProcess", new BasicDBObject("$ne", new BasicDBList()));

MongoDB/Java SDK: Query elements with a value in array

I am very new to MongoDB and its Java... SDK? Api? I have a very simple question, but I haven't been able to find a satisfactory answer.
Let's say I have a collection of instances that are like:
{
"_id": {
"$oid": "5156171e5d451c136236e738"
},
"_types": [
"Sample"
],
"last_z": {
"$date": "2012-12-30T09:12:12.250Z"
},
"last": {
"$date": "2012-12-30T04:12:12.250Z"
},
"section": "5156171e5d451c136236e70f",
"s_mac": "AA:AA:AA:AA:AA:AA",
"_cls": "Sample",
}
And I have a hard-coded Java list:
static List<String> MAC_LIST = Arrays.asList("90:27:E4:0E:3D:D2", "A8:26:D9:E6:1D:8B");
What I would like to know is how to query the MongoDB so it will give me all the objects whose s_mac field has a value that appears in the MAC_LIST List.
I'm guessing I should use the $in operator, but I don't know how to translate it to Java code.
Any hint or link to pages with explanations of the use of the $in operator through the Java SDK would be appreciated!

Here is a contrived example that works for me (driver version 2.10.1) - you can adjust the IP address and run it as is to check if you get the same outcome:
public void gss() throws Exception{
MongoClient mongo = new MongoClient("192.168.1.1");
DB db = mongo.getDB("test");
DBCollection collection = db.getCollection("stackoverflow");
DBObject o1 = new BasicDBObject();
o1.put("s_mac", "AA:AA:AA:AA:AA:AA");
o1.put("_cls", "Sample1");
DBObject o2 = new BasicDBObject();
o2.put("s_mac", "90:27:E4:0E:3D:D2");
o2.put("_cls", "Sample2");
DBObject o3 = new BasicDBObject();
o3.put("s_mac", "A8:26:D9:E6:1D:8B");
o3.put("_cls", "Sample3");
collection.insert(o1, o2, o3);
System.out.println(collection.find().count());
List<String> MAC_LIST = Arrays.asList("90:27:E4:0E:3D:D2", "A8:26:D9:E6:1D:8B");
System.out.println(collection.find(new BasicDBObject("s_mac", new BasicDBObject("$in", MAC_LIST))).count());
}
It inserts the following documents:
{ "_id" : ObjectId("5159ff98567e143bff0668e9"),
"s_mac" : "AA:AA:AA:AA:AA:AA",
"_cls" : "Sample1"
}
{ "_id" : ObjectId("5159ff98567e143bff0668ea"),
"s_mac" : "90:27:E4:0E:3D:D2",
"_cls" : "Sample2"
}
{ "_id" : ObjectId("5159ff98567e143bff0668eb"),
"s_mac" : "A8:26:D9:E6:1D:8B",
"_cls" : "Sample3"
}
A call to collection.find().count() returns 3 and a call to collection.find(new BasicDBObject("s_mac", new BasicDBObject("$in", MAC_LIST))).count() returns 2 which I think is what you expected.