We Keep Coding

How to get the count of element with non-empty-array-field when group in mongodb aggregate using Spring Data Mongo?

I have the following documents in one collection named as mail_test. Some of them have a tags field which is an array:
/* 1 */
{
"_id" : ObjectId("601a7c3a57c6eb4c1efb84ff"),
"email" : "aaaa#bbb.com",
"content" : "11111"
}
/* 2 */
{
"_id" : ObjectId("601a7c5057c6eb4c1efb8590"),
"email" : "aaaa#bbb.com",
"content" : "22222"
}
/* 3 */
{
"_id" : ObjectId("601a7c6d57c6eb4c1efb8675"),
"email" : "aaaa#bbb.com",
"content" : "33333",
"tags" : [
"x"
]
}
/* 4 */
{
"_id" : ObjectId("601a7c8157c6eb4c1efb86f4"),
"email" : "aaaa#bbb.com",
"content" : "4444",
"tags" : [
"yyy",
"zzz"
]
}
There are two documents with non-empty-tags, so I want the result to be 2.
I use the the following statement to aggregate and get the correct tag_count:
db.getCollection('mail_test').aggregate([{$group:{
"_id":null,
"all_count":{$sum:1},
"tag_count":{"$sum":{$cond: [ { $ne: ["$tags", undefined] }, 1, 0]}}
//if replace `undefined` with `null`, I got the tag_count as 4, that is not what I want
//I also have tried `$exists`, but it cannot be used here.
}}])
and the result is:
{
"_id" : null,
"all_count" : 4.0,
"tag_count" : 2.0
}
and I use spring data mongo in java to do this:
private void test(){
Aggregation agg = Aggregation.newAggregation(
Aggregation.match(new Criteria()),//some condition here
Aggregation.group(Fields.fields()).sum(ConditionalOperators.when(Criteria.where("tags").ne(null)).then(1).otherwise(0)).as("tag_count")
//I need an `undefined` instead of `null`,or is there are any other solution?
);
AggregationResults<MailTestGroupResult> results = mongoTemplate.aggregate(agg, MailTest.class, MailTestGroupResult.class);
List<MailTestGroupResult> mappedResults = results.getMappedResults();
int tag_count = mappedResults.get(0).getTag_count();
System.out.println(tag_count);//get 4,wrong
}
I need an undefined instead of null but I don't know how to do this,or is there are any other solution?

You can use Aggregation operators to check if the field tags exists or not with one of the following constructs in the $group stage of your query (to calculate the tag_count value):
"tag_count":{ "$sum": { $cond: [ { $gt: [ { $size: { $ifNull: ["$tags", [] ] }}, 0 ] }, 1, 0] }}
// - OR -
"tag_count":{ "$sum": { $cond: [ $eq: [ { $type: "$tags" }, "array" ] }, 1, 0] }
Both, return the same result (as you had posted).

How to write mongo cli query in mongo-template for $in aggregation

This is how my data looks like
{
"_id" : "2011250546437843117",
"name" : "Book",
"textbook" : [
"Maths",
"Science"
],
"language" : [
"English"
],
"isRead" : true,
"isAvailable" : true
}
I have to filter documents based on textbook,and based on that isRead field should be true or false.
my mongo query is
db.user.aggregate([
{
$match: {
"isAvailable": true
}
},
{
$project: {
"textbook": 1,
"name": 1,
"isread": {
$in: [
"Maths",
"$textbook"
]
}
}
}
]);
I have tried to write this using mongo-template
Aggregation aggregation = newAggregation(match(Criteria.where("isAvailable").is(true)),
project("textbook","name"));
I dont understand how to write the $in operator in project stage.
Thankyou in advance.

Get count of unique ObjectId from array MongoDB

I'm new to working with MongoDb and do not know a lot of things.
I need to write an aggregation request.
Here is the JSON document structure.
{
"_id" : ObjectId("5a72f7a75ef7d430e8c462d2"),
"crawler_id" : ObjectId("5a71cbb746e0fb0007adc6c2"),
"skill" : "stack",
"created_date" : ISODate("2018-02-01T13:19:03.522+0000"),
"modified_date" : ISODate("2018-02-01T13:22:23.078+0000"),
"connects" : [
{
"subskill" : "we’re",
"weight" : NumberInt(1),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec11")
]
},
{
"subskill" : "b1",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec11"),
ObjectId("5a71d88d5ef7d41964fbec1b")
]
},
{
"subskill" : "making",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec1b"),
ObjectId("5a71d88d5ef7d41964fbec1c")
]
},
{
"subskill" : "delivery",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec1c"),
ObjectId("5a71d88d5ef7d41964fbec1e")
]
}
]
}
I need the result return the name of skill and the number of unique parser_id.
In this case, the result should be:
[
{
"skill": "stack",
"quantity": 4
}
]
where "stack" - skill name,
and "quantity" - count of unique parser_id.
ObjectId("5a71d88d5ef7d41964fbec11")
ObjectId("5a71d88d5ef7d41964fbec1b")
ObjectId("5a71d88d5ef7d41964fbec1c")
ObjectId("5a71d88d5ef7d41964fbec1e")
Can some one help me with this request ???

Given the document supplied in your question, this command ...
db.collection.aggregate([
{ $unwind: "$connects" },
// count all occurrences
{ "$group": { "_id": {skill: "$skill", parser_id: "$connects.parser_id"}, "count": { "$sum": 1 } }},
// sum all occurrences and count distinct
{ "$group": { "_id": "$_id.skill", "quantity": { "$sum": 1 } }},
// (optional) rename the '_id' attribute to 'skill'
{ $project: { 'skill': '$_id', 'quantity': 1, _id: 0 } }
])
... will return:
{
"quantity" : 4,
"skill" : "stack"
}
The above command groups by skill and connects.parser_id and then gets a distinct count of those groups.
Your command includes the java tag so I suspect you are looking to execute the same command using the MongoDB Java driver. The code below (using MongoDB Java driver v3.x) will return the same result:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
List<Document> documents = collection.aggregate(Arrays.asList(
Aggregates.unwind("$connects"),
new Document("$group", new Document("_id", new Document("skill", "$skill").append("parser_id", "$connects.parser_id"))
.append("count", new Document("$sum", 1))),
new Document("$group", new Document("_id", "$_id.skill").append("quantity", new Document("$sum", 1))),
new Document("$project", new Document("skill", "$_id").append("quantity", 1).append("_id", 0))
)).into(new ArrayList<>());
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Note: this code deliberately uses the form new Document(<pipeline aggregator>, ...) instead of the Aggregators utilities to make it easier to see the translation between the shell command and its Java equivalent.

try $project with $reduce
$setUnion is used to keep only the distinct ids and finally $size used to get the distinct array count
db.col.aggregate(
[
{$project : {
_id : 0,
skill : 1,
quantity : {$size :{$reduce : {input : "$connects.parser_id", initialValue : [] , in : {$setUnion : ["$$value", "$$this"]}}}}
}
}
]
).pretty()
result
{ "skill" : "stack", "quantity" : 4 }

MongoDB Query to match both single entry and array elements

I have a problem with MongoDB QueryBuilder.
Assume I have a number of documents, that can contain one or more users:
{
"_id": "document1",
"data": {
"user": {
"credentials": {
"name": "John",
"lastname": "Watson",
"middle": "Hemish"
}
}
}
}
{
"_id": "document2",
"data": {
"user": [
{
"credentials": {
"name": "John",
"lastname": "Nicholson",
"middle": "Joseph"
}
},
{
"credentials": {
"name": "Mary",
"lastname": "Watson",
"middle": ""
}
}
]
}
}
{
"_id": "document3",
"data": {
"user": [
{
"credentials": {
"name": "John",
"lastname": "Watson",
"middle": "Hemish"
}
},
{
"credentials": {
"name": "John",
"lastname": "Nicholson",
"middle": "Joseph"
}
},
{
"credentials": {
"name": "Mary",
"lastname": "Watson",
"middle": ""
}
}
]
}
}
What I am trying to do is the query, that will return only those documents containing John Watson as a user.
Here what I got so far:
1.
QueryBuilder qb = QueryBuilder.start("credentials.lastname").is("Watson").and("credentials.name").is("John");
DBObject query = QueryBuilder.start("data.user").elemMatch(qb.get()).get();
this query will return only document3: there is no array in document1 and no match in document2 (but I would like it to return document1 and document3)
2.
DBObject query = QueryBuilder.start("data.user.credentials.lastname").is("Watson").and("data.user.credentials.name").is("John").get();
this one will return all three documents: document1 and document3 are desired match, but the query will match as well document2, for it has Watson and John in query fields in the array, no matter that they are separate entries.
Is there any way to make a right query that will return document1 and document3 for John Watson?
I am trying to do it in Java, but any other example would be fine.
Right now I use a workaround combining results from both queries: first I get limit(100) results from the query with elementMatch(), then, if there are less than 100 results, I do the second query and filter all wrong matches. But I hope there is a better and more effective way to get those results.

I could give you at best like the following where user would be in an array as unwind value of the key data. I think a little bit more effort would lead you to the exact format as you want.
I am sharing it as I think it should serve the purpose or anyhow it should help you.
The aggregation query:
db.tuttut.aggregate([
{$unwind:"$data.user"},
{ $project: {
_id:1,
data:1,
temp: {name:"$data.user.credentials.name",
lastname:"$data.user.credentials.lastname"}
} } ,
{ $group:{
_id:"$_id" ,
data: {$addToSet: "$data"} ,
temp:{ $addToSet: "$temp" } } },
{ $match:{ temp:{name:"John",lastname:"Watson"} } } ,
{$project:{_id:1, data:1}}
]).pretty()
Returned Result:
{
"_id" : "document1",
"data" : [
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Watson",
"middle" : "Hemish"
}
}
}
]
}
{
"_id" : "document3",
"data" : [
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Watson",
"middle" : "Hemish"
}
}
},
{
"user" : {
"credentials" : {
"name" : "Mary",
"lastname" : "Watson",
"middle" : ""
}
}
},
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Nicholson",
"middle" : "Joseph"
}
}
}
]
}

retrieve values from nested json array in mongodb

My mongo collection has entries in the following format
{
"myobj" : {
"objList" : [
{ "location" : "Texas" },
{ "location" : "Houston"},
{ "name":"Sam" }
]
},
"category" : "cat1"
}
{
"myobj" :
{
"objList" : [
{ "location" : "Tennesy" },
{ "location" : "NY"},
{ "location" : "SF" }
]
},
"category" : "cat2"
}
I want to extract the "**category**" where location is "Houston". In case of simple JSON object I have to just pass it as query like:
BasicDBObject place = new BasicDBObject();
place.put("location", "Houston");
But in case of nested JSON I don't know how to pass it as a query and get the appropriate category. ie If I pass my location as"Houston" then it should return it's appropriate category "cat1"...i hope my question is clear now....

Ok, you have your documents:
db.coll1.insert({
"myobj" : {
"objList" : [
{ "location" : "Texas" },
{ "location" : "Houston"},
{ "name":"Sam" }
]
},
"category" : "cat1"
})
and
db.coll1.insert({
"myobj" : {
"objList" : [
{ "location" : "Tennesy" },
{ "location" : "Houston"},
{ "location" : "SF" }
]
},
"category" : "cat1"
})
Now you can find what you want using the dot operator:
db.coll1.find({"myobj.objList.location": "Texas"}).pretty() will return one object which has Texas
db.coll1.find({"myobj.objList.location": "SF"}).pretty() will return one object which has SF
db.coll1.find({"myobj.objList.location": "Houston"}).pretty() will return both objects
And now I hope you will be able to write it in Java. I have never used Java, but based on this question you can do something like this. If it will not work, just look how to use dot operator in java driver for mongo:
DBCursor cursor = coll1.find(new BasicDBObject("myobj.objList.location", "Texas"));
P.S. you told, that you wanted to retrieve category. In such a way, you will need to use a projection db.coll1.find({<the query I provided}, {category: 1, _id: 0})