We Keep Coding

Spring mongodb aggregation pipeline

MatchOperation matchDepartmentStage = Aggregation.match(Criteria.where("department").is("DEPT_A"));
ProjectionOperation projectEmployeeStage = Aggregation.project()
.andExpression("concat(emp_name,'-', emp_age)").as("emp_name_age");
Criteria criteriaNameAndAge = new Criteria();
List<Pair<String,String>> nameAndAgeList = new ArrayList<>();
nameAndAgeList.add(Pair.of("abcd", "30"));
nameAndAgeList.forEach(nameAndAge -> {
criteriaNameAndAge.and("emp_name_age").ne(nameAndAge.getFirst()+"-"+nameAndAge.getSecond());
});
MatchOperation matchNameAndAgeStage = Aggregation.match(criteriaNameAndAge);
SortOperation sortStage = Aggregation.sort(Sort.Direction.ASC, "dateCreated");
Aggregation aggregation = Aggregation.newAggregation(matchDepartmentStage,
projectEmployeeStage,
matchNameAndAgeStage,
sortStage);
Employee employee = null;
AggregationResults<Employee> employeeAggregationResults =
mongoTemplate.aggregate(aggregation, "employees", Employee.class);
My attempt is to do as following
Filter employees by department
From the result, for every employee, project a new field that combines 2 fields, emp_name & emp_age to emp_name_age
Create a dynamic match criteria to match this new field
Sort by date
It does not work as expected. Error is invalid reference 'dateCreated'
Sample documents:
{ "_id" : "1", "department" : "DEPAT_A", "name" : "abcd", "age" : "30", "dateCreated" : ISODate("2019-12-31T10:30:00Z"), "_class" : "org.example.Employee" }
{ "_id" : "2", "department" : "DEPAT_A", "name" : "abcd", "age" : "40", "dateCreated" : ISODate("2019-12-31T10:30:00Z"), "_class" : "org.example.Employee" }
EDIT
changed a few lines and it is working. 2 problems identified.
Project has to mention the fields. In my case, name all fields, which I didnt like.
Use nin instead of ne
ProjectionOperation projectEmployeeStage = Aggregation.project(Field.fields("department","name","age","dateCreated"))
.andExpression("concat(emp_name,'-', emp_age)").as("emp_name_age");
List<String> nameAges = nameAndAgeList.stream()
.map(nameAge -> nameAge.getFirst()+"-"+nameAge.getSecond())
.collect(Collectors.toList());
Criteria criteriaNameAndAge = new Criteria("emp_name_age").nin(nameAges);
MatchOperation matchNameAndAgeStage = Aggregation.match(criteriaNameAndAge);

Get Average from Mongo Collection using Aggrerate

I am trying to learn how to use the Mongo in Java and have been able to make some simple queries but I have been having trouble with the aggregate operator.
The document structure is a simple one, as the following:
{
"_id" : ObjectId("57dbe94f0507a4d8710ac5b2"),
"name" : "Name1",
"age" : 23
}
{
"_id" : ObjectId("57dbe9750507a4d8710ac5b3"),
"name" : "Name2",
"age" : "String for examble"
}
{
"_id" : ObjectId("57dbee630507a4d8710ac5b5"),
"name" : "Name3",
"age" : 24
}
All I want to do is get the average of the ages in the collection ( name example ).
Simply using mongo I can get the desirable result with the following consult:
db.example.aggregate([
{
$group: {
_id: null,
averageAge: { $avg: "$age" }
}
}
]);
I have tried the following:
BasicDBObject groupFields = new BasicDBObject("_id", "null");
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("mediaIdade", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemplo").aggregate(Arrays.asList (group));
Which is almost a direct translation but got a "java.lang.NoSuchMethodError: org.bson.codecs.BsonTypeClassMap.keys()Ljava/util/Set;" , unsurprisingly.
But I cannot translate that to Java. I have checked and found this question but could not understand it due to the use of opperators such as $unwind. So I'm trying to make query as simple as possible to better understand how the Java framework for aggregation works.
Can someone help?

Try something like this.
MongoCollection<Document> dbCollection = db.getCollection("example", Document.class);
AggregateIterable<org.bson.Document> aggregate = dbCollection.aggregate(Arrays.asList(Aggregates.group("_id", new BsonField("averageAge", new BsonDocument("$avg", new BsonString("$age"))))));
Document result = aggregate.first();
double age = result.getDouble("averageAge");
Input:
{ "_id" : ObjectId("58136d6ed96cc299c224d529"), "name" : "Name1", "age" : 23 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52a"), "name" : "Name2", "age" : 26 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52b"), "name" : "Name3", "age" : 24 }
Output:
24.333333333333332

I noticed a small typo in your code:
db.getCollection("exemplo").aggregate(Arrays.asList (group));
Should be example instead of examplo
Also the easiest way to translate working mongo expression is using Document.parse method.
In your case it could be:
db.getCollection("exemplo").aggregate(Arrays.asList(
Document.parse("{ $group: { _id: null, averageAge: { $avg: '$age' } } }")
));
Your impl also almost correct, two minor issues:
replace "null" string with just null value
Use "averageAge" instead of "mediaIdade"
BasicDBObject groupFields = new BasicDBObject("_id", null);
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("averageAge", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemple").aggregate(Arrays.asList (group));
To get result:
agregate.first().getDouble("averageAge");

Try this:
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(group);
Iterable<DBObject> list = output.results();
If required add a filter to the query you can add a match parameter:
DBObject match = new BasicDBObject();
match.put("age", new BasicDBObject("$gte", 25));
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(match, group);
Iterable<DBObject> list = output.results();

How can I compose a query such as (A || B || C) && (X || Y) in MongoDB Java Driver 3.2

Unfortunately, I cannot find an example for Mongo 3.2 java driver for query like "(A or B or C) and (D or E or F or G)"
Number of parameters inside parentheses will be variable - up to hundred.
Funny thing that I've found example for "(A && B) || (X && Y)" but it doesn't help me.
How to execute queries with both AND and OR clauses in MongoDB with Java
My code produces error:
MongoQueryException: Query failed with error code 2 and error message '$or/$and/$nor entries need to be full objects'
List<Document> docs = new ArrayList<>();
for (Integer ln: input.getLastnames()) {
docs.add(new Document("lastname",ln));
}
Document queryLN = new Document(
"$or", Arrays.asList(docs)
);
docs.clear();
for (Integer fn: input.getFirstnames()) {
docs.add(new Document("firstname",fn));
}
Document queryFN = new Document(
"$or", Arrays.asList(docs)
);
Document query = new Document(
"$and", Arrays.asList(queryFN,queryLN));
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());

You should use an "in" query in such condition when you have a long unknown list of OR conditions.
An example code:
try {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("so");
DBCollection coll = db.getCollection("employees");
List<Integer> ageList = new ArrayList<>();
ageList.add(30);
ageList.add(35);
List<String> nameList = new ArrayList<>();
nameList.add("Anna");
BasicDBObject query = new BasicDBObject("$and", Arrays.asList(
new BasicDBObject("age", new BasicDBObject("$in", ageList)),
new BasicDBObject("name", new BasicDBObject("$in", nameList)))
);
DBCursor cursor = coll.find(query);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
}catch (Exception ex){
ex.printStackTrace();
}
To experiment with the above code, you can add the following entries in your MongoDB:
db.employees.insert({"name":"Adma","dept":"Admin","languages":["german","french","english","hindi"],"age":30, "totalExp":10});
db.employees.insert({"name":"Anna","dept":"Admin","languages":["english","hindi"],"age":35, "totalExp":11});
db.employees.insert({"name":"Bob","dept":"Facilities","languages":["english","hindi"],"age":36, "totalExp":14});
db.employees.insert({"name":"Cathy","dept":"Facilities","languages":["hindi"],"age":31, "totalExp":4});
db.employees.insert({"name":"Mike","dept":"HR","languages":["english", "hindi", "spanish"],"age":26, "totalExp":3});
db.employees.insert({"name":"Jenny","dept":"HR","languages":["english", "hindi", "spanish"],"age":25, "totalExp":3});
The above code produces this query:
db.employees.find({"$and":[{"age":{"$in":[30, 35]}},{"name":{"$in":["Anna"]}}]});
And the output is:
{ "_id" : { "$oid" : "57ff3e5e3dedf0228d4862ad"} , "name" : "Anna" , "dept" : "Admin" , "languages" : [ "english" , "hindi"] , "age" : 35.0 , "totalExp" : 11.0}
A good article on this topic: https://www.mkyong.com/mongodb/java-mongodb-query-document/
Read these as well: https://stackoverflow.com/a/8219679/3896066 and https://stackoverflow.com/a/14738878/3896066

Let's first understand your code. We will make it simple by replacing the for loops with simple statements and add some print statements.
List<Document> docs = new ArrayList<>();
docs.add(new Document("lastname","Walker"));
docs.add(new Document("lastname","Harris"));
Document queryLN = new Document("$or", Arrays.asList(docs));
docs.clear();
System.out.println(queryLN.toJson());//{ "$or" : [[]] }
docs.add(new Document("firstname", "Pat"));
docs.add(new Document("firstname", "Matt"));
Document queryFN = new Document("$or", Arrays.asList(docs));
System.out.println(queryLN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
System.out.println(queryFN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());//{ "$and" : [{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }, { "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }] }
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Observations:
docs is already a list. Using Arrays.asList on docs, creates a list of list, which is not acceptable for the $and, $or, $nor. These operators accept a list of Documents. That explains the error message.
Arrays.asList does not create a copy of the array or the list that it receives. It just creates a wrapper over it. Also, new document() does not copy the list that it receives with "$or", just references the original list. Hence, calling docs.clear() will reset the content in $or of queryLN.
Also, the above concept explains why the 2nd and 3rd print statements give the same output.
Let us get the code working now.
List<Document> docsLN = new ArrayList<Document>();
List<Document> docsFN = new ArrayList<Document>();
for (Integer ln: input.getLastnames()) {
docsLN.add(new Document("lastname",ln));
}
Document queryLN = new Document("$or", docsLN);
for (Integer fn: input.getFirstnames()) {
docsFN.add(new Document("firstname",fn));
}
Document queryFN = new Document("$or", docsFN);
System.out.println(queryLN.toJson());
System.out.println(queryFN.toJson());
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Also, consider replacing the $or with $in.
From the docs:
When using $or with that are equality checks for the
value of the same field, use the $in operator instead of the $or
operator.

Add new field to mongoDB embeded Doc

I have a mongo Document structure like this with comments field as embedded doc.
i want to add "newField" : "something" to embedded comments field with "cid" : "17426944" :
in java driver i tried :
BasicDBObject query = new BasicDBObject(); // MongoDB query
BasicDBObject record = new BasicDBObject(); // MongoDB record
BasicDBObject dbObject = new BasicDBObject(); // fieldsToUpdate
query.put("comments.cid","17426944");
dbObject.put("comments.newField","something");
record.put("$set",dbObject );
mongoCtrl.updateCollection(query, record, false, true); // mongoCtrl is my connection contrl
problem here is :
com.mongodb.MongoException: cannot use the part (comments of comments.newField) to traverse the element ({comments:[ .......... ]})
at com.mongodb.CommandResult.getException(CommandResult.java:100)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:134)
at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:142)
sample doc :
{
"_id" : ObjectId("53abb8d17bfd6b92e2398d34"),
"name" : "satish",
"number": "1122112",
"comments" : [
{
"cid" : "17426944"
},
{
"cid" : "607395840"
},
{
"cid" : "393084416"
}
]
}
what i need :
{
"_id" : ObjectId("53abb8d17bfd6b92e2398d34"),
"name" : "satish",
"number": "1122112",
"comments" : [
{
"cid" : "17426944",
"newField" : "something"
},
{
"cid" : "607395840"
},
{
"cid" : "393084416"
}
]
}
Plz help me.Thanks vijay

Pretty close, you just missed the positional $ operator to match the position of the array found in your query portion:
BasicDBObject query = new BasicDBObject(); // MongoDB query
BasicDBObject record = new BasicDBObject(); // MongoDB record
BasicDBObject dbObject = new BasicDBObject(); // fieldsToUpdate
query.put("comments.cid","17426944");
dbObject.put("comments.$.newField","something");
record.put("$set",dbObject );
mongoCtrl.updateCollection(query, record, false, true);

BasicDBObject update duplicate same document

I have a big problem with the api java mongodb. I use a request with the update methods of DBCollection class and in the mongodb i get a multiple same document while the value doesn't change,help me please. I don't want to have a duplicate document.
BasicDBObject query = new BasicDBObject();
query.append("ad", "man2ist").append("list.id", new BasicDBObject("$ne", "5")); // "list.id" : {$ne : 0 }
BasicDBObject a = new BasicDBObject("id",String.valueOf(5)).append("value", 100);
BasicDBObject upd = new BasicDBObject("$addToSet",new BasicDBObject("list",a));
System.out.println(query);
System.out.println(upd);
WriteResult r = dbc.update(query,upd,true,false);
//db.friends.update({ "ad" : "man2ist" , "list.id" : { $ne : "4"} },{ $addToSet : { "list" : { "id" : "4","value" : 100}}},true,true);
my document here :
{
"ad" : "man2ist",
"createdDate" : ISODate(),
"list" : [
{
"id" : "45",
"value" : 489
},
{
"id" : "5",
"value" : 20,
},
{
"id" : "4578",
"value" : 21,
} ]}

The problem is that you set the upset flag to true, meaning that the document should be created if no document matching the criteria is found. If there is no document matching the criteria in the db, the mongo shell will do the same.
If you change your query to this,
WriteResult r = dbc.update(query,upd,false,false);
it should work all the time.