Spring mongodb aggregation pipeline - java

MatchOperation matchDepartmentStage = Aggregation.match(Criteria.where("department").is("DEPT_A"));
ProjectionOperation projectEmployeeStage = Aggregation.project()
.andExpression("concat(emp_name,'-', emp_age)").as("emp_name_age");
Criteria criteriaNameAndAge = new Criteria();
List<Pair<String,String>> nameAndAgeList = new ArrayList<>();
nameAndAgeList.add(Pair.of("abcd", "30"));
nameAndAgeList.forEach(nameAndAge -> {
criteriaNameAndAge.and("emp_name_age").ne(nameAndAge.getFirst()+"-"+nameAndAge.getSecond());
});
MatchOperation matchNameAndAgeStage = Aggregation.match(criteriaNameAndAge);
SortOperation sortStage = Aggregation.sort(Sort.Direction.ASC, "dateCreated");
Aggregation aggregation = Aggregation.newAggregation(matchDepartmentStage,
projectEmployeeStage,
matchNameAndAgeStage,
sortStage);
Employee employee = null;
AggregationResults<Employee> employeeAggregationResults =
mongoTemplate.aggregate(aggregation, "employees", Employee.class);
My attempt is to do as following
Filter employees by department
From the result, for every employee, project a new field that combines 2 fields, emp_name & emp_age to emp_name_age
Create a dynamic match criteria to match this new field
Sort by date
It does not work as expected. Error is invalid reference 'dateCreated'
Sample documents:
{ "_id" : "1", "department" : "DEPAT_A", "name" : "abcd", "age" : "30", "dateCreated" : ISODate("2019-12-31T10:30:00Z"), "_class" : "org.example.Employee" }
{ "_id" : "2", "department" : "DEPAT_A", "name" : "abcd", "age" : "40", "dateCreated" : ISODate("2019-12-31T10:30:00Z"), "_class" : "org.example.Employee" }
EDIT
changed a few lines and it is working. 2 problems identified.
Project has to mention the fields. In my case, name all fields, which I didnt like.
Use nin instead of ne
ProjectionOperation projectEmployeeStage = Aggregation.project(Field.fields("department","name","age","dateCreated"))
.andExpression("concat(emp_name,'-', emp_age)").as("emp_name_age");
List<String> nameAges = nameAndAgeList.stream()
.map(nameAge -> nameAge.getFirst()+"-"+nameAge.getSecond())
.collect(Collectors.toList());
Criteria criteriaNameAndAge = new Criteria("emp_name_age").nin(nameAges);
MatchOperation matchNameAndAgeStage = Aggregation.match(criteriaNameAndAge);

Related

Add an array of string to ProjectionOperation in MongoDB

I need to add the fields(get from ui) which need to be fetched using mongodb Aggregation,
From uri i will get param as fields which has comma seperated string of fields
http://foo.com?fields=id,name
A document looks like:
{
"_id" : "3a237c007a87d",
"name" : "Available",
"is_active" : true,
}
The below will work as i want and produce the result
Aggregation aggregation = newAggregation(
project(fields.contains("name") ? "name" : "",
fields.contains("id") ? "id" : ""),
fields.contains("is_active") ? "is_active" : ""),
skip((page-1)*limit),
limit(limit)
);
The above query gets what i want and its shown belo
{
"_id" : "3a237c007a87d",
"name" : "Available"
}
IF i run with below query i m getting atleast one field need to specify in project
And the code:
ProjectionOperation project = project();
for(String field : fields) {
project.andInclude(field);
}
but the field is not adding in projectionOperation
if projectOperation need to like to have
{ "$project" : {"id":1, "name":1 } }
Aggregation aggregation = newAggregation(
project,
skip((page-1)*limit),
limit(limit)
);
The output need to be
{
"_id" : "3a237c007a87d",
"name" : "Available"
}
I like to avoid check whether the list contain the field as they want in project.
Is it possible that you're just confusing some variables here (fields vs fieldList)? Also, you need to use the return value of the andInclude() call Try this instead:
List<String> fieldList = new ArrayList<>();
fieldList.add("id");
fieldList.add("name");
ProjectionOperation project = project();
for(String field : fieldList) { // note that I'm using 'fieldList' here
project = project.andInclude(field);
}

query to get results from mongoDB

I have the following collections in MongoDB
"userDetails" : [
{
"user" : DBRef("users", "RAVI"),
"class1" : DBRef("classes", "1"),
"class2" : DBRef("classes", "2")
},
{
"user" : DBRef("users", "TEJA"),
"class1" : DBRef("classes", "1"),
"class2" : DBRef("classes", "2")
}]
classes
{
"_id" : "1",
"maxScore" : "50",
"subject" : DBRef("subjects", "class1")
}
{
"_id" : "2",
"maxScore" : "80",
"subject" : DBRef("subjects", "class2")
}
users{
"_id" : "RAVI",
"address" : "3-2-2222",
"lastClass" : "1"
"lastScore" : ""
}
{
"_id" : "TEJA",
"address" : "5-23",
"lastClass" : "1"
}
From java program, I want to query such that when I pass input as user name I want to fetch all the details of that user and his classes details.can anyone help me out with this?
previously i tried
BasicDBObject fields = new BasicDBObject("userDetails", 1).append("userDetails", new BasicDBObject("$elemMatch", new BasicDBObject("user.$id", "RAVI")));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
List<UserDetails> usrDetailsList = mongoTemplate.find(query, UserDetails.class);
//Considering you have data inserted in your collection, now this following code will fetch a specific record from collection:
BasicDBObject whereQuery = new BasicDBObject();
whereQuery.put("user","RAVI"); //here you are specifying that you want all details of user-RAVI
DBCursor cursor = collection.find(whereQuery);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
I got my solution with a minor change.I removed $ symbol before id and i got my answer
BasicDBObject fields = new BasicDBObject("userDetails", 1).append("userDetails", new BasicDBObject("$elemMatch", new BasicDBObject("user.id", "RAVI")));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
List<UserDetails> usrDetailsList = mongoTemplate.find(query, UserDetails.class);
but i got stuck in another point.I want to search my query based on subject id something like following
BasicDBObject fields = new BasicDBObject("userDetails", 1).append("userDetails", new BasicDBObject("$elemMatch", new BasicDBObject("class1.$subject.$id", "class1")));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
List<UserDetails> usrDetailsList = mongoTemplate.find(query, UserDetails.class);

How can I compose a query such as (A || B || C) && (X || Y) in MongoDB Java Driver 3.2

Unfortunately, I cannot find an example for Mongo 3.2 java driver for query like "(A or B or C) and (D or E or F or G)"
Number of parameters inside parentheses will be variable - up to hundred.
Funny thing that I've found example for "(A && B) || (X && Y)" but it doesn't help me.
How to execute queries with both AND and OR clauses in MongoDB with Java
My code produces error:
MongoQueryException: Query failed with error code 2 and error message '$or/$and/$nor entries need to be full objects'
List<Document> docs = new ArrayList<>();
for (Integer ln: input.getLastnames()) {
docs.add(new Document("lastname",ln));
}
Document queryLN = new Document(
"$or", Arrays.asList(docs)
);
docs.clear();
for (Integer fn: input.getFirstnames()) {
docs.add(new Document("firstname",fn));
}
Document queryFN = new Document(
"$or", Arrays.asList(docs)
);
Document query = new Document(
"$and", Arrays.asList(queryFN,queryLN));
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
You should use an "in" query in such condition when you have a long unknown list of OR conditions.
An example code:
try {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("so");
DBCollection coll = db.getCollection("employees");
List<Integer> ageList = new ArrayList<>();
ageList.add(30);
ageList.add(35);
List<String> nameList = new ArrayList<>();
nameList.add("Anna");
BasicDBObject query = new BasicDBObject("$and", Arrays.asList(
new BasicDBObject("age", new BasicDBObject("$in", ageList)),
new BasicDBObject("name", new BasicDBObject("$in", nameList)))
);
DBCursor cursor = coll.find(query);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
}catch (Exception ex){
ex.printStackTrace();
}
To experiment with the above code, you can add the following entries in your MongoDB:
db.employees.insert({"name":"Adma","dept":"Admin","languages":["german","french","english","hindi"],"age":30, "totalExp":10});
db.employees.insert({"name":"Anna","dept":"Admin","languages":["english","hindi"],"age":35, "totalExp":11});
db.employees.insert({"name":"Bob","dept":"Facilities","languages":["english","hindi"],"age":36, "totalExp":14});
db.employees.insert({"name":"Cathy","dept":"Facilities","languages":["hindi"],"age":31, "totalExp":4});
db.employees.insert({"name":"Mike","dept":"HR","languages":["english", "hindi", "spanish"],"age":26, "totalExp":3});
db.employees.insert({"name":"Jenny","dept":"HR","languages":["english", "hindi", "spanish"],"age":25, "totalExp":3});
The above code produces this query:
db.employees.find({"$and":[{"age":{"$in":[30, 35]}},{"name":{"$in":["Anna"]}}]});
And the output is:
{ "_id" : { "$oid" : "57ff3e5e3dedf0228d4862ad"} , "name" : "Anna" , "dept" : "Admin" , "languages" : [ "english" , "hindi"] , "age" : 35.0 , "totalExp" : 11.0}
A good article on this topic: https://www.mkyong.com/mongodb/java-mongodb-query-document/
Read these as well: https://stackoverflow.com/a/8219679/3896066 and https://stackoverflow.com/a/14738878/3896066
Let's first understand your code. We will make it simple by replacing the for loops with simple statements and add some print statements.
List<Document> docs = new ArrayList<>();
docs.add(new Document("lastname","Walker"));
docs.add(new Document("lastname","Harris"));
Document queryLN = new Document("$or", Arrays.asList(docs));
docs.clear();
System.out.println(queryLN.toJson());//{ "$or" : [[]] }
docs.add(new Document("firstname", "Pat"));
docs.add(new Document("firstname", "Matt"));
Document queryFN = new Document("$or", Arrays.asList(docs));
System.out.println(queryLN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
System.out.println(queryFN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());//{ "$and" : [{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }, { "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }] }
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Observations:
docs is already a list. Using Arrays.asList on docs, creates a list of list, which is not acceptable for the $and, $or, $nor. These operators accept a list of Documents. That explains the error message.
Arrays.asList does not create a copy of the array or the list that it receives. It just creates a wrapper over it. Also, new document() does not copy the list that it receives with "$or", just references the original list. Hence, calling docs.clear() will reset the content in $or of queryLN.
Also, the above concept explains why the 2nd and 3rd print statements give the same output.
Let us get the code working now.
List<Document> docsLN = new ArrayList<Document>();
List<Document> docsFN = new ArrayList<Document>();
for (Integer ln: input.getLastnames()) {
docsLN.add(new Document("lastname",ln));
}
Document queryLN = new Document("$or", docsLN);
for (Integer fn: input.getFirstnames()) {
docsFN.add(new Document("firstname",fn));
}
Document queryFN = new Document("$or", docsFN);
System.out.println(queryLN.toJson());
System.out.println(queryFN.toJson());
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Also, consider replacing the $or with $in.
From the docs:
When using $or with that are equality checks for the
value of the same field, use the $in operator instead of the $or
operator.

Spring Data Mongo Template - Counting an array

Mongo document:
{
"_id" : "1",
"array" : [
{
"item" : "item"
},
{
"item" : "item"
}
]
}
My mongo shell query looks like so:
db.getCollection('collectionName').aggregate(
{$match: { _id: "1"}},
{$project: { count: { $size:"$array" }}}
)
Is there anyway to implement this using the Mongo Template from Spring?
So far I have this:
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
ProjectionOperation project = new ProjectionOperation();
Aggregation aggregate = Aggregation.newAggregation(match, project);
mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
I think I am only missing the project logic but I'm not sure if it is possible to do $size or equivalent here.
It's quite possible, the $size operator is supported (see DATAMONGO-979 and its implementation here). Your implementation could follow this example:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = new Aggregation(
match(where("_id").is("1")), //
project() //
.and("array") //
.size() //
.as("count")
);
AggregationResults<IntegerCount> results = mongoTemplate.aggregate(
agg, collectionName, Integer.class
);
List<IntegerCount> intCount = results.getMappedResults();
Please find below the sample code. You can change it accordingly for your requirement with collection name, collection name class and array field name.
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
Aggregation aggregate = Aggregation.newAggregation(match, Aggregation.project().and("array").project("size").as("count"));
AggregationResults<CollectionNameClass> aggregateResult = mongoOperations.aggregate(aggregate, "collectionName", <CollectionNameClass>.class);
if (aggregateResult!=null) {
//You can find the "count" as an attrribute inside "result" key
System.out.println("Output ====>" + aggregateResult.getRawResults().get("result"));
System.out.println("Output ====>" + aggregateResult.getRawResults().toMap());
}
Sample output:-
Output ====>[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}]
Output ====>{serverUsed=127.0.0.1:27017, waitedMS=0, result=[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}], ok=1.0}
You can write query as
Aggregation aggregate = Aggregation.newAggregation(Aggregation.match(Criteria.where("_id").is(1)),
Aggregation.project().and("array").size().as("count")); mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
It will execute the following query { "aggregate" : "collectionName" , "pipeline" : [ { "$match" : { "_id" : 1}} , { "$project" : { "count" : { "$size" : [ "$array"]}}}]}

How to match a document with existing array elements in mongodb using java driver

Hello all i am trying to match a document using mongodb java driver for eg :
{
"fName" : "abc",
"lName" : "456",
"dob" : "00",
"address" : "xyz"
}
with
"nameIdentity" : [
{
"fName" : "abc",
"lName" : "def",
"dob" : "00",
"address" : "xyz"
},
{
"fName" : "123",
"lName" : "456",
"dob" : "00",
"address" : "789"
}
If i found the document then i don't do anything else add the document. My problem here is if my source document contains fname : abc and lname: 456 this is matching fname in the first set of nameIdentity and lname in the second set of identity. I want this to be a one complete match. I have tried something like this
List<Document> nameIdentities = (List<Document>) matchedDocument.get("nameIdentity");
for (int i=0;i<nameIdentities.size();i++)
{
temp.add(nameIdentities.get(0));
quBasicDBObject=new BasicDBObject("$and",temp);
}
iterable = mongoDatabase.getCollection("entity").find(updatedDocumentTypeOne);
if (iterable.first() == null)
{
updateResult = mongoDatabase.getCollection("entity")
.updateOne(
new Document("_id", new ObjectId(objectId)),
new Document("$push", new Document("nameIdentity", nameList.get(0))));
}
any suggestions where am i going wrong?
UPDATE
You may have to use the aggregation framework.
Maybe something like:
List<Bson> filterList = new ArrayList<>();
filterList.add(new BsonDocument().append("nameIdentity.fName", new BsonString("abc") ));
filterList.add(new BsonDocument().append("nameIdentity.lName", new BsonString("456") ));
FindIterable<org.bson.Document> it = collection.find(Filters.and(filterList));

Categories