Mongo document:
{
"_id" : "1",
"array" : [
{
"item" : "item"
},
{
"item" : "item"
}
]
}
My mongo shell query looks like so:
db.getCollection('collectionName').aggregate(
{$match: { _id: "1"}},
{$project: { count: { $size:"$array" }}}
)
Is there anyway to implement this using the Mongo Template from Spring?
So far I have this:
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
ProjectionOperation project = new ProjectionOperation();
Aggregation aggregate = Aggregation.newAggregation(match, project);
mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
I think I am only missing the project logic but I'm not sure if it is possible to do $size or equivalent here.
It's quite possible, the $size operator is supported (see DATAMONGO-979 and its implementation here). Your implementation could follow this example:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = new Aggregation(
match(where("_id").is("1")), //
project() //
.and("array") //
.size() //
.as("count")
);
AggregationResults<IntegerCount> results = mongoTemplate.aggregate(
agg, collectionName, Integer.class
);
List<IntegerCount> intCount = results.getMappedResults();
Please find below the sample code. You can change it accordingly for your requirement with collection name, collection name class and array field name.
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
Aggregation aggregate = Aggregation.newAggregation(match, Aggregation.project().and("array").project("size").as("count"));
AggregationResults<CollectionNameClass> aggregateResult = mongoOperations.aggregate(aggregate, "collectionName", <CollectionNameClass>.class);
if (aggregateResult!=null) {
//You can find the "count" as an attrribute inside "result" key
System.out.println("Output ====>" + aggregateResult.getRawResults().get("result"));
System.out.println("Output ====>" + aggregateResult.getRawResults().toMap());
}
Sample output:-
Output ====>[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}]
Output ====>{serverUsed=127.0.0.1:27017, waitedMS=0, result=[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}], ok=1.0}
You can write query as
Aggregation aggregate = Aggregation.newAggregation(Aggregation.match(Criteria.where("_id").is(1)),
Aggregation.project().and("array").size().as("count")); mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
It will execute the following query { "aggregate" : "collectionName" , "pipeline" : [ { "$match" : { "_id" : 1}} , { "$project" : { "count" : { "$size" : [ "$array"]}}}]}
Related
I am new bee to mongo. Below is the Aggregation operation I am doing in mongodb shell . But In my java ProjectionAggregation , am not able to give the $toObjectId. Please correct me what am I missing.
db shell query
db.getCollection('UserData').aggregate([
{
$project : {
"username" : "$username",
"beneficiaries" : "$beneficiaries"
}
},
{
$unwind : {
path : "$beneficiaries",
preserveNullAndEmptyArrays: true
}
},
{
$project : {
"username" : "$username",
"beneficiaries" : "$beneficiaries",
---- dont know how to give $toObjectId in java ProjectionOperation .
"beneficiaryStudId" : { $toObjectId : "$beneficiaries.studentId" }
}
},
{
$lookup:
{
from: "StudentProfileData",
localField: "beneficiaryStudId",
foreignField: "_id",
as: "studProfile"
}
}
])
Java code Projection Operation
ProjectionOperation projectUserAndBeneficiaries = Aggregation.project()
.andExpression("username").as("username")
.andExpression("beneficiaries").as("beneficiaries");
ProjectionOperation projectUserAndOtherDetails = Aggregation.project()
.andExpression("username").as("username")
.andExpression("beneficiaries").as("beneficiaries")
---- How to give $toObjectId in projection operation .andExpression("beneficiaries.studentId").as("beneficiaryStudId");
LookupOperation lookupOperation = LookupOperation.newLookup().
from("StudentProfileData").
localField("beneficiaryStudId").
foreignField("_id").
as("studProfile");
Aggregation agg = Aggregation.newAggregation(projectUserAndBeneficiaries, unwindBeneficiars,
projectUserAndOtherDetails
,lookupOperation);
AggregationResults<UserAndStudentData> output
= mongotemplate.aggregate(agg, "UserData", UserAndStudentData.class);
Sample Ouput
Output in db shell
{
"_id" : ObjectId("5d2f08574de2690001c281ac"),
"username" : "ks241#goo.com",
"beneficiaries" : {
"studentId" : "5d2f0e9c3bcf3e0001a7e562",
"mcBeneficiaryId" : "597418",
"enabled" : true
},
"beneficiaryStudId" : ObjectId("5d2f0e9c3bcf3e0001a7e562"),
"studProfile" : [
{
"_id" : ObjectId("5d2f0e9c3bcf3e0001a7e562"),
"lastName" : "Sharma",
"firstName" : "Kapil",
"studentRegisterCustomFieldValues" : [
{
"bcfdValue" : "One",
"bcfdName" : "Year"
}
],
"gender" : "M",
"merchantId" : "38788943"
}
]
}
where as in java
the studProfile array is always empty if I add $toObjectId the above java aggregation projection query and ran it produces the values as same as that db shell .
Spring data doesn't support to few type of methods. This problem may include into it. But we may use this solution.
Aggregation aggregation=newAggregation(
p-> new Document("$project",
new Document()
.append("username","$username"),
.append("beneficiaries","$beneficiaries)
.append("beneficiaryStudId",
new Document("$toObjectId","$beneficiaries.studentId")
)
)
)
There is a way to do this with Spring's ProjectionOperation and ConvertOperators classes:
ProjectionOperation projection = Aggregation.project()
.andInclude("username", "beneficiaries")
.and(ConvertOperators.ToObjectId.toObjectId("$beneficiaries.studentId"))
.as("beneficiaryStudId");
I need to add the fields(get from ui) which need to be fetched using mongodb Aggregation,
From uri i will get param as fields which has comma seperated string of fields
http://foo.com?fields=id,name
A document looks like:
{
"_id" : "3a237c007a87d",
"name" : "Available",
"is_active" : true,
}
The below will work as i want and produce the result
Aggregation aggregation = newAggregation(
project(fields.contains("name") ? "name" : "",
fields.contains("id") ? "id" : ""),
fields.contains("is_active") ? "is_active" : ""),
skip((page-1)*limit),
limit(limit)
);
The above query gets what i want and its shown belo
{
"_id" : "3a237c007a87d",
"name" : "Available"
}
IF i run with below query i m getting atleast one field need to specify in project
And the code:
ProjectionOperation project = project();
for(String field : fields) {
project.andInclude(field);
}
but the field is not adding in projectionOperation
if projectOperation need to like to have
{ "$project" : {"id":1, "name":1 } }
Aggregation aggregation = newAggregation(
project,
skip((page-1)*limit),
limit(limit)
);
The output need to be
{
"_id" : "3a237c007a87d",
"name" : "Available"
}
I like to avoid check whether the list contain the field as they want in project.
Is it possible that you're just confusing some variables here (fields vs fieldList)? Also, you need to use the return value of the andInclude() call Try this instead:
List<String> fieldList = new ArrayList<>();
fieldList.add("id");
fieldList.add("name");
ProjectionOperation project = project();
for(String field : fieldList) { // note that I'm using 'fieldList' here
project = project.andInclude(field);
}
I want to write a generic code for aggregation using mongoTempalte and Aggregation.group() method. So I have the problem of passing generic fields into the group method with the first() operator
here is my demo native query as follows:
db.subscriberProfile.aggregate([{"$unwind":"$usage_history"},
{ "$group" : { "_id" :"$_id" ,"birthdate" : { "$first":"$birthdate"} , "category" : { "$first":"$category"} , "control_group" : { "$first":"$control_group"} , "sumOfTotalUsage" : { "$sum" :{"$cond": [ { "$gte" :[ "$usage_history.date" , ISODate( "2017-01-13T10:43:55.306Z")] }, "$usage_history.total_usage", 0]}}}},
{ "$match" : { "$and" : [ { "birthdate" : { "$lte" : ISODate( "2017-07-12T10:43:55.306Z")}} , { "birthdate" : { "$gte" : ISODate( "1917-07-12T10:20:35.306Z")}} , { "category" : { "$in" : [ "Prepaid"]}} , { "control_group" : false} , { "sumOfTotalUsage" : { "$gte" : 0}}]}}])
And here is my Sample code in Java.
UnwindOperation unwind = Aggregation.unwind("usage_history");
GroupOperation group = Aggregation.group(fields.toArray(new String[fields.size()])).sum("usage_history.total_usage").as("sumOfTotalUsage");
I just want to know how to add multiple fields in group operation with $first operator.
So, Is there any way to pass list of fields with list of first operator to the group method.
Thanks,
Try this code,I hope this will help you
UnwindOperation unwind = Aggregation.unwind("usage_history");
BasicDBObject object = new BasicDBObject("_id", "$_id");
for (String string : fields) {
object.append(string, new BasicDBObject("$first", "$" + string));
}
object.append("total", new BasicDBObject("$sum", new BasicDBObject("$cond",
new Object[] { new BasicDBObject("$gte", new Object[] { "$usage_history.date", calendarMin.getTime() }),
"$usage_history.total_usage", 0 })));
BasicDBObject groupObject = new BasicDBObject("$group", object);
DBObject groupOperation = (DBObject) groupObject;
MatchOperation matchMain = Aggregation
.match(new Criteria().andOperator(criteriaList.toArray(new Criteria[criteriaList.size()])));
Aggregation aggregation = Aggregation.newAggregation(unwind, new CustomGroupOperation(groupOperation),
matchMain);
This is the values of my data stored in mongo db. How am I able to retrieve all the data of "HomeTown" and store it all into a list? My list would contain AA, AA, BB, BB, etc... I want to use that array list to create a for loop of each Hometown.
Sample mongo data
[{ "_id" : { "$oid" : "4ceb753a70fdf877ef5113ca"} , "HomeTown" : "AA" ,
"PhoneNumber" : { "CustName" : "xxx" , "Number" : "3403290"},
"MobileNumber" : { "CustName" : "yyy" , "Number" : "9323304302"}}]
[{ "_id" : { "$oid" : "4ceb753a70fdf877ef5113ca"} , "HomeTown" : "AA" ,
"PhoneNumber" : { "CustName" : "xxx" , "Number" : "3403290"},
"MobileNumber" : { "CustName" : "yyy" , "Number" : "9323304302"}}]
[{ "_id" : { "$oid" : "4ceb753a70fdf877ef5113ca"} , "HomeTown" : "BB" ,
"PhoneNumber" : { "CustName" : "xxx" , "Number" : "3403290"},
"MobileNumber" : { "CustName" : "yyy" , "Number" : "9323304302"}}]
[{ "_id" : { "$oid" : "4ceb753a70fdf877ef5113ca"} , "HomeTown" : "BB" ,
"PhoneNumber" : { "CustName" : "xxx" , "Number" : "3403290"},
"MobileNumber" : { "CustName" : "yyy" , "Number" : "9323304302"}}]
How can I get all of the values of "HomeTown" in Java into an array? I am trying to create a for loop with the HomeTown Names. I am currently using mongodb dependency through Spring boot. I am not sure how would I implement mongodb into my java to use mongo db.
Attempt/Problem
I was able to retrieve mongodb values in a list using the following code. I am trying to convert this list to a arraylist.
public List<AppPortModel> getAppPortList() {
List<ApServerModel> objLst = mongoOperations.findAll(ApServerModel.class);
String[] apServerArray = new String[objLst.size()];
for(int i = 0; i < objLst.size(); i++) {
apServerArray[i] = objLst.get(i);
}
Error on objLst.get(i)
Type mismatch: cannot convert from ApServerModel to String
Attempt #2 Following Sagar Example
#Autowired
MongoOperations mongoOperations;
MongoCollection<ApServerModel> myCollection = **mongoOperations.getCollection("apAllCustomer");**
List<ApServerModel> result = myCollection.find().into(new ArrayList<>());
Error on mongoOperations.getCollection
Type mismatch: cannot convert from DBCollection to MongoCollection<ApServerModel>
Looks like you're using mongo 3.x driver.You'll need to use something like this.
MongoClient mongoClient = new MongoClient();
MongoDatabase db = mongoClient.getDatabase("mkoydb");
MongoCollection<Document> myCollection = db.getCollection("apAllCustomer");
List<Document> result = myCollection.find().into(new ArrayList<>());
Fix for Attempt 1:
public List<AppPortModel> getAppPortList() {
List<ApServerModel> objLst = mongoOperations.findAll(ApServerModel.class);
String[] apServerArray = new String[objLst.size()];
for(int i = 0; i < objLst.size(); i++) {
apServerArray[i] = objLst.get(i).getHomeTown();
}
Fix for Attempt 2:
DBCollection dbCollection = mongoOperations.findAll(AppServreModel.class, "apAllCustomer");
You can call toArray() which, intuitively, returns a List<DBObject>.
List<DBObject> myList = null;
DBCursor myCursor=myCollection.find(new BasicDBObject(),"HomeTown");
myList = myCursor.toArray();
if you are using java driver with version 3.0 and above you can also do
collection.find().projection(Projections.include("HomeTown"));
You can use projection to retrieve only required fields.
db.apAllCustomer.find( ..., { HomeTown: 1 } );
In Java it depends on the API you use. See this for Spring Data:
SpringData MongoDB Using projection
MongoDB-Java:
Restrict fields in Result
I am looking for a solution without spring data. My project requirement is to do without spring data.
To calculate the sum using aggregate function by mongo command, able to get output. But same by using spring data getting exception.
Sample mongo query :
db.getCollection('events_collection').aggregate(
{ "$match" : { "store_no" : 3201 , "event_id" : 882800} },
{ "$group" : { "_id" : "$load_dt", "event_id": { "$first" : "$event_id" }, "start_dt" : { "$first" : "$start_dt" }, "count" : { "$sum" : 1 } } },
{ "$sort" : { "_id" : 1 } },
{ "$project" : { "load_dt" : "$_id", "ksn_cnt" : "$count", "event_id" : 1, "start_dt" : 1, "_id" : 0 } }
)
Same thing done in java as,
String json = "[ { \"$match\": { \"store_no\": 3201, \"event_id\": 882800 } }, { \"$group\": { \"_id\": \"$load_dt\", \"event_id\": { \"$first\": \"$event_id\" }, \"start_dt\": { \"$first\": \"$start_dt\" }, \"count\": { \"$sum\": 1 } } }, { \"$sort\": { \"_id\": 1 } }, { \"$project\": { \"load_dt\": \"$_id\", \"ksn_cnt\": \"$count\", \"event_id\": 1, \"start_dt\": 1, \"_id\": 0 } } ]";
BasicDBList pipeline = (BasicDBList) JSON.parse(json);
System.out.println(pipeline);
AggregationOutput output = col.aggregate(pipeline);
exception is :
com.mongodb.CommandFailureException: { "serverUsed" : "somrandomserver/10.10.10.10:27001" , "errmsg" : "exception: pipeline element 0 is not an object" , "code" : 15942 , "ok" : 0.0}
Could someone please suggest how to use aggregate function with spring?
Try the following (untested) Spring Data MongoDB aggregation equivalent
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
MongoTemplate mongoTemplate = repository.getMongoTemplate();
Aggregation agg = newAggregation(
match(Criteria.where("store_no").is(3201).and("event_id").is(882800)),
group("load_dt")
.first("event_id").as("event_id")
.first("start_dt").as("start_dt")
.count().as("ksn_cnt"),
sort(ASC, previousOperation()),
project("ksn_cnt", "event_id", "start_dt")
.and("load_dt").previousOperation()
.and(previousOperation()).exclude()
);
AggregationResults<OutputType> result = mongoTemplate.aggregate(agg,
"events_collection", OutputType.class);
List<OutputType> mappedResult = result.getMappedResults();
As a first step, filter the input collection by using a match operation which accepts a Criteria query as an argument.
In the second step, group the intermediate filtered documents by the "load_dt" field and calculate the document count and store the result in the new field "ksn_cnt".
Sort the intermediate result by the id-reference of the previous group operation as given by the previousOperation() method.
Finally in the fourth step, select the "ksn_cnt", "event_id", and "start_dt" fields from the previous group operation. Note that "load_dt" again implicitly references an group-id field. Since you do not want an implicit generated id to appear, exclude the id from the previous operation via and(previousOperation()).exclude().
Note that if you provide an input class as the first parameter to the newAggregation method the MongoTemplate will derive the name of the input collection from this class. Otherwise if you don’t not specify an input class you must provide the name of the input collection explicitly. If an input-class and an input-collection is provided the latter takes precedence.