I have a collection ref consisting of
{
"_id": "refId"
"ref": "itemId"
}
and a collection item
{
"_id": "itemId"
"field1": ...
"array": [
{
"field2": "a"
},
{
"field2": "b"
}
]
}
Here i need to 'join' a.k.a perform a lookup on ref to item, and project the fields field1 and the last item of array as arrayItem into the top level of the return document, to return
{
"_id": "itemId",
"field1": ...
"arrayItem: "b"
}
Using the mongo shell, this works perfectly using the following statement:
db.ref.aggregate([
{ "$lookup": {
"from": "item",
"localField": "ref",
"foreignField": "_id",
"as": "item"
}},
{ "$unwind": "$item" },
{ "$project": {
"_id": "$item._id",
"field1": "$item.field1",
"arrayItem": { $slice: [ "$item.array", -1, 1 ]}
}},
{ "$unwind": "$arrayItem" }
])
Now, i have to make this happen in Java with Spring and Spring-MongoDB, which i have tried using this Aggregation:
Aggregation.newAggregation(
new LookupAggreationOperation("item", "ref", "_id", "item"),
Aggregation.unwind("item"),
Aggregation.project(Fields.from(
Fields.field("_id", "$item._id"),
Fields.field("field1", "$item.field1"),
Fields.field("array", "$item.array"))),
Aggregation.project("_id", "field1", "array")
.and("$array").project("slice", -1, 1).as("$arrayItem"),
Aggregation.unwind("$array"));
As lookup is only available in Spring-Mongo 1.9.2, I had to rebuild it myself like that:
public class LookupAggregationOperation implements AggregationOperation {
private DBObject operation;
public LookupAggregationOperation(String from, String localField,
String foreignField, String as) {
this.operation = new BasicDBObject("$lookup", //
new BasicDBObject("from", from) //
.append("localField", localField) //
.append("foreignField", foreignField) //
.append("as", as));
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
The problem is that the slice command is not yet implemented as method in Aggregation (and isn't in 1.9.2), hence i need to call project("slice", -1, 1).as("$arrayItem") as suggested in DATAMONGO-1457. The slicing is simply not executed, arrayItem is null in the result.
I'm using Spring 1.3.5 with Spring MongoDB 1.8.4 and MongoDB 3.2.8.
Related
Assume I have two mongo collections as follows.
Collection A
{
"_id" : ObjectId("582abcd85d2dfa67f44127e0"),
"level" : "super"
"dataReference" : Object
B : DbRef(B, 5b618a570550de0021aaa2ef, undefined)
}
Collection B
{
"_id" : ObjectId("5b618a570550de0021aaa2ef"),
"role" : "admin"
}
What I need is retrieve the records from Collection A, which records have "level" field's value as "super" and its related Collection B record's "role" value as "admin".
For this, I am trying to use aggregation and java mongoTemplate.
Following is the code that I tried but it returns 0 records.
final TypedAggregation<A> typedAggregation = Aggregation.newAggregation(A.class,
Aggregation.match(Criteria.where("level").equals(level)),
Aggregation.lookup("B", "_id", "dataReference.B.$id", "Basb"),
Aggregation.match(new Criteria().andOperator(
Criteria.where("B.role").regex("admin")
)));
final AggregationResults<Map> A = mongoTemplate.aggregate(typedAggregation, "A", Map.class);
Please note that I am new to Mongo aggregation.
It's quiet ugly solution:
MongoTemplate
You cannot use TypedAggregation because we need to transform A collection to be able join with B collection
Aggregation typedAggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("level").is("super")),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return Document.parse("{\"$addFields\":{\"dataReference\":{\"$reduce\":{\"input\":{\"$objectToArray\":\"$dataReference\"},\"initialValue\":null,\"in\":{\"$cond\":[{\"$eq\":[{\"$type\":\"$$this.v\"},\"objectId\"]},\"$$this.v\",\"$$value\"]}}}}}");
}
},
Aggregation.lookup("B", "dataReference", "_id", "B"),
Aggregation.match(new Criteria().andOperator(
Criteria.where("B.role").regex("admin")
)
)
);
final AggregationResults<Document> A = mongoTemplate.aggregate(typedAggregation, "A", Document.class);
MongoDB Aggregation
db.A.aggregate([
{
"$match": {
"level": "super"
}
},
{
"$addFields": {
"B": {
"$reduce": {
"input": {
"$objectToArray": "$dataReference"
},
"initialValue": null,
"in": {
"$cond": [
{
"$eq": [
{
"$type": "$$this.v"
},
"objectId"
]
},
"$$this.v",
"$$value"
]
}
}
}
}
},
{
"$lookup": {
"from": "B",
"localField": "B",
"foreignField": "_id",
"as": "B"
}
},
{
"$match": {
"$and": [
{
"B.role": {
"$regex": "admin",
"$options": ""
}
}
]
}
}
])
MongoPlayground
The following is a app_relation Collection :
{
"_id" : ObjectId("5bf518bb1e9f9d2f34a8299b"),
"app_id" : "123456789",
"dev_id" : "1",
"user_id" : "1",
"status" : "active",
"created" : NumberLong(1542789294)
}
The other collection is app :
{
"_id" : ObjectId("5bd02abb1e9f9d2adc211138"),
"app_id" : "123456789",
"custom_app_name" : "Demo",
"price" : 10,
"created" : NumberLong(1540369083)
}
Using Lookup in mongodb I want to embed App collection in AppRelation
For the same my mongodb query is:
db.app_relation.aggregate([
{
$lookup: {
"from": "app",
"localField": "app_id",
"foreignField": "app_id",
"as": "data"
}
},
{
$match: {
"data": {
"$size": 1
}
}
}
])
The equivalent code in Spring Java is :
LookupOperation lookupOperation = LookupOperation.newLookup().from("app").localField("app_id")
.foreignField("app_id").as("data");
AggregationOperation match = Aggregation.match(Criteria.where("data").size(1));
Aggregation aggregation = Aggregation.newAggregation(lookupOperation, match)
.withOptions(Aggregation.newAggregationOptions().cursor(new BasicDBObject()).build());
List<AppRelation> results = mongoTemplate.aggregate(aggregation, AppRelation.class, AppRelation.class)
.getMappedResults();
When executing the above code it provides the empty collection, whereas executing mongo db query it provides proper result.
The query generated in Debug logs is:
{
"aggregate": "app_relation",
"pipeline": [
{
"$lookup": {
"from": "app",
"localField": "app_id",
"foreignField": "app_id",
"as": "data"
}
},
{
"$match": {
"data": {
"$size": 1
}
}
}
],
"cursor": {}
}
I am having a difficult time creating the Java equivalent of the $push part of the following mongo $group:
db.crewtrades.aggregate(
{
$group:
{
_id: {staffId: '$initiatingStaffId', status: '$status', firstName: '$initiatingStaffFirstName'},
count:{'$sum':1}
}
},
{
$group:
{
_id:'$_id.staffId',
name: {$first: '$_id.firstName'},
statuses:
{
$push:
{
status:'$_id.status',
count:'$count'
}
},
staffCount: {$sum: '$count'}
}
}
)
The result should look something like this:
{
"_id" : "00238061",
"name" : "Kirstie Rachel Wong",
"statuses" : [
{
"status" : "Pending",
"count" : 1.0
},
{
"status" : "Approved",
"count" : 2.0
}
],
"staffCount" : 3.0
}
I have tried the following:
private List<Bson> createPipeline(String companyCode, String startDate, String endDate, String eventType, List<String> eventConfigIds, String requestType, String staffId, String status) {
return Arrays.asList(
match(and(getBsons(companyCode, startDate, endDate, eventConfigIds, requestType, staffId, status)
)),
(group("_id", createIdFields(startDate, endDate, eventType))),
(group("$staffId",
first("firstName", "$firstName"),
first("lastName", "$lastName"),
first("startDate", "$startDate"),
first("endDate", "$endDate"),
first("eventType", "$eventType"),
first("requestType", "$requestType"),
push("statuses", createPushFields()),
sum("staffCount", "$count")
)));
}
private DBObject createPushFields() {
return new BasicDBObject("status","$status").append("count","$count");
}
private List<BsonField> createIdFields(String startDate, String endDate, String eventType) {
ArrayList<BsonField> fields = new ArrayList<>();
fields.add(first("staffId", "$initiatingStaffId"));
fields.add(first("firstName", "$initiatingStaffFirstName"));
fields.add(first("lastName", "$initiatingStaffLastName"));
fields.add(push("status", "$status"));
fields.add(first("startDate", startDate));
fields.add(first("endDate", endDate));
fields.add(first("eventType", eventType));
fields.add(first("requestType", "$tradeBoardId"));
fields.add(sum("count", 1));
return fields;
}
But it ends up returning:
[
{
"_id": "00238061",
"firstName": "Kirstie Rachel Wong",
"lastName": "Wong",
"startDate": "2018-01-01T16:30:40Z",
"endDate": "2018-12-12T16:30:40Z",
"eventType": "DutySwap",
"requestType": 0,
"statuses": [
{
"status": [
"Approved",
"Approved",
"Pending"
],
"count": 3
}
],
"staffCount": 3
}
]
How should I specify in the Mongo Java code that the status ("Approved", "Pending") and count for each status should be a JSON object within the statuses array? The Mongo native query is able to handle this well but I cannot get the Java driver to do the same.
Appreciate any help with this!
I have a document in collection as :
{
"_id": ObjectId("5ab273ed31fa764560a912f8"),
"hourNumber": 21,
"errorSegments": [{
"agentName": "agentX"
},
{
"agentName": "agentY"
}]
}
I am trying to perform following aggregation function in spring boot where i want to retrieve "errorSegments" of a particular hour that matches an agent , which is working fine in mongo shell :
Working shell :
db.errorsegment.aggregate([{
"$match": {
"hourNumber": 21,
"errorSegments.agentName": "agentX"
}
},
{
"$project": {
"errorSegments": {
"$filter": {
"input": "$errorSegments",
"as": "e",
"cond": {
"$eq": ["$$e.agentName",
"agentX"]
}
}
}
}
},
{
"$unwind": "$errorSegments"
},
{
"$replaceRoot": {
"newRoot": "$errorSegments"
}
}])
So it provides output only , which is desired result :
{ "agentName" : "agentX" }
But my following code in spring gives error :
MatchOperation match = Aggregation.match(Criteria.where("hourNumber").is(21).and("errorSegments.agentName").is("agentX"));
ProjectionOperation project = Aggregation.project()
.and(new AggregationExpression() {
#Override
public DBObject toDbObject(AggregationOperationContext context) {
DBObject filterExpression = new BasicDBObject();
filterExpression.put("input", "$errorSegments");
filterExpression.put("as", "e");
filterExpression.put("cond", new BasicDBObject("$eq", Arrays.<Object> asList("$$e.agentName","agentX")));
return new BasicDBObject("$filter", filterExpression);
} }).as("prop");
UnwindOperation unwind = Aggregation.unwind("$errorSegments");
ReplaceRootOperation replaceRoot = Aggregation.replaceRoot("$errorSegments");
Aggregation aggregation = Aggregation.newAggregation(match,project,unwind,replaceRoot);
AggregationResults<ErrorSegment> errorSegments = mongoOps.aggregate(aggregation, SegmentAudit.class , ErrorSegment.class);
Following is the Logs :
java.lang.IllegalArgumentException: Invalid reference 'errorSegments'!
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:99) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:71) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.aggregation.UnwindOperation.toDBObject(UnwindOperation.java:95) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.aggregation.AggregationOperationRenderer.toDBObject(AggregationOperationRenderer.java:56) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.aggregation.Aggregation.toDbObject(Aggregation.java:580) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1567) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1502) ~[spring-data-mongodb-1.10.3.RELEASE.jar:na]
The source of error is the alias that you use with filter operation. It should be errorSegments instead of prop but you you other problems too. Use fieldname i.e $ prefix is not right.
Here is the updated aggregation. You can use $filter helper.
MatchOperation match = Aggregation.match(Criteria.where("hourNumber").is(21).and("errorSegments.agentName").is("agentX"));
ProjectionOperation project = Aggregation.
project().and(ArrayOperators.Filter.filter("errorSegments")
.as("e")
.by(ComparisonOperators.Eq.valueOf(
"e.agentName")
.equalToValue(
"agentX")))
.as("errorSegments");
UnwindOperation unwind = Aggregation.unwind("errorSegments");
ReplaceRootOperation replaceRoot = Aggregation.replaceRoot("errorSegments");
Aggregation aggregation = Aggregation.newAggregation(match,project,unwind,replaceRoot);
Below is the generated query.
[
{
"$match": {
"hourNumber": 21,
"errorSegments.agentName": "agentX"
}
},
{
"$project": {
"errorSegments": {
"$filter": {
"input": "$errorSegments",
"as": "e",
"cond": {
"$eq": [
"$$e.agentName",
"agentX"
]
}
}
}
}
},
{
"$unwind": "$errorSegments"
},
{
"$replaceRoot": {
"newRoot": "$errorSegments"
}
}
]
i have following structure
{
"name": "abc",
"lname": "xyz",
"data": {
"1": {
"info": {
"test": "test"
},
"info1": {
"test": "test"
}
}
}
}
now i want to add following object in 'data' object
"2": {
"info": {
"test": "test1"
},
"info1": {
"test": "test1"
}
}
how to do that in mongodb using mongodb java driver?
In MongoDB shell you can do it as below :
db.collection.update( {_id:id} , { $set: { "data.2":
{
"info": {"test": "test1" }, "info1": {"test": "test1"}
}
}});
In Java driver :
DBObject query = new BasicDBObject("_id", "123");
DBObject update = new BasicDBObject();
DBObject info = new BasicDBObject("test","test1");
update.put("$set", new BasicDBObject("data.2",
new BasicDBObject("info",info).append("info1",info));
collection.update(query, update);