I'm performing an aggregation operation using the java mongodb driver, and I followed the example from the docs (pasted below). According to this, the _id field should be hidden. However, in my experience with my own code as well as the output of this example, the _id field doesn't hide even when setting the projection value to 0 (it works from the mongo shell). Does anyone know if this is a bug in the mongodb java driver? Or am I doing something incorrectly?
// create our pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match", new BasicDBObject("type", "airfare") );
// build the $projection operation
DBObject fields = new BasicDBObject("department", 1);
fields.put("amount", 1);
fields.put("_id", 0);
DBObject project = new BasicDBObject("$project", fields );
// Now the $group operation
DBObject groupFields = new BasicDBObject( "_id", "$department");
groupFields.put("average", new BasicDBObject( "$avg", "$amount"));
DBObject group = new BasicDBObject("$group", groupFields);
// run aggregation
AggregationOutput output = collection.aggregate( match, project, group );
The _id field you are getting at the end is from the $group operator. If you want to rename it back to department, add another $project to the end of the pipeline and translate _id to department.
Related
I'm trying to load the matched documents into a temporary collection using aggregation query. Actually, I'm able to load all the matched documents into the temporary collection of MongoDB but my java program is throwing Null pointer exception at the for loop.
I'm totally stuck over here. May I know the reason for Null Pointer exception in this scenario . And can anyone please suggest me regarding the same ...
Document query = {"$or":[{"roll":1,"joiningDate":{"$gte":ISODate("2017-04-11T00:00:00Z")}},{"roll":2,"joiningDate":{"$gte": ISODate("2017-03-17T00:00:00Z")}}]};
Document match = new Document("$match",new Document("$or",query));
Document out =new Document("$out","TempCol");
System.out.println("Before Aggregation");
AggregateIterable<Document> resultAgg = collection.aggregate(Arrays.asList(match,out));
System.out.println("After aggregation");
for (Document doc : resultAgg){
System.out.println("The result of aggregation match:-");
}
System.out.println("Completed");
I generally prefer to keep the pipeline structured in one variable.
But the general idea here is use Document where you see {} and Arrays.asList where you see []:
List<Document> pipeline = Arrays.<Document>asList(
new Document("$match",
new Document("$or", Arrays.<Document>asList(
new Document("roll", 1)
.append("joiningDate", new Document(
"$gte", new DateTime(2017,04,11,0,0,0, DateTimeZone.UTC).toDate()
)),
new Document("controlId", 2)
.append("joiningDate", new Document(
"$gte", new DateTime(2017,03,17,0,0,0, DateTimeZone.UTC).toDate()
))
))
),
new Document("$out","TempCol")
);
AggregateIterable<Document> resultAgg = controlIssueCollection.aggregate(pipeline);
Also make sure when constructing a Date object with whatever your favorite construction method here is ( for me org.joda.time.DateTime ) that you are working with time in UTC, unless you really mean otherwise. And if you are comparing with values stored in MongoDB as shown in the shell, then you mean UTC.
In the example found here (http://mongodb.github.io/mongo-java-driver/3.2/builders/aggregation/), the aggregation grouping is based on "$customerId":
collection.aggregate(Arrays.asList(match(eq("author", "Dave")),
group("$customerId", sum("totalQuantity", "$quantity"),
avg("averageQuantity", "$quantity"))
out("authors")));
How would one add another grouping field into the aggregation pipepline, like say, "$systemId"?
Within the mongo-shell, you would do something like this:
db.coll.aggregate(
[
{
$group : {
_id : {"customerId":"$customerId", "systemId":"$systemId"},
//etc
}
}
]
)
In Java, you could build a query:
DBObject groupFields = new BasicDBObject();
DBObject groupIdFields = new BasicDBObject();
groupIdFields.put("customerId", "$customerID");
groupIdFields.put("systemId", "$systemId");
groupFields.put( "_id", groupIdFields );
//...
//... here some more aggregate criteria
//...
final AggregationOutput output = coll.aggregate(group);
Now, this seems like an awful lot of code just to get more than one grouping criterion. How is this possible with the aggregation pipeline in the mongo 3 java driver?
I'm trying to query and sort documents as followed:
Query only for documents older than SOMETIME.
Within range of AROUNDME_RANGE_RADIUS_IN_RADIANS.
Get distance for each document.
Sort them by time. New to Old.
Overall it should return up to 20 results.
But it seems that since $geoNear is by default limited to 100 results, I get unexpected results.
I see $geoNear working in the following order:
Gets docs from the entire collection, by distance.
And only then executes the given Query.
Is there a way to reverse the order?
MongoDB v2.6.5
Java Driver v2.10.1
Thank you.
Example document in my collection:
{
"timestamp" : ISODate("2014-12-27T06:52:17.949Z"),
"text" : "hello",
"loc" : [
34.76701564815013,
32.05852053407342
]
}
I'm using aggregate since from what I understood it's the only way to sort by "timestamp" and get the distance.
BasicDBObject query = new BasicDBObject("timestamp", new BasicDBObject("$lt", SOMETIME));
// aggregate: geoNear
double[] currentLoc = new double[] {
Double.parseDouble(myLon),
Double.parseDouble(myLat)
};
DBObject geoNearFields = new BasicDBObject();
geoNearFields.put("near", currentLoc);
geoNearFields.put("distanceField", "dis");
geoNearFields.put("maxDistance", AROUNDME_RANGE_RADIUS_IN_RADIANS));
geoNearFields.put("query", query);
//geoNearFields.put("num", 5000); // FIXME: a temp solution I would really like to avoid
DBObject geoNear = new BasicDBObject("$geoNear", geoNearFields);
// aggregate: sort by timestamp
DBObject sortFields = new BasicDBObject("timestamp", -1);
DBObject sort = new BasicDBObject("$sort", sortFields);
// aggregate: limit
DBObject limit = new BasicDBObject("$limit", 20);
AggregationOutput output = col.aggregate(geoNear, sort, limit);
You could add a $match stage at the top of the pipleine, to filter the documents before the $geonear stage.
BasicDBObject match = new BasicDBObject("timestamp",
new BasicDBObject("$lt", SOMETIME));
AggregationOutput output = col.aggregate(match,geoNear, sort, limit);
The below piece of code now, is not required,
geoNearFields.put("query", query);
I have a collection full of taxi data which looks something like this (simplified):
{
"TaxiLicense" : "TET123",
"GetOff" : "2015-01-10,00:02:11",
"GetOffLongitude" : 121.41
}
Since this real time data, the Taxi is constantly sending new documents to the collection with a new GetOff time along with GPS coordinates. I only want the GPS coordinates for each distinct Taxi License at the most recent GetOff time.
Is there a way for the Aggregation Framework to do this in Java or do I need to aggregate all the entries and then have my Java program find the latest time for each unique taxi?
I'm currently working with
DBObject taxigroup = new BasicDBObject("$group",
new BasicDBObject("_id",
new BasicDBObject("License","$TaxiLicense")
.append("getoff","$GetOff").append("longitude","GetOffLongitude"))
);
AggregationOutput aggout = taxistationOfCollection.aggregate( Arrays.asList(taxigroup));
You are basically looking for the $last operator. This is commonly used with $sort and it returns the "last" document properties found at the grouping boundary.
Basic pipeline:
[
{ "$sort": { "TaxiLicence": 1, "GetOff": 1 } },
{ "$group": {
"_id": "$TaxiLicence",
"GetOff": { "$last": "$GetOff" },
"GetOffLongitude": { "$last": "$GetOffLongitude" }
}}
]
Or specifically to construct with the Java Driver:
DBObject sort = new BasicDBObject("$sort",
new BasicDBObject("TaxiLicence", 1)
.append("GetOff",1)
);
DBObject group = new BasicDBObject("$group",
new BasicDBObject("_id", "$TaxiLicence"
.append("GetOff", new BasicDBObject( "$first", "$GetOff" ) )
.append("GeOffLongitude", new BasicDBObject( "$first", "$GeOffLongitude" ))
);
AggregationOutput aggout = taxistationOfCollection.aggregate(sort,group);
I need two to add new items to the existing data in mongo db.
This is mongo db I have the following data.
{
"_id" : ObjectId("53ce11e7d0881d32d9fa935f"),
"name" : "massive riots",
"lastFeachedTime" : "Jul 15, 2014 12:55:27 PM"
}
Here I have to find the data based on name and the I have to add another two items two it.
Here is my code.
DBObject queryObject = new BasicDBObject().append("name", keyword);
if (null == newFetchTime) {
}
DBObject updateObject = new BasicDBObject();
updateObject.put("nextPageToken", nextPageToken);
updateObject.put("prevPageToken", prevPageToken);
Utils utils = new Utils();
DBCollection collection = utils.getStaging().getCollection("test");
collection.update(queryObject, updateObject, true, false);
But I am do update the existing value get removed and the new data get added.
Can any one tell me how to add the items to the existing data in mongo db.
You want the $set operator in your update. This allows the specified fields to be altered without affecting any of the existing fields in the document, unless the specified field exists in which case that field is overwritten:
DBObject update = new BasicDBObject(
"$set", new BasicDBObject()
.append("nextPageToken",nextPageToken)
.append("prevPageToken",prevPageToken)
);
Works out to the equivalent in shell:
{ "$set" : { "nextPageToken" : nextPageToken , "prevPageToken" : prevPageToken }}