I have collection like
id : 1, url : youtube.com
also one url can be many times,
I need get whole collections and count unique elements
like
youtube 10
google 8
lycos 5
here is the code
public List<URLEntity> findAll() {
List<URLEntity> list = new ArrayList<URLEntity>();
String sort = "searchDate";
String order = "desc";
DBObject sortCriteria = new BasicDBObject(sort, "desc".equals(order) ? -1 : 1);
BasicDBObject query = new BasicDBObject();
DBCursor cursor = mongoCoreService.getDomainCollection().find(query).sort(sortCriteria);
try {
while (cursor.hasNext()) {
DBObject document = cursor.next();
URLEntity entity = new URLEntity();
entity = Converter.toObject(URLEntity.class, document);
list.add(entity);
}
} finally {
cursor.close();
}
return list;
}
thanks
You can achieve this by using the aggregate framework from MongoDB.
db.yourcollection.aggregate({ $group: { _id: '$url', total: {$sum: 1} }})
You'll obtain something like this, that you can manipulate in JAVA:
{
"result" : [
{
"_id" : "youtube.com",
"total" : 10
},
{
"_id" : "google.com",
"total" : 8
},
{
"_id" : "lycos",
"total" : 5
}
],
"ok" : 1
}
Today I found and test answer in java, here is a code
DBCollection mycoll= db.getCollection("domain");
DBObject fields = new BasicDBObject("domain", 1);
DBObject project = new BasicDBObject("$project", fields );
// Now the $group operation
DBObject groupFields = new BasicDBObject( "_id", "$domain");
groupFields.put("total", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields);
// run aggregation
AggregationOutput output = mycoll.aggregate(project, group);
System.out.println(output.getCommand().toString());
for (DBObject dbObject : output.results()) {
System.out.println(dbObject);
}
Related
I am trying to learn how to use the Mongo in Java and have been able to make some simple queries but I have been having trouble with the aggregate operator.
The document structure is a simple one, as the following:
{
"_id" : ObjectId("57dbe94f0507a4d8710ac5b2"),
"name" : "Name1",
"age" : 23
}
{
"_id" : ObjectId("57dbe9750507a4d8710ac5b3"),
"name" : "Name2",
"age" : "String for examble"
}
{
"_id" : ObjectId("57dbee630507a4d8710ac5b5"),
"name" : "Name3",
"age" : 24
}
All I want to do is get the average of the ages in the collection ( name example ).
Simply using mongo I can get the desirable result with the following consult:
db.example.aggregate([
{
$group: {
_id: null,
averageAge: { $avg: "$age" }
}
}
]);
I have tried the following:
BasicDBObject groupFields = new BasicDBObject("_id", "null");
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("mediaIdade", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemplo").aggregate(Arrays.asList (group));
Which is almost a direct translation but got a "java.lang.NoSuchMethodError: org.bson.codecs.BsonTypeClassMap.keys()Ljava/util/Set;" , unsurprisingly.
But I cannot translate that to Java. I have checked and found this question but could not understand it due to the use of opperators such as $unwind. So I'm trying to make query as simple as possible to better understand how the Java framework for aggregation works.
Can someone help?
Try something like this.
MongoCollection<Document> dbCollection = db.getCollection("example", Document.class);
AggregateIterable<org.bson.Document> aggregate = dbCollection.aggregate(Arrays.asList(Aggregates.group("_id", new BsonField("averageAge", new BsonDocument("$avg", new BsonString("$age"))))));
Document result = aggregate.first();
double age = result.getDouble("averageAge");
Input:
{ "_id" : ObjectId("58136d6ed96cc299c224d529"), "name" : "Name1", "age" : 23 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52a"), "name" : "Name2", "age" : 26 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52b"), "name" : "Name3", "age" : 24 }
Output:
24.333333333333332
I noticed a small typo in your code:
db.getCollection("exemplo").aggregate(Arrays.asList (group));
Should be example instead of examplo
Also the easiest way to translate working mongo expression is using Document.parse method.
In your case it could be:
db.getCollection("exemplo").aggregate(Arrays.asList(
Document.parse("{ $group: { _id: null, averageAge: { $avg: '$age' } } }")
));
Your impl also almost correct, two minor issues:
replace "null" string with just null value
Use "averageAge" instead of "mediaIdade"
BasicDBObject groupFields = new BasicDBObject("_id", null);
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("averageAge", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemple").aggregate(Arrays.asList (group));
To get result:
agregate.first().getDouble("averageAge");
Try this:
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(group);
Iterable<DBObject> list = output.results();
If required add a filter to the query you can add a match parameter:
DBObject match = new BasicDBObject();
match.put("age", new BasicDBObject("$gte", 25));
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(match, group);
Iterable<DBObject> list = output.results();
I have a collection in a Mongo database that looks like this:
{
"_id" : ObjectId("561b42d4e4b0d4227d011d2c"),
"product_related_data" : {
"depth" : 6,
"height" : 23,
"product_barcode" : "54491472",
"product_name" : "xyz product",
"product_uuid" : "009b9846-b3ad-49f7-a7a0-d35a04f83480",
"width" : 6
},
"sensostrip_data" : {
"barcode" : "130150208299",
"battery_level" : 2.894
},
"stock_related_data" : {
"calculated_max_product_percentage" : 15.625,
"calculated_min_product_percentage" : 12.5,
"current_stock" : 2,
"max_stock" : 6,
"stock_difference_percentage" : 0,
"stock_difference_value" : 0,
"stock_percentage" : 37.5
},
"store_data" : {
"city_uuid" : "cbb4dfe8-172b-11e4-a1f0-00163ed23ec2",
"ip" : "10.0.1.1",
"retailer_uuid" : "8c33c32c-5903-11e4-a1f0-00163ed23ec2",
"store_name" : "xyz store",
"store_uuid" : "15a6cc90-081f-11e5-b213-001e6745ff8d"
},
"time" : {
"check_date" : "2015-10-11 11:53:55",
"previous_check_date" : "2015-10-11 11:48:57"
},
"id" : "6be54bef-0aa3-456c-b912-1731f8154e7d"
}
The mongo query I'm currently executing returns all the documents for a list of store_uuid's and for a product_uuid, and looks like this:
db.readings
.find({ $and:[ {"store_data.store_uuid": {$in:["15a6cc90-081f-11e5-b213-001e6745ff8d","217b983b-5904-11e4-a1f0-00163ed23ec2","5337d78d-5904-11e4-a1f0-00163ed23ec2"]}}
,{"product_related_data.product_uuid": "f44aa29d-09ce-4902-bf12-d45d44b3dfd0"}]})
My current Java implementation (where I make use of projection) looks like this:
DBCollection table = databaseConncetion().getCollection("readings");
BasicDBObject sensorReturn = new BasicDBObject("sensostrip_data.barcode",1);
BasicDBObject clause1 = new BasicDBObject("store_data.store_uuid", new BasicDBObject("$in", StoreIds));
BasicDBObject clause2 = new BasicDBObject("product_related_data.product_uuid", productId);
BasicDBList and = new BasicDBList();
and.add(clause1);
and.add(clause2);
DBObject query = new BasicDBObject("$and", and);
DBCursor cursor = table.find(query, sensorReturn
.append("stock_related_data.stock_percentage",1)
.append("store_data.store_uuid",1)
.append("time.check_date", 1))
.sort(new BasicDBObject("time.check_date", -1))
.limit(100);
However I need this query to group the results for the latest check_date by barcode
The aggregation framework is at your disposal. Running the following aggregation pipeline will give you the desired result:
Mongo shell:
pipeline = [
{
"$match": {
"store_data.store_uuid": {
"$in": [
"15a6cc90-081f-11e5-b213-001e6745ff8d",
"217b983b-5904-11e4-a1f0-00163ed23ec2",
"5337d78d-5904-11e4-a1f0-00163ed23ec2"
]
},
"product_related_data.product_uuid": "f44aa29d-09ce-4902-bf12-d45d44b3dfd0"
}
},
{ "$sort": { "time.check_date": -1 } },
{
"$group": {
"_id": "$sensostrip_data.barcode",
"stock_percentage": { "$first": "$stock_related_data.stock_percentage" },
"store_uuid": { "$first": "$store_data.store_uuid" },
"check_date": { "$first": "$time.check_date" }
}
},
{ "$limit": 100 }
];
db.readings.aggregate(pipeline);
Java test implementation
public class JavaAggregation {
public static void main(String args[]) throws UnknownHostException {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("test");
DBCollection coll = db.getCollection("readings");
// create the pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match",
new BasicDBObject("store_data.store_uuid", new BasicDBObject("$in", StoreIds))
.append("product_related_data.product_uuid", productId)
);
// sort pipeline
DBObject sort = new BasicDBObject("$sort",
new BasicDBObject("time.check_date", -1)
);
// build the $group operations
DBObject groupFields = new BasicDBObject( "_id", "$sensostrip_data.barcode"); // group by barcode
groupFields.put("stock_percentage", new BasicDBObject( "$first", "$stock_related_data.stock_percentage")); // get the first when ordered documents are grouped
groupFields.put("store_uuid", new BasicDBObject( "$first", "$store_data.store_uuid"));
groupFields.put("check_date", new BasicDBObject( "$first", "$time.check_date"));
// append any other necessary fields
DBObject group = new BasicDBObject("$group", groupFields);
// limit step
DBObject limit = new BasicDBObject("$limit", 100);
// put all together
List<DBObject> pipeline = Arrays.asList(match, sort, group, limit);
AggregationOutput output = coll.aggregate(pipeline);
for (DBObject result : output.results()) {
System.out.println(result);
}
}
}
Example Date (collection name 'people')
{"person" : "Frank",
"coords" : [
{
"point":new GLatLng(40.266044,-74.718479),
},
{
"point":new GLatLng(40.211600,-74.695702),
}
] }
{"person : ""Bob",
"coords" : [
{
"point":new GLatLng(40.266044,-74.718479),
},
{
"point":new GLatLng(40.211600,-74.695702),
},
{
"point":new GLatLng(40.294535,-74.682012),
}
] }
Goal : obtain a total number of 'coord' entries for each person document (accumulated overall)
Expected result 5
What I've tried
public int getTotalCoordinateCount(){
BasicDbObject project= new BasicDBObject()
.append("count",new BasicDBObject("$size","coords"))
AggregationOutput output = db.getPeopleCollection().aggregate(Arrays. <DBObject>asList(
new BasicDBObject("$project", project);
BasicDbObject result = (BasicDBObject)output.results().iterator.next();
return result.getInt("count");
}
Problem I get : 2
Clueless on what I'm doing wrong in the mongo aggregation framework. I suspect i need to include a $sum but not sure where
You need to do an unwind operation on coords field, in the aggregation pipeline. After that you do a group by Null with a count field, to count the results set and you will get the total number of elements from all arrays on all documents.
public int getTotalCoordinateCount(){
BasicDbObject group = new BasicDBObject()
.append("_id", 1)
.append("count", new BasicDBObject("$sum", 1));
BasicDbObject project= new BasicDBObject()
.append("_id", 0)
.append("count", 1);
AggregationOutput output = db.getPeopleCollection().aggregate(Arrays. <DBObject>asList(
new BasicDBObject("$unwind", "coords"),
new BasicDBObject("$group", group),
new BasicDBObject("$project", project)
);
BasicDbObject result = (BasicDBObject)output.results().iterator.next();
return result.getInt("count");
}
I has the following json document in Mongo db. The show element will have several season elements which will also have several episodes elements that in turn have multiple questionEntry elements.
I want to return multiple questionElements where the questionElements metaTag entry equals my search. E.G. if a metaTag element equals my string the return it's parent questionEntry element and search across all elements nested in show.
{
"show":[
{
"season":[
{
"episodes":[
{
"questionEntry":{
"id":1,
"info":{
"seasonNumber":1,
"episodeNumber":5,
"episodeName":"A Hero Sits Next Door"
},
"questionItem":{
"theQuestion":"What is the name of the ringer hired by Mr. Weed?",
"attachedElement":{
"type":1,
"value":""
}
},
"options":[
{
"type":1,
"value":"Johnson"
},
{
"type":1,
"value":"Hideo"
},
{
"type":1,
"value":"Guillermo"
}
],
"answer":{
"questionId":1,
"answer":3
},
"metaTags":[
"Season 1",
"Episode 5",
"Trivia",
"Arya Stark",
"House Stark"
]
}
}
]
}
]
}
]
}
I'm usingthe latest Java Mongo driver in Windows 8.1 and using Mongodb 2.4.4. So my question is what is the best method to return a single or multiple qestionEntry element(s) over this entire show collection that match my search string?
Hopefully somebody here can help me with this.
EDIT:
private DB mongoDatabase;
private DBCollection mongoColl;
private DBObject dbObject;
// Singleton class
// Create client (server address(host,port), credential, options)
mongoClient = new MongoClient(new ServerAddress(host, port),
Collections.singletonList(credential),
options);
mongoDatabase = ClientSingleton.getInstance().getClient().getDB("MyDB");
Please try the below :
db.exp.aggregate([{"$redact":{"$cond": { if: {$gt:[ {"$size": {
$setIntersection : [ { "$ifNull": [ "$metaTags", []]},
["House Stark"]]} } , 0 ]} , then:"$$PRUNE",
else:"$$DESCEND" }}}]).pretty();
[OR]
EDIT :
db.exp.aggregate([{"$unwind":"$show"},
{"$unwind":"$show.season"},
{"$unwind":"$show.season.episodes"},
{"$match" : {"show.season.episodes.questionEntry.metaTags":{"$in":
["Trivia"]}}},
{"$group":{"_id":"$_id","episodes":{"$push":"$show.season.episodes"}
]);
JAVA CODE :
MongoClient client = new MongoClient();
List<String> continentList = Arrays.asList(new String[]{"Trivia"});
DB db = client.getDB("example");
DBCollection coll = db.getCollection("exp");
DBObject matchFields = new
BasicDBObject("show.season.episodes.questionEntry.metaTags",
new BasicDBObject("$in", continentList));
DBObject groupFields = new BasicDBObject( "_id",
"$_id").append("episodes",
new BasicDBObject("$push","$show.season.episodes"));
DBObject unwindshow = new BasicDBObject("$unwind","$show");
DBObject unwindsea = new BasicDBObject("$unwind", "$show.season");
DBObject unwindepi = new BasicDBObject("$unwind",
"$show.season.episodes");
DBObject match = new BasicDBObject("$match", matchFields);
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output =
coll.aggregate(unwindshow,unwindsea,unwindepi,match,group);
for (DBObject result : output.results()) {
System.out.println(result);
}
I want to count number of sub document where type="User.Notice". My database is as below.
i wrote following query but it's returning 1 or 0 always. What's possibly wrong with it.
long countss = eventlist.count(new BasicDBObject("192_168_10_17.type", new BasicDBObject("$eq", "User.Notice")));
System.out.println(countss);
Update:
How can i get all records under particular array. I want all the documents under array `192_168_10_17. can you suggest a way?
First you should unwind 192_168_10_17 and used mongo aggregation as below
db.collectionName.aggregate({
"$unwind": "$192_168_10_17"
}, {
"$match": {
"192_168_10_17.type": "User.Notice"
}
}, {
"$group": {
"_id": "$192_168_10_17.type",
"count": {
"$sum": 1
}
}
}, {
"$project": {
"_id": 0,
"count": "$count"
}
})
Above query return all matching User.Notice count. Now convert this query in java using mongo java aggregation . I tried following java code as below
// unwind 192_168_10_17
DBObject unwind = new BasicDBObject("$unwind", "$192_168_10_17");
// create pipeline operations, with the $match
DBObject match = new BasicDBObject("$match",new BasicDBObject("192_168_10_17.type", "User.Notice"));
// Now the $group operation
DBObject groupFields = new BasicDBObject("_id", "$192_168_10_17.type");
groupFields.put("count", new BasicDBObject("$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields);
// build the $projection operation
DBObject fields = new BasicDBObject("_id", 0);
fields.put("count", "$count");
DBObject project = new BasicDBObject("$project", fields);
// run aggregation
List < DBObject > pipeline = Arrays.asList(match, group, project);
AggregationOutput output = collectionName.aggregate(pipeline);
for (DBObject result: output.results()) {
System.out.println(result);
}