We Keep Coding

Get Average from Mongo Collection using Aggrerate

I am trying to learn how to use the Mongo in Java and have been able to make some simple queries but I have been having trouble with the aggregate operator.
The document structure is a simple one, as the following:
{
"_id" : ObjectId("57dbe94f0507a4d8710ac5b2"),
"name" : "Name1",
"age" : 23
}
{
"_id" : ObjectId("57dbe9750507a4d8710ac5b3"),
"name" : "Name2",
"age" : "String for examble"
}
{
"_id" : ObjectId("57dbee630507a4d8710ac5b5"),
"name" : "Name3",
"age" : 24
}
All I want to do is get the average of the ages in the collection ( name example ).
Simply using mongo I can get the desirable result with the following consult:
db.example.aggregate([
{
$group: {
_id: null,
averageAge: { $avg: "$age" }
}
}
]);
I have tried the following:
BasicDBObject groupFields = new BasicDBObject("_id", "null");
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("mediaIdade", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemplo").aggregate(Arrays.asList (group));
Which is almost a direct translation but got a "java.lang.NoSuchMethodError: org.bson.codecs.BsonTypeClassMap.keys()Ljava/util/Set;" , unsurprisingly.
But I cannot translate that to Java. I have checked and found this question but could not understand it due to the use of opperators such as $unwind. So I'm trying to make query as simple as possible to better understand how the Java framework for aggregation works.
Can someone help?

Try something like this.
MongoCollection<Document> dbCollection = db.getCollection("example", Document.class);
AggregateIterable<org.bson.Document> aggregate = dbCollection.aggregate(Arrays.asList(Aggregates.group("_id", new BsonField("averageAge", new BsonDocument("$avg", new BsonString("$age"))))));
Document result = aggregate.first();
double age = result.getDouble("averageAge");
Input:
{ "_id" : ObjectId("58136d6ed96cc299c224d529"), "name" : "Name1", "age" : 23 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52a"), "name" : "Name2", "age" : 26 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52b"), "name" : "Name3", "age" : 24 }
Output:
24.333333333333332

I noticed a small typo in your code:
db.getCollection("exemplo").aggregate(Arrays.asList (group));
Should be example instead of examplo
Also the easiest way to translate working mongo expression is using Document.parse method.
In your case it could be:
db.getCollection("exemplo").aggregate(Arrays.asList(
Document.parse("{ $group: { _id: null, averageAge: { $avg: '$age' } } }")
));
Your impl also almost correct, two minor issues:
replace "null" string with just null value
Use "averageAge" instead of "mediaIdade"
BasicDBObject groupFields = new BasicDBObject("_id", null);
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("averageAge", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemple").aggregate(Arrays.asList (group));
To get result:
agregate.first().getDouble("averageAge");

Try this:
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(group);
Iterable<DBObject> list = output.results();
If required add a filter to the query you can add a match parameter:
DBObject match = new BasicDBObject();
match.put("age", new BasicDBObject("$gte", 25));
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(match, group);
Iterable<DBObject> list = output.results();

How can I compose a query such as (A || B || C) && (X || Y) in MongoDB Java Driver 3.2

Unfortunately, I cannot find an example for Mongo 3.2 java driver for query like "(A or B or C) and (D or E or F or G)"
Number of parameters inside parentheses will be variable - up to hundred.
Funny thing that I've found example for "(A && B) || (X && Y)" but it doesn't help me.
How to execute queries with both AND and OR clauses in MongoDB with Java
My code produces error:
MongoQueryException: Query failed with error code 2 and error message '$or/$and/$nor entries need to be full objects'
List<Document> docs = new ArrayList<>();
for (Integer ln: input.getLastnames()) {
docs.add(new Document("lastname",ln));
}
Document queryLN = new Document(
"$or", Arrays.asList(docs)
);
docs.clear();
for (Integer fn: input.getFirstnames()) {
docs.add(new Document("firstname",fn));
}
Document queryFN = new Document(
"$or", Arrays.asList(docs)
);
Document query = new Document(
"$and", Arrays.asList(queryFN,queryLN));
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());

You should use an "in" query in such condition when you have a long unknown list of OR conditions.
An example code:
try {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("so");
DBCollection coll = db.getCollection("employees");
List<Integer> ageList = new ArrayList<>();
ageList.add(30);
ageList.add(35);
List<String> nameList = new ArrayList<>();
nameList.add("Anna");
BasicDBObject query = new BasicDBObject("$and", Arrays.asList(
new BasicDBObject("age", new BasicDBObject("$in", ageList)),
new BasicDBObject("name", new BasicDBObject("$in", nameList)))
);
DBCursor cursor = coll.find(query);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
}catch (Exception ex){
ex.printStackTrace();
}
To experiment with the above code, you can add the following entries in your MongoDB:
db.employees.insert({"name":"Adma","dept":"Admin","languages":["german","french","english","hindi"],"age":30, "totalExp":10});
db.employees.insert({"name":"Anna","dept":"Admin","languages":["english","hindi"],"age":35, "totalExp":11});
db.employees.insert({"name":"Bob","dept":"Facilities","languages":["english","hindi"],"age":36, "totalExp":14});
db.employees.insert({"name":"Cathy","dept":"Facilities","languages":["hindi"],"age":31, "totalExp":4});
db.employees.insert({"name":"Mike","dept":"HR","languages":["english", "hindi", "spanish"],"age":26, "totalExp":3});
db.employees.insert({"name":"Jenny","dept":"HR","languages":["english", "hindi", "spanish"],"age":25, "totalExp":3});
The above code produces this query:
db.employees.find({"$and":[{"age":{"$in":[30, 35]}},{"name":{"$in":["Anna"]}}]});
And the output is:
{ "_id" : { "$oid" : "57ff3e5e3dedf0228d4862ad"} , "name" : "Anna" , "dept" : "Admin" , "languages" : [ "english" , "hindi"] , "age" : 35.0 , "totalExp" : 11.0}
A good article on this topic: https://www.mkyong.com/mongodb/java-mongodb-query-document/
Read these as well: https://stackoverflow.com/a/8219679/3896066 and https://stackoverflow.com/a/14738878/3896066

Let's first understand your code. We will make it simple by replacing the for loops with simple statements and add some print statements.
List<Document> docs = new ArrayList<>();
docs.add(new Document("lastname","Walker"));
docs.add(new Document("lastname","Harris"));
Document queryLN = new Document("$or", Arrays.asList(docs));
docs.clear();
System.out.println(queryLN.toJson());//{ "$or" : [[]] }
docs.add(new Document("firstname", "Pat"));
docs.add(new Document("firstname", "Matt"));
Document queryFN = new Document("$or", Arrays.asList(docs));
System.out.println(queryLN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
System.out.println(queryFN.toJson());//{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());//{ "$and" : [{ "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }, { "$or" : [[{ "firstname" : "Pat" }, { "firstname" : "Matt" }]] }] }
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Observations:
docs is already a list. Using Arrays.asList on docs, creates a list of list, which is not acceptable for the $and, $or, $nor. These operators accept a list of Documents. That explains the error message.
Arrays.asList does not create a copy of the array or the list that it receives. It just creates a wrapper over it. Also, new document() does not copy the list that it receives with "$or", just references the original list. Hence, calling docs.clear() will reset the content in $or of queryLN.
Also, the above concept explains why the 2nd and 3rd print statements give the same output.
Let us get the code working now.
List<Document> docsLN = new ArrayList<Document>();
List<Document> docsFN = new ArrayList<Document>();
for (Integer ln: input.getLastnames()) {
docsLN.add(new Document("lastname",ln));
}
Document queryLN = new Document("$or", docsLN);
for (Integer fn: input.getFirstnames()) {
docsFN.add(new Document("firstname",fn));
}
Document queryFN = new Document("$or", docsFN);
System.out.println(queryLN.toJson());
System.out.println(queryFN.toJson());
Document query = new Document("$and", Arrays.asList(queryFN, queryLN));
System.out.println(query.toJson());
List<Document> result = collectionMain.find(query).into(new ArrayList<Document>());
Also, consider replacing the $or with $in.
From the docs:
When using $or with that are equality checks for the
value of the same field, use the $in operator instead of the $or
operator.

Mongo : count number of array elements for each entry in collection

Example Date (collection name 'people')
{"person" : "Frank",
"coords" : [
{
"point":new GLatLng(40.266044,-74.718479),
},
{
"point":new GLatLng(40.211600,-74.695702),
}
] }
{"person : ""Bob",
"coords" : [
{
"point":new GLatLng(40.266044,-74.718479),
},
{
"point":new GLatLng(40.211600,-74.695702),
},
{
"point":new GLatLng(40.294535,-74.682012),
}
] }
Goal : obtain a total number of 'coord' entries for each person document (accumulated overall)
Expected result 5
What I've tried
public int getTotalCoordinateCount(){
BasicDbObject project= new BasicDBObject()
.append("count",new BasicDBObject("$size","coords"))
AggregationOutput output = db.getPeopleCollection().aggregate(Arrays. <DBObject>asList(
new BasicDBObject("$project", project);
BasicDbObject result = (BasicDBObject)output.results().iterator.next();
return result.getInt("count");
}
Problem I get : 2
Clueless on what I'm doing wrong in the mongo aggregation framework. I suspect i need to include a $sum but not sure where

You need to do an unwind operation on coords field, in the aggregation pipeline. After that you do a group by Null with a count field, to count the results set and you will get the total number of elements from all arrays on all documents.

public int getTotalCoordinateCount(){
BasicDbObject group = new BasicDBObject()
.append("_id", 1)
.append("count", new BasicDBObject("$sum", 1));
BasicDbObject project= new BasicDBObject()
.append("_id", 0)
.append("count", 1);
AggregationOutput output = db.getPeopleCollection().aggregate(Arrays. <DBObject>asList(
new BasicDBObject("$unwind", "coords"),
new BasicDBObject("$group", group),
new BasicDBObject("$project", project)
);
BasicDbObject result = (BasicDBObject)output.results().iterator.next();
return result.getInt("count");
}

How to calculate unique elements in collections?

I have collection like
id : 1, url : youtube.com
also one url can be many times,
I need get whole collections and count unique elements
like
youtube 10
google 8
lycos 5
here is the code
public List<URLEntity> findAll() {
List<URLEntity> list = new ArrayList<URLEntity>();
String sort = "searchDate";
String order = "desc";
DBObject sortCriteria = new BasicDBObject(sort, "desc".equals(order) ? -1 : 1);
BasicDBObject query = new BasicDBObject();
DBCursor cursor = mongoCoreService.getDomainCollection().find(query).sort(sortCriteria);
try {
while (cursor.hasNext()) {
DBObject document = cursor.next();
URLEntity entity = new URLEntity();
entity = Converter.toObject(URLEntity.class, document);
list.add(entity);
}
} finally {
cursor.close();
}
return list;
}
thanks

You can achieve this by using the aggregate framework from MongoDB.
db.yourcollection.aggregate({ $group: { _id: '$url', total: {$sum: 1} }})
You'll obtain something like this, that you can manipulate in JAVA:
{
"result" : [
{
"_id" : "youtube.com",
"total" : 10
},
{
"_id" : "google.com",
"total" : 8
},
{
"_id" : "lycos",
"total" : 5
}
],
"ok" : 1
}

Today I found and test answer in java, here is a code
DBCollection mycoll= db.getCollection("domain");
DBObject fields = new BasicDBObject("domain", 1);
DBObject project = new BasicDBObject("$project", fields );
// Now the $group operation
DBObject groupFields = new BasicDBObject( "_id", "$domain");
groupFields.put("total", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields);
// run aggregation
AggregationOutput output = mycoll.aggregate(project, group);
System.out.println(output.getCommand().toString());
for (DBObject dbObject : output.results()) {
System.out.println(dbObject);
}

Insert Array inside an object in MongoDB

I am new at MongoDB and I want to insert to mongodb data like this but I couldn't figure out how
{
image = "cab"
tags = [
[ "NNP", 0 ],
[ "NN", 1 ]
]
},
{
image = "castle"
tags = [
[ "NNP", 2 ],
[ "NN", 1 ],
]
}
my code is
BasicDBObject obj = new BasicDBObject();
obj.put("images", ....);
for(Tag tag:tags){
BasicDBObject tagsObj = new BasicDBObject();
tagsObj.put("NNP",tag.getNNP());
tagsObj.put("NN",tag.getNN());
obj.put("tags",tagsObj);
}
UPDATE:
using this code
Mongo m = new Mongo();
DB db = m.getDB("test");
DBCollection coll = db.getCollection("tags");
for(Tag tag:tags){
BasicDBList dbl = new BasicDBList();
dbl.add(new BasicDBObject("NNP",tag.getNNP()));
dbl.add(new BasicDBObject("NN", tag.getNNP()));
BasicDBObject outer=new BasicDBObject("images", currentImageName).append("tags", dbl);
coll.insert(outer);
}
I store every image alone cause the tags might be like this for the same image
{
image = "cab",
tags = [
{ "NNP", 0 },
{ "NN", 1 }
],
[ {"NNP", 4 },
{ "NN", 5 }
],
[
{"NNP", 0 },
{ "NN", 4 }
]
},
Thanks

Basically you use BasicDBObject for key-value mappings and BasicDBList for array objects. For the object in your question, you'd do this:
BasicDBList dbl = new BasicDBList();
dbl.add(new BasicDBObject("NNP",0));
dbl.add(new BasicDBObject("NN", 1));
BasicDBOBject outer=new BasicDBObject("image", "cab").append("tags", dbl);
There's some convenience methods in the api to make this a bit less verbose.
The mapping works like this:
for: {"A":1} use: new BasicDBObject("A",1)
for: {"A":1, "B":2} use: new BasicDBObject("A",1).append("B",2)
for: {"A":{"B":2}} use: new BasicDBObject("A",new BasicDBObject("B",2))
for: {"A":["B","C"]} use:
BasicDBList dbl = new BasicDBList();
dbl.add("B");
dbl.add("C");
-> new BasicDBObject("A",dbl);

Did you mean like this?
BasicDBObject obj = new BasicDBObject();
obj.put("image", ....);
for(Tag tag:tags){
BasicDBObject tagsObj = new BasicDBObject();
tagsObj.put("NNP",tag.getNNP());
tagsObj.put("NN",tag.getNN());
obj.put("tags",tagsObj);
}

Here is how i use it when using mongo3.x:
suppose you want the result to be like this: {"data": [{"key":"v1"}, {"key":"v1"}, {"key":"v1"}] }
[step1]: use Java Map to create json object which maps to the elements inside the array; that is, the {} inside []
[step1 Ans]: Map m1,m2,m3 = new HashMap(); m1.put("key", "v1"); m2.put("key", "v1"); m3.put("key", "v1");
[step2]: use Java List to add all Java Map into one element.
[step2 Ans]: List list = new ArrayList(); list.add(m1); list.add(m2); list.add(m3);
[step3]: add Java list into mongo
[step3 Ans]: Document dc = new Document("key", list);