We Keep Coding

"or" condition in embedded document java

What is the correct syntax for creating $or condition in embedded document in MongoDB Java driver?
Is it actually possible to get a cursor to embedded documents?
Suppose I have the following document:
{
statuses: [{
{
streamName: A
}{
statusA: 0
}{
statusB: 1
}
},
{
{
streamName: B
}{
statusA: 0
}{
statusB: 1
}
}]
}
I would also like to get cursor to sub documents (in array of statuses) that has at least one status bigger than 0.
This is how I did it but it didn't work:
List<BasicDBObject> obj = new ArrayList<BasicDBObject>();
DBObject query = new BasicDBObject();
obj.add(new BasicDBObject ("statuses",
new BasicDBObject ("statusA",
new BasicDBObject ("$gt",0 ) )));
obj.add(new BasicDBObject ("statuses",
new BasicDBObject ("statusB" ,
new BasicDBObject ("$gt",0 ) )));
query.put("$or",obj)
db.find(collectionName,query)
I didn't find any documentation on that.

What you have translates to
{ "$or" : [ { "statuses" : { "statusA" : { "$gt" : 0}}} , { "statuses" : { "statusB" : { "$gt" : 0}}}]}
which is used for whole document comparison.
For comparing fields inside an embedded arrays/doc you've to use dot notation.
{ "$or" : [ { "statuses.statusA" : { "$gt" : 0}} , { "statuses.statusB" : { "$gt" : 0}}]}
The equliavent java code is below
List<BasicDBObject> obj = new ArrayList<BasicDBObject>();
DBObject query = new BasicDBObject();
obj.add(new BasicDBObject ("statuses.statusA", new BasicDBObject ("$gt",0 ) ));
obj.add(new BasicDBObject ("statuses.statusB" , new BasicDBObject ("$gt",0 ) ));
query.put("$or",obj);
Alternatively you can use $elemMatch to run matches on embedded arrays. Similar to what you've but $elemMatch applies condition to each fields.
Something like
{ "statuses" : { "$elemMatch" : { "$or" : [ { "statusA" : { "$gt" : 0}} , { "statusB" : { "$gt" : 0}}]}}}
Java Code
BasicDBList obj = new BasicDBList();
obj.add(new BasicDBObject ("statusA",new BasicDBObject ("$gt",0 ) ));
obj.add(new BasicDBObject ("statusB",new BasicDBObject ("$gt",0 ) ));
DBObject query = new BasicDBObject("statuses", new BasicDBObject("$elemMatch", new BasicDBObject("$or",obj)));
Count the no of matching occurrences.
Bson count = new Document("statuses", Document.parse("{$size:{\n" +
" $filter: {\n" +
" input: \"$statuses\",\n" +
" as: \"status\",\n" +
" cond: { \"$or\" : [ {$gt:[\"$$status.statusA\", 0]} , {$gt:[\"$$status.statusB\", 0]}]}\n" +
" }\n" +
" }}"));
Bson project = new Document("$project", count);
col.aggregate(Arrays.asList(project));

Get Average from Mongo Collection using Aggrerate

I am trying to learn how to use the Mongo in Java and have been able to make some simple queries but I have been having trouble with the aggregate operator.
The document structure is a simple one, as the following:
{
"_id" : ObjectId("57dbe94f0507a4d8710ac5b2"),
"name" : "Name1",
"age" : 23
}
{
"_id" : ObjectId("57dbe9750507a4d8710ac5b3"),
"name" : "Name2",
"age" : "String for examble"
}
{
"_id" : ObjectId("57dbee630507a4d8710ac5b5"),
"name" : "Name3",
"age" : 24
}
All I want to do is get the average of the ages in the collection ( name example ).
Simply using mongo I can get the desirable result with the following consult:
db.example.aggregate([
{
$group: {
_id: null,
averageAge: { $avg: "$age" }
}
}
]);
I have tried the following:
BasicDBObject groupFields = new BasicDBObject("_id", "null");
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("mediaIdade", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemplo").aggregate(Arrays.asList (group));
Which is almost a direct translation but got a "java.lang.NoSuchMethodError: org.bson.codecs.BsonTypeClassMap.keys()Ljava/util/Set;" , unsurprisingly.
But I cannot translate that to Java. I have checked and found this question but could not understand it due to the use of opperators such as $unwind. So I'm trying to make query as simple as possible to better understand how the Java framework for aggregation works.
Can someone help?

Try something like this.
MongoCollection<Document> dbCollection = db.getCollection("example", Document.class);
AggregateIterable<org.bson.Document> aggregate = dbCollection.aggregate(Arrays.asList(Aggregates.group("_id", new BsonField("averageAge", new BsonDocument("$avg", new BsonString("$age"))))));
Document result = aggregate.first();
double age = result.getDouble("averageAge");
Input:
{ "_id" : ObjectId("58136d6ed96cc299c224d529"), "name" : "Name1", "age" : 23 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52a"), "name" : "Name2", "age" : 26 }
{ "_id" : ObjectId("58136d6ed96cc299c224d52b"), "name" : "Name3", "age" : 24 }
Output:
24.333333333333332

I noticed a small typo in your code:
db.getCollection("exemplo").aggregate(Arrays.asList (group));
Should be example instead of examplo
Also the easiest way to translate working mongo expression is using Document.parse method.
In your case it could be:
db.getCollection("exemplo").aggregate(Arrays.asList(
Document.parse("{ $group: { _id: null, averageAge: { $avg: '$age' } } }")
));
Your impl also almost correct, two minor issues:
replace "null" string with just null value
Use "averageAge" instead of "mediaIdade"
BasicDBObject groupFields = new BasicDBObject("_id", null);
BasicDBObject media = new BasicDBObject("$avg", "$age");
groupFields.put("averageAge", media);
BasicDBObject group = new BasicDBObject("$group", groupFields );
AggregateIterable<org.bson.Document> agregate = db.getCollection("exemple").aggregate(Arrays.asList (group));
To get result:
agregate.first().getDouble("averageAge");

Try this:
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(group);
Iterable<DBObject> list = output.results();
If required add a filter to the query you can add a match parameter:
DBObject match = new BasicDBObject();
match.put("age", new BasicDBObject("$gte", 25));
DBObject groupFields = new BasicDBObject( "_id", 0);
groupFields.put("average", new BasicDBObject( "$avg", "$age"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = db.getCollection("exemplo").aggregate(match, group);
Iterable<DBObject> list = output.results();

How do I group the results of mongodb query and implement in Java?

I have a collection in a Mongo database that looks like this:
{
"_id" : ObjectId("561b42d4e4b0d4227d011d2c"),
"product_related_data" : {
"depth" : 6,
"height" : 23,
"product_barcode" : "54491472",
"product_name" : "xyz product",
"product_uuid" : "009b9846-b3ad-49f7-a7a0-d35a04f83480",
"width" : 6
},
"sensostrip_data" : {
"barcode" : "130150208299",
"battery_level" : 2.894
},
"stock_related_data" : {
"calculated_max_product_percentage" : 15.625,
"calculated_min_product_percentage" : 12.5,
"current_stock" : 2,
"max_stock" : 6,
"stock_difference_percentage" : 0,
"stock_difference_value" : 0,
"stock_percentage" : 37.5
},
"store_data" : {
"city_uuid" : "cbb4dfe8-172b-11e4-a1f0-00163ed23ec2",
"ip" : "10.0.1.1",
"retailer_uuid" : "8c33c32c-5903-11e4-a1f0-00163ed23ec2",
"store_name" : "xyz store",
"store_uuid" : "15a6cc90-081f-11e5-b213-001e6745ff8d"
},
"time" : {
"check_date" : "2015-10-11 11:53:55",
"previous_check_date" : "2015-10-11 11:48:57"
},
"id" : "6be54bef-0aa3-456c-b912-1731f8154e7d"
}
The mongo query I'm currently executing returns all the documents for a list of store_uuid's and for a product_uuid, and looks like this:
db.readings
.find({ $and:[ {"store_data.store_uuid": {$in:["15a6cc90-081f-11e5-b213-001e6745ff8d","217b983b-5904-11e4-a1f0-00163ed23ec2","5337d78d-5904-11e4-a1f0-00163ed23ec2"]}}
,{"product_related_data.product_uuid": "f44aa29d-09ce-4902-bf12-d45d44b3dfd0"}]})
My current Java implementation (where I make use of projection) looks like this:
DBCollection table = databaseConncetion().getCollection("readings");
BasicDBObject sensorReturn = new BasicDBObject("sensostrip_data.barcode",1);
BasicDBObject clause1 = new BasicDBObject("store_data.store_uuid", new BasicDBObject("$in", StoreIds));
BasicDBObject clause2 = new BasicDBObject("product_related_data.product_uuid", productId);
BasicDBList and = new BasicDBList();
and.add(clause1);
and.add(clause2);
DBObject query = new BasicDBObject("$and", and);
DBCursor cursor = table.find(query, sensorReturn
.append("stock_related_data.stock_percentage",1)
.append("store_data.store_uuid",1)
.append("time.check_date", 1))
.sort(new BasicDBObject("time.check_date", -1))
.limit(100);
However I need this query to group the results for the latest check_date by barcode

The aggregation framework is at your disposal. Running the following aggregation pipeline will give you the desired result:
Mongo shell:
pipeline = [
{
"$match": {
"store_data.store_uuid": {
"$in": [
"15a6cc90-081f-11e5-b213-001e6745ff8d",
"217b983b-5904-11e4-a1f0-00163ed23ec2",
"5337d78d-5904-11e4-a1f0-00163ed23ec2"
]
},
"product_related_data.product_uuid": "f44aa29d-09ce-4902-bf12-d45d44b3dfd0"
}
},
{ "$sort": { "time.check_date": -1 } },
{
"$group": {
"_id": "$sensostrip_data.barcode",
"stock_percentage": { "$first": "$stock_related_data.stock_percentage" },
"store_uuid": { "$first": "$store_data.store_uuid" },
"check_date": { "$first": "$time.check_date" }
}
},
{ "$limit": 100 }
];
db.readings.aggregate(pipeline);
Java test implementation
public class JavaAggregation {
public static void main(String args[]) throws UnknownHostException {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("test");
DBCollection coll = db.getCollection("readings");
// create the pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match",
new BasicDBObject("store_data.store_uuid", new BasicDBObject("$in", StoreIds))
.append("product_related_data.product_uuid", productId)
);
// sort pipeline
DBObject sort = new BasicDBObject("$sort",
new BasicDBObject("time.check_date", -1)
);
// build the $group operations
DBObject groupFields = new BasicDBObject( "_id", "$sensostrip_data.barcode"); // group by barcode
groupFields.put("stock_percentage", new BasicDBObject( "$first", "$stock_related_data.stock_percentage")); // get the first when ordered documents are grouped
groupFields.put("store_uuid", new BasicDBObject( "$first", "$store_data.store_uuid"));
groupFields.put("check_date", new BasicDBObject( "$first", "$time.check_date"));
// append any other necessary fields
DBObject group = new BasicDBObject("$group", groupFields);
// limit step
DBObject limit = new BasicDBObject("$limit", 100);
// put all together
List<DBObject> pipeline = Arrays.asList(match, sort, group, limit);
AggregationOutput output = coll.aggregate(pipeline);
for (DBObject result : output.results()) {
System.out.println(result);
}
}
}

Aggregation Query adjusting dates for timezone

I want to run this mongodb query using java code.
Query :
db.log.aggregate([
{ $match : {
"ts" : {
$gte: ISODate("2015-07-31T18:30:00.000Z"),
$lt: ISODate("2015-08-01T18:30:00.000Z")
},
"dup": {$exists:false}
}},
{ $project : {
'lts': {
'$add': ['$ts',5.5*3600*1000]
}
}},
{ $group : {
_id : {
day: { $dayOfMonth: "$lts" },
month: { $month: "$lts" },
year: { $year: "$lts" }
},
count: { $sum: 1 }
}}
])
i have tried with this code but its not worked
DBObject query = QueryBuilder.start().put("ts").greaterThanEquals(startdate).lessThanEquals(enddate).and("dup").exists(false).get();
DBObject match = new BasicDBObject("$match", query);
DBObject project=new BasicDBObject("$project", new BasicDBObject("ts",new BasicDBObject("$add",5.5*3600*1000)));
DBObject group = new BasicDBObject("$group", new BasicDBObject("_id",new BasicDBObject("day", new BasicDBObject("$dayOfMonth", "$ts"))
).append("count", new Document("$sum", 1)));
AggregationOutput output = collection.aggregate(match,group);
for (DBObject result : output.results()) {
System.out.println(result);
}

Your inital query could have been better written. Presumably you want to group results by "day" whilst also ajusting from UTC to a local timzezone. It usually is better to just stick with the date math rather than mix in the date aggregation operators in such a case, and also this means that the $project here is not required, and you can just go straight to $group:
db.log.aggregate([
{ "$match": {
"ts": {
"$gte": ISODate("2015-07-31T18:30:00.000Z"),
"$lt": ISODate("2015-08-01T18:30:00.000Z")
},
"dup": { "$exists": false }
}},
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [
{ "$add": [ "$ts", 5.5*1000*60*60 ] },
new Date(0)
]},
{ "$mod": [
{ "$subtract": [
{ "$add": [ "$ts", 5.5*1000*60*60 ] },
new Date(0)
]},
1000*60*60*24
]}
]},
new Date(0)
]
},
"count": { "$sum": 1 }
}}
])
Note the additional Date(0) statements in there. This is "epoch" and just like when you $add a numeric value to a date you get a Date type in return, when you $subtract you get the integer value. So correcting by adding back "epoch" makes everything a Date type again.
Hence the "date math" is employed everywhere, to "round" to a "day":
( 1000 millseconds * 60 seconds * 60 minutes * 24 hours )
And the modulo $mod operator works out the remainder of this which is subtracted from the current date value, to come to the "rounded" date.
All done in $group and efficiently as well:
Translating to Java is just following the same indentation:
Date startdate = new DateTime(2015, 7, 31, 18, 30, 0, DateTimeZone.UTC).toDate();
Date enddate = new DateTime(2015, 8, 1, 18, 30, 0, DateTimeZone.UTC).toDate();
DBObject query = QueryBuilder.start()
.put("ts").greaterThanEquals(startdate)
.lessThan(enddate)
.and("dup").exists(false).get();
DBObject match = new BasicDBObject("$match",query);
DBObject group = new BasicDBObject("$group",
new BasicDBObject("_id",
new BasicDBObject(
"$add", Arrays.asList(
new BasicDBObject(
"$subtract", Arrays.asList(
new BasicDBObject(
"$subtract", Arrays.asList(
new BasicDBObject(
"$add",Arrays.asList( "$ts", 5.5*1000*60*60 )
),
new Date(0)
)
),
new BasicDBObject(
"$mod", Arrays.asList(
new BasicDBObject(
"$subtract", Arrays.asList(
new BasicDBObject(
"$add",Arrays.asList( "$ts", 5.5*1000*60*60 )
),
new Date(0)
)
),
1000*60*60*24
)
)
)
),
new Date(0)
)
)
)
.append("count", new BasicDBObject("$sum",1))
);
AggregationOutput output = collection.aggregate(Arrays.asList(match, group));
for ( DBObject result : output.results() ) {
System.out.println(result);
}
Also note that your "timezeone adjustment" here is presuming 5.5 hours "behind" UTC, which I hope is correct. If you mean "after" or positive, then that operation is a "subtraction" and not an addition to correct the UTC hours to days.
So all is good, and your grouping keys are real Date objects as well, rather than the composite values returned by date aggregation operators.

change field ts to east 8 timezone:
BasicDBObject addEightHour = new BasicDBObject("$add", Arrays.asList( "$ts", 8*1000*60*60 ));
final BasicDBObject day = new BasicDBObject("$dateToString", new BasicDBObject("format", "\"%Y%m%d\"").append("date", addEightHour));
DBObject groupFields = new BasicDBObject("_id", new BasicDBObject(
"value", day));
DBObject group = new BasicDBObject("$group", groupFields);

Retrieve all match records from mongodb depend on multiple condition

I have the my mongodb collection schema in following format
{
"_id" : 1,
"sid" : 11,
"shapes" : [
{"shape" : "square", "color" : "red"},
{"shape" : "circle", "color" : "green"},
{"shape" : "rectangle", "color" : "green"},
......,
......,
{"shape" : "elipse", "color" : "green"}
]
},
........,
........,
{
"_id" : 100
"sid" : 111,
"shapes" : [
{"shape" : "square", "color" : "red"},
{"shape" : "circle", "color" : "green"},
......,
{"shape" : "rectangle", "color" : "green"}
]
}
And I want to Retrieve record from it where sid = 11 and shape like %r% using java driver.
I used the following code but it give me only first record ,please suggest me what I doing wrong ?
DBObject query = new BasicDBObject("sid", 1);
DBObject searchQuery = new BasicDBObject();
Pattern regex = Pattern.compile(".*r.*");
searchQuery.put("shape", regex);
DBObject elemMatchQuery = new BasicDBObject("$elemMatch", searchQuery);
DBObject fields = new BasicDBObject();
fields.put("shapes", elemMatchQuery);
DBCursor cursor = collection.find(query, fields);
System.out.println(cursor.count());
while (cursor.hasNext()) {
System.out.println(cursor.next());
}

Use mongo aggregation query as below :
db.collectionName.aggregate({"$match":{"sid":11}},{"$unwind":"$shapes"},{"$match":{"shapes.shape":{"$regex":".r."}}})
and equivalent java code as :
BasicDBObject match = new BasicDBObject("sid",11);
BasicDBObject firstmatchObj = new BasicDBObject();
firstmatchObj.put("$match", match);
BasicDBObject unwind = new BasicDBObject("$unwind","$shapes");
BasicDBObject matchAfterUnwind = new BasicDBObject("shapes.shape",new BasicDBObject("$regex",".r."));
BasicDBObject secondmatchObj = new BasicDBObject();
secondmatchObj.put("$match", matchAfterUnwind);
List<DBObject> pipeline = new ArrayList<>();
pipeline.add(firstmatchObj);
pipeline.add(unwind);
pipeline.add(secondmatchObj);
AggregationOutput output = collection.aggregate(pipeline);
for (DBObject res : output.results()) {
System.out.println(res);
}