We Keep Coding

Mongodb inner join in java spring using AggregationOperation :Error [The 'cursor' option is required, except for aggregate with the explain argument]

I am excecuting following mongodb query I am new to mongo db ,please tell me what i am doing wrong
db.entityCounter.aggregate([
{
$lookup:
{
from: "fields",
localField: "code",
foreignField: "fieldCode",
as: "fieldsresult"
}
},{
$match:{
$and: [{
"fieldsresult.isVisible":"1"
},{"type":"field"
}]
}
}])
below is java spring code
LookupOperation lookupOperation = LookupOperation.newLookup()
.from("fields")
.localField("code")
.foreignField("fieldCode")
.as("fieldsresult");
AggregationOperation match1 = Aggregation.match(Criteria.where("fieldsresult.isVisible").is("1"));
// AggregationOptions aggregationOptions = Aggregation.newAggregationOptions();
DBObject ob=new BasicDBObject();
((BasicDBObject) ob).put("batchSize",10);
Aggregation aggregation = Aggregation.newAggregation(lookupOperation,match1).withOptions(Aggregation.newAggregationOptions().cursor(ob).build());
long val=0;
try {
AggregationResults<EntityCounter> result = mongoOperations.aggregate(aggregation, Fields.class, EntityCounter.class);
// val= result.getMappedResults();
}catch (Exception e){
e.printStackTrace();
}
but I am getting below error
org.springframework.dao.InvalidDataAccessApiUsageException: Command execution failed: Error [The 'cursor' option is required, except for aggregate with the explain argument], Command = { "aggregate" : "entityCounter" , "pipeline" : [ { "$match" : { "fieldsresult.isVisible" : "1"}} , { "$lookup" : { "from" : "fields" , "localField" : "code" , "foreignField" : "fieldCode" , "as" : "fieldsresult"}}]}; nested exception is com.mongodb.MongoCommandException: Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }

The lookup was introduced in mongodb 3.4 please upgrade your dB

Mongodb Java query for date range

I needed to find all the records in mongo db within two date ranges using Mongo Driver[3.4.0] for Java.
Example:
I have books Collection.
{
"_id" : ObjectId("5acb40d27d63b61cb002bafe"),
"title" : "WingsOfFire",
"pub-date" : ISODate("2013-10-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-11-02T00:00:00.000Z")
}
Like above I have 100s of documents.
I need to find all records wherein pub-date > rel-date.
I am using Mongo DB version 3.2.6
I tried to use $expr operator but it seems to work with only Mongo 3.6+
Not able find cleaner solutions for above requirement.
Please clarify.

The MongoDB (prior to v3.4) shell command for your use case is:
db.collection.aggregate([
{
"$redact": {
"$cond": [
{ "$gt": [ "$pub-date", "$rel-date" ] },
"$$KEEP",
"$$PRUNE"
]
}
}
])
Translating this command into Java you'll get:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
List<Document> documents = collection.aggregate(Arrays.asList(
new Document("$redact", new Document("$cond",
Arrays.asList(new Document("$gt", Arrays.asList("$pub-date", "$rel-date")), "$$KEEP", "$$PRUNE"))
))).into(new ArrayList<>());
for (Document document : documents) {
System.out.println(document.toJson());
}
Given a collection with these documents ...
{
"_id" : ObjectId("5acb40d27d63b61cb002bafe"),
"title" : "WingsOfFire",
"pub-date" : ISODate("2013-10-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-11-02T00:00:00.000Z")
}
{
"_id" : ObjectId("5acb662756539a6734e64e4a"),
"title" : "WingsOfSmoke",
"pub-date" : ISODate("2013-11-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-10-02T00:00:00.000Z")
}
.. the above Java code will print ...
{ "_id" : { "$oid" : "5acb662756539a6734e64e4a" }, "title" : "WingsOfSmoke", "pub-date" : { "$date" : 1383350400000 }, "rel-date" : { "$date" : 1380672000000 } }
... because this document's pub-date (2013-11-02T00:00:00.000Z) is after its rel-date (2013-10-02T00:00:00.000Z).
Note: the $where operator is functionally equivalent but use of that operator comes with some limitations:
$where evaluates JavaScript and cannot take advantage of indexes. Therefore, query performance improves when you express your query using the standard MongoDB operators (e.g., $gt, $in).
In general, you should use $where only when you can’t express your query using another operator. If you must use $where, try to include at least one other standard query operator to filter the result set. Using $where alone requires a collection scan.

You might want to try $where-Operator:
db.books.find({ "$where": "this.pub-date > this.rel-date"});

how to get document comparing two felids of same document (nested fields) mongodb

{
"_id" : ObjectId("5a3218bbdcb7c38fd3731232"),
"json" : {
"query_by_gtin_response:queryByGtinResponse" : {
"xmlns" : "urn:gs1:tsd:query_by_gtin_response:xsd:1",
"productData" : {
"productDataRecord" : [
{
"module" : [
{
"product_tracking_information_module" : {
"createdDate" : "2017-12-13T13:30:08.297Z",
"updatedDate" : "2017-12-13T13:30:08.297Z",
}
},
]
}
]
}
}
},
get above document if createdDate = updatedDate.I have used $where but i think because of nesting it is not working.Can you please suggest how to get this document .I am building query using java.

On the $where filter use the this or obj keyword to access the current document properties and compare those values:
this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.module.product_tracking_information_module.createDate == this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.module.product_tracking_information_module.updateDate
EDIT
The $where filter expects a javascript expression or a function that returns a boolean. With this and since you are using arrays, functions some and every can be used on those arrays.
Let's suppose that we have a collection T like this:
a: {b : [{c: Date, d: Date}]}
If we want documents that at least have one match element we use some like this:
db.T.find({$where: "this.a.b.some((el) => el.c.valueOf() == el.d.valueOf())"})
If all elements of the array have to match the condition, we use every:
db.T.find({$where: "this.a.b.every((el) => el.c.valueOf() == el.d.valueOf())"})
Now, you collection has a nested array. With these, 4 posibilitys are present:
1: Select all documents that at least have one createdDate == updatedDate, no matter in wich "productDateRecord" element it is.
db.T.find({$where: "this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.some((outside) => outside.module.some((inside) inside.createdDate.valueOf() == inside.updatedDate.valueOf()))"})
2: Select all documents where all "productDateRecord" object have at least one createdUpdate == updatedDate, no matter in wich "module" it is.
db.T.find({$where: "this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.every((outside) => outside.module.some((inside) => inside.createdDate.valueOf() == inside.updatedDate.valueOf()))"})
3: Select all documents that at least have one "productDateRecord" where all "module" elements have createdUpdate == updatedDate
db.T.find({$where: "this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.some((outside) => outside.module.every((inside) => inside.createdDate.valueOf() == inside.updatedDate.valueOf()))"})
4: Select all documents where every "productDateRecord" have all "module" elements with createdUpdate == updatedDate
db.T.find({$where: "this.json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.every((outside) => outside.module.every((inside) => inside.createdDate.valueOf() == inside.updatedDate.valueOf()))"})

The following code gives me output 5a3218bbdcb7c38fd3731232 :
MongoClient mongoClient = new MongoClient(new MongoClientURI("mongodb://localhost:27017"));
DB database = mongoClient.getDB("test");
DBObject query = new BasicDBObject("json.query_by_gtin_response:queryByGtinResponse.productData.productDataRecord.module.product_tracking_information_module.updatedDate", "2017-12-13T13:30:08.297Z");
DBCollection collection = database.getCollection("testData");
DBCursor cursor = collection.find(query);
DBObject jo = cursor.one();
System.out.println((String)cursor.one().get("_id"));
Note that on the db console I ran the following commands:
use test
t = { "_id" : "5a3218bbdcb7c38fd3731232", "json" : { "query_by_gtin_response:queryByGtinResponse" : { "xmlns" : "urn:gs1:tsd:query_by_gtin_response:xsd:1", "productData" : { "productDataRecord" : [ { "module" : [ { "product_tracking_information_module" : { "createdDate" : "2017-12-13T13:30:08.297Z", "updatedDate" : "2017-12-13T13:30:08.297Z" } } ] } ] } } } }
db.testData.insert( t );
I am using a maven project with following entry in pom.xml:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.6.1</version>
</dependency>

Morphia query date range

What is the equivalent of the following Mongo query in Morphia?
db.events.find({ "date": { $gte: ISODate("2001-01-01") , $lt: ISODate("2001-01-02")} })
Currently I have the following code
Query<Event> query = dataStore.find(Event.class);
query.field("date").greaterThanOrEq(startDate).field("date").lessThan(endDate);
but it results in the following Mongo query
{ "$and" : [ { "date" : { "$gte" : { "$date" : "2001-01-01T00:00:00.000Z"}}} , { "date" : { "$lt" : { "$date" : "2001-01-02T00:00:00.000Z"}}}]}
I suppose the end result is the same, but the resulting query is more verbose.

Use criteria with add method
Something like
Query<Event> query = datastore.find(Event.class);
query.criteria("date").greaterThanOrEq(startDate).add(query.criteria("date").lessThan(endDate));

You need to create a query then add date range condition like followed.
Query<Event> queryForEvent = ds.createQuery(Event.class);
queryForEvent.field("date").greaterThanOrEq(startDate);
queryForEvent.field("date").lessThan(endDate);
List<Event> eventList = queryForEvent.asList();
Hoping you will find it useful.

MongoDB Java Driver: Multiple Date query

I am trying to create a query using MongoDB Java Driver as part of an aggregation command. Currently I allow a date range or an array of specific dates as an argument. eg
<date>
<start>2013-12-10 00:00:00.000</start>
<end>2013-12-12 23:59:59.999</end>
</date>
or
<date>
<specificDates>2013-12-10 00:00:00.000,2013-12-13 00:00:00.000</specificDates>
</date>
The date range query works fine, I parse and convert the xml into a DBObject that produces the following query in mongo;
{ "$match" : { "d" : { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}}}
For the specificDates I want to return only results that occur between 00:00:00.000 on the given day and 00:00:00.000 of the next day. From my pretty basic knowledge of mongo querys i had hoped to do a similar $match as the date range, but have it use $in on an array of date ranges similar to the following;
{ "$match" : { "d" : { "$in" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}
The above query fails to return any results. I have noticed that $in is not listed in the mongodb manual under the Mongo Aggregation Framework section, but its not throwing any kind of errors that I would have expected for an unsupported operation.
I think the issue may come from this line in the MongoDB Manual;
If the field holds an array, then the $in operator selects the documents whose field holds an array that contains at least one element that matches a value in the specified array (e.g. , , etc.)
In my collection the date isn't stored in an array, I suppose I could store it in the collections in an single element array? (Actually, decided to try this quickly before I posted, no documents returned when the date entry in the document is stored in a single element array)
Document entry example
{ "_id" : ObjectId("52aea5b0065991de1a56d5b0"), "d" : ISODate("2013-12-15T00:00:11.088Z"), "t" : 1501824, "s" : 0, "e" : 601, "tld" : "uk", "y" : "domain:check", "n" : "removed.co.uk" }
Is anyone able to give me some advice as to how I should do this query? Thank you.
EDIT: I left the Java tag here in case anyone needs my DBObject creation code, though it shouldn't be necessary as the queries posted have been generated by my build.
EDIT2: So as Alan Spencer pointed out I should be using $or rather than $in, a working $or function is below (ignore the different formatting like the use of ISODate(), its just copy pasted from the mongo shell rather than getting output from my program)
{ $match : { $or : [ { d : { $gte : ISODate("2013-10-01T00:00:00.000Z"), $lt : ISODate("2013-10-02T00:00:00.000Z") } }, { d : { $gte : ISODate("2013-10-03T00:00:00.000Z"), $lt : ISODate("2013-10-04T00:00:00.000Z") } } ] } }

I think you're inverting the meaning of the $in.
$in is used to match exactly against a list of possible values, like
{"color":{"$in": ["red","green","blue"]}}
For your use case, you are trying to match if it satisfies the first or second, etc. So, you can use $or - http://docs.mongodb.org/manual/reference/operator/query/or/
{ "$match" : { "d" : { "$or" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}