I needed to find all the records in mongo db within two date ranges using Mongo Driver[3.4.0] for Java.
Example:
I have books Collection.
{
"_id" : ObjectId("5acb40d27d63b61cb002bafe"),
"title" : "WingsOfFire",
"pub-date" : ISODate("2013-10-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-11-02T00:00:00.000Z")
}
Like above I have 100s of documents.
I need to find all records wherein pub-date > rel-date.
I am using Mongo DB version 3.2.6
I tried to use $expr operator but it seems to work with only Mongo 3.6+
Not able find cleaner solutions for above requirement.
Please clarify.
The MongoDB (prior to v3.4) shell command for your use case is:
db.collection.aggregate([
{
"$redact": {
"$cond": [
{ "$gt": [ "$pub-date", "$rel-date" ] },
"$$KEEP",
"$$PRUNE"
]
}
}
])
Translating this command into Java you'll get:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
List<Document> documents = collection.aggregate(Arrays.asList(
new Document("$redact", new Document("$cond",
Arrays.asList(new Document("$gt", Arrays.asList("$pub-date", "$rel-date")), "$$KEEP", "$$PRUNE"))
))).into(new ArrayList<>());
for (Document document : documents) {
System.out.println(document.toJson());
}
Given a collection with these documents ...
{
"_id" : ObjectId("5acb40d27d63b61cb002bafe"),
"title" : "WingsOfFire",
"pub-date" : ISODate("2013-10-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-11-02T00:00:00.000Z")
}
{
"_id" : ObjectId("5acb662756539a6734e64e4a"),
"title" : "WingsOfSmoke",
"pub-date" : ISODate("2013-11-02T00:00:00.000Z"),
"rel-date" : ISODate("2013-10-02T00:00:00.000Z")
}
.. the above Java code will print ...
{ "_id" : { "$oid" : "5acb662756539a6734e64e4a" }, "title" : "WingsOfSmoke", "pub-date" : { "$date" : 1383350400000 }, "rel-date" : { "$date" : 1380672000000 } }
... because this document's pub-date (2013-11-02T00:00:00.000Z) is after its rel-date (2013-10-02T00:00:00.000Z).
Note: the $where operator is functionally equivalent but use of that operator comes with some limitations:
$where evaluates JavaScript and cannot take advantage of indexes. Therefore, query performance improves when you express your query using the standard MongoDB operators (e.g., $gt, $in).
In general, you should use $where only when you can’t express your query using another operator. If you must use $where, try to include at least one other standard query operator to filter the result set. Using $where alone requires a collection scan.
You might want to try $where-Operator:
db.books.find({ "$where": "this.pub-date > this.rel-date"});
Related
I'm needing to retrieve just with two dates, all the documents from my MongoDB's collection, with the filtered items from the array.
This is an example of 2 of my documents;
{
"_id" : ObjectId("5f18fa823406b7000132d097"),
"last_date" : "22/07/2020 23:48:32",
"history_dates" : [
"22/07/2020 23:48:32",
"22/07/2020 00:18:53",
"23/07/2020 00:49:12",
"23/07/2020 01:19:30"
],
"hostname" : "MyHostname1",
"ip" : "142.0.111.79",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "LogonUI.exe",
"date" : "23/07/2020 10:26:04",
}
{
"_id" : ObjectId("5f18fa823406b7000132d098"),
"last_date" : "22/07/2020 23:48:33",
"history_dates" : [
"22/07/2020 23:48:33",
"23/07/2020 00:18:53",
],
"hostname" : "MyHostName2",
"ip" : "142.0.111.54",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "svchost.exe",
"date" : "23/07/2020 10:26:04",
}
I'm needing to make a find to my database (Using Spring Data), to retrieve the same objects, but with the "history_dates"'s array filtered between the 2 dates recieved.
For example, if my 2 recieved dates are: "23/07/2020" and "24/07/2020", I want MongoDB to return the next objects;
{
"_id" : ObjectId("5f18fa823406b7000132d097"),
"last_date" : "22/07/2020 23:48:32",
"history_dates" : [
"23/07/2020 00:49:12",
"23/07/2020 01:19:30"
],
"hostname" : "MyHostname1",
"ip" : "142.0.111.79",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "LogonUI.exe",
"date" : "23/07/2020 10:26:04",
}
{
"_id" : ObjectId("5f18fa823406b7000132d098"),
"last_date" : "22/07/2020 23:48:33",
"history_dates" : [
"23/07/2020 00:18:53"
],
"hostname" : "MyHostName2",
"ip" : "142.0.111.54",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "svchost.exe",
"date" : "23/07/2020 10:26:04",
}
I'm really ignorant about MongoDB's queries, and I have been trying to make this with Spring Data all the week.
UPDATE 1.
Thanks varman, and do you know how can i just retrieve the documents with filtered arrays not empty?
So basically you need to do filter. MongoTemplate offers a lot of operation for mongodb, if some methods don't exist in MongoTemplate, we can go with Bson Document pattern. In that case, try this article: Trick to covert mongo shell query.
Actually you need a Mongo query something like following. Using $addFields one of the methods shown below. But you can use $project, $set etc. Here $addFields overwrites your history_dates. (It uses to add new fields to document too).
{
$addFields: {
history_dates: {
$filter: {
input: "$history_dates",
cond: {
$and: [{
$gt: ["$$this", "23/07/2020"]
},
{
$lt: ["$$this", "24/07/2020"]
}
]
}
}
}
}
}
Working Mongo playground.
You need to convert this into spring data. So #Autowired the MongoTemplate in you class.
#Autowired
MongoTemplate mongoTemplate;
The method is,
public List<Object> filterDates(){
Aggregation aggregation = Aggregation.newAggregation(
a->new Document("$addFields",
new Document("history_dates",
new Document("$filter",
new Document("input","$history_dates")
.append("cond",
new Document("$and",
Arrays.asList(
new Document("$gt",Arrays.asList("$$this","23/07/2020")),
new Document("$lt",Arrays.asList("$$this","24/07/2020"))
)
)
)
)
)
)
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_CLASS.class), Object.class).getMappedResults();
}
Mongo template doesn't provide add methods for $addFields and $filter. So we just go with bson document pattern. I haven't tested this in Spring.
suppose that I have mongoDB collection that has documents like below
{
"_id" : ObjectId("5cb2dd4d378a8e2484e7bb69"),
"name":"ali",
"creator" : {
"name":"user1",
"code":2,
"type" : "desk",
"type_id" : "desk::bb36640a-e384-45ec-aafa-ce71b724c389"
},
"creatorPosition" : {
"name":"user2",
"role":"admin",
"type" : "position",
"type_id" : "position::9f711bb3-3aad-4936-b75a-4ddd024b4bb1"
},
"creatorDetails" : {
"name":"user1",
"type" : "deskUserAssignment",
"type_id" : "deskUserAssignment::dd46fa14-0fd3-43d7-8312-471bc253ee80"
}
}
I want to keep two key values ("type", "type_id") from each fields and remove extra keys (such as "name", "code", "role")
Can I handle this in one mongo query?
In addition, I want to write this code in java
The simplest way to do is using Projection that MongoTemplate offers
For example including fields using org.springframework.data.mongodb.core.query.Query and org.springframework.data.mongodb.core.MongoTemplate for retrieving the results:
Query query = new Query();
query.fields().include("creator.type").include("creator.type_id");
query.fields().include("creatorPosition.type").include("creatorPosition.type_id");
query.fields().include("creatorDetails.type").include("creatorDetails.type_id");
List<Document> documents = mongoTemplate.find(query, Document.class, "your_collection");
For more information see this Article: https://www.mkyong.com/mongodb/spring-data-mongodb-select-fields-to-return/
I have this two documents in my mongoDB database:
db.DocumentFile.find().pretty()
{
"_id" : ObjectId("587f39910cc0fec092bdb10c"),
"_class" : "com.smartinnotec.legalprojectmanagement.dao.domain.DocumentFile",
"fileName" : "DocumentFile1",
"ending" : "jpg",
"projectId" : "587f39910cc0fec092bdb10b",
"active" : true,
"userIdBlackList" : [
"587f39910cc0fec092bdb10a"
]
}
{
"_id" : ObjectId("587f39910cc0fec092bdb10d"),
"_class" : "com.smartinnotec.legalprojectmanagement.dao.domain.DocumentFile",
"fileName" : "DocumentFile2",
"ending" : "jpg",
"projectId" : "587f39910cc0fec092bdb10b",
"active" : true,
"userIdBlackList" : [ ]
}
I have this code in order to get amount of query:
final Query query = new Query();
query.addCriteria(Criteria.where("userIdBlackList").nin(userId));
final Long amount = mongoTemplate.count(query, DocumentFile.class);
return amount.intValue();
The amount is 2 in this case what is wrong - it should be 1.
The query in Query object looks like this:
Query: { "userIdBlackList" : { "$nin" : [ "587f39910cc0fec092bdb10a"]}}
If I copy this query and made a query for the mongodb console like this:
db.DocumentFile.find({ "userIdBlackList" : { "$nin" : [ "587f39910cc0fec092bdb10a"]}}).pretty()
I get an amount of two, what if wrong because one document includes 587f39910cc0fec092bdb10a in userIdBlackList -> it should be one.
With this query command:
db.DocumentFile.find({userIdBlackList: { "$nin": ["587f39910cc0fec092bdb10a"] } }).pretty();
I get the right result, I am really confused at the moment.
Does anyone have any idea?
Maybe the problem ist that one time userIdBlackList is with quotation mark ("userIdBlackList") and the other time it isn't.
I think the problem is with the unintentional formatting picked up for "userIdBlackList". Your string is interpreted with non printing characters in the "??userIdBlackList" for all your search queries. I see little transparent square boxes when I copy your queries to mongo shell.
That tells me their is some encoding issue. Clear that formatting and see if that helps you.
Both $ne and $nin should work!
I have a mongo collection named firma which has one of the document structure as below:
{
"_id" : ObjectId("5729af099b3ebf1d0ca7ff05"),
"musteriler" : [
{
"_id" : "de0bf813-b707-4a8d-afc2-9752e05c3aa5",
"yetkiliListesi" : [
{
"_id" : "a5e487fa-2034-4817-94f2-3bd837b76284",
"ad" : "Burak",
"soyad" : "Duman 1",
"cepTel" : "3333333333333",
"mail" : "asdf#asdf.com"
},
{
"_class" : "com.bisoft.entity.MusteriYetkili",
"_id" : "bc4b537d-522a-4c9a-9f67-8ca243e18f46",
"ad" : "Ridvan",
"soyad" : "ENİŞ",
"cepTel" : "222222222222",
"mail" : "asdf#asdf.com"
}
]
}
],
"defaultTimezone" : "Europe/Istanbul"
}
In the above json, I need to update element of second array(yetkiliListesi) which _id = "a5e487fa-2034-4817-94f2-3bd837b76284"
Since I am using a java application(using mongo java driver and spring boot MongoTemplate) to access it and execute this query :
mongoTemplate.updateFirst(Query.query(Criteria.where("_id").is("5729af099b3ebf1d0ca7ff05").and("musteriler.yetkiliListesi._id").is("a5e487fa-2034-4817-94f2-3bd837b76284")),
new Update().set("musteriler.yetkiliListesi.$", yetkiliDBO), Firma.class);
In the above query, yetkiliDBO is a BasicDBObject and its content :
yetkiliDBO = {
'_class': 'com.bisoft.entity.MusteriYetkili',
'_id': "a5e487fa-2034-4817-94f2-3bd837b76284",
'ad': 'wer',
'soyad': 'xyz',
'cepTel': "222222222222",
mail: "asdf#asdf.com"
}
when execute my query I have an error
com.mongodb.WriteConcernException: { "serverUsed" : "192.168.2.250:27017" , "ok" : 1 , "n" : 0 , "updatedExisting" : false , "err" : "cannot use the part (musteriler of musteriler.yetkiliListesi.0) to traverse the element
What I need to do?
You can not use the '$' placeholder when traversing nested arrays.
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
source
I would suggest restructuring your data into separate, less-nested collections.
I am trying to create a query using MongoDB Java Driver as part of an aggregation command. Currently I allow a date range or an array of specific dates as an argument. eg
<date>
<start>2013-12-10 00:00:00.000</start>
<end>2013-12-12 23:59:59.999</end>
</date>
or
<date>
<specificDates>2013-12-10 00:00:00.000,2013-12-13 00:00:00.000</specificDates>
</date>
The date range query works fine, I parse and convert the xml into a DBObject that produces the following query in mongo;
{ "$match" : { "d" : { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}}}
For the specificDates I want to return only results that occur between 00:00:00.000 on the given day and 00:00:00.000 of the next day. From my pretty basic knowledge of mongo querys i had hoped to do a similar $match as the date range, but have it use $in on an array of date ranges similar to the following;
{ "$match" : { "d" : { "$in" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}
The above query fails to return any results. I have noticed that $in is not listed in the mongodb manual under the Mongo Aggregation Framework section, but its not throwing any kind of errors that I would have expected for an unsupported operation.
I think the issue may come from this line in the MongoDB Manual;
If the field holds an array, then the $in operator selects the documents whose field holds an array that contains at least one element that matches a value in the specified array (e.g. , , etc.)
In my collection the date isn't stored in an array, I suppose I could store it in the collections in an single element array? (Actually, decided to try this quickly before I posted, no documents returned when the date entry in the document is stored in a single element array)
Document entry example
{ "_id" : ObjectId("52aea5b0065991de1a56d5b0"), "d" : ISODate("2013-12-15T00:00:11.088Z"), "t" : 1501824, "s" : 0, "e" : 601, "tld" : "uk", "y" : "domain:check", "n" : "removed.co.uk" }
Is anyone able to give me some advice as to how I should do this query? Thank you.
EDIT: I left the Java tag here in case anyone needs my DBObject creation code, though it shouldn't be necessary as the queries posted have been generated by my build.
EDIT2: So as Alan Spencer pointed out I should be using $or rather than $in, a working $or function is below (ignore the different formatting like the use of ISODate(), its just copy pasted from the mongo shell rather than getting output from my program)
{ $match : { $or : [ { d : { $gte : ISODate("2013-10-01T00:00:00.000Z"), $lt : ISODate("2013-10-02T00:00:00.000Z") } }, { d : { $gte : ISODate("2013-10-03T00:00:00.000Z"), $lt : ISODate("2013-10-04T00:00:00.000Z") } } ] } }
I think you're inverting the meaning of the $in.
$in is used to match exactly against a list of possible values, like
{"color":{"$in": ["red","green","blue"]}}
For your use case, you are trying to match if it satisfies the first or second, etc. So, you can use $or - http://docs.mongodb.org/manual/reference/operator/query/or/
{ "$match" : { "d" : { "$or" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}