Some other process inserts documents into mongo collection and below is the sample data
{ "_id" : ObjectId("597b89c8da52380b04ee6948"), "_class" : "com.test.mongo", "clientId" : "CAQ123999", "isValid" : false, "isParent" : true }
{ "_id" : ObjectId("597b89c8da52380b04ee6949"), "_class" : "com.test.mongo", "clientId" : "CAQ123999", "isValid" : false, "isParent" : true }
{ "_id" : ObjectId("597b89c8da52380b04ee6950"), "_class" : "com.test.mongo", "clientId" : "CAQ123998", "isValid" : true, "isParent" : true }
{ "_id" : ObjectId("597b89c8da52380b04ee6951"), "_class" : "com.test.mongo", "clientId" : "CAQ123997", "isValid" : true, "isParent" : false }
I'm trying to grab one record for the clientId, using QueryByExampleExecutor.
Here is my model
package com.test.cfp.model;
public class TFSModel {
private String clientId;
private boolean isValid;
private boolean isParent;
...
}
Here is the code to build the example:
TFSModel tfs = new TFSModel();
tfs.setClientId(CAQ123999);
tfs.setValid(false);
tfs.setParent(true);
ExampleMatcher matcher =ExampleMatcher.matching().withIgnoreNullValues().withIgnorePaths("_id","_class");
Example<TFSModel > example = Example.of(tfs,matcher);
TFSModel oneTfsRecord = tflsRepository.findOne(example);
This is not working , below is the generated query
findOne using query: { "isValid" : false , "isParent" : true , "clientId" : "CAQ123999" , "_class" : { "$in" : [ "com.test.cfp.model.TFSModel"]}} in db.collection: returns.tfs;
Obviously the _class has the different package than the one in the mongo collection. How can I tell mongo to build a query without _class. I tried withIgnoredPaths and its not working.
MongoExampleMapper inspects the probe type and writes type restrictions according to known types in MappingContext. Types assignable to the probe get included in the $in operator.
class One {
#Id String id;
String value;
// ...
}
class Two extends One {
// ...
}
One probe = new One();
probe.value = "firefight";
Example<One> example = Example.of(probe, ExampleMatcher.matchingAny());
{
value: firefight,
_class: { $in : [ "com.example.One" , "com.example.Two" ] }
}
This behavior cannot be altered by using .withIgnorePaths() as the _class specifier is not part of the domain model. Please file an issue in jira.spring.io if you think this should be considered.
Looking at the provided sample data from the mongo collection you provided the _class attribute is not matching your domain type and therfore cannot be loaded.
Related
I'm needing to retrieve just with two dates, all the documents from my MongoDB's collection, with the filtered items from the array.
This is an example of 2 of my documents;
{
"_id" : ObjectId("5f18fa823406b7000132d097"),
"last_date" : "22/07/2020 23:48:32",
"history_dates" : [
"22/07/2020 23:48:32",
"22/07/2020 00:18:53",
"23/07/2020 00:49:12",
"23/07/2020 01:19:30"
],
"hostname" : "MyHostname1",
"ip" : "142.0.111.79",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "LogonUI.exe",
"date" : "23/07/2020 10:26:04",
}
{
"_id" : ObjectId("5f18fa823406b7000132d098"),
"last_date" : "22/07/2020 23:48:33",
"history_dates" : [
"22/07/2020 23:48:33",
"23/07/2020 00:18:53",
],
"hostname" : "MyHostName2",
"ip" : "142.0.111.54",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "svchost.exe",
"date" : "23/07/2020 10:26:04",
}
I'm needing to make a find to my database (Using Spring Data), to retrieve the same objects, but with the "history_dates"'s array filtered between the 2 dates recieved.
For example, if my 2 recieved dates are: "23/07/2020" and "24/07/2020", I want MongoDB to return the next objects;
{
"_id" : ObjectId("5f18fa823406b7000132d097"),
"last_date" : "22/07/2020 23:48:32",
"history_dates" : [
"23/07/2020 00:49:12",
"23/07/2020 01:19:30"
],
"hostname" : "MyHostname1",
"ip" : "142.0.111.79",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "LogonUI.exe",
"date" : "23/07/2020 10:26:04",
}
{
"_id" : ObjectId("5f18fa823406b7000132d098"),
"last_date" : "22/07/2020 23:48:33",
"history_dates" : [
"23/07/2020 00:18:53"
],
"hostname" : "MyHostName2",
"ip" : "142.0.111.54",
"component" : "C:\\Windows\\System32\\es-ES\\KernelBase.dll.mui",
"process" : "svchost.exe",
"date" : "23/07/2020 10:26:04",
}
I'm really ignorant about MongoDB's queries, and I have been trying to make this with Spring Data all the week.
UPDATE 1.
Thanks varman, and do you know how can i just retrieve the documents with filtered arrays not empty?
So basically you need to do filter. MongoTemplate offers a lot of operation for mongodb, if some methods don't exist in MongoTemplate, we can go with Bson Document pattern. In that case, try this article: Trick to covert mongo shell query.
Actually you need a Mongo query something like following. Using $addFields one of the methods shown below. But you can use $project, $set etc. Here $addFields overwrites your history_dates. (It uses to add new fields to document too).
{
$addFields: {
history_dates: {
$filter: {
input: "$history_dates",
cond: {
$and: [{
$gt: ["$$this", "23/07/2020"]
},
{
$lt: ["$$this", "24/07/2020"]
}
]
}
}
}
}
}
Working Mongo playground.
You need to convert this into spring data. So #Autowired the MongoTemplate in you class.
#Autowired
MongoTemplate mongoTemplate;
The method is,
public List<Object> filterDates(){
Aggregation aggregation = Aggregation.newAggregation(
a->new Document("$addFields",
new Document("history_dates",
new Document("$filter",
new Document("input","$history_dates")
.append("cond",
new Document("$and",
Arrays.asList(
new Document("$gt",Arrays.asList("$$this","23/07/2020")),
new Document("$lt",Arrays.asList("$$this","24/07/2020"))
)
)
)
)
)
)
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_CLASS.class), Object.class).getMappedResults();
}
Mongo template doesn't provide add methods for $addFields and $filter. So we just go with bson document pattern. I haven't tested this in Spring.
Mongo document:
{
"_id" : "1",
"array" : [
{
"item" : "item"
},
{
"item" : "item"
}
]
}
My mongo shell query looks like so:
db.getCollection('collectionName').aggregate(
{$match: { _id: "1"}},
{$project: { count: { $size:"$array" }}}
)
Is there anyway to implement this using the Mongo Template from Spring?
So far I have this:
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
ProjectionOperation project = new ProjectionOperation();
Aggregation aggregate = Aggregation.newAggregation(match, project);
mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
I think I am only missing the project logic but I'm not sure if it is possible to do $size or equivalent here.
It's quite possible, the $size operator is supported (see DATAMONGO-979 and its implementation here). Your implementation could follow this example:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = new Aggregation(
match(where("_id").is("1")), //
project() //
.and("array") //
.size() //
.as("count")
);
AggregationResults<IntegerCount> results = mongoTemplate.aggregate(
agg, collectionName, Integer.class
);
List<IntegerCount> intCount = results.getMappedResults();
Please find below the sample code. You can change it accordingly for your requirement with collection name, collection name class and array field name.
MatchOperation match = new MatchOperation(Criteria.where("_id").is("1"));
Aggregation aggregate = Aggregation.newAggregation(match, Aggregation.project().and("array").project("size").as("count"));
AggregationResults<CollectionNameClass> aggregateResult = mongoOperations.aggregate(aggregate, "collectionName", <CollectionNameClass>.class);
if (aggregateResult!=null) {
//You can find the "count" as an attrribute inside "result" key
System.out.println("Output ====>" + aggregateResult.getRawResults().get("result"));
System.out.println("Output ====>" + aggregateResult.getRawResults().toMap());
}
Sample output:-
Output ====>[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}]
Output ====>{serverUsed=127.0.0.1:27017, waitedMS=0, result=[ { "_id" : "3ea8e671-1e64-4cde-bd78-5980049a772b" , "count" : 47}], ok=1.0}
You can write query as
Aggregation aggregate = Aggregation.newAggregation(Aggregation.match(Criteria.where("_id").is(1)),
Aggregation.project().and("array").size().as("count")); mongoTemplate.aggregate(aggregate, collectionName, Integer.class);
It will execute the following query { "aggregate" : "collectionName" , "pipeline" : [ { "$match" : { "_id" : 1}} , { "$project" : { "count" : { "$size" : [ "$array"]}}}]}
I have a mongo collection named firma which has one of the document structure as below:
{
"_id" : ObjectId("5729af099b3ebf1d0ca7ff05"),
"musteriler" : [
{
"_id" : "de0bf813-b707-4a8d-afc2-9752e05c3aa5",
"yetkiliListesi" : [
{
"_id" : "a5e487fa-2034-4817-94f2-3bd837b76284",
"ad" : "Burak",
"soyad" : "Duman 1",
"cepTel" : "3333333333333",
"mail" : "asdf#asdf.com"
},
{
"_class" : "com.bisoft.entity.MusteriYetkili",
"_id" : "bc4b537d-522a-4c9a-9f67-8ca243e18f46",
"ad" : "Ridvan",
"soyad" : "ENİŞ",
"cepTel" : "222222222222",
"mail" : "asdf#asdf.com"
}
]
}
],
"defaultTimezone" : "Europe/Istanbul"
}
In the above json, I need to update element of second array(yetkiliListesi) which _id = "a5e487fa-2034-4817-94f2-3bd837b76284"
Since I am using a java application(using mongo java driver and spring boot MongoTemplate) to access it and execute this query :
mongoTemplate.updateFirst(Query.query(Criteria.where("_id").is("5729af099b3ebf1d0ca7ff05").and("musteriler.yetkiliListesi._id").is("a5e487fa-2034-4817-94f2-3bd837b76284")),
new Update().set("musteriler.yetkiliListesi.$", yetkiliDBO), Firma.class);
In the above query, yetkiliDBO is a BasicDBObject and its content :
yetkiliDBO = {
'_class': 'com.bisoft.entity.MusteriYetkili',
'_id': "a5e487fa-2034-4817-94f2-3bd837b76284",
'ad': 'wer',
'soyad': 'xyz',
'cepTel': "222222222222",
mail: "asdf#asdf.com"
}
when execute my query I have an error
com.mongodb.WriteConcernException: { "serverUsed" : "192.168.2.250:27017" , "ok" : 1 , "n" : 0 , "updatedExisting" : false , "err" : "cannot use the part (musteriler of musteriler.yetkiliListesi.0) to traverse the element
What I need to do?
You can not use the '$' placeholder when traversing nested arrays.
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
source
I would suggest restructuring your data into separate, less-nested collections.
I have a collection with a dataset that looks like:
{
"resource" : "10.10.10.10",
"statistics" : {
"connections" : 17
}
}
{
"resource" : "10.10.10.10",
"statistics" : {
"connections" : 24
}
}
I want to use Mongo's $group/$push mechanism to return a dataset that looks like:
{ "_id" : "10.10.10.10", "statTotals" : [ 17, 24 ] }
In Mongo shell, I can that by doing:
db.testcol.aggregate([ { "$project" : { "resource" : 1 , "total" : "$statistics.connections"}} , { "$group" : { "_id" : "$resource" , "statTotals" : { "$push" : "$total"}}} ])
Now I want to do this using Spring's Mongo data solution in Java. The operations I'm currently trying to use are:
ProjectionOperation projOper = Aggregation.project("resource").and("statistics.connections").as("total");
GroupOperation groupOper = Aggregation.group("resource").push("total").as("statTotals");
Unfortunately, this is generating a pipeline which looks like:
{ "aggregate" : "event" , "pipeline" : [ { "$project" : { "resource" : 1 , "total" : "$statistics.connections"}} , { "$group" : { "_id" : "$resource" , "statTotals" : { "$push" : "$statistics.connections"}}}]}
In the $group, the $push is being done against $statistics.connections instead of $total so the results come back blank.
Any help would be greatly appreciated on this. At first I thought this might be https://jira.spring.io/browse/DATAMONGO-1254 but I've tried using both 1.7.2 as well as 1.8.1 and get the same results.
I am trying to create a query using MongoDB Java Driver as part of an aggregation command. Currently I allow a date range or an array of specific dates as an argument. eg
<date>
<start>2013-12-10 00:00:00.000</start>
<end>2013-12-12 23:59:59.999</end>
</date>
or
<date>
<specificDates>2013-12-10 00:00:00.000,2013-12-13 00:00:00.000</specificDates>
</date>
The date range query works fine, I parse and convert the xml into a DBObject that produces the following query in mongo;
{ "$match" : { "d" : { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}}}
For the specificDates I want to return only results that occur between 00:00:00.000 on the given day and 00:00:00.000 of the next day. From my pretty basic knowledge of mongo querys i had hoped to do a similar $match as the date range, but have it use $in on an array of date ranges similar to the following;
{ "$match" : { "d" : { "$in" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}
The above query fails to return any results. I have noticed that $in is not listed in the mongodb manual under the Mongo Aggregation Framework section, but its not throwing any kind of errors that I would have expected for an unsupported operation.
I think the issue may come from this line in the MongoDB Manual;
If the field holds an array, then the $in operator selects the documents whose field holds an array that contains at least one element that matches a value in the specified array (e.g. , , etc.)
In my collection the date isn't stored in an array, I suppose I could store it in the collections in an single element array? (Actually, decided to try this quickly before I posted, no documents returned when the date entry in the document is stored in a single element array)
Document entry example
{ "_id" : ObjectId("52aea5b0065991de1a56d5b0"), "d" : ISODate("2013-12-15T00:00:11.088Z"), "t" : 1501824, "s" : 0, "e" : 601, "tld" : "uk", "y" : "domain:check", "n" : "removed.co.uk" }
Is anyone able to give me some advice as to how I should do this query? Thank you.
EDIT: I left the Java tag here in case anyone needs my DBObject creation code, though it shouldn't be necessary as the queries posted have been generated by my build.
EDIT2: So as Alan Spencer pointed out I should be using $or rather than $in, a working $or function is below (ignore the different formatting like the use of ISODate(), its just copy pasted from the mongo shell rather than getting output from my program)
{ $match : { $or : [ { d : { $gte : ISODate("2013-10-01T00:00:00.000Z"), $lt : ISODate("2013-10-02T00:00:00.000Z") } }, { d : { $gte : ISODate("2013-10-03T00:00:00.000Z"), $lt : ISODate("2013-10-04T00:00:00.000Z") } } ] } }
I think you're inverting the meaning of the $in.
$in is used to match exactly against a list of possible values, like
{"color":{"$in": ["red","green","blue"]}}
For your use case, you are trying to match if it satisfies the first or second, etc. So, you can use $or - http://docs.mongodb.org/manual/reference/operator/query/or/
{ "$match" : { "d" : { "$or" : [ { "$gte" : { "$date" : "2013-10-01T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-02T00:00:00.000Z"}} , { "$gte" : { "$date" : "2013-10-03T00:00:00.000Z"} , "$lt" : { "$date" : "2013-10-04T00:00:00.000Z"}}]}}}