I have a problem with MongoDB QueryBuilder.
Assume I have a number of documents, that can contain one or more users:
{
"_id": "document1",
"data": {
"user": {
"credentials": {
"name": "John",
"lastname": "Watson",
"middle": "Hemish"
}
}
}
}
{
"_id": "document2",
"data": {
"user": [
{
"credentials": {
"name": "John",
"lastname": "Nicholson",
"middle": "Joseph"
}
},
{
"credentials": {
"name": "Mary",
"lastname": "Watson",
"middle": ""
}
}
]
}
}
{
"_id": "document3",
"data": {
"user": [
{
"credentials": {
"name": "John",
"lastname": "Watson",
"middle": "Hemish"
}
},
{
"credentials": {
"name": "John",
"lastname": "Nicholson",
"middle": "Joseph"
}
},
{
"credentials": {
"name": "Mary",
"lastname": "Watson",
"middle": ""
}
}
]
}
}
What I am trying to do is the query, that will return only those documents containing John Watson as a user.
Here what I got so far:
1.
QueryBuilder qb = QueryBuilder.start("credentials.lastname").is("Watson").and("credentials.name").is("John");
DBObject query = QueryBuilder.start("data.user").elemMatch(qb.get()).get();
this query will return only document3: there is no array in document1 and no match in document2 (but I would like it to return document1 and document3)
2.
DBObject query = QueryBuilder.start("data.user.credentials.lastname").is("Watson").and("data.user.credentials.name").is("John").get();
this one will return all three documents: document1 and document3 are desired match, but the query will match as well document2, for it has Watson and John in query fields in the array, no matter that they are separate entries.
Is there any way to make a right query that will return document1 and document3 for John Watson?
I am trying to do it in Java, but any other example would be fine.
Right now I use a workaround combining results from both queries: first I get limit(100) results from the query with elementMatch(), then, if there are less than 100 results, I do the second query and filter all wrong matches. But I hope there is a better and more effective way to get those results.
I could give you at best like the following where user would be in an array as unwind value of the key data. I think a little bit more effort would lead you to the exact format as you want.
I am sharing it as I think it should serve the purpose or anyhow it should help you.
The aggregation query:
db.tuttut.aggregate([
{$unwind:"$data.user"},
{ $project: {
_id:1,
data:1,
temp: {name:"$data.user.credentials.name",
lastname:"$data.user.credentials.lastname"}
} } ,
{ $group:{
_id:"$_id" ,
data: {$addToSet: "$data"} ,
temp:{ $addToSet: "$temp" } } },
{ $match:{ temp:{name:"John",lastname:"Watson"} } } ,
{$project:{_id:1, data:1}}
]).pretty()
Returned Result:
{
"_id" : "document1",
"data" : [
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Watson",
"middle" : "Hemish"
}
}
}
]
}
{
"_id" : "document3",
"data" : [
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Watson",
"middle" : "Hemish"
}
}
},
{
"user" : {
"credentials" : {
"name" : "Mary",
"lastname" : "Watson",
"middle" : ""
}
}
},
{
"user" : {
"credentials" : {
"name" : "John",
"lastname" : "Nicholson",
"middle" : "Joseph"
}
}
}
]
}
Related
In the project flow i am getting the data in form of json collection in apache camel exchange and to process it further i need to transform it in java object
try{
List<RespModel> records = (List<RespModel>) exchange.getIn().getBody(RespModel.class);
System.out.println(records.size());
}catch (Exception e){
System.out.println("NO LUCK "+e.getLocalizedMessage());
}
But i am getting records as null.
Could you please help me to transform this ?
exchange data is as below -
"identifier": {
"domain": "transport",
"id": "123",
"version": 1
},
"record": "NEW",
"payload": {
"pesonalDetails" : {
"name" : "bla bla bla"
"dob" :
},
"reason" :
}
},{
"identifier": {
"domain": "transport",
"id": "123",
"version": 1
},
"record": "NEW",
"payload": {
"pesonalDetails" : {
"name" : "bla bla bla"
"dob" :
},
"reason" :
}]```
I am getting null here, how can i achieve this ? pls reply if you know. Thanks
Got the answer finally -
String result = IOUtils.toString((InputStream) exchange.getIn().getBody(), StandardCharsets.UTF_8);
List<RespModel> = new ObjectMapper().readValue(result,
new TypeReference<List<RespModel>>() {
});
I am trying to flatten nested arrays using aggregation framework but I can not get the result I which.
my collection is :
[
{
"id" : "xxx",
"countryName" : "xxx",
"cities" : [
{
"id" : "xxx",
"cityName" : "xxx"
},
{
"id" : "xxx",
"cityName" : "xxx"
}
]
}
]
I want to get the cities from all countries, the result I am looking for is :
[
{
"id" : "xxx",
"cityName" : "xxx"
},
{
"id" : "xxx",
"cityName" : "xxx"
}
]
I tried this request :
val aggregation = Aggregation.newAggregation(
Aggregation.group("cities")
)
return mongoDb.aggregate(aggregation, Country::class.java, Any::class.java).mappedResults
But, I got this result :
[
{
"_id": [
{
"id": "xxx",
"cityName": "xxx"
},
{
"id": "xxx",
"cityName": "xxx"
}
]
}
]
Can someone help me please?
This aggregation will help you achieve your result, except that you have to adapt it with Java driver:
db.countries.aggregate([
{
"$unwind": "$cities"
},
{
"$project": {
"_id": 0,
"cities": 1
}
},
{
"$replaceRoot": {
"newRoot": "$cities"
}
}
])
I have a field called Description which is a text field and has data like:
This is a good thing for versions before 3.2 but bad for 3.5 and later
I want to run range query on this type of text. I know that for a field containing only Dates/Age(Numbers) or even String Ids, we can use queries like
{
"query": {
"range" : {
"age" : {
"gte" : 10,
"lte" : 20,
"boost" : 2.0
}
}
}
}
But i have a mixed field like mentioned above and I need to perform range query on that. Also, i cannot change the index structure. I can only perform queries or do some post processing after retrieving results. So anyone has any idea how to run this type of query, or even obtain my goal after getting results in the post processing? I am using Java.
I hope i fully understand what you are looking for.
I've managed to create a simple working example.
Mappings
Using char_group tokenizer:
The char_group tokenizer breaks text into terms whenever it encounters a character which is in a defined set. It is mostly useful for cases where a simple custom tokenization is desired, and the overhead of use of the pattern tokenizer is not acceptable.
Char Group Tokenizer
PUT my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"type": "custom",
"tokenizer": "my_tokenizer"
}
},
"tokenizer": {
"my_tokenizer": {
"type": "char_group",
"tokenize_on_chars": [
"letter",
"whitespace"
]
}
}
}
},
"mappings": {
"properties": {
"text": {
"type": "text",
"fields": {
"digit": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
}
}
Post a few documents
PUT my_index/_doc/1
{
"text": "This is a good thing for versions before 3.2 but bad for 3.5 and later"
}
PUT my_index/_doc/2
{
"text": "This is a good thing for versions before 5 but bad for 6 and later"
}
Search Query
GET my_index/_search
{
"query": {
"range": {
"text.digit": {
"gte": 3.2,
"lte": 3.5
}
}
}
}
Results
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"text" : "This is a good thing for versions before 3.2 but bad for 3.5 and later"
}
}
]
}
Another Search Query
GET my_index/_search
{
"query": {
"range": {
"text.digit": {
"gt": 3.5
}
}
}
}
Results
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"text" : "This is a good thing for versions before 5 but bad for 6 and later"
}
}
]
}
Analyze Query
Play with the following query till you get the desired results.
It is already compatible to your example.
This is a good thing for versions before 3.2 but bad for 3.5 and later
POST _analyze
{
"tokenizer": {
"type": "char_group",
"tokenize_on_chars": [
"letter",
"whitespace"
]
},
"text": "This is a good thing for versions before 3.2 but bad for 3.5 and later"
}
Hope this helps
I am using MongoDB Java Driver 3.6.3.
I want to create regex query with group by aggregation to retrieve distinct values.
Let's say I have json:
[{
"name": "John Snow",
"category": 1
},
{
"name": "Jason Statham",
"category": 2
},
{
"name": "John Lennon",
"category": 2
},
{
"name": "John Snow",
"category": 3
}]
I want to create query where regex is like "John.*" and group it by name so there would be only one "John Snow"
Expected result is:
[{
"name": "John Snow",
"category": 1
},
{
"name": "John Lennon",
"category": 2
}]
The answer provided by felix is correct, in terms of Mongo Shell commands. The equivalent expression of that command using the MongoDB Java driver is:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
// Java equivalent of the $match stage
Aggregates.match(Filters.regex("name", "John")),
// Java equivalent of the $group stage
Aggregates.group("$name", Accumulators.first("category", "$category"))
));
for (Document document : documents) {
System.out.println(document.toJson());
}
The above code will print out:
{ "_id" : "John Lennon", "category" : 2 }
{ "_id" : "John Snow", "category" : 1 }
You can achieve this with a $regex in $match stage, followed by a $group stage:
db.collection.aggregate([{
"$match": {
"name": {
"$regex": "john",
"$options": "i"
}
}
}, {
"$group": {
"_id": "$name",
"category": {
"$first": "$category"
}
}
}])
output:
[
{
"_id": "John Lennon",
"category": 2
},
{
"_id": "John Snow",
"category": 1
}
]
you can try it here: mongoplayground.net/p/evw6DP_574r
You can use Spring Data Mongo
like this
Aggregation agg = Aggregation.newAggregation(
ggregation.match(ctr.orOperator(Criteria.where("name").regex("john", "i")),
Aggregation.group("name", "category")
);
AggregationResults<CatalogNoArray> aggResults = mongoTemp.aggregate(agg, "demo",demo.class);
I'm trying to combine mutiple queries in elasticsearch using a boolean query but the result is not what I'm expecting. For example:
If I have the following documents (among others):
DOC 1:
{
"name":"Iphone 5",
"product_suggestions":{
"input":[
"iphone 5",
"apple"
]
},
"description":"Iphone 5 - The almost last version",
"brand":"Apple",
"brand_facet":"Apple",
"state_id":"2",
"user_state_description":"Almost New",
"product_type_id":"1",
"current_price":350,
"finish_date":"2014/06/20 14:12",
"finish_date_ms":1403273520
}
DOC 2:
{
"name":"Apple II Lisa",
"product_suggestions":{
"input":[
"apple ii lisa",
"apple"
]
},
"description":"Make a offer and I Apple II Lisa!!",
"brand":"Apple",
"brand_facet":"Apple",
"state_id":"2",
"user_state_description":"Used",
"product_type_id":"1",
"current_price":150,
"finish_date":"2014/06/15 16:12",
"finish_date_ms":1402848720
}
DOC 3:
{
"name":"Iphone 5s",
"product_suggestions":{
"input":[
"iphone 5s",
"apple"
]
},
"description":"Iphone 5s 32Gb like new with a few scratches bla bla bla",
"brand":"Apple",
"brand_facet":"Apple",
"state_id":"1",
"user_state_description":"New",
"product_type_id":"2",
"current_price":510.1,
"finish_date":"2014/06/10 14:12",
"finish_date_ms":1402409520
}
DOC 4:
{
"name":"Iphone 4s",
"product_suggestions":{
"input":[
"iphone 4s",
"apple"
]
},
"description":"Iphone 4s 16Gb Mint conditions and unlocked to all network",
"brand":"Apple",
"brand_facet":"Apple",
"state_id":"1",
"user_state_description":"Almost New",
"product_type_id":"2",
"current_price":385,
"finish_date":"2014/06/12 16:12",
"finish_date_ms":1402589520
}
And if I run the following query (Get all documents and facets with the keyword "Apple" that the finish_date_ms is bigger than 1402869581)
{
"from" : 1,
"size" : 20,
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "apple",
"default_operator" : "and",
"analyze_wildcard" : true
}
},
"must_not" : {
"range" : {
"finish_date_ms" : {
"from" : null,
"to" : 1402869581,
"include_lower" : true,
"include_upper" : false
}
}
}
}
},
"facets" : {
"brand" : {
"terms" : {
"field" : "brand_facet",
"size" : 10
}
},
"product_type_id" : {
"terms" : {
"field" : "product_type_id",
"size" : 10
}
},
"state_id" : {
"terms" : {
"field" : "state_id",
"size" : 10
}
}
}
}
This returns:
{
"took":5,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":0.18392482,
"hits":[
]
},
"facets":{
"brand":{
"_type":"terms",
"missing":0,
"total":1,
"other":0,
"terms":[
{
"term":"Apple",
"count":1
}
]
},
"product_type_id":{
"_type":"terms",
"missing":0,
"total":1,
"other":0,
"terms":[
{
"term":1,
"count":1
}
]
},
"state_id":{
"_type":"terms",
"missing":0,
"total":1,
"other":0,
"terms":[
{
"term":2,
"count":1
}
]
}
}
}
And should return only the document DOC1. If I remove the range query, returns all the documents that has Apple word. If I remve the "term" query then n document is returns, so I presume the problem is in the range query.
Can anyone point me in the right direction with this?
One other important thing, all this query is to be implemented in java (if this help).
Thanks!
(sory for this huge post)
I found my mistake. (newbie mistake to be honest)
The problem was not in the range query but in the begging of the Json: The from field is set to 1 but the result is only one record so this should be 0!!
Thanks for everything!!