This is my one elastic document example:
"_source": {
"leg_b":{
"team_name" : "aaaa",
"channel_uuid" : "12121"
},
wrapup":[
"0":{
"name": "sss",
"channel_uuid": "12121"
},
"1":{
"name": "bbb",
"channel_uuid" : "21212"
},
"2":{
"name": "acbs",
"channel_uuid" : "111211"
}
]
}
I have to perform search on "leg_b.team_name" value and find out its "channel_uuid" value on the basis of leg_b's "channel_uuid",NEXT I will search in wrapup object for common "channel_uuid" values in the Same Query.
Example,
lets consider above document, First I will search on leg_b.team_name = "aaaa" after that I will take leg_b.channel_uuid = "12121" and search on wrapup object channel_uuid.
Explaination:
So there are two searches are going on 1) leg_b.team_name="aaaa" which returns this document and after that 2) on the returned doc which I got from 1st search, I will take its leg_b.channel_uuid to search on wrapup common channel uuid. (i.e, leg_b channel_uuid is "12121" so I should get "wrapup" channel_uuid : "12121" objects. It exclude "acbc" because its channel_uuid : "111211".
So it should return,
"_source": {
"leg_b":{
"team_name" : "aaaa",
"channel_uuid" : "12121"
},
"wrapup":[
"0":{
"name": "sss",
"channel_uuid": "12121"
},
"1":{
"name": "bbb",
"channel_uuid" : "21212"
}
]
}
Please suggest some approach, Thank you.
Related
Dear MongoDb Experts out there! I am new to Mongo and I am currently working with a MongoDb with Java.
Having a MongoDb with a collection called "teams" with the following example structure:
{[
{
"id": "123",
"name": "Dev1",
"employees": [
{"name": "John", "age": 30},
{"name": "Jane", "age": 30}
]
},
{
"id": "456",
"name": "Dev2",
"employees": [
{"name": "Mike", "age": 30},
{"name": "Oscar", "age": 27}
]
}
]}
I want to have a query which returns an array with all employees, that are 30 years old. So the expected result would be:
{[
{"name": "John", "age": 30},
{"name": "Jane", "age": 30},
{"name": "Mike", "age": 30}
]}
It would be even better to only get the employees name (since I know the age I searched for), like:
{[
{"name": "John"},
{"name": "Jane"},
{"name": "Mike"}
]}
I have a MongoCollection object:
MongoCollection<Document> collection = mongoClient
.getDatabase(databaseName)
.getCollection("teams")
My question: Is it possible to retrieve my expected result from the MongoDb? If so, which operations do I have to call on my collection object?
Approach 1: Find with $elemMatch projection
db.getCollection('teams').find(
{"employees.age":30},
{ "employees": { "$elemMatch": { "age": 30 } },"_id":0 }
)
Output Format:
{
"employees" : [
{
"name" : "John",
"age" : 30.0
}
]
}
{
"employees" : [
{
"name" : "Mike",
"age" : 30.0
}
]
}
Approach 2: Aggregate with $unwind and key renaming using $projection
db.getCollection('teams').aggregate([
{"$match": {"employees.age":30}} ,
{"$unwind": "$employees"},
{"$match": {"employees.age":30}} ,
{"$project": {"name": "$employees.name","_id":0}}
])
Output format:
{
"name" : "John"
}
{
"name" : "Jane"
}
{
"name" : "Mike"
}
Approach 1 would be faster, but require additional work at app layer. It is recommended to offload formatting tasks to app servers rather than on db itself.
Approach 2 can instantly give to the formatted results, doing the querying and formatting both in the database servers
I have indexed sample documents in elasticsearch and trying to search using fuzzy query. But am not getting any results when am search by using Java fuzzy query api.
Please find my below mapping script :
PUT productcatalog
{
"settings": {
"analysis": {
"analyzer": {
"attr_analyzer": {
"type": "custom",
"tokenizer": "letter",
"char_filter": [
"html_strip"
],
"filter": ["lowercase", "asciifolding", "stemmer_minimal_english"]
}
},
"filter" : {
"stemmer_minimal_english" : {
"type" : "stemmer",
"name" : "minimal_english"
}
}
}
},
"mappings": {
"doc": {
"properties": {
"values": {
"type": "text",
"analyzer": "attr_analyzer"
},
"catalog_type": {
"type": "text"
},
"catalog_id":{
"type": "long"
}
}
}
}
}
Please find my sample data.
PUT productcatalog/doc/1
{
"catalog_id" : "343",
"catalog_type" : "series",
"values" : "Activa Rooftop, valves, VG3000, VG3000FS, butterfly, ball"
}
PUT productcatalog/doc/2
{
"catalog_id" : "12717",
"catalog_type" : "product",
"values" : "Activa Rooftop, valves"
}
Please find my search script :
GET productcatalog/_search
{
"query": {
"match" : {
"values" : {
"query" : " activa rooftop VG3000",
"operator" : "and",
"boost": 1.0,
"fuzziness": 2,
"prefix_length": 0,
"max_expansions": 100
}
}
}
}
Am getting the below results for the above query :
{
"took": 239,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.970927,
"hits": [
{
"_index": "productcatalog",
"_type": "doc",
"_id": "1",
"_score": 0.970927,
"_source": {
"catalog_id": "343",
"catalog_type": "series",
"values": "Activa Rooftop, valves, VG3000, VG3000FS, butterfly, ball"
}
}
]
}
}
But if i use the below Java API for the same fuzzy search am not getting any results out of it.
Please find my below Java API query for fuzzy search :
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.fuzzyQuery("values", keyword).boost(1.0f).prefixLength(0).maxExpansions(100));
Update 1
I have tried with the below query
QueryBuilder qb = QueryBuilders.matchQuery(QueryBuilders.fuzzyQuery("values", keyword).boost(1.0f).prefixLength(0).maxExpansions(100));
But am not able to pass QueryBuilders inside matchQuery. Am getting this suggestion while am writing this query The method matchQuery(String, Object) in the type QueryBuilders is not applicable for the arguments (FuzzyQueryBuilder)
The mentioned java query is not a match query. It's a must query. you should use matchQuery instead of boolQuery().must(QueryBuilders.fuzzyQuery())
Update 1:
fuzzy query is a term query while match query is a full text query.
Also don't forget that in match query the default Operator is or operator which you should change it to and like your dsl query.
How to project embedded array element field in Spring Data MongoDB Aggregation with the document sample below, I tried:
project("customers.id")
project("customers.[].id")
project("customers.?.id")
project("$customers.id")
but didn't work.
Result document without projection:
{
"id": "group1",
"name": "Default Identity Management",
"warningThreshold": 900000,
"tariffId": "TR_0001",
"active": false,
"customers": [
{
"id": "1",
"name": "David",
"properties": [
{
"name": "phone",
"value": "678"
}
],
"roles": [
"dev"
]
},
{
"id": "2",
"name": "Peter",
"properties": [
{
"name": "phone",
"value": "770"
}
],
"roles": [
"techsales",
"dev"
]
}
]
}
Expected document like this:
{
"id" : "group1",
"name" : "Group1",
"tariffId" : "TR_0001",
"warningThreshold" : 900000,
"customers" : [
{
"id" : "1",
"name" : "David",
"properties" : [
{
"name" : "phone",
"value" : "678"
}
]
},
{
"id" : "2",
"name" : "Peter",
"properties" : [
{
"name" : "phone",
"value" : "770"
}
]
}
]
}
I would like to include customers[].id, customers[].name, customers[].properties.
I'd been trying to figure this out for a while now, but couldn't. And the other posts here on stackoverflow, and other places on the internet, didn't provide the solution I was looking for.
My problem was similar to the original author's: There's a document, which has a field which is an array of documents. I wanted to query all the top level fields in the document, and exclude a single field from the documents within the array.
s7vr's answer in the comments for the question did the job for me! Just re-posting that here since most people don't go through all the comments, and it is a really useful answer, that saved me from writing a lot of crappy code! :D
AggregationOperation project = new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext) {
return new Document("$project", new Document("arrayField.nestedFieldToExclude", 0));
}
};
With Lambda:
AggregationOperation project = aggregationOperationContext -> new Document("$project", new Document("arrayField.nestedFieldToExclude", 0));
Overall pipeline:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(criteria),
Aggregation.sort(Sort.Direction.DESC, "createdOn"),
project);
I just wish there was a cleaner way to do this with the Spring MongoDB Data API directly, rather than using it this way with lambda functions.
Also, please note that the method AggregationOperation.toDocument(AggregationOperationContext aggregationOperationContext) has been deprecated as of spring-data-mongodb version 2.2.
I am using MongoDB Java Driver 3.6.3.
I want to create regex query with group by aggregation to retrieve distinct values.
Let's say I have json:
[{
"name": "John Snow",
"category": 1
},
{
"name": "Jason Statham",
"category": 2
},
{
"name": "John Lennon",
"category": 2
},
{
"name": "John Snow",
"category": 3
}]
I want to create query where regex is like "John.*" and group it by name so there would be only one "John Snow"
Expected result is:
[{
"name": "John Snow",
"category": 1
},
{
"name": "John Lennon",
"category": 2
}]
The answer provided by felix is correct, in terms of Mongo Shell commands. The equivalent expression of that command using the MongoDB Java driver is:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
// Java equivalent of the $match stage
Aggregates.match(Filters.regex("name", "John")),
// Java equivalent of the $group stage
Aggregates.group("$name", Accumulators.first("category", "$category"))
));
for (Document document : documents) {
System.out.println(document.toJson());
}
The above code will print out:
{ "_id" : "John Lennon", "category" : 2 }
{ "_id" : "John Snow", "category" : 1 }
You can achieve this with a $regex in $match stage, followed by a $group stage:
db.collection.aggregate([{
"$match": {
"name": {
"$regex": "john",
"$options": "i"
}
}
}, {
"$group": {
"_id": "$name",
"category": {
"$first": "$category"
}
}
}])
output:
[
{
"_id": "John Lennon",
"category": 2
},
{
"_id": "John Snow",
"category": 1
}
]
you can try it here: mongoplayground.net/p/evw6DP_574r
You can use Spring Data Mongo
like this
Aggregation agg = Aggregation.newAggregation(
ggregation.match(ctr.orOperator(Criteria.where("name").regex("john", "i")),
Aggregation.group("name", "category")
);
AggregationResults<CatalogNoArray> aggResults = mongoTemp.aggregate(agg, "demo",demo.class);
I have a json file that contains three datas together. I want to insert all three datas seperatly into the mongodB. Is that possible? if yes then how?
{
"docs": [
{
"_id": "First",
"count": 4,
"name": "Fish",
},
{
"_id": "Second",
"count": 6,
"name": "Meat"
},
{
"_id": "Third",
"count": 8,
"name": "Vegetables"
}
]
}
Inserting a group of documents from the mongo client shell:
let,
var input = {
"docs": [
{
"_id": "First",
"count": 4,
"name": "Fish",
},
{
"_id": "Second",
"count": 6,
"name": "Meat"
},
{
"_id": "Third",
"count": 8,
"name": "Vegetables"
}
]
}
Inserting the docs array:
db.collection.insert(input["docs"]);
This would insert each item in the docs array as separate documents in the collection.
db.collection.find();
would give us, three different documents that were inserted.
{ "_id" : "First", "count" : 4, "name" : "Fish" }
{ "_id" : "Second", "count" : 6, "name" : "Meat" }
{ "_id" : "Third", "count" : 8, "name" : "Vegetables" }
To do it in Java, you need to load and parse the JSON file using JSON parsing libraries such as Jackson parser, get the docs array and persist it.