I've been struggling with Elasticsearch templates, specifically in optional parameters. I'd like to add optional filters there. This is the code snippet I was trying out:
{
"filter" : {
"bool" : {
"must" : [
{{#ProductIDs.0}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.0}}
]
}
}
}
Of course I replaced " with \", uglified it, wrapped it up in { "template" :"_snippet_above_" }.
Now when I'm trying to call it using the following:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": [1,2]
}
}
It ignores parameter that I've provided, however when I try to do that in official mustache.io demo page - it works just fine.
I tried {{#ProductIDs.length}} option too - it didn't work out. After doing some research I've found out that there is one difference between mustache.js and mustache.java. I assumed that Elasticsearch uses JAVA version and it doesn't support length parameter, so I have to rely on isEmpty. So I've rewritten my query as follows:
{
"filter" : {
"bool" : {
"must" : [
{{^ProductIDs.isEmpty}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.isEmpty}}
]
}
}
}
Now when I query template with ProductIDs list - it works fine, however if I remove parameter, it brings no results. I assume it generates this:
{
"filter" : {
"bool" : {
"must" : [
{ "terms" : { "Product.ProductID" : [] } }
]
}
}
}
If I send empty array as Parameter - it works fine.
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
I assume this happens because "ProductIDs" are undefined and not empty.
Is there a way to cath this condition in mustache.java so I can ignore these parameters?
tl;dr;
The issue is that if I don't specify parameter in my search request via template, my condition is rendered as an empty array, see this:
{
"filter" : {
"bool" : {
"must" : [
{ "terms" : { "Product.ProductID" : [] } }
]
}
}
}
If I pass empty array as a parameter, see this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
It works as expected and doesn't generate filter condition as described in my template, because array doesn't have any data in it.
I want this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
}
}
To work same as this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
A workaround probably not the most elegant would be to change template query to be a should clause and add a match_all clause for empty list.
example:
{
"filter" : {
"bool" : {
"should" : [
{ "terms" : { "status" : [ "{{#ProductIDs}}","{{.}}","{{/ProductIDs}}"] }}
{{^ProductIDs}},
{"match_all":{}}
{{/ProductIDs}}
]
}
}
}
Didn't try it, but shouldn't something like this work?
{
"filter" : {
"bool" : {
"must" : [
{{#ProductIDs}}
{{^ProductIDs.isEmpty}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.isEmpty}}
{{#ProductIDs.isEmpty}}
{"match_all":{}}
{{/ProductIDs.isEmpty}}
{{/ProductIDs}}
{{^ProductIDs}}
{"match_all":{}}
{{/ProductIDs}}
]
}
}
}
Ain't pretty, maybe there's better way.
My suggestion to overcome this using a JSON template is:
{
"query": {
"bool": {
"must": [
{
"script": {
"script": {
"inline": "1==1 {{#ProductIDs}} && [\"{{#ProductIDs}}\",\"{{.}}\",\"{{/ProductIDs}}\"].contains(doc['Product.ProductID'].value){{/ProductIDs}}",
"lang": "painless"
}
}
}
]
}
}
Related
Please find my mapping query below for the filename field.
PUT /articles
{
"settings" : {
"analysis" : {
"analyzer" : {
"filename_search" : {
"tokenizer" : "filename",
"filter" : ["lowercase"]
},
"filename_index" : {
"tokenizer" : "filename",
"filter" : ["lowercase","edge_ngram"]
}
},
"tokenizer" : {
"filename" : {
"pattern" : "[^\\p{L}\\d]+",
"type" : "pattern"
}
},
"filter" : {
"edge_ngram" : {
"side" : "front",
"max_gram" : 50,
"min_gram" : 1,
"type" : "edgeNGram"
}
}
}
},
"mappings" : {
"doc" : {
"properties" : {
"filename" : {
"type" : "text",
"search_analyzer" : "filename_search",
"analyzer" : "filename_index"
}
}
}
}
}
If am trying to query series1333372 doc623258 and am expecting karthik_series1333372_oracle_page_doc623258_v1_en-EU.pdf. But it's giving all the files which is having series1333372, not even checking for doc623258.
Please find my query below
get articles/_search
{
"query" : {
"match" : {
"filename" : "series1333372 doc623258"
}
}
}
I am inserting the following sample documents for testing from Kibana
POST articles/doc/1
{
"filename" : "karthik_series1333372_oracle_page_doc623258_v1_en-EU.pdf"
}
POST articles/doc/2
{
"filename" : "karthik_series1333372_sun_page_doc658_v1_en-EU.pdf"
}
POST articles/doc/3
{
"filename" : "series1333372_oracle_page_doc623_v1_en-US.pdf"
}
POST articles/doc/4
{
"filename" : "Engineering series1333372 valve_page doc6232 v1_en-US.pdf"
}
POST articles/doc/5
{
"filename" : "Machines_series1333372_page_doc62258_v1_en-US.pdf"
}
POST articles/doc/6
{
"filename" : "AIX series1333372 IBM page doc62358 v1_en-EU.pdf"
}
The default operator of match is OR. If you want all your terms to be present change it like this
GET articles/_search
{
"query" : {
"match" : {
"filename" : {
"query": "series1333372 doc623258",
"operator" : "and"
}
}
}
}
We are using haschild query to find the parent documents based on the condition.
We have two types
funnels
pages
funnels sample doc
{
"funnel_id": "12345",
"path": "a -> b -> c"
}
{
"funnel_id": "56789",
"path": "a -> d"
}
** pages sample doc**
{
"_parent": "12345",
"visited_page": "/home"
}
{
"_parent": "12345",
"visited_page": "/cart"
}
{
"_parent": "12345",
"visited_page": "/cart"
}
Condition1:
Find parent doc based child doc "visited_page" value contains "home".
"must" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"value" : ".*home.*",
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
It works perfectly.
Condition2
Find parent doc based child doc "visited_page" value does NOT contains "home".
"must_not" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"value" : ".*home.*",
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
But this query returned wrong results.
Output of the query
{
"funnel_id": "12345",
"path": "a -> b -> c"
}
{
"funnel_id": "56789",
"path": "a -> d"
}
You can see the parent id(funnel_id:12345) child doc contains visited page with value "home". But that also returns.
Expected Result
{
"funnel_id": "56789",
"path": "a -> d"
}
I believe you are "must_not"ing in the wrong spot
try:
"must" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"must_not": {
"value" : ".*home.*"
},
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
I was trying to search the following case
I want to search a name that ends with the particular word. For example:
name : group Test
name : group test
name : group test org
Here is my wild card query
"bool" : {
"must" : [
{
"wildcard" : {
"name.keyword"" : {
"wildcard" : "*test",
"boost" : 1.0
}
}
}
]
}
It returns me "group test" for case sensitive search
But I need to get both "group Test" and "group test" for case-insensitive search.
my mapping as follows:
"name":{
"type":"text",
"fielddata":true
"fields":{
"keyword":{
"type":"keyword"
}
}
}
Can anyone help me to find out queries in elasticsearch java api or any other way to search it.
Elastic search version 6.1.2
Any help is really appreciated.
Unfortunately there is no direct way to do this with ES configuration as keyword type does not have the analyzer property but I found a workaround. Please take a look on this solution:
PUT test
{
"settings": {
"analysis": {
"analyzer": {
"folding": {
"tokenizer": "standard",
"filter": [ "lowercase", "asciifolding" ]
}
},
"normalizer": {
"lowerasciinormalizer": {
"type": "custom",
"filter": [ "lowercase", "asciifolding" ]
}
}
}
},
"mappings": {
"_default_": {
"dynamic_templates": [
{
"string_as_keyword": {
"match_mapping_type": "string",
"match": "*_k",
"mapping": {
"type": "keyword",
"normalizer": "lowerasciinormalizer"
}
}
}
]
}
}
}
PUT test/1/123
{
"str_k" : "string âgáÈÒU is cool"
}
GET test/_search
{
"query": {
"wildcard": {
"str_k": "*agaeou*"
}
}
}
Hello i have this ood error where in elasticsearch searching for data i get this error.
`
[2017-11-09 18:14:30,977][DEBUG][action.search ] [Xorn] All shards failed for phase: [query]
RemoteTransportException[[Xorn][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: SearchParseException[failed to parse search source [{"from":0,"size":15,"query":{"bool":{"must":[{"query_string":{"query":"ctDossierType:(TrademarkApplication)"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"query_string":{"query":"design:(false)"}}]}},"sort":[{"stComparableApplicationNumber":{"order":"desc","missing":"_last","ignore_unmapped":true}}]}]]; nested: QueryParsingException[[nested] failed to find nested object under path [ntComments]];
Caused by: SearchParseException[failed to parse search source [{"from":0,"size":15,"query":{"bool":{"must":[{"query_string":{"query":"ctDossierType:(TrademarkApplication)"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"query_string":{"query":"design:(false)"}}]}},"sort":[{"stComparableApplicationNumber":{"order":"desc","missing":"_last","ignore_unmapped":true}}]}]]; nested: QueryParsingException[[nested] failed to find nested object under path [ntComments]];
`
the funny thing is it works when no argument is inputed in search field. System is duplicate to other system, and on other one it works. The thing is its all auto generated.
nested query
`
{
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
}
"path" : "ntComments"
}
} `
nested bool query
`{
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
}
ntComments
must : {
"bool" : {
"must" : [ {
"query_string" : {
"query" : "ctDossiertype:(typetypetype)"
}
}, {
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
},
"path" : "ntComments"
}
}, {
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
},
"path" : "ntComments"
}
}]
}
}
`
im going crazy over it, any ideas why its not working would be great.
I have 3 document,
{
"category":[{"id":"1"},{"id":"2"},{"id":"3"}]
}
{
"category":[{"id":"1"},{"id":"4"}]
}
{
"category":[]
}
How I can find document which have category.id in (2,3) like mysql,
also which type of DSL query I need to use in java api querybuilder
You can use the bool query with should clause to find all documents which contain either of the requested ids. Here is how the Query DSL would like
{
"query": {
"bool": {
"should": [
{"term": {
"category.id": {
"value": "2"
}
}},
{"term": {
"category.id": {
"value": "3"
}
}
}
]
}
}
}
Here's how you would use the Java API
QueryBuilders.boolQuery().should(QueryBuilders.matchQuery("category.id", "2"))
.should(QueryBuilders.matchQuery("category.id", "1"));
If the id field is not-analysed, you can also use the terms query. More info here https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html
Here's how your query should look like:
{
"query": {
"bool": {
"filter": [
{
"term": {
"category.id": "1"
}
},
{
"term": {
"category.id": "2"
}
}
]
}
}
}
Like this:
{
"filter": {
"terms": {
"category.id": ["2", "3"]
}
}
}