Hello i have this ood error where in elasticsearch searching for data i get this error.
`
[2017-11-09 18:14:30,977][DEBUG][action.search ] [Xorn] All shards failed for phase: [query]
RemoteTransportException[[Xorn][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: SearchParseException[failed to parse search source [{"from":0,"size":15,"query":{"bool":{"must":[{"query_string":{"query":"ctDossierType:(TrademarkApplication)"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"query_string":{"query":"design:(false)"}}]}},"sort":[{"stComparableApplicationNumber":{"order":"desc","missing":"_last","ignore_unmapped":true}}]}]]; nested: QueryParsingException[[nested] failed to find nested object under path [ntComments]];
Caused by: SearchParseException[failed to parse search source [{"from":0,"size":15,"query":{"bool":{"must":[{"query_string":{"query":"ctDossierType:(TrademarkApplication)"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"nested":{"query":{"bool":{"must":{"query_string":{"query":"ntComments.comment:(xcbxcv)"}}}},"path":"ntComments"}},{"query_string":{"query":"design:(false)"}}]}},"sort":[{"stComparableApplicationNumber":{"order":"desc","missing":"_last","ignore_unmapped":true}}]}]]; nested: QueryParsingException[[nested] failed to find nested object under path [ntComments]];
`
the funny thing is it works when no argument is inputed in search field. System is duplicate to other system, and on other one it works. The thing is its all auto generated.
nested query
`
{
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
}
"path" : "ntComments"
}
} `
nested bool query
`{
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
}
ntComments
must : {
"bool" : {
"must" : [ {
"query_string" : {
"query" : "ctDossiertype:(typetypetype)"
}
}, {
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
},
"path" : "ntComments"
}
}, {
"nested" : {
"query" : {
"bool" : {
"must" : {
"query_string" : {
"query" : "ntComments.comment:(blablabla)"
}
}
}
},
"path" : "ntComments"
}
}]
}
}
`
im going crazy over it, any ideas why its not working would be great.
Related
I'm currently learning about Spring Boot and am undertaking a project where users can make posts, view those posts, etc.
A user's post(s) can be viewed via http://localhost:8080/users/{user_id}/posts and http://localhost:8080/users/{user_id}/posts/{post_id}
As a result I have the following UserPostController
#RestController
#RequestMapping("/users")
public class UserPostController {
#Autowired
private UserPostService postService;
#GetMapping("/{user_id}/posts")
public List<Post> retrieveUserPosts(#PathVariable int user_id) {
return postService.retrieveUserPostList(user_id);
}
#GetMapping("/{user_id}/posts/{post_id}")
public EntityModel<Post> retrieveUserPost(#PathVariable int user_id, #PathVariable int post_id) {
return postService.retrieveUserPost(user_id, post_id);
}
#PostMapping("/{user_id}/posts")
public ResponseEntity<Object> createUserPost(#PathVariable int user_id, #Valid #RequestBody Post post) {
return postService.saveUserPost(user_id, post);
}
}
Every request to the links work correctly. For example a GET request to http://localhost:8080/users/1/posts returns [{"id":1,"description":"This is a post"},{"id":2,"description":"This is another post"}], which is the expected action.
However, for some reason I am able to visit http://localhost:8080/posts which then returns a list of all posts:
{
"_embedded" : {
"posts" : [ {
"description" : "This is a post",
"_links" : {
"self" : {
"href" : "http://localhost:8080/posts/1"
},
"post" : {
"href" : "http://localhost:8080/posts/1"
},
"user" : {
"href" : "http://localhost:8080/posts/1/user"
}
}
}, {
"description" : "This another post",
"_links" : {
"self" : {
"href" : "http://localhost:8080/posts/2"
},
"post" : {
"href" : "http://localhost:8080/posts/2"
},
"user" : {
"href" : "http://localhost:8080/posts/2/user"
}
}
}, {
"description" : "This is yet another post",
"_links" : {
"self" : {
"href" : "http://localhost:8080/posts/3"
},
"post" : {
"href" : "http://localhost:8080/posts/3"
},
"user" : {
"href" : "http://localhost:8080/posts/3/user"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "http://localhost:8080/posts"
},
"profile" : {
"href" : "http://localhost:8080/profile/posts"
}
},
"page" : {
"size" : 20,
"totalElements" : 3,
"totalPages" : 1,
"number" : 0
}
}
Through HATEOAS I am able to also see available links of the format http://localhost:8080/posts/{user_id}/user which I have also not created methods for, but they still exist.
Is there a reason why these unwanted routes exist? If so how do I change this?
Thank you :)
Please find my mapping query below for the filename field.
PUT /articles
{
"settings" : {
"analysis" : {
"analyzer" : {
"filename_search" : {
"tokenizer" : "filename",
"filter" : ["lowercase"]
},
"filename_index" : {
"tokenizer" : "filename",
"filter" : ["lowercase","edge_ngram"]
}
},
"tokenizer" : {
"filename" : {
"pattern" : "[^\\p{L}\\d]+",
"type" : "pattern"
}
},
"filter" : {
"edge_ngram" : {
"side" : "front",
"max_gram" : 50,
"min_gram" : 1,
"type" : "edgeNGram"
}
}
}
},
"mappings" : {
"doc" : {
"properties" : {
"filename" : {
"type" : "text",
"search_analyzer" : "filename_search",
"analyzer" : "filename_index"
}
}
}
}
}
If am trying to query series1333372 doc623258 and am expecting karthik_series1333372_oracle_page_doc623258_v1_en-EU.pdf. But it's giving all the files which is having series1333372, not even checking for doc623258.
Please find my query below
get articles/_search
{
"query" : {
"match" : {
"filename" : "series1333372 doc623258"
}
}
}
I am inserting the following sample documents for testing from Kibana
POST articles/doc/1
{
"filename" : "karthik_series1333372_oracle_page_doc623258_v1_en-EU.pdf"
}
POST articles/doc/2
{
"filename" : "karthik_series1333372_sun_page_doc658_v1_en-EU.pdf"
}
POST articles/doc/3
{
"filename" : "series1333372_oracle_page_doc623_v1_en-US.pdf"
}
POST articles/doc/4
{
"filename" : "Engineering series1333372 valve_page doc6232 v1_en-US.pdf"
}
POST articles/doc/5
{
"filename" : "Machines_series1333372_page_doc62258_v1_en-US.pdf"
}
POST articles/doc/6
{
"filename" : "AIX series1333372 IBM page doc62358 v1_en-EU.pdf"
}
The default operator of match is OR. If you want all your terms to be present change it like this
GET articles/_search
{
"query" : {
"match" : {
"filename" : {
"query": "series1333372 doc623258",
"operator" : "and"
}
}
}
}
This is how the json looks like
"Node" : {
"1-5-2018" : {
"10001" : {
"centre" : "centre_1",
"name" : "name_1",
"paidAmt" : "25000"
},
"10002" : {
"centre" : "centre_2",
"name" : "name_2",
"paidAmt" : "25000"
},
"10003" : {
"centre" : "centre_3",
"name" : "name_3",
"paidAmt" : "10000"
}
},
"2-5-2018 : {
"10004" : {
"centre" : "centre_4",
"name" : "name_4",
"paidAmt" : "20000"
}
}
I want to retrieve the values(centre, name and paidAmt) inside the given dates
I've tried using this particular query but it always shows data does not exist
Query query = databaseReference1.orderByKey().startAt(fromDate.getText().toString()).endAt(toDate.getText().toString());
query.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
if(dataSnapshot.exists()) {
Log.d(TAG, "Data exists within dates");
for(DataSnapshot ds : dataSnapshot.getChildren()) {
Log.d(TAG , "Data are " + ds.getValue().toString());
}
}
else {
Log.d(TAG, "Data does not exist within dates");
}
}
Please let me know if it's possible to retrieve all the data within dates. In future i'm planning to add multiple data inside different dates and want to retrieve them based on requirement. Maybe monthly or weekly.
You are using timestamp instead of a date it may look like this.
"Node" : {
"1525188857" : {
"10001" : {
"centre" : "centre_1",
"name" : "name_1",
"paidAmt" : "25000"
},
"10002" : {
"centre" : "centre_2",
"name" : "name_2",
"paidAmt" : "25000"
},
"10003" : {
"centre" : "centre_3",
"name" : "name_3",
"paidAmt" : "10000"
}
},
"1525188873" : {
"10004" : {
"centre" : "centre_4",
"name" : "name_4",
"paidAmt" : "20000"
}
}
We are using haschild query to find the parent documents based on the condition.
We have two types
funnels
pages
funnels sample doc
{
"funnel_id": "12345",
"path": "a -> b -> c"
}
{
"funnel_id": "56789",
"path": "a -> d"
}
** pages sample doc**
{
"_parent": "12345",
"visited_page": "/home"
}
{
"_parent": "12345",
"visited_page": "/cart"
}
{
"_parent": "12345",
"visited_page": "/cart"
}
Condition1:
Find parent doc based child doc "visited_page" value contains "home".
"must" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"value" : ".*home.*",
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
It works perfectly.
Condition2
Find parent doc based child doc "visited_page" value does NOT contains "home".
"must_not" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"value" : ".*home.*",
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
But this query returned wrong results.
Output of the query
{
"funnel_id": "12345",
"path": "a -> b -> c"
}
{
"funnel_id": "56789",
"path": "a -> d"
}
You can see the parent id(funnel_id:12345) child doc contains visited page with value "home". But that also returns.
Expected Result
{
"funnel_id": "56789",
"path": "a -> d"
}
I believe you are "must_not"ing in the wrong spot
try:
"must" : {
"has_child" : {
"query" : {
"regexp" : {
"url" : {
"must_not": {
"value" : ".*home.*"
},
"flags_value" : 65535
}
}
},
"child_type" : "session_pages"
}
}
I've been struggling with Elasticsearch templates, specifically in optional parameters. I'd like to add optional filters there. This is the code snippet I was trying out:
{
"filter" : {
"bool" : {
"must" : [
{{#ProductIDs.0}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.0}}
]
}
}
}
Of course I replaced " with \", uglified it, wrapped it up in { "template" :"_snippet_above_" }.
Now when I'm trying to call it using the following:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": [1,2]
}
}
It ignores parameter that I've provided, however when I try to do that in official mustache.io demo page - it works just fine.
I tried {{#ProductIDs.length}} option too - it didn't work out. After doing some research I've found out that there is one difference between mustache.js and mustache.java. I assumed that Elasticsearch uses JAVA version and it doesn't support length parameter, so I have to rely on isEmpty. So I've rewritten my query as follows:
{
"filter" : {
"bool" : {
"must" : [
{{^ProductIDs.isEmpty}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.isEmpty}}
]
}
}
}
Now when I query template with ProductIDs list - it works fine, however if I remove parameter, it brings no results. I assume it generates this:
{
"filter" : {
"bool" : {
"must" : [
{ "terms" : { "Product.ProductID" : [] } }
]
}
}
}
If I send empty array as Parameter - it works fine.
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
I assume this happens because "ProductIDs" are undefined and not empty.
Is there a way to cath this condition in mustache.java so I can ignore these parameters?
tl;dr;
The issue is that if I don't specify parameter in my search request via template, my condition is rendered as an empty array, see this:
{
"filter" : {
"bool" : {
"must" : [
{ "terms" : { "Product.ProductID" : [] } }
]
}
}
}
If I pass empty array as a parameter, see this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
It works as expected and doesn't generate filter condition as described in my template, because array doesn't have any data in it.
I want this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
}
}
To work same as this:
GET /statistic/_search/template
{
"template": {
"id": "test"
},
"params": {
"ProductIDs": []
}
}
A workaround probably not the most elegant would be to change template query to be a should clause and add a match_all clause for empty list.
example:
{
"filter" : {
"bool" : {
"should" : [
{ "terms" : { "status" : [ "{{#ProductIDs}}","{{.}}","{{/ProductIDs}}"] }}
{{^ProductIDs}},
{"match_all":{}}
{{/ProductIDs}}
]
}
}
}
Didn't try it, but shouldn't something like this work?
{
"filter" : {
"bool" : {
"must" : [
{{#ProductIDs}}
{{^ProductIDs.isEmpty}}
{ "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
{{/ProductIDs.isEmpty}}
{{#ProductIDs.isEmpty}}
{"match_all":{}}
{{/ProductIDs.isEmpty}}
{{/ProductIDs}}
{{^ProductIDs}}
{"match_all":{}}
{{/ProductIDs}}
]
}
}
}
Ain't pretty, maybe there's better way.
My suggestion to overcome this using a JSON template is:
{
"query": {
"bool": {
"must": [
{
"script": {
"script": {
"inline": "1==1 {{#ProductIDs}} && [\"{{#ProductIDs}}\",\"{{.}}\",\"{{/ProductIDs}}\"].contains(doc['Product.ProductID'].value){{/ProductIDs}}",
"lang": "painless"
}
}
}
]
}
}