I'm trying to create index following this guide: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-create-index.html#_providing_the_whole_source
The problem is, that index is not created properly. Looks like whole settings section as well as completion type is ignored.
My json file:
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"analysis": {
"filter": {},
"analyzer": {
"keyword_analyzer": {
"filter": [
"lowercase",
"asciifolding",
"trim"
],
"char_filter": [],
"type": "custom",
"tokenizer": "keyword"
}
}
}
},
"mappings": {
"my_type": {
"properties": {
"first": {
"type": "text",
"fields": {
"keywordstring": {
"type": "text",
"analyzer": "keyword_analyzer"
},
"completion": {
"type": "completion"
}
},
"analyzer": "standard"
},
"second": {
"type": "text",
"fields": {
"keywordstring": {
"type": "text",
"analyzer": "keyword_analyzer"
},
"completion": {
"type": "completion"
}
},
"analyzer": "standard"
},
"third": {
"type": "text",
"fields": {
"keywordstring": {
"type": "text",
"analyzer": "keyword_analyzer"
},
"completion": {
"type": "completion"
}
},
"analyzer": "standard"
},
"fourth": {
"type": "text",
"fields": {
"keywordstring": {
"type": "text",
"analyzer": "keyword_analyzer"
},
"completion": {
"type": "completion"
}
},
"analyzer": "standard"
}
}
}
}
}
Java code:
CreateIndexRequest indexRequest = new CreateIndexRequest(ESClientConfiguration.INDEX_NAME);
URL url = Resources.getResource(TERYT_INDEX_CONFIGURATION_FILE_NAME);
return Try.of(() -> Resources.toString(url, Charsets.UTF_8))
.map(jsonIndexConfiguration -> indexRequest.source(jsonIndexConfiguration, XContentType.JSON))
.get();
createIndexRequest.setTimeout(TimeValue.timeValueMinutes(2));
Try.of(() -> client.indices().create(createIndexRequest, RequestOptions.DEFAULT))...
Index is created but when I look into Index Metadata, it looks completly wrong:
{
"state": "open",
"settings": {
"index": {
"creation_date": "1556379012380",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "L5fmkrjeQ6eKmuDyZ3MP3g",
"version": {
"created": "7000099"
},
"provided_name": "my_index"
}
},
"mappings": {
"my_type": {
"properties": {
"first": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
},
"second": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
},
"third": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
},
"fourth": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
}
}
},
"aliases": [],
"primary_terms": {
"0": 1
},
"in_sync_allocations": {
"0": [
"Cx6tBeohR8mzbTO74dwsCw",
"FcTUhpb_SL2LiaEyy_uwkg"
]
}
}
There is only 1 shard without replicas, I also don't see any informations about completion type. Does someone could tell me what I'm doing wrong here?
I think that this line:
Try.of(() -> client.indices().create(createIndexRequest, RequestOptions.DEFAULT))...
is hiding an important exception.
Here you are using elasticsearch 7.0.0 which does not allow anymore giving a "type" name in your mapping.
Instead of
"mappings": {
"my_type": {
"properties": {
You should write:
"mappings": {
"properties": {
Because of the exception and the fact that probably after index creation you are indexing some documents, default index settings and mapping are applied.
Which explains what you are seeing.
You need to fix your index settings first.
I'd recommend doing that in Kibana dev console.
Related
Request JSON:
{
"apiVersion":"0.0.9b",
"apiDate":"18.01.19",
"general":{
"documentType": "invoice",
},
"references":{
"invoiceId":"123",
"invoiceDate":"01.01.1970",
"creditNoteId":"123",
"creditNoteDate":"01.01.1970"
}
}
I have to validate request json based on documentType value if documentType is invoice then invoiceId and invoiceDate required if documentType is creditNote then creditNoteId and creditNoteDate required.
I tried the below schema but it is not working as expected
Updated JSON Schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"apiVersion",
"apiDate"
],
"oneOf": [
{
"properties": {
"general": {
"properties": {
"documentType": {
"enum": [
"invoice"
]
}
}
},
"references": {
"required": [
"invoiceId",
"invoiceDate"
]
}
}
},
{
"properties": {
"general": {
"properties": {
"documentType": {
"enum": [
"creditNote"
]
}
}
},
"references": {
"required": [
"creditNoteId",
"creditNoteDate"
]
}
}
}
],
"properties": {
"apiVersion": {
"type": "string",
"description": "The version of the json file"
},
"apiDate": {
"type": "string",
"description": "The date when the json version was published"
},
"general": {
"$ref": "#/definitions/general_identifiers"
},
"references": {
"type": "object",
"properties": {
"invoiceId": {
"type": "string"
},
"invoiceDate": {
"type": "string"
},
"creditNoteId": {
"type": "string"
},
"creditNoteDate": {
"type": "string"
}
}
}
},
"definitions": {
"general_identifiers": {
"type": "object",
"required": [
"documentType"
],
"properties": {
"documentType": {
"enum": [
"invoice",
"creditNote"
]
}
}
}
}
}
No validation Error even if m not setting any of required field.
What have I missed?
Thanks in advance :)
The validator response is correct. Your schema requires the "references" property to be under the "general" node.
But this is not the case in your sample json (it is on the same level, not as child)
"general":{
"documentType": "invoice",
},
"references":{
...}
I only want to highlight the words I search in the query, not including the synonym, but I also hope es can return the search result can contain the synonym search result, here is an example.
PUT /my_test_index/
{
"settings": {
"analysis": {
"filter": {
"native_synonym": {
"type": "synonym",
"ignore_case": true,
"expand": true,
"synonyms": [
"apple,fruit"
]
}
},
"analyzer": {
"test_analyzer": {
"tokenizer": "whitespace",
"filter": [
"native_synonym"
]
}
}
}
},
"mappings": {
"properties": {
"desc": {
"type": "text",
"analyzer": "test_analyzer"
}
}
}
}
POST /my_test_index/_doc
{
"desc": "apple"
}
POST /my_test_index/_doc
{
"desc": "fruit"
}
GET /my_test_index/_search
{
"query": {
"match": {
"desc": "apple"
}
},
"highlight": {
"fields": {
"desc": {}
}
}
}
However, es highlight both fruit and apple while I only want the apple get highlight.
Anyone knows how to solve this? Thanks in advance :)
"hits": [
{
"_index": "my_test_index",
"_type": "_doc",
"_id": "RMyZrXAB7JsJEwsbVF33",
"_score": 0.29171452,
"_source": {
"desc": "apple"
},
"highlight": {
"desc": [
"<em>apple</em>"
]
}
},
{
"_index": "my_test_index",
"_type": "_doc",
"_id": "RcyarXAB7JsJEwsboF2V",
"_score": 0.29171452,
"_source": {
"desc": "fruit"
},
"highlight": {
"desc": [
"<em>fruit</em>"
]
}
}
]
You can add a highlight query that behaves different to your actual search query. All you need then is a field indexed without the synonyms, and you should be able to get what you want:
PUT /my_test_index/
{
"settings": {
"analysis": {
"filter": {
"native_synonym": {
"type": "synonym",
"ignore_case": true,
"expand": true,
"synonyms": [
"apple,fruit"
]
}
},
"analyzer": {
"test_analyzer": {
"tokenizer": "whitespace",
"filter": [
"native_synonym"
]
}
}
}
},
"mappings": {
"properties": {
"desc": {
"type": "text",
"analyzer": "test_analyzer",
"fields": {
"raw": {
"type": "text",
"analyzer": "whitespace"
}
}
}
}
}
}
GET /my_test_index/_search
{
"query": {
"match": {
"desc": "apple"
}
},
"highlight": {
"fields": {
"desc.raw": {
"highlight_query": {
"match": {
"desc.raw": "apple"
}
}
}
}
}
}
I have a JSON
{
"Id": "xxx",
"Type": "Transaction.Create",
"Payload": {
"result": 2,
"description": "Pending",
"body": {
"redirect": {
"url": "xxx",
"fields": {
"MD": "8a829449620619e80162252adeb66a39"
}
},
"card": {
"expiryMonth": "1",
"expiryYear": "2033"
},
"order": {
"amount": 1
}
}
}
}
And I want to remove the card info of it like this:
{
"Id": "xxx",
"Type": "Transaction.Create",
"Payload": {
"result": 2,
"description": "Pending",
"body": {
"redirect": {
"url": "xxx",
"fields": {
"MD": "8a829449620619e80162252adeb66a39"
}
},
"order": {
"amount": 1
}
}
}
}
How can I do this with Apache velocity?
What works is:
#set($content = $util.urlEncode($input.json('$')))
#set($new = $content.replaceAll("2033","2055"))
Action=SendMessage&MessageBody={"body": "$new","Event-Signature": "$util.urlEncode($input.params('Event-Signature'))"}
This gives me
{
"Id": "xxx",
"Type": "Transaction.Create",
"Payload": {
"result": 2,
"description": "Pending",
"body": {
"redirect": {
"url": "xxx",
"fields": {
"MD": "8a829449620619e80162252adeb66a39"
}
},
"card": {
"expiryMonth": "1",
"expiryYear": "2050"
},
"order": {
"amount": 1
}
}
}
}
But now I want to remove the card part but it does not work:
#set($content = $util.urlEncode($input.json('$')))
#set($new = $content.delete("$.Payload.body.card"))
Action=SendMessage&MessageBody={"body": "$new","Event-Signature": "$util.urlEncode($input.params('Event-Signature'))"}
what am I doing wrong?
Main goal is transform a mapping template in API Gateway for a webhook. The webhook contains to many information and we want to remove some part of the JSON POST call.
Try using the below
#set($dummy=$content.Payload.remove("card"))
I'm trying to insert an object to a nested array through the java api, but I get the following error
MapperParsingException[object mapping for [X] tried to parse field [null] as object, but found a concrete value]
doing it through Kibana as shown below the same script works.
Any ideas on how to fix this?
The java code is the following
HashMap<String, Object> params = new HashMap<>();
params.put("object", objectAsString);
Script script = new Script(ScriptType.INLINE, "painless", "ctx._source.media.add(params.object)", params);
UpdateResponse result = elasticClient.prepareUpdate(indexName, "Type", documentId).setScript(script).execute().actionGet();
Trhough Kibana
POST index/document/id/_update
{
"script": {
"lang": "painless",
"inline": "ctx._source.media.add(params.object)",
"params": {
"object": {
"description" : "A second image",
"height" : 5,
"weight": 5,
"name" : "Test",
"orientation" : "Vertical",
"url" : "htttp://newurl.jpg",
"tags":["first","second"],
"type":"image"
}
}
}
}
The template mapping is the following:
"mappings": {
"FOLDER_MODULE": {
"properties": {
"name": {
"type": "keyword"
},
"publisherId": {
"type": "keyword"
},
"username": {
"type": "keyword"
},
"media": {
"type": "nested",
"properties": {
"id": {
"type": "text"
},
"name": {
"type": "text"
},
"publisherId": {
"type": "short"
},
"tags": {
"type": "text"
},
"description": {
"type": "text"
},
"url": {
"type": "keyword"
},
"createdDate": {
"format": "strict_date_optional_time||epoch_millis",
"type": "date"
},
"height": {
"type": "float"
},
"width": {
"type": "float"
},
"weight": {
"type": "float"
},
"orientation": {
"type": "keyword"
},
"status": {
"type": "keyword"
},
"type": {
"type": "keyword"
},
"username": {
"type": "keyword"
}
}
}
}
}
}
I want to create an API with following spec.:
project-api.json
{
"Project": {
"required": [
"uuid"
],
"properties": {
"uuid": {
"type": "string",
"format": "string"
}
"Tasks": {
"type": "array",
"items": {
"$ref": "models/task.json"
}
}
}
...
task.json
{
"Task": {
"required": [
"uuid"
],
"properties": {
"user": {
"type": "string"
},
"title": {
"type": "string"
},
...
Instead of making a reference to models/tasks.json, the generated api generates:
List<ModelstasksJson>
Anyone know how to reference to models from another json file?