I have an object that contains a Map<String, String> field:
#Document(indexName = "custom-creative-template")
#Setting(settingPath = "/elasticsearch/custom-creative-template-index.json")
public class CustomCreativeTemplateIndexDto {
Map<String, String> customValues;
}
When querying data for this index, I am getting the following error:
org.springframework.data.mapping.MappingException: Couldn't find PersistentEntity for type java.lang.String!
When using Map<String, Object> however, it works.
That's the Settings file for the index:
{
"index": {
"analysis": {
"normalizer": {
"sortable": {
"filter": [
"lowercase"
]
}
}
}
}
}
I have tried setting the FieldType in the Field annotation to FieldType.Text & FieldType.Flattened but I keep getting the same error.
Mappings:
{
"custom-creative-template": {
"mappings": {
"properties": {
"_class": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"customValues": {
"properties": {
"offerBullets": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"offerDescription": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"key": {
"type": "long"
},
"name": {
"type": "text",
"fields": {
"sort": {
"type": "keyword",
"normalizer": "sortable"
}
}
}
}
}
}
}
Works for me with field type object:
#Document(indexName = "custom-creative-template")
#Setting(settingPath = "/elasticsearch/custom-creative-template-index.json")
public class CustomCreativeTemplateIndexDto {
#Field(type = FieldType.Object)
Map<String, String> customValues;
}
Related
I have one abstract class and three other subclass. I would like those three subclasses as example on the Swagger/OpenAPI interface.
But the Swagger/OpenApi interface show just the abstract class and the first subclass fields.
OperationRequest
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "subType", visible = true)
#JsonSubTypes({
#JsonSubTypes.Type(value = InstallmentOperationRequest.class, name = "I"),
#JsonSubTypes.Type(value = CreditCardOperationRequest.class, name = "C"),
#JsonSubTypes.Type(value = SingleOperationRequest.class, name = "S")
})
#Schema(
description = "Parent operation request",
discriminatorProperty = "subType",
discriminatorMapping = {
#DiscriminatorMapping(value = "SingleOperation", schema = SingleOperationRequest.class),
#DiscriminatorMapping(value = "InstallmentOperation", schema = InstallmentOperationRequest.class),
#DiscriminatorMapping(value = "CreditCardOperation", schema = CreditCardOperationRequest.class)
})
public abstract class OperationRequest {
private String description;
private OperationTypeEnum type;
private OperationSubTypeEnum subType;
private BigDecimal value;
private String observations;
}
CreditCardOperationRequest
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonTypeName("C")
#Schema(allOf = OperationRequest.class)
public class CreditCardOperationRequest extends OperationRequest {
private String creditCard;
private LocalDate creditCardOperationDate;
private Integer creditCardInstallments;
}
SingleOperationRequest
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonTypeName("S")
#Schema(allOf = OperationRequest.class)
public class SingleOperationRequest extends OperationRequest{
private BigDecimal paidValue;
private YearMonth period;
}
InstallmentOperationRequest
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonTypeName("I")
#Schema(allOf = OperationRequest.class)
public class InstallmentOperationRequest extends OperationRequest{
private Integer installments;
private YearMonth initialInstallment;
}
Json generated
...
"components": {
"schemas": {
"CreditCardOperationRequest": {
"required": [
"creditCard",
"creditCardInstallments",
"description",
"subType",
"type",
"value"
],
"type": "object",
"allOf": [
{
"$ref": "#/components/schemas/OperationRequest"
},
{
"type": "object",
"properties": {
"creditCard": {
"type": "string",
"description": "Credit card identification",
"example": "abcd"
},
"creditCardOperationDate": {
"type": "string",
"description": "Credit card operation date",
"format": "date",
"example": "2021-01-10"
},
"creditCardInstallments": {
"minimum": 1,
"type": "integer",
"description": "Number of credit card installments",
"format": "int32",
"example": 5
}
}
}
]
},
"InstallmentOperationRequest": {
"required": [
"description",
"initialInstallment",
"installments",
"subType",
"type",
"value"
],
"type": "object",
"allOf": [
{
"$ref": "#/components/schemas/OperationRequest"
},
{
"type": "object",
"properties": {
"installments": {
"minimum": 2,
"type": "integer",
"description": "Number of installments",
"format": "int32",
"example": 5
},
"initialInstallment": {
"type": "string",
"description": "Initial installment period",
"example": "2021-01"
}
}
}
]
},
"OperationRequest": {
"required": [
"description",
"subType",
"type",
"value"
],
"type": "object",
"properties": {
"description": {
"type": "string",
"description": "Operation description",
"example": "abc"
},
"type": {
"type": "string",
"description": "Operation type",
"enum": [
"I",
"E"
]
},
"subType": {
"type": "string",
"description": "Operation subType",
"enum": [
"S",
"C",
"I"
]
},
"value": {
"type": "number",
"description": "Operation value",
"format": "double",
"example": 32.56
},
"observations": {
"maxLength": 2147483647,
"minLength": 2,
"type": "string",
"description": "Observations",
"example": "abc"
}
},
"description": "Parent operation request",
"discriminator": {
"propertyName": "subType"
}
},
"SingleOperationRequest": {
"required": [
"description",
"paidValue",
"subType",
"type",
"value"
],
"type": "object",
"allOf": [
{
"$ref": "#/components/schemas/OperationRequest"
},
{
"type": "object",
"properties": {
"paidValue": {
"type": "number",
"description": "Operation paid value",
"example": 12.56
},
"period": {
"type": "string",
"description": "Operation period",
"example": "2021-01"
}
}
}
]
}
...
}
}
...
My config:
#Configuration
#EnableElasticsearchRepositories(
basePackages = { "com.aj.new.repositories" })
public class ElasticDataSourceConfig {
#Bean
public RestHighLevelClient client() {
ClientConfiguration clientConfiguration = ClientConfiguration.builder()
.connectedTo("localhost:9200")
.build();
return RestClients.create(clientConfiguration).rest();
}
#Bean
public ElasticsearchRestTemplate elasticsearchTemplate() {
return new ElasticsearchRestTemplate(client());
}
}
Document:
#Document(indexName = "provider_search")
public class Provider {
#Id #Field
private Long id;
#Field(type = FieldType.Keyword)
private String search;
#Field(name = "ProviderName", type = FieldType.Keyword)
private String providerName;
#Field(name = "Address")
private String address;
#Field(name = "City")
private String city;
#Field(name = "State")
private String state;
...getters...
...setters...
}
Usage:
#Service
public class MySearchService {
#Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
public List<Provider> searchProvidersUsingElastic(final String
providerName, final AddressSearchCriteriaBean addressCriteria) {
final NativeSearchQueryBuilder searchQueryBuilder = new
NativeSearchQueryBuilder();
if (providerName != null) {
final String regex = ".*" + providerName + ".*";
searchQueryBuilder.withQuery(regexpQuery("providerName", regex));
}
if (addressCriteria.getState() != null) {
searchQueryBuilder.withFilter(matchQuery("state",
addressCriteria.getState())
.fuzziness(Fuzziness.ONE));
}
SearchHits<Provider> articles =
elasticsearchRestTemplate.search(searchQueryBuilder.build(),
Provider.class, IndexCoordinates.of("provider_search"));
final List<Provider> providers = articles
.stream()
.map(SearchHit::getContent)
.collect(Collectors.toList());
return providers;
}
}
When debugging with or without filters, I get providers with only their ID field populated. Every other field like "search", "state", etc. is null.
This is my first venture in ElasticSearch world and I'm not sure what's wrong here. Any help is appreciated.
Edit:
Provider Mappings from Elasticsearch
{
"provider_search": {
"mappings": {
"properties": {
"Address": {
"type": "text",
"analyzer": "autocomplete"
},
"City": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"FaxNumber": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"PhoneNumber": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"ProviderName": {
"type": "text",
"analyzer": "autocomplete"
},
"State": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Status": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Zip": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"fac_dbk": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"search": {
"type": "text",
"analyzer": "autocomplete"
}
}
}
}
}
Please note that for testing purposes, I have not mapped every field of Provider on Java side. If that's problematic, let me know.
Second Update:
I have changed the Provider document to map the Field names as is. Still, everything except id is still null.
#Document(indexName = "provider_search")
public class Provider {
#Id
private Long id;
private String search;
private String ProviderName;
private String Address;
private String City;
private String State;
...getters...
...setters...
}
UPDATE:
Turns out the Elasticsearch index had a bug and the fields I had mapped on the Java side were not available on the ES index. It has been fixed and I'm seeing all the values populate correctly.
You have an inconsistency in using the property names (the properties of the Provider entity) and the corresponding field names (the names in Elasticsearch).
Your properties start with a lowercase letter and then are camelcase (providerName, state), for Spring Data Elasticsearch you define them to map to versions starting with a uppercase letter (ProviderName, State) with the #Field annotations. So Spring Data Elasticsearch expects Elasticsearch to return a value for ProviderName and will then store this in the providerName property.
Are the field names in your index starting with an uppercase letter? What does http://localhost:9200/provider_search/_mappings show? This is a guess, as you did not show what Elasticsearch returns, but would explain your error.
I'm using elasticsearch as indexing engine in my java project. I'm asking if there is anyway to index a java.util.Map a key/value Map.
For example i have this java class:
public class NextEvent extends NewtonResource{
private Map<String, String> metadata;
private Instant executionDate;
private String type;
protected Boolean executed;
private List<PatchOperation> jsonPatch;
private List<String> crons;
...
}
I want to create an elasticsearch mapping including the Map of metadata variable. somethings like this:
{
"aliases": {
"posc-alias-nextevent": {}
},
"mappings": {
"nextevent": {
"properties": {
"executed": {
"type": "boolean"
},
"executionDate": {
"type": "date"
},
"type": {
"type": "string",
"index": "not_analyzed"
},
"metadata": {
"type": "nested",
"properties": {
"key": {
"type": "string",
"index": "not_analyzed"
},
"value": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
I am new to the Elastic search world.Basically I am trying to retrieve the nested objects based on the ID.This is the JSON representation of my document.
{
"_index": "xyz",
"_type": "abc",
"_id": "12",
"_version": 1,
"found": true,
"_source":
{
"lastModifiedBy": "12",
"lastModifiedDate": "2015-12-31T19:45:29.493Z",
"profile":
[
{
"type": "nested",
"views":
[
{
"type": "nested",
"id": "view1",
"name": "view1",
"properties":
[
{
"name": "default",
"value": false
}
],
"widgets":
[
{
"type": "nested",
"id": "graph",
"name": "graph",
"elementId": "ui_graph",
"properties":
[
{
"name": "currency",
"value": "YEN"
}
]
}
]
}
} ] } ]
I am trying to get the widgets based on the view id.This is the my search query.
"query" : {
"term" : {
"_id" : "12"
}
},
"post_filter" : {
"nested" : {
"query" : {
"filtered" : {
"query" : {
"match_all" : { }
},
"filter" : {
"term" : {
"profile.views.id" : "view1"
}
}
}
},
"path" : "profile.views"
}
}
}
I am not sure what is wrong here.But getting "nested object under path [profile.views] is not of nested type]".
Below is my mapping structure
{
"xyz": {
"mappings": {
"abc": {
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"name": {
"type": "string"
},
"profile": {
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"type": {
"type": "string"
},
"views": {
"properties": {
"id": {
"type": "string"
},
"isDefault": {
"type": "boolean"
},
"name": {
"type": "string"
},
"properties": {
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
"type": {
"type": "string"
},
"viewId": {
"type": "string"
},
"widgets": {
"properties": {
"elementId": {
"type": "string"
},
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"properties": {
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
"type": {
"type": "string"
}
}
}
}
}
}
}
}
}
}
}
}
Please help!
You are getting the error because you have not specified type as nested for profile and views. Refer to the Docs for how to created nested objects . You should be defining type as nested for every nested object like this
{
"xyz": {
"mappings": {
"abc": {
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"name": {
"type": "string"
},
"profile": {
"type": "nested", <--- here, you need this for every nested object
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"type": {
"type": "string"
},
"views": {
"type": "nested",
"properties": {
"id": {
"type": "string"
},
"isDefault": {
"type": "boolean"
},
"name": {
"type": "string"
},
"properties": {
"type": "nested",
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
"type": {
"type": "string"
},
"viewId": {
"type": "string"
},
"widgets": {
"type": "nested",
"properties": {
"elementId": {
"type": "string"
},
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"properties": {
"type": "nested",
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
"type": {
"type": "string"
}
}
}
}
}
}
}
}
}
}
}
}
Hope this helps!!
Here is my scenario:
I have a schema for my class that is validated by Avro:
{
"type": "record",
"name": "MyCLass",
"namespace": "com.somepackage",
"fields": [
{
"name": "attributes",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "KeyValuePair",
"fields": [
{
"name": "key",
"type": "string"
},
{
"name": "value",
"type": "string"
}
]
},
"java-class": "java.util.List"
}
},
{
"name": "someString",
"type": "string"
},
{
"name": "myclass1",
"type": {
"type": "record",
"name": "MyClass1",
"fields": [
{
"name": "attributes",
"type": {
"type": "record",
"name": "ArrayOfKeyValuePair",
"fields": [
{
"name": "item",
"type": {
"type": "array",
"items": "KeyValuePair",
"java-class": "java.util.List"
}
}
]
}
},
{
"name": "labels",
"type": {
"type": "record",
"name": "ArrayOfXsdString",
"fields": [
{
"name": "item",
"type": {
"type": "array",
"items": "string",
"java-class": "java.util.List"
}
}
]
}
},
{
"name": "uniqueID",
"type": "string"
}
]
}
},
{
"name": "MyClass2",
"type": {
"type": "record",
"name": "MyClass2",
"fields": [
{
"name": "attributes",
"type": "ArrayOfKeyValuePair"
},
{
"name": "someString",
"type": "string"
},
{
"name": "someLabel",
"type": "ArrayOfXsdString"
},
{
"name": "someId",
"type": "string"
}
]
}
}
]
}
This schema was generated with `GenericRecord obj = new GenericData.Record(ReflectData.get().getSchema(MyClass.class));
next I build the list of 4 parameters that the obj expects in it's values list:
KeyValuePair kv1 = new KeyValuePair();
kv1.setKey("key1");
kv1.setValue("val1");
KeyValuePair kv2 = new KeyValuePair();
kv2.setKey("key2");
kv2.setValue("val2");
List<KeyValuePair> attr = new ArrayList<KeyValuePair>();
attr.add(kv1);
attr.add(kv2);
ArrayOfKeyValuePair arrKV = new ArrayOfKeyValuePair();
arrKV.getItem().add(kv1);
MyClass1 ud = new MyClass1();
ud.setAttributes(arrKV);
ud.setUniqueID("SomeID");
MyCLass2 sd = new MyCLass2();
sd.setAttributes(arrKV);
sd.setExternalCaseID("SomeID");
sd.setUniqueID("SomeId");
obj.put("attributes", attr);
obj.put("workFlowName", "Nume workflow");
obj.put("userDescriptor", ud);
obj.put("subscriberDescriptor", sd);
And when I try:
ByteArrayOutputStream out = new ByteArrayOutputStream();
DatumWriter<GenericRecord> writerS = new GenericDatumWriter<GenericRecord>(schemaMentionedAbove);
Encoder encoder = EncoderFactory.get().binaryEncoder(out, null);
writerS.write(obj, encoder);
encoder.flush();
out.close();
The code fails at line: writerS.write(obj, encoder); with error:
KeyValuePair cannot be cast to org.apache.avro.generic.IndexedRecord
KeyValuePair class is a simple class with 2 fields: String key, String value.
After debugging I can see that DatumWriter fails on iterating through the first "attributes" array of records.
Any help appreciated! Thanks