We Keep Coding

Convert XML to JSON with different property names using Jackson

I have the next task: read XML file from some directory and convert it to JSON string.
The problem: initial XML and JSON have different names for corresponding properties, e.g. x_date in XML and j_date in JSON.
I have created the class with required field for JSON with such annotations:
public class Card {
#JacksonXmlProperty(localName = "x_date")
#JsonProperty("j_date")
private String date;
// other fields
I have tried to serialize/deserialize test XML file, and it's seems work correctly.
But I'm not sure that is ok to annotate fields with #JacksonXmlProperty and #JsonProperty annotations at the same time. Maybe it's better to create one class per XML part and one for the JSON and transfer the data between them some mapper (e.g. Orika)?
Any suggestions?

Finally solved this by splitting logic in two separate classes: Card.class for XML data with help of #JacksonXmlProperty annotation and CardDto.class which uses #JsonProperty. Mapping between these classes is handled by Orika mapper.
This split will ease further customization of both classes and will allow add new functionality (e.g. persist data to the database using new entity class).

Validate JSON data against Yaml shema

I have a java object (let's name it JsonValidator) that can be configured in YAML file.
I would like to describe a schema of JSON objects in YAML notation something like this
And then I need to validate JSON objects according to the schema. Does anybody know any Java libs I can use or any examples?
Thanks

The schema of a json document can be defined using json schema (actually, OpenAPI uses a custom flavor of json schema). It is essentially a json document defining the structure of an other json document. There are a few java implementations out there. If you want to stick to YAML for defining the schema, then will first need to convert YAML to JSON and then use the json schema validator, see this SO question for doing that.

You can find some Java validators for JSON Schema here.

Spring Data Elasticsearch - Create keyword field with normalizer

We are using the spring-data-elasticsearch project to interface with our elasticsearch clusters, and have been using it now for around a year. Recently, we moved to elasticsearch 5.x (from 2.x) where we now have the "keyword" datatype.
I would like to index these keywords as lowercase values, which I know can be done with field normalizers. I can't find anywhere in the documentation or online where I can add a normalizer to a field through the annotation based mapping.
E.g
#Field(type = FieldType.keyword, <some_other_param = some_normalizer>)
Is this something that can be done? I know that we can use JSON based mapping definitions as well, so I will fall back to that option if needed, but would like to be able to do it this way if possible.
Any help would be very appreciated!

Since the pull request of #xhaggi has been merged (spring-data-elasticsearch 3.1.3+ or Spring Boot 2.1.1), we have a normalizer field in the #Field annotation.
To use it, we need:
declare a #Field or an #InnerField with params type = FieldType.Keyword, normalizer = "%NORMALIZER_NAME%"
add #Setting(settingPath = "%PATH_TO_NORMALIZER_JSON_FILE%") at the class level.
put the normalizer mapping into a json file at %PATH_TO_NORMALIZER_JSON_FILE%
Example of usage

FYI, for anyone looking at this, the answer is there is not a way to do this at this time.
You can do this, however, by creating your mappings file as JSON in the Elasticsearch format. See:
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html
You can then create that JSON file and link it to your Domain model with.
#Mapping(mappingPath = "some/path/mapping.json")
Note that this is not, in my experience, compatible with the provided annotation based mapping for fields.

There is a pending issue https://jira.spring.io/browse/DATAES-492 waiting for review.

Elasticsearch - what to do if fields have the same name but multiple mapping

I use Elasticsearch for storing data sent from multiple sources outside of my system, i.e. I'm not controlling the incoming data - I just receive json document and store it. I have no logstash with its filters in the middle, only ES and Kibana. Each data source sent its own data type and all of them are stored in the same index (per tenant) but in different types. However since I cannot control the data that is sent to me, it is possible to receive documents of different types with the field having the same name and different structure.
For example, assume that I have type1 and type2 with field FLD, which is an object in both cases but the structure of this object is not the same. Specifically FLD.name is a string field in type1 but an object in type2. And in this case, when type1 data arrives it is stored successfully but when type2 data arrives, it is rejected:
failed to put mappings on indices [[myindex]], type [type2]
java.lang.IllegalArgumentException: Mapper for [FLD] conflicts with existing mapping in other types[Can't merge a non object mapping [FLD.name] with an object mapping [FLD.name]]
ES documentation clearly declare that fields with the same name in the same index in different mapping types mapped to the same field internally and must have the same mapping (see here).
My question is what can I do in this case? I'd prefer to keep all the types in the same index. Is it possible to add a unique-per-type suffix to field names or something like this? Any other solution? I'm a newbie in Elasticsearch so maybe I'm missing something simple... Thanks in advance.

There is no way to do index arbitrary JSON without pre-processing before it's indexed - not even Dynamic templates are flexible enough.
You can flatten nested objects into key-value pairs and use a Nested datatype, Multi-fields, and ignore_malformed to index arbitrary JSON (even with type conflicts) as described here. Unfortunately, Elasticsearch can still throw an exception at query time if you try to, for example, match a string to kv_pairs.value.long, so you'll have choose appropriate fields based on format of the value.

It's not the best practice I suppose, but you can store the field content as a String and make the deserialization manually after retrieve the information.
So, imagine a class like:
class Person {
private final Object name;
}
That can receive a List of String or a List of any other Object, just for example
So, instead of serialize the Person to a String and save it, you can serialize to a String and save the content on another class, like:
String personContent new ObjectMapper().writeValueAsString(person);
RequestDto dto = new RequestDto(personContent);
String dtoContent new ObjectMapper().writeValueAsString(dto);
And save the dtoContent:
IndexRequest request = new IndexRequest("persons")
request.source(dtoContent, XContentType.JSON);
IndexResponse response = client.index(request, RequestOptions.DEFAULT);
The RequestDto will be a simple class with a String field:
class RequestDto {
private String content;
}
I'm not a expert on ElasticSearch, but probably you will loose a lot of features from the ElasticSearch by passing his validations doing that.

How do I get Jersey to show a List in an object persisted by Hibernate?

Jersey is not showing a list in the JSON output when I retrieve the object using Hibernate. The list within the object is defined like this:
#OneToMany(cascade=CascadeType.ALL)
#OrderColumn
private List<Project> projects = new ArrayList<Project>();
When I retrieve the object (which also contains the projects list), I get the normal fields (ints and Strings and such), but not this list. When I use the debugger, I can see that the list is indeed there, but Jersey doesn't output it in JSON.

It looks like you need to configure a JSON Serializer such as Jackson. The answers to this question have some guidance on how to do that.
Once you have Jackson with JAXB support configured, you will need to add appropriate JAXB annotations to the Project class (either XML based one or JSON based ones, the serializer can be configured to support either or both). So, for example adding this to Project
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name = "")
#XmlRootElement(name = "project"))
public class Project {
Should be enough to serialize Project and it's fields to JSON.