We Keep Coding

Spring Data Elasticsearch (4.x) - Using #Id forces id field in _source

Summary
Recently we upgraded to Spring Data Elasticsearch 4.x. Part of this major release meant that Jackson is no longer used to convert our domain objects to json (using MappingElasticsearchConverter instead) [1]. This means we are now forced to add a new id field to all our documents.
Previously we had domain objects like this:
import org.springframework.data.annotation.Id;
public ESDocument {
#Id
private String id;
private String field1;
#JsonIgnore
public String getId() {
return id;
}
public String getField1() {
return field1;
}
Which resulted in documents like this in ES:
{
"_index" : "test_index",
"_type" : "_doc",
"_id" : "d5bf7b5c-7a44-42f9-94d6-d59fe3988482",
"_score" : 1.0,
"_source" : {
"field1" : "blabla"
}
}
Note that:
The #JsonIgnore annotation used to ensure that we were not required to have a id field in the _source.
We are setting the document id ourselves and it ends up in _id.
Problem
With Spring Data Elastic 4.x the #JsonIgnore annotation is no longer respected which means we are now forced to have an id field in the _source as shown below:
{
"_index" : "test_index",
"_type" : "_doc",
"_id" : "d5bf7b5c-7a44-42f9-94d6-d59fe3988482",
"_score" : 1.0,
"_source" : {
"id": "d5bf7b5c-7a44-42f9-94d6-d59fe3988482",
"field1" : "blabla"
}
}
Questions
Is it no longer possible to omit the duplication of the identifier of the document (i.e. in the _id and id fields)? If so how? (Note we already tried #org.springframework.data.annotation.Transient which does not work because spring-data-elastic then thinks our document does not have an id).
Was our previous approach of suppressing the id field in _source incorrect or problematic?
Versions
java: 1.8.0_252
elasticsearch: 7.6.2
spring-boot: 2.3.1.RELEASE
spring-data-elastic: 4.0.1.RELEASE
References
[1] - https://spring.io/blog/2020/05/27/what-s-new-in-spring-data-elasticsearch-4-0

Question 1:
To omit the id field from the _source, you would normally use the #Transient annotation, but as you wrote, this does not work for the id property. Transient properties are ignored in Spring Data modules (not only Spring Data Elasticsearch).
But you you can use the org.springframework.data.annotation.ReadOnlyProperty annotation for this:
#Id
#ReadOnlyProperty
private String id;
To be honest, I didn't know up to now that this exists, this comes from Spring Data Commons as well and is checked in the isWriteable() method of the property when properties are written by the MappingElasticsearchConverter .
Question 2:
Surely not incorrect, but problematic as you found out. We always consider the whole entity when storing it, so we never thought about not writing the id. Strictly speaking, it is not necessary, there you're right, because we always get the id back in the _id field together with the _source, so we can easily put the entity back together, but we never considered this a necessary feature to have.
Note:
When you look at the data in your ES index you will find that with the MappingElasticsearchConverter an additional _source field named _class is written which contains the name of the entity class (or a defined alias). This allows for mapping generics; for further info check the documentation - just in case you wonder where this comes from.
Edit 18.11.2022:
Recently (with version 4.4.3) we had a change that fixed a wrong behaviour in Spring Data Elasticsearch: Spring Data Elasticsearch must not write data into a property that is marked with #ReadOnlyProperty. This leads to the proposed solution not working any longer because on reading data from Elasticsearch the id property is not filled anymore.
To get the id property being set in this case it is necessary to add an AfterConvertCallback to your application:
#import org.springframework.data.elasticsearch.core.event.AfterConvertCallback;
#Component
public class EntityAfterConvertCallback implements AfterConvertCallback<EsDocument> {
#Override
public EsDocument onAfterConvert(EsDocument entity, Document document, IndexCoordinates indexCoordinates) {
entity.setId(document.getId());
return entity;
}
}

type-preserving deserialization of GraphQL response using schema

I am trying to write deserialization code for responses of user-defined GraphQL queries. The code has access to the query response in JSON-serialized form and the underlying GraphQL schema (by querying the endpoint's schema.json or making introspection requests).
Assume the following schema:
scalar Date
type User {
name: String
birthday: Date
}
type Query {
allUsers: [User]
}
schema {
query: Query
}
And the following query:
query {
allUsers {
name
birthday
}
}
The response may look like this (only includes the data.allUsers-field from the full response for brevity):
[
{"name": "John Doe", "birthday": "1983-12-07"}
]
What I am attempting to do is deserialize the above response in a manner that preserves type information, including for any custom scalars. In the above example, I know by convention that the GraphQL scalar Date should be deserialized as LocalDate in Java, but just from the response alone I do not know that the birthday field represents the GraphQL scalar type Date, since it's serialized as a regular string in JSON.
What I can do is try to utilize the GraphQL schema for this. For the above example, the schema may look something like this (shortened for brevity):
...
"types": [
{
"kind": "OBJECT",
"name": "User",
"fields": [
{
"name": "name",
"type": {
"kind": "SCALAR",
"name": "String"
}
},
{
"name": "birthday"
"type": {
"kind": "SCALAR",
"name": "Date"
}
}
...
From this information I can deduce that that response's birthday field is of type Date, and deserialize it accordingly. However, things get more complicated if the query uses non-trivial GraphQL features. Take aliasing for example:
query {
allUsers {
name
dayOfBirth: birthday
}
}
At this point I would already need to keep track of any aliasing (which I could do since that information is available if I parse the query), and backtrack those to find the correct type. I fear it might get even more complicated if e.g. fragments are used.
Given that I use graphql-java, and it appears to already need to handle all of these cases for serialization, I wondered if there was an easier way to do this than to manually backtrack the types from the query and schema.

How about generating java classes from the schema and then using those classes to deserialize. There is one plugin which I have used before for this - graphql-java-generator
You may need to enhance the plugin a bit to support your custom scalars though
It basically generates a java client for invoking your GraphQL queries in a Java way.

I had the same problem to deserialize an LocalDate attribute, even using the graphql-java-extended-scalars library.
Researching I found that this library works well for queries but not so well for mutations.
I fixed my problem by customizing SchemaParserOptions, like this:
#Bean
public SchemaParserOptions schemaParserOptions() {
return SchemaParserOptions.newOptions().objectMapperConfigurer((mapper, context) -> {
mapper.registerModule(new JavaTimeModule());
}).build();
}
In the object i didn't use any serialization and deserialization annotations.

Issue with Spring-boot API bidirectional creating/updating

So I built the API with a crud on spring boot, the issue arises due to the bidirectional nature of the entities.
I can create it fine manually through the application (non-api) and it appears with children and all.
However, once the API is up, I try to post it (to create) a JSON such as this:
{
"idReserva": 1,
"comentarios": "",
"fechaIngreso": "0019-07-15",
"fechaSalida": "0019-10-30",
"cantidadDePersonas": 3,
"usuario": {
"idUsuario": 1,
"nombres": "test",
"apellidos": "test",
"contrasena": "1234",
"codUsuario": "USU01",
"email": "test#gmail.com",
"foto": ""
},
"pagos": [
{
"idPago": 1,
"tipo": "Efectivo",
"total": 1500
}
],
"habitaciones": [
{
"idHabitacion": 1,
"descripcion": "Habitación Ejecutiva",
"tipo": 3,
"numero": "5",
"codHabitacion": "HAB01",
"precio": "1500 dolares"
}
]
}
The issue comes that in my "create" method inside the repository, I can't receive the nested entities, it does create the "reserve" entry in the database, but it doesn't give it its children
List<Pago> listPagos = new ArrayList<>();
for (Pago pago : reserva.getPagos()){
log.info(pago.getIdPago()+"");
pagoService.create(pago);
listPagos.add(pago);
}
reserva.setPagos(listPagos);
I tried something such as that above to obtain each "pago"(payment) entity from the json and then create it/add it to reserve, since I need it to have the fields of its children payments in the database, but when I log the entities I receive "null" as if it's not receiving anything, is there any specific way I need to obtain the nested entities?

Alright so after a few hours of working around it, I found the issue. the API itself was missing something crucial, when you want to save inside the resource (api) layer, before you actually .save() using the service layer, you want to create an instance of the child entity, using a For: loop pass each entity inside the Json to an instance of that child, and JPA automatically will create them, and add them to the parent entity as well.
Example:
for (Habitacion habitacion : reserva.getHabitaciones()){
habitacion.setReserva(reserva);
}
for (Pago pago : reserva.getPagos()){
pago.setReserva(reserva);
}
Usuario usuario = reserva.getUsuario();
usuario.setReserva(reserva);
(this is inside the createReserva method from the resource layer)

Spring Data JPA - OneToMany child should be treated as a single resouce

Using Spring Data JPA 1.11.6.RELEASE, I have the following trouble;
With a simple 1-to-n relationship between them, I have TableA, and TableB, where A contains multiple B. And I have RepoA, that modifies not only TableA, but also TableB as a dependent child.
In this sense, when I have the following in DB (writing entities in JSON);
{
"uuid": "f10cdd75-ffbe-49e6-b7a5-ad6f8e15b2b5",
"name": "title",
"listOfB": [{
"pk": 1
}, {
"pk": 2
}]
}
and I'd like to update TableA, and consequentially TableB's with the following through RepoA;
{
"uuid": "f10cdd75-ffbe-49e6-b7a5-ad6f8e15b2b5",
"name": "title",
"listOfB": [{
"pk": 2
}, {
"pk": 3
}]
}
But I am getting the constraint violation due to Hibernate following its famous order of operations, it tries to insert all dependent TableB values, without removing the original ones.
Is there no way to overwrite the TableB entities in any way? I was able to find this solution;
select `TableA` with `TableB` values
clear `"listOfB"`
save & flush `TableA` // deletes all current `TableB`
add new `B`'s to `"listOfB"`
save again
But it is very laborious, and ugly, plus the more such tables I have, the more such code I have to write. Can't I have some definition in JPA to allow this behaviour automatically? Do not treat this table as a proper table, but only as a basic resource, that should be overwritten in all update requests?

Java Client for POSTing complex entities to a Spring Data REST / HATEOAS service

From what I can tell, there are provided means for converting a complex object to proper HAL format. This is of course leveraged in marshalling the objects in the framework itself. Resource and Link objects, etc.
For the sake of a use-case:
Company 1 is an existing Company in my system. I want to add a new Employee that works for Company 1
Below is an example Employee object that you'd receive from a Spring Data REST based service. Spring HATEOAS also provides the means to construct these objects yourself.
{
"id": null,
"firstName": "bZWthNFk",
"lastName": "GtTnrqka",
"loginId": "zTk5rT",
"active": true,
"_links": {
"company": {
"href": "http://localhost/companies/1";
}
}
}
However, this seems to not work for POSTing the object. As I understand it, that same object would have to be POSTed as:
{
"id": null,
"firstName": "bZWthNFk",
"lastName": "GtTnrqka",
"loginId": "zTk5rT",
"active": true,
"company": "http://localhost/companies/1"
}
As far as I can tell, there are no means provided by either the HATEOAS or Data REST project to produce this object for posting to a valid HAL based service, either via RestTemplate or some other means. In fact, I can't find any means of readily POSTing a complex object without some hand-marshalling. Am I wrong in assuming this?
How is one supposed to build out a valid Java SDK for service-to-service communication that leverages HATEOAS principles without this tooling to actually POST objects reliably?
Long story short, I want to post this object without having to hand serialize the URIs for associations.
public class Employee {
private Integer id;
#NotNull
private Company company;
private String firstName;
private String lastName;
}
I've created the following improvement request in reference to this:
https://jira.spring.io/browse/SPR-12678

The approach you suggested should actually work, provided you use at least version 2.0 of Spring Data REST.
You should also have an association resource like http://app.com/employee/10/company. You can PUT a new link to that location using the media type text/uri-list or remove the company from Employee with a DELETE.
UDATE
It seems I didn't address your main concern, that was clarified by your update and comments. So let's take your Employee class that has an association with a Customer.
As you can see from the JSON response you posted, the data structure that the REST API works with doesn't contain a Customer object (or Company in that case), only a link. A client would usually work with the data structure defined by the API. So customerwould be link in the first place and there would be no need for serializing an object to a link.
If the client uses a different data structure internally, then some kind of conversion is necessary anyway. But the reason would be the different structure, not HAL or association links.