I have a general question about how best to build an API that can modify records in a database.
Suppose we have a table with 10 columns and we can query these 10 columns using REST (GET). The JSON response will contain all 10 fields. This is easy and works without problems.
The next step is that someone wants to create a new record via POST. In this case the person sends only 8 of the 10 fields in the JSON Request. We would then only fill the 8 fields in the database (the rest would be NULL). This also works without problems.
But what happens if someone wants to update a record? We see here different possibilities with advantages and disadvantages.
Only what should be updated is sent.
Problem: How can you explicitly empty / delete a field? If a "NULL" is passed in the JSON, we get NULL in the object, but any other field that is not passed is NULL as well. Therefore we cannot distinguish which field can be deleted and which field cannot be touched.
The complete object is sent.
Problem: Here the object could be fetched via a GET before, changed accordingly and returned via PUT. Now we get all information back and could write the information directly back into the database. Because empty fields were either already empty before or were cleared by the user.
What happens if the objects are extended by an update of the API. Suppose we extend the database by five more fields. The user of the API makes a GET, gets the 15 fields, but can only read the 10 fields he knows on his page (because he hasn't updated his side yet). Then he changes some of the 10 fields and sends them back via PUT. We would then update only the 10 fields on our site and the 5 new fields would be emptied from the database.
Or do you have to create a separate endpoint for each field? We have also thought about creating a map with key / value, what exactly should be changed.
About the technique: We use the Wildfly 15 with Resteasy and Jackson.
For example:
Database at the beginning
+----+----------+---------------+-----+--------+-------+
| ID | Name | Country | Age | Weight | Phone |
+----+----------+---------------+-----+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 56789 |
| 3 | Person 3 | Canada | 52 | 102 | 99999 |
+----+----------+---------------+-----+--------+-------+
GET .../person/2
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :62,
"phone": "56789"
}
Now I want to update his weight and remove the phone number
PUT .../person/2
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :78
}
or
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :78,
"phone" : null
}
Now the database should look like this:
+----+----------+---------------+-----+--------+-------+
| ID | Name | Country | Age | Weight | Phone |
+----+----------+---------------+-----+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | 99999 |
+----+----------+---------------+-----+--------+-------+
The problem is
We extend the table like this (salery)
+----+----------+---------------+-----+--------+--------+-------+
| ID | Name | Country | Age | Weight | Salery | Phone |
+----+----------+---------------+-----+--------+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 1929 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 2831 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | 3921 | 99999 |
+----+----------+---------------+-----+--------+--------+-------+
The person using the API does not know that there is a new field in JSON for the salary. And this person now wants to change the phone number of someone again, but does not send the salary. This would also empty the salary:
{
"id" : 3,
"name" : "Person 3",
"country" : "Cananda",
"age" : 52,
"weight" :102,
"phone" : null
}
+----+----------+---------------+-----+--------+--------+-------+
| ID | Name | Country | Age | Weight | Salery | Phone |
+----+----------+---------------+-----+--------+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 1929 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 2831 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | NULL | NULL |
+----+----------+---------------+-----+--------+--------+-------+
And salary should not be null, because it was not set inside the JSON request
You could deserialize your JSON to a Map.
This way, if a property has not been sent, the property is not present in the Map. If its null, its inside the map will a null value.
ObjectMapper mapper = new ObjectMapper();
TypeReference<HashMap<String, Object>> typeReference = new TypeReference<>() {};
HashMap<String, Object> jsonMap = mapper.readValue(json, typeReference);
jsonMap.entrySet().stream().map(Map.Entry::getKey).forEach(System.out::println);
Not a very convenient solution, but it might work for you.
A common technique is to track changes on the entity POJO.
Load Dog with color = black, size = null and age = null
Set size to null (the setter will mark this field as changed)
Run update SQL
The POJO will have an internal state knowning that size was changed, and thus include that field in the UPDATE. age, on the other hand, was never set, and is thus left unchanged. jOOQ works like that, I'm sure there's others.
Only what should be updated is sent. Problem: How can you explicitly empty / delete a field? If a "NULL" is passed in the JSON, we get NULL in the object, but any other field that is not passed is NULL as well. Therefore we cannot distinguish which field can be deleted and which field cannot be touched.
The problem you have identified is genuine; I have faced this too. I think it is reasonable to not provide a technical solution for this, but rather document the API usage to let the caller know the impact of leaving out a field or sending it as null. Of course, assuming that the validations on the server side are tight and ensure sanity.
The complete object is sent. Problem: Here the object could be fetched via a GET before, changed accordingly and returned via PUT. Now we get all information back and could write the information directly back into the database. Because empty fields were either already empty before or were cleared by the user.
This is "straighter-forward" and should be documented in the API.
What happens if the objects are extended by an update of the API.
With the onus put on the caller through the documentation, this too is handled implicitly.
Or do you have to create a separate endpoint for each field?
This, again, is a design issue, the solution to which varies from person-to-person. I would rather retain the API at a record level than at the level of individual value. However, there may be cases where they are needed to be that way. Eg, status updates.
Suppose we extend the database by five more fields. The user of the API makes a GET, gets the 15 fields, but can only read the 10 fields he knows on his page (because he hasn't updated his side yet). Then he changes some of the 10 fields and sends them back via PUT. We would then update only the 10 fields on our site and the 5 new fields would be emptied from the database.
So let's start with an example - what would happen on the web, where clients are interacting with your API via HTML rendered in browsers. The client would GET a form, and that form would have input controls for each of the fields. Client updates the fields in the form, submits it, and you apply those changes to your database.
When you want to extend the API to include more fields, you add those fields to the form. The client doesn't know about those fields. So what happens?
One way to manage this is that you make sure that you include in the form the correct default values for the new fields; then, if the client ignores the new fields, the correct value will be returned when the form is submitted.
More generally, the representations we exchange in our HTTP payloads are messages; if we want to support old clients, then we need the discipline of evolving the message schema in a backwards compatible way, and our clients have to be written with the understanding that the message schema may be extended with additional fields.
The person using the API does not know that there is a new field in JSON for the salary.
The same idea holds here - the new representation includes a field "salary" that the client doesn't know about, so it is the responsibility of the client to forward that data back to you unchanged, rather than just dropping it on the floor assuming it is unimportant.
There's a bunch of prior art on this from 15-20 years ago, because people writing messages in XML were facing exactly the same sort of problems. They have left some of their knowledge behind. The easiest way to find it is to search for some key phases; for instance must ignore or must forward.
See:
Versioning XML Vocabularies
Extensibility, XML Vocabularies, and XML Schema
Events in an event store have the same kinds of problems. Greg Young's book Versioning in an Event Sourced System covers a lot of the same ground (representations of events are also messages).
The accepted answer works well but it has a huge caveat which is that it's completely untyped. If the object's fields change then you'll have no compile time warning that you're looking for the wrong fields.
Therefore I would argue that it's better to force all fields to be present in the request body. Therefore a null means the user explicitly set it to null while if the user misses a field they'll receive a 400 Bad Request with the request body describing the error in detail.
Here's a great post on how to achieve this: Configure Jackson to throw an exception when a field is missing
Here's my example in Kotlin:
data class PlacementRequestDto(
val contentId: Int,
#param:JsonProperty(required = true)
val tlxPlacementId: Int?,
val keywords: List<Int>,
val placementAdFormats: List<Int>
)
Notice that the nullable field is marked as required. This way the user has to explicitly include it in the request body.
You can control empty or null values as below
public class Person{
#JsonInclude(JsonInclude.Include.NON_NULL)
private BigDecimal salary; // this will make sure salary can't be null or empty//
private String phone; //allow phone Number to be empty
// same logic for other fields
}
i) As you're updating weight and removing the phone number,Ask client to send fields which needs to updated along with record identifier i.e id in this case
{
"id" : 2,
"weight" :78,
"phone" : null
}
ii) As you're adding salary as one more column which is mandatory field & client should be aware of it..may be you have to redesign contract
Hello I'm desperate trying to find a way in ESPER - CEP to output the events that have the max value. Here's a good example to illustrate my problem:
| value | category | date |
| 12.2 | A | yyyy-MM-dd HH:mm:ss |
| 13.3 | A | yyyy-MM-dd HH:mm:ss |
I want the following output:
| value | category | date |
| 13.3 | A | yyyy-MM-dd HH:mm:ss |
Very basic in SQL : select max(value), category date from tab group by category
Now in Esper, i have tried many things: output every, output last, contexts.. But couldn't find a solution :/ It either outputs nothing or outputs all the lines. With "output first every", it only ouputs the first line, regardless of the max() comparison.
Is there someone who has an idea of how to proceed to obtain the max(value) and group by a parameter from a stream?
Thanks for your help :)
Doc link for controlling time keeping: http://esper.codehaus.org/esper-5.1.0/doc/reference/en-US/html_single/index.html#api-controlling-time
I have an SQLite table like:
+---+-------------+----------------+
|_id| lap_time_ms |formatted_elapse|
+---+-------------+----------------+
| 1 | 5600 | 00:05.6 |
| 2 | 4612 | 00:04.6 |
| 3 | 4123 | 00:04.1 |
| 4 | 15033 | 00:15.0 |
| 5 | 4523 | 00:04.5 |
| 6 | 6246 | 00:06.2 |
Where lap_time_ms is an of the type long and represents the amount of time in milliseconds for a lap while formatter_elapse is a String that represents the formatted (displayable) form of the first column (elapse).
My question is that if I want to add (say) 5 seconds (5000) to each lap_time_ms then I use a statement like:
DB.execSQL("update table_name set KEY_ELAPSE=KEY_ELAPSE+5000);
Which works fine however the problem is that formatted_elapse still retains its outdated value!
So, what is the best way to update the values in the formatted_elapse column if I have a function like:
public static String getFormattedTime(long milliseconds) {
// custom code that processes the long
return processedString;
}
It may be a long shot (metaphorically speaking of course ;) but is it possible to have SQLite link the two columns such that if I update a lap_time_ms row, the formatted_elapse will automatically update appropriately.
Thanks in advance!
In theory, it would be possible to create a trigger to update that column, but only if the formatting can be done with some built-in SQLite function (Android does not allow user-defined functions):
CREATE TRIGGER update_formatted_elapse
AFTER UPDATE OF lap_time_ms ON MyTable
FOR EACH ROW
BEGIN
UPDATE MyTable
SET formatted_elapse = strftime('%M:%f', NEW.lap_time_ms, 'unixepoch')
WHERE _id = NEW._id;
END;
However, it would be bad design to store the formatted string in the database; it would be duplicated information that is in danger of becoming inconsistent.
Drop the formatted_elapse column and just call getFormattedTime in your code whenever you need it.
I have a TIME data type column in my database. According to this, it supports hh[:mm] {AM | PM} format. But when I tried insert this sql :
INSERT INTO POSISI (ID, TUJUAN, PELABUHAN_TERAKHIR, LATLNG, AREA, KECEPATAN, HALUAN, STATUS, KETERANGAN, WAKTU, JAM) VALUES ((SELECT ID FROM KAPAL WHERE UPPER(KAPAL.NAMA)=UPPER('Aura Kasih')), 'Surabaya', 'Karimun', '2.02, 100.2', 'Kiclik', '19 knot', '11°', 'Anchor', 'Aman', '2014-03-16', '01:50 AM')
I got this on my database :
2 | Surabaya | Karimun | 2.02, 100.2 | Kiclik | 19 knot | 11° | Anchor | Aman | 2014-03-16 | 01:50:00
As you can see, I am using 01:50 AM but it stored as 01:50:00.
So, can I tell JavaDB to store TIME as hh:mm a format?
Thank you.
Time will always be stored as hh:mm:ss in the db. The format that you mentioned is for the input. You will need to format the time again when you want to retrieve / display from the db.
I am trying to map a normalized Java model to a legacy database schema using Hibernate 3.5. One particular table encodes a foreign keys in a one-to-many relationship as a bit array column.
Consider tables person and club that describes people's affiliations to clubs:
person: .----.------. club: .----.---------.---------------------------.
| id | name | | id | name | members | binary(members) |
|----+------| |----+---------|---------+-----------------|
| 1 | Bob | | 10 | Cricket | 0 | 000 |
| 2 | Joe | | 11 | Tennis | 5 | 101 |
| 3 | Sue | | 12 | Cooking | 7 | 111 |
'----'------' | 13 | Golf | 3 | 100 |
'----'---------'---------'-----------------'
So hopefully it is clear that person.id is used as the bit index in the bit array club.members:
.---.---.---.
| S | J | B |
| u | o | o |
| e | e | b |
|---+---+---|
| 1 | 0 | 1 |
'---'---'---'
In this example the members column tells us that:
no one is a member of Cricket --- no flags set
Bob/Sue -> Tennis --- flags at positions 1 and 3 are set
Bob/Sue/Joe -> Cooking --- flags at positions 1, 2 and 3 are set
Sue -> Golf --- flag at position 3 is set
Now, for this example a join table could've been used instead which would simplify matters and avoid many potential issues - e.g: the maximum range of members placing an upper bound on the number of people rows. However, I am stuck with this schema and it seems that there were factors in favour of using a bit array column way back when.
In my Java domain I'd like to model this schema with entities like so:
class Person {
private int id;
private String name;
...
}
class Club {
private Set<Person> members;
private int id;
private String name;
...
}
I am assuming that I must use a UserType implementation but have been unable to find any examples where the items described by the user type are references to entities - not literal field values - or composites thereof. Additionally I am aware that I'll have to consider how the person entities are fetched when a club instance is loaded.
Can anyone tell me how I can tame this legacy schema with Hibernate?
Update
I have recently had to revisit this type of mapping in a legacy database. This time around it became apparent that our equivalent of the members table was actually a static set and could be hardcoded as an Enum. With this simplification it was fairly straightforward to implement a Hibernate UserType that converted between the bit array and a set of enums.
I've never faced this situation but I think that you'll need to implement a custom UserCollectionType (see chapter 5.2.3. Custom value types in Hibernate Core documentation), the UserCollectionType being an extension point which may be used to support any damn collection and collection semantics you like.
I'm not sure how well they are supported by annotations though (according to HHH-4417, you may have to use a hack). Using hbm.xml for this would be a good idea here IMO.
Some more pointers/discussions:
https://forum.hibernate.org/viewtopic.php?f=9&t=938522&start=0
http://www.javalobby.org/java/forums/m91832311.html (different case but may give you an idea of how to start)