I have a general question about how best to build an API that can modify records in a database.
Suppose we have a table with 10 columns and we can query these 10 columns using REST (GET). The JSON response will contain all 10 fields. This is easy and works without problems.
The next step is that someone wants to create a new record via POST. In this case the person sends only 8 of the 10 fields in the JSON Request. We would then only fill the 8 fields in the database (the rest would be NULL). This also works without problems.
But what happens if someone wants to update a record? We see here different possibilities with advantages and disadvantages.
Only what should be updated is sent.
Problem: How can you explicitly empty / delete a field? If a "NULL" is passed in the JSON, we get NULL in the object, but any other field that is not passed is NULL as well. Therefore we cannot distinguish which field can be deleted and which field cannot be touched.
The complete object is sent.
Problem: Here the object could be fetched via a GET before, changed accordingly and returned via PUT. Now we get all information back and could write the information directly back into the database. Because empty fields were either already empty before or were cleared by the user.
What happens if the objects are extended by an update of the API. Suppose we extend the database by five more fields. The user of the API makes a GET, gets the 15 fields, but can only read the 10 fields he knows on his page (because he hasn't updated his side yet). Then he changes some of the 10 fields and sends them back via PUT. We would then update only the 10 fields on our site and the 5 new fields would be emptied from the database.
Or do you have to create a separate endpoint for each field? We have also thought about creating a map with key / value, what exactly should be changed.
About the technique: We use the Wildfly 15 with Resteasy and Jackson.
For example:
Database at the beginning
+----+----------+---------------+-----+--------+-------+
| ID | Name | Country | Age | Weight | Phone |
+----+----------+---------------+-----+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 56789 |
| 3 | Person 3 | Canada | 52 | 102 | 99999 |
+----+----------+---------------+-----+--------+-------+
GET .../person/2
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :62,
"phone": "56789"
}
Now I want to update his weight and remove the phone number
PUT .../person/2
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :78
}
or
{
"id" : 2,
"name" : "Person 2",
"country" : "United States",
"age" : 22,
"weight" :78,
"phone" : null
}
Now the database should look like this:
+----+----------+---------------+-----+--------+-------+
| ID | Name | Country | Age | Weight | Phone |
+----+----------+---------------+-----+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | 99999 |
+----+----------+---------------+-----+--------+-------+
The problem is
We extend the table like this (salery)
+----+----------+---------------+-----+--------+--------+-------+
| ID | Name | Country | Age | Weight | Salery | Phone |
+----+----------+---------------+-----+--------+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 1929 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 2831 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | 3921 | 99999 |
+----+----------+---------------+-----+--------+--------+-------+
The person using the API does not know that there is a new field in JSON for the salary. And this person now wants to change the phone number of someone again, but does not send the salary. This would also empty the salary:
{
"id" : 3,
"name" : "Person 3",
"country" : "Cananda",
"age" : 52,
"weight" :102,
"phone" : null
}
+----+----------+---------------+-----+--------+--------+-------+
| ID | Name | Country | Age | Weight | Salery | Phone |
+----+----------+---------------+-----+--------+--------+-------+
| 1 | Person 1 | Germany | 22 | 60 | 1929 | 12345 |
| 2 | Person 2 | United States | 32 | 78 | 2831 | NULL |
| 3 | Person 3 | Canada | 52 | 102 | NULL | NULL |
+----+----------+---------------+-----+--------+--------+-------+
And salary should not be null, because it was not set inside the JSON request
You could deserialize your JSON to a Map.
This way, if a property has not been sent, the property is not present in the Map. If its null, its inside the map will a null value.
ObjectMapper mapper = new ObjectMapper();
TypeReference<HashMap<String, Object>> typeReference = new TypeReference<>() {};
HashMap<String, Object> jsonMap = mapper.readValue(json, typeReference);
jsonMap.entrySet().stream().map(Map.Entry::getKey).forEach(System.out::println);
Not a very convenient solution, but it might work for you.
A common technique is to track changes on the entity POJO.
Load Dog with color = black, size = null and age = null
Set size to null (the setter will mark this field as changed)
Run update SQL
The POJO will have an internal state knowning that size was changed, and thus include that field in the UPDATE. age, on the other hand, was never set, and is thus left unchanged. jOOQ works like that, I'm sure there's others.
Only what should be updated is sent. Problem: How can you explicitly empty / delete a field? If a "NULL" is passed in the JSON, we get NULL in the object, but any other field that is not passed is NULL as well. Therefore we cannot distinguish which field can be deleted and which field cannot be touched.
The problem you have identified is genuine; I have faced this too. I think it is reasonable to not provide a technical solution for this, but rather document the API usage to let the caller know the impact of leaving out a field or sending it as null. Of course, assuming that the validations on the server side are tight and ensure sanity.
The complete object is sent. Problem: Here the object could be fetched via a GET before, changed accordingly and returned via PUT. Now we get all information back and could write the information directly back into the database. Because empty fields were either already empty before or were cleared by the user.
This is "straighter-forward" and should be documented in the API.
What happens if the objects are extended by an update of the API.
With the onus put on the caller through the documentation, this too is handled implicitly.
Or do you have to create a separate endpoint for each field?
This, again, is a design issue, the solution to which varies from person-to-person. I would rather retain the API at a record level than at the level of individual value. However, there may be cases where they are needed to be that way. Eg, status updates.
Suppose we extend the database by five more fields. The user of the API makes a GET, gets the 15 fields, but can only read the 10 fields he knows on his page (because he hasn't updated his side yet). Then he changes some of the 10 fields and sends them back via PUT. We would then update only the 10 fields on our site and the 5 new fields would be emptied from the database.
So let's start with an example - what would happen on the web, where clients are interacting with your API via HTML rendered in browsers. The client would GET a form, and that form would have input controls for each of the fields. Client updates the fields in the form, submits it, and you apply those changes to your database.
When you want to extend the API to include more fields, you add those fields to the form. The client doesn't know about those fields. So what happens?
One way to manage this is that you make sure that you include in the form the correct default values for the new fields; then, if the client ignores the new fields, the correct value will be returned when the form is submitted.
More generally, the representations we exchange in our HTTP payloads are messages; if we want to support old clients, then we need the discipline of evolving the message schema in a backwards compatible way, and our clients have to be written with the understanding that the message schema may be extended with additional fields.
The person using the API does not know that there is a new field in JSON for the salary.
The same idea holds here - the new representation includes a field "salary" that the client doesn't know about, so it is the responsibility of the client to forward that data back to you unchanged, rather than just dropping it on the floor assuming it is unimportant.
There's a bunch of prior art on this from 15-20 years ago, because people writing messages in XML were facing exactly the same sort of problems. They have left some of their knowledge behind. The easiest way to find it is to search for some key phases; for instance must ignore or must forward.
See:
Versioning XML Vocabularies
Extensibility, XML Vocabularies, and XML Schema
Events in an event store have the same kinds of problems. Greg Young's book Versioning in an Event Sourced System covers a lot of the same ground (representations of events are also messages).
The accepted answer works well but it has a huge caveat which is that it's completely untyped. If the object's fields change then you'll have no compile time warning that you're looking for the wrong fields.
Therefore I would argue that it's better to force all fields to be present in the request body. Therefore a null means the user explicitly set it to null while if the user misses a field they'll receive a 400 Bad Request with the request body describing the error in detail.
Here's a great post on how to achieve this: Configure Jackson to throw an exception when a field is missing
Here's my example in Kotlin:
data class PlacementRequestDto(
val contentId: Int,
#param:JsonProperty(required = true)
val tlxPlacementId: Int?,
val keywords: List<Int>,
val placementAdFormats: List<Int>
)
Notice that the nullable field is marked as required. This way the user has to explicitly include it in the request body.
You can control empty or null values as below
public class Person{
#JsonInclude(JsonInclude.Include.NON_NULL)
private BigDecimal salary; // this will make sure salary can't be null or empty//
private String phone; //allow phone Number to be empty
// same logic for other fields
}
i) As you're updating weight and removing the phone number,Ask client to send fields which needs to updated along with record identifier i.e id in this case
{
"id" : 2,
"weight" :78,
"phone" : null
}
ii) As you're adding salary as one more column which is mandatory field & client should be aware of it..may be you have to redesign contract
Imagine there are two entities Children and Gifts. Let's say there are 1000 children and 10 gifts. Children will always try to grab available gifts, and gifts will be tagged to children on a "first come, first serve" basis.
Table structure
children table
+----+------+
| id | name |
+----+------+
| 1 | Sam |
| 2 | John |
| 3 | Sara |
+----+------+
gift table
+----+---------+-------------+
| id | gift | children_id |
+----+---------+-------------+
| 1 | Toy Car | 2 |
| 2 | Doll | 3 |
+----+---------+-------------+
Here the children_id is the child who grabbed the gift first.
In my application, I want to update the children_id in such a way that only the first child who initiated the request will get it and rest get a GiftUnavailableException.
How will I ensure that, even if a 1000 requests come at a time to grab a specific gift, only the first one will get it. Or how will I ensure that there are no race conditions for this update.
Are there any spring specific feature that I can make use of or are there any other ways.
I'm using spring-boot as my backend.
I can't post a comment so here I go !
I assume you are using Spring Data JPA.
Then you should use #Transactional annotation : This mean that everytime you are requesting your database, you do a transaction :
Begin Transaction
Execute Transaction
Commit Transaction
Lot of usefull informations about this (Read it !!): Spring #Transactional - isolation, propagation
You will need to seed your Transactional Isolation to Serializable and maybe change the propagation method.
And if you are not using Spring data JPA.. Well There is a synchronized keywords but I think it's a bit awful to use it here.
2suppose i have a Scenario Outline like
#Scenario1
Scenario Outline:Scenario one
Given fill up login fields "<email>" and "<password>"
And click the login button
Examples:
| email | password |
| someEmailAddress | SomePassword |
| someEmailAddress2| SomePassword2 |
and another Scenario like
#Scenario2
Scenario Outline:Scenario two
Given fill up fields "<value1>" and "<value2>"
Examples:
| value1 | value2 |
| value11 | value21 |
| value12 | value22 |
How could i run scenario like login with 'someEmailAddress' and fill up with all scenario2 value and then login with 'someEmailAddress2' and do the same.
Cucumber scenarios are tools we use to describe behaviour i.e. what is happening and why its important. They are not tools to program tests. The way to use Cucumber effectively is to keep your scenarios simple, and let code called by step definitions do your programming for you.
Step definitions and the methods they call are written in a programming language. This gives you all the power you need to deal with the details of how you interact with your system.
The art of writing Cucumber scenarios is for each one to talk about
The state we need setup so we can do something (Givens)
Our interaction (When)
What we expect to see after our interaction. (Then)
So for your scenario we have
Scenario: Login
Given I am registered
When I login
Then I should be logged in
When we make this scenario work our program has the behaviour that we can login. So then we can use that behaviour in other scenarios e.g.
Scenario: See my profile
Given I am logged in
When I view my profile
Then I should see my profile
Now to make this work we might need a bit more work because this scenario doesn't have a registered user yet. We can deal with this in a number of ways
1) Add another Given, perhaps in a background
Background:
Given I am registered
Scenario ...
Given I am logged in
2) We can register in the login step e.g.
Given "I am logged in" do
#i = register_user
login_as user: #i
end
Notice how in this step we are calling helper methods register_user and login_as to do the work for us.
This is the way to start using Cucumber. Notice how my scenarios have no mention of how we login, no email, no password, no filling in anything. To use Cucumber effectively you have to push these details down into the step definitions and the helper methods they call.
Summary
Keep you scenarios simple and use them to describe WHAT and explain WHY. Use the step definitions and helper methods to deal with HOW. There is no need to use Scenario Outlines when using Cucumber and you should never be nesting them.
There is no support for nested scenario outline in cucumber. but you can use following way to overcome it.
Scenario Outline:Scenario one and two
Given fill up login fields "<email>" and "<password>"
And click the login button
And fill up fields "<value1>" and "<value2>"
Examples:
| email | password | value1 | value2 |
| someEmailAddress | SomePassword | value11 | value21 |
| someEmailAddress | SomePassword | value12 | value22 |
| someEmailAddress2| SomePassword2 | value11 | value21 |
| someEmailAddress2| SomePassword2 | value12 | value22 |
I have a database with a table called Car. The car table looks like this:
+----+------+--------------+-----------+----------+------+
| Id | Name | Desccription | Make | Model | Year |
+----+------+--------------+-----------+----------+------+
| 1 | A | something1 | Ford | Explorer | 2010 |
| 2 | B | something2 | Nissan | Ultima | 2005 |
| 3 | C | something3 | Chevrolet | Malibu | 2012 |
+----+------+--------------+-----------+----------+------+
Different pages on my website want to display different information. Some pages only want to display the name, others wants to display the make and model, etc.
I have an api that the web calls to retrieve all this information. The api uses JPA and QueryDSL to communicate with the database and fetch information. I want to only fetch the information that I want for that particular page. I'm thinking about implementing some sort of builder patter to my repo to allow for me to only retrieve what I want but I'm not quite sure how to go about it.
For example, my home page only wants to display the Name of the car. So it'll call the HomeController and the controller will call the HomeService which will call the repository layer something like this:
carRepository.getCarById(1).withName().build();
Some other page that wants to display the make and model would make a repo call like this:
carRepository.getCarById(1).withMake().withModel.build();
What is the best way to implement something like this in Java/Jpa?
If I understand the question correctly, you want queries for different projections of your entities to be built dynamically.
In that case, dynamic entity graphs are what you want (see e.g. here: https://www.thoughts-on-java.org/jpa-21-entity-graph-part-2-define/). You start with an empty entity graph, and each call to one of your with() method simply adds a field to the graph.
The base query remains unchanged, you just need to set the fetch graph hint (javax.persistence.fetchgraph) upon calling build() (note that the samples in the above link use load graphs instead of fetch graphs; the subtle difference between the two is described here: What is the diffenece between FETCH and LOAD for Entity graph of JPA?)
We're building a Java web application. We're using EJB containers and JPA with Wildfly 9.2.
Now we want to integrate a permission system in which a user has a certain role, but this role is only granted in combination with a certain entity. I'll name this access resource Department.
So we would have a list of permission for the user stored in such a table:
| User ID | Department | Role |
| ------- | ---------- | ------- |
| 1 | A | MANAGER |
| 1 | B | ADMIN |
| 2 | B | MANAGER |
In addition we have global roles. A user has a set of global roles that will be applied should there not be an entry in the table above with the deparment in question.
| User ID | Role |
| ------- | ------- |
| 1 | VIEWER |
| 3 | MANAGER |
How can we easily check if a user is in a certain role, given a department?
By just using the annotation #RolesAllowed we can check for a certain role, but not restricted to a department.
Remember that enterprise java security roles are about authorization.
So, you could model the department as a role in your security system as well. It's quite OK for a user to be assigned more than one security role.
Therefore, given a manager in department B:
#RolesAllowed({"manager", "B"}
public void someBusinessMethod(...) {
...
}
Only users who are managers and in department B are authorized to invoke this method.
Turns out EJB does not have the features we wanted.
We migrated to Apache Shiro Security and are super happy with it.
Resource based permissions always have to be checked programmatically. Static role and permission based checks can be annotations.