What is the accepted practice to use multiple database 'models' in a Spring / Hibernate combo ?
Would appreciate some help on an issue that has me scratching my head, as I am no expert in Spring/Hibernate !
We are developing a web project that up until now has been built to the specification of its first customer. All the database tables are mapped to POJOs and we use Hibernate Annotations. Very simply, the project is a CRUD application that allows end users to update database information via a front end.
We now need to support a new customer, but this customer has a slightly different specification; we will have changes to a number of tables' columns and datatypes.
We don't want to combine into one table, as this bloats up the database with lots of NULL columns (there will be 10+ unique columns per customer in some table).
The front end we can handle easily enough, as we just convert to and from JSON and the front end has been implemented per-customer from the ground up. But the backend is a bit more complicated.
We have tried a prototype where we override two tables at runtime; we converted two corresponding model classes to interfaces and implemented a concrete class per customer, loaded via the Spring configuration, e.g for a "products" table, we tried:
package com.mycompany.generic.model;
public interface Product
and then
package com.mycompany.customera.model;
#Table(name="products")
public class CustomerAProduct implements Properties {
String colour;
int weight;
}
or
package com.mycompany.customerb.model;
#Table(name="products")
public class CustomerBProduct implements Properties {
String colour;
int volume;
Double price;
}
..where Spring loads, as a runtime option, com.mycompany.customera or customerb depending on configuration, along with any customer-specific validations/service classes to act on the new POJO's. You see that the underlying columns can be different.
However, I am hesitate to pitch into implementing this, because..
Almost every model class will need converting and implementing per-customer, not just the ones that are different. This is because we have many joins (OneToMany etc) within the model classes, Hibernate does not let you join to an interface (eg HHH-4338)
some sort of util/factory is required to generate new instances of classes, as where we used to do new Properties we do not know whether to use new CustomerAProperties() or new CustomerBProperties() (for the same reason, we cannot use TargetEntity to solve the above problem, because this requires a .class file which cannot be set in the code)
These seem pretty major downsides so I am trying to research a better way to do it. We can build per-customer, via Maven, but then I am not sure how to implement this in a way that isn't going to break Eclipse while we develop locally..
Or, is there some other method that I am not aware of ?
thanks in advance for any suggestions or pointers on what to research etc.
Or, is there some other method that I am not aware of ?
Any reason you cannot switch from annotation based mapping to xml mapping?
Each customer gets a set of XML mapping files. Now you only need to subclass when the model truly differs between customers.
Related
I'm (trying to :) using spring-boot-starter-data-rest in my spring boot app to quickly serve the model through true, fullblown, restFULL api. That works great.
Question 1 (Security):
The advantage of Spring JpaRepository is I don't need to code basic functions (save, findAll, etc). Is it possible to secure these auto-implemented methods without overriding all of them (wasting what Spring provided for me)? i.e.:
public interface BookRepository extends JpaRepository<Book, Long> {
#PreAuthorize("hasRole('ROLE_ADMIN')")
<S extends Book> Book save(Book book);
}
.
Question 2 (Security):
How to secure a JpaRepository to prevent updating items the loggeg-in user is not an owner?
i.e.: User is allowed to modify only his/her own properties.
i.e.2: User is allowed to modify/delete only the Posts he/she created.
Sample code is highly welcome here.
.
Question 3 (DTOs):
Some time ago I had an argue with a developer friend: He ensisted that there MUST be DTOs returned from Spring MVC controllers. Even if the DTO is 1-1 copy of the model object. Then I reserched, asked other guys and confirmed it: DTOs are required to divide/segregate the application layers.
How this relates to JpaRepositories? How to use DTOs with Spring auto serverd rest repos? Should I DTOs at all?
Thanks for your hints/answers in advance !
Question 1: Security
Some old docs mention:
[...] you expose a pre-defined set of operations to clients that are not under you control, it’s pretty much all or nothing until now. There’s seemingly no way to only expose read operations while hiding state changing operations entirely.
which implies that all methods are automatically inherited (also, as per standard java inheritance behavior).
As per the #PreAuhtorize docs, you can place the annotation also on a class / interface declaration.
So you could just have one basic interface extend JpaRepository
#NoRepositoryBean // tell Spring not create instances of this one
#PreAuthorize("hasRole('ROLE_ADMIN')") // all methods will inherit this behavior
interface BaseRepository<T, ID extends Serializable> extends Repository<T, ID> {}
and then have all of your Repository's extend BaseRepository.
Question 2: Security
I'm going to be a little more general on this one.
In order to correctly regulate access to entities within your application and define what-can-see-what, you should always separate your project into different layers.
A good starting point would be:
layer-web (or presentation-layer): access to layer-business, no access to the db-layer. Can see DTO models but not DB models
layer-business (or business-layer): access to the db-layer but no access to the DAO
layer-db (or data-layer): convert DTO -> DB model. Persist objects and provide query results
In your case, I believe that the right thing to do, would be therefore to check the role in the layer-business, before the request even reaches the Repository class.
#Service
public interface BookService {
#PreAuthorize("hasRole('ROLE_ADMIN')")
ActionResult saveToDatabase(final BookDTO book);
}
or, as seen before
#Service
#PreAuthorize("hasRole('ROLE_ADMIN')")
public interface BookService {
ActionResult saveToDatabase(final BookDTO book);
}
Also, ensuring a user can modify only its own objects can be done in many ways.
Spring provides all necessary resources to do that, as this answer points out.
Or, if you are familiar with AOP you can implement your own logic.
E.g (dummyCode):
#Service
public interface BookService {
// custom annotation here
#RequireUserOwnership(allowAdmin = false)
ActionResult saveToDatabase(final BookDTO book);
}
And the check:
public class EnsureUserOwnershipInterceptor implements MethodInterceptor {
#Autowired
private AuthenticationService authenticationService;
#Override
public Object invoke(Invocation invocation) throws Throwable {
// 1. get the BookDTO argument from the invocation
// 2. get the current user from the auth service
// 3. ensure the owner ID and the current user ID match
// ...
}
}
Useful resources about AOP can be found here and here.
Question 3: DTO's and DB models
Should I DTOs at all?
Yes, yes you should. Even if your projects has only a few models and your are just programming for fun (deploying only on localhost, learning, ...).
The sooner you get into the habit of separating your models, the better it is.
Also, conceptually, one is an object coming from an unknown source, the other represents a table in your database.
How this relates to JpaRepositories?
How to use DTOs with Spring auto serverd rest repos?
Now that's the point! You can't put DTO's into #Repositorys. You are forced to convert one to another. At the same point you are also forced to verify that the conversion is valid.
You are basically ensuring that DTOs (dirty data) will not touch the database in any way, and you are placing a wall made of logical constraints between the database and the rest of the application.
Also I am aware of Spring integrating well with model-conversion frameworks.
So, what are the advantages of a multi-layer / modular web-application?
Applications can grow very quickly. Especially when you have many developers working on it. Some developers tend to look for the quickest solution and implement dirty tricks or change access modifiers to finish the job asap. You should force people to gain access to certain resources only through some explicitly defined channels.
The more rules you set from the beginning, the longer the correct programming pattern will be followed. I have seen banking application become a complete mess after less then a year. When a hotfix was required, changing some code would create two-three other bugs.
You may reach a point where the application is consuming too many OS resources. If you, let's say, have a module module-batch containing background-jobs for your application, it will be way easier to extract it and implement it into another application. If your module contains logic that queries the database, access any type of data, provides API for the front-end, ecc... you will be basically forced to export all your code into your new application. Refactoring will be a pain in the neck at that point.
Imagine you want to hire some database experts to analyze the queries your application does. With a well-defined and separated logic you can give them access only to the necessary modules instead of the whole application. The same applies to front-end freelancers ecc... I have lived this situation as well. The company wanted database experts to fix the queries done by the application but did not want them to have access to the whole code. At the end, they renounced to the database optimization because that would have exposed too much sensitive information externally.
And what are the advantages of DTO / DB model separation?
DTO's will not touch the database. This gives you more security against attacks coming from the outside
You can decide what goes on the other side. Your DTO's do not need to implement all the fields as the db model. Actually you can even have a DAO map to many DTO's or the other way around. There is lots of information that shouldn't reach the front-end, and with the DTO's you can easily do that.
DTO are in general liter than #Entity models. Whereas entities are mapped (e.g #OneToMany) to other entities, DTO's may just contain the id field of the mapped objects.
You do not want to have database objects hanging around for too long; and neither being passed around by methods of your application. Many framework commit database transactions at the end of each method, which means any involuntary change done onto the database entity may be committed into the db.
Personally, I believe that any respectful web-application should strongly separate layers, each with its responsibility and limited visibility to other layers.
Differentiation between database models and data transfer objects is also a good pattern to follow.
At the end this is only my opinion though; many argue that the DTO pattern is outdated and causes unnecessary code repetition any many argue that to much separation leans to difficulty in maintaining the code. So, you should always consult different sources and then apply what works best for you.
Also interesting:
SE: What is the point of using DTO (Data Transfer Objects)?
Lessons Learned: Don't Expose EF Entities to the Client Directly
Guice Tutorial – method interception (old but gold)
SO: Large Enterprise Java Application - Modularization
Microsoft Docs: Layered Application Guidelines
The 5-layer architecture
I have one #Table domain and want to create multiple tables using that. How can I do it with JPARepository.
#Table
public class structure implements Serializable{
...
}
I want to create a lot of tables with different names and same structure in run time and save records to them.
How can I do this with JPARepository?
Creating JPA mapped classes dynamically is not possible. But there are possibles to minimize the code you have to write.
MappedSuperClass
Create one class annotated with MappedSuperclass and let all other classes inherit from it. This way the actual mapped classes contain very little code.
Inheritance
You mentioned in the comments that your classes are different types. This suggests that maybe inheritance with a table per class is the correct way to go.
Again create a superclass with all your proper classes to inherit from, but this time annotate them as with #Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
Partitioning
You write in the comments
Because I'm working on a large amount of data and if I insert all of them in one table my query performance is low.
Which makes me think you might be approaching this on the wrong level and you should really look into partitioning. The availability of this and how it exactly works depends on the database you use. Therefore I just leave a link to Wikipedia.
My application has about 50 entities that are displayed in grid format in the UI. All 50 entities have CRUD operations. Most of the operations have the standard flow
ie. for get, read entities from repository, convert to DTO and return a list of DTO's.
for create/update/delete - get DTO's - convert to entities, use repository to create/update/delete on DB, return updated DTOs
Mind you that for SOME entities, there are also some entity specific operations that have to be done.
Currently, we have a get/create/update/delete method for all our entities like
getProducts
createProducts
updateProducts
getCustomers
createCustomers
updateCustomers
in each of these methods, we use the Product/Customer repository to perform the CRUD operation AFTER conversion from entity -> dto and vice versa.
I feel there is a lot of code repetition and there must be a way by which we can remove so many of these methods.
Can i use some pattern (COMMAND PATTERN) to get away with code repetition?
Have a look at the Spring Data JPA or here project. It does away with boilerplate code for DAO.
I believe it basically uses AOP to interpret calls like
findByNameandpassword (String name,String passwd)
to do a query based upon the parameters passed in selecting the fields in the method name (only an interface).
Being a spring project it has very minimal requirements for spring libraries.
Basically, you have 2 ways to do this.
First way: Code generation
Write a class that can generate the code given a database schema.
Note that this you will create basic classes for each entity.
If you have custom code (code specific to certain entities) you can put that in subclasses so that it doesn't get overwritten when you regenerate the basic classes.
Object instatiation should be via Factory methods so that the correct subclass is used.
Make sure you add comments in the generated code that clearly states that the code is generated automatically (so that people don't start editing them directly).
Second way: Reflection
This solution, while being more elegant, is also more complex.
Instead of generating one basic class for each entity you have one basic class that can handle any entity. The class would be using reflection to access the DTO:s.
If you have custom code (code specific to certain entities) you can put that in other classes. These other classes would be injected into the generic class.
Using reflection would require a strict naming policy on your DTO:s.
Conclusion
I have been in a project using the first method in a migration project to generate DTO classes for the service interface between the new application server (running java) and the fat clients and it worked quite well. We had more than 100 generated DTO classes. I am aware that what you are attempting is slighty different. Editing database records is a generic problem (all projects need it) but there aren't (m)any frameworks for it.
I have been thinking about creating a generic tool or framework for it but I have never gotten around to it.
Environment:
Java
Spring
MVC pattern
Hibernate
Description:
Ok, let's say I have a web application with two domain objects:
User
Report
a User can have a lot of Reports (one to many relation). Please consider that a Report is a very complex object, with a lot of attributes.
User class:
public class User {
private Set<Report> reports = new HashSet<Report>();
}
Report class:
public class Report {
//...so maaaaaaany attributes
private String name;
}
Let's say I need to show an html page displaying a User profile with the list of associated Reports. Only reports' names appear in the list. Please comment these considerations:
I don't think to eagerly load the Reports because of memory saving issues, so I'll go lazy.
Even if I go lazy, the point is that I actually need the reports'name only. I don't want to load tons of information just to cherry-pick the report name!
So a possible solution is to modify the User class as follows:
public class User {
private Set<Report> reports = new HashSet<Report>();
private List<String> reportNames;
}
taking the needed information from the report to the user. In my opinion, it brings two consequences:
the list of report names must be kept updated
i break the separation between objects domain, filling the User with information I can't easily retrieve. This approach might even be effective, but it is very ugly.
So is there a nice solution to cope with this problem? I think it is common issue for developers.
One way would be to use the following pattern:
Create a view object which represents exactly what you want to be displayed, let's call it UserViewObject. I would not modify the domain objects just to adapt them for the view, that would break the MVC design.
Implements a service method in a service class which returns a list of UserViewObject.
Let the service method call a DAO method in a DAO class that actually does the job.
The DAO method could make sure to only read the required data, or you could do that transformation in the service layer. It is really a bit of what you prefer, and how it fits in. But do not make the DAO layer aware of your UserViewObject.
I'm developing (another) java web framework for personal use, and in addition to that I also want to develop somekind of persistence framework.
I have already developed and engine that in order to access the table, you must only extend a class and create your fields with the same type and name of those in the table. Ex:
private int id;
private String nome;
So you only need now to build the query and execute. The engine put the values on the right fields.
Recently I've had a quite good experience with Django, wich in order to update, create and filter the table you just need to call .save(), .create(field1=field, field2=213) and, .filter(filterfield=value).
I want to build this to, but I am having some trouble, because the extending class would actually have to write more, fact that I don't want to force. I have had the idea to the extending class write an enum implementing an Interface. So the super class get those fields, along with a private HashMap (the key is the enum) and keep the values, so the client just calls:
String nome = Usuarios.get(Usuarios.fields.name);
To retrieve the value.
Usuarios.update(new Pair(Usuarios.fields.name, "José Leal"), new Pair(Usuarios.fields.sobrenome, "Domingues"));
To update and so on.
Anyone have a better idea? I also use velocity framework, so I would like my field to be accessible by its getter.
ps: I know the existence of hibernate and etc.
I would suggest that you not force users to extend one of your own classes for their model objects. This locks them out of extending their own classes, and makes them more vulnerable to API changes you might make later.
Seeing as javax.persistence already defines annotations that can be used for mapping properties to a database, I'd suggest instead that you let the users make POJOs for their model objects with annotated properties. That way, along with the other mentioned benefits, you're following an already established standard and thereby lowering the barrier to entry for your users.