In my EclipseLink JPA system I want to access a custom query using a OneToMany. Unfortunately the database is not normalized.
Simplified my DB looks like this:
Company
ID Name
-----------
01 CompanyA
02 CompanyB
Person
ID Company_Id Name Occupation
------------------------------
01 01 Alice Management
02 01 Bob Accounting
03 01 Carl Accounting
The occupation is given as a natural key.
Very simplified I have the following code:
#Entity
class Company {
#Id
#Column private Integer id;
#Column private String name;
#OneToMany()
#JoinColumn(name="Company_Id")
private List<Person> persons;
}
#Entity
class Person {
#Id
#Column private Integer id;
#Column private Integer company_id;
#Column private String name;
#Column private String occupation;
}
So far it is running fine. Now I want to add a OneToMany relationship that that lists the occupations and gives me the possibility to get a list of persons that have that occupation.
I want to do:
for (Company c : myCollectionOfCompanyEntities) {
System.out.println(c.name);
for (OccupationsInCompany o : c.getOccupations()) {
System.out.println(o.name);
for (Person p : o.getPersons()) {
System.out.println(p.name);
}
}
}
How do I start to write that OccupationsInCompany class? If I simply create an entity class, JPA wants to read the data from a table.
I know how to get all the data by hand using custom queries. But is it possible to do this using OneToMany annotations?
You should be able to do this by using the #ElementCollection annotation.
#ElementCollection
#CollectionTable(name = "PEOPLE", joinColumns = {#JoinColumn(name="Company_Id")})
private List<Person> people;
Useful link at WikiBooks.
There are many ways to do this, but unless listing people by the occupation in a company is large part of the application, I don't know that it makes sense to add objects to your model that might affect any other use cases negatively - you will be forced to maintain this model when you may only want it sorted at one point in time.
Other options:
1) take your list of people and sort them by occupation in memory within the getOccupations method. This can then be stored in a transient occupation list so it doesn't need to be repeatedly done.
2)Query for it in one query. "Select c.name, p.occupation, p.name from company c join c.persons p order by c.name, p.occupation where..". This one query gives you everything without the need to traverse your object model. Depending on how often this is needed, you can add options to make either more efficient for your use cases.
Option 1 can be combined with JPA so that the organization list is populated upfront, such in a postLoad method. Any model changes will result in extra complexity and result in JPA doing the same work the model itself can handle just as well if not better.
I understand that the N+1 problem is where one query is executed to fetch N records and N queries to fetch some relational records.
But how can it be avoided in Hibernate?
The problem
The N+1 query issue happens when you forget to fetch an association and then you need to access it.
For instance, let's assume we have the following JPA query:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
where pc.review = :review
""", PostComment.class)
.setParameter("review", review)
.getResultList();
Now, if we iterate the PostComment entities and traverse the post association:
for(PostComment comment : comments) {
LOGGER.info("The post title is '{}'", comment.getPost().getTitle());
}
Hibernate will generate the following SQL statements:
SELECT pc.id AS id1_1_, pc.post_id AS post_id3_1_, pc.review AS review2_1_
FROM post_comment pc
WHERE pc.review = 'Excellent!'
INFO - Loaded 3 comments
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 1
INFO - The post title is 'Post nr. 1'
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 2
INFO - The post title is 'Post nr. 2'
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 3
INFO - The post title is 'Post nr. 3'
That's how the N+1 query issue is generated.
Because the post association is not initialized when fetching the PostComment entities, Hibernate must fetch the Post entity with a secondary query, and for N PostComment entities, N more queries are going to be executed (hence the N+1 query problem).
The fix
The first thing you need to do to tackle this issue is to add [proper SQL logging and monitoring][1]. Without logging, you won't notice the N+1 query issue while developing a certain feature.
Second, to fix it, you can just JOIN FETCH the relationship causing this issue:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
where pc.review = :review
""", PostComment.class)
.setParameter("review", review)
.getResultList();
If you need to fetch multiple child associations, it's better to fetch one collection in the initial query and the second one with a secondary SQL query.
How to automatically detect the N+1 query issue
This issue is better to be caught by integration tests.
You can use an automatic JUnit assert to validate the expected count of generated SQL statements. The db-util project already provides this functionality, and it's open-source and the dependency is available on Maven Central.
Suppose we have a class Manufacturer with a many-to-one relationship with Contact.
We solve this problem by making sure that the initial query fetches all the data needed to load the objects we need in their appropriately initialized state. One way of doing this is using an HQL fetch join. We use the HQL
"from Manufacturer manufacturer join fetch manufacturer.contact contact"
with the fetch statement. This results in an inner join:
select MANUFACTURER.id from manufacturer and contact ... from
MANUFACTURER inner join CONTACT on MANUFACTURER.CONTACT_ID=CONTACT.id
Using a Criteria query we can get the same result from
Criteria criteria = session.createCriteria(Manufacturer.class);
criteria.setFetchMode("contact", FetchMode.EAGER);
which creates the SQL :
select MANUFACTURER.id from MANUFACTURER left outer join CONTACT on
MANUFACTURER.CONTACT_ID=CONTACT.id where 1=1
in both cases, our query returns a list of Manufacturer objects with the contact initialized. Only one query needs to be run to return all the contact and manufacturer information required
for further information here is a link to the problem and the solution.
Native solution for 1 + N in Hibernate, is called:
20.1.5. Using batch fetching
Using batch fetching, Hibernate can load several uninitialized proxies if one proxy is accessed. Batch fetching is an optimization of the lazy select fetching strategy. There are two ways we can configure batch fetching: on the 1) class level and the 2) collection level...
Check these Q & A:
#BatchSize but many round trip in #ManyToOne case
Avoiding n+1 eager fetching of child collection element association
With annotations we can do it like this:
A class level:
#Entity
#BatchSize(size=25)
#Table(...
public class MyEntity implements java.io.Serializable {...
A collection level:
#OneToMany(fetch = FetchType.LAZY...)
#BatchSize(size=25)
public Set<MyEntity> getMyColl()
Lazy loading and batch fetching together represent optimization, which:
does not require any explicit fetching in our queries
will be applied on any amount of references which are (lazily) touched after the root entity is loaded (while explicit fetching effects only these named in query)
will solve issue 1 + N with collections (because only one collection could be fetched with root query) without need to farther processing To get DISTINCT root values (check: Criteria.DISTINCT_ROOT_ENTITY vs Projections.distinct)
You can even get it working without having to add the #BatchSize annotation everywhere, just set the property hibernate.default_batch_fetch_size to the desired value to enable batch fetching globally. See the Hibernate docs for details.
While you are at it, you will probably also want to change the BatchFetchStyle, because the default (LEGACY) is most likely not what you want. So a complete configuration for globally enabling batch fetching would look like this:
hibernate.batch_fetch_style=PADDED
hibernate.default_batch_fetch_size=25
Also, I'm suprised that one of the proposed solutions involves join-fetching. Join-fetching is rarely desirable because it causes more data to be transferred with every result row, even if the dependent entity has already been loaded into the L1 or L2 cache. Thus I would recommend to disable it completey by setting
hibernate.max_fetch_depth=0
This is a frequently asked question so I created the article Eliminate Spring Hibernate N+1 Queries to detail the solutions
To help you detect all the N+1 queries in your application and avoid adding more queries, I created the library spring-hibernate-query-utils that auto-detects the Hibernate N+1 queries.
Here is some code to explain how to add it to your application:
Add the library to your dependencies
<dependency>
<groupId>com.yannbriancon</groupId>
<artifactId>spring-hibernate-query-utils</artifactId>
<version>1.0.0</version>
</dependency>
Configure it in your application properties to return exceptions, default is error logs
hibernate.query.interceptor.error-level=EXCEPTION
If you are using Spring Data JPA to implement your repositories, you can specify lazy fetching in the JPA associations:
#Entity
#Table(name = "film", schema = "public")
public class Film implements Serializable {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "language_id", nullable = false)
private Language language;
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "film")
private Set<FilmActor> filmActors;
...
}
#Entity
#Table(name = "film_actor", schema = "public")
public class FilmActor implements Serializable {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "film_id", nullable = false, insertable = false, updatable = false)
private Film film;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "actor_id", nullable = false, insertable = false, updatable = false)
private Actor actor;
...
}
#Entity
#Table(name = "actor", schema = "public")
public class Actor implements Serializable {
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "actor")
private Set<FilmActor> filmActors;
...
}
And add #EntityGraph to your Spring Data JPA-based repository:
#Repository
public interface FilmDao extends JpaRepository<Film, Integer> {
#EntityGraph(
type = EntityGraphType.FETCH,
attributePaths = {
"language",
"filmActors",
"filmActors.actor"
}
)
Page<Film> findAll(Pageable pageable);
...
}
My blog post at https://tech.asimio.net/2020/11/06/Preventing-N-plus-1-select-problem-using-Spring-Data-JPA-EntityGraph.html helps you preventing the N+1 select problem using Spring Data JPA and #EntityGraph.
Here are some snippet codes that would help you to fix the N+1 Problem.
One to Many Relationship with Manager and Client Entity.
Client JPA Repository -
public interface ClientDetailsRepository extends JpaRepository<ClientEntity, Long> {
#Query("FROM clientMaster c join fetch c.manager m where m.managerId= :managerId")
List<ClientEntity> findClientByManagerId(String managerId);
}
Manager Entity -
#Entity(name = "portfolioManager")
#Table(name = "portfolio_manager")
public class ManagerEntity implements Serializable {
// some fields
#OneToMany(fetch = FetchType.LAZY, mappedBy = "manager")
protected List<ClientEntity> clients = new ArrayList<>();
// Getter & Setter
}
Client Entity -
#Entity(name = "clientMaster")
#Table(name = "clientMaster")
public class ClientEntity implements Serializable {
// some fields
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "manager_id", insertable = false, updatable = false)
protected ManagerEntity manager;
// Getter & Setter
}
And finally, Generate output -
Hibernate: select cliententi0_.client_id as client_id1_0_0_, cliententi0_.manager_id as manager_id2_0_0_, managerent1_.manager_id as manager_id1_2_1_, cliententi0_.created_by as created_by7_0_0_, cliententi0_.created_date as created_date3_0_0_, cliententi0_.client_name as client_name4_0_0_, cliententi0_.sector_name as sector_name5_0_0_, cliententi0_.updated_by as updated_by8_0_0_, cliententi0_.updated_date as updated_date6_0_0_, managerent1_.manager_name as manager_name2_2_1_ from client_master cliententi0_, portfolio_manager managerent1_ where cliententi0_.manager_id=managerent1_.manager_id and managerent1_.manager_id=?```
i have the following two entities and one join table:
entity:
#Entity
#Table(name = "PERSON")
public class parent {
#OneToOne(mappedBy="person", cascade= CascadeType.ALL)
private Reader reader;
//more fields and getters and setters and of course #Id.....
}
and another entity:
#Entity
#Table(name="READER")
public class Reader{
#Id
#Column(name= "READER_ID")
#GeneratedValue(generator="gen")
#GenericGenerator(name="gen", strategy="foreign", parameters=#Parameter(name="property", value="Person"))
private long readerId;
#OneToOne
#PrimaryKeyJoinColumn
private Person person;
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(name = "READER_BOOKS", joinColumns={#JoinColumn(name="READER_ID")}, inverseJoinColumns={#JoinColumn(name="BOOK_ID")})
private List<Books> books;
//more fields and getters and setters
}
And i have a join table in the data base which is named "READER_BOOKS" which is consisted with two columns - readerId and bookId - both are a primary key as a pair, and each of the columns is a foreign key to the corresponding table.
A few details:
I have defined a "ManyToMany" relationship in Reader.java, meaning
that a reader can have many books ,and books can be reused in other
readers ( for example another reader can have the same books as other
readers). Also as you can see the reader id and the person id is the
same - so i defined a "OneToOne" relationship - which works fine.
In the books entity class i didn't add a reference or stated any
relationship because i understood that i don't need to.
The problem:
When i add a Person and set the reader field with several books, it works and the database is filled with the right values.(a person,reader and join tables are inserted with new rows)
The problem is when i delete the Person instance OR the Reader instance AND the books instance is not null(books are assigned to reader). hibernate is just stuck and not throwing any execption.
but when the books instance is null (didnt assign any books to the reader) the deletion is successfull.
I think that something might be wrong with the "ManyToMany" , because when the join table is empty the deletion is successful.
what is wrong in my implementation?
Edited: i found out sometimes hibernate gets stuck even if that list is empty, so i think something is also wrong with the "OneToOne" anottation, maybe i missed out something
there was no problem in my implementation.
i use PL/SQL developer and i didn't notice but it acquired a lock on one of the tables :|.
so when i tried to delete a record, hibernate got stuck and from some reason didn't throw any error.
Any way, the implementation is correct.
I have an entity setup like this:
#Entity
#Table(name = "APP", uniqueConstraints = #UniqueConstraint(columnNames = "APP_KEY"))
public class Application implements java.io.Serializable {
...
#OneToMany(fetch = FetchType.LAZY)
#JoinColumn(name = "application_Id", referencedColumnName = "application_Id")
private Set<Document> documents = new HashSet<Document>(0);
}
Now, in some situations I don't want the list of documents to be returned. However when I serialize this object the "getDocuments()" method will be called.
There will not be an active transaction at this time, so I don't want one of those "no session" errors. I just want to ignore it and have the getDocuments() method return empty, not throw an exception and not try to fetch more data.
Take a look at the FetchGroup capability ref:
Take a look at the jpql fetch join's...
Select e from Employee e join fetch e.phones p where p.areaCode = '613'
This was covered for a similar question here: How to properly express JPQL "join fetch" with "where" clause as JPA 2 CriteriaQuery?
I do not know your serialization setup, however:
If you are using Jackson, you can use this : https://github.com/FasterXML/jackson-datatype-hibernate which works quite nicely.
If you have a manual serialization setup, you should probably not be calling methods directly, without prior knowledge of their initialization status. What you could do is use something like Hibernate.isInitialized(object.Method)
https://docs.jboss.org/hibernate/orm/5.2/javadocs/org/hibernate/Hibernate.html#isInitialized-java.lang.Object-
I'm using Spring 3.2 with Roo 1.2.3 to build a database-backed Java application via Hibernate. I have several bidirectional OneToMany/ManyToOne relationships among the tables in my database. When I set up the ManyToOne side of the relationship using #JoinColumn (via "field reference" in Roo), a new field whose type is the related entity (the "one" in ManyToOne) is created. However, once this is done, there seems to be no way to access the underlying column value on which the ManyToOne relationship is based. This is a problem when the underlying join column contains data needed by the application (i.e. when the join column contains product stock numbers).
Is there any way to set up my entity class so that the column on which its ManyToOne relationship is based remains accessible without traversing the new join property? How can I define an accessor method for the value of this column?
I've been looking online for an answer to this question for several days, but to no avail. Thanks in advance for your help.
just map the column a second time with insertable=false and updateable=false
To make it more concrete. It's possible to do a HQL-SELCT and restrict a ManyToOne relationship, without any join in the resulting SQL:
Instead of using a join in
session.createQuery("FROM Person person WHERE person.adress.id = 42")
we use can use the adress_idcolumn
session.createQuery("FROM Person person WHERE person.adressId = 42")
This works, if you specify an additional adressId field, which is only used as mapping info for Hibernate:
#Entity
#Access(AccessType.FIELD)
public class Person{
#Id
String id;
#JoinColumn(name = "adress_id")
#ManyToOne(fetch = FetchType.LAZY)
#Nullable
public Adress adress;
#Column(name = "adress_id", insertable = false, updatable = false)
private String adressId;
}
#Entity
#Access(FIELD)
public class Adress{
#Id
String id;
}
The AccessType.FIELD is not needed (But we can leave getters/setters in example). The FetchType.LAZY and #Nullable are also optional, but make it clear when it makes sense to use it. We are able to load Person entities which have a specific Address (we know the address id). But we don't need a join because it's not needed for the WHERE-clause and not for the initial fetch (the address can be fetched lazy).