We Keep Coding

Hibernate is making extra SQL statement with #ManyToOne and #Lazy fetching object

I would like someone to explain me why Hibernate is making one extra SQL statement in my straight forward case. Basically i have this object:
#Entity
class ConfigurationTechLog (
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
val id: Long?,
val configurationId: Long,
val type: String,
val value: String?
) {
#JsonIgnore
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "configurationId", insertable = false, updatable = false)
val configuration: Configuration? = null
}
So as you can see, nothing special there. And when i execute this query :
#Query(value = "SELECT c FROM ConfigurationTechLog c where c.id = 10")
fun findById10() : Set<ConfigurationTechLog>
In my console i see this:
Hibernate:
/* SELECT
c
FROM
ConfigurationTechLog c
where
c.id = 10 */ select
configurat0_.id as id1_2_,
configurat0_.configuration_id as configur2_2_,
configurat0_.type as type3_2_,
configurat0_.value as value4_2_
from
configuration_tech_log configurat0_
where
configurat0_.id=10
Hibernate:
select
configurat0_.id as id1_0_0_,
configurat0_.branch_code as branch_c2_0_0_,
configurat0_.country as country3_0_0_,
configurat0_.merchant_name as merchant4_0_0_,
configurat0_.merchant_number as merchant5_0_0_,
configurat0_.org as org6_0_0_,
configurat0_.outlet_id as outlet_i7_0_0_,
configurat0_.platform_merchant_account_name as platform8_0_0_,
configurat0_.store_type as store_ty9_0_0_,
configurat0_.terminal_count as termina10_0_0_
from
configuration configurat0_
where
configurat0_.id=?
Can someone please explain me, what is happening here ? From where this second query is coming from ?

I assume you are using Kotlin data class. The kotlin data class would generate toString, hashCode and equals methods utilizing all the member fields. So if you are using the returned values in your code in a way that results in calling of any of these method may cause this issue.
BTW, using Kotlin data claases is against the basic requirements for JPA Entity as data classes are final classes having final members.

In order to make an association lazy, Hibernate has to create a proxy instance instead of using the real object, i.e. it needs to create an instance of dynamically generated subclass of the association class.
Since in Kotlin all classes are final by default, Hibernate cannot subclass it so it has to create the real object and initialize the association right away. In order to verify this, try declaring the Configuration class as open.
To solve this without the need to explicitly declare all entities open, it is easier to do it via the kotlin-allopen compiler plugin.

This Link can be useful for understand what kind (common) problem is that N + 1 Problem
Let me give you an example:
I have three Courses and each of them have Students related.
I would like to perform a "SELECT * FROM Courses". This is the first query that i want (+ 1) but Hibernate in background, in order to get details about Students for each Course that select * given to us, will execute three more queries, one for each course (N, there are three Course coming from select *). In the end i will see 4 queries into Hibernate Logs
Considering the example before, probably this is what happen in your case: You execute the first query that you want, getting Configuration Id = 10 but after, Hibernate, will take the entity related to this Configuration, then a new query is executed to get this related entity.
This problem should be related in specific to Relationships (of course) and LAZY Fetch. This is not a problem that you have caused but is an Hibernate Performance Issue with LAZY Fetch, consider it a sort of bug or a default behaviour
To solve this kind of problem, i don't know if will be in your case but ... i know three ways:
EAGER Fetch Type (but not the most good option)
Query with JOIN FETCH between Courses and Students
Creating EntityGraph Object that rappresent the Course and SubGraph that rappresent Students and is added to EntityGraph

Looking at your question, it seems like an expected behavior.
Since you've set up configuration to fetch lazily with #ManyToOne(fetch = FetchType.LAZY), the first sql just queries the other variables. When you try to access the configuration object, hibernate queries the db again. That's what lazy fetching is. If you'd like Hibernate to use joins and fetch all values at once, try setting #ManyToOne(fetch = FetchType.EAGER).

When use a JoinType in Criteria ? Hibernate mapping

I'm a bit confused about when use a join with Criteria. Example of what I'm talking about:
.createAlias("cars", "c", JoinType.LEFT_OUTER_JOIN)
Here are the different JoinType:
LEFT_OUTER_JOIN
INNER_JOIN
LEFT_OUTER_JOIN
NONE
RIGHT_OUTER_JOIN
When we map entities with Hibernate there is already a "JoinType" automatically (is that an INNER_JOIN ?).
Simple example with user - car:
User class (table name USERS)
class User {
#Id
#Column(name="ID_USER")
private int idUser;
#Column(name="NAME")
private String name;
#OneToMany
#JoinColumn(name="ID_USER")
private Set<Car> cars;
}
And Car class (table name CARS):
class Car{
#Id
#Column(name="ID_CAR")
private int idCar;
#Column(name="MODEL)
private String model;
#Column(name="ID_USER")
private int idUser;
}
If I have a user u and i type u.getCars() which Join is it mapped by default ?
If I want for example results of:
SELECT * FROM Users u LEFT JOIN Cars c on u.id_user = c.id_user
Then is that correct to use that :
Criteria c = getSession().createCriteria(User.class);
c.createAlias("cars", "c", JoinType.LEFT_OUTER_JOIN);
return c.list();
(And then I loop on User and Car to search what I'm looking for).
And if I'm looking for that:
SELECT * FROM Users u INNER JOIN Cars c on u.id_user = c.id_user
Am I right saying I have no need to create a JoinType.INNER_JOIN ?
If I'm, JoinType.INNER_JOIN is useless when I already have map entites ?
I have to map a lot of complex tables using a lot of join and I'm a little muddled !

When we map entities with Hibernate there is already a "JoinType" automatically (is that an INNER_JOIN ?).
Assuming you have marked the oneToMany association as Fetch.EAGER. In that case when you load User object via hibernate criteria, it uses LEFT OUTER JOIN by default, unless you override it in Criteria. If you have not marked it as FetchType.EAGER, it will be lazy and so no join by default, again, unless you override it in Criteria.
If I have a user u and i type u.getCars() which Join is it mapped by default ?
For EAGER loading refer above. If lazy and it is within the same session you are calling u.getCars, it will fire a new SELECT to load cars based on the userId and so no join.
Am I right saying I have no need to create a JoinType.INNER_JOIN ? If I'm, JoinType.INNER_JOIN is useless when I already have map entites ?
Refer above, by default it is LEFT OUTER JOIN if it is EAGER and unless overidden in Criteria.
On suggestion, - It would be good idea to view the SQLs that hibernate is actually firing in the background via enable show_sql property with different FetchTypes and Criteria JOIN Types, because based on your entiy mappings the count/complexity of SQL queries might be different.

Hibernate - How do I force an Outer Join in a "not in" #Filter?

Ran into this issue today and I was not able to find any solution anywhere else. Actual entities in my application are much bigger, so here is a simplified version of the problem I am facing. I have an entity which has multiple child entities:
public class ParentEntity {
#OneToMany(fetch = FetchType.LAZY, mappedBy = "parentEntity", cascade = CascadeType.ALL)
#Filter(name = "childEntity_1_filter", condition="state = 'ACTIVE'")
private Set<ChildEntity_1> childEntity_1;
#OneToMany(fetch = FetchType.LAZY, mappedBy = "parentEntity", cascade = CascadeType.ALL)
#Filter(name = "childEntity_2_filter", condition = "entity_name not in ('x', 'y')")
private Set<ChildEntity_2> childEntity_2;
}
Now I am fetching the parent entity from the database using Hibernate Criteria.
So I specify both the joins and the filters from above, and then I specify the FetchMode for all these associations as FetchMode.JOIN.
This is where things don't work as expected. My records were not being returned properly and I debugged the SQL generated by hibernate. I saw something like:
# Note that this is hibernate generated sql
select
........
from
parent_entity pe,
childentity_1 ce1,
childentity_2 ce2
where
pe.id=ce1.id(+) and
ce1.state (+)= 'ACTIVE' and -- Notice the (+) here
pe.id=ce2.id(+) and
ce2.entity_name not in ('x', 'y'); -- Notice that (+) is missing here
As you can see above, the filter on the first child entity has the outer join properly applied to the query, while the "not in" filters do not have this join applied to them.
This is causing my query to not return records in some cases. Eg: If there are entities corresponding to the parent entity in the childentity_2 table, but those entities have entity_name = 'x'. This means that even though there are records after the join (since it is an outer join), all of them will be filtered out with the not in filter.
I tried changing the filter to use not equals condition:
#Filter(name = "childEntity_2_filter", condition = "entity_name != 'x' and ...")
In this case hibernate tried to add the (+), but failed spectacularly. It added the (+) between the ! and =. I tried various other combinations of the query like <>, but could not get anything to do what I wanted.
Just to be clear, I know how to fix the query. My question is how to tell hibernate to fix the query.
How do I fix the above #Filter annotation? My problem is that I have a lot of child entities which all have the equals #Filter and they all work fine, but hibernate is not generating the correct query when using not in #Filter.

What is the solution for the N+1 issue in JPA and Hibernate?

I understand that the N+1 problem is where one query is executed to fetch N records and N queries to fetch some relational records.
But how can it be avoided in Hibernate?

The problem
The N+1 query issue happens when you forget to fetch an association and then you need to access it.
For instance, let's assume we have the following JPA query:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
where pc.review = :review
""", PostComment.class)
.setParameter("review", review)
.getResultList();
Now, if we iterate the PostComment entities and traverse the post association:
for(PostComment comment : comments) {
LOGGER.info("The post title is '{}'", comment.getPost().getTitle());
}
Hibernate will generate the following SQL statements:
SELECT pc.id AS id1_1_, pc.post_id AS post_id3_1_, pc.review AS review2_1_
FROM post_comment pc
WHERE pc.review = 'Excellent!'
INFO - Loaded 3 comments
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 1
INFO - The post title is 'Post nr. 1'
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 2
INFO - The post title is 'Post nr. 2'
SELECT pc.id AS id1_0_0_, pc.title AS title2_0_0_
FROM post pc
WHERE pc.id = 3
INFO - The post title is 'Post nr. 3'
That's how the N+1 query issue is generated.
Because the post association is not initialized when fetching the PostComment entities, Hibernate must fetch the Post entity with a secondary query, and for N PostComment entities, N more queries are going to be executed (hence the N+1 query problem).
The fix
The first thing you need to do to tackle this issue is to add [proper SQL logging and monitoring][1]. Without logging, you won't notice the N+1 query issue while developing a certain feature.
Second, to fix it, you can just JOIN FETCH the relationship causing this issue:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
where pc.review = :review
""", PostComment.class)
.setParameter("review", review)
.getResultList();
If you need to fetch multiple child associations, it's better to fetch one collection in the initial query and the second one with a secondary SQL query.
How to automatically detect the N+1 query issue
This issue is better to be caught by integration tests.
You can use an automatic JUnit assert to validate the expected count of generated SQL statements. The db-util project already provides this functionality, and it's open-source and the dependency is available on Maven Central.

Suppose we have a class Manufacturer with a many-to-one relationship with Contact.
We solve this problem by making sure that the initial query fetches all the data needed to load the objects we need in their appropriately initialized state. One way of doing this is using an HQL fetch join. We use the HQL
"from Manufacturer manufacturer join fetch manufacturer.contact contact"
with the fetch statement. This results in an inner join:
select MANUFACTURER.id from manufacturer and contact ... from
MANUFACTURER inner join CONTACT on MANUFACTURER.CONTACT_ID=CONTACT.id
Using a Criteria query we can get the same result from
Criteria criteria = session.createCriteria(Manufacturer.class);
criteria.setFetchMode("contact", FetchMode.EAGER);
which creates the SQL :
select MANUFACTURER.id from MANUFACTURER left outer join CONTACT on
MANUFACTURER.CONTACT_ID=CONTACT.id where 1=1
in both cases, our query returns a list of Manufacturer objects with the contact initialized. Only one query needs to be run to return all the contact and manufacturer information required
for further information here is a link to the problem and the solution.

Native solution for 1 + N in Hibernate, is called:
20.1.5. Using batch fetching
Using batch fetching, Hibernate can load several uninitialized proxies if one proxy is accessed. Batch fetching is an optimization of the lazy select fetching strategy. There are two ways we can configure batch fetching: on the 1) class level and the 2) collection level...
Check these Q & A:
#BatchSize but many round trip in #ManyToOne case
Avoiding n+1 eager fetching of child collection element association
With annotations we can do it like this:
A class level:
#Entity
#BatchSize(size=25)
#Table(...
public class MyEntity implements java.io.Serializable {...
A collection level:
#OneToMany(fetch = FetchType.LAZY...)
#BatchSize(size=25)
public Set<MyEntity> getMyColl()
Lazy loading and batch fetching together represent optimization, which:
does not require any explicit fetching in our queries
will be applied on any amount of references which are (lazily) touched after the root entity is loaded (while explicit fetching effects only these named in query)
will solve issue 1 + N with collections (because only one collection could be fetched with root query) without need to farther processing To get DISTINCT root values (check: Criteria.DISTINCT_ROOT_ENTITY vs Projections.distinct)

You can even get it working without having to add the #BatchSize annotation everywhere, just set the property hibernate.default_batch_fetch_size to the desired value to enable batch fetching globally. See the Hibernate docs for details.
While you are at it, you will probably also want to change the BatchFetchStyle, because the default (LEGACY) is most likely not what you want. So a complete configuration for globally enabling batch fetching would look like this:
hibernate.batch_fetch_style=PADDED
hibernate.default_batch_fetch_size=25
Also, I'm suprised that one of the proposed solutions involves join-fetching. Join-fetching is rarely desirable because it causes more data to be transferred with every result row, even if the dependent entity has already been loaded into the L1 or L2 cache. Thus I would recommend to disable it completey by setting
hibernate.max_fetch_depth=0

This is a frequently asked question so I created the article Eliminate Spring Hibernate N+1 Queries to detail the solutions
To help you detect all the N+1 queries in your application and avoid adding more queries, I created the library spring-hibernate-query-utils that auto-detects the Hibernate N+1 queries.
Here is some code to explain how to add it to your application:
Add the library to your dependencies
<dependency>
<groupId>com.yannbriancon</groupId>
<artifactId>spring-hibernate-query-utils</artifactId>
<version>1.0.0</version>
</dependency>
Configure it in your application properties to return exceptions, default is error logs
hibernate.query.interceptor.error-level=EXCEPTION

If you are using Spring Data JPA to implement your repositories, you can specify lazy fetching in the JPA associations:
#Entity
#Table(name = "film", schema = "public")
public class Film implements Serializable {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "language_id", nullable = false)
private Language language;
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "film")
private Set<FilmActor> filmActors;
...
}
#Entity
#Table(name = "film_actor", schema = "public")
public class FilmActor implements Serializable {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "film_id", nullable = false, insertable = false, updatable = false)
private Film film;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "actor_id", nullable = false, insertable = false, updatable = false)
private Actor actor;
...
}
#Entity
#Table(name = "actor", schema = "public")
public class Actor implements Serializable {
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "actor")
private Set<FilmActor> filmActors;
...
}
And add #EntityGraph to your Spring Data JPA-based repository:
#Repository
public interface FilmDao extends JpaRepository<Film, Integer> {
#EntityGraph(
type = EntityGraphType.FETCH,
attributePaths = {
"language",
"filmActors",
"filmActors.actor"
}
)
Page<Film> findAll(Pageable pageable);
...
}
My blog post at https://tech.asimio.net/2020/11/06/Preventing-N-plus-1-select-problem-using-Spring-Data-JPA-EntityGraph.html helps you preventing the N+1 select problem using Spring Data JPA and #EntityGraph.

Here are some snippet codes that would help you to fix the N+1 Problem.
One to Many Relationship with Manager and Client Entity.
Client JPA Repository -
public interface ClientDetailsRepository extends JpaRepository<ClientEntity, Long> {
#Query("FROM clientMaster c join fetch c.manager m where m.managerId= :managerId")
List<ClientEntity> findClientByManagerId(String managerId);
}
Manager Entity -
#Entity(name = "portfolioManager")
#Table(name = "portfolio_manager")
public class ManagerEntity implements Serializable {
// some fields
#OneToMany(fetch = FetchType.LAZY, mappedBy = "manager")
protected List<ClientEntity> clients = new ArrayList<>();
// Getter & Setter
}
Client Entity -
#Entity(name = "clientMaster")
#Table(name = "clientMaster")
public class ClientEntity implements Serializable {
// some fields
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "manager_id", insertable = false, updatable = false)
protected ManagerEntity manager;
// Getter & Setter
}
And finally, Generate output -
Hibernate: select cliententi0_.client_id as client_id1_0_0_, cliententi0_.manager_id as manager_id2_0_0_, managerent1_.manager_id as manager_id1_2_1_, cliententi0_.created_by as created_by7_0_0_, cliententi0_.created_date as created_date3_0_0_, cliententi0_.client_name as client_name4_0_0_, cliententi0_.sector_name as sector_name5_0_0_, cliententi0_.updated_by as updated_by8_0_0_, cliententi0_.updated_date as updated_date6_0_0_, managerent1_.manager_name as manager_name2_2_1_ from client_master cliententi0_, portfolio_manager managerent1_ where cliententi0_.manager_id=managerent1_.manager_id and managerent1_.manager_id=?```

Hibernate and criteria that return the same items multiple times

I've "Student" and "Class" in relation many to many. In between there is a linking table.
If I want to retrieve all the students with HQL in the following way, everything is fine:
Query queryObject = getSession().createQuery("from Student");
return queryObject.list();
With the above code if there are three students I get a list of three students.
If I use criterias, however, it's fine only as long as there are no associations in the linking table.
Criteria crit = getSession().createCriteria(Student.getClass());
return crit.list();
With the second code, I get three results only when the linking table is empty. However when I add an association, I get 6 results. Looking at the log, Hibernate generates multiple selects. How is it possible?
Can someone explain why does this happen? How can I fix the criteria in order to return the three records only once?
EDIT
In the Student class I mapped in this way:
#ManyToMany(fetch=FetchType.EAGER)
#JoinTable(
name="room_student_rel"
, joinColumns={
#JoinColumn(name="id_student")
}
, inverseJoinColumns={
#JoinColumn(name="id_room")
}
)
private List<Room> rooms;
In the room (class) I mapped in this way:
#OneToMany(fetch=FetchType.EAGER, mappedBy="room")
#Fetch(FetchMode.SELECT)
private List<RoomStudent> roomStudents;

To expand on my previous comment, Hibernate will try to select using an outer join when the FetchType is set to EAGER.
See 2.2.5.5 at the following:
http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html_single/
For the implications of this see here:
Hibernate Criteria returns children multiple times with FetchType.EAGER
Now you can also specify a FetchMode (on your criteria.setFetchMode("assocFiled",FetchMode.LAZY) ) to set it to something other than JOIN or you can use something like criteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY); to filter the duplicates.
Ideally you should avoid EAGER on your entity mappings and where eager fetching is required enable it explicitly on entity load using some hint or via FetchProfile and then deal with the duplicates if required.