I have a list of projects and a list of customers. A project can be for one customer and every customer can have many projects. So it's a simple 1:n relationship where the project is the owning side.
Simplified to the essential it is
#Entity
public class Project {
#Id
long id;
#ManyToOne(optional = true)
#JoinColumn(name = "customer", nullable = true, updatable = true)
Customer customer;
}
#Entity
public class Customer {
#Id
long id;
}
When I load a list of projects, I want to retrieve the customers efficiently at the same time. This is not the case. There is one single query for the projects and then for every distinct customer that is encountered a separate query is issued.
So say I have 100 projects that are assigned to 50 different customers. This would result in one query for the projects and 50 queries for the customers.
This quickly adds up and for large project/customer lists our application gets rather slow. Also this is just one example. All our entities with relationships are affected by this behavior.
I already tried #Fetch(FetchMode.JOIN) on the customers field as suggested here but it does nothing and FetchMode.SUBQUERY is not applicable according to Hibernate:
org.hibernate.AnnotationException: Use of FetchMode.SUBSELECT not allowed on ToOne associations
How can I fix this problem?
If you are using Spring Data JPA to implement your repositories, you can specify lazy fetching in the JPA entities:
#Entity
public class Project {
#Id
long id;
#ManyToOne(fetch = FetchType.LAZY, optional = true)
#JoinColumn(name = "customer", nullable = true, updatable = true)
Customer customer;
}
#Entity
public class Customer {
#Id
long id;
...
}
And add #EntityGraph to your Spring Data JPA-based repository:
#Repository
public interface ProjectDao extends JpaRepository<Project, Long> {
#EntityGraph(
type = EntityGraphType.FETCH,
attributePaths = {
"customer"
}
)
Optional<Project> findById(Long id);
...
}
My blog post at https://tech.asimio.net/2020/11/06/Preventing-N-plus-1-select-problem-using-Spring-Data-JPA-EntityGraph.html helps you preventing the N+1 select problem using Spring Data JPA and #EntityGraph.
Yes, it is a by-the-book example of the n+1 selects problem.
The approach I use in most cases is to make the association lazy and define a batch size.
Alternatively, you could use a JPQL query with [left] join fetch to initialize the association directly from the query result set:
select p from Project p left join fetch p.customer
Yes, it is a by-the-book example of the n+1 selects problem as #dragan-bozanovic said.
In Spring-Boot 2.1.3 #Fetch(FetchMode.JOIN) can be used to solve it:
#ManyToOne(optional = true)
#Fetch(FetchMode.JOIN)
#JoinColumn(name = "customer", nullable = true, updatable = true)
Customer customer;
Warning: If the relationship can be invalid, for example when marked with #NotFound(action = NotFoundAction.IGNORE), each invalid relationship will trigger another SELECT query.
Related
Those are my classes. I want to delete items from likedCourses. So I expect JPA to delete item from course_like table. I looked and try to understand from other examples but couldn't. It is like deletion doesn't exist in JPA realm when there is relation. It is good for selecting though. I'd want to share what I tried but I couldn't find anything about it.
Note : I see that in ManyToMany relationship there is not orphanRemoval option.
#Entity
class Student {
#Id
Long id;
#ManyToMany
#JoinTable(
name = "course_like",
joinColumns = #JoinColumn(name = "student_id"),
inverseJoinColumns = #JoinColumn(name = "course_id"))
Set<Course> likedCourses
}
#Entity
class Course {
#Id
Long id;
#ManyToMany(mappedBy = "likedCourses")
Set<Student> likes;
}
If you want to remove an entry from the course_like table, you will have to load the Student and remove the element from the likedCourses set that should be removed. If you do that in a #Transactional method, you will see that Hibernate will emit delete statements to delete the rows that represent the removed objects from the likedCourses set. That's the magic of an ORM, it synchronizes the object graph with the database, without you telling it to emit statement A, B, ...
In my REST API project (Java 8, Spring Boot 2.3.1) I have a problem with some queries triggering massive query chains by loading lazy relations, even though the related objects are never accessed.
I have a UserEntity and a polymorphic CompanyEntity that are related with a ManyToMany relationship. I have an endpoint that returns all users and I include the IDs of the related companies in the JSON. I excpect a query to the user table and a query to the company table, however all related entities of one sub-entity of CompanyEntity are always loaded for each of those sub-entities resulting in large query chains.
Here are snippets of my classes:
User entity
#Entity(name = "USERS")
public class UserEntity {
#Id
#GeneratedValue
private UUID id;
#EqualsAndHashCode.Exclude
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(fetch = FetchType.LAZY)
#JoinTable(
name = "users_company",
joinColumns = #JoinColumn(name = "USER_ID"),
inverseJoinColumns = #JoinColumn(name = "COMPANY_ID")
)
private Set<CompanyEntity> companies = new HashSet<>();
public List<UUID> getCompanyIds() {
return companies.stream()
.map(CompanyEntity::getId)
.collect(Collectors.toList());
}
}
Polymorphic company entity
#Entity(name = "COMPANY")
#Inheritance(strategy = InheritanceType.JOINED)
public abstract class CompanyEntity {
#Id
#GeneratedValue
private UUID id;
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(mappedBy = "companies", fetch = FetchType.LAZY)
private Set<UserEntity> users = new HashSet<>();
}
Concrete company subclass that triggers the problem
#Entity(name = "CUSTOMER")
public class CustomerEntity extends CompanyEntity {
#NotNull
#OneToOne(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY)
private ContactPersonEntity contactPerson;
#Fetch(FetchMode.SUBSELECT)
#OneToMany(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY, mappedBy = "customer")
private Set<TransactionEntity> transactions = new HashSet<>();
public Set<UUID> getTransactionIds() {
return this.transactions.stream()
.map(TransactionEntity::getId)
.collect(Collectors.toSet());
}
}
In the REST controller I return the following mapping:
#GetMapping(value = "", produces = MediaType.APPLICATION_JSON_VALUE)
public List<UserReadModel> getUsers() {
return userRepository.findAll().stream()
.map(userEntity -> new UserReadModel(userEntity))
.collect(Collectors.toList());
}
Where the UserReadModel is a DTO:
#Data
public class UserReadModel {
private UUID id;
private List<UUID> companyIds;
}
Logging the database queries results in the following output:
// Expected
Hibernate: select userentity0_.id as id1_47_, ... from users userentity0_
Hibernate: select companies0_.user_id ... case when companyent1_1_.id is not null then 1 when companyent1_2_.id is not null then 2 when companyent1_.id is not null then 0 end as clazz_0_ from users_company companies0_ inner join company companyent1_ on companies0_.company_id=companyent1_.id left outer join customer companyent1_1_ on companyent1_.id=companyent1_1_.id left outer join external_editor companyent1_2_ on companyent1_.id=companyent1_2_.id where companies0_.user_id in (select userentity0_.id from users userentity0_)
// Unexpected as they are marked lazy and never accessed
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
...
I've read through loads of articles on entity mapping and lazy loading but I can't seem to find a reason why this behavior persists. Did anyone have this problem before?
You are accessing the collection, so Hibernate has to load the collection. Since you only need the ids and already have a DTO, I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(UserEntity.class)
public interface UserReadModel {
#IdMapping
UUID getId();
#Mapping("companies.id")
Set<UUID> getCompanyIds();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
UserReadModel a = entityViewManager.find(entityManager, UserReadModel.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<UserReadModel> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary! In your case, a query like the following will be generated:
select u.id, uc.company_id
from users u
left join users_company uc on uc.user_id = u.id
left join company c on c.id = uc.company_id
Depending on the Hibernate version, the join for the company might even be omitted.
I eventually figured out the solution and want to post it here, in case anyone stumbles upon this question. This was purely a mistake on my side and is not reproducible from the examples I posted.
I used lombok annotations to generate equals and hashcode methods on the customer entity (and all other entities for that matter) and forgot to annotate the contactPerson and transactions fields with #EqualsAndHashcode.Exclude. As the equals method was called somewhere along the execution, it triggered the lazy loading of those fields. Implementing equals and hashcode manually and using the guidelines from this article for that solved the problem.
I'm working with Spring Boot 2.3, Spring Data and Hibernate.
I've the following entities
#Entity
#Getter
#Setter
#EqualsAndHashCode(of = "id")
public class User {
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Id
private Long id;
private String name;
#OneToOne(mappedBy = "user", fetch = FetchType.LAZY, cascade = CascadeType.REMOVE)
private Address address;
#Version
private Long version;
}
#Entity
#Getter
#Setter
#EqualsAndHashCode(of = "id")
public class Address {
#Id
private Long id;
private String fullAddress;
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "id")
#MapsId
private User user;
#Version
private Long version;
}
When the following code is executed, any query related to the user repository is performed (and for me it is the expected behavior).
Address addressFromDb = addressRepository.findAll().get(0);
log.info("" + addressFromDb.getUser().getId());
// select address0_.id as id1_0_, address0_.full_address as full_add2_0_, address0_.version as version3_0_ from address address0_
but when I execute the following code, then there are multiple queries and I don't understanding why. Apparently the FetchType.LAZY from user to address is not honored.
User userFromDb = userRepository.findAll().get(0);
// select user0_.id as id1_4_, user0_.name as name2_4_, user0_.version as version3_4_ from user user0_
// select address0_.id as id1_0_0_, address0_.full_address as full_add2_0_0_, address0_.version as version3_0_0_ from address address0_ where address0_.id=?
What am I missing?
In order to be more helpful and more clear I've created the following github repo
Hibernate (or more specifically PersistenceContext) needs to know, whether the entity exists or not, so that it can decide, whether to provide a proxy for the entity or null. This does not apply for XToMany relationships, because the whole collection can be wrapped in a proxy and in special case it will be empty.
It is also important to point out, that FetchType is just a suggestion for the JPa implementation and there is no guarantee, that in every case it will be fulfilled. You can read more about #OneToOne here, especially in terms of fetching strategy:
While the unidirectional #OneToOne association can be fetched lazily, the parent-side of a bidirectional #OneToOne association is not. Even when specifying that the association is not optional and we have the FetchType.LAZY, the parent-side association behaves like a FetchType.EAGER relationship. And EAGER fetching is bad.
Even if the FK is NOT NULL and the parent-side is aware about its non-nullability through the optional attribute (e.g. #OneToOne(mappedBy = "post", fetch = FetchType.LAZY, optional = false)), Hibernate still generates a secondary select statement.
For every managed entity, the Persistence Context requires both the entity type and the identifier,
so the child identifier must be known when loading the parent entity, and the only way to find the associated post_details primary key is to execute a secondary query.
Bytecode enhancement is the only viable workaround. However, it only works if the parent side is annotated with #LazyToOne(LazyToOneOption.NO_PROXY) and the child side is not using #MapsId.
I want to define a bidirectional #OneToMany relation. I did it the following way:
public class Customer {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#OneToMany(mappedBy = "customer", cascade = CascadeType.ALL, orphanRemoval = true)
#JsonIgnore
#JsonProperty(access = JsonProperty.Access.WRITE_ONLY)
private List<CustomerEmployeeRole> roles;
// more fields
}
public class CustomerEmployeeRole {
#ManyToOne
#NotNull
private Customer customer;
// more fields
}
Everything works as expected, the only problem is that n+1 queries are executed when i delete a customer. n is the number of roles the customer has. For every role the following query is executed:
delete from customer_employee_role where id=?
It would be possible to delete all roles in one query:
delete from customer_employee_role where customer_id=?
I assume hibernate does n+1 queries because of mappedBy = "customer". I want to keep the annotation, because i want to avoid a join table. Is there any way to tell hibernate to only execute one query instead of n queries in this case? Otherwise i would have to write my own queries which would work too, but it´s not really elegant.
The n+1 queries are because of the CascadeType.ALL. Since you set this attribute, the CustomerEmployeeRole's are deleted too. So if you want to remove a Customer let's see what happens assuming that the customer have 2 roles :
1) delete from CustomerEmployeeRole where id = ..
2) delete from CustomerEmployeeRole where id = ..
3) delete from Customer where id = .. here is your +1 query.
P.S.: In case you have any more questions about which is the best way to map a #OneToMany, #ManyToMany, #OneToOne or what is happening behind the scenes this is a good tutorial. ( Same for the other annotations )
I have entity Workflow which has #OneToMany relation with ValidationResults class. It's fetch Lazy but sometimes I would like to get all the Workflows and interate on them accessing the ValidationResults. In that moment I want jpa to get all the data eagerly not query each time I access ValidationResults. I use springDataJpa, How to do it, is there any way to do it with #Query ?
I try to achieve something like that but I don't know how
//here all the workflows has corresponding data eagerly
List<Workflow> workflows = workflowService.getAllWorkflowsWithValidationResultsEagerly();
//here validationResults ref is lazy, when I try to access it it does query
List<Workflow> workflows = workflowService.getAllWorkflowsUsually();
Here are my entities.
#Entity
#Table(name = "workflow")
public class Workflow {
..............
#OneToMany(fetch = FetchType.LAZY, mappedBy = "workflow", cascade = CascadeType.ALL)
private Set<ValidationResults> validationResultsSet = new HashSet<>();
public Set<ValidationResults> getValidationResultsSet(){return this.validationResultsSet;}
...............
}
And ValidationResult class
#Entity
#Table(name = "validation_results")
public class ValidationResults {
...
#ManyToOne
#JoinColumn(name = "workflow_id", insertable = false, updatable = false)
private Workflow workflow;
....
}
The spring boot-ish way of doing this is by using the #EntityGraph as described in the documentation.
You can use fetch join in order to do it on #Query https://www.logicbig.com/tutorials/java-ee-tutorial/jpa/fetch-join.html
#Query("SELECT DISTINCT e FROM Employee e INNER JOIN FETCH e.tasks t")
If you don't want to create another query, just call .size() of your list