We Keep Coding

How to paginate entities with OneToMany relationship with Fetch without warning firstResult/maxResults specified with collection fetch?

I would like to be able to create pagination for pulling all customers from the database (MYSQL), but I encountered a hibernate n+1 problem, which I then solved, but I encountered another problem: 2023-02-09 16:57:04.933 WARN 11660 --- [io-8080-exec-10] o.h.h.internal.ast.QueryTranslatorImpl : HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
This problem I tried to solve with EntityGraph, but still nothing. Then I tried to use two Query, which collected the id and then used the IN clause, but this caused a huge sql query, which led to the generation of many "IN" which, with a huge dataset, can be problematic.
I am currently in a quandary and do not know how to solve this problem. I would like the figures to be fetched along with the customers, but I have no idea how to do it in such a way that the pagination works properly
I want to return CustomerDTO who have numberOfCreatedFigures attribute which is mapping from method in customer entity. This method is returning a size of customer figures.
I am using lombok for args/getters/setters. I've been trying to do everything, but nothing seems to fix the issue.
Config class with a mapper
#Bean
public ModelMapper modelMapper() {
ModelMapper modelMapper = new ModelMapper();
modelMapper.createTypeMap(Customer.class, CustomerDTO.class)
.addMappings(mapper -> mapper
.map(Customer::numberOfCreatedFigures, CustomerDTO::setNumberOfFigures));
return modelMapper;
}
Customer class
public class Customer implements UserDetails, Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#NotBlank(message = "Your name cannot be blank")
private String name;
#NotBlank(message = "Your name cannot be blank")
private String surname;
#NotBlank(message = "Your login cannot be blank")
private String login;
#NotBlank(message = "Your password cannot be blank")
private String password;
#Enumerated(EnumType.STRING)
private Role role;
private Boolean locked = false;
private Boolean enabled = true;
#OneToMany(mappedBy = "createdBy",
cascade = {CascadeType.MERGE, CascadeType.PERSIST},
fetch = FetchType.LAZY,
orphanRemoval = true)
#ToString.Exclude
private Set<Figure> figures = new HashSet<>() ...;
Figure class
public abstract class Figure implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(updatable = false, insertable = false)
private String figureType;
#Version
private Integer version;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "created_by_id")
#CreatedBy
#ToString.Exclude
private Customer createdBy;
#CreatedDate
private LocalDate createdAt;
#LastModifiedDate
private LocalDate lastModifiedAt;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "last_modified_by_id")
#LastModifiedBy
#ToString.Exclude
private Customer lastModifiedBy;
private Integer numberOfModification = 0 ...;
CustomerDTO class
public class CustomerDTO {
private Long id;
private String name;
private String surname;
private String login;
private Integer numberOfFigures;
private Role role;}
Method from Customer Controller
#GetMapping
public ResponseEntity<Page<CustomerDTO>> listAll(#PageableDefault Pageable pageable) {
return new ResponseEntity<>(customerService.listAll(pageable)
.map(customer -> modelMapper
.map(customer, CustomerDTO.class)), HttpStatus.OK);
}

I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Customer.class)
public interface CustomerDTO {
#IdMapping
Long getId();
String getName();
String getSurname();
String getLogin();
#Mapping("SIZE(figures)")
Integer getNumberOfFigures();
Role getRole();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
CustomerDTO a = entityViewManager.find(entityManager, CustomerDTO.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<CustomerDTO> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!

You could load the customers with the figures relationship eagerly initialized.
For this case, an entity graph would be suitable. You'd need to create a new repository method like this:
#Repository
public interface CustomerRepository extends JpaRepository<Customer, Long> {
#EntityGraph(attributePaths = "figures")
List<Customer> findWithFiguresBy(Pageable pageable);
}
Then, you'd need call this repository method when searching instead of the one you are using now. With this approach, your figures relationship can remain lazily fetched (which is generally important as eager fetching is a code smell), but whenever you need to fetch customers with the figures eagerly loaded, you can use this method.
If you want to lear more about entity graphs, I recommend these articles:
JPA Entity Graph by Hibernate maintainer Vlad Mihalcea
JPA Entity Graph by Baeldung
Side note: if you had more than one association which needs to be loaded eagerly, you couldn't use an entity graph for that as it would result in a MultipleBagFetchException. Instead, you would load your parent entities as usual and then collect all ids into a list (say customerIds). Then, you'd need to load all child associations (say figures and otherFigures) by the customer id (JPQL example: select f from Figure f where f.customer.id in :customerIds) and place the figures in a Map<Long, List<Figure> (where the Long parameter is the customer id). Your mapper logic would then need to use the entities from the Maps for the DTOs instead of directly from the parent entity.

Mapping DTO with circular self dependency

I have a entity user with self dependency. When i Map this entity to DTO I have the problem of circular dependency. .
User.class:
public class User {
#Id
#GeneratedValue
private Long id;
#NotNull
#Column(name = "first_name")
private String firstName;
#NotNull
#Column(name = "last_name")
private String lastName;
#JsonBackReference
#ManyToMany(
private List<User> friedns_of = new ArrayList<>();
#ManyToMany(cascade = CascadeType.ALL,
mappedBy = "followers")
private List<User> friends = new ArrayList<>();
UserMapper method in UserMapper:
public static UserResponse toUser(User user) {
UserResponse userResponse = new UserResponse();
userResponse.setId(user.getId());
userResponse.setFollowers(user.getFollowers().stream().map(UserMapper::toUser).toList());
userResponse.setFollowing(user.getFollowing().stream().map(UserMapper::toUser).toList());
return userResponse;
}
When i run the method toUser() I get stackOverFlowError exception caused by the infinite circular dependency. Any advise how to solve this?

One way to resolve this is to model the 'follows' relationship as a separate entity:
#Table(name="user_followers")
public class Follows {
#Id
#GeneratedValue
private Long id;
#NotNull
#Column(name = "follower_Id")
private User follower;
#NotNull
#Column(name = "user_id")
private User user;
}
Then you could give your user two one-to-many lists of these entities:
public class User {
#Id
#GeneratedValue
private Long id;
#OneToMany(mappedBy = "user_id")
private List<Follows> followers;
#OneToMany(mappedBy = "follower_Id")
private List<Follows> following;
}
EDIT: instead of the id field in Follows you could use the user_id and follower_id as a composite primary key using #Embeddable. Omitted here for brevity. See here for more details: https://www.baeldung.com/jpa-many-to-many

Since you already have a DTO of UserResponse, you are on the right path towards a correct solution. My suggestion would be to avoid #ManyToMany on an entity level, and manage followers on a service level.
This means you will have to split relation ManyToMany join column into a separate entity, such as UserFollowsEntity with fields userId and followsUserId. Then remove followers and following lists from your User entity entirely.
Now when creating UserResponse in a service, you will have to
Select the actual user from repository – userRepository.findById(userId)
Select followers – userFollowsRepository.findByFollowsUserId(userId)
Select following – userFollowsRepository.findByUserId(userId)
It is a good practice to try and avoid bidirectional in entities relationships entirely if possible.
EDIT: This will give you two lists: followers and following. You will probably want to know their user names, so what you can do is to merge followers and following lists into one, then extract all user ids from that list. Then query user repository with a list of those IDs, and just attach the required user information to your response model.
Yes it does sound like a bit more work compared to the seeming simplicity of utilizing JPA annotations, but this is the best way to avoid circular dependency as well as decouple the Follower functionality from your user entity.

JPA Hibernate SQL Query to NOT fetch data from one of the associated tables

I am trying to make use of JPA query to fetch the data from DB using below query:
from Candidate c where c.id = :id
where, id = candidate_id(primary key of main table).
This query fetch the data from all of its associated tables as well, whereas my requirement is to fetch the data only from 2 of its associated tables. Because 3rd table contains heavy JSON data which degrades the response time of above query.
I tried using JOIN, Fetch JOIN of parent with its 2 associated tables but it didn't work. I was also looking into how we can skip just one column data(column of 3rd table with heavy JSON) while fetching the Candidate data but with no luck.
Is it possible to achieve it using JPA query or I need to try something else? I am good to save candidate and all its associated tables data in one go using repo.save(candidate) but don't want to fetch one of the related tables data.
This is how I have made the association in spring-boot:
CANDIDATE ENTITY:
#OneToOne(cascade = CascadeType.ALL,
fetch = FetchType.LAZY,
mappedBy = "candidate")
#JsonManagedReference
private Address address;
#OneToMany(cascade = CascadeType.ALL,
fetch = FetchType.LAZY,
mappedBy = "candidate")
#JsonManagedReference
private Set<Skills> skills= new HashSet<>();
#OneToMany(cascade = CascadeType.ALL,
fetch = FetchType.LAZY,
mappedBy = "candidate")
#JsonManagedReference
private Set<Prefrence> prefrences = new HashSet<>();
ADDRESS ENTITY:
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "candidate_id")
#JsonBackReference
private Candidate candidate;
SKILLS ENTITY:
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "candidate_id")
#JsonBackReference
private Candidate candidate;
//rest of the fields
PREFERENCE ENTITY:
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "candidate_id")
#JsonBackReference
private Candidate candidate;
//rest of the fields
I don't want to fetch Preference entity data when using candidateRepo.findById(id) It should be always null in fetched Candidate
Attached the diagram
diagram

What you need here is a DTO and I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Candidate.class)
public interface CandidateDto {
#IdMapping
Long getId();
String getName();
AddressDto getAddress();
Set<SkillsDto> getSkills();
#EntityView(Address.class)
interface AddressDto {
#IdMapping
Long getId();
String getName();
}
#EntityView(Skills.class)
interface SkillsDto {
#IdMapping
Long getId();
String getName();
}
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
CandidateDto a = entityViewManager.find(entityManager, CandidateDto.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<CandidateDto> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!

Why are lazy fields on related entities loaded

In my REST API project (Java 8, Spring Boot 2.3.1) I have a problem with some queries triggering massive query chains by loading lazy relations, even though the related objects are never accessed.
I have a UserEntity and a polymorphic CompanyEntity that are related with a ManyToMany relationship. I have an endpoint that returns all users and I include the IDs of the related companies in the JSON. I excpect a query to the user table and a query to the company table, however all related entities of one sub-entity of CompanyEntity are always loaded for each of those sub-entities resulting in large query chains.
Here are snippets of my classes:
User entity
#Entity(name = "USERS")
public class UserEntity {
#Id
#GeneratedValue
private UUID id;
#EqualsAndHashCode.Exclude
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(fetch = FetchType.LAZY)
#JoinTable(
name = "users_company",
joinColumns = #JoinColumn(name = "USER_ID"),
inverseJoinColumns = #JoinColumn(name = "COMPANY_ID")
)
private Set<CompanyEntity> companies = new HashSet<>();
public List<UUID> getCompanyIds() {
return companies.stream()
.map(CompanyEntity::getId)
.collect(Collectors.toList());
}
}
Polymorphic company entity
#Entity(name = "COMPANY")
#Inheritance(strategy = InheritanceType.JOINED)
public abstract class CompanyEntity {
#Id
#GeneratedValue
private UUID id;
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(mappedBy = "companies", fetch = FetchType.LAZY)
private Set<UserEntity> users = new HashSet<>();
}
Concrete company subclass that triggers the problem
#Entity(name = "CUSTOMER")
public class CustomerEntity extends CompanyEntity {
#NotNull
#OneToOne(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY)
private ContactPersonEntity contactPerson;
#Fetch(FetchMode.SUBSELECT)
#OneToMany(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY, mappedBy = "customer")
private Set<TransactionEntity> transactions = new HashSet<>();
public Set<UUID> getTransactionIds() {
return this.transactions.stream()
.map(TransactionEntity::getId)
.collect(Collectors.toSet());
}
}
In the REST controller I return the following mapping:
#GetMapping(value = "", produces = MediaType.APPLICATION_JSON_VALUE)
public List<UserReadModel> getUsers() {
return userRepository.findAll().stream()
.map(userEntity -> new UserReadModel(userEntity))
.collect(Collectors.toList());
}
Where the UserReadModel is a DTO:
#Data
public class UserReadModel {
private UUID id;
private List<UUID> companyIds;
}
Logging the database queries results in the following output:
// Expected
Hibernate: select userentity0_.id as id1_47_, ... from users userentity0_
Hibernate: select companies0_.user_id ... case when companyent1_1_.id is not null then 1 when companyent1_2_.id is not null then 2 when companyent1_.id is not null then 0 end as clazz_0_ from users_company companies0_ inner join company companyent1_ on companies0_.company_id=companyent1_.id left outer join customer companyent1_1_ on companyent1_.id=companyent1_1_.id left outer join external_editor companyent1_2_ on companyent1_.id=companyent1_2_.id where companies0_.user_id in (select userentity0_.id from users userentity0_)
// Unexpected as they are marked lazy and never accessed
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
...
I've read through loads of articles on entity mapping and lazy loading but I can't seem to find a reason why this behavior persists. Did anyone have this problem before?

You are accessing the collection, so Hibernate has to load the collection. Since you only need the ids and already have a DTO, I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(UserEntity.class)
public interface UserReadModel {
#IdMapping
UUID getId();
#Mapping("companies.id")
Set<UUID> getCompanyIds();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
UserReadModel a = entityViewManager.find(entityManager, UserReadModel.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<UserReadModel> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary! In your case, a query like the following will be generated:
select u.id, uc.company_id
from users u
left join users_company uc on uc.user_id = u.id
left join company c on c.id = uc.company_id
Depending on the Hibernate version, the join for the company might even be omitted.

I eventually figured out the solution and want to post it here, in case anyone stumbles upon this question. This was purely a mistake on my side and is not reproducible from the examples I posted.
I used lombok annotations to generate equals and hashcode methods on the customer entity (and all other entities for that matter) and forgot to annotate the contactPerson and transactions fields with #EqualsAndHashcode.Exclude. As the equals method was called somewhere along the execution, it triggered the lazy loading of those fields. Implementing equals and hashcode manually and using the guidelines from this article for that solved the problem.

How to prevent additional query for OneToOne relation

I have tables:
users (id, name, email, password)
user_statuses (user_id, is_premium, is_advanced, user_rank_id)
user_ranks (id, name, ordinal)
So the relation between User and UserStatus is 1-1, and I have following entity clasess:
#Entity
#Table(name = "users")
#Getter
#Setter
#NoArgsConstructor
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String name;
private String email;
private String password;
#OneToOne(mappedBy = "user", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
private UserStatus status;
}
#Entity
#Table(name = "user_statuses")
#Getter
#Setter
#NoArgsConstructor
public class UserStatus {
#Id
private long id;
#MapsId
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "user_id")
private User user;
private boolean isPremium;
private boolean isAdvanced;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "user_rank_id")
private UserRank rank;
}
#Entity
#Table(name = "user_ranks")
#Getter
#Setter
#NoArgsConstructor
public class UserRank {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String name;
private int ordinal;
}
Then i created endpoint "/users/{id}" which should return user's email address as a string:
#GetMapping("/users/{id}")
public String getUserEmail(#PathVariable("id") long userId) {
User user = service.getUser(userId);
return user.getEmail();
}
When I call above endpoint I get user's email address as a response, however looking at the console log I see that hibernate executed 2 queries but noone asked him to do so:
First one for fetching the user:
SELECT
user0_.id AS id1_2_0_,
user0_.email AS email2_2_0_,
user0_.name AS name3_2_0_,
user0_.password AS password4_2_0_
FROM
users user0_
WHERE
user0_.id = 1;
And second one for fetching User Status that is associated with this User object:
SELECT
userstatus0_.user_id AS user_id1_1_0_,
userstatus0_.is_advanced AS is_advan2_1_0_,
userstatus0_.is_premium AS is_premi3_1_0_,
userstatus0_.user_rank_id AS user_ran4_1_0_
FROM
user_statuses userstatus0_
WHERE
userstatus0_.user_id = 1;
So I am confused: Why is hibernate running second query when I set fetch = FetchType.LAZY on each relation... It looks like that LAZY is ignored for #OneToOne annotation?
I do not use EntityGraph.
How to stop hibernate for running second query?
EDIT
So, it turns out Hibernate ignores my Lazy hint because it needs to decide should it initialize property with NULL or ProxyObject which makes sense. This link explains it well:
https://thorben-janssen.com/hibernate-tip-lazy-loading-one-to-one/
However this link also suggests that the best way to model this is Unidirectional One to One and it says that I can always fetch UserStatus based on User's ID (because both tables "shares" primary key)
However this confuses me a little bit, because I can fetch both rows using single query (SELECT * FROM users LEFT JOIN user_statuses ON users.id = user_statuses.user_id), but with approach described in the link I need 2 queries, and as far as I know (which I might be wrong) is 1 query is better than executing 2 queries, also if I want to fetch 25 users and their User Statuses, then I would also need 2 queries, one for fetching users and then fetching corespoinding user statuses and finally write nested for each loops to join these objects. I could have just executed one single query to fetch everything...

It is possible to make OTO lazy even if it's not the owning side. You just need to mark it as optional = false. This way Hibernate will know that it can safely a create proxy (and null is not possible) as the association always exists. Note, though it really must be non-optional - the 2nd entity must always exist. Otherwise you'll get an exception once Hibernate tries to load it lazily.
As for the number of queries, with native Hibernate (not JPA!) you can select org.hibernate.annotations.FetchMode. Which gives options to:
Use a separate select
Or use a join to load association
Alternatively, you can stay with JPA and write a JPQL query and use fetch join to keep it as a single query.
PS: before doing additional select Hibernate will check if the element already exists within the Session. If it is, then no select is going to be issued. But with fetch join or FetchMode.JOIN you won't have this luxury - join will always happen.

For one to one relation in hibernate it is always loading reference object whether you keep Fetch type Lazy or Eager. So alternate solution is select only those columns which are needed, it should not contain that reference column. So in this case hibernate will not fire another query.
Query for below class will be :
#Query("select new Example(id,field1) from Example")
#Entity
#Table(name = "example")
class Example implements Serializable {
private static final long serialVersionUID = 1L;
public Example(Long id, String field1) {
this.id = id;
this.field1 = field1;
}
#Id
#Column(name = "id", nullable = false, updatable = false)
private Long id;
#OneToOne(mappedBy = "example", fetch = LAZY, cascade = ALL)
private CustomerDetails customerDetails;
#Column(name = "field1", nullable = false, updatable = false)
private String field1;
}