We Keep Coding

How to paginate entities with OneToMany relationship with Fetch without warning firstResult/maxResults specified with collection fetch?

I would like to be able to create pagination for pulling all customers from the database (MYSQL), but I encountered a hibernate n+1 problem, which I then solved, but I encountered another problem: 2023-02-09 16:57:04.933 WARN 11660 --- [io-8080-exec-10] o.h.h.internal.ast.QueryTranslatorImpl : HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
This problem I tried to solve with EntityGraph, but still nothing. Then I tried to use two Query, which collected the id and then used the IN clause, but this caused a huge sql query, which led to the generation of many "IN" which, with a huge dataset, can be problematic.
I am currently in a quandary and do not know how to solve this problem. I would like the figures to be fetched along with the customers, but I have no idea how to do it in such a way that the pagination works properly
I want to return CustomerDTO who have numberOfCreatedFigures attribute which is mapping from method in customer entity. This method is returning a size of customer figures.
I am using lombok for args/getters/setters. I've been trying to do everything, but nothing seems to fix the issue.
Config class with a mapper
#Bean
public ModelMapper modelMapper() {
ModelMapper modelMapper = new ModelMapper();
modelMapper.createTypeMap(Customer.class, CustomerDTO.class)
.addMappings(mapper -> mapper
.map(Customer::numberOfCreatedFigures, CustomerDTO::setNumberOfFigures));
return modelMapper;
}
Customer class
public class Customer implements UserDetails, Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#NotBlank(message = "Your name cannot be blank")
private String name;
#NotBlank(message = "Your name cannot be blank")
private String surname;
#NotBlank(message = "Your login cannot be blank")
private String login;
#NotBlank(message = "Your password cannot be blank")
private String password;
#Enumerated(EnumType.STRING)
private Role role;
private Boolean locked = false;
private Boolean enabled = true;
#OneToMany(mappedBy = "createdBy",
cascade = {CascadeType.MERGE, CascadeType.PERSIST},
fetch = FetchType.LAZY,
orphanRemoval = true)
#ToString.Exclude
private Set<Figure> figures = new HashSet<>() ...;
Figure class
public abstract class Figure implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(updatable = false, insertable = false)
private String figureType;
#Version
private Integer version;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "created_by_id")
#CreatedBy
#ToString.Exclude
private Customer createdBy;
#CreatedDate
private LocalDate createdAt;
#LastModifiedDate
private LocalDate lastModifiedAt;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "last_modified_by_id")
#LastModifiedBy
#ToString.Exclude
private Customer lastModifiedBy;
private Integer numberOfModification = 0 ...;
CustomerDTO class
public class CustomerDTO {
private Long id;
private String name;
private String surname;
private String login;
private Integer numberOfFigures;
private Role role;}
Method from Customer Controller
#GetMapping
public ResponseEntity<Page<CustomerDTO>> listAll(#PageableDefault Pageable pageable) {
return new ResponseEntity<>(customerService.listAll(pageable)
.map(customer -> modelMapper
.map(customer, CustomerDTO.class)), HttpStatus.OK);
}

I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Customer.class)
public interface CustomerDTO {
#IdMapping
Long getId();
String getName();
String getSurname();
String getLogin();
#Mapping("SIZE(figures)")
Integer getNumberOfFigures();
Role getRole();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
CustomerDTO a = entityViewManager.find(entityManager, CustomerDTO.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<CustomerDTO> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!

You could load the customers with the figures relationship eagerly initialized.
For this case, an entity graph would be suitable. You'd need to create a new repository method like this:
#Repository
public interface CustomerRepository extends JpaRepository<Customer, Long> {
#EntityGraph(attributePaths = "figures")
List<Customer> findWithFiguresBy(Pageable pageable);
}
Then, you'd need call this repository method when searching instead of the one you are using now. With this approach, your figures relationship can remain lazily fetched (which is generally important as eager fetching is a code smell), but whenever you need to fetch customers with the figures eagerly loaded, you can use this method.
If you want to lear more about entity graphs, I recommend these articles:
JPA Entity Graph by Hibernate maintainer Vlad Mihalcea
JPA Entity Graph by Baeldung
Side note: if you had more than one association which needs to be loaded eagerly, you couldn't use an entity graph for that as it would result in a MultipleBagFetchException. Instead, you would load your parent entities as usual and then collect all ids into a list (say customerIds). Then, you'd need to load all child associations (say figures and otherFigures) by the customer id (JPQL example: select f from Figure f where f.customer.id in :customerIds) and place the figures in a Map<Long, List<Figure> (where the Long parameter is the customer id). Your mapper logic would then need to use the entities from the Maps for the DTOs instead of directly from the parent entity.

Mapping DTO with circular self dependency

I have a entity user with self dependency. When i Map this entity to DTO I have the problem of circular dependency. .
User.class:
public class User {
#Id
#GeneratedValue
private Long id;
#NotNull
#Column(name = "first_name")
private String firstName;
#NotNull
#Column(name = "last_name")
private String lastName;
#JsonBackReference
#ManyToMany(
private List<User> friedns_of = new ArrayList<>();
#ManyToMany(cascade = CascadeType.ALL,
mappedBy = "followers")
private List<User> friends = new ArrayList<>();
UserMapper method in UserMapper:
public static UserResponse toUser(User user) {
UserResponse userResponse = new UserResponse();
userResponse.setId(user.getId());
userResponse.setFollowers(user.getFollowers().stream().map(UserMapper::toUser).toList());
userResponse.setFollowing(user.getFollowing().stream().map(UserMapper::toUser).toList());
return userResponse;
}
When i run the method toUser() I get stackOverFlowError exception caused by the infinite circular dependency. Any advise how to solve this?

One way to resolve this is to model the 'follows' relationship as a separate entity:
#Table(name="user_followers")
public class Follows {
#Id
#GeneratedValue
private Long id;
#NotNull
#Column(name = "follower_Id")
private User follower;
#NotNull
#Column(name = "user_id")
private User user;
}
Then you could give your user two one-to-many lists of these entities:
public class User {
#Id
#GeneratedValue
private Long id;
#OneToMany(mappedBy = "user_id")
private List<Follows> followers;
#OneToMany(mappedBy = "follower_Id")
private List<Follows> following;
}
EDIT: instead of the id field in Follows you could use the user_id and follower_id as a composite primary key using #Embeddable. Omitted here for brevity. See here for more details: https://www.baeldung.com/jpa-many-to-many

Since you already have a DTO of UserResponse, you are on the right path towards a correct solution. My suggestion would be to avoid #ManyToMany on an entity level, and manage followers on a service level.
This means you will have to split relation ManyToMany join column into a separate entity, such as UserFollowsEntity with fields userId and followsUserId. Then remove followers and following lists from your User entity entirely.
Now when creating UserResponse in a service, you will have to
Select the actual user from repository – userRepository.findById(userId)
Select followers – userFollowsRepository.findByFollowsUserId(userId)
Select following – userFollowsRepository.findByUserId(userId)
It is a good practice to try and avoid bidirectional in entities relationships entirely if possible.
EDIT: This will give you two lists: followers and following. You will probably want to know their user names, so what you can do is to merge followers and following lists into one, then extract all user ids from that list. Then query user repository with a list of those IDs, and just attach the required user information to your response model.
Yes it does sound like a bit more work compared to the seeming simplicity of utilizing JPA annotations, but this is the best way to avoid circular dependency as well as decouple the Follower functionality from your user entity.

How to get Count of OneToMany field in JPA Enity?

How can we get the count of OneToMany field of JPA entity as querying count for each parent entity while fetching as a list is costly and there is no way in JPA Repository.
I want to get the number of likes and comments for each PostEntity. The field is Lazy fetch type and if I call likes.size() or comments.size() then it will load all of the comments and likes from database and there can be thousands of comments and likes.
I know I can create a seperate repo for likes and comments to get the counts but while calling method from PostRepository how to get the counts for each and every entity? What is the best and efficient way?
Parent Entity
#Entity
#Table(name = "posts")
#Getter
#Setter
public class PostEntity extends MappedSuperClassEntity<UserEntity> {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int id;
#Nullable
private String title;
#Nullable
private String postText;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name="user_id")
private UserEntity user;
#Nullable
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "community_id")
private CommunityEntity community;
#OneToMany(fetch = FetchType.LAZY)
private List<CommentEntity> comments;
#OneToMany(fetch = FetchType.LAZY)
private List<LikeEntity> likes;
#Transient
private int numberOfLikes;
#Transient
private int numberOfComments;
}
I would like to get the likes and comments count for each PostEntity while querying for the list of posts.
My Repo
public interface PostsRepository extends PagingAndSortingRepository<PostEntity, Integer> {
#Query(value = "SELECT P FROM PostEntity P WHERE P.user.id = :userId ORDER BY P.createdDate DESC")
Page<PostEntity> getUserPosts(int userId, Pageable pageable);
#Query(value = "select P from PostEntity P where p.community.id = :communityId order by P.createdDate desc")
Page<PostEntity> getCommunityPosts(int communityId, Pageable pageable);
}
I searched for a lot and someone suggested to use #Formula annotation for custom queries on the entity field but #Formula is hibernate specific and don't know if it works with #Transient field. Is there any JPA specific way to do that as it's a common problem.

You need "LazyCollection" annotation with EXTRA option.
#OneToMany(fetch = FetchType.LAZY)
#LazyCollection(LazyCollectionOption.EXTRA)
private List<CommentEntity> comments;
This annotation would allow to access "size()" without loading.
You can check this article.
https://www.baeldung.com/hibernate-lazycollection
Sometimes, we're only concerned with the properties of the collection, and we don't need the objects inside it right away. For example, going back to the Branch and the Employees example, we could just need the number of employees in the branch while not caring about the actual employees' entities. In this case, we consider using the EXTRA option. Let's update our example to handle this case. Similar to the case before, the Branch entity has an id, name, and an #OneToMany relation with the Employee entity. However, we set the option for #LazyCollection to be EXTRA:
I try to add comment but i have no writing comment access because of reputation so i send an answer.

How to prevent additional query for OneToOne relation

I have tables:
users (id, name, email, password)
user_statuses (user_id, is_premium, is_advanced, user_rank_id)
user_ranks (id, name, ordinal)
So the relation between User and UserStatus is 1-1, and I have following entity clasess:
#Entity
#Table(name = "users")
#Getter
#Setter
#NoArgsConstructor
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String name;
private String email;
private String password;
#OneToOne(mappedBy = "user", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
private UserStatus status;
}
#Entity
#Table(name = "user_statuses")
#Getter
#Setter
#NoArgsConstructor
public class UserStatus {
#Id
private long id;
#MapsId
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "user_id")
private User user;
private boolean isPremium;
private boolean isAdvanced;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "user_rank_id")
private UserRank rank;
}
#Entity
#Table(name = "user_ranks")
#Getter
#Setter
#NoArgsConstructor
public class UserRank {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String name;
private int ordinal;
}
Then i created endpoint "/users/{id}" which should return user's email address as a string:
#GetMapping("/users/{id}")
public String getUserEmail(#PathVariable("id") long userId) {
User user = service.getUser(userId);
return user.getEmail();
}
When I call above endpoint I get user's email address as a response, however looking at the console log I see that hibernate executed 2 queries but noone asked him to do so:
First one for fetching the user:
SELECT
user0_.id AS id1_2_0_,
user0_.email AS email2_2_0_,
user0_.name AS name3_2_0_,
user0_.password AS password4_2_0_
FROM
users user0_
WHERE
user0_.id = 1;
And second one for fetching User Status that is associated with this User object:
SELECT
userstatus0_.user_id AS user_id1_1_0_,
userstatus0_.is_advanced AS is_advan2_1_0_,
userstatus0_.is_premium AS is_premi3_1_0_,
userstatus0_.user_rank_id AS user_ran4_1_0_
FROM
user_statuses userstatus0_
WHERE
userstatus0_.user_id = 1;
So I am confused: Why is hibernate running second query when I set fetch = FetchType.LAZY on each relation... It looks like that LAZY is ignored for #OneToOne annotation?
I do not use EntityGraph.
How to stop hibernate for running second query?
EDIT
So, it turns out Hibernate ignores my Lazy hint because it needs to decide should it initialize property with NULL or ProxyObject which makes sense. This link explains it well:
https://thorben-janssen.com/hibernate-tip-lazy-loading-one-to-one/
However this link also suggests that the best way to model this is Unidirectional One to One and it says that I can always fetch UserStatus based on User's ID (because both tables "shares" primary key)
However this confuses me a little bit, because I can fetch both rows using single query (SELECT * FROM users LEFT JOIN user_statuses ON users.id = user_statuses.user_id), but with approach described in the link I need 2 queries, and as far as I know (which I might be wrong) is 1 query is better than executing 2 queries, also if I want to fetch 25 users and their User Statuses, then I would also need 2 queries, one for fetching users and then fetching corespoinding user statuses and finally write nested for each loops to join these objects. I could have just executed one single query to fetch everything...

It is possible to make OTO lazy even if it's not the owning side. You just need to mark it as optional = false. This way Hibernate will know that it can safely a create proxy (and null is not possible) as the association always exists. Note, though it really must be non-optional - the 2nd entity must always exist. Otherwise you'll get an exception once Hibernate tries to load it lazily.
As for the number of queries, with native Hibernate (not JPA!) you can select org.hibernate.annotations.FetchMode. Which gives options to:
Use a separate select
Or use a join to load association
Alternatively, you can stay with JPA and write a JPQL query and use fetch join to keep it as a single query.
PS: before doing additional select Hibernate will check if the element already exists within the Session. If it is, then no select is going to be issued. But with fetch join or FetchMode.JOIN you won't have this luxury - join will always happen.

For one to one relation in hibernate it is always loading reference object whether you keep Fetch type Lazy or Eager. So alternate solution is select only those columns which are needed, it should not contain that reference column. So in this case hibernate will not fire another query.
Query for below class will be :
#Query("select new Example(id,field1) from Example")
#Entity
#Table(name = "example")
class Example implements Serializable {
private static final long serialVersionUID = 1L;
public Example(Long id, String field1) {
this.id = id;
this.field1 = field1;
}
#Id
#Column(name = "id", nullable = false, updatable = false)
private Long id;
#OneToOne(mappedBy = "example", fetch = LAZY, cascade = ALL)
private CustomerDetails customerDetails;
#Column(name = "field1", nullable = false, updatable = false)
private String field1;
}

What is the most efficient way to retrieve specific data from DB without doing loop for all records?

I want to ask about what is the most efficient way to search about specific data from a database without doing a for loop in all of the records?
I have a project on java spring and I have this Entity:
#Entity
#Table(name = "USERS") public class USERS {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "UID")
private Integer id;
#Column(name = "FName")
private String firstName;
#Column(name = "SName")
private String secondName;
#Column(name = "TName")
private String thirdName;
#Column(name = "LName")
private String fourthName;
#Column(name = "Email")
private String email;
#Column(name = "PW")
private String password;
#Column(name = "MNumber")
private String mobileNumber;
#Column(name = "ISDeleted")
private boolean isUserDeleted;
//---------------------- Getters and Setters ----------------------
and I made this service:
public List<USERS> findAllActive() {
List<USERS> usersList = new ArrayList<USERS>();
for (USERS users: usersRepository.findAll()){
if (!users.isUserDeleted()){
usersList.add(users);
}
}
return usersList;
}
For example; I have one property for User, if he is active or not.
So, my question; what is the most efficient way to do get specific data like retrieving all of the active users from the DB without doing a for loop like in the code above? Because if the list of users is a 1 Million or more, it could have performance issues.

Assuming that you are using JpaRepository then you can create custom query.
#Query("SELECT u FROM USERS u WHERE u.userDeleted = false")
List<USERS> findNotDeletedUsers();
and then call usersRepository.findNotDeletedUsers();

First of all, use an index on the field you want to search on (this won't help you much if the column has only two distinct values, but will make a huge difference if the value has high sparsity).
#Entity
#Table(name = "USERS",
indexes = {
// not a huge performance gain, since the column values are true/false
#Index(name = "index_by_active", columnList="ISDeleted", unique = false),
// possible huge performance gain, since only the relevant records are scanned
#Index(name = "index_by_first_name", columnList="FName", unique = false)})
public class USERS {...}
Then, define a query method that uses the indexed field (if you are using spring data it would look as follows).
public interface UsersRepository extends CrudRepository<USERS, Long> {
List<USERS> findUsersByISDeleted(boolean deleted);
List<USERS> findUsersByFName(String name);
List<USERS> findUsersByFNameAndISDeleted(String name, boolean deleted);
}
Queries on indexed fields will leverage the underlying index and provide an efficient access plan (so you won't end up scanning the whole table in order to extract a subset of entities matching a given criteria).

The solution from #Madis is okay. But if you always want to get users which are not deleted in all queries, you can specify it on Entity:
#Entity
#Table(name = "USERS")
#Where("ISDeleted = false")
public class USERS {
So now the condition "ISDeleted = false" is automatically append to all queries from the UserRepository. You can use usersRepository.findAll() instead of.

You don't need to specify any sql query or where clause. CrudRepository will do it for you automatically. Just use below code and pass true/false on need basis
List<Users> findIsUserDeleted(boolean isDeleted)