We Keep Coding

How to paginate entities with OneToMany relationship with Fetch without warning firstResult/maxResults specified with collection fetch?

I would like to be able to create pagination for pulling all customers from the database (MYSQL), but I encountered a hibernate n+1 problem, which I then solved, but I encountered another problem: 2023-02-09 16:57:04.933 WARN 11660 --- [io-8080-exec-10] o.h.h.internal.ast.QueryTranslatorImpl : HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
This problem I tried to solve with EntityGraph, but still nothing. Then I tried to use two Query, which collected the id and then used the IN clause, but this caused a huge sql query, which led to the generation of many "IN" which, with a huge dataset, can be problematic.
I am currently in a quandary and do not know how to solve this problem. I would like the figures to be fetched along with the customers, but I have no idea how to do it in such a way that the pagination works properly
I want to return CustomerDTO who have numberOfCreatedFigures attribute which is mapping from method in customer entity. This method is returning a size of customer figures.
I am using lombok for args/getters/setters. I've been trying to do everything, but nothing seems to fix the issue.
Config class with a mapper
#Bean
public ModelMapper modelMapper() {
ModelMapper modelMapper = new ModelMapper();
modelMapper.createTypeMap(Customer.class, CustomerDTO.class)
.addMappings(mapper -> mapper
.map(Customer::numberOfCreatedFigures, CustomerDTO::setNumberOfFigures));
return modelMapper;
}
Customer class
public class Customer implements UserDetails, Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#NotBlank(message = "Your name cannot be blank")
private String name;
#NotBlank(message = "Your name cannot be blank")
private String surname;
#NotBlank(message = "Your login cannot be blank")
private String login;
#NotBlank(message = "Your password cannot be blank")
private String password;
#Enumerated(EnumType.STRING)
private Role role;
private Boolean locked = false;
private Boolean enabled = true;
#OneToMany(mappedBy = "createdBy",
cascade = {CascadeType.MERGE, CascadeType.PERSIST},
fetch = FetchType.LAZY,
orphanRemoval = true)
#ToString.Exclude
private Set<Figure> figures = new HashSet<>() ...;
Figure class
public abstract class Figure implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(updatable = false, insertable = false)
private String figureType;
#Version
private Integer version;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "created_by_id")
#CreatedBy
#ToString.Exclude
private Customer createdBy;
#CreatedDate
private LocalDate createdAt;
#LastModifiedDate
private LocalDate lastModifiedAt;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "last_modified_by_id")
#LastModifiedBy
#ToString.Exclude
private Customer lastModifiedBy;
private Integer numberOfModification = 0 ...;
CustomerDTO class
public class CustomerDTO {
private Long id;
private String name;
private String surname;
private String login;
private Integer numberOfFigures;
private Role role;}
Method from Customer Controller
#GetMapping
public ResponseEntity<Page<CustomerDTO>> listAll(#PageableDefault Pageable pageable) {
return new ResponseEntity<>(customerService.listAll(pageable)
.map(customer -> modelMapper
.map(customer, CustomerDTO.class)), HttpStatus.OK);
}

I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Customer.class)
public interface CustomerDTO {
#IdMapping
Long getId();
String getName();
String getSurname();
String getLogin();
#Mapping("SIZE(figures)")
Integer getNumberOfFigures();
Role getRole();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
CustomerDTO a = entityViewManager.find(entityManager, CustomerDTO.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<CustomerDTO> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!

You could load the customers with the figures relationship eagerly initialized.
For this case, an entity graph would be suitable. You'd need to create a new repository method like this:
#Repository
public interface CustomerRepository extends JpaRepository<Customer, Long> {
#EntityGraph(attributePaths = "figures")
List<Customer> findWithFiguresBy(Pageable pageable);
}
Then, you'd need call this repository method when searching instead of the one you are using now. With this approach, your figures relationship can remain lazily fetched (which is generally important as eager fetching is a code smell), but whenever you need to fetch customers with the figures eagerly loaded, you can use this method.
If you want to lear more about entity graphs, I recommend these articles:
JPA Entity Graph by Hibernate maintainer Vlad Mihalcea
JPA Entity Graph by Baeldung
Side note: if you had more than one association which needs to be loaded eagerly, you couldn't use an entity graph for that as it would result in a MultipleBagFetchException. Instead, you would load your parent entities as usual and then collect all ids into a list (say customerIds). Then, you'd need to load all child associations (say figures and otherFigures) by the customer id (JPQL example: select f from Figure f where f.customer.id in :customerIds) and place the figures in a Map<Long, List<Figure> (where the Long parameter is the customer id). Your mapper logic would then need to use the entities from the Maps for the DTOs instead of directly from the parent entity.

JPA Criteria API Specification for Many to Many

I have got three classes as mentioned below. I am trying to create a specification to filter data where there is a match in the linked table.
public class Album {
private Long id;
private List<AlbumTag> albumTags;
}
public class Tag {
private Long id;
private String category;
}
public class AlbumTag{
private Long id;
private Album album;
private Tag tag;
}
In the schema given above what I am trying to find is a list of all albums from Album table with the link in AlbumTag. The SQL that I want to achieve, doesn't have to be same, is below
select *
from Album A
where (A.Id in (select [AT].AlbumId
from AlbumTag [AT]))
What I have tried so far which is not working, of course, is below
public class AlbumWithTagSpecification implements Specification<Album> {
#Override
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> cq, CriteriaBuilder cb) {
final Subquery<Long> personQuery = cq.subquery(Long.class);
final Root<Album> album = personQuery.from(Album.class);
final Join<Album, AlbumTag> albumTags = album.join("albumTags");
personQuery.select((albumTags.get("album")).get("id"));
personQuery.where(cb.equal(album.get("id"), (albumTags.get("album")).get("id")));
return cb.in(root.get("id")).value(personQuery);
}
}

Using spring boot and spring data JPA, you can prefer entity relationship to fetch the data.
1.Annotate the domain class with the entity relationship which given below:
#Entity
#Table(name="Album")
public class Album {
#Id
#Column(name="id")
private Long id;
#OneToMany(targetEntity = AlbumTag.class, mappedBy = "album")
private List<AlbumTag> albumTags;
//getter and setter
}
#Entity
#Table(name="Tag")
public class Tag {
#Id
#Column(name="id")
private Long id;
#Column(name="category")
private String category;
//getter and setter
}
#Entity
#Table(name="AlbumTag")
public class AlbumTag{
#Id
#Column(name="id")
private Long id;
#ManyToOne(optional = false, targetEntity = Album.class)
#JoinColumn(name = "id", referencedColumnName="id", insertable = false, updatable = false)
private Album album;
#ManyToOne(optional = false, targetEntity = Tag.class)
#JoinColumn(name = "id", referencedColumnName="id", insertable = false, updatable = false)
private Tag tag;
//getter and setter
}
2.use the spring data to fetch the details using the below:
Album album = ablumRepository.findOne(1); // get the complete details about individual album.
List<AlbumTag> albumTags = ablum.getAlbumTags(); // get the all related albumTags details for particular album.
I hope this will help you to solve it.

Subqueries in JPA only really work with CriteriaBuilder.exists() so i would try:
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> cq, CriteriaBuilder cb) {
final Subquery<Long> subQuery = cq.subquery(Long.class);
final Root<AlbumTag> albumTag = subQuery.from(AlbumTag.class);
// it doesn't really matter what we select
subQuery.select(cb.literal(1));
subQuery.where(cb.equal(root.get("id"), (albumTag.get("album")).get("id")));
return cb.exists(subQuery);
}
which is equivalent to
select *
from Album A
where exists(
select 1 from AlbumTag AT
where AT.AlbumId = A.Id
)

Well, I wouldn't go for in operation in this case - it just complicates the query and the specification. The problem you described is actually matter of joining records from Table A with related records from Table B so the query in your case would be like:
SELECT a from Album a join AlbumTag at on a.id = at.albumId - as you needed it will return all albums that have album tags. Inner join explained
So in your case I would create this "factory" method that would create for you this specification.
public static Specification<Album> withTags() {
return new Specification<Album>() {
#Override
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
return root.join("albumTags").getOn();
}
};
}
Also I would suggest you to have a look at static metamodel library from hibernate - link to introduction. It generates for you static model from your entity classes that helps you avoid creating queries/specifications using hardcoded strings.

creteria query for join tables
CriteriaQuery<Album> query = cb.createQuery(Album.class);
Root<Album> album = query.from(Teacher.class);
Join<Album, AlbumTag> tag = teacher.join("id");
query.select(tag).where(cb.equal(album.get("album")));
List<Album> results = em.createQuery(query).getResultList();
for (Album al : results) {
System.out.println("album-->+al.get(name));
}

This looks like a classic many to many example. The three classes you have map directly to the tables you would expect in the database. JPA is an Object Relational Mapping (ORM) library which means we can structure the classes in a more OO style and map to the underlying relational database.
The AlbumTag class can be omitted and the #ManyToMany relationship added to both Album and Tag.
public class Album {
private Long id;
#ManyToMany
#JoinTable(name="AlbumTag",
joinColumns=
#JoinColumn(name="album", referencedColumnName="id"),
inverseJoinColumns=
#JoinColumn(name="tag", referencedColumnName="id"))
private List<Tag> tags;
}
public class Tag {
private Long id;
private String category;
#ManyToMany(mappedBy="tags")
private List<Album> albums;
}
To find albums by Tag you would first retrieve the Tag from the repository using something like findById(1l); or findByCategory("Rock"); and then simply call getAlbums() on the Tag object.
Note: One slight difference here is that the AlbumTag table would have only two columns (album and tag). The extra id column on AlbumTag is unnecessary since the combination of album and tag would be a unique id and you would never need to find by id in this table anyway.

Since you are using spring-data-jpa you should really take advantage of the features it provides.
My first question is related to your entity classes. I do not understand why is it necesary to store a list of album tags in the album class. Since you have a join table this information is reduntant.
Secondly you should adnotate your entity clases:
#Entity
public class Album {
#Id
#Column
private Long id;
}
#Entity
public class Tag {
#Id
#Column
private Long id;
#Column
private String category;
}
#Entity
#Table
public class AlbumTag{
#Id
#Column
private Long id;
#ManyToOne
#JoinColumn
private Album album;
#ManyToOne
#JoinColumn
private Tag tag;
}
Next you should create repositories for your entity classes.
interface AlbumRepository extends JpaRepository<Album, Long>{
#Query
("select DISTINCT(a) from AlbumTag at "+
"join at.album a "
"where at.tag is not null")
List<Album> findAlbumWithTag();
}
Then simply call the repository function which will return a list of albums which have at least one tag.

How to query and persist static objects in MySQL Database with Spring Boot?

I am working on a Restful service built with Java Spring and I have some issues modeling the data. I want to store shelfs with books. The books belong to a given category. I have a POST request to store shelfs to a mysql database (via service and CrudRepository). However I am not able to store more than one book of the same category. Here are my (simplified) entities.
A Shelf with an id and a collection of books.
#Entity
public class Shelf{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, mappedBy = "shelf")
private List<Book> books= new ArrayList<>();
...
}
The class Book is defined as follows:
#Entity
public class Book{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#ManyToOne(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
#JoinColumn(name = "category_id")
private Category category;
#OneToOne(fetch = FetchType.EAGER, cascade = CascadeType.ALL)
#JsonIgnore
private Shelf shelf;
Each book belongs to a category(e.g. thriller, fiction, etc.). Here is the category entity:
#Entity
public class Category {
private Long id;
private String name;
And finally my Controller:
#RestController
public class ShelfController {
#Autowired
private ShelfService shelfService;
#PostMapping("/shelfs")
public Shelf addShelf(#RequestBody Shelf shelf) {
return shelfService.addShelf(shelf);
}
Now here is my problem: The categories will be given and there will be no option to change these, I would therefore like to have them stored in the database or hard code them as static objects. In the Post request for new shelfs I would like to provide only the category id and make the controller find the corresponding object itself.
What I did so far was to treat the categories as a usual Entity, so whenever I added a new shelf with books having a category_id, the category was created with the given id and an empty name. But as soon as I used the same category id again, the application threw a com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '1' for key 'PRIMARY' exception. I don't want the controller to create new category objects, but instead want it to fetch the corresponding objects from a service or a static Collection.
So my question is: How can I achieve this?
Hints for solutions/tricks to improve the design are most welcome, I am new to the topic.

One solution is to create Data Transfer Objects (DTO). Example:
class ShelfDTO {
private Long id;
private List<Long> bookIds;
}
Then use this class to receive the POST requests:
#RestController
public class ShelfController {
#Autowired private ShelfService shelfService;
#PostMapping("/shelfs")
public Shelf addShelf(#RequestBody ShelfDTO shelfDto) {
return shelfService.addShelf(shelfDto);
}
}
Then modify your ShelfService to convert the DTO to an Entity:
#Service
public class ShelfService {
#Autowired private ShelfRepository shelfRepository;
#Autowired private BookRepository bookRepository;
#Transactional
public Shelf addShelf(ShelfDTO shelfDto) {
List<Book> books = bookRepository.findAllById(shelfDto.getBookIds());
return shelfService.addShelf(new Shelf(books));
}
}
Final comment: I noticed that you have a bidirectional relationship. You are responsible for keeping it in a consistent state.
The easiest way is to create the methods addTo(shelf, book) and removeFrom(shelf, book) that encapsulate the logic of both adding the book to the list in the shelf and setting the shelf in the book.

Error in filtering by child object in Spring Data JPA Query

My code structure looks like the following.
Article:
#Entity
public class NewsArticle{
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
[Other class properties such as title, publisher, publishedDate, etc.]
#OneToMany(mappedBy = "article")
private Set<UserReadNewsArticle> userReadNewsArticles = new HashSet<>();
[Getters and Setters]
}
Article read by User:
#Entity
public class UserReadNewsArticle {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
private Long readAccountId;
private Long readArticleId;
#JsonIgnore
#ManyToOne
private Account account;
#JsonIgnore
#ManyToOne
private NewsArticle article;
[Getters and Setters]
}
Account:
#Entity
public class Account {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
[Other class properties]
#OneToMany(mappedBy = "account")
private Set<UserReadNewsArticle> userReadNewsArticles = new HashSet<>();
[Getters and Setters]
}
I want to have a query method in my NewsArticleRepository to get all the Read News Articles for a user.
public interface NewsArticleRepository extends PagingAndSortingRepository<NewsArticle, Long>{
Collection<NewsArticle> findByUserReadNewsArticlesReadAccountId(Long readAccountId);
}
This method works great. But how can I write a Spring Data JPA Query/Method to get the "Unread News Articles for a user". What I have tried is the following.
Collection<NewsArticle> findByUserReadNewsArticlesReadAccountIdNot(Long readAccountId);
This one does return a list of articles which have been read by other users. But my requirement is to get all the unread news articles. I have gone through Spring Data JPA Documentation but failed to come up with an easier soultion. How can I overcome this issue? Or am I doing something wrong?

You could achieve your result by using a JPQL query with also a subquery:
public interface NewsArticleRepository extends PagingAndSortingRepository<NewsArticle, Long> {
#Query("SELECT n FROM NewsArticle n WHERE n NOT IN "
+ "(SELECT ur.article FROM UserReadNewsArticle ur JOIN ur.account a WHERE a.id = :readAccountId)")
Collection<NewsArticle> findByUserReadNewsArticlesReadAccountIdNotIn(#Param("readAccountId") Long readAccountId);
}
http://localhost:8080/newsArticles/search/findByUserReadNewsArticlesReadAccountIdNotIn?readAccountId=1
So first get the read articels from the current user and then exlude them from the whole article list.
I don't think that spring data is able to get you the same, since a subquery is definitetly needed. If I'm wrong, somebody can correct me.

Best architectural choice to merge 2 Spring Data JPA repository classes

I am pretty new in Spring Data JPA and I have the following doubt about the best way to implement the following situation:
So basically I have the following 2 model classes:
Room (representing a room of an accomodation):
#Entity
#Table(name = "room")
public class Room implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#ManyToOne
#JoinColumn(name = "id_accomodation_fk", nullable = false)
private Accomodation accomodation;
#ManyToOne
#JoinColumn(name = "id_room_tipology_fk", nullable = false)
private RoomTipology roomTipology;
#Column(name = "room_number")
private String number;
#Column(name = "room_name")
private String name;
#Column(name = "room_description")
#Type(type="text")
private String description;
#Column(name = "max_people")
private Integer maxPeople;
#Column(name = "is_enabled")
private Boolean isEnabled;
public Room() {
}
// GETTER AND SETTER METHODS
}
And RoomTipology that represent a tipology of room (something like: single room, double bed room, etcetc):
#Entity
#Table(name = "room_tipology")
public class RoomTipology implements Serializable{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#Column(name = "tipology_name")
private String name;
#Column(name = "tipology_description")
private String description;
#Column(name = "time_stamp")
private Date timeStamp;
#OneToMany(mappedBy = "roomTipology")
private List<Room> rooms;
#OneToOne(mappedBy = "roomTipology")
private RoomRate roomRate;
// GETTER AND SETTER METHODS
}
Ok, using Spring Data JPA I will have 2 different repository classes (one for the Room entity class and another one for the RoomTipology entity class, something like this:
#Repository
#Transactional(propagation = Propagation.MANDATORY)
public interface RoomDAO extends JpaRepository<Room, Long> {
//#Query("FROM Room WHERE accomodation = :id")
List<Room> findByAccomodation(Accomodation accomodation);
}
#Repository
#Transactional(propagation = Propagation.MANDATORY)
public interface RoomTipologyDAO extends JpaRepository<RoomTipologyDAO , Long> {
// METHOD RELATED TO THE ACCESS TO ROOM TIPOLOGY ENTITIES
}
Ok, I have the following architectural doubt:
I have 2 little repositories classes that access to something that are semantically similar (the room concept and the room tipology concept are both related to the room).
Furthermore, as you can see in the code of the RoomTipology entity class there is the following field:
#OneToMany(mappedBy = "roomTipology")
private List<Room> rooms;
that is mapped by the #OneToMany annotation (because starting from a specific room tipology I want to access to all the room of this accomodation of this tipology: all the single bed room or all the double bed room and so on...).
So, following this architectural style, I will have the method that return the List associated to a room tipology into the RoomTipologyDAO repository class and not into the RoomTipology repository class..it works fine but it is semantically bad because I will have a method of RoomTipologyDAO that doesn't return something related to RoomTipology instance but a list of Room object.
Is it not nasty?
So what is the best way to create an architecture that uses Spring Data JPA in this case?
I can't not do something like:
public interface RoomDAO extends JpaRepository<Room, Long> extends JpaRepository<RoomTipology, Long> {
........................................................
........................................................
........................................................
}
because Java doesn't support multiple heredity, but I think that the best choice should obtain something like this.
Maybe can I create something like a RoomMetaDAO class that have the RoomDAO and the RoomTipologyDAO as field? Can it work?
What do you think could be the best architectural choice for my situation?

You are absolutely correct in being sceptical about this.
The mistake is to assume that you should have one repository per entity. Instead you should look into the concept of aggregate roots from domain driven design.
An aggregate root is an entity that is used to manipulate a bunch of entities that can only accessed and modified through the aggregate root.
You want one repository per such aggregate root, which would be in your case the Room.
This is explained in much more detail in this article by Oliver Gierke, lead of the Spring Data project.