The following entity relationship exists: A 1 <-> * B (edited down for brevity)
#Entity
public class EntityA {
#Id
UUID id;
#Column
String searchText;
#OneToMany(mappedBy = "entity_a", fetch = FetchType.LAZY)
List<EntityB> listOfB;
}
#Entity
public class EntityB {
#Id
UUID id;
#Column
String searchText;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "entity_a_id")
EntityA entityA;
}
I would like to retrieve all entities (EntityA) where a specific field (searchText) matches a criteria - easy enough. Now I want to go through all linked entities (EntityB) with a field (searchText) matching the criteria as well, only returning the matching entities. This result can be in any format for eg:
Map<EntityA, List<EntityB>> (The parent EntityA, possibly matching, with matching children EntityB)
List<EntityA_EntityBSearchResult> (some custom domain model)
List<EntityA> (bad: don't use entities with not conforming to expected relationship)
There are two main constrains complicating this
The outer entity (EntityA) should stay pageable
The resulting list (of EntityB) is limited to a maximum of 10 items
What I've tried:
Building the SQL before converting to JPA Query
For simplicity the criteria is a simple like '%textToFind%' on field searchText
1. Retrieve all of EntityB where EntityA or EntityB matches criteria.
select a.*, b.* from entity_a a
inner join entity_b b on b.entity_a_id = a.id
where a.search_text like '%textToFind%' or b.search_text like '%textToFind%'
Good: Pageable on EntityA
Bad: EntityB not limited to max of 10 items. Grouping will have to be done afterwards to Map<EntityA, List<EntityB>>.
2. Retrieve all of EntityB where EntityA or EntityB matches criteria - limited to 10 of EntityB.
select * from entity_b where id in (
select limitedB.id from (
select ROW_NUMBER() OVER(PARTITION BY a.id ORDER BY b.created_date_time DESC) AS RowNumber, b.* from entity_a a
inner join entity_b b on b.entity_a_id = a.id
where a.search_text like '%textToFind%' or b.search_text like '%textToFind%') limitedB
WHERE RowNumber <= 10)
Good: EntityB limited to 10
Bad: Not pageable on EntityA
3. Retrieve all of EntityA first (where EntityA or EntityB matches criteria). Followed by retrieval of EntityB (where EntityA or EntityB matches criteria) with parent EntityA id in previous list.
select DISTINCT a.* from entity_a a
inner join entity_b b on b.entity_a_id = a.id
where a.search_text like '%textToFind%' or b.search_text like '%textToFind%'
use the result to create an EntityA id list (List<UUID> EntityA_Ids). Send this list to the next query:
select * from entity_b where id in (
select limitedB.id from (
select ROW_NUMBER() OVER(PARTITION BY a.id ORDER BY b.created_date_time DESC) AS RowNumber, b.* from entity_a a
inner join entity_b b on b.entity_a_id = a.id
where a.search_text like '%textToFind%' or b.search_text like '%textToFind%') limitedB
WHERE RowNumber <= 10) and entity_a_id in EntityA_Ids
From this result a map (Map<EntityA, List<EntityB>>) can easily be created with Collectors.groupingBy.
Good: EntityA is pageable and EntityB is limited to 10
Bad: Multiple queries
I have the following entity relationship in my data model.
ERD
Entity A: one-to-many :Entity B
Entity B: one-to-many :Entity C
Entity B: one-to-many :Entity D
Hibernate Entities:
public class EntityA {
private Integer id;
#OneToMany
private List<EntityB> entityBList;
}
public class EntityB {
private Integer id;
#ManyToOne
private EntityA entityA;
#OneToMany(fetch=FetchType.LAZY)
private List<EntityC> entityCList;
#OneToMany(fetch=FetchType.LAZY)
private List<EntityD> entityDList;
}
public class EntityC {
private Integer id;
#ManyToOne
private EntityB entityB;
}
public class EntityD {
private Integer id;
#ManyToOne
private EntityB entityB;
}
Requirement: I want to query on Entity C with join fetch of Entity B and also eagerly fetch Entity D at the same time. After the query is completed I expect that if I do entityC.getEntityB().getEntityDList(), it should not lead to N+1 query problem in hibernate.
I was trying the following JPQL query:
select ec from EntityC ec
join fetch ec.entityB eb
join fetch eb.entityD ed
where ec.id = :id
This leads to duplicates in the result because of cross join with Entity D. Instead of one result, I get n results where n is the size of list of Entity D.
How can I avoid this? Is there any way to fetch the Entity D without cross join in JPQL?
The first thing is to use the DISTINCT JPQL keyword on your query, e.g. as:
TypedQuery<EntityC> query = em.createQuery("SELECT DISTINCT ec FROM EntityC ec JOIN FETCH ec.entityB eb JOIN FETCH eb.entityDList ed WHERE ec.id = :id", EntityC.class);
// ^^^^^^^^
This will eliminate the duplicates but has the side-effect of passing the DISTINCT to the SQL query as well, which is not something that you want. Read details here, here and in this excellent answer. Long story short - quoting Vlad Mihalcea's answer:
By passing DISTINCT to the SQL query, the EXECUTION PLAN is going to execute an extra Sort phase which adds overhead without bringing any value[...]
The solution is in the linked articles and answers, but in short, if you are on Hibernate >= 5.2.2, use the hibernate.query.passDistinctThrough hint:
TypedQuery<EntityC> query = em.createQuery("SELECT DISTINCT ec FROM EntityC ec JOIN FETCH ec.entityB eb JOIN FETCH eb.entityDList ed WHERE ec.id = :id", EntityC.class);
query.setHint("hibernate.query.passDistinctThrough", false);
// -OR-
query.setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false); // import org.hibernate.jpa.QueryHints
I have 4 tables in relations. A,B,C,D.
So I wrote select bellow:
select NEW org.example.ExtendsA(a,b.name,c.name,d.name)
from A a LEFT JOIN a.bItems b LEFT JOIN a.cItems c LEFT JOIN b.dItems d
order by b.name ASC;
A is unique but the relations are incomplette.
and I tried this:
select NEW org.example.ExtendsA(a,b.name,c.name,d.name)
from A a LEFT JOIN FETCH a.bItems b LEFT JOIN FETCH a.cItems c
LEFT JOIN FETCH b.dItems d order by b.name ASC;
A is not unique.
A object definition is:
#Entity
public class A implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "ID")
private Long id;
#NotNull
#Size(min = 1, max = 100)
#Column(name = "NAME")
private String name;
#OneToMany(mappedBy = "aId")
private List<B> bItems;
#OneToMany(mappedBy = "aId")
private List<C> cItems;
}
Some relations are empty but need A object with null relations.
Some A object has more than one relations between B and C and I want to select with all in one A object (distinct A).
Would you help me how to solve this issue? Maybe the approach is bad?
I use EclipeLink data provider.
This is a typical problem with loading OneToMany relathionships.
This is because of the nature of SQL result sets. Imagine a SQL result of a join of entity with it's other entities linked to it. On the left side, fields of this entity will be duplicated as many times as many related entities it has.
Unforunately, EclipseLink doesn't filter them out and you get many items of the same entity in your result. Although, EclipseLink is smart enough and each item will actually be the same Java object instance.
It's also the reason why you can't use setMaxResults in such queries.
You need to use distinct keyword that in this particular keys will not be mapped to a real SQL distinct, but will filter duplicated entities. Or, you can filter them manually.
I have the following situation:
#Entity
public class Period
{
String Name;
}
#Entity
public class Bill
{
Period period;
#OneToMany(mappedBy = "bill", fetch = FetchType.LAZY)
private List<Entry> entry = new ArrayList<Entry>(0);
}
#Entity
public class Entry
{
#NotNull
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "BILL_ID", nullable = false)
Bill bill;
String text;
BigDecimal amount;
}
So what I need is to fetch all the data in a single query, either with the root being the Bill or the Entry using JPA 2.0 criteria (with Hibernate behind). I've read few posts about this problem HERE and HERE and it seems that I can't use subqueries in the result or fetch data two levels deep.
EDIT: To make my problem more clear: When I use Entry as root, I can't fetch Period and when I use Bill as root I can't fetch all other tables in Entry. Also I can't use eager fetch because there are other use cases that need those tables.
Are there any other ways to do this?
Thanks!
To fetch data from association, you use left join fetch clauses:
select distinct b from Bill b
left join fetch b.period
left join fetch b.entry
where b...
or
select distinct e from Entry e
left join fetch e.bill b
left join fetch b.period
where e...
Regarding Criteria, its fetch() method returns a Fetch, which itself has a method fetch() returning a Fetch(), which itself has a method fetch() returning a Fetch, etc. So yes, its supports as many levels you want.
I am using Spring Data JpaRepository and I find it extremely easy to use. I actually need all those features - paging, sorting, filtering. Unfortunately there is one little nasty thing that seems to force me to fall back to use of plain JPA.
I need to order by a size of associated collection. For instance I have:
#Entity
public class A{
#Id
private long id;
#OneToMany
private List<B> bes;
//boilerplate
}
and I have to sort by bes.size()
Is there a way to somehow customize the ordering still taking the advantage of pagination, filtering and other Spring Data great features?
I've solved the puzzle using hints and inspirations from:
Limiting resultset using #Query anotations by Koitoer
How to order by count() in JPA by MicSim
Exhaustive experiments on my own
The first and most important thing I've not been aware of about spring-data is that even using #Query custom methods one can still create paging queries by simply passing the Pageable object as parameter. This is something that could have been explicitely stated by spring-data documentation as it is definitely not obvious though very powerful feature.
Great, now the second problem - how do I actually sort the results by size of associated collection in JPA? I've managed to come to a following JPQL:
select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a
where AwithBCount is a class that the query results are actually mapped to:
public class AwithBCount{
private Long bCount;
private A a;
public AwithBCount(Long bCount, A a){
this.bCount = bCount;
this.a = a;
}
//getters
}
Excited that I can now simply define my repository like the one below
public interface ARepository extends JpaRepository<A, Long> {
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCount(Pageable pageable);
}
I hurried to try my solution out. Perfect - the page is returned but when I tried to sort by bCount I got disappointed. It turned out that since this is a ARepository (not AwithBCount repository) spring-data will try to look for a bCount property in A instead of AwithBCount. So finally I ended up with three custom methods:
public interface ARepository extends JpaRepository<A, Long> {
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCount(Pageable pageable);
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a order by bCount asc",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCountOrderByCountAsc(Pageable pageable);
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a order by bCount desc",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCountOrderByCountDesc(Pageable pageable);
}
...and some additional conditional logic on service level (which could be probably encapsulated with an abstract repository implementation). So, although not extremely elegant, that made the trick - this way (having more complex entities) I can sort by other properties, do the filtering and pagination.
One option, which is much simpler than the original solution and which also has additional benefits, is to create a database view of aggregate data and link your Entity to this by means of a #SecondaryTable or #OneToOne.
For example:
create view a_summary_view as
select
a_id as id,
count(*) as b_count,
sum(value) as b_total,
max(some_date) as last_b_date
from b
Using #SecondaryTable
#Entity
#Table
#SecondaryTable(name = "a_summary_view",
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id", referencedColumnName= "id")})
public class A{
#Column(table = "a_summary_view")
private Integer bCount;
#Column(table = "a_summary_view")
private BigDecimal bTotal;
#Column(table = "a_summary_view")
private Date lastBDate;
}
You can now then sort, filer, query etc purely with reference to entity A.
As an additional advantage you have within your domain model data that may be expensive to compute in-memory e.g. the total value of all orders for a customer without having to load all orders or revert to a separate query.
Thank you #Alan Hay, this solution worked fine for me. I just had to set the foreignKey attribute of the #SecondaryTable annotation and everything worked fine (otherwise Spring Boot tried to add a foreignkey constraint to the id, which raise an error for a sql View).
Result:
#SecondaryTable(name = "product_view",
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id", referencedColumnName = "id")},
foreignKey = #javax.persistence.ForeignKey(ConstraintMode.NO_CONSTRAINT))
I don't know much about Spring Data but for JPQL, to sort the objects by size of associated collection, you can use the query
Select a from A a order by a.bes.size desc
You can use the name of an attribute found in the select clause as a sort property:
#Query(value = "select a, count(b) as besCount from A a join a.bes b group by a", countQuery = "select count(a) from A a")
Page<Tuple> findAllWithBesCount(Pageable pageable);
You can now sort on property besCount :
findAllWithBesCount(PageRequest.of(1, 10, Sort.Direction.ASC, "besCount"));
I used nativeQuery to arrange sorting by number of records from another table, pagable works.
#Query(value = "SELECT * FROM posts where posts.is_active = 1 and posts.moderation_status = 'ACCEPTED' " +
"group by posts.id order by (SELECT count(post_id) FROM post_comments where post_id = posts.id) desc",
countQuery = "SELECT count(*) FROM posts",
nativeQuery = true)
Page <Post> findPostsWithPagination(Pageable pageable);
For SpringBoot v2.6.6, accepted answer isn't working if you need to use pageable with child's side field especially when using #ManyToOne.
For the accepted answer:
You can return new object with static query method, which have to include order by count(b.id)
And also order by bCount isn't working.
Please use #AlanHay solution, it is working, but you can't use primitive field and change foreign key constraint. For instance, change long with Long. Because:
When saving a new entity Hibernate does think a record has to be written to the secondary table with a value of zero. (if you use primitive type)
Otherwise you will get an exception:
Caused by: org.postgresql.util.PSQLException: ERROR: cannot insert into view "....view"
Here is the example:
#Entity
#Table(name = "...")
#SecondaryTable(name = "a_summary_view,
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id",
referencedColumnName= "id")},
foreignKey = #javax.persistence.ForeignKey(name = "none"))
public class UserEntity {
#Id
private String id;
#NotEmpty
private String password;
#Column(table = "a_summary_view",
name = "b_count")
private Integer bCount;
}