We Keep Coding

Query using specifications gives error when ordering on joined column that is not in SELECT DISTINCT

I get following error when doing a rather complicated query: for SELECT DISTINCT, ORDER BY expressions must appear in select list
In the query I need to find all distinct Requests that have an ExploitationSite that contains a search term in their dutch or french name. The result has to be ordered by the Activity's dutch name and limited to the first 10 for pagination.
To do this query I use the Page <T> findAll(Specification<T> spec, Pageable pageable) method of JpaSpecificationExecutor.
This will result in a SELECT DISTINCT query which has to be ORDERed BY a property that is not in SELECT. (details below)
I tried to fetch the activities eagerly in the hope it would place those differently in the SELECT. I did my best trying to get the DISTINCT in a subquery and then have the ORDER BY + LIMIT around that, but I did not succeed in that.
Has someone an idea how I can get this query to work?
The (simplified) Request entity
#Entity
#Table(name = "request_requests")
#History("Request")
public class Request extends EqualByStateObject {
#GeneratedValue
#Id
private int id;
#Embedded
private RequestNumber requestNumber;
#OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
#JoinColumn(name = "fk_request")
private List<ExploitationSite> exploitationSites = new ArrayList<>();
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(unique = true, name = "fk_activity")
private Activity activity;
...
}
The Specification (I have to use distinct here because since a Request contains a List of ExploitationSites it was possible I got the same request multiple times back if multiple ExploitationSites contained the search term)
public class ExploitationSiteSpecification extends EqualByStateObject implements Specification<Request> {
private final String exploitationSiteName;
protected ExploitationSiteSpecification(String exploitationSiteName) {
this.exploitationSiteName = exploitationSiteName;
}
#Override
public Predicate toPredicate(Root<Request> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {
query.distinct(true);
ListJoin<Object, Object> exploitationSites = root.joinList("exploitationSites");
return criteriaBuilder.or(
criteriaBuilder.like(
criteriaBuilder.lower(exploitationSites.get("name").get("dutchName")), "%" + exploitationSiteName.toLowerCase() + "%"),
criteriaBuilder.like(
criteriaBuilder.lower(exploitationSites.get("name").get("frenchName")), "%" + exploitationSiteName.toLowerCase() + "%")
);
}
}
The Pageable
public Pageable getPageable() {
Sort sort = Sort.by(Sort.Order.asc("activity.name.dutchName"));
PageRequest.of(0, 10, sort);
}
This results in a generated query like this one
select distinct request0_.id as id1_23_,
request0_.fk_activity as fk_acti15_23_,
request0_.request_number as request12_23_
from request_requests request0_
inner join request_exploitation_sites exploitati1_ on request0_.id=exploitati1_.fk_request
left outer join request_activity activity2_ on request0_.fk_activity=activity2_.id
where lower(exploitati1_.dutch_name) like $1
or lower(exploitati1_.french_name) like $2
order by activity2_.dutch_name asc limit $3
which then gives the for SELECT DISTINCT, ORDER BY expressions must appear in select list error

Assuming you put the distinct because the join with exploitationSites would return multiple rows, the following two options would work without using distinct.
right after the join you could do an additional fetch
ListJoin<Object, Object> exploitationSites = root.joinList("exploitationSites");
root.fetch("exploitationSites")
this would result in hibernate to create an additional join of ExploitationSites as well as selecting additional columns
select request0_.id as id1_23_,
request0_.fk_activity as fk_acti15_23_,
request0_.request_number as request12_23_,
exploitati3_.id as exploitati3_id,
exploitati3_.name as exploitati3_name,
...
from request_requests request0_
inner join request_exploitation_sites exploitati1_ on request0_.id=exploitati1_.fk_request
left outer join request_activity activity2_ on request0_.fk_activity=activity2_.id
inner join request_exploitation_sites exploitati3_ on request0_.id=exploitati3_.fk_request
where lower(exploitati1_.dutch_name) like $1
or lower(exploitati1_.french_name) like $2
order by activity2_.dutch_name asc limit $3
use fetch in the first place and cast it to Join
Join<Object, Object> exploitationSites = (Join<Object, Object>) root.fetch("exploitationSites");
By casting the Fetch to a Join you can still use where clauses.
Note that this will also select additional columns, but won't do an additional join in the resulting query.
In both cases the fetch will result in a join fetch which hibernate internally will remove duplicates from the parent entity (see https://stackoverflow.com/a/51177569)

HQL: How to query certain fields when one of those fields is a one-to-many List?

I am having difficulty writing a HQL query to select ONLY the caseid, title, and caseStatus fields from my Cases entity. The cases returned have to be distinct based on caseid. I do not want the name and userid fields to be included. I also do not want to use Lazy fetching for caseid, title, and caseStatus fields. Note that the caseStatus field is a one-to-many List. Below are the entities. The getters/setters are omitted to save space.
#Entity
#Table(name = "Cases")
public class Cases {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "caseid", nullable = false)
private Integer caseid;
private Integer userid;
private String name;
private String title;
#OrderBy("caseStatusId DESC")
#OneToMany(mappedBy = "cases", fetch = FetchType.EAGER)
private List<CaseStatus> caseStatus;
}
#Entity
#Table(name = "CaseStatus")
public class CaseStatus {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "caseStatusId", nullable = false)
private Integer caseStatusId;
private String info;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "caseid")
private Cases cases;
}
My goal is to retrieve a distinct List<Cases> or List<Object[]> of the Cases entity containing only caseid, title, and a List<CaseStatus>. The List<CaseStatus> will contain CaseStatus objects with all of its fields populated.
public List<Object[]> getCases(String title) {
TypedQuery<Object[]> q = em.createQuery("select distinct c.caseid, c.title, cs "
+ "FROM Cases c join c.caseStatus cs "
+ "where c.title like :title", Object[].class);
q.setParameter("title", "%" + title + "%");
List<Object[]> results = q.getResultList();
return results;
}
The above method is close, but not correct because rather than returning a List<CaseStatus> in one of the indexes, it is only returning a single CaseStatus entity.
For example, if my DB contains a single Case with a List<CaseStatus> having a size of n for example, the results will be similar to the example below:
Example of results I'm getting now. Not correct:
List<Object[]> index 0:
Contains an Object[] where:
Object[0] = {some caseid}
Object[1] = {some title}
Object[2] = {1st CaseStatus}
List<Object[]> index 1:
Contains an Object[] where:
Object[0] = {same caseid as the one found in index 0 above}
Object[1] = {same title as the one found in index 0 above}
Object[2] = {2nd CaseStatus}
...
List<Object[]> index n-1:
Contains an Object[] where:
Object[0] = {same caseid as all the previous}
Object[1] = {same title as all the previous}
Object[2] = {nth CaseStatus}
Example of results I hope to achieve:
List<Object[]> index 0:
Contains an Object[] where:
Object[0] = {unique caseid}
Object[1] = {some title}
Object[2] = List<CaseStatus> with size of n
Updated the question. Instead of name, title, and List<CaseStatus>, the fields I want to retrieve are caseid, title, and List<CaseStatus>. caseid is the primary key of Cases.
I found various threads Select Collections with HQL - hibernate forum and Select collections with HQL - stackoverflow. It's pretty much the problem I ran into. Looks like no one found a solution in these threads.

Hibernates a bit confused about the query; in HQL do your join like this (apologies, I've not been able to test before posting due to wonky computer, but you should get the idea)
select distinct c from Cases c left join fetch c.caseStatus cs where....
the "fetch" makes it eager. Note that this will return an array of type Cases. You where clauses look about right.

In fact HQL is fully object-oriented and uses your classes structure in the Query, so by writing c.caseStatus HQL expects that your Cases class has a caseStatus property, which is wrong because it's a collection.
If you take a look at Hibernate HQL documentation you can see that:
Compared with SQL, however, HQL is fully object-oriented and understands notions like inheritance, polymorphism and association.
I think what you need to do here is to change your query so it matches your classes structures:
Query q = em.createQuery("select distinct c.name, c.title, cs.caseStatus FROM Cases c left join c.caseStatus where "
+ "c.name like :name and "
+ "c.title like :title");

Correct syntax should be
TypedQuery<Object[]> q = em.createQuery("select c.name, c.title, cs FROM Cases c "
+ "join c.caseStatus cs where "
+ "c.name = :name and "
+ "c.title = :title", Object[].class);
Return type will be List<Object[]>, where in first index of Object[] is c.name, second is c.title and third is associated caseStatus entity. It is possible to query for multiple instances (rows).
We need JOIN because relationship between CaseStatus and Case is mapped via collection.
SELECT cs
FROM Case c JOIN c.cases cs;

Why don't you just use
Query q = em.createQuery("select distinct c from Cases c where "
+ "c.name like :name and "
+ "c.title like :title");
Just try this. This may be a naive approach but should be able to solve the problem. You may be getting more fields than you required but the return type would be list of Cases.

Jpa recursive query

consider the following schema
#Entity
Class employee{
#OneToMany()
List<employee> manaagedEmps;
#OneToOne
employee manager;
}
how to write a query that get all the managed employee for a certain manager , direct(the list of managedEmps) and indirect (managed by managed employee).

It seems that JPA does not support recursive queries. Recently I solved the smilar problem by adding "path" field of type ltree (postgresql). Path is generated by adding id separated by dot to path of parent and path of root nodes is just id. With that field you are able to query subtree (direct and indirect employees) of some node (manager):
SELECT * FROM nodes WHERE path ~ '*.42.*{1,}'; /* for path of type ltree */
SELECT * FROM nodes WHERE path LIKE '%.42.%'; /* for path of type varchar */
The following JPQL query returns flat list of subs for employee with id 2.
List<Employee> subs = em.createQuery(
"SELECT e FROM Employee e LEFT JOIN FETCH e.subs WHERE e.path LIKE '%.' || ?1 || '.%'",
Employee.class
).setParameter(1, '2').getResultList();

//Returns a list of the managed employee of the manager with the specified ID.
#NamedQuery(name="queryName", query="SELECT p.managedEmps FROM employee p WHERE p.manager.uuid = :uuid")

I am using postgresql here.
I did this through native query like this:
Suppose following entity
#Entity
#Table(name = "employee")
public class Employee {
#Id
private Long id;
#ManyToOne
#JoinColumn(name = "parent_id")
private Employee parent;
}
Now, following query can be used to get all childs and sub childs under one manager recursively:
public interface IEmployeeRepository extends JpaRepository<Employee, Long> {
#Query(value = "with recursive subordinates as ("
+ " select e1.id as id, e1.parent_id as parent from employee e1 where e1.parent_id = :parentId"
+ " union"
+ " select e2.id, e2.parent_id from employee e2"
+ " inner join subordinates s on (s.id = e2.parent_id)"
+ " ) select * from subordinates", nativeQuery = true)
Collection<Employee2> getChilds(#Param("parentId") Long parentId);
public static interface Employee2 {
Long getId();
Long getParent();
}
}
Now, you have to convert this result Collection into List in your service layer. That's it.
References:
postgres recursive queries
Jpa Projections to get result
Hope this helps.

I usually prefer to offer some code, but in this case I think the article itself does a better job of explaining.

Order by count using Spring Data JpaRepository

I am using Spring Data JpaRepository and I find it extremely easy to use. I actually need all those features - paging, sorting, filtering. Unfortunately there is one little nasty thing that seems to force me to fall back to use of plain JPA.
I need to order by a size of associated collection. For instance I have:
#Entity
public class A{
#Id
private long id;
#OneToMany
private List<B> bes;
//boilerplate
}
and I have to sort by bes.size()
Is there a way to somehow customize the ordering still taking the advantage of pagination, filtering and other Spring Data great features?

I've solved the puzzle using hints and inspirations from:
Limiting resultset using #Query anotations by Koitoer
How to order by count() in JPA by MicSim
Exhaustive experiments on my own
The first and most important thing I've not been aware of about spring-data is that even using #Query custom methods one can still create paging queries by simply passing the Pageable object as parameter. This is something that could have been explicitely stated by spring-data documentation as it is definitely not obvious though very powerful feature.
Great, now the second problem - how do I actually sort the results by size of associated collection in JPA? I've managed to come to a following JPQL:
select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a
where AwithBCount is a class that the query results are actually mapped to:
public class AwithBCount{
private Long bCount;
private A a;
public AwithBCount(Long bCount, A a){
this.bCount = bCount;
this.a = a;
}
//getters
}
Excited that I can now simply define my repository like the one below
public interface ARepository extends JpaRepository<A, Long> {
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCount(Pageable pageable);
}
I hurried to try my solution out. Perfect - the page is returned but when I tried to sort by bCount I got disappointed. It turned out that since this is a ARepository (not AwithBCount repository) spring-data will try to look for a bCount property in A instead of AwithBCount. So finally I ended up with three custom methods:
public interface ARepository extends JpaRepository<A, Long> {
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCount(Pageable pageable);
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a order by bCount asc",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCountOrderByCountAsc(Pageable pageable);
#Query(
value = "select new package.AwithBCount(count(b.id) as bCount,c) from A a join a.bes b group by a order by bCount desc",
countQuery = "select count(a) from A a"
)
Page<AwithBCount> findAllWithBCountOrderByCountDesc(Pageable pageable);
}
...and some additional conditional logic on service level (which could be probably encapsulated with an abstract repository implementation). So, although not extremely elegant, that made the trick - this way (having more complex entities) I can sort by other properties, do the filtering and pagination.

One option, which is much simpler than the original solution and which also has additional benefits, is to create a database view of aggregate data and link your Entity to this by means of a #SecondaryTable or #OneToOne.
For example:
create view a_summary_view as
select
a_id as id,
count(*) as b_count,
sum(value) as b_total,
max(some_date) as last_b_date
from b
Using #SecondaryTable
#Entity
#Table
#SecondaryTable(name = "a_summary_view",
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id", referencedColumnName= "id")})
public class A{
#Column(table = "a_summary_view")
private Integer bCount;
#Column(table = "a_summary_view")
private BigDecimal bTotal;
#Column(table = "a_summary_view")
private Date lastBDate;
}
You can now then sort, filer, query etc purely with reference to entity A.
As an additional advantage you have within your domain model data that may be expensive to compute in-memory e.g. the total value of all orders for a customer without having to load all orders or revert to a separate query.

Thank you #Alan Hay, this solution worked fine for me. I just had to set the foreignKey attribute of the #SecondaryTable annotation and everything worked fine (otherwise Spring Boot tried to add a foreignkey constraint to the id, which raise an error for a sql View).
Result:
#SecondaryTable(name = "product_view",
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id", referencedColumnName = "id")},
foreignKey = #javax.persistence.ForeignKey(ConstraintMode.NO_CONSTRAINT))

I don't know much about Spring Data but for JPQL, to sort the objects by size of associated collection, you can use the query
Select a from A a order by a.bes.size desc

You can use the name of an attribute found in the select clause as a sort property:
#Query(value = "select a, count(b) as besCount from A a join a.bes b group by a", countQuery = "select count(a) from A a")
Page<Tuple> findAllWithBesCount(Pageable pageable);
You can now sort on property besCount :
findAllWithBesCount(PageRequest.of(1, 10, Sort.Direction.ASC, "besCount"));

I used nativeQuery to arrange sorting by number of records from another table, pagable works.
#Query(value = "SELECT * FROM posts where posts.is_active = 1 and posts.moderation_status = 'ACCEPTED' " +
"group by posts.id order by (SELECT count(post_id) FROM post_comments where post_id = posts.id) desc",
countQuery = "SELECT count(*) FROM posts",
nativeQuery = true)
Page <Post> findPostsWithPagination(Pageable pageable);

For SpringBoot v2.6.6, accepted answer isn't working if you need to use pageable with child's side field especially when using #ManyToOne.
For the accepted answer:
You can return new object with static query method, which have to include order by count(b.id)
And also order by bCount isn't working.
Please use #AlanHay solution, it is working, but you can't use primitive field and change foreign key constraint. For instance, change long with Long. Because:
When saving a new entity Hibernate does think a record has to be written to the secondary table with a value of zero. (if you use primitive type)
Otherwise you will get an exception:
Caused by: org.postgresql.util.PSQLException: ERROR: cannot insert into view "....view"
Here is the example:
#Entity
#Table(name = "...")
#SecondaryTable(name = "a_summary_view,
pkJoinColumns = {#PrimaryKeyJoinColumn(name = "id",
referencedColumnName= "id")},
foreignKey = #javax.persistence.ForeignKey(name = "none"))
public class UserEntity {
#Id
private String id;
#NotEmpty
private String password;
#Column(table = "a_summary_view",
name = "b_count")
private Integer bCount;
}

JPA Query.getResultList() - use in a generic way

I'm creating a complex query with multiple tables and need to list the result. Usually, I'm using the EntityManager and map the result to the JPA-Representation:
UserEntity user = em.find(UserEntity.class, "5");
Then I can access all values as the user UserEntity class defines it. But how can I access the field-values returned from a native, multiple-table query? What I get is a List of Objects. That's fine so far, but what "is" that Object? Array? Map? Collection? ...
//simpleExample
Query query = em.createNativeQuery("SELECT u.name,s.something FROM user u, someTable s WHERE s.user_id = u.id");
List list = query.getResultList();
//do sth. with the list, for example access "something" for every result row.
I guess the answer is quite simple, but most examples out there just show the usage when directly casting to a targetClass.
PS: In the example I could use the class-mappings of course. But in my case someTable is not managed by JPA, and therefore I don't have the entity nor do I have a class-representation of it, and since I'm joining like 20 tables, I don't want to create all the classes just to access the values.

General rule is the following:
If select contains single expression and it's an entity, then result is that entity
If select contains single expression and it's a primitive, then result is that primitive
If select contains multiple expressions, then result is Object[] containing the corresponding primitives/entities
So, in your case list is a List<Object[]>.

Since JPA 2.0 a TypedQuery can be used:
TypedQuery<SimpleEntity> q =
em.createQuery("select t from SimpleEntity t", SimpleEntity.class);
List<SimpleEntity> listOfSimpleEntities = q.getResultList();
for (SimpleEntity entity : listOfSimpleEntities) {
// do something useful with entity;
}

If you need a more convenient way to access the results, it's possible to transform the result of an arbitrarily complex SQL query to a Java class with minimal hassle:
Query query = em.createNativeQuery("select 42 as age, 'Bob' as name from dual",
MyTest.class);
MyTest myTest = (MyTest) query.getResultList().get(0);
assertEquals("Bob", myTest.name);
The class needs to be declared an #Entity, which means you must ensure it has an unique #Id.
#Entity
class MyTest {
#Id String name;
int age;
}

The above query returns the list of Object[]. So if you want to get the u.name and s.something from the list then you need to iterate and cast that values for the corresponding classes.

I had the same problem and a simple solution that I found was:
List<Object[]> results = query.getResultList();
for (Object[] result: results) {
SomeClass something = (SomeClass)result[1];
something.doSomething;
}
I know this is defenitly not the most elegant solution nor is it best practice but it works, at least for me.

Here is the sample on what worked for me. I think that put method is needed in entity class to map sql columns to java class attributes.
//simpleExample
Query query = em.createNativeQuery(
"SELECT u.name,s.something FROM user u, someTable s WHERE s.user_id = u.id",
NameSomething.class);
List list = (List<NameSomething.class>) query.getResultList();
Entity class:
#Entity
public class NameSomething {
#Id
private String name;
private String something;
// getters/setters
/**
* Generic put method to map JPA native Query to this object.
*
* #param column
* #param value
*/
public void put(Object column, Object value) {
if (((String) column).equals("name")) {
setName(String) value);
} else if (((String) column).equals("something")) {
setSomething((String) value);
}
}
}

What if you create a bean with all required properties and cast the result using Java 8+ streams?
Like this:
public class Something {
private String name;
private String something;
// getters and setters
}
And then:
import javax.persistence.Query;
...
Query query = em.createNativeQuery("SELECT u.name,s.something FROM user u, someTable s WHERE s.user_id = u.id", Something.class);
List<?> list = query.getResultList();
return list
.stream()
.map(item -> item instanceof Something ? (Something) item : null)
.collect(Collectors.toList());
That way, you don't need to return List<Object[]> nor hide the warning with #SuppressWarnings("unchecked")
Ps.:
1 - I know that this post is very old. But... I'm here in 2021, so others will be coming here too =)
2 - This is wrong or bad practice? Let me know :D

You can also update your hibernate to a version greater than 5.4.30.final