We Keep Coding

Convert postgresql join and group by query to JPA criteria API

I am having trouble to converting the following postgresql query (with a join and a group by) to JPA criteria API for a Spring Boot, JPA, Hibernate application:
select u.id, u.full_name, count(*) project_applications_count from users u
join project_applications pa on pa.created_by = u.id
group by u.id, u.full_name
having count(*) >= 1 and count(*) <= 5
The tables look like this:
create table project_applications (
id serial primary key,
...
city_id integer not null references cities (id),
created_by integer not null references users (id)
);
create table users (
id serial primary key,
...
full_name varchar(100) not null
);
And the entities look like this:
#Entity
#Table(name = "project_applications")
public class ProjectApplication {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "created_by")
private User createdBy;
...
}
#Entity
#Table(name = "users")
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "full_name")
private String fullName;
...
}
I tried searching online for a solution but every exemple I found was using either a join or group by, but not both.

Using #akortex's idea with projections, I think something like this should work:
public class UserSummary {
private Long id;
private String fullName;
private Long count;
public UserSummary() {
}
public UserSummary(Long id, String fullName, Long count) {
this.id = id;
this.fullName = fullName;
this.count = count;
}
... (getters and setters)
}
public List<UserSummary> getSummaries(Integer minProjectAppsCount, Integer maxProjectAppsCount) {
CriteriaBuilder cb = _entityManager.getCriteriaBuilder();
CriteriaQuery<UserSummary> query = cb.createQuery(UserSummary.class);
Root<ProjectApplication> projectApp = query.from(ProjectApplication.class);
Join<ProjectApplication, User> userJoin = projectApp.join("createdBy", JoinType.INNER);
query.multiselect(userJoin.get("id"), userJoin.get("fullName"), cb.count(projectApp))
.groupBy(userJoin.get("id"), userJoin.get("fullName"));
List<Predicate> predicates = new ArrayList<>();
if (minProjectAppsCount != null ) {
Predicate p = cb.ge(cb.count(projectApp), minProjectAppsCount);
predicates.add(p);
}
if (maxProjectAppsCount != null ) {
Predicate p = cb.le(cb.count(projectApp), maxProjectAppsCount);
predicates.add(p);
}
query.having(predicates.toArray(new Predicate[0]));
return _entityManager.createQuery(query).getResultList();
}

You could potentially look into projections in order to achieve what you want.
For example consider the following projection and repository:
#Data
#AllArgsConstructor
public class ProjectApplicationSummary {
private Long id;
private String fullName;
private Long count;
}
And:
#Repository
public interface ProjectApplicationRepository extends JpaRepository<ProjectApplication, Long> {
#Query(
"""
SELECT new com.example.springdemo.entities.ProjectApplicationSummary(u.id, u.fullName, count(pa))
FROM User u, ProjectApplication pa
GROUP BY u.id, u.fullName
"""
)
List<ProjectApplicationSummary> getSummaries();
}
You will most likely need to tweak the query a bit (which revolves experimenting with JPQL) but other than that, the basic idea is there.

I'm not sure in my solution, but it should be similar. I took an idea from here. Maybe it helps you to resolve your problem.
public static Specification<User> getUsers() {
return Specification.where((root, query, criteriaBuilder) -> {
CriteriaQuery<User> criteriaQuery = criteriaBuilder.createQuery(User.class);
Subquery<Long> subQuery = criteriaQuery.subquery(Long.class);
Root<ProjectApplication> subRoot = subQuery.from(ProjectApplication.class);
subQuery
.select(criteriaBuilder.count(subRoot))
.where(criteriaBuilder.equal(root.get("id"), subRoot.get("createdBy").get("id")));
query
.multiselect(criteriaBuilder.construct(root.get("id"), root.get("fullName")))
.groupBy(root.get("id"), root.get("fullName"))
.having(criteriaBuilder.and(
criteriaBuilder.greaterThanOrEqualTo(subQuery.getSelection(), 1L),
criteriaBuilder.lessThanOrEqualTo(subQuery.getSelection(), 5L)));
return query.getRestriction();
});
}

JPA Criteria for MAX id GROUP BY FOREIGN KEY in a ManyToMany relationship

I'm developing an application using Spring Boot and Spring Data JPA.
The following are the relevant entities:
#Entity
public class Certification {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "certification_type_id")
private CertificationType certificationType;
#OneToMany(fetch = FetchType.EAGER, mappedBy = "certification", cascade = CascadeType.ALL)
private Set<CertificationStatus> certificationStatuses;
}
#Entity
public class CertificationType {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Integer id;
#Enumerated(EnumType.STRING)
private Code code;
private String name;
private String description;
...
public enum Code {
...
}
}
#Entity
public class CertificationStatus {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "certification_id")
private Certification certification;
#ManyToOne
#JoinColumn(name = "certification_status_type_id")
private CertificationStatusType certificationStatusType;
private String remarks;
}
#Entity
public class CertificationStatusType {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Integer id;
#Enumerated(EnumType.STRING)
private Code code;
private String name;
private String description;
#OneToMany(fetch = FetchType.EAGER, mappedBy = "certificationStatusType", cascade = CascadeType.ALL)
private Set<CertificationStatus> certificationStatuses;
public enum Code {
...
}
}
In my service, I have a method, CertificationService.findAll(). The idea is to retrieve certification records whose latest certification_status is based on MAX(id), GROUP BY certification_id and WHERE certification_status_type_id = ?. This method has five (5) parameters:
Integer certificationTypeId,
Integer certificationStatusTypeId,
LocalDate dateFrom,
LocalDate dateTo,
String client
The combination could be zero (0) to five (5). Thus, I chose to apply JPA Criteria. So far, I have coded the following:
#Override
public List<CertificationDto> findAll(
Integer certificationTypeId,
Integer certificationStatusTypeId,
LocalDate dateFrom,
LocalDate dateTo,
String client) {
var certifications = this.certificationRepository.findAll((root, criteriaQuery, criteriaBuilder) -> {
var predicates = new ArrayList<Predicate>();
// This is working
if (certificationTypeId != null && certificationTypeId > 0) {
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.join(Certification_.certificationType, JoinType.INNER), certificationTypeId)));
}
// To do
if (certificationStatusTypeId != null && certificationStatusTypeId > 0) {
/*
* I have no idea how to apply my SQL statement:
* SELECT * FROM certification_status WHERE id IN (SELECT MAX(id) FROM certification_status GROUP BY certification_id) and certification_status_type_id = ?;
*/
// Not sure what is the direction of the ff codes
var certificationStatuses = root.join(Certification_.certificationStatuses);
predicates.add(criteriaBuilder.and(criteriaBuilder.max(certificationStatuses.get(CertificationStatus_.id))));
// It says Cannot resolve method 'and(javax.persistence.criteria.Expression<N>)'
// Not also sure how to pass in parameter certificationStatusTypeId
}
// This is working
if (dateFrom != null) {
var offsetDateTimeFrom = OffsetDateTime.of(dateFrom, LocalTime.MIDNIGHT, ZoneOffset.ofHours(8));
var dateTimeFrom = Date.from(offsetDateTimeFrom.toInstant());
var offsetDateTimeTo = OffsetDateTime.of(Objects.requireNonNullElse(dateTo, dateFrom), LocalTime.MAX, ZoneOffset.ofHours(8));
var dateTimeTo = Date.from(offsetDateTimeTo.toInstant());
predicates.add(criteriaBuilder.and(criteriaBuilder.between(
root.get(Certification_.createdAt),
criteriaBuilder.literal(dateTimeFrom),
criteriaBuilder.literal(dateTimeTo)))
);
}
// This is working
if (client != null && !client.isBlank()) {
predicates.add(criteriaBuilder.and(criteriaBuilder.like(criteriaBuilder.lower(root.join(Certification_.client).get(Client_.name)), "%" + client.toLowerCase() + "%")));
}
return criteriaBuilder.and(predicates.toArray(new Predicate[0]));
});
return certifications.stream()
.map(this.certificationMapper::fromEntityToDto)
.collect(Collectors.toUnmodifiableList());
}
My criteria for certificationTypeId, dateFrom, dateTo, and client are working. For the certificationStatusTypeId, I want to create criteria based on the following SQL statement:
SELECT * FROM certification_status WHERE id IN (SELECT MAX(id) FROM certification_status GROUP BY certification_id) and certification_status_type_id = ?;
This statement is working in the database. Not sure though if this is the best composition of query to select the record based on MAX(id), GROUP BY certification_id and WHERE certification_status_type_id = ?.
Appreciate any help. Thank you.
**Update
This is now my code:
if (certificationStatusTypeId != null && certificationStatusTypeId > 0) {
Subquery<Long> subQuery = criteriaQuery.subquery(Long.class);
Root<CertificationStatus> subRoot = subQuery.from(CertificationStatus.class);
subQuery.select(criteriaBuilder.max(subRoot.get(CertificationStatus_.id)))
.groupBy(subRoot.get(CertificationStatus_.certification).get(Certification_.id));
predicates.add(criteriaBuilder.and(criteriaBuilder.in(subRoot.get(CertificationStatus_.id)).value(subQuery)));
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(subRoot.get(CertificationStatus_.certificationStatusType).get(CertificationStatusType_.id), certificationStatusTypeId)));
}
But it throws the following error:
org.hibernate.hql.internal.ast.QuerySyntaxException: Invalid path: 'generatedAlias1.id' [select generatedAlias0 from entity.Certification as generatedAlias0 where ( generatedAlias1.id in (select max(generatedAlias1.id) from entity.CertificationStatus as generatedAlias1 group by generatedAlias1.certification.id) ) and ( generatedAlias1.certificationStatusType.id=2 )]; nested exception is java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: Invalid path: 'generatedAlias1.id' [select generatedAlias0 from entity.Certification as generatedAlias0 where ( generatedAlias1.id in (select max(generatedAlias1.id) from entity.CertificationStatus as generatedAlias1 group by generatedAlias1.certification.id) ) and ( generatedAlias1.certificationStatusType.id=2 )]
Looking at the generated JPQL, it seems it looks similar to my SQL statement. I just don't understand why it's Invalid path.
**Update
My apologies! My SQL statement is actually similar to the following:
select
// props of Certification
from certification c
inner join certification_type ct on ct.id = c.certification_type_id
inner join client cl on cl.certification_id = c.id
inner join certification_status cs on cs.certification_id = c.id
inner join certification_status_type cst on cst.id = cs.certification_status_type_id
where cs.id in (select max(cs2.id) form certification_status cs2 group by cs2.certification_id) and cs.certification_status_type_id = ?;
Update
This is solved. Thanks to Thorben Janssen for being patient looking into my issue.
This is the final portion of my codes:
if (certificationStatusTypeId != null && certificationStatusTypeId > 0) {
var certificationStatuses = root.join(Certification_.certificationStatuses);
Subquery<Long> subQuery = criteriaQuery.subquery(Long.class);
Root<CertificationStatus> subRoot = subQuery.from(CertificationStatus.class);
subQuery.select(criteriaBuilder.max(subRoot.get(CertificationStatus_.id)))
.groupBy(subRoot.get(CertificationStatus_.certification).get(Certification_.id));
predicates.add(criteriaBuilder.and(certificationStatuses.get(CertificationStatus_.id).in(subQuery)));
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(certificationStatuses.get(CertificationStatus_.certificationStatusType).get(CertificationStatusType_.id), certificationStatusTypeId)));
}

The following code snippet creates a CriteriaQuery that's identical to your SQL statement. Because you're using Spring Data JPA, you can ignore the first block and the last line. Spring Data JPA provides them for you.
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<CertificationStatus> criteriaQuery = cb.createQuery(CertificationStatus.class);
Root<CertificationStatus> root = criteriaQuery.from(CertificationStatus.class);
criteriaQuery.select(root);
Subquery<Long> sub = criteriaQuery.subquery(Long.class);
Root<CertificationStatus> subRoot = sub.from(CertificationStatus.class);
sub.select(cb.max(subRoot.get("id")));
sub.groupBy(subRoot.get("certification").get("id"));
criteriaQuery.where(root.get("id").in(sub));
em.createQuery(criteriaQuery).getResultList();
You can create a subquery by calling the subquery method on your CriteriaQuery. This returns a Subquery interface which you can use in the same way as the CriteriaQuery interface.
After you defined your Subquery, you can define the IN clause and include it in the WHERE clause of your query. This is the part that always looks a little strange. Instead of using the CriteriaBuilder to define the Expression, you get the entity attribute and call the in method on it with a reference to your Subquery.
In the final step, you use the CriteriaQuery to instantiate a TypedQuery and execute it. After you activated the logging of SQL statements, you can see that Hibernate executes the following SQL statement:
select
certificat0_.id as id1_4_,
certificat0_.certification_id as certific3_4_,
certificat0_.certification_status_type_id as certific4_4_,
certificat0_.remarks as remarks2_4_
from
CertificationStatus certificat0_
where
certificat0_.id in (
select
max(certificat1_.id)
from
CertificationStatus certificat1_
group by
certificat1_.certification_id
)
**Update
It looks like you're referencing the wrong table when defining the IN clause. Please try to replace
predicates.add(criteriaBuilder.and(criteriaBuilder.in(subRoot.get(CertificationStatus_.id)).value(subQuery)));
with
predicates.add(criteriaBuilder.and(certificationStatuses.get(CertificationStatus_.id).in(subQuery)));

how to select multiple columns in subquery(hql)

Is there a way to select multiple columns in a subquery of hql?
I want to convert the following mysql query to hql
select u.name, sub.cnt from user u
inner join (select user_id, count(user_id) cnt from user_log group by user_id order by cnt desc limit 5) sub
on u.id = sub.user_id;
But in hql, I know that can use subquery only in where clause

I'm not aware of a pure Hibernate solution. I talk about that in my Blog post.
But there is a FluentJPA solution that you may consider:
FluentQuery query = FluentJPA.SQL((User u) -> {
UserIdCount sub = subQuery((UserLog log) -> {
int count = alias(COUNT(log.getUserId()), UserIdCount::getCount);
SELECT(log.getUserId(), count);
FROM(log);
ORDER(BY(count).DESC());
LIMIT(5);
});
SELECT(u.getName(), sub.getCount());
FROM(u).JOIN(sub).ON(u.getId() == sub.getUserId());
});
return query.createQuery(em, UserNameCount.class).getSingleResult();
which produces the following SQL:
SELECT t0.name, q0.count
FROM USER t0 INNER JOIN (SELECT t1.user_id, COUNT(t1.user_id) AS count
FROM USER_LOG t1
ORDER BY COUNT(t1.user_id) DESC
LIMIT 5 ) q0 ON (t0.id = q0.user_id)
Entities I used (declared with lombok):
#Entity
#Getter
#Table(name = "USER")
class User {
private Long id;
private String name;
}
#Entity
#Getter
#Table(name = "USER_LOG")
class UserLog {
private Long userId;
}
#Tuple
#Getter
class UserIdCount {
private Long userId;
private int count;
}
#Tuple
#Data
class UserNameCount {
private int count;
private String name;
}

JPA2 Criteria Query to select records in Table A whose reference is not found in Table B

I have the following tables with the following structure
Table A {
id <-- Primary key
someColumn
}
Table B {
id <-- Primary key
someColumn
idOfA <-- Foreign key mapping to Table A
}
Entity classes look like below
#Entity
#Table(name = "A")
public class A implements Serializable {
private static final long serialVersionUID = -78448557049178402L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
.......
.......
#OneToMany(mappedBy = "a")
private List<B> bs = new ArrayList<>();
}
#Entity
#Table(name = "B")
public class B implements Serializable {
private static final long serialVersionUID = -659500557015441771L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
.......
.......
#OneToOne
#JoinColumn(name = "a_id", nullable = false)
private A a;
}
Using JPA2, I want to select records from table A which do not have a reference in Table B.
The Expected native postgres query is
select * from A a
where a.id not in
(select b.idOfA from B b);
What I have so far managed to do is
public List<A> getANotInB() {
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
// Select From Table B
CriteriaQuery<B> criteriaQueryB = criteriaBuilder
.createQuery(B.class);
Root<B> rootB = criteriaQueryB.from(B.class);
criteriaQueryB.select(rootB);
// Select records from Table A
CriteriaQuery<A> criteriaQueryA = criteriaBuilder.createQuery(A.class);
Root<A> rootA = criteriaQueryA.from(A.class);
criteriaQueryA.select(A);
// Create predicate
Predicate predicate = rootAttemptA.in(criteriaQueryB.getSelection());
criteriaQueryA.where(criteriaBuilder.not(predicate));
// Create query
TypedQuery<A> query = entityManager.createQuery(criteriaQueryA);
List<A> as= query.getResultList();
System.out.println(as);
return as;
}
I know the code above is incorrect and I have got a lot of basics wong.
Kindly help
Note: I Want to use JPA2 Criteria Query

Try this
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
// Select distinct aid from B
CriteriaQuery<B> bQuery = cb.createQuery(B.class);
Root<B> bRoot = bQuery.from(B.class);
bQuery.select(bRoot.get("a").get("id")).distinct(true);
// Select * from A where aid not in ()
CriteriaQuery<A> aQuery = cb.createQuery(A.class);
Root<A> aRoot = aQuery.from(A.class);
aQuery.select(aRoot).where(cb.not(aRoot.get("id").in(bQuery)));
TypedQuery<A> query = entityManager.createQuery(aQuery);
List<A> result = query.getResultList();
Basically, you will construct part of the query and glue them together.
More information here:
JPA Criteria

I was able to get it done using subquery() as below. Posting it so that it can help others
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
// select a from A a
CriteriaQuery<A> queryA = criteriaBuilder.createQuery(A.class);
Root<A> rootA = queryA.from(A.class);
queryA.select(rootA);
// Select distinct aId from B
CriteriaQuery<B> subQueryB = queryA.subquery(B.class);
Root<B> rootB = subQueryB.from(B.class);
bQuery.select(rootB.get("a")).distinct(true);
queryA.where(criteriaBuilder.not(criteriaBuilder.in(rootA.get("id").value(subQueryB))));
TypedQuery<A> query = entityManager.createQuery(aQuery);
List<A> result = query.getResultList();
Thanks #Mạnh for showing the way

ANY clause on a subquery using jpa criteria API

I have two entities Document and Property where a document has a set of properties:
#Entity
public class Document{
#Id
private Integer id;
#OneToMany
private Set<Property> properties;
}
And
#Entity
public class Property{
#Id
private Integer id;
private String key;
private String value;
}
I want to implement the following JPQL query using Criteria API:
SELECT d FROM Document d
WHERE "value11" = ANY(SELECT p.value FROM d.properties p WHERE p.key="key11")
I tried the following:
EntityManager em = PersistenceManager.INSTANCE.getEntityManager();
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Document> query = criteriaBuilder.createQuery(Document.class);
Root<Document> fromDoc = query.from(Document.class);
Subquery<String> subQuery = query.subquery(String.class);
Root<Property> subRoot = subQuery.from(Property.class);
subQuery.select(subRoot.get(Property_.value));
SetJoin<Document, Property> join = fromDoc.join(Document_.properties, JoinType.INNER);
subQuery.select(join.<String> get("value"));
subQuery.where(criteriaBuilder.equal(join.<String> get("key"), "key11"));
query.where(criteriaBuilder.equal(criteriaBuilder.any(subQuery), "value11"));
Query q = em.createQuery(query);
List<Document> docs = q.getResultList();
PersistenceManager.INSTANCE.close();
But I got this exception:
Exception Description: The query has not been defined correctly, the
expression builder is missing. For sub and parallel queries ensure
the queries builder is always on the left. Query:
ReadAllQuery(referenceClass=Document ) at
org.eclipse.persistence.exceptions.QueryException.invalidBuilderInQuery(QueryException.java:689)
at
org.eclipse.persistence.internal.expressions.SQLSelectStatement.appendFromClauseToWriter(SQLSelectStatement.java:537)
at
org.eclipse.persistence.internal.expressions.SQLSelectStatement.printSQL(SQLSelectStatement.java:1704)
Any help would be appreciated!