We Keep Coding

Spring native query with GROUP BY and ORDER BY also pagination

What if I have my native query (with PostgreSQL) in a more "complex" way, which I need to GROUP the data in order to apply an ORDER BY clause? Well, doing so with native query (as explained here) the ORDER BY attributes are placed at the end of the query, which raises a SQL syntax error (of course) cause the ORDER BY should stay BEFORE my pagination clauses.
#Query(
" SELECT * FROM (" +
"SELECT CAST(t.id AS VARCHAR) AS id, t.name, CAST(COUNT(p.id) AS SMALLINT) AS usedByProjectsCount " +
"FROM tag t " +
"LEFT JOIN project_tag pt ON pt.tag_id = t.id " +
"LEFT JOIN project p ON p.id = pt.project_id " +
"GROUP BY t.id, t.name) AS t " +
"ORDER BY #pageable " +
"LIMIT :limit OFFSET :offset ",
countQuery = " SELECT COUNT(*) FROM (" +
"SELECT CAST(t.id AS VARCHAR) AS id, t.name, CAST(COUNT(p.id) AS SMALLINT) AS usedByProjectsCount " +
"FROM tag t " +
"LEFT JOIN project_tag pt ON pt.tag_id = t.id " +
"LEFT JOIN project p ON p.id = pt.project_id " +
"GROUP BY t.id, t.name) AS t ",
nativeQuery = true
)
I already tried putting #pageable at the end, but I get the same error message.
Moving on, Spring generates my query like: SELECT * FROM (SELECT CAST(t.id AS VARCHAR) AS id, t.name, CAST(COUNT(p.id) AS SMALLINT) AS usedByProjectsCount FROM tag t LEFT JOIN project_tag pt ON pt.tag_id = t.id LEFT JOIN project p ON p.id = pt.project_id GROUP BY t.id, t.name) AS t ORDER BY {#pageable} LIMIT ? OFFSET ? , t.usedByProjectsCount desc
and I need like: SELECT * FROM (SELECT CAST(t.id AS VARCHAR) AS id, t.name, CAST(COUNT(p.id) AS SMALLINT) AS usedByProjectsCount FROM tag t LEFT JOIN project_tag pt ON pt.tag_id = t.id LEFT JOIN project p ON p.id = pt.project_id GROUP BY t.id, t.name) AS t ORDER BY t.usedByProjectsCount desc LIMIT ? OFFSET ? {#pageable}
I do know the {#pageable} attribute should stay there, just so my page values can be placed in the right variables.
Anyway, any help would be very appreciated.

In the one of my project I have a native query and just put the pageable inside.
#Query(
value = "SELECT * " +
" FROM (SELECT CAST(t.id AS VARCHAR) AS id, " +
" t.name, " +
" CAST(COUNT(p.id) AS SMALLINT) AS usedByProjectsCount " +
" FROM tag t " +
" LEFT JOIN project_tag pt ON pt.tag_id = t.id " +
" LEFT JOIN project p ON p.id = pt.project_id " +
" GROUP BY t.id, t.name) AS t ",
countQuery = " SELECT COUNT(*) " +
" FROM (SELECT 1 " +
" FROM tag t " +
" LEFT JOIN project_tag pt ON pt.tag_id = t.id " +
" LEFT JOIN project p ON p.id = pt.project_id " +
" GROUP BY t.id, t.name) AS t ",
nativeQuery = true)
Page<CustomProjection> customQuery(Pageable pageable)
and just call your method with the needed pageRequest.
Pageable pageRequest = PageRequest.of(0, 10, Sort.of(Order.desc("usedByProjectsCount"))

HQL Hibernate query search check if list contains all elements of another list

I came across a problem that I cannot think of a solution.
So I got this piece of code:
public List<Post> getPosts(List<PostStrategy> allowedStrategyList, Set<Tag> allowedTags, int page, int resultsPerPage) {
return entityManager.createQuery("select post from Post post join post.tags tag where post.postStrategy in (:postStrategyList) and (:allowedTagsSize = 0 or tag in (:allowedTags))", Post.class)
.setParameter("postStrategyList", allowedStrategyList)
.setParameter("allowedTags", allowedTags)
.setParameter("allowedTagsSize", allowedTags.size())
.setFirstResult((page - 1) * resultsPerPage)
.setMaxResults(resultsPerPage)
.getResultList();
}
The problem with this piece of code is that when someone searches with more than one tag (for example: #video, #image), it returns both posts with two tags and one tag.
I would like it to return a post with both #video, and #image in its tags. For it to work, I somehow need to check if the list contains all elements of another list.
I searched for a solution for a while and tried different approaches so far, nothing.
I tried replacing "tag in (:allowedTags)" with "post.tags in (:allowedTags)" but that throws that my SQL is invalid.

Two years later I got my answer. I am now using Spring Boot and SpEL expressions so bear with me. This query does a little more than just my original idea but you can extract what you need from it.
#Override
#Query("select p from PostSnapshot p where " +
// Tags
"(:#{#query.withTags.size()} = 0 or p.id in (" +
"select post.id from PostSnapshot post inner join post.tags tag " +
"where tag.value in (:#{#query.withTags}) " +
"group by post.id " +
"having count(distinct tag.value) = :#{#query.withTags.size() * 1L}" +
")) and (:#{#query.withoutTags.size()} = 0 or p.id not in (" +
"select post.id from PostSnapshot post inner join post.tags tag " +
"where tag.value in (:#{#query.withoutTags}) " +
"group by post.id " +
"having count(distinct tag.value) = :#{#query.withoutTags.size() * 1L}" +
")) " +
// Artists
"and (:#{#query.withArtists.size()} = 0 or p.id in (" +
"select post.id from PostSnapshot post inner join post.artists artist " +
"where artist.preferredNickname in (:#{#query.withArtists}) " +
"group by post.id " +
"having count(distinct artist.preferredNickname) = :#{#query.withArtists.size() * 1L}" +
")) and (:#{#query.withoutArtists.size()} = 0 or p.id not in (" +
"select post.id from PostSnapshot post inner join post.artists artist " +
"where artist.preferredNickname in (:#{#query.withoutArtists}) " +
"group by post.id " +
"having count(distinct artist.preferredNickname) = :#{#query.withoutArtists.size() * 1L}" +
")) " +
// Title like words
"and lower(p.title) like concat('%', lower(:#{(#query.words.isEmpty()) ? '' : #query.words.toArray()[0]}) ,'%')" +
// OwnerId
"and p.ownerId = :ownerId")
Page<PostSnapshot> findAllByOwnerIdAndQuery(#Param("ownerId") UUID ownerId, #Param("query") PostQuerySearchDTO query, Pageable pageable);

Bad SQL grammar -The column name was not found in this ResultSet?

Hello I am using a JDBC query and I am having this error but I don't understand why, or what exactly is the meaning of this error and what is wrong with the query.
The query is as below
private List<StatisticsLog> runStatistics(Integer partnerId, LocalDate ldStart, LocalDateTime ldtEnd, String statType)
{
return jdbcTemplate.query("SELECT S.APP_ID, S.LOG_TYPE, ACC.CONTACT_PERSON_FIRST_NAME, ACC.CONTACT_PERSON_LAST_NAME, ACV.VERSION_APP_NAME, "
+ "'" + statType.trim() + "' STATTYPE, "
+ "ACV.APP_CATALOG_ID, ACV.APP_CATALOG_VERSION_ID, ACV.VERSION, ACV.UPDATED_AT, ACV.VERSION_LOGO_EXT, ACV.HAS_LOGO, "
+ "LANG.LANGUAGE_NAME_SHORT, LANG.LANGUAGE_ID , S.count, S.PARTNER_ID, APP.CREATED_AT "
+ "FROM (SELECT partner_id, log_type, app_id, language_id, count(*) as count FROM public.statistics_log "
+ " WHERE partner_id = ? "
+ " and logged_at between ? and ? "
+ "group by 1, log_type, app_id, language_id) as S "
+ "INNER JOIN APP_CATALOG_ACCOUNT ACP ON ACP.APP_CATALOG_SERIAL_ID = S.APP_ID "
+ "INNER JOIN APP_CATALOG APP ON APP.APP_CATALOG_SERIAL_ID = S.APP_ID "
+ "INNER JOIN ACCOUNT ACC ON ACC.ACCOUNT_ID = S.PARTNER_ID "
+ "INNER JOIN APP_CATALOG_VERSION ACV on ACV.APP_CATALOG_SERIAL_ID = APP.APP_CATALOG_SERIAL_ID AND ACV.STATUS = 3 "
+ "INNER JOIN LANGUAGE LANG ON LANG.LANGUAGE_ID = ACV.LANGUAGE_ID "
+ "ORDER BY S.count desc ",
new Object[] { partnerId, ldStart, ldtEnd }, new StatisticsLogRowMapper());
}
and I am getting the error from this column CREATED_AT that is part of the app catalog, and I think it should be there, also for more detailed also I will attach StatisticsLogRowMapper
public class StatisticsLogRowMapper implements RowMapper<StatisticsLog>
{
#Override
public StatisticsLog mapRow(ResultSet rs, int rowNum) throws SQLException
{
StatisticsLog stat = new StatisticsLog();
stat.setAppId(rs.getInt("APP_ID"));
stat.setPartnerId(rs.getInt("PARTNER_ID"));
stat.setCount(rs.getLong("COUNT"));
stat.setCreatedAt(rs.getTimestamp("CREATED_AT").toLocalDateTime());
stat.setPartnerFirstName(rs.getString("CONTACT_PERSON_FIRST_NAME"));
stat.setPartnerLastName(rs.getString("CONTACT_PERSON_LAST_NAME"));
stat.setAppNameDefault(rs.getString("VERSION_APP_NAME"));
stat.setAppCatalogId(rs.getInt("APP_CATALOG_ID"));
stat.setStatType(rs.getString("STATTYPE"));
stat.setLogType(LogTypeEnum.valueOf(rs.getString("LOG_TYPE").trim()));
stat.setAppCatalogVersionId(rs.getInt("APP_CATALOG_VERSION_ID"));
stat.setVersion(rs.getInt("VERSION"));
stat.setUpdatedAt(rs.getDate("UPDATED_AT"));
stat.setVersionLogoExt(rs.getString("VERSION_LOGO_EXT"));
stat.setHasLogo(rs.getBoolean("HAS_LOGO"));
stat.setLanguageNameShort(rs.getString("LANGUAGE_NAME_SHORT"));
stat.setLanguageId(rs.getInt("LANGUAGE_ID"));
return stat;
}
}
THE ERROR IS: "PreparedStatementCallback; bad SQL grammar [SELECT S.APP_ID, S.LOG_TYPE, ACC.CONTACT_PERSON_FIRST_NAME, ACC.CONTACT_PERSON_LAST_NAME, ACV.VERSION_APP_NAME, 'days30' STATTYPE, ACV.APP_CATALOG_ID, ACV.APP_CATALOG_VERSION_ID, ACV.VERSION, ACV.UPDATED_AT, ACV.VERSION_LOGO_EXT, ACV.HAS_LOGO, LANG.LANGUAGE_NAME_SHORT, S.count, S.PARTNER_ID FROM (SELECT partner_id, log_type, app_id, language_id, count(*) as count FROM public.statistics_log WHERE logged_at between ? and ? group by 1, log_type, app_id, language_id, partner_id) as S INNER JOIN APP_CATALOG_ACCOUNT ACP ON ACP.APP_CATALOG_SERIAL_ID = S.APP_ID INNER JOIN APP_CATALOG APP ON APP.APP_CATALOG_SERIAL_ID = S.APP_ID INNER JOIN ACCOUNT ACC ON ACC.ACCOUNT_ID = S.PARTNER_ID INNER JOIN APP_CATALOG_VERSION ACV on ACV.APP_CATALOG_SERIAL_ID = APP.APP_CATALOG_SERIAL_ID AND ACV.STATUS = 3 INNER JOIN LANGUAGE LANG ON LANG.LANGUAGE_ID = ACV.LANGUAGE_ID ORDER BY S.count desc ]; nested exception is org.postgresql.util.PSQLException: The column name CREATED_AT was not found in this ResultSet."
IN THE ERROR SEEMS THIS FIELD ISN'T THERE SOMEHOW
If you have any idea what may be wrong or thing I can try please don't hesitate to comment, I would appreciate d even the smallest help thank you.

I can see there is an error in line 2 of the query - + "'" + statType.trim() + "' STATTYPE, ", right after the quote. STATTYPE does not have a table reference and is appended with a string. That could be another issue. I cannot comment on the question as I dont have enough reputation.

JPA: ORDER BY before GROUP BY in JPARepository

This is the mysql query that I've:
SELECT * FROM (SELECT vev.* FROM vital_entry ve
INNER JOIN vital_entry_values vev ON ve.vital_entry_id=vev.vital_entry_id
WHERE ve.account_id=146
ORDER BY date_entered DESC) t1
GROUP BY vital_id ;
How do I do this in JPA?? I tried:
#Query("SELECT t1 FROM (SELECT ve FROM VitalEntry ve " +
"INNER JOIN ve.vitalEntryValues vev " +
"WHERE ve.accountId=:accountId " +
"ORDER BY ve.dateEntered DESC) t1")
List<VitalEntry> getRecentVitalValues(#Param("accountId") int accountId);
But IntelIj shows error expected identifer, got '('

This gave me the expected result
#Query("SELECT new com.v2.model.VitalValue(vev.vitalId, vev.value, max(ve.dateEntered), ve.vitalEntryType, vev.type) FROM VitalEntry ve " +
"INNER JOIN ve.vitalEntryValues vev " +
"WHERE ve.accountId=:accountId " +
"GROUP BY vev.vitalId " +
"ORDER BY max(ve.dateEntered) DESC")
List<VitalValue> getRecentVitalValues(#Param("accountId") int accountId);

Use native query instead:
#Query("SELECT * FROM (SELECT ve.* FROM vital_entry ve INNER JOIN vital_entry_values vev ON ve.vital_entry_id=vev.vital_entry_id WHERE ve.account_id=:accountId ORDER BY date_entered DESC) t1 GROUP BY vital_id", nativeQuery=true)
List<VitalEntry> getRecentVitalValues(#Param("accountId") int accountId);

Named Query to SELECT rows with MAX(column name), DISTINCT by another column

I have a case similar to the one described in this question, I wrote an identical query which works, but when I try to write it as a jpql named query, I'm getting an error.
My query:
#NamedQuery(
name = "findRankingsBetween",
query = "SELECT rt FROM Rankingtable rt " +
"INNER JOIN " +
"(SELECT teamId, MAX(lastupdate) as MaxDateTime " +
"FROM Rankingtable " +
"GROUP BY teamId) grouped " +
"ON rt.teamId = grouped.teamId " +
"AND rt.lastupdate = grouped.MaxDateTime " +
"WHERE rt.lastupdate BETWEEN :from AND :to"
)
Error:
Error in named query: findRankingsBetween: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: ( near line 1, column 79
How to write the query properly in jpql?

As noted in this answer, a subquery in JPQL can only occur in select and where clauses.
Hibernate doc.
An equivalent query in JPQL is:
"SELECT rt FROM Rankingtable rt " +
"WHERE rt.lastupdate = (SELECT MAX(r2.lastupdate) " +
"FROM Rankingtable r2 " +
"WHERE r2.teamid = rt.teamid) " +
"AND rt.lastupdate BETWEEN :from AND :to"