I am using Spring Boot 2.0.2.RELEASE with Spring Data JPA. I am trying to do implement pagination with native query in MySql, my code is :
#Query(nativeQuery=true, value = "SELECT DISTINCT ud.latitude,ud.longitude,u.user_id userId FROM users u \n" +
"INNER JOIN user_devices ud ON u.id = ud.user_id\n" +
"WHERE ud.access_token IS NOT NULL AND ud.user_id <> 1\n" +
"ORDER BY calculateDistanceByLatLong(:userLat, :userLong, ud.latitude, ud.longitude) ASC \n#pageable\n",
countQuery = "SELECT COUNT(DISTINCT u.id) FROM users u \n" +
"INNER JOIN user_devices ud ON u.id = ud.user_id\n" +
"WHERE ud.access_token IS NOT NULL AND ud.user_id <> 1 \n#pageable\n")
public Page<LocationProjection> listNearByUsers(#Param("userLat")String userLatitude,#Param("userLong") String userLongitude, Pageable pageable) throws Exception;
I got the reference from this link.
And also reviewed this link.
But it's not adding any pagination code.
When I try to use :
userDao.listNearByUsers(userDeviceEntity.getLatitude(),userDeviceEntity.getLongitude(), PageRequest.of(pageNo, maxResults))
And for example set pageNo=0 and maxResults =1,all the results are displayed. So no pagination is being implemented.
I printed the fired query, it is :
SELECT
DISTINCT ud.latitude,
ud.longitude,
u.user_id userId
FROM
users u
INNER JOIN
user_devices ud
ON u.id = ud.user_id
WHERE
ud.access_token IS NOT NULL
AND ud.user_id <> 1
ORDER BY
calculateDistanceByLatLong(?,
?,
ud.latitude,
ud.longitude) ASC #pageable
And the count query as :
SELECT
COUNT(DISTINCT u.id)
FROM
users u
INNER JOIN
user_devices ud
ON u.id = ud.user_id
WHERE
ud.access_token IS NOT NULL
AND ud.user_id <> 1 #pageable
I thought Spring Data would add "LIMIT 0,1" in the main query but it's not working. The DAO interface is :
#Repository
public interface UserDao extends JpaRepository<UserEntity, Integer>{}
Please suggest some solution.
After going over several Q/A here, and none working properly (for my setup, at least). After reading this bit at Spring official documentation, I've managed to get it working by simply removing the "pageable" bit from my query.
I'm working with spring-boot version 2.0.1.RELEASE.
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "SELECT * FROM USERS WHERE LASTNAME = ?1",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
Related
While writing a pagination query in repository as below code segment, I'm getting following error.
Code
#Repository
public interface Aaaa extends PagingAndSortingRepository<TxnDealerInventoryItem, Long> {
#Query(value = "SELECT EM.PART_NO, EM.PART_NAME FROM TXN_DEALER_INVENTORY_ITEM E INNER JOIN MST_PRODUCT EM ON E.PRODUCT_ID = EM.PRODUCT_ID WHERE E.ACCOUNT_ID= :accountId AND EM.ALLOW_SERIAL_NUM = :isSerialized ORDER BY ?#{#pageable}",
countQuery = "SELECT COUNT(*) FROM TXN_DEALER_INVENTORY_ITEM E INNER JOIN MST_PRODUCT EM ON E.PRODUCT_ID = EM.PRODUCT_ID WHERE E.ACCOUNT_ID= :accountId AND EM.ALLOW_SERIAL_NUM = :isSerialized",
nativeQuery = true)
Page<Object[]> getNonSerializedDeviceList(#Param("accountId") Long accountId, #Param("isSerialized") String isSerialized, Pageable pageable);
}
Error
HQL: SELECT COUNT(*) FROM TXN_DEALER_INVENTORY_ITEM E INNER JOIN MST_PRODUCT EM ON E.PRODUCT_ID = EM.PRODUCT_ID WHERE E.ACCOUNT_ID= :accountId AND EM.ALLOW_SERIAL_NUM = :isSerialized
2023-02-10 18:52:52,753 ERROR [org.hibernate.hql.internal.ast.ErrorCounter] (http-/127.0.0.1:8881-1) line 1:76: unexpected token: ON
Native Query doesn't have any error when run from sql developer.
Framework versions are as follows, Unfortunately I'cant update these any further as there are limitations in deployment environment. You inputs are highly welcome on this !!
<spring.version>4.3.30.RELEASE</spring.version>
<spring.data.version>1.11.23.RELEASE</spring.data.version>
<hibernate.version>4.2.18.Final</hibernate.version>
PS : for testing purpose when I change the SQL to a very basic like a select *, It gives following error.
org.hibernate.hql.internal.ast.QuerySyntaxException: TXN_DEALER_INVENTORY_ITEM is not mapped [SELECT COUNT(*) FROM TXN_DEALER_INVENTORY_ITEM E ]
Got the problem resolved by removing the count query and make the return type to a List of object Array instead of Page as below code segment.
#Repository
public interface Aaaa extends PagingAndSortingRepository<TxnDealerInventoryItem, Long> {
#Query(value = "SELECT EM.PART_NO, EM.PART_NAME FROM TXN_DEALER_INVENTORY_ITEM E INNER JOIN MST_PRODUCT EM ON E.PRODUCT_ID = EM.PRODUCT_ID WHERE E.ACCOUNT_ID= :accountId AND EM.ALLOW_SERIAL_NUM = :isSerialized ORDER BY ?#{#pageable}",
nativeQuery = true)
List<Object[]> getNonSerializedDeviceList(#Param("accountId") Long accountId, #Param("isSerialized") String isSerialized, Pageable pageable);
}
Finally having good old Paginations :) Thanks everyone who looked into this.
#Query("select u from User u join u.favoriteRestos s where u.id = ?1 ")
User findByIdWithFavoriteRestos (Long userId);
O have this query in the my repo, but instead of returning a User with empty favoriteRestos collection is returning a null user
that returns also a null user:
#Query("select u from User u join fetch u.favoriteRestos s where u.id = ?1 ")
User findByIdWithFavoriteRestos (Long userId);
I also tried with:
#Query("select u from User u left join u.favoriteRestos s where u.id = ?1 ")
User findByIdWithFavoriteRestos (Long userId);
It doesn't find any user, because join is an inner join. You need to add left keyword:
#Query("select u from User u left join fetch u.favoriteRestos s where u.id = ?1 ")
User findByIdWithFavoriteRestos (Long userId);
P.S.: fetch is also needed if you have a default (lazy) to-many mapping if you want to populate the collection.
I have this sql query that works fine
SELECT ppd.userId, AVG(Coalesce(sm.score, 0)) AS avgScore
FROM Table1 ppd
LEFT JOIN Table2 sm ON ppd.userId = sm.userId AND sm.created BETWEEN start AND end
WHERE ppd.someId IN (listOfIds)
GROUP BY ppd.userId ORDER BY avgScore DESC LIMIT 1 OFFSET 0;
I have written a query for a method in a JPARepository that looks like the following
#Query(value = "SELECT ppd.userId, AVG(Coalesce(sm.score, 0)) AS avgScore " +
" FROM Table1 ppd " +
" LEFT JOIN Table2 sm ON sm.userId = ppd.userId AND sm.created BETWEEN :start AND :end " +
" WHERE ppd.someId IN (:IdList) GROUP BY ppd.userId",
nativeQuery = true)
Page<MyDtoProjection> getAvg(#Param("IdList") List<String> IdList,
#Param("start") Long start,
#Param("end") Long end,
Pageable pageable);
Which works just as expected except when the pageable size is 1 (SQL query with Limit 1 works fine).
For the pageable parameter the service sends the following pageable object
Sort sort = JpaSort.unsafe(Sort.Direction.ASC, "(avgScore)");
Pageable pageable = PageRequest.of(offset, limit, sort);
The request sets offset=0 and limit=1
The error I'm receiving from request using repository code is this one:
Unknown column 'ppd' in 'field list'\nQuery is: select count(ppd)..., I don't know why it is taking the alias ppd as a column name, also, I read from another post that this way of executing native queries performs a count (which can be seen in the error message, I did not specify that count query), and that it can be specified by setting countQuery parameter, but I'm not sure what to write there.
Solve it>
Just added the countQuery to the #Query annotation, I was not so sure what to place there, but it was simple
#Query(value = "SELECT ppd.userId, AVG(Coalesce(sm.score, 0)) AS avgScore " +
" FROM Table1 ppd " +
" LEFT JOIN Table2 sm ON sm.userId = ppd.userId AND sm.created BETWEEN :start AND :end " +
" WHERE ppd.someId IN (:IdList) GROUP BY ppd.userId",
countQuery = "SELECT COUNT(ppd.userId) FROM Table1 ppd " +
"WHERE ppd.userId IN (:IdList) GROUP BY ppd.userId",
nativeQuery = true)
Page<MyDtoProjection> getAvg(#Param("IdList") List<String> IdList,
#Param("start") Long start,
#Param("end") Long end,
Pageable pageable);
This is the only post I found that mentions the countQuery and how it works in one of the answers, go check it out if someone is going through the same
Spring Data and Native Query with pagination
I want to make a distinct select in my table with pagination, but it is claiming this error. Does anyone know how to solve it?
org.postgresql.util.PSQLException: ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list
#Query(value = "SELECT DISTINCT budget.* FROM budget LEFT JOIN user_budget ON budget.id = user_budget.budget_id ORDER BY budget.created DESC, ?#{#pageable}",
countQuery = "SELECT DISTINCT count(*) FROM budget LEFT JOIN user_budget ON budget.id = user_budget.budget_id",
nativeQuery = true)
public Page<Budget> findAllByfilter(Pageable pageable);
I am trying to use HQL fetching my entity along with sub-entities using JOIN FETCH, this is working fine if I want all the results but it is not the case if I want a Page
My entity is
#Entity
#Data
public class VisitEntity {
#Id
#Audited
private long id;
.
.
.
#OneToMany(cascade = CascadeType.ALL,)
private List<VisitCommentEntity> comments;
}
and because I have millions of visits I need to use Pageable and I want to Fetch the comments in a single database query like :
#Query("SELECT v FROM VisitEntity v LEFT JOIN FETCH v.comments WHERE v.venue.id = :venueId and ..." )
public Page<VisitEntity> getVenueVisits(#Param("venueId") long venueId,...,
Pageable pageable);
That HQL call throws the following exception:
Caused by: java.lang.IllegalArgumentException: org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list [FromElement{explicit,not a collection join,fetch join,fetch non-lazy properties,classAlias=null,role=com.ro.lib.visit.entity.VisitEntity.comments,tableName=visitdb.visit_comment,tableAlias=comments1_,origin=visitdb.visit visitentit0_,columns={visitentit0_.visit_id ,className=com.ro.lib.visit.entity.VisitCommentEntity}}] [select count(v) FROM com.ro.lib.visit.entity.VisitEntity v LEFT JOIN FETCH v.comments WHERE v.venue.id = :venueId and (v.actualArrival > :date or v.arrival > :date)]
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1374)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
at org.hibernate.ejb.AbstractEntityManagerImpl.createQuery(AbstractEntityManagerImpl.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
and once I remove the paging everything works fine
#Query("SELECT v FROM VisitEntity v LEFT JOIN FETCH v.comments WHERE v.venue.id = :venueId and ..." )
public List<VisitEntity> getVenueVisits(#Param("venueId") long venueId,...);
Obviously the problem is the count query from Spring-Data, but how can we fix it?
The easiest way is to use the countQuery attribute of the the #Query annotation to provide a custom query to be used.
#Query(value = "SELECT v FROM VisitEntity v LEFT JOIN FETCH v.comments …",
countQuery = "select count(v) from VisitEntity v where …")
List<VisitEntity> getVenueVisits(#Param("venueId") long venueId, …);
Alternatively in newest versions of Spring (supporting JPA 2.1 specification) you can use entity graph like this:
#EntityGraph(attributePaths = "roles")
#Query("FROM User user")
Page<User> findAllWithRoles(Pageable pageable);
Of course named entity graphs work as well.
You have to specify countQuery param for #Query and now you can use Page or List as return value.
#Query(value = "SELECT v FROM VisitEntity v LEFT JOIN FETCH v.comments WHERE v.venue.id = :venueId and ...",
countQuery = "SELECT count(v) FROM VisitEntity v LEFT JOIN v.comments WHERE v.venue.id = :venueId and ..." )
public Page<VisitEntity> getVenueVisits(#Param("venueId") long venueId,...,
Pageable pageable);
If you want completely control your query build by Specification with join fetch you can check CriteriaQuery return type and change join fetch logic according to query type like this:
public class ContactSpecification implements Specification<Contact> {
#Override
public Predicate toPredicate(Root<Contact> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
if(query.getResultType() == Long.class) {
root.join(Contact_.company);
} else {
root.fetch(Contact_.company);
}
return cb.equal(root.get(Contact_.company).get(Company_.name), "Company 123");
}
}
I was not able to find this info in documentation, but from SimpleJpaRepository.getCountQuery() method you can see query for count request first build for Long return type, and later fetch for expected class is running.
CriteriaBuilder builder = em.getCriteriaBuilder();
CriteriaQuery<Long> query = builder.createQuery(Long.class);
Root<S> root = applySpecificationToCriteria(spec, domainClass, query);
It can be not reliable since it is an implementation details which can be changed, but it works.
Try countProjection
#Query(value="SELECT v FROM VisitEntity v LEFT JOIN FETCH v.comments WHERE v.venue.id = :venueId and ..." ,
countProjection = "v.id")
public Page<VisitEntity> getVenueVisits(#Param("venueId") long venueId,...,
Pageable pageable);