We Keep Coding

EntityManager.createNativeQuery returning list of objects instead of list of BigDecimal when using Pagination

I am trying to use Pagination with EntityManager.createNativeQuery(). Below is the skeleton code that I am using:
var query = em.createNativeQuery("select distinct id from ... group by ... having ...");
List<BigDecimal> results = query
.setMaxResults(pageSize)
.setFirstResult(pageNumber * pageSize)
.getResultList();
When pageNumber is 0 (first page), I get the expected List of BigDecimals:
But as soon as pageNumber > 0 (example, second page), I get a List of Objects, and each object in this list seems to contain two BigDecimals, the first of which contains the value from the db, and the second BigDecimal seems to be the position of this row.
and obviously I get this exception
java.lang.ClassCastException: class [Ljava.lang.Object; cannot be cast to class java.math.BigDecimal
Can someone please explain this discrepancy, and how this can be fixed to always return a List of BigDecimals? Thank you.
Update-1 : I have created a sample project to reproduce this issue. I was able to reproduce this issue only with an Oracle database. With H2 database, it worked fine, and I consistently got a list of BigDecimals irrelevant of the page number.
Update-2 : I have also created a sample project with H2 where it works without this issue.

The problem that you are running into is that your OracleDialect adds a column to its selected ResultSet. It wraps the query that you are running as discussed in SternK's answer.
If you were using the Hibernate SessionFactory and the Session interfaces, then the function that you would be looking for would be the "addScalar" method. Unfortunately, there doesn't seem to be an implementation in pure JPA (see the question asked here: Does JPA have an equivalent to Hibernate SQLQuery.addScalar()?).
I would expect your current implementation to work just fine in DB2, H2, HSQL, Postgres, MySQL (and a few other DB engines). However, in Oracle, it adds a row-number column to the ResultSet which means that Hibernate gets 2 columns from the ResultSet. Hibernate does not implement any query parsing in this case, which means that it simply parses the ResultSet into your List. Since it gets 2 values, it converts them into an Object[] rather than a BigDecimal.
As a caveat, relying on the JDBC driver to provide the expected-data-type is a bit dangerous, since Hibernate will ask the JDBC driver which data-type it suggests. In this case, it suggests a BigDecimal, but under certain conditions and certain implementations would be allowed to return a Double or some other type.
You have a couple options then.
You can modify your oracle-dialect (as SternK) suggests. This will take advantage of an alternate oracle-paging implementation.
If you are not opposed to having hibnerate-specific aspects in your JPA implementation, then you can take advantage of additional hibernate functions that are not offered in the JPA standard. (See the following code...)
List<BigDecimal> results = entitymanager.createNativeQuery("select distinct id from ... group by ... having ...")
.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("id", BigDecimalType.INSTANCE)
.getResultList();
System.out.println(results);
This does have the advantage of explicitly telling hibnerate, that you are only interested in the "id" column of your ResultSet, and that hibernate needs to explicitly convert to the returned object to a BigDecimal, should the JDBC-driver decide that a different type would be more appropriate as a default.

The root cause of your problem in the way how the pagination implemented in your hibernate oracle dialect.
There are two cases:
When we have setFirstResult(0) the following sql will be generated:
-- setMaxResults(5).setFirstResult(0)
select * from (
select test_id from TST_MY_TEST -- this is your initial query
)
where rownum <= 5;
As you can see, this query returns exactly the same columns list as your initial query, and therefore you do not have problem with this case.
When we set setFirstResult in not 0 value the following sql will be generated:
-- setMaxResults(5).setFirstResult(2)
select * from (
select row_.*, rownum rownum_
from (
select test_id from TST_MY_TEST -- this is your initial query
) row_
where rownum <= 5
)
where rownum_ > 2
As you can see, this query returns the columns list with additional rownum_ column, and therefore you do have the problem with casting this result set to the BigDecimal.
Solution
If you use Oracle 12c R1 (12.1) or higher you can override this behavior in your dialect using new row limiting clause in this way:
import org.hibernate.dialect.Oracle12cDialect;
import org.hibernate.dialect.pagination.AbstractLimitHandler;
import org.hibernate.dialect.pagination.LimitHandler;
import org.hibernate.dialect.pagination.LimitHelper;
import org.hibernate.engine.spi.RowSelection;
public class MyOracleDialect extends Oracle12cDialect
{
private static final AbstractLimitHandler LIMIT_HANDLER = new AbstractLimitHandler() {
#Override
public String processSql(String sql, RowSelection selection) {
final boolean hasOffset = LimitHelper.hasFirstRow(selection);
final StringBuilder pagingSelect = new StringBuilder(sql.length() + 50);
pagingSelect.append(sql);
/*
see the documentation https://docs.oracle.com/database/121/SQLRF/statements_10002.htm#BABHFGAA
(Restrictions on the row_limiting_clause)
You cannot specify this clause with the for_update_clause.
*/
if (hasOffset) {
pagingSelect.append(" OFFSET ? ROWS");
}
pagingSelect.append(" FETCH NEXT ? ROWS ONLY");
return pagingSelect.toString();
}
#Override
public boolean supportsLimit() {
return true;
}
};
public MyOracleDialect()
{
}
#Override
public LimitHandler getLimitHandler() {
return LIMIT_HANDLER;
}
}
and then use it.
<property name="hibernate.dialect">com.me.MyOracleDialect</property>
For my test data set for the following query:
NativeQuery query = session.createNativeQuery(
"select test_id from TST_MY_TEST"
).setMaxResults(5).setFirstResult(2);
List<BigDecimal> results = query.getResultList();
I got:
Hibernate:
/* dynamic native SQL query */
select test_id from TST_MY_TEST
OFFSET ? ROWS FETCH NEXT ? ROWS ONLY
val = 3
val = 4
val = 5
val = 6
val = 7
P.S. See also HHH-12087
P.P.S I simplified my implementation of the AbstractLimitHandler by removing checking presents FOR UPDATE clause. I think we will not have nothing good in this case and with this checking.
For example for the following case:
NativeQuery query = session.createNativeQuery(
"select test_id from TST_MY_TEST FOR UPDATE OF test_id"
).setMaxResults(5).setFirstResult(2);
hibernate (with Oracle12cDialect) will generate the following sql:
/* dynamic native SQL query */
select * from (
select
row_.*,
rownum rownum_
from (
select test_id from TST_MY_TEST -- initial sql without FOR UPDATE clause
) row_
where rownum <= 5
)
where rownum_ > 2
FOR UPDATE OF test_id -- moved for_update_clause
As you can see, hibernate tries to fix query by moving FOR UPDATE to the end of the query. But anyway, we will get:
ORA-02014: cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc.

I've simulated your consult and everything works fine. I've used DataJpaTest to instance entityManager for me, h2 memory database and JUnit 5 to run the test. See below:
#Test
public void shouldGetListOfSalaryPaginated() {
// given
Person alex = new Person("alex");
alex.setSalary(BigDecimal.valueOf(3305.33));
Person john = new Person("john");
john.setSalary(BigDecimal.valueOf(33054.10));
Person ana = new Person("ana");
ana.setSalary(BigDecimal.valueOf(1223));
entityManager.persist(alex);
entityManager.persist(john);
entityManager.persist(ana);
entityManager.flush();
entityManager.clear();
// when
List<BigDecimal> found = entityManager.createNativeQuery("SELECT salary FROM person").setMaxResults(2).setFirstResult(2*1).getResultList();
// then
Assertions.assertEquals(found.size(), 1);
Assertions.assertEquals(found.get(0).longValue(), 1223L);
}
I suggest that you review your native query. It's preferable that you use Criteria API instead and let native queries for extreme cases like complex consults.
Update
After the author posted the project, I could reproduce the problem and it was related to the oracle dialect. For unknown reason the query which is running for the second call is: select * from ( select row_.*, rownum rownum_ from ( SELECT c.SHOP_ID FROM CUSTOMER c ) row_ where rownum <= ?) where rownum_ > ?, and that's why this is generating a bug, because it's querying 2 columns instead of only one. The undesired one is this rownum. For other dialects there is no such problem.
I suggest you try other oracle dialect version and whether none of them work, my final tip is try to do the pagination yourself.

After a lot of trails with different versions of different spring libraries, I was finally able to figure out the issue. In one of my attempts, the issue seems to have disappeared, as soon as I updated the spring-data-commons library from v2.1.5.RELEASE to v2.1.6.RELEASE. I looked up the changelog of this release, and this bug, which is related to this bug in spring-data-commons, is the root cause of this issue. I was able to fix the issue after upgrading the spring-data-commons library.

Hibernate exception on encountering mysql := operator

When I execute the following code the exception occurs:
Exception: org.springframework.orm.hibernate3.HibernateQueryException:
Not all named parameters have been set
Here is my code:
queryString = SET #quot=0,#latest=0,#comp='';
select B.* from (
select A.time,A.change,IF(#comp<>A.company,1,0) as LATEST,#comp:=A.company as company from (
select time,company,quote-#quot as `change`, #quot:=quote curr_quote
from stocks order by company,time) A
order by company,time desc) B where B.LATEST=1;
list = getHibernateTemplate().executeFind(new HibernateCallback(){
public Object doInHibernate(Session session)throws HibernateException,SQLException {
SQLQuery query = session.createSQLQuery(queryString);
query.setParameterList("list", custIds);
return query.list();
}
What is the reason for this behavior?

It's a little bit hard to understand, what is exactly the query you are executing, but if you need to use the colon character in native query, in your case as "assign a value" operator, you should escape all the colon occurances with \\ in your java String with the query, so it could be like:
select B.* from (
select A.time,A.change,IF(#comp<>A.company,1,0) as LATEST,#comp\\:=A.company as company from (
select time,company,quote-#quot as `change`, #quot\\:=quote curr_quote
from stocks order by company,time) A
order by company,time desc) B where B.LATEST=1;
Update: seems, it is not possible yet to escape the colons in Hibernate native queries, there is an open issue about it. That means, that you are not able to use a colons in Hibernate native queries not for the named parameters. You can try to create a function and call it instead of calling a query.

you can create a named query and then use it in spring jpa repository or hibernate. This link helped me from similar problem.

what is the equelent of sql query in hibernate

In SQL Server i am using this query
select *
from Unit c
ORDER BY CONVERT(INT, LEFT(name, PATINDEX('%[^0-9]%', name + 'z')-1)) desc;
I want this query to use in Hibernate. when I use this in Hibernate I got error
java.lang.IllegalArgumentException: org.hibernate.hql.ast.QuerySyntaxException:unexpected token: LEFT near line 1, column 122
[SELECT c FROM models.entities.UnitEntity c WHERE c.expSetId = :expSetId AND isWaitArea=:isWaitArea ORDER BY CONVERT(INT, LEFT(name, PATINDEX('%[^0-9]%', name + 'z')-1)) asc]

some of that is not in the hibernate dialect, you can change left with substring, convert with cast. and as for patindex i couldn't find the substitution. you either can add pathindex to the constructor of the dialect that you use
registerFunction( "patindex", new StandardSQLFunction("patindex") );
or create patindex() to a stored procedure.
then you can use something like this:
from Unit c order by cast(substring(name, 0, PATINDEX('%[^0-9]%', name + 'z')-1) as integer);
or you can use locate() instead of patindex(), but i think it doesn't support regular expression.

I am pretty sure that you can use SQL query with hibernate as well. When you create your hibernate session, you can use something like
session.createSQLQuery("Your Query Here")
Hope this helps.

JPA Criteria API using ISNULL in a order by statement

I use MySQL 5.5 with Hibernate 3.6 and JPA 2.0. I have a User table with a firstName which could also be null or an empty string. I want to have those empty firstName results last in my search results. For that I wrote the following SQL query which works just fine:
SELECT * FROM User ORDER BY ISNULL(firstName), firstName = "", firstName ASC LIMIT 100
Now want to translate this to JPA using the criteria API and I am not quite so sure about the order by. Here is what I have:
criteriaQuery = criteriaQuery.orderBy(cb.asc(cb.isNull(users.get(User_.firstName))), cb.asc(cb.equal(users.get(User_.firstName), "")), cb.asc(users.get(User_.firstName)));
However, the code snippet above does not work, because the CriteriaBuilder.isNull() method is traslated to IS NULL and not to the ISNULL() function of MySQL. I get the following exception:
org.hibernate.hql.ast.QuerySyntaxException: unexpected AST node: is null
Any ideas on how to check for null in the Order by statement with JPA 2.0

That is not possible. In JPA you can ORDER BY fields only that you select (that are in the SELECT part of your query). The problem is that there is no IS_NULL function, that can be used in the SELECT part.

I got the exact same problem as you do, finally I solve it using this way, maybe you can try:
CriteriaQuery<> query;
query.orderBy(cb.desc(cb.selectCase().
when(cb.isNull("field name"),0).otherwise(1)),
cb.asc("field name");

How can I use MySQL assign operator(:=) in hibernate native query?

I'm using Hibernate. I wrote some native query because I need to use sub select statement.
Query looks like this:
SELECT sub.rownum FROM
(SELECT k.`news_master_id` AS id, #row := #row + 1 AS rownum
FROM keyword_news_list k
JOIN (SELECT #row := 0) r
WHERE k.`keyword_news_id` = :kid
ORDER BY k.`news_master_id` ASC) AS sub
WHERE sub.id = :nid
When I run this query like this:
sessionFactory.getCurrentSession()
.createSQLQuery(query)
.setParameter("kid", kid)
.setParameter("nid", nid)
.uniqueResult();
This exception comes out:
org.hibernate.QueryException: Space is not allowed after parameter prefix ':' ....
This might because of := operator. I found some Hibernate issue about this. This issue is still open. Isn't there any solution for this problem?

Note that HHH-2697 is now fixed for Hibernate 4.1.3 You can now escape with backslash:
SELECT k.`news_master_id` AS id, #row \:= #row + 1 AS rownum
FROM keyword_news_list k
JOIN (SELECT #row \:= 0) r
WHERE k.`keyword_news_id` = :kid
ORDER BY k.`news_master_id` ASC

Another solution for those of us who can't make the jump to Hibernate 4.1.3.
Simply use /*'*/:=/*'*/ inside the query. Hibernate code treats everything between ' as a string (ignores it). MySQL on the other hand will ignore everything inside a blockquote and will evaluate the whole expression to an assignement operator.
I know it's quick and dirty, but it get's the job done without stored procedures, interceptors etc.

you can implement this is a slightly different way.. you need to replace the : operator with something else (say '|' char ) and in your interceptor replace the '|' with the : .
this way hibernate will not try to think the : is a param but will ignore it
For the interceptor logic you can refer to the hibernate manual
This has worked for me using MySQL 5.
remember, this replacing of : must be only done to ':=' and other MySQL specific requirments.. don't try to replace the : for the param-placeholders. (hibernate will not be able to identify the params then)

I prefer to include Spring JDBC and execute the query rather than fight against Hibernate interceptors.

in Hibernate exception on encountering mysql := operator Stanislav gave another option other than interceptor to solve this issue

If you keep your SQL-files away from Java code - try this piece of code. Played a lot to get the right number of escaping slashes:
String sqlPattern = FileUtils.readFile(this.getClass(), /sql/my_query.sql");
sqlPattern = sqlPattern.replaceAll(":=", "\\\\:=");
Query query = entityManager.createNativeQuery(sqlPattern);

I guess there should not be a space after = , the operator should be written as =: (without any spaces)