I have the SQL like:
select *
from some_table
where (select array(select unnest(first_array_field) except select unnest(second_array_field))) #> '{some_value}';
This is my try to write the where condition for that query in a JOOQ way:
private Condition selectAwayValue(
List<Long> list) {
var allSelect = dslContext
.select(TABLE.ARRAY_1)
.from(DSL.unnest(TABLE.ARRAY_1));
var homeSelect = dslContext
.select(TABLE.ARRAY_2)
.from(DSL.unnest(TABLE.ARRAY_2));
var awaySelect = allSelect.except(homeSelect);
var awayArray = dslContext.select(DSL.array(awaySelect));
return awayArray.asField().contains(awayCompetitorIdsList);
}
I'm facing 2 problems here:
I got the different SQL than I expected (see above)
Also got the exception like Cannot convert from 5 (class java.lang.Long) to class [Ljava.lang.Long;
I understand what's the exceptions says to me, but don't understand why :)
What's would be the best way to write my SQL in a jooq syntax?
I'm using the latest stable jooq version, the latest stable Postgres version
thanks
Alternative solution
If you think of your problem this way:
select *
from some_table
where first_array_field #> '{some_value}'
and not second_array_field #> '{some_value}';
Then, the problem is a lot simpler to express in jOOQ as well
ctx.selectFrom(SOME_TABLE)
.where(SOME_TABLE.FIRST_ARRAY_FIELD.contains(new Long[] { someValue }))
.andNot(SOME_TABLE.SECOND_ARRAY_FIELD.contains(new Long[] { someValue }))
.fetch();
Why your solution didn't work
Of course, you could pull this off using your own approach. Regarding the problems you've observed:
I got the different SQL than I expected (see above)
That syntax where you put UNNEST() in SELECT is not supported by jOOQ. It is quite the weird syntax in PostgreSQL as well, and doesn't do what you might think it does in many ways. Have you tried putting two instances of UNNEST() in SELECT? The second one won't form a cartesian product like the first one. Rows from it will be matched with the first one's rows by ordinality.
However, using UNNEST in FROM is correct and also more intuitive in native PostgreSQL, even if it's a bit longer. And it's supported by jOOQ.
Also got the exception like Cannot convert from 5 (class java.lang.Long) to class [Ljava.lang.Long;
This is because you seem to be calling Field.contains() with a scalar argument, instead of an array. See the example from the Javadoc, or my example above:
// Use this expression
val(new Integer[] { 1, 2, 3 }).contains(new Integer[] { 1, 2 })
// ... to render this SQL
ARRAY[1, 2, 3] #> ARRAY[1, 2]
But again, I think my alternative approach is much simpler.
Related
Using Jooq, I am trying to fetch from a table by id first, if no matches found, then fetch by handle again.
And I want all fields of the returned rows, not just one.
Field<?> firstMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.ID.eq(id))
.asfield(); // This is wrong, because it supports only one field, but above we selected Tables.MY_TABLE.fields(), which is plural.
Field<?> secondMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.HANDLE.eq(handle))
.asfield(); // Same as above.
dslContext.select(DSL.coalesce(firstMatch, secondMatch))
.fetchInto(MyClass.class);
Due to the mistake mentioned above in the code, the following error occurs:
Can only use single-column ResultProviderQuery as a field
I am wondering how to make firstMatch and secondMatch two lists of fields, instead of two fields?
I tried
Field<?>[] secondMatch = DSL.select(Tables.MY_TABLE.fields())
.from(Tables.MY_TABLE.fields())
.where(Tables.MY_TABLE.HANDLE.eq(handle))
.fields();
but the following error occurred in the line containing DSL.coalesce
Type interface org.jooq.Field is not supported in dialect DEFAULT
Thanks in advance!
This sounds much more like something you'd do with a simple OR?
dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.eq(id))
// The ne(id) part might not be required...
.or(MY_TABLE.ID.ne(id).and(MY_TABLE.HANDLE.eq(handle))
.fetchInto(MyClass.class);
If the two result sets should be completely exclusive, then you can do this:
dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.eq(id))
.or(MY_TABLE.HANDLE.eq(handle).and(notExists(
selectFrom(MY_TABLE).where(MY_TABLE.ID.eq(id))
)))
.fetchInto(MyClass.class);
If on your database product, a query using OR doesn't perform well, you can write an equivalent query with UNION ALL, which might perform better.
I am trying to use Pagination with EntityManager.createNativeQuery(). Below is the skeleton code that I am using:
var query = em.createNativeQuery("select distinct id from ... group by ... having ...");
List<BigDecimal> results = query
.setMaxResults(pageSize)
.setFirstResult(pageNumber * pageSize)
.getResultList();
When pageNumber is 0 (first page), I get the expected List of BigDecimals:
But as soon as pageNumber > 0 (example, second page), I get a List of Objects, and each object in this list seems to contain two BigDecimals, the first of which contains the value from the db, and the second BigDecimal seems to be the position of this row.
and obviously I get this exception
java.lang.ClassCastException: class [Ljava.lang.Object; cannot be cast to class java.math.BigDecimal
Can someone please explain this discrepancy, and how this can be fixed to always return a List of BigDecimals? Thank you.
Update-1 : I have created a sample project to reproduce this issue. I was able to reproduce this issue only with an Oracle database. With H2 database, it worked fine, and I consistently got a list of BigDecimals irrelevant of the page number.
Update-2 : I have also created a sample project with H2 where it works without this issue.
The problem that you are running into is that your OracleDialect adds a column to its selected ResultSet. It wraps the query that you are running as discussed in SternK's answer.
If you were using the Hibernate SessionFactory and the Session interfaces, then the function that you would be looking for would be the "addScalar" method. Unfortunately, there doesn't seem to be an implementation in pure JPA (see the question asked here: Does JPA have an equivalent to Hibernate SQLQuery.addScalar()?).
I would expect your current implementation to work just fine in DB2, H2, HSQL, Postgres, MySQL (and a few other DB engines). However, in Oracle, it adds a row-number column to the ResultSet which means that Hibernate gets 2 columns from the ResultSet. Hibernate does not implement any query parsing in this case, which means that it simply parses the ResultSet into your List. Since it gets 2 values, it converts them into an Object[] rather than a BigDecimal.
As a caveat, relying on the JDBC driver to provide the expected-data-type is a bit dangerous, since Hibernate will ask the JDBC driver which data-type it suggests. In this case, it suggests a BigDecimal, but under certain conditions and certain implementations would be allowed to return a Double or some other type.
You have a couple options then.
You can modify your oracle-dialect (as SternK) suggests. This will take advantage of an alternate oracle-paging implementation.
If you are not opposed to having hibnerate-specific aspects in your JPA implementation, then you can take advantage of additional hibernate functions that are not offered in the JPA standard. (See the following code...)
List<BigDecimal> results = entitymanager.createNativeQuery("select distinct id from ... group by ... having ...")
.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("id", BigDecimalType.INSTANCE)
.getResultList();
System.out.println(results);
This does have the advantage of explicitly telling hibnerate, that you are only interested in the "id" column of your ResultSet, and that hibernate needs to explicitly convert to the returned object to a BigDecimal, should the JDBC-driver decide that a different type would be more appropriate as a default.
The root cause of your problem in the way how the pagination implemented in your hibernate oracle dialect.
There are two cases:
When we have setFirstResult(0) the following sql will be generated:
-- setMaxResults(5).setFirstResult(0)
select * from (
select test_id from TST_MY_TEST -- this is your initial query
)
where rownum <= 5;
As you can see, this query returns exactly the same columns list as your initial query, and therefore you do not have problem with this case.
When we set setFirstResult in not 0 value the following sql will be generated:
-- setMaxResults(5).setFirstResult(2)
select * from (
select row_.*, rownum rownum_
from (
select test_id from TST_MY_TEST -- this is your initial query
) row_
where rownum <= 5
)
where rownum_ > 2
As you can see, this query returns the columns list with additional rownum_ column, and therefore you do have the problem with casting this result set to the BigDecimal.
Solution
If you use Oracle 12c R1 (12.1) or higher you can override this behavior in your dialect using new row limiting clause in this way:
import org.hibernate.dialect.Oracle12cDialect;
import org.hibernate.dialect.pagination.AbstractLimitHandler;
import org.hibernate.dialect.pagination.LimitHandler;
import org.hibernate.dialect.pagination.LimitHelper;
import org.hibernate.engine.spi.RowSelection;
public class MyOracleDialect extends Oracle12cDialect
{
private static final AbstractLimitHandler LIMIT_HANDLER = new AbstractLimitHandler() {
#Override
public String processSql(String sql, RowSelection selection) {
final boolean hasOffset = LimitHelper.hasFirstRow(selection);
final StringBuilder pagingSelect = new StringBuilder(sql.length() + 50);
pagingSelect.append(sql);
/*
see the documentation https://docs.oracle.com/database/121/SQLRF/statements_10002.htm#BABHFGAA
(Restrictions on the row_limiting_clause)
You cannot specify this clause with the for_update_clause.
*/
if (hasOffset) {
pagingSelect.append(" OFFSET ? ROWS");
}
pagingSelect.append(" FETCH NEXT ? ROWS ONLY");
return pagingSelect.toString();
}
#Override
public boolean supportsLimit() {
return true;
}
};
public MyOracleDialect()
{
}
#Override
public LimitHandler getLimitHandler() {
return LIMIT_HANDLER;
}
}
and then use it.
<property name="hibernate.dialect">com.me.MyOracleDialect</property>
For my test data set for the following query:
NativeQuery query = session.createNativeQuery(
"select test_id from TST_MY_TEST"
).setMaxResults(5).setFirstResult(2);
List<BigDecimal> results = query.getResultList();
I got:
Hibernate:
/* dynamic native SQL query */
select test_id from TST_MY_TEST
OFFSET ? ROWS FETCH NEXT ? ROWS ONLY
val = 3
val = 4
val = 5
val = 6
val = 7
P.S. See also HHH-12087
P.P.S I simplified my implementation of the AbstractLimitHandler by removing checking presents FOR UPDATE clause. I think we will not have nothing good in this case and with this checking.
For example for the following case:
NativeQuery query = session.createNativeQuery(
"select test_id from TST_MY_TEST FOR UPDATE OF test_id"
).setMaxResults(5).setFirstResult(2);
hibernate (with Oracle12cDialect) will generate the following sql:
/* dynamic native SQL query */
select * from (
select
row_.*,
rownum rownum_
from (
select test_id from TST_MY_TEST -- initial sql without FOR UPDATE clause
) row_
where rownum <= 5
)
where rownum_ > 2
FOR UPDATE OF test_id -- moved for_update_clause
As you can see, hibernate tries to fix query by moving FOR UPDATE to the end of the query. But anyway, we will get:
ORA-02014: cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc.
I've simulated your consult and everything works fine. I've used DataJpaTest to instance entityManager for me, h2 memory database and JUnit 5 to run the test. See below:
#Test
public void shouldGetListOfSalaryPaginated() {
// given
Person alex = new Person("alex");
alex.setSalary(BigDecimal.valueOf(3305.33));
Person john = new Person("john");
john.setSalary(BigDecimal.valueOf(33054.10));
Person ana = new Person("ana");
ana.setSalary(BigDecimal.valueOf(1223));
entityManager.persist(alex);
entityManager.persist(john);
entityManager.persist(ana);
entityManager.flush();
entityManager.clear();
// when
List<BigDecimal> found = entityManager.createNativeQuery("SELECT salary FROM person").setMaxResults(2).setFirstResult(2*1).getResultList();
// then
Assertions.assertEquals(found.size(), 1);
Assertions.assertEquals(found.get(0).longValue(), 1223L);
}
I suggest that you review your native query. It's preferable that you use Criteria API instead and let native queries for extreme cases like complex consults.
Update
After the author posted the project, I could reproduce the problem and it was related to the oracle dialect. For unknown reason the query which is running for the second call is: select * from ( select row_.*, rownum rownum_ from ( SELECT c.SHOP_ID FROM CUSTOMER c ) row_ where rownum <= ?) where rownum_ > ?, and that's why this is generating a bug, because it's querying 2 columns instead of only one. The undesired one is this rownum. For other dialects there is no such problem.
I suggest you try other oracle dialect version and whether none of them work, my final tip is try to do the pagination yourself.
After a lot of trails with different versions of different spring libraries, I was finally able to figure out the issue. In one of my attempts, the issue seems to have disappeared, as soon as I updated the spring-data-commons library from v2.1.5.RELEASE to v2.1.6.RELEASE. I looked up the changelog of this release, and this bug, which is related to this bug in spring-data-commons, is the root cause of this issue. I was able to fix the issue after upgrading the spring-data-commons library.
I am stuck trying to get a query (QueryDSL) to work that gives me a count of distinct categories.
For example, what I am trying to achieve:
categoryA -> 10 entries
categoryB -> 20 entries
This is what i have so far:
query().from(application)
.transform(groupBy(application.category).as(list(application)));
However, this gives me for each category a list of all whole entries, I just want to get a count of this.
I tried messing around with count() but no luck.
Anybody know how to do this?
Note that as of Querydsl 4.x, the accepted answer by Timo Westkämper is no longer valid. Breaking changes in Querydsl 4.0 have removed the query.list() method.
A solution working with Querydsl 4.0+ would now be:
query.from(application)
.groupBy(application.category)
.select(application.category, application.category.count())
.fetch();
transform combined groupBy is meant to construct tree structures out of flat result sets. What you need can be easier expressed as
query.from(application)
.groupBy(application.category)
.list(application.category, application.category.count())
If using 4.1.4, you may need to use a JPAQueryFactory instead of JPAQuery
If using JPAQuery, there is no select() or fetch() on the return from groupBy.
JPAQueryFactory queryFactory = new JPAQueryFactory(entityManager);
queryFactory
.select(application.category, application.category.count())
.from(application)
.groupBy(application.category)
.fetch();
In QueryDSL 4.4.0:
query.from(application)
.groupBy(application.category)
.transform(GroupBy.groupBy(application.category)).as(application.category.count()))
Learn more on: https://www.tabnine.com/code/java/methods/com.querydsl.jpa.JPQLQuery/transform
I'm using CriteriaBuilder.concat to concatenate 2 Strings, using code below:
Expression<String> concat = criteriaBuilder.concat(expr1, expr2)
But the generated SQL is something like:
select distinct col_1 || col_2
which causes org.hibernate.hql.ast.QuerySyntaxException:
expecting CLOSE, found '||' near line 1, column 48 [
select count(distinct generatedAlias0.hostname || generatedAlias0.device) from ...
^(1,48)
I wonder how to force it to generate the following SQL which uses the concat() function, instead of the || operator?
select distinct concat(col_1, col_2)
Update:
From the error we can see that the problem is more on the Hibernate (v3.6.10.Final) side, which is why making MySQL to accept || for concatenation doesn't help, also updating to a newer version is not an option for me.
Thank you
I've actually found a workaround. by using #Formula (from Hibernate) instead of CriteriaBuilder for the same task, like this:
#Entity
public class MyEntity {
#Column(name="col_a")
private String colA;
#Column(name="col_b")
private String colB;
#Formula("concat(col_a, col_b)")
private String concated;
//...
}
This way I can use the concated field for CriteriaBuilder.countDistinct:
//...
Expression<?> exp = criteriaBuilder.countDistinct(entity.get("concated"));
criteriaQuery.select(exp);
TypedQuery<Long> query = entityManager.createQuery(criteriaQuery);
return query.getSingleResult();
I wish JPA would (or hopefully already) support countDistinct with multiple columns, then all these mess could have been avoided (see: How to countDistinct on multiple columns, the answer was NO).
I had a similar problem with the concat function.
I have used the concat function in a selectCase and this also returns the same QuerySyntaxException.
My workaround is to use the concat function via criteria builder function:
cb().selectCase().when(cb().equal(root.get(Person_.flag), cb().literal("1")),
cb().function("CONCAT", String.class, root.get(Person_.something), cb().literal(" bla bla bla")))
.otherwise(root.get(Person_.something)))
Hibernate Version 4.3.11.Final
I had this issue as well. The JPA/HQL generated sql query use pipes as concat (which is ||).
I am using Mariadb 10, Springboot-data-jpa2.0.6 (with Hibernate 5.2.17)
Issue example
Given HQL: select x from Xxx x where concat(x.field1, x.field2) = $1
Generated SQL: select ..... where (x.field1 || x.field2) = ?
Reason:
Use of || is deprecated, since mysql 8.0, unless the PIPES_AS_CONCAT SQL mode is enabled: https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html
For mariadb: https://mariadb.com/kb/en/or/
Work around:
(preferred) ranther then use concat function, there is a similar one concat_ws: https://mariadb.com/kb/en/concat_ws/
Use JPA native query #Query(nativeQuery = true, value ="select * from ....")
set global sql_mode=<list of modes which contains PIPES_AS_CONCAT>
(I didn't try this)
Note: This may be a simple question but I can't find a short way to make it clear. So, sorry for this long question.
In a project I am responsible, I use Spring 2.5, Hibernate 3.3.2 as middleware and Oracle database. As database is related to many other projects, some queries as really very complicated and I can't get a solution with Hibernate's solutions (HQL, Criteria, etc...). So I feel more comfortable with JdbcTemplate's queryForX() methods, as an example;
String sql = "select * from myTable";
jdbc.queryForList(sql);
Sure there are mostly "where" conditions and params indeed:
jdbc.querForList(sql, new Object[]{obj1,obj2,obj3 /* and many more arguments... */})
In this case, I must write question marks "?" for my parameters, so my SQL query string turns out some messy and hard to read; something like this:
select t1.col1, t2.col2, t1.col, --...some cols ,
sum(nvl(some_col1,?)-nvl(other_col2,?)) over (partition by col1,col2,col3,col4) sum_of_cols
from weird_table t1, another_table t2
where t1.col20=? and sum_of_cols>? and t1.col3=t2.col3 --and many ?'s...
and not exists (
select ? from boring_table t3 where -- many ?'s
)
--group by and order by order by etc
So now, which question mark is for which parameter? It is obvious but hard to read. But there are some other solutions for binded params like:
select * from a_table t where t.col1= :col1 and t.col2= :col2 -- and many more ":param"s
For this type query, we can write if it were Hibernate:
Query q = hibernateTemplate.createQuery();
q.setString("col1","a value");
q.setInteger("col2", 3);
I think it is more readable and easy to understand which value is what. I know I can do this with SQLQuery;
SQLQuery sq = hibernateTemplate.createSQLQuery();
/* same as above setInteger() etc. */
But this sq.list() gives me a list without a column name. so I have a basic array which is difficult to use:
[[1,2,"a"],[1,2,"b"], ...]
But with queryForList() I get better one:
[{COL1=1,COL2=2,COL3="a"},{COL1=1,COL2=2,COL3="b"},...]
So if I use queryForList(), I must write a very messy params Object;
or I use SQLQuery and then I have to get my list without a map as column names.
Is there a simple solution with mapped list using more readable param setting (like query.setX()) ?
Well you can use NamedParameterJdbcTemplate to do just that
Heres a sample
String query = "INSERT INTO FORUMS (FORUM_ID, FORUM_NAME, FORUM_DESC)
VALUES (:forumId,:forumName,:forumDesc)";
Map namedParameters = new HashMap();
namedParameters.put("forumId", Integer.valueOf(forum.getForumId()));
namedParameters.put("forumName", forum.getForumName());
namedParameters.put("forumDesc", forum.getForumDesc());
namedParameterJdbcTemplate.update(query, namedParameters);
You check the complete example with the source code in the below link
Spring NamedParameterJdbcTemplate Tutorial