I want to issue a query like the following
select max(col1), f(:1, col2) from t group by f(:1, col2)
where :1 is a bind variable. Using PreparedStatement, if I say
connection.prepareStatement
("select max(col1), f(?, col2) from t group by f(?, col2)")
I get an error from the DBMS complaining that f(?, col2) is not a GROUP BY expression.
How does one normally solve this in JDBC?
I suggest re-writing the statement so that there is only one bind argument.
This approach is kind of ugly, but returns the result set:
select max(col1)
, f_col2
from (
select col1
, f(? ,col2) as f_col2
from t
)
group
by f_col2
This re-written statement has a reference to only a single bind argument, so now the DBMS sees the expressions in the GROUP BY clause and the SELECT list are identical.
HTH
[EDIT]
(I wish there were a prettier way, this is why I prefer the named bind argument approach that Oracle uses. With the Perl DBI driver, positional arguments are converted to named arguments in the statement actually sent to Oracle.)
I didn't see the problem at first, I didn't understand the original question. (Apparently, several other people missed it too.) But after running some test cases, it dawned on me what the problem was, what the question was working.
Let me see if I can state the problem: how to get two separate (positional) bind arguments to be treated (by the DBMS) as if it were two references to the same (named) bind argument.
The DBMS is expecting the expression in the GROUP BY to match the expression in the SELECT list. But the two expressions are considered DIFFERENT even when the expressions are identical, when the only difference is that each expression references a different bind variable. (We can demonstrate some test cases that at least some DBMS will allow, but there are more general cases that will raise an exception.)
At this point the short answer is, that's got me stumped. The suggestion I have (which may not be an actual answer to the original question) is to restructure the query.
[/EDIT]
I can provide more details if this approach doesn't work, or if you have some other problem figuring it out. Or if there's a problem with performance (I can see the optimizer choosing a different plan for the re-written query, even though it returns the specified result set. For further testing, we'd really need to know what DBMS, what driver, statistics, etc.)
EDIT (eight and a half years later)
Another attempt at a query rewrite. Again, the only solution I come up with is a query with one bind placeholder. This time, we stick it into an inline view that returns a single row, and join that to t. I can see what it's doing; I'm not sure how the Oracle optimizer will see this. We may want (or need) to do an explicit conversion e.g. TO_NUMBER(?) AS param, TO_DATE(?,'...') AS param, TO_CHAR(?) AS param, depending on the datatype of the bind parameter, and the datatype we want to be returned as from the view.)
This is how I would do it in MySQL. The original query in my answer does the join operation inside the inline view (MySQL derived table). And we want to avoid materializing a hughjass derived table if we can avoid it. Then again, MySQL would probably let the original query slide as long as sql_mode doesn't include ONLY_FULL_GROUP_BY. MySQL would also let us drop the FROM DUAL)
SELECT MAX(t.col1)
, f( v.param ,t.col2)
FROM t
CROSS
JOIN ( SELECT ? AS param FROM DUAL) v
GROUP
BY f( v.param ,t.col2)
According to the answer from MadusankaD, within the past eight years, Oracle has added support for reusing the same named bind parameters in the JDBC driver, and retaining equivalence. (I haven't tested that, but if that works now, then great.)
Even though you have issued a query through JDBC driver(using PreparedStatement) like this:
select max(col1), f(:1, col2) from t group by f(:1, col2)
At last JDBC driver replaces these like below query before parsing to the database , even though you have used the same binding variable name in the both places.
select max(col1), f(*:1*, col2) from t group by f(*:2*, col2)
But in oracle this will not be recognized as a valid group by clause.
And also normal JDBC driver doesn't support named bind variables.
For that you can use OraclePreparedStatement class for you connection. That means it is oracle JDBC. Then you can use named bind variables. It will solve your issue.
Starting from Oracle Database 10g JDBC drivers, bind by name is supported using the setXXXAtName methods.
http://docs.oracle.com/cd/E24693_01/java.11203/e16548/apxref.htm#autoId20
Did you try using ? rather than the named bind variables? As well, which driver are you using? I tried this trivial example using the thin driver, and it seemed to work fine:
PreparedStatement ps = con.prepareStatement("SELECT COUNT(*), TO_CHAR(SYSDATE, ?) FROM DUAL GROUP BY TO_CHAR(SYSDATE, ?)");
ps.setString(1, "YYYY");
ps.setString(2, "YYYY");
ps.executeQuery();
In the second case, there are actually two variables - you will need to send them both with the same value.
Related
Given below is a gist of the query, which I'm able to run successfully in MySQL
SELECT a.*,
COALESCE(SUM(condition1 or condition2), 0) as countColumn
FROM table a
-- left joins with multiple tables
GROUP BY a.id;
Now, I'm trying to use it with JOOQ.
ctx.select(a.asterisk(),
coalesce(sum("How to get this ?")).as("columnCount"))
.from(a)
.leftJoin(b).on(someCondition)
.leftJoin(c).on(someCondition))
.leftJoin(d).on(someCondition)
.leftJoin(e).on(someCondition)
.groupBy(a.ID);
I'm having a hard time preparing the coalesce() part, and would really appreciate some help.
jOOQ's API is more strict about the distinction between Condition and Field<Boolean>, which means you cannot simply treat booleans as numbers as you can in MySQL. It's usually not a bad idea to be explicit about data types to prevent edge cases, so this strictness isn't necessarly a bad thing.
So, you can transform your booleans to integers as follows:
coalesce(
sum(
when(condition1.or(condition2), inline(1))
.else_(inline(0))
),
inline(0)
)
But even better than that, why not use a standard SQL FILTER clause, which can be emulated in MySQL using a COUNT(CASE ...) aggregate function:
count().filterWhere(condition1.or(condition2))
Ok, just to cut it short, I've done the actual JPQL without using any parameter first and it looks like this.
SELECT count(dt)
FROM transaction dt
WHERE dt.transactionType = 'TEST'
AND dt.date
BETWEEN FUNC('TO_DATE','01-2019','mm-yyyy')
AND FUNC('TO_DATE','02-2019','mm-yyyy')
This thing work! But the thing is now I need to make the transactionType and date as a parameter and this is how it looks like
SELECT count(dt)
FROM transaction dt
WHERE dt.transactionType = :transType
AND dt.date
BETWEEN FUNC('TO_DATE',:lastMonth,'mm-yyyy')
AND FUNC('TO_DATE',:nextMonth,'mm-yyyy')
So for :transType it's fine, but inside this FUNC() seems like I shouldnt put the parameter just like that and need some workaround. I've been googling and can't find any result.
The error was like this
You have attempted to set a parameter value using a name of
lastMonth,'mm-yyyy') that does not exist in the query string
As you can see, the parameter inside FUNC() take along the parameter behind it that meant for FUNC(). What did I miss? Enlighten me please.
Make sure you're using setString for the parameter type.
I always had difficulty with named parameters within JPA, depending upon how the query was created - try using ordinal parameters, eg: ?1 and set them by index.
I'd avoid FUNC as it can carry some major overhead if you're not extremely careful.
There's a workaround for this problem.
Initially, the simplified SQL as below:
SELECT count(*)
FROM table tb
WHERE tb.date between to_date('01-2020','mm-yyyy') and to_date('02-2020','mm-yyyy');
And by directly convert the simplified SQL to JPQL, it turns out as such:
SELECT count(tb)
FROM table tb
WHERE tb.date BETWEEN FUNC('TO_DATE','01-2020','mm-yyyy') AND FUNC('TO_DATE','02-2020','mm-yyyy')
But, the JPQL need to be dynamic as the date will not be static, so by using JPQL parameter to ensure this JPQL can be used at any date, instinctively I thought to use as such:
SELECT count(tb)
FROM table tb
WHERE tb.date BETWEEN FUNC('TO_DATE',:fromDate,'mm-yyyy') AND FUNC('TO_DATE',:toDate,'mm-yyyy')
But as my initial question when this thread first started, such JPQL will not work. So how did I found a workaround? Relatively quite simple actually.
Instead of using this to get the ranging date (as from sql wise I use to_date)
WHERE tb.date BETWEEN FUNC('TO_DATE','','') AND FUNC('TO_DATE','','')
I used this
WHERE FUNC('TO_CHAR','','') between (--fromDate) and (--toDate)
Which finally resulted in final working JPQL of
SELECT count(tb)
FROM table tb
WHERE FUNC('TO_CHAR',tb.date,'mm-yyyy') BETWEEN (:fromDate) AND (:toDate)
I have created the following query which is now in one of my java classes being used by Hibernate.
private static final String COUNT_INTERQUARTILE_SQL
= " SET #number_of_rows \\:= (SELECT COUNT(*) FROM carecube.visit)" +
" SET #quartile \\:= (ROUND(#number_of_rows*0.25))" +
" SET #medianquartile \\:= (ROUND(#number_of_rows*0.50))" +
" SET #sql_q1 \\:= (CONCAT('(SELECT 'Q1' AS quartile, visit.id FROM carecube.visit order by visit.id LIMIT 1 OFFSET ', #quartile, ')'))" +
" SET #sql \\:= (CONCAT_WS(' UNION ', #sql_q1, #sql_med))" +
" PREPARE stmt1 from #sql;" +
" EXECUTE stmt1;";`
The stack trace complains of a syntax errors for each line where I've set a mysql variable. Obviously it works in MySQL just fine.
I read that I can use double backslashes with assignments in Hibernate. This is the first time I've tried to use MySQL variables with Hibernate so am unsure if I'm missing anything out and whether 'PREPARE' and 'EXECUTE' are necessary?
Can someone with more knowledge point me where I am going wrong?
Also, where I am selecting Q1, I've placed that in single quotes, in MySQL workbench it is double quotes.
EDIT: I've added double quotes so hibernate doesn't throw a sissy fit with the assignments. I still can't for the life of me, figure out why I cannot just use '#sql' after i've prepared it.
EDIT: I receive the following error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'PREPARE stmt1 from #sql_max; EXECUTE stmt1' at line 1
Thanks
I'm not sure if this is possible, but in my opinion this doesn't make much sense.
Some people have asked similar questions (they have some code samples in the answers if you want to check):
How to use Mysql variables with Hibernate?
How can I use MySQL assign operator(:=) in hibernate native query?
Hibernate is an ORM (Object Relational Mapping), so it's supposed to convert data between incompatible type systems (data from database) in objects. As far as I could understand your query, you're returning a COUNT, so it would be just one single result (row) and one single column, is that right?
Due the complexity of your query, I would say there are some options I could think of:
Use Criteria or HQL to run each query using Hibernate and then in Java work with the logic to have your desired result (may become much slower than the query in MySQL)
Create a VIEW with this SELECT (if possible), map it into an object as an #Entity and query directly to it
Create a FUNCTION/PROCEDURE (this is possible) and call it using CallableStatement
I like to think that the two elements here (Hibernate x Database) should have a well define responsibility in the project. First of all, I would try to use only Criteria/HQL for the queries (to use object properties), but if needed to use SQL I would keep all queries ANSI to allow interoperability. If it's not possible, I would create an object in the database to return what I want (view/procedure/function).
Mixing specific database provider code in the query, like your example, doesn't look a good practice.
If possible, I would definitely go for option 2. If it can't be done, surely for number 3.
Hope it's somehow helpful.
I have to execute below query through JDBC call
select primaryid from data where name in ("abc", adc", "anx");
Issue is inside in clause I have to pass 11000 strings. Can I use prepared statement here? Or any other solution any one can suggest. I dont want to execute the query for each record, as it is consuming time. I need to run this query in very less time.
I am reading the strings from an XML file using DOMParser. and I am using sql server db.
I'm just wondering why you would need to have a manual set of 11,000 items where you need to specify each item. It sounds like you need to bring the data into a staging table
(surely it's not been selected from the UI..?), then join to that to get your desired resultset.
Using an IN clause with 11k literal values is a really bad idea - off the top of my head, I know one major RDBMS (Oracle) that doesn't support more than 1k values in the IN list.
What you can do instead:
create some kind of (temporary) table T_NAMES to hold your names; if your RDBMS doesn't support "real" (session-specific) temporary tables, you'll have to add some kind of session ID
fill this table with the names you're looking for
modify your query to use the temporary table instead of the IN list: select primaryid from data where name in (select name from T_NAMES where session_id = ?session_id) or (probably even better) select primaryid from data join t_names on data.name = t_names.name and t_names.session_id = ?session_id (here, ?session_id denotes the bind variable used to pass your session id)
A prepared statement will need to know the number of arguments in advance - something along the lines of :
PreparedStatement stmt = conn.prepareStatement(
"select id, name from users where id in (?, ?, ?)");
stmt.setInt(1);
stmt.setInt(2);
stmt.setInt(3);
11,000 is a large number of parameters. It may be easiest to use a 'batch' approach as described here (in summary - looping over your parameters, using a prepared statement
each time)
Note - if your 11,000 strings are the result of an earlier database select, then the best approach is to write a stored procedure to do the whole calculation in the database (avoiding passing the 11,000 strings back and forth with your code)
You can merge all your parameter strings into one bitg string separating by ';' char
bigStrParameter=";abc;adc;anx;"
And use LOCATE to find substring.
select primaryid from data where LOCATE(concat(';',name,';'),?)>=0;
i have a table containing 15+ million records in oracle. its sort of a log table which has a created_ts column of type "date" . i have a simple "non-unique" type index on created_ts column.
i have a simple range query :
select * from table1 where created_ts >= ? and created_ts <= ?;
when i run this query from SQLPlus or SQL Developer etc like this :
select * from table1
where created_ts >= TO_DATE( '2009-11-10 00:00:00', 'YYYY-MM-DD HH24:MI:SS')
and created_ts <= TO_DATE( '2009-11-10 23:59:59', 'YYYY-MM-DD HH24:MI:SS');
the query returns within 1-2 second max.
but when I run the exact same query in java over JDBC and set the corresponding "?" params using java.sql.Timestamp object . the query takes long time . Analyzing the oracle process it goes for full table scan and doesnt use the index.
the jdbc driver i am using is ojdbc5 11.1.0.7.0
Can any one please help .. how to create the index correctly so that it uses the index.
My problem was resolved when i used "oracle.sql.DATE" objects to set the bind variables instead of "java.sql.timestamp" . The query used the index and executed almost within 1-2 seconds.
Thanks to all who replied and helped.
But its problematic for me as this solution is DB dependent and my app receives DB connection and query as param and load and process data in a generic way. The DB connection can be of any RDBMS like oracle, mysql, etc.
This is classic behaviour for an implicit datatype conversion. Because the database is having to convert the datatype of the column it cannot use any index on that column.
In your case I suspect this is due to your use of java.sql.Timestamp. Would it be possible to use the equivalent type from the Oracle datatypes package, oracle.sql.Timestamp? Obviously that may have some knock-on effects but I think you should at least test it, to see whether that solves your problem.
The difference may because of bind variables vs. literal values. You are not comparing the same things.
Try this in SQL*Plus:-
explain plan for
select * from table1 where created_ts >= :1 and created_ts <= :2;
set markup html preformat on
set linesize 100
set pagesize 0
select plan_table_output
from table(dbms_xplan.display('plan_table',null,'serial'));
This will show you the plan Oracle will pick when using bind variables. In this scenario, Oracle has to make up a plan before you have provided values for your date range. It does not know if you are selecting only a small fraction of the data or all of it. If this has the same plan (full scan?) as your plan from java, at least you konw what is happening.
Then, you could consider:-
Enabling bind peeking (but only after testing this does not cause anything else to go bad)
Carefully binding literal values from java in a way that does not allow SQL injection
Putting a hint in the statement to indicate it should use the index you want it to.
You should try a hint of the form /*+ USE_INDEX(table_name, index_name) */
My guess is that the optimizer is choosing a full table scan because it sees that as the best option in absence of knowing the bind values.