I have below the native sql query. i am using Oracle database.
select
*
from
(select
row_.*,
rownum rownumber
from
(select
colmn1,
colmn2,
colmn3,
colmn4,
colmn5,
colmn6,
from
Table5
where
colmn5 In (
'19901','10001'
)
order by
colmn1 ) row_ )
Where
Rownumber <= 50000
and rownumber > 0
Above query returns 50000 records. if i execute above query in sqldeveloper, it takes only 30 seconds but in the spring and hibernate integrated application it takes 15 minutes. How can i improve the performance?
Thanks!
You have two inner selects. An inner select always is a possible source of bad performance, because it might hinder the database from finding the optimal search strategy.
As far ar I can see you use the inner selects only for handling the row number. If you use only the most inner select and handle the row number on java/hibernate level, then you'll get a much better performance.
You only use this select
select colmn1, colmn2, colmn3, colmn4, colmn5, colmn6,
from Table5
where colmn5 In ('19901','10001')
order by colmn1
which, as it does not have any database specialities any more easily can be replaced by an HQL statement and so making your program independent of the used database (Java class and property names should be replaced by the real ones):
from Table5_Class
where colmn5_Prop in ('19901','10001')
order by colmn1_prop
Then you replace your where condition Where Rownumber <= 50000 and rownumber > 0 by the hibernate methods Query.setMaxResults(50000) and Query.setFirstResult(0) (remark: setFirstResult(0) is superfluous as row 0 always is the first one, but I guess you also want to get the next 50000 rows, and then you can use setFirstResult(n)).
If you need the rownumber as a parameter then you can use the index of the resulting List for this.
P. S: I can't tell you why your select is so much faster in the SQL developer than in Hibernate.
Related
Hey all I have the query below that I wrote when I was using SQLite:
SELECT
ad.ID,ad.Script_Name,ad.Current_Status,
ad.Issues_found_during_run,ad.Testers,
ad.Run_Date,ad.Tools,u.fTag,u.role,u.avatar
FROM
allData ad
INNER JOIN
users u
ON
u.fTag = ad.lastUserWhoUpdated
GROUP BY
ad.ID
ORDER BY
ad.ID ASC
That produces the 6 records I would imagine would come up with that query.
However, I have since moved to using Oracle database. This is my information on the Oracle server I am using:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
So when I convert the SQLite data to Oracle and run the same query with just a modification for the GROUP BY since it seems Oracle wants all the names that are in the SELECT statement and not just the one I need to group by:
SELECT
ad.ID,ad.Script_Name,ad.Current_Status,
ad.Issues_found_during_run,ad.Testers,
ad.Run_Date,ad.Tools,u.fTag,u.role,u.avatar
FROM
allData ad
INNER JOIN
users u
ON
u.fTag = ad.lastUserWhoUpdated
GROUP BY
ad.ID,ad.Script_Name,ad.Current_Status,ad.Issues_found_during_run,ad.Testers,ad.Run_Date,ad.Tools,u.fTag,u.role,u.avatar
ORDER BY
ad.ID ASC;
That above produces 0 records when ran in SQL Developer. So, what would Oracle need me to do in order to fix this so it pulls the 6 records as did the SQLite version?
You can use something like this:
SELECT
ad.ID,MAX(ad.Script_Name), MAX(ad.Current_Status),
MAX(ad.Issues_found_during_run), MAX(ad.Testers),
MAX(ad.Run_Date), MAX(ad.Tools), MAX(u.fTag),MAX(u.role),MAX(u.avatar)
FROM
allData ad
INNER JOIN
users u
ON
u.fTag = ad.lastUserWhoUpdated
GROUP BY
ad.ID
ORDER BY
ad.ID ASC;
More on Aggregrate functions here:
https://docs.oracle.com/database/121/SQLRF/functions003.htm
It doesn't make sense (to me, at least). There's no WHERE clause which would restrict number of rows returned. GROUP BY certainly doesn't have to do anything with that. The only suspicious thing is
on u.fTag = ad.lastUserWhoUpdated
which caused no rows to be retrieved.
Therefore:
are you sure that there are matching values in those two tables?
check those two columns' datatypes, especially if one (or both) are CHAR (I mean CHAR, not VARCHAR2) which right-pads values with blanks up to the whole column size, so you might try with
on trim(u.fTag) = trim(ad.lastUserWhoUpdated)
which has its drawbacks (index won't be used, unless it is a function-based) so - if it turns out that it is the case, modify datatype to VARCHAR2.
I've got some quarrels with hibernate.
My query, yet optimized, is quite heavy. One of my optimization consist on limiting the resultset returned.
So with hibernate I've used the method setMaxResultSet, but I hit the same problem described in this post:
Hibernate: Pagination with setFirstResult and setMaxResult
(the issue is that using setMaxResultSet hibernate in some cases wrap the query like this:
select * from (your query) where rownum <= :rownum)
So, the solution in that case was to add an orderBy, bu I've millions of records and an orderBy kills the execution time of the query.
I've managed to overcome the problem using the createNativeQuery and passing the exact query I need (something like "my query where rownum <= :rownum" instead of "select * from (your query) where rownum <= :rownum", and goodbye portability), but honestly I don't get why Hibernate acts like this...
As the previous post suggests, hibernate resolve an SQL like that as long as your query "is not stable" because, if I haven't misunderstand, the order of the records may not be the same between two executions, but I don't get how that method could solve this stability problem.
I am using the same pagination in hibernate.the HQL is given below.it may be useful for you.
(i) initially you should use this Query
List<Object> Entity_Cls_Lst= Objclass.createQuery("from library where book_id>Book_ID order by book_id").list();
(ii) after scrolling you should take last result data's Book_ID and pass to the query in where condition.
List<Object> Entity_Cls_Lst= Objclass.createQuery("from library where book_id>Book_ID order by book_id").setMaxResults(MAX_RECORDS).list();
I have the following prepared statement in java:
with main_select as
(select request_id,rownum iden
from
(select request_id
from queue_requests
where request_status = 0 and
date_requested <= sysdate and
mod(request_id,?) = ?
order by request_priority desc, oper_id, date_requested)
where rownum < ?)
select *
from queue_requests qr, main_select ms
where qr.request_id in ms.request_id
order by ms.iden for update skip locked;
It doesn't execute:
ORA-02014: cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc.
I'll try to explain why i need all the select statements:
the first (inner) select obtains the data i need
the second one limits the number of lines to a number (i can't put it in the first select, because oracle firstly limits the results and only after orders them, which is not what i want)
the third (outside with) select preserves the order (i tried using 3 nested selects - so, no with clause - but i can't find a way to preserve the order in this case). Also, it should lock the lines in the queue_requests table, but because i selected data from the with clause, it gives the above error.
So, i want to select data from queue_requests, keep the first x lines, preserve the order of the select and lock the lines.
Is there a way to do it?
The problem seems to be, that you want to set a lock on the result of main_select. I would just guess, that you can do select for update in the select in the with clause like:
with main_select as
(select request_id,rownum iden
from (subselect)
where rownum < ?
for update skip locked)
But as I said lucky guess.
I am generating reports in my system but some reports are returning a huge number of results. To remedy this I hit the database with a count first, then in my code I check if this count is above a certain threshold (e.g. 2000), then don't generate the report.
This is fine in most cases but some reports have over a million results, this means it takes the count a good few seconds to return a result.
Ideally, what I would like to do is put my threshold (2000) into my sql statement, stop the count if it reaches this value, and return some value (e.g. true or false, 0 or 1, anything) so that I know it has exceeded its limit. Is this possible in sql, so far I cannot find a solution.
Pseudocode: select count(1) from table while count <= threshold
I am working with java, hibernate, sql server 2005.
Any help will be much appreciated.
Regards,
Eamon
I think something like this should work:
select count(1) from (
select top 2000 *
from table
where ...
)
I don't like the idea of running the query twice, once to check the row count and then again to actually get the data. As a result, if you want to limit the report to 2000 rows, just do something like this:
SELECT TOP 2001
...
from...
in the application check if the rowcount is greater than your limit >2000 return an error, or skip displaying it.
try something like limit clause? this page suggests how that could be done in sql server: http://blogs.msdn.com/b/sqlserver/archive/2006/10/25/limit-in-sql-server.aspx
I'm not really aware of equivalent in Hibernate for TOP instruction, but even if there exists first you will have to retrieve the 2000 rows and then count it what probably will be slow as is now. So you can featch allredy 2001 row and then validate it in application.
But I think that what you should optimize is the query by itself.
If in the query does not exists some outer join just specify index column in statement
SELECT count(*) from test1 inner join test2 on test1.id = test2.id;
888 ms
SELECT count(test.id) from test1 inner join test2 on test1.id = test2.id;
88ms
relation 35 rows in test1 match to 1 in test2;
rows 1252080;
i have a table containing 15+ million records in oracle. its sort of a log table which has a created_ts column of type "date" . i have a simple "non-unique" type index on created_ts column.
i have a simple range query :
select * from table1 where created_ts >= ? and created_ts <= ?;
when i run this query from SQLPlus or SQL Developer etc like this :
select * from table1
where created_ts >= TO_DATE( '2009-11-10 00:00:00', 'YYYY-MM-DD HH24:MI:SS')
and created_ts <= TO_DATE( '2009-11-10 23:59:59', 'YYYY-MM-DD HH24:MI:SS');
the query returns within 1-2 second max.
but when I run the exact same query in java over JDBC and set the corresponding "?" params using java.sql.Timestamp object . the query takes long time . Analyzing the oracle process it goes for full table scan and doesnt use the index.
the jdbc driver i am using is ojdbc5 11.1.0.7.0
Can any one please help .. how to create the index correctly so that it uses the index.
My problem was resolved when i used "oracle.sql.DATE" objects to set the bind variables instead of "java.sql.timestamp" . The query used the index and executed almost within 1-2 seconds.
Thanks to all who replied and helped.
But its problematic for me as this solution is DB dependent and my app receives DB connection and query as param and load and process data in a generic way. The DB connection can be of any RDBMS like oracle, mysql, etc.
This is classic behaviour for an implicit datatype conversion. Because the database is having to convert the datatype of the column it cannot use any index on that column.
In your case I suspect this is due to your use of java.sql.Timestamp. Would it be possible to use the equivalent type from the Oracle datatypes package, oracle.sql.Timestamp? Obviously that may have some knock-on effects but I think you should at least test it, to see whether that solves your problem.
The difference may because of bind variables vs. literal values. You are not comparing the same things.
Try this in SQL*Plus:-
explain plan for
select * from table1 where created_ts >= :1 and created_ts <= :2;
set markup html preformat on
set linesize 100
set pagesize 0
select plan_table_output
from table(dbms_xplan.display('plan_table',null,'serial'));
This will show you the plan Oracle will pick when using bind variables. In this scenario, Oracle has to make up a plan before you have provided values for your date range. It does not know if you are selecting only a small fraction of the data or all of it. If this has the same plan (full scan?) as your plan from java, at least you konw what is happening.
Then, you could consider:-
Enabling bind peeking (but only after testing this does not cause anything else to go bad)
Carefully binding literal values from java in a way that does not allow SQL injection
Putting a hint in the statement to indicate it should use the index you want it to.
You should try a hint of the form /*+ USE_INDEX(table_name, index_name) */
My guess is that the optimizer is choosing a full table scan because it sees that as the best option in absence of knowing the bind values.