i have a simple stored procedure. i use this to show data from my db. i want to optimize this query because i think this is slow. 31 seconds for 220.000 rows. so this is my query
BEGIN
sql_base :=
'SELECT
p.id
,p.cis
,p.account_no
,p.name
,p.address
,p.village_name
,p.postal_code
,p.subdistrict_name
,p.regency_name
,p.province_name
,p.country_name
,m.group_id
,i.name
,m.id
,m.name
,m.address
,m.village_name
,m.postal_code
,m.subdistrict_name
,m.regency_name
,m.province_name
,m.country_name
,listagg (mMcc.name, '','') within group (order by mMcc.name) merchantMccName
,o.mid
,o.name
,o.mcc_id outletMccId
,REPLACE(REPLACE(o.jenis_outlet,''ecomm'',''e-commerce''),''retail'',''toko'') jenisOutlet
,o.MODIFIEDDATE
,o.CREATEDDATE
,o.CC_testing_ACCOUNTNO outletCcAccountNo
,o.DEBIT_testing_ACCNO outletDebitAccountNo
,MCC.name outletMccName
,o.MCC_CODE outletMccCode
,m.code merchantCode
,p.jenisn abah jenisn abah
,p.segmenn abah segmenn abah
,p.IDENTITY_NO identityNo
,p.IDENTITY_TYPE identityType
,o.status status
,NVL(o.ISOUTLET, 0) isOutlet
,NVL(p.IS_ACCOUNT_testing, 0) isAccounttesting
,p.ACCOUNT_OTHERBANK_NO accountOtherbankNo
,p.ACCOUNT_OTHERBANK_BANKNAME accountOtherbankBankname
,p.ACCOUNT_OTHERBANK_NO pemilikOtherBankAccountNo
,o.ADDRESS1 outletAddress1
,o.ADDRESS2 outletAddress2
,o.VILLAGE_NAME outletVillageName
,o.POSTAL_CODE outletPostalCode
,o.SUBDISTRICT_NAME outletSubdistrictName
,o.REGENCY_NAME outletRegencyName
,o.PROVINCE_NAME outletProvinceName
,C E WHEN NVL(o.IS_CC_TESTING,0)=1 THEN ''TESTING'' ELSE TO_CHAR(o.CC_OTHERBANK_NAME) END outletCcBankName
,C E WHEN NVL(o.IS_DEBIT_TESTING,0)=1 THEN ''TESTING'' ELSE TO_CHAR(o.CC_OTHERBANK_NAME) END outletDebitBankName
,NVL(o.IS_CC_OTHERBANK,0) isCcOtherBank
,NVL(o.IS_CC_TESTING,0) is_cc_testing
,NVL(o.IS_DEBIT_testing,0) is_debit_testing
,o.CC_OTHERBANK_ACCNO ccOtherBankAccNo
,o.QRIS_STATIS_NMID nmid
,o.AGENT_BANK_CODE outletAgentBank
FROM PEMILIK p
LEFT JOIN mytable m ON p.ID = m.PEM_ID
LEFT JOIN mytable2 i ON m.GROUP_ID = i.ID
INNER JOIN mytable3 o ON m.ID = o.MERCH_ID
LEFT JOIN mytable4 mMcc ON m.ID = mMcc.MERCH_ID
LEFT JOIN mytable5 mcc ON MCC.ID = o.MCC_ID
GROUP BY
*SAME LIKE SELECT*;
final_result:='SELECT rec.*, count (*) over ()CountData FROM ('
|| sql_base || ') rec '
||WHERE_Q
||ORDERBY_Q||' '||PAGING_Q ;
OPEN O_RESULT_REC FOR
final_result;
i already tried to change the query but still i think my query still not good. any tips to improve my query? should i change the connection in java? or this is just query problem?
Related
tl;dr I have a long native query in Hibernate, that is split in multiple queries (with table as ...). This Query use inner joins that not work in hibernate (empty result), but work excellent in the SQL Console. Also if I copy/paste the query from the hibernate logs into the console.
with params as (select ca.id as calculation_id, chart_id, range_size, range_size * interval '1' minute as size
from calculation ca
inner join chart c on c.id = ca.chart_id
inner join time_range tr on tr.id = c.time_range_id
where ca.id = :calculationid),
tuple_diff as (select t.calculation_id,
t.id,
t.ohlc_id,
t.time, t.time - lag(t.time, 1) over (order by t.time) as diff
from tuple t inner join params p
on t.calculation_id = p.calculation_id),
ohlc_diff as (select o.chart_id,
o.id,
o.time,
o.time - lag(o.time, 1) over (order by o.time) as diff
from ohlc o inner join params p on o.chart_id = p.chart_id),
tuple_filtered as (select id, ohlc_id, time, diff, p.size, p.range_size
from tuple_diff t
inner join params p on chart_id = p.chart_id
where diff <> p.size),
ohlc_filtered as (select o.chart_id, o.id as ohlc_id, time
from ohlc_diff o
inner join params p on o.chart_id = p.chart_id
where o.diff <> p.size)
select t.time as time, t.range_size as size
from tuple_filtered t
left join ohlc_filtered o on t.time = o.time
where o.ohlc_id is null
order by t.time;
The long version, I have a microservice, generated with jhipster and the project runs productive with PostgreSQL and in development mode with a H2 Database. I have ohlc stockexchange time series and calculate calculations with an other microservice and store the computation in a tuple table. Sometimes I have holes in the timeline and want to recalculate them. I need to get the time difference to the last calculation. This is not an easy SQL Query, because you need to access the previous row like in Excel. I use the lag function to solve this problem. But the lag function is not the reason.
I generate a test project with an integration test to analyze the problem. The Junit Test is de.bitc.se.service.CalculationServiceIT and run with a Postgresql Testcontainer. The test also fill the SQL tables with testdata. I start to split the query in small parts.
with params as (select ca.id as ca_id,
chart_id, range_size,
range_size * interval '1' minute as size
from calculation ca
inner join chart c on c.id = ca.chart_id
inner join time_range tr on tr.id = c.time_range_id
where ca.id = :calculationid)
select ca_id, chart_id, range_size
from params
order by ca_id;
This part works like the console.
with tuple_diff as (select t.calculation_id,
t.id,
t.ohlc_id,
t.time,
t.time - lag(t.time, 1) over (order by t.time) as diff
from tuple t
where calculation_id = :calculationid)
select calculation_id, time
from tuple_diff
order by calculation_id;
The second query too, but when I try to join the query's, I got an empty result:
with params as (select ca.id as ca_id,
chart_id,
range_size, range_size * interval '1' minute as size
from calculation ca
inner join chart c on c.id = ca.chart_id
inner join time_range tr on tr.id = c.time_range_id
where ca.id = :calculationid),
tuple_diff as (select t.calculation_id,
t.id,
t.ohlc_id,
t.time,
t.time - lag(t.time, 1) over (order by t.time) as diff
from tuple t
inner join params p on p.ca_id = t.calculation_id
where calculation_id = ca_id)
select calculation_id
from tuple_diff
order by calculation_id;
When I try to join the param query (one result) with the tuple table I got no result. In the SQL console I got a result of multiple lines. I used the EntityManager to exclude Spring Data code in the splitted queries in the integration test.
I have no idea why I got no result. Is it a bug in Hibernate or a mistake?
I am working on a Spring web application that utilizes hibernate to connect to a DB2 database. I am try to optimize a service method that gets called may times during a wed service call by reducing the number of DB queries.
So my question is whether or not this query
SELECT DISTINCT a.* FROM TABLE_A a
LEFT JOIN TABLE_B b ON a.ID = b.FK_ID
LEFT JOIN TABLE_C c ON a.ID = c.FK_ID
LEFT JOIN TABLE_D d ON c.DATA_RQST_ID = d.ID
WHERE (b.REQUEST_ID = 1234 AND b.TYPE = 'TYPE_A')
OR (c.REQUEST_ID = 1234 AND (c.TYPE = 'TYPE_A' OR c.TYPE = 'TYPE_B'))
is equivalent/better then this query
SELECT * FROM TABLE_A a
WHERE a.ID IN
(
SELECT b.FK_ID FROM TABLE_B b
WHERE b.REQUEST_ID = 1234 AND eb.TYPE = 'TYPE_A'
)
OR a.ID IN
(
SELECT c.FK_ID FROM TABLE_C
WHERE ( c.REQUEST_ID = 1234 AND c.TYPE = 'TYPE_A' )
OR
(
c.TYPE = 'TYPE_B' AND c.REQUEST_ID IN
(
SELECT d.ID FROM TABLE_D d
WHERE d.REQUEST_ID = 1234 AND v.TYPE = 'TYPE_A'
)
)
)
or is there a better option?
Both queries seem to run about the same time (<50ms) but that may depend on the resulting data. I would need to test more to know for sure.
The point of these two queries is for one of them to replace three other queries where their resulting data is processed in Java to get the required data.
I will also have to be able to convert the SQL query to HQL. I was struggling to convert the first query.
I have a feeling that I maybe wasting my time since the java objects for tables B and C are a one-to-many relationship in the object for table A and they are load by hibernate anyway. Meaning I may not be saving anytime in the long run. Is my thinking here correct?
Thanks!
If I understand correctly, exists would be the best solution:
SELECT a.*
FROM TABLE_A a
WHERE EXISTS (SELECT 1
FROM TABLE_B b
WHERE a.ID = b.FK_ID AND b.REQUEST_ID = 1234 AND b.TYPE = 'TYPE_A'
) OR
EXISTS (SELECT 1
FROM TABLE_C c JOIN
TABLE_D d
ON c.DATA_RQST_ID = d.ID
WHERE a.ID = c.FK_ID AND
c.REQUEST_ID = 1234 AND
(c.TYPE IN ('TYPE_A', 'TYPE_B'))
);
One big gain is just in removing the select distinct.
Then for performance, you want indexes on table_b(fk_id, request_id, type_id) and table_c(fk_id, request_id, type, DATA_RQST_ID) and table_d(id).
I got this exception when calling the method that calls this query:
SELECT s FROM Survey s
WHERE s.idsurvey NOT IN
(SELECT DISTINCT s FROM Useranswer ua
JOIN ua.iduser u
JOIN ua.idanswer a
JOIN a.idquestion q
JOIN q.idsurvey s
WHERE
ua.iduser.iduser = u.iduser
AND ua.idanswer.idanswer = a.idanswer
AND a.idquestion.idquestion = q.idquestion
AND q.idsurvey.idsurvey = s.idsurvey
AND u.iduser = :iduser
)
order by s.dateEnd
Any help?
The query seems fine.
I was trying to solve this problem for almost 2 hours.
I've found a solution using native sql.
String query = "select s.IDSURVEY, s.DATE_END from survey s where s.IDSURVEY not IN (select distinct s.IDSURVEY from survey s join question q on s.IDSURVEY = q.IDSURVEY join answer a on q.IDQUESTION = a.IDQUESTION join useranswer ua on a.IDANSWER = ua.IDANSWER where ua.IDUSER = "+iduser+") order by s.DATE_END";
return (List<Survey>)em.createNativeQuery(query).getResultList();
Can someone help me?
I got that SQL query and need to represent that in JPQL, but i faced trouble with right join:
SELECT alrt.*
FROM
REACTION.ALERT alrt, REACTION.INVESTIGATION inv,
REACTION.CLASSIFICATION_TYPE clst, REACTION.FRAUD_TYPE frt,
REACTION.TRANS trns, REACTION.CARD crd
WHERE
alrt.ISS_INST IN(1201, 1101) AND
alrt.MODULE_TYPE = 0 AND
0 < (SELECT COUNT(*) FROM REACTION.INVESTIGATION WHERE REACTION.INVESTIGATION.ALERT_ID = alrt.ID) AND
inv.CLASSIFICATION_TYPE_ID IS NOT NULL AND
clst.CLASSIFICATION_TYPE = 10 AND
(alrt.REMINDER_USER_LOGIN = 'qwr' OR alrt.REMINDER_USER_LOGIN IS NULL) AND
alrt.ID = inv.ALERT_ID AND
alrt.TRANSACTION_ID = trns.ID(+) AND inv.CLASSIFICATION_TYPE_ID =
clst.ID AND inv.FRAUD_TYPE_ID = frt.ID(+) AND trns.HPAN = crd.HPAN(+);
After read tutorials and docs i create that JPQL query:
SELECT alrt
FROM INVESTIGATION inv
JOIN inv.CLASSIFICATION_TYPE_ID clst
RIGHT JOIN inv.FRAUD_TYPE_ID frt
JOIN inv.alert_id alrt
RIGHT JOIN alrt.transactio_id trns
RIGHT JOIN trns.HPAN crd
WHERE
alrt.ISS_INST IN(1201, 1101) AND
alrt.MODULE_TYPE = 0 AND 0 < (SELECT COUNT(inv1) FROM INVESTIGATION inv1 WHERE inv1.ALERT_ID = alrt.ID) AND
inv.CLASSIFICATION_TYPE_ID IS NOT NULL AND
clst.CLASSIFICATION_TYPE = 2 AND
(alrt.REMINDER_USER_LOGIN = 'qwr' OR alrt.REMINDER_USER_LOGIN IS NULL);
But i got error then try to execute that. Can someone tell what i did wrong pls?
If it make sense i use JPA 1.0 version
Your error is join ID, but not OR-mapping object.
Try this:
JOIN inv.CLASSIFICATIONTYPE(your mapping property name) clst
If u do not have mapping relationship between entities, u'd better to user join table i.o. OR-mapping.
Read more information on JPQL
I'm learning JPQL and I wonder how to translate this query into JPQL ?
I have entities Astre for astres table and Planetefor planetes table. generate_series() is PostgreSQL function.
SELECT t1.galaxie, t1.ss, t1.position FROM
(
SELECT galaxie, ss, position FROM
generate_series(1, 9) galaxie
CROSS JOIN generate_series(1, 500) ss
CROSS JOIN generate_series(1, 15) position
) t1
LEFT JOIN
(
SELECT a.galaxie, a.ss FROM astres a
INNER JOIN planetes p
ON a.id=p.astre_id
WHERE p.planete_mere=TRUE
GROUP BY a.galaxie, a.ss
HAVING COUNT(*)=3
) t2
ON t1.galaxie = t2.galaxie
AND t1.ss = t2.ss
LEFT JOIN
(
SELECT a.galaxie, a.ss, a.position FROM astres a
INNER JOIN planetes p
ON a.id=p.astre_id
) t3
ON t1.galaxie=t3.galaxie
AND t1.ss=t3.ss
AND t1.position=t3.position
WHERE t2.galaxie IS NULL
AND t3.galaxie IS NULL
ORDER BY t1.galaxie, t1.ss, t1.position
I can supply entities code if needed.
Thanks for help.