I have a JDBI query that is as simple as it can be
#Override
#SqlQuery("SELECT COUNT(*) FROM " + TABLE_NAME + " WHERE (" + COLUMN_MODERATOR_CATEGORY_ID
+ " in (<categories>) OR " + COLUMN_EXPERT_CATEGORY_ID
+ " in (<categories>)) AND (" + COLUMN_STATUS + " <> 0)")
long getIdeasCountInCategories(#BindIn("categories") List<Long> categories);
The <> 0 fails with a syntax error at the end of input.... It works as soon as I change it to > 0 (which also serves the purpose).
Using Java 1.8.0 and Postgres 9.6. Please let me know if any more info is needed.
If you absolutely can't change the SQL query (e.g. switch operator to != or >) then you need to escape < with preceding \\.
Related
Here is my requirements, there are 3 Kafka topics:
topic1-device & topic2-consumer & topic3-order.
we are going to calculate the order(amount) per device from the past 12hours using flink SQL.
I did the following things:
Regiter 3 tables corresponding 3 kafka topics.
// java code
String creConsumer = "CREATE TABLE " + consumerTable + " (" +
" deviceId STRING" +
",deviceFingerprintHash STRING" +
",consumer ROW(consumerUuid STRING)" +
",eventInfo ROW<eventTime BIGINT>" +
",id BIGINT" +
",ts AS TO_TIMESTAMP(FROM_UNIXTIME(eventInfo.eventTime/1000, 'yyyy-MM-dd HH:mm:ss'))" +
",WATERMARK FOR ts AS ts - INTERVAL '5' SECOND" +
") WITH (...)";
String createToken = "CREATE TABLE " + orderTokenTable + " (" +
"sessionId BIGINT" +
",token STRING" +
",eventInfo ROW(eventTime BIGINT)" +
",ts AS TO_TIMESTAMP(FROM_UNIXTIME(eventInfo.eventTime/1000, 'yyyy-MM-dd HH:mm:ss'))" +
",WATERMARK FOR ts AS ts - INTERVAL '5' SECOND" +
") WITH (...)";
String createTransaction = "CREATE TABLE " + orderTransactionTable + " (" +
"orderTransactionId BIGINT" +
",consumer ROW(`consumerUuid` STRING)" +
",token STRING" +
",countryCode STRING" +
",consumerTotalAmount ROW<amount STRING>" +
",status STRING" +
",eventInfo ROW<eventTime BIGINT>" +
",ts AS TO_TIMESTAMP(FROM_UNIXTIME(eventInfo.eventTime/1000, 'yyyy-MM-dd HH:mm:ss'))" +
",WATERMARK FOR ts AS withOffset(ts,1000)" +
") WITH (...)";
Join the 3 tables and generate a View:
// java code
String createWideTable = "CREATE VIEW view_order_consumer AS " +
"SELECT " +
" otc.eventTime " +
",otc.orderTransactionId " +
",otc.token" +
",otc.consumerUuid " +
",otc.countryCode " +
",CAST(otc.amount AS DOUBLE) AS amount " +
",otc.status " +
",csc.deviceId " +
",csc.deviceFingerprintHash " +
",otc.ts " +
"FROM " +
" order_transaction_completed otc " +
"INNER JOIN order_token_added ota " +
" ON (otc.token=ota.token AND otc.ts BETWEEN ota.ts - INTERVAL '10' DAY AND ota.ts + INTERVAL '10' DAY)" +
"INNER JOIN consumer_session_created csc " +
" ON (ota.sessionId=csc.id AND csc.ts BETWEEN otc.ts - INTERVAL '10' DAY AND otc.ts) ";
Do the aggregation job using window agg(Hop Window):
// SQL
select deviceId
,HOP_START(voc.ts, INTERVAL '5' SECOND , INTERVAL '10' SECOND)
,count(1) as cnt
from consumer_session_created as voc
group by HOP(voc.ts, INTERVAL '5' SECOND , INTERVAL '10' SECOND)
,deviceId
Output result table data:
// java
DataStream<Tuple2<Boolean, Row>> retractResultStream = tableEnvironment
.toRetractStream(table, Row.class);
retractResultStream.print();
However I can't get any result(No error messages).
Change the sql to :
// SQL
select * from view_order_consumer
Result:
6> (true,1628564685939,100100113280,002.qa2dtem5k6umlokop1boud4c8p77c9lhclb8i5ug0na383ed,94e44b95-223b-4479-82b9-b4f710f7f8c3,US,10.0,APPROVED,740baadd20e544e8bdcdc8d2a76cbdc9,c718225f5f1d4876ffc1ce2bb5ab3852,2021-08-10T11:04:45)
6> (true,1628564687358,100100113280,002.qa2dtem5k6umlokop1boud4c8p77c9lhclb8i5ug0na383ed,94e44b95-223b-4479-82b9-b4f710f7f8c3,US,11.0,APPROVED,740baadd20e544e8bdcdc8d2a76cbdc9,c718225f5f1d4876ffc1ce2bb5ab3852,2021-08-10T11:04:47)
6> (true,1628564688364,100100113280,002.qa2dtem5k6umlokop1boud4c8p77c9lhclb8i5ug0na383ed,94e44b95-223b-4479-82b9-b4f710f7f8c3,US,12.0,APPROVED,740baadd20e544e8bdcdc8d2a76cbdc9,c718225f5f1d4876ffc1ce2bb5ab3852,2021-08-10T11:04:48)
After some research, I found that the time-attr will be dropped after join operator in Flink.
Can anyone tell me how can I get the correct data using FLink-sql ?
I'm not sure what's wrong, but here are some ideas:
FWIW, it's easier to iterate and debug these situations using an interactive tool, such as the SQL client that ships with Flink, or the one built into Ververica Platform.
You can use "explain ..." to see the plan that has been produced for the query. This often provides some clues.
I'm not sure what to expect from WATERMARK FOR ts AS withOffset(ts,1000). Are you sure that works? In most cases where a Flink job or SQL query produces no results, the cause turns out to be a problem with the watermarking.
I would try tearing apart the 3-way join, and instead do a sequence of two joins. That may produce a different plan; possibly one that works better.
I have a query in jpql that normally walk with a reduced data count, the problem is when the table to them over 600000 data records.
I use spring data, with an entity that contains no relation (no OneToMany, OneToOne, ManyToOne .....)with oracle as database..
I used JpaRepository, crudRepository, I even tried with JDBC directly, the return takes between 8 min up to 30 min.
I thought the problem was coming from the request, so I tried a findAll () and the processing time remained the same.
I changed the settings of the JVM -Xmx and -Xms to give more memory, but nothing helps.
Here is the request that I make:
public interface TestRepository extends CrudRepository {
#Query(value = "select new Test(CONCAT(t.date1, t.stringTarget, t.numInfo), t.name, t.phone, t.numInfo, t.date1, t.cotations, t.stringTarget, p.code, p.design)"
+ " from Test t, PointVente p"
+ " WHERE t.ePdv = p.numero"
+ " AND t.date1 BETWEEN :dateBegin AND :dateEnd"
+ " AND (t.state <> 'ANCL' or t.state is null)"
+ " AND t.game in :game"
+ " AND t.type in :type"
+ " AND t.participe = 1"
+ " AND NOT EXISTS (select t2.numInfo, t2.date1"
+ " from Test t2"
+ " WHERE t2.date1 BETWEEN :dateBegin AND :dateEnd"
+ " AND (t2.state <> 'ANCL' or t2.state is null)"
+ " AND t2.game in :game"
+ " AND t2.type in :type"
+ " AND t2.participe = 1"
+ " AND t2.numInfo = t.numInfo"
+ " AND t2.date1 = t.date1"
+ " AND (t2.phone is null or t2.phone NOT IN (select b.phone from BlacklistTest b))"
+ " group by CONCAT(t2.date1, t2.numInfo), t2.name, t2.phone, t2.numInfo, t2.date1, t2.ePdv"
+ " having sum(t2.cotations) <= :target)"
+ " AND t.cotations > :target"
+ " AND (t.phone is null or t.phone NOT IN (select b.phone from BlacklistTest b))")
List<TestResult> findTest(#Param("dateBegin") Date dateBegin, #Param("dateEnd") Date dateEnd, #Param("game") List<String> game, #Param("type") List<String> type, #Param("target") BigDecimal target);
}
is it possible to reduce the response time?
May I have your help please.
Unable to pass named parameters in #NamedNativeQuery in spring data jpa
my repo:
#Query(value = "select stat.desc as desc," +
" stat.priority as priority," +
" (case when sum(activeUser) is null then 0 else sum(activeUser) end) as activeUser," +
" (case when sum(totalUser) is null then 0 else sum(totalUser) end) as totalUser" +
" from lookup.user stat left outer join" +
" (" +
" select user.role as role, " +
" sum (case when user.STATUS = 'ACTIVE' then 1 else 0 end) as activeUser," +
" count(*) as totalUser," +
" user.group as group" +
" from Ctrl.user user" +
" where user.group =:userGroup " +
" and user.branch_code =:branchCode " +
" group by user.role,user.group" +
" ) as tbl on stat.role = tbl.role and stat.group = tbl.group" +
" where stat.group =:userGroup " +
" group by stat.desc, stat.priority" +
"", nativeQuery = true)
public List<com.cimb.dto.UserStatusSummary> getSummaryReport(#Param(value = "userGroup") String userGroup, #Param(value = "branchCode") String branchCode);
The underlying database is DB2
When I tried to access that method I am hitting following error.
DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.25.13
if I hard code those named parameters with values then it's working.
I can not use jpql as real queries have some subqueries in it, so I cant use JPQL
Edit Update
After some digging, I have found out that, the parameters I am passing are in the subquery, since JPA don't have subquery concept it's not injecting into named parameters which resulting in a syntax error.
Now how to work with Subqueries in JPA
Please help.
I am trying to run PostgreSQL native query that contains ltree functions and operators.
Here's the definition:
#NamedNativeQuery(
name = "pathSegmentQuery",
query = "select ltree2text(okm_path) as okm_path, " +
" index(okm_path, text2ltree(:lastSegment)) + 2 <> nlevel(okm_path) as haschild, " +
" case " +
" when index(okm_path, text2ltree(:lastSegment)) + 1 <> nlevel(okm_path) " +
" then ltree2text(subpath(okm_path, index(okm_path, text2ltree(:lastSegment)) + 1, 1)) " +
" end as child " +
"from document " +
"where okm_path ~ :pathLike " +
"and " +
"index(okm_path, text2ltree(:path)) + 1 <> nlevel(okm_path) ",
resultSetMapping = "pathSegmentQueryRSMapping")
invoked like:
public List<PathSegment> getPathChildren(String path, String lastSegment) {
Query query = entityManager.createNamedQuery("pathSegmentQuery");
String pathLike = "'*." + path + ".*'";
query.setParameter("path", path);
query.setParameter("pathLike", pathLike);
query.setParameter("lastSegment", lastSegment);
return query.getResultList();
}
result is error ERROR: operator does not exist: ltree ~ character varying.
when I try to run the query directly against database it runs ok:
select ltree2text(okm_path) as okm_path,
index(okm_path, text2ltree('_root_')) + 2 <> nlevel(okm_path) as haschild,
case
when index(okm_path, text2ltree('_root_')) + 1 <> nlevel(okm_path)
then ltree2text(subpath(okm_path, index(okm_path, text2ltree('_root_')) + 1, 1))
end as child
from document
where
okm_path ~ '*._root_.*'
and
index(okm_path, text2ltree('_root_')) + 1 <> nlevel(okm_path)
from the error it's obvious that hibernate(?) dislikes the type on the right side of th ~ operator, but as you can see, I am using the string in the later query and works fine.
So what do I need to do with hibernate query to run the query successfully?
EDIT:
when I replace okm_path ~ :pathLike for "where okm_path ~ '*._root_.*' " I will be given:
org.postgresql.util.PSQLException: ERROR: syntax error at position 0 error
hibernate: 5.2.9.Final
postgresql: 9.2.23
it turned out that there is lquery() function that needs to be called when you do operations against lquery.
so my query translates to
...
where okm_path ~ lquery(:pathLike)
...
and this solves the problem
The error
operator does not exist: ltree ~ character varying
should be read as
operator does not exist: <left_data_type> <operator> <right_data_type> varying
Which means the operator is not defined for these data types. This happens when, for example, the left side of the operator is a integer and the right side a varchar, the error that time would be ERROR: operator does not exist: integer = character varying.
The problem here is when you set the value for right side,
query.setParameter("pathLike", pathLike)
pathLike is a string. So Postgres sees this as comparing a ltree to a string. When you execute the SQL directly the right hand side is taken as a ltree expression than a string.
I am not sure if this will work but can you try ltree can be directly cast to a varchar, but can you try this?:
query.setParameter("pathLike", pathLike, Hibernate.OBJECT)
See also Java type in JDBC to Postgres ltree
I have the following SQL query in my j2ee web app that I am unable to get to work, as-is. The named parameters sourceSystem and sourceClientId do not appear to get passed to the query, and therefore it does not return any records. I added a watch to the Query object querySingleView and it maintained a value of null as the debugger ran through the code. I also inserted a System.out.println statement just under the method declaration and confirmed that the correct values sourceSystem and sourceClientId are being passed to the method signature. I am using NetBeans 8.0, JPA 2.1, running on a JBoss EAP 6.21 server. I have multiple Entities mapped to several tables in an Oracle database.
Here is the query (I should note that the items in the query follow a schema.table.column format - not sure if that is part of the problem, but it does work in one test, see my comment and sample below the main query below):
public List<String> searchSingleView (String sourceSystem, String sourceClientId) {
//Change the underscore character in the source system value to a dash, and make it uppercase. This is what single view accepts
if (!sourceSystem.equals("")) {
sourceSystemSingleView = sourceSystem.replace('_', '-').toUpperCase();
}
String sqlSingleViewQuery = "SELECT DISTINCT " +
"MDMCUST_ORS.C_BO_CONTRACT.LAST_ROWID_SYSTEM AS POLICY_SYSTEM, " +
"MDMCUST_ORS.C_BO_CONTRACT.SRC_POLICY_ID AS POLICY_ID, " +
"MDMCUST_ORS.C_BO_CONTRACT.ISSUE_DT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.SRC_CLIENT_ID AS SRC_CLIENT_ID, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.PERS_FULL_NAME_TXT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.ORG_LEGAL_NAME_TXT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.ORG_LEGAL_SFX_TXT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.ORG_NAME_TXT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.SIN_BIN_TEXT, " +
"MDMCUST_ORS.C_BO_PARTY_XREF.PERS_BIRTH_DT, " +
"MDMCUST_ORS.C_LU_CODES.CODE_DESCR_EN AS ADDRESS_PURPOSE_CD, " +
"MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.COMPLETE_ADDRESS_TXT, " +
"MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.CITY_NAME, " +
"MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.COUNTRY_NAME, " +
"MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.POSTAL_CD, " +
"MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.STATE_PROVINCE_NAME " +
"FROM MDMCUST_ORS.C_BO_PARTY_XREF " +
"LEFT JOIN MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR ON MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.PARTY_ROWID = MDMCUST_ORS.C_BO_PARTY_XREF.ROWID_OBJECT " +
"LEFT JOIN MDMCUST_ORS.C_LU_CODES ON MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.POSTAL_ADDR_PURPOSE_CD = MDMCUST_ORS.C_LU_CODES.CODE " +
"LEFT JOIN MDMCUST_ORS.C_BO_PARTY_REL ON MDMCUST_ORS.C_BO_PARTY_XREF.ROWID_OBJECT = MDMCUST_ORS.C_BO_PARTY_REL.FROM_PARTY_ROWID " +
"LEFT JOIN MDMCUST_ORS.C_BO_CONTRACT ON MDMCUST_ORS.C_BO_PARTY_REL.CONTRACT_ROWID = MDMCUST_ORS.C_BO_CONTRACT.ROWID_OBJECT " +
"WHERE MDMCUST_ORS.C_BO_CONTRACT.LAST_ROWID_SYSTEM = :sourceSystemSingleView " +
"AND MDMCUST_ORS.C_BO_PARTY_XREF.SRC_CLIENT_ID = :sourceClientId " +
"AND MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.POSTAL_ADDR_PURPOSE_CD = '56|07' " +
"AND MDMCUST_ORS.C_BO_PARTY_POSTAL_ADDR.LAST_ROWID_SYSTEM = MDMCUST_ORS.C_BO_CONTRACT.LAST_ROWID_SYSTEM " +
"ORDER BY MDMCUST_ORS.C_BO_CONTRACT.LAST_ROWID_SYSTEM";
querySingleView = emSingleView.createQuery(sqlSingleViewQuery);
querySingleView.setParameter("sourceSystemSingleView", sourceSystemSingleView);
querySingleView.setParameter("sourceClientId", sourceClientId);
querySingleViewResult = (List<String>) querySingleView.getResultList();
return querySingleViewResult;
}
}
However, if I put literal values in the SQL query in place of the positional parameters it works fine (without using the setParameter method).
WHERE MDMCUST_ORS.C_BO_CONTRACT.LAST_ROWID_SYSTEM = 'ADMIN' " +
"AND MDMCUST_ORS.C_BO_PARTY_XREF.SRC_CLIENT_ID = '0000001234' " +
I have looked online but haven't as yet found anything that seems to address this specific question. Any help would be greatly appreciated. Thank you!