I am trying to perform a query like the following, with selecting by a case statement and grouping by the same case statement..
Select USER,
(CASE
WHEN value between 0 AND 2 then '0-2'
WHEN value between 3 AND 4 then '3-4'
ELSE '5+'
END) as CASE_STATEMENT ,
SUM(value)
.....
Group by user, CASE_STATEMENT
using JPA 2.0 Criteria API, with Hibernate.
My test case looks like ...
CriteriaBuilder cb = em.getCriteriaBuilder()
CriteriaQuery cq = cb.createQuery(Tuple)
def root = cq.from(TestEntity)
def userGet = root.get('user')
def valueGet = root.get('value')
def caseExpr =
cb.selectCase()
.when(cb.between(valueGet, 0, 2), '0-2')
.when(cb.between(valueGet, 3, 4), '3-4')
.otherwise('5+')
def sumExpr = cb.sum(valueGet)
cq.multiselect([userGet, caseExpr, sumExpr])
cq.groupBy([userGet, caseExpr])
log(typedQuery.unwrap(Query).queryString)
List<Tuple> tuples = typedQuery.getResultList()
The log statement of the queryString reads
SELECT generatedAlias0.USER,
CASE
WHEN generatedAlias0.value BETWEEN 0 AND 2 THEN Cast(:param0 AS STRING)
WHEN generatedAlias0.value BETWEEN 3 AND 4 THEN Cast(:param1 AS STRING)
ELSE Cast(:param2 AS STRING)
END,
Sum(generatedAlias0.value)
FROM test AS generatedAlias0
GROUP BY generatedAlias0.USER,
CASE
WHEN generatedAlias0.value BETWEEN 0 AND 2 THEN Cast(
:param3 AS STRING)
WHEN generatedAlias0.value BETWEEN 3 AND 4 THEN Cast(
:param4 AS STRING)
ELSE Cast(:param5 AS STRING)
END
When calling the typedQuery.getResultList(), I get the following error statement
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet
Caused by: org.h2.jdbc.JdbcSQLException: Column "TESTENTITY0_.VALUE" must be in the GROUP BY list; SQL statement:
select testentity0_.user as col_0_0_, case when testentity0_.value between 0 and 2 then cast(? as varchar(255)) when testentity0_.value between 3 and 4 then cast(? as varchar(255)) else cast(? as varchar(255)) end as col_1_0_, sum(testentity0_.value) as col_2_0_ from test testentity0_ group by testentity0_.user , case when testentity0_.value between 0 and 2 then cast(? as varchar(255)) when testentity0_.value between 3 and 4 then cast(? as varchar(255)) else cast(? as varchar(255)) end [90016-194]
Is there something wrong with the way I am trying to group by the Expression? I have also tried grouping by alias names, and by number literals (1, 2)
Is there another way I can go about structuring the SQL to get the same results?
Thanks.
As the exception message suggests, the problem is related to the Group By statement at DBMS level. See: https://www.percona.com/blog/2019/05/13/solve-query-failures-regarding-only_full_group_by-sql-mode/
To solve the error, you must either
Set the Group By Mode of the underlying DBMS to a less restrictive level (MySQL allows to disable only-full-group-by, but H2 does not (you may try setting MODE=MYSQL in jdbc connection string)
or (better)
Add all columns that are part of the select statement to the GROUP BY statement or to an aggregate function as described above.
You should be able to build a nested query which fulfills the GROUP BY RESTRICTIONS.
For the rescue, there are some (maybe DBMS specific) aggregate
functions (at least in MySQL). To trick JPA and Hibernate to understand these, there are
several ways to achieve this, as described at
https://vladmihalcea.com/hibernate-sql-function-jpql-criteria-api-query/
and
https://vladmihalcea.com/the-jpa-entitymanager-createnativequery-is-a-magic-wand/
Edit
In contrast and addition to the statement above, the findings after discussion below are:
The exception is raised by the h2 driver in the org.h2.expression.ExpressionColumn class, while it's verifying the query syntax
The solution requires setting and referencing an alias in the query (at the case statement or subquery), which is currently not possible in Criteria API (see column aliases usually can't be referenced in the query itself)
A workaround would be creating of a NativeQuery like this:
List<Tuple> tuples = em.createNativeQuery(
"SELECT generatedAlias0.USER, " +
" CASE " +
" WHEN generatedAlias0.value BETWEEN 0 AND 2 THEN Cast(:param0 AS VARCHAR) " +
" WHEN generatedAlias0.value BETWEEN 3 AND 4 THEN Cast(:param1 AS VARCHAR) " +
" ELSE Cast(:param2 AS VARCHAR) " +
" END c, " +
" Sum(generatedAlias0.value) as sumvalue " +
"FROM test AS generatedAlias0 " +
"GROUP BY generatedAlias0.USER, c "
)
.setParameter("param0", "0-2")
.setParameter("param1", "3-4")
.setParameter("param2", "5+")
.getResultList();
Related
I have a spring boot app connected to oracle DB.
I am trying to order a list of records and select the top most record.
I wrote a JPA query as below but it fails.
#Query("SELECT id FROM UploadedFile uploadedFile "
+ "WHERE uploadedFile.p_file_type = 'branch' "
+ "and p_file_status='Processed' "
+ "and p_is_file_process_with_error = 0 "
+ "order by c_created_date desc "
+ "FETCH FIRST 1 rows only ")
public String findLatestBranchCodeFile();
The error received was
creating bean with name 'uploadedFileRepo': Invocation of init method
failed; nested exception is java.lang.IllegalArgumentException:
Validation failed for query for method public abstract
java.lang.String
com.rhb.pintas.repo.UploadedFileRepo.findLatestBranchCodeFile()!
org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token:
FETCH near line 1, column 204 [SELECT id FROM
com.rhb.pintas.entities.UploadedFile uploadedFile WHERE
uploadedFile.p_file_type = 'branch' and p_file_status='Processed' and
p_is_file_process_with_error = 0 order by c_created_date desc FETCH
FIRST 1 rows only ] -> [Help 1]
The issue seems to be with fetch,not sure whats wrong.
It seems you are mixing HQL and native query dialects:
If this will be a naviveQuery (like most of the columns would mention), then replace the entity name with table name and add nativeQuery option. And because You are using only a single table, You can skip the alias name:
#Query("SELECT id FROM uploaded_file "
+ "WHERE p_file_type = 'branch' and p_file_status='Processed' and "
+ "p_is_file_process_with_error = 0 "
+ "order by c_created_date desc "
+ "FETCH FIRST 1 rows only ", nativeQuery = true)
public String findLatestBranchCodeFile();
If You want to keep it as a HQL, then replace all column names with entity property names, like p_file_type > fileType (I guess column names). Secondly You will need to pass a Pageable parameter, to replace Your 'Fetch first' statement.
You can find more materials here:
Bealdung
NativeQ
StackOverflow
You are trying to execute SQL query, in this case you need to add nativeQuery=true attribute to #Query annotation
UPD.
got confused because FETCH FIRST - is a SQL syntax, for JPA way please check another solution - it is possible to return list with at most one element.
I guess, you can try passing pagable to limit result set size and unlimit your query:
public String findLatestBranchCodeFile(Pageable pageable); // pass new PageRequest(0,1)
I am having code something like this.
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
Calculation of fullTableName is something like:
public String getFullTableName(final String table) {
if (this.schemaDB != null) {
return this.schemaDB + "." + table;
}
return table;
}
Here schemaDB is the name of the environment(which can be changed over time) and table is the table name(which will be fixed).
Value for schemaDB is coming from an XML file which makes the query vulnerable to SQL injection.
Query: I am not sure how the table name can be used as a prepared statement(like the name used in this example), which is the 100% security measure against SQL injection.
Could anyone please suggest me, what could be the possible approach to deal with this?
Note: We can be migrated to DB2 in future so the solution should compatible with both Oracle and DB2(and if possible database independent).
JDBC, sort of unfortunately, does not allow you to make the table name a bound variable inside statements. (It has its reasons for this).
So you can not write, or achieve this kind of functionnality :
connection.prepareStatement("SELECT * FROM ? where id=?", "TUSERS", 123);
And have TUSER be bound to the table name of the statement.
Therefore, your only safe way forward is to validate the user input. The safest way, though, is not to validate it and allow user-input go through the DB, because from a security point of view, you can always count on a user being smarter than your validation.
Never trust a dynamic, user generated String, concatenated inside your statement.
So what is a safe validation pattern ?
Pattern 1 : prebuild safe queries
1) Create all your valid statements once and for all, in code.
Map<String, String> statementByTableName = new HashMap<>();
statementByTableName.put("table_1", "DELETE FROM table_1 where name= ?");
statementByTableName.put("table_2", "DELETE FROM table_2 where name= ?");
If need be, this creation itself can be made dynamic, with a select * from ALL_TABLES; statement. ALL_TABLES will return all the tables your SQL user has access to, and you can also get the table name, and schema name from this.
2) Select the statement inside the map
String unsafeUserContent = ...
String safeStatement = statementByTableName.get(usafeUserContent);
conn.prepareStatement(safeStatement, name);
See how the unsafeUserContent variable never reaches the DB.
3) Make some kind of policy, or unit test, that checks that all you statementByTableName are valid against your schemas for future evolutions of it, and that no table is missing.
Pattern 2 : double check
You can 1) validate that the user input is indeed a table name, using an injection free query (I'm typing pseudo sql code here, you'd have to adapt it to make it work cause I have no Oracle instance to actually check it works) :
select * FROM
(select schema_name || '.' || table_name as fullName FROM all_tables)
WHERE fullName = ?
And bind your fullName as a prepared statement variable here. If you have a result, then it is a valid table name. Then you can use this result to build a safe query.
Pattern 3
It's sort of a mix between 1 and 2.
You create a table that is named, e.g., "TABLES_ALLOWED_FOR_DELETION", and you statically populate it with all tables that are fit for deletion.
Then you make your validation step be
conn.prepareStatement(SELECT safe_table_name FROM TABLES_ALLOWED_FOR_DELETION WHERE table_name = ?", unsafeDynamicString);
If this has a result, then you execute the safe_table_name. For extra safety, this table should not be writable by the standard application user.
I somehow feel the first pattern is better.
You can avoid attack by checking your table name using regular expression:
if (fullTableName.matches("[_a-zA-Z0-9\\.]+")) {
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
}
It's impossible to inject SQL using such a restricted set of characters.
Also, we can escape any quotes from table name, and safely add it to our query:
fullTableName = StringEscapeUtils.escapeSql(fullTableName);
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
StringEscapeUtils comes with Apache's commons-lang library.
I think that the best approach is to create a set of possible table names and check for existance in this set before creating query.
Set<String> validTables=.... // prepare this set yourself
if(validTables.contains(fullTableName))
{
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
//and so on
}else{
// ooooh you nasty haker!
}
create table MYTAB(n number);
insert into MYTAB values(10);
commit;
select * from mytab;
N
10
create table TABS2DEL(tname varchar2(32));
insert into TABS2DEL values('MYTAB');
commit;
select * from TABS2DEL;
TNAME
MYTAB
create or replace procedure deltab(v in varchar2)
is
LvSQL varchar2(32767);
LvChk number;
begin
LvChk := 0;
begin
select count(1)
into LvChk
from TABS2DEL
where tname = v;
if LvChk = 0 then
raise_application_error(-20001, 'Input table name '||v||' is not a valid table name');
end if;
exception when others
then raise;
end;
LvSQL := 'delete from '||v||' where n = 10';
execute immediate LvSQL;
commit;
end deltab;
begin
deltab('MYTAB');
end;
select * from mytab;
no rows found
begin
deltab('InvalidTableName');
end;
ORA-20001: Input table name InvalidTableName is not a valid table name ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 21
ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 16
ORA-06512: at line 2
ORA-06512: at "SYS.DBMS_SQL", line 1721
I have a JPQL query in a repository that is the equivalent of the MySQL query below:
SELECT DISTINCT ji.* FROM tracker_job_item AS ji
JOIN tracker_work AS w ON ji.id=w.tracker_job_item_id
JOIN tracker_work_quantity AS wq on w.id=wq.tracker_work_id
WHERE w.work_type = 'CUTTING' AND ji.is_finished=0
GROUP BY wq.tracker_work_id
HAVING ji.quantity != SUM(wq.received_quantity)
The MySQL version works just fine, but the JPQL equivalent gives an exception: Unknown column 'jobitem0_.quantity' in 'having clause'
The JPQL query is like below:
#Query("select distinct ji from JobItem ji" +
" join Work w on ji.id=w.jobItem.id" +
" join WorkQuantity wq on w.id=wq.work.id" +
" where w.workType='CUTTING' and ji.isFinished=false and ji.jobItemName like %:search%" +
" group by ji.id" +
" having ji.quantity != sum(wq.receivedQuantity)")
Page<JobItem> findAllActiveCuttingJobs(Pageable pageable, #Param("search") String search);
Please help me with why I'm getting the error even though the field quantity exists in JobItem.
You can't reference a column in the having clause that isn't in the group by clause, definitely in JPA and (normally at least) in SQL. Looks like MySQL is letting you get away with it but JPA isn't.
See here:
http://learningviacode.blogspot.co.uk/2012/12/group-by-and-having-clauses-in-hql.html
I made a query in Java which changes one column's datatype in another. It works fine until it tries to change type. It finds all columns in DB with specified datatype, but cannt change it.
Here is my code:
st = conn.createStatement();
rs = st.executeQuery("SELECT a.name as ColName, o.name AS TableName"
+ "FROM sys.syscolumns AS a"
+ "INNER JOIN sys.systypes AS b ON a.xtype = b.xtype AND b.name = 'char' AND a.length = 255"
+ "INNER JOIN sys.objects AS o ON a.id = o.object_id WHERE (o.type = 'u') AND (o.schema_id = 1)");
ResultSet rsPom;
while (rs.next()){
String tName=rs.getString("TableName");
String cName=rs.getString("ColName");
System.out.println(tName+" "+cName);
rsPom=st.executeQuery("ALTER TABLE "+ tName+" ALTER COLUMN "+cName+" nvarchar(255)");
}
And here is my Exception:
com.microsoft.sqlserver.jdbc.SQLServerException: The object 'CK_TimeInstant_frame_default' is dependent on column 'frame'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:792)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:616)
at DataTypeChanger.changeDataType(DataTypeChanger.java:50)
at DataTypeChanger.main(DataTypeChanger.java:36)
Does anyone knows what is all about, and what can I do?
Firstly, my apologies about the sarky comment above.
The reason you receive this error is because your alter script isn't taking any action with respect to constraints. Columns which are the target of constraints (Unique, Foreign Key, Default, etc) can't be modified unless the constraint is first dropped. In which case you'll probably need to add the constraints back afterwards.
I've assumed your earlier (deleted) comment still holds, viz that you do not require to create the constraints again after they have been dropped.
Re : How do I drop all constraints
Disclaimer : Back up your database before you try this, but the following MIGHT work. It is a destructive one way operation.
declare #sql nvarchar(2000);
while(exists(select 1 from sys.objects WHERE type_desc LIKE '%CONSTRAINT%'))
begin
BEGIN TRY
SELECT TOP 1 #sql=('ALTER TABLE ' + SCHEMA_NAME(schema_id) + '.[' + OBJECT_NAME(parent_object_id)
+ '] DROP CONSTRAINT [' + OBJECT_NAME(OBJECT_ID) + ']')
FROM sys.objects
WHERE type_desc LIKE '%CONSTRAINT%'
ORDER BY NEWID();
exec (#sql);
END TRY
BEGIN CATCH
END CATCH
end;
GO
Rationale: The TRY CATCH is required because there is no guarantee that we will get the order of dependencies correct when dropping constraints (e.g. FK dependent on PK), so we basically squash the error and try drop another random constraint (ORDER BY NEWID())
Reference : Based on this query here, but extended to all constraints
SqlFiddle Here
Following is the one of property of ExecutionHistory class, which value is fetched from
#Formula using JPA/Hibernate from exectution_history table,
#Formula("(SELECT SUM(dividend) || '/' || SUM(divisor) " +
"FROM (SELECT CAST(substr(sr.result, 1, position('/' in sr.result)-1 ) AS int) AS dividend ," +
"CAST(substr(sr.result, position('/' in sr.result)+1 ) AS int) AS divisor " +
"FROM suite_results as sr WHERE sr.execution_history=id) AS division)")
private String result;
When I tried to get instance of ExecutionHistory class, I found that above formula query
is converted by JPA/Hibernate like this:
select executionh0_.id as id7_1_, executionh0_.execution_plan as execution3_7_1_, executionh0_.start_time as start2_7_1_,
(SELECT SUM(sr.duration) FROM suite_results as sr WHERE sr.execution_history=executionh0_.id) as formula0_1_,
(SELECT SUM(executionh0_.dividend) || '/' || SUM(executionh0_.divisor) FROM
(SELECT CAST(substr(sr.result, 1, position('/' in sr.result)-1 ) AS executionh0_.int) AS executionh0_.dividend ,
CAST(substr(sr.result, position('/' in sr.result)+1 ) AS executionh0_.int) AS executionh0_.divisor
FROM suite_results as sr WHERE sr.execution_history=executionh0_.id) AS executionh0_.division) as
formula1_1_, executionp1_.id as id6_0_, executionp1_.build_number as
build2_6_0_, executionp1_.name as name6_0_, executionp1_.owner as owner6_0_, executionp1_.sut as sut6_0_,
executionp1_.wait_for_page_to_load as wait6_6_0_ from execution_history executionh0_
left outer join execution_plans executionp1_ on executionh0_.execution_plan=executionp1_.id where executionh0_.id=?
So the problem is that, here formula query contains "CAST() AS int", but during query conversion by Hibernate, it puts unnecessary table reference and execute it as "CAST() AS executionh0_.int" so it giving sql grammer exeception while execution.
I've no idea about how to avoid this problem, Can anybody help me in this?
Thanks.
It's an old question, but I'll post an answer anyway.
If you are using a SQL Server database, you can add double quotes around the type you are casting.
Something like this:
#Formula("CAST(FLOOR(CAST( dat_criacao AS \"float\")) AS \"datetime\")")
Don't know which database you're using, but in SQL Server you should use CONVERT rather than CAST in you Hibernate queries.