We Keep Coding

Hibernate Native Query bad Performance and struggling to convert to JQPL

I wrote this SQL Query and using it as a native query in hibernate.
#Query(
value = "SELECT DISTINCT tp.* FROM TWITTER_POST AS tp " +
"JOIN TWITTER_LIST AS tl " +
"ON tl.owner_id = ?1 " +
"JOIN REL_TWITTER_LIST__ACCOUNTS_TRACKED_BY_LIST AS atbl " +
"ON tl.id = atbl.twitter_list_id " +
"JOIN TWITTER_ACCOUNT AS ta " +
"ON ta.id = atbl.accounts_tracked_by_list_id " +
"LEFT OUTER JOIN REL_TWITTER_POST__TWITTER_USERS_HIDING_POST uhp " +
"ON tp.id = uhp.twitter_post_id " +
"AND uhp.TWITTER_USERS_HIDING_POST_ID = ?1 " +
"WHERE uhp.twitter_post_id is NULL AND ta.id = tp.author_id",
countQuery = "SELECT DISTINCT count(tp.*) FROM TWITTER_POST AS tp " +
"JOIN TWITTER_LIST AS tl " +
"ON tl.owner_id = ?1 " +
"JOIN REL_TWITTER_LIST__ACCOUNTS_TRACKED_BY_LIST AS atbl " +
"ON tl.id = atbl.twitter_list_id " +
"JOIN TWITTER_ACCOUNT AS ta " +
"ON ta.id = atbl.accounts_tracked_by_list_id " +
"LEFT OUTER JOIN REL_TWITTER_POST__TWITTER_USERS_HIDING_POST uhp " +
"ON tp.id = uhp.twitter_post_id " +
"AND uhp.TWITTER_USERS_HIDING_POST_ID = ?1 " +
"WHERE uhp.twitter_post_id is NULL AND ta.id = tp.author_id",
nativeQuery = true
)
Page<TwitterPost> findAllNonHiddenForListsFromTwitterAccountId(Long twitterAccountId, Pageable pageable);
I noticed that the query executes very slowly when I'm running it through hibernate as opposed to a SQL tool. I assumed it was because I am using a native query as opposed to JQPL, which (from what I read) immediately does caching and pagination without requiring a definition for "count". Trying to convert it to JQPL failed, because I cannot find a good tutorial for more complicated queries on JQPL across join tables.
#Query(
value = "SELECT DISTINCT twitterPost " +
"FROM TwitterPost twitterPost " +
"JOIN TwitterList twitterList " +
"ON twitterList.owner.id = ?1 " +
"JOIN TwitterAccount tweetAuthorFromList " +
"ON tweetAuthorFromList IN twitterList.accountsTrackedByLists " +
"WHERE twitterPost.author = tweetAuthorFromList " +
"AND twitterList.owner NOT IN twitterPost.twitterUsersHidingPosts"
)
Page<TwitterPost> findAllNonHiddenPostsFromListsForTwitterAccountId(Long twitterAccountId, Pageable pageable);
Apparently my Syntax is off
org.hibernate.exception.SQLGrammarException: could not prepare
statement
but the compiler only shows me problems with the generated SQL, not the JQPL so I'm left in the dark.
Also checked for typical bad performance culprits i.e. eager fetching of entities which I set to lazy everywhere.
Any help regarding whether my performance problem assumptions are correct, or converting the query, are highly appreciated - thanks in advance!

There are many things that are wrong here:
Using SELECT DISTINCT with JOIN indicates that you should have used a Semi Join instead.
The ON tl.owner_id = ?1 is done for filtering, not for the projection, hence you are better off doing an EXISTS query.
Assuming why the query runs slow instead of profiling it. The reason why it runs faster in the DB tool is that DB tools usually truncate the result set while Spring Data consumes the entire result set. Or, if you run EXPLAIN, the output might come from the Optimizer without even running the query.
So, here's what you can do:
Use Semi Joins instead of Joins for filtering.
Use Blaze Persistence to write better entity queries dynamically.
Configure Statement Caching at the JDBC Driver level.
Use the slow query log to log the execution plan when the query is slower than N seconds.

Got an exception! You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax

I got this error i am using java and javafx and it is connected to MYsql DB i got this error while excute this statment from java to my sql please help
Got an exception!
You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near 'update kstds.match SET
kstds.match.Team1Goals=kstds.match.Team1Goals+1 where kst' at line 1
String Query ="use kstds; update kstds.match SET kstds.match.Team1Goals=kstds.match.Team1Goals+1 "
+ "where kstds.match.Team1ID= ( select kstds.team.TeamID from kstds.team where kstds.team.Name='AHLI' ) "
+ "and kstds.match.Matchid = 1 ; "
+ "Update kstds.match SET kstds.match.Team2Goals=kstds.match.Team2Goals+1 "
+ "where kstds.match.Team2ID= ( select kstds.team.TeamID from kstds.team where kstds.team.Name='AHLI' ) "
+ "and kstds.match.Matchid=1;";

you are trying to execute multiple sql queries which should be done using addBatch & executeBatch.
you don't have to execute use kstds because the connection to the database is set via Java
try this:
String Query1 ="update match SET match.Team1Goals=match.Team1Goals+1 "
+ "where match.Team1ID= ( select team.TeamID from team where team.Name='AHLI' ) "
+ "and match.Matchid = 1 ; "
String Query2 ="Update match SET match.Team2Goals=match.Team2Goals+1 "
+ "where match.Team2ID= ( select team.TeamID from team where team.Name='AHLI' ) "
+ "and match.Matchid=1;";
//stmt is your Statement and conn is your Connection
con.setAutoCommit(false);
stmt.addBatch(Query1);
stmt.addBatch(Query2);
stmt.executeBatch();
con.commit();

How to rewrite postgresSql query as Spring-data jpa query

I am stuck at the moment while trying to write the next query as a spring-data JPA query:
with recursive s as (
select *
from t
where file_id = '12345'
union all
select dfs.*
from t dfs
join s on s.file_id = dfs.parent_folder_id
)
select * from s;
I have tried the next:
#Query(value = "with recursive subfiles as (
select * from t where file_id=?1
union all
dfs.* from t dfs join subfiles s
on s.file_id = dfs.parent_folder_id)
select file_id from subfiles", nativeQuery = true)
But I get the next error:
Method threw 'org.springframework.dao.InvalidDataAccessResourceUsageException' exception.
could not extract ResultSet; SQL [n/a]
org.hibernate.exception.SQLGrammarException: could not extract ResultSet
org.postgresql.util.PSQLException: ERROR: syntax error at or near "dfs"
The query should list all direct or indirect dependent children for a specific id. (a similar post here)

I have managed to fix it using the next format:
#Query(nativeQuery = true,
value = "with recursive subfiles as " +
"(select * " +
"from t " +
"where file_id=?1 " +
"union all " +
"select dfs.* " +
"from t dfs " +
"join subfiles s " +
"on s.file_id = dfs.parent_folder_id) " +
"select file_id from subfiles")
List<String> listAllByParentId(String folderId);

Join Query for unrelated entities in Spring boot JPA

I am trying to write a query like the below on unrelated entities A and B
Select a.x,a.y,a.z FROM A a ,B b Where
b.Parameter_Name='APPLICABLE_RULE_TYPE' And b.Parameter_KEY='MC' AND
b.Parameter_Value=a.rule_type And a.Rule_Status='ON' Order By Rule_Priority;
I am not able to figure out how this should be written in the Repository class of a Spring boot application.
Can someone please suggest a way do this
Thanks in advance.
Edit:
I have tried it as a native query like
#Query(value = "Select Rule_Id, Rule_Sql,S_System,S_Entity, s_table_name,Rule_Sql_Type "
+ "FROM rule_master TRM ,T_SYSTEM_CONFIG TSC Where"
+ " Tsc.Parameter_Name='APPLICABLE_RULE_TYPE' "
+ " And Tsc.Parameter_KEY= :systemType"
+ " AND Tsc.Parameter_Value=trm.rule_type "
+ " And Rule_Status='ON'"
+ " Order By Rule_Priority", nativeQuery = true)
public List<RuleMaster> findByRuleStatus(#Param("systemType" String systemType);
but getting this error
o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 17006, SQLState: null
o.h.engine.jdbc.spi.SqlExceptionHelper : Invalid column name
org.springframework.orm.jpa.JpaSystemException: could not execute query; nested exception is org.hibernate.exception.GenericJDBCException: could not execute query

I was able to solve this by using native query itself.
The error was because i was selecting only part of the columns from the table. Instead i have used the below query which works fine:
#Query(value = "Select * "
+ "FROM rule_master TRM ,T_SYSTEM_CONFIG TSC Where"
+ " Tsc.Parameter_Name='APPLICABLE_RULE_TYPE' "
+ " And Tsc.Parameter_KEY= :systemType"
+ " AND Tsc.Parameter_Value=trm.rule_type "
+ " And Rule_Status='ON'"
+ " Order By Rule_Priority", nativeQuery = true)
public List<RuleMaster> findByRuleStatus(#Param("systemType" String systemType);

JPA use IN clause with objects

I am facing a problem usign JPA, and more specifically using a IN clause.
The best way is, I think, to show you my code :
#NamedQuery(name = "Commande.findCustom", query = "SELECT DISTINCT [myFields] "
+ "FROM Commande c WHERE "
+ "[SomeCriterias] AND "
+ "c.ID IN (SELECT t.ID FROM SubTable t "
+ "WHERE t.IDX IN :param) AND [otherCriterias]"),
I then get an error from MySQL :
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.ID FROM SubTable t1 WHERE (t1.IDX IN (168)'
I am looking for a response but can't find anything ...
I tied to delete a IN clause, the problem is still the same (so it's not a double IN use problem)
my :param is a List of objects, that i got by using the Object.find() method. As you can see, it returns the ID (168). But I can't seem to find the problem ...
Any help would be greatly appreciated, thank you
EDIT : Full query
#NamedQuery(name = "Commande.findCustom", query = "SELECT DISTINCT c.idChargement, c.libelle, "
+ "c.codeTransporteur, c.reference, c.dateCreation, c.dateChargementPrevu, "
+ "c.dateValidationChargement, c.dateLivraisonPrevue, c.codeDestinataire, "
+ "c.raisonSocialeDestinataire, c.adresseDestinataire, c.codePostalDestinataire, "
+ "c.villeDestinataire, c.paysDestinataire, c.contactDestinataire, "
+ "c.telephoneDestinataire, c.mailDestinataire, c.poidsCommande, c.nombreColis, "
+ "c.nombreUniteManutention, c.typeUniteManutention, c.prendreRDV, c.commentaires "
+ "FROM Commande c WHERE "
+ "c.idChargement = :idChargement AND c.codeTransporteur = :codeTransporteur AND "
+ "(c.dateCreation BETWEEN :dateDebut AND :dateFin) AND "
+ "c.idDernierStatut IN (SELECT l.idListeStatutsCommande FROM Listestatutscommande l "
+ "WHERE l.idStatut IN :idStatut) AND c.raisonSocialeDestinataire = :raisonSociale AND "
+ "c.adresseDestinataire = :adresseDestinataire AND c.codeDestinataire = :codeDestinataire "
+ "AND c.codePostalDestinataire = :codePostal AND c.villeDestinataire = :villeDestinataire "
+ "AND c.paysDestinataire = :codePays")
And the Error Message
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.idListeStatutsCommande FROM listestatutscommande t1 WHERE (t1.IdStatut IN (168)'

As you are getting error from MYSQL not Hibernate, you could try to find out what query actually is generated. To do this, use proxy invoker like p6spy and then everything should be clear. Check p6spy site. When you do, try to invoke such generated SQL yourself and try to fix it. I used such method when I had some troubles using JPA's joins fetches and stuff. Very helpfull in diagnosing such problems.

Ok so I found the problem, and then the answer. Thanks #Antoniossss, the fact is that I was looking at the wrong part of the query.
The error was here : c.idDernierStatut IN ...
The fact is that this part is a foreign key. And when you want to search on it, you have to consider it as an object. So the correct form is c.idDernierStatut.idListeStatutsCommande IN to get the ID.
Thank you to both of you for your time anyway !