Problem Synopsis:
When attempting to execute a SQL query in Java with a SQLite Database, the SQL statement fails to return from the execute() or executeQuery() method. In other words, the system "hangs" when executing this SQL statement.
Question:
What am I doing wrong to explain why the ResultSet never "returns?"
TroubleShooting
I tried to narrow the problem and the problem seems to be with the Java execute() or executeQuery(). A ResultSet never seems to return. For example, I tried executing exactly the same query directly in SQLite (that is, using a SQLite DB manager). The query (outside Java) executes in about 5ms and returns the valid result set.
NOTE: No exception is thrown. The system merely seems to "hang" and becomes unresponsive until a manual kill. (waiting more than 10 minutes.)
Code:
I heavily edited this code to make the problem simpler to see. (In production, this uses Prepared Statements. But, the error occurs in both methods--straight Statement and prepared Statement versions.)
Basically, the SELECT returns a single DB item so the user can review that item.
Statement st = conn.createStatement() ;
ResultSet rs = st.executeQuery("SELECT DISTINCT d1.id, d1.sourcefullfilepath, " +
"d1.sourcefilepath, d1.sourcefilename, d1.classificationid, d1.classid, " +
"d1.userid FROM MatterDataset, (SELECT MatterDataset.id, " +
"MatterDataset.sourcefullfilepath, MatterDataset.sourcefilepath, " +
"MatterDataset.sourcefilename, MatterDataset.matterid , " +
"DocumentClassification.classificationid, DocumentClassification.classid," +
" DocumentClassification.userid FROM MatterDataset " +
"LEFT JOIN DocumentClassification ON " +
"DocumentClassification.documentid = Matterdataset.id " +
"WHERE ( DocumentClassification.classid = 1 OR " +
"DocumentClassification.classid = 2 ) AND " +
"DocumentClassification.userid < 0 AND " +
"MatterDataset.matterid = \'100\' ) AS d1 " +
"LEFT JOIN PrivilegeLog ON " +
"d1.id = PrivilegeLog.documentparentid AND " +
"d1.matterid = PrivilegeLog.matterid " +
"WHERE PrivilegeLog.privilegelogitemid IS NULL " +
"AND MatterDataset.matterid = \'100\' " +
"ORDER BY d1.id LIMIT 1 ;") ;
Configuration:
Java 6,
JDBC Driver = Xerial sqlite-jdbc-3.7.2,
SQLite 3,
Windows
Update
Minor revision: as I continue to work with this, adding a MIN(d1.id) to the beginning of the SQL statement at least returns a ResultSet (rather than "hanging"). But, this is not really what I wanted as the MIN obviates the LIMIT function.
Statement st = conn.createStatement() ;
ResultSet rs = st.executeQuery("SELECT DISTINCT MIN(d1.id), d1.id,
d1.sourcefullfilepath, " +
"d1.sourcefilepath, d1.sourcefilename, d1.classificationid, d1.classid, " +
"d1.userid FROM MatterDataset, (SELECT MatterDataset.id, " +
"MatterDataset.sourcefullfilepath, MatterDataset.sourcefilepath, " +
"MatterDataset.sourcefilename, MatterDataset.matterid , " +
"DocumentClassification.classificationid, DocumentClassification.classid," +
" DocumentClassification.userid FROM MatterDataset " +
"LEFT JOIN DocumentClassification ON " +
"DocumentClassification.documentid = Matterdataset.id " +
"WHERE ( DocumentClassification.classid = 1 OR " +
"DocumentClassification.classid = 2 ) AND " +
"DocumentClassification.userid < 0 AND " +
"MatterDataset.matterid = \'100\' ) AS d1 " +
"LEFT JOIN PrivilegeLog ON " +
"d1.id = PrivilegeLog.documentparentid AND " +
"d1.matterid = PrivilegeLog.matterid " +
"WHERE PrivilegeLog.privilegelogitemid IS NULL " +
"AND MatterDataset.matterid = \'100\' " +
"ORDER BY d1.id LIMIT 1 ;") ;
What a messy SQL statement (sorry)! I don't know SQLite, but why not simplify to:
SELECT DISTINCT md.id, md.sourcefullfilepath, md.sourcefilepath, md.sourcefilename,
dc.classificationid, dc.classid, dc.userid
FROM MatterDataset md
LEFT JOIN DocumentClassification dc
ON dc.documentid = md.id
AND (dc.classid = 1 OR dc.classid = 2 )
AND dc.userid < 0
LEFT JOIN PrivilegeLog pl
ON md.id = pl.documentparentid
AND md.matterid = pl.matterid
WHERE pl.privilegelogitemid IS NULL
AND md.matterid = \'100\'
ORDER BY md.id LIMIT 1 ;
I was uncertain whether you wanted to LEFT JOIN or INNER JOIN to DocumentClassification (using LEFT JOIN and then put requirements on classid and userid in the WHERE statement is - in my opinion - contradictory). If DocumentClassification has to exist, then change to INNER JOIN and put the references to classid and userid into the WHERE clause, if DocumentClassification may or may not exist in your result set, then keep the query as I suggested above.
I went back and started over. The SQL syntax, while it worked outside Java, simply seemed too complex for the JDBC driver. This cleaned-up revision seems to work:
SELECT DISTINCT
MatterDataset.id, MatterDataset.sourcefullfilepath, MatterDataset.sourcefilepath,
MatterDataset.sourcefilename
FROM MatterDataset , DocumentClassification
ON DocumentClassification.documentid = MatterDataset.id
AND MatterDataset.matterid = DocumentClassification.matterid
LEFT JOIN PrivilegeLog ON MatterDataset.id = PrivilegeLog.documentparentid
AND MatterDataset.matterid = PrivilegeLog.matterid
WHERE PrivilegeLog.privilegelogitemid IS NULL
AND MatterDataset.matterid = '100'
AND (DocumentClassification.classid = 1 OR DocumentClassification.classid = 2)
AND DocumentClassification.userid = -99
ORDER BY MatterDataset.id LIMIT 1;
A nice lesson in: just because you can in SQL doesn't mean you should.
What this statement does is essentially locates items in the MatterDataset Table that are NOT in the PrivilegeLog table. The LEFT JOIN and IS NULL syntax locate the items that are "missing." That is, i want to find items that are in MatterDataset but not yet in PrivilegeLog and return those items.
Related
I wrote this SQL Query and using it as a native query in hibernate.
#Query(
value = "SELECT DISTINCT tp.* FROM TWITTER_POST AS tp " +
"JOIN TWITTER_LIST AS tl " +
"ON tl.owner_id = ?1 " +
"JOIN REL_TWITTER_LIST__ACCOUNTS_TRACKED_BY_LIST AS atbl " +
"ON tl.id = atbl.twitter_list_id " +
"JOIN TWITTER_ACCOUNT AS ta " +
"ON ta.id = atbl.accounts_tracked_by_list_id " +
"LEFT OUTER JOIN REL_TWITTER_POST__TWITTER_USERS_HIDING_POST uhp " +
"ON tp.id = uhp.twitter_post_id " +
"AND uhp.TWITTER_USERS_HIDING_POST_ID = ?1 " +
"WHERE uhp.twitter_post_id is NULL AND ta.id = tp.author_id",
countQuery = "SELECT DISTINCT count(tp.*) FROM TWITTER_POST AS tp " +
"JOIN TWITTER_LIST AS tl " +
"ON tl.owner_id = ?1 " +
"JOIN REL_TWITTER_LIST__ACCOUNTS_TRACKED_BY_LIST AS atbl " +
"ON tl.id = atbl.twitter_list_id " +
"JOIN TWITTER_ACCOUNT AS ta " +
"ON ta.id = atbl.accounts_tracked_by_list_id " +
"LEFT OUTER JOIN REL_TWITTER_POST__TWITTER_USERS_HIDING_POST uhp " +
"ON tp.id = uhp.twitter_post_id " +
"AND uhp.TWITTER_USERS_HIDING_POST_ID = ?1 " +
"WHERE uhp.twitter_post_id is NULL AND ta.id = tp.author_id",
nativeQuery = true
)
Page<TwitterPost> findAllNonHiddenForListsFromTwitterAccountId(Long twitterAccountId, Pageable pageable);
I noticed that the query executes very slowly when I'm running it through hibernate as opposed to a SQL tool. I assumed it was because I am using a native query as opposed to JQPL, which (from what I read) immediately does caching and pagination without requiring a definition for "count". Trying to convert it to JQPL failed, because I cannot find a good tutorial for more complicated queries on JQPL across join tables.
#Query(
value = "SELECT DISTINCT twitterPost " +
"FROM TwitterPost twitterPost " +
"JOIN TwitterList twitterList " +
"ON twitterList.owner.id = ?1 " +
"JOIN TwitterAccount tweetAuthorFromList " +
"ON tweetAuthorFromList IN twitterList.accountsTrackedByLists " +
"WHERE twitterPost.author = tweetAuthorFromList " +
"AND twitterList.owner NOT IN twitterPost.twitterUsersHidingPosts"
)
Page<TwitterPost> findAllNonHiddenPostsFromListsForTwitterAccountId(Long twitterAccountId, Pageable pageable);
Apparently my Syntax is off
org.hibernate.exception.SQLGrammarException: could not prepare
statement
but the compiler only shows me problems with the generated SQL, not the JQPL so I'm left in the dark.
Also checked for typical bad performance culprits i.e. eager fetching of entities which I set to lazy everywhere.
Any help regarding whether my performance problem assumptions are correct, or converting the query, are highly appreciated - thanks in advance!
There are many things that are wrong here:
Using SELECT DISTINCT with JOIN indicates that you should have used a Semi Join instead.
The ON tl.owner_id = ?1 is done for filtering, not for the projection, hence you are better off doing an EXISTS query.
Assuming why the query runs slow instead of profiling it. The reason why it runs faster in the DB tool is that DB tools usually truncate the result set while Spring Data consumes the entire result set. Or, if you run EXPLAIN, the output might come from the Optimizer without even running the query.
So, here's what you can do:
Use Semi Joins instead of Joins for filtering.
Use Blaze Persistence to write better entity queries dynamically.
Configure Statement Caching at the JDBC Driver level.
Use the slow query log to log the execution plan when the query is slower than N seconds.
I'm trying to get result from 3 inner join in java code with sql.
I tried this
BeanHandler rsh=new BeanHandler(evaluation.class);
evaluation eval=(evaluation) runner.query(query,new Object[]{userid}, rsh);
This code return the result from query, but with "join" statement it's not working, I want to get result from 3 tables.
any ideas?
Edit: (the query)
String query=" select users.username, sum(evaluation.mark)\n" +
"from users inner JOIN user_has_eval \n" +
"on users.id= user_has_eval.user_id inner JOIN\n" +
"evaluation ON evaluation.id=user_has_eval.eval_id\n" +
"where evaluation.types=\"Secretaria\" and users.id = ? \n" +
"\n" ;
I am trying to write native query:
#NamedNativeQuery(name = "getUncheckedTests",
query = "SELECT t.test_name AS test_name, q.question AS question, #counter \\:= COUNT(ua.is_checked=0) AS uncheckedAnswers FROM user_answer ua" +
" JOIN result r on r.id = ua.result_id" +
" JOIN test t on r.test_id = t.id" +
" JOIN question q on ua.question_id = q.id" +
" WHERE r.is_checked = false AND q.is_open = true AND ua.is_checked = false" +
" AND r.test_id IN (SELECT t.id FROM result r" +
" JOIN test t on t.id = r.test_id" +
" JOIN user u on u.id = r.user_id" +
" WHERE r.permission = 2 AND u.id = :userId) and r.permission = 1 and #counter > 0" +
" GROUP BY ua.question_id")
However, i am getting empty result as #counter > 0 does not work properly. I can execute this query right in Workbench, however no result if it's put in Hibernate as native query
You should probably check the actual SQL Hibernate executes by enabling logging of the statements or by using log4jdbc and make sure you execute the query with the proper parameters.
Good day. I have a query in my Java code that deletes duplicate rows in a table. Initially it worked and for a while i didn't touch the project. But on running the file a few days ago, my code was throwing exceptions. This is my code:
String query = "DELETE error_log FROM error_log INNER JOIN "
+ "(SELECT min(id) minid, service_source, channel,transaction_type, provider_name, pido_account, beneficiary_id, error_description, error_date FROM error_log "
+ "GROUP BY service_source, channel, transaction_type, provider_name, pido_account, beneficiary_id, error_description, error_date "
+ "HAVING COUNT(1) > 1 AS duplicates ON "
+ "(duplicates.service_source = error_log.service_source AND duplicates.channel = error_log.channel "
+ "AND duplicates.transaction_type = error_log.transaction_type AND duplicates.provider_name = error_log.provider_name "
+ "AND duplicates.pido_account = error_log.pido_account AND duplicates.beneficiary_id = error_log.beneficiary_id "
+ "AND duplicates.error_description = error_log.error_description AND duplicates.error_date = error_log.error_date "
+ "AND duplicates.minid <> error_log.id"
+ ")"
+ ")";
int deploy = duplicate.executeUpdate(query);
I get this afterwards:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AS duplicates ON (duplicates.service_source = error_log.service_source AND dupli' at line 1.
How do i correct this and have the duplicates deleted from the table?
You have a missing ) on line + "HAVING COUNT(1) > 1 AS duplicates ON " and have an extra ) at the end.
String query = "DELETE error_log FROM error_log INNER JOIN "
+ "(SELECT min(id) minid, service_source, channel,transaction_type, provider_name, pido_account, beneficiary_id, error_description, error_date FROM error_log "
+ "GROUP BY service_source, channel, transaction_type, provider_name, pido_account, beneficiary_id, error_description, error_date "
+ "HAVING COUNT(1) > 1 ) AS duplicates ON "
+ "(duplicates.service_source = error_log.service_source AND duplicates.channel = error_log.channel "
+ "AND duplicates.transaction_type = error_log.transaction_type AND duplicates.provider_name = error_log.provider_name "
+ "AND duplicates.pido_account = error_log.pido_account AND duplicates.beneficiary_id = error_log.beneficiary_id "
+ "AND duplicates.error_description = error_log.error_description AND duplicates.error_date = error_log.error_date "
+ "AND duplicates.minid <> error_log.id"
+ ")";
int deploy = duplicate.executeUpdate(query);
If you haven't made any changes and it stopped working, 1) are you sure you tested this code? and 2) has anyone else made any changes without your knowledge?
I'm trying to write a small function that gets an id u (integer), and returns the number of friends u have with distance <=3 (the friends information is stored in the table Likes: Likes(uid1, uid2) means: uid1 likes uid2).
I wrote this simple query:
String likesDistance =
"SELECT uid2 " + //DISTANCE = 1
"FROM Likes " +
"WHERE uid1 = ? " +
"UNION " +
"SELECT L2.uid2 " + //DISTANCE = 2
"FROM Likes L1, Likes L2 " +
"WHERE L1.uid1 = ? and L1.uid2 = L2.uid1 " +
"UNION "+
"SELECT L3.uid2 " + //DISTANCE = 3
"FROM Likes L1, Likes L2 , Likes L3 " +
"WHERE L1.uid1 = ? and L1.uid2 = L2.uid1 and L2.uid2 = L3.uid1 ";
Then, I do the following:
PreparedStatement pstmt2 = con.prepareStatement(likesDistance);
pstmt1.setInt(1, u);
pstmt1.setInt(2, u);
System.out.println("2");
pstmt1.setInt(3, u);
System.out.println("3");
pstmt1.setInt(4, u);
pstmt1.setInt(5, u);
Where u, as mention before, is an integer passed to the function.
All this code is within a 'try' block. When I try to run it, it prints the first printout ("2"), but not the second one ("3"), and I get the following exception message:
The column index is out of range: 3, number of columns: 2.
Why is it like this, and how can I change it to work as I want?
Thanks a lot.
Copy and paste? Guess you wanted to set parameters for statement 2.
Your prepared statement is pstmt2.
You are setting properties on pstmt1.
Try setting the properties on pstmt2 and it should work.
Though you set properties on pstmt2 you get exception as there are only three placeholders in the statement and you are setting for five placeholders.