We Keep Coding

Spring JPA - Reading data along with relations - performance improvement

I am reading data from a table using Spring JPA.
This Entity object has one-to-many relationship to other six tables.
All tables together has 20,000 records in them.
I am using below query to fetch data from DB.
SELECT * FROM A WHER ID IN (SELECT ID FROM B WHERE COL1 = '?')
A table has relationship to other 6 tables.
Spring JPA is taking around 30 seconds of time to read this data from DB.
Any idea to improve the data fetch time here.
I am using native Queries here and i am looking for query rewriting that will optimize the data fetch time.
Please suggest thanks.

You might need consider below to identify the root cause:
Check if you are ending up with n+1 query issue. Your query might end up calling n queries for each join table, where n is no. of associations with the join table. You can check this by setting spring.jpa.show-sql=true
If you see the issue as n+1 then you need set appropriate FetchMode, refer https://www.baeldung.com/hibernate-fetchmode for detailed explanation of using different FetchModes.
If it is not n+1 query issue you might need to check the performance of the genarated queries using EXPLAIN command. Usually IN clause on a non indexed columns have performance impact.
So set spring.jpa.show-sql=true and check queries generated and run to debug and optimize your code or query.

Hibernate NonUniqueObjectException

I'm getting NonUniqueObjectException in hibernate.
There is one Item class, I saved list of Item objects using session.save of hibernate.
Now in the same transaction, I'm trying to update same Items using raw sql query which has join with another table. This gives me NonUniqueObjectException. The two tables I'm joining are unrelated as entities for hibernate, that is, there is no foreign key relation.
So I have 2 questions:
First, is there any way of using hql for writing inner join queries in hibernate.
Second, how to avoid NonUniqueObjectException.

One of the things that is working is that I clear the session before making any raw sql query. Any better approach is welcomed.

Fetch efficiently huge number of entities from DB meeting criteria

Let's say I have about half of million records in table A. Of course table A is joined with table B, C, etc. Now I have to fetch entities from table A which meet my criteria. My criteria consists of about 20-30 rules e.g. name in table A has to be like 'something' or date from table B for joined record from table A with ID=1 should be earlier than today. I see three solutions:
Write native query and put parameters from my criteria. But in this case I will join so many tables that it doesn't seem to me as a good example.
Fetch all records from table A and then check in Java every rule for each record. But this seems for me as the worst possible solution.
Use JPA Criteria. But to be honest I do not know how efficient they are while there are so many records and so many joined tables. What's more it seems to me like working with Criteria can be a little irritating when I have so many rules to match.
Maybe there is another (better) solution to my problem but I cannot see it now. I need to add that I need these fetched entities stored in Java collection because when they are all fetched then I have to work with them (e.g. generate report or create some updates in DB basing on these information).
I hope I described my problem clear and I will be thankful for every tip how to optimize such query.

What method of joining entities is faster?

Here are two queries retrieving equivalent data:
SELECT DISTINCT packStatus
FROM PackStatus packStatus JOIN FETCH packStatus.vars, ProcessEntity requestingProcess
WHERE
packStatus.status.code='OGVrquestExec'
AND packStatus.procId=requestingProcess.activityRequestersProcessId
AND requestingProcess.id='1000323733_GU_OGVProc'
SELECT DISTINCT packStatus
FROM PackStatus packStatus JOIN FETCH packStatus.vars
WHERE
packStatus.status.code='OGVrquestExec'
AND packStatus.procId=(SELECT requestingProcess.activityRequestersProcessId FROM ProcessEntity requestingProcess WHERE requestingProcess.id='1000323733_GU_OGVProc')
These queries differ in the method how requstingProcess is joined with packStatus. In general, which of these two methods is more preferable in terms of performance? I'm using JPA 1.2 provided by Hibernate 3.3 on Postgres 8.4.
UPD: I've replaced fake queries with real queries from my app. Here is SQL generated by Hibernate for first and second query. Links to query plans: first, second. Query plans look pretty the same. The only difference is what moment data from bpms_process table is aggregated to query result at. But I don't know is it right to generalize these results? Would query plans be almost the same for queries differing only in joining method? Is it possible to get a big difference in query cost by changing joining method?

Use EXPLAIN ANALYZE and see.
I won't be surprised if they get turned into the same query plan.
See: https://stackoverflow.com/tags/postgresql-performance/info

Change of design of queries to improve performance

This is more like a design question but related to SQL optimization as well.
My project has to import a large number of records into the database (more than 100k records). In the meantime, the project has logic to check each record to make sure it meets the criteria which are configurable. It then will mark the record as no warning or has warning in the database. The inserting and warning checking are done within one importing process.
For each criteria it has to query the database. The query needs to join two other tables and sometimes add additional nested query inside the conditions, such as
select * from TableA a
join TableB on ...
join TableC on ...
where
(select count(*) from TableA
where TableA.Field = Bla) > 100
Although the queries take unnoticeable time, to query the entire record set takes a considerable amount of time which may be 4 - 5 hours on a server. Especially if there are many criteria, at the end the project will stop running the import and rollback.
I've tried changing "SELECT * FROM" to "SELECT TableA.ID FROM" but it seems it has no effect at all. Is there a better design to improve the performance of this process?

How about making a temp table (or more than one) that stores the aggregated results of the sub-queries, then indexing that/those with covering indexes.
From your code above, we'd make a temp table grouping on TableA.Field1 and including a count, then index on Field1, theCount. On SQL server the fastest approach would then be:
select * from TableA a
join TableB on ...
join TableC on ...
join (select Field1 from #temp1 where theCount > 100) t on...
The reason this works is that we are doing the same trick twice.
First, we pre-aggregate into the temp table, which is a simple operation and very easy for SQL Server to optimize. So we have taken a piece of the problem and solved in an optimizable way.
Then we repeat this trick by joining to a subquery, putting the filter inside the subquery, so that the join acts as a filter.

I would suggest you batch your records together (500 or so at a time) and send it to a stored proc which can do the calculation.
Use simple statements instead of joins in there. That saves as well. This link might help as well.

Good choice is using indexed view.
http://msdn.microsoft.com/en-us/library/dd171921(SQL.100).aspx