I have a problem with hibernate where it is doing N+1 SELECTS when using a DTO object.
For example, this JPQL query:
SELECT com.acme.MyDto(t) FROM Thing t
Where the constructor of MyDto is something like:
public MyDto(Thing t) {
...
}
Which results in a query of something like:
SELECT t.id FROM thing t WHERE condition
followed by all the single queries for each row, ie:
SELECT t.id, t.column1, t.column2 FROM thing t WHERE t.id = 1
SELECT t.id, t.column1, t.column2 FROM thing t WHERE t.id = 2
SELECT t.id, t.column1, t.column2 FROM thing t WHERE t.id = 3
...
However if the constructor doesn't accept the Entity, but instead each individual column, then hibernate behaves as you would expect, ie:
public MyDto(Integer id, String column1, String column2) {
...
}
Then the generated SQL looks like this:
SELECT t.id, t.column1, t.column2 FROM thing t WHERE condition
Aside from creating DTO constructors that takes every column, is there a way to coax hibernate to just select all the columns at once from the beginning?
The table that we're working with has 100+ columns spread across embeddables, so it's pretty annoying to maintain a huge constructor. The table is hugely normalised and has no joins.
Read your question wrong the first time... I don't recall using DTOs if they just take the whole entity and not just some specific columns, so I'm not sure why Hibernate behaves like that when you use a whole entity as a parameter in DTO constructor. Anyway, you could work around it by just gettin the actual Things via a query and then construct the DTOs in a loop, something along the lines of:
public List<ThingDTO> getThingDTOs( ... )
{
Query query = em().createQuery("FROM Thing t WHERE ...");
query.setParameter( ... );
List<Thing> things = query.getResultList();
List<ThingDTO> thingDTOs = new ArrayList(things.size());
for(Thing t : things)
{
thingDTOs.add(new ThingDTO(t));
}
return thingDTOs
}
It ain't pretty, but this way Hibernate should fetch all the needed rows in one go
As you have probably noticed already, the constructor expression approach has quite some downsides. If you need nested associations it going to get worse. The main problem with working with entity objects here is that you might still run into N + 1 queries problems. I wrote a blog post on that topic a while ago that justifies why I created Blaze-Persistence Entity Views, a library that allows to map DTOs as interfaces.
Related
Hi I used spring data to map my Entity and Repository. The mapping is very simple:
public class Car {
Set<Part> parts;
}
public class Part {
}
I use the findAllByIds(Iterable) interface of my spring data repository. And it generates a nice sql in the form of:
select from CAR where id in (?, ?, ?, ?)
for each Car it executes exactly one SQL.
Select from Part where car_id = ?
My problem starts when the related parts are fetch. It apears that it is fetching them one by one. Is there in spring data jdbc something equivalent to the batch fetching in hibernate ?
If the anser is negative is there some relatively easy way to implement it ?
Unfortunately, the answer is short answer is "No" to both questions right now.
If you want to implement batching for selects what you would need to do is to come up with
a) a new implementation of the DataAccessStrategy which essentially implements all the CRUD functionality, and/or
b) a new EntityRowMapper which converts ResultSet rows into entities.
The first one is needed if you want to execute a different SQL statement to start with.
The second one if you consider changing subsequent SQL sufficient.
There are issues around batching that you might want to track or if the exact variant you are looking for doesn't exist, feel free to create another one.
I would like to create conditions from this select in JOOQ. Because in my case I want to declare a dynamic query and check if TABLE_C.FIELDC contains "foo" only when I need...
Example:
create.select().from(TABLE_A).join(TABLE_B).onKey(Keys.FK_TABLEA_TABLEB)
.join(TABLE_C).onKey(Keys.FK_TABLEB_TABLEC)
.where(TABLE_C.FIELDC.containsIgnoreCase("foo");
to:
SelectFinalStep select = create.select().from(TABLEA);
if (isFooSearched) {
query.addCondition( <JOIN> and <CONTAINS> like first example)
}
How can I do this?
There are several ways to solve this:
Using implicit joins
In relatively simple cases, when the optional join follows a to-one relationship, you may be able to use an implicit join (if you're using the code generator):
create.select()
.from(TABLE_A)
.join(TABLE_B).onKey(Keys.FK_TABLEA_TABLEB)
.where(isFooSearched
? TABLE_B.tableC().FIELDC.containsIgnoreCase("foo")
: noCondition())
.fetch();
Using SEMI JOIN instead of INNER JOIN, which makes dynamic SQL much easier
create.select()
.from(TABLE_A)
.where(
isFooSearched
? TABLE_A.TABLE_B_ID.in(
select(TABLE_B.ID)
.from(TABLE_B)
.join(TABLE_C).onKey(FK_TABLEB_TABLEC)
.where(TABLE_C.FIELDC.containsIgnoreCase("foo"))
)
: trueCondition())
.fetch();
Note that a semi join is also more formally correct in this case than an inner join, as you will not get any duplicate rows on TABLE_A for any matches in to-many relationships (removing them with DISTINCT might be wrong and certainly is inefficient).
Side-note: Not all databases recognise semi-joins in EXISTS or IN syntax, and may thus not optimally run this statement, compared to a JOIN based solution.
Using INNER JOIN as you asked for
// I'm assuming DISTINCT is required here, as you
// - are not interested in TABLE_B and TABLE_C results (semi join semantics)
// - do not want any duplicate TABLE_A values
create.selectDistinct(TABLE_A.fields())
.from(
isFooSearched
? TABLE_A
.join(TABLE_B).onKey(FK_TABLEA_TABLEB)
.join(TABLE_C).onKey(FK_TABLEB_TABLEC)
)
: TABLE_A)
.where(
isFooSearched
? TABLE_C.FIELDC.containsIgnoreCase("foo")
: trueCondition())
.fetch();
I've mad a few assumptions here, including the fact that DISTINCT usage could be correct on your joined variant of the query, but it is hurting you (probably) on your "default" query variant, so perhaps, shoe horning this into a single dynamic query might be overkill.
Thus...
Using two different queries
For my taste, the two queries are simple enough to allow for some duplication and simply run two different queries depending on the flag:
if (isFooSearched)
create.select().from(TABLE_A) /* joins or semi joins here */ .fetch();
else
create.select().from(TABLE_A).fetch();
Side note
All solutions are assuming you have these static imports in your code:
import static org.jooq.impl.DSL.*;
I'm doing a left join of Table A and Table B and trying to fetch the results into a custom POJO which has fields from both Table A and Table B as follows:
List<MyCustomPojo> res = create.select()
.from(TABLE_A)
.leftJoin(TABLE_B)
.on(TABLE_A.MY_CODE.eq(TABLE_B.MY_CODE))
.fetchInto(MyCustomPojo.class);
It works fine for all the fields except for the field myCode the one on which these two tables are joined. For me the values for myCode were picked up from the right table, Table B, which is NULL for all of those records in Table A that do not have a corresponding entry in Table B. I would like to know how jooQ decides which field to map to POJO and if this behavior is documented anywhere.
My goal is to fetch all the fields from Table A and Table B into the custom POJO such that myCode is picked up from the left table. I would appreciate your advice on the right way to achieve it.
The default behaviour of ResultQuery.fetchInto(Class) (and most other into(Class) methods) is specified in DefaultRecordMapper. It can be overridden globally by providing a custom RecordMapperProvider in your Configuration.
In your particular case, DefaultRecordMapper will map all values from your records in field order. If there's a column that appears twice, it will be mapped twice, meaning that the second value will persist in your resulting object. There are two easy workarounds:
Don't select the "wrong" myCode. This is really the most robust solution
List<MyCustomPojo> res = create
.select(TABLE_A.fields())
.select(/* all fields in TABLE_B except the ones you don't want */)
.from(TABLE_A)
.leftJoin(TABLE_B)
.on(TABLE_A.MY_CODE.eq(TABLE_B.MY_CODE))
.fetchInto(MyCustomPojo.class);
Use RIGHT JOIN instead:
Perhaps a bit of a hack, but this will quickly reverse the table order in your SELECT statement.
List<MyCustomPojo> res = create
.select()
.from(TABLE_B)
.rightJoin(TABLE_A)
.on(TABLE_A.MY_CODE.eq(TABLE_B.MY_CODE))
.fetchInto(MyCustomPojo.class);
Finally a use-case for RIGHT JOIN :)
Note, this is the only solution that will also prevent wrong values for other columns that "accidentally" share the same name.
Add the "correct" myCode field once more.
Another hack, but it will work around the issue you're experiencing:
List<MyCustomPojo> res = create
.select(TABLE_A.fields())
.select(TABLE_B.fields())
.select(TABLE_A.MY_CODE)
.from(TABLE_A)
.leftJoin(TABLE_B)
.on(TABLE_A.MY_CODE.eq(TABLE_B.MY_CODE))
.fetchInto(MyCustomPojo.class);
I am attempting to create a domain/entity class based on a complex query. The query unions a bunch of tables together and unfortunately I am not able to create a view on the database for this query. I have been trying to set up the entity object but I am unsure of how to ensure that the marshaling works properly (and ensure the entity acts as read-only object).
As an example of the query, I am doing something like this:
Select
U_T.a,
U_T.b,
U_T.c,
C_T.a
FROM
(select
A_T.a,
null as b,
A_T.c,
1 as ind
from A_T
UNION
select
B_T.a,
B_T.b,
null,
0 as ind
FROM B_T
) U_T
left outer join C_T on C_T.fk_a = U_T.a;
The other issues are that this union can result in instances where there is no unique key column. This is fine as this data is for viewing only, and never editing. However the #Entity annotation wants a column to be listed with the #ID annotation. Another issue is that I do not believe I can use the other entity classes as the goal is to reduce the number of DB transactions from this query (as the actual one can result in hundreds of recursive queries being performed).
If I need to give any more information please let me know.
Please help me with these Hibernate querying issues.
Consider the following structure:
#Entity
class Manager {
#OneToMany
List<Project> projects;
}
0) there are 2 possible ways of dynamic fetching in HQL:
select m from Manager m join m.projects
from Manager m join fetch m.projects
In my setup second one always returns a result of cartesian product with wrong number of objects in a list, while the first one always returns correct number of entities in a list. But the sql queries look the same. Does this mean that "select" clause removes redundant objects from the list in-memory? In this case its strange to see an advice in a book to use select distinct ... to get rid of redundant entities, while "select" does the job. If this is a wrong assumption than why these 2 queries return different results?
If I utilize dynamic fetching by one of the 2 methods above I see a classic n+1 select problem output in my hibernate SQL log. Indeed, FetchMode annotations (subselect or join) do not have power while fetching dynamically. Do I really can't solve the n+1 problem in this particular case?
Looks like Hibernate Criteria API does not support generics. Am I right? Looks like I have to use JPA Criteria API instead?
Is it possible to write HQL query with an entity name parameter inside? For example "from :myEntityParam p where p.id=1" and call setParameter("myEntityParam", MyClass.class) after this. Actually what I want is generic HQL query to replace multiple non-generic dao's by one generic one.
0) I always use a select clause, because it allows telling what you want to select, and is mandatory in JPQL anyway. If you want to select the managers with their projects, use
select distinct m from Manager m left join fetch m.projects
If you don't use the distinct keyword, the list will contain n instances of each manager (n being the number of projects of the manager): Hibernate returns as many elements as there are rows in the result set.
1) If you want to avoid the n + 1 problem, fetch the other association in the same query:
select distinct m from Manager m
left join fetch m.projects
left join fetch m.boss
You may also configure batch fetching to load 10 bosses (for example) at a time when the first boss is accessed. Search for "batch fetching" in the reference doc.
2) The whole Hibernate API is not generified. It's been made on JDK 1.4, before generics. That doesn't mean it isn't useful.
3) No. HQL query parameters are, in the end, prepared statement parameters. You must use String concatenation to do this.