We Keep Coding

Hibernate is making extra SQL statement with #ManyToOne and #Lazy fetching object

I would like someone to explain me why Hibernate is making one extra SQL statement in my straight forward case. Basically i have this object:
#Entity
class ConfigurationTechLog (
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
val id: Long?,
val configurationId: Long,
val type: String,
val value: String?
) {
#JsonIgnore
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "configurationId", insertable = false, updatable = false)
val configuration: Configuration? = null
}
So as you can see, nothing special there. And when i execute this query :
#Query(value = "SELECT c FROM ConfigurationTechLog c where c.id = 10")
fun findById10() : Set<ConfigurationTechLog>
In my console i see this:
Hibernate:
/* SELECT
c
FROM
ConfigurationTechLog c
where
c.id = 10 */ select
configurat0_.id as id1_2_,
configurat0_.configuration_id as configur2_2_,
configurat0_.type as type3_2_,
configurat0_.value as value4_2_
from
configuration_tech_log configurat0_
where
configurat0_.id=10
Hibernate:
select
configurat0_.id as id1_0_0_,
configurat0_.branch_code as branch_c2_0_0_,
configurat0_.country as country3_0_0_,
configurat0_.merchant_name as merchant4_0_0_,
configurat0_.merchant_number as merchant5_0_0_,
configurat0_.org as org6_0_0_,
configurat0_.outlet_id as outlet_i7_0_0_,
configurat0_.platform_merchant_account_name as platform8_0_0_,
configurat0_.store_type as store_ty9_0_0_,
configurat0_.terminal_count as termina10_0_0_
from
configuration configurat0_
where
configurat0_.id=?
Can someone please explain me, what is happening here ? From where this second query is coming from ?

I assume you are using Kotlin data class. The kotlin data class would generate toString, hashCode and equals methods utilizing all the member fields. So if you are using the returned values in your code in a way that results in calling of any of these method may cause this issue.
BTW, using Kotlin data claases is against the basic requirements for JPA Entity as data classes are final classes having final members.

In order to make an association lazy, Hibernate has to create a proxy instance instead of using the real object, i.e. it needs to create an instance of dynamically generated subclass of the association class.
Since in Kotlin all classes are final by default, Hibernate cannot subclass it so it has to create the real object and initialize the association right away. In order to verify this, try declaring the Configuration class as open.
To solve this without the need to explicitly declare all entities open, it is easier to do it via the kotlin-allopen compiler plugin.

This Link can be useful for understand what kind (common) problem is that N + 1 Problem
Let me give you an example:
I have three Courses and each of them have Students related.
I would like to perform a "SELECT * FROM Courses". This is the first query that i want (+ 1) but Hibernate in background, in order to get details about Students for each Course that select * given to us, will execute three more queries, one for each course (N, there are three Course coming from select *). In the end i will see 4 queries into Hibernate Logs
Considering the example before, probably this is what happen in your case: You execute the first query that you want, getting Configuration Id = 10 but after, Hibernate, will take the entity related to this Configuration, then a new query is executed to get this related entity.
This problem should be related in specific to Relationships (of course) and LAZY Fetch. This is not a problem that you have caused but is an Hibernate Performance Issue with LAZY Fetch, consider it a sort of bug or a default behaviour
To solve this kind of problem, i don't know if will be in your case but ... i know three ways:
EAGER Fetch Type (but not the most good option)
Query with JOIN FETCH between Courses and Students
Creating EntityGraph Object that rappresent the Course and SubGraph that rappresent Students and is added to EntityGraph

Looking at your question, it seems like an expected behavior.
Since you've set up configuration to fetch lazily with #ManyToOne(fetch = FetchType.LAZY), the first sql just queries the other variables. When you try to access the configuration object, hibernate queries the db again. That's what lazy fetching is. If you'd like Hibernate to use joins and fetch all values at once, try setting #ManyToOne(fetch = FetchType.EAGER).

Hibernate creating N+1 queries for #ManyToOne JPA annotated property

I have these classes:
#Entity
public class Invoice implements Serializable {
#Id
#Basic(optional = false)
private Integer number;
private BigDecimal value;
//Getters and setters
}
#Entity
public class InvoiceItem implements Serializable {
#EmbeddedId
protected InvoiceItemPK invoiceItemPk;
#ManyToOne
#JoinColumn(name = "invoice_number", insertable = false, updatable = false)
private Invoice invoice;
//Getters and setters
}
When i run this query:
session.createQuery("select i from InvoiceItem i").list();
It executes one query to select the records from InvoiceItem, and if I have 10000 invoice items, it generates 10000 additional queries to select the Invoice from each InvoiceItem.
I think it would be a lot better if all the records could be fetched in a single sql. Actually, I find it weird why it is not the default behavior.
So, how can I do it?

The problem here is not related to Hibernate but to JPA.
Prior to JPA 1.0, Hibernate 3 used lazy loading for all associations.
However, the JPA 1.0 specification uses FetchType.LAZY only for collection associations:
#OneToMany,
#ManyToMany
#ElementCollection)
The #ManyToOne and #OneToOne associations use FetchType.EAGER by default, and that's very bad from a performance perspective.
The behavior described here is called the [N+1 query issue][5], and it happens because Hibernate needs to make sure that the #ManyToOne association is initialized prior to returning the result to the user.
Now, if you are using direct fetching via entityManager.find, Hibernate can use a LEFT JOIN to initialize the FetchTYpe.EAGER associations.
However, when executing a query that does not explicitly use a JOIN FETCH clause, Hibernate will not use a JOIN to fetch the FetchTYpe.EAGER associations, as it cannot alter the query that you already specified how to be constructed. So, it can only use secondary queries.
The fix is simple. Just use FetchType.LAZY for all associations:
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "invoice_number", insertable = false, updatable = false)
private Invoice invoice;
More, you should use the Hypersistence Utils to assert the number of statements executed by JPA and Hibernate.

Try with
session.createQuery("select i from InvoiceItem i join fetch i.invoice inv").list();
It should get all the data in a single SQL query by using joins.

Yes there is setting you need: #BatchSize(size=25). Check it here:
20.1.5. Using batch fetching
small cite:
Using batch fetching, Hibernate can load several uninitialized proxies if one proxy is accessed. Batch fetching is an optimization of the lazy select fetching strategy. There are two ways you can configure batch fetching: on the class level and the collection level.
Batch fetching for classes/entities is easier to understand. Consider the following example: at runtime you have 25 Cat instances loaded in a Session, and each Cat has a reference to its owner, a Person. The Person class is mapped with a proxy, lazy="true". If you now iterate through all cats and call getOwner() on each, Hibernate will, by default, execute 25 SELECT statements to retrieve the proxied owners. You can tune this behavior by specifying a batch-size in the mapping of Person:
<class name="Person" batch-size="10">...</class>
With this batch-size specified, Hibernate will now execute queries on demand when need to access the uninitialized proxy, as above, but the difference is that instead of querying the exactly proxy entity that being accessed, it will query more Person's owner at once, so, when accessing other person's owner, it may already been initialized by this batch fetch with only a few ( much less than 25) queries will be executed.
So, we can use that annotation on both:
collections/sets
classes/Entities
Check it also here:
#BatchSize but many round trip in #ManyToOne case

In this Method there are Multiple SQLs fired. This first one is fired for retrieving all the records in the Parent table. The remaining are fired for retrieving records for each Parent Record. The first query retrieves M records from database, in this case M Parent records. For each Parent a new query retrieves Child.

JQPL Update Query to update entity without using the primary key

This may be a simple question, but I'm trying to find out if there is a way that I can create a JPQL update query that would allow me to update a single Persisted Entity using a unique column identifier that is not the primary key.
Say I have and entity like the following:
#Entity
public class Customer {
#ID
private Long id;
#Column
private String uniqueExternalID;
#Column
private String firstname;
....
}
Updating this entity with a Customer that has the id value set is easy, however, id like to update this customer entity using the uniqueExternalId without having to pre-query for the local entity and merge the changes in or manually construct a jpql query with all the fields in it manually.
Something like
UPDATE Customer c SET c = :customer WHERE c.uniqueExternalId = :externalId
Is something like this possible in JQPL?

You cannot do it in the exact way you describe - by passing an entity reference, but you can use bulk queries to achieve the same effect.
UPDATE Customer c SET c.name = :name WHERE c.uniqueExternalId = :externalId
Please note that you will have to explicitly define each updated attribute.
It is important to note that bulk queries bypass the persistence context. Entity instances that are managed within the persistence context will not reflect the changes to the records that are changed by the bulk update. Further, if you use optimistic locking, consider incrementing the #Version field of your entities with the bulk update:
UPDATE Customer c SET c.name = :name, c.version = c.version + 1 WHERE c.uniqueExternalId = :externalId
EDIT: The JPA 2.0 spec advises in § 4.10:
In general, bulk update and delete operations should only be performed
within a transaction in a new persistence context or before fetching
or accessing entities whose state might be affected by such
operations.

Why is JPA retrieving one column using a seperate select

Problem description
Hello, I've been using OpenJPA 2.1.2 (JPA 2.0) to retrieve data from a table called LOGISCHRAPPORTwhich has about 28 columns. In some cases OpenJPA decides to issue seperate select statements for one of the columns for no particular reason, leading to a dramatic decrease in query performance.
Everything is fine until...
Initially everything goes fine and all my columns are retrieved in a performant, single SELECT statement by JPA.
As soon as I add a relationship to another entity called RAPTAALMETADATA
#OneToMany(fetch=FetchType.EAGER, cascade = CascadeType.ALL)
#JoinColumns({
#JoinColumn(name = "RAPPORTNR", referencedColumnName = "RAPPORTNR"),
#JoinColumn(name = "RAPPORTTYPE", referencedColumnName = "RAPPORTTYPE") })
private List<Raptaalmetadata> raptaalmetadata;
---
Queried using Criteria API as follows:
---
Join<LogischRapport, Raptaalmetadata> metadata = reportRoot.join(
"raptaalmetadata");
JPA no longer includes one of my original columns called REPORT_COMMENTS instead it is issuing separate select statements to retrieve the REPORT_COMMENTS column for each instance of LOGISCHRAPPORT. All other columns (including the ones coming from RAPTAALMETADATA are retrieved properly as part of the intial SELECT.
REPORT_COMMENTS is of the HUGEBLOB type in Oracle and I've mapped in in my Entity as follows:
#Lob
#Basic
#Column(name = "REPORT_COMMENTS")
private byte[] reportComments;
I now get tons of these:
SELECT t0.REPORT_COMMENTS
FROM dwhsd001.LogischRapport t0
WHERE t0.rapportnr = ? AND t0.rapporttype = ?
[params=(long) 1473, (String) RAP]
Can anyone point out why I might be seeing this behavior and how I can avoid it?
Its not a LAZY loading problem as I've specifically tested that case. I don't see any other reason why OpenJPA decides to retrieve this one column using separate statements.
Update 1
Interesting addition: as soon as I remove the fetch=FetchType.EAGER attribute from the #OneToMany annotation described above I start seeing the exact same behavior for the relationship as I've been getting for the REPORT_COMMENTS column. This means I'm also getting separate SELECT statements for retrieving the entity relationship on top of the seperate selects for the column thereby further degrading performance.
In other words I'm then also getting tons of these:
SELECT t0.isotaalcode, t0.rapportnr, t0.rapporttype,
t0.FUNCDESC_MODIFIED_BY, t0.FUNCDESC_MODIFIED_DATE,
t0.FUNCTIONAL_DESCRIPTION, t0.omschrijving, t0.titel
FROM dwhsd001.Raptaalmetadata t0
WHERE t0.rapportnr = ? AND t0.rapporttype = ?

Hibernate: When to use #Index annotation

In my scenario, I have a schema generation script to create tables and required indexes. I am wondering is there any need to define #Index annotation in hibernate entities as well, if so why?
Script:
create table issues (id, project_id, .., status_id)
create index idx_issues_projid on issues (project_id)
Entity:
#Table(name="issues")
public class DBIssue {
..
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "PROJECT_ID")
#Index(name="INDEX_TFW_ISSUE_PROJECT_ID")
private DBProject project;
}
Hibernate configuration:
<property name="hibernate.hbm2ddl.auto">off</property>

I presume you're asking about the Hibernate #Index annotation, which has essentially been imported into JPA 2.1. You would use #Index anywhere you would otherwise proactively tell a relational database to index a column, primarily on a field where you know you'll be doing lots of lookups. In this case, for example, you are probably going to want to "select the DBIssues belonging to a particular DBProject frequently, so it would make sense to index that column in the table holding DBIssue.