I have a Hibernate entity that is comprised of many other entities that are used within the application. The other entities that make up this MainEntity are joined by using #ManyToOne and #JoinColumn. This MainEntity class has 5 columns (#Column) and 7 #ManyToOne/#JoinColumn entities that are used.
I seem to be running into performance issues when retrieving all of these MainEntity classes. We want to serialize the MainEntity to JSON as well as the other entities that are associated with it. Note that there aren't that many that we are retrieving - less than 30 total.
Below is an example of what the class looks like along with my findAll() method to retrieve these classes. I know that #ManyToOne is EAGER by default, so I'm wondering if there's a better way to get all of these entities that is easier on the system. Thank you in advance.
#Entity(name = "MainEntity")
#Table(name = "main_entity")
public class MainEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Integer id;
// Other #Columns defined here
#ManyToOne()
#JoinColumn(name = "entity_1_id")
private Entity1 entity1;
#ManyToOne()
#JoinColumn(name = "entity_2_id")
private Entity2 entity2;
#ManyToOne()
#JoinColumn(name = "entity_3_id")
private Entity3 entity3;
// ... and so on, for a total of 7 #ManyToOne() columns
}
Here is the findAll() method that I have:
final List<E> findAllOrdered(Class<E> clazz, Order order) {
final Session session = sessionManager.openNewSession();
try {
return session.createCriteria(clazz)
.addOrder(order)
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY)
.list();
} finally {
sessionManager.closeSession(session);
}
}
I found myself having to add the Criteria.DISTINCT_ROOT_ENTITY because we were getting duplicate MainEntity results if a child had multiple associated with it. I suspect this is big part of my performance problem.
If you are retrieving unwanted response and if you want to filter then you may use #JsonIgnore
eg:
#ManyToOne()
#JoinColumn(name = "entity_1_id")
#JsonIgnore
private Entity1 entity1;
Few pointers to consider:
Consider making associations Lazy by default unless you really want to load all the association data and its associations along the parent.
Use JOIN in HQL/criteria based on which association we really want to fetch and the depth of associations.
Or use EntityGraph to decide which associations to be fetch.
Enable show_sql as this show the number of SQLs and the exact SQLs that are getting fired to the DB. This would be a good starting point and subsequently you can tune you associations to LAZY/EAGER, SELECT/JOIN/SUBSELECT based on your use case.
You can run these queries against the DB and see if tuning the query/DB (indexes, partitioning etc) will help reduce the query times.
See if second level cache would help for your use case. Note that second level cache will come with its own complexity and extra overhead and especially if the data is of transactional type and not read-only mostly. With application deployed on nodes maintaining the cache coherence will be another aspect to think about. Need to validate if the extra overhead and complexity is really worth the efficiency outcome of the second level cache.
From an application design perspective, you can also consider and see if you really want to retrieve the MainEntity and the associations in a single request or UI. Instead we could first show the MainEntity with some paging and based on the selection we could fetch the associations for that MainEntity with paging.
Note that, this is not a complete list. But a good starting point and based on your use case you can see which one would fit for you and any other additional techniques.
Related
I have a class Entry which has two fields serving auditing purposes: startAuditAction and endAuditAction. One audit action can affect several entries, therefore the class Entry describes ManyToOne relationships as follows:
public class Entry{
#Id
#Column(nullable = false)
protected String path;
#Id
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(nullable = false, name = "start_action_id")
protected AuditAction startAction;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(updatable = true, nullable = true, name = "end_action_id")
protected AuditAction endAction;
}
I want to retrieve instances of Entry based on conditions on the path field and the audit fields. For example to retrieve entries which have not yet been deleted, the HQL would look something like that:
SELECT DISTINCT entry FROM ENTRY_TABLE entry JOIN FETCH entry.startAction startAct LEFT JOIN FETCH entry.endAction endAct WHERE entry.path LIKE myPath and endAct IS NULL
I am using lazy loading together with JOIN FETCH to avoid the N+1 problem while still being able to access the audit fields. However, I have two problems with this:
Firstly, this really does not seem clean to me: if I know I want to access the audit fields (namely the audit actions timestamp), then they should not be lazy loaded. But if I use eager loading I am facing the n+1 problem even if I use JOIN FETCH (and in that case I do not understand why fetch = FetchType.EAGER would ever be useful)...
Secondly, even though I am avoiding the n+1 problem and therefore firing less SQL queries, I get some performance issues for the overall use of my database, probably because of the joins.
What is the proper way to avoid firing additional queries while preserving a good throughput ?
Thanks!
1- Using join fetch is useful when you have FetchType.LAZY in a field that you know you'll need in that specific case whereas using FetchType.EAGER will force that entity to always load the collection independently from the query
(e.g. with your same configuration example you can do multiple query and only when you need the collection use the JOIN FETCH)
2- You probably have problems somewhere else, i doubt the join is what is slowing you down
I have mapped an 1:N relation with a #OneToMany List, but when I access the list, the results are duplicated due to an OUTER JOIN.
This is how the mapping looks like:
#Entity
public class Programmer
#ElementCollection(fetch=FetchType.EAGER)
#CollectionTable(name="emails", joinColumns=#JoinColumn(name="id", nullable=false))
#Column(name="email", nullable=false)
protected Set<String> emails = new HashSet<String>();
#OneToMany(mappedBy="programmer", fetch=FetchType.EAGER)
private List <Game> games = new ArrayList<Game>();
When I get the attribute with prog.getGames(), the results comes duplicated because the Hibernate SQL makes an OUTER JOIN:
from programmer
left outer join emails on programmer.id=emails.id
left outer join game on programmer.id=game.id
where programmer.id=?
Is there any solution without transforming the List into a Set? I need to get the games with prog.getGames(), can not use a custom HQL or Criteria.
While the use of Set<> fundamentally resolves your issue, I'd argue that is simply a bandaid to get the expected results you're after but it doesn't technically address the underlying problem.
You should ultimately be using the default lazy fetch strategy because I'm of the opinion that eagerly loading any associations, particularly collection-based ones, are specific to a query and therefore should be toggled when you construct specific queries and not influenced as a part of your entity mapping model as you're doing.
Consider the future where you add a new query but you're only interestesd in attributes from the aggregate root entity. Your mapping model will still impose eagerly fetching those associations, you'll consume additional resources to by having a larger persistence context which means more memory consumption and impose unnecessary database joins for something which you aren't going to use.
If there are multiple collections that you need to hydrate, I would instead recommend you consider using FetchMode.SUBSELECT instead.
If we assume your query has 10 entities being returned, the default lazy strategy with 2 collections would issue 21 queries (1 for the base result set and 2 for each loaded entity).
The benefit of SUBSELECT is that Hibernate will actually only issue 3 queries (1 for the base result set and 1 for each collection to load all collection elements for all entities). And obviously, depending on certain queries, breaking one query with left-joins into 3 queries could actually perform better at the database level too.
Ive resolved this problem with #Fetch(FetchMode.SUBSELECT)
#OneToMany(mappedBy = "user", fetch = FetchType.EAGER, cascade = CascadeType.ALL)
#Fetch(FetchMode.SUBSELECT)
private List<CompanyUserEntity> companyUserRelations;
I had the same problem. companyUserRelations had duplicate objects (I mean the same pointers to the same object, not duplicated data)
So after reading #dimitry response, I added #Fetch(FetchMode.SUBSELECT) and it worked
I'm using Hibernate 4.2.3 and I have a class similar to the following:
#Entity
#DynamicInsert
#DynamicUpdate
#SelectBeforeUpdate
public class Test {
#Id
private BigInteger theId;
#Lob
#Basic(fetch = FetchType.LAZY)
#JsonIgnore
private Blob data;
#Lob
#Basic(fetch = FetchType.LAZY)
#JsonIgnore
private Blob otherData;
// Getters and setters....
}
The sql that this is generating for an update includes the data column, even though it hasn't changed. (To be precise, what I do is get the object, detach it, read the data and use that to generate otherData, set that and then call saveOrUpdate on the session.)
Can anyone explain why this would happen? Does this functionality work with Blobs? I've searched for documentation but found none.
PS I'm not using #DynamicUpdate for performance reasons. I know that it would be questionable to use it from that standpoint.
The safest and most portable (between different databases and JPA providers) way to achieve real lazy loading of Lobs is to create an artificial lazy one-to-one association between the original entity and a new one to which you move the Lob.
This approach is suitable for other kinds of optimizations as well, for example when I want to enable second-level caching of a complex entity, but a few columns of the entity are updated frequently. Then I extract those columns to a separate non-second-level-cacheable entity.
However, keep in mind general pitfalls specific to one-to-one associations. Basically, either map it with a mandatory (optional = false) one-to-one association with #PrimaryKeyJoinColumn or make sure the foreign key is in the entity (table) which declares the lazy association (in this case the original entity from which the Lob is moved out). Otherwise, the association could be effectively eager, thus defeating the purpose of introducing it.
How to make hierarhical mapping Hibernate?
For example:
Category
id parent_id name
1 0 Root
2 1 Sub-root 1
3 1 Sub-root 2
4 2 Sub-(sub-root 1)
Is it possible to make lazy mapping for such Category object?
It is not exactly clear what you are asking.
However it would appear you are talking about a self-referencing relationship rather than Inheritance so you can then map as below. The default fetch strategy should be same as for any other #OneToMany i.e. LAZY.
#Entity
public class Category{
#Id
private Long id;
#ManyToOne
#JoinColumn(name = "parent_id")
private Category parent;
#OneToMany(mappedBy = "parent")
private Set<Category> subCategories;
}
I believe you want to ask about inheritance of entities. I recommend using JPA inheritance strategies. There are 3 available.
Single Table: Uses only one database table. columns need to be nullable and hence wastes database space
Joined Strategy: Uses multiple table which can be joined for insertion and retrieval of entity data. Saves database space but performance becomes an issue when inheritance hierarchy becomes wide and deep
Table per concrete class: Uses separate database tables which are not joined.
Different strategies have different advantages and disadvantages. You can choose according to your need.
I have a #ManyToMany relationship between two entities. When I perform an update on the owning side, it appears that JPA deletes all the linked records from my database and re-inserts them. For me this is a problem because I have a MySQL trigger that fires before a record is deleted. Any ideas on how to get around this problem?
#Entity
public class User {
#Id
#Column(name="username")
private String username;
...
#ManyToMany
#JoinTable(name="groups", joinColumns=
#JoinColumn(name="username", referencedColumnName="username"),
inverseJoinColumns=#JoinColumn(name="groupname",
referencedColumnName="type_id"))
private List<UserType> types;
...
}
#Entity
public class UserType {
#Id
#Column(name="type_id")
private String id;
#ManyToMany(mappedBy="types")
private List<User> users;
...
}
Use Set instead of List solved the problem. But I have no idea why it works.
Another solution provided by Hibernate is to split the #ManyToMany association into two bidirectional #OneTo#Many relationships. See Hibernate 5.2 documentation for example.
If a bidirectional #OneToMany association performs better when
removing or changing the order of child elements, the #ManyToMany
relationship cannot benefit from such an optimization because the
foreign key side is not in control. To overcome this limitation, the
link table must be directly exposed and the #ManyToMany association
split into two bidirectional #OneToMany relationships.
Try this one:
1) change declaration to:
private List<UserType> types = new Vector<UserType>();
2) never call
user.setTypes(newTypesList)
3) only call
user.getTypes().add(...);
user.getTypes().remove(...);
Its probably related to this question. You have to ensure you have an appropriately defined hashCode an equals method in your mapped object so that Eclipselink can determine equality and thus determine that the existing objects map to existing objects in the DB. Otherwise it has no choice but to recreate the child objects every time.
Alternatively, I've read that this kind of join can only support efficient adding and removing of list items if you use an index column, but that's going to be EclipseLink specific, since the JPA annotations don't seem to support such a thing. I know there is an equivalent Hibernate annotation, but I don't know what it would be in Eclipselink, if such a thing exists.
It appears my problem was that I was not merging the entity.