I have a database table mapped with ORMlite, it contains some data (18 columns) and it also contains ForeignCollectionField(eager = true).
Problem is when loading all data from this table ... ORMlite is creating query for every item instead using joins. Which is resulting in 67124 queries and taking forever to load all objects from this table.
This could be however done in right join query under few seconds? Why to generate thousands of queries instead?
How can I speed it up? Do I have to write raw query and then RawRowMapper , which makes using ORM pointless..
How to deal with loading eager collections in ormlite? Because queryForAll is not way..
Problem is when loading all data from this table ... ORMlite is creating query for every item instead using joins. Which is resulting in 67124 queries and taking forever to load all objects from this table.
It's ORM_Lite_ for a reason. Lot of people have asked for the join support on foreign collections but I've not gotten to it yet. It's not easy.
If you still want to use ORMLite then I'd recommend not using eager = true and doing 2 queries instead. One query for your main item and then another query using the DAO associated with the collection entity using IN. Something like:
qb = accountDao.queryBuilder();
qb.where()...;
List<Account> accounts = qb.query();
// build a list of account-ids
List<Long> accountIds = new ArrayList<>();
for (Account account : accounts) {
accountIds.add(account.getId());
}
// now use this list of ids to get your other entities
List<Order> orders = orderDao.queryBuilder().where().in("accountId", accountIds).query();
// now you have a list of orders for all of your ids
// you will need to associate each order with its account
Hope this helps.
Related
I am using hibernate in my application to update bulk entries in list.
I have a list of employees which I get from the database by running particular criteria query. As the whole method is under transactional annotation.I want to udpate each employee in this list. So i can update the employee objects in the list, by just calling setter methods on employee objects , there is no need to call update method explicitly, as the whole list objects are under persistent context.
As of now, I am using for loop to iterate the list and updating each employee by calling setter method
like below:
for(Employee emp:empList){
emp.setAge(5);
emp.setAddress("ABC");
}
Now updating like this takes lot of time ,if I have 1000 employees in the list. how can i apply bulk operation when I am updating the fields in the persistent context by setting corresponding setter values and not by calling update method.
Thanks
try this if you are using JPA 2.1.
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaUpdate<Employee> update = cb.createCriteriaUpdate(Employee.class);
Root<Employee> root = update.from(Employee.class);
//Acces properties by metamodel entity
root.set(Employee_.age, 5);
root.set(Employee_.address, "ABC");
//Add conditions where
update.where([...])
Query query = em.createQuery(update);
query.executeUpdate();
Be careful if you perform more write operations in the same method because the bulk operations of criteria do not respect the order of execution of the transaction.
A solution we implemented in order to collect a large amount of heavy objects in reasonable time and without memory overflow (I'm speaking of objects with multiple fetchType.eager relations, to entities that themselves have eager fetched relations), was to first select ids of those objects, and then select the objects themselves based on these ids.
Coming the time to optimise our code, we noticed (using hibernate.show_sql=true) that the query we used to gather those objects (select a from A a where a.id in :ids) is translated by JPA/Hibernate into thousands of queries of the form select a from ...endless join list... where a.id = ?
The question is the following:
Why JPA/Hibernate transforms our initial query with an "in" clause into so many queries with "=" clauses. Isn't it inefficent ? If so, is there a way to prevent that ?
Here's how the query is called in our code:
Query q = this.getContext().createQuery("select a from A a where a.id in :ids");
q.setParameter("ids", idList);
return (List<A>) q.getResultList();
Hi you should use setParameterList instead of setParameter.
Also no need to use select in hibernate if you want to get the complete entity object.
Query q = this.getContext().createQuery("from A a where a.id in (:ids)");
q.setParameterList("ids", idList);
return (List<A>) q.getResultList();
If your idList is more that 1000 elements, you probably hit this Oracle-specific deoptimization https://hibernate.atlassian.net/browse/HHH-9299.
One way to solve this is to break idList into multiple chunks, execute a query for each and concatenate the results.
Though I still can't explain why a query is generated for each ID provided in the in clause (provided in my original question), I found a solution to this problem on this blog (https://thoughts-on-java.org/fetch-multiple-entities-id-hibernate/).
The solution consists in using Hibernate's session API as follows:
//get session object from entity manager
Session session = em.unwrap(Session.
MultiIdentifierLoadAccess<A> multiLoadAccess = session.byMultipleIds(A.class);
List<A> aObjects= multiLoadAccess.withBatchSize(1000).multiLoad(idList);
return aObjects;
I have entity named Test1, it has 2 fields id and name.
I put about 1000 entity Test1 into a ArrayList with first entity has id = 1, second entity has id = 2, and so on... to 1000.
I use EclipseLink 2.4 provider to communication Postgresql database.
When I store List into database using following code:
EntityManager em;
List<Test1> list = new ArrayList<Test1>();
//populate data into list
for(Test1 test1 : list){
em.merge(test1);
}
When I check database, the rows is not ordered as expected. The first row does not have id = 1, and the second row does not have id = 2 and so on...
What's wrong with EclipseLink or I got mistake?
From what you wrote, you cannot be sure who is responsible for that: either your DB / DB client (when you check the order), or the EcliplseLink. Anyway, if you want a certain order for your JPA operations, you should use EntityManager.flush() in order to hit the DB, otherwise the persistence provider can decide when and in which order to flush its query cache. Also, if you want a certain order in your client, you should sort the results, as some databases (like MySQL) do not guarantee that the rows are returned in the same order in which they have been inserted.
I think it depends, depends on what?
Depends if the List has been marked with #OrderColumn annotation, default is not store the order of the list, but it depends on your model an eclipselink can be configured to store order and be the responsible for the correct update of that field.
Also use OrderColumn could be a hit in the performance so use with caution
I need to query database for different combination of elements from the already received result object.
For instance, I get a list of Person entities. For each person in Person entities, I need to get List of address (for each person).
There are two ways to do it:
Iterate the Person entity and fire a query for each Person entity to get the list of Addresses for that person.
Build a query dynamically with elements from Person entity and fire ONE single query to pull all addresses lists for all Persons and then iterate the Person entity again and match the Address list for each Person.
I don't know much many Person entities I might get. So what is the better approach in terms of performance and practice.
So, if I have 100 Person entities, in the first approach its going to be 100 queries vs 2nd approach with huge query like below
from address where (person.id = 1 and person.zip = 393)
or (person.id = 2 and person.zip = 123)
or (person.id = 3 and person.zip = 345)
.... // 10 times.
Which one is better? Any restrictions / limitation on Or conditions in Oracle?
Is there a better approach? Batch queries?
You can use hibernate with eager loading to directly get the results what you want by loading the person with the restrictions required. Or else if you want to stick to lazy loading try using an inner join with person and Address so that you can then get a list of array which consist the results
I have an Class with a unidirectional one to many relationship as follows:
public class Order {
#OneToMany(cascade = CascadeType.ALL)
#JoinTable(name="order_item", joinColumns={#JoinColumn(name="order_id")}, inverseJoinColumns={#JoinColumn(name="item_id")})
public Set<Item> getItems() {
return items;
}
}
Normally getting the contents of this order is straightforward:
List<Item> items = order.getItems();
But for whatever reason I might want to filter my results in some way and retrieve only part of a collection of items such as all items more than a certain price, below a certain stock etc in the fastest way possible (Not returning then all then filtering afterwards). To do this I would run a HQL query to retrieve the items for a particular order and add some more stuff into my where clause or onto my query object.
Intuitively I would want this behaviour (Which is completely wrong):
SELECT jointable.ITEM from order_item as jointable inner join jointable.order where order = :order
But of course this is wrong as HQL works in terms of mapped entities, so I cannot use the join table in the query. So what is the correct way to do this?
Edit:
I have found the answer to this question, I want the following Query:
Select o.items from Order o where o = ?
This allows me to fetch the collection of items for an order, without having to use a bidirectional relationship. I am now however confused on the second stage of this question, which is how to filter the results of this collection, the most simple example being:
Select o.items from Order o where o = ? order by o.items.somenumberfield asc
Which returns illegal attempt to dereference collection, so how would I filter my items?
Edit:
The ticket solution is actually correct, I misread the solution originally.
select item from Order order
inner join order.items item
where order = :order
and ...
HQL queries use entities and their associations. The fact that the association uses a join table or not is not important for HQL: you navigate through associations and Hibernate does the appropriate translation to SQL.
It sounds like you want to map the other side of this relationship i.e. make this bi-directional mapping.
Then you can use the order number in your HQL:
from Item as item where item.amount > :amount and item.order.id = :orderId
From the HQL documentation:
from Cat as cat where cat.mate.id = 69
The query is efficient and does not require a table join.
The fact that you are trying to do this query suggests that the relationship between a Line and its Order is important!