I'm using Hibernate for my ORM layer.
I'm try to run batch of HQL queries in one transaction (I cannot use session.update).
The issue is that even the transaction.commit() is at the end of the loop, the update queries run one by one.
Is there a way to run multiple HQL queries in one transaction?
public void updateItems() {
t = session.beginTransaction();
for (int i = 0; i < itemList.size(); i++) {
Query q = createUpdateQuery(session, itemList.get(i));
q.executeUpdate(); //updating one by one, and not waiting for transaction commit
}
t.commit();
}
Query createUpdateQuery(Session session, Item item) {
Query q = session.createQuery(
"Update Item i set i.notes=:notes, i.time=:time, i.counter=:counter, i.status=:status Where i.id=:id and i.time=:time");
q.setParameter("time", item.getTime());
q.setParameter("status", item.getStatus());
q.setParameter("notes", item.getNotes());
q.setParameter("id", item.getId());
return q;
}
Appreciate any help.
You are using a database transaction to enroll all your statements, but I think you want to use batch updates.
Just add the following configuration property:
<property name="hibernate.jdbc.batch_size" value="10"/>
Even so, I think you should use Hibernate to manage the insert/update/delete statements, as you should only focus on entity state transitions. The dirty checking mechanism can automatically detect entities that have been modified, and Hibernate can generate the update statement for you, which is much more convenient.
Related
I am using Spring Boot JPA, I have enabled batching by ensuring the following lines are in the my application.properties:
spring.jpa.properties.hibernate.jdbc.batch_size=1000
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
I now have a loop where I am doing a findById on an entity and then saving that entity like so:
var entity = dao.findById(id)
// Do some stuff
dao.save(entity) //This line is not really required but I am being explicit here
Putting the above in a loop I see that the save(update) statements are batched to the DB.
My issue is that if I do a findOneByX where X is a property on the entity then the batching does not work (batch size of 1), requests get sent one at a time i.e.:
var entity = dao.findOneByX(x)
// Do some stuff
dao.save(entity)
Why is this happening? Is JPA/JDBC only equipped to batch when we findById only?
Solution
Refer to How to implement batch update using Spring Data Jpa?
Fetch the list of entity you want to update to a list
Update as desired
Call saveAll
PS: beware of memory usage for this solution, when your list size is large.
Why findById and findOneByX behave differently?
As suggested by M. Deinum, hibernate will auto flush your change
prior to executing a JPQL/HQL query that overlaps with the queued entity actions
Since both findById and findOneByX will execute query, what is the different between them?
First, the reason to flush is to make sure session and Database are in same state, hence you can get consistent result from session cache(if available) and database.
When calling findById, hibernate will try to get it from session cache, if entity is not available, fetch it from database. While for findOneByX, we always need to fetch it from database as it is impossible to cache entity by X.
Then we can consider below example:
#Entity
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
public class Student {
#Id
private Long id;
private String name;
private int age;
}
Suppose we have
id
name
age
1
Amy
10
#Transactional
public void findByIdAndUpdate() {
dao.save(new Student(2L, "Dennis", 14));
// no need to flush as we can get from session
for (int i = 0; i < 100; i++) {
Student dennis = dao.findById(2L).orElseThrow();
dennis.setAge(i);
dao.save(dennis);
}
}
Will result in
412041 nanoseconds spent executing 2 JDBC batches;
1 for insert 1 one for update.
Hibernate: I'm sure that result can be fetch from session (without flush) or database if record is not in session, so let's skip flushing as it is slow!
#Transactional
public void findOneByNameAndUpdate() {
Student amy = dao.findOneByName("Amy");
// this affect later query
amy.setName("Tammy");
dao.save(amy);
for (int i = 0; i < 100; i++) {
// do you expect getting result here?
Student tammy = dao.findOneByName("Tammy");
// Hibernate not smart enough to notice this will not affect later result.
tammy.setAge(i);
dao.save(tammy);
}
}
Will result in
13964088 nanoseconds spent executing 101 JDBC batches;
1 for first update and 100 for update in loop.
Hibernate: Hmm, I'm not sure if stored update will affect the result, better flush the update or I will be blamed by developer.
I want to update a column value in a table containing 800k rows. So, I have created a simple Java application with Hibernate 4.3.6.Final as an ORM framework.
I have configured the JDBC batch with 45 as a value and I have disabled the use of second level cache.
<property name="hibernate.jdbc.batch_size">45</property>
<property name="hibernate.cache.use_second_level_cache">false</property>
Can I increase the batch_size for example to 200? Because in hibernate docs they mention:
Before batch processing, enable JDBC batching. To enable JDBC batching, set the property hibernate.jdbc.batch_size to an integer between 10 and 50.
This is the code (simplified):
session.beginTransaction();
List<MyEntity> entities = findAllEntities();
logger.info("Number of fetched rows: " + entities.size());
int count = 0;
for (MyEntity entity : entities) {
// change some fields of the entity
session.update(entity);
if ( ++count % HIBERNATE_BACH_SIZE == 0 ) {
//flush a batch of updates and release memory:
session.flush();
session.clear();
}
}
session.getTransaction().commit();
Fetching all the entities and then looping through them one by one is quite tedious and will always result in suboptimal performance.
Since it seems that you perform an unconditional update (i.e no checks are in place to define which object will get its fields updated) you should use a simple HQL query to perform the update in a single action.
For example, given the table you want to update is MyEntity then your query would looke like this:
int rows = session.createQuery("UPDATE MyEntity ME SET me.myField1=:newField1, me.myField2=:newField2)
.setString("newField1", "Something")
.setString("newField2", "Something")
.executeUpdate();
Using that should improve performance a lot.
I'm trying to do insert via a native query with JPA, but I don't want to create a transaction:
Query query = em.createNativeQuery("INSERT INTO person (id, firstname, lastname) VALUES ('1','Ronnie','Dio')");
int count = query.executeUpdate();
this ends up with the TransactionRequiredException:
javax.persistence.TransactionRequiredException: Executing an update/delete query
at org.hibernate.ejb.AbstractQueryImpl.executeUpdate(AbstractQueryImpl.java:99)
However, if I unwrap the Hibernate session and execute the same query - it works:
Session session = em.unwrap(Session.class);
Query query = session.createSQLQuery("INSERT INTO person (id, firstname, lastname) VALUES ('1','Ronnie','Dio')");
int count = query.executeUpdate();
So my question is - what am I doing wrong in the first sample? Is transaction really required to execute insert/update/delete queries in JPA?
If it is, is there any link to documentation that specifies it clearly? If it's not - what am I doing wrong?
It seems you are building an application that does not run inside a container supporting JTA managed transactions. In such an environment, you have to handle/manage transactions for yourself, i.e., you have to control when transactions are opened, committed or rolled back. This scenario is referred to as resource-local entity manager.
In section 7.5.2 Resource-local EntityManagers of the official JPA 2.2 specification (p. 345) we find:
An entity manager whose transactions are controlled by the application through the EntityTransaction
API is a resource-local entity manager. A resource-local entity manager transaction is mapped
to a resource transaction over the resource by the persistence provider. Resource-local entity managers
may use server or local resources to connect to the database and are unaware of the presence of JTA
transactions that may or may not be active
Further down in the spec document the EntityTransaction interface is given. It allows you to call
begin() to "Start a resource transaction"
commit() to "Commit the current resource transaction, writing any
unflushed changes to the database."
rollback() to "Roll back the current resource transaction." in case something went wrong on the database side while committing changes.
That's for the theory part.
For your code example, you might want to change it as follows:
EntityTransaction tx = null;
try {
tx = em.getTransaction();
// start a new transaction, i.e. gather changes from here on...
tx.begin();
// do your changes here
Query query = em.createNativeQuery("INSERT INTO person (id, firstname, lastname) VALUES ('1','Ronnie','Dio')");
int count = query.executeUpdate();
// write changes to database via commit
tx.commit();
} catch(RuntimeException re) {
if(tx != null && tx.isActive()) {
// ensure no semi-correct changes are pending => cleanup
tx.rollback();
}
// handle exception, log it somewhere...
}
This should avoid the TransactionRequiredException you encounter. Moreover, you should avoid the use createNativeQuery, as you are mis-using a fundamental concept of an Object-Relational-Mapper (ORM), i.e. the mapper will transform objects into tuples and vice versa for you. This - in general - should ease the pain of writing insert/update queries for a large amount of domain entities.
Have a look into section 3.1.1 EntityManager Interface (p. 65) of the spec document linked above and make use of the methods
persist(..) - "Make an instance managed and persistent." or
merge(..) - "Merge the state of the given entity into the current persistence context."
For more infos on the difference of both approaches see the posts here and here.
Hope it helps.
Instead of creating a native query , I would recommend to create a JPA entity for the person and with JPARepository you can use a save method for the person to insert any record.
I am currently upgrading our application and we are migrating from Hibernate 3.x and Spring/Spring Security 3.x to Hibernate 5.x and Spring/Spring Security 5.x respectively. We had some methods that executed native sql update queries (example 1), but after upgrading to 5.x the methods started throwing TransactionRequiredException: Executing an update/delete query exceptions. Well I tried adding the #Transactional annotation on the methods but it doesn't seem to help. I will share the old method and the upgraded method (example 2).
I don't understand how it is not working on the new version, did Hibernate change the way they treat native sql queries? Thanks for the responses.
Example 1
public void myTestMethod() {
String sql = "Update myTable set State = 1";
Session session = getSessionFactory().openSession();
Query query = session.createSQLQuery(sql);
query.executeUpdate();
session.close();
}
Example 2
public void myTestMethod() {
String sql = "Update myTable set State = 1";
Session session = sessionFactory.openSession();
Query query = session.createNativeQuery(sql);
query.executeUpdate();
session.close();
}
What am I doing wrong here? How can I execute this update without changing much in the methods (we have thousands of methods implemented). The sessionFactory in the example 2 is injected with #Inject annotation.
Try to use #Transactional over your methods and use getCurrentSession() instead of the openSession()
And your code has session.close() statement which doesn't make sense any more, since the connection was already closed and managed by the spring. Try removing the session.close() statement and try again
I am using hibernate. i am deleting record using delete method as below.
Session session = sessionFactory.getCurrentSession();
session.delete(pojotobedeleted);
my requirement is after deleting the record, it should return the same deleted record. How can i get the deleted record?
Thanks!
I'd say
Session session = sessionFactory.getCurrentSession();
pojotobedeleted = session.load(Pojo.class, id); /* not necessary if loaded before */
session.delete(pojotobedeleted);
return pojotobedeleted;
Sure there is an extra select statement for doing so, but that is the same with native SQL. The SQL delete does not return the deleted rows, and hibernate can't be better that native SQL.
(If this is not what you wants then I don't understand your question: What is the goal, why not like in my example?)