We Keep Coding

Does a hibernate transaction commit changes asynchronously, independent of the program flow?

I have a test method which sometimes fails during deploy and sometimes does not. I have never seen it fail on my local. You can see my code below.
I have the following retry mechanism which is asynchronously called from another service:
#Transactional
public boolean retry(NotificationOrder order) {
notificationService.send(order);
return true;
}
public void resolveOnFailedAttempt(Long orderId) { //automatically called if `retry` method fails
notificationOrderCommonTransactionsService.updateNotificationOrderRetryCount(orderId);
}
The notification service is like this :
#Service
#RequiredArgsConstructor
public class NotificationServiceImpl implements NotificationService {
private final NotificationOrderCommonTransactionsService notificationOrderCommonTransactionsService;
#Override
#Transactional
public NotificationResponse send(NotificationOrder order) {
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e;
}
...
return response;
}
private void prepareNotificationRequest(NotificationOrder order) {
...
throw new Exception("ERROR");
}
}
And the commmon transactions service is like this :
#Transactional(propagation = Propagation.REQUIRES_NEW)
public NotificationOrder saveNotificationOrderErrorMessage(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setErrorDescription(errorMessage);
notificationRepository.save(order);
return order;
}
public NotificationOrder updateNotificationOrderRetryCount(Long orderId) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
notificationRepository.save(order);
return order;
}
Here is my integration test :
#Test
public void test() {
NotificationOrderRequest invalidRequest = invalidRequest();
ResponseEntity<NotificationOrderResponse> responseEntity = send(invalidRequest);
NotificationOrder notificationOrder = notificationOrderRepository.findOne(1);
softly.assertThat(notificationOrder.getOrderStatus().isEqualTo(NotificationOrderStatus.IN_PROGRESS))
softly.assertThat(notificationOrder.getErrorDescription())
.isEqualTo("ERROR"); //This the line that fails.
softly.assertThat(responseEntity.getStatusCode()).isEqualTo(HttpStatus.OK);
}
In the test method it is confirmed that updateNotificationOrderRetryCount is called and the order status is updated as IN_PROGRESS. However, the error message is null and I get the following assertion error :
-- failure 1 --
Expecting:
<null>
to be equal to:
<"ERROR">
but was not.
I expect saveNotificationOrderErrorMessage transaction to be completed and the changes to be committed before updateNotificationOrderRetryCount method is called. But it seems like it does work that way. Could anyone help me find out why my code behave like this ?
How can I reproduce this error on my local? And what can I do to fix it ?
Thanks.

Try enabling SQL logging and parameter bind logging and look through the statements. I don't know all of your code, but maybe your are setting the message to null somewhere? It could also be, that the actions are interleaving somehow such that updateNotificationOrderRetryCount is called before/while saveNotificationOrderErrorMessage in a way that causes this. If both run right before commit, but saveNotificationOrderErrorMessage commits before updateNotificationOrderRetryCount, you could see the error message being overwritten with null.

If the code snippet of the question is accurate, pay attention to the fact that you are rethrowing the exception raised in the prepareNotificationRequest method, I assume in order to enable the retry mechanism:
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e; // You are rethrowing the exception
}
For your comment, the exception thrown extends RuntimeException.
As the Spring documentation indicates:
In its default configuration, the Spring Framework’s transaction infrastructure code marks a transaction for rollback only in the case of runtime, unchecked exceptions. That is, when the thrown exception is an instance or subclass of RuntimeException. ( Error instances also, by default, result in a rollback). Checked exceptions that are thrown from a transactional method do not result in rollback in the default configuration.
Probably Spring is performing rollback of the initial transaction, that one associated with saveNotificationOrderErrorMessage. I realize that this method is annotated as #Transactional(propagation = Propagation.REQUIRES_NEW) and that it is initiating a new transaction, but perhaps the problem could be related with it.
When the retry mechanism takes place, another transaction, associated with the invocation of the method updateNotificationOrderRetryCount is performed, and this transaction is successfully committed. This is the reason why the changes performed in this second method are properly committed.
The solution of the problem will depend on how your retry mechanism is implemented, but you can, for example, raise the original exception and, as a first step in the retry mechanism, trace the problem in the database or, raise a checked exception - Spring by default will not perform rollback for it - and handle it as appropriate.
Update
Another possible reason of the problem could be the transaction demarcations in the send method.
This method is annotated as #Transactional. As a consequence, Spring will initiate a new transaction for it.
The error occurs, and you trace the error in the database, in a new transaction but please, be aware that the initial transaction is still there.
Although not described in your code, in some way, the retry mechanism takes place, and updates the retry count. It this operation is performed within the initial transaction (or a higher level one), due to the transaction boundaries, database isolation levels, and related stuff, it is possible that this transaction, the initial, fetch an actually outdated, but current from the transaction boundary point of view, NotificationOrder. And this information is the one that finally is committed, overwriting the information of the error. I hope you get the idea.
One simple solution, maybe for both possibilities, could be to include the error message in the updateNotificationOrderRetryCount method itself, reducing the problem to a single transaction:
/* If appropriate, mark it as Transactional */
#Transactional
public NotificationOrder updateNotificationOrderRetryCount(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
order.setErrorDescription(errorMessage);
// It is unnecessary, all the changes performed in the entity within the transaction will be committed
// notificationRepository.save(order);
return order;
}

How to guarantee atomic batch insert with pure Hibernate inside #Transactional method in Spring Boot application

To simplify my scenario I would give the following example:
I'm using a Spring Boot application with spring.jpa.properties.hibernate.order_inserts=true property.
Inside a java method marked with #Transactional annotation I want to perform a batch insert of a few thousand records with pure hibernate. The batch insert is performed by a separate method which is called by the main method with #Transactional. I'm doing it like that:
#Transactional
public void doSomeStuff() {
......
insertRecords(records, session);
......
}
private void insertRecords(final List<Record> records, final Session session) {
int counter = 0;
int batchSize = 50;
for(Record r: records){
session.save(r);
counter ++;
if(counter > 0 && counter % batchSize == 0){
session.flush();
session.clear();
}
}
}
The problem is that I want the operation to be atomic and if one of the batch inserts, fails everything to be rolled back. I know #Transactional anotation will guarantee rollback automatically for RunTimeException or Error. Does this mean that if one of the batches fails #Transactional will cover the rollback or I should surround the for cycle with try/catch block and throw a checked exception and use the rollbackFor propery of #Transactional? I'm not sure if the exceptions thrown during batch failure are checked or unchecked.
Another important question is does in this scenario hibernate commit every batch automatically in case of successfull insert? This would mean that if one batch fails I wouldn't be able to rollback the already committed once which breaks the atomicity.
In addition, I don't want to manage the commits manually and want to handle this to the transactional context provided by Spring.

You are right. By default , #Transactional will only rollback for RuntimeException and Error but not for checked exception. That means if you want to also rollback the checked exception , you have to catch all the checked exception and re-throw it as the RuntimeException or simply use rollbackFor setting in #Transactional.
I'm not sure if the exceptions thrown during batch failure are checked
or unchecked.
The java compiler will help to check it.If some of the inner methods throw checked exception ,it requires you must handle it which you can neither catch it or specify to throw this checked exception in the method declaration. The codes cannot be compiled if you do not do so.
That means if your codes can compile, no checked exception is thrown from the inner methods and you can simply stick to the current settings. On the other hands, if some checked exception are thrown from the inner methods , you can choose to catch it and re-throw it as RuntimeException :
#Transactional
public void doSomeStuff() {
try{
}catch(Exception ex){
throw new RuntimeException("something goes wrong.." ,ex);
}
}
or configure rollbackFor
#Transactional(rollbackFor={Exception.class})
public void doSomeStuff() throws Exception{
}
Also , please note that session.flush() is not the same as committing transaction, so your codes will not commit for every batch. Instead, transaction will commit after the method marked with #Transactional return successfully which in your case is the returned of doSomeStuff() when all the records are inserted.
The point of flush() is for the subsequent session.clear() which clear the memory from the session such that it will not cause the JVM to run out of memory if you insert a lot of records.

SpringBoot 2 transaction propagation NESTED not supported

I have a SpringBoot 2 project and i'm using spring data jpa with hibernate with MySQL5.7
I have problems with the following use case: i have a service method that calls another service's method. If second service's method generates a runtime exception, also the first method is marked as rollback and i cannot commit things anymore. I'd like to only rollback second method and still commit something in the first one.
I tried to use propagation.NESTED but nested transaction are not allowed with hibernate (even if jpaTransactionManager supports them and MySQL supports savepoints).
How can i solve this problem? Can i configure nested in some way?
Please remember i need second method to see changes committed by first so i can't mark the second method as propagation.REQUIRES_NEW
Here is come sample code to clarify my problem:
FirstServiceImpl.java
#Service
public class FirstServiceImpl implements FirstService
#Autowired
SecondService secondService;
#Autowired
FirstServiceRepository firstServiceRepository;
#Transactional
public void firstServiceMethod() {
//do something
...
FirstEntity firstEntity = firstServiceRepository.findByXXX();
firstEntity.setStatus(0);
firstServiceRepository.saveAndFlush(firstEntity);
...
boolean runtimeExceptionHappened = secondService.secondServiceMethod();
if (runtimeExceptionHappened) {
firstEntity.setStatus(1);
firstServiceRepository.save();
} else {
firstEntity.setStatus(2);
firstServiceRepository.save();
}
}
SecondServiceImpl.java
#Service
public class SecondServiceImpl implements SecondService
#Transactional
public boolean secondServiceMethod() {
boolean runtimeExceptionHappened = false;
try {
//do something that saves to db but that may throw a runtime exception
...
} catch (Exception ex) {
runtimeExceptionHappened = true;
}
return runtimeExceptionHappened;
}
So the problem is that when secondServiceMethod() raises a runtime exception it rollback its operations (and that's OK) and then set its return variable runtimeExceptionHappened to false, but then firstServiceMethod is marked as rollback only and then
firstEntity.setStatus(1);
firstServiceRepository.save();
isn't committed.
Since i can't use NESTED propagation how can i achieve my goal?

I would suggest you break them up into two separate transactions.
In the first transaction do all of the work presently in firstServiceMethod that you know you want to commit. (e.g. through saveAndFlush). Now as you exit this method the changes are committed, so they will be available to subsequent calls.
Then have whatever called firstServiceMethod call a new Transactional method setFirstEntityStatus() that calls secondServiceMethod and sets the status of the entity as appropriate.
Basically, instead of attempting to NEST the transactions, split them into two fully separate transactions and use the ordering to ensure the result of the 1st is available to the 2nd.

DB constraint violation not throwing Exception in Hibernate

I have the following code:
try {
userDAO1.save(userRecord);
userDAO2.save(userRecord);
}
catch(DataIntegrityViolationException e) {
throw new ApplicationException("Contraint violated")
}
userDAO1.save(userRecord) violates an integrity constraint - so after the entire code has been run, there is nothing written to the table userDAO1 refers to.
However, the userDAO1.save() statement doesn't throw an error/exception - so userDAO2.save() is executed as well.
But the DataIntegrityViolationException is caught, and the stack trace is null.
How do I check where the DataIntegrityViolationException is thrown from, and prevent userDAO2.save() from being executed if userDAO1.save() violates a constraint?
I tried adding a #Transactional annotation around this code, but that didn't work either.
Stack trace:
org.springframework.dao.DataIntegrityViolationException: ORA-00001: unique constraint (UNIQUE_EMAIL) violated
; SQL [n/a]; constraint [UNIQUE_EMAIL]; nested exception is org.hibernate.exception.ConstraintViolationException: ORA-00001: unique constraint (UNIQUE_EMAIL) violated
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:643)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:104)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:516)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy76.updateUser(Unknown Source)
at com.osiris.UserReg.UpdateUserCommand.execute(UpdateUserCommand.java:63)
The code I've posted is in UpdateUserCommand, which is annotated with #Transactional(rollbackFor=Exception.class, propagation=Propagation.REQUIRES_NEW)

Ok, this is a bit of a tricky one, but I'll do my best.
Hibernate will only commit a transaction when the method annotated with #Transactional exits. Hence your DataIntegrityViolationException will only be catchable after that method returns. There is no way that you can get Hibernate to not call UserDAO2.save() because it can't detect that a violation has occurred. I'll provide an example below
#Service
/*These variable names are used for clarity's sake, I don't actually use these names myself*/
public UserServiceImpl implements UserService{
#Autowired
private HibernateUserDAO1 userDao1;
#Autowired
private HibernateUserDAO2 userDao2
#Transactional
/*Put your try catch block around where this method is called*/
public void saveUserDao1(User user){
userDao1.saveOrUpdate(user);
}
#Transactional
/*Only call this if saveUserDao1 succeeds*/
public void saveUserDao2(User user){
userDao2.saveOrUpdate(user)
}
}
Then in your HibernateUserDAO1:
public void saveOrUpdate(User user){
currentSession().saveOrUpdate(user);
}
The exception can only be caught above your service layer. Ideally what you want to be doing, is individual saves using 2 different DAO's and checking that the first succeeded before doing the second.
EDITED:
Also be aware that Hibernate will not pick up private methods annotated with #Transactional because Hibernate depends on creating Proxy objects from the interface that your class implements. No interface definition = no proxy object = no Hibernate Session. So you can't call a private method annotated with #Transactional. I'd try to make your SessionFactory an object in an abstract superclass and have both DAO's inherit from this. A better option is to use 2 transaction managers each pointing to your different databases, then specify which database things are saving too. That way you can use just 1 DAO, and use whichever session factory you require to do your saves.

What made you to believe that DataIntegrityViolationException was not thrown while the statement userDAO1.save() was executed? Also, why do you believe that the statement userDAO2.save() was executed as well?
If the above opinions were made upon observations of code execution progression in an IDE debug console such as that of Eclipse then the interpretations might be wrong.
Please try to observe the results by punching-in some debug statements like the ones below, and executing the code. This may help you to find out the root cause of the failure -
try {
userDAO1.save(userRecord);
System.out.println("-- After userDAO1.save(userRecord) --");
userDAO2.save(userRecord);
System.out.println("-- After userDAO2.save(userRecord) --");
} catch(DataIntegrityViolationException e) {
e.printStackTrace();
throw new ApplicationException("Contraint violated")
}

Clear the in memory database after every testcase

I am using hsqldb for testing some of the data access layer in Java. I have certain test cases like 100 around. I create a in memory database and then insert some values in the table so that with my test case i can load it, but the problem is for every test case i need to clear the in memory database, only the values not the tables.
Is it possible, one thing is i need to manually delete the rows from the table and is there some thing else I can use.
Thanks

If you use DbUnit in unit-tests, you can specify that DbUnit should perform a clean-and-insert operation before every test to ensure that the contents of the database are in a valid state before every test. This can be done in a manner similar to the one below:
#Before
public void setUp() throws Exception
{
logger.info("Performing the setup of test {}", testName.getMethodName());
IDatabaseConnection connection = null;
try
{
connection = getConnection();
IDataSet dataSet = getDataSet();
//The following line cleans up all DbUnit recognized tables and inserts and test data before every test.
DatabaseOperation.CLEAN_INSERT.execute(connection, dataSet);
}
finally
{
// Closes the connection as the persistence layer gets it's connection from elsewhere
connection.close();
}
}
Note that it is always recommended to perform any setup activities in a #Before setup method, rather than in a #After teardown method. The latter indicates that you are creating new database objects in a method being tested, which IMHO does not exactly lend easily to testable behavior. Besides, if you are cleaning up after a test, to ensure that a second test runs correctly, then any such cleanup is actually a part of the setup of the second test, and not a teardown of the first.
The alternative to using DbUnit is to start a new transaction in your #Before setup method, and to roll it back in the #After teardown method. This would depend on how your data access layer is written.
If your data access layer accepts Connection objects, then your setup routine should create them, and turn off auto-commit. Also, there is an assumption that your data access layer will not invoke Connection.commit. Assuming the previous, you can rollback the transaction using Connection.rollback() in your teardown method.
With respect to transaction control, the below snippet demonstrates how one would do it using JPA for instance:
#Before
public void setUp() throws Exception
{
logger.info("Performing the setup of test {}", testName.getMethodName());
em = emf.createEntityManager();
// Starts the transaction before every test
em.getTransaction.begin();
}
#After
public void tearDown() throws Exception
{
logger.info("Performing the teardown of test {}", testName.getMethodName());
if (em != null)
{
// Rolls back the transaction after every test
em.getTransaction().rollback();
em.close();
}
}
Similar approaches would have to be undertaken for other ORM frameworks or even your custom persistence layer, if you have written one.

Could you use HSQLDB transactions?
Before every test, start a new transaction:
START TRANSACTION;
After every test, roll it back:
ROLLBACK;
This would also allow you to have some permanent data.

Depending on your test framework, it is possible to execute a delete call after each test. In Junit the annotation is #After and a method with this annotation will be run after each [#Test] method.

You have to use Truncate Query for the Destroy Database Memory or this link can be helpful to you.
http://wiki.apache.org/db-derby/InMemoryBackEndPrimer