I have a test method which sometimes fails during deploy and sometimes does not. I have never seen it fail on my local. You can see my code below.
I have the following retry mechanism which is asynchronously called from another service:
#Transactional
public boolean retry(NotificationOrder order) {
notificationService.send(order);
return true;
}
public void resolveOnFailedAttempt(Long orderId) { //automatically called if `retry` method fails
notificationOrderCommonTransactionsService.updateNotificationOrderRetryCount(orderId);
}
The notification service is like this :
#Service
#RequiredArgsConstructor
public class NotificationServiceImpl implements NotificationService {
private final NotificationOrderCommonTransactionsService notificationOrderCommonTransactionsService;
#Override
#Transactional
public NotificationResponse send(NotificationOrder order) {
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e;
}
...
return response;
}
private void prepareNotificationRequest(NotificationOrder order) {
...
throw new Exception("ERROR");
}
}
And the commmon transactions service is like this :
#Transactional(propagation = Propagation.REQUIRES_NEW)
public NotificationOrder saveNotificationOrderErrorMessage(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setErrorDescription(errorMessage);
notificationRepository.save(order);
return order;
}
public NotificationOrder updateNotificationOrderRetryCount(Long orderId) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
notificationRepository.save(order);
return order;
}
Here is my integration test :
#Test
public void test() {
NotificationOrderRequest invalidRequest = invalidRequest();
ResponseEntity<NotificationOrderResponse> responseEntity = send(invalidRequest);
NotificationOrder notificationOrder = notificationOrderRepository.findOne(1);
softly.assertThat(notificationOrder.getOrderStatus().isEqualTo(NotificationOrderStatus.IN_PROGRESS))
softly.assertThat(notificationOrder.getErrorDescription())
.isEqualTo("ERROR"); //This the line that fails.
softly.assertThat(responseEntity.getStatusCode()).isEqualTo(HttpStatus.OK);
}
In the test method it is confirmed that updateNotificationOrderRetryCount is called and the order status is updated as IN_PROGRESS. However, the error message is null and I get the following assertion error :
-- failure 1 --
Expecting:
<null>
to be equal to:
<"ERROR">
but was not.
I expect saveNotificationOrderErrorMessage transaction to be completed and the changes to be committed before updateNotificationOrderRetryCount method is called. But it seems like it does work that way. Could anyone help me find out why my code behave like this ?
How can I reproduce this error on my local? And what can I do to fix it ?
Thanks.
Try enabling SQL logging and parameter bind logging and look through the statements. I don't know all of your code, but maybe your are setting the message to null somewhere? It could also be, that the actions are interleaving somehow such that updateNotificationOrderRetryCount is called before/while saveNotificationOrderErrorMessage in a way that causes this. If both run right before commit, but saveNotificationOrderErrorMessage commits before updateNotificationOrderRetryCount, you could see the error message being overwritten with null.
If the code snippet of the question is accurate, pay attention to the fact that you are rethrowing the exception raised in the prepareNotificationRequest method, I assume in order to enable the retry mechanism:
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e; // You are rethrowing the exception
}
For your comment, the exception thrown extends RuntimeException.
As the Spring documentation indicates:
In its default configuration, the Spring Framework’s transaction infrastructure code marks a transaction for rollback only in the case of runtime, unchecked exceptions. That is, when the thrown exception is an instance or subclass of RuntimeException. ( Error instances also, by default, result in a rollback). Checked exceptions that are thrown from a transactional method do not result in rollback in the default configuration.
Probably Spring is performing rollback of the initial transaction, that one associated with saveNotificationOrderErrorMessage. I realize that this method is annotated as #Transactional(propagation = Propagation.REQUIRES_NEW) and that it is initiating a new transaction, but perhaps the problem could be related with it.
When the retry mechanism takes place, another transaction, associated with the invocation of the method updateNotificationOrderRetryCount is performed, and this transaction is successfully committed. This is the reason why the changes performed in this second method are properly committed.
The solution of the problem will depend on how your retry mechanism is implemented, but you can, for example, raise the original exception and, as a first step in the retry mechanism, trace the problem in the database or, raise a checked exception - Spring by default will not perform rollback for it - and handle it as appropriate.
Update
Another possible reason of the problem could be the transaction demarcations in the send method.
This method is annotated as #Transactional. As a consequence, Spring will initiate a new transaction for it.
The error occurs, and you trace the error in the database, in a new transaction but please, be aware that the initial transaction is still there.
Although not described in your code, in some way, the retry mechanism takes place, and updates the retry count. It this operation is performed within the initial transaction (or a higher level one), due to the transaction boundaries, database isolation levels, and related stuff, it is possible that this transaction, the initial, fetch an actually outdated, but current from the transaction boundary point of view, NotificationOrder. And this information is the one that finally is committed, overwriting the information of the error. I hope you get the idea.
One simple solution, maybe for both possibilities, could be to include the error message in the updateNotificationOrderRetryCount method itself, reducing the problem to a single transaction:
/* If appropriate, mark it as Transactional */
#Transactional
public NotificationOrder updateNotificationOrderRetryCount(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
order.setErrorDescription(errorMessage);
// It is unnecessary, all the changes performed in the entity within the transaction will be committed
// notificationRepository.save(order);
return order;
}
Related
To simplify my scenario I would give the following example:
I'm using a Spring Boot application with spring.jpa.properties.hibernate.order_inserts=true property.
Inside a java method marked with #Transactional annotation I want to perform a batch insert of a few thousand records with pure hibernate. The batch insert is performed by a separate method which is called by the main method with #Transactional. I'm doing it like that:
#Transactional
public void doSomeStuff() {
......
insertRecords(records, session);
......
}
private void insertRecords(final List<Record> records, final Session session) {
int counter = 0;
int batchSize = 50;
for(Record r: records){
session.save(r);
counter ++;
if(counter > 0 && counter % batchSize == 0){
session.flush();
session.clear();
}
}
}
The problem is that I want the operation to be atomic and if one of the batch inserts, fails everything to be rolled back. I know #Transactional anotation will guarantee rollback automatically for RunTimeException or Error. Does this mean that if one of the batches fails #Transactional will cover the rollback or I should surround the for cycle with try/catch block and throw a checked exception and use the rollbackFor propery of #Transactional? I'm not sure if the exceptions thrown during batch failure are checked or unchecked.
Another important question is does in this scenario hibernate commit every batch automatically in case of successfull insert? This would mean that if one batch fails I wouldn't be able to rollback the already committed once which breaks the atomicity.
In addition, I don't want to manage the commits manually and want to handle this to the transactional context provided by Spring.
You are right. By default , #Transactional will only rollback for RuntimeException and Error but not for checked exception. That means if you want to also rollback the checked exception , you have to catch all the checked exception and re-throw it as the RuntimeException or simply use rollbackFor setting in #Transactional.
I'm not sure if the exceptions thrown during batch failure are checked
or unchecked.
The java compiler will help to check it.If some of the inner methods throw checked exception ,it requires you must handle it which you can neither catch it or specify to throw this checked exception in the method declaration. The codes cannot be compiled if you do not do so.
That means if your codes can compile, no checked exception is thrown from the inner methods and you can simply stick to the current settings. On the other hands, if some checked exception are thrown from the inner methods , you can choose to catch it and re-throw it as RuntimeException :
#Transactional
public void doSomeStuff() {
try{
}catch(Exception ex){
throw new RuntimeException("something goes wrong.." ,ex);
}
}
or configure rollbackFor
#Transactional(rollbackFor={Exception.class})
public void doSomeStuff() throws Exception{
}
Also , please note that session.flush() is not the same as committing transaction, so your codes will not commit for every batch. Instead, transaction will commit after the method marked with #Transactional return successfully which in your case is the returned of doSomeStuff() when all the records are inserted.
The point of flush() is for the subsequent session.clear() which clear the memory from the session such that it will not cause the JVM to run out of memory if you insert a lot of records.
I have a SpringBoot 2 project and i'm using spring data jpa with hibernate with MySQL5.7
I have problems with the following use case: i have a service method that calls another service's method. If second service's method generates a runtime exception, also the first method is marked as rollback and i cannot commit things anymore. I'd like to only rollback second method and still commit something in the first one.
I tried to use propagation.NESTED but nested transaction are not allowed with hibernate (even if jpaTransactionManager supports them and MySQL supports savepoints).
How can i solve this problem? Can i configure nested in some way?
Please remember i need second method to see changes committed by first so i can't mark the second method as propagation.REQUIRES_NEW
Here is come sample code to clarify my problem:
FirstServiceImpl.java
#Service
public class FirstServiceImpl implements FirstService
#Autowired
SecondService secondService;
#Autowired
FirstServiceRepository firstServiceRepository;
#Transactional
public void firstServiceMethod() {
//do something
...
FirstEntity firstEntity = firstServiceRepository.findByXXX();
firstEntity.setStatus(0);
firstServiceRepository.saveAndFlush(firstEntity);
...
boolean runtimeExceptionHappened = secondService.secondServiceMethod();
if (runtimeExceptionHappened) {
firstEntity.setStatus(1);
firstServiceRepository.save();
} else {
firstEntity.setStatus(2);
firstServiceRepository.save();
}
}
SecondServiceImpl.java
#Service
public class SecondServiceImpl implements SecondService
#Transactional
public boolean secondServiceMethod() {
boolean runtimeExceptionHappened = false;
try {
//do something that saves to db but that may throw a runtime exception
...
} catch (Exception ex) {
runtimeExceptionHappened = true;
}
return runtimeExceptionHappened;
}
So the problem is that when secondServiceMethod() raises a runtime exception it rollback its operations (and that's OK) and then set its return variable runtimeExceptionHappened to false, but then firstServiceMethod is marked as rollback only and then
firstEntity.setStatus(1);
firstServiceRepository.save();
isn't committed.
Since i can't use NESTED propagation how can i achieve my goal?
I would suggest you break them up into two separate transactions.
In the first transaction do all of the work presently in firstServiceMethod that you know you want to commit. (e.g. through saveAndFlush). Now as you exit this method the changes are committed, so they will be available to subsequent calls.
Then have whatever called firstServiceMethod call a new Transactional method setFirstEntityStatus() that calls secondServiceMethod and sets the status of the entity as appropriate.
Basically, instead of attempting to NEST the transactions, split them into two fully separate transactions and use the ordering to ensure the result of the 1st is available to the 2nd.
We have an application with three databases. Two of them are only very seldomly updated. We tried JPA to create transactions around it and it worked for the databases, but grails then did not work on different places (gsp related I am told). This was tried quite a while ago (and not by me).
Due to delivery pressure we needed a solution that at least works for us, so I created a new aspect for the methods changing data in multiple databases. I got this to work, it is a fairly simple approach.
In the aspect we request to start a transaction for each data source, by calling getTransaction(TransactionDefinition def) with the propagation set to REQUIRES_NEW. We then proceed and finally rollback or commit depending on the outcome of the call.
However, one test flow failed. This is the scenario where the code requests a rollback by calling TransactionAspectSupport.currentTransactionStatus().setRollbackOnly(). Of the three TransactionStatusses obtained initially, none actually returns isRollbackOnly() with true. However calling TransactionAspectSupport.currentTransationStatus().isRollbackOnly() does return true. So this seems to point to a different transaction status.
I have not been able to figure out how to make this work, other than checking this additional status. I could not find a way to change the currentTransactionStatus to the one of created TransactionStatus. Looking at the TransactionTemplate implementation, I seem to do things correctly (it also just calls getTransaction() on the datasource).
The code calling the decorated method has specified #Transactional(propagation=Propagation.NOT_SUPPORTED), so I expected no currentTransactionStatus, but one is there.
However, if it is not there the proxied code will not be able to request a rollback the standard way, which I want to be able to fix.
So the question is, how to start a transaction correctly from an Aspect so that the currentTransactionStatus is set correctly or how to set the currentTransactionStatus to what I think is the correct one.
Regards,
Wim Veldhuis.
I finally figured it out.
#Transactional leads to a different code path, where eventually TransactionAspectSupport.invokeWithinTransaction is invoked. This method will set up the current transaction correctly.
So in order to make my approach working, I needed to derive from TransactionAspectSupport, do a number of cast operations so I could get to the correct values for the invokeWithinTransaction call, and within the guarded function block use getTransaction(def) to obtain txns for the OTHER databases. I have choose the most important database to be the one used for invoke...
To make it work I had also to provide a TransactionAttributeSource, that returned my default transaction attributes.That one is stored into the TransactionAspectSupport base class during initialization.
#Around("#annotation(framework.db.MultiDbTransactional)")
public Object multiDbTransaction(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
// Get class and method, needed for parent invocation. We need to cast to the actual
// implementation
MethodInvocationProceedingJoinPoint mipJoinPoint = (MethodInvocationProceedingJoinPoint) proceedingJoinPoint;
MethodSignature signature = (MethodSignature) mipJoinPoint.getSignature();
Class<?> clazz = mipJoinPoint.getTarget().getClass();
Method method = signature.getMethod();
return invokeWithinTransaction(method, clazz, new InvocationCallback() {
#Override
public Object proceedWithInvocation() throws Throwable {
// This class will create the other transactions, not of interest here.
MultiDbTxnContext ctx = new MultiDbTxnContext();
ctx.startTransactions();
/*
* We have started the transactions, so do the job. We mimic DEFAULT spring behavior
* regarding exceptions, so runtime exceptions roll back, the rest commits.
*/
try {
Object result = proceedingJoinPoint.proceed();
ctx.finishTransactions();
return result;
} catch (Error | RuntimeException re) {
ctx.rollbackTransactions();
throw re;
} catch (Throwable t) {
ctx.commitTransactions();
throw t;
}
}
});
}
Method countryDao.delete(countryEntity) should throw ConstraintViolationException and the test should pass only if the exception is thrown.
However, the test in the form hereunder does not pass and I get message "java.lang.AssertionError: Expected test to throw an instance of org.hibernate.exception.ConstraintViolationException"
#Test
public void testDeleteWithCities() throws Exception {
exception.expect(ConstraintViolationException.class);
CountryEntity countryEntity = countryDao.find(1L);
countryDao.delete(countryEntity);
}
After I add the last line to the test, it does pass:
#Test
public void testDeleteWithCities() throws Exception {
exception.expect(ConstraintViolationException.class);
CountryEntity countryEntity = countryDao.find(1L);
countryDao.delete(countryEntity);
Assert.assertEquals(3, countryDao.findAll().size());
}
Should it behave like this? I beleive that the Assert.assertEquals(3, countryDao.findAll().size()); should not be needed.
Thanks for any help.
My guess is that your test is transactional. So the DAO.delete() method is done in the transaction used for the test method.
delete() or remove() doesn't immediately deletes the entity, just like persist() doesn't immediately inserts it. It simply marks the entity as deleted and, at the next flush, Hibernate will execute the delete query.
When calling findAll(), you're implicitely flushing the Hibernate session (to make sure that findAll() doesn't return the previously deleted entity). So the delete query is executed at this moment, and that causes the ConstraintViolationException.
You can confirm that by turning on SQL logging, executing the code line by line, and examining the SQL queries generated at each step.
So, either don't make your test transactional, or call flush() explicitely inside your test (which makes the intent clearer than calling findAll()).
I'm implementing simultaneous write into database and Oracle Coherence 3.7.1 and want to make whole operation transactional.
I would like to have a critique on my approach.
Currently, I've created façade class like this:
public class Facade {
#EJB
private JdbcDao jdbcDao;
#EJB
private CoherenceDao coherenceDao;
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
private void updateMethod(List<DomainObject> list) {
jdbcDao.update(list);
coherenceDao.update(list);
}
}
I guess JDBC DAO would not need to do anything specific about transactions, it something happens Hibernate would throw some kind of RuntimeException.
public class JdbcDao {
private void update(List<DomainObject> list) {
// I presume there is nothing specific I have to do about transactions.
// if I don't catch any exceptions it would work just fine
}
}
Here is interesting part. How do I make Coherence support transactions?
I guess I should open coherence transaction inside update() method and on any exceptions inside it I should throw RuntimeException myself?
I currently thinking of something like this:
public class CoherenceDao {
private void update(List<DomainObject> list) {
// how should I make it transactional?
// I guess it should somehow throw RuntimeException?
TransactionMap mapTx = CacheFactory.getLocalTransaction(cache);
mapTx.setTransactionIsolation(TransactionMap.TRANSACTION_REPEATABLE_GET);
mapTx.setConcurrency(TransactionMap.CONCUR_PESSIMISTIC);
// gather the cache(s) into a Collection
Collection txnCollection = Collections.singleton(mapTx);
try {
mapTx.begin();
// put into mapTx here
CacheFactory.commitTransactionCollection(txnCollection, 1);
} catch (Throwable t) {
CacheFactory.rollbackTransactionCollection(txnCollection);
throw new RuntimeException();
}
}
}
Would this approach work as expected?
I know that you asked this question a year ago and my answer now might not be as much as value for you after a year but I still give it a try.
What you are trying to do works as long as there is no RuneTimeException after the method call of coherenceDao.update(list); You might be assuming that you don't have any line of codes after that line but that's not the whole story.
As an example: You might have some deferrable constraints in your Database. Those constraints will be applied when the container is trying to commit the transaction which is on method exit of updateMethod(List<DomainObject> list) and after your method call to coherenceDao.update(list). Another cases would be like a connection timeout to database after that coherenceDao.update(list) is executed but still before the transaction commit.
In both cases your update method of CoherenceDAO class is executed safe and sound and your coherence transaction is not rollbacked anymore which will put your cache in an inconsistent state because you will get a RuneTimeException because of those DB or Hibernate Exceptions and that will cause your container managed transaction to be rollbacked!