In this queue processing I wanted to put the sending itself into separate transactions to avoid rolling back the sent status. However I run into problem due to the closed session or detached entity at the lazy fetching.
This is my code:
// Before this is called queueToSend is fetched in a separate transaction
#TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void sendMails(List<QueuedMail> queueToSend)
{
for (QueuedMail queuedMail: queueToSend) {
sendMail(queuedMail);
}
}
public void sendMail(QueuedMail queuedMail)
{
System.out.println("Checking "+queuedMail);
crud.getEntityManager().merge(queuedMail);
Mail mail = (Mail)crud.find(queuedMail.getMail().getId(),Mail.class);
System.out.println("mail id: "+queuedMail.getMail().getId());
crud.getEntityManager().merge(mail);
Boolean unsubscribed = false;
Set<Campaign> campaigns = mail.getCampaigns();
for (Campaign campaign: campaigns) {
if (campaign.getUnsubscribedUsers().contains(queuedMail.getUser())) {
unsubscribed = true;
}
}
// ...
}
The error is:
failed to lazily initialize a collection of role: dps.simplemailing.entities.Mail.campaigns, could not initialize proxy - no Session
And comes first at the line:
for (Campaign campaign: campaigns) {
I thought it can be because the queuedMail is already detached, however I am trying to reattach both queuedMail and mail using merge, but it doesn't help.
Also maybe it's already in cache, and that is why it doesn't start a new session.
Basically I want it to do the same as before (before I added TransactionAttribute), but as a separate transaction within the loop. I don't think I should do any vendor-specific solution as it seems to be a trivial task.
Update:
I have done some research, and found that I have to use the result of the merge operation in order to reattach the detached entity. This changed the vendor-specific error (as using lazy loading properties of a detached entity undefined) to a vendor-independent error, and I think it shows the real problem.
The modified code is:
public void sendMail(QueuedMail queuedMail)
{
System.out.println("Checking "+queuedMail);
queuedMail = crud.getEntityManager().merge(queuedMail);
Mail mail = (Mail)crud.getEntityManager().merge(queuedMail.getMail());
Boolean unsubscribed = false;
Set<Campaign> campaigns = mail.getCampaigns();
for (Campaign campaign: campaigns) {
if (campaign.getUnsubscribedUsers().contains(queuedMail.getUser())) {
unsubscribed = true;
}
}
I have also changed the entity to define the cascade type:
#ManyToMany(mappedBy = "mails",cascade = CascadeType.MERGE)
private Set<Campaign> campaigns;
Now the error is this:
Transaction is required to perform this operation (either use a transaction or extended persistence context)
What I don't really understand about this error is that the transaction should have been started as the class default is TransactionAttributeType.REQUIRED. The intention for setting the other methods to TransactionAttributeType.NOT_SUPPORTED was to start the transaction in this method. I am beginner in transactions and entity manager, so I will just continue research to understand it better, but maybe I get answer here sooner, or if I find the solution I will post it.
I found the glitch, it's pretty tricky, enough to say, that no further research shown the real problem, but I got the answer during taking time out and walking.
As I told, the main idea for this method was that the sendMails has no transaction, while the sendMail (should) have a transaction, and this would have to provide one transaction for every item to be processed.
The error in the update shown to me that there is no transaction in the sendMail, and I were really confused about it.
The real reason is found in the fact that the bean is executed using a proxy. Normally this is done from outside of the bean itself. If the bean calls it's own method directly, the container doesn't have the chance to proxy the request, nor it can start a transaction.
The corrected code has an injection point in the bean injecting it's own instance, but using the container:
#Stateless
public class MailQueue {
#Inject MailQueue mailQueue;
// ...
}
And using this injection point one method can call the other method via the proxy:
#TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void sendMails(List<QueuedMail> queueToSend)
{
for (QueuedMail queuedMail: queueToSend) {
mailQueue.sendMail(queuedMail);
}
}
public void sendMail(QueuedMail queuedMail)
{
// ...
}
With this solution the container has the chance to start the new transaction before the call to sendMail.
Related
I have a test method which sometimes fails during deploy and sometimes does not. I have never seen it fail on my local. You can see my code below.
I have the following retry mechanism which is asynchronously called from another service:
#Transactional
public boolean retry(NotificationOrder order) {
notificationService.send(order);
return true;
}
public void resolveOnFailedAttempt(Long orderId) { //automatically called if `retry` method fails
notificationOrderCommonTransactionsService.updateNotificationOrderRetryCount(orderId);
}
The notification service is like this :
#Service
#RequiredArgsConstructor
public class NotificationServiceImpl implements NotificationService {
private final NotificationOrderCommonTransactionsService notificationOrderCommonTransactionsService;
#Override
#Transactional
public NotificationResponse send(NotificationOrder order) {
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e;
}
...
return response;
}
private void prepareNotificationRequest(NotificationOrder order) {
...
throw new Exception("ERROR");
}
}
And the commmon transactions service is like this :
#Transactional(propagation = Propagation.REQUIRES_NEW)
public NotificationOrder saveNotificationOrderErrorMessage(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setErrorDescription(errorMessage);
notificationRepository.save(order);
return order;
}
public NotificationOrder updateNotificationOrderRetryCount(Long orderId) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
notificationRepository.save(order);
return order;
}
Here is my integration test :
#Test
public void test() {
NotificationOrderRequest invalidRequest = invalidRequest();
ResponseEntity<NotificationOrderResponse> responseEntity = send(invalidRequest);
NotificationOrder notificationOrder = notificationOrderRepository.findOne(1);
softly.assertThat(notificationOrder.getOrderStatus().isEqualTo(NotificationOrderStatus.IN_PROGRESS))
softly.assertThat(notificationOrder.getErrorDescription())
.isEqualTo("ERROR"); //This the line that fails.
softly.assertThat(responseEntity.getStatusCode()).isEqualTo(HttpStatus.OK);
}
In the test method it is confirmed that updateNotificationOrderRetryCount is called and the order status is updated as IN_PROGRESS. However, the error message is null and I get the following assertion error :
-- failure 1 --
Expecting:
<null>
to be equal to:
<"ERROR">
but was not.
I expect saveNotificationOrderErrorMessage transaction to be completed and the changes to be committed before updateNotificationOrderRetryCount method is called. But it seems like it does work that way. Could anyone help me find out why my code behave like this ?
How can I reproduce this error on my local? And what can I do to fix it ?
Thanks.
Try enabling SQL logging and parameter bind logging and look through the statements. I don't know all of your code, but maybe your are setting the message to null somewhere? It could also be, that the actions are interleaving somehow such that updateNotificationOrderRetryCount is called before/while saveNotificationOrderErrorMessage in a way that causes this. If both run right before commit, but saveNotificationOrderErrorMessage commits before updateNotificationOrderRetryCount, you could see the error message being overwritten with null.
If the code snippet of the question is accurate, pay attention to the fact that you are rethrowing the exception raised in the prepareNotificationRequest method, I assume in order to enable the retry mechanism:
NotificationRequest request;
try {
request = prepareNotificationRequest(order);
} catch (Exception e) {
notificationOrderCommonTransactionsService.saveNotificationOrderErrorMessage(order.getId(),
e.getMessage());
throw e; // You are rethrowing the exception
}
For your comment, the exception thrown extends RuntimeException.
As the Spring documentation indicates:
In its default configuration, the Spring Frameworkâs transaction infrastructure code marks a transaction for rollback only in the case of runtime, unchecked exceptions. That is, when the thrown exception is an instance or subclass of RuntimeException. ( Error instances also, by default, result in a rollback). Checked exceptions that are thrown from a transactional method do not result in rollback in the default configuration.
Probably Spring is performing rollback of the initial transaction, that one associated with saveNotificationOrderErrorMessage. I realize that this method is annotated as #Transactional(propagation = Propagation.REQUIRES_NEW) and that it is initiating a new transaction, but perhaps the problem could be related with it.
When the retry mechanism takes place, another transaction, associated with the invocation of the method updateNotificationOrderRetryCount is performed, and this transaction is successfully committed. This is the reason why the changes performed in this second method are properly committed.
The solution of the problem will depend on how your retry mechanism is implemented, but you can, for example, raise the original exception and, as a first step in the retry mechanism, trace the problem in the database or, raise a checked exception - Spring by default will not perform rollback for it - and handle it as appropriate.
Update
Another possible reason of the problem could be the transaction demarcations in the send method.
This method is annotated as #Transactional. As a consequence, Spring will initiate a new transaction for it.
The error occurs, and you trace the error in the database, in a new transaction but please, be aware that the initial transaction is still there.
Although not described in your code, in some way, the retry mechanism takes place, and updates the retry count. It this operation is performed within the initial transaction (or a higher level one), due to the transaction boundaries, database isolation levels, and related stuff, it is possible that this transaction, the initial, fetch an actually outdated, but current from the transaction boundary point of view, NotificationOrder. And this information is the one that finally is committed, overwriting the information of the error. I hope you get the idea.
One simple solution, maybe for both possibilities, could be to include the error message in the updateNotificationOrderRetryCount method itself, reducing the problem to a single transaction:
/* If appropriate, mark it as Transactional */
#Transactional
public NotificationOrder updateNotificationOrderRetryCount(Long orderId, String errorMessage) {
NotificationOrder order = notificationRepository.findOne(orderId);
order.setRetryCount(order.getRetryCount() + 1);
order.setOrderStatus(NotificationOrderStatus.IN_PROGRESS);
order.setErrorDescription(errorMessage);
// It is unnecessary, all the changes performed in the entity within the transaction will be committed
// notificationRepository.save(order);
return order;
}
I have the following code that I am expecting to persist all or none of the entities in the method.
However, it is behaving that some entities are created and others are not - i.e. the whole transaction is not being rolled back.
Why is this so?
Note - I am running my code as an EAR file in JBOSS EAP server
#TransactionAttribute(value = TransactionAttributeType.REQUIRES_NEW)
public void createCompanyStatuses(String client, CompanyStatusPostDTO companyStatusPostDTO) {
EntityManager entityManager = null;
try {
CompanyStatus companyStatus = new CompanyStatus();
companyStatus.setCompanyLabel(candidateMaskedStatusPostDTO.getCompanyLabel());
entityManager = entityManagement.createEntityManager(client);
entityManager.persist(companyStatus);
for(Integer employeeStatusId: companyStatusPostDTO.getEmployeeStatuses()){
CompanyStatusEmployeeStatus companyStatusEmployeeStatus = new CompanyStatusEmployeeStatus();
companyStatusEmployeeStatus.setEmployeeId(employeeStatusId);
companyStatusEmployeeStatus.setCompanyId(companyStatus.getCompanyId()); //todo - how will get this?
entityManager.persist(CompanyStatusEmployeeStatus);
}
} catch(Exception e){
log.error("An exception has occurred in inserting data into the table" + e.getMessage(), e);
} finally {
entityManagement.closeEntityManager(client, entityManager);
}
}
Answer valid for Hibernate.
TransactionAttributeType.REQUIRES_NEW is not supported.
It is hard to implement rollback on java object. Imagine scenario when:
Transaction started
Object created and saved
Subtransaction started
Object modified
Subtransaction rolled back.
You expect object to be in state it was before subtransaction started, so you need to trace information about modification of that object inside subtransaction and ability to roll those modification back.
Alternatively you could reload state from DB, but you need to track which object belong to which transaction.
I assume developers just decided that it is too much effort for little gain.
I have a SpringBoot 2 project and i'm using spring data jpa with hibernate with MySQL5.7
I have problems with the following use case: i have a service method that calls another service's method. If second service's method generates a runtime exception, also the first method is marked as rollback and i cannot commit things anymore. I'd like to only rollback second method and still commit something in the first one.
I tried to use propagation.NESTED but nested transaction are not allowed with hibernate (even if jpaTransactionManager supports them and MySQL supports savepoints).
How can i solve this problem? Can i configure nested in some way?
Please remember i need second method to see changes committed by first so i can't mark the second method as propagation.REQUIRES_NEW
Here is come sample code to clarify my problem:
FirstServiceImpl.java
#Service
public class FirstServiceImpl implements FirstService
#Autowired
SecondService secondService;
#Autowired
FirstServiceRepository firstServiceRepository;
#Transactional
public void firstServiceMethod() {
//do something
...
FirstEntity firstEntity = firstServiceRepository.findByXXX();
firstEntity.setStatus(0);
firstServiceRepository.saveAndFlush(firstEntity);
...
boolean runtimeExceptionHappened = secondService.secondServiceMethod();
if (runtimeExceptionHappened) {
firstEntity.setStatus(1);
firstServiceRepository.save();
} else {
firstEntity.setStatus(2);
firstServiceRepository.save();
}
}
SecondServiceImpl.java
#Service
public class SecondServiceImpl implements SecondService
#Transactional
public boolean secondServiceMethod() {
boolean runtimeExceptionHappened = false;
try {
//do something that saves to db but that may throw a runtime exception
...
} catch (Exception ex) {
runtimeExceptionHappened = true;
}
return runtimeExceptionHappened;
}
So the problem is that when secondServiceMethod() raises a runtime exception it rollback its operations (and that's OK) and then set its return variable runtimeExceptionHappened to false, but then firstServiceMethod is marked as rollback only and then
firstEntity.setStatus(1);
firstServiceRepository.save();
isn't committed.
Since i can't use NESTED propagation how can i achieve my goal?
I would suggest you break them up into two separate transactions.
In the first transaction do all of the work presently in firstServiceMethod that you know you want to commit. (e.g. through saveAndFlush). Now as you exit this method the changes are committed, so they will be available to subsequent calls.
Then have whatever called firstServiceMethod call a new Transactional method setFirstEntityStatus() that calls secondServiceMethod and sets the status of the entity as appropriate.
Basically, instead of attempting to NEST the transactions, split them into two fully separate transactions and use the ordering to ensure the result of the 1st is available to the 2nd.
I have a code where transaction control is done manually,
But the code calls methods of other EJB's of the same application where the transaction is not controlled manually, for example, there are methods using the Hibernate Session that is managed by the Container.
How to prevent my manually managed transaction from downloading the commit when there is a query method using eg a Session.createCriteria.
When this occurs, my transaction unloads the commit before my process is actually completed.
Example
private void exe() throws Exception {
#EJB Businessbusiness;
this.beginTransaction();
business.processar(); // Exemplo
this.commit();
}
#Stateless
public class Business() {
#EJB
private DAO dao;
private void processar() throws Exception {
// executando processo 1
this.save();
// executando processo 2
this.update();
// Saving and updating has not yet been committed. So far it is correct.
Teste = dao.buscarTeste(1L);
// Here, after performing the search, my transaction downloads the commit to the bank without completing the whole process.
}
}
#Stateless
public class DAO() {
public Teste buscarTeste(Long codigo) {
Criteria cri = getSession().createCriteria(Teste.class);
cri.add(Restrictions.eq("codigo", codigo));
return (Teste) cri.uniqueResult();
}
}
I am not sure I actually got your point.
But assuming you are not getting any error, when you invoke the Business.processar() method it inherits the transaction. Which remains 'pending' until the exe client commit.
So, I would investigate what your getSession() does in the middle, I am quite sure it starts a brand new transaction, which is going to retrieve uncommitted data.
By the way, is there a reason for using hibernate instead of JPA with hibernate as a concrete implementation ?
In my service code, I am trying to create or update a Person domain object:
#Transactional
def someServiceMethod(some params....) {
try{
def person = Person.findByEmail(nperson.email.toLowerCase())
if (!person) {
person = new Person()
person.properties = nperson.properties
} else {
// update the person parameters (first/last name)
person.firstName = nperson.firstName
person.lastName = nperson.lastName
person.phone = nperson.phone
}
if (person.validate()) {
person.save(flush: true)
//... rest of code
}
// rest of other code....
} catch(e) {
log.error("Unknown error: ${e.getMessage()}", e)
e.printStackTrace()
return(null)
}
Now above code OCCASIONALLY when trying to save a Person object with already existing email throws following exception:
Hibernate operation: could not execute statement; SQL [n/a]; Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'
This is very strange because I am already finding the person by email and hence the save() should try to update the record instead of creating the new one.
I was wondering why is this happening!
EDIT:
I am on grails 2.4.5 and Hibernate plugin in BuildConfig is:
runtime ':hibernate4:4.3.8.1'
EDIT2:
My application is on multiple servers hence synchronized block won't work
If this is concurrency issue, here is what we do in such case. We have a lot of concurrent background processes which work on the same tables. If there is such operation it indeed is in synchronized block, so code may look like:
class SomeService {
static transactional = false //service cannot be transactional
private Object someLock = new Object() //synchronized block on some object must be used
def someConcurrentSafeMethod(){
synchronized(someLock){
def person = Person.findByEmail(nperson.email.toLowerCase())
...
person.save(flush: true) // flush is very important, must be done in synchronized block
}
}
}
There are few important points to make this working (from our experience, not official):
Service cannot be transactional - if service is transactional, transaction is commited after the method returns value and synchronization inside method will not be enough. Programmatic transactions may be another way
synchronized method is not enough synchronized def someConcurrentSafeMethod() will not work - probably because service is wrapped in proxy
Session MUST be flushed inside synchronized block
every object which will be saved, should be read in synchronized block, if you pass it from external method, you may run into optimistic locking failed exception
UPDATED
Because application is deployed on distributed system, above will not solve the issue here (still may help others). After discussion we had on Slack, I just summarize potential ways to do that:
pessimistic locking of updated objects and lock of whole table for inserts (if possible)
moving 'dangerous' database related methods to single server with some API like REST and calling it from other deployments (and using synchronized approach from above)
using multiple save approach - if operation fails, catch exception and try again. This is supported by integration libraries like Spring Integration or Apache Camel and is one of enterprise patterns. See request-handler-advice-chain for Spring Integration as an example
use something to queue operations, for example JMS server
If anyone has more ideas please share them.