I am trying to understand Spring AMQP. I can't get my head aroud the transaction.
What is the use case of transaction? Is it similar to JTA? Why would someone rollback a Message?
See the documentation about transaction semantics.
No, it's not "similar to JTA" it applies only to RabbitMQ itself - it can't participate in a global transaction.
One use case:
publish multiple records and one fails - then all are rolled back
i.e. all published records are secured in the broker, or none are.
Related
I was wondering on a Spring Boot method marked as #Transactional, when should a message appear on the queue? I’m asking because I’ve noticed 2 different behaviours on 2 different applications.
Both applications have the following in common:
Spring Boot 2.0 based
JMS message sending is using JmsTemplate, with setSessionTransacted set to true
No explicit transaction settings configured
There is a Mongo DB used (using Spring Data) and a record is being modified in the same method as the message is sent
The major difference between the two applications is:
One has a JPA data source (using Spring Dataj and a record is read and/or written in this method. The other application does not have this data source.
The difference in observed behaviour is that when the JPA source is present, the message is sent at the end of method. Without it, the message is sent immediately.
Is this the expected behaviour?
Can I configure the applications to behave the same way? Ideally I’d like the message to be sent at the end (so any Mongo changes that fail would cancel the message send and rollback any JPA changes made)? I realise that Mongo changes are not part of any transaction created.
Thanks
With JMS and a DB you have two resources.
To have a full transactional behavior you need distributed transactions support.
If you don't have this even when the message is sent as last operation if the sending fails the data is changed in the database anyway.
To configure distributed transaction you need JTA. This is described here:
With JMS and a DB you have two resources. To have a full transactional behavoir you need distributed transactions. We use Bistronix in our application and this works very well.
Have a look at the docs: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-jta.html
Recently I was asked a question which left me thinking..want to get the community views on the same question.
I have a CustomerEJB which has say a createCustomer method. My EJB is exposed as a web service and hence createCustomer is one of its operations.
When a request hits createCustomer, 2 operations need to be performed
An INSERT SQL query into the database which may be adding certain data into db that came in input request
creation of a text file, say .txt in the file system.
Now the question is I want to couple these two tasks into a transaction. If any one task fails, I rollback the other task as well.
Without mentioning any hot technologies, like Spring/Hibernate what is the approach I can follow for Transaction management
My thoughts:
1. I can use JTA, demarcate the transaction boundaries and perform commit and rollback accordingly. JDBC can be used for the SQL task
2. I can use DAOs
Inviting your kind suggestions/comments
You would need to wrap the file creating in a XA capable JCA connector (not sure whether there's a ready made one out there, a quick good only found this fsconnector which doesn't support transactions yet), and use an XA driver for your DB transaction (most DBs will will be able handle this) and then wrap your EJB in an XA transaction (should be straightforward).
As long as both resources can handle the XA transactions, you'll get the benefit of 2-phase commits, which is what you're after.
I think this is a fairly common question: how to put my business logic in a global transaction in distributed systems environment? Cite an example, I have a TaskA containing couples of sub tasks:
TaskA {subtask1, subtask2, subtask3 ... }
each of these subtasks may execute on local machine or a remote one, I hope TaskA executes in an atomic manner(success or fail) by means of transaction. Every subtask has a rollback function, once TaskA thinks the operation fails(because one of subtask fails), it calls each rollback function of subtasks. Otherwise TaskA commits the whole transaction.
To do this, I follow "Audit trial" transaction pattern to have a record for each subtask, so TaskA can know operation results of subtasks then decide rollback or commit. This sounds like simple, however, the hard part is how to associate each subtask to the global transaction?
When TaskA begins, it starts a global transaction about which subtask knows nothing. To make subtask aware of it, I have to pass the transaction context to every invocation of subtask. This is really dreadful! My subtask may either execute in a new thread or execute in remote by sending a message through AMQP broker, it's hard to consolidate the way of context propagation.
I did some research like "Transaction Patterns - A Collection of Four Transaction Related Patterns", "Checked Transactions in an Asynchronous Message Passing Environment", none of these solve my problem. They either don't have practical example or don't solve the context propagation issue.
I wonder how people solve this? as this sort of transaction must be common in enterprise software.
Is X/Open XA only the solution for this? Can JTA help here(I have't look into JTA as most stuff for it relate to database transaction, and I am using Spring, I don't want to involve another Java EE application server in my software).
Can some expert share some thoughts with me? thank you.
Conclusion
Arjan and Martin gave out really good answers, thank you.
Finally I didn't go with this way. After more research, I chose another pattern "CheckPoint" 1.
Looking into my requirement, I found my intention to "Audit Trial Transaction Pattern" is to know which level the operation has proceeded to, if it's failed, I can restart it at the failed spot after reloading some context. Actually this is not transaction, it didn't rollback other successful steps after failure. This is essence of CheckPoint pattern.
However, studying distributed transaction stuff makes me learned lot of interesting things. Apart from what Arjan and Martin have mentioned. I also suggest people digging into this area take a look at CORBA which is a well-known protocol for distributed system.
You're right, you need two-phase commit support thanks to a XA transaction manager provided by the JTA API.
As far as I know Spring does not provide a transaction manager implementation itself. The JtaTransactionManager only delegates to existing implementation usually provided from JavaEE implementations.
So you will have to plugin a JTA implementation into Spring to get effective job done. Here are some proposals:
JOTM
JBossTS based on Arjuna.
Atokimos
Then you will have to implement your resource manager to support two-phase commit. In the JavaEE worlds, it consists in a resource adapter packaged as a RAR archive. Depending on the type of resource, the following aspects are to read/implement:
XAResource interface
JCA Java Connector Architecture mainly if a remote connection is involved
JTS Transaction Service if transaction propagation between nodes is required
As examples, I recommend you to look at implementations for the classical "transactions with files" issue:
the transactional file manager from JBoss Transactions
XADisk project
If you want to write your own transactional resource, you indeed need to implement an XAResource and let it join an ongoing transaction, handle prepare and commit requests from the transaction manager etc.
Datasources are the most well known transactional resources, but as mentioned they are not the only ones. You already mentioned JMS providers. A variety of Caching solutions (e.g. Infinispan) are also transactional resources.
Implementing XAResources and working with the lower level part of the JTA API and the even lower level JTS (Java Transaction Service) is not a task for the faint of heart. The API can be archaic and the entire process is only scarcely documented.
The reason is that regular enterprise applications creating their own transactional resources is extremely rare. The entire point of being transactional is to do an action that is atomic for an external observer.
To be observable in the overwhelming majority of cases means the effects of the action are present in a database. Nearly each and every datasource is already transactional, so that use case is fully covered.
Another observable effect is whether a message has been send or not, and that too is fully covered by the existing messaging solutions.
Finally, updating a (cluster-wide) in memory map is yet another observable effect, which too is covered by the major cache providers.
There's a remnant of demand for transactional effects when operating with external enterprise information systems (EIS), and as a rule of thumb the vendors of such systems provide transaction aware connectors.
The shiver of use cases that remain is so small that apparently nobody ever really bothered to write much about it. There are some blogs out there that cover some of the basics, but much will be left to your own experimentation.
Do try to verify for yourself if you really absolutely need to go down this road, and if your need can't be met by one of the existing transactional resources.
You could do the following (similar to your checkpoint strategy):
TaskA executes in a (local) JTA transaction and tries to reserve the necessary resources before it delegates to your subtasks
Subtask invocations are done via JMS/XA: if step 1 fails, no subtask will ever be triggered and if step 1 commits then the JMS invocations will be received at each subtask
Subtasks (re)try to process their invocation message with a JMS max redelivery limit set (see your JMS vendor docs on how to do this)
Configure a "dead letter queue" for any failures in 3.
This works, assuming that:
-retrying in step 3 makes sense
-there is a bit of human intervention needed for messages in the dead letter queue
If this is not acceptable then there is also TCC: http://www.atomikos.com/Main/DownloadWhitepapers?article=TccForRestApi.pdf - this can be seen as a "reservation" pattern for REST services.
Hope this helps
Guy
http://www.atomikos.com
I have a spring web app running on jboss that is currently configured to use the HibernateTransactionManager for db transactions and the JmsTransactionManager for jms. For jms we use Camel and ActiveMQ, our database is DB2. Within a transaction I need to write a number of records to the database and send two asynchronous jms messages. The jms messages are event notifications and I only want them to be sent if the database transaction commits.
I am willing to accept the risk of the communication with the broker failing after the jdbc transaction has already committed (and thus no messages sent but db committed) so I do not think I need proper XA.
I believe that what I need is "best efforts" transaction management using spring transaction synchronization.
The spring documentation sort of hints at the fact that spring will synchronize the two transactions and commit the jms transaction only after the jdbc transaction has been committed - but I don't think it is very clear. The spring documentation here http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/transaction.html#tx-resource-synchronization doesn't go into enough detail about how it works.
I have found a couple of other sources that say spring will do what I want including some javadoc below, and I have written some integration tests that also show it.
http://static.springsource.org/spring/docs/3.0.x/api/org/springframework/jms/support/JmsAccessor.html#setSessionTransacted%28boolean%29 The javadoc on setSessionTransacted here sounds like exactly what I want.
From what I have seen I think creating the Camel JmsConfiguration with transacted set to true like this is enough:
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="transacted" value="true"/>
<property name="concurrentConsumers" value="10"/>
</bean>
However I need to convince someone I work with who is a bit skeptical and thinks that my integration test only works because of a poorly documented side effect rather than a intentional spring feature.
So my question is - Am I correct that spring can be relied upon to synchronize the transactions and always commit the jms transaction after the jdbc transaction or is that not something that I should rely on, and could you point me at any official documentation that says that clearly? And I guess in general is this a good approach to take or should we be managing these transactions in a different way?
This article might be of help Distributed transactions in Spring, with and without XA. I don't think it covers your case specifically - sending message + updating database.
Official Spring Boot repository contain JTA examples that combine JMS with JDBC based on Atomikos, Bitronix or Java EE server JBoss WildFly.
Additionally I also created few examples that are located in my Github repository. This contains also non-Spring Boot (pure Spring) example.
If you are using Local transactions
And the usecase is save to database and then send to jms
Then there could be three cases :
Exception just after receiving(before DB and JMS)
No problem everything will be rolledback
After saving to DB , we have exception
If there is an insert operation,there will be mutiple rows in DB due to retries.With each retry , an insert will be done.And for JMS , message will go to DeadLetterQueue
After saving to DB and sending to JMS , we have an exception
If there is an insert operation,there will be mutiple rows in DB due to retries.With each retry , an insert will be done.And for JMS , message will go to DeadLetterQueue
Now you dont want to use XA, so the solutions could be
1)Check If(message.getJmsRedelivered() {…}
If not , process it
If its redelivered , check if you processed it already
Check if the data is in database based on details in message
Note that re-deliveries are rare so , this check is also rare and there is no overhead
2)If your method is idempotent , then you dont need this check
And regarding XA , XA guarantees that message is delivered only once
And synchronize the transaction across multiple resources
But with XA , you have overhead
So if you can manage without XA , it is preferable
I have a web-app, which I deploy on Tomcat 6 and it uses Hibernate.
It receives messages on a JMS queue which trigger changes both to my DB, via Hibernate and to an Object of mine (Agent).
The web-requests also access the DB, via Hibernate, and access the shared object (there's a ConcurrentHashMap<AgentId,Agent> held by a singleton).
My problem is that I have a JMS message which changes several different Agents and several tables and I need the changes in the Agents to be available if and only if the DB transaction completed successfully. In addition I do not want to employ read locks as that is too much of a performance hazard for me.
I was thinking of somehow implementing the XAResource interface for my singleton and then use JTA to manage both my singleton and my Hibernate transaction.
What do you think? Does it sound reasonable? Am I way off?
If any additional details are needed please don't hesitate to ask
Ittai
Instead of implementing XAResource, you could use a transactional cache like EHCache which supports JTA since 2.0 (i.e. it can act as an XA resource and participate in a XA transaction alongside other XA resources).