I need to read a message from a queue and call a REST service doing a post.
Then, I was looking about the JMS transaction with Camel, it looks like you can set a maximumRedeliveries to process the queue message again, so rolling back the transaction in case of a failure, I was wondering how it can work if in the same camel route we have to call a REST service to post something, how that part can be rollback??
maxDelivery conf:
errorHandler(new TransactionErrorHandlerBuilder()
.loggingLevel(LoggingLevel.ERROR)
.useOriginalMessage()
.maximumRedeliveries(2)
.logHandled(false)
.logExhausted(true)
);
Pseudo code for the route:
//Reading message from the queue
from("activemq:AMQ.App.EMC2.In.PMQueue?jmsMessageType=Bytes").
transacted().
unmarshal(jaxbDataFormat).bean(pmMessageEnricher).
to("direct:start-post");
//Then doing the post
from("direct:start-post").
setHeader(Exchange.HTTP_METHOD, constant("POST")).
setHeader(Exchange.CONTENT_TYPE, constant("application/json")).
setBody(constant(pmMessageEnricher.toJson())).
to("http://xxx").
to("direct:start-somethingelse");
//Then doing something else
from("direct:start-somethingelse").
blabla...
Let say an exception occurs in the start-somethingelse, How the REST post call can be roll backed ? since we call an external service in a stateless way.
Your doubt is correct. In case of a JMS transaction rollback, the POST request cannot be rolled back because the service provider is not part of the JMS transaction. The transaction is just between the JMS broker and the Camel JMS consumer (see also Camel transactional client).
However, if you catch the processing error, you can apply the needed compensation logic. For example delete the already posted data with another request.
By the way: don't confuse Camel redeliveries and broker redeliveries!
Camel redeliveries are done by the Camel Errorhandler (not the broker). In your example it does up to 2 redeliveries. But be aware, that Camel redeliveries are not reprocessing the entire route, but only the failed processor.
So if to("http://xxx") fails and the Camel Errorhandler does redeliveries, Camel retries only the to("http://xxx").
In contrast, if your JMS transaction is rolled back, the broker redelivers the message to Camel and the entire route is processed again.
Take care that you don't "mask" the JMS redelivery with your Camel Errorhandler.
Related
I am trying to run a camel transacted() route (a standalone java process) with JPATransactionManager is the spring PlatformTransactionManager (as I want camel route to run in a single DB transaction) but I am not able to suppress redelivery from MQ Broker in case a transactional method fails even though I have used handled(true) in onException clause along with my custom redelivery policy (which is executed successfully). I only want MQ to redeliver when there is a service crash.
Tried below but it doesn't work:
Setting setTransacted(false) in JMSComponent config so as to prevent camel jms to run is transacted_session jms mode but it does not work
doTry and doCatch the exception from transactional block
camel redeliveries followed by handled(true).
onException(Exception.class)
.log("ERROR OCCURRED")
.redeliveryPolicyRef("myRedeliveryPolicy")
.handled(true)
.to(getPostExceptionRoute());
#Bean
#Autowired
public RedeliveryPolicy myRedeliveryPolicy() {
RedeliveryPolicy myRedeliveryPolicy= new RedeliveryPolicy();
myRedeliveryPolicy.setMaximumRedeliveries(2);
myRedeliveryPolicy.setMaximumRedeliveryDelay(2000);
return myRedeliveryPolicy;
}
#Bean
#Autowired
public JmsComponent jms(IJMSConnectionFactory cf) throws JMSException {
JmsComponent jmsComponent = new JmsComponent();
jmsComponent.setConfiguration(jmsConfig(cf));
jmsComponent.setTransacted(false);
return jmsComponent;
}
from("jms:queue:TestQueue?acknowledgementModeName=CLIENT_ACKNOWLEDGE")
.unmarshal().json(JsonLibrary.Jackson, TestObject.class)
.transacted()
.processRef("myPersistInDBProcessor")
I expect camel to try redeliveries as per redelivery policy (working) but MQ should not redeliver.
I expect my camel route to run in a single db transaction.
I expect MQ broker to redeliver only when my java service crashes in middle of processing, so that I do not lose the message.
I expect camel to try redeliveries as per redelivery policy (working) but MQ should not redeliver
When MQ must never do a redelivery (because you handle errors in Camel), you should remove acknowledgementModeName=CLIENT_ACKNOWLEDGE or explicitly set AUTO_ACKNOWLEDGE (default value).
As long as a message is not acknowledged, it is from the broker perspective not delivered. AUTO_ACKNOWLEDGE immediately acknowledges the message after consumption what makes sense if you never want to get redeliveries.
CLIENT_ACKNOWLEDGE on the other hand only acknowledges a message under certain conditions, see this post for some more info about this.
EDIT due to comment with new information
If you want MQ redeliveries, but "override" them with Camel in most cases, you have to consume messages transacted.
Use local JMS broker transactions by configuring your JMS component like this
jmsComponent.setLazyCreateTransactionManager(false);
jmsComponent.setTransacted(true);
For this type of transaction you don't need a Spring TransactionManager at all. So I guess the JPATransactionManager is ignored by JMS and your JMS consumptions should be transactional.
Now, when your Camel error handler "swallows" an Exception by using handled(true), there must be no MQ redelivery. But MQ does a redelivery when an Exception is propagated back to the broker.
I expect my camel route to run in a single db transaction
I did not find anything in your question about not working database transactions. There seems to be just one processor that does database stuff. If this is not working correctly, please describe the problem in your question or a separate question.
According to the Apache Karaf Transaction Guide should doTry and doCatch work as expected. What is probably the problem in your case is the Exception triggering the error scenario. Only checked exceptions (no RuntimeException or it's descendant) don't mark ongoing transaction for rolling back.
I have a REST service defined in Spring Boot, which exposes a synchronous REST API, to be invoked by a user from a Web based UI. The service is expected to provide a real time response.
The service talks to an external endpoint in an asynchronous fashion. That is:
- A single one - way outbound message for the request
- A single one - way inbound message for the response
How can I combine the two messages to provide an impression of a synchronous behavior? I am thinking of a few approaches like:
The Rest Service posts a request to the endpoint. Once the endpoint responds, the response is added to a ConcurrentHashMap. The Rest Service queries the HashMap every few milliseconds and once it finds the right response it exits with a valid HTTP reason code.
It is akin to polling and I am thinking if we can avoid that.
The Rest Service posts a request to the endpoint. Once the endpoint responds, the waiting thread in the Rest Service is notified. However the waiting thread should conclude only if the right response is received (i.e. matching correlation Ids etc.)
Is this possible?
I realize that this is similar to a JMS Queue Request Response scenario, where each JMS queue request opens up a listener on the response queue with a message selector criteria.
However in this case I have to implement this using HTTP.
Any thoughts on this are welcome. I am fully convinced I am missing something very basic, but am not sure what.
Thanks a lot!
I have camel route which suspends the route using control bus on exception.
onException(Exception.class).maximumRedeliveries(1).onRedeliveryRef("controlBusProcessor");
from("quartz2://myGroup/myTimerName?trigger.repeatInterval=2000&trigger.repeatCount=0")
.routeId("myRoute")
.process(simpleProcessor)
.to("stream:out")
The route initially calls a simple processor which raises an Exception, the onException has a redeliveryRef which calls a control bus processor which suspends the route, however it still calls simple processor to redeliver the message.
If the route is suspended, why is simple processor is still called, should it not wait until the route resumes?
No the route will complete its in-flight messages before being suspended. In your custom redelivery processor, you can mark the exchange to stop continue routing by setting the property on the exchange with Exchange.STOP=true.
See how the StopProcessor does that: https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/processor/StopProcessor.java
I suspect the redelivery starts at the point of failure which is at .process(simpleProcessor). But if you only want to suspend the route why not just add .onException(Exception.class).to("controlBusProcessor")
I am using Apache Camel to send messages to my Java service. I have kept transacted=true on consumer route. I also need to send e-mail on successfully processing of JMS messages.
I am using below code to register synchronization and send e-mail only after transaction is committed.
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter(){
#Override
public void afterCommit(){
sendMail(mailBody);
}
});
Problem: incoming transaction from Camel is not synchronized and I am getting
java.lang.IllegalStateException: Transaction synchronization is not active
I tried calling transactionsynchronizationmanager.initsynchronization() - I am not getting any exception but afterCommit() method is never called.
transactionsynchronizationmanager.initsynchronization();
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter(){
#Override
public void afterCommit(){
sendMail(mailBody); //never called
}
});
Same code is working when request is received via spring mvc controller (through Spring Transaction).
You likely need to turn on transacted on the route to enable spring transaction. The option transacted=true on the JMS endpoint is NOT spring-transaction, but its only for JMS acknowledge mode to be set as transacted. They are not the same.
So in your Camel route, setup spring transaction as well, eg
from jms
transacted
See more details in the Camel docs: http://camel.apache.org/transactional-client.html or even better if you have a copy of the Camel in Action book (1st or 2nd ed) then it has a fully chapter devoted to transactions.
I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}
This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.