We Keep Coding

How to prevent message redelivery from MQ Broker after camel internal redeliveries are successfully processed (transacted camel route)

I am trying to run a camel transacted() route (a standalone java process) with JPATransactionManager is the spring PlatformTransactionManager (as I want camel route to run in a single DB transaction) but I am not able to suppress redelivery from MQ Broker in case a transactional method fails even though I have used handled(true) in onException clause along with my custom redelivery policy (which is executed successfully). I only want MQ to redeliver when there is a service crash.
Tried below but it doesn't work:
Setting setTransacted(false) in JMSComponent config so as to prevent camel jms to run is transacted_session jms mode but it does not work
doTry and doCatch the exception from transactional block
camel redeliveries followed by handled(true).
onException(Exception.class)
.log("ERROR OCCURRED")
.redeliveryPolicyRef("myRedeliveryPolicy")
.handled(true)
.to(getPostExceptionRoute());
#Bean
#Autowired
public RedeliveryPolicy myRedeliveryPolicy() {
RedeliveryPolicy myRedeliveryPolicy= new RedeliveryPolicy();
myRedeliveryPolicy.setMaximumRedeliveries(2);
myRedeliveryPolicy.setMaximumRedeliveryDelay(2000);
return myRedeliveryPolicy;
}
#Bean
#Autowired
public JmsComponent jms(IJMSConnectionFactory cf) throws JMSException {
JmsComponent jmsComponent = new JmsComponent();
jmsComponent.setConfiguration(jmsConfig(cf));
jmsComponent.setTransacted(false);
return jmsComponent;
}
from("jms:queue:TestQueue?acknowledgementModeName=CLIENT_ACKNOWLEDGE")
.unmarshal().json(JsonLibrary.Jackson, TestObject.class)
.transacted()
.processRef("myPersistInDBProcessor")
I expect camel to try redeliveries as per redelivery policy (working) but MQ should not redeliver.
I expect my camel route to run in a single db transaction.
I expect MQ broker to redeliver only when my java service crashes in middle of processing, so that I do not lose the message.

I expect camel to try redeliveries as per redelivery policy (working) but MQ should not redeliver
When MQ must never do a redelivery (because you handle errors in Camel), you should remove acknowledgementModeName=CLIENT_ACKNOWLEDGE or explicitly set AUTO_ACKNOWLEDGE (default value).
As long as a message is not acknowledged, it is from the broker perspective not delivered. AUTO_ACKNOWLEDGE immediately acknowledges the message after consumption what makes sense if you never want to get redeliveries.
CLIENT_ACKNOWLEDGE on the other hand only acknowledges a message under certain conditions, see this post for some more info about this.
EDIT due to comment with new information
If you want MQ redeliveries, but "override" them with Camel in most cases, you have to consume messages transacted.
Use local JMS broker transactions by configuring your JMS component like this
jmsComponent.setLazyCreateTransactionManager(false);
jmsComponent.setTransacted(true);
For this type of transaction you don't need a Spring TransactionManager at all. So I guess the JPATransactionManager is ignored by JMS and your JMS consumptions should be transactional.
Now, when your Camel error handler "swallows" an Exception by using handled(true), there must be no MQ redelivery. But MQ does a redelivery when an Exception is propagated back to the broker.
I expect my camel route to run in a single db transaction
I did not find anything in your question about not working database transactions. There seems to be just one processor that does database stuff. If this is not working correctly, please describe the problem in your question or a separate question.

According to the Apache Karaf Transaction Guide should doTry and doCatch work as expected. What is probably the problem in your case is the Exception triggering the error scenario. Only checked exceptions (no RuntimeException or it's descendant) don't mark ongoing transaction for rolling back.

How reliable is JMS rollback?

Code...
#Transactional
#JmsListener(destination = "QueueA")
public void process(String input) {
doSomethingWhichMayThrowException(input);
}
Consider following situation where...
Transaction is started (using Spring #Transactional annotation)
Persistent JMS message is read from QueueA (Queue use disk as message storage)
Disk is full and do not accept any write operations
Exception happens and transaction is rolled back
Is message lost?
If it's not then how message is read from queue under transaction (step 2)?
Is some kind of a queue browser used so message is read from queue but not consumed?

Is message lost?
No, the message is NOT lost as the transaction is rolledback.
If it's not then how message is read from queue under transaction
(step 2) ?
Once after the message listener's process()/onMessage() method completes and returns with a success or exception, then internally message acknowledgment (default is AUTO_ACKNOWLEDGE) happens (which is the last thing implicitly happens) to the JMS provider(IBMMQ, ActiveMQ, SonicMQ, etc..) which tells that the transaction is successful or not.
If the Transaction is successful, JMS provider deletes the message from the queue/topic.
If the Transaction is NOT successful, JMS provider preserve the message as is (until message TimetoLive expires).
Is some kind of a queue browser used so message is read from queue but
not consumed ?
You can think that it is like queue browser concept, but it is upto the implementation of the JMS provider how do they implement this internally. In order to achieve this, the message broker just reads the message content, but do not delete the actual message from the queue/topic until the acknowledgement is received from the message listener's process()/onMessage() method for the current transaction.

Spring AMQP MessageProperties:all headers were removed during deadlettering

I have got a question about Spring AMQP Message:
During processing I was able to update headers of message properties in String AMQP Message with some specific values.
After DeadLettering of this message, all specific headers were disappeared/removed.
Is this behaviour correct ?
Looking forward to your response.
Regards, Anton.
spring-rabbit.version: 1.3.5.RELEASE
spring.version: 4.1.1.RELEASE

The broker knows nothing about your client-side consumer changes; the original message (with its orignal headers) is dead-lettered by the broker (with an x-death header added to indicate the reason - rejection, expiry etc).
In order to do what you want, you need to publish your modified message yourself rather than using dead-lettering.
See the RepublishMessageRecoverer for an example using Spring retry. You can make a custom recover, or simply catch the exception in your listener to republish.

Different dead-letter-queues in Spring Amqp + RabbitMQ based on exception

Given a basic MessageListener implementation which consumes messages from a RabbitMQ queue, how can I send the message to different dead-letter-queues based on the type of exceptions that could be thrown while processing it?
The queue were the messages are originally published has the x-dead-letter-exchange and x-dead-letter-routing-key set on it, but this is not enough in my case.
In case it matters, my application is using Spring 4 and Spring Amqp.

As far as I understand RabbitMQ documentation and Spring AMQP, it is not possible to send a message to different DLQs based on conditions from inside the code. The reason I say this is that my understanding from debugging the code is that when a message has to be send to a DLQ, the code doesn't specify the exchange or the routing key and RabbitMQ uses the ones defined when the message was published.
So, the only solution I found is to implement something like this:
try {
try_to_do_useful_stuff(message);
} catch (BusinessException e) {
resend_the_message_to_business_dlq(message);
}
This way, if a business exception is thrown, then the message is manually send to the business DLQ. Of course, some details get lost, like originating queue, but this is not an issue if they're not used.
When a non-business exception is thrown then the standard path is followed: retry (if configured) and then route to the defined DLQ.

JMS and MDB with setRollbackOnly

I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}

This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.