Given a basic MessageListener implementation which consumes messages from a RabbitMQ queue, how can I send the message to different dead-letter-queues based on the type of exceptions that could be thrown while processing it?
The queue were the messages are originally published has the x-dead-letter-exchange and x-dead-letter-routing-key set on it, but this is not enough in my case.
In case it matters, my application is using Spring 4 and Spring Amqp.
As far as I understand RabbitMQ documentation and Spring AMQP, it is not possible to send a message to different DLQs based on conditions from inside the code. The reason I say this is that my understanding from debugging the code is that when a message has to be send to a DLQ, the code doesn't specify the exchange or the routing key and RabbitMQ uses the ones defined when the message was published.
So, the only solution I found is to implement something like this:
try {
try_to_do_useful_stuff(message);
} catch (BusinessException e) {
resend_the_message_to_business_dlq(message);
}
This way, if a business exception is thrown, then the message is manually send to the business DLQ. Of course, some details get lost, like originating queue, but this is not an issue if they're not used.
When a non-business exception is thrown then the standard path is followed: retry (if configured) and then route to the defined DLQ.
Related
we have a larger multi service java spring app that declares about 100 exchanges and queues in RabbitMQ on startup. Some are declared explicitly via Beans, but most of them are declared implicitly via #RabbitListener Annotations.
#Component
#RabbitListener(
bindings = #QueueBinding(key = {"example.routingkey"},
exchange = #Exchange(value = "example.exchange", type = ExchangeTypes.TOPIC),
value = #Queue(name = "example_queue", autoDelete = "true", exclusive = "true")))
public class ExampleListener{
#RabbitHandler
public void handleRequest(final ExampleRequest request) {
System.out.println("got request!");
}
There are quite a lot of these listeners in the whole application.
The services of the application sometimes talk to each other via RabbitMq, so take a example Publisher that publishes a message to the Example Exchange that the above ExampleListener is bound to.
If that publish happens too early in the application lifecycle (but AFTER all the Spring Lifecycle Events are through, so after ApplicationReadyEvent, ContextStartedEvent), the binding of the Example Queue to the Example Exchange has not yet happend and the very first publish and reply chain will fail. In other words, the above Example Listener would not print "got request".
We "fixed" this problem by simply waiting 3 seconds before we start sending any RabbitMq messages to give it time to declare all queues,exchanges and bindings but this seems like a very suboptimal solution.
Does anyone else have some advice on how to fix this problem? It is quite hard to recreate as I would guess that it only occurs with a large amount of queues/exchanges/bindings that RabbitMq can not create fast enough. Forcing Spring to synchronize this creation process and wait for a confirmation by RabbitMq would probably fix this but as I see it, there is no built in way to do this.
Are you using multiple connection factories?
Or are you setting usePublisherConnection on the RabbitTemplate? (which is recommended, especially for a complex application like yours).
Normally, a single connection is used and all users of it will block until the admin has declared all the elements (it is run as a connection listener).
If the template is using a different connection factory, it will not block because a different connection is used.
If that is the case, and you are using the CachingConnectionFactory, you can call createConnection().close() on the consumer connection factory during initialization, before sending any messages. That call will block until all the declarations are done.
I am implementing a global error handler in a complex system (many queues, many listeners). Inside the handling method, I need to retrieve the name of the queue the message was consumed from. Is that even possible?
My scenario (for full context, but feel free to ignore what follows and focus on the question only)
I want to use the global error handler to catch any non-fatal exception and enqueue the message into a "retry" exchange bound to a "retry" queue with an x-message-ttl of, say, a few seconds and a x-dead-letter-exchange set to the default exchange. I want to set the message's routing key to the queue the message came from so the default exchange will resend it to its original queue. This way all consumers will retry consuming any failed message with a delay, preventing the infamous infinite-retry loop. Hardcoding each queue manually on each consumer is obviously not suitable because there are so many consumers that the solution would be unmaintainable.
EDIT: if not within the error handler, is there any other amqp construct that I can use to intercept the listener and add the queue name to, for example, the message headers so that the error handler would have access to it?
I figured it out. I found out that the message carries information about the queue it comes from.
class MyGlobalErrorHandler implements ErrorHandler {
public void handleError(Throwable t) {
String queueName = ((ListenerExecutionFailedException) t)
.getFailedMessage()
.getMessageProperties()
.getConsumerQueue();
// ...
}
}
I have an amqp-backed channel <int-amqp:channel> to temporarily persist messages coming from a JDBC inbound adapter. when no exception is thrown, the message is ack'd and removed from the rabbit queue. when an exception occurs, the message is returned to the queue and is reprocessed continuosly. There are some circumstances where I'd like the request to go back to the queue, but in most cases I just want to log the error and acknowledge the request (remove from the rabbit queue).
I've implemented an errorHandler to deal with thrown exceptions and allow for logging and "successful" completion, however even after handling, the original request is redelivered to the rabbit queue (unacknowledged)
in the inbound-rabbit-adapter, there is a property for error-channel and handling the message on the errorChannel allows me to achieve the desired behavior described above. the only error property on the amqp channel is errorhandler.
any suggestions on a configuration that would allow me to meet my requirements?
thanks
Throw an AmqpRejectAndDontRequeueException. See 3.9 Exception Handling.
The default error handler does that for message conversion exceptions (which are likely irrecoverable).
In fact you can use that error handler by injecting a custom FatalExceptionStrategy.
I have got a question about Spring AMQP Message:
During processing I was able to update headers of message properties in String AMQP Message with some specific values.
After DeadLettering of this message, all specific headers were disappeared/removed.
Is this behaviour correct ?
Looking forward to your response.
Regards, Anton.
spring-rabbit.version: 1.3.5.RELEASE
spring.version: 4.1.1.RELEASE
The broker knows nothing about your client-side consumer changes; the original message (with its orignal headers) is dead-lettered by the broker (with an x-death header added to indicate the reason - rejection, expiry etc).
In order to do what you want, you need to publish your modified message yourself rather than using dead-lettering.
See the RepublishMessageRecoverer for an example using Spring retry. You can make a custom recover, or simply catch the exception in your listener to republish.
I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}
This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.