I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}
This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.
Related
we have a larger multi service java spring app that declares about 100 exchanges and queues in RabbitMQ on startup. Some are declared explicitly via Beans, but most of them are declared implicitly via #RabbitListener Annotations.
#Component
#RabbitListener(
bindings = #QueueBinding(key = {"example.routingkey"},
exchange = #Exchange(value = "example.exchange", type = ExchangeTypes.TOPIC),
value = #Queue(name = "example_queue", autoDelete = "true", exclusive = "true")))
public class ExampleListener{
#RabbitHandler
public void handleRequest(final ExampleRequest request) {
System.out.println("got request!");
}
There are quite a lot of these listeners in the whole application.
The services of the application sometimes talk to each other via RabbitMq, so take a example Publisher that publishes a message to the Example Exchange that the above ExampleListener is bound to.
If that publish happens too early in the application lifecycle (but AFTER all the Spring Lifecycle Events are through, so after ApplicationReadyEvent, ContextStartedEvent), the binding of the Example Queue to the Example Exchange has not yet happend and the very first publish and reply chain will fail. In other words, the above Example Listener would not print "got request".
We "fixed" this problem by simply waiting 3 seconds before we start sending any RabbitMq messages to give it time to declare all queues,exchanges and bindings but this seems like a very suboptimal solution.
Does anyone else have some advice on how to fix this problem? It is quite hard to recreate as I would guess that it only occurs with a large amount of queues/exchanges/bindings that RabbitMq can not create fast enough. Forcing Spring to synchronize this creation process and wait for a confirmation by RabbitMq would probably fix this but as I see it, there is no built in way to do this.
Are you using multiple connection factories?
Or are you setting usePublisherConnection on the RabbitTemplate? (which is recommended, especially for a complex application like yours).
Normally, a single connection is used and all users of it will block until the admin has declared all the elements (it is run as a connection listener).
If the template is using a different connection factory, it will not block because a different connection is used.
If that is the case, and you are using the CachingConnectionFactory, you can call createConnection().close() on the consumer connection factory during initialization, before sending any messages. That call will block until all the declarations are done.
I am implementing a global error handler in a complex system (many queues, many listeners). Inside the handling method, I need to retrieve the name of the queue the message was consumed from. Is that even possible?
My scenario (for full context, but feel free to ignore what follows and focus on the question only)
I want to use the global error handler to catch any non-fatal exception and enqueue the message into a "retry" exchange bound to a "retry" queue with an x-message-ttl of, say, a few seconds and a x-dead-letter-exchange set to the default exchange. I want to set the message's routing key to the queue the message came from so the default exchange will resend it to its original queue. This way all consumers will retry consuming any failed message with a delay, preventing the infamous infinite-retry loop. Hardcoding each queue manually on each consumer is obviously not suitable because there are so many consumers that the solution would be unmaintainable.
EDIT: if not within the error handler, is there any other amqp construct that I can use to intercept the listener and add the queue name to, for example, the message headers so that the error handler would have access to it?
I figured it out. I found out that the message carries information about the queue it comes from.
class MyGlobalErrorHandler implements ErrorHandler {
public void handleError(Throwable t) {
String queueName = ((ListenerExecutionFailedException) t)
.getFailedMessage()
.getMessageProperties()
.getConsumerQueue();
// ...
}
}
I have an application with a main thread and a JMS thread which talk to each other through ActiveMQ 5.15.11. I am able to send messages just fine, however I would like a way to send back status or errors. I noticed that the MessageListener allows for onSuccess() and onException(ex) as two events to listen for, however I am finding that only onSuccess() is getting called.
Here are snippets of my code.
JMS Thread:
ConnectionFactory factory = super.getConnectionFactory();
Connection connection = factory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(super.getQueue());
MessageConsumer consumer = session.createConsumer(queue);
consumer.setMessageListener(m -> {
try {
super.processRmbnConfigMsg(m);
} catch (JMSException | IOException e) {
LOG.error(e.getMessage(), e);
// I can only use RuntimeException.
// Also this exception is what I am expecting to get passed to the onException(..)
// call in the main thread.
throw new RuntimeException(e);
}
});
connection.start();
Main thread (sending messages to JMS):
sendMessage(xml, new AsyncCallback() {
#Override
public void onException(JMSException e) {
// I am expecting this to be that RuntimeException from the JMS thread.
LOG.error("Error", e);
doSomethingWithException(e);
}
#Override
public void onSuccess() {
LOG.info("Success");
}
});
What I am expecting is that the exceptions thrown in the new RuntimeException(e) will get picked up on the onException(JMSException e) event listener, in some way, even if the RuntimeException is wrapped.
Instead, I am always getting onSuccess() events. I suppose the onException(..) event happens during communication issues, but I would like a way to send back to the caller exceptions.
How do I accomplish that goal of collecting errors in the JMS thread and sending it back to my calling thread?
Your expectation is based on a fundamental misunderstanding of JMS.
One of the basic tenets of brokered messaging is that producers and consumers are logically disconnected from each other. In other words...A producer sends a message to a broker and it doesn't necessarily care if it is consumed successfully or not, and it certainly won't know who consumes it or have any guarantee when it will be consumed. Likewise, a consumer doesn't necessarily know when or why the message was sent or who sent it. This provides great flexibility between producers and consumers. JMS adheres to this tenet of disconnected producers and consumers.
There is no direct way for a consumer to inform a producer about a problem with the consumption of the message it sent. That said, you can employ what's called a "request/response pattern" so that the consumer can provide some kind of feedback to the producer. You can find an explanation of this pattern along with example code here.
Also, the AsyncCallback class you're using is not part of JMS. I believe it's org.apache.activemq.AsyncCallback provided exclusively by ActiveMQ itself and it only provides callbacks for success or failure for the actual send operation (i.e. not for the consumption of the message).
Lastly, you should know that throwing a RuntimeException from the onMessage method of a javax.jms.MessageListener is considered a "programming error" by the JMS specification and should be avoided. Section 8.7 of the JMS 2 specification states:
It is possible for a listener to throw a RuntimeException; however, this is considered a client programming error. Well behaved listeners should catch such exceptions and attempt to divert messages causing them to some form of application-specific 'unprocessable message' destination.
The result of a listener throwing a RuntimeException depends on the session's acknowledgment mode.
AUTO_ACKNOWLEDGE or DUPS_OK_ACKNOWLEDGE - the message will be immediately redelivered. The number of times a JMS provider will redeliver the same message before giving up is provider-dependent. The JMSRedelivered message header field will be set, and the JMSXDeliveryCount message property incremented, for a message redelivered under these circumstances.
CLIENT_ACKNOWLEDGE - the next message for the listener is delivered. If a client wishes to have the previous unacknowledged message redelivered, it must manually recover the session.
Transacted Session - the next message for the listener is delivered. The client can either commit or roll back the session (in other words, a RuntimeException does not automatically rollback the session).
What's the difference between SimpleMessageListenerContainer and DirectMessageListenerContainer in Spring AMQP? I checked both of their documentation pages, SimpleMessageListenerContainer has almost no explanation on inner workings, and DirectMessageListenerContainer has the following explanation:
The SimpleMessageListenerContainer is not so simple. Recent changes to the rabbitmq java client has facilitated a much simpler listener container that invokes the listener directly on the rabbit client consumer thread. There is no txSize property - each message is acked (or nacked) individually.
I don't really understand what these mean. It says listener container that invokes the listener directly on the rabbit client consumer thread. If so, then how does SimpleMessageListenerContainer do the invocation?
I wrote a small application and used DirectMessageListenerContainer and just to see the difference, I switched to SimpleMessageListenerContainer, but as far as I can see there was no difference on RabbitMQ side. From Java side the difference was in methods (SimpleMessageListenerContainer provides more) and logs (DirectMessageListenerContainer logged more stuff)
I would like to know the scenarios to use each one of those.
The SMLC has a dedicated thread for each consumer (concurrency) which polls an internal queue. When a new message arrives for a consumer on the client thread, it is put in the internal queue and the consumer thread picks it up and invokes the listener. This was required with early versions of the client to provide multi-threading. With the newer client that is not a problem so we can invoke the listener directly (hence the name).
There are a few other differences aside from txSize.
See Choosing a Container.
In the DirectMessageListenerContainer some of the logic is moved into the AMQP implementation as opposed to ListenerContainer as is SimpleMessageListenerContainer
This is what the Javadocs in SimpleMessageListenerContainer say for setTxSize() -
/**
* Tells the container how many messages to process in a single transaction (if the channel is transactional). For
* best results it should be less than or equal to {#link #setPrefetchCount(int) the prefetch count}. Also affects
* how often acks are sent when using {#link AcknowledgeMode#AUTO} - one ack per txSize. Default is 1.
* #param txSize the transaction size
*/
The client sends an ack every time txSize number of messages are processed. This is controlled in the method
private boolean doReceiveAndExecute(BlockingQueueConsumer consumer) throws Throwable { //NOSONAR
Channel channel = consumer.getChannel();
for (int i = 0; i < this.txSize; i++) {
logger.trace("Waiting for message from consumer.");
Message message = consumer.nextMessage(this.receiveTimeout);
.
.
In the newer implementations, each message is acked on the thread directly and based on the transactional model (Single or publisher confirms) the consumer sends Acknowledgments to Rabbit MQ
Given a basic MessageListener implementation which consumes messages from a RabbitMQ queue, how can I send the message to different dead-letter-queues based on the type of exceptions that could be thrown while processing it?
The queue were the messages are originally published has the x-dead-letter-exchange and x-dead-letter-routing-key set on it, but this is not enough in my case.
In case it matters, my application is using Spring 4 and Spring Amqp.
As far as I understand RabbitMQ documentation and Spring AMQP, it is not possible to send a message to different DLQs based on conditions from inside the code. The reason I say this is that my understanding from debugging the code is that when a message has to be send to a DLQ, the code doesn't specify the exchange or the routing key and RabbitMQ uses the ones defined when the message was published.
So, the only solution I found is to implement something like this:
try {
try_to_do_useful_stuff(message);
} catch (BusinessException e) {
resend_the_message_to_business_dlq(message);
}
This way, if a business exception is thrown, then the message is manually send to the business DLQ. Of course, some details get lost, like originating queue, but this is not an issue if they're not used.
When a non-business exception is thrown then the standard path is followed: retry (if configured) and then route to the defined DLQ.