We Keep Coding

Pending Messages in ActiveMQ

I have deployed my Java-MDB based application using ActiveMQ as messaging service . I could see that a few messages have been in pending status for quite some time on some queues. I have read that this happens when ActiveMQ delivers the message and consumer consumes the message but doesn't send the ack back. But I could not see any related loggers on the consumer/application side which proves that the message is consumed.
Could anyone please help me understand the reason of message being stuck in pending state.
Edit - Adding the details:
We are using Auto-acknowledge as acknowledgeMode and below is the onMessage method used on consumer side.
public void onMessage(Message message) {
try {
// Clear all ThreadLocal in SQLQueryHelper.
SQLQueryHelper.clearCache();
String messageOut = processMessage(message);
// if there is a reply, send it out
if (messageOut != null) {
logger.warn(LoggerKeys.LOG_1_ARGS,
new String[] {"Reply from MDB not supported. " + messageOut});
}
} catch (Throwable e) {
logger.error(LoggerKeys.LOG_1_ARGS,
new String[] {"Error encountered: " + e.toString()});
try {
//put message on error queue
handleError(message, e);
} catch (Throwable e2) {
//retry to put message on error queue
handleErrorAndRollBack(message, e2);
}
}
}

How to requeue a message to the Back of a Rabbit MQ Queue via Spring

I am writing a SpringBoot RabbitMQ Consumer and I have a need to occasionally re queue a message to the BACK of the queue
I thought this was how negative acknowledgment worked, but
basicReject(deliveryTag, true) simply places the message back as close to its original position in the queue as it can, which in my one-at-a-time case is right back at the FRONT of queue.
My first thought was to use a Dead Letter Queue feeding back into the Message Queue on some time interval (similar to the approach mentioned in this answer) but I would rather not create an additional queue if there is some way to simply re queue to the BACK of the initial queue
My below structure simply consumes the message and fails to re-add it to the queue.
How can this be accomplished without a DLQ?
#ServiceActivator(inputChannel = "amqpInputChannel")
public void handle(#Payload String message,
#Header(AmqpHeaders.CHANNEL) Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long deliveryTag){
try{
methodThatThrowsRequeueError();
methodThatThrowsMoveToErrorQueueError();
} catch (RequeueError re) {
channel.basicAck(deliveryTag, false);
sendMessageToBackOfQueue(message);
return;
} catch (MoveToErrorQueueError me) {
//Structured the same as sendMessageToBackOfQueue, works fine
moveMessageToErrorQueue(message);
}
channel.basicAck(deliveryTag, false);
}
private void sendMessageToBackOfQueue(String message) {
try {
rabbitTemplate.convertAndSend(
exchangeName,
routingKeyRequeueMessage,
message,
message -> {
message.getMessageProperties().setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN);
return message;
}
);
} catch (AmqpException amqpEx) {
//error handling which is not triggered...
}
}

TL;DR : There is no way I have found to forward a Message from a listening Service back into the originating Queue with no intermediary.
There are several options that revolve around Dead Letter Queues/Dead Letter Exchanges, but a non-DLQ/DLX solution we found was a timed Exchange, a psuedo DLX if you will. Essentially:
Message enters MessageExchange (MsgX), which propagates to the Service Queue (SvcQ).
The Service (Svc) Gets a Message from the SvcQ.
Once you have determined that the message should be sent to the back of the SvcQ, Svc should:
Send an Acknowledgement to SvcQ.
Send the message to another exchange, our timed psuedo-DLX
The psuedo-DLX can be configured to release messages to the (BACK OF!!) SvcQ on some timed interval

Rabbit-mq hangs to process message after MessageConversionException

I am using Rabbit-mq messaging broker in my application for queuing purpose. Where I will send a chunk of data to one queue, where another consumer which is listening to this queue will convert this message into an user-defined object. . Here is the consumer class code
#RabbitListener(queues = "queue-name")
public void receiveMessage(Message message) {
try {
TestObject o = (TestObject ) new TestObject().fromMessage(message);
//do other processes
} catch (MessageConversionException ex){
//exception thrown
}
Here for some reason, if MessageConversionException is thrown, then all message queue stops its process, and no queue would accept or process any messages. Is there any way to recover from exception?
Even catching this exception is not helping me.

JMS queue receive message?

In the JMS API doc, it said:
public Message receive() throws JMSException
Receives the next message
produced for this message consumer. This call blocks indefinitely
until a message is produced or until this message consumer is closed.
If this receive is done within a transaction, the consumer retains the message until the transaction commits.
Here I have three questions:
1. in the code, do we need while-loop to receive message ? like:
while(true){
Message msg = queue.receive();
....
}
what is the transaction setting ? how to commit a transaction ? like this:
boolean transacted = false;
session = connection.createQueueSession(transacted, Session.AUTO_ACKNOWLEDGE);
receiveNoWait() has transaction support ? how to use it ?
Thanks

If you are going to use receive then you will need some sort of loop to keep receiving messages after the first one is received. Remember that you can also setup a messagelistener and get the received messages async via a callback method and not have to block.
The transaction is generally set to AUTO_ACKNOWLEDGE by default which means that as soon as the message is taken from the queue it is gone and cannot be rolled back. If you want to setup a transaction you need to set the session to transacted and the method to SESSION_TRANSACTED. When you call commit() on the session the messages will be acknowledged on the queue.
receiveNoWait() can have transaction support if you setup the acknowledgement mode correctly and you use commit() and rollback() on the session.
If I were you I would create a MessageListener and not have to worry about spinning a thread to poll the receive methods. Keep in mind that an implicit transaction is started once the session is created.
public class JmsAdapter implements MessageListener, ExceptionListener
{
private ConnectionFactory connFactory = null;
private Connection conn = null;
private Session session = null;
public void receiveMessages()
{
try
{
this.session = this.conn.createSession(true, Session.SESSION_TRANSACTED);
this.conn.setExceptionListener(this);
Destination destination = this.session.createQueue("SOME_QUEUE_NAME");
this.consumer = this.session.createConsumer(destination);
this.consumer.setMessageListener(this);
this.conn.start();
}
catch (JMSException e)
{
//Handle JMS Exceptions Here
}
}
#Override
public void onMessage(Message message)
{
try
{
//Do Message Processing Here
//Message sucessfully processed... Go ahead and commit the transaction.
this.session.commit();
}
catch(SomeApplicationException e)
{
//Message processing failed.
//Do whatever you need to do here for the exception.
//NOTE: You may need to check the redelivery count of this message first
//and just commit it after it fails a predefined number of times (Make sure you
//store it somewhere if you don't want to lose it). This way you're process isn't
//handling the same failed message over and over again.
this.session.rollback()
}
}
}

JMS - Going from one to multiple consumers

I have a JMS client which is producing messages and sending over a JMS queue to its unique consumer.
What I want is more than one consumer getting those messages. The first thing that comes to my mind is converting the queue to a topic, so current and new consumers can subscribe and get the same message delivered to all of them.
This will obviously involve modifying the current clients code in both producer and consumer side of things.
I would like to also look at other options like creating a second queue, so that I don't have to modify the existing consumer. I believe there are advantages in this approach like (correct me if I am wrong) balancing the load between two different queues rather than one, which might have a positive impact on performance.
I would like to get advise on these options and cons / pros that you might see. Any feedback is highly appreciated.

You have a few options as you stated.
If you convert it to a topic to get the same effect you will need to make the consumers persistent consumers. One thing the queue offers is persistence if your consumer isn't alive. This will depend on the MQ system you are using.
If you want to stick with queues, you will create a queue for each consumer and a dispatcher that will listen on the original queue.
Producer -> Queue_Original <- Dispatcher -> Queue_Consumer_1 <- Consumer_1
-> Queue_Consumer_2 <- Consumer_2
-> Queue_Consumer_3 <- Consumer_3
Pros of Topics
Easier to dynamically add new consumers. All consumers will get new messages without any work.
You can create round-robin topics, so that Consumer_1 will get a message, then Consumer_2, then Consumer_3
Consumers can be pushed new messages, instead of having to query a queue making them reactive.
Cons of Topics
Messages are not persistent unless your Broker supports this configuration. If a consumer goes off line and comes back it is possible to have missed messages unless Persistent consumers are setup.
Difficult to allow Consumer_1 and Consumer_2 to receive a message but not Consumer_3. With a Dispatcher and Queues, the Dispatcher can not put a message in Consumer_3's queue.
Pros of Queues
Messages are persistent until a Consumer removes them
A dispatcher can filter which consumers get which messages by not placing messages into the respective consumers queues. This can be done with topics through filters though.
Cons of Queues
Additional Queues need to be created to support multiple consumers. In a dynamic environment this wouldn't be efficient.
When developing a Messaging System I prefer topics as it gives me the most power, but seeing as you are already using Queues it would require you to change how your system works to implement Topics instead.
Design and Implementation of Queue System with multiple consumers
Producer -> Queue_Original <- Dispatcher -> Queue_Consumer_1 <- Consumer_1
-> Queue_Consumer_2 <- Consumer_2
-> Queue_Consumer_3 <- Consumer_3
Source
Keep in mind there are other things you'll need to take care of such as problem exception handling, reconnection to the connection and queues if you lose your connection, etc. This is just designed to give you an idea of how to accomplish what I described.
In a real system I probably wouldn't exit out at the first exception. I would allow the system to continue operating the best it could and log errors. As it stands in this code if putting a message in a single consumers queue fails, the whole dispatcher will stop.
Dispatcher.java
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package stackoverflow_4615895;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueSession;
import javax.jms.Session;
public class Dispatcher {
private static long QUEUE_WAIT_TIME = 1000;
private boolean mStop = false;
private QueueConnectionFactory mFactory;
private String mSourceQueueName;
private String[] mConsumerQueueNames;
/**
* Create a dispatcher
* #param factory
* The QueueConnectionFactory in which new connections, session, and consumers
* will be created. This is needed to ensure the connection is associated
* with the correct thread.
* #param source
*
* #param consumerQueues
*/
public Dispatcher(
QueueConnectionFactory factory,
String sourceQueue,
String[] consumerQueues) {
mFactory = factory;
mSourceQueueName = sourceQueue;
mConsumerQueueNames = consumerQueues;
}
public void start() {
Thread thread = new Thread(new Runnable() {
public void run() {
Dispatcher.this.run();
}
});
thread.setName("Queue Dispatcher");
thread.start();
}
public void stop() {
mStop = true;
}
private void run() {
QueueConnection connection = null;
MessageProducer producer = null;
MessageConsumer consumer = null;
QueueSession session = null;
try {
// Setup connection and queues for receiving the messages
connection = mFactory.createQueueConnection();
session = connection.createQueueSession(false, Session.DUPS_OK_ACKNOWLEDGE);
Queue sourceQueue = session.createQueue(mSourceQueueName);
consumer = session.createConsumer(sourceQueue);
// Create a null producer allowing us to send messages
// to any queue.
producer = session.createProducer(null);
// Create the destination queues based on the consumer names we
// were given.
Queue[] destinationQueues = new Queue[mConsumerQueueNames.length];
for (int index = 0; index < mConsumerQueueNames.length; ++index) {
destinationQueues[index] = session.createQueue(mConsumerQueueNames[index]);
}
connection.start();
while (!mStop) {
// Only wait QUEUE_WAIT_TIME in order to give
// the dispatcher a chance to see if it should
// quit
Message m = consumer.receive(QUEUE_WAIT_TIME);
if (m == null) {
continue;
}
// Take the message we received and put
// it in each of the consumers destination
// queues for them to process
for (Queue q : destinationQueues) {
producer.send(q, m);
}
}
} catch (JMSException ex) {
// Do wonderful things here
} finally {
if (producer != null) {
try {
producer.close();
} catch (JMSException ex) {
}
}
if (consumer != null) {
try {
consumer.close();
} catch (JMSException ex) {
}
}
if (session != null) {
try {
session.close();
} catch (JMSException ex) {
}
}
if (connection != null) {
try {
connection.close();
} catch (JMSException ex) {
}
}
}
}
}
Main.java
QueueConnectionFactory factory = ...;
Dispatcher dispatcher =
new Dispatcher(
factory,
"Queue_Original",
new String[]{
"Consumer_Queue_1",
"Consumer_Queue_2",
"Consumer_Queue_3"});
dispatcher.start();

You may not have to modify the code; it depends on how you wrote it.
For example, if your code sends messages using MessageProducer rather than QueueSender, then it will work for topics as well as queues. Similarly if you used MessageConsumer rather than QueueReceiver.
Essentially, it is good practice in JMS applications to use non-specific interfaces to interact with the JMS system, such as MessageProducer, MessageConsumer, Destination, etc. If that's the case, it's a "mere" matter of configuration.