We Keep Coding

Pending Messages in ActiveMQ

I have deployed my Java-MDB based application using ActiveMQ as messaging service . I could see that a few messages have been in pending status for quite some time on some queues. I have read that this happens when ActiveMQ delivers the message and consumer consumes the message but doesn't send the ack back. But I could not see any related loggers on the consumer/application side which proves that the message is consumed.
Could anyone please help me understand the reason of message being stuck in pending state.
Edit - Adding the details:
We are using Auto-acknowledge as acknowledgeMode and below is the onMessage method used on consumer side.
public void onMessage(Message message) {
try {
// Clear all ThreadLocal in SQLQueryHelper.
SQLQueryHelper.clearCache();
String messageOut = processMessage(message);
// if there is a reply, send it out
if (messageOut != null) {
logger.warn(LoggerKeys.LOG_1_ARGS,
new String[] {"Reply from MDB not supported. " + messageOut});
}
} catch (Throwable e) {
logger.error(LoggerKeys.LOG_1_ARGS,
new String[] {"Error encountered: " + e.toString()});
try {
//put message on error queue
handleError(message, e);
} catch (Throwable e2) {
//retry to put message on error queue
handleErrorAndRollBack(message, e2);
}
}
}

How to requeue a message to the Back of a Rabbit MQ Queue via Spring

I am writing a SpringBoot RabbitMQ Consumer and I have a need to occasionally re queue a message to the BACK of the queue
I thought this was how negative acknowledgment worked, but
basicReject(deliveryTag, true) simply places the message back as close to its original position in the queue as it can, which in my one-at-a-time case is right back at the FRONT of queue.
My first thought was to use a Dead Letter Queue feeding back into the Message Queue on some time interval (similar to the approach mentioned in this answer) but I would rather not create an additional queue if there is some way to simply re queue to the BACK of the initial queue
My below structure simply consumes the message and fails to re-add it to the queue.
How can this be accomplished without a DLQ?
#ServiceActivator(inputChannel = "amqpInputChannel")
public void handle(#Payload String message,
#Header(AmqpHeaders.CHANNEL) Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long deliveryTag){
try{
methodThatThrowsRequeueError();
methodThatThrowsMoveToErrorQueueError();
} catch (RequeueError re) {
channel.basicAck(deliveryTag, false);
sendMessageToBackOfQueue(message);
return;
} catch (MoveToErrorQueueError me) {
//Structured the same as sendMessageToBackOfQueue, works fine
moveMessageToErrorQueue(message);
}
channel.basicAck(deliveryTag, false);
}
private void sendMessageToBackOfQueue(String message) {
try {
rabbitTemplate.convertAndSend(
exchangeName,
routingKeyRequeueMessage,
message,
message -> {
message.getMessageProperties().setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN);
return message;
}
);
} catch (AmqpException amqpEx) {
//error handling which is not triggered...
}
}

TL;DR : There is no way I have found to forward a Message from a listening Service back into the originating Queue with no intermediary.
There are several options that revolve around Dead Letter Queues/Dead Letter Exchanges, but a non-DLQ/DLX solution we found was a timed Exchange, a psuedo DLX if you will. Essentially:
Message enters MessageExchange (MsgX), which propagates to the Service Queue (SvcQ).
The Service (Svc) Gets a Message from the SvcQ.
Once you have determined that the message should be sent to the back of the SvcQ, Svc should:
Send an Acknowledgement to SvcQ.
Send the message to another exchange, our timed psuedo-DLX
The psuedo-DLX can be configured to release messages to the (BACK OF!!) SvcQ on some timed interval

RabbitMq: works through times, not as I expect

And so, step by step, what do I want to do:
I receive data if an error occurs when sending to another system, then I want to send data to rabbitMQ:
#Override
public void updateAnketaIfThrowThenSendMessageInRabbit(ProfileId profileId, ChangeClientAnketaRequest anketa, String profileVersion) {
try {
anketaService.updateAnketa(profileId, anketa, profileVersion);
} catch (ClubProNotAvailableException e) {
rabbitTemplate.convertAndSend(config.getExchange(), config.getRoutingKey(), clubProNotAvailableRabbit);
Anketa a = conversionService.convert(conversionService.convert(anketa, UgAnketa.class), Anketa.class);
profileService.updateProfileAnketa(profileId, a, null);
}
}
}
Next, I want to accept these data and queues and try sending them again at a certain time interval.
For this i:
I accept messages
I'm trying to resend it:
a) If everything was successful, I delete it from the queue
b) If an error occurred, I call the stop method for the container. After a certain time I use the scheduler to call the start method for the container
#Override
public void onMessage(Message message, Channel channel) throws IOException {
ClubProNotAvailableRabbit data = null;
try {
data = OBJECT_MAPPER.readValue(message.getBody(), ClubProNotAvailableRabbit.class);
MDC.put(data.getRequestContextRabbit().getRequestId(), UUID.randomUUID().toString());
requestContextService.put(createRequestContext(data.getRequestContextRabbit(), data.getRequestContextRabbit().getFront()));
methodCall(data);
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} catch (ClubProNotAvailableException e) {
listenerContainer.stop();
throw new ClubProNotAvailableException();
}
}
public void startContainer() {
listenerContainer.start();
}
I have encountered such problems:
The message is not delivered to the queue every time. Sometimes I have to call the convert And Send method several times.
When I got messages from the queue and an error occurred, I turn off the container, then when it turns on, the queue is empty, and when I turn off, I see this message:
2020-07-20 21:36:59.878 [INFO ] o.s.a.r.l.SimpleMessageListenerContainer - Workers not finished.
2020-07-20 21:36:59.878 [WARN ] o.s.a.r.l.SimpleMessageListenerContainer - Closing channel for unresponsive consumer: Consumer#77416991: tags=[[amq.ctag-E62UisbYdAAOQIM2bWr08w]], channel=Cached Rabbit Channel: AMQChannel(amqp://usergate_tst#10.64.177.12:5672/,35), conn: Proxy#6c60c170 Shared Rabbit Connection: SimpleConnection#5fb65b3a [delegate=amqp://usergate_tst#10.64.177.12:5672/, localPort= 59801], acknowledgeMode=MANUAL local queue size=0
How can I fix this situation?
CONTINUED QUESTION.
I corrected the code like this:
#Override
public void onMessage(Message message, Channel channel) throws IOException, InterruptedException {
ClubProNotAvailableRabbit data = null;
try {
data = OBJECT_MAPPER.readValue(message.getBody(), ClubProNotAvailableRabbit.class);
MDC.put(data.getRequestContextRabbit().getRequestId(), UUID.randomUUID().toString());
requestContextService.put(createRequestContext(data.getRequestContextRabbit(), data.getRequestContextRabbit().getFront()));
methodCall(data);
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} catch (ClubProNotAvailableException e) {
channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
Thread.sleep(20000);
}
}
Thread.sleep here for experiment.
I expect that when I grab a message from the queue in the rabbitmq admin console, I will see it go to Unacked status, this is how it happens.
Then, when an error occurs, I call the basicReject method, and I want the status to become ready, immediately after the basicReject call line, but it becomes ready as soon as the method completes completely.
Unacked status:
Although the baseReject method has already worked.
Why is this happening? how is it supposed to work and what mechanism? why doesn't the message become immediately ready (status in console rabbit) after calling the baseReject method?

Closing channel for unresponsive consumer:
This means the listener is "stuck" in your code - you can't call stop() from the listener itself - the container.stop() waits for the listener to exit. You should use stop(() -> log.info("stopped container")) instead.
You need to basicReject in the catch case - the container won't handle it for you with MANUAL acks.
You MUST use MANUAL acks if you ack/nack the message yourself.
It's generally better to let the container take care of acking your messages.

How to receive message from wildfly jms queue using consumer

I encountered a knotty problem when receiving message from WildFly JMS queue. My code is below:
Session produceSession = connectionFactory.createConnection().createSession(false, Session
.CLIENT_ACKNOWLEDGE);
Session consumerSession = connectionFactory.createConnection().createSession(false, Session
.CLIENT_ACKNOWLEDGE);
ApsSchedule apsSchedule = new ApsSchedule();
boolean success;
MessageProducer messageProducer = produceSession.createProducer(outQueueMaxusOrder);
success = apsSchedule.sendD90Order(produceSession,messageProducer, d90OrderAps);
if (!success) {
logger.error("Can't send APS schedule msg ");
} else {
MessageConsumer consumer = consumerSession.createConsumer(inQueueDeliveryDate);
data = apsSchedule.receiveD90Result(consumerSession,consumer);
}
then getting into the receiveD90Result():
public DeliveryData receiveD90Result(Session session, MessageConsumer consumer) {
DeliveryData data = null;
try {
Message message = consumer.receive(10000);
if (message == null) {
return null;
}
TextMessage msg = (TextMessage) message;
String text = msg.getText();
logger.debug("Receive APS d90 result: {}", text);
ObjectMapper mapper = new ObjectMapper();
data = mapper.readValue(text, DeliveryData.class);
} catch (JMSException je) {
logger.error("Can't receive APS d90 order result: {}", je.getMessage());
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
consumer.close();
} catch (JMSException e) {
e.printStackTrace();
}
}
return data;
}
But when implementing the consumer.receive(10000), the project can't get a message from queue. If I use asynchronous way of MDB to listen the queue, I can get the message from queue. How to resolve it?

There are multiple modes you can choose to get a message from the queue. Message Queues are by default asynchronous in usage. There are however cases when you want to read it synchronously , for example sending a message with account number and using another queue to read the response and match it with a message id or a message correlation id. When you do a receive , the program is waiting for a message to arrive within that polling interval specified in receive.
The code snippet you have , as i see it uses the psuedo synchronous approach. If you have to use it as an MDB , you will have to implement message driven bean (EJB Resource) or message listener.
The way that MDB/Message Listener works is more event based , instead of a poll with a timeout (like the receive) , you implement a callback called onMessage() that is invoked every time there is a message. Instead of a synchronous call , this becomes asynchronous. Your application may require some changes both in terms of design.

I don't see where you're calling javax.jms.Connection.start(). In fact, it doesn't look like you even have a reference to the javax.jms.Connection instance used for your javax.jms.MessageConsumer. If you don't have a reference to the javax.jms.Connection then you can't invoke start() and you can't invoke close() when you're done so you'll be leaking connections.
Furthermore, connections are "heavy" objects and are meant to be re-used. You should create a single connection for both the producer and consumer. Also, if your application is not going to use the javax.jms.Session from multiple threads then you don't need multiple sessions either.

JMS queue receive message?

In the JMS API doc, it said:
public Message receive() throws JMSException
Receives the next message
produced for this message consumer. This call blocks indefinitely
until a message is produced or until this message consumer is closed.
If this receive is done within a transaction, the consumer retains the message until the transaction commits.
Here I have three questions:
1. in the code, do we need while-loop to receive message ? like:
while(true){
Message msg = queue.receive();
....
}
what is the transaction setting ? how to commit a transaction ? like this:
boolean transacted = false;
session = connection.createQueueSession(transacted, Session.AUTO_ACKNOWLEDGE);
receiveNoWait() has transaction support ? how to use it ?
Thanks

If you are going to use receive then you will need some sort of loop to keep receiving messages after the first one is received. Remember that you can also setup a messagelistener and get the received messages async via a callback method and not have to block.
The transaction is generally set to AUTO_ACKNOWLEDGE by default which means that as soon as the message is taken from the queue it is gone and cannot be rolled back. If you want to setup a transaction you need to set the session to transacted and the method to SESSION_TRANSACTED. When you call commit() on the session the messages will be acknowledged on the queue.
receiveNoWait() can have transaction support if you setup the acknowledgement mode correctly and you use commit() and rollback() on the session.
If I were you I would create a MessageListener and not have to worry about spinning a thread to poll the receive methods. Keep in mind that an implicit transaction is started once the session is created.
public class JmsAdapter implements MessageListener, ExceptionListener
{
private ConnectionFactory connFactory = null;
private Connection conn = null;
private Session session = null;
public void receiveMessages()
{
try
{
this.session = this.conn.createSession(true, Session.SESSION_TRANSACTED);
this.conn.setExceptionListener(this);
Destination destination = this.session.createQueue("SOME_QUEUE_NAME");
this.consumer = this.session.createConsumer(destination);
this.consumer.setMessageListener(this);
this.conn.start();
}
catch (JMSException e)
{
//Handle JMS Exceptions Here
}
}
#Override
public void onMessage(Message message)
{
try
{
//Do Message Processing Here
//Message sucessfully processed... Go ahead and commit the transaction.
this.session.commit();
}
catch(SomeApplicationException e)
{
//Message processing failed.
//Do whatever you need to do here for the exception.
//NOTE: You may need to check the redelivery count of this message first
//and just commit it after it fails a predefined number of times (Make sure you
//store it somewhere if you don't want to lose it). This way you're process isn't
//handling the same failed message over and over again.
this.session.rollback()
}
}
}