We Keep Coding

How to requeue a message to the Back of a Rabbit MQ Queue via Spring

I am writing a SpringBoot RabbitMQ Consumer and I have a need to occasionally re queue a message to the BACK of the queue
I thought this was how negative acknowledgment worked, but
basicReject(deliveryTag, true) simply places the message back as close to its original position in the queue as it can, which in my one-at-a-time case is right back at the FRONT of queue.
My first thought was to use a Dead Letter Queue feeding back into the Message Queue on some time interval (similar to the approach mentioned in this answer) but I would rather not create an additional queue if there is some way to simply re queue to the BACK of the initial queue
My below structure simply consumes the message and fails to re-add it to the queue.
How can this be accomplished without a DLQ?
#ServiceActivator(inputChannel = "amqpInputChannel")
public void handle(#Payload String message,
#Header(AmqpHeaders.CHANNEL) Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long deliveryTag){
try{
methodThatThrowsRequeueError();
methodThatThrowsMoveToErrorQueueError();
} catch (RequeueError re) {
channel.basicAck(deliveryTag, false);
sendMessageToBackOfQueue(message);
return;
} catch (MoveToErrorQueueError me) {
//Structured the same as sendMessageToBackOfQueue, works fine
moveMessageToErrorQueue(message);
}
channel.basicAck(deliveryTag, false);
}
private void sendMessageToBackOfQueue(String message) {
try {
rabbitTemplate.convertAndSend(
exchangeName,
routingKeyRequeueMessage,
message,
message -> {
message.getMessageProperties().setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN);
return message;
}
);
} catch (AmqpException amqpEx) {
//error handling which is not triggered...
}
}

TL;DR : There is no way I have found to forward a Message from a listening Service back into the originating Queue with no intermediary.
There are several options that revolve around Dead Letter Queues/Dead Letter Exchanges, but a non-DLQ/DLX solution we found was a timed Exchange, a psuedo DLX if you will. Essentially:
Message enters MessageExchange (MsgX), which propagates to the Service Queue (SvcQ).
The Service (Svc) Gets a Message from the SvcQ.
Once you have determined that the message should be sent to the back of the SvcQ, Svc should:
Send an Acknowledgement to SvcQ.
Send the message to another exchange, our timed psuedo-DLX
The psuedo-DLX can be configured to release messages to the (BACK OF!!) SvcQ on some timed interval

Set maximum consumer count jms solace

I am trying to set the maximum consumer count of a topic endpoint with jms using solace as a broker, so for increasing load, multiple instances of the app can be started in cloudfoundry, and multiple subscribers can consume messages of the same topic.
I have tried multiple combinations of the below settings (setConcurrency(), setConcurrentConsumers(), setMaxConcurrentConsumers(), (20 as an arbitrary high number). Judging from the documentation, I definitely need to use setMaxConcurrentConsumers() and set this to an appropriately high value.
When I deploy the app, the topic endpoint gets created, but when I look at the solace management interface, the maximum consumer count is always 1 (as can be seen here: Queues -> Topic Endpoints -> select endpoint -> Configured Limit), even though it should be 20. So the second consumer is not able to connect. I don't want to set this manually every time I deploy the app.
import javax.jms.*;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.connection.CachingConnectionFactory;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.listener.DefaultMessageListenerContainer;
#Configuration
public class ProducerConfiguration {
private static final Log logger = LogFactory.getLog(SolaceController.class);
#Value("${durable_subscription}")
private String subscriptionName;
#Value("${topic_name}")
private String topic_name;
#Autowired
private ConnectionFactory connectionFactory;
#Bean
public JmsTemplate jmsTemplate() {
CachingConnectionFactory ccf = new CachingConnectionFactory(connectionFactory);
JmsTemplate jmst = new JmsTemplate(ccf);
jmst.setPubSubDomain(true);
return jmst;
}
#Bean
public Session configureSession(ConnectionFactory connectionFactory) throws JMSException {
return connectionFactory.createConnection().createSession(false, Session.AUTO_ACKNOWLEDGE);
}
private TextMessage lastReceivedMessage;
public class SimpleMessageListener implements MessageListener {
#Override
public void onMessage(Message message) {
if (message instanceof TextMessage) {
lastReceivedMessage = (TextMessage) message;
try {
logger.info("Received message : " + lastReceivedMessage.getText());
} catch (JMSException e) {
logger.error("Error getting text of the received TextMessage: " + e);
}
} else {
logger.error("Received message that was not a TextMessage: " + message);
}
}
}
#Bean
public DefaultMessageListenerContainer orderMessageListenerContainer() {
DefaultMessageListenerContainer lc = new DefaultMessageListenerContainer();
lc.setConnectionFactory(connectionFactory);
lc.setDestinationName(topic_name);
lc.setMessageListener(new SimpleMessageListener());
lc.setDurableSubscriptionName(subscriptionName);
lc.setPubSubDomain(true);
//tried multiple combinations here, also setting only setMaxConcurrentConsumers
lc.setConcurrency("2-20");
lc.setConcurrentConsumers(20);
lc.setMaxConcurrentConsumers(20);
lc.setSubscriptionDurable(true);
lc.initialize();
lc.start();
return lc;
}
}

I think for your use case, your consumer is stuck with queues. See https://solace.com/blog/topic-subscription-queues/
"... while multiple consumers can bind to Queues
Durable endpoints are limited to a single topic subscription. Queues allow multiple topic subscriptions as well as topic wildcards."
If you don't want to change your publisher you can try "Topic Subscription on Queues". That is a queue can be configured to listen on a topic. And then your consumers would get messages from that queue.

You need to create a non-exclusive queue/endpoint.
By default, the queue you create are exclusive queues/endpoints, which means only one subscriber can bind to it at any time.
The easiest way to create such a queue/endpoint is through the Solace CLI.
To create a non-exclusive queue in your JMS program, you have to go into Solace specific JMS implementation like this:
if (queueName != null) {
EndpointProperties props = new EndpointProperties();
props.setAccessType(EndpointProperties.ACCESSTYPE_NONEXCLUSIVE);
try {
((SolConnection)connection).getProperties().getJCSMPSession()
.provision(JCSMPFactory.onlyInstance().createQueue(queueName), props, 0L);
} catch (Exception e) {
e.printStackTrace();
}
queue = session.createQueue(queueName);
}

How to receive message from wildfly jms queue using consumer

I encountered a knotty problem when receiving message from WildFly JMS queue. My code is below:
Session produceSession = connectionFactory.createConnection().createSession(false, Session
.CLIENT_ACKNOWLEDGE);
Session consumerSession = connectionFactory.createConnection().createSession(false, Session
.CLIENT_ACKNOWLEDGE);
ApsSchedule apsSchedule = new ApsSchedule();
boolean success;
MessageProducer messageProducer = produceSession.createProducer(outQueueMaxusOrder);
success = apsSchedule.sendD90Order(produceSession,messageProducer, d90OrderAps);
if (!success) {
logger.error("Can't send APS schedule msg ");
} else {
MessageConsumer consumer = consumerSession.createConsumer(inQueueDeliveryDate);
data = apsSchedule.receiveD90Result(consumerSession,consumer);
}
then getting into the receiveD90Result():
public DeliveryData receiveD90Result(Session session, MessageConsumer consumer) {
DeliveryData data = null;
try {
Message message = consumer.receive(10000);
if (message == null) {
return null;
}
TextMessage msg = (TextMessage) message;
String text = msg.getText();
logger.debug("Receive APS d90 result: {}", text);
ObjectMapper mapper = new ObjectMapper();
data = mapper.readValue(text, DeliveryData.class);
} catch (JMSException je) {
logger.error("Can't receive APS d90 order result: {}", je.getMessage());
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
consumer.close();
} catch (JMSException e) {
e.printStackTrace();
}
}
return data;
}
But when implementing the consumer.receive(10000), the project can't get a message from queue. If I use asynchronous way of MDB to listen the queue, I can get the message from queue. How to resolve it?

There are multiple modes you can choose to get a message from the queue. Message Queues are by default asynchronous in usage. There are however cases when you want to read it synchronously , for example sending a message with account number and using another queue to read the response and match it with a message id or a message correlation id. When you do a receive , the program is waiting for a message to arrive within that polling interval specified in receive.
The code snippet you have , as i see it uses the psuedo synchronous approach. If you have to use it as an MDB , you will have to implement message driven bean (EJB Resource) or message listener.
The way that MDB/Message Listener works is more event based , instead of a poll with a timeout (like the receive) , you implement a callback called onMessage() that is invoked every time there is a message. Instead of a synchronous call , this becomes asynchronous. Your application may require some changes both in terms of design.

I don't see where you're calling javax.jms.Connection.start(). In fact, it doesn't look like you even have a reference to the javax.jms.Connection instance used for your javax.jms.MessageConsumer. If you don't have a reference to the javax.jms.Connection then you can't invoke start() and you can't invoke close() when you're done so you'll be leaking connections.
Furthermore, connections are "heavy" objects and are meant to be re-used. You should create a single connection for both the producer and consumer. Also, if your application is not going to use the javax.jms.Session from multiple threads then you don't need multiple sessions either.

Cannot create a subscriber on the durable subscription

I'm working on a small project for a Systems Integration subject, and I'm using JMS (JBOSS). We have to use durable topics, and that part is quite easy. The thing is, let's say I use the following code:
TopicConnectionFactory topicConnectionFactory = InitialContext.doLookup("jms/RemoteConnectionFactory");
try(JMSContext jmsContext = topicConnectionFactory.createContext(<username>,<password>)) {
Topic topic = InitialContext.doLookup(<topic>);
JMSConsumer jmsConsumer = jmsContext.createDurableConsumer(topic, <client-id>);
Message message = jmsConsumer.receive();
if(message != null) {
result = message.getBody(ArrayList.class);
}
}
This try-with-resources is useful, since it destroys the connection when the block ends. But let's say I interrupt the program while the JMSConsumer waits for the message. When I restart the program, it will throw:
javax.jms.IllegalStateRuntimeException: Cannot create a subscriber on the durable subscription since it already has subscriber(s)
Is there a way to close the connection/unsubscribe/something when the program is interrupted?

If you need to do some cleanup but not swallow the exception, you can catch the exception, do some cleanup, then rethrow the original exception:
try(JMSContext jmsContext = topicConnectionFactory.createContext(<username>,<password>)) {
// ...
} catch (InterruptedException e) {
// Do some cleanup.
throw e;
}
(I'm assuming that it's an InterruptedException, because you said "say I interrupt the program" - but maybe it is some other type: same idea applies)

Basically, I used the following code:
TopicConnectionFactory topicConnectionFactory = InitialContext.doLookup("jms/RemoteConnectionFactory");
try(JMSContext jmsContext = topicConnectionFactory.createContext(<username>,<password>)) {
Topic topic = InitialContext.doLookup(<topic>);
JMSConsumer jmsConsumer = jmsContext.createDurableConsumer(topic, <client-id>);
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
jmsConsumer.close();
this.interrupt();
}
});
Message message = jmsConsumer.receive();
if(message != null) {
result = message.getBody(ArrayList.class);
}
}
I was trying to close the connection using jmsContext.stop(), I think. Anyhow, it wasn't working, now it is. Yay me.

JMS queue receive message?

In the JMS API doc, it said:
public Message receive() throws JMSException
Receives the next message
produced for this message consumer. This call blocks indefinitely
until a message is produced or until this message consumer is closed.
If this receive is done within a transaction, the consumer retains the message until the transaction commits.
Here I have three questions:
1. in the code, do we need while-loop to receive message ? like:
while(true){
Message msg = queue.receive();
....
}
what is the transaction setting ? how to commit a transaction ? like this:
boolean transacted = false;
session = connection.createQueueSession(transacted, Session.AUTO_ACKNOWLEDGE);
receiveNoWait() has transaction support ? how to use it ?
Thanks

If you are going to use receive then you will need some sort of loop to keep receiving messages after the first one is received. Remember that you can also setup a messagelistener and get the received messages async via a callback method and not have to block.
The transaction is generally set to AUTO_ACKNOWLEDGE by default which means that as soon as the message is taken from the queue it is gone and cannot be rolled back. If you want to setup a transaction you need to set the session to transacted and the method to SESSION_TRANSACTED. When you call commit() on the session the messages will be acknowledged on the queue.
receiveNoWait() can have transaction support if you setup the acknowledgement mode correctly and you use commit() and rollback() on the session.
If I were you I would create a MessageListener and not have to worry about spinning a thread to poll the receive methods. Keep in mind that an implicit transaction is started once the session is created.
public class JmsAdapter implements MessageListener, ExceptionListener
{
private ConnectionFactory connFactory = null;
private Connection conn = null;
private Session session = null;
public void receiveMessages()
{
try
{
this.session = this.conn.createSession(true, Session.SESSION_TRANSACTED);
this.conn.setExceptionListener(this);
Destination destination = this.session.createQueue("SOME_QUEUE_NAME");
this.consumer = this.session.createConsumer(destination);
this.consumer.setMessageListener(this);
this.conn.start();
}
catch (JMSException e)
{
//Handle JMS Exceptions Here
}
}
#Override
public void onMessage(Message message)
{
try
{
//Do Message Processing Here
//Message sucessfully processed... Go ahead and commit the transaction.
this.session.commit();
}
catch(SomeApplicationException e)
{
//Message processing failed.
//Do whatever you need to do here for the exception.
//NOTE: You may need to check the redelivery count of this message first
//and just commit it after it fails a predefined number of times (Make sure you
//store it somewhere if you don't want to lose it). This way you're process isn't
//handling the same failed message over and over again.
this.session.rollback()
}
}
}