Problem:
Somehow producer is sending event to "ActiveMQ.Advisory.Producer.Queue.Queue" instead of "Queue"
Active-MQ admin console in Topics section Screenshot with producer-queue: (Not sure why it has queue and 0 consumers and number of message enqueued = 38)
Active-MQ admin console in Queues section Screenshot with consumer-queue: (it shows consumers = 1 but number of message enqueued = 0)
Attaching Producer, Consumer and Config code.
Producer
public void sendMessage(WorkflowRun message){
var queue = "Queue";
try{
log.info("Attempting Send message to queue: "+ queue);
jmsTemplate.convertAndSend(queue, message);
} catch(Exception e){
log.error("Recieved Exception during send Message: ", e);
}
}
Listener
#JmsListener(destination = "Queue")
public void messageListener(SystemMessage systemMessage) {
LOGGER.info("Message received! {}", systemMessage);
}
Config
#Value("${spring.active-mq.broker-url}")
private String brokerUrl;
#Bean
public ConnectionFactory connectionFactory() throws JMSException {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
activeMQConnectionFactory.setWatchTopicAdvisories(false);
activeMQConnectionFactory.createQueueConnection(ActiveMQConnectionFactory.DEFAULT_USER,
ActiveMQConnectionFactory.DEFAULT_PASSWORD);
return activeMQConnectionFactory;
}
When your producer starts, the ActiveMQ broker produces an 'Advisory Message' and sends it to that topic. The count indicates how many producers have been created for the queue://Queuee-- in this case 38 producers have been created.
Since the message is not being produced, it appears that in your Spring wiring, you have the connection, session and producer objects being created-- but the messages are not being sent.
Additionally, if you are showing queue://ActiveMQ.Advisory.. showing up you probably have a bug in some other part of the app (or monitoring tool?) that should be configured to consume from topic://ActiveMQ.Advisory.. instead of queue://
Related
I'm am using Virtual Destinations to implement Publish Subscribe model in ActiveMQ 5.15.13.
I have a virtual topic VirtualTopic and there are two queues bound to it. Each queue has its own redelivery policy. Let's say Queue 1 will retry message 2 times in case there is an exception while processing the message and Queue 2 will retry message 3 times. Post retry message will be sent to deadletter queue. I'm also using Individual Dead letter Queue strategy so that each queue has it's own deadletter queue.
I've observed that when a message is sent to VirtualTopic, the message with same message id is delivered to both the queues. I'm facing an issue where if the consumers of both queues are not able to process the message successfully. The message destined for Queue 1 is moved to deadletter queue after retrying for 2 times. But there is no deadletter queue for Queue 2, though message in Queue 2 is retried for 3 times.
Is it the expected behavior?
Code:
public class ActiveMQRedelivery {
private final ActiveMQConnectionFactory factory;
public ActiveMQRedelivery(String brokerUrl) {
factory = new ActiveMQConnectionFactory(brokerUrl);
factory.setUserName("admin");
factory.setPassword("password");
factory.setAlwaysSyncSend(false);
}
public void publish(String topicAddress, String message) {
final String topicName = "VirtualTopic." + topicAddress;
try {
final Connection producerConnection = factory.createConnection();
producerConnection.start();
final Session producerSession = producerConnection.createSession(false, AUTO_ACKNOWLEDGE);
final MessageProducer producer = producerSession.createProducer(null);
final TextMessage textMessage = producerSession.createTextMessage(message);
final Topic topic = producerSession.createTopic(topicName);
producer.send(topic, textMessage, PERSISTENT, DEFAULT_PRIORITY, DEFAULT_TIME_TO_LIVE);
} catch (JMSException e) {
throw new RuntimeException("Message could not be published", e);
}
}
public void initializeConsumer(String queueName, String topicAddress, int numOfRetry) throws JMSException {
factory.getRedeliveryPolicyMap().put(new ActiveMQQueue("*." + queueName + ".>"),
getRedeliveryPolicy(numOfRetry));
Connection connection = factory.createConnection();
connection.start();
final Session consumerSession = connection.createSession(false, CLIENT_ACKNOWLEDGE);
final Queue queue = consumerSession.createQueue("Consumer." + queueName +
".VirtualTopic." + topicAddress);
final MessageConsumer consumer = consumerSession.createConsumer(queue);
consumer.setMessageListener(message -> {
try {
System.out.println("in listener --- " + ((ActiveMQDestination)message.getJMSDestination()).getPhysicalName());
consumerSession.recover();
} catch (JMSException e) {
e.printStackTrace();
}
});
}
private RedeliveryPolicy getRedeliveryPolicy(int numOfRetry) {
final RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(0);
redeliveryPolicy.setMaximumRedeliveries(numOfRetry);
redeliveryPolicy.setMaximumRedeliveryDelay(-1);
redeliveryPolicy.setRedeliveryDelay(0);
return redeliveryPolicy;
}
}
Test:
public class ActiveMQRedeliveryTest {
private static final String brokerUrl = "tcp://0.0.0.0:61616";
private ActiveMQRedelivery activeMQRedelivery;
#Before
public void setUp() throws Exception {
activeMQRedelivery = new ActiveMQRedelivery(brokerUrl);
}
#Test
public void testMessageRedeliveries() throws Exception {
String topicAddress = "testTopic";
activeMQRedelivery.initializeConsumer("queue1", topicAddress, 2);
activeMQRedelivery.initializeConsumer("queue2", topicAddress, 3);
activeMQRedelivery.publish(topicAddress, "TestMessage");
Thread.sleep(3000);
}
#After
public void tearDown() throws Exception {
}
}
I recently came across this problem. To fix this there are 2 attributes that needs to be added to individualDeadLetterStrategy as below
<deadLetterStrategy>
<individualDeadLetterStrategy destinationPerDurableSubscriber="true" enableAudit="false" queuePrefix="DLQ." useQueueForQueueMessages="true"/>
</deadLetterStrategy>
Explanation of attributes:
destinationPerDurableSubscriber - To enable a separate destination per durable subscriber.
enableAudit - The dead letter strategy has a message audit that is enabled by default. This prevents duplicate messages from being added to the configured DLQ. When the attribute is enabled, the same message that isn't delivered for multiple subscribers to a topic will only be placed on one of the subscriber DLQs when the destinationPerDurableSubscriber attribute is set to true i.e. say two consumers fail to acknowledge the same message for the topic, that message will only be placed on the DLQ for one consumer and not the other.
For an application I am doing some tests with Spring Boot and RabbitMQ.
I set up a very simple Sender - Receiver application:
Sender:
public class Tut1Sender
{
private final Gson gson = new Gson();
#Autowired
private RabbitTemplate template;
#Autowired
private Queue queue;
public static int count = 1;
#Scheduled(fixedDelay = 1000, initialDelay = 500)
public void send() throws InterruptedException
{
String message = "Hello World! "+" Nr. "+count;
MessageObject mo = new MessageObject(message);
String toJson = gson.toJson(mo);
this.template.convertAndSend(queue.getName(), toJson);
System.out.println(" [x] Sent '" + toJson + "'");
Thread.sleep(5);
count++;
}
}
This part works just fine and fill my queue with messages.
Here is my receiver:
#RabbitListener(queues = "hello")
public class Tut1Receiver
{
private final Gson gson = new Gson();
#RabbitHandler
public void receive(String in) throws InterruptedException
{
System.out.println("Received Raw: " + in);
MessageObject fromJson = gson.fromJson(in, MessageObject.class);
System.out.println("Received Message '" + fromJson + "'");
int nextInt = ThreadLocalRandom.current().nextInt(1000, 5000);
System.out.println("Sleep for " + nextInt + " ms");
Thread.sleep(nextInt);
}
}
Messages created by the Sender are handled correctly by the receiver. I get a nice output, the message is acknowledged and deleted from the queue.
Then I put a message directly into the queue by the Web-GUI of RabbitMQ.
The sender grabs this message. I can say this because the message created by me switched from status "Ready" to "Unacked" (as displayed in Web-GUI)
The sender gave me no output.
Then I configured the ContainerFactory:
#Profile("receiver")
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(
SimpleRabbitListenerContainerFactoryConfigurer configurer,
ConnectionFactory connectionFactory)
{
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setErrorHandler(e -> {
System.out.println("Error: "+e);
System.out.println("Raw: "+((ListenerExecutionFailedException) e).getFailedMessage().toString());
});
return factory;
}
Now I am getting the following error (in an endless loop)
Error: org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method 'no match' threw exception
Raw: (Body:'[B#53452feb(byte[11])' MessageProperties [headers={content_type=text/plain, content_encoding=UTF-8}, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=true, receivedExchange=, receivedRoutingKey=hello, deliveryTag=1, consumerTag=NOT_SET, consumerQueue=hello])
How can I handle this error? The sender should just display the error, acknowledging the message and proceed with the next message.
What is the right way to handle faulty messages in general?
For broken message, consumers can reject or deliver the message. If you are sure the broken message can't be processed by any other consumers, you should tell the broker to discard the message or deliver it to a dead-letter-exchange.
From official document of spring amqp, I find:
Another alternative is to set the container's rejectRequeued property to false. This causes all failed messages to be discarded. When using RabbitMQ 2.8.x or higher, this also facilitates delivering the message to a Dead Letter Exchange.
Or, you can throw a AmqpRejectAndDontRequeueException; this prevents message requeuing, regardless of the setting of the rejectRequeued property.
I want to load the JMS messages in a queue in real time, if is a consumer.setMessageListener(myListener) viable and how it works?
I'm not sure how a listener working. I wrote a listener like below.
public class JmsMessageListenerExample {
public static void main(String[] args) throws URISyntaxException, Exception {
BrokerService broker = BrokerFactory.createBroker(new URI(
"broker:(tcp://localhost:61616)"));
broker.start();
Connection connection = null;
try {
// Producer
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(
"tcp://localhost:61616");
connection = connectionFactory.createConnection();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("customerQueue");
String payload = "Important Task";
Message msg = session.createTextMessage(payload);
MessageProducer producer = session.createProducer(queue);
System.out.println("Sending text '" + payload + "'");
producer.send(msg);
// Consumer
MessageConsumer consumer = session.createConsumer(queue);
consumer.setMessageListener(new ConsumerMessageListener("Consumer"));
connection.start();
} finally {
if (connection != null) {
connection.close();
}
broker.stop();
}
}
After I run the JmsMessageListenerExample, it completed quickly and I didn't received any messages. If a listener should keep on running until I stop it?
When you set a JMS message listener it will receive messages asynchronously in its own thread (invoked by the JMS implementation). In your case you need to prevent main from exiting and stopping your application because when that happens the MessageListener will be terminated.
Also, when you say "load the JMS messages in a queue in real time" I assume you mean "consume JMS messages from a queue as soon as possible when queue receives them." If that's the case then a JMS message listener is the right approach.
I've used the subscriber example from the google documentation for Google PubSub
the only modification I've made is commenting out the acknowledgement of the messages.
The subscriber doesn't add messages to the queue anymore while messages should be resent according to the interval set in the google cloud console.
Why is this happening or am I missing something?
public class SubscriberExample {
use the default project id
private static final String PROJECT_ID = ServiceOptions.getDefaultProjectId();
private static final BlockingQueue<PubsubMessage> messages = new LinkedBlockingDeque<>();
static class MessageReceiverExample implements MessageReceiver {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
messages.offer(message);
//consumer.ack();
}
}
/** Receive messages over a subscription. */
public static void main(String[] args) throws Exception {
// set subscriber id, eg. my-sub
String subscriptionId = args[0];
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(
PROJECT_ID, subscriptionId);
Subscriber subscriber = null;
try {
// create a subscriber bound to the asynchronous message receiver
subscriber = Subscriber.newBuilder(subscriptionName, new MessageReceiverExample()).build();
subscriber.startAsync().awaitRunning();
// Continue to listen to messages
while (true) {
PubsubMessage message = messages.take();
System.out.println("Message Id: " + message.getMessageId());
System.out.println("Data: " + message.getData().toStringUtf8());
}
} finally {
if (subscriber != null) {
subscriber.stopAsync();
}
}
}
}
When you do not acknowledge a messages, the Java client library calls modifyAckDeadline on the message until maxAckExtensionPeriod passes. By default, this value is one hour. Therefore, if you don't ack/nack the message or change this value, it is likely the message will not be redelivered for an hour. If you want to change the max ack extension period, set it on the builder:
subscriber = Subscriber.newBuilder(subscriptionName, new MessageReceiverExample())
.setMaxAckExtensionPeriod(Duration.ofSeconds(60))
.build();
It is also worth noting that when you don't ack or nack messages, then flow control may prevent the delivery of more messages. By default, the Java client library allows up to 1000 messages to be outstanding, i.e., waiting for ack or nack or for the max ack extension period to pass.
I have ActiveMQ running on the localhost using the default port, 61616. I put some messages in a Topic called topic.STUDENTS. I have a client program that I am trying to use to read the messages in the Topic. Here is the main part of the program that does not work as expected. I was expecting to get some of the message back but instead I just receive null.
public void receiveMessages() throws URISyntaxException, JMSException, Exception {
try {
//Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
//Create a Connection
Connection connection = connectionFactory.createConnection();
connection.start();
//Create a session
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
//Create the destination (Topic or Queue)
Destination destination = session.createTopic("topic.STUDENTS");
//Create a MessageConsumer from the Session to the Topic or Queue
MessageConsumer consumer = session.createConsumer(destination);
//Wait for a message
Message message = consumer.receive(1000);
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
String text = textMessage.getText();
System.out.println("Received: " + text);
} else {
System.out.println("Received: " + message);
}
consumer.close();
session.close();
connection.close();
} catch(Exception e) {
System.out.println("Caught: " + e);
e.printStackTrace();
}
}
This code basically apes the code from ActiveMQ 'Hello World' example: http://activemq.apache.org/hello-world.html
Does anyone know why I do not receive any messages from the Topic? Here is one message that is in the Topic that I can see through the ./activemq browse topic.STUDENTS command:
JMS_BODY_FIELD:JMSText = Invention-my-dear-friends-is-93%-perspiration-6%-electricity
JMS_HEADER_FIELD:JMSExpiration = 0
JMS_HEADER_FIELD:JMSMessageID = ID:user-thinkpad-335i-59919-1457977678059-1:1:1:1:3
JMS_HEADER_FIELD:JMSPriority = 4
JMS_HEADER_FIELD:JMSDestination = topic.STUDENTS
JMS_HEADER_FIELD:JMSTimestamp = 1457977678409
JMS_HEADER_FIELD:JMSRedelivered = false
JMS_HEADER_FIELD:JMSDeliveryMode = persistent
Topics do not retain messages by default, so if you start the consumer after you've produced the messages then any message that was sent before the consumer arrived is gone. If you want to consume messages that were produced while your consumer was offline then you need to use a Queue or register a durable Topic subscriber on the Topic prior to producing the messages.