We Keep Coding

Multiple queues receiving same message from virtual topic creates a deadletter entry for one queue only

I'm am using Virtual Destinations to implement Publish Subscribe model in ActiveMQ 5.15.13.
I have a virtual topic VirtualTopic and there are two queues bound to it. Each queue has its own redelivery policy. Let's say Queue 1 will retry message 2 times in case there is an exception while processing the message and Queue 2 will retry message 3 times. Post retry message will be sent to deadletter queue. I'm also using Individual Dead letter Queue strategy so that each queue has it's own deadletter queue.
I've observed that when a message is sent to VirtualTopic, the message with same message id is delivered to both the queues. I'm facing an issue where if the consumers of both queues are not able to process the message successfully. The message destined for Queue 1 is moved to deadletter queue after retrying for 2 times. But there is no deadletter queue for Queue 2, though message in Queue 2 is retried for 3 times.
Is it the expected behavior?
Code:
public class ActiveMQRedelivery {
private final ActiveMQConnectionFactory factory;
public ActiveMQRedelivery(String brokerUrl) {
factory = new ActiveMQConnectionFactory(brokerUrl);
factory.setUserName("admin");
factory.setPassword("password");
factory.setAlwaysSyncSend(false);
}
public void publish(String topicAddress, String message) {
final String topicName = "VirtualTopic." + topicAddress;
try {
final Connection producerConnection = factory.createConnection();
producerConnection.start();
final Session producerSession = producerConnection.createSession(false, AUTO_ACKNOWLEDGE);
final MessageProducer producer = producerSession.createProducer(null);
final TextMessage textMessage = producerSession.createTextMessage(message);
final Topic topic = producerSession.createTopic(topicName);
producer.send(topic, textMessage, PERSISTENT, DEFAULT_PRIORITY, DEFAULT_TIME_TO_LIVE);
} catch (JMSException e) {
throw new RuntimeException("Message could not be published", e);
}
}
public void initializeConsumer(String queueName, String topicAddress, int numOfRetry) throws JMSException {
factory.getRedeliveryPolicyMap().put(new ActiveMQQueue("*." + queueName + ".>"),
getRedeliveryPolicy(numOfRetry));
Connection connection = factory.createConnection();
connection.start();
final Session consumerSession = connection.createSession(false, CLIENT_ACKNOWLEDGE);
final Queue queue = consumerSession.createQueue("Consumer." + queueName +
".VirtualTopic." + topicAddress);
final MessageConsumer consumer = consumerSession.createConsumer(queue);
consumer.setMessageListener(message -> {
try {
System.out.println("in listener --- " + ((ActiveMQDestination)message.getJMSDestination()).getPhysicalName());
consumerSession.recover();
} catch (JMSException e) {
e.printStackTrace();
}
});
}
private RedeliveryPolicy getRedeliveryPolicy(int numOfRetry) {
final RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(0);
redeliveryPolicy.setMaximumRedeliveries(numOfRetry);
redeliveryPolicy.setMaximumRedeliveryDelay(-1);
redeliveryPolicy.setRedeliveryDelay(0);
return redeliveryPolicy;
}
}
Test:
public class ActiveMQRedeliveryTest {
private static final String brokerUrl = "tcp://0.0.0.0:61616";
private ActiveMQRedelivery activeMQRedelivery;
#Before
public void setUp() throws Exception {
activeMQRedelivery = new ActiveMQRedelivery(brokerUrl);
}
#Test
public void testMessageRedeliveries() throws Exception {
String topicAddress = "testTopic";
activeMQRedelivery.initializeConsumer("queue1", topicAddress, 2);
activeMQRedelivery.initializeConsumer("queue2", topicAddress, 3);
activeMQRedelivery.publish(topicAddress, "TestMessage");
Thread.sleep(3000);
}
#After
public void tearDown() throws Exception {
}
}

I recently came across this problem. To fix this there are 2 attributes that needs to be added to individualDeadLetterStrategy as below
<deadLetterStrategy>
<individualDeadLetterStrategy destinationPerDurableSubscriber="true" enableAudit="false" queuePrefix="DLQ." useQueueForQueueMessages="true"/>
</deadLetterStrategy>
Explanation of attributes:
destinationPerDurableSubscriber - To enable a separate destination per durable subscriber.
enableAudit - The dead letter strategy has a message audit that is enabled by default. This prevents duplicate messages from being added to the configured DLQ. When the attribute is enabled, the same message that isn't delivered for multiple subscribers to a topic will only be placed on one of the subscriber DLQs when the destinationPerDurableSubscriber attribute is set to true i.e. say two consumers fail to acknowledge the same message for the topic, that message will only be placed on the DLQ for one consumer and not the other.

How does the MessageListener running in JMS? Can it load messages in the queue in real time?

I want to load the JMS messages in a queue in real time, if is a consumer.setMessageListener(myListener) viable and how it works?
I'm not sure how a listener working. I wrote a listener like below.
public class JmsMessageListenerExample {
public static void main(String[] args) throws URISyntaxException, Exception {
BrokerService broker = BrokerFactory.createBroker(new URI(
"broker:(tcp://localhost:61616)"));
broker.start();
Connection connection = null;
try {
// Producer
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(
"tcp://localhost:61616");
connection = connectionFactory.createConnection();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("customerQueue");
String payload = "Important Task";
Message msg = session.createTextMessage(payload);
MessageProducer producer = session.createProducer(queue);
System.out.println("Sending text '" + payload + "'");
producer.send(msg);
// Consumer
MessageConsumer consumer = session.createConsumer(queue);
consumer.setMessageListener(new ConsumerMessageListener("Consumer"));
connection.start();
} finally {
if (connection != null) {
connection.close();
}
broker.stop();
}
}
After I run the JmsMessageListenerExample, it completed quickly and I didn't received any messages. If a listener should keep on running until I stop it?

When you set a JMS message listener it will receive messages asynchronously in its own thread (invoked by the JMS implementation). In your case you need to prevent main from exiting and stopping your application because when that happens the MessageListener will be terminated.
Also, when you say "load the JMS messages in a queue in real time" I assume you mean "consume JMS messages from a queue as soon as possible when queue receives them." If that's the case then a JMS message listener is the right approach.

How to start ActiveMQ when tomcat starts?

How do I configure my J2EE application so that I can invoke ActiveMQ service along with tomcat server? I am aware about embedded broker, here asking how to start the ActiveMQ whenever I start tomcat
Current Code (works fine) :
Now I want to remove main() method and use the code to run when tomcat runs.
public class JMSService {
public void produceJMS() throws NamingException, JMSException {
ConnectionFactory connFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection conn = connFactory.createConnection();
conn.start();
Session session = conn.createSession(false,Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("testQueue");
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
TextMessage message = session.createTextMessage("Test Message ");
// send the message
producer.send(message);
System.out.println("sent: " + message);
}}
Here is my consumer :
public class JMSReceiver implements MessageListener,ExceptionListener {
public static void main(String args[]) throws Exception {
JMSReceiver re = new JMSReceiver();
re.receiveJMS();
}
public void receiveJMS() throws NamingException, JMSException {
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
// Getting the queue 'testQueue'
Destination destination = session.createQueue("testQueue");
MessageConsumer consumer = session.createConsumer(destination);
// set an asynchronous message listener
JMSReceiver asyncReceiver = new JMSReceiver();
consumer.setMessageListener(asyncReceiver);
connection.setExceptionListener(asyncReceiver);
}
#Override
public void onMessage(Message message) {
System.out.println("Received message : " +message);
}
}

What #Tim Bish said is correct. You either need to have a timer say for example receiver should listen for 1 hour- or make it available until program terminate. Either case you need to start your consumer program once:
Change your receiveJMS method as follows:
public void receiveJMS() throws NamingException, JMSException {
try{
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection connection = connectionFactory.createConnection();
connection.start(); // it's the start point
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
// Getting the queue 'testQueue'
Destination destination = session.createQueue("testQueue");
MessageConsumer consumer = session.createConsumer(destination);
// set an asynchronous message listener
// JMSReceiver asyncReceiver = new JMSReceiver();
//no need to create another object
consumer.setMessageListener(this);
connection.setExceptionListener(this);
// connection.close(); once this is closed consumer no longer active
Thread.sleep(60 *60 * 1000); // receive messages for 1 hour
}finally{
connection.close();// after 1 hour close it
}
}
The above program will listen upto 1 hour. If you want it as long as the program run, remove the finally block. But the recommended way is to close it somehow. since your application seems to be standalone ,you can check the java runtime shutdown hook, where you can specify how to release such resources while program terminates.
If your consumer is a web application you can close it in a ServletContextlistner.

You aren't giving the consumer application any time to actually receive a message, you create it, then you close it. You either need to use a timed receive call to do an sync receive of the message from the Queue or you need to add some sort of wait in the main method such as a CountDownLatch etc to allow the async onMessage call to trigger shutdown once processing of the message is complete.

Unable to consume messages from an embedded remote broker in ActiveMQ

I am an ActiveMQ begginer. My main looks like this:
public static void main(String[] args) throws Exception {
BrokerService broker = new BrokerService();
if(isProducer(args)){
broker.addConnector("tcp://localhost:8001");
broker.start();
// start producer...
}
else{
broker.addConnector("tcp://localhost:9000");
broker.addNetworkConnector("static:(tcp://localhost:8001)");
broker.start(); // Getting stuck here!!!
// start consumer
}
waitForever();
}
I start this application twice, once a a producer and once as a consumer.
When I start the consumer, it gets stuck on the broker.start() line.
What am I missing?!

Basicly you start the broker once (embedding it into a jvm).
BrokerService broker = new BrokerService();
broker.setUseJmx(true);
broker.addConnector("tcp://localhost:61616");
broker.start();
Then you connect to the broker (this code is needed in both consumer and producer application):
url = "vm://localhost:61616" //if you are in same jvm
url2 = "tcp://localhost:61616" //if you are in diff jvm or other host
connectionFactory = new ActiveMQConnectionFactory(username,password,url);
connection = (ActiveMQConnection) connectionFactory.createConnection();
connection.start();
session = connection.createSession(transacted, ackMode);
Then setup a consumer
Destination queue = session.createQueue("queuename");
MessageConsumer consumer = session.createConsumer(queue);
consumer.setMessageListener(new MessageConsumer());
Setup a producer and send a message
MessageProducer producer = session.createProducer(queue);
ObjectMessage objectMessage = session.createObjectMessage();
objectMessage.setObject(object);
producer.send(objectMessage);
look at for example: http://jmsexample.zcage.com/index2.html
or http://svn.apache.org/repos/asf/activemq/branches/activemq-5.6/assembly/src/release/example/src/

ACTIVEMQ- publisher subscriber hello world example

There are two programs: subscriber and publisher...
Subscriber is able to put the message onto the topic and the message is sent successfully.
When I check the activemq server on my browser it shows 1 msg enqueued . But when I run the consumer code, it is not receiving the message
Here is the producer code:
import javax.jms.*;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
public class producer {
private static String url = ActiveMQConnection.DEFAULT_BROKER_URL;
public static void main(String[] args) throws JMSException {
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
// JMS messages are sent and received using a Session. We will
// create here a non-transactional session object. If you want
// to use transactions you should set the first parameter to 'true'
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Topic topic = session.createTopic("testt");
MessageProducer producer = session.createProducer(topic);
// We will send a small text message saying 'Hello'
TextMessage message = session.createTextMessage();
message.setText("HELLO JMS WORLD");
// Here we are sending the message!
producer.send(message);
System.out.println("Sent message '" + message.getText() + "'");
connection.close();
}
}
After I run this code the output at the console is:
26 Jan, 2012 2:30:04 PM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://localhost:61616
Sent message 'HELLO JMS WORLD'
And here is the consumer code:
import javax.jms.*;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
public class consumer {
// URL of the JMS server
private static String url = ActiveMQConnection.DEFAULT_BROKER_URL;
// Name of the topic from which we will receive messages from = " testt"
public static void main(String[] args) throws JMSException {
// Getting JMS connection from the server
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Topic topic = session.createTopic("testt");
MessageConsumer consumer = session.createConsumer(topic);
MessageListener listner = new MessageListener() {
public void onMessage(Message message) {
try {
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
System.out.println("Received message"
+ textMessage.getText() + "'");
}
} catch (JMSException e) {
System.out.println("Caught:" + e);
e.printStackTrace();
}
}
};
consumer.setMessageListener(listner);
connection.close();
}
}
After I run this code it doesnt show anything.
Can someone help to me to overcome this problem?

Your issue is that your consumer is running and then shutting down immediately.
Try adding this into your consumer:
consumer.setMessageListener(listner);
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}
connection.close();
This will wait until you hit a key before stopping.
Other things to consider:
Use a finally block for the close
Java naming conventions encourage using uppercase for the first letter of a class

The main problem (besides the app closing down to quickly) is that you are sending to a Topic. Topics don't retain messages so if you run your application that produces and then run the consumer, the consumer won't receive anything because it was not subscribed to the topic at the time the message was sent. If you fix the shutdown issue and then run the consumer in one terminal and then run the producer you should then see the message received by your consumer. If you want message retention then you need to use a Queue which will hold onto the message until someone consumes it.

Your producer class is correct. It runs smoothly.
But, your consumer is incorrect & you have to modify it.
First, add setClientID("any_string_value") after creating connection object;
eg: Connection connection = connectionFactory.createConnection();
// need to setClientID value, any string value you wish
connection.setClientID("12345");
secondly, use createDurableSubscriber() method instead of createConsumer() for transmitting message via topic.
MessageConsumer consumer = session.createDurableSubscriber(topic,"SUB1234");
Here is the modified comsumer class:
package mq.test;
import javax.jms.*;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
public class consumer {
// URL of the JMS server
private static String url = ActiveMQConnection.DEFAULT_BROKER_URL;
// Name of the topic from which we will receive messages from = " testt"
public static void main(String[] args) throws JMSException {
// Getting JMS connection from the server
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
// need to setClientID value, any string value you wish
connection.setClientID("12345");
try{
connection.start();
}catch(Exception e){
System.err.println("NOT CONNECTED!!!");
}
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Topic topic = session.createTopic("test_data");
//need to use createDurableSubscriber() method instead of createConsumer() for topic
// MessageConsumer consumer = session.createConsumer(topic);
MessageConsumer consumer = session.createDurableSubscriber(topic,
"SUB1234");
MessageListener listner = new MessageListener() {
public void onMessage(Message message) {
try {
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
System.out.println("Received message"
+ textMessage.getText() + "'");
}
} catch (JMSException e) {
System.out.println("Caught:" + e);
e.printStackTrace();
}
}
};
consumer.setMessageListener(listner);
//connection.close();
}
}
Now, your code will run successfully.

just some:
work with a queue not a topic. messages in topics will be discarded when no consumer is available, they are NOT persistend.
add connection.start() after setting the message listener. you should start a connection when all consumers/producers are properly set up.
wait some time before before closing the connection again.
the topic will probably be your most important source of failure.