We Keep Coding

Remove "ActiveMQ.Advisory.Producer.x" prefix

Problem:
Somehow producer is sending event to "ActiveMQ.Advisory.Producer.Queue.Queue" instead of "Queue"
Active-MQ admin console in Topics section Screenshot with producer-queue: (Not sure why it has queue and 0 consumers and number of message enqueued = 38)
Active-MQ admin console in Queues section Screenshot with consumer-queue: (it shows consumers = 1 but number of message enqueued = 0)
Attaching Producer, Consumer and Config code.
Producer
public void sendMessage(WorkflowRun message){
var queue = "Queue";
try{
log.info("Attempting Send message to queue: "+ queue);
jmsTemplate.convertAndSend(queue, message);
} catch(Exception e){
log.error("Recieved Exception during send Message: ", e);
}
}
Listener
#JmsListener(destination = "Queue")
public void messageListener(SystemMessage systemMessage) {
LOGGER.info("Message received! {}", systemMessage);
}
Config
#Value("${spring.active-mq.broker-url}")
private String brokerUrl;
#Bean
public ConnectionFactory connectionFactory() throws JMSException {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
activeMQConnectionFactory.setWatchTopicAdvisories(false);
activeMQConnectionFactory.createQueueConnection(ActiveMQConnectionFactory.DEFAULT_USER,
ActiveMQConnectionFactory.DEFAULT_PASSWORD);
return activeMQConnectionFactory;
}

When your producer starts, the ActiveMQ broker produces an 'Advisory Message' and sends it to that topic. The count indicates how many producers have been created for the queue://Queuee-- in this case 38 producers have been created.
Since the message is not being produced, it appears that in your Spring wiring, you have the connection, session and producer objects being created-- but the messages are not being sent.
Additionally, if you are showing queue://ActiveMQ.Advisory.. showing up you probably have a bug in some other part of the app (or monitoring tool?) that should be configured to consume from topic://ActiveMQ.Advisory.. instead of queue://

Multiple queues receiving same message from virtual topic creates a deadletter entry for one queue only

I'm am using Virtual Destinations to implement Publish Subscribe model in ActiveMQ 5.15.13.
I have a virtual topic VirtualTopic and there are two queues bound to it. Each queue has its own redelivery policy. Let's say Queue 1 will retry message 2 times in case there is an exception while processing the message and Queue 2 will retry message 3 times. Post retry message will be sent to deadletter queue. I'm also using Individual Dead letter Queue strategy so that each queue has it's own deadletter queue.
I've observed that when a message is sent to VirtualTopic, the message with same message id is delivered to both the queues. I'm facing an issue where if the consumers of both queues are not able to process the message successfully. The message destined for Queue 1 is moved to deadletter queue after retrying for 2 times. But there is no deadletter queue for Queue 2, though message in Queue 2 is retried for 3 times.
Is it the expected behavior?
Code:
public class ActiveMQRedelivery {
private final ActiveMQConnectionFactory factory;
public ActiveMQRedelivery(String brokerUrl) {
factory = new ActiveMQConnectionFactory(brokerUrl);
factory.setUserName("admin");
factory.setPassword("password");
factory.setAlwaysSyncSend(false);
}
public void publish(String topicAddress, String message) {
final String topicName = "VirtualTopic." + topicAddress;
try {
final Connection producerConnection = factory.createConnection();
producerConnection.start();
final Session producerSession = producerConnection.createSession(false, AUTO_ACKNOWLEDGE);
final MessageProducer producer = producerSession.createProducer(null);
final TextMessage textMessage = producerSession.createTextMessage(message);
final Topic topic = producerSession.createTopic(topicName);
producer.send(topic, textMessage, PERSISTENT, DEFAULT_PRIORITY, DEFAULT_TIME_TO_LIVE);
} catch (JMSException e) {
throw new RuntimeException("Message could not be published", e);
}
}
public void initializeConsumer(String queueName, String topicAddress, int numOfRetry) throws JMSException {
factory.getRedeliveryPolicyMap().put(new ActiveMQQueue("*." + queueName + ".>"),
getRedeliveryPolicy(numOfRetry));
Connection connection = factory.createConnection();
connection.start();
final Session consumerSession = connection.createSession(false, CLIENT_ACKNOWLEDGE);
final Queue queue = consumerSession.createQueue("Consumer." + queueName +
".VirtualTopic." + topicAddress);
final MessageConsumer consumer = consumerSession.createConsumer(queue);
consumer.setMessageListener(message -> {
try {
System.out.println("in listener --- " + ((ActiveMQDestination)message.getJMSDestination()).getPhysicalName());
consumerSession.recover();
} catch (JMSException e) {
e.printStackTrace();
}
});
}
private RedeliveryPolicy getRedeliveryPolicy(int numOfRetry) {
final RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(0);
redeliveryPolicy.setMaximumRedeliveries(numOfRetry);
redeliveryPolicy.setMaximumRedeliveryDelay(-1);
redeliveryPolicy.setRedeliveryDelay(0);
return redeliveryPolicy;
}
}
Test:
public class ActiveMQRedeliveryTest {
private static final String brokerUrl = "tcp://0.0.0.0:61616";
private ActiveMQRedelivery activeMQRedelivery;
#Before
public void setUp() throws Exception {
activeMQRedelivery = new ActiveMQRedelivery(brokerUrl);
}
#Test
public void testMessageRedeliveries() throws Exception {
String topicAddress = "testTopic";
activeMQRedelivery.initializeConsumer("queue1", topicAddress, 2);
activeMQRedelivery.initializeConsumer("queue2", topicAddress, 3);
activeMQRedelivery.publish(topicAddress, "TestMessage");
Thread.sleep(3000);
}
#After
public void tearDown() throws Exception {
}
}

I recently came across this problem. To fix this there are 2 attributes that needs to be added to individualDeadLetterStrategy as below
<deadLetterStrategy>
<individualDeadLetterStrategy destinationPerDurableSubscriber="true" enableAudit="false" queuePrefix="DLQ." useQueueForQueueMessages="true"/>
</deadLetterStrategy>
Explanation of attributes:
destinationPerDurableSubscriber - To enable a separate destination per durable subscriber.
enableAudit - The dead letter strategy has a message audit that is enabled by default. This prevents duplicate messages from being added to the configured DLQ. When the attribute is enabled, the same message that isn't delivered for multiple subscribers to a topic will only be placed on one of the subscriber DLQs when the destinationPerDurableSubscriber attribute is set to true i.e. say two consumers fail to acknowledge the same message for the topic, that message will only be placed on the DLQ for one consumer and not the other.

How to identify the MQTT topic that received the message?

The client is subscribed to a x / # topic. There is the possibility of receiving message in the topics x / start and x / stop, and depending on the topic, it performs an action. I wonder how I can identify if it's coming up in the start or stop topic.
In the current code, I send an "action" key in the JSON: "start" or "stop". I want to delete this key and use the format that said above, identifying the topic.
Any further information they deem necessary, please request that I edit the post!
JDK 8
The code:
private MqttCallback callback = new MqttCallback() {
public void connectionLost(Throwable throwable) {
try {
connect();
} catch (MqttException e) {
e.printStackTrace();
}
}
public void messageArrived(String s, MqttMessage mqttMessage) throws Exception {
String messageReceived = new String(mqttMessage.getPayload());
actionPerformed(messageReceived);
}
public void deliveryComplete(IMqttDeliveryToken iMqttDeliveryToken) {
}
};
private void actionPerformed(String message) throws IOException {
ClientDTO clientDTO = new ObjectMapper().readValue(message, ClientDTO.class);
if (clientDTO.getAction().equalsIgnoreCase("start")) {
startView(clientDTO);
} else if (clientDTO.getAction().equalsIgnoreCase("stop")) {
stopView();
}
}
public void connect() throws MqttException {
MqttConnectOptions options = new MqttConnectOptions();
options.setUserName("a_nice_username");
options.setPassword("a_cool_password".toCharArray());
options.setAutomaticReconnect(true);
MqttClient client = new MqttClient("someaddress", MqttClient.generateClientId());
client.setCallback(callback);
try {
client.connect(options);
client.subscribe(topic);
TaskbarIcon.alteraIconeOnline();
} catch (Exception e) {
TaskbarIcon.alteraIconeOffline();
}
}
public void tipoConexao(int tipoConex) throws IOException {
switch (tipoConex) {
case 0:
topic += "/operador/" + getIdReceived() + "/#";
System.out.println(topic);
break;
//etc
}

The s in this method is the topic: public void messageArrived(String s, MqttMessage mqttMessage)
As is very well documented here:
messageArrived
void messageArrived(java.lang.String topic, MqttMessage message) throws java.lang.Exception
This method is called when a message arrives from the server.
This method is invoked synchronously by the MQTT client. An acknowledgment is not sent back to the server until this method
returns cleanly.
If an implementation of this method throws an Exception, then the client will be shut down. When the client is next re-connected, any
QoS 1 or 2 messages will be redelivered by the server.
Any additional messages which arrive while an implementation of this method is running, will build up in memory, and will then back up
on the network.
If an application needs to persist data, then it should ensure the data is persisted prior to returning from this method, as after
returning from this method, the message is considered to have been
delivered, and will not be reproducible.
It is possible to send a new message within an implementation of this callback (for example, a response to this message), but the
implementation must not disconnect the client, as it will be
impossible to send an acknowledgment for the message being processed,
and a deadlock will occur.
Parameters:
topic - name of the topic on the message was published to
message - the actual message.
Throws:
java.lang.Exception - if a terminal error has occurred, and the client should be shut down.

Intercepting Spring Cloud Stream Messages from Consumer only

I am currently using Spring Cloud Stream with Kafka binders with a GlobalChannelInterceptor to perform message-logging for my Spring Boot microservices.
I have:
a producer to publish messages to a SubscribableChannel
a consumer to listen from the Stream (using the #StreamListener annotation)
Throughout the process when a message is published to the Stream from the producer and listened by the consumer, it is observed that the preSend method was triggered twice:
Once at producer side - when the message is published to the Stream
Once at consumer side - when the message is listened from the Stream
However, for my logging purposes, I only need to intercept and log the message at consumer side.
Is there any way to intercept the SCS message ONLY at one side (e.g. consumer side)?
I would appreciate any thoughts on this matter. Thank you!
Ref:
GlobalChannelInterceptor documentation - https://docs.spring.io/spring-integration/api/org/springframework/integration/config/GlobalChannelInterceptor.html
EDIT
Producer
public void sendToPushStream(PushStreamMessage message) {
try {
boolean results = streamChannel.pushStream().send(MessageBuilder.withPayload(new ObjectMapper().writeValueAsString(message)).build());
log.info("Push stream message {} sent to {}.", results ? "successfully" : "not", StreamChannel.PUSH_STREAM);
} catch (JsonProcessingException ex) {
log.error("Unable to parse push stream message.", ex);
}
}
Producer's streamChannel
public interface StreamChannel {
String PUSH_STREAM = "PushStream";
#Output(StreamChannel.PUSH_STREAM)
SubscribableChannel pushStream();
}
Consumer
#StreamListener(StreamChannel.PUSH_STREAM)
public void handle(Message<PushStreamMessage> message) {
log.info("Incoming stream message from {}, {}", streamChannel.pushStream(), message);
}
Consumer's streamChannel
public interface StreamChannel {
String PUSH_STREAM = "PushStream";
#Input(StreamChannel.PUSH_STREAM)
SubscribableChannel pushStream();
}
Interceptor (Common Library)
public class GlobalStreamInterceptor extends ChannelInterceptorAdapter {
#Override
public Message<?> preSend(Message<?> msg, MessageChannel mc) {
log.info("presend " + msg);
return msg;
}
#Override
public void postSend(Message<?> msg, MessageChannel mc, boolean sent) {
log.info("postSend " + msg);
}
}

Right, why don't follow GlobalChannelInterceptor options and don't apply
An array of simple patterns against which channel names will be matched.
?
So, you may have something like this:
#GlobalChannelInterceptor(patterns = Processor.INPUT)
Or use a custom name of input channel to your SCSt app.

Java Producer, Stompy Python Consumer, ActiveMQ

I have a java ActiveMQ producer which produces Integer messages into an ObjectMessage instance.
On the python side, I use stomp python for listening on the queue. However, I receive empty message body although all the headers are received right.
Moreover, if I change the message type to TextMessage on the java side, I get correct message on python-consumer side.
I have also tried with PyactiveMQ but with the same effect
Any suggestions will be appreciated!!!
EDIT: Here is a boilerplate java producer code and python subscriber code which i wrote to test stomp on python
public class App
{
Connection conn;
Session session;
MessageProducer producer;
public void registerPublisher(String queueName, String url) throws JMSException {
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory("system", "manager" ,url);
conn = cf.createConnection();
conn.start();
session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue(queueName);
producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
}
public void send(int c) {
for (int i=0; i<c; ++i) {
try {
TextMessage tm = session.createTextMessage(new Integer(i).toString());
// ObjectMessage tm = session.createObjectMessage();
producer.send(tm);
} catch (JMSException e) {
e.printStackTrace();
}
}
}
public static void main(String []arg) {
App app = new App();
try {
app.registerPublisher(arg[0], arg[1]);
System.out.println(app.session);
} catch (JMSException e) {
e.printStackTrace();
}
app.send(1000);
}
}
And Python Stomp listener
import time
import sys
import logging
import stomp
from stomp import ConnectionListener
queuename = sys.argv[1]
logging.basicConfig( level=logging.DEBUG)
class MyListener(ConnectionListener):
def on_error(self, headers, message):
print 'received an error %s' % message
def onMessage(self, headers, message):
print headers
print str(message)
print type(message)
print 'received a message ...%s...' % message
conn = stomp.Connection([('localhost', 61613)])
conn.set_listener('', MyListener())
conn.start()
conn.connect()
conn.subscribe(destination='/queue/'+queuename, ack='auto')
while 1:
time.sleep(2)

In order to send an receive ObjectMessage types over Stomp you need to use the message transformation feature of ActiveMQ to have the object payload delivered in a form a STOMP client can understand. ActiveMQ provides XML and JSON transformation support out of the box however you can add your own transformer to get whatever format you want on the content.

PROBLEM: Sending ObjectMessage from a java producer to ActiveMQ Broker. Stomp Python consumer client was getting empty message body
SOLUTION: Use transformation header while subscribing to the activemq broker in the python client,
for example:
connection.subscribe(destination='/queue/'+queuename, ack='auto', transformation="jms-json")
So that the broker knows in what form the message is to be sent to the stomp client