We have a producer service which is publishing a message to topic and we have 3 instances of consumers reading that message from topic. Producer needs to do further processing (update DB) only when all consumers processed and replied a success message in another reply queue. Even if one of the consumer service failed due to some exception or any failure message in reply queue, producer should not do update DB for that publish request.
For next request (new message published in topic), if all consumers processed and replied in reply queue, producer should update DB. How can we achieve this with camel and activemq. Is there any EIP pattern(s) to achieve this?
Any
Yes, the producer can just use the aggregator enterprise integration pattern on the reply queue to ensure all 3 consumers processed the previous message.
Doc: Apache Camel aggregator2 component.
You will need a unique key and use a proper aggregation strategy.
Related
I have a queue with one producer and two consumers using CLIENT_ACKNOWLEDGE. The two consumers share the connections, but they live in different threads so each uses its own session.
What happens if consumer A does not acknowledge the last message received? In which scenario the message will be re-delivered and consumed by consumer B? Is it when the session consumer A is using is closed? Do I need some configuration on JMS provider to dictate what happens in such a scenario?
I don't think this situation is directly addressed in the JMS specification so the answer will ultimately depend on what JMS broker you're using. However, having worked on multiple JMS brokers in the past I would say that generally speaking any message that has been dispatched by the broker to a consumer but which hasn't yet been acknowledged by that consumer will be cancelled back to the broker and will be made available for redelivery once the consumer in question is closed.
I have recently started working on Azure Service Bus module which is currently used in our project. The current model is that, we send message to a topic and the topic has multiple subscriptions. There is no filter for subscriptions and the subscribers consume it(Currently not looking to add filters).
Question: Let's say there is 1 topic and 3 subscriptions. The message sent to topic is broadcasted to all subscriptions and applications from two subscriptions consume it. Application for the third subscription goes down and the message is not consumed.
What happens to the message from the 3rd subscription, will it be sent to Dead Letter Queue after TTL
Is there a way to find out, for which subscription the message wasn't consumed
What happens to the message from the 3rd subscription, will it be sent to Dead Letter Queue after TTL
If subscription is configured to dead-letter expired messages, yes, that's what will happen.
Is there a way to find out, for which subscription the message wasn't consumed
You shouldn't do that. Topics and subscriptions are intended to implement Pub/Sub pattern and is about decoupling the senders from the receivers.
I have 2 JMS queues where my application can publish the message to any one of the queue based on the node to which the request is being received. Pentaho should actively look to both the queues and should be able to process as soon as the message arrives in any one of both the queue.
Currently, I have implemented a job to actively listen to one queue and process the message and post a response for the same.
How do we configure pentaho to actively listen to two queues at the same time and perform the same action when any of the queue is posted with a message?
EDIT I am not aware of any such direct feature available in Pentaho for such intra service communication.
Will Clustering help this cause?
Finally cracked it
Job1
Start (Run next entries in Parallel) -> Transformation1 (JMS Consumer1)
|-> Transformation2 (JMS Consumer2)
With the task and message prioritisation logic put on the application side.
If I understand JMS correctly, the consumer sends acknowledgement to the broker, and the broker understands that the consumer accepted the message. But how does the producer make sure that broker accepted the message? I don't see any acknowledgements in the API.
Does the method send sync or async?
Acknowledgement is a consumer side concept. Acknowledgement is the way
that a consumer informs the JMS provider that it has successfully
received a message. On the producer side, the only notion of
acknowledgement consists of a successful invocation of either the
topic publishe’s publish method or the queue sender’s send method. If
an acknowledgement is given for a message it indicates that the JMS
provider must not deliver that same message to the consumer in
question again and also the JMS provider can release any resources it
is holding on behalf of the said message (i.e if a JMS queue is
considered after acknowledgement of successful delivery, that message
is removed from the queue). In order to minimize resource consumption,
consumer applications need to acknowledge messages as quickly as
possible after successful delivery.
source: http://wso2.com/library/articles/2013/01/jms-message-delivery-reliability-acknowledgement-patterns/
I am using seda queue as dead letter channel endpoint. It works fine if network is down or other application is down.
What will happen if I restart my own system?
Will I loose my messages in dead letter channel endpoint SEDA queue?
The seda endpoint is not a reliable message solution, meaning any messages sent to a seda destination are subject to loss in the event of a failure/restart. If JMS is not an available solution you will need to provide your own persistence logic to provide message recovery.
Additionally, given that seda endpoints are asynchronous, you must ensure that a pattern of message acknowledgement is used post persistence of the DLQ message so your producer to the DLQ will be notified of success or failure to ensure reliability.
This of course when using a DLQ. You could also use a persistence preprocessor that would store messages meant for delivery and only delete them in the case of a failure to deliver them.