I was wondering if someone here could clarify the following for me.
Let's say that I have a Topic, with 5 Durable Subscribers attached to it. Each subscriber was created with the following call (the name is obviously different for every one):
session.createDurableSubscriber(
(Topic) destination,
"NAME_n", // Unique for every subscriber
"PARTY_ID = '123'", // Message selector
true );
So, when I publish a message with PARTY_ID = '123', it will be retained in the Topic until all 5 durable subscribers got a copy, correct?
Now, let's say that we have 3 other durable subscribers, with message selector "PARTY_ID = '666'". When I publish a '666' message to the Topic, will the message be removed from the Topic after the 3 durable '666' subscribers receive the message, or will it keep it there until ALL subscribers receive the message, which in this case will never happen due to selector restrictions?
In other words, does message retention on a Topic take the message selector of the Message and Subscriber into account?
When a message is published, the messaging provider will put a copy of that message to all the subscribers that match the selection criteria specified by the subscribers. In your case a copy of the message will be put to 5 subscribers as they match the selector 'PARTY_ID=123'. Then message is discarded by the messaging provider. Similarly for selector 'PARTY_ID=666', messaging provider will copy the message to matching subscribers only and then discard the message.
Please note the messaging provider will put a copy of the message to the storage (typically a queue) assigned to the subscribers and will go on to process next message from publishers. It will not wait for the subscriber to consume the message from the queue.
Message retention or Retain Publication is another concept where the messaging provider will retain (or cache) a publication for topic until the next publication is received for the same topic. The next publication may be published in the next second or after 5 minutes or 5 hrs. Basically the time of next publication is unpredicted. This concept is quite useful when subscriber comes in after a publication has been maid and the next publication is made after sometime. With Retain publication, the a copy of the retained publication is delivered immediately to the subscriber so that the subscriber does not have to wait long for the next publication.
Let's assume that a publisher is publishing on score of a FIFA2014 ongoing match between The Netherlands and Spain on topic FIFA2014/MATCHUPDATE/NED-ESP. The publisher publishes a new message every time a goal is scored. Since the scoring of goal is unpredictable, messages are published with 'Retain Publication' property which tells the message provider to keep a copy of the message till the next message is published on FIFA2014/MATCHUPDATE/NED-ESP. Assume that there are two subscribers when the first publication was made and with score being 0:0. Now let's assume that a third subscriber comes in after the first publication was made, it will get a publication immediately with score being 0:0. It does not have to wait to know the score till a next goal is scored. See this link.
Hope I am clear.
Related
A case where senders are sending messages to a Queue, for example message1 is sent by sender1 to a queue. Now a consumer named consumer1 connects to queue and reads the message message1.
There is another consumer named consumer2. But the message message1 is already consumed by consumer1 so it will not be available for consumer2.
When a next message arrives in queue, consumer2 might receive that message if it reads the queue before consumer1.
Does it mean that it all is a case whether one consumer reads the queue before the other in order to get the first message available from the queue?
This is the nature of a Queue in JMS, messages are sent to one consumer and once ack'd they are gone, the next consumer can get the next message and so on. This is often referred to as competing consumers or load balancing. The consumers can share the work as jobs or work items are enqueued which allows for higher throughput when the work associated with the items in the Queue can take significant time.
There are options depending on the messaging broker to make a consumer exclusive such that only that consumer can read messages from the queue while the other consumers sit and wait for the exclusive consumer to leave which makes them backups of a sort.
Other options are to use something like Apache Camel to route a given message to more than one queue, or to use AcitveMQ Virtual Topics to send messages to a Topic and have that message then enqueue onto specific consumer Queues.
The solution depends on the broker you are using and the problem you are trying to solve, none of which you've really made clear in the question.
Is there a way (third party software or programming) to monitor the time a message arrive to an specific queue and the time it's consumed?
something like a message arrive at 17:14:22 565 and consumed at 17:14:22 598 or the message was enqueued N miliseconds
I have read about Statistics plugin but it just give max and min times of enqueued messages
You can use http://activemq.apache.org/advisory-message.html
First example below to be notified when a message is delivered to the broker.
Second example to be notified when the message is consumed.
AdvisorySupport.getMessageDeliveredAdvisoryTopic()
AdvisorySupport.getMessageConsumedAdvisoryTopic()
See example here below to have access to Message properties like creation time, in or out time when a messages arrived or left the broker.
Here is the list of properties http://activemq.apache.org/activemq-message-properties.html
Spring JMS Producer and Consumer interaction
One way is to write your own plugin. (http://activemq.apache.org/developing-plugins.html)
It's quite simple, and the effect is similar as you change activemq broker code.
You can extends BrokerFilter class, and override it's methods, like postProcessDispatch(), send(). Then you can record the time, or whatever you want in your own code.
I write a simple example (https://github.com/lcy362/FoxActivemqPlugin/blob/b54d375a6a91a9ec418e779deb69a8b11f7d985a/src/main/java/com/mallow/activemq/FoxBrokerPlugin.java), hoping that's helpful.
Consumer is listening on queue(FIFO or standard queue ),Producer produces the message on queue.
Does Amazon SQS queue deletes the message from queue automatically once it gets acknowledgement from consumer ? Is there a way/configuration where queue keeps the message instead of deleting it and ensures it is not delivered again.
Producer produces the message on queue. Consumer becomes offline because of network issue. After some time he/she get backs to online. will queue deliver the message
to consumer when he gets online ? I think yes as queue has not received ACK from consumer.
I believe you are asking from rabbitmq perspective. There is some difference. There is no ack in sqs. Messages are not automatically deleted, they stay in queue even after a consumer accepts it. The messages need to be explictly deleted by the consumer after it has done processing it.
Sqs does not bother about the online offline status of a consumer. The consumer periodically polls sqs for new items. If a message is available, it is handed out. Once consumer is done, it calls sqs to delete that message. Then again poll for new message.
In your scenario, once the consumer is done processing a message, it can make two requests: one to enqueue the message in a different queue and second to delete the message from original queue.
If you have multiple consumers listning on the same queue, then a concept of message-invisibility-period comes to play. If you have such setup, ask in comments and i will update with more info.
Hope it helps.
I have a dead letter exchange which works as expected - when I nack a message it goes there:
#Override
public void onMessage(Message message, Channel channel) throws Exception {
// How to specify when the message will be put back to the original queue?
// This doesn't work.
message.getMessageProperties().setExpiration("3000");
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, false);
}
But what I cannot find is how to specify per-message when the message should be returned to the original queue. Please advise.
The expiration header is removed when the message is dead lettered. Secondly, messages do not automatically get sent back to your original queue from your deadletter queue.
It sounds like you want a retry system with per message wait times. This can be done but not the way you are doing it now.
The basic pattern is that you Ack your message and then send it to a "delay" exchange and queue. This queue sets it's dead letter exchange as your application's exchange. When a message arrives to the delay queue, it sits there until the message TTL is reached and gets dead lettered to your application exchange. However, this does not work when you have variable message expirations. Messages get dead lettered from the head of the queue so a message with a long expiration will block messages with shorter expirations behind it.
So you have various options to overcome this problem.
Have a few standard delay exchanges to select from which have different queue based TTLs and don't set message expiration.
Just before sending your message, declare a temporary delay exchange (that auto deletes) and queue (with a queue TTL that expires a few seconds after your message expires). Send your message and it will get dead lettered after the correct amount of time and then the exchange will auto delete and the queue will expire and be deleted. This could be very expensive if you have peaks of retries. However, if you name the temporary exchange and queue after the expiry time limit, then it will get reused by other messages with the same TTL being sent within the same period.
NServiceBus has an ingenious technique using multiple delay exchanges and queues: https://docs.particular.net/nservicebus/rabbitmq/delayed-delivery
I want to build a broadcasting system. It consists of several groups. Each group has one User who can broadcast a message to the other members of the same group. What is the appropriate JMS architecture for this kind of system?
Should I use a topic with durable subscription? I don't know much about pub/sub messaging style, can topic have multiple subscription? If so, then each subscription represents a group in the broadcasting system. If not, should I use a queue whereas each message has a header specifying the group and then use a JMS selector to filter the message so that each member receive only messages from groups he's in?
Also, I am thinking of persisting the messages after consumption. I decided to make each message to expire after one hour, and each member should check the database for every message he misses. But, how can I fire an action upon message expiration?
Topics can be used... Topics can have multiple subscribers. Each group can use a different topic. The user can send message to that topic and all the subscribers would receive it.
The durable subscribers are required only if the subscribers can go offline for sometime and the messages for the subscriber shouldn't be lost.
Queue doesn't suite well in one-to-many scenarios. But if you have a pre-defined set of receivers you can use a queue for each one of them and route messages to that. But this is a overhead to route the messages to the receiver's queue. JMS selector idea you have suggested would work but for a queue only one client can receive a message. In topics it distributed to all the clients interested in that topic.
Normally one would persist data to database and not the message itself. So you can persist to database and then create the message for delivery.