Consumer is listening on queue(FIFO or standard queue ),Producer produces the message on queue.
Does Amazon SQS queue deletes the message from queue automatically once it gets acknowledgement from consumer ? Is there a way/configuration where queue keeps the message instead of deleting it and ensures it is not delivered again.
Producer produces the message on queue. Consumer becomes offline because of network issue. After some time he/she get backs to online. will queue deliver the message
to consumer when he gets online ? I think yes as queue has not received ACK from consumer.
I believe you are asking from rabbitmq perspective. There is some difference. There is no ack in sqs. Messages are not automatically deleted, they stay in queue even after a consumer accepts it. The messages need to be explictly deleted by the consumer after it has done processing it.
Sqs does not bother about the online offline status of a consumer. The consumer periodically polls sqs for new items. If a message is available, it is handed out. Once consumer is done, it calls sqs to delete that message. Then again poll for new message.
In your scenario, once the consumer is done processing a message, it can make two requests: one to enqueue the message in a different queue and second to delete the message from original queue.
If you have multiple consumers listning on the same queue, then a concept of message-invisibility-period comes to play. If you have such setup, ask in comments and i will update with more info.
Hope it helps.
Related
I have a consumer process which has to do heavy lifting, it is a process that takes more than 30 sec to complete. And we acknowledge the message when process is completed successfully. However, looks like queue is waiting for acknowledgement and as it does not receive the acknowledgement within time, it's putting the message back to queue and same message is consumed by other consumer instance. Are there any config that I can tweak?? I don't want to auto acknowledge the message as it's an important flow and autoscaling down of the cluster may cause message loss.
I am looking if there is any config that can help me with it or is my understanding incorrect? I don't want same message getting consumed by more than one consumer. We're using IBM MQ in this instance.
However, looks like queue is waiting for acknowledgement and as it
does not receive the acknowledgement within time, it's putting the
message back to queue and same message is consumed by other consumer
instance.
Neither the queue nor the queue manager by itself puts the message back to the queue. There is one exception to that rule and that is if the client application crashes. If the queue manager determines that the application has crashed then it will rollback the message to the queue.
Or are you saying that if the sending application does not receive an acknowledgement within a specified amount of time then it resends the same message? If that is the case, then tell the sending application to double or triple the wait time.
I have some code after the consumer consumes, right now since the message queue is asynchronous the flow will continue even if the message is not consumed by the consumer. I want to wait till the consumer receives the message and till then the flow needs to be paused. Is there any way to do it? I'm using RabbitMQ Java implementation and I'm using topic exchange.
In your case you can make your Queue a Lazy queue so that message will be stored on disk until the consumer comes back and picks your messages.
https://www.rabbitmq.com/lazy-queues.html
I have 2 applications A and B, trying to send messages from both to one queue.
Placed a while loop at both places which is sending message to queue.
if i start application A and start while loop it starts sending message to queue and consumer consumes message sent from A, now at same time if i start while loop from B application it doesn't publish messages to queue as consumer doesn't consumes any message sent from B.
So can someone clear the doubt if messages are being sent at same time from multiple producers to single queue or not.
PS- using IBM queue and using a single consumer.
Yes, we can have multiple producers for single queue.
Multiple producers can also publish messages at the same time.
I want to write a CommunicationSupervisor for connection to a device with these features.
Outgoing messages should be queued in a data structure.
There is just one Consumer. Consumer should check if there is an element in queue.
Consumer should send the outgoing message and wait for an incoming response.
It is possible to produce new outgoing messages while consumer
is waiting for a responce but they will not be handled until consumer
is getting a response for first element, then second and so on
If consumer is getting a timeout for a message an exception should be raised but consumer should proceed with the other outgoing messages in the queue.
My question is what is the best method to implement this? Shall I use a SynchronousQueue as a data structure? Or do we have a pattern example for this? By the way We are using Java 6. So solution for java 8 is not going to work.
Thanks for the help
A case where senders are sending messages to a Queue, for example message1 is sent by sender1 to a queue. Now a consumer named consumer1 connects to queue and reads the message message1.
There is another consumer named consumer2. But the message message1 is already consumed by consumer1 so it will not be available for consumer2.
When a next message arrives in queue, consumer2 might receive that message if it reads the queue before consumer1.
Does it mean that it all is a case whether one consumer reads the queue before the other in order to get the first message available from the queue?
This is the nature of a Queue in JMS, messages are sent to one consumer and once ack'd they are gone, the next consumer can get the next message and so on. This is often referred to as competing consumers or load balancing. The consumers can share the work as jobs or work items are enqueued which allows for higher throughput when the work associated with the items in the Queue can take significant time.
There are options depending on the messaging broker to make a consumer exclusive such that only that consumer can read messages from the queue while the other consumers sit and wait for the exclusive consumer to leave which makes them backups of a sort.
Other options are to use something like Apache Camel to route a given message to more than one queue, or to use AcitveMQ Virtual Topics to send messages to a Topic and have that message then enqueue onto specific consumer Queues.
The solution depends on the broker you are using and the problem you are trying to solve, none of which you've really made clear in the question.