Our architecture is based on a ActiveMQ 5.10.0 backbone that consists of about 70 queues. Different applications send messages to queues and different applications consumes messages from queues.
In details, only 5 queues have more than one consumer while the remaining have one single consumer per queue.
Everything works fine except for the queues with multiple consumers. For these queues, messages are correctly queued but they are not dequeued untill we access to the ActiveMQ Web portal and click on the queue name thus enlisting the full message list. When we do this, suddendly pending messages are dequeued.
Some additional notes:
the queue only contains TEXT messages
we have 10 consumers registered to that queue. Every consumer defines a proper selector in order to consume only some of the published messages.
every message set a timeout since there are messages that doesn't match any selector rule and we don't want to keep messages in the queue indefinitely.
every consumer defines a connection pool via BiTronix pool. According to what suggested in another thread, for every consumer we set the prefetch to 0
Can someone give us any advice? Why accessing the ActiveMQ Web message list unlock the unqueued messages?
Related
I need to build scalable application that have several Java applications (Spring RabbitMQ producers) that consume messages from other applications by HTTP protocol, calculating priority of message, and send them in to 'priority queue' that matches the params.
So far under load of hundreds of messages per second, one application works just fine, but there is need to scale up applications.
The problem is that I don't really understand how does RabbitMQ producers work with 'priority queues'. I've been searching information in RabbitMQ documentation and I found docs that says that every producer needs to get acks to make sure that messages have proceed successfully.
So the questions are
Docs says that priority of messages calculated under hood AMQP protocol, so do RabbitMQ will send acks to producer after the position for messages will be selected or before
How does messages will be treated if assume that we have 2 producers that produce 2 different messages with same priority to the same 'priority queue'
I will be appreciated for any hint that will help me with that!
I'm using rabbitmqclient for RabbitMQ (from Scala). I subscribe to a queue via DefaultConsumer and consume the messages from few instances concurrently.
The problem is that when the first consumer starts, it immediately takes all existing messages from the queue, so other nodes will consume only newer messages. I'd like to configure the consumers to take, say, not more than 10 messages at a time. It's definitely possible to rewrite it using pull-based API and manage back pressure manually, but I'd like to avoid it.
I am pretty new to RabbitMQ, I want to to consume multiple messages from RabbitMQ so that work can be done parallely also sending acknowledgement only when any of the actor has finished it's task so as not to loose messages. How should I proceed, I want to use spring support for AKKA.
Can I use an actor as a consumer or it should be a plain consumer that can consume multiple messages without sending acknowledgement for any of the message or it should be that I have multiple classes/threads working as consumer instantiated to listen a single message at a time than calling actor (but that would be as if it had no actor or parallelism via AKKA model).
I haven't worked with RabbitMQ per se, but I would probably designate one actor as a dispatcher, that would:
Handle RabbitMQ connection.
Receive messages (doesn't matter if one-by-one or in a batch for efficiency).
Distribute work between worker actors, either by creating a new worker for each message, or by sending messages to a pre-created pool of workers.
Receive confirmation from worker once task is completed and results are committed, and send acknowledgement back to RabbitMQ. Acknowledgment token may be included as a part of worker's task, so no need to track the mappings inside the dispatcher.
Some additional things to consider are:
Supervision strategy: who supervises the dispatcher? Who supervises the workers? What happens if they crash?
Message re-sends when running in a distributed environment.
Iam working on ActiveMQ application where iam using a consumer which uses Session.CLIENT_ACKNOWLEDGE.
Iam sending the messages received from queue in consumer to a webservice.Assume if i don do message.acknowledge() all the messages sent to webservice are back on the queue in enqueued state.
My question is how to retrieve the messages again from the queue and use it.I used retroactive=true and tried redelivery also but all of them are failing.
How to avoid this.
if you use message.acknowledge() all consumed messages are not available again in the same queueu because the are considered as delivered!
can you explain why you need to consume again the messages already consumed.
retroactive is for consumers who was offline and when starting connection to receive messages sent before the connection.
You need to setup the prefetch policy for the consumer to 400 in this case.
You can read to understand the concept http://activemq.apache.org/what-is-the-prefetch-limit-for.html
If you want to treat messages one by one with counter you need to set prefetch to 1 and acknowledge each message when you treat 200 you don't acknowledge.
Weblogic application servers that I am using are clustered. I have a created a JMS queue and it has a JNDI name. When a consumer looks up the jndi name and publishes the event on a queue , would it be published in the queue created in both the app servers? The same MDB will be running on both the servers - which one will get the message posted on to the queue? In case I need to delete the message put on the queue , should I iterate through all the nodes and delete the message?
Thanks.
Using a queue means the message is guaranteed to be consumed exactly once. Meaning, that the message will be delivered to both nodes, but it will only be processed once globally by one of the nodes. WebLogic handles the synchronization and coordination between nodes in your cluster to simultaneously guarantee the delivery, but assure that it is processed exactly once globally.
This is contrast to a topic, where each subscriber gets a copy of the message. Each message will be processed once by each subscriber.
You don't need to iterate through the nodes to delete the message... just grab a jndi reference to the queue and delete the message before any consumer consumes it.
You don't say what type of queue you're creating within Weblogic for this. For a clustered environment it's better to use a DistributedQueue, rather than a standard Queue. I believe it allows Weblogic to better handle how to deal with messages on the queue when one of the nodes in the cluster goes down. There is also the option to view the contents of a queue and delete messages from the Weblogic Admin Console.