I want to setup multiple JMS nodes (brokers) which have multiple topics. Recently I discovered failover feature (http://activemq.apache.org/failover-transport-reference.html#FailoverTransportReference-BrokersideOptionsforFailover) which allows consumers to be distributed among all the broker nodes + redirects in case if target node failed.
I'm new to JMS and to ActiveMQ and perhaps my question would sound stupidly, but anyway:
I wonder if ActiveMQ supports distributed Topics from producer point of view so when producer publishes the message then it appears in a cluster rather than in a single cluster node (to where producer publishes it). The reason why I'm interested in this kind of feature is because I'm afraid that if this single node (where producer publishes message) fails, then producer will not be able to publish messages until this node is up again. But it would be much more reliable if producer could publish a message to a cluster (just as producer uses failover feature) and if the original topic holder node is down, then message is just redirected to other broker nodes. I've been looking for some examples and was unable to find them. Could anybody give a hint if ActiveMQ supports this kind of feature? Thanks
Yes, you combine the failover: scheme to provide client-side recovery and then use the network-of-broker on the server-side to distribute the messages to other consumers in the cluster.
Related
I have a three node rabbitmq cluster.
This is the cluster I am using (not mine) https://github.com/bijukunjummen/docker-rabbitmq-cluster.
I am running into an issue where if I send a large amount of messages to a queue with ha-policy=all, and ungracefully shutdown the server, the messages are not all available on the other nodes.
I think this issue is because the node does not have enough time to propagate the message to the other nodes.
I am seeking a way to know if the client needs to resend the message because the server was terminated before message propagation.
Is this possible in the Java RMQ library?
Thanks.
Take a look into Publisher Confirms https://www.rabbitmq.com/confirms.html
What happens behind the scene, when receiving messages with (spring or ejb) message listener container in ActiveMQ/HornetQ?
Does broker pushing messages to consumers? If so, how consumers register
themselves to broker?
Or consumers polling messages on the queue? If so, why each queue (in admin console) has a consumer-number field that shows number of registered consumers of the queue?
This link of O'Reilly book said:
The p2p messaging model has traditionally been a pull- or
polling-based model, where messages are requested from the queue
instead of being pushed to the client automatically. (The JMS
specification does not specifically state how the p2p and pub/sub
models must be implemented. Either one may use push or pull, but at
least conceptually pub/sub is push and p2p is pull).
You are not stating the protocol, since ActiveMQ and HornetQ are multi protocol brokers the exact implementation may vary a bit. However, most protocols except HTTP/REST based ones pushes messages to the client. It's not possible to achive high throughput without a push strategy on the wire protocol level.
The application level API allows for "polling", i.e. JMS MessageConsumer.receive, but that's really just a "sleep until a message is pushed" mechanism.
I have JMS implementation based on JBoss (to be precise, JBossMQ on JBoss4.2). There are 5 clusters, with each cluster having few nodes. One node in each of the cluster acts as master node. Out of the 5 clusters, one of the cluster is supposed to publish messages to a persistent Topic, and the other 4 clusters consumes those messages. The publishing and consuming is done by only the master node of each cluster.
I want to device a mechanism where the publisher knows that the message was consumed by all the subscribers or a subscriber knows that it has consumed all the messages produced by the publisher. How can this be achieved?
In principle, you use a JMS system in order to not care about that and only configure it the way you need. You could save state information in a shared resource like a database, but I wouldn't do it. Better use monitor features of the JMS system in order to track that. In case your application really needs to know about the successful processing of a message, then you could have a queue where process acknowledge go back to the sender.
For HornetQ, which you might use with JBoss, you'll find an example of a clustered topic here.
There is a cluster of tomcats, each tomcat node generates "tasks" which can be performed by any other node. I'd prefer task to be performed by the node which created it.
I thought that it would be good idea to use an embedded broker for each tomcat and configure it as a store-and-forward network. The problem is that a node can go down and the tasks/messages should then be performed by other tomcat instead of waiting for current one to get up.
On the other hand - when using master/slave cluster how to prioritize the node which sent the message?
How to configure it in activemq?
The priority of a local consumer should be default. In AMQ Docs:
ActiveMQ uses Consumer Priority so that local JMS consumers are always
higher priority than remote brokers in a store and forward network.
However, you will not really achive what you want. If one tomcat node goes down, so will the embedded ActiveMQ (and any messages still attached to that instance). A message will not automatically get copied to all other brokers.
But you ask something about Master/slave cluster. Do you intend to have a network of brokers OR a master/slave setup? Or do you intend to have a combo?
I have a JMS topic on an ActiveMQ network of brokers cluster (aka distributed topic). I have an external JMS consumer (Weblogic portal) that needs to subscribe to this topic and get all the messages sent to it (across all brokers).
If the consumer subscribes to the topic on one of the brokers, it will only get the subset of the messages that the broker receives, correct?
I guess I could create a consumer for each broker and aggregate the messages together, but then I'm also on the hook for dealing with connection issues and needing to know which brokers are available, etc.
Question, is there a way to configure the network of brokers or consumer to get all the messages from a distributed JMS topic?
If the consumer subscribes to the
topic on one of the brokers, it will
only get the subset of the messages
that the broker receives, correct?
Technically, yes, but the broker network is responsible for knowing which consumers are interested in which messages, and making sure that the right brokers get the right messages.
Normally, this means that every broker gets every message, but if a broker only has consumers with a given message selector, it will only get messages that those clients are interested in.
In practise, this means you pick a broker, connect to it, and let the broker network sort it out amongst themselves. In theory.
You just connect to the cluster. It is up to the cluster to deliver the messages to the consumer.