I have JMS implementation based on JBoss (to be precise, JBossMQ on JBoss4.2). There are 5 clusters, with each cluster having few nodes. One node in each of the cluster acts as master node. Out of the 5 clusters, one of the cluster is supposed to publish messages to a persistent Topic, and the other 4 clusters consumes those messages. The publishing and consuming is done by only the master node of each cluster.
I want to device a mechanism where the publisher knows that the message was consumed by all the subscribers or a subscriber knows that it has consumed all the messages produced by the publisher. How can this be achieved?
In principle, you use a JMS system in order to not care about that and only configure it the way you need. You could save state information in a shared resource like a database, but I wouldn't do it. Better use monitor features of the JMS system in order to track that. In case your application really needs to know about the successful processing of a message, then you could have a queue where process acknowledge go back to the sender.
For HornetQ, which you might use with JBoss, you'll find an example of a clustered topic here.
Related
I want to setup multiple JMS nodes (brokers) which have multiple topics. Recently I discovered failover feature (http://activemq.apache.org/failover-transport-reference.html#FailoverTransportReference-BrokersideOptionsforFailover) which allows consumers to be distributed among all the broker nodes + redirects in case if target node failed.
I'm new to JMS and to ActiveMQ and perhaps my question would sound stupidly, but anyway:
I wonder if ActiveMQ supports distributed Topics from producer point of view so when producer publishes the message then it appears in a cluster rather than in a single cluster node (to where producer publishes it). The reason why I'm interested in this kind of feature is because I'm afraid that if this single node (where producer publishes message) fails, then producer will not be able to publish messages until this node is up again. But it would be much more reliable if producer could publish a message to a cluster (just as producer uses failover feature) and if the original topic holder node is down, then message is just redirected to other broker nodes. I've been looking for some examples and was unable to find them. Could anybody give a hint if ActiveMQ supports this kind of feature? Thanks
Yes, you combine the failover: scheme to provide client-side recovery and then use the network-of-broker on the server-side to distribute the messages to other consumers in the cluster.
I have a three node rabbitmq cluster.
This is the cluster I am using (not mine) https://github.com/bijukunjummen/docker-rabbitmq-cluster.
I am running into an issue where if I send a large amount of messages to a queue with ha-policy=all, and ungracefully shutdown the server, the messages are not all available on the other nodes.
I think this issue is because the node does not have enough time to propagate the message to the other nodes.
I am seeking a way to know if the client needs to resend the message because the server was terminated before message propagation.
Is this possible in the Java RMQ library?
Thanks.
Take a look into Publisher Confirms https://www.rabbitmq.com/confirms.html
I will be publishing to a single Activemq Topic and I will have many subscribers consuming from this Activemq. Some of my subscribers may connect at a later date, but when they do I want them to receive ALL MESSAGES ever published to that Activemq topic. How do I do this and what is this pub-sub type called where you get a full picture on first subscribe?
It's typically a lot better to create a separate initial load service. New clients connecting and wanting years of missed updates can trigger some sync from the source application and receive these message through some other channel (a queue for instance). Once up to sync, you simply use durable subscribers on your topic to guarantee that you miss no further updates.
ActiveMQ is not really built to store huge amount of data in the middle for long term. Kahadb is not like a regular database (although you can back it with a JDBC data source if you wish). Storing messages long term in MOM software is actually an anti-pattern.
I'm facing a design issue in which I would like to have only one JMS producer sending messages to two consumers. There are only two servers, and the producer will start generating messages that will be load balanced (with round robin) to both consumers.
In the hypothetical case of one server failing, I do have a mechanism so a new producer will be activated in the remaining server. But what will happen to the messages that were being processed in the server that went down?
Will they be reassigned to the remaining server thus being processed by the remaining consumer? or they will be lost?
If the last case is true there will be another problem. The producer creates messages based on files in a NAS so when a server goes down, the newly activated producer will start creating messages based on the contents of the NAS and that may replicate messages (but that case is handled) the problem is that if the server that goes down is not the server with the active producer then when the server goes up again it will not have messages to consume and also no messages will replace the ones lost.
How can I achieve a design so that no messages are lost?
Note: When one server goes down, the journal and bindings are lost.
Once the message is transferred to a particular node it belongs to that node.
If a node goes down, you would have to activate that node with its journal and the message state would be recovered from disk. You could eventually have messages being redistributed if you don't have more consumers (that will depend on redistribution configuration of course).
Or the best approach would be to have a backup node for each node.
We have been advising the use of collocated topologies, where one VM has an active instance and a backup instance for the other Server... That way each alive server would also have a backup config. That's being improved on 2.4.0 as we speak as you need a lot of manual configuration at the moment.
So, in summary either:
Restart the node
configure backup nodes
Weblogic application servers that I am using are clustered. I have a created a JMS queue and it has a JNDI name. When a consumer looks up the jndi name and publishes the event on a queue , would it be published in the queue created in both the app servers? The same MDB will be running on both the servers - which one will get the message posted on to the queue? In case I need to delete the message put on the queue , should I iterate through all the nodes and delete the message?
Thanks.
Using a queue means the message is guaranteed to be consumed exactly once. Meaning, that the message will be delivered to both nodes, but it will only be processed once globally by one of the nodes. WebLogic handles the synchronization and coordination between nodes in your cluster to simultaneously guarantee the delivery, but assure that it is processed exactly once globally.
This is contrast to a topic, where each subscriber gets a copy of the message. Each message will be processed once by each subscriber.
You don't need to iterate through the nodes to delete the message... just grab a jndi reference to the queue and delete the message before any consumer consumes it.
You don't say what type of queue you're creating within Weblogic for this. For a clustered environment it's better to use a DistributedQueue, rather than a standard Queue. I believe it allows Weblogic to better handle how to deal with messages on the queue when one of the nodes in the cluster goes down. There is also the option to view the contents of a queue and delete messages from the Weblogic Admin Console.