I am using Java with ActiveMQ for sending messages from Embedded-Linux. My requirements are that messages should be persisted if the destination broker is not available. I chose KahaDB as the persistence adapter in order for the messages to survive a broker restart.
broker = new SslBrokerService();
KahaDBPersistenceAdapter kahaDBPersistenceAdapter=new KahaDBPersistenceAdapter();
kahaDBPersistenceAdapter.setJournalMaxFileLength(16 * 1024 * 1024);
broker.setPersistenceAdapter(kahaDBPersistenceAdapter);
/** Connector setup code */
broker.start();
Even though I have limited the maximum file size to 16MB i would like to ONLY persist messages if the destination broker/consumer is not available. The reason for this is to limit the CPU and disk usage. For example: Produce message 1, attempt to send message. If successful persistence should not occur since I am not interested in re-sending already sent data ( or should occur only in memory ) . Produce messages 2,3,4 but destination broker is unavailable, then persist messages on disk such that in the unlikely shutdown of borker messages will survive.
After going back and forth through the documentation of ActiveMQ I have not found any answer for my use case so I am asking this great community if what I am looking for is actually possible.
You would have to write a custom plugin to get that exact scenario. JMS does not describe a use case to persist if only the consumer is not available, b/c the reliability contract is also with the producer. I think a store-and-forward design would be what you are looking for. Configure embeddedBroker to use a static: networkConnector to send it to the destination broker. ActiveMQ will automatically push messages up to the destination server when available, and queue them up on the embedded server if the remote broker is unavailable.
Related
I have a situation where I need to read a(on going) messages from a topic and put them on another Queue . I have doubts do I need jms Queue or I can be satisfied with an in memory java Queue . I will do the reading from the Queue by other thread(s) in same jvm and will do client acknowledge of the message to the topic after reading the message from the (in memory) queue and process it as necessary (send it to remote IBM MQ) .So if my client crash the messages that were exist in the in memory queue will be lost but will still exist on topic and will be redeliver to me . Am I right ?
Some of this depends on how you have set up the queue/topic and the connection string you are using to read from IBM's MQ but if you are using the defaults you WILL lose messages if you're reading it to an in-memory queue.
I'd use ActiveMQ, either in the same JVM as a library so you have it taking care of receipt, delivery and persistence.
Also if you are listening to a topic you're not going to be sent missed messages after a crash even if you reconnect afterwards unless you've
configured your client as a durable subscriber
reconnect in the time (before the expireMessagesPeriod is reached)
The ActiveMQ library is not large and worth using if ensure delivery of every message is important, especially in an asynchronous environment.
Main difference is that in-memory loses data when the application goes down; JMS queue loses data when the server goes down IF the topic/queue is not persistent. The former is much more likely than the latter, so I'd also say go with JMS.
I'm using XMS (i.e IBM Message Services Client for .NET) to connect to IBM MQ and get the messages.
With that said, I wanted to know how to get count of messages on a Queue at any given point of time.
(Explored on IQueueBrowser.GetEnumerator, but it will download the messages onto client.)
XMS .NET is an implementation of JMS specification and JMS specific does not define a method or a property that retrieves count of messages in a queue. Hence XMS does not provide a way to do that.
Anything to do with queue attributes is a provider specific administrative job, so you have to use message provider specific APIs.
But why do you want to know the count of messages? Application should be coded to receive messages continuously. Your application can choose to quit receiving messages when there are no more messages in the queue or wait for further messages. If there are no messages, the receive call will return with null message object. Your application can check for this and decide to make further receive calls or quit.
HTH
I understand JMS as depicted by the following diagram:
(source: techhive.com)
Is there any way for me to access the underlying database using JMS or some other thing? Further, the JDBC connections that the JMS server maintains, can I add new connections in it so as to access other databases also and do CRUD operations on them? If yes, how?
Where did you get this from?
Normally JMS is used to send messages to queue (or topics). You have message producers that push messages in the queue and message consumers consume them and process it.
In your exemple it seems that you have multiple queues. One for the messages that need to be processed, and one for each client to retrieve the result the processing of its messages.
With JMS Server you don't necessarily have a database behind. Everything can stay in memory, or can be written to files. You will need database server behind only if you configure your JMS server to be persistent (and to assure that even if server/application crash your messages won't be lost). But in that case you will never have to interact with the database. Only the JMS server will and you will interact with the JMS server sending and consuming messages.
I have JMS implementation based on JBoss (to be precise, JBossMQ on JBoss4.2). There are 5 clusters, with each cluster having few nodes. One node in each of the cluster acts as master node. Out of the 5 clusters, one of the cluster is supposed to publish messages to a persistent Topic, and the other 4 clusters consumes those messages. The publishing and consuming is done by only the master node of each cluster.
I want to device a mechanism where the publisher knows that the message was consumed by all the subscribers or a subscriber knows that it has consumed all the messages produced by the publisher. How can this be achieved?
In principle, you use a JMS system in order to not care about that and only configure it the way you need. You could save state information in a shared resource like a database, but I wouldn't do it. Better use monitor features of the JMS system in order to track that. In case your application really needs to know about the successful processing of a message, then you could have a queue where process acknowledge go back to the sender.
For HornetQ, which you might use with JBoss, you'll find an example of a clustered topic here.
I'm facing a design issue in which I would like to have only one JMS producer sending messages to two consumers. There are only two servers, and the producer will start generating messages that will be load balanced (with round robin) to both consumers.
In the hypothetical case of one server failing, I do have a mechanism so a new producer will be activated in the remaining server. But what will happen to the messages that were being processed in the server that went down?
Will they be reassigned to the remaining server thus being processed by the remaining consumer? or they will be lost?
If the last case is true there will be another problem. The producer creates messages based on files in a NAS so when a server goes down, the newly activated producer will start creating messages based on the contents of the NAS and that may replicate messages (but that case is handled) the problem is that if the server that goes down is not the server with the active producer then when the server goes up again it will not have messages to consume and also no messages will replace the ones lost.
How can I achieve a design so that no messages are lost?
Note: When one server goes down, the journal and bindings are lost.
Once the message is transferred to a particular node it belongs to that node.
If a node goes down, you would have to activate that node with its journal and the message state would be recovered from disk. You could eventually have messages being redistributed if you don't have more consumers (that will depend on redistribution configuration of course).
Or the best approach would be to have a backup node for each node.
We have been advising the use of collocated topologies, where one VM has an active instance and a backup instance for the other Server... That way each alive server would also have a backup config. That's being improved on 2.4.0 as we speak as you need a lot of manual configuration at the moment.
So, in summary either:
Restart the node
configure backup nodes