I have a requirement that I need to process JMS Messages (via MDB) in a way that Messages belonging to a certain group (a group ID is set) are consumed by the same bean instance. The behaviour I require in this is that Messages with the same group ID are processed sequentially (though message ordering is irrelevant), and tying them to the same MDB-instance should provide that.
The messages do not carry any kind of sequence number (as it is irrelevant) and we do not know what the first or last message in a group is (there could theoratically "never" be a last message in a group). We want them to be delivered as soon as the consumer is able to receive them.
ActiveMQ provides this exact feature (http://activemq.apache.org/message-groups.html) by simply setting JMSXGroupID. We are bound to WebSphere MQ, though. All I've found out so far is that it is possible to collect Messages of the same group in the queue and using a MessageSelector to receive a "Last Message in Group" Message as described in http://www.ibm.com/developerworks/websphere/library/techarticles/0602_currie/0602_currie.html. We would prefer a cleaner way though (like in ActiveMQ). Does anyone know how to achieve that behaviour in WebSphere?
Thanks!
Generally you use MessageSelectors in IBM product implementations of JMS (both WebSphere MQ and SIBus implementations). These are the equivalent of a filter that would scan the header of an HTTP or SOAP message for web-based protocols.
Though it may not be what you want, it actually is a clean and well thought through design.
However, if you do not want to use MessageSelectors, you will probably have to build your own and "process" a message with a fronting MDB that scans the headers, and then forwards the message to an appropriate queue, where only the MDB that cares about the grouped messages will process them (sort of a gateway/message selector pattern).
If you are using the "pure" JMS API's, then you can ask the Session object to create a MessageConsumer with the specified selector string (value in the header) you want to filter on (again you have to set this yourself).
//assume we have created a JMS Connection and Session object already.
//and looked up the Queue we want already.
MessageConsumer consumerWithSelector = session.createConsumer(queue, groupId);
This is all the pure JMS API's give you. Anything else is purely up to the implementer of the messaging technology, which is then proprietary to their implementation and not portable code.
Related
I'm not sure if ActiveMQ is a right tool here...
I have a task queue and multiple consumers, so my idea was to use ActiveMQ to post tasks, which are then consumed by consumers.
But I need to be able to cancel the task, if it was not processed yet...
Is there an API for removing Message from Queue in ActiveMQ?
Destination destination = session.createQueue(TOPIC_NAME);
MessageProducer producer = session.createProducer(destination);
ObjectMessage message = session.createObjectMessage(jobData);
producer.send(message);
...
producer.cancel(message); (?)
The use-case is that, for any reason, performing the task is no longer needed, and the task is resource-consuming.
What about setting an expiry time on the message?
http://activemq.apache.org/how-do-i-set-the-message-expiration.html
If you want a message to be deleted if it has not been processed / consumed in a particular time frame, then message expiry seems the answer to me.
ActiveMQ exposes a JMX interface that allows for operations of this kind. The MBean that models a Queue (e.g., org.apache.activemq:type=broker,brokerName=amq,destinationType=Queue,destinationName=my_queue) exposes a method removeMessage (String id). There are also methods that remove messages that match a particular pattern.
So far as I know, this functionality is not exposed outside JMX.
But...
I have a nasty feeling that JMX operations that work on specific messages only work on messages that are paged into memory. By default that would usually be the 400 messages nearest the head of the queue. I know this is true for selector operations, although I'm not sure about JMX.
Some ActiveMQ message stores (e.g., the JDBC store) might also provide a way to get to the underlying message data and manipulate it. On a relational database this is usually safe to do, because messages that are 'in flight' in a JMS operation will be locked at the database level. However, this is a lot of hassle for what ought to be a simple job.
I wonder if JMS is really the right technology for this job? It isn't really intended for random access. Perhaps some sort of distributed data cache would work better (jgroups, Hazelcast,...)?
For those who are looking for a direct answer, there's a JMS API to control this behaviour:
Per JMS API docs:
setTimeToLive(long timeToLive)
Specifies the time to live of messages that are sent using this JMSProducer.
So you can set this value on the producer before sending:
...
producer.setTimeToLive(30000L);
producer.send();
With this particular setting, messages will be retained for 30 seconds before being deleted by the Message Broker
I am trying to get the number of consumers of a particular Websphere MQ queue from Java? I need to know whether someone is going to consume the messages before placing them on the queue.
First, it is worth noting that the design proposed is a very, VERY bad design. The effect is to turn async messaging back into synchronous messaging. This couples message producers to consumers, introduces location and resolution dependencies, breaks clustering, defeats WMQ's load distribution and balancing, embeds network topology into the application, and makes the whole system brittle. Please do not blame WMQ for not working correctly after intentionally defeating all its best features except the actual queue/dequeue operations.
However, to answer your question more directly, use the getOpenInputCount method of the queue object to obtain the number of open input handles. Here's how:
MQQueue outQ = qMgr.accessQueue(qName,
openOptions,
null, // default q manager
null, // no dynamic q name
null); // no alternate user id
int inCount = outQ.getOpenInputCount();
Note that you can only inquire the input handles on a local queue. If the queue is hosted on a QMgr other than the one where the message sender is connected, this method will not work. Of course it is the normal case that the message sender and receiver would reside on different QMgrs. However since you do not mention much about the design, I'll assume for purposes of this answer that connections from the message producer and consumer attach to the same QMgr. If that's not the case, we need to have a discussion about PCF and even stronger warnings about the design.
I'm trying to move off of ActiveMQ but one feature we'd like to keep is the message group. By adding a session ID to the JMS header ActiveMQ will route all other messages on the queue with the same ID to the same consumer (our consumers may be on different machines) allowing the receiver to treat the group of messages as one unit of work.
My first thought was simply to put the session into CLIENT_ACKNOWLEDGE mode. My thinking was that if consumer A looked at the header and saw it wasn't an ID it was handling then it could just drop the message and consumer B would pick it up. I've hit several issues, including ActiveMQ's prefecting, and the more I read, the more it looks like that's not what that was designed for to begin with.
The one idea I can think of is to have a dispatch queue which would then route messages to each consumer's, for lack of a better word, sub-queue and manage matching the session IDs to the sub-queues ourselves.
Before I head down this path, which we're leery of since it'd add more complexity to the code then we'd like, is there anything I'm missing about CLIENT_ACKNOWLEDGE? Or something else entirely I should try first?
Is this what you are trying to do http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/5/html-single/HornetQ_User_Guide/index.html#message-grouping
I have 2 Message driven beans. 2 Activation Specs for these beans. I have one Message bus and both the activation specs are configured to this one bus. I have 2 different queues and one queue connection factory configured for that one Message bus.
Now, I would write my code to send a message to one of the queues at runtime after determining the queue. However, both my MDBs receive the same message.
How is this configuration done in general? Do I always configure 1 Queue -> 1 Queue Connection Factory -> 1 Message Bus -> 1 MDB? Is it all a one-to-one relationship?
Oh, I forgot to mention this: I am using Websphere Application Server v6.1
In general the concept is that:
a message is sent(Queue)/published(Topic) to a destination (Queue/Topic)
the ActivationSpec listens to messages at a particular destintation (Queue/Topic)
ActivationSpec : Destination is a 1:1 relationship
A bean (MDB which is a consumer) is configured to listen to an ActivationSpec.
What this means is that in effect the bean is linked to a destination with a layer of indirection provided by the activationSpec.
Where does the bus come in - SIBus is the messaging infrastructure that makes all this possible. Destinations are hosted on the bus.
Coming to the question - the ActivationSpec would be configured to listen to a destination on the bus to which messages would be sent. The connection factory decides the bus to which message would be sent. As long as the destination name is unique and targetted to a specific queue (JMS Queue is linked to destination on the bus) one message would only be received by one ActivationSpec.
how many destinations (under SIBus link in WAS admin console) have been created on the bus ? Could you check/validate if the configuration is correct?
to answer your questions - "Is it one bus per activation spec and one queue connection factory per queue." - the answer is NO.
Bus is the underlying infrastructure that can host "n" destinations. One ActivationSpec listens to one destination.
With queue connection factory is a factory (J2EE factory pattern) for creating queues.
I think you're saying you want both MDBs to receive the same message, right?
If this is the case then the MDBs should be listening to a topic not a queue.
Alternatively, there are ways you can configure IBM MQ to forward messages, so for example a message posted to a particular queue could be re-posted to n other queues, but I've only seen that used when some sort of message enrichment takes place before the re-posting and so I suspect would be overkill for what you're trying to achieve.
Why do you need the message bus?
Usually I associate an MDB with a queue - it's a 1:1 relationship. Send a message to a queue, the listener gets it. What is the bus buying you?
I've done JMS with WebLogic, and there's no such construct as a message bus required. I think it's an IBM thing.
Here's an example of how to do JMS with Spring. That's how I'd recommend proceeding.
UPDATE: I misinterpreted your question. When you said both your queues were getting the same message, I didn't think that was the desired behavior. If it is, then topics are the correct way to go. Queues are point-to-point messaging; topics are publish/subscribe.
I suspect the configuration you have is not set up the way you think it is. We use the same configuration your described, with many MDB's (with a queue and activation spec), a single factory and message bus and everything works as expected.
To get the behaviour you are seeing is not actually possible unless you either send the same message to both queues, or have defined a topic instead of a queue. I am pretty sure that even if both MDB's are reading from the same queue, only one will get the message, since a queue only supports point to point messaging. What you have described is topic based behaviour.
Say I load messages in a queue from multiple nodes.
Then, one or many nodes are pulling messages from the queue.
Is it possible (or is this normal usage?) that the queue guarantees to not hand out a message to more than one server/node?
And does that server/node have to tell the queue it has completed the operation and the queue and delete the message?
A message queuing system that did not guarantee to hand out a given message to just one recipient would not be worth the using. Some message queue systems have transactional controls. In that case, if a message is collected by one receiver as part of a transaction, but the receiver does not then commit the transaction (and the message queue can identify that the original recipient is no longer available), then it would be reissued. However, the message would not be made available to two processes concurrently.
What messaging/queuing technology are you using ? AMQP can certainly guarantee this behaviour (amongst many others, including pub/sub models)
If you want this in Java - then a JMS compliant messaging system will do what you want - and most messaging systems have a JMS client. You can Use Spring's JmsTemplate for real ease of use too.
With JMS - a message from a Queue will only be consumed by one and only one client - and once it is consumed (acknowledged) - it will be removed from the messaging system. Also when you publish a message using JMS - if its persistent - it will be sent synchronously, and the send() method won't return until the message is stored on the broker's disk - this is important - if you don't want to run the risk of loosing messages in the event of failure.