We have an application that synchronously reads 4MB messages from an IBM Websphere queue (version 7.5) using JMS. Under certain circumstances, I want to discard messages from the queue without reading them. I am trying to figure out if there is a way to do this programatically without reading the entire 4MB message, which takes several seconds (there could be hundreds of messages that need to be discarded). In the absense of a discard() method (or similar), here is what I have tried:
BytesMessage msg = (BytesMessage)queueReceiver.receiveNoWait();
bytesRead = msg.readBytes(msgBytes, 1024); // just read 1024 bytes
queueReceiver.close();
The above code is no quicker than retrieving the entire 4MB message from the queue (by reading into a larger buffer). This leads me to believe that the receiveNoWait() call is downloading the entire message into an internal buffer before the readBytes() call is made. The only other information I can provide is that the queue is set to "auto acknowledge" when the session is started:
queueSession = queueConnection.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
If I were to change this to CLIENT_ACKNOWLEDGE and acknowlege the message using msg.acknowledge(), would that have the desired effect? Or is there something I am missing?
Thanks for any help,
Doug
There is no other way, a message must be consumed to take remove it from a queue.
Changing to CLIENT_ACKNOWLEDGE from AUTO_ACKNOWLEDGE will not make any difference as the acknowledge is way to tell the messaging provider to remove a message from queue. The AUTO_ACKNOWLEDGE option tells the JMS client to automatically send a confirmation to provider to remove a message whereas CLIENT_ACKNOWLEDGE is used by the application to explicitly tell the provider to remove message(s).
You could probably take a look at setting an expiry time on messages that you don't plan to consume. Messages with an expiry time set, will not be available for delivery after the expiry time is over. Read through JMSExpiration property of a message.
Given this some more thought and there is potentially one other way here; MQ has the concept of PCF messages - simply this is being able to send an administrative command as a message to a queue manager.
JMS can send these messages - so one open would be to send a CLEAR_QUEUE command when you know you don't want any more messages.
It's quite a broad approach - clearing the entire queue but it depends on what your criteria are for removing messages.
I can see the use case however for selectively removing messages - maybe worth raising a 'request for enhancement' RFE on the IBM developerWorks site?
As far as I know, JMS cannot read part of a message. You can only do that with C or Java (non-JMS).
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_NO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING;
MQMessage getMsg = new MQMessage();
try
{
/* get the message with only 1 byte of message data */
_inQ.get(getMsg, gmo, 1);
}
catch (MQException e)
{
System.err.println(e.getLocalizedMessage() );
}
Related
We're using Oracle JMS APIs to read messages from Advanced Queue. We use the following piece of code to read the messages from the queue:
MessageConsumer consumer = sess.createConsumer(q);
for (Message m; (m = consumer.receive()) != null;)
{
new Timer().schedule(new QueueExample(m), 0);
}
The problem is, after the message is received from the queue, it is not completely removed from the queue table, only the STATE field is changed from 0 to 2. Is this the default behavior of the Oracle JMS Client? We would like to completely remove the record from the queue table, after the message has been read from the queue with consumer.receive() method. What is the appropriate api method to do that ?
I think you are experiencing this due to the retention_time parameter on your queue being configured to some high value.
Retention is used for:
Users can specify that messages be retained after consumption. The
systems administrator can specify the duration for which messages will
be retained. Oracle AQ stores information about the history of each
message, preserving the queue and message properties of delay,
expiration, and retention for messages destined for local or remote
recipients. The information contains the ENQUEUE/DEQUEUE time and the
identification of the transaction that executed each request. This
allows users to keep a history of relevant messages. The history can
be used for tracking, data warehouse and data mining operations.
You can verify this by checking the queue-creation script and alter the setting via admin interface or using ALTER_QUEUE.
I'm trying to build a custom mq exit to archive messages that hit a queue. I have the following code.
class MyMqExits implements WMQSendExit, WMQReceiveExit{
#Override
public ByteBuffer channelReceiveExit(MQCXP arg0, MQCD arg1, ByteBuffer arg2) {
// TODO Auto-generated method stub
if ( arg2){
def _bytes = arg2.array()
def results = new String(_bytes)
println results;
}
return arg2;
}
...
The content of the message (header/body) is in the byte buffer, along with some unreadable binary information. How can I parse the message (including the body and the queue name) from arg2? We've gone through IBM's documentation, but haven't found an object or anything that makes this easy.
Assuming the following two points:
1) Your sender application has not hard coded the queue name where it puts messages. So you can change the application configuration to send messages to a different object.
2) MessageId of the archived message is not important, only message body is important.
Then one alternative I can think of is to create an Alias queue that resolves to a Topic and use two subscribers to receive messages.
1) Subscriber 1: An administratively defined durable subscriber with a queue provided to receive messages. Provide the same queue name from which your existing consumer application is receiving messages.
2) Subscriber 2: Another administratively defined durable subscriber with queue provided. You can write a simple java application to get messages from this queue and archive.
3) Both subscribers subscribe to the same topic.
Here are steps:
// Create a topic
define topic(ANY.TOPIC) TOPICSTR('/ANY_TOPIC')
// Create an alias queue that points to above created topic
define qalias(QA.APP) target(ANY.TOPIC) targtype(TOPIC)
// Create a queue for your application that does business logic. If one is available already then no need to create.
define ql(Q.BUSLOGIC)
// Create a durable subscription with destination queue as created in previous step.
define sub(SB.BUSLOGIC) topicstr('/ANY_TOPIC') dest(Q.BUSLOGIC)
// Create a queue for application that archives messages.
define ql(Q.ARCHIVE)
// Create another subscription with destination queue as created in previous step.
define sub(SB.ARCHIVE) topicstr('/ANY_TOPIC') dest(Q.ARCHIVE)
Write a simple MQ Java/JMS application to get messages from Q.ARCHIVE and archive messages.
A receive exit is not going to give you the whole message. Send and receive exits operate on the transmission buffers sent/received by channels. These will contain various protocol flows which are not documented because the protocol is not public, and part of those protocol flows will be chunks of the messages broken down to fit into 32Kb chunks.
You don't give enough information in your question for me to know what type of channel you are using, but I'm guessing it's on the client side since you are writing it in Java and that is the only environment where that is applicable.
Writing the exit at the client side, you'll need to be careful you deal with the cases where the message is not successfully put to the target queue, and you'll need to manage syncpoints etc.
If you were using QMgr-QMgr channels, you should use a message exit to capture the MQXR_MSG invocations where the whole message is given to you. If you put any further messages in a channel message exit, the messages you put are included in the channel's Syncpoint and so committed if the original messages were committed.
Since you are using client-QMgr channels, you could look at an API Exit on the QMgr end (currently client side API Exits are only supported for C clients) and catch all the MQPUT calls. This exit would also give you the MQPUT return codes so you could code your exit to look out for, and deal with failed puts.
Of course, writing an exit is a complicated task, so it may be worth finding out if there are any pre-written tools that could do this for you instead of starting from scratch.
I fully agree with Morag & Shashi, wrong approach. There is an open source project called Message Multiplexer (MMX) that will get a message from a queue and output it to one or more queues. Context information is maintained across the message put(s). For more info on MMX go to: http://www.capitalware.com/mmx_overview.html
If you cannot change the source or target queues to insert MMX into the mix then an API Exit may do the trick. Here is a blog posting about message replication via an API Exit: http://www.capitalware.com/rl_blog/?p=3304
This is quite an old question but it's worth replying with an update that's relevant to MQ 9.2.3 or later. There is a new feature called Streaming Queues (see https://www.ibm.com/docs/en/ibm-mq/9.2?topic=scenarios-streaming-queues) and one of the use-cases it is designed to support is putting a copy of every message sent to a given queue, to an alternative queue. Another application can then consume the duplicate messages and archive them separately to the application that is processing the original messages.
I have a Java client which monitors RabbitMQ queue. I am able to get the count of messages currently in queue with this code
#Resource
RabbitAdmin rabbitAdmin;
..........
DeclareOk declareOk = rabbitAdmin.getRabbitTemplate().execute(new ChannelCallback<DeclareOk>() {
public DeclareOk doInRabbit(Channel channel) throws Exception {
return channel.queueDeclarePassive("test.pending");
}
});
return declareOk.getMessageCount();
I want to get some more additional details like -
Message body of currently enqueued items.
Total number of messages that was enqueued in the queue since the queue was created.
Is there any way to retrieve these data in Java client?
With AMQP protocol (including RabbitMQ implementation) you can't get such info with 100% guarantee.
The closest number to messages count is messages count returned with queue.declare-ok (AMQP.Queue.DeclareOk in java AMQP client library).
Whilst messages count you receive with queue.declare-ok may match exact messages number enqueues, you can't rely on it as it doesn't count messages which waiting acknowledges or published to queue during transaction but not committed yet.
It really depends what kind of precission do you need.
As to enqueued messages body, you may want to manually extract all messages in queue, view their body and put them back to queue. This is the only way to do what you want.
You can get some information about messages count with Management Plugin, RabbitMQ Management HTTP API and rabbitmqctl util (see list_queues, list_channels).
You can't get total published messages count since queue was created and I think nobody implement such stats while it useless (FYI, with messages flow in average 10k per second you will not even reach uint64 in a few thousand years).
AMQP.Queue.DeclareOk dok = channel.queueDeclare(QUEUE_NAME, true, false, false, queueArgs);
dok.getMessageCount();
To access queue details via http api,
http://public-domain-name:15672/api/queues/%2f/queue_name
To access queue details via command from localhost cli promt,
curl -i -u guest_uname:guest_password http://localhost:15672/api/queues/%2f/queue_name
Where,
%2f is default vhost "/"
I am novice in IBM WebSphere MQ and I would like to ask you about the best approach to solve the following task.
I use WebSphere MQ 7.0 and I have implemented an java app to check MQ queue on incoming messages.
Incoming queue opened via the following code:
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_INQUIRE;
MQQueue incomingQueue =
qManager.accessQueue(qName, openOptions, null, null, null);
Now, the task is to check in real-time mode when new messages appear in incomingQueue and process them.
I permanently check queue depth via invocation of incomingQueue.getCurrentDepth() in while-loop and check if it is bigger than zero then I get new messages.
That works, but I believe it is not a good approach.
What is the best approach to be notified when a new incoming message appeared in MQ Queue?
Thank you.
Just call the queue.Get(msg) method. This is a blocking call and will return only when there is a message on a queue.
If the above is not suitable as it is a blocking call, you could look at WMQ JMS that provides a message listener. The message listener is used to receive messages on a callback method while the main thread can continue to do other work.
There are good samples that comes with MQ. You can find them under (on Windows) \tools\jms\samples and tools\wmqjava\samples.
Try to use the below open options to access a queue
openOptions = MQConstants.MQOO_INQUIRE + MQConstants.MQOO_FAIL_IF_QUIESCING
+ MQConstants.MQOO_INPUT_AS_Q_DEF + MQConstants.MQOO_READ_AHEAD;
And following get options to get the messages
MQGetMessageOptions getOptions = new MQGetMessageOptions();
getOptions.options = MQConstants.MQGMO_WAIT + MQConstants.MQGMO_PROPERTIES_COMPATIBILITY
+ MQConstants.MQGMO_ALL_SEGMENTS_AVAILABLE + MQConstants.MQGMO_COMPLETE_MSG
+ MQConstants.MQGMO_ALL_MSGS_AVAILABLE;
MQConstants.MQGMO_WAIT option will help us to read the messages when arrives to the queue. But make sure a Java thread/program should be there to run your class all the time to listen to the queue
I'm invoking:
GetResponse response = channel.basicGet("some.queue", false); // no auto-ack
....
channel.basicAck(deliveryTag, ...);
However, when I invoke basicGet, the messages in the queue stay in "Ready", rather than in "Unacknowledged". I want them to be in unacknowledged, so that I can either basic.ack them (thus discarding them from the queue), or basic.nack them
I'm doing the following to mimic Delaying the ack:
At consumption time
Get(consume) the message form the initial Queue.
Create a "PendingAck_123456" Queue.
123456 is a unique id of the message.
Set the following properties
x-message-ttl (to requeue after
timeout)
x-expires (to make sure the temp queue will be deleted)
x-dead-letter-exchange and x-deal-letter-routing-key to requeue to
the initial Queue upon TTL expiration.
Publish the message Pending ack to this "PendingAck_123456" Queue
Ack the message to delete it from the initial queue
At Acknowledge time
Calculate Queue Name from Message Id and Get from the "PendingAck_123456" Queue
Acknowledge it (no need to call .getBody() ).
That'll delete it from this pending queue, preventing the TTL to requeue it
Remarks
A Queue for only 1 message.. Is that an issue if there are a lot of such Queues ?
A requeued message will be sent at the queue input side.. not at the queue output (as would do a real ack).. There is an impact on the messages order.
Message is copied by the application to the Pending Queue.. This is an additional step that may have impacts on the overall performance.
To mimic a Nack/Reject, you you may want to Copy the message to the Initial Queue, and Ack it from the PendingAck queue. By default, the TTL would do it (later).
When doing ack immediately after the get it works fine. However, in my case, they were separated by a request. And spring's template closes the channel and connection on each execution. So there are three options:
keep one channel and connection open throughout the whole lifetime of the application
have some kind of conversation-scope (or worst-case: use the session) to store the same channel and reuse it.
use one channel per request, acknowledge receipt immediately, and store the messages in memory.
In the former two cases you can't do it with spring's RabbitTemplate