I am using Spring AMQP with asynchronous messaging. My model assumes that there are two applications A and B, both producers and consumers.
A sends job request to B and starts listening.
B are listening for job request, and when it comes, starts job and periodically sends progress messages to A.
B sends job finish message to A after job is finished.
A consumes progress messages until job finish message comes, then A exists.
I am using #RabbitListener on class level and #RabbitHandler on method level, for message consuming. Everything works nice and design is clean, I like Spring's solution.
My problem is - I have no idea how to detect, and how to act, when A is expecting to receive progress message from B (any message) and it's not coming in. Is there any timeout for such cases? If so, how to implement callback method?
I found some timeout settings, but they usually works for connection itself, or only when using RPC pattern (one request one response).
Desired solution is - A should receive progress message every minute. If no progress message is consumed for, say, 3 minutes, I want to cancel the job.
When using async consumers, there's no mechanism to generate an event if a message is not received within some time period.
You can schedule your own task though and cancel/reschedule the task when a message arrives.
Use a TaskScheduler with
future = schedule(myRunnable, new Date(System.currentTimeMillis() + 180000));
use future.cancel() when a message arrives.
Related
Java & RabbitMQ here. I need to implement sort of a poison pill pattern where, upon handling a particular message, the consumer needs to cancel itself and stop receiving/handling any further messages. Full stop and clean up. The message kills the consumer and releases the thread, memory, etc.
I see consumers have a handleCancel method that they can implement to respond to cancellation commands from the outside, but how do I handle a poison pill message inside a consumer that tells the consumer to fall over dead?
I don't think RabbitMQ handles this scenario for some reason.
My solution which appears to be working:
Implement a stateful consumer that exists in one of two states: Processing (default) and Terminating
When its in the Processing state it consumes and handles messages off the queue like normal. When it receives the magical poison pill (perhaps a value in the message header/properties, or maybe a specific value in the message itself) it sets its status to Terminating and does not process the message. It also uses an async event bus to send a custom ShutdownConsumerEvent to an external handler. This event is instantiated with both the channel and consumerTag sent to the consumer (e.g. ShutdownConsumerEvent event = new ShutdownConsumerEvent(channel, consumerTag);)
Any more messages the consumer receives while in the Terminating state get republished to the queue, with ACKs enabled so we don't lose them and have pseudo-transactionality
When the external ShutdownConsumerSubscriber (a registered handler to receive ShutdownConsumerEvents) receives the command to shut down the consumer, it does this by issuing a channel.basicCancel(consumerTag)
I have an application which should use JMS to queue several long running tasks asynchronously in response to a specific request. Some of these tasks might complete within seconds while others might take a longer time to complete. The original request should already complete after all the tasks have been started (i.e. the message to start the task has been queued) - i.e. I don't want to block the request while the tasks are being executed.
Now, however, I would like to execute another action per request once all of the messages have been processed successfully. For this, I would like to send another message to another queue - but only after all messages have been processed.
So what I am doing is a bit similar to a reply-response pattern, but not exactly: The responses of multiple messages (which were queued in the same transaction) should be aggregated and processed in a single transaction once they are all available. Also, I don't want to "block" the transaction enqueuing the messages by waiting for replies.
My first, naive approach would be the following:
When a requests comes in:
Queue n messages for each of the n actions to be performed. Give them all the same correlation id.
Store n (i.e. the number of messages sent) in a database along with the correlation id of the messages.
Complete the request successfully
Each of the workers would do the following:
Receive a message from the queue
Do the work that needs to be done to handle the message
Decrement the counter stored in the database based on the correlation id.
If the counter has reached zero: Send a "COMPLETED" message to the completed-queue
However, I am wondering if there is an alternative solution which doesn't require a database (or any other kind of external store) to keep track whether all messages have already been processed or not.
Does JMS provide some functionality which would help me with this?
Or do I really have to use the database in this case?
If your system is distributed, and I presume it is, it's very hard to solve this problem without some kind of global latch lock like the one you have implemented. The main thing to notice is that "tasks" have to signal within "global storage" that they are over. Your app is essentially creating a new countdown latch lock instance (identified by CorrelationID) each time a new request comes by inserting a row in a db. Your tasks are "signaling" the end of jobs by counting that latch down. The job which ends holding a lock has to clean the row.
Now global storage doesn't have to be a database, but it still has to be some kind of global access state. And you have to keep on counting. And if only thing you have is a JMS you have to create latch and count down by sending messages.
The simplest solution which comes to a mind is by having each job sends a TASK_ENDED message to a JOBS_FINISHED queue. TASK_ENDED message stands for: "task X triggered by request Y with CorrelationID Z has ended" signal. Just as counting down in db. Recipient of this q is a special task whose only job is to trigger COMPLETED messages when all messages are received for a request with given correlation id. So this jobs is just reading messages sequentially. And counts each unique correlation id which it encounters. Once it has counted to an expected number it should clear that counter and send COMPLETED message.
You can encode number of triggered tasks and any other specifics within JMS header of messages created when processing request. For example:
// pretend this request handling triggers 10 tasks
// here we are creating first of ten START TASK messages
TextMessage msg1 = session.createTextMessage("Start a first task");
msg1.setJMSCorrelationID(request.id);
msg1.setIntProperty("TASK_NUM", 1);
msg1.setIntProperty("TOTAL_TASK_COUNT", 10);
And than you just pass that info to a TASK_ENDED messages all the way to a final job. You have to make sure that all messages sent to an ending job are received to same instance of a job.
You could go from here by expanding idea with publish subscribe messaging, and error handling and temporary queues and stuff like that, but that is becoming very specific of you needs so I'll end here.
I have a stream of incoming data that is sent to RabbitMQ as individual messages.
I want to send these to a service that requires a batch of messages. I need to send the request to the service when I either have a batch of 1000 messages or when 5 seconds have expired. Is this possible using SimpleMessageListenerContainer?
The SimpleMessageListenerContainer supports transactions, however this won't help with the 5 second timeout. I did look at the method doReceiveAndExecute(BlockingQueueConsumer consumer) and "receiveTimeout", but as this variable is inside the transaction loop I could end up waiting 5 seconds per message (1000*5 sec= 83 min).
I currently have a channel aware listener that batches the messages into a bulk processor that will manage my timeouts and queue length. The SimpleMessageListenerContainer is set to manual ack. However as the listener returns before the message has actually been sent to the service I occasionally have issues when I do come to ack the message as the channel has been closed.
I have thought about writing my own ListenerContainer that sends the whole BlockingQueueConsumer to the Listener. Is this the only solution or has anyone managed to do something similar already?
You can use a ChannelAwareMessageListener, set acknowledgeMode=MANUAL; accumulate the deliveries in the listener; start a timer (scheduled task) to execute in +5 seconds and keep a reference to the channel. When a new delivery arrives, cancel the task, add the new delivery to the collection.
When 1000 deliveries arrive (or the scheduled task fires); invoke your service; then use channel.basicAck() (multiple) to ack the processed messages.
You'll have some race conditions to deal with but it should be pretty easy. Perhaps another queue of batches would be easiest with some other thread waiting for batches to arrive in that queue.
EDIT
As of 2.2, the SimpleMessageListenerContainer supports delivering batches of messages natively - see Batched Messages.
Starting with version 2.2, the SimpleMessageListeneContainer can be use to create batches on the consumer side (where the producer sent discrete messages).
Set the container property consumerBatchEnabled to enable this feature. deBatchingEnabled must also be true so that the container is responsible for processing batches of both types. Implement BatchMessageListener or ChannelAwareBatchMessageListener when consumerBatchEnabled is true. See #RabbitListener with Batching for information about using this feature with #RabbitListener.
After reading some document of JMS, I totally puzzled by the phrase synchronous and asynchronouns.
See this page: http://docs.oracle.com/cd/E19798-01/821-1841/bncdq/index.html
Synchronous
You use the receive method to consume a message synchronously.
You can use this method at any time after you call the start method:
connection.start();
Message m = consumer.receive();
connection.start();
Message m = consumer.receive(1000); // time out after a second
To consume a message asynchronously, you use a message listener, described in the next section.
Asynchronous
JMS Message Listeners
A message listener is an object that acts as an asynchronous event handler for messages. This object implements the MessageListener interface, which contains one method, onMessage. In the onMessage method, you define the actions to be taken when a message arrives.
You register the message listener with a specific MessageConsumer by using the setMessageListener method. For example, if you define a class named Listener that implements the MessageListener interface, you can register the message listener as follows:
Listener myListener = new Listener();
consumer.setMessageListener(myListener);
I have two questions:
As what I understood, the nature of JMS is asynchronous. Producer publishes messages to the queue/topic, it doesn't need to wait consumer. This is asynchronous behaviour. How can it be "synchronous"?
If the "mesageListener" is asynchronous, but in my test with spring-jms, I found it always running in a thread. That means, if I write Thread.sleep(2000) in onMessage, it have to be wait 2 seconds before processing next message. Is it "asynchronous"?
If you understand it better like this, consumer.receive() uses a pull model: you read from a queue and are blocked waiting for this message until it comes, or some timeout has elapsed.
Using a listener uses a push model: you register a listener and, when a message comes in, the listener is called, in a separate thread.
Everything is done in a thread in Java, and the listener call is no exception. Whether the listener message handling prevents the processing of other messages in the queue depends on how many threads are dedicated to message processing. If you configure Spring to use a pool of 5 threads to process messages asynchronously, then 5 listeners will be able to process messages in parallel.
Like I understand this:
asynchronous - MessageListener: Use this on a server that listens to a queue. When a message arrives, then deal with it immediately. The server keeps listening to this queue.
synchronous - consumer.receive(1000): Use this on a client applications that now and then needs to check if a message is intend for this client. Example: poll every 60 seconds. This only opens a connection to the server shortly. The 1000 milliseconds will keep this connection open. If a message arrives within these 1000 milliseconds, then the message is consumed and the connection is closed.
You are looking at it end-to-end: from publisher to the consumer. Yes, it is asynchronous delivery from publisher to consumer irrespective of Sync/Async consumer. However Sync/Async in your question is for consumer only, i.e from the JMS broker (eg: ApacheMQ) to the consumer. As others have pointed out, Sync consumers pull messages sequentially from the broker and are waiting for messages. Async consumers register a callback where messages pushed to them (onMessage). Async consumers can go about doing other things while these messages are delivered to them asynchronously from the JMS broker.
I understand synchronous/asynchronous differently.
Synchronous: Caller(Sender) has to wait till the response from consumer has been received(till the time-out) -- request/reply pattern
Asynchronous: Caller(Sender) just post message and continue with its work, while the consumer processes as soon as the message reaches it -- one way request
Any MOM(Message Oriented Middle ware) follows service activator pattern which promotes asynchronous communication. One of my project has implemented a framework around JMS to make communication really synchronous.
Any message has 2 parts.
a. Metadata attributes
b. Payload
Set attribute "reply-to-queue" to a randomly generated value
Make sure the MOM framework creates temporary queue with name from #2
Make sure the sender spawns thread, which listens to temporary queue created in #3
Publish message and block sender till it receives message to temporary queue
Make sure the consumer intercepts "reply-to-queue" header and publishes response to it
This is one of the ways to make MOM based communication acts like synchronous. You may find other implementations like request-reply mechanism.
I have a single threaded message listener that listens for the incoming messages.The received messages are persisted in a database as they are recieved.
There is a message A and associated message B follows it with a reference it.In Some Odd occurences, B arrives before A. Now in this case, There has to be 3retries after some 'x' equal intervals to see if A has arrived and then persists the association.
As the message listener is single threaded if we put the thread to sleep the whole system would be affected. So there has to be a separate thread retrying.
Can we use Quartz Job Scheduler for this purpose to avoid handling multiThreading issues and to have a PERSISTENT STORE of the in any of the following 2 ways,
Schedule a Job in Quartz for 3 times and keep track of a flag in the JobDataMap to check if previous retries succeeds then return without doing anything
OR
2.Schedule a Job to retry once and then if the retry fails schedule the same job after a few seconds.
Can quartz be used only for repetitive jobs and not have some state information span across job or Is there any other better way to do this.
You should configure your JMS provider to set the redelivery delay on your message queue. In your code, you call context.setRollbackOnly to abort a message that does not pass prerequisites check.
In that case, the code execution scenario becomes:
consume "B", check for prerequisite and detect A is lacking
roll back the transaction and the message returns to the queue, it will be re-delivered after the configured delay
consume and process next message "A"
after the delay, the MDB consumes and processes again "B" with success