I am working on implementing Akka Alpakka for consuming from and producing to ActiveMQ queues, in Java. I can consume from the queue successfully, but I haven't yet been able to implement application-level message acknowledgement.
My goal is to consume messages from a queue and send them to another actor for processing. When that actor has completed processing, I want it to be able control the acknowledgement of the message in ActiveMQ. Presumably this would be done by sending a message to another actor that can do the acknowledgement, calling an acknowledge function on the message itself, or some other way.
In my test, 2 messages are put into the AlpakkaTest queue, and then this code attempts to consume and acknowledge them. However, I don't see a way to set the ActiveMQ session to CLIENT_ACKNOWLEDGE, and I don't see any difference in behavior with or without the call to m.acknowledge();. Because of this, I think messages are still being auto-acknowledged.
Does anyone know the accepted way to configure ActiveMQ sessions for CLIENT_ACKNOWLEDGE and manually acknowledge ActiveMQ messages in Java Akka systems using Alpakka?
The relevant test function is:
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://0.0.0.0:2999"); // An embedded broker running in the test.
Source<Message, NotUsed> jmsSource = JmsSource.create(
JmsSourceSettings.create(connectionFactory)
.withQueue("AlpakkaTest")
.withBufferSize(2)
);
Materializer materializer = ActorMaterializer.create(system); // `system` is an ActorSystem passed to the function.
try {
List<Message> messages = jmsSource
.take(2)
.runWith(Sink.seq(), materializer)
.toCompletableFuture().get(4, TimeUnit.SECONDS);
for(Message m:messages) {
System.out.println("Found Message ID: " + m.getJMSMessageID());
try {
m.acknowledge();
} catch(JMSException jmsException) {
System.out.println("Acknowledgement Failed for Message ID: " + m.getJMSMessageID() + " (" + jmsException.getLocalizedMessage() + ")");
}
}
} catch (InterruptedException e1) {
e1.printStackTrace();
} catch (ExecutionException e1) {
e1.printStackTrace();
} catch (TimeoutException e1) {
e1.printStackTrace();
} catch (JMSException e) {
e.printStackTrace();
}
This code prints:
Found Message ID: ID:jmstest-43178-1503343061195-1:26:1:1:1
Found Message ID: ID:jmstest-43178-1503343061195-1:27:1:1:1
Update: The acknowledgement mode is configurable in the JMS connector since Alpakka 0.15. From the linked documentation:
Source<Message, NotUsed> jmsSource = JmsSource.create(JmsSourceSettings
.create(connectionFactory)
.withQueue("test")
.withAcknowledgeMode(AcknowledgeMode.ClientAcknowledge())
);
CompletionStage<List<String>> result = jmsSource
.take(msgsIn.size())
.map(message -> {
String text = ((ActiveMQTextMessage)message).getText();
message.acknowledge();
return text;
})
.runWith(Sink.seq(), materializer);
As of version 0.11, Alpakka's JMS connector does not support application-level message acknowledgment. Alpakka creates internally a Session with the CLIENT_ACKNOWLEDGE mode here and acknowledges each message here in the internal MessageListener. The API does not expose these settings for overriding.
There is an open ticket that discusses enabling downstream acknowledgement of queue-based sources, but that ticket has been inactive for a while.
Currently you cannot prevent Alpakka from acknowledging the messages at the JMS level. However, that doesn't preclude you from adding a stage to your stream that sends each message to an actor for processing and uses the actor's replies as backpressure signals. The Akka Streams documentation describes how to do this with either a combination of mapAsync and ask or with Sink.actorRefWithAck. For example, to use the former:
Timeout askTimeout = Timeout.apply(4, TimeUnit.SECONDS);
jmsSource
.mapAsync(2, msg -> ask(processorActor, msg, askTimeout))
.runWith(Sink.seq(), materializer);
(Side note: In the related Streamz project, there is a recently opened ticket to allow application-level acknowledgement. Streamz is the replacement for the old akka-camel module and, like Alpakka, is built on Akka Streams. Streamz also has a Java API and is listed in the Alpakka documentation as an external connector.)
Looking at the source code for the Alpakka JmsSourceStage it already acknowledges each incoming message for you (and it's session is a Client Ack session). From what I can tell from the source there is no mode that allows you to do the acknowledgement of messages.
You can view the source code for Alpakka here.
Related
I have a problem when reading messages from multiple JMS Queues in a single transaction using WebLogic JMS client (wlthin3client.jar) from WebLogic 11g (WebLogic Server 10.3.6.0). I am trying to read first one message from queue Q1 and then, if this message satisfy some requirements, read other message (if available at that time) from queue Q2.
I expect that after committing the transaction both messages should disappear from Q1 and Q2. In case of rollback - messages should remain in both Q1 and Q2.
My first approach was to use an asynchronous queue receiver to read from Q1 and then synchronously read from Q2 when it is needed:
void run() throws JMSException, NamingException {
QueueConnectionFactory cf = (QueueConnectionFactory) ctx.lookup(connectionFactory);
// create connection and session
conn = cf.createQueueConnection();
session = conn.createQueueSession(true, Session.SESSION_TRANSACTED);
Queue q1 = (Queue) ctx.lookup(queue1);
// setup async receiver for Q1
QueueReceiver q1Receiver = session.createReceiver(q1 );
q1Receiver.setMessageListener(this);
conn.start();
// ...
// after messages are processed
conn.close();
}
#Override
public void onMessage(Message q1msg) {
try {
QueueReceiver q2receiver = session.createReceiver(queue2);
if(shouldReadFromQ2(q1msg)){
// synchronous receive from Q2
Message q2msg = q2receiver.receiveNoWait();
process(q2msg);
}
session.commit();
} catch (JMSException e) {
e.printStackTrace();
} finally {
q2receiver.close();
}
}
Unfortunately even though I issue a session.commit() the message from Q1 remains uncommitted. It is in receive state until the connection or receiver is closed. Then is seems to be rolled back as it gets delayed state.
Other observations:
Q1 message is correctly committed if Q2 is empty and there is nothing to read from it.
The problem does not occur when I am using synchronous API in similar, nested way for both Q1 and Q2. So if I use q1Receiver.receiveNoWait() everything is fine.
If I use asynchronous API in similar, nested way for Q1 and Q2, then only Q1 message listener is called and commit works on Q1. But Q2 message listener is not called at all and Q2 is not committed (message stuck in receive / delayed).
Am I misusing the API somehow? Or is this a WLS JMS bug? How to combine reading from multiple queues with asynchronous API?
It turns out this is an WLS JMS bug 28637420.
The bug status says it is fixed, but I wouldn't rely on this - a WLS 11g patch with this fix doesn't work (see bug 29177370).
Oracle says that this happens because two different delivery mechanisms (synchronous messages vs asynchronous messages) were not designed to work together on the same session.
Simplest way to work around the problem is just use synchronous API (polling) for cases when you need to work on multiple queues in a single session. I decided on this approach.
Another option suggested by oracle is to to use UserTransactions with two different sessions, one session for the async consumer and another session for the synchronous consumer. I didn't test that though.
I'm using Qpid Proton (proton-j-0.13.0) to send messages over AMQP to an ActiveMQ 5.12.0 queue. On a development machine, where ActiveMQ and the Java program run on the same machine, this is working fine. On a test environment, where ActiveMQ is running on a separate server, we see the send() method hangs in 15 to 20 percent of the cases. The CPU also remains around 100% when the send() method hangt. When the send() succeeds, it completes within 0.1 seconds.
Statements to perform a send are similar to this:
final Messenger messenger = Messenger.Factory.create();
messenger.start;
messenger.put(message); // one message of 1 KByte
messenger.send(1);
messenger.stop();
I'm aware Messenger.send(int n) is a blocking method. However, I don't know why it would block my calls. I can add a timeout and try to resend the message, but that's a workaround instead of a proper solution.
Statements to receive the sent messages from ActiveMQ are similar to this:
this.messenger = Messenger.Factory.create();
this.messenger.start();
this.messenger.subscribe(this.address);
while (this.isRunning) {
try {
this.messenger.recv(1);
while (this.messenger.incoming() > 0) {
final Message message = this.messenger.get();
this.messageListener.onMessage(message);
} catch (final Exception e) {
LOGGER.error("Exception while receiving messages", e);
}
}
Am I missing something simple, being a Qpid newbie? Could this be configuration in ActiveMQ? Is it normal to add a timeout and retry? Any help to resolve this would appreciated.
on my java program, some kind of messages are being sent over RabbitMQ queues as below :
if(!con.isConnected()){
log.error("Not connected !!!");
return false;
}
con.getChannel().basicPublish("",queueName, MessageProperties.PERSISTENT_BASIC, bytes)
I deleted queues via RabbitMQ management GUI plugin
try to send a message over that deleted queue
Result: queues were deleted from RabbitMQ GUI but when I am trying to send message over that deleted RabbitMQ queues, connection is still alive.(con.isConnected() == true ) I need to find a way to listen the queue, if it is deleted , I shouldn't send any message to the deleted queue.
Note: After deleting queue, I am not restarting RabbitMQ.
channel creation :
channel = connection.createChannel();
channel.queueDeclare(prop.getQueueName(), true, false, false, null);
example code channel, queue,exchange creation :
ConnectionFactory cf = new ConnectionFactory();
cf.setUsername("guest");
cf.setPassword("guest");
cf.setHost("localhost");
cf.setPort(5672);
cf.setAutomaticRecoveryEnabled(true);
cf.setConnectionTimeout(10000);
cf.setNetworkRecoveryInterval(10000);
cf.setTopologyRecoveryEnabled(true);
cf.setRequestedHeartbeat(5);
Connection connection = cf.newConnection();
channel = connection.createChannel();
channel.queueDeclare("test", true, false, false, null);
channel.exchangeDeclare("testExchange", "direct",true);
channel.queueBind("test", "testExchange", "testRoutingKey");
connection.addShutdownListener(new ShutdownListener() {
#Override
public void shutdownCompleted(ShutdownSignalException cause) {
System.out.println("test"+cause);
}
});
Sending message :
channel.basicPublish("testExchange", "testRoutingKey", null,messageBodyBytes);
From RabbitMQ google
Messages in AMQP 0-9-1 are not published to queues; they are published to exchanges, from where they
are routed to a queue (or another exchange) or not. [1]
basic.publish is a completely asynchronous protocol method by design: there is no response for it
unless you ask for it [2]. Messages that are unroutable can be returned to the publisher
if you define a return listener and publish with the mandatory flag set to true.
Note that publisher confirms and the mandatory flag/returns are orthogonal and one does not imply
the other.
Defining return listener and setting mandatory flag true was solved my problem. If any message was not routed , I can catch them by using ReturnListener and add to my persisted queue to send another time when system becomes active.
I am sending some messages to a JMS queue. What are the possible ways to search for a particular message in a queue to consume?
I tried out in the following way: I am setting the JMSCorrelationID while sending a message to the queue:
public void createDQueue(String queuename, String json, Integer userid) {
try {
QueueSession.AUTO_ACKNOWLEDGE );
Queue queue = session.createQueue(queuename);
ObjectMessage objectMessage = session.createObjectMessage();
objectMessage.setJMSCorrelationID(String.valueOf(userid));
objectMessage.setObject(json);
session.createSender(queue).send(objectMessage);
session.close();
connection.close();
}catch(Exception e){
e.printStackTrace();
}
}
In the consumer code I want to get that particular message based on the JMSCorrelationID. I am not able to get that particular message. Can you suggest a solution?
public void getSpecificMessage(String queuename, Integer userid) {
try {
QueueConnectionFactory connectionFactory = new ActiveMQConnectionFactory( "tcp://localhost:61616");
((ActiveMQConnectionFactory) connectionFactory).setUseAsyncSend(true);
QueueConnection connection = connectionFactory.createQueueConnection();
connection.start();
QueueSession session = connection.createQueueSession( false,
QueueSession.AUTO_ACKNOWLEDGE );
String id = String.valueOf(userid);
Queue queue = session.createQueue(queuename);
QueueReceiver receiver = session.createReceiver(queue, "JMSCorrelationID="+id);
Message message = receiver.receive();
} catch (JMSException e) {
e.printStackTrace();
}
}
Your first problem is that you are trying to think about the message broker as a database, you must always remember this sage piece of advice, "A message broker is not a database".
There are certain limits on how deep a consumer or Queue browser can go into a destination before the broker will not page in more messages from disk, so you need to check your depth and see if its large than you maxPageSize setting and adjust as needed, but remember that messages paged in remain in memory until consumed.
Just wrap the id value in single quotes
"JMSCorrelationID='"+id+"'"
This functionality is not recommended to be used , there are lot more complications as explained by Tim , but if you want to obsolutely work with it make the change
You can search messages using the MeessageID of a message. This would be fast as messaging providers index messages on message id. There are other way to search based on CorrelationId, meta data etc.
But please remember the primary objective of using a messaging provider is to connect applications in a time independent manner. The receiving application must get messages as soon as possible. If messages are piling up in a queue, it indicates a problem that must be addressed.
I'm trying to hook up a message listener to an MQ.
Queue primaryQueue = session.createQueue("Q1");
MessageConsumer primaryConsumer = session.createConsumer(primaryQueue);
primaryConsumer.setMessageListener(new MessageListener() {
#Override
public void onMessage(Message message) {
ObjectMessage objectMessage = (ObjectMessage) message;
try {
Payload payload = (Payload) objectMessage.getObject();
System.out.println("Payload\n" + payload);
} catch (JMSException e) {
e.printStackTrace();
}
}
});
It's just a simple console app that's connecting to a queue. What happens though is, after i start the application and attach the listener, it reads the current set of messages and just... stops. If i send another message into the queue a couple of seconds later, the listener isn't even up and ready to receive it. I'm working with a Websphere MQ that i'm running locally.
I was kinda expecting the app to like hang on and wait to be triggered again right? The point of hooking up a listener is so that it can react to incoming messages rather than me physically doing a recieve() on it.
I'm really new to messaging and queues, so i guess i'm missing something obvious. Appreciate the help.