I have a problem when reading messages from multiple JMS Queues in a single transaction using WebLogic JMS client (wlthin3client.jar) from WebLogic 11g (WebLogic Server 10.3.6.0). I am trying to read first one message from queue Q1 and then, if this message satisfy some requirements, read other message (if available at that time) from queue Q2.
I expect that after committing the transaction both messages should disappear from Q1 and Q2. In case of rollback - messages should remain in both Q1 and Q2.
My first approach was to use an asynchronous queue receiver to read from Q1 and then synchronously read from Q2 when it is needed:
void run() throws JMSException, NamingException {
QueueConnectionFactory cf = (QueueConnectionFactory) ctx.lookup(connectionFactory);
// create connection and session
conn = cf.createQueueConnection();
session = conn.createQueueSession(true, Session.SESSION_TRANSACTED);
Queue q1 = (Queue) ctx.lookup(queue1);
// setup async receiver for Q1
QueueReceiver q1Receiver = session.createReceiver(q1 );
q1Receiver.setMessageListener(this);
conn.start();
// ...
// after messages are processed
conn.close();
}
#Override
public void onMessage(Message q1msg) {
try {
QueueReceiver q2receiver = session.createReceiver(queue2);
if(shouldReadFromQ2(q1msg)){
// synchronous receive from Q2
Message q2msg = q2receiver.receiveNoWait();
process(q2msg);
}
session.commit();
} catch (JMSException e) {
e.printStackTrace();
} finally {
q2receiver.close();
}
}
Unfortunately even though I issue a session.commit() the message from Q1 remains uncommitted. It is in receive state until the connection or receiver is closed. Then is seems to be rolled back as it gets delayed state.
Other observations:
Q1 message is correctly committed if Q2 is empty and there is nothing to read from it.
The problem does not occur when I am using synchronous API in similar, nested way for both Q1 and Q2. So if I use q1Receiver.receiveNoWait() everything is fine.
If I use asynchronous API in similar, nested way for Q1 and Q2, then only Q1 message listener is called and commit works on Q1. But Q2 message listener is not called at all and Q2 is not committed (message stuck in receive / delayed).
Am I misusing the API somehow? Or is this a WLS JMS bug? How to combine reading from multiple queues with asynchronous API?
It turns out this is an WLS JMS bug 28637420.
The bug status says it is fixed, but I wouldn't rely on this - a WLS 11g patch with this fix doesn't work (see bug 29177370).
Oracle says that this happens because two different delivery mechanisms (synchronous messages vs asynchronous messages) were not designed to work together on the same session.
Simplest way to work around the problem is just use synchronous API (polling) for cases when you need to work on multiple queues in a single session. I decided on this approach.
Another option suggested by oracle is to to use UserTransactions with two different sessions, one session for the async consumer and another session for the synchronous consumer. I didn't test that though.
Related
How can I check whether a queue exists on a JMS server using the Java API? I don't want to send or receive any data to the queue for now, just verify that the queue exists. Also, the queue may be empty.
Here is my code sample. I have removed the error handling for simplicity.
Connection connection = null;
Session session = null;
connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
//I was hoping this next line would throw an exception if the queue does not exist
Queue queue = session.createQueue(queueName);
My JMS server is TIBCO EMS. I'm hoping for a solution that works on versions 5-7.
Solution
I followed the recommendation in the accepted answer but created a browser instead. The following line threw an exception as desired:
QueueBrowser browser = session.createBrowser(queue);
This is dependent on the provider, but you wont know in most cases until you create the session type, such as session.createConsumer. Simply creating a consumer this way will not consume any messages until you do a receive. And it is here the behavior may change from provider to provider and configuration of the server.
For example with ActiveMQ, assuming there are no permissions blocking the user you are connecting with, the queue is created automatically when you create the session type.
With WebSphere MQ, the queue has to be defined by an admin. If it does not exist, the queue manager will return an exception with a reason code of 2085 (UNKNOWN_OBJECT_NAME).
Outside of this, you'd need to see if the particular provider had a way to access a list of queues. Using the above examples, ActiveMQ you can get the list of queues using JMX, with WebSphere MQ, you can do this if you have permissions to send PCF commands to the queue manager.
Try creating a consumer or producer off the Session passing in the queue object you just created:
session.createConsumer(queue);
This should throw an InvalidDestinationException if the queue (or topic) does not exist.
I have a JBoss-6 server with HornetQ and a single queue:
<queue name="my.queue">
<entry name="/queue/test"/>
</queue>
There a different consumers (on different machines) connected to this queue, but only a single consumer is active at a time. If I shut down this consumer, the messages are immediately processed by one of the other consumers.
Since my messages have some time consuming processing, I want multiple consumer process their unique messages concurrently.
I remember a similar in earlier versions of JBoss where this setup worked without problems. Here in Jboss-6 the messaging system is working well -- except of the issue described above. This question is similar to Are multiple client consumers possible in hornetq?, but the scenario is not similar to mine.
Update 1: If I close (STRG+C) one consumer there is a short timeout (until the server recognized the lost consumer) until the next consumer gets the message.
Update 2: Code Snippet
VoidListener ml = new VoidListener();
QueueConnectionFactory qcf = (QueueConnectionFactory)
ctx.lookup("ConnectionFactory");
QueueConnection conn = qcf.createQueueConnection();
Queue queue = (Queue) ctx.lookup(queueName);
QueueSession session = conn.createQueueSession(false,
QueueSession.AUTO_ACKNOWLEDGE);
QueueReceiver recv = session.createReceiver(queue,"");
recv.setMessageListener(ml);
conn.start();
And the MessageListerner:
public class OlVoidListener implements MessageListener
{
public void onMessage(Message msg)
{
counter++;
logger.debug("Message ("+counter+") received");
try {Thread.sleep(15*1000);} catch (InterruptedException e) {}
}
}
With multiple consumers on a queue, messages are load balanced between the consumers.
As you have some time consuming the message, you should disable buffering by setting consumer-window-size.
On hornetQ there's an example on the distribution, about how to disable client buffering and give a better support for slow consumers. (a slow consumer is a consumer that will have some time processing the message)
message systems will pre-fetch/read-ahead messages to the client buffer to speed up processing and avoid network latency. This is not an issue if you have fast processing queues and a single consumer.
JBoss Messaging offered the slow-consumer option at the connection factory and hornetq offers the consumer window size.
Most Message systems will provide you a way to enable or disable client pre-fetching.
I am sorry but I cannot understand what exactly the problem is. We've used hornetq in 2.0.0.GA version and 2.2.2.Final. In both cases, queue-based load balancing works fine. If you will define multiple consumers for one queue and all of them are active, messages will be distributed between them automatically. First message to consumer A, second to consumer B, third to consumer C and so on. This is how queues with multiple consumers works - it's free load balancing :) That's normal that when you shut down one consumer, others would receive more messages.
I'm having a bit of trouble with understanding JMS from a performance perspective. We have this very straightforward code in our application:
QueueConnection connection = null;
QueueSession session = null;
QueueSender sender = null;
TextMessage msg = null;
try {
// The JNDIHelper uses InitialContext to look up things
QueueConnectionFactory qcf = JNDIHelper.lookupFactory();
Queue destQueue = JNDIHelper.lookupQueue();
// These objects are created for every message, which is quite slow
connection = qcf.createQueueConnection();
session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
sender = session.createSender(destQueue);
// This is the actual message
msg = session.createTextMessage(xmlMsg);
sender.setTimeToLive(0);
sender.send(msg);
}
finally {
// Close all objects again
JMSUtilities.safeClose(sender);
JMSUtilities.safeClose(session);
JMSUtilities.safeClose(connection);
}
The code is correct, but probably some of the above artefacts could be reused for several messages. These are our configurations:
We use Oracle Weblogic 10.3.3
Weblogic connects to IBM MQ 7.0 (Problem also appears with 6.0) for JMS
The above logic is executed by a single thread on a backend server. It would be simple to keep some objects (QueueConnection, QueueSession, QueueSender) in memory as there is no concurrency involved.
My questions
Which types of objects can be shared among several messages? (of course we'd include error recovery, restoring those shared objects)
What are best practices to improve performance?
Here are some relevant parts of the jms spec:
section 2.8 Multithreading
JMS Object Supports Concurrent Use
Destination YES
ConnectionFactory YES
Connection YES
Session NO
MessageProducer NO
MessageConsumer NO
section 4.4.14 Serial Execution of Client Code
JMS does not cause concurrent execution of client code unless a client explicitly requests it. One way this is done is to define that a session serializes all asynchronous delivery of messages
So as already mentioned reuse as much as possible. Reuse the ConnectionFactory, Connection and Destinations for all Threads. For each Thread reuse consumers and producers.
If you are reusing a JMS connection beware, that the JMS Provider will multiplex different sessions on that connections. So even if it is safe to reuse connections it might be faster to create a connection for every session you need.
The only thing you need to create again and again is the msg itself - if you are sending to the same queue.
So yes, you can remember the Connection, Session and Sender.
Define "to share".
If you mean to share among different threads this is very dangerous. You can safely share QueueConnectionFactory object as well as the JMS Connection object. You must not share Session, Sender/Consumer or Message objects. Thats the way how TIBCO EMS works I am not sure about IBM platform but I guess this is very same.
If you can be sure your "send" method is not called by different threads you can encapulate this into a MySender class with Connection, Session and Sender member variables. But watch out! Do properly close the resources on exit. Thats what Heiko Rupp recommends. Somthing like this:
class MySender {
private QueueConnection connection = null;
private QueueSession session = null;
private QueueSender sender = null;
public MySender(...) { /* initialize conn/sess/sender */ }
public send(String xmlMsg) { /* sender.send(session.createTextMessage(xmlMsg)) */ }
public close() { /* close all resources */ }
}
Regarding performance. There is no much room for improvement in JMS standard. Keep messages small and optimize server setting. Use durable destinations only when you need it etc. Read documentation for your platform. But on the client side there is not much room. Some platforms offers additional features to JMS that allows some extra performance gain (batch sends etc) but it depends on the platform. I dont know IBM.
I am using Glassfish JMS.
I am able to add messages to a queue.
I can see the messages using the QueueBrowser object.
However the MessageConsumer (nor the QueueReceiver) cannot receice any message and return null.
Message expiration is set to 0 and I remember to open the connection.
Any ideas?
Here is the code:
Session session = null;
Connection conn = null;
try
{
InitialContext jndi = new InitialContext();
ConnectionFactory qFactory = (ConnectionFactory)jndi.
lookup("myConnectionFactory");
conn = qFactory.createConnection();
conn.start();
Queue queue = (Queue)jndi.lookup("myQueueName");
session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
QueueReceiver mc = ((QueueSession)session).createReceiver(queue);
Object m = mc.receive(2000);
//m is NULL!
QueueBrowser browser = session.createBrowser(queue);
for(Enumeration e = browser.getEnumeration(); e.hasMoreElements(); )
{
//there are many messages here...
}
That would be good to have the client code.
Similar thing happened to me when not properly committing/closing the connection on the sender side. The message would be visible when using the admin console, however, not available yet to the MDB.
Hope it helps.
Does this code run in the appserver? If it does, I'd obtain the required objects via annotations, and for a message receiver I'd use a MDB.
If this is a piece of standalone code, I had a hell of a time getting a JNDI based client working, I reverted to using the "raw" Java API.
I witnessed the same behavior happening after the first session commit, meaning that before the messages where received correctly. In my case the issue was that I was re-creating the receiver while keeping the same session.
As pointed out in this article:
Creating temporary destinations, consumers, producers and connections
are all synchronous request-response operations with the broker and so
should be avoided for processing each request as it results in lots of
chat with the JMS broker.
The solution was as simple as reusing the same receiver.
I'm using ActiveMQ on a simulation of overloading servers in Java. And mainly it goes ok, but when I get over 600 requests the thing just go WTF!
I think the bottleneck is my Master Server which is this guy below. I'm already reusing the connection and creating various sessions to consume messages from clients. Like I said, I'm using about 50-70 sessions per connection, reutilizing the connection and queue. Any idea of what I can reuse/optimize of my components/listener below?
The architecture is the follow:
* = various
Client ---> JMS MasterQueue ---> * Master ---> JMS SlavaQueue ---> * SlaveQueue
Mainly I'm creating a Temp Queue for each session of Master --> Slave communication, is that a big problem on performance?
/**
* This subclass implements the processing log of the Master JMS Server to
* propagate the message to the Server (Slave) JMS queue.
*
* #author Marcos Paulino Roriz Junior
*
*/
public class ReceiveRequests implements MessageListener {
public void onMessage(Message msg) {
try {
ObjectMessage objMsg = (ObjectMessage) msg;
// Saves the destination where the master should answer
Destination originReplyDestination = objMsg.getJMSReplyTo();
// Creates session and a sender to the slaves
BankQueue slaveQueue = getSlaveQueue();
QueueSession session = slaveQueue.getQueueConnection()
.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
QueueSender sender = session
.createSender(slaveQueue.getQueue());
// Creates a tempQueue for the slave tunnel the message to this
// master and also create a masterConsumer for this tempQueue.
TemporaryQueue tempDest = session.createTemporaryQueue();
MessageConsumer masterConsumer = session
.createConsumer(tempDest);
// Setting JMS Reply Destination to our tempQueue
msg.setJMSReplyTo(tempDest);
// Sending and waiting for answer
sender.send(msg);
Message msgReturned = masterConsumer.receive(getTimeout());
// Let's check if the timeout expired
while (msgReturned == null) {
sender.send(msg);
msgReturned = masterConsumer.receive(getTimeout());
}
// Sends answer to the client
MessageProducer producerToClient = session
.createProducer(originReplyDestination);
producerToClient.send(originReplyDestination, msgReturned);
} catch (JMSException e) {
logger.error("NO REPLY DESTINATION PROVIDED", e);
}
}
}
Well, After some reading I found out how to optimize.
We should reuse some session variables, such as the sender and tempqueue. Instead of creating new ones.
Another approach is put the stack size for thread in Java lower, following this link
ActiveMQ OutOfMemory Can't create more threads
It could have to do with the configuration of the listener thread pool. It could be that up to a certain threshold number of requests per second the listener is able to keep up and process the incoming requests in a timely way, but above that rate it starts to fall behind. it depends on the work done for each incoming request, the incoming request rate, the memory and CPU available to each listener, and the number of listeners allocated.
If this is true, you should be able to watch the queue and see when the number of incoming messages start to back up. That's the point at which you need to increase the resources and number of listeners to process efficiently.