We Keep Coding

JMSCode to send and receive messages one by one using Database

After I went to multiple sites and learned JMS I wrote a JMS standalone client to read messages from a database and send them one by one. I also want to receive message one by one message and then update the database.
I need to send a message to a queue and the other application using standard JMS which will consume a TextMessage and whose body will be read as an ISO-8859-1 string. Also they will similarly send reply as a TextMessage.
I wrote a for loop to read the message one by one from the DB.
I am new to JMS so could you please correct me whether my below code works properly to read and send messages to a queue and receive and update the DB. Is there any thing I need to change in the JMS Type or any thing I need to correct. Does the for loop work fine?
/*MQ Configuration*/
MQQueueConnectionFactory mqQueueConnectionFactory = new MQQueueConnectionFactory();
mqQueueConnectionFactory.setHostName(url);
mqQueueConnectionFactory.setChannel(channel);//communications link
mqQueueConnectionFactory.setPort(port);
mqQueueConnectionFactory.setQueueManager(qmgr);//service provider
mqQueueConnectionFactory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
/*Create Connection */
QueueConnection queueConnection = mqQueueConnectionFactory.createQueueConnection();
queueConnection.start();
/*Create session */
QueueSession queueSession = queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
/*Create response queue */
// Queue queue = queueSession.createQueue("QUEUE.RESPONSE");
int messageCount = 0;
Queue queue = queueSession.createQueue(replytoQueueName);
QueueSender queueSender = null;
QueueReceiver queueReceiver=null;
for (Testbean testBean : testbeanList) {
String testMessage = testBean.getMessage();
/*Create text message */
textMessage = queueSession.createTextMessage(testMessage);
logger.info("Text messages sent: " + messageCount);
textMessage.setJMSReplyTo(queue);
textMessage.setJMSType("mcd://xmlns");//message type
textMessage.setJMSExpiration(2*1000);//message expiration
textMessage.setJMSDeliveryMode(DeliveryMode.PERSISTENT); //message delivery mode either persistent or non-persistemnt
/*Create sender queue */
// QueueSender queueSender = queueSession.createSender(queueSession.createQueue("QUEUE.REQEST"));
queueSender = queueSession.createSender(queueSession.createQueue(outputQName));
queueSender.setTimeToLive(2*1000);
queueSender.send(textMessage);
/*After sending a message we get message id */
System.out.println("after sending a message we get message id "+ textMessage.getJMSMessageID());
String jmsCorrelationID = " JMSCorrelationID = '" + textMessage.getJMSMessageID() + "'";
/*Within the session we have to create queue reciver */
queueReceiver = queueSession.createReceiver(queue,jmsCorrelationID);
/*Receive the message from*/
Message message = queueReceiver.receive(60*1000);
// String responseMsg = ((TextMessage) message).getText();
byte[] by = ((TextMessage) message).getText().getBytes("ISO-8859-1");
logger.info(new String(by));
String responseMsg = new String(by,"UTF-8");
testDAO rmdao = new testDAO();
rmdao.updateTest(responseMsg, jmsCorrelationID);
messageCount += 1;
}
queueSender.close();
queueReceiver.close();
queueSession.close();
queueConnection.close();

Couple of things:
I would create your QueueSender and the Queue object its sending messages to outside the for loop since they don't appear to be changing.
Without the corresponding consumer code it's ultimately impossible to tell if the selector will work or not, but not invoking setCorrelationID() on the message you send looks a bit strange to me. Using the provider-assigned message ID may be a common pattern with IBM MQ request/reply applications, but the general pattern for using a correlation ID is to invoke setJMSCorrelationID() on the sent message. This makes the code more clear and also allows the application to directly control the uniqueness of the correlation ID. This is potentially important for application portability (e.g. if you migrated from IBM MQ to a different JMS provider) since different JMS providers use styles/formats of message ID specific to their particular implementation. Also, regarding the message ID the JMS spec states, "The exact scope of uniqueness is provider defined," which in my opinion is not a strong enough guarantee of uniqueness especially when using something like java.util.UUID.randomUUID().toString() is so simple.
You should ensure that you're using an XA transaction for both the JMS & database work so that they are atomic.
Close your JMS resources in a finally block.

Manual Acknowledgement of ActiveMQ Messages with Alpakka

I am working on implementing Akka Alpakka for consuming from and producing to ActiveMQ queues, in Java. I can consume from the queue successfully, but I haven't yet been able to implement application-level message acknowledgement.
My goal is to consume messages from a queue and send them to another actor for processing. When that actor has completed processing, I want it to be able control the acknowledgement of the message in ActiveMQ. Presumably this would be done by sending a message to another actor that can do the acknowledgement, calling an acknowledge function on the message itself, or some other way.
In my test, 2 messages are put into the AlpakkaTest queue, and then this code attempts to consume and acknowledge them. However, I don't see a way to set the ActiveMQ session to CLIENT_ACKNOWLEDGE, and I don't see any difference in behavior with or without the call to m.acknowledge();. Because of this, I think messages are still being auto-acknowledged.
Does anyone know the accepted way to configure ActiveMQ sessions for CLIENT_ACKNOWLEDGE and manually acknowledge ActiveMQ messages in Java Akka systems using Alpakka?
The relevant test function is:
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://0.0.0.0:2999"); // An embedded broker running in the test.
Source<Message, NotUsed> jmsSource = JmsSource.create(
JmsSourceSettings.create(connectionFactory)
.withQueue("AlpakkaTest")
.withBufferSize(2)
);
Materializer materializer = ActorMaterializer.create(system); // `system` is an ActorSystem passed to the function.
try {
List<Message> messages = jmsSource
.take(2)
.runWith(Sink.seq(), materializer)
.toCompletableFuture().get(4, TimeUnit.SECONDS);
for(Message m:messages) {
System.out.println("Found Message ID: " + m.getJMSMessageID());
try {
m.acknowledge();
} catch(JMSException jmsException) {
System.out.println("Acknowledgement Failed for Message ID: " + m.getJMSMessageID() + " (" + jmsException.getLocalizedMessage() + ")");
}
}
} catch (InterruptedException e1) {
e1.printStackTrace();
} catch (ExecutionException e1) {
e1.printStackTrace();
} catch (TimeoutException e1) {
e1.printStackTrace();
} catch (JMSException e) {
e.printStackTrace();
}
This code prints:
Found Message ID: ID:jmstest-43178-1503343061195-1:26:1:1:1
Found Message ID: ID:jmstest-43178-1503343061195-1:27:1:1:1

Update: The acknowledgement mode is configurable in the JMS connector since Alpakka 0.15. From the linked documentation:
Source<Message, NotUsed> jmsSource = JmsSource.create(JmsSourceSettings
.create(connectionFactory)
.withQueue("test")
.withAcknowledgeMode(AcknowledgeMode.ClientAcknowledge())
);
CompletionStage<List<String>> result = jmsSource
.take(msgsIn.size())
.map(message -> {
String text = ((ActiveMQTextMessage)message).getText();
message.acknowledge();
return text;
})
.runWith(Sink.seq(), materializer);
As of version 0.11, Alpakka's JMS connector does not support application-level message acknowledgment. Alpakka creates internally a Session with the CLIENT_ACKNOWLEDGE mode here and acknowledges each message here in the internal MessageListener. The API does not expose these settings for overriding.
There is an open ticket that discusses enabling downstream acknowledgement of queue-based sources, but that ticket has been inactive for a while.
Currently you cannot prevent Alpakka from acknowledging the messages at the JMS level. However, that doesn't preclude you from adding a stage to your stream that sends each message to an actor for processing and uses the actor's replies as backpressure signals. The Akka Streams documentation describes how to do this with either a combination of mapAsync and ask or with Sink.actorRefWithAck. For example, to use the former:
Timeout askTimeout = Timeout.apply(4, TimeUnit.SECONDS);
jmsSource
.mapAsync(2, msg -> ask(processorActor, msg, askTimeout))
.runWith(Sink.seq(), materializer);
(Side note: In the related Streamz project, there is a recently opened ticket to allow application-level acknowledgement. Streamz is the replacement for the old akka-camel module and, like Alpakka, is built on Akka Streams. Streamz also has a Java API and is listed in the Alpakka documentation as an external connector.)

Looking at the source code for the Alpakka JmsSourceStage it already acknowledges each incoming message for you (and it's session is a Client Ack session). From what I can tell from the source there is no mode that allows you to do the acknowledgement of messages.
You can view the source code for Alpakka here.

IBM MQ failure to send/receive all JMS Messages

I am using IBM MQ to produce messages while receiving it through a consumer on my client. To create the connection I'm using JmsConnectionFactory, along with provided properties to set up the connection with the server. So from what I understand is, as the consumer the only way to recognize the messages produced by the server is through the onMessage call. I'm currently testing this by creating a local producer and local consumer and assuring that every message sent by the producer is received by the consumer.
I'm running into the following problems:
I'm not receiving all messages produced.
Depending on the size of the message, more of them are received if they are smaller.
Here is code for the creation of the producer:
JmsConnectionFactory cf = ff.createConnectionFactory();
cf.setStringProperty(WMQConstants.WMQ_HOST_NAME, qm.getHost());
int port = ###;
cf.setIntProperty(WMQConstants.WMQ_PORT, port);
cf.setStringProperty(WMQConstants.WMQ_CHANNEL, qm.getChannel());
cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT);
cf.setStringProperty(WMQConstants.WMQ_QUEUE_MANAGER, qm.getQueueManagerName());
Connection connection = cf.createConnection(qm.getUser().getUsername(), qm.getUser().getPassword());
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue(qm.getDestinationName());
LOG.debug("Destination Created at " +qm.getDestinationName());
msgSender = session.createProducer(destination);
msgSender.setDeliveryMode(DeliveryMode.PERSISTENT);
And this is how the producer is sending messages:
/**
* msgSender is the MessageProducer object
**/
private void produceMessages(int numOfMessages) throws JMSException, InterruptedException {
for (int i = 0; i < numOfMessages; i++) {
String text = "Message #" +i;
TextMessage message = session.createTextMessage(text);
msgSender.send(message);
}
}
On the consumer side, I am simply printing received messages and verifying visually:
#Override
public void onMessage(Message m) {
System.out.println(((TextMessage)m).getText());
}
I am not fully familiar with how IBM MQ works. Could the reason for the missing messages reside on the MQ simply ignoring messages that are produced before a message is fully sent?

I would say the issue is residing on your consumer side, rather than your simulated producer. Your message producer should be sending messages to MQ just fine, but multiple consumers are probably competing to retrieve these messages from the connection you have set up (given the same queue manager properties). So unless no one else is trying to consume from your IBM MQ, you're going to be expected to miss some messages.

You should use other method of send(Message m, CompletionListener l) to send new messages only after completion.
And if you use "Best Effort", it still will lose messages. You can try "Express" instead.

OpenMQ and JMX - Is it possible to query for messages?

I am able to query for queues by invoking a GET_DESTINATIONS operation using JMX. With that I will receive the queue info (attributes).
I would like now to query the messages that are stored in this queue, is that possible? Could someone give me some direction?
I have already tried using this code
ConnectionFactory connectionFactory = new
com.sun.messaging.QueueConnectionFactory();
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue myQueue = session.createQueue(string);
QueueBrowser browser = session.createBrowser(myQueue);
Enumeration msgs = browser.getEnumeration();
if (!msgs.hasMoreElements()) {
System.out.println("No messages in queue");
} else {
while (msgs.hasMoreElements()) {
Message tempMsg = (Message) msgs.nextElement();
System.out.println("Message: " + tempMsg);
}
}
connection.close();
But for some reason O can't access the same queue as using JMX. I didn't made any research on that because I want to use JMX as the access standard.
I am still trying to find any JMX operation that could help me, but I am not finding anything that could help me.
Could you please give me some hints what can I look for?
thank you,
Oscar
Edit: just to let you know: I don't want to consume the queues, I want a similar behavior to the Browser, in which I can read the messages without removing them from the queue.

QueueBrowser browser = null;
try{
Queue myQueue = session.createQueue(getName());
//Create the browser and session to be able to iterate
browser = session.createBrowser(myQueue);
Enumeration msgs = browser.getEnumeration();
This code will give you the messages, then just iterate through it and you can get get infos about the message and its content

How to optimize activemq

I'm using ActiveMQ on a simulation of overloading servers in Java. And mainly it goes ok, but when I get over 600 requests the thing just go WTF!
I think the bottleneck is my Master Server which is this guy below. I'm already reusing the connection and creating various sessions to consume messages from clients. Like I said, I'm using about 50-70 sessions per connection, reutilizing the connection and queue. Any idea of what I can reuse/optimize of my components/listener below?
The architecture is the follow:
* = various
Client ---> JMS MasterQueue ---> * Master ---> JMS SlavaQueue ---> * SlaveQueue
Mainly I'm creating a Temp Queue for each session of Master --> Slave communication, is that a big problem on performance?
/**
* This subclass implements the processing log of the Master JMS Server to
* propagate the message to the Server (Slave) JMS queue.
*
* #author Marcos Paulino Roriz Junior
*
*/
public class ReceiveRequests implements MessageListener {
public void onMessage(Message msg) {
try {
ObjectMessage objMsg = (ObjectMessage) msg;
// Saves the destination where the master should answer
Destination originReplyDestination = objMsg.getJMSReplyTo();
// Creates session and a sender to the slaves
BankQueue slaveQueue = getSlaveQueue();
QueueSession session = slaveQueue.getQueueConnection()
.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
QueueSender sender = session
.createSender(slaveQueue.getQueue());
// Creates a tempQueue for the slave tunnel the message to this
// master and also create a masterConsumer for this tempQueue.
TemporaryQueue tempDest = session.createTemporaryQueue();
MessageConsumer masterConsumer = session
.createConsumer(tempDest);
// Setting JMS Reply Destination to our tempQueue
msg.setJMSReplyTo(tempDest);
// Sending and waiting for answer
sender.send(msg);
Message msgReturned = masterConsumer.receive(getTimeout());
// Let's check if the timeout expired
while (msgReturned == null) {
sender.send(msg);
msgReturned = masterConsumer.receive(getTimeout());
}
// Sends answer to the client
MessageProducer producerToClient = session
.createProducer(originReplyDestination);
producerToClient.send(originReplyDestination, msgReturned);
} catch (JMSException e) {
logger.error("NO REPLY DESTINATION PROVIDED", e);
}
}
}

Well, After some reading I found out how to optimize.
We should reuse some session variables, such as the sender and tempqueue. Instead of creating new ones.
Another approach is put the stack size for thread in Java lower, following this link
ActiveMQ OutOfMemory Can't create more threads

It could have to do with the configuration of the listener thread pool. It could be that up to a certain threshold number of requests per second the listener is able to keep up and process the incoming requests in a timely way, but above that rate it starts to fall behind. it depends on the work done for each incoming request, the incoming request rate, the memory and CPU available to each listener, and the number of listeners allocated.
If this is true, you should be able to watch the queue and see when the number of incoming messages start to back up. That's the point at which you need to increase the resources and number of listeners to process efficiently.