I am Using Amazon Mq as my Mqtt broker and when around 1000 requests are received simultaneously the mqtt broker breaks and disconnects. Can Anyone tell me how to use Amazon Mq as my broker & simultaneously solve the scaling problem also.
I'm assuming that you have created ActiveMQ as a singleton class. Right?
-For producing a message, you create an instance of PooledConnectionFactory like
-------//some code here
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(MQTT_END_POINT);
connectionFactory.setUserName();
connectionFactory.setPassword();
PooledConnectionFactory pooledConnectionFactory = getActiveMQInstance().configurePooledConnectionFactory(activeMQConnectionFactory);
-------
This pooledConnectionFactory is used to create a connection then session and then destination is entered (as mentioned on AmazonMQ documentation). You send the message using MessageProducer object and close the MessageProducer, session and connection
-For consumption, there will be an always-alive-listener that is always ready for message to arrive. The consumer part, it follows the same process like consumerConnection, then session and then destination queue to listen on.
As far as I remember, this part is also mentioned in amazonMQ documentation.
There is one problem that the connection to broker is lost for consumer sometimes, (since producer reopens the connections, produces and closes, it is not observed in it). Remember, you will have to reestablish the connection for consumer.
If there is any variance from the above approach please mention. Also, add your amazonMQ broker picture showing the connection, queue, active consumers.
Just out of curiosity, what are the maximum connections you have set for the PooledConnectionFactory?
Related
In my Java application I am using the failover transport to connect to a local ActiveMQ broker:
failover:(tcp://0.0.0.0:61616)
I create one single connection that I reuse in the rest of the application:
ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
In another part of the application when I receive some external call I need to send a message to the broker, and so, for doing that I create a new "Session":
Session locSession = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
When the broker is down my app tries to reconnect to the broker forever (this is the expected behavior I really want to have).
However, the problem is that if the broker is down and I receive a call that invokes the code that executes the connection.createSession(false, Session.AUTO_ACKNOWLEDGE) then my app hangs forever on this line of code waiting for the app to reconnect successfully to the broker and then create the session.
Please, do you know any way to check before I execute createSession if the connection object is trying to reconnect or it is really connected? If I am able to know this I could avoid the creation of the session if the app is not connected to the broker (only trying to reconnect) and therefore I would avoid to hanging on connection.createSession forever (I would raise an exception).
I wasn't able to find any property or method on ActiveMQConnection to gather this information.
The failover: url provides a setting startupMaxReconnectAttempts to prevent infinite retry when connecting to the broker the first time.
Also note-- If you want an exception to bubble up, that conflicts with requirement to have infinite retry. You would need to adjust the failover settings to match your intended behavior, by setting a max count or max time to perform retry, then throw an exception and unblock your caller.
For example, you could indicate you only want to retry for 5 minutes, then receive an exception to handle in the code to prevent the infinite blocking.
Thank you all for your help and suggestions. They helped me a lot in re-focusing the problem.
However I f found the answer to my question using the method "getTransport().isConnected()".
I am trying to figure out how to conveniently pause all consumers/message-listeners, while my application is in controlled maintenance mode. The application is using ActiveMQ 5.13.3 client libraries.
Some time ago I have switched from a single ActiveMQConnectionFactory to a PooledConnectionFactory. It is being setup like so:
ActiveMQConnectionFactory amcf = new ActiveMQConnectionFactory(config.getMessageBrokerUrl());
amcf.setTrustedPackages(Arrays.asList(new String[] {
"some.package.or.other",
"java.lang",
"java.util"
}));
connectionFactory = new PooledConnectionFactory(amcf);
connectionFactory.setCreateConnectionOnStartup(true);
Consumers and producers "create" (= fetch) a connection from the connection pool and "close" it when they are done, returning it to the pool. Obviously in the case of MessageListeners, it is obtained once at startup and returned on application shutdown.
ActiveMQConnection.stop() says it Temporarily stops a connection's delivery of incoming messages. Perfect for what I want, only the pool obviously contains many connections, not just one.
How do you pause all connections of an ActiveMQ connection pool?
I guess you have to resort to other means of pausing the message delivery using pooled connection pools. See this question for example when using spring DMLC (may not be the case for you): Start and Stop JMS Listener using Spring
You can also pause that queue from the broker side. There is a pause/resume operation on the JMX MBean of the queue. See attached screenshot.
It does not answer the question about pausing the client, but may solve your problem.
How can I check whether a queue exists on a JMS server using the Java API? I don't want to send or receive any data to the queue for now, just verify that the queue exists. Also, the queue may be empty.
Here is my code sample. I have removed the error handling for simplicity.
Connection connection = null;
Session session = null;
connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
//I was hoping this next line would throw an exception if the queue does not exist
Queue queue = session.createQueue(queueName);
My JMS server is TIBCO EMS. I'm hoping for a solution that works on versions 5-7.
Solution
I followed the recommendation in the accepted answer but created a browser instead. The following line threw an exception as desired:
QueueBrowser browser = session.createBrowser(queue);
This is dependent on the provider, but you wont know in most cases until you create the session type, such as session.createConsumer. Simply creating a consumer this way will not consume any messages until you do a receive. And it is here the behavior may change from provider to provider and configuration of the server.
For example with ActiveMQ, assuming there are no permissions blocking the user you are connecting with, the queue is created automatically when you create the session type.
With WebSphere MQ, the queue has to be defined by an admin. If it does not exist, the queue manager will return an exception with a reason code of 2085 (UNKNOWN_OBJECT_NAME).
Outside of this, you'd need to see if the particular provider had a way to access a list of queues. Using the above examples, ActiveMQ you can get the list of queues using JMX, with WebSphere MQ, you can do this if you have permissions to send PCF commands to the queue manager.
Try creating a consumer or producer off the Session passing in the queue object you just created:
session.createConsumer(queue);
This should throw an InvalidDestinationException if the queue (or topic) does not exist.
According to this config page on the ActiveMQ site, the connection.sendTimeout property is:
Time to wait on Message Sends for a Response, default value of zero indicates to wait forever. Waiting forever allows the broker to have flow control over messages coming from this client if it is a fast producer or there is no consumer such that the broker would run out of memory if it did not slow down the producer. Does not affect Stomp clients as the sends are ack'd by the broker. (Since ActiveMQ-CPP 2.2.1)
I'm having difficulty interpreting what this means (and what the sendTimeout property really is/what it does):
What is a "Message Sends" object?
Why would ActiveMQ be waiting for a response? Isn't it on the server-side of a JMS connection? Shouldn't it be waiting for a request?
What does it actually timeout? When should it be used?
Thanks in advance!
The timeout affects the send of a Message by the client to the Broker. In the case where a send is not async then the client waits for the Broker to return a response indicating that the Message was received and added to the Message store. In some cases this can block for a long time if the Broker has engaged producer flow control because one of its preset memory limits has been reached. If the client application can't tolerate a long wait on send it could configure this timeout so that MessageProducer::send doesn't indefinitely block.
Messages are sent in synchronous mode either because the Connection was configured with alwaysSyncSend=true or because the MessageProducer is sending with the delivery mode set to Persistent.
In general this setting shouldn't need to be used if you've configured your Broker with limits that match your use case.
I have a JBoss-6 server with HornetQ and a single queue:
<queue name="my.queue">
<entry name="/queue/test"/>
</queue>
There a different consumers (on different machines) connected to this queue, but only a single consumer is active at a time. If I shut down this consumer, the messages are immediately processed by one of the other consumers.
Since my messages have some time consuming processing, I want multiple consumer process their unique messages concurrently.
I remember a similar in earlier versions of JBoss where this setup worked without problems. Here in Jboss-6 the messaging system is working well -- except of the issue described above. This question is similar to Are multiple client consumers possible in hornetq?, but the scenario is not similar to mine.
Update 1: If I close (STRG+C) one consumer there is a short timeout (until the server recognized the lost consumer) until the next consumer gets the message.
Update 2: Code Snippet
VoidListener ml = new VoidListener();
QueueConnectionFactory qcf = (QueueConnectionFactory)
ctx.lookup("ConnectionFactory");
QueueConnection conn = qcf.createQueueConnection();
Queue queue = (Queue) ctx.lookup(queueName);
QueueSession session = conn.createQueueSession(false,
QueueSession.AUTO_ACKNOWLEDGE);
QueueReceiver recv = session.createReceiver(queue,"");
recv.setMessageListener(ml);
conn.start();
And the MessageListerner:
public class OlVoidListener implements MessageListener
{
public void onMessage(Message msg)
{
counter++;
logger.debug("Message ("+counter+") received");
try {Thread.sleep(15*1000);} catch (InterruptedException e) {}
}
}
With multiple consumers on a queue, messages are load balanced between the consumers.
As you have some time consuming the message, you should disable buffering by setting consumer-window-size.
On hornetQ there's an example on the distribution, about how to disable client buffering and give a better support for slow consumers. (a slow consumer is a consumer that will have some time processing the message)
message systems will pre-fetch/read-ahead messages to the client buffer to speed up processing and avoid network latency. This is not an issue if you have fast processing queues and a single consumer.
JBoss Messaging offered the slow-consumer option at the connection factory and hornetq offers the consumer window size.
Most Message systems will provide you a way to enable or disable client pre-fetching.
I am sorry but I cannot understand what exactly the problem is. We've used hornetq in 2.0.0.GA version and 2.2.2.Final. In both cases, queue-based load balancing works fine. If you will define multiple consumers for one queue and all of them are active, messages will be distributed between them automatically. First message to consumer A, second to consumer B, third to consumer C and so on. This is how queues with multiple consumers works - it's free load balancing :) That's normal that when you shut down one consumer, others would receive more messages.