We Keep Coding

RabbitMQ Java Consumer app best practice for Ubuntu linux deployment

I'm studing how RabbitMQ works with Java, it is almost clear how Producer works and how to implement the Consumer, but I'm still not sure how to deploy the Consumer application to server with a correct threading handling.
Let's say I have a web-application that has to sent some transaction email as producer, this application will push in the queue the messages and it is managed by a container, like Tomcat, that increase/descrease threads to serve multiple requests.
What is the best practice to deployt the consumer application?
I have read many tutorials and the RabbitMQ commands are well explained, usually this is the code (from official RabbitMq Hello World):
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.DeliverCallback;
import java.nio.charset.StandardCharsets;
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
String message = new String(delivery.getBody(), StandardCharsets.UTF_8);
System.out.println(" [x] Received '" + message + "'");
};
channel.basicConsume(QUEUE_NAME, true, deliverCallback, consumerTag -> { });
}
}
However it is not clear to me how I should run this application in my server (sure I can start it by command line!) and how am I sure the process will not crash for some reason?
Should I put the channel opening commands in a loop inside a try/catch to be sure it will not exit?
Am I'm in charge of handling thread pool opening/increasing?
Does exists something for this purpose?

How to start ActiveMQ when tomcat starts?

How do I configure my J2EE application so that I can invoke ActiveMQ service along with tomcat server? I am aware about embedded broker, here asking how to start the ActiveMQ whenever I start tomcat
Current Code (works fine) :
Now I want to remove main() method and use the code to run when tomcat runs.
public class JMSService {
public void produceJMS() throws NamingException, JMSException {
ConnectionFactory connFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection conn = connFactory.createConnection();
conn.start();
Session session = conn.createSession(false,Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("testQueue");
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
TextMessage message = session.createTextMessage("Test Message ");
// send the message
producer.send(message);
System.out.println("sent: " + message);
}}
Here is my consumer :
public class JMSReceiver implements MessageListener,ExceptionListener {
public static void main(String args[]) throws Exception {
JMSReceiver re = new JMSReceiver();
re.receiveJMS();
}
public void receiveJMS() throws NamingException, JMSException {
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
// Getting the queue 'testQueue'
Destination destination = session.createQueue("testQueue");
MessageConsumer consumer = session.createConsumer(destination);
// set an asynchronous message listener
JMSReceiver asyncReceiver = new JMSReceiver();
consumer.setMessageListener(asyncReceiver);
connection.setExceptionListener(asyncReceiver);
}
#Override
public void onMessage(Message message) {
System.out.println("Received message : " +message);
}
}

What #Tim Bish said is correct. You either need to have a timer say for example receiver should listen for 1 hour- or make it available until program terminate. Either case you need to start your consumer program once:
Change your receiveJMS method as follows:
public void receiveJMS() throws NamingException, JMSException {
try{
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
Connection connection = connectionFactory.createConnection();
connection.start(); // it's the start point
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
// Getting the queue 'testQueue'
Destination destination = session.createQueue("testQueue");
MessageConsumer consumer = session.createConsumer(destination);
// set an asynchronous message listener
// JMSReceiver asyncReceiver = new JMSReceiver();
//no need to create another object
consumer.setMessageListener(this);
connection.setExceptionListener(this);
// connection.close(); once this is closed consumer no longer active
Thread.sleep(60 *60 * 1000); // receive messages for 1 hour
}finally{
connection.close();// after 1 hour close it
}
}
The above program will listen upto 1 hour. If you want it as long as the program run, remove the finally block. But the recommended way is to close it somehow. since your application seems to be standalone ,you can check the java runtime shutdown hook, where you can specify how to release such resources while program terminates.
If your consumer is a web application you can close it in a ServletContextlistner.

You aren't giving the consumer application any time to actually receive a message, you create it, then you close it. You either need to use a timed receive call to do an sync receive of the message from the Queue or you need to add some sort of wait in the main method such as a CountDownLatch etc to allow the async onMessage call to trigger shutdown once processing of the message is complete.

Netty 4.0.23 multiple hosts single client

My question is about creating multiple TCP clients to multiple hosts using the same event loop group in Netty 4.0.23 Final, I must admit that I don't quite understand Netty 4's client threading business, especially with the loads of confusing references to Netty 3.X.X implementations I hit through my research on the internet.
with the following code, I establish a connection with a single server, and send random commands using a command queue:
public class TCPsocket {
private static final CircularFifoQueue CommandQueue = new CircularFifoQueue(20);
private final EventLoopGroup workerGroup;
private final TcpClientInitializer tcpHandlerInit; // all handlers shearable
public TCPsocket() {
workerGroup = new NioEventLoopGroup();
tcpHandlerInit = new TcpClientInitializer();
}
public void connect(String host, int port) throws InterruptedException {
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.remoteAddress(host, port);
b.handler(tcpHandlerInit);
Channel ch = b.connect().sync().channel();
ChannelFuture writeCommand = null;
for (;;) {
if (!CommandQueue.isEmpty()) {
writeCommand = ch.writeAndFlush(CommandExecute()); // commandExecute() fetches a command form the commandQueue and encodes it into a byte array
}
if (CommandQueue.isFull()) { // this will never happen ... or should never happen
ch.closeFuture().sync();
break;
}
}
if (writeCommand != null) {
writeCommand.sync();
}
} finally {
workerGroup.shutdownGracefully();
}
}
public static void main(String args[]) throws InterruptedException {
TCPsocket socket = new TCPsocket();
socket.connect("192.168.0.1", 2101);
}
}
in addition to executing commands off of the command queue, this client keeps receiving periodic responses from the serve as a response to an initial command that is sent as soon as the channel becomes active, in one of the registered handlers (in TCPClientInitializer implementation), I have:
#Override
public void channelActive(ChannelHandlerContext ctx) {
ctx.writeAndFlush(firstMessage);
System.out.println("sent first message\n");
}
which activates a feature in the connected-to server, triggering a periodic packet that is returned from the server through the life span of my application.
The problem comes when I try to use this same setup to connect to multiple servers,
by looping through a string array of known server IPs:
public static void main(String args[]) throws InterruptedException {
String[] hosts = new String[]{"192.168.0.2", "192.168.0.4", "192.168.0.5"};
TCPsocket socket = new TCPsocket();
for (String host : hosts) {
socket.connect(host, 2101);
}
}
once the first connection is established, and the server (192.168.0.2) starts sending the designated periodic packets, no other connection is attempted, which (I think) is the result of the main thread waiting on the connection to die, hence never running the second iteration of the for loop, the discussion in this question leads me to think that the connection process is started in a separate thread, allowing the main thread to continue executing, but that's not what I see here, So what is actually happening? And how would I go about implementing multiple hosts connections using the same client in Netty 4.0.23 Final?
Thanks in advance

ConsoleAppender is blocking

I am not sure what is happening here. I am starting a RMI server in a separate JVM. While connecting to the instance calling the remote method, the method get stuck after a very short time. The execution continues as soon as I shutdown the client process.
What am I doing wrong?
class Client
...
//name some kind of name
String name= "HelloService";
//libname points to a runnable jar with the server class as main class
ProcessBuilder jvm= new ProcessBuilder(javaPath, "-jar", libname, "-n", name);
jvm.start();
//Waiting for RMI server to start
try { Thread.sleep(10000); } catch ...
try {
Registry registry= LocateRegistry.getRegistry(1199);
//String as input and String as output
IRemoteService<String, String> service= (IRemoteService<String, String>) registry.lookup(name)
String returnVal= service.execute("SomeValue");
return returnVal;
} catch ...
Following by the server code snip. The server code is packed in a runnable jar with itself as the MainClass.
class Server implements IRemoteService<String, String>
//Is not returning a value, due the fact that I liked to examine the behaviour of
//this method. Doing this by running an infinite loop.
public String execute(String str) {
log.info("Running doExectue from "+getClass().getSimpleName());
int i=0;
while(true) {
i++;
log.info(String.valueOf(i));
}
}
protected static void register(String name, IRemoteService service) {
try {
IRemoteService rsStub= (IRemoteService) UnicastRemoteObject.exportObject(service,0);
Registry registry= LocateRegistry.getRegistry(1199);
try {
registry.bind(name, rsStub);
} catch (ConnectException ce) {
registry= LocateRegistry.createRegistry(1199);
registry.bind(name, rsStub);
}
} catch ...
}
public static void main(String[] args) {
String rmiName= args[1];
IRemoteService<String, String> service= (IRemoteService<String, String>) new Server();
register(rmiName, service);
}
Now if I start the client the log file displayes 36 runs of the loop in method "execute". Than it stops. There is no other client getting this object or calling this method too.
It starts a again and is running forever as soon as I killed the Client process.
For me it looks like that the client is blocking the execution of the remote server methods. But I have no clue how to overcome this situation.
Thanks for any help in advance,
Danny

What you describe is impossible. The client can't block the server while the server is executing a remote method. Find another explanation.

thanks for your support. You are right a RMI client can't block the server. So I was really confused. But I found the failure.
It about the process is writing to the console. As soon as the buffer is full the process is stopping to wait for someone collecting the output.
After I removed the ConsoleAppender from the log configuration the job runs as expected.

Restart embedded broker in unit test : VMTransportServer already bound

I'm trying to write a test that simulates a "broker down" phase.
Therefore I want to
start a local broker
send message1
stop the broker
send message2 (which will of course not arrive)
start the broker again
send message3
According to http://activemq.apache.org/how-do-i-restart-embedded-broker.html it is recommended to init a new BrokerService to start the broker again.
So the code looks (almost) like this:
private BrokerService _broker;
private void startBroker() throws Exception {
_broker = new BrokerService();
_broker.addConnector("vm://localhost?broker.persistent=false");
_broker.start();
_broker.waitUntilStarted();
}
private void stopBroker() throws Exception {
_broker.stop();
_broker.waitUntilStopped();
}
#Test
public void publishMessagesWithServerBreakdownInBetween()
throws Exception
{
startBroker();
... send and receive message (works fine)
stopBroker();
... send message (fails of course)
startBroker(); // this fails with java.io.IOException: VMTransportServer already bound at: vm://localhost?broker.persistent=false
... send and receive message
}
The problem is already mentioned as comment in code:
The restart of the broker fails due to the error : java.io.IOException: VMTransportServer already bound at: vm://localhost?broker.persistent=false
I found a similar problem at ActiveMQ forum (http://activemq.2283324.n4.nabble.com/VMTransportServer-already-bound-td2364603.html), but in my case the hostname isn't null.
Another idea was to set 2 different broker names, but that also didn't help.
What am I doing wrong?

You want to control what the VM Transport does by telling it not to try and create a broker for you since you are adding it to an already created broker. The rest is pretty simply then:
public class AMQRestartTest {
private BrokerService broker;
private String connectorURI;
private ActiveMQConnectionFactory factory;
#Before
public void startBroker() throws Exception {
createBroker(true);
factory = new ActiveMQConnectionFactory("failover://" + connectorURI);
}
private void createBroker(boolean deleteAllMessages) throws Exception {
broker = new BrokerService();
TransportConnector connector = broker.addConnector("vm://localhost?create=false");
broker.setPersistent(false);
broker.start();
broker.waitUntilStarted();
connectorURI = connector.getConnectUri().toString();
}
#Test(timeout = 60_000)
public void test() throws Exception {
Connection connection = factory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("test");
MessageConsumer consumer = session.createConsumer(queue);
MessageProducer producer = session.createProducer(queue);
connection.start();
broker.stop();
broker.waitUntilStopped();
createBroker(false);
producer.send(session.createTextMessage("help!"));
Message received = consumer.receive();
assertNotNull(received);
assertTrue(received instanceof TextMessage);
}
}