In my application I have 3 classes:
- Company, which hires Workers for any of 3 jobs
- Workers, each can do 2 jobs
- Administrator, which receives copies of all messages in the program and can send messages to all companies, all workers or just everyone
I'm using work.companies.companyName for companies keys and work.workers.workerName for workers keys, they both use default exchange and queue for communication. The Administrator receives messages with admin Topic Exchange.
The problem is with the Administrator -> everyone else communication. It works exactly like Direct exchange - I can get Companies and Workers any names, even like "#", "company1.#" etc. and they won't receive anything, unless in the Administrator I explicitly send the message with key like "work.companies.company1".
I would like to be able to use just e. g. "work.companies.#" to send message to all companies. What am I doing wrong?
Administrator.java:
public class Administrator
{
public static void main(String[] args) throws IOException, TimeoutException
{
new Thread(new TopicListener("admin", ign -> {})).start();
TopicWriter writer = new TopicWriter();
// lots of code
TopicListener.java:
public class TopicListener implements Runnable
{
private final String EXCHANGE_NAME = "space";
private String key;
private Consumer<String> msgHandler;
public TopicListener(String key, Consumer<String> msgHandler)
{
this.key = key;
this.msgHandler = msgHandler;
}
#Override
public void run()
{
try
{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
String queueName = channel.queueDeclare().getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, key);
com.rabbitmq.client.Consumer consumer = new DefaultConsumer(channel)
{
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
{
String msg = new String(body, StandardCharsets.UTF_8);
System.out.println("Received: \"" + msg + "\"");
msgHandler.accept(msg);
}
};
channel.basicConsume(queueName, true, consumer);
}
catch (IOException | TimeoutException e)
{ e.printStackTrace(); }
}
}
TopicWriter.java:
public class TopicWriter
{
private final String EXCHANGE_NAME = "space";
private final Channel channel;
public TopicWriter() throws IOException, TimeoutException
{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
this.channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
}
public void send(String msg, String key) throws IOException
{
channel.basicPublish(
EXCHANGE_NAME,
key,
null,
msg.getBytes(StandardCharsets.UTF_8));
}
}
Company.java contains:
new Thread(new TopicListener("space.agencies." + agencyID, ign -> {})).start();
Worker.java contains:
new Thread(new TopicListener("space.carriers." + carrierID, consumer)).start();
I found out where the problem was: I was trying to send message to everyone using Topic, where in RabbitMQ Topic is used to specify who should receive the message. The "#" or "*" should be used in the queue key declaration, not while sending the message with a given key.
Related
I have a tcp client, which is based on the mina (V2.0.21 and J8) framework. It is working fine.
Here is the minimal example:
private static IoConnector connector;
public static void main(String[] args) throws InterruptedException {
connector = new NioSocketConnector();
connector.getFilterChain().addLast( "logger", new LoggingFilter() );
connector.getFilterChain().addLast( "codec", new ProtocolCodecFilter( new TextLineCodecFactory( Charset.forName( "UTF-8" ))));
connector.setHandler(new Handler());
try {
ConnectFuture connFuture = connector.connect(new InetSocketAddress("x.x.x.x", 9999));
connFuture.awaitUninterruptibly();
connFuture.getSession();
} catch (Exception e) {
System.err.println(e);
}
while(true) {
System.out.println("sleep.");
Thread.sleep(3000);
}
}
This is my handler:
public class Handler implements IoHandler {
#Override
public void messageReceived(IoSession session, Object message) throws Exception {
String str = (String)message;
System.out.println("->" + str);
}
#Override
public void sessionCreated(IoSession session) throws Exception {
System.out.println("CREATED.");
}
#Override
public void sessionOpened(IoSession session) throws Exception {
System.out.println("OPENED.");
}
...
}
Now, i have changed the line
connector = new NioSocketConnector();
to
connector = new NioDatagramConnector();
to be able to receive data via UDP.
If i now send packages via UDP (e.g. using a network test tool) to the port 9999 this program will not receive anything anymore. But i can see the log information, that the session was opened and created. Can somebody explain, why UDP is not working (to be more specific: messageReceived() is not called), but TCP does?
UPDATE: As a test tool i am using this method here:
public static void main(String[] args) throws IOException {
InetAddress ia = InetAddress.getByName("x.x.x.x");
int port = 9999;
String s = "Message";
byte[] data = s.getBytes();
DatagramPacket packet = new DatagramPacket( data, data.length, ia, port );
DatagramSocket toSocket = new DatagramSocket();
toSocket.send( packet );
toSocket.close();
System.out.println("Send.");
}
Thanks
Ok, the secret is to know, that in UDP case both, the "connector" and "acceptor" side is handled by the class NioDatagramAcceptor.
This piece of code does the magic for the UDP-"connector"-side:
NioDatagramAcceptor acceptor = new NioDatagramAcceptor();
acceptor.getFilterChain().addLast( "logger", new LoggingFilter() );
acceptor.getFilterChain().addLast( "codec", new ProtocolCodecFilter( new TextLineCodecFactory( Charset.forName( "UTF-8" ))));
acceptor.setHandler(new Handler());
acceptor.bind(new InetSocketAddress(9999));
I am new to rabbitMQ.I have written a java consumer in the following way.Please advice me whether it is correct implementation of the Thread+RabbitMQ .I have created three threads which consumer data from the queue and do the processing.
public class TileJobs implements Runnable {
private static final String EXCHANGE_NAME = "fanout_logs";
Connection connection;
ConnectionFactory factory;
Channel channel;
String name;
int count = 0;
TileJobs(String name) throws IOException, TimeoutException {
factory = new ConnectionFactory();
factory.setHost("192.168.2.4");
factory.setUsername("manish");
factory.setPassword("mm#1234");
connection = factory.newConnection();
channel = connection.createChannel();
this.name = name;
channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
channel.queueDeclare("test", true, false, false, null);
channel.queueBind("test", EXCHANGE_NAME, "");
channel.basicQos(1);
}
#Override
public void run() {
// TODO Auto-generated method stub
System.out.println("inside the run");
Consumer consumer = new DefaultConsumer(channel) {
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println(TileJobs.this.name);
TileJobs.this.count = TileJobs.this.count + 1;
System.out.println(TileJobs.this.count);
System.out.println(" [x] Received '" + envelope.getRoutingKey() + "':'" + message + "'");
channel.basicAck(envelope.getDeliveryTag(), false);
}
};
try {
channel.basicConsume("test", false, consumer);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public class ReceiveLogsDirect {
private static final String EXCHANGE_NAME = "fanout_logs";
public static void main(String[] argv) throws Exception {
TileJobs consumer = new TileJobs("manish");
Thread consumerThread = new Thread(consumer);
consumerThread.start();
TileJobs consumer1 = new TileJobs("manish1");
Thread consumerThread1 = new Thread(consumer1);
consumerThread1.start();
TileJobs consumer2 = new TileJobs("manish2");
Thread consumerThread2 = new Thread(consumer2);
consumerThread2.start();
}
}
Regards
Manish
You can make a class to get connection to rabbitmq server.In program use one connection multi channels to consume queue.Note that one channel for one consumer.
I'm using rabbitMQ in order to send tasks to workers (consumers) which are created on the run. Currently, each time a new task is created, a new worker is created. The problem goes like that :
-A user creates a task
-A worker is created then the task is sent on the queue for the workers to process
-The worker starts processing the queue (the worker basically sleeps for a time)
-Another user creates a task
-New worker is created and task sent on the queue
-The new worker doesn't process the new task and does absolutly nothing meanwhile, and the new task is processed by the first worker once he's done with the first task
I've checked on the admin part of rabbitmq and there are two consumers bound to the queue, but one of them seems to do all the work while the other just waits.
Here's the code for the worker:
public class Worker extends Thread {
private final static String QUEUE_NAME = "Tasks";
private final static String QUEUE_COMPL = "Completed";
public static int id = 0;
private static final String EXCHANGE_NAME = "logs";
public int compteur;
String identifier;
public Worker() {
Worker.id++;
compteur = id;
}
public void run() {
try {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicQos(1);
final Consumer consumer = new DefaultConsumer(channel) {
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println("WORKER " + compteur + " : Message received :" + message);
String taskName = message.split(" ")[0];
String length = message.split(" ")[1];
try {
System.out.println("WORKER " + compteur + " : Commencing job :" + message);
doWork(length);
System.out.println("WORKER " + compteur + " : Job's finished :" + message);
taskName += " done by " + compteur;
// confirm(taskName);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
System.out.println("WORKER " + compteur + " : Waiting for a new task...");
}
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
} catch (IOException ex) {
Logger.getLogger(Worker.class.getName()).log(Level.SEVERE, null, ex);
} catch (TimeoutException ex) {
Logger.getLogger(Worker.class.getName()).log(Level.SEVERE, null, ex);
}
}
private static void doWork(String taskLength) throws InterruptedException {
int temps = Integer.parseInt(taskLength);
Thread.sleep(temps);
}
}
and the code for the part which puts the messages into the queue:
public class serveurSD {
private final static String QUEUE_NAME = "Tasks";
private Manager MANAGER = new Manager();
#WebMethod(operationName = "processTask")
public String processTask(#WebParam(name = "message") String txt, #WebParam(name = "duree") int duree) throws IOException, TimeoutException {
if (MANAGER == null){
MANAGER= new Manager();
MANAGER.listen();
}
System.out.println("SERVER : Message received : " + txt + " " + duree);
MANAGER.giveTask();
ConnectionFactory factory = new ConnectionFactory();
String message = txt + " " + duree;
System.out.println("SERVER : Sending message to workers : " + message);
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
channel.close();
connection.close();
return "Your task is being processed";
}
}
(Manager is the class creating the workers on the go.)
I'm sorry if a similar question has already been asked but I couldn't find it.
Thanks for any possible help :)
the second parameter of the basicConsume method is "auto acknowledge". Having this parameter set to true means the consumer will tell RabbitMQ that the message has been acknowledged, as soon as it receives the message.
When the consumer is set to autoAck true, it is highly likely that it will immediately receive the next available message from the queue, even when basicQos is set to 1. this happens, because the 1 limit is immediately decremented by the consumer, to say it no longer has any message and it can accept the next one.
Changing the auto ack parameter to false prevents this problem, when combined with the basic QoS setting of 1, because it forces your consumer to say "hey, i've got a message and i'm currently working on it. don't send me anything else until i'm done."
this allows the other consumer to say "hey, i have a spot open. go ahead and send me the message"
Okay, I seem to have found a way to make it work, problem is I don't know why it works, so if anyone knows, I'd be glad to have an explanation
I changed the second argument of channel.basicConsume(QUEUE_NAME, true, consumer); to false, but I'm not sure to have understood why does it fix my problem.
I have a Spring 3 application that receives messages via a non-RabbitMQ receiver, processes them and forwards via RabbitMQ. Each time a message is to be sent a new RabbitMQ connection is built. This seems a bit wasteful. I am just wondering if this is really necessary or if there is a reason why the connection cannot be held in a Singleton and only built once (for multiple sends). This is the sending method:
private void send(String routingKey, String message) throws Exception {
String exchange = applicationConfiguration.getAMQPExchange();
String ipAddress = applicationConfiguration.getAMQPHost();
String exchangeType = applicationConfiguration.getAMQPExchangeType();
String password = applicationConfiguration.getAMQPUser();
String user = applicationConfiguration.getAMQPPassword();
String virtualHost = applicationConfiguration.getAMQPVirtualHost();
String port = applicationConfiguration.getAMQPPort();
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(user);
factory.setPassword(password);
factory.setVirtualHost(virtualHost);
factory.setPort(Integer.parseInt(port));
factory.setHost(ipAddress);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(exchange, exchangeType);
channel.basicPublish(exchange, routingKey, null, message.getBytes());
log.debug(" [AMQP] Sent message with key {} : {}",routingKey, message);
connection.close();
}
or a possible singleton:
public class MyConnection {
private static MyConnection singleton = new MyConnection();
private static Connection connection;
private MyConnection() {
ConnectionFactory factory = new ConnectionFactory();
String exchange = applicationConfiguration.getAMQPExchange();
String ipAddress = applicationConfiguration.getAMQPHost();
String exchangeType = applicationConfiguration.getAMQPExchangeType();
String password = applicationConfiguration.getAMQPUser();
String user = applicationConfiguration.getAMQPPassword();
String virtualHost = applicationConfiguration.getAMQPVirtualHost();
String port = applicationConfiguration.getAMQPPort();
try {
factory.setUsername(user);
factory.setPassword(password);
factory.setVirtualHost(virtualHost);
factory.setPort(Integer.parseInt(port));
factory.setHost(ipAddress);
connection = factory.newConnection();
}
catch (Exception e) {
e.printStackTrace();
}
}
public static Connection getInstance( ) {
return connection;
}
}
The connection could be a singleton, and you can share it for multiple send.
The channel should be one for thread.
you code could be:
private void send(String routingKey, String message) throws Exception {
Connection connection = MyConnection().getInstance();
Channel channel = connection.createChannel();
channel.exchangeDeclare(exchange, exchangeType);
channel.basicPublish(exchange, routingKey, null, message.getBytes());
log.debug(" [AMQP] Sent message with key {} : {}",routingKey, message);
channel.close();
}
You can decide to create and destroy a channel for each publish, or create it for your thread and reuse always the same channel.
EDIT**
In order to create a sigleton read here:
http://javarevisited.blogspot.it/2012/12/how-to-create-thread-safe-singleton-in-java-example.html
public class MySingletonConnection{
private static final MySingletonConnection INSTANCE = new MySingletonConnection();
private Connection myConnection;
private Singleton(){
// here you can init your connection parameter
}
public static MySingletonConnection getInstance(){
return INSTANCE;
}
public Connection getConnection( ) {
return connection;
}
}
This is one way to create a Singleton
private void send(String routingKey, String message) throws Exception {
Connection connection = MySingletonConnection().getInstance().getConnection();
I have an app which receives messages from some producer using RabbitMQ, parses them and sends somwhere else. The problem is that my app doesn't use the CPU power fully. it uses not more than 25% of the CPU. look at this screenshot from profiler:
Here is the code in which the biggest part of the processing executing:
public class Consumer {
private static final String QUEUE_NAME = "MdnaMessagesQueue";
private static int i = 1;
private final ConnectionFactory factory;
private final boolean autoAck = false;
private final Connection connection;
private final Channel channel;
private String response;
private Sender sender;
private Logger LOG = Logger.getLogger(Consumer.class);
private ExecutorService es;
public Consumer() throws IOException{
es = Executors.newFixedThreadPool(8);
factory = new ConnectionFactory();
factory.setHost("localhost");
connection = factory.newConnection(es);
channel = connection.createChannel();
sender = new Sender();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
channel.basicQos(25);
Properties props = new Properties();
try {
props.load(new FileInputStream("/home/mikhail/bzrrep/DLP/DLPServer/src/main/java/log4j.properties"));
} catch (Exception e){
LOG.error(e);
}
PropertyConfigurator.configure(props);
}
/**
* is used to receive messages from the queue
* #param customerId the id of current customer
* #throws IOException
* #throws InterruptedException
*/
public void receive(final String customerId) throws IOException,InterruptedException{
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
final QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume(QUEUE_NAME, autoAck, "myConsumerTag",
new DefaultConsumer(channel){
#Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body)
throws IOException{
BasicProperties replyProperties = new BasicProperties.Builder().correlationId(properties.getCorrelationId()).build();
long deliveryTag = envelope.getDeliveryTag();
try{
LOG.info(" [" +(i++) +"] Received from first queue ");
byte[] dataArr = body;
MimeParser mimeParser = new MimeParser(true);
Filter filter = new Filter();
ByteArrayInputStream bais1 = new ByteArrayInputStream(dataArr);
MessageInfo mi = mimeParser.parseMessages(bais1);
//checking for compliance with rules
boolean messageAccepted = filter.getMessageAcceptability(mi.getPlainText(), customerId);
response = filter.getResult();
if(messageAccepted){
//sending to the other queue
sender.send(dataArr);
channel.basicPublish("", properties.getReplyTo(), replyProperties, response.getBytes());//need to add responce
} else {
channel.basicPublish("", properties.getReplyTo(), replyProperties, response.getBytes());
}
} catch (Exception e){
LOG.error(e);
}finally {
channel.basicAck(deliveryTag, false);
}
}
});
}
}
Here is the snapshot from the profiler:
How to solve this problem?