We Keep Coding

RabbitMQ using Direct Exchange when Topic was specified

In my application I have 3 classes:
- Company, which hires Workers for any of 3 jobs
- Workers, each can do 2 jobs
- Administrator, which receives copies of all messages in the program and can send messages to all companies, all workers or just everyone
I'm using work.companies.companyName for companies keys and work.workers.workerName for workers keys, they both use default exchange and queue for communication. The Administrator receives messages with admin Topic Exchange.
The problem is with the Administrator -> everyone else communication. It works exactly like Direct exchange - I can get Companies and Workers any names, even like "#", "company1.#" etc. and they won't receive anything, unless in the Administrator I explicitly send the message with key like "work.companies.company1".
I would like to be able to use just e. g. "work.companies.#" to send message to all companies. What am I doing wrong?
Administrator.java:
public class Administrator
{
public static void main(String[] args) throws IOException, TimeoutException
{
new Thread(new TopicListener("admin", ign -> {})).start();
TopicWriter writer = new TopicWriter();
// lots of code
TopicListener.java:
public class TopicListener implements Runnable
{
private final String EXCHANGE_NAME = "space";
private String key;
private Consumer<String> msgHandler;
public TopicListener(String key, Consumer<String> msgHandler)
{
this.key = key;
this.msgHandler = msgHandler;
}
#Override
public void run()
{
try
{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
String queueName = channel.queueDeclare().getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, key);
com.rabbitmq.client.Consumer consumer = new DefaultConsumer(channel)
{
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
{
String msg = new String(body, StandardCharsets.UTF_8);
System.out.println("Received: \"" + msg + "\"");
msgHandler.accept(msg);
}
};
channel.basicConsume(queueName, true, consumer);
}
catch (IOException | TimeoutException e)
{ e.printStackTrace(); }
}
}
TopicWriter.java:
public class TopicWriter
{
private final String EXCHANGE_NAME = "space";
private final Channel channel;
public TopicWriter() throws IOException, TimeoutException
{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
this.channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
}
public void send(String msg, String key) throws IOException
{
channel.basicPublish(
EXCHANGE_NAME,
key,
null,
msg.getBytes(StandardCharsets.UTF_8));
}
}
Company.java contains:
new Thread(new TopicListener("space.agencies." + agencyID, ign -> {})).start();
Worker.java contains:
new Thread(new TopicListener("space.carriers." + carrierID, consumer)).start();

I found out where the problem was: I was trying to send message to everyone using Topic, where in RabbitMQ Topic is used to specify who should receive the message. The "#" or "*" should be used in the queue key declaration, not while sending the message with a given key.

channel is already closed due to channel error rabbitmq

I am triying to develop a publish/subscribe system using RabbitMQ. The RabbitMQ server is in my laptop.
I have coded the Consumer and Producer Classes. The producer is connected correctly to the broker, but I have a com.rabbitmq.client.ShutdownSignalException when I run the consumer class.
My code is as follow.
I have a Functions.class :
public class Functions {
public static String EXCHANGE_NAME = "exchange";
private static ConnectionFactory factory;
/**
* Initialization for ConnectionFactory
*/
static {
factory = new ConnectionFactory();
factory.setAutomaticRecoveryEnabled(true);
factory.setHost("localhost");
}
public static Connection createConnection() {
try {
return factory.newConnection();
} catch (Exception e) {
e.printStackTrace();
return null;
}
}}
Then the MyCosumer.java Class:
public class MyConsumer {
private Connection connection;
private Channel channel;
private String queueName = "consumer";
public MyConsumer() {
}
public void connect() throws IOException {
connection = Functions.createConnection();
channel = connection.createChannel();
channel.exchangeDeclare(Functions.EXCHANGE_NAME, BuiltinExchangeType.TOPIC);
channel.queueDeclare(queueName, true, false, false, new HashMap<String, Object>()).getQueue();
}
public void setRoutingKeys(String... keys) throws IOException {
for (String bindingKey : keys) {
channel.queueBind(queueName, Functions.EXCHANGE_NAME, bindingKey);
}
}
public void clearRoutingKeys(String... keys) throws IOException {
for (String bindingKey : keys) {
channel.queueUnbind(queueName, Functions.EXCHANGE_NAME, bindingKey);
}
}
public void recieveMessages() throws IOException {
Consumer consumer = new DefaultConsumer(channel) {
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + envelope.getRoutingKey() + "':'" + message + "'");
}
};
channel.basicConsume(queueName, true, consumer);
}
public void disconnect() throws IOException, TimeoutException {
channel.close();
connection.close();
}
}
And the MyProducer.java Class :
public class MyProducer {
private Connection connection;
private Channel channel;
public MyProducer() {
}
public void connect() throws Exception {
connection = Functions.createConnection();
channel = connection.createChannel();
channel.exchangeDeclare(Functions.EXCHANGE_NAME, "topic");
}
public void sendMessage(String routingKey, String message) throws IOException {
channel.basicPublish(Functions.EXCHANGE_NAME, routingKey, null, message.getBytes());
System.out.println(" [x] Sent '" + routingKey + "':'" + message + "'");
}
public void disconnect() throws IOException, TimeoutException {
channel.close();
connection.close();
}}
The main classes are as follow :
the Producer main:
public class MainProducer {
public static String[] categorias = { "Beauty", "Drinks", "Diary", "Vegetable" };
public static void main(String[] args) throws Exception {
MyProducer productor = new MyProducer();
productor.connect();
send(productor, categorias[1], "Makeup", "Discount 20% in loreal lipstick");
send(productor, categorias[0], "Soap", "2x1 in Herbal Essence soap");
send(productor, categorias[0], "Makeup", "Discount 10% in l'ocitane lipstick");
send(productor, categorias[0], "Loreal Makeup", "Discount 20% in loreal lipstick");
productor.disconnect();
}
private static void send(MyProducer productor, String categoria, String producto, String mensaje) {
String routingKey = "offers." + categoria + "." + producto;
try {
productor.sendMessage(routingKey, mensaje);
} catch (IOException e) {
e.printStackTrace();
}
}}
THe consumer Main:
public class MainConsumer {
public static void main(String[] args) throws Exception {
MyConsumer consumer = new MyConsumer();
consumer.connect();
consumer.setRoutingKeys("kern");
consumer.recieveMessages();
}}
The obtained exception is :
Exception in thread "main" java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:124)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:120)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:142)
at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:952)
at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.queueDeclare(AutorecoveringChannel.java:333)
at main.MyConsumer.connect(MyConsumer.java:44)
at main.MainConsumer.main(MainConsumer.java:7)
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit#souhaila-UX430UAR' of durable queue 'consumer' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:443)
at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:263)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:136)
... 4 more
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit#souhaila-UX430UAR' of durable queue 'consumer' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:509)
at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:340)
at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:162)
at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:109)
at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:643)
at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:47)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:581)
at java.lang.Thread.run(Thread.java:748)

How to manipulate Message coming from Netty server/client

I am prototyping a Netty client/server transfer for strings, now I want to pass these strings to file when it arrives to server side.
Client:
private ClientBootstrap bootstrap;
private Channel connector;
private MyHandler handler=new MyHandler();
public boolean start() {
// Standard netty bootstrapping stuff.
Executor bossPool = Executors.newCachedThreadPool();
Executor workerPool = Executors.newCachedThreadPool();
ChannelFactory factory =
new NioClientSocketChannelFactory(bossPool, workerPool);
this.bootstrap = new ClientBootstrap(factory);
// Declared outside to fit under 80 char limit
final DelimiterBasedFrameDecoder frameDecoder =
new DelimiterBasedFrameDecoder(Integer.MAX_VALUE,
Delimiters.lineDelimiter());
this.bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
public ChannelPipeline getPipeline() throws Exception {
return Channels.pipeline(
handler,
frameDecoder,
new StringDecoder(),
new StringEncoder());
}
});
ChannelFuture future = this.bootstrap
.connect(new InetSocketAddress("localhost", 12345));
if (!future.awaitUninterruptibly().isSuccess()) {
System.out.println("--- CLIENT - Failed to connect to server at " +
"localhost:12345.");
this.bootstrap.releaseExternalResources();
return false;
}
this.connector = future.getChannel();
return this.connector.isConnected();
}
public void stop() {
if (this.connector != null) {
this.connector.close().awaitUninterruptibly();
}
this.bootstrap.releaseExternalResources();
System.out.println("--- CLIENT - Stopped.");
}
public boolean sendMessage(String message) {
if (this.connector.isConnected()) {
// Append \n if it's not present, because of the frame delimiter
if (!message.endsWith("\n")) {
this.connector.write(message + '\n');
} else {
this.connector.write(message);
}
System.out.print(message);
return true;
}
return false;
}
Server:
private final String id;
private ServerBootstrap bootstrap;
private ChannelGroup channelGroup;
private MyHandler handler= new MyHandler();
public Server(String id) {
this.id = id;
}
// public methods ---------------------------------------------------------
public boolean start() {
// Pretty standard Netty startup stuff...
// boss/worker executors, channel factory, channel group, pipeline, ...
Executor bossPool = Executors.newCachedThreadPool();
Executor workerPool = Executors.newCachedThreadPool();
ChannelFactory factory =
new NioServerSocketChannelFactory(bossPool, workerPool);
this.bootstrap = new ServerBootstrap(factory);
this.channelGroup = new DefaultChannelGroup(this.id + "-all-channels");
// declared here to fit under the 80 char limit
final ChannelHandler delimiter =
new DelimiterBasedFrameDecoder(Integer.MAX_VALUE,
Delimiters.lineDelimiter());
this.bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
#Override
public ChannelPipeline getPipeline() throws Exception {
SimpleChannelHandler handshakeHandler =
new SimpleChannelHandler();
return Channels.pipeline(
handler,
delimiter,
new StringDecoder(),
new StringEncoder(),
handshakeHandler);
}
});
Channel acceptor = this.bootstrap.bind(new InetSocketAddress(12345));
if (acceptor.isBound()) {
System.out.println("+++ SERVER - bound to *:12345");
this.channelGroup.add(acceptor);
return true;
} else {
System.err.println("+++ SERVER - Failed to bind to *:12345");
this.bootstrap.releaseExternalResources();
return false;
}
}
public void stop() {
this.channelGroup.close().awaitUninterruptibly();
this.bootstrap.releaseExternalResources();
System.err.println("+++ SERVER - Stopped.");
}
Handlers used:
Client handler:
public class MyHandler extends SimpleChannelUpstreamHandler{
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e)
throws Exception {
if(e.getMessage() instanceof String){
System.out.println((String)e.getMessage());
}
System.out.println(e.getMessage().toString());
}
}
Server handler:
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e)
throws Exception {
Channel channel= ctx.getChannel();
channel.write(e.getMessage());
if(e.getMessage() instanceof String){
System.out.println((String)e.getMessage());
}
System.out.println(e.getMessage().toString());
}
client runner:
public static void main(String[] args) throws InterruptedException {
final int nMessages = 5;
try {
Client c = new Client();
if (!c.start()) {
return;
}
for (int i = 0; i < nMessages; i++) {
Thread.sleep(1L);
c.sendMessage((i + 1) + "\n");
}
c.stop();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Server Runner:
public static void main(String[] args) {
final Server s = new Server("server1");
if (!s.start()) {
return;
}
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
s.stop();
}
});
}
now what I really need is to print the message that I wrote on the channel on both client and server side and I am really puzzled on this.

Your pipeline creation seems to be wrong at first look. At server side when decoding, the Delimiter needs to come first, then the StringDecoder and then the business handler. You could resolve this probably by just putting breakpoints in these decoders and encoders. Also take a look at this link for very good documentation on how this works.

RabbitMQ perfomance issue

I have an app which receives messages from some producer using RabbitMQ, parses them and sends somwhere else. The problem is that my app doesn't use the CPU power fully. it uses not more than 25% of the CPU. look at this screenshot from profiler:
Here is the code in which the biggest part of the processing executing:
public class Consumer {
private static final String QUEUE_NAME = "MdnaMessagesQueue";
private static int i = 1;
private final ConnectionFactory factory;
private final boolean autoAck = false;
private final Connection connection;
private final Channel channel;
private String response;
private Sender sender;
private Logger LOG = Logger.getLogger(Consumer.class);
private ExecutorService es;
public Consumer() throws IOException{
es = Executors.newFixedThreadPool(8);
factory = new ConnectionFactory();
factory.setHost("localhost");
connection = factory.newConnection(es);
channel = connection.createChannel();
sender = new Sender();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
channel.basicQos(25);
Properties props = new Properties();
try {
props.load(new FileInputStream("/home/mikhail/bzrrep/DLP/DLPServer/src/main/java/log4j.properties"));
} catch (Exception e){
LOG.error(e);
}
PropertyConfigurator.configure(props);
}
/**
* is used to receive messages from the queue
* #param customerId the id of current customer
* #throws IOException
* #throws InterruptedException
*/
public void receive(final String customerId) throws IOException,InterruptedException{
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
final QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume(QUEUE_NAME, autoAck, "myConsumerTag",
new DefaultConsumer(channel){
#Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body)
throws IOException{
BasicProperties replyProperties = new BasicProperties.Builder().correlationId(properties.getCorrelationId()).build();
long deliveryTag = envelope.getDeliveryTag();
try{
LOG.info(" [" +(i++) +"] Received from first queue ");
byte[] dataArr = body;
MimeParser mimeParser = new MimeParser(true);
Filter filter = new Filter();
ByteArrayInputStream bais1 = new ByteArrayInputStream(dataArr);
MessageInfo mi = mimeParser.parseMessages(bais1);
//checking for compliance with rules
boolean messageAccepted = filter.getMessageAcceptability(mi.getPlainText(), customerId);
response = filter.getResult();
if(messageAccepted){
//sending to the other queue
sender.send(dataArr);
channel.basicPublish("", properties.getReplyTo(), replyProperties, response.getBytes());//need to add responce
} else {
channel.basicPublish("", properties.getReplyTo(), replyProperties, response.getBytes());
}
} catch (Exception e){
LOG.error(e);
}finally {
channel.basicAck(deliveryTag, false);
}
}
});
}
}
Here is the snapshot from the profiler:
How to solve this problem?

Syncronizing a multithreaded server

Hello everyone i have created a multi threaded chat server that looks like this:
public class Main {
public static ServerSocket server;
public static Socket connection;
public static int backLog = 100;
public static int numberOfConnected;
public static boolean connected = false;
public final static int potNumber = 6080;
public static PrintWriter pw;
public static Scanner input;
public static int i = 0;
public static void main(String[] args) {
startServer();
}
public static void startServer(){
try {
server = new ServerSocket(potNumber, backLog);
waitingForConnection();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private static void waitingForConnection() {
connected = false;
i++;
while (!connected) {
try {
if (connected) {
}
connection = server.accept();
Server s = new Server(connection, pw = new PrintWriter(connection.getOutputStream()), input = new Scanner(connection.getInputStream()));
s.start();
numberOfConnected++;
waitingForConnection();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
The idea is that this is suppose to be a chat server so when one connects to the server it starts the following thread:
threads
public void run(){
while (connection.isConnected()) {
if (input.hasNext()) {
String fullMessage = input.nextLine();
if (fullMessage.equalsIgnoreCase("Connect")) {
connectHim();
}else {
chatMessage(fullMessage);
}
}
}
}
private void chatMessage(String fullMessage) {
String name = fullMessage.substring(0, fullMessage.indexOf(" "));
String message = fullMessage.substring(fullMessage.indexOf(" "), fullMessage.length());
pw.println(name+": "+message);
pw.flush();
}
private void connectHim() {
String name = input.nextLine();
pw.println(0);
pw.flush();
pw.println(1);
pw.flush();
pw.println();
pw.flush();
pw.println(name);
pw.flush();
}
So my problem is the following:
if the user that is bound to thread 1 (this is an example) and the user bound to thread 2 sends a message to the server how will i send that message to the user bound on thread 1?

One of options is to use Hashtable or HashMap (just call Collections.synchronizedMap(myMap) in case of Map usage). When you start new Thread, give him unique name (for example user nick name ) and put it to your collection where key - Thread name, and value - Thread as Object.
if the user that is bound to thread 1 (this is an example) and the user bound to thread 2 sends a message to the server how will i send that message to the user bound on thread 1?
For example you have user1, user2, user3. Now you build 3 Threads and put them to HashMap, like:
Map<String, Thread> threadMap = new HashMap<String,Thread>();
threadMap = Collections.synchronizedMap(threadMap);
YourThread th1 = new YourThread();
threadMap.put("user1", th);
YourThread th2 = new YourThread();
threadMap.put("user2", th);
YourThread th3 = new YourThread();
threadMap.put("user3", th);
....
Set<String> userSet = threadMap.keySet();
Iterator<String> it = userSet.iterator();
Thread currThread = null;
while(it.hasNext()){
String key = it.next();
currThread = threadMap.get(key);
// do something with currThread
}