We Keep Coding

why rabbitmq message consumption is so slow

Currently I am making logic to consume Message using Rabbitmq. However, contrary to expectations, it takes too long to consume the message.
If you look at the Queued messages graph in the picture above, you can see Unacked and Ready stacking up.
Looking at the message rates below, the publish speed is fast, but the consumer ack speed is too slow.
I'm not sure if the Rabbitmq Configuration I've developed is wrong or if I forgot to set the listener configuration.
The rabbitmq message I receive is a callback message.
Any help would be greatly appreciated.
This is Rabbitmq configuration and RabbitListener configuration
#Configuration
#Profile({ProfileConfig.RABBITMQ})
public class RabbitmqConfig {
#Value("${rabbitmq.queue.name}")
private String queueName;
#Value("${rabbitmq.exchange.name}")
private String exchangeName;
#Value("${rabbitmq.routing.key.callback}")
private String routingKey;
#Value("${rabbitmq.fetch-count}")
private Integer fetchCount;
#Bean
Queue queue() {
return new Queue(queueName, true);
}
#Bean
DirectExchange directExchange() {
return new DirectExchange(exchangeName);
}
#Bean
Binding binding(DirectExchange directExchange, Queue queue) {
return BindingBuilder.bind(queue).to(directExchange).with(routingKey);
}
#Bean
public RabbitListenerContainerFactory<SimpleMessageListenerContainer> prefetchOneContainerFactory(
SimpleRabbitListenerContainerFactoryConfigurer configurer, ConnectionFactory factory)
{
SimpleRabbitListenerContainerFactory simpleFactory = new SimpleRabbitListenerContainerFactory();
configurer.configure(simpleFactory, factory);
simpleFactory.setPrefetchCount(fetchCount);
return simpleFactory;
}
}
#RabbitListener(queues = {"${rabbitmq.queue.name}"}, concurrency = "3", containerFactory = "prefetchOneContainerFactory")
public void receiveMessage(final String message, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag) {
try {
JSONParser parser = new JSONParser();
JSONObject json = (JSONObject) parser.parse(message);
String messageType = json.get("messageType").toString();
log.debug("Receive Queue Key={}, Message = {}", messageType, message);
AsyncType asyncType = AsyncType.valueOf(messageType);
executeMessage(asyncType, message);
} catch (Exception e) {
traceService.removeTraceId();
traceService.printErrorLog(log, "Fail to deal receive message.", e, PrintStackPolicy.ALL);
} finally {
try {
channel.basicAck(tag, false);
}
catch (IOException e) {
traceService.printErrorLog(log, "Fail to send ack to RabbitMQ", e, PrintStackPolicy.ALL);
}
}
}
The goal is to consume messages to Rabbitmq faster.
However, the current consumption speed is too slow.

RabbitMQ, headers exchange, messages not routed by headers x-match = all

I'm trying to setup a headers exchange with a queue where messages are routed based on a recipient header.
The exchange is of type headers.
So far the class is able to connect to the exchange and feed messages to the queue.
It's also able to subscribe to the queue and receive messages. It also closes the connection whenever the subscriber's connection is cancelled.
The current problem is that the message is not routed by the recipient's header value.
Given the following class:
import com.rabbitmq.client.*;
import lombok.extern.slf4j.Slf4j;
import reactor.core.publisher.Flux;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;
#Slf4j
public class MyQueue {
private final ConnectionFactory connectionFactory;
private Channel channel;
public MyQueue(ConnectionFactory connectionFactory) {
this.connectionFactory = connectionFactory;
}
public String sendMessage(TestTextMessage message) throws UndeliverableMessageException {
try (Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel()) {
Map<String, Object> headers = new HashMap<>();
headers.put(RabbitMqConfig.MATCH_HEADER, message.getRecipient());
AMQP.BasicProperties props = new AMQP.BasicProperties.Builder()
.deliveryMode(MessageProperties.PERSISTENT_TEXT_PLAIN.getDeliveryMode())
.priority(MessageProperties.PERSISTENT_TEXT_PLAIN.getPriority())
.headers(headers).build();
log.info("Sending message to {}", headers);
channel.basicPublish(RabbitMqConfig.EXCHANGE_NAME, "", props,
message.getMessage().getBytes(StandardCharsets.UTF_8));
log.info("RabbitMQ sent message {} to {}", message.getMessage(), message.getRecipient());
return "ok";
} catch (TimeoutException e) {
log.error("Rabbit mq timeout", e);
} catch (IOException e) {
log.error("Rabbit mq io error", e);
}
throw new UndeliverableMessageException();
}
public Flux<String> listenMessages(String recipient) throws IOException, TimeoutException {
Connection connection = connectionFactory.newConnection();
this.channel = connection.createChannel();
// The map for the headers.
Map<String, Object> headers = new HashMap<>();
headers.put("x-match", "all");
headers.put(RabbitMqConfig.MATCH_HEADER, recipient);
final String[] consumerTag = new String[1];
Flux<String> as = Flux.create(sink -> new MessageListener<String>() {
{
try {
log.info("Binding to {}", headers);
channel.queueBind(RabbitMqConfig.QUEUE_NAME, RabbitMqConfig.EXCHANGE_NAME, "",
headers);
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
String message = new String(delivery.getBody(), StandardCharsets.UTF_8);
log.info("Subscriber {} received a message {} with headers {}", recipient, delivery.getEnvelope(),
delivery.getProperties().getHeaders());
sink.next(delivery.getEnvelope().getDeliveryTag() + "--" + message);
//channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
};
consumerTag[0] = channel.basicConsume(RabbitMqConfig.QUEUE_NAME,
true, deliverCallback, tag -> {
sink.complete();
});
} catch (IOException e) {
log.error("RabbitMQ IOException ", e);
}
}
});
return as.doOnCancel(() -> {
try {
if (consumerTag[0] == null) {
log.error("RabbitMQ uncloseable subscription, consumerTag is null!");
channel.close();
return;
}
channel.basicCancel(consumerTag[0]);
channel.close();
log.info("RabbitMQ CANCEL subscription for recipient {}", recipient);
} catch (IOException | TimeoutException e) {
log.error("RabbitMQ channel close error", e);
}
});
}
interface MessageListener<T> {
}
}
The exchange is declared by the following call
channel.exchangeDeclare(RabbitMqConfig.EXCHANGE_NAME, BuiltinExchangeType.HEADERS, true);
Binding recipient log:
Binding to {x-match=all, message-recipient=mary}
Binding to {x-match=all, message-recipient=james}
Binding to {x-match=all, message-recipient=john}
Bound 3 recipients with x-match:
However, messages are not matched, as if they were routed randomly
Sending message to {message-recipient=james}
RabbitMQ sent message Hey there to james
Subscriber mary received a message Envelope(deliveryTag=1, redeliver=false, exchange=my-exchange, routingKey=) with headers {message-recipient=james}
Sending message to {message-recipient=james}
RabbitMQ sent message Hey there to james
Subscriber james received a message Envelope(deliveryTag=1, redeliver=false, exchange=my-exchange, routingKey=) with headers {message-recipient=james}
Sending message to {message-recipient=james}
RabbitMQ sent message Hey there to james
Subscriber john received a message Envelope(deliveryTag=1, redeliver=false, exchange=my-exchange, routingKey=) with headers {message-recipient=james}
Why isn't x-match: all, matching?

After reading the comment posted by #Gryphon, on the subscriber side, I ended up creating a queue for each participant.
channel.queueDeclare(RabbitMqConfig.QUEUE_NAME + "-" + recipient,
true,
false,
false,
null)
On the publisher side, code remains unchanged, the messages are sent to the exchange, and the exchange will handle routing based on the x-match: all configuration, routing the messages to the appropiate queue.

Supervisor of akka doesn't work correctly with RestTemplate

I'm using akka with Java and Spring. And my main goal is notify other application in a asynchronous way. But sometimes this other app isn't notified. So, I implemented a Supervisor to make use of retries. But, when I receive an Exception, specifically InternalServerErrorException my supervisor does not make another retry. Seems that the RestTemplate and your treatement of exceptions is causing the problem.
Below follows my code:
#Scope(SCOPE_PROTOTYPE)
public class NotificacaoSupervisor extends AbstractActor {
private static final int RETRIES = 5;
private OneForOneStrategy ONE_FOR_ONE_STRATEGY = new OneForOneStrategy(
RETRIES,
Duration.create("5 minutes"),
true,
DeciderBuilder.match(NotificacaoException.class, ex -> SupervisorStrategy.restart())
.build());
#Inject #Qualifier("notifyActor")
private ActorRef notifyActor;
#Override
public Receive createReceive() {
return receiveBuilder()
.matchAny(any -> notifyActor.forward(any, getContext()))
.build();
}
#Override
public SupervisorStrategy supervisorStrategy() {
return ONE_FOR_ONE_STRATEGY;
}
}
Block that sends a notification
try{
if(content.isPresent()) {
this.logInfo(content.get());
HttpEntity<String> entity = new HttpEntity<>(content.get(), headers);
String urlDeCallback = integracao.getUrlDeCallback();
URI uri = URI.create(urlDeCallback);
restTemplate.postForObject(uri, entity, Void.class);
}
} catch (Exception e) {
this.logError(new ErrorResponse(notificacao).toString(), e);
throw new NotificacaoException(e);
}

How to get SOAP fault message have no mapped in wsdl

I generated my client soap from wsimport JAX-WS, I have already consumed others webservice that it had fault message mapped, but the service current doesn't have.
When I call the service and it returns fault message I can't get the message in the Java, but if call from soapUI I can see the error.
The fault message is the same of the success, generated from JAX-WS.
My code:
//before I setter my request
try{
IPGApiOrderService iPGApiOrderService = new IPGApiOrderService();
IPGApiOrder client = iPGApiOrderService.getIPGApiOrderSoap11();
IPGApiOrderResponse response = client.ipgApiOrder(request)
}catch (SOAPFaultException soapEx) {
System.out.println("Fault ............. " + soapEx.getFault());
System.out.println("Detail ............ " + soapEx.getFault().getDetail());
System.out.println("FaultCode.......... " + soapEx.getFault().getFaultCode());
System.out.println("FaultActor......... " + soapEx.getFault().getFaultActor());
System.out.println("Message............ " + soapEx.getMessage());
soapEx.printStackTrace();
}
follow the out
Fault ............. [SOAP-ENV:Fault: null]
Detail ............ [detail: null]
FaultCode.......... SOAP-ENV:Client
FaultActor......... null
Message............ Client received SOAP Fault from server: ProcessingException Please see the server log to find more detail regarding exact cause of the failure.
com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: ProcessingException Please see the server log to find more detail regarding exact cause of the failure.
at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178)
at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:124)
at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(StubHandler.java:238)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:189)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:276)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:104)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147)
at com.sun.proxy.$Proxy36.ipgApiOrder(Unknown Source)
at com.firstdata.test.demo.MainTest.main(MainTest.java:53)

I resolved my problem with following steps.
Create SOAPHandler;
It'll be necessary implement 4 methods;
On method handleFault get SOAPMessageContext -> SOAPMessage -> SOAPBody -> Fault -> Detail -> add detail with xml error or some information do you want.
3.1 Fault you can put fault code, if API you was consuming always return one code error to API fault.
4. On exception you find that detail you set and work with it.
Code:
public class SOAPHandlerImpl implements SOAPHandler<SOAPMessageContext> {
public static final QName JSON_ERROR = new QName("json-error");
public boolean handleMessage(SOAPMessageContext smc) {
SOAPMessage message = smc.getMessage();
Boolean isOut = (Boolean) smc.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
return isOut;
}
#Override
public boolean handleFault(SOAPMessageContext context) {
SOAPMessage message = context.getMessage();
try {
StringOutputStream str = new StringOutputStream();
message.writeTo(str);
ErrorDTO dto = XmlUtil.buildErroDTO(str.toString());
Detail detail = message.getSOAPBody().getFault().getDetail();
Gson gson = new Gson();
String obj = gson.toJson(dto);
detail.addDetailEntry(JSON_ERROR).addTextNode(obj);
message.getSOAPBody().getFault().setFaultCode(String.valueOf(dto.getTransactionId()));
} catch (Exception e) {
System.out.println("Exception in handler: " + e);
}
return true;
}
#Override
public void close(MessageContext context) {
// TODO Auto-generated method stub
}
#Override
public Set<QName> getHeaders() {
// TODO Auto-generated method stub
return null;
}
}
Catch exception
} catch (SOAPFaultException sopex) {
ErrorDTO error = null;
Iterator childElements = sopex.getFault().getDetail().getChildElements();
while (childElements.hasNext()) {
DetailEntry next = (DetailEntry) childElements.next();
if (SOAPHandlerImpl.JSON_ERROR.getLocalPart().equals(next.getNodeName())) {
Gson gson = new Gson();
error = gson.fromJson(next.getValue(), ErrorDTO.class);
}
}
String message = null;
if(error.getProcessorResponseCode() != null) {
message = ErrorApiUtil.getInstance().getMessage(error.getProcessorResponseCode());
}else {
message = error.getMessage();
}
throw new BusinessException(message);
}

can't send object via websocket

So i'm trying to send an object from client->server & server->client with WebSocket. Sending object from client->server works fine, meanwhile server->client throw an exception
org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Can not deserialize instance of java.lang.String out of START_OBJECT token
Here is the class i'm trying to send
#Data
#AllArgsConstructor
#NoArgsConstructor
public class TextMessage {
private String sender;
private String room;
private String message;
}
and this is the code on the client-side
public class TelepatiClient {
public static void main(String[] args) {
WebSocketClient client = new StandardWebSocketClient();
WebSocketStompClient stompClient = new WebSocketStompClient(client);
stompClient.setMessageConverter(new MappingJackson2MessageConverter());
stompClient.setTaskScheduler(new ConcurrentTaskScheduler());
String url = "ws://localhost:8000/connect";
StompSessionHandler handler = new TelepatiSessionHandler();
stompClient.connect(url, handler);
new Scanner(System.in).nextLine();
}
}
public class TelepatiSessionHandler extends StompSessionHandlerAdapter {
#Override
public void afterConnected(StompSession session, StompHeaders connectedHeaders) {
session.subscribe("/room/global", this);
session.send("/test", new TextMessage("test", "test", "test"));
}
#Override
public void handleFrame(StompHeaders headers, Object payload) {
System.out.println(payload.toString());
}
#Override
public void handleException(StompSession session, StompCommand command, StompHeaders headers, byte[] payload, Throwable exception) {
exception.printStackTrace();
super.handleException(session, command, headers, payload, exception);
}
}
and this is message controller on the server-side
#Controller
public class TelepatiController {
#MessageMapping("/test")
#SendTo("/room/global")
public TextMessage getMessage(TextMessage message) {
System.out.println("get message :" + message.toString());
return new TextMessage("test2", "test2", "test2");
}
}
i was able to run System.out.println("get message :" + message.toString());, but get message convertion exception on the client-side when returning new TextMessage("test2", "test2", "test2");. From my test before, returning a String object works fine, why returning TextMessage object not working? How can i send any object (in this case TextMessage) from server->client? Thanks!

Well the problem is the content. In this line:
stompClient.setMessageConverter(new MappingJackson2MessageConverter());
You indicate that the client uses a json converter. So, your client is always expecting a JSON object.
But in your test, in this line:
return new TextMessage("test2", "test2", "test2");
You are sending plain text. Due the StompClient is thrown an exception
org.springframework.messaging.converter.MessageConversionException
Because the message in text plain is not JSON object.

I hope someone helps you, I had the same problem, what I did was tell the topler handler, to which I subscribe, the type of payload that will return.
This is the handler of my stompClient:
public class TelepatiSessionHandler extends StompSessionHandlerAdapter {
#Override
public void afterConnected(StompSession session, StompHeaders connectedHeaders) {
session.send("/test", new TextMessage("test", "test", "test"));
}
#Override
public void handleException(StompSession session, StompCommand command, StompHeaders headers, byte[] payload, Throwable exception) {
exception.printStackTrace();
super.handleException(session, command, headers, payload, exception);
}
}
And this is the handler for the topic to which I subscribe
WebSocketClient client = new StandardWebSocketClient();
WebSocketStompClient stompClient = new WebSocketStompClient(client);
stompClient.setMessageConverter(new MappingJackson2MessageConverter());
stompClient.setTaskScheduler(new ConcurrentTaskScheduler());
String url = "ws://localhost:8000/connect";
StompSessionHandler handler = new TelepatiSessionHandler();
StompSession session = stompClient.connect(url, handler).get();
session.subscribe("/room/global", new StompFrameHandler() {
#Override
public Type getPayloadType(StompHeaders headers) {
return TextMessage.class;
}
#Override
public void handleFrame(StompHeaders headers, Object payload) {
TextMessage textMessage = (TextMessage) payload;
System.out.println(textMessage.toString());
}
});
Here is a complete example:
https://github.com/jaysridhar/spring-websocket-client/blob/master/src/main/java/sample/Application.java