I have an application that creates queues (if it doesn't exist) with a naming convention. Example:
"test. {Something} .demandas"
Since {something} is passed at the time of its creation, and then there are several queues with different {something}.
Now I need to read these queues on the consumer, that is, get all the queues created by the producer. I saw some examples using the RabbitListenerEndpointRegistry, or even getting the names of the queues by jenkins (using variables from the vm).
But would you have any alternative?
This is the rabbit configuration class:
#Configuration
#EnableRabbit
public class RabbitConfig {
public static final String S_S = "%s.%s";
public static final String PREFIX = "test.laa.aaa";
public static final String QUEUE_NAME = "demandas";
public static final String APPLICATION_NAME = "name:test.laa.aaa";
private final String exchange;
private final String routingKey;
private final Integer maxConsumers;
public RabbitConfig(
#Value("${crawler.exchange.name:demandas}")
String exchange,
#Value("${crawler.exchange.routing-key:default}")
String routingKey,
#Value("${crawler.max-consumers:1}")
Integer maxConsumers) {
this.routingKey = routingKey;
this.maxConsumers = maxConsumers;
this.exchange = exchange;
}
#Bean
#Primary
public String routingKey() {
return routingKey;
}
#Bean
#Primary
public String prefixName() {
return PREFIX;
}
#Bean
#Primary
public String queueName() {
return QUEUE_NAME;
}
private String exchangeName() {
return String.format(S_S, PREFIX, exchange);
}
#Bean
#Primary
public RabbitAdmin rabbitAdmin(ConnectionFactory connectionFactory) {
AbstractConnectionFactory abstractConnectionFactory = (AbstractConnectionFactory) connectionFactory;
abstractConnectionFactory.setConnectionNameStrategy(con -> String.format("%s", APPLICATION_NAME));
final RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory);
rabbitAdmin.afterPropertiesSet();
return rabbitAdmin;
}
#Bean(name = "queueConsumer")
#Primary
public Queue queue() {
Map<String, Object> map = new HashMap<>();
map.put("x-max-priority", 10);
return new Queue(String.format(S_S, PREFIX, QUEUE_NAME), true, false, false, map);
}
#Bean
#Primary
public DirectExchange exchange() {
return new DirectExchange(exchangeName());
}
#Bean
#Primary
public Binding binding(Queue queue, DirectExchange exchange) {
if (Objects.nonNull(routingKey)) {
return BindingBuilder.bind(queue).to(exchange).with(routingKey);
}
return BindingBuilder.bind(queue).to(exchange).with("*");
}
#Bean
#Primary
public MessageConverter messageConverter(ObjectMapper objectMapper) {
return new Jackson2JsonMessageConverter(objectMapper);
}
#Bean
#Primary
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory, MessageConverter messageConverter) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setConcurrentConsumers(maxConsumers);
factory.setMaxConcurrentConsumers(maxConsumers);
factory.setPrefetchCount(1); //Default
factory.setMessageConverter(messageConverter);
factory.setAfterReceivePostProcessors(message -> {
message.getMessageProperties().setContentType("application/json");
message.getMessageProperties().setContentEncoding("UTF-8");
return message;
});
return factory;
}
public String getPrefix() {
return PREFIX;
}
public String getQueueName() {
return QUEUE_NAME;
}
Since queues are created by your producer, I assume that you are publishing messages directly into queues (as describe in the Rabbitmq documentation). If you want to keep this approach, then you have no other choice than finding a way to communicate queue names to consumers.
However, I recommend you to take a look at a different approach based on publish/subscribe pattern (you can find it in the official documentation too). Producers will then push messages into an exchange, with a specific routing key (for example: test.{Something}.demandas).
Then consumers will be in charge on creating they own queue and bind it (for example: receive messages from test.*.demandas, making the value of {Something} irrelevant to route your message).
This way, you don't have to share queue names (though you have to share the exchange name). It also helps to reduce the coupling between producer and consumer.
Related
How can I deserialize a message using convertSendAndReceive() method? It gives me NullPointerException due to not being able to find the required class for deserialization in another package. Packages are marked in the code.
Listener receives and sends messages normally
package org.dneversky.user;
#EnableRabbit
#Component
public class TestListener {
private static final Logger logger = LoggerFactory.getLogger(TestListener.class);
#Autowired
private RabbitTemplate rabbitTemplate;
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE)
public void doGet(UserReplyMessage message) {
logger.info("Received message: {}", message);
UserReplyMessage response = new UserReplyMessage();
logger.info("Sending message: {}", response);
rabbitTemplate.convertSendAndReceive(RabbitMQConfig.RPC_EXCHANGE,
RabbitMQConfig.REPLY_QUEUE, response);
}
}
Configuration of the listener
package org.dneversky.user.config;
#Configuration
public class RabbitMQConfig {
public static final String RECEIVE_QUEUE = "rpc_queue";
public static final String REPLY_QUEUE = "reply_queue";
public static final String RPC_EXCHANGE = "rpc_exchange";
#Bean
public TopicExchange rpcExchange() {
return new TopicExchange(RPC_EXCHANGE);
}
#Bean
public Queue receiveQueue() {
return new Queue(RECEIVE_QUEUE);
}
#Bean
public Queue replyQueue() {
return new Queue(REPLY_QUEUE);
}
#Bean
public Binding receiveBinding() {
return BindingBuilder.bind(receiveQueue()).to(rpcExchange()).with(RECEIVE_QUEUE);
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Sender sends a message normally, but it can't to deserialize returning message
package org.dneversky.gateway.servie.impl;
#Service
public class UserServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(UserServiceImpl.class);
#Autowired
private RabbitTemplate rabbitTemplate;
public UserPrincipal getUserByUsername(String username) {
UserResponse message = new UserResponse(username);
logger.info("Sending created message: {}", message);
UserResponse result = (UserResponse) rabbitTemplate.convertSendAndReceive(RabbitMQConfig.RPC_EXCHANGE, RabbitMQConfig.RPC_QUEUE, message);
logger.info("Getting response: {}", result);
return null;
}
}
Configuration of the Sender
package org.dneversky.gateway.config;
#Configuration
public class RabbitMQConfig {
public static final String RPC_QUEUE = "rpc_queue";
public static final String REPLY_QUEUE = "reply_queue";
public static final String RPC_EXCHANGE = "rpc_exchange";
#Bean
public Queue rpcQueue() {
return new Queue(RPC_QUEUE);
}
#Bean
public Queue replyQueue() {
return new Queue(REPLY_QUEUE);
}
#Bean
public TopicExchange rpcExchange() {
return new TopicExchange(RPC_EXCHANGE);
}
#Bean
public Binding binding() {
return BindingBuilder.bind(replyQueue()).to(rpcExchange()).with(REPLY_QUEUE);
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setExchange(RPC_EXCHANGE);
rabbitTemplate.setReplyAddress(REPLY_QUEUE);
rabbitTemplate.setReplyTimeout(6000);
rabbitTemplate.setMessageConverter(messageConverter());
return rabbitTemplate;
}
#Bean
public MessageConverter messageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public SimpleMessageListenerContainer replyContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(REPLY_QUEUE);
container.setMessageListener(rabbitTemplate(connectionFactory));
return container;
}
}
Error log
2022-05-22 17:12:31.344 ERROR 16920 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.amqp.support.converter.MessageConversionException: failed to resolve class name. Class not found [org.dneversky.user.model.UserReplyMessage]] with root cause
java.lang.ClassNotFoundException: org.dneversky.user.model.UserReplyMessage
by default, the producer set the _TypeID_ header as the class name used for the serialization of the object
then consumer uses _TypeID_ header to know the class that should use to convert the JSON to java instance
you use two different classes to serialize and deserialize the object and you have to configure the converter
inside your replyContainer I couldn't see your messageConverter bean. In default it uses java objects to send and receive messages without converting them into human readable json.
#Bean
public SimpleRabbitListenerContainerFactory customListenerContainerFactory(ConnectionFactory connectionFactory,
MessageConverter jsonMessageConverter) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(jsonMessageConverter);
return factory;
}
for your consumer;
#RabbitListener(queues = RabbitConstants.YOUR_QUEUE_NAME, containerFactory = "customListenerContainerFactory")
public void onMessage(#Valid YourEvent YourEvent){
//your code
}
Inside the Listener class, you need to add this line to bind your message converter
#Bean
public SimpleRabbitListenerContainerFactory jsaFactory(ConnectionFactory connectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
Also, in the TestListener class, you should replace this line
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE)
with this one
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE,containerFactory="jsaFactory")
I have a problem with Spring Boot Rabbit Mq. I have 2 listeners which listen the same queue but handle different objects:
#Service
#RabbitListener(queues = "#{changeDataQueue.name}")
public class CreateDayAheadTradeListener implements DayAheadTradeEventListener<CreateDayAheadTradeRequest> {
#Autowired
private DayAheadTradeRepository repository;
#Override
#RabbitHandler
public void process(final CreateDayAheadTradeRequest event) {
// Do something
}
}
And the next one:
#Service
#RabbitListener(queues = "#{changeDataQueue.name}")
public class UpdateDayAheadTradeListener implements DayAheadTradeEventListener<ModifyDayAheadTradeStatusRequest> {
#Autowired
private DayAheadTradeRepository repository;
#Override
#RabbitHandler
public void process(final ModifyDayAheadTradeStatusRequest event) {
// Do something
}
}
The Rabbit Config is:
#EnableRabbit
#Configuration
public class RabbitMqConfig {
private final String createDayAheadTradesRouting = CreateDayAheadTradeRequest.class
.getAnnotation(Routing.class)
.routingKey();
private final String updateDayAheadTradesRouting = ModifyDayAheadTradeStatusRequest.class
.getAnnotation(Routing.class)
.routingKey();
private final String requestDayAheadTradesRouting = DayAheadTradeRequest.class
.getAnnotation(Routing.class)
.routingKey();
#Bean
public TopicExchange topicExchange() {
return new TopicExchange("schedule"); // exchange name
}
#Bean
public Queue changeDataQueue() {
return QueueBuilder
.durable("dayahead.trades.data") // queue template name
.build();
}
#Bean
public Queue requestQueue() {
return QueueBuilder
.nonDurable("dayahead.trades.request") // queue template name
.exclusive()
.build();
}
#Bean
public Binding createDataBinding(final Queue changeDataQueue, final TopicExchange topicExchange) {
return BindingBuilder
.bind(changeDataQueue)
.to(topicExchange)
.with(createDayAheadTradesRouting);
}
#Bean
public Binding updateDataBinding(final Queue changeDataQueue, final TopicExchange topicExchange) {
return BindingBuilder
.bind(changeDataQueue)
.to(topicExchange)
.with(updateDayAheadTradesRouting);
}
#Bean
public Binding requestBinding(final Queue requestQueue, final TopicExchange topicExchange) {
return BindingBuilder
.bind(requestQueue)
.to(topicExchange)
.with(requestDayAheadTradesRouting);
}
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
final ObjectMapper mapper = EventUtils.createMapper();
mapper.registerModule(new MarketdataModule());
return new Jackson2JsonMessageConverter(mapper);
}
}
The problem is that if I create 2 separate queues for different types of objects - everything works fine, but the idea is to use one queue for several types of objects. But I have the following exception:
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener method 'no match' threw exception
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:219)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:143)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:132)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1569)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1488)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1476)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1467)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1411)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:958)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:908)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:81)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1279)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1185)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.springframework.amqp.AmqpException: No method found for class java.util.LinkedHashMap
at org.springframework.amqp.rabbit.listener.adapter.DelegatingInvocableHandler.getHandlerForPayload(DelegatingInvocableHandler.java:149)
at org.springframework.amqp.rabbit.listener.adapter.DelegatingInvocableHandler.invoke(DelegatingInvocableHandler.java:129)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:61)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:211)
... 13 more
I can't find any information about it, I hope someone knows about it. Thanks
I have two queues and they each have messages on them. Queue one has bird objects and queue two has birdspotting object. I'm using a defaultclassmapper to convert the messages back into objects. Is there a way for me to add different configurations on both my rabbitlisteners.
My listeners.
#Qualifier("bird")
#RabbitListener(queues = "vogels")
public void receiveBird(Bird in)
BirdSpotting birdSpotting = new BirdSpotting();
birdSpotting.setBird(in);
rabbitTemplate.convertAndSend("vogelspottings",birdSpotting);
}
#Qualifier("birdspotting")
#RabbitListener(queues = "vogelspottingmetlocatie")
public void receiveBirdWithLocation(BirdSpotting birdSpotting){
service.saveBirdSpotting(birdSpotting);
}
My configuration class.
#Configuration
#EnableRabbit
public class RabbitConf2 implements RabbitListenerConfigurer {
#Autowired
DefaultClassMapper mapper;
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(consumerJackson2MessageConverter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
#Bean
public RabbitTemplate rabbitTemplateService2(final ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(producerJackson2MessageConverterService2());
return rabbitTemplate;
}
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverterService2() {
final Jackson2JsonMessageConverter jackson2JsonMessageConverter = new Jackson2JsonMessageConverter();
jackson2JsonMessageConverter.setClassMapper(mapper);
return jackson2JsonMessageConverter;
}
My two defaultclassmappers for both queues:
#Bean(value = "bird")
public DefaultClassMapper classMapperService2() {
DefaultClassMapper classMapper = new DefaultClassMapper();
Map<String, Class<?>> idClassMapping = new HashMap<>();
idClassMapping.put("be.kdg.birdgeneratorservice.Bird", Bird.class);
classMapper.setIdClassMapping(idClassMapping);
return classMapper;
}
#Bean(value = "birdspotting")
public DefaultClassMapper classMapperService3() {
DefaultClassMapper classMapper = new DefaultClassMapper();
Map<String, Class<?>> idClassMapping = new HashMap<>();
idClassMapping.put("be.kdg.locationservice.BirdSpotting", BirdSpotting.class);
classMapper.setIdClassMapping(idClassMapping);
return classMapper;
}
You need to introduce one more RabbitListenerContainerFactory bean with an appropriate configuration and use its name from the second #RabbitListener:
/**
* The bean name of the {#link org.springframework.amqp.rabbit.listener.RabbitListenerContainerFactory}
* to use to create the message listener container responsible to serve this endpoint.
* <p>If not specified, the default container factory is used, if any.
* #return the {#link org.springframework.amqp.rabbit.listener.RabbitListenerContainerFactory}
* bean name.
*/
String containerFactory() default "";
This way you will distinguish a default one provided by the Spring Boot and will have your own custom for another use-case.
See more info in the Docs: https://docs.spring.io/spring-amqp/docs/2.1.4.RELEASE/reference/#async-annotation-driven
I am using Spring Boot framework. I want to send an object from a service to another service via RabbitMQ like this:
Service A:
rabbitTemplate.convertAndSend("queue", createAccountRequestMessage);
Service B:
#RabbitListener(queues = "queue")
public void onAccountRequested(#Valid CreateAccountRequestMessage createAccountRequestMessage, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG, long tag) throws IOException
{
}
In CreateAccountRequestMessage class I have defined some validation annotations like #NotEmpty, #NotNull and etc, but when I'm sending wrong message from service A to service B, #Valid annotation doesn't work and CreateAccountRequestMessage object is not validated before invoke onAccountRequested method.
You need to set the validator in DefaultMessageHandlerMethodFactory.
#Autowired
SmartValidator validator;
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(this.validator);
return factory;
}
Then you also need to specify the #Payload annotation along with the #Valid annotation.
#RabbitListener(queues = "queue")
public void onAccountRequested(#Valid #Payload CreateAccountRequestMessage
createAccountRequestMessage, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG, long tag) throws IOException
{
}
Now MethodArgumentNotValidException will be thrown and the message will be discarded, or you can send the message to a dead letter exchange.
I had the same problem. The answer of #Praveer works well except SmartValidator. I post here my solution, which is inspired by this article https://blog.trifork.com/2016/02/29/spring-amqp-payload-validation/
#Configuration
#EnableRabbit
#Slf4j
public class CmsMQConfig implements RabbitListenerConfigurer {
#Value("${dw.rabbitmq.hosts}")
private String hosts;
#Value("${dw.rabbitmq.username}")
private String username;
#Value("${dw.rabbitmq.password}")
private String password;
#Value("${dw.rabbitmq.virtual-host}")
private String virtualHost;
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses(hosts);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
connectionFactory.setVirtualHost(virtualHost);
return connectionFactory;
}
#Bean
public Jackson2JsonMessageConverter messageConverter() {
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JodaModule());
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
return new Jackson2JsonMessageConverter(mapper);
}
#Bean
public DefaultMessageHandlerMethodFactory defaultHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(amqpValidator());
return factory;
}
#Bean
public Validator amqpValidator() {
return new OptionalValidatorFactoryBean();
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setContainerFactory(rabbitListenerContainerFactory());
registrar.setMessageHandlerMethodFactory(defaultHandlerMethodFactory());
}
}
I'm using spring 4 annotation based configuration, and would like to set up a simple telnet/socket client.
This is what I have so far:
#MessageEndpoint
public class MySocket {
#Bean
public TcpConnectionFactoryFactoryBean clientFactory() {
TcpConnectionFactoryFactoryBean fact = new TcpConnectionFactoryFactoryBean();
fact.setType("client");
fact.setHost(host);
fact.setPort(port);
fact.setUsingNio(true);
fact.setSingleUse(true);
fact.setSoTimeout(timeout);
return fact;
}
#Bean
public MessageChannel clientChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "clientChannel")
public TcpOutboundGateway outGateway(TcpNioClientConnectionFactory factory,
#Qualifier("clientChannel") MessageChannel clientChannel) throws Exception {
TcpOutboundGateway gate = new TcpOutboundGateway();
gate.setConnectionFactory(factory);
gate.setReplyChannel(clientChannel);
return gate;
}
}
#Component
public class MyMessageService {
#Autowired
#Qualifier("clientChannel")
private MessageChannel clientChannel;
public void run() {
Message<String> msg = MessageBuilder.withPayload("test").build();
Message<?> rsp = new MessagingTemplate(clientChannel).sendAndReceive(msg);
}
}
Result: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
What am I missing here to send the message via the socket and receive the reply?
You don't need the #MessageEndpoint annotation, but you need a consumer on the channel...
#ServiceActivator(inputChannel = "clientChannel")
#Bean
public TcpOutboundGateway outGateway(AbstractClientConnectionFactory scf) {
...
}
The gateway needs a reference to the connection factory. Since you are using a factory bean, it's easiest to add it as a parameter to the bean's factory method.