I am using Spring AMQP to send messages and be able to perform retries on a "custom" Exception. Lets say I have a Receiver which throws a custom exception "EventException" and for that, I want there to be a n number of retries (in our example 5). Between the retries I also want there to be a 5 seconds delay as well. Here is my source code:
#SpringBootApplication
public class DemoApplication implements CommandLineRunner {
final static String queueName = "testing-queue";
#Autowired
AnnotationConfigApplicationContext context;
#Autowired
RabbitTemplate rabbitTemplate;
#Bean
Queue queue() {
Map<String, Object> arguments = new HashMap<String, Object>();
arguments.put("x-dead-letter-exchange", "dead-letter-exchange");
Queue queue = new Queue(queueName, true, false, false, arguments);
return queue;
}
#Bean
TopicExchange exchange() {
return new TopicExchange("testing-exchange");
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(queueName);
}
#Bean
Queue deadLetterQueue() {
return new Queue("dead-letter-queue", true);
}
#Bean
FanoutExchange deadLetterExchange() {
return new FanoutExchange("dead-letter-exchange");
}
#Bean
Binding deadLetterBinding(Queue deadLetterQueue, FanoutExchange deadLetterExchange) {
return BindingBuilder.bind(deadLetterQueue).to(deadLetterExchange);
}
#Bean
ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
SimpleMessageListenerContainer container(
ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter,
RetryOperationsInterceptor interceptor) {
Advice[] adviceChain = { interceptor };
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queueName);
container.setAdviceChain(adviceChain);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
Receiver receiver() {
return new Receiver();
}
#Bean
MessageListenerAdapter listenerAdapter(Receiver receiver) {
MessageListenerAdapter adapter =
new MessageListenerAdapter(receiver, "receiveMessage");
return adapter;
}
#Bean
RetryOperations retryTemplate() {
Map<Class<? extends Throwable>, Boolean> retryableExceptions =
new HashMap<Class<? extends Throwable>, Boolean>();
retryableExceptions.put(EventException.class, false);
FixedBackOffPolicy backoffPolicy = new FixedBackOffPolicy();
backoffPolicy.setBackOffPeriod(5000);
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setBackOffPolicy(backoffPolicy);
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(5, retryableExceptions));
return retryTemplate;
}
#Bean
RetryOperationsInterceptor interceptor(RetryOperations retryTemplate) {
RetryOperationsInterceptor interceptor = new RetryOperationsInterceptor();
interceptor.setRecoverer(new CustomMessageRecover());
interceptor.setRetryOperations(retryTemplate);
return interceptor;
// return RetryInterceptorBuilder
// .stateless()
// //.retryOperations(retryTemplate)
// .maxAttempts(5)
// .recoverer(new CustomMessageRecover()).build();
}
public static void main(String[] args) throws InterruptedException {
SpringApplication.run(DemoApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
System.out.println("Sending message...");
rabbitTemplate.convertAndSend(queueName, "Hello from RabbitMQ!");
context.close();
}
public class Receiver {
public void receiveMessage(String message) throws Exception {
System.out.println("!!!!!!!!Message has been recieved!!!!!!");
throw new EventException("TESTING");
}
}
public class CustomMessageRecover implements MethodInvocationRecoverer<Void> {
#Override
public Void recover(Object[] args, Throwable cause) {
System.out.println("IN THE RECOVER ZONE!!!");
throw new AmqpRejectAndDontRequeueException(cause);
}
}
class EventException extends Exception {
private static final long serialVersionUID = 1L;
public EventException() {}
public EventException(String message) {
super(message);
}
}
}
Now in the code, as you can see I am using RetryOperationsInterceptor in order to intercept and check to see what type of exception it is being thrown and base on that, make the decision to either do the retry or not, along with the delay between the retries.
For this I am setting the backoffPolicy and retryPolicy of the RetryTemplate Bean and having that injected into the RetryOperationsInterceptor.
I would appreciate if anyone can help me out and tell me why the retry and the delay between the retries are not working. My messages are going directly to the dead letter exchange without the retries and the delays happening.
THANK YOU!
Your issue is here:
retryableExceptions.put(EventException.class, false);
Please, find the SimpleRetryPolicy code:
public boolean canRetry(RetryContext context) {
Throwable t = context.getLastThrowable();
return (t == null || retryForException(t)) && context.getRetryCount() < maxAttempts;
}
and further:
private boolean retryForException(Throwable ex) {
return retryableClassifier.classify(ex);
}
Since you specify a false for your EventException, it won't be retryable. Hence any retries and backoffs.
Related
How can I deserialize a message using convertSendAndReceive() method? It gives me NullPointerException due to not being able to find the required class for deserialization in another package. Packages are marked in the code.
Listener receives and sends messages normally
package org.dneversky.user;
#EnableRabbit
#Component
public class TestListener {
private static final Logger logger = LoggerFactory.getLogger(TestListener.class);
#Autowired
private RabbitTemplate rabbitTemplate;
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE)
public void doGet(UserReplyMessage message) {
logger.info("Received message: {}", message);
UserReplyMessage response = new UserReplyMessage();
logger.info("Sending message: {}", response);
rabbitTemplate.convertSendAndReceive(RabbitMQConfig.RPC_EXCHANGE,
RabbitMQConfig.REPLY_QUEUE, response);
}
}
Configuration of the listener
package org.dneversky.user.config;
#Configuration
public class RabbitMQConfig {
public static final String RECEIVE_QUEUE = "rpc_queue";
public static final String REPLY_QUEUE = "reply_queue";
public static final String RPC_EXCHANGE = "rpc_exchange";
#Bean
public TopicExchange rpcExchange() {
return new TopicExchange(RPC_EXCHANGE);
}
#Bean
public Queue receiveQueue() {
return new Queue(RECEIVE_QUEUE);
}
#Bean
public Queue replyQueue() {
return new Queue(REPLY_QUEUE);
}
#Bean
public Binding receiveBinding() {
return BindingBuilder.bind(receiveQueue()).to(rpcExchange()).with(RECEIVE_QUEUE);
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Sender sends a message normally, but it can't to deserialize returning message
package org.dneversky.gateway.servie.impl;
#Service
public class UserServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(UserServiceImpl.class);
#Autowired
private RabbitTemplate rabbitTemplate;
public UserPrincipal getUserByUsername(String username) {
UserResponse message = new UserResponse(username);
logger.info("Sending created message: {}", message);
UserResponse result = (UserResponse) rabbitTemplate.convertSendAndReceive(RabbitMQConfig.RPC_EXCHANGE, RabbitMQConfig.RPC_QUEUE, message);
logger.info("Getting response: {}", result);
return null;
}
}
Configuration of the Sender
package org.dneversky.gateway.config;
#Configuration
public class RabbitMQConfig {
public static final String RPC_QUEUE = "rpc_queue";
public static final String REPLY_QUEUE = "reply_queue";
public static final String RPC_EXCHANGE = "rpc_exchange";
#Bean
public Queue rpcQueue() {
return new Queue(RPC_QUEUE);
}
#Bean
public Queue replyQueue() {
return new Queue(REPLY_QUEUE);
}
#Bean
public TopicExchange rpcExchange() {
return new TopicExchange(RPC_EXCHANGE);
}
#Bean
public Binding binding() {
return BindingBuilder.bind(replyQueue()).to(rpcExchange()).with(REPLY_QUEUE);
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setExchange(RPC_EXCHANGE);
rabbitTemplate.setReplyAddress(REPLY_QUEUE);
rabbitTemplate.setReplyTimeout(6000);
rabbitTemplate.setMessageConverter(messageConverter());
return rabbitTemplate;
}
#Bean
public MessageConverter messageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public SimpleMessageListenerContainer replyContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(REPLY_QUEUE);
container.setMessageListener(rabbitTemplate(connectionFactory));
return container;
}
}
Error log
2022-05-22 17:12:31.344 ERROR 16920 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.amqp.support.converter.MessageConversionException: failed to resolve class name. Class not found [org.dneversky.user.model.UserReplyMessage]] with root cause
java.lang.ClassNotFoundException: org.dneversky.user.model.UserReplyMessage
by default, the producer set the _TypeID_ header as the class name used for the serialization of the object
then consumer uses _TypeID_ header to know the class that should use to convert the JSON to java instance
you use two different classes to serialize and deserialize the object and you have to configure the converter
inside your replyContainer I couldn't see your messageConverter bean. In default it uses java objects to send and receive messages without converting them into human readable json.
#Bean
public SimpleRabbitListenerContainerFactory customListenerContainerFactory(ConnectionFactory connectionFactory,
MessageConverter jsonMessageConverter) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(jsonMessageConverter);
return factory;
}
for your consumer;
#RabbitListener(queues = RabbitConstants.YOUR_QUEUE_NAME, containerFactory = "customListenerContainerFactory")
public void onMessage(#Valid YourEvent YourEvent){
//your code
}
Inside the Listener class, you need to add this line to bind your message converter
#Bean
public SimpleRabbitListenerContainerFactory jsaFactory(ConnectionFactory connectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
Also, in the TestListener class, you should replace this line
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE)
with this one
#RabbitListener(queues = RabbitMQConfig.RECEIVE_QUEUE,containerFactory="jsaFactory")
I have a challenge to set up a service in SpringBoot, where it will be listening to several queues. I searched a lot and couldn't find what I was looking for. I have queues, which can grow dynamically.
Exemple: queue-1, queue-2, queue-3...
What could I use in this service to get this service up by listening to these queues dynamically?
Using spring JMS you can do this like it is done here
Your config file is like this:
#Configuration
#EnableJms
public class AppConfig implements JmsListenerConfigurer {
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
List<QueueInformation> queueInformationList = consumersStatic.getQueueInformationList();
int i = 0;
for (QueueInformation queueInformation :
queueInformationList) {
SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId("myJmsEndpoint-" + i++);
endpoint.setDestination(queueInformation.getMqQueueName());
endpoint.setMessageListener(message -> {
logger.debug("***********************************************receivedMessage:" + message);
});
registrar.registerEndpoint(endpoint);
logger.debug("registered the endpoint for queue" + queueInformation.getMqQueueName());
}
}
Another way is using RabbitListenerConfigurer. You can get more idea from here
code from this link:
for rabbitconfig:
#Configuration
public class RabbitMqConfiguration implements RabbitListenerConfigurer {
#Autowired
private ConnectionFactory connectionFactory;
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
return rabbitTemplate;
}
#Bean
public RabbitAdmin rabbitAdmin() {
return new RabbitAdmin(connectionFactory);
}
#Bean
public RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry() {
return new RabbitListenerEndpointRegistry();
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(consumerJackson2MessageConverter());
return factory;
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Override
public void configureRabbitListeners(final RabbitListenerEndpointRegistrar registrar) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setPrefetchCount(1);
factory.setConsecutiveActiveTrigger(1);
factory.setConsecutiveIdleTrigger(1);
factory.setConnectionFactory(connectionFactory);
registrar.setContainerFactory(factory);
registrar.setEndpointRegistry(rabbitListenerEndpointRegistry());
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
}
service you can find here
I want to pragmatically control when to start/stop my kafka listeners.So looking through some previous posts and discussions it looks like I could use KafkaListenerEndpointRegistry.getListenerContainer(id).stop() to do that .However I verified that no containers are registered with my KafkaListenerEndpointRegistry bean.How do I register my container with KafkaListenerEndpointRegistry ?
#Autowired
KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;
#Bean
public KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry() {
kafkaListenerEndpointRegistry = new KafkaListenerEndpointRegistry();
return kafkaListenerEndpointRegistry;
}
#Bean
public ConcurrentKafkaListenerContainerFactory<?, ?>
kafkaListenerContainerFactory(
ConsumerFactory<String, SpecificRecord> kafkaConsumerFactory
) {
ConcurrentKafkaListenerContainerFactory<String, SpecificRecord> factory
= new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
factory.getContainerProperties().setTransactionManager(kafkaTransactionManager());
factory.getContainerProperties().setIdleEventInterval(60000L);
factory.getContainerProperties().setAckOnError(false);
factory.setRetryTemplate(getRetryTemplate());
factory.setConcurrency(2);
factory.getContainerProperties().setErrorHandler(rawLogsErrorHandler(KafkaTemplate));
return factory;
}
#Bean
KafkaTransactionManager<String,SpecificRecord> kafkaTransactionManager() {
return new KafkaTransactionManager<>(producerFactory());
}
#Bean
public RetryTemplate getRetryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
backOffPolicy.setBackOffPeriod(1000);
RetryPolicy retryPolicy = new SimpleRetryPolicy();
retryTemplate.setBackOffPolicy(backOffPolicy);
retryTemplate.setRetryPolicy(retryPolicy);
retryTemplate.registerListener(retryListener());
return retryTemplate;
}
#Bean
public LoggingErrorHandler rawLogsErrorHandler(KafkaTemplate<String,SpecificRecord> kafkaTemplate) {
return new LoggingErrorHandler() {
#SuppressWarnings({ "rawtypes", "unchecked" })
#Override
public void handle(Exception thrownException, ConsumerRecord<?,?> record) {
// record send to a dead letter here
//stop all listeners
kafkaListenerEndpointRegistry.stop();
}
#Bean
public LogReceiver receiver() {
return new LogReceiver();
}
// and on Logreciever class
public class Logreciever
#KafkaListener(topics = RAWLLOGTOPIC,id="rawLogConsumer",containerFactory="kafkaListenerContainerFactory")
public void onMessage(#Payload Log log,
#Header(KafkaHeaders.RECEIVED_PARTITION_ID) int partition,
#Header(KafkaHeaders.OFFSET) Long offset) throws Exception
{
//processing code
}
}
See the documentation. Only containers for #KafkaListeners are registered in the registry.
Containers retrieved from the factory as #Beans are registered with the application context.
If you manually create containers using the container factory, no registration is performed.
I am using Spring Boot framework. I want to send an object from a service to another service via RabbitMQ like this:
Service A:
rabbitTemplate.convertAndSend("queue", createAccountRequestMessage);
Service B:
#RabbitListener(queues = "queue")
public void onAccountRequested(#Valid CreateAccountRequestMessage createAccountRequestMessage, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG, long tag) throws IOException
{
}
In CreateAccountRequestMessage class I have defined some validation annotations like #NotEmpty, #NotNull and etc, but when I'm sending wrong message from service A to service B, #Valid annotation doesn't work and CreateAccountRequestMessage object is not validated before invoke onAccountRequested method.
You need to set the validator in DefaultMessageHandlerMethodFactory.
#Autowired
SmartValidator validator;
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(this.validator);
return factory;
}
Then you also need to specify the #Payload annotation along with the #Valid annotation.
#RabbitListener(queues = "queue")
public void onAccountRequested(#Valid #Payload CreateAccountRequestMessage
createAccountRequestMessage, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG, long tag) throws IOException
{
}
Now MethodArgumentNotValidException will be thrown and the message will be discarded, or you can send the message to a dead letter exchange.
I had the same problem. The answer of #Praveer works well except SmartValidator. I post here my solution, which is inspired by this article https://blog.trifork.com/2016/02/29/spring-amqp-payload-validation/
#Configuration
#EnableRabbit
#Slf4j
public class CmsMQConfig implements RabbitListenerConfigurer {
#Value("${dw.rabbitmq.hosts}")
private String hosts;
#Value("${dw.rabbitmq.username}")
private String username;
#Value("${dw.rabbitmq.password}")
private String password;
#Value("${dw.rabbitmq.virtual-host}")
private String virtualHost;
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses(hosts);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
connectionFactory.setVirtualHost(virtualHost);
return connectionFactory;
}
#Bean
public Jackson2JsonMessageConverter messageConverter() {
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JodaModule());
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
return new Jackson2JsonMessageConverter(mapper);
}
#Bean
public DefaultMessageHandlerMethodFactory defaultHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(amqpValidator());
return factory;
}
#Bean
public Validator amqpValidator() {
return new OptionalValidatorFactoryBean();
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setContainerFactory(rabbitListenerContainerFactory());
registrar.setMessageHandlerMethodFactory(defaultHandlerMethodFactory());
}
}
I have two different apps for sender and receiver.
sender:
#SpringBootApplication
public class RabbitJmsApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(RabbitJmsApplication.class, args);
}
#Autowired
private JmsTemplate template;
#Autowired
private JmsTemplate topicTemplate;
#Override
public void run(String... arg0) throws Exception {
for (int i = 0; i < 10; i++) {
template.convertAndSend("my_queue", "msg_" + i);
Thread.sleep(100);
}
for (int i = 0; i < 10; i++) {
topicTemplate.convertAndSend("my_topic", "topic_msg_" + i);
Thread.sleep(100);
}
}
#Bean
public RMQConnectionFactory connectionFactory() {
return new RMQConnectionFactory();
}
#Bean
public JmsTemplate template() {
return new JmsTemplate(connectionFactory());
}
#Bean
public JmsTemplate topicTemplate() {
final JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory());
jmsTemplate.setPubSubDomain(true);
return jmsTemplate;
}
}
and receiver:
#Component
public class Listener {
#JmsListener(destination = "my_queue")
public void receive(String str){
System.out.println(str);
}
#JmsListener(destination = "my_topic")
public void receiveTopic(String str){
System.out.println(str);
}
}
I see
msg_1
msg_2
...
on the receiver but I don't see the topic messages.
What am I doing wrong?
P.S.
management console:
Subscriptions to topics are not durable by default - you are probably sending the messages before the listener has started.
Try adding a Thread.sleep() before sending the messages to the topic.
My receiver became to receive mesagges after adding following bean to the context:
#Bean
public JmsListenerContainerFactory<?> myFactory(DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory());
// You could still override some of Boot's default if necessary.
factory.setPubSubDomain(true);
return factory;
}