My Sring Boot application listens Amazon SQS queue. Right now I need to implement correct message acknowledgement - I need to receive a message, do some business logic a only after that in case of success I need to ack the message(delete the message from the queue). For example, in case of error in my business logic the message must be re-enqueued.
This is my SQS config:
/**
* AWS Credentials Bean
*/
#Bean
public AWSCredentials awsCredentials() {
return new BasicAWSCredentials(accessKey, secretAccessKey);
}
/**
* AWS Client Bean
*/
#Bean
public AmazonSQS amazonSQSAsyncClient() {
AmazonSQS sqsClient = new AmazonSQSClient(awsCredentials());
sqsClient.setRegion(Region.getRegion(Regions.US_EAST_1));
return sqsClient;
}
/**
* AWS Connection Factory
*/
#Bean
public SQSConnectionFactory connectionFactory() {
SQSConnectionFactory.Builder factoryBuilder = new SQSConnectionFactory.Builder(
Region.getRegion(Regions.US_EAST_1));
factoryBuilder.setAwsCredentialsProvider(new AWSCredentialsProvider() {
#Override
public AWSCredentials getCredentials() {
return awsCredentials();
}
#Override
public void refresh() {
}
});
return factoryBuilder.build();
}
/**
* Registering QueueListener for queueName
*/
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer() {
DefaultMessageListenerContainer messageListenerContainer = new DefaultMessageListenerContainer();
messageListenerContainer.setConnectionFactory(connectionFactory());
messageListenerContainer.setMessageListener(new MessageListenerAdapter(new MyQueueListener()));
messageListenerContainer.setDestinationName(queueName);
return messageListenerContainer;
}
My queue listener:
public class MyQueueListener {
public void handleMessage(String messageContent) throws JMSException {
//do some job
//TODO: ack the message
}
}
Right now I don't know how to ack the message from my listener.
Normally DefaultMessageListenerContainer acknowledges the message before or after execution of handleMessage automatically. So you don't need to do anything.
But recommended with DefaultMessageListenerContainer is to use transactions instead of client ack mode. Not sure if Amazon SQS has such option.
Related
This is my JMS configuration:
#EnableJms
#Configuration
public class VmsJmsConfig implements JmsListenerConfigurer {
#Value("${spring.activemq.broker-url}")
String brokerUrl;
#Value("${spring.activemq.ssl.trustStorePath}")
String trustStorePath;
#Value("${spring.activemq.ssl.trustStorePass}")
String trustStorePass;
#Bean
public DefaultJmsListenerContainerFactory defaultJmsListenerContainerFactory(ConnectionFactory conFactory) {
DefaultJmsListenerContainerFactory defaultJmsListenerContainerFactory = new DefaultJmsListenerContainerFactory();
defaultJmsListenerContainerFactory.setConnectionFactory(conFactory);
defaultJmsListenerContainerFactory.setConcurrency("10-20");
return defaultJmsListenerContainerFactory;
}
#Bean("conFactory")
public ConnectionFactory activeMQSslConnectionFactory() throws Exception {
ActiveMQSslConnectionFactory activeMQSslConnectionFactory = new ActiveMQSslConnectionFactory(brokerUrl);
activeMQSslConnectionFactory.setTrustStore(trustStorePath);
activeMQSslConnectionFactory.setTrustStorePassword(trustStorePass);
return activeMQSslConnectionFactory;
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public MessageConverter messageConverter() {
return new MappingJackson2MessageConverter();
}
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
My app is listening to an ActiveMQ queue. It consumes a message, transforms it, sends a downstream request, waits for the response, and sends that response to another queue.
I want it to consume multiple messages at the same time and process them in parallel, but no matter how many consumer I set in setConcurrency() it only consumes 1 message at a time, and there are more than 1000 messages pending in queue.
I tried changing the concurency, but no luck. But when I comment downstream call, it consumes 10-20 messages at a time, I couldn't find reason for that.
Using JMSConfig i am creating MQ Connection factory and i have InboundGatewayConfig and OutboundGatewayConfig , in Inbound Config i am reading the Message payload from one queue and in Outbound config sending the message to another queue and my goad is toset reply channel to acknowledge the sender queue once queue receive the message.
InboundGatewayConfig:
#Configuration
#EnableIntegration
public class InboundGatewayConfig {
#Autowired
JmsConfig jmsConfig;
#Value("${fcb.inbound.receiver.queue.name}")
private String orderRequestDestination;
#Bean
public MessageChannel inboundOrderRequestChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel inboundOrderResponseChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "inboundOrderRequestChannel")
public OrderService orderService() {
return new OrderService();
}
#Bean
public JmsInboundGateway jmsInboundGateway() {
JmsInboundGateway gateway = new JmsInboundGateway(
defaultMessageListenerContainer(),
channelPublishingJmsMessageListener());
gateway.setRequestChannel(inboundOrderRequestChannel());
return gateway;
}
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConfig.fcbCachingConnectionFactory());
container.setDestinationName(orderRequestDestination);
container.setRecoveryInterval(5000);
container.setErrorHandler(t -> System.out.println("Error in JMS Configurations \t" + t.getCause()));
return container;
}
#Bean
public ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener() {
ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener = new ChannelPublishingJmsMessageListener();
channelPublishingJmsMessageListener.setExpectReply(true);
return channelPublishingJmsMessageListener;
}
}
OutboundGatewayConfig :
#Configuration
public class OutboundGatewayConfig {
#Value("${fcb.inbound.receiver.queue.name}")
private String orderRequestDestination;
#Value("${fcb.inbound.response.queue.name}")
private String orderResponseDestination;
#Autowired
JmsConfig jmsConfig;
#Bean
public MessageChannel outboundOrderRequestChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel outboundOrderResponseChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "outboundOrderRequestChannel")
public JmsOutboundGateway jmsOutboundGateway() {
JmsOutboundGateway gateway = new JmsOutboundGateway();
gateway.setConnectionFactory(jmsConfig.fcbCachingConnectionFactory());
gateway.setRequestDestinationName(orderRequestDestination);
gateway.setReplyDestinationName(orderResponseDestination);
gateway.setReplyChannel(outboundOrderResponseChannel());
return gateway;
}
}
and this my service class
public class OrderService {
private static final Logger LOGGER =
LoggerFactory.getLogger(OrderService.class);
public Message<?> order(Message<?> order) {
LOGGER.info("received order='{}'", order);
Message<?> status = MessageBuilder.withPayload("Accepted")
.setHeader("jms_correlationId",
order.getHeaders().get("jms_messageId"))
.setReplyChannelName("inboundOrderResponseChannel").build();
LOGGER.info("sending status='{}'", status);
return status;
}
}
While running the application InboundGateway able to receive the message payload from request queue but while reply , giving bellow warning message.
2022-04-07 16:06:24.303 WARN 19408 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'xxx.yyy.queue' - trying to recover. Cause: Cannot determine reply destination: Request message does not contain reply-to destination, and no default reply destination set.
I have simple RabbitMQ configuration in Spring (not SpringBoot):
#Bean
public Queue queueTest() {
return QueueBuilder.durable("test")
.withArgument("x-dead-letter-exchange", "myexchange")
.withArgument("x-dead-letter-routing-key", "mykey")
.build();
}
#Bean
public Queue queueTestDlq() {
return new Queue("test.dlq", true);
}
#Bean
public Binding bindingTest(DirectExchange directExchange, Queue queueTest) {
return BindingBuilder
.bind(queueTest)
.to(directExchange)
.with("testkey");
}
#Bean
public Binding bindingTestDlq(DirectExchange directExchange, Queue queueTestDlq) {
return BindingBuilder
.bind(queueTestDlq)
.to(directExchange)
.with("testdlqkey");
}
Processing works properly and message is moved to DLQ in case of exception.
How can I append exception details (eg. message, stacktrace) to message headers for message going to DLQ?
I'm using AyncRabbitTemplate for publishing messages. Giving an incorrect(non-existing) queue name while publishing - it drops the message silently.
I have tried enabling "confirm" and "mandate" on the AyncRabbitTemplate and added the required callback methods as below:
#Bean
AsyncRabbitTemplate template() {
RabbitTemplate rabbit = rabbitTemplate();
rabbit.setChannelTransacted(true); //to throw error when channel shuts down in case of incorrect exchange names
AsyncRabbitTemplate asyncRabbitTemplate = new AsyncRabbitTemplate(rabbit, rpcReplyMessageListenerContainer(connectionFactory()));
asyncRabbitTemplate.setEnableConfirms(true);
asyncRabbitTemplate.setMandatory(true); //if the message cannot be delivered to a queue an AmqpMessageReturnedException will be thrown
return asyncRabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer rpcReplyMessageListenerContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer(connectionFactory);
simpleMessageListenerContainer.setQueueNames(Constants.REPLY_QUEUE);
simpleMessageListenerContainer.setTaskExecutor(Executors.newCachedThreadPool());
simpleMessageListenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return simpleMessageListenerContainer;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("localhost");
return connectionFactory;
}
And the callback methods as:
RabbitConverterFuture<String> future = this.asyncRabbitTemplate.convertSendAndReceive("",Constants.SNS_QUEUE, "This is the request message ",new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) {
message.getMessageProperties().setTimestamp(new Date());
message.getMessageProperties().setMessageId(UUID.randomUUID().toString());
return message;
}
});
ListenableFuture<Boolean> future2 = future.getConfirm();
future2.addCallback(new ListenableFutureCallback<Boolean>() {
#Override
public void onSuccess(Boolean result) {
System.out.println("Publish Result " + result);
}
#Override
public void onFailure(Throwable ex) {
System.out.println("Publish Failed: " + ex);
}
});
As discussed in the documentation you have to enable returned messages on the connection factory.
The send will not throw an exception but undeliverable messages will be returned to the ReturnCallback (if mandatory is true).
Confirms are not sent for undeliverable messages. A negative confirmation is only received if there is some kind of problem in the broker; they are rare.
I am reading mails using spring mail inbound channel adapter once message is read i am performing some db operations in service activator of corresponding channel. My requirement is if any db operation fails adapter should read same message again.
Mail configuration :
#Bean
public DirectChannel inputChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow pop3MailFlow() {
String url = "[url]";
return IntegrationFlows
.from(Mail.pop3InboundAdapter(url)
.javaMailProperties(p -> p.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory")),e -> e.autoStartup(true)
.poller(Pollers.fixedDelay(2000).transactionSynchronizationFactory(transactionSynchronizationFactory())))
.channel(inputChannel())
.handle(inboundEmailProcessor(),"messageProcess")
.get();
}
#Bean
public InboundEmailProcessor inboundEmailProcessor() {
return new InboundEmailProcessor();
}
#Bean
public TransactionSynchronizationFactory transactionSynchronizationFactory() {
TransactionSynchronizationFactory synchronizationFactory = new DefaultTransactionSynchronizationFactory(expressionEvaluatingTransactionSynchronizationProcessor());
return synchronizationFactory;
}
#Bean
public ExpressionEvaluatingTransactionSynchronizationProcessor expressionEvaluatingTransactionSynchronizationProcessor() {
ExpressionEvaluatingTransactionSynchronizationProcessor processor = new ExpressionEvaluatingTransactionSynchronizationProcessor();
ExpressionParser parser = new SpelExpressionParser();
processor.setAfterRollbackExpression(parser.parseExpression("new com.muraai.ex.config.Exp().process(payload)"));
return processor;
}
public class InboundEmailProcessor {
#Autowired
AttachmentsRepository attachmentsRepository;
#Transactional(rollbackFor = Exception.class)
public void messageProcess() {
// some db operations
// if it fails the same message should be read again
}
}
I thought this would work but its not working. Is there any way to achieve my requirement
public class Exp {
public void process(MimeMessage message) throws MessagingException {
message.setFlag(Flags.Flag.SEEN, false);
}
}
You need IMAP for that; with POP3, the server always marks them read.
You can add a spring-retry interceptor advice to the poller's advice chain and/or send the failed message to an error channel.
The retry advice can be configured for number of retries, back off policy etc.