We Keep Coding

Spring Boot JMS ActiveMQ redeliver on failover

I have a consumer application that use Spring Boot JMS to listen message from a queue. This application is connected to an ActiveMQ cluster with failover to have HA. But I have the following problem, when I shutdown one of the broker and the application is processing one the message this message is not dequeue from the queue, and when the application connect to the other broker the message is redelivered. The problem is that the message has been processed the first time and I don't need to process it again.
I've been searching for Acknowledgement modes and I tried to use client mode to force acknowledge before message processing. But I didn't work. Any idea??
I've declared the following beans:
#Bean
public ActiveMQConnectionFactory jmsConnectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerUrl);
connectionFactory.setUserName(brokerName);
connectionFactory.setPassword(brokerPassword);
connectionFactory.setTrustAllPackages(true);
return connectionFactory;
}
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(jmsConnectionFactory());
jmsTemplate.setSessionTransacted(false);
jmsTemplate.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
jmsTemplate.setDefaultDestinationName(REMOTE_T);
return jmsTemplate;
}
#Bean
public JmsListenerContainerFactory<?> jmsListenerContainerFactory(
#Qualifier("jmsConnectionFactory") ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setSessionTransacted(false);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
configurer.configure(factory, connectionFactory);
return factory;
}
My listener's code is:
#JmsListener(destination = "Consumer.consumer1.VirtualTopic.TopicPrueba", containerFactory="jmsListenerContainerFactory")
public void receiveMessageFromContacts(Message message) {
try {
message.acknowledge();
TextMessage txtMessage = (TextMessage)message;
mensajesConsumer1++;
System.out.println("First Consumer:"+ txtMessage.getText()+ " received:"+mensajesConsumer1);
}catch(JMSException e) {
e.printStackTrace();
}
}
I'm not sure if I've understood correctly the client acknowledge mode.
Please help! :)
Thanks in advance!

Giving incorrect(non-existing) queue name while publishing in RabbitMQ

I'm using AyncRabbitTemplate for publishing messages. Giving an incorrect(non-existing) queue name while publishing - it drops the message silently.
I have tried enabling "confirm" and "mandate" on the AyncRabbitTemplate and added the required callback methods as below:
#Bean
AsyncRabbitTemplate template() {
RabbitTemplate rabbit = rabbitTemplate();
rabbit.setChannelTransacted(true); //to throw error when channel shuts down in case of incorrect exchange names
AsyncRabbitTemplate asyncRabbitTemplate = new AsyncRabbitTemplate(rabbit, rpcReplyMessageListenerContainer(connectionFactory()));
asyncRabbitTemplate.setEnableConfirms(true);
asyncRabbitTemplate.setMandatory(true); //if the message cannot be delivered to a queue an AmqpMessageReturnedException will be thrown
return asyncRabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer rpcReplyMessageListenerContainer(ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer(connectionFactory);
simpleMessageListenerContainer.setQueueNames(Constants.REPLY_QUEUE);
simpleMessageListenerContainer.setTaskExecutor(Executors.newCachedThreadPool());
simpleMessageListenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return simpleMessageListenerContainer;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("localhost");
return connectionFactory;
}
And the callback methods as:
RabbitConverterFuture<String> future = this.asyncRabbitTemplate.convertSendAndReceive("",Constants.SNS_QUEUE, "This is the request message ",new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) {
message.getMessageProperties().setTimestamp(new Date());
message.getMessageProperties().setMessageId(UUID.randomUUID().toString());
return message;
}
});
ListenableFuture<Boolean> future2 = future.getConfirm();
future2.addCallback(new ListenableFutureCallback<Boolean>() {
#Override
public void onSuccess(Boolean result) {
System.out.println("Publish Result " + result);
}
#Override
public void onFailure(Throwable ex) {
System.out.println("Publish Failed: " + ex);
}
});

As discussed in the documentation you have to enable returned messages on the connection factory.
The send will not throw an exception but undeliverable messages will be returned to the ReturnCallback (if mandatory is true).
Confirms are not sent for undeliverable messages. A negative confirmation is only received if there is some kind of problem in the broker; they are rare.

JMS Local Transactions not working when using a CachingConnectionFactory in Spring Integration

I am using CachingConnectionFactory to cache my JMS Broker Connections for IBM MQ. Messages under the local transaction are not rolled back even if I throw an explicit exception. But when I remove the Caching and use just a plain JMS Connection Factory the message is rolled back in case an Exception is thrown.
#Gary Russell specified that we can simply throw an exception to roll back any message in transaction.
Gary's Answer
edit:
This is how is set up my JMS Broker :
#Bean
public IntegrationFlow primaryInitiationListenerFlow() {
return IntegrationFlows.from(Jms
.messageDrivenChannelAdapter(
context.getBean("connection" + environment.getProperty("primaryConnection"), ConnectionFactory.class),
DefaultMessageListenerContainer.class)
.autoStartup(false)
.destination(environment.getProperty("sourceName"))
.configureListenerContainer(listenerContainerSpec -> listenerContainerSpec
.destinationResolver((session, destinationName, pubSubDomain) -> destinationName.toUpperCase().endsWith("TOPIC") ?
session.createTopic(destinationName) : session.createQueue(destinationName))
.subscriptionDurable(false))
.id(environment.getProperty("id") + "PrimaryIn")
.get())
.channel("idEnrichmentChannel")
.get();
}
#Bean
public ConnectionFactory connection301() {
MQConnectionFactory factory = new MQConnectionFactory();
try {
factory.setHostName("xxxxxxx");
factory.setPort(1416);
factory.setQueueManager("xxxxxxx");
factory.setChannel("xxxxxxx");
factory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
} catch (JMSException e) {
e.printStackTrace();
}
return factory;
}
Here is my configuration with CachingConnectionFactory :
#Bean
public JmsTemplate template301() {
CachingConnectionFactory cachedConnection = new CachingConnectionFactory();
cachedConnection.setTargetConnectionFactory(connection301());
return new JmsTemplate(cachedConnection);
}
And this is post removal of CachingConnectionFactory :
#Bean
public JmsTemplate template301() {
return new JmsTemplate(connection301());
}

RabbitMQ slow receiving speed

I need to create application which should receive data from a data producer and process it. I chose RabbitMQ as a message-broker. My tests show me not the best results:
Sent - 100 msg;
Produce - 100 msg/s;
Consume - 6 msg/s;
To solve it I set listenerContainer.setAcknowledgeMode(AcknowledgeMode.NONE);
But I need acknowledge for some queues. And I can't do it in parallel using workers, because the order of messages is important for the data processing.
Is it possible to increase the speed of receiving, having acknowledges?
Producer:
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("ip");
connectionFactory.setUsername("name");
connectionFactory.setPassword("pswd");
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
public FanoutExchange exchange() {
return new FanoutExchange("exchange-1");
}
#Bean
public Binding binding(){
return BindingBuilder.bind(queue()).to(exchange());
}
...
rabbitTemplate.setExchange("exchange-1");
rabbitTemplate.convertAndSend(data);
Consumer:
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("ip");
connectionFactory.setUsername("name");
connectionFactory.setPassword("pswd");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
Queue queue() {
return new Queue("queue-1", false);
}
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(queue());
listenerContainer.setMessageListener(new Receiver());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
...
#Override
public void onMessage(Message message) {
System.out.println("Received message: " + fromBytes(message.getBody()) + " \n Time = " + System.currentTimeMillis());
}
Tested on instance with 2 vCPU and 4 Gb memory.

You can increase the container's prefetchCount, which will greatly improve performance. However, if you reject and requeue a message, ordering will be lost (the requeued message will be behind the prefetched messages).

Re downloading of message when using spring integration inbound channel adapter

I am reading mails using spring mail inbound channel adapter once message is read i am performing some db operations in service activator of corresponding channel. My requirement is if any db operation fails adapter should read same message again.
Mail configuration :
#Bean
public DirectChannel inputChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow pop3MailFlow() {
String url = "[url]";
return IntegrationFlows
.from(Mail.pop3InboundAdapter(url)
.javaMailProperties(p -> p.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory")),e -> e.autoStartup(true)
.poller(Pollers.fixedDelay(2000).transactionSynchronizationFactory(transactionSynchronizationFactory())))
.channel(inputChannel())
.handle(inboundEmailProcessor(),"messageProcess")
.get();
}
#Bean
public InboundEmailProcessor inboundEmailProcessor() {
return new InboundEmailProcessor();
}
#Bean
public TransactionSynchronizationFactory transactionSynchronizationFactory() {
TransactionSynchronizationFactory synchronizationFactory = new DefaultTransactionSynchronizationFactory(expressionEvaluatingTransactionSynchronizationProcessor());
return synchronizationFactory;
}
#Bean
public ExpressionEvaluatingTransactionSynchronizationProcessor expressionEvaluatingTransactionSynchronizationProcessor() {
ExpressionEvaluatingTransactionSynchronizationProcessor processor = new ExpressionEvaluatingTransactionSynchronizationProcessor();
ExpressionParser parser = new SpelExpressionParser();
processor.setAfterRollbackExpression(parser.parseExpression("new com.muraai.ex.config.Exp().process(payload)"));
return processor;
}
public class InboundEmailProcessor {
#Autowired
AttachmentsRepository attachmentsRepository;
#Transactional(rollbackFor = Exception.class)
public void messageProcess() {
// some db operations
// if it fails the same message should be read again
}
}
I thought this would work but its not working. Is there any way to achieve my requirement
public class Exp {
public void process(MimeMessage message) throws MessagingException {
message.setFlag(Flags.Flag.SEEN, false);
}
}

You need IMAP for that; with POP3, the server always marks them read.
You can add a spring-retry interceptor advice to the poller's advice chain and/or send the failed message to an error channel.
The retry advice can be configured for number of retries, back off policy etc.