We Keep Coding

How to understand async vs sync in Spring AMQP?

I'm currently reading through Spring AMQP's official sample project along with it's corresponding explanations from Spring AMQP docs. The project involves an sync and async version, and the two only differs slightly. Here's the async version:
Producer config:
#Configuration
public class ProducerConfiguration {
protected final String helloWorldQueueName = "hello.world.queue";
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(this.helloWorldQueueName);
return template;
}
#Bean
public ConnectionFactory connectionFactory() {
return new CachingConnectionFactory();
}
#Bean
public ScheduledProducer scheduledProducer() {
return new ScheduledProducer();
}
#Bean
public BeanPostProcessor postProcessor() {
return new ScheduledAnnotationBeanPostProcessor();
}
static class ScheduledProducer {
#Autowired
private volatile RabbitTemplate rabbitTemplate;
private final AtomicInteger counter = new AtomicInteger();
#Scheduled(fixedRate = 3000)
public void sendMessage() {
rabbitTemplate.convertAndSend("Hello World " + counter.incrementAndGet());
}
}
}
Consumer config:
#Configuration
public class ConsumerConfiguration {
protected final String helloWorldQueueName = "hello.world.queue";
#Bean
public ConnectionFactory connectionFactory() {
return new CachingConnectionFactory();
}
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setQueueNames(this.helloWorldQueueName);
container.setMessageListener(new MessageListenerAdapter(new HelloWorldHandler()));
return container;
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(this.helloWorldQueueName);
template.setDefaultReceiveQueue(this.helloWorldQueueName);
return template;
}
#Bean
public Queue helloWorldQueue() {
return new Queue(this.helloWorldQueueName);
}
}
HelloWorldHandler:
public class HelloWorldHandler {
public void handleMessage(String text) {
System.out.println("Received: " + text);
}
}
As the docs explains:
Since this sample demonstrates asynchronous message reception, the producing side is designed to continuously send messages (if it were a message-per-execution model like the synchronous version, it would not be quite so obvious that it is, in fact, a message-driven consumer). The component responsible for continuously sending messages is defined as an inner class within the ProducerConfiguration. It is configured to run every three seconds.
I failed to understand what's "async" about this code, since, from my understanding, in a basic "synchronous fashion", operations like amqpTemplate.converAndSend() and amqpTemplate.receiveAndConvert() already peforms Rabbitmq's async actions, neither producer nor consumer are blocking when sending/receiving messages.
So, how's async in this example manifested? And how to understand async vs sync in the Spring AMQP context?

With async, the MessageListener is invoked by the framework; messages arrive whenever they are available.
With sync, the application calls a receive method which either returns immediately if no message is available, or blocks until a message arrives or a timeout expires.
In the sync case, the application controls when messages are received, with async, the framework is in control.

Spring-Boot 2.7 Redis PUB/SUB fails startup on missing Redis connection

I have this configuration for my pub/sub implementation:
#Bean
public RedisMessageListenerContainer container(LettuceConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.addMessageListener(listenerAdapter, new ChannelTopic(publishChannel));
return container;
}
#Bean
public MessageListenerAdapter listenerAdapter(RedisReceiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
#Bean
public StringRedisTemplate template(LettuceConnectionFactory connectionFactory) {
return new StringRedisTemplate(connectionFactory);
}
This code worked fine until I updated to Spring-Boot 2.7 (previously 2.6.7).
Now this code throws the following error on startup, when my Redis is not running:
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'container'; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect to localhost/:6379
("container" is the Bean at the top in my code snippet)
Where or how can I configure that it catches the thrown exception on startup and just retries it again until the connection to Redis is available?
UPDATE:
Did some digging and this part of the code is the culprit:
https://github.com/spring-projects/spring-data-redis/blob/main/src/main/java/org/springframework/data/redis/listener/RedisMessageListenerContainer.java#L1169
In comparison to the 2.6.x branch, where this was inside a try/catch (not sure though as it seems to be a completely different implementation), it is missing in version 2.7.

I think you have to wait for the next release, someone has fixed it and just merged into maim

It looks like the PR didn't fix the issue in the latest release (2.7.1), it really "works as intended" to Spring team :\
Here's my workaround:
RedisMessageListenerContainer.Subscriber#initialize(...) method is called when adding listener to the container, so what you need is to make sure the redis connection is available first, and then add the listener to container.
#AllArgsConstructor
public class MessageListenerSubscriber implements ApplicationListener<ApplicationReadyEvent> {
private RedisConnectionFactory connectionFactory;
private RedisMessageListenerContainer container;
private MessageListener listener;
private Collection<? extends Topic> topics;
#Async
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
var template = new RetryTemplateBuilder()
.maxAttempts(Integer.MAX_VALUE)
.fixedBackoff(5000)
.build();
template.execute(context -> {
try {
var connection = connectionFactory.getConnection();
if (connection.isSubscribed()) {
log.debug("Retrieved connection is already subscribed; aborting listening");
return null;
}
} catch (Exception e) {
log.error("Connection failure occurred. Restarting subscription task after 5000 ms");
throw e;
}
this.container.addMessageListener(listener, topics);
log.debug("Listeners registered successfully after {} retries.", context.getRetryCount());
return null;
});
}
}
and the configuration looks like:
#Bean
public RedisMessageListenerContainer container(LettuceConnectionFactory connectionFactory) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
return container;
}
#Bean
public MessageListenerAdapter listenerAdapter(RedisReceiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
#Bean
public MessageListenerSubscriber messageListenerSubscriber(
RedisConnectionFactory connectionFactory,
RedisMessageListenerContainer container,
MessageListenerAdapter listener){
return new MessageListenerSubscriber(
connectionFactory,
container,
listener,
List.of(new ChannelTopic(publishChannel))
);
}

Dispatcher has no subscribers for outgoing sftp channel

I'm using spring-integration-sftp and my goal is to push local file to SFTP (just that for now, without confirmation or anything else). My configuration is as follows:
#EnableIntegration
#IntegrationComponentScan
#Configuration
#Lazy
public class SftpConfiguration {
#Bean(name = "toSftpChannel")
public MessageChannel sftpMessageChannel() {
return new DirectChannel();
}
#Bean
public DefaultSftpSessionFactory sftpSessionFactory(
#Qualifier("sftpDestination") SftpPropertiesService sftpPropertiesService
) {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(sftpPropertiesService.getServiceHost());
factory.setPort(22);
factory.setUser(sftpPropertiesService.getUsername());
factory.setPassword(sftpPropertiesService.getPassword());
factory.setAllowUnknownKeys(true);
return factory;
}
#Bean
public SftpRemoteFileTemplate sftpRemoteFileTemplate(DefaultSftpSessionFactory dssf,
#Value("${sftp.output.directory}") String outputDirectory) {
SftpRemoteFileTemplate template = new SftpRemoteFileTemplate(dssf);
template.setRemoteDirectoryExpression(new LiteralExpression(outputDirectory));
return template;
}
#Bean
#ServiceActivator(inputChannel = "toSftpChannel")
public MessageHandler handler(SftpRemoteFileTemplate sftpRemoteFileTemplate) {
SftpOutboundGateway gateway =
new SftpOutboundGateway(sftpRemoteFileTemplate, Command.PUT.getCommand(), "payload");
gateway.setFileExistsMode(FileExistsMode.FAIL);
return gateway;
}
#MessagingGateway
public interface OutputSftpGateway {
#Gateway(requestChannel = "toSftpChannel")
void sendToSftp(File file);
}
}
and sending is just
private final OutputSftpGateway outputSftpGateway;
...
outputSftpGateway.sendToSftp(file);
When I'm running my code I at first get
A bean definition with name 'toSftpChannel' exists, but failed to be created; nested exception is org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'toSftpChannel': Requested bean is currently in creation: Is there an unresolvable circular reference?
which is kind of expected with lazy init (though still will have to fix it), but at the second and subsequent runs I'm getting stuck with
Exception occurred during request processing. org.springframework.messaging.MessageDeliveryException. Dispatcher has no subscribers for channel 'application.toSftpChannel'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
I'll honestly admit that I'm new with Spring messaging magic, so the cause is probably very stupid but can someone give me a hint why this is happening and how can I fix it?

The #Lazy may have some effect on those beans initialization. Consider to divide your configuration logic to extract only those beans which cannot live with #Lazy. And don't apply it for those Spring Integration components.

Close connections related to SimpleMessageListenerContainer in Spring AMQP

I am currently working on an event-based async AMQP message listener, like this:
#Configuration
public class ExampleAmqpConfiguration {
#Bean(name = "container")
public SimpleMessageListenerContainer messageListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueueName("some.queue");
container.setMessageListener(exampleListener());
return container;
}
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public MessageListener exampleListener() {
return new MessageListener() {
public void onMessage(Message message) {
System.out.println("received: " + message);
}
};
}
}
I have changed the container bean's autostart property to false. And I have autowired this bean to an event lister that starts the container when StartEvent happens:
#EventListener
public void startContainer(StartEvent startEvent) {
this.container.start();
}
At the same time, I have also autowired the bean to another event that stops the container and shutdowns the container, hoping that the container will be stopped and that there will be no lingering connection:
#EventListener
public void endContainer(EndEvent endEvent) {
this.container.stop();
this.container.shutdown();
}
However, after the EndEvent, I find in my RabbitMQ admin console that all the channels are closed but there is still a lingering connection.
Does that mean that shutdown() is not the right method to use to remove the lingering connection? If that's the case, what is the right method to use?
Thanks.

You need to call resetConnection() on the CachingConnectionFactory to close the connection; as implied by the class name; the connection is cached.

spring rabbit and spring transactions

I have a spring rabbit consumer like:
#Override public void onMessage(Message amqpMessage, Channel channel)
throws Exception {
//..some code goes here - I want it to be in spring transaction
}
The issue is the code which is in onMessage method is not under transaction. I checked it, I save data to 3 tables, then throw exception, then save to 4th table. And data from 3 previos tables is not being rolled back. How to do that properly in spring? I want all code in onMessage method to be within a transaction. Thanks
UPDATE
My rabbit conf:
#Configuration #ComponentScan(basePackages = {"com.mycompany"})
public class TicketModeRabbit {
#Bean TicketModeConsumer ticketModeConsumer() {
return new TicketModeConsumer();
}
#Bean(name = TicketModeRabbitData.QUEUE_BEAN_NAME) Queue queue() {
return new Queue(TicketModeRabbitData.QUEUE_BEAN_NAME);
}
#Bean(name = TicketModeRabbitData.QUEUE_BINDING_NAME) Binding binding(
#Qualifier(TicketModeRabbitData.QUEUE_BEAN_NAME) Queue q, TopicExchange e) {
return BindingBuilder.bind(q).to(e).with(TicketModeRabbitData.QUEUE_TOKEN_NAME);
}
#Bean(name = TicketModeRabbitData.CONTAINER_NAME)
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,
#Qualifier(TicketModeRabbitData.LISTENER_ADAPTED_NAME)
MessageListenerAdapter listenerAdapter) {
return WorkerConfigHelper
.rabbitConfigListenerContainer(connectionFactory, listenerAdapter,
TicketModeRabbitData.QUEUE_BEAN_NAME,
WorkerConfigHelper.GLOBAL_CONCURRENT_CONSUMERS);
}
#Bean(name = TicketModeRabbitData.LISTENER_ADAPTED_NAME)
MessageListenerAdapter listenerAdapter() {
return new MessageListenerAdapter(ticketModeConsumer());
}
}

If your transaction manager is properly set up for your database, the only thing you need to do is add the #Transactional annotation on the onMessage method. Note that the consumer (MessageListener) needs to be a bean managed by the Spring container.
#Override
#Transactional
public void onMessage(Message amqpMessage, Channel channel)
throws Exception {
//..some code goes here - I want it to be in spring transaction
}