I am now working on a project with rabbitmq, Spring, and Hibernate, where once my program is running, the user can change a boolean field in the database.
If true, my program will create a new queue and bind an predetermined exchange to it.
If false, my program will unbind the queue and delete it.
However, the tutorials I have seen all seem to use annotation to create the queue and the binding when the program first runs:
import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
#Profile({"tut3", "pub-sub", "publish-subscribe"})
#Configuration
public class Tut3Config {
#Bean
public FanoutExchange fanout() {
return new FanoutExchange("tut.fanout");
}
#Profile("receiver")
private static class ReceiverConfig {
#Bean
public Queue autoDeleteQueue1() {
return new AnonymousQueue();
}
#Bean
public Queue autoDeleteQueue2() {
return new AnonymousQueue();
}
#Bean
public Binding binding1(FanoutExchange fanout,
Queue autoDeleteQueue1) {
return BindingBuilder.bind(autoDeleteQueue1).to(fanout);
}
#Bean
public Binding binding2(FanoutExchange fanout,
Queue autoDeleteQueue2) {
return BindingBuilder.bind(autoDeleteQueue2).to(fanout);
}
#Bean
public Tut3Receiver receiver() {
return new Tut3Receiver();
}
}
#Profile("sender")
#Bean
public Tut3Sender sender() {
return new Tut3Sender();
}
}
In my case, rather than using annotations, should I implement interfaces such as AmqpAdmin and use methods such as declareQueue() and deleteQueue explicitly to be able to create and delete queue constantly?
If that's the case, does Spring have a specific place in projects to implment those methods?
Thanks.
In your example, the queues beans autoDeleteQueue1 and autoDeleteQueue1 are created with scope singleton and so they live within Spring container for a whole app lifecycle.
If you need more flexible way to handle queue beans, you can use the default implementation of AmqpAdmin, like it's described below.
if (condition) {
Queue queue = admin.declareQueue(new Queue("queueOne"));
admin.declareBinding(new Binding(queue, fanout))
} else {
admin.removeBindings(new Binding(new Queue("queueOne"), fanout))
}
You may want to put this code into some Service or whatever the class which handles this logic. Within that service you may wire the beans
#Bean
public ConnectionFactory connectionFactory() {
return new CachingConnectionFactory();
}
#Bean
public AmqpAdmin admin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public FanoutExchange fanout() {
return new FanoutExchange("tut.fanout");
}
More details can be found here
Here's also nice article with nice examples
If you look at the tutorial for RabbitMQ using Java, you will find #2 and #5 tutorial apply to your case.
For example, to create a queue you could use this:
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(TASK_QUEUE_NAME, true, false, false, null);
To create binding, you can do this:
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
String queueName = channel.queueDeclare().getQueue();
if (argv.length < 1) {
System.err.println("Usage: ReceiveLogsTopic [binding_key]...");
System.exit(1);
}
for (String bindingKey : argv) {
channel.queueBind(queueName, EXCHANGE_NAME, bindingKey);
}
For your case, basically you want to explictly create the channel and create/bind explicitly rather than through beans.
Each of those operations can be its own method gathered in a service class.
Then you can simply utilize the service class for using each method. This way you can create/bind anytime you want.
Related
We're using RabbitMq for communication between some of our services. Sometimes there are a lot of messages beeing queued at once. We want to be able to see that there are still unhandled messages, i.e. if the Service handling the messages is busy.
I've been looking around for a programmatical way to check if a queue has messages and found this
channel.queueDeclarePassive(queueName).getMessageCount()
The problem is: I dont have a channel object. Our RabbitMq setup has been created a couple of years ago and usually looks like this:
#Configuration
#EnableRabbit
public class RabbitMqConfig {
public static final String RENDER_HTML_QUEUE = "render.html";
private String rabbitUri;
private int connectionTimeout;
private String exchangeName;
private int concurrentConsumers;
public RabbitMqConfig(
#Value("${rabbitmq.uri}") String rabbitUri,
#Value("${rabbitmq.exchange.name}") String exchangeName,
#Value("${rabbitmq.connection.timeout}") int timeout,
#Value("${rabbitmq.concurrent-consumers:1}") int concurrentConsumers) {
this.exchangeName = exchangeName;
this.rabbitUri = rabbitUri;
this.connectionTimeout = timeout;
this.concurrentConsumers = concurrentConsumers;
}
#Bean
DirectExchange directExchangeBean() {
return new DirectExchange(this.exchangeName, true, false);
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(RENDER_HTML_QUEUE);
container.setConcurrentConsumers(concurrentConsumers);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(RenderItemMessageConsumer receiver) {
return new MessageListenerAdapter(receiver, "reciveMessageFromRenderQueue");
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory;
try {
connectionFactory = new CachingConnectionFactory(new URI(this.rabbitUri));
connectionFactory.setConnectionTimeout(this.connectionTimeout);
} catch (URISyntaxException e) {
throw new ApiException(e, BaseErrorCode.UNKOWN_ERROR, e.getMessage());
}
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
public Queue renderRenderQueue() {
return new Queue(RENDER_HTML_QUEUE, true);
}
#Bean
Binding rendererRenderBinding() {
return BindingBuilder.bind(renderRenderQueue()).to(directExchangeBean()).with(
RENDER_HTML_QUEUE);
}
}
Messages are then consumed like this:
#Component
public class RenderItemMessageConsumer {
#RabbitListener(queues = RENDER_HTML_QUEUE)
public void reciveMessageFromRenderQueue(String message) {
//...
}
The exchangeName is shared across services. So generally I need a way to get the channel that probably is created for the queue and connection to see how many messages are inside. Ideally I want to access that information at the other service that produces the messages consumed in the rendering service.
Or am I doing something wrong? Do I have to explicitly create a channel and connect the queue to it? I'm not even sure what channels are created under the hood, as I mentioned I've set this up some years ago and didnt dig deeper after everything was running fine.
Can I maybe somehow use the amqpAdmin to get all channels?
Turns out I can answer my own question after just a little more trying around:
#Autowired
private ConnectionFactory connectionFactory;
public Boolean isBusy(String queue) throws IOException {
Connection connection = connectionFactory.createConnection();
Channel channel = connection.createChannel(false);
return channel.queueDeclarePassive(queue).getMessageCount() > 0;
}
Since all my services have a similar setup exposing the connectionFactory as a bean and they all connect to the shared rabbitMq server using the same exchange name, I can just use any service to do the above. I can put that snippet behind a rest resource and thus from my management UI can request all the information about the queues of which I know the names to decide if I want to post another batch of messages to it.
In My Scenario I need to create a lots of queues dynamically at run time that is why I don't want to use #Bean instead want to write a function that create queue and I will call it whenever necessary.
Here When i use #bean annotation it creates queue on rabbitmq server.
#Bean
public Queue productQueue(final String queueName) {
return new Queue(queueName);
}
But with the same code without #Bean
public Queue productQueue(final String queueName) {
return new Queue(queueName);
}
when call this function doesn't create queue on rabbitmq server
Queue queue = <Object>.productQueue("product-queue");
To create rabbitmq queue Dynamically I used following approach and this is best approach if you also want to create exchanges and bind to queue.
#Autowired
private ConnectionFactory connectionFactory;
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory);
}
Now you can define a class that creates queue, exchange and bind them
public class rabbitHelper {
#Autowired
private RabbitAdmin rabbitAdmin;
public Queue createQueue(String queueName) {
Queue q = new Queue(queueName);
rabbitAdmin.declareQueue(q);
return q;
}
public Exchange createExchange(String exchangeName) {
Exchange exchange = new DirectExchange(exchangeName);
rabbitAdmin.declareExchange(exchange);
return exchange;
}
public void createBinding(String queueName, String exchangeName, String routingKey) {
Binding binding = new Binding(queueName, Binding.DestinationType.QUEUE, queueName, routingKey, null);
rabbitAdmin().declareBinding(binding);
}
}
The Queue object must be a bean in the context and managed by Spring. To create queues dynamically at runtime, define the bean with scope prototype:
#Bean
#Scope("prototype")
public Queue productQueue(final String queueName) {
return new Queue(queueName);
}
and create queues at runtime using ObjectProvider:
#Autowired
private ObjectProvider<Queue> queueProvider;
Queue queue1 = queueProvider.getObject("queueName1");
Queue queue2 = queueProvider.getObject("queueName2");
I have a requirement to add multiple listeners as mentioned in the application.properties file. Like Below,
InTopics=Sample.QUT4,Sample.T05,Sample.T01,Sample.JT7
NOTE: This number can be lot more or less.
I am thinking of getting them in an array,
#Value("${InTopics}")
private String[] inTopics;
But i don't know how to create multiple listeners from the array.
Currently, for one Topic i am doing as below,
#Configuration
#EnableJms
public class JmsConfiguration {
#Value("${BrokerURL}")
private String brokerURL;
#Value("${BrokerUserName}")
private String brokerUserName;
#Value("${BrokerPassword}")
private String brokerPassword;
#Bean
TopicConnectionFactory connectionFactory() throws JMSException {
TopicConnectionFactory connectionFactory = new TopicConnectionFactory(brokerURL, brokerUserName, brokerPassword);
return connectionFactory;
}
#Bean
JmsListenerContainerFactory<?> jmsContainerFactory(TopicConnectionFactory connectionFactory) throws JMSException {
SimpleJmsListenerContainerFactory factory = new SimpleJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setPubSubDomain(Boolean.TRUE);
return factory;
}
}
And My Listener,
#JmsListener(destination = "${SingleTopicName}", containerFactory = "jmsContainerFactory")
public void receiveMessage(Message msg) {
//Do Some Stuff
}
Is there any way i can achieve this?
You can't do it with annotated #JmsListeners but you can register each listener programmatically by extending JmsListenerConfigurer as described in the reference documentation.
EDIT
Since you are injecting the property as an array...
#Value("${InTopics}")
private String[] inTopics;
Spring will split up the list an create an array based on the number of queues in the list.
You can then iterate over the array in JmsListenerConfigurer.configureJmsListeners() and create an endpoint for each element in the array - you don't need to know ahead of time how big the array is.
for (String inTopic : inTopics) {
...
}
Here is the customized code to define number of listeners dynamically.
JmsConfiguration jmsConfiguration;
private List<String> queueList;
#Bean
public DefaultJmsListenerContainerFactory mqJmsListenerContainerFactory() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(jmsConfiguration.jmsConnectionFactory());
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setSessionTransacted(true);
factory.setConcurrency("5");
return factory;
}
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
queueList.forEach(queue -> {
SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId(queue);
endpoint.setDestination(queue);
try {
endpoint.setMessageListener(message -> {
try {
logger.info("Receieved ID: {} Destination {}", message.getJMSMessageID(), message.getJMSDestination());
}
catch (JMSException e) {
logger.info("Exception while reading message - " + e);
}
});
registrar.setContainerFactory(mqJmsListenerContainerFactory());
}
catch (JMSException e) {
logger.info("Exception - " + e);
}
registrar.registerEndpoint(endpoint);
});
}
I did not know this was exist and I had to manually write all these code down. So, another way to do this is to implement BeanFactoryPostProcessor in your bean and manually add all the required components of a jms listener.
jndiTemplate
jndiQueueConnectionFactory (depends on jndiTemplate from step 1)
queueConnectionFactory (depends on jndiQueueConnectionFactory from step 2)
jndiDestinationResolver (uses jndiTemplate from stem 1)
messageListenerContiner (uses all above created items)
so, as you can see I do not only multiply the jms listener, I als dynamically generate the listener container multiple too. Ofc, this was my requirement. And may vary based on requirements.
One thing to keep in mind is that, there is no resource (like properties etc.. loaded) while you manipulate the BeanFactoryPostProcessor. You have to manually load the properties. I did via, afterPropertiesSet method coming from InitializingBean
I am using Spring to configure transactions in my application. I have two transaction managers defined for two RabbitMQ servers.
....
#Bean(name = "devtxManager")
public PlatformTransactionManager devtxManager() {
return new RabbitTransactionManager(devConnectionFactory());
}
#Bean(name = "qatxManager")
public PlatformTransactionManager qatxManager() {
return new RabbitTransactionManager(qaConnectionFactory());
}
#Bean
public ConnectionFactory devConnectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost(propertyLoader.loadProperty("dev.rabbit.host"));
factory.setPort(Integer.parseInt(propertyLoader.loadProperty("dev.rabbit.port")));
factory.setVirtualHost("product");
factory.setUsername(propertyLoader.loadProperty("dev.sender.rabbit.user"));
factory.setPassword(propertyLoader.loadProperty("dev.sender.rabbit.password"));
return factory;
}
#Bean
public ConnectionFactory qaConnectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost(propertyLoader.loadProperty("qa.rabbit.host"));
factory.setPort(Integer.parseInt(propertyLoader.loadProperty("qa.rabbit.port")));
factory.setVirtualHost("product");
factory.setUsername(propertyLoader.loadProperty("qa.sender.rabbit.user"));
factory.setPassword(propertyLoader.loadProperty("qa.sender.rabbit.password"));
return factory;
}
...
In my service class I need to pick the right transaction manager by the 'env' variable passed in. ( i.e if env=='qa' I need to choose 'qatxManager' else if 'env==dev' I need to choose 'devtxManager'.
....
#Transactional(value = "qatxManager")
public String requeue(String env, String sourceQueue, String destQueue) {
// read from queue
List<Message> messageList = sendReceiveImpl.receive(env, sourceQueue);
....
How can I get it done?
I think you need a Facade. Define an interface and create 2 classes implementing the same interface but with different #Transactional(value = "qatxManager")
Then define one Facade class which keeps 2 implementations (use #Qualifier to distinguish them) The Facade gets the env String and call method of proper bean
I need to implement with the help of spring-intagration libraries a message pipeline. At the beginning, as I see now, it needs to contain several elements:
a. Messaging gateway,
#MessagingGateway(name = "entryGateway", defaultRequestChannel = "requestChannel")
public interface MessageGateway {
public boolean processMessage(Message<?> message);
}
which is called when I want to start the pipeline:
messageGateway.processMessage(message);
b. Channel for transmitting the messages:
#Bean
public MessageChannel requestChannel() {
return new DirectChannel();
}
c.Router which decides then where flow the messages
#MessageEndpoint
#Component
public class MessageTypeRouter {
Logger log = Logger.getLogger(MessageTypeRouter.class);
#Router(inputChannel="requestChannel")
public String processMessageByPayload(Message<?> message){...}
There can be many incoming messages in a small period of time, so I wanted to realize a channel (b) as QueueChannel:
#Bean
public MessageChannel requestChannel() {
return new QueueChannel();
}
On the other hand I would like the router to start as soon as a message comes through gateway and the other messages to wait in the queue. But in this case I received an error, which said that I should have used a poller.
May be you could give me a piece of advice, how I can realize my scheme. Thank you in advance.
As we know with XML config we must declare a <poller> component and mark it with default="true". That allows any PollingConsumer endpoint to pick up the default Poller from the context.
With Java config we must declare a #Bean for similar purpose:
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
pollerMetadata.setTrigger(new PeriodicTrigger(10));
return pollerMetadata;
}
Where the PollerMetadata.DEFAULT_POLLER is specific constant to define the default poller. Although the same name is used from XML config in case of default="true".
From other side #Router annotation has poller attribute to specify something similar like we do with nested <poller> in XML.