I am learning about ActiveMQ, and so far I have made a simple Spring Boot producer+consumer application (call it App1 for the purposes of this question) that communicates with a local instance of ActiveMQ and everything works as expected.
Now I am trying to run a different Spring Boot application (on the same computer but after ensuring App1 is not running) that has only a consumer (no producer), but when I start up this application, the messages in the queue (that I put using a modified App1 in which I removed the consumer portion of the application) do not get picked up. In App1, as soon as the message was published, the consumer printed out the system.out print statement, but not so in this consumer-only application. Below is my listener component class:
package com.demo.listener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.stereotype.Component;
#Component
public class Consumer {
#JmsListener(destination = "testqueue")
public void consume(String message) {
System.out.println("Picked up message: " + message);
}
}
What changes would I need to make in order achieve the desired behavior?
App1 application.properties file:
spring.activemq.in-memory=false
spring.activemq.pool.enabled=false
server.port=9000
activemq.broker-url=tcp://localhost:61616
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration, org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
security.basic.enabled=false
management.security.enabled=false
App1 JmsConfig class
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.command.ActiveMQQueue;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.core.JmsTemplate;
#Configuration
public class JmsConfig {
#Value("${activemq.broker-url}")
private String brokerUrl;
#Bean
public Queue queue() {
return new ActiveMQQueue("testqueue");
}
#Bean
public ActiveMQConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();
factory.setBrokerURL(brokerUrl);
return factory;
}
#Bean
public JmsTemplate jmsTemplate() {
return new JmsTemplate(activeMQConnectionFactory());
}
}
App1 Producer class
import javax.jms.Queue;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
#RestController
#RequestMapping("/rest/publish")
public class ProducerResource {
#Autowired
JmsTemplate jmsTemplate;
#Autowired
Queue queue;
#GetMapping("/{message}")
public String publishMessage(#PathVariable("message") final String message) {
jmsTemplate.convertAndSend(queue, message);
return "Published successfully";
}
}
App1 consumer class is the same class I used in the consumer only application (listed above).
For your consumer app, you do need to add Pool connection factory & JMS message listener factory for your consumer JMStemplate to start receiving messages.
#Configuration
#EnableJms
public class ConsumerConfig {
#Value("${activemqbrokerurl}")
private String brokerUrl;
#Bean
public ActiveMQConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
return activeMQConnectionFactory;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(activeMQConnectionFactory());
factory.setConcurrency("{#setDesiredConcurrency}");
return factory;
}
}
Spring's MessagListenerContainer should be used for message consumption. This provides all the power of MDBs - efficient JMS consumption and pooling of the message listeners - but without requiring a full EJB container.You can use the activemq-pool org.apache.activemq.pool.PooledConnectionFactory for efficient pooling of the connections and sessions for your collection of consumers, or you can use the Spring JMS org.springframework.jms.connection.CachingConnectionFactory to achieve the same effect.
You can read more about it here
Related
Im setting up the connection factory for my rabbitmq application. Ive got the beans created in the config, however for some reason the ConnectionFactory that ive passed into the RabbitTemplate doesnt seem to be creating the bean automatically as it should.
the error its saying is Could not autowire. No beans of 'ConnectionFactory' type found.
However this bean should be created automatically. It works if i create a bean of ConnectionFactory manually, but struggling to understand why it doesnt work without it.
package com.example.redistorabbit;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
#SpringBootApplication
public class RedisToRabbitApplication {
public static void main(String[] args) {
SpringApplication.run(RedisToRabbitApplication.class, args);
}
public static final String TOPIC_EXCHANGE_NAME = "covid-exchange";
public static final String QUEUE_NAME = "cases-entity";
#Bean
Queue queue(){
return new Queue(QUEUE_NAME,false);
}
#Bean
TopicExchange exchange(){
return new TopicExchange(TOPIC_EXCHANGE_NAME);
}
#Bean
Binding binding(Queue queue, TopicExchange topicExchange){
return BindingBuilder.bind(queue).to(topicExchange).with("#");
}
#Bean
public RabbitTemplate publishingRabbitTemplate(final ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jsonConverter());
rabbitTemplate.setChannelTransacted(true);
return rabbitTemplate;
}
#Bean
public Jackson2JsonMessageConverter jsonConverter() {
return new Jackson2JsonMessageConverter();
}
}
another issue i seem to have is even though i have the Jackson2JsonMessageConverter when the lisetener received and prints the message it is showing as bytes instead of a json message:
#Component
#RequiredArgsConstructor
public class Receiver {
private final CasesRepository casesRepository;
private static final Logger log = LoggerFactory.getLogger(Receiver.class);
#RabbitListener(queues = RedisToRabbitApplication.QUEUE_NAME)
public void consumeMessage(final Message message) {
log.info("received: " + message.toString());
}
}
received: (Body:'[B#1b68f46c(byte[91])'
Have you tried Injecting the Spring Object Mapper into the Jackson Message converter?
#Bean
public Jackson2JsonMessageConverter jsonConverter(ObjectMapper objectMapper) {
return new Jackson2JsonMessageConverter(objectMapper);
}
It seems the issue is with the version of spring-boot-autoconfiguration dependency, i could see the bean in the version for 2.7.3, in the JacksonAutoConfiguration class, however Intellij was still saying no bean was found. I added the following dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
<version>1.5.15.RELEASE</version>
</dependency>
and both ConnectionFactory and ObjectMapper beans have been created automatically
I am using Spring Kafka first time and I am not able to use Acknowledgement.acknowledge() method for manual commit in my consumer code as mentioned here https://docs.spring.io/spring-kafka/reference/html/_reference.html#committing-offsets. Mine is spring-boot application. If I am not using manual commit process than my code is working fine. But when I use
Acknowledgement.acknowledge() for manual commit it shows error related to bean. Also If I am not using manual commit properly please suggest me the right way to do it.
Error message:
***************************
APPLICATION FAILED TO START
***************************
Description:
Field ack in Receiver required a bean of type 'org.springframework.kafka.support.Acknowledgment' that could not be found.
Action:
Consider defining a bean of type 'org.springframework.kafka.support.Acknowledgment' in your configuration.
I googled this error I found that I need to add #Component but that is already there in my consumer code.
My consumer code looks like this: Receiver.java
import java.util.concurrent.CountDownLatch;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.stereotype.Component;
#Component
public class Receiver {
#Autowired
public Acknowledgment ack;
private CountDownLatch latch = new CountDownLatch(1);
#KafkaListener(topics = "${kafka.topic.TestTopic}")
public void receive(ConsumerRecord<?, ?> consumerRecord){
System.out.println(consumerRecord.value());
latch.countDown();
ack.acknowledge();
}
}
My producer code looks like this: Sender.java
import java.util.Map;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;
#Component
public class Sender {
#Autowired
private KafkaTemplate<String, Map<String, Object>> kafkaTemplate;
public void send(Map<String, Object> map){
kafkaTemplate.send("TestTopic", map);
}
}
EDIT 1:
My new consumer code looks like this: Receiver.java
import java.util.concurrent.CountDownLatch;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.stereotype.Component;
#Component
public class Receiver {
private CountDownLatch latch = new CountDownLatch(1);
#KafkaListener(topics = "${kafka.topic.TestTopic}", containerFactory = "kafkaManualAckListenerContainerFactory")
public void receive(ConsumerRecord<?, ?> consumerRecord, Acknowledgment ack){
System.out.println(consumerRecord.value());
latch.countDown();
ack.acknowledge();
}
}
I changed my configuration class also:
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
#Configuration
#EnableKafka
public class ReceiverConfig {
#Value("${kafka.bootstrap-servers}")
private String bootstrapServers;
#Value("${spring.kafka.consumer.group-id}")
private String consumerGroupId;
#Bean
public Map<String, Object> consumerConfigs() throws SendGridException {
Map<String, Object> props = new HashMap<>();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.GROUP_ID_CONFIG, consumerGroupId);
return props;
}
#Bean
public ConsumerFactory<String, String> consumerFactory(){
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
/*#Bean
public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(){
ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
return factory;
}*/
#Bean
public ConcurrentKafkaListenerContainerFactory<String, String> kafkaManualAckListenerContainerFactory(){
ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
return factory;
}
#Bean
public Receiver receiver() {
return new Receiver();
}
}
After adding containerFactory = "kafkaManualAckListenerContainerFactory" to my receive() method I am getting the below error.
***************************
APPLICATION FAILED TO START
***************************
Description:
Parameter 1 of method kafkaListenerContainerFactory in org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration required a bean of type 'org.springframework.kafka.core.ConsumerFactory' that could not be found.
- Bean method 'kafkaConsumerFactory' in 'KafkaAutoConfiguration' not loaded because #ConditionalOnMissingBean (types: org.springframework.kafka.core.ConsumerFactory; SearchStrategy: all) found bean 'consumerFactory'
Action:
Consider revisiting the conditions above or defining a bean of type 'org.springframework.kafka.core.ConsumerFactory' in your configuration.
For those still looking for a solution to these errors concerning manual acknowledgment, you don't need to specify containerFactory = "kafkaManualAckListenerContainerFactory", instead you can just add:
factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE);
to your receiver config just before you return the factory object.
Then you also need:
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
in consumer config props.
So in the end your listener method can simply look like:
#KafkaListener(topics = "${spring.kafka.topic}")
private void listen(#Payload String payload, Acknowledgment acknowledgment) {
//Whatever code you want to do with the payload
acknowledgement.acknowledge(); //or even pass the acknowledgment to a different method and acknowledge even later
}
You really should follow documentation:
When using manual AckMode, the listener can also be provided with the Acknowledgment; this example also shows how to use a different container factory.
#KafkaListener(id = "baz", topics = "myTopic",
containerFactory = "kafkaManualAckListenerContainerFactory")
public void listen(String data, Acknowledgment ack) {
...
ack.acknowledge();
}
There is really nowhere noted that Acknowledgment is a bean. So, change your receive() #KafkaListener method signature appropriately and remove that #Autowired for suspicious Acknowledgment bean - it just doesn't exists because this object is a part (header) of each received message.
Those who are using spring boot application, simple add below to your application.yml (or environment specific file).
spring:
kafka:
listener:
ack-mode: manual
Above change will make Acknowledgment ack argument available inside
receive(ConsumerRecord<?, ?> consumerRecord, Acknowledgment ack) method automatically.
I'm building JPA configuration with multiple persistence units using different in-memory datasources, but the configuration fails resolving the qualified datasource for entity manager factory bean with the following error:
***************************
APPLICATION FAILED TO START
***************************
Description:
Parameter 0 of method emfb in datasources.Application$PersistenceConfiguration required a single bean, but 2 were found:
- ds1: defined by method 'ds1' in class path resource [datasources/Application$PersistenceConfiguration.class]
- ds2: defined by method 'ds2' in class path resource [datasources/Application$PersistenceConfiguration.class]
Action:
Consider marking one of the beans as #Primary, updating the consumer to accept multiple beans, or using #Qualifier to identify the bean that should be consumed
Here is the sample application
package datasources;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.PersistenceContextType;
import javax.sql.DataSource;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import org.apache.log4j.Logger;
import org.glassfish.jersey.server.ResourceConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.transaction.jta.JtaAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.stereotype.Component;
#Configuration
#EnableAutoConfiguration(exclude = {
// HibernateJpaAutoConfiguration.class,
// DataSourceAutoConfiguration.class
JtaAutoConfiguration.class
})
#ComponentScan
public class Application {
public static void main(String[] args) {
new SpringApplicationBuilder(Application.class)
.build()
.run(args);
}
#Component
#Path("/ds")
public static class DsApi {
private final static Logger logger = Logger.getLogger(DsApi.class);
#Autowired(required = false)
#Qualifier("ds1")
private DataSource ds;
#GET
public String ds() {
logger.info("ds");
return ds.toString();
}
}
#Component
#Path("/em")
public static class EmApi {
private final static Logger logger = Logger.getLogger(EmApi.class);
#PersistenceContext(unitName = "ds2", type = PersistenceContextType.TRANSACTION)
private EntityManager em;
#GET
public String em() {
logger.info("em");
return em.toString();
}
}
#Configuration
#ApplicationPath("/jersey")
public static class JerseyConfig extends ResourceConfig {
public JerseyConfig() {
register(DsApi.class);
register(EmApi.class);
}
}
#Configuration
public static class PersistenceConfiguration {
#Bean
#Qualifier("ds1")
public DataSource ds1() {
return new EmbeddedDatabaseBuilder().build();
}
#Bean
#Qualifier("ds2")
public DataSource ds2() {
return new EmbeddedDatabaseBuilder().build();
}
#Bean
#Primary
#Autowired
public LocalContainerEntityManagerFactoryBean emfb(#Qualifier("ds1") DataSource ds, EntityManagerFactoryBuilder emfb) {
return emfb.dataSource(ds)
.packages(Application.class)
.persistenceUnit("ds1")
.build();
}
#Bean
#Autowired
public LocalContainerEntityManagerFactoryBean emfb2(#Qualifier("ds2") DataSource ds, EntityManagerFactoryBuilder emfb) {
return emfb.dataSource(ds)
.packages(Application.class)
.persistenceUnit("ds2")
.build();
}
}
}
The error is indicating that at some point in the application, a bean is being injected by the type DataSource and not being qualified by name at that point.
It does not matter that you have added #Qualifier in one location. The injection is failing in some other location that has not been qualified. It's not your fault though because that location is in Spring Boot's DataSourceAutoConfiguration which you should be able to see in your stack trace, below the piece that you have posted.
I would recommend excluding DataSourceAutoConfiguration i.e. #SpringBootApplication(exclude = DataSourceAutoConfiguration.class). Otherwise, this configuration is only being applied to the bean you have made #Primary. Unless you know exactly what that is, it is likely to result in subtle and unexpected differences in behaviour between your DataSources.
Declare one of your DataSource as #Primary.
Also you have 2 beans of same type - LocalContainerEntityManagerFactoryBean, declare one of them #Primary as well, as follows:
#Configuration
public static class PersistenceConfiguration {
#Bean
#Primary
public DataSource ds1() {
return new EmbeddedDatabaseBuilder().build();
}
#Bean
public DataSource ds2() {
return new EmbeddedDatabaseBuilder().build();
}
#Bean
#Primary
#Autowired
public LocalContainerEntityManagerFactoryBean emfb(#Qualifier("ds1") DataSource ds, EntityManagerFactoryBuilder emfb) {
return emfb.dataSource(ds)
.packages(DemoApplication.class)
.persistenceUnit("ds1")
.build();
}
#Bean
#Autowired
public LocalContainerEntityManagerFactoryBean emfb2(#Qualifier("ds2") DataSource ds, EntityManagerFactoryBuilder emfb) {
return emfb.dataSource(ds)
.packages(DemoApplication.class)
.persistenceUnit("ds2")
.build();
}
}
Try declaring the datasource beans outside the static class . I.e directly in Application.java
To me, this appears to be just about the simplest possible spring integration example. I'm trying to learn from the si4demo. But when I run it, I get this exception:
Exception in thread "main"
org.springframework.messaging.MessageDeliveryException: Dispatcher has
no subscribers for channel 'application.inbox'.; nested exception is
org.springframework.integration.MessageDispatchingException:
Dispatcher has no subscribers
Where am I going wrong? Doesn't the defined flow create a subscription to the inbox channel?
import org.springframework.boot.SpringApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.IntegrationComponentScan;
import org.springframework.integration.annotation.MessagingGateway;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.messaging.MessageChannel;
#Configuration
#ComponentScan
#IntegrationComponentScan
public class App {
public static void main(String[] args) {
try (ConfigurableApplicationContext ctx = SpringApplication.run(App.class, args)) {
final Gateway gateway = ctx.getBean(Gateway.class);
final String rs = gateway.send("hullo");
System.out.println(rs);
}
}
private static final String INBOX = "inbox";
#MessagingGateway(defaultRequestChannel = INBOX)
public interface Gateway {
String send(String msg);
}
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(INBOX)
.transform(p -> "world")
.get();
}
#Bean(name = INBOX)
public MessageChannel inbox() {
return new DirectChannel();
}
}
Looks like you have missed the main player - #EnableIntegraion:
Starting with version 4.0, the #EnableIntegration annotation has been introduced, to allow the registration of Spring Integration infrastructure beans (see JavaDocs). This annotation is required when only Java & Annotation configuration is used, e.g. with Spring Boot and/or Spring Integration Messaging Annotation support and Spring Integration Java DSL with no XML integration configuration.
http://docs.spring.io/spring-integration/docs/4.3.0.BUILD-SNAPSHOT/reference/html/overview.html#configuration-enable-integration
I am implementing WebSocket in spring 4.0.5. The client(browser) keep a connection to the server. When my service layer update some record at the back end, I want to inform the client which subscribe to the particular topic.
I want my Controller to listen to my ApplicationEvent and publish it to the WebSocket client. Is this possible?
I have another listener implemented in my service layer and it can capture the event.
Is it true that the controller cannot listen to the ApplicationEvent?
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationListener;
import org.springframework.messaging.simp.SimpMessagingTemplate;
import org.springframework.stereotype.Controller;
import com.geneoz.service.event.MessageEvent;
#Controller
public class ApplicationEventObservorController implements ApplicationListener<MessageEvent> {
private static Logger logger = Logger.getLogger(ApplicationEventObservorController.class);
private SimpMessagingTemplate template;
#Autowired
public ApplicationEventObservorController(SimpMessagingTemplate template) {
logger.debug("Initialising ApplicationEventObservorController");
this.template = template;
}
#Override
public void onApplicationEvent(MessageEvent event) {
logger.debug("Captured event. " + event);
this.template.convertAndSend("/topic/update", event.toString());
}
}