We Keep Coding

How to disable KAFKA consumer being autostarted without having to make any code changes including setting the autoStartup = "{xyz}" in Spring Boot?

Below is my KAFKA consumer
#Service
public class Consumer {
private static final Logger LOGGER = Logger.getLogger(Consumer.class.getName());
public static Queue<ProductKafka> consumeQueue = new LinkedList<>();
#KafkaListener(topics = "#{'${spring.kafka.topics}'.split('\\\\ ')}", groupId = "#{'${spring.kafka.groupId}'}")
public void consume(ProductKafka productKafka) throws IOException {
consumeQueue.add(productKafka);
LOGGER.info(String.format("#### -> Logger Consumed message -> %s", productKafka.toString()));
System.out.printf("#### -> Consumed message -> %s", productKafka.toString());
}
}
and below is my "application.properties" file
spring.kafka.topics=Product
spring.kafka.groupId=Product-Group
My KAFKA consumer is getting started automatically.
However I want to disable KAFKA consumer being autostarted without having to make any changes to the existing code including setting autoStartup = "{xyz}" in the consumer class due to the requirement.
I am looking an existing properties which would disable KAFKA consumer being autostarted, something like this
spring.kafka.consumer.enable=false
Note: I have multiple KAFKA consumers and the above property should disable all the consumers in the project.
do we have any existing properties which would disable KAFKA consumer being autostarted without having to make any changes to the existing code?

There is no standard out-of-the-box property; you have to provide your own.
autoStartup="${should.start:true}"
will start the container if property should.start is not present.
EDIT
Just add something like this in your application.
#Component
class Customizer {
Customizer(AbstractKafkaListenerContainerFactory<?, ?, ?> factory,
#Value("${start.containers:true}") boolean start) {
factory.setAutoStartup(start);
}
}
start:
containers: false

Spring-Kafka usage of ConcurrentKafkaListenerContainerFactory for more than One #Kafkalistener

I am working on implementing consumption of messages from Kafka Topics using Spring-Kafka framework. I am trying to understand some usage of the ConcurrentKafkaListenerContainerFactory that i am creating for my Kafka Listener. The #KafkaListener works fine and as expected, however, in my scenario, i have more than one independent Listeners, listening to more than one Topic respectively. I would like to know if i can reuse the ConcurrentKafkaListenerContainerFactory among all my Listeners, or do i have to create one containerFactory per #KafkaListener. Is there a way of having a generic containerFactory that can be shared among all #Kafkalisteners
Thanks you

Yes; that's the whole point - it's a factory for listener containers; you typically only need the one factory that boot auto configures.
If you need different properties (e.g. deserializers) for a listener, recent versions (since spring-kafka 2.2.4) allow you to override consumer properties on the annotation.
To override other properties, e.g. container properties, for individual listeners, add a listener container customizer to the factory.
#Component
class ContainerFactoryCustomizer {
ContainerFactoryCustomizer(AbstractKafkaListenerContainerFactory<?, ?, ?> factory) {
factory.setContainerCustomizer(
container -> {
String groupId = container.getContainerProperties().getGroupId();
if (groupId.equals("foo")) {
container.getContainerProperties().set...
}
else {
container.getContainerProperties().set...
}
});
}
As you can see, you can tell which container we are creating when it is called by accessing the groupId() container property.
You might want to use 2 factories if your listeners have vastly different configuration, but then you lose boot's auto configuration features (at least for the factory).

Use KafkaListener in class not managed by Spring

In my project, I have a lot of spring managed components that does about the same thing. I want to create a common Util class that does all the common operations for all my components. Since this Util class needs to access environment variables and beans it's instantiated like this:
// Util class:
public class FooUtil {
public FooUtil(Environment env) {
env.getProperty("FOO_TOPIC", "foo")
}
}
// Example configuration for one of my components:
#Configuration
public class ComponentConfig {
#Bean
FooUtil fooUtil(Environment env) {
return new FooUtil(env);
}
}
This allows FooUtil to access all environment variables and beans without itself being a component.
Now, this Util class also need to listen to kafka topics. Each component currently has a listener set up like this:
#KafkaListener(topics = "${FOO"_TOPIC:foo2}", containerFactory = "kafkaListenerContainerFactory")
private void fooListener(ConsumerRecord<String, Foo> rec) {
// Stuff...
}
I want to move this kafka listener into FooUtil. How can I do this? To be clear, I want FooUtil to start listening as soon as it's instantiated and initialized by a component.

Since the FooUtil is not managed by Spring you're unable to use the #KafkaListener annotation. If FooUtil was a bean managed by Spring it would be picked up by Spring and the listener annotation will cause Spring to connect the listener. All of this is done by Spring in KafkaListenerAnnotationBeanPostProcessor I believe.
Does the FooUtil have to be an unmanaged bean? I might be missing some details but from the question I can't see why it shouldn't be possible. If you need different instances for every bean using it you can use #Scope("prototype") on the FooUtil.

Turns out, you can make a kafka listener without using the #KafkaListener annotation (thanks Gary Russell). Just follow the instructions here (douevencode.com) for instructions of how to do it.

Configure Amazon SQS queue name in Spring Boot

I'm using AmazonSQS & Spring Boot (spring-cloud-aws-messaging). I've configured a message listener to receive messages from the queue with the annotation #SqsListener.
#SqsListener(value = "indexerQueue", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void queueListener(String rawMessage) {
...
}
This is a very simple approach but I didn't find the way to make the queue name load from a configuration file because I have different environments. Any ideas on this regard?

What version of spring-cloud-aws-messaging are you using? Version 1.1 should allow you to use a placeholder as a queue name, e.g.
#SqsListener(value = "${sqs.queue.indexer}", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void queueListener(String rawMessage) {
...
}
Then, in your application-env.properties files you can put different values. For instance in application-dev.properties:
sqs.queue.indexer=devIndexerQueue
and in application-production.properties
sqs.queue.indexer=indexerQueue

Extract headers from message using spring-rabbit without declaring queues in annotation

Now I can do like this:
#RabbitListener(queues = {ENTITY_KEY + "-snapshots", ENTITY_KEY + "-updates"})
public void handleMessage(ProviderOddsOffer offer, #Header("update_type") Long updateType) {
...
}
Can I do it without declaring queues in annotation itself?

It's not clear what you mean; the listener has to be configured to consume from some queue, or queues.
If you that mean you wish to externalize the queue name(s) rather than hard-coding in java, you can use a property placeholder ${...} or a SpEL expression #{...} for the queue name(s); they will be resolved during bean initialization.