Configure Thread Names in Apache Camel - java

I'm kind of new to Apache Camel and am testing it to use it on my application (replacing already implemented Spring Integration).
I've been searching all over the web, Apache Camel's documentation site and stackoverflow for the last days but can't seem to find an answer on how to configure thread's names in Apache Camel via Java DSL.
I only got to see this and this other question but it only says how to do it via Spring DSL. Same on Apache Camel documentation page.
To give some context:
Right now I'm building two operation flows (first and second) and each one has it's one route.
Both routes read from different ActiveMQ queues, process the messages in a different way and the send the response back to different queues.
I already managed to configure different concurrentConsumers and maxConcurrentConsumers for each route (via properties file).
I would like to assign thread names (or at least patterns, since I have many consumers on each route); in a way that I could have something like "FirstOp-X" and "SecondOp-X" (where X is the thread number).
Here's the code snippet:
public class SampleCamelRouter extends RouteBuilder {
/**
* The first operation name
*/
public static final String FIRST_NAME = "first";
/**
* The second operation name
*/
public static final String SECOND_NAME = "second";
/**
* The ActiveMQ outbound queue format
*/
public static final String OUTBOUND_QUEUE_FORMAT = "activemq:queue:aq.%1$s.response";
/**
* The ActiveMQ inbound queue format
*/
public static final String INBOUND_QUEUE_FORMAT = "activemq:queue:aq.%1$s.request"
+ "?concurrentConsumers={{queue.%1$s.concurrentConsumers}}"
+ "&maxConcurrentConsumers={{queue.%1$s.maxConcurrentConsumers}}";
/*
* (non-Javadoc)
* #see org.apache.camel.builder.RouteBuilder#configure()
*/
#Override
public void configure() throws Exception {
from(String.format(INBOUND_QUEUE_FORMAT, FIRST_NAME))
.unmarshal().json(JsonLibrary.Jackson, FirstRequestMessage.class)
.bean(TestBean.class, "doFirst")
.marshal().json(JsonLibrary.Jackson)
.to(String.format(OUTBOUND_QUEUE_FORMAT, FIRST_NAME));
from(String.format(INBOUND_QUEUE_FORMAT, SECOND_NAME))
.unmarshal().json(JsonLibrary.Jackson, SecondRequestMessage.class)
.bean(TestBean.class, "doSecond")
.marshal().json(JsonLibrary.Jackson)
.to(String.format(OUTBOUND_QUEUE_FORMAT, SECOND_NAME));
}
I used to do it in Spring Integration with something like this (per flow):
#Bean
public static IntegrationFlow setUpFirstFlow() {
final DefaultMessageListenerContainer messageListenerContainer = new DefaultMessageListenerContainer();
messageListenerContainer.setBeanName("FirstOp");
messageListenerContainer.setDestinationName("aq.first.request");
messageListenerContainer.setConcurrentConsumers(concurrentConsumers);
messageListenerContainer.setMaxConcurrentConsumers(maxConcurrentConsumers);
return IntegrationFlows.from(Jms.messageDrivenChannelAdapter(messageListenerContainer))
.transform(new JsonToObjectTransformer(FirstRequestMessage.class))
.handle(TestBean.class, "doFirst")
.transform(new ObjectToJsonTransformer())
.handle(Jms.outboundAdapter(.......)).get();
}
So basically: I created different message listener containers, and that way, I have different thread names per flow.
At any time if a thread is stoped, blocked, inside a thread-dump (or even as simple as printing a log) I could easily see, wich flow that thread belongs to.
I see Apache Camel has some kind of workaround (but not per route, but per camelContext) but only implemented using Spring DSL not Java.
I wouldn't mind to change my configuration to XML files if this per-route-configuration is possible only with Spring.
Please help me, it's kind of a tie-breaker for me right now. For the application I'm building, it's very important to be able to identify each thread isolated; and don't really like the default thread names (Camel (camel-1) thread #27 - JmsConsumer[aa.first.request]) :'-(.

You can set an ExecutorServiceManager per Camel context via org.apache.camel.impl.DefaultCamelContext#setExecutorServiceManager - if you use a DefaultExecutorServiceManager you can set a threadNamePattern.
Alternatively, you can use the Threads DSL and assign a thread pool to a route by using org.apache.camel.model.ProcessorDefinition#threads(int, int, java.lang.String), e.g.
from(String.format(INBOUND_QUEUE_FORMAT, FIRST_NAME))
.threads(1, 2, "first")
...
from(String.format(INBOUND_QUEUE_FORMAT, SECOND_NAME))
.threads(1, 2, "second")
...
Note that using the threads() method would effectively mean that you would be using an asynchronous processing model in Camel.

Related

Opentracing context not propagated on Quarkus when using reactive messaging

I've two microservices interacting with each other via Kafka, that is the one publishes messages while the other consumes them. Both the publisher and the consumer run on Quarkus (1.12.0.Final) and use reactive messaging and Mutiny.
Producer:
package myproducer;
import myavro.MyAvro;
import io.smallrye.mutiny.Uni;
import org.eclipse.microprofile.reactive.messaging.Channel;
import org.eclipse.microprofile.reactive.messaging.Emitter;
import org.eclipse.microprofile.reactive.messaging.Message;
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import java.util.concurrent.CompletableFuture;
#ApplicationScoped
public class Publisher {
#Channel("mytopic")
#Inject
public Emitter<MyAvro> myTopic;
#Override
public Uni<Void> publish(MyModel model) {
MyAvro avro = MyModelMapper.INSTANCE.modelToAvro(model);
return Uni.createFrom().emitter(e -> myTopic.send(Message.of(avro)
.addMetadata(toOutgoingKafkaRecordMetadata(avro))
.withAck(() -> {
e.complete(null);
return CompletableFuture.completedFuture(null);
})));
}
}
Consumer:
package myconsumer;
import myavro.MyAvro;
import io.smallrye.mutiny.Uni;
import io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord;
import org.eclipse.microprofile.reactive.messaging.Incoming;
import javax.enterprise.context.ApplicationScoped;
#ApplicationScoped
public class Consumer {
#Incoming("mytopic")
public Uni<Void> consume(IncomingKafkaRecord<String, MyAvro> message) {
MyModel model = MyModelMapper.INSTANCE.avroToModel(message.getPayload());
return ...;
}
}
Dependencies:
include among others the artefacts
quarkus-smallrye-reactive-messaging-kafka
quarkus-resteasy-mutiny
quarkus-smallrye-opentracing
quarkus-mutiny
opentracing-kafka-client
Quarkus configuration (application.properties):
includes among others
quarkus.jaeger.service-name=myservice
quarkus.jaeger.sampler-type=const
quarkus.jaeger.sampler-param=1
quarkus.log.console.format=%d{yyyy-MM-dd HH:mm:ss} %-5p traceId=%X{traceId}, spanId=%X{spanId}, sampled=%X{sampled} [%c{2.}] (%t) %s%e%n
mp.messaging.incoming.mytopic.topic=abc
mp.messaging.incoming.mytopic.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.incoming.mytopic.value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
...
mp.messaging.incoming.mytopic.interceptor.classes=io.opentracing.contrib.kafka.TracingConsumerInterceptor
With this setup no traceId or spanId is logged at all (even though they should according to Quarkus' "Using OpenTracing" guide). Only after adding #org.eclipse.microprofile.opentracing.Traced a traceId and a spanId is set, but both are completely unrelated to each other on the producer and the consumer.
I checked my opentracing configuration against the beforementioned Quarkus' guide "Using OpenTracing" but found no hints for a misconfiguration on my side.
After reading discussions about issues in some Quarkus extensions relying on ThreadLocals when using with Mutiny I added the artefact quarkus-smallrye-context-propagation to my dependencies, but to no avail.
I suspect that the issue might be related to https://github.com/quarkusio/quarkus/issues/15182, though there it's about reactive routes instead of reactive messaging.
Any ideas?
This issue is not easy to solve, first I will try to explain what happens.
OpenTracing has the concepts of transactions and spans. A span is a block of execution (a method, a database call, a send to a Kafka topic), whereas a transaction is a distributed process that span multiple components (a group of spans).
The issue here is that, each time a span is created, it didn't find any OpenTracing transaction so it creates a new one. This is why none of your spans are correlated to each others.
In OpenTracing, when you create a span, you'll create it based on a span context. Each OpenTracing integration will creates a span context based on the extension technology (I didn't find a better term), for example, HTTP span context is based on HTTP headers and Kafka span context is based on Kafka Headers.
So, to correlate two spans, you need to have the span context created with some context from the underlying technology providing the right OpenTracing ID.
For example, to correlate two Kafka spans, you need to have a uber-trace-id header (this is the default name of the OpenTracing id in Jaeger) with the trace identifier (see tracespan-identity for the format of this header).
Knowing this, there is multiple things to do.
First, you need to add an uber-trace-id Kafka header inside your outgoing message in your #Traced method to correlate the span from the method with the span created inside the Kafka producer interceptor.
Tracer tracer = GlobalTracer.get(); // you can also inject it
JaegerSpanContext spanCtx = ((JaegerSpan)tracer.activeSpan()).context();
// uber-trace-id format: {trace-id}:{span-id}:{parent-span-id}:{flags}
//see https://www.jaegertracing.io/docs/1.21/client-libraries/#tracespan-identity
var uberTraceId = spanCtx.getTraceId() + ":" +
Long.toHexString(spanCtx.getSpanId()) + ":" +
Long.toHexString(spanCtx.getParentId()) + ":" +
Integer.toHexString(spanCtx.getFlags());
headers.add("uber-trace-id", openTracingId.getBytes());
Then, you need to correlate your #Traced method with the span from the incoming message, if any. For this, the easiest way is to add a CDI interceptor that will try to create a span context for all methods annotated with #Traced based on the method parameters (it will search for a Message parameter). For this to work, this interceptor needs to be executed before the OpenTracing interceptor, and sets the span context in the interceptor context.
This is our interceptor implementation, feel free to use it or adapt it for your needs.
public class KafkaRecordOpenTracingInterceptor {
#AroundInvoke
public Object propagateSpanCtx(InvocationContext ctx) throws Exception {
for (int i = 0 ; i < ctx.getParameters().length ; i++) {
Object parameter = ctx.getParameters()[i];
if (parameter instanceof Message) {
Message message = (Message) parameter;
Headers headers = message.getMetadata(IncomingKafkaRecordMetadata.class)
.map(IncomingKafkaRecordMetadata::getHeaders)
.get();
SpanContext spanContext = getSpanContext(headers);
ctx.getContextData().put(OpenTracingInterceptor.SPAN_CONTEXT, spanContext);
}
}
return ctx.proceed();
}
private SpanContext getSpanContext(Headers headers) {
return TracingKafkaUtils.extractSpanContext(headers, GlobalTracer.get());
}
}
This code uses both the Quarkus OpenTracing extension and the Kafka OpenTracing contrib library.
With both the correlation of outgoing message thanks to the addition of the OpenTracing Kafka Header created from the current span context, and the creation of a context from incoming message's header, the correlation should happens in any case.

Publish / Subscribe MQTT using SmallRye reactive messaging dynamically

We try to publish and subscribe to MQTT protocol using smallrye reactive messaging. We managed to actually publish a message into a specific topic/channel through the following simple code
import io.smallrye.mutiny.Multi;
import org.eclipse.microprofile.reactive.messaging.Outgoing;
import javax.enterprise.context.ApplicationScoped;
import java.time.Duration;
#ApplicationScoped
public class Publish {
#Outgoing("pao")
public Multi<String> generate() {
return Multi.createFrom().ticks().every(Duration.ofSeconds(1))
.map(x -> "A Message in here");
}
}
What we want to do is to call whenever we want the generate() method somehow with a dynamic topic, where the user will define it. That one was our problem but then we found these classes from that repo in github. Package name io.smallrye.reactive.messaging.mqtt
For example we found that there is a class that says it makes a publish call to a MQTT broker(Mosquitto server up).
Here in that statement SendingMqttMessage<String> message = new SendingMqttMessage<String>("myTopic","A message in here",0,false);
We get the a red underline under the SendingMqttMessage<String> saying 'SendingMqttMessage(java.lang.String, java.lang.String, io.netty.handler.codec.mqtt.MqttQoS, boolean)' is not public in 'io.smallrye.reactive.messaging.mqtt.SendingMqttMessage'. Cannot be accessed from outside package
UPDATE(Publish done)
Finally made a Publish request to the mqtt broker(a mosquitto server) and all this with a dynamic topic configured from user. As we found out the previous Class SendingMqttMessage was not supposed to be used at all. And we found out that we also needed and emitter to actually make a publish request with a dynamic topic.
#Inject
#Channel("panatha")
Emitter<String> emitter;
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response createUser(Device device) {
System.out.println("New Publish request: message->"+device.getMessage()+" & topic->"+device.getTopic());
emitter.send(MqttMessage.of(device.getTopic(), device.getMessage()));
return Response.ok().status(Response.Status.CREATED).build();
}
Now we need to find out about making a Subscription to a topic dynamically.
first to sett us to the same page:
Reactive messaging does not work with topics, but with channels.
That is important to note, because you can exclusively read or write to a channel. So if you want to provide both, you need to configure two channels pointing at the same topic, one incoming and one outgoing
To answer your question:
You made a pretty good start with Emitters, but you still lack the dynamic nature you'd like.
In your example, you acquired that Emitter thru CDI.
Now that is all we need, to make this dynamic, since we cann dynamically inject Beans at runtime using CDI like this:
Sending Messages
private Emitter<byte[]> dynamicEmitter(String topic){
return CDI.current().select(new TypeLiteral<Emitter<byte[]>>() {}, new ChannelAnnotation(topic)).get();
}
please also note, that i am creating a Emitter of type byte[], as this is the only currently supportet type of the smallrye-mqtt connector (version 3.4.0) according to its documentation.
Receiving Messages
To read messages from a reactive messaging channel, you can use the counterpart of the Emitter, which is the Publisher.
It can be used analog:
private Publisher<byte[]> dynamicReceiver(String topic){
return CDI.current().select(new TypeLiteral<Publisher<byte[]>>() {}, new ChannelAnnotation(topic)).get();
}
You can then process these Date in any way you like.
As demo, it hung it on a simple REST Endpoint
#GET
#Produces(MediaType.SERVER_SENT_EVENTS)
public Multi<String> stream(#QueryParam("topic") String topic) {
return Multi.createFrom().publisher(dynamicReceiver(topic)).onItem().transform(String::new);
}
#GET
#Path("/publish")
public boolean publish(#QueryParam("msg") String msg, #QueryParam("topic") String topic) {
dynamicEmitter(topic).send(msg.getBytes());
return true;
}
One more Thing
When creating this solution I hit a few pitfalls you should know about:
Quarkus removes any CDI-Beans that are "unused". So if you want to inject them dynamically, you need to exclude those, or turne off that feature.
All channels injected that way must be configured. Otherwise the injection will fail.
For some Reason, (even with removal completely disabled) I was unable to inject Emitters dynamically, unless they are ever injected elsewhere.

Using MarshallingWebServiceOutboundGateway with dynamic URI based on message header

I have the following configuration for sending SOAP requests as part of a integration flow, where uriDefinedInApplicationProperties is a fixed uri, defined in 'application.properties' file :
#Bean
public MarshallingWebServiceOutboundGateway outboundSOAPGateway()
{
final MarshallingWebServiceOutboundGateway outboundGateway = new MarshallingWebServiceOutboundGateway(
uriDefinedInApplicationProperties,
requestMarshaller,
responseMarshaller);
outboundGateway.setAdviceChain(Collections.singletonList(retryOutboundGatewayAdvice));
if (soapActionCallback!= null) {
outboundGateway.setRequestCallback(soapActionCallback);
}
return outboundGateway;
}
Now i have the requirement that the URI of the remote SOAP server should be dynamically generated ( i'm planning on using message headers to store the URI).
I wanted to do something like the following, but MarshallingWebServiceOutboundGateway does not seem to support it, and i haven't been able to find how to do something similar using spring integration dsl:
#Bean
public MarshallingWebServiceOutboundGateway outboundSOAPGateway()
{
final MarshallingWebServiceOutboundGateway outboundGateway = new MarshallingWebServiceOutboundGateway(
message -> (message -> message.getHeaders().get("remote.uri.header"),
requestMarshaller,
responseMarshaller);
outboundGateway.setAdviceChain(Collections.singletonList(retryOutboundGatewayAdvice));
if (soapActionCallback!= null) {
outboundGateway.setRequestCallback(soapActionCallback);
}
return outboundGateway;
}
I have noted that MarshallingWebServiceOutboundGateway has a setUriVariableExpressions(Map<String, Expression> uriVariableExpressions) method, but i didn't find any clear documentation on what it is supposed to do and how it works.
Also i tried to do something like the following to create the outbound gateway, but it does not seem to support requestCallbacks nor advice chain.
Http.outboundGateway(message -> message.getHeaders().get("remote.uri.header"))
.messageConverters(new MarshallingHttpMessageConverter(
remoteRequestMarshaller,
remoteResponseMarshaller));
What is the best way to create a SOAP outbound gateway with retry advice and dynamically generated uri?
The advice config is not a MessageHandler responsibility. If you use Java DSL, see a second argument (a GenericEndpointSpec lambda) of the handle() you use for that MarshallingWebServiceOutboundGateway:
/**
* Configure a list of {#link Advice} objects to be applied, in nested order, to the
* endpoint's handler. The advice objects are applied only to the handler.
* #param advice the advice chain.
* #return the endpoint spec.
*/
public S advice(Advice... advice) {
Yes, I agree that MarshallingWebServiceOutboundGateway (and its super class) doesn't support URI resolution against message at the moment. Feel free to raise a GH issue to fix a gap with a SpEL configuration like we have it for the mentioned Http.outboundGateway.
Meanwhile as a workaround you can consider to implement a DestinationProvider which reads an URI from a TheadLocal store. Before calling this gateway you should consult your message and store built URI into that ThreadLocal variable.

Testing Camel with MockEndpoints

I've got a series of "pipelined" components that all communicate through ActiveMQ message queues. Each component uses Camel to treat each of these queues as an Endpoint. Each component uses the same basic pattern:
Where each component consumes messages off of an input queue, processes the message(s), and then places 1+ messages on an outbound/output queue. The "output" queue then becomes the "input" queue for the next component in the chain. Pretty basic.
I am now trying to roll up my sleeves and provide unit testing for each component using the MockEndpoints provided by Camel's test API. I have been pouring over the javadocs and the few examples on Camel's website, but am having difficulty connecting all the dots.
It seems to me that, for each component, a portion of my unit testing is going to want to accomplish the following three things:
Test to see if there are messages waiting on a particular "input" queue
Pull those messages down and process them
Push new messages to an "output" queue and verify that they made it there
I believe I need to create MockEndpoints for each queue like so:
#EndpointInject(uri = "mock:inputQueue")
protected MockEndpoint intputQueue;
#EndpointInject(uri = "mock:outputQueue")
protected MockEndpoint outputQueue;
So now, in my JUnit test methods, I can set up expectations and interact with these endpoints:
#Test
public final void processMethodShouldSendToOutputQueue()
{
Component comp = new Component();
comp.process();
outputQueue.assertIsSatisfied();
}
I'm just not understanding how to wire everything up correctly:
How do I connect comp to the inputQueue and outputQueue MockEndpoints?
For each MockEndpoint, how do I set up expectations so that assertIsSatisfied() checks that a message is present inside a particular queue, or that a particular queue contains messages?
Adam, there are several ways to do this.
For POJO components, blackbox test them separately from any Camel context/routing to focus on business logic.
If you want to do end-to-end testing of the routes, consider using one of these approaches to validate that each step in the route is satisfied.
use NotifyBuilder to build Exchange validation expressions (somewhat complex to get your head around)
use AdviceWith to dynamically change the route before its run (add Log/Mock endpoints, etc)
I prefer AdviceWith because its very flexible and leverages the familiar MockEndpoints. For a complete example of this, see this unit test
In short, you will create a unit test to inject MockEndpoints into your route and then validate against them as usual...
context.getRouteDefinition("myRouteId").adviceWith(context, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
// mock all endpoints
mockEndpoints();
}
});
getMockEndpoint("mock:direct:start").expectedBodiesReceived("Hello World");
template.sendBody("direct:start", "Hello World");

How can I handle multiple messages concurrently from a JMS topic (not queue) with java and spring 3.0?

Note that I'd like multiple message listeners to handle successive messages from the topic concurrently. In addition I'd like each message listener to operate transactionally so that a processing failure in a given message listener would result in that listener's message remaining on the topic.
The spring DefaultMessageListenerContainer seems to support concurrency for JMS queues only.
Do I need to instantiate multiple DefaultMessageListenerContainers?
If time flows down the vertical axis:
ListenerA reads msg 1 ListenerB reads msg 2 ListenerC reads msg 3
ListenerA reads msg 4 ListenerB reads msg 5 ListenerC reads msg 6
ListenerA reads msg 7 ListenerB reads msg 8 ListenerC reads msg 9
ListenerA reads msg 10 ListenerB reads msg 11 ListenerC reads msg 12
...
UPDATE:
Thanks for your feedback #T.Rob and #skaffman.
What I ended up doing is creating multiple DefaultMessageListenerContainers with concurrency=1 and then putting logic in the message listener so that only one thread would process a given message id.
You don't want multiple DefaultMessageListenerContainer instances, no, but you do need to configure the DefaultMessageListenerContainer to be concurrent, using the concurrentConsumers property:
Specify the number of concurrent
consumers to create. Default is 1.
Specifying a higher value for this
setting will increase the standard
level of scheduled concurrent
consumers at runtime: This is
effectively the minimum number of
concurrent consumers which will be
scheduled at any given time. This is a
static setting; for dynamic scaling,
consider specifying the
"maxConcurrentConsumers" setting
instead.
Raising the number of concurrent
consumers is recommendable in order to
scale the consumption of messages
coming in from a queue. However, note
that any ordering guarantees are lost
once multiple consumers are
registered. In general, stick with 1
consumer for low-volume queues.
However, there's big warning at the bottom:
Do not raise the number of concurrent consumers for a topic.
This would lead to concurrent
consumption of the same message, which
is hardly ever desirable.
This is interesting, and makes sense when you think about it. The same would occur if you had multiple DefaultMessageListenerContainer instances.
I think perhaps you need to rethink your design, although I'm not sure what I'd suggest. Concurrent consumption of pub/sub messages seems like a perfectly reasonable thing to do, but how to avoid getting the same message delivered to all of your consumers at the same time?
At least in ActiveMQ what you want is totally supported, his name is VirtualTopic
The concept is:
You create a VirtualTopic (Simply creating a Topic using the prefix VirtualTopic. ) eg. VirtualTopic.Color
Create a consumer subscribing to this VirtualTopic matching this pattern Consumer.<clientName>.VirtualTopic.<topicName> eg. Consumer.client1.VirtualTopic.Color, doing it, Activemq will create a queue with that name and that queue will subscribe to VirtualTopic.Color then every message published to this Virtual Topic will be delivered to client1 queue, note that it works like rabbitmq exchanges.
You are done, now you can consume client1 queue like every queue, with many consumers, DLQ, customized redelivery policy, etc.
At this point I think you understood that you can create client2, client3 and how many subscribers you want, all of them will receive a copy of the message published to VirtualTopic.Color
Here the code
#Component
public class ColorReceiver {
private static final Logger LOGGER = LoggerFactory.getLogger(MailReceiver.class);
#Autowired
private JmsTemplate jmsTemplate;
// simply generating data to the topic
long id=0;
#Scheduled(fixedDelay = 500)
public void postMail() throws JMSException, IOException {
final Color colorName = new Color[]{Color.BLUE, Color.RED, Color.WHITE}[new Random().nextInt(3)];
final Color color = new Color(++id, colorName.getName());
final ActiveMQObjectMessage message = new ActiveMQObjectMessage();
message.setObject(color);
message.setProperty("color", color.getName());
LOGGER.info("status=color-post, color={}", color);
jmsTemplate.convertAndSend(new ActiveMQTopic("VirtualTopic.color"), message);
}
/**
* Listen all colors messages
*/
#JmsListener(
destination = "Consumer.client1.VirtualTopic.color", containerFactory = "colorContainer"
selector = "color <> 'RED'"
)
public void genericReceiveMessage(Color color) throws InterruptedException {
LOGGER.info("status=GEN-color-receiver, color={}", color);
}
/**
* Listen only red colors messages
*
* the destination ClientId have not necessary exists (it means that his name can be a fancy name), the unique requirement is that
* the containers clientId need to be different between each other
*/
#JmsListener(
// destination = "Consumer.redColorContainer.VirtualTopic.color",
destination = "Consumer.client1.VirtualTopic.color",
containerFactory = "redColorContainer", selector = "color='RED'"
)
public void receiveMessage(ObjectMessage message) throws InterruptedException, JMSException {
LOGGER.info("status=RED-color-receiver, color={}", message.getObject());
}
/**
* Listen all colors messages
*/
#JmsListener(
destination = "Consumer.client2.VirtualTopic.color", containerFactory = "colorContainer"
)
public void genericReceiveMessage2(Color color) throws InterruptedException {
LOGGER.info("status=GEN-color-receiver-2, color={}", color);
}
}
#SpringBootApplication
#EnableJms
#EnableScheduling
#Configuration
public class Config {
/**
* Each #JmsListener declaration need a different containerFactory because ActiveMQ requires different
* clientIds per consumer pool (as two #JmsListener above, or two application instances)
*
*/
#Bean
public JmsListenerContainerFactory<?> colorContainer(ActiveMQConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setConcurrency("1-5");
configurer.configure(factory, connectionFactory);
// container.setClientId("aId..."); lets spring generate a random ID
return factory;
}
#Bean
public JmsListenerContainerFactory<?> redColorContainer(ActiveMQConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
// necessary when post serializable objects (you can set it at application.properties)
connectionFactory.setTrustedPackages(Arrays.asList(Color.class.getPackage().getName()));
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setConcurrency("1-2");
configurer.configure(factory, connectionFactory);
return factory;
}
}
public class Color implements Serializable {
public static final Color WHITE = new Color("WHITE");
public static final Color BLUE = new Color("BLUE");
public static final Color RED = new Color("RED");
private String name;
private long id;
// CONSTRUCTORS, GETTERS AND SETTERS
}
Multiple Consumers Allowed on the Same Topic Subscription in JMS 2.0, while this was not the case with JMS 1.1. Please refer:
https://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html
This is one of those occasions where the differences in transport providers bubble up through the abstraction of JMS. JMS wants to provide a copy of the message for each subscriber on a topic. But the behavior that you want is really that of a queue. I suspect that there are other requirements driving this to a pub/sub solution which were not described - for example other things need to subscribe to the same topic independent of your app.
If I were to do this in WebSphere MQ the solution would be to create an administrative subscription which would result in a single copy of each message on the given topic to be placed onto a queue. Then your multiple subscribers could compete for messages on that queue. This way your app could have multiple threads among which the messages are distributed, and at the same time other subscribers independent of this application could dynamically (un)subscribe to the same topic.
Unfortunately, there's no generic JMS-portable way of doing this. You are dependent on the transport provider's implementation to a great degree. The only one of these I can speak to is WebSphere MQ but I'm sure other transports support this in one way or another and to varying degrees if you are creative.
Here's a possibility:
1) create only one DMLC configured with the bean and method to handle the incoming message. Set its concurrency to 1.
2) Configure a task executor with its #threads equal to the concurrency you desire. Create an object pool for objects which are actually supposed to process a message. Give a reference of task executor and object pool to the bean you configured in #1. Object pool is useful if the actual message processing bean is not thread-safe.
3) For an incoming message, the bean in DMLC creates a custom Runnable, points it to the message and the object pool, and gives it to task executor.
4) The run method of Runnable gets a bean from the object pool and calls its 'process' method with the message given.
#4 can be managed with a proxy and the object pool to make it easier.
I haven't tried this solution yet, but it seems to fit the bill. Note that this solution is not as robust as EJB MDB. Spring e.g. will not discard an object from the pool if it throws a RuntimeException.
Creating a custom task executor seemingly solved the issue for me, w/o duplicate processing:
#Configuration
class BeanConfig {
#Bean(destroyMethod = "shutdown")
public ThreadPoolTaskExecutor topicExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setAllowCoreThreadTimeOut(true);
executor.setKeepAliveSeconds(300);
executor.setCorePoolSize(4);
executor.setQueueCapacity(0);
executor.setThreadNamePrefix("TOPIC-");
return executor;
}
#Bean
JmsListenerContainerFactory<?> topicListenerFactory(ConnectionFactory connectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer, #Qualifier("topicExecutor") Executor topicExecutor) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setPubSubDomain(true);
configurer.configure(factory, connectionFactory);
factory.setPubSubDomain(true);
factory.setSessionTransacted(false);
factory.setSubscriptionDurable(false);
factory.setTaskExecutor(topicExecutor);
return factory;
}
}
class MyBean {
#JmsListener(destination = "MYTOPIC", containerFactory = "topicListenerFactory", concurrency = "1")
public void receiveTopicMessage(SomeTopicMessage message) {}
}
I've run into the same problem. I'm currently investigating RabbitMQ, which seems to offer a perfect solution in a design pattern they call "work queues." More info here: http://www.rabbitmq.com/tutorials/tutorial-two-java.html
If you're not totally tied to JMS you might look into this. There might also be a JMS to AMQP bridge, but that might start to look hacky.
I'm having some fun (read: difficulties) getting RabbitMQ installed and running on my Mac but think I'm close to having it working, I will post back if I'm able to solve this.
on server.xml configs:
so , in maxSessions you can identify the number of sessions you want.
Came across this question. My configuration is :
Create a bean with id="DefaultListenerContainer", add property name="concurrentConsumers" value="10" and property name="maxConcurrentConsumers" value ="50".
Works fine, so far. I printed the thread id and verified that multiple threads do get created and also reused.

Categories