I'm trying to implement the following JMS message flow using camel routes:
there is a topic published on external message broker. My program is listening for messages on this topic. Each incoming message triggers specific route to be executed - ONE TIME ONLY (some kind of ad-hoc, disposable route). This route is supposed to move messages between queues within my internal message broker based on some selector (get all messages from queue A matching given selector and move them to queue B). I'm only starting with camel and so far I figured out just the first part - listening for messages on topic:
<bean id="somebroker" class="org.apache.camel.component.jms.JmsComponent"
p:connectionFactory-ref="rmAdvisoriesConnectionFactory"/>
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<endpoint id="jms" uri="somebroker:topic:sometopic"/>
<route id="routeAdvisories">
<from ref="jms"/>
<to>???</to>
</route>
</camelContext>
Can you suggest a destination for these advisory messages? I need to read some of their JMS properties and use these values to construct JMS selector that will be used for the "move messages" operation. But I have no idea how to declare and trigger this ad-hoc route. It would be ideal if I could define it within the same camelContext using only Spring DSL. Or alternatively, I could route advisories to some java method, which would be creating and executing this ad-hoc routes. But if this isn't possible, I'll be grateful for any suggestion.
Thanks.
As far as I understand, it will be useful to use the 'selector' option, in your JMS consumer route, for example:
from("activemq:queue:test?selector=key='value1'").to("mock:a");
from("activemq:queue:test?selector=key='value2'").to("mock:b");
Maybe, another option is to implement some routes based on 'Content Based Router Pattern" through "choice" option. You can find more info here: http://camel.apache.org/content-based-router.html
I hope it helps.
I couldn't get it working the way I intended, so I had to abandon my original approach. Instead of using camel routes to move messages between queues (now I'm not sure camel routes are even intended to be used this way) I ended up using ManagedRegionBroker - the way JMX operation "moveMatchingMessagesTo" is implemented - to move messages matching given selector.
Related
I have a kafka consumer route from where I get some data.
from("Kafka:foo?brokers=localhost:9092")
Once I receive data from the consumer, use that data in the topic name for a paho mqtt consumer.
from("paho:#?brokerUrl=tcp://localhost:1883")
I'm not able to figure out how to set the dynamic header CamelMqttTopic, from first consumer, as both seems independent flows. I'm using camel with Spring framework. Excuse me if my basic camel understanding is flawed.
You can override the MQTT topic using the CamelPahoOverrideTopic message header with a value being the Kafka topic accessed through the kafka.TOPIC message header:
from("kafka:foo?brokers=localhost:9092")
.setHeader(PahoConstants.CAMEL_PAHO_OVERRIDE_TOPIC, simple("${headers[kafka.TOPIC]"))
.to("paho:#?brokerUrl=tcp://localhost:1883");
I am trying to write a camel route that consumes incoming message from a JMS queue and handles them concurrently in different threads. The camel route I have got is like this:
<camel:endpoint id="requestQueue" uri="jms:queue.request" camelContextId="camel"/>
<camel:endpoint id="responseQueue" uri="jms:queue.response" camelContextId="camel"/>
<camel:camelContext id="camel">
<camel:threadPool id="serviceThreadPool" poolSize="10" threadName="workerThread" maxPoolSize="20"/>
<camel:route id="requestServingRoute">
<camel:from ref="requestQueue"/>
<camel:threads executorServiceRef="serviceThreadPool">
<camel:to uri="bean:doSomething"/>
<camel:to ref="responseQueue"/>
</camel:threads>
</camel:route>
</camel:camelContext>
However, what I can observe is that the incoming message is indeed handled by separate threads, but they are handled in a sequential order.
What I tried to achieve is that camel handles each request in the doSomething bean in separate threads for each incoming request.
How can I achieve this?
Thank you very much.
You need to specify the number of concurrent consumers for the JMS request queue. See http://camel.apache.org/jms.html. You don't need to message around with the thread pools. eg in Java DSL syntax you would write something like:
from("jms:queue.request?concurrentConsumers=10")
.beanRef("bean", "doSomething")
.to("jms:queue.response)
I am using doing integration work with Camel and using the HTTP endpoint as a proxy to route certain messages to an HTTP endpoint. I have my route configured to use my custom error handler which places failed messages in a queue that I specified (Dead Letter Channel pattern).
<route>
...
<to uri="direct:MessageTypeGuaranteed"/>
</route>
<route errorHandlerRef="MyCustomErrorHandler">
<from uri="direct:MessageTypeGuaranteed">
<to uri="http://dummyUri?throwExceptionOnFailure=true"/>
</route>
When anything fails to get delivered to my http endpoint, it's being added to my custom queue ("CustomFailedMessageQueue"), and I have a separate route that attempts to retry these messages:
<route>
<from uri="jms:queue:CustomFailedMessageQueue">
<to uri="direct:MessageTypeGuaranteed"/>
</route>
What I'd like to do is to be able to specify that I only want a message to live say for 10 seconds. So I am trying to set time to live on my http destination itself.
For example, I have a processor that does something like this:
exchange.getIn().setHeader(Exchange.HTTP_URI, "http://localhost/nodeserver?timeToLive=10000");
However, I think I have misunderstood the documentation. The timeToLive option is only valid when passing it to the jms component, correct? In other words, if I want to make use of time to live with this end point, I will need to do that handling myself in a processor, correct?
Yes TimeToLive is an option from the JMS spec that the Camel JMS components support. That options has no meaning for other components such as HTTP.
It seems like you may want to use a filter EIP and discard the message if its to old, and you can use a java method etc to implement some code that figures out if its to old or not, and return a boolean
public boolean isNotToOld(Exchange exchange) {
...
return true // to accept and process the message
}
See more about the filter eip here
http://camel.apache.org/message-filter
And you can use it in a route something a like
<from uri="direct:MessageTypeGuaranteed">
<filter>
<method ref="myBean" method="isNotToOld"/>
<to uri="http://dummyUri?throwExceptionOnFailure=true"/>
</filter>
I am bit confused about Camel routes and its two endpoints : Direct and Seda. Well let's say i have a route like this :
public void configure()
{
from("direct:services")
.process(//Some processing here)
.to("http://ThirdPartyServers")
}
On top of this I have a rest web service which receives several requests, does some processing and then hands over the message to this route to get response from some third party servers. I have instantiated Camel Context through Spring framework like this :
<camelContext id="appCamelContext" xmlns="http://camel.apache.org/schema/spring"
trace="true" streamCache="true">
<propertyPlaceholder id="properties"
location="classpath:camel.properties" />
<camel:routeBuilder ref="oneRouteBuilder" />
<camel:routeBuilder ref="photosRouteBuilder" />
</camelContext>
Now the question is that in a instant I send multiple different messages to this route. Now Camel documentation says that direct component is invoked in single thread and is synchronous. So will all the messages be processed concurrently or one by one processing will happen ?
Also if i change the direct component to seda, will it make any difference ?
TIA
Update [after Petter's answer]:
Although Petter's answer has clarified but i have new doubt regarding the same Direct and Seda components. Lets say my route is now like this :
public void configure(){
from("direct:services")
.choice()
.when("some predicate here-Predicate1")
.to("seda:predicate1")
.otherwise()
.to("seda:fallback")
.end();
from("seda:predicate1")
.process("some processing")
.to("http://ThirdPartyServers");
from("seda:fallback")
.process("some processing")
.to("jms:fallbackqueue");
}
Now if i send 5 messages to direct component from different threads, so these messages would be processed concurrently. As you can see in the above route, direct component sends the message to seda component. So now will there be only one thread of seda component which will process all the different 5 messages? Meaning in the end all the messages will be processed one by one ?
The direct component runs in the thread of the caller. Simplified, it's as a regular java method invocation. Multiple messages can run through the route as long as multiple threads are calling the direct endpoint.
When invoking the SEDA (or VM) endpoint, you message is put on a queue (in memory). Another thread (in the route) picks messages from the queue one by one and processes them. You can configure how many threads the seda consumer should have by setting the concurrentConsumers option. By default, the one thread consuming messages makes sure only one message at a time is processed no matter how many threads are producing to this route.
It boils down to your statement
in a instant I send multiple different messages to this route
If this means different threads, or just from one single thread in a sequence,
We're currentyly developping an application using JMS and activemq (5.5.1).
We would like to define a higher priority for some messages, which would make them consumed first.
After setting the producer and the consumer (through spring (3.1) JMSTemplate), the priority does not fully work.
Indeed, when we "turn off" the consumer, and send some messages, the priority is respected, but when we add messages while the consumer is on, the messages are received in the same order they were sent.
The configuration is quite simple:
Priority was activated in the activemq config file:
<policyEntries>
<policyEntry queue=">" prioritizedMessages="true"/>
...
</policyEntries>
And QoS was enabled in the producer template configuration:
<bean id="jmsOCRTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory" />
<property name="defaultDestination" ref="destination_ocr" />
<property name="explicitQosEnabled" value="true" />
</bean>
To send a message with a high priority, we just change the template priority property on the producer side:
template.setPriority(9);
Any idea? Is this the normal behaviour, or is there some configuration we would have forgotten?
If my opinion you are not missing anything, I had a similar issue a couple of weeks ago (but with TTL and QPid).
First the JMS is not Strict about this :
JMS does not require that a provider strictly implement priority ordering of messages; however, it should do its best to deliver expedited messages ahead of normal messages.
Second, ActiveMQ does not YET implement priority queues, they say it will somewhere in 6.x version.
So, what you see is actually normal.
As a work-around you can use the Resequencer pattern if it fits your case.
http://camel.apache.org/resequencer.html
Here is another discussion on this subject:
http://activemq.2283324.n4.nabble.com/Priority-message-td2352179.html
I know it is late but this answers may help somebody.
If you want your consumer to consume message based on priority (Priority Queue) then you can use client side message priority. This means, when messages are being sent to your consumer (even before your consumer is receiving them, using prefetch), they will be cached on the consumer side and prioritized by default. This is regardless of whether you’re using priority support on the broker side. This could impact the ordering you see on the consumer so just keep this in mind.
To enable it, set the following configuration option on your broker URL, e.g.,
tcp://0.0.0.0:61616?jms.messagePrioritySupported=true
To disable it, tcp://0.0.0.0:61616?jms.messagePrioritySupported=false
So you do not require to use Camel (if you want to avoid complication)