Need to execute #Handler(import org.apache.camel.Handler) method in multi-threading environment. below is my current code and camelroute.xml file. Any Idea or suggestion would be appreciable.
#Component("messagehandler")
public class HandleMessages {
#Handler
public void handle(String body, Exchange exchange) throws Exception {
// do some business operation
}
}
<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<route id="IncomingRoute">
<from uri="someSourceURL" />
<to uri="bean:messagehandler" />
<log message="Message Body - ${body}" />
</route>
</routes>
In general terms, thread safety is all about execution – that's all. A given method / routine / piece of code is thread-safe if it guarantees the manipulation of shared data (data structures, etc.) safely in a way that multiple threads don't corrupt that data.
So it really depends on how you structure the execution of a given workflow; and that's the same for any other piece of code you can think of.
Since you are using Apache Camel, take a look at their threading model. If I remember correctly, you would have to define your routes / workflows in such a way they run concurrently by using parallelProcessing (CC EIP), a custom thread pool, or using staged event-driven architecture (SEDA); at that point you would need to take care about what you do in the handlers (or any other "processor(s)" that handle shared data), otherwise you should be OK.
Another thing you need to think about is how Camel uses its routing engine to route messages synchronously or asynchronously; be aware that synchronicity and MEP affects the threading model.
Related
Is it a best practice to use end() for every route?
The following works:
from("jms:some-queue")
.beanRef("bean1", "method1")
.beanRef("bean2", "method2")
and so is this,
from("jms:some-queue")
.beanRef("bean1", "method1")
.beanRef("bean2", "method2")
.end()
No! Calling end() to "end" a Camel route is not a best practice and won't yield any functional benefits.
For common ProcessorDefinition functions like to(), bean() or log() it simply leads to a call to the endParent() method, which as one can see from the Camel source code, does very little:
public ProcessorDefinition<?> endParent() {
return this;
}
The call to end() is required, once you have called processor definitions that start their own block and most prominently includes TryDefinitions aka doTry() and ChoiceDefinitions aka choice(), but also well know functions like split(), loadBalance(), onCompletion() or recipientList().
You must use the end() when you want to end specific route which is in action. It can be best explained in example of onCompletion
from("direct:start")
.onCompletion()
// this route is only invoked when the original route is complete as a kind
// of completion callback
.to("log:sync")
.to("mock:sync")
// must use end to denote the end of the onCompletion route
.end()
// here the original route contiues
.process(new MyProcessor())
.to("mock:result");
Here you must put end to indicate operation related to onCompletion is done and you are resuming operation on the original rote.
This becomes more lucid and easy to understand if you are using XML DSL instead of java. Because in this you don't have to use end tag. The closing tags of XML will take care of writing end(). Below is exactly same example written in XML DSL
<route>
<from uri="direct:start"/>
<!-- this onCompletion block will only be executed when the exchange is done being routed -->
<!-- this callback is always triggered even if the exchange failed -->
<onCompletion>
<!-- so this is a kinda like an after completion callback -->
<to uri="log:sync"/>
<to uri="mock:sync"/>
</onCompletion>
<process ref="myProcessor"/>
<to uri="mock:result"/>
The question may be generic but it's quite just what the title says.
I have an external API using HTTPS which I need to invoke within a Camel route to get some JSON response back however, I cannot seem to find a good way to do this.
I tried Camel's component 'restlet' to invoke the API but no luck. I tried to make use of CXFRS which requires a bean to be setup, which in turn requires a 'serviceClass' as far as I understand. Obviously with the API being a third-party external service there is no way to supply that.
Does anyone have any ideas or directions that they can point me in to merely invoke an external REST API which returns a JSON response?
Much appreciated.
Ok, it turns out I was utterly confused!
#Component
public class WeatherRESTRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer:aTimer?fixedRate=true&period=10s")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.to("ahc:https://restcountries.p.mashape.com/callingcode/90")
.routeId("TEST")
.log("${body}");
}
This is the working route as per my question and troubles I had the REST API URL in the .from which in Camel land means I want to expose that as a REST Endpoint rather than invoke it.
I was able to come to my sense while reading through the mailing list linked below.
http://camel.465427.n5.nabble.com/Making-Periodic-HTTP-Request-Using-Timer-td5749336.html
P.S.Thank you #6ton I have tried the solutions on that page before hand.
Using spring DSL with timer component
<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<route id="fetchdata">
<from uri="timer:somename?period=24h"/>
<toD uri="https://some/api/xxx?httpMethod=GET"/>
<to uri="file://abcd?fileName=${exchangeId}&fileExist=Append"/>
</route>
</routes>
In above we are storing it in file, if you want you can you can send it to other route or queue
<to uri="activemq:queuename?jmsMessageType=Text&exchangePattern=InOnly"/>
We have an XML file which consists of multiple customer elements in it and we want to save customer information to the DB using transactions. From my understanding, transactions needs to be run in a single thread to roll back the whole transaction in case of errors.
Here's my XML:
<root>
<customers>
<customer>...</customer>
<customer>...</customer>
<customer>...</customer>
<customer>...</customer>
</customers
</root>
Here is my route:
<route id="routeA">
<from uri="direct-vm:sample" />
<transacted />
<splitter parallelProcessing = "true" stopOnException="true"
strategyRef="combine" />
<xpath>/root/customers/customer</xpath>
<bean ref="customerService" method="saveCustomer" />
<onException>java.lang.Exception</onException>
<handled><constant>true</constant></handled>
<rollback markRollbackOnly="true" />
</route>
The method saveCustomer() runs a lot of business logic before saving customers to the Database and if for some reason an exception is thrown for 1 or 2 customers, I see multiple rollback messages, and it seems like this is happening for each thread. Do transactions in camel routes with parallel processing work? Is there any other way to save customers in parallel to the DB in a single DB transaction?
No you cannot do parallel work in the same transaction. The work must occur on the same thread.
You can use shareUnitOfWork() in combination with your parallelProcessing.
As the Camel Documentation of the Splitter EIP mentions it will rollback the entire unit of work not only the sub units:
When the Splitter is done, it checks the state of the shared unit of work and checks if any errors occurred. And if an error occurred it will set the exception on the Exchange and mark it for rollback. The error handler will yet again kick in, as the Exchange has been marked as rollback and it had an exception as well. No redelivery attempts is performed (as it was marked for rollback) and the Exchange will be moved into the dead letter queue.
see Sharing Unit of work chapter in the link.
I am bit confused about Camel routes and its two endpoints : Direct and Seda. Well let's say i have a route like this :
public void configure()
{
from("direct:services")
.process(//Some processing here)
.to("http://ThirdPartyServers")
}
On top of this I have a rest web service which receives several requests, does some processing and then hands over the message to this route to get response from some third party servers. I have instantiated Camel Context through Spring framework like this :
<camelContext id="appCamelContext" xmlns="http://camel.apache.org/schema/spring"
trace="true" streamCache="true">
<propertyPlaceholder id="properties"
location="classpath:camel.properties" />
<camel:routeBuilder ref="oneRouteBuilder" />
<camel:routeBuilder ref="photosRouteBuilder" />
</camelContext>
Now the question is that in a instant I send multiple different messages to this route. Now Camel documentation says that direct component is invoked in single thread and is synchronous. So will all the messages be processed concurrently or one by one processing will happen ?
Also if i change the direct component to seda, will it make any difference ?
TIA
Update [after Petter's answer]:
Although Petter's answer has clarified but i have new doubt regarding the same Direct and Seda components. Lets say my route is now like this :
public void configure(){
from("direct:services")
.choice()
.when("some predicate here-Predicate1")
.to("seda:predicate1")
.otherwise()
.to("seda:fallback")
.end();
from("seda:predicate1")
.process("some processing")
.to("http://ThirdPartyServers");
from("seda:fallback")
.process("some processing")
.to("jms:fallbackqueue");
}
Now if i send 5 messages to direct component from different threads, so these messages would be processed concurrently. As you can see in the above route, direct component sends the message to seda component. So now will there be only one thread of seda component which will process all the different 5 messages? Meaning in the end all the messages will be processed one by one ?
The direct component runs in the thread of the caller. Simplified, it's as a regular java method invocation. Multiple messages can run through the route as long as multiple threads are calling the direct endpoint.
When invoking the SEDA (or VM) endpoint, you message is put on a queue (in memory). Another thread (in the route) picks messages from the queue one by one and processes them. You can configure how many threads the seda consumer should have by setting the concurrentConsumers option. By default, the one thread consuming messages makes sure only one message at a time is processed no matter how many threads are producing to this route.
It boils down to your statement
in a instant I send multiple different messages to this route
If this means different threads, or just from one single thread in a sequence,
I have a remote service that I'm calling to load pricing data for a product, when a specific event occurs. Once loaded, the product pricing is then broadcast for another consumer to process elsewhere.
Rather than call the remote service on every event, I'd like to batch the events into small groups, and send them in one go.
I've cobbled together the following pattern based on an Aggregator. Although it works, lots of it 'smells' -- especially my SimpleCollatingAggregator. I'm new to Spring Integration, and EIP in general, and suspect I'm misusing components.
The Code
My code is triggered elsewhere in code by calling a method on the below #Gateway:
public interface ProductPricingGateway {
#Gateway(requestChannel="product.pricing.outbound.requests")
public void broadcastPricing(ProductIdentifer productIdentifier);
}
This is then wired to an aggregator, as follows:
<int:channel id="product.pricing.outbound.requests" />
<int:channel id="product.pricing.outbound.requests.batch" />
<int:aggregator input-channel="product.pricing.outbound.requests"
output-channel="product.pricing.outbound.requests.batch" release-strategy="releaseStrategy"
ref="collatingAggregator" method="collate"
correlation-strategy-expression="0"
expire-groups-upon-completion="true"
send-partial-result-on-expiry="true"/>
<bean id="collatingAggregator" class="com.mangofactory.pricing.SimpleCollatingAggregator" />
<bean id="releaseStrategy" class="org.springframework.integration.aggregator.TimeoutCountSequenceSizeReleaseStrategy">
<!-- Release when: 10 Messages ... or ... -->
<constructor-arg index="0" value="10" />
<!-- ... 5 seconds since first request -->
<constructor-arg index="1" value="5000" />
</bean>
Here's the aggregator implementation:
public class SimpleCollatingAggregator {
public List<?> collate(List<?> input)
{
return input;
}
}
Finally, this gets consumed on the following #ServiceActivator:
#ServiceActivator(inputChannel="product.pricing.outbound.requests.batch")
public void fetchPricing(List<ProductIdentifer> identifiers)
{
// omitted
}
Note: In practice, I'm also using #Async, to keep the calling code as quick-to-return as possible. I have a bunch of questions about that too, which I'll move to a seperate question.
Question 1:
Given what I'm trying to acheive, is an aggregator pattern an appropriate choice here? This feels like a lot of boilerplate -- is there a better way?
Question 2:
I'm using a fixed collation value of 0, to effectively say : 'It doesn't matter how you group these messages, take 'em as they come.'
Is this an appropriate way of achieving this?
Question 3:
SimpleCollatingAggregator simply looks wrong to me.
I want this to receive my individual inbound ProductIdentifier objects, and group them into batches, and then pass them along. This works, but is it appropriate? Are there better ways of acheiving the same thing?
Q1: Yes, but see Q3 and the further discussion below.
Q2: That is the correct way to say 'no correlation needed' (but you need the expire-groups-on-completion, which you have).
Q3: In this case, you don't need a custom Aggregator, just use the default (remove the ref and method attributes).
Note that the aggregator is a passive component; the release is triggered by the arrival of a new message; hence the second part of your release strategy will only kick in when a new message arrives (it won't spontaneously release the group after 5 seconds).
However, you can configure a MessageGroupStoreReaper for that purpose: http://static.springsource.org/spring-integration/reference/html/messaging-routing-chapter.html#aggregator