I have camel route which suspends the route using control bus on exception.
onException(Exception.class).maximumRedeliveries(1).onRedeliveryRef("controlBusProcessor");
from("quartz2://myGroup/myTimerName?trigger.repeatInterval=2000&trigger.repeatCount=0")
.routeId("myRoute")
.process(simpleProcessor)
.to("stream:out")
The route initially calls a simple processor which raises an Exception, the onException has a redeliveryRef which calls a control bus processor which suspends the route, however it still calls simple processor to redeliver the message.
If the route is suspended, why is simple processor is still called, should it not wait until the route resumes?
No the route will complete its in-flight messages before being suspended. In your custom redelivery processor, you can mark the exchange to stop continue routing by setting the property on the exchange with Exchange.STOP=true.
See how the StopProcessor does that: https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/processor/StopProcessor.java
I suspect the redelivery starts at the point of failure which is at .process(simpleProcessor). But if you only want to suspend the route why not just add .onException(Exception.class).to("controlBusProcessor")
Related
I have the following Integration Flow:
Integration Flow
If an exception is thrown during the second split method, inside the parser, I would like to channel that message to an error channel, is that possible somehow?
One way is to have an output channel for spitter as an ExecutorChannel or QueueChannel. This way every splitted item is going to be processed in the separate thread. You can then apply any available error handling options for those asyn channels.
See docs for more info: https://docs.spring.io/spring-integration/docs/5.2.0.RELEASE/reference/html/error-handling.html#error-handling
Another one is to use a .gateway() with its errorChannel option downstream after the second splitter, so every item is going to be processed in isolation again.
Also an ExpressionEvaluatingRequestHandlerAdvice (maybe together with a RequestHandlerRetryAdvice) can be used downstream on the specific endpoint to deal with its own exceptions: https://docs.spring.io/spring-integration/docs/5.2.0.RELEASE/reference/html/messaging-endpoints.html#message-handler-advice-chain
I need to read a message from a queue and call a REST service doing a post.
Then, I was looking about the JMS transaction with Camel, it looks like you can set a maximumRedeliveries to process the queue message again, so rolling back the transaction in case of a failure, I was wondering how it can work if in the same camel route we have to call a REST service to post something, how that part can be rollback??
maxDelivery conf:
errorHandler(new TransactionErrorHandlerBuilder()
.loggingLevel(LoggingLevel.ERROR)
.useOriginalMessage()
.maximumRedeliveries(2)
.logHandled(false)
.logExhausted(true)
);
Pseudo code for the route:
//Reading message from the queue
from("activemq:AMQ.App.EMC2.In.PMQueue?jmsMessageType=Bytes").
transacted().
unmarshal(jaxbDataFormat).bean(pmMessageEnricher).
to("direct:start-post");
//Then doing the post
from("direct:start-post").
setHeader(Exchange.HTTP_METHOD, constant("POST")).
setHeader(Exchange.CONTENT_TYPE, constant("application/json")).
setBody(constant(pmMessageEnricher.toJson())).
to("http://xxx").
to("direct:start-somethingelse");
//Then doing something else
from("direct:start-somethingelse").
blabla...
Let say an exception occurs in the start-somethingelse, How the REST post call can be roll backed ? since we call an external service in a stateless way.
Your doubt is correct. In case of a JMS transaction rollback, the POST request cannot be rolled back because the service provider is not part of the JMS transaction. The transaction is just between the JMS broker and the Camel JMS consumer (see also Camel transactional client).
However, if you catch the processing error, you can apply the needed compensation logic. For example delete the already posted data with another request.
By the way: don't confuse Camel redeliveries and broker redeliveries!
Camel redeliveries are done by the Camel Errorhandler (not the broker). In your example it does up to 2 redeliveries. But be aware, that Camel redeliveries are not reprocessing the entire route, but only the failed processor.
So if to("http://xxx") fails and the Camel Errorhandler does redeliveries, Camel retries only the to("http://xxx").
In contrast, if your JMS transaction is rolled back, the broker redelivers the message to Camel and the entire route is processed again.
Take care that you don't "mask" the JMS redelivery with your Camel Errorhandler.
I've recently started playing with Apache Camel, and one of the things I've been having issues with is properly performing shutdown logic on selective routes. Since the shutdown logic would vary between routes, Camel's RoutePolicy made the most sense. Here's an example of why I'm trying to do.
public class ProcessingRouteBuilder extends RouteBuilder {
private ProducerTemplate prodTemplate;
public class ProcessingRouteBuilder(ProducerTemplate aProdTemplate) {
prodTemplate = aProdTemplate;
}
#Override
public void configure() {
from("direct://processing")
.routePolicy(new RoutePolicySupport() {
#Override
public void onStop(Route route) {
super.onStop(route);
prodTemplate.sendBody("direct://shutdownRoute", "msg");
}
})
.process(ex -> // Do stuff)
from("direct://shutdownRoute")
.log("Running shutdown A route body - ${body}");
}
}
The shutdown is done like (http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html). The ProducerTemplate comes from the primary CamelContext (read that it is good practice to create one ProducerTemplate per context).
Running this gives me a DirectConsumerNotAvailableException, I've used seda and vm (i don't plan to interact with multiple contexts, but I gave this a shot anyways), both don't exception, but the shutdown routes are never hit. Some questions I have
I might be using the Producer Template wrong? It doesn't look like it's creating an exchange.
Can I even use the ProducerTemplate once the Shutdown hook has been initiated? I'm not sure how Camel performs the shutdown, but it makes sense that it wouldn't allow new messages to be sent, and if the shutdown route is even available at the time of sending.
One thing to note, that I'm not handling here, is ensuring that the shutdown route is performed after the processing route finishes processing all messages in its queue. I'm not entirely sure if the onStop() method is called after there are no more inflight messages and if not, how to enforce it?
I figure another approach is to use when/choice at the beginning of each route and send some sort of shutdown notifier or message, but this seems a little more clunkier.
Thanks guys!
To programmatic shut down a route you can also use the Control Bus EIP.
However the "stop" logic is not clear as you'd want to send a message to the shutdownroute when the processing route stops, but if the stop happen because you are shutting down the camel context it may be possible that the shutdownRoute has already been stopped.
I have a java class which consumes messages from a queue, sending HTTP calls to some urls. I have made some search on google and also on stackoverflow (and really sorry if i have missed any sources mentioning about the problem) but couldnt find anything in details about setRollbackOnly call.
My question is... in case I rollback, the message which is consumed from the queue will be blocking the rest of the queue and will be looping until it is processed successfully or it will be requeued at the end of the current queue?
My code which I use for consuming from the queue and sending HTTP calls is below and the whole application is running on Glassfish server:
public class RequestSenderBean implements MessageListener
{
#Resource
private MessageDrivenContext mdbContext;
public RequestSenderBean(){}
public void onMessage(final Message message)
{
try
{
if(message instanceof ObjectMessage)
{
String responseOfCall=sendHttpPost(URL, PARAMS_FROM_MESSAGE);
if(responseOfCall.startsWith("Success"))
{
//Everything is OK, do some stuff
}
else if(responseOfCall.startsWith("Failure"))
{
//Failure, do some other stuff
}
}
catch(final Exception e)
{
e.printStackTrace();
mdbContext.setRollbackOnly();
}
}
}
This is fundamental JMS/messaging knowledge.
Queues implement "load balancing" scenarios, whereby a message hits a queue and is dequed to be processed by one consumer. Increasing the number of consumers increases potential throughput of that queue's processing. Each message on a queue will be processed by one and only one consumer.
Topics provide publish-subscribe semantics: all consumers of a topic will receive the message that is pushed to the topic.
With that in mind, once a message is dequed and handed (transactionally) to a consumer, it is by no means blocking the rest of the queue if it is asynchronous (as is the case with MDBs).
As the Java EE Tutorial states:
Message Consumption
Messaging products are inherently asynchronous: There is no fundamental timing dependency between the production and the consumption of a message. However, the JMS specification uses this term in a more precise sense. Messages can be consumed in either of two ways:
Synchronously: A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit.
Asynchronously: A client can register a message listener with a consumer. A message listener is similar to an event listener. Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener’s onMessage method, which acts on the contents of the message.
Because you use a MessageListener which is by definition asynchronous, you are not blocking the queue or its subsequent processing.
Also from the tutorial is the following:
Using Session Beans to Produce and to Synchronously Receive Messages
An application that produces messages or synchronously receives them can use a session bean to perform these operations. The example in An Application That Uses the JMS API with a Session Bean uses a stateless session bean to publish messages to a topic.
Because a blocking synchronous receive ties up server resources, it is not a good programming practice to use such a receive call in an enterprise bean. Instead, use a timed synchronous receive, or use a message-driven bean to receive messages asynchronously. For details about blocking and timed synchronous receives, see Writing the Clients for the Synchronous Receive Example.
As for message failure, it depends on how your queue is configured. You can set error-queues (in the case of containers like Glassfish or Weblogic) that failed messages are pushed to for later inspection. In your case, you're using setRollbackOnly which is handled thus:
7.1.2 Coding the Message-Driven Bean: MessageBean.java
The message-driven bean class, MessageBean.java, implements the
methods setMessageDrivenContext, ejbCreate, onMessage, and ejbRemove.
The onMessage method, almost identical to that of TextListener.java,
casts the incoming message to a TextMessage and displays the text. The
only significant difference is that it calls the
MessageDrivenContext.setRollbackOnly method in case of an exception.
This method rolls back the transaction so that the message will be
redelivered.
I recommend you read the Java EE Tutorial as well as the Enterprise Integration Patterns book which covers messaging concepts in good detail that's also product/technology-agnostic.
I am bit confused about Camel routes and its two endpoints : Direct and Seda. Well let's say i have a route like this :
public void configure()
{
from("direct:services")
.process(//Some processing here)
.to("http://ThirdPartyServers")
}
On top of this I have a rest web service which receives several requests, does some processing and then hands over the message to this route to get response from some third party servers. I have instantiated Camel Context through Spring framework like this :
<camelContext id="appCamelContext" xmlns="http://camel.apache.org/schema/spring"
trace="true" streamCache="true">
<propertyPlaceholder id="properties"
location="classpath:camel.properties" />
<camel:routeBuilder ref="oneRouteBuilder" />
<camel:routeBuilder ref="photosRouteBuilder" />
</camelContext>
Now the question is that in a instant I send multiple different messages to this route. Now Camel documentation says that direct component is invoked in single thread and is synchronous. So will all the messages be processed concurrently or one by one processing will happen ?
Also if i change the direct component to seda, will it make any difference ?
TIA
Update [after Petter's answer]:
Although Petter's answer has clarified but i have new doubt regarding the same Direct and Seda components. Lets say my route is now like this :
public void configure(){
from("direct:services")
.choice()
.when("some predicate here-Predicate1")
.to("seda:predicate1")
.otherwise()
.to("seda:fallback")
.end();
from("seda:predicate1")
.process("some processing")
.to("http://ThirdPartyServers");
from("seda:fallback")
.process("some processing")
.to("jms:fallbackqueue");
}
Now if i send 5 messages to direct component from different threads, so these messages would be processed concurrently. As you can see in the above route, direct component sends the message to seda component. So now will there be only one thread of seda component which will process all the different 5 messages? Meaning in the end all the messages will be processed one by one ?
The direct component runs in the thread of the caller. Simplified, it's as a regular java method invocation. Multiple messages can run through the route as long as multiple threads are calling the direct endpoint.
When invoking the SEDA (or VM) endpoint, you message is put on a queue (in memory). Another thread (in the route) picks messages from the queue one by one and processes them. You can configure how many threads the seda consumer should have by setting the concurrentConsumers option. By default, the one thread consuming messages makes sure only one message at a time is processed no matter how many threads are producing to this route.
It boils down to your statement
in a instant I send multiple different messages to this route
If this means different threads, or just from one single thread in a sequence,