I'm trying to use a Camel poll-once route that will use a file if it's present and log an error if not.
By default the route does nothing if the file does not exist so I've started by adding consumer.sendEmptyMessageWhenIdle=true to the URI. I then check for null body to decide whether to log an exception or continue:
from(theFileUri)
.onCompletion()
.onCompleteOnly()
.log("SUCCESS")
.bean(theOtherAction, "start")
.end()
.onException(Exception.class)
.logStackTrace(true)
.log(ERROR, "Failed to load file")
.handled(true)
.end()
.choice()
.when(body().isNotNull())
.to(NEXT_ROUTE_URI)
.endChoice()
.otherwise()
.throwException(new FileNotFoundException(theFileUri))
.endChoice();
There are two problems with this:
if the file is missing, the success log still happens (but not the bean!)
the stack trace is not printed
If there is a better way to do this then I'd welcome suggestions but I'd also like to know what I'm doing wrong in this method.
It's still not completely clear to me what is going on. However, I believe that the onException call needs to be separated from the chain. It looks to me that logStackTrace applies only to redelivery attempts. The number of attempts defaults to 0 and this is what I want. The only way to access the exception from the Java DSL appears to be a custom Processor. The getException() method will return null if you are using handled(true) so you must use Exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class).
I also suspect that the log message from the onCompletion is due to it running in parallel before the exception aborts:
Camel 2.13 or older - On completion runs in separate thread Icon The
onCompletion runs in a separate thread in parallel with the original
route. It is therefore not intended to influence the outcome of the
original route. The idea for on completion is to spin off a new thread
to eg send logs to a central log database, send an email, send alterts
to a monitoring system, store a copy of the result message etc.
Therefore if you want to do some work that influence the original
route, then do not use onCompletion for that. Notice: if you use the
UnitOfWork API as mentioned in the top of this page, then you can
register a Synchronization callback on the Exchange which is executed
in the original route. That way allows you to do some custom code when
the route is completed; this is how custom components can enlist on
completion services which they need, eg the File component does that
for work that moves/deletes the original file etc.
Since I want to not run this code on exception, I think I can just abort the route with the exception.
I currently have this:
onException(Exception.class)
.handled(true)
.process(new Processor()
{
#Override
public void process(Exchange anExchange) throws Exception
{
Exception myException = anExchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class);
LOGGER.error("Failed to load", myException);
}
});
from(theFileUri)
.choice()
.when(body().isNotNull())
.to(NEXT_ROUTE_URI)
.log("SUCCESS")
.bean(theOtherAction, "start")
.endChoice()
.otherwise()
.throwException(new FileNotFoundException(theFileUri));
Related
I try to use Camel to deliver files from one folder to a rest call and Im trying to achieve that on Error it's tried to redeliver twice and then moved to an error folder if the second redelivery fails as well. My code in the RouteBuilder's configure method looks like this:
errorHandler(deadLetterChannel("file:///home/camelerror").useOriginalMessage());
from("file:///home/camelefiles")
.onException(RetryableException.class)
.log("RetryableException handled")
.maximumRedeliveries(2)
.end()
.routeId(port.id())
.throwException(new RetryableException());
I get the "RetryableException handled" logs so I guess the exception is handled correctly but it redelivers the message an infinite number of times.
What am I doing wrong and how can I achieve that the message is only redelivered twice and then the deadLetterChannel is used?
With camel, I want to send an HTTP request and, if it fails, take some corrective action and try it once more.
Sending the request is wrapped in a direct route and requires the generation of a unique token, apart from that the request should be exactly the same. If the request fails the second time, it should fail the "normal" way.
I want the sending logic in it's own route because sometimes I want to call it without retry.
How can I set up this scenario with the camel DSL? I tried various combinations of onException and errorHandler but it doesn't even seem to catch the exception.
Example (that doesn't work):
from("direct:send-request-no-retry")
.setHeader("token").exchange(this::generateToken)
.to("http://example.com");
from("direct:fix-the-error")
// ...
;
from("direct:send-request")
.onException(HttpOperationFailedException.class)
.to("direct:fix-the-error")
.maximumRedeliveries(1)
.useOriginalMessage()
.end()
.to("direct:send-request-no-retry");
I have mainly used onException() clause as a global exception handler. With quick testing it does not indeed catch the exceptions when put directly into a route. Couple of options that come to my mind are:
1st option do-try clauses:
from("direct:send-request")
.doTry()
.to("direct:send-request-no-retry")
.doCatch(HttpOperationFailedException.class)
.to("direct:fix-the-error")
.to("direct:send-request-no-retry")
.doFinally()
.whatevs()
.endDoTry();
But this is a bit janky since you would have to handle the exception again from the doCatch clause. Another one is to throw custom exception from the "send-request-no-retry".
from("direct:send-request-no-retry")
.setHeader("token").exchange(this::generateToken)
.doTry()
.to("http://example.com")
.doCatch(HttpOperationFailedException.class)
.throwException(new CustomExceptionThatExtendsRunTimeException())
.doFinally();
onException(CustomExceptionThatExtendsRunTimeException.class)
.handled(true)
.to("direct:fix-the-error")
.maximumRedeliveries(1)
.useOriginalMessage()
.to("direct:send-request-no-retry")
.end();
I didn't test this so I'm not sure if this creates an infinite feedback loop or if the maximumRedeliveries will cut it off. If it does you can implement another "direct:send-request-no-retry" route that does not throw the customException but just fails silently or throws the HttpOperationFailedException and pass the request there from onException(). Or set the retryAmount to a header and check that header in onException handler to see if we want to try again.
The customException is for the purposes that the onException() clause catches all exceptions from every route of the specified type. And I think you don't want to pass every failed http request to your retryroute.
I have a spring rabbit consumer:
public class SlackIdle1Consumer extends AbstractMessageConsumer {
#Override public void process(Message amqpMessage, Channel channel)
throws Exception {
/*very bad exception goes here.
it causes amqp message to be rejected and if no other consumer is available and error
still persists, the message begins looping over and over.
And when the error is fixed,
those messages are being processed but the result of this procession may be harmful.
*/
}
}
}
And somewhere inside an exception happens. Lets imagine this is a bad exception - development logic error. So amqp message begins to spin indefinitely, and when error is fixed and consumer restarted, all old messages are being processed, and it's bad, because logic and data may change since those messages were sent. How to handle it properly?
So the question is: how to fix this situation properly? Should I wrap all my code to try-catch clause or will I have to develop 'checks' in each consumer to prevent consistency issues in my app?
There are several options:
Set the container's defaultRequeueRejected property to false so failed messages are always rejected (discarded or sent to a dead letter exchange depending on queue configuration).
If you want some exceptions to be retried and others not, then add a try catch and throw an AmqpRejectAndDontRequeueException to reject those you don't want retried.
Add a custom ErrorHandler to the container, to do the same thing as #2 - determine which exceptions you want retried - documentation here.
Add a retry advice with a recoverer - the default recoverer simply logs the error, the RejectAndDontRequeueRecoverer causes the message to be rejected after retries are exhausted, the RepublishMessageRecoverer is used to write to a queue with additional diagnostics in headers - documentation here.
Within a single camel route I have two url calls, making calls to two different applications.
to("http://datasource1/data)
//some process
to("http://datasource2/data)
//some process
Both are capable of throwing UnKnowHostException.
So, if the URL1 throws the exception i have to handled and set the exchange body as "Datasource 1 not available" and if URL2 throws the same exception , I want to show a different message.
How to handle this using onException
You can use onWhen. Set some header (in my example "httpDatasource") before each request, and after use different handlers.
onException(UnKnowHostException.class).onWhen(header("httpDatasource").isEqualTo("1")).to("...");
onException(UnKnowHostException.class).onWhen(header("httpDatasource").isEqualTo("2")).to("...");
.....
setHeader("httpDatasource").constant("1")
to("http://datasource1/data)
//some process
setHeader("httpDatasource").constant("2")
to("http://datasource2/data)
//some process
I would use the camel try catch blocks (as suggested by #soilworker).
.doTry()
.to("http://datasource1/data")
.doCatch(UnknownHostException.class)
// Add message 1 here
.end()
//process
.doTry()
.to("http://datasource2/data")
.doCatch(UnknownHostException.class)
// Add message 2 here
.end()
// process
It's more verbose, but it's easy to understand and clearly associates the message with the exception. And in the event you wish to make the to calls asynchronous, you can.
You can use a route specific onException but you would need to split your route into multiple routes:
from("somewhere")
.to("direct:datasource1")
//process
.to("direct:datasource2")
//process
from("direct:datasource1")
.onException(UnknownHostException.class)
// add message 1 here
.end()
.to("http://datasource/data")
from("direct:datasource2")
.onException(UnknownHostException.class)
// add message 2 here
.end()
.to("http://datasource2/data")
I don't believe there's a way of using onException with the one route but applied to different to calls (other than using #Alexeys or #Ewouts suggestion). Would love to hear about it if there is.
I want to put continue behaviour in route, my route is like following
from("file:D:\\?fileName=abc.csv&noop=true").split().unmarshal().csv()
.to("direct:insertToDb").end();
from("direct:insertToDb")
.to("direct:getDataId")
.to("direct:getDataParameters")
.to("direct:insertDataInDb");
from("direct:getDataId")
.to("sql:SELECT id FROM data WHERE name = :#name)
.choice()
.when(header("id").isGreaterThan(0) )
.setProperty("id", header("id"))
.otherwise()
.log("Error for")
.endChoice().end();
I want that if direct:getDataId dont find any record , my execution of route for current record from CSV get skip and program process next request. it would be equal to continue keyword.
How i can achieve this in Apache Camel route?
You can modify your routes like this:
from("file:D:\\?fileName=abc.csv&noop=true").split().unmarshal().csv()
.to("sql:SELECT id FROM data WHERE name = :#name?outputHeader=id&outputType=SelectOne)
.choice().when(header("id").isGreaterThan(0))
.to("direct:getDataParameters")
.to("direct:insertDataInDb")
.end();
Have you got a test for this? I suggest you try using CamelTestSupport because what you want is how camel will execute by default.
From Camel Split Docs:
stopOnException
default:false
description: Whether or not to stop continue processing immediately when an exception occurred. If disable, then Camel continue splitting and process the sub-messages regardless if one of them failed. You can deal with exceptions in the AggregationStrategy class where you have full control how to handle that.