I have set up an errorHandler in a Camel route that will retry a message several times before sending the message to a dead letter channel (an activemq queue in this case). What I would like is to see an ERROR log when the message failed to be retried the max number of times and was then sent to the dead letter queue.
Looking at the docs for error handling and dead letter channels, it seems that there are 2 options available on the RedeliveryPolicy: retriesAttemptedLogLevel and retriesExhaustedLogLevel. Supposedly by default the retriesExhaustedLogLevel is already set at LoggingLevel.ERROR, but it does not appear to actually log anything when it has expended all retries and routes the message to the dead letter channel.
Here is my errorHandler definition via Java DSL.
.errorHandler(this.deadLetterChannel(MY_ACTIVE_MQ_DEAD_LETTER)
.useOriginalMessage()
.maximumRedeliveries(3)
.useExponentialBackOff()
.retriesExhaustedLogLevel(LoggingLevel.ERROR)
.retryAttemptedLogLevel(LoggingLevel.WARN))
I have explicitly set the level to ERROR now and it still does not appear to log out anything (to any logging level). On the other hand, retryAttemptedLogLevel is working just fine and will log to the appropriate LoggingLevel (ie, I could set retryAttemptedLogLevel to LoggingLevel.ERROR and see the retries as ERROR logs). However I only want a single ERROR log in the event of exhaustion, instead of an ERROR log for each retry when a subsequent retry could potentially succeed.
Maybe I am missing something, but it seems that the retriesExhaustedLogLevel does not do anything...or does not log anything if the ErrorHandler is configured as a DeadLetterChannel. Is there a configuration that I am still needing, or does this feature of RedeliveryPolicy not execute for this specific ErrorHandlerFactory?
I could also set up a route to send my exhausted messages that simply logs and routes to my dead letter channel, but I would prefer to try and use what is already built into the ErrorHandler if possible.
Updated the ErrorHandler's DeadLetterChannel to be a direct endpoint. Left the 2 logLevel configs the same. I got the 3 retry attempted WARN logs, but no ERROR log telling me the retries were exhausted. I did, however, set up a small route listening to the direct dead letter endpoint that logs, and that is working.
Not a direct solution to my desire to have the ERROR log work for the exhaustion, but is an acceptable workaround for now.
Please try with this code:
.errorHandler(deadLetterChannel("kafka:sample-dead-topic")
.maximumRedeliveries(4).redeliveryDelay(60000)
.retriesExhaustedLogLevel(LoggingLevel.WARN)
.retryAttemptedLogLevel( LoggingLevel.WARN)
.retriesExhaustedLogLevel(LoggingLevel.ERROR)
.logHandled(true)
.allowRedeliveryWhileStopping(true)
.logRetryStackTrace(true)
.logExhausted(true)
.logStackTrace(true)
.logExhaustedMessageBody(true)
)
retry is configured for 1 minute interval.
Camel application logged the errors for evry retry with the detailed information.
Related
I try to use Camel to deliver files from one folder to a rest call and Im trying to achieve that on Error it's tried to redeliver twice and then moved to an error folder if the second redelivery fails as well. My code in the RouteBuilder's configure method looks like this:
errorHandler(deadLetterChannel("file:///home/camelerror").useOriginalMessage());
from("file:///home/camelefiles")
.onException(RetryableException.class)
.log("RetryableException handled")
.maximumRedeliveries(2)
.end()
.routeId(port.id())
.throwException(new RetryableException());
I get the "RetryableException handled" logs so I guess the exception is handled correctly but it redelivers the message an infinite number of times.
What am I doing wrong and how can I achieve that the message is only redelivered twice and then the deadLetterChannel is used?
I have configured ActiveMQ redelivery plugin as follows (with max 4 redeliveries)
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<defaultEntry>
<redeliveryPolicy initialRedeliveryDelay="5000" maximumRedeliveries="4" redeliveryDelay="10000"/>
</defaultEntry>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
If a client fails to send ACK the message is redelivered. So far so good... However, the max redeliveries are completely ignored by the broker and it keeps redelivering the messages "infinitely" many times. Also the message is never moved to DLQ.
I also tried using:
?jms.redeliveryPolicy.maximumRedeliveries=4 on the connection URI (STOMP Connector), but also with no luck.
Any help is most appreciated!
For a STOMP client I would assume that the broker will not consider the message as being delivered unless the client either ACKs it or NACKs it otherwise it must assume that it never made it to a client and therefore treats it as always having a delivery count of zero. The broker redelivery plugin keys off the message's delivery count so if the message is treated as not having been delivered which in this case it likely is then it will take no action on the message.
I've got a route which is defined in Java in a route builder as follows:
from("jms:topic:trigger?maxConcurrentConsumers=1")
.autoStartup(true)
.beanRef("someProcessorBeen");
This has worked well for a few years.
However, under certain circumstances I want to be able to throw a specific 'retry' exception in the java processor bean, so that the same exchange is re-sent to the Bean.
I've tried adding &transacted=true to the JMS topic reference URI in the 'from' part, but I just get an ERROR log message from the DefaultErrorHandler which says "Failed delivery for ......" when I throw my retry Exception.
I've also tried adding:
onException(MyRetryException.class).maximumRedeliveries(10);
Before the from route definition, but the route doesn't seem to get created, as the route does not process any topic messages.
Note that this is all running within an OSGi environment, with ActiveMQ handling the JMS stuff.
UPDATE: OK, I found that adding the following in the route builder gives me what I want:
RedeliveryPolicyDefinition redeliveryPolicy = new RedeliveryPolicyDefinition();
// Set redelivery policy so it retries every 5 seconds for 10 minutes,
// then log an error when the retries have been exhausted
redeliveryPolicy.maximumRedeliveries(120)
.redeliveryDelay(5000)
.logExhausted(true)
.retriesExhaustedLogLevel(LoggingLevel.ERROR);
onException(MyRetryException.class)
.setRedeliveryPolicy(redeliveryPolicy);
So the question now is: Is this the correct way to achieve this?
Is it possible to rollback async processed message in ActiveMQ? I'm consuming next message while first one is still processing, so while I'm trying to rollback the first message on another (not activemq pool) thread, I'm getting above error. Eventually should I sednd message to DLQ manually?
Message error handling can work a couple ways:
Broker-side 'redelivery policy'. Where the client invokes a rollback n number (default is usually 6 retries) of times and the broker automatically moves the message to a Dead Letter Queue (DLQ)
Client-side. Application consumes the message and then produces to the DLQ.
Option #1 is good for unplanned/planned outages-- database down, etc. Where you want automatic retry. The re-delivery policy can also be configured when the client connects to the broker.
Option #2 is good for 'bad data' scenarios where you know the message will never be able to be processed. This is ideal, because you can move the message on the 1st consumption and not have to reject the message n number of times.
When you combine infinite retry with #1 and include #2 in your application flow, you can have a robust process flow of automatic retry, and move-bad-data-out-of-the-way-quickly. Best of breed =)
ActiveMQ Redelivery policy
We are using MQ JMS standalone client application (NO app server) to consume WebSphere MQ messages. Our Queue definition is as follows:
APP_QUEUE1 - (QA, PUT enabled)
APP_QUEUE1.CL - (QL and target of above APP_QUEUE1)
APP_QUEUE1_BOQ - (QA and BOQNAME of APP_QUEUE1.CL, PUT enabled)
APP_QUEUE1_BOQ.CL - (QL and target of above APP_QUEUE1_BOQ )
BOTHERH of APP_QUEUE1 = 3.
With above set up, when exception occurs for the first time, I am getting exception saying backout queue is not defined and attempt to add to dead letter queue also fails. Can someone explain why message is not getting recede to main queue (APP_QUEUE1) even though BOTHRESH is 3.
My understanding is, in case of exception, message will be recede to APP_QUEUE1 3 times and after that it will be routed to back out queue. If back out queue is full or fails then only message is added to dead letter queue.
Can someone please answer if there's anything wrong with queue definition ? Or something needs to be done in the application code ?