I'm not adding a retry against EJB exception in listener but somehow I m thinking that retry is happening. How can I be sure if retry is happening or not. Note: I'm using spring.retry.RetryTemplate
Related
We are trying to solve an edge case scenario, in spring boot kafka listener:
#KafkaListener(topics="topicName", groupId="consumerGoupId")
public void consumeNotification(#Payload Datarequest request,
#Header(KafkaHeaders.OFFSET) Long offset,
#Header(KafkaHeaders.RECEIVED_PARTITION_ID)Integer partitionId)
{
processRequest(request);
}
Now for some reason when processRequest is going on, my spring boot app gets Shutdown,the offset doesn't get committed and when it is up,the listener picks up the same message for processing again.
My use case is : Is there any way to know Listener has got this message as part of Retry?
Because since process request was stopped in between due to app Crash, if the same message has picked up by listener again, I have to write some logic in processRequest to start processing from where it stopped.
Right; the only guarantee is at least once delivery; there is no way to know if a message has been redelivered (after a restart - we can tell if it's a redelivery in the same application instance - see https://docs.spring.io/spring-kafka/docs/current/reference/html/#delivery-header).
Since you can't tell in all cases, you have to make your listener idempotent - e.g. by storing the topic/partition/offset along with the data, so you can check if it has already been processed.
In one of the steps of my Spring Batch job, I'm trying to configure it so that when ObjectOptimisticLockingFailureException happens, the step can be retried and hopefully the retry will work.
#Bean
public Step myStep(StaxEventItemReader<Response> staxEventResponseReader,
ItemWriter<Response> itemWriter,
ItemProcessor<? super Response, ? extends Response> responseProcessor) {
return stepBuilderFactory.get("myStep")
.<Response, Response>chunk(1)
.reader(staxEventResponseReader)
.processor(responseProcessor)
.writer(itemWriter)
//.faultTolerant().retryLimit(3).retry(Exception.class)
.build();
}
The logic of writer for the step is pretty simple: it tries to read a row from the database, and once it finds the row, it updates it. I was able to reproduce the ObjectOptimisticLockingFailureException by setting a breakpoint right after the find method, manually bump the version column for the row in database and commit it, then resume.
However, after uncommenting the retry definition in my step, no retries were attempted. After some debugging, it seems that the Spring retry logic is inside the chunk's transaction; but since the ObjectOptimisticLockingFailureException is not thrown by my code in the writer, but by Spring's chunk transaction committing logic, no retries were attempted at all.
Chunk Transaction Begin
Begin Retry loop in FaultTolerantChunkProcessor.write()
Writer logic in my Step
End Retry loop
Chunk Transaction Commit - Throws ObjectOptimisticLockingFailureException
When I tried to explicitly throw ObjectOptimisticLockingFailureException in my writer, the retry logic worked perfectly as expected. My questions are:
How to make the retry logic work if the exception is not thrown from my writer code in the step, but by the time the chunk transaction is committed by Spring Batch?
Another weird behavior is, when I manually cause the ObjectOptimisticLockingFailureException by bumping the version column in database, with the retry definition commented in the step, the final status of the step is FAILED which is expected. But with the retry definition uncommented, the final status of the step is COMPLETE. Why is that?
How to make the retry logic work if the exception is not thrown from my writer code in the step, but by the time the chunk transaction is committed by Spring Batch?
There is an open issue for that here: https://github.com/spring-projects/spring-batch/issues/1826. The workaround is to (try to anticipate and) throw any exception that might happen at the commit time in the writer. This is what you tried already and confirmed that works when you say When I tried to explicitly throw ObjectOptimisticLockingFailureException in my writer, the retry logic worked perfectly as expected.
Another weird behavior is, when I manually cause the ObjectOptimisticLockingFailureException by bumping the version column in database, with the retry definition commented in the step, the final status of the step is FAILED which is expected. But with the retry definition uncommented, the final status of the step is COMPLETE. Why is that?
This is related to the previous issue, but caused by a different one: https://github.com/spring-projects/spring-batch/issues/1189. That said, it is ok to play with the version field during a debugging session to understand how things work, but I would not recommend changing the version column in your code. Spring Batch relies on this column heavily in its optimistic locking strategy, and it is not expected to change values of this column in user code, otherwise unexpected behaviour might happen.
so I have a Saga and the Saga sends a command to a different microservice on a specific event. I wanted to configure the commandGateway with a RetryScheduler, so that it retries to send the command in case that the other microservice is down. The RetryScheduler will only perform retries if the exception is a RuntimeException, which the NoHandlerForCommandException that is thrown when the other service if offline definately is.
If i dont set the maxRetryCount then the error message is o.a.c.gateway.IntervalRetryScheduler : Processing of Command [XXXCommand] resulted in an exception 1 times. Giving up permanently
If I do set the attribute the error message is o.a.c.gateway.IntervalRetryScheduler : Processing of Command [XXXCommand] resulted in an exception and will not be retried
If the other microservice is running, then the command is handled correctly, no problems.
Does anybody have an idea what could be the issue?
This is my configuration for the commandGateway with a RetryScheduler:
#Bean
public CommandGateway commandGateway(){
Configurer configurer = DefaultConfigurer.defaultConfiguration();
CommandBus commandBus = configurer.buildConfiguration().commandBus();
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
RetryScheduler rs = IntervalRetryScheduler.builder().retryExecutor(scheduledExecutorService).maxRetryCount(100).retryInterval(1000).build();
CommandGateway commandGateway = DefaultCommandGateway.builder().commandBus(commandBus).retryScheduler(rs).build();
return commandGateway;
}
To resolve the problem at hand, you could provide your own implementation of the IntervalRetryScheduler, which overrides the IntervalRetryScheduler#isExplicitlyNonTransient(Throwable) method to also take into account retrying the NoHandlerForCommandException.
Note though that the IntervalRetryScheduler is intentionally only retrying on exceptions which are of type AxonNonTransientException, is those typically signal recoverable exceptions. The NoHandlerForCommandException means that the CommandBus implementation being used has no clue whom to give the command too, something which in general suggests an thing which cant be retried.
It however seems you have a scenario where it does make sense. Thus, like I point out at the start, overriding the isExplicitlyNonTransient(Throwable) method to exclude NoHandlerForCommandException would be the way to go for you I think.
Hope this helps you out!
So I currently have a spring batch process that has a composite skip policy implemented for a few custom exception types. So the issue that I am now running into is the fact that I don't always just want to skip when I get an exception.
For some database related exceptions I would like to retry a few times and then if it still fails move on and skip the record. Unfortunately I don't see a way to do this.
I tried implementing my own RetryPolicy but the only option for canRetry is true or false (rather than false I would like to throw my skippable exception).
So am I missing something here or is this not really functionality that spring batch has?
Thanks
From a StepBuilderFactory, you can do that:
stepBuilder.reader(reader).writer(writer).faultTolerant().retryPolicy(retryPolicy).skipPolicy(skipPolicy).
And yes it is working. I had the same issue and after test, I see that my items are retried following my RetryPolicy and then they are skipped following my SkipPolicy.
I have a question about handling org.apache.shiro.cache.CacheException.
What is shiro's responsibility when a cache exception occurs? This RTE ripples through the whole stack, but I'm unclear where it is handled and in what manner.
Can a shiro enabled web application recover?
Is the cache manager reinitialised?
Do I have to restart the application when a cache exception occurs?
For example, a timeout exception or a failed to connect to node exception occurs on cache.get(key). What's the expected outcome apart from bubble to the stop and die?
Thanks
I have checked all usages of CacheException and the only thing shiro does is throw it. It never gets catched let alone handled.
So the only expected outcome is to "bubble to the stop and die".
What you can do about it is have a servlet filter in front of shiro that catches this exception, then get a hold of the CacheManager instance (how you do that depends on your setup) and call cacheManager.getCache().clear() so the cache is reset.