I’ve a SQS queue which has max receives value of 3 & default visibility timeout of 30 seconds.
Currently I’m listening to its messages using annotation #SqsListener, which works fine.
Now I want to implement exponential backoff for retries on this queue.
The only pointer I got in this direction in AWS documentation is to use ClientConfiguration.
But I'm not able to find any exmample on how to use it.
I thought SimpleMessageListenerContainer might provide a setter to apply backoff strategy, but it just provides to add a timer.
If there any examples to guide how to add backoff in spring in SQS listener, that will be great.
Exponential backoff can be achieved in a Spring SQS listener by using a custom error handler in combination with the SimpleMessageListenerContainer.
#EnableScheduling
#Configuration
public class ExponentialBackoffSqsListener {
private QueueMessagingTemplate queueMessagingTemplate;
private String queueUrl;
#Autowired
public ExponentialBackoffSqsListener(AmazonSQSAsync amazonSqs, String queueUrl) {
this.queueMessagingTemplate = new QueueMessagingTemplate(amazonSqs);
this.queueUrl = queueUrl;
}
#SqsListener(value = "${queue.name}")
public void receiveMessage(String message) {
// Your business logic goes here
}
#Bean
public SimpleMessageListenerContainer messageListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setAmazonSqs(this.queueMessagingTemplate.getAmazonSqs());
listenerContainer.setErrorHandler(t -> {
if (RetryUtils.isRetryableServiceException(t)) {
RetryPolicy retryPolicy = RetryUtils.getDefaultRetryPolicy();
int backoffTime = retryPolicy.getBackoffStrategy().computeBackoffInMilliseconds(retryPolicy.getRetryCondition().getRetryCount());
// Schedule a retry for the failed message after the backoff time
scheduleRetry(backoffTime, message);
}
});
listenerContainer.setQueueUrl(this.queueUrl);
return listenerContainer;
}
private void scheduleRetry(int backoffTime, String message) {
// Schedule a retry using the #Scheduled annotation
new ScheduledThreadPoolExecutor(1).schedule(() -> {
this.queueMessagingTemplate.convertAndSend(this.queueUrl, message);
}, backoffTime, TimeUnit.MILLISECONDS);
}
}
Basically the title. I am using a JDBC item reader and JDBC item writer and I'm changing a particular status through the processor by using an API, if the API fails to change the status, I want to use exponential backoff to retry this at a later instant. I'm not able to figure out how to implement this
You have two options:
1. Handle the retry operation manually in your item processor
There are basically two ways to do that, either programmatically or in a declarative way.
1.1 Programmatic approach
You first define your retry template with a backoff policy as needed:
#Bean
public RetryTemplate retryTemplate() {
// configure backoff policy
ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
exponentialBackOffPolicy.setInitialInterval(1000);
exponentialBackOffPolicy.setMultiplier(2.0);
exponentialBackOffPolicy.setMaxInterval(10000);
// configure retry policy
SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(5);
// configure retry template
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setBackOffPolicy(exponentialBackOffPolicy);
retryTemplate.setRetryPolicy(simpleRetryPolicy);
return retryTemplate;
}
Then use that retry template in your item processor:
import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.support.RetryTemplate;
public class MyRetryableItemProcessor implements ItemProcessor {
RetryTemplate retryTemplate;
public MyRetryableItemProcessor(RetryTemplate retryTemplate) {
this.retryTemplate = retryTemplate;
}
#Override
public Object process(Object item) throws Exception {
return retryTemplate.execute(new RetryCallback<Object, Exception>() {
#Override
public Object doWithRetry(RetryContext retryContext) throws Exception {
// API call
return item;
}
});
}
}
1.2 Declarative approach using annotations
Here is an example:
import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.annotation.Backoff;
import org.springframework.retry.annotation.Retryable;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.stereotype.Component;
#Component
public class MyAnnotationBasedRetryableItemProcessor implements ItemProcessor {
#Override
#Retryable(backoff = #Backoff(delay = 1000L, maxDelay = 10000, multiplier = 2.0))
public Object process(Object item) throws Exception {
// Do API call
return item;
}
}
2. Let Spring Batch handle the retry for you by using a fault-tolerant step
In this case, you can set a custom RetryPolicy in your fault-tolerant step:
#Bean
public Step step(StepBuilderFactory stepBuilderFactory) {
// configure backoff policy
ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
exponentialBackOffPolicy.setInitialInterval(1000);
exponentialBackOffPolicy.setMultiplier(2.0);
exponentialBackOffPolicy.setMaxInterval(10000);
// configure retry policy
SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(5);
return stepBuilderFactory.get("myStep")
.<Integer, Integer>chunk(5)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.faultTolerant()
.retryPolicy(simpleRetryPolicy)
.build();
}
Note that in this case, whenever your processor throws an exception for an item, the entire chunk is retried item by item (and each item will be re-processed in its own transaction).
The examples above use spring-retry since you mentioned you have a preference for that. But the same ideas can be applied with any other fault-tolerance library.
I am new to spring-retry. Basically, for retrying calls to REST APIs, I have integrated spring-retry into my spring-boot application. To do this, I have made following changes:
Added spring-retry to pom.xml.
Added following configuration:
#Configuration
#EnableRetry
public class RetryConfiguration {
}
Finally added #Retryable annotation to the class (this class is not a Spring Bean) method that I would like to be retried for various exceptions as follows:
public class OAuth1RestClient extends OAuthRestClient {
#Override
#Retryable(maxAttempts = 3, value = {
Exception.class},
backoff = #Backoff(delay = 100, multiplier = 3))
public Response executeRequest(OAuthRequest request)
throws InterruptedException, ExecutionException, IOException {
System.out.println("Inside Oauth1 client");
return myService.execute(request);
}
Now, the executeRequest method is not retrying. I am not able to understand if I am missing anything here.
Could anyone please help? Thanks.
If your class is not Spring managed (e.g. #Component/#Bean) the
annotation processor for #Retryable won't pick it up.
You can always manually define a retryTemplate and wrap calls with it:
RetryTemplate.builder()
.maxAttempts(2)
.exponentialBackoff(100, 10, 1000)
.retryOn(RestClientException.class)
.traversingCauses()
.build();
and then
retryTemplate.execute(context -> myService.execute(request));
If you want to retry on multiple exception, this can happen via custom RetryPolicy
Map<Class(? extends Throwable), Boolean> exceptionsMap = new HashMap<>();
exceptionsMap.put(InternalServerError.class, true);
exceptionsMap.put(RestClientException.class, true);
SimpleRetryPolicy policy = new SimpleRetryPolicy(5, exceptionsMap, true);
RetryTemplate.builder()
.customPolicy(policy)
.exponentialBackoff(100, 10, 1000)
.build();
FYI: RetryTemplate is blocking and you might want to explore a non-blocking async retry approach like async-retry. - and the retryOn() supports a list of exceptions.
I want to implement the circuit breaker design pattern using Spring-Retry. The main problem that I am facing is opening the circuit for all requests. It still keeps retrying if I make a new request from the browser.
I have a RetryTemplate with a CircuitBreakerRetryPolicy defined as follows:
#Configuration
public class MyApplicationConfig {
#Bean
public RetryTemplate retryTemplate(RetryPolicy cbRetry) {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(1000);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(circuitBreakerRetryPolicy);
return retryTemplate;
}
#Bean("cbRetry")
public RetryPolicy circuitBreaker() {
CircuitBreakerRetryPolicy circuitBreakerRetryPolicy = new CircuitBreakerRetryPolicy();
circuitBreakerRetryPolicy.setResetTimeout(20000);
circuitBreakerRetryPolicy.setOpenTimeout(2000);
return circuitBreakerRetryPolicy;
}
}
Then I use RestTemplate to make a call to a non-existing url, just to make the call fail and I count the retries. The circuit is supposed to open in 2 seconds, but if I try another call right after the first one fails, it keeps retrying again.
How do I make the circuit stay open for a given openTimeOut period for all incoming requests?
I experimented with RetryState by providing it to the retryTemplate call, but that doesn't help either.
Thank you!
spring-retry state is thread-bound.
You would need to write a custom policy to maintain global state.
The following code is not retrying. What am I missing?
#EnableRetry
#SpringBootApplication
public class App implements CommandLineRunner
{
.........
.........
#Retryable()
ResponseEntity<String> authenticate(RestTemplate restTemplate, HttpEntity<MultiValueMap<String, String>> entity) throws Exception
{
System.out.println("try!");
throw new Exception();
//return restTemplate.exchange(auth_endpoint, HttpMethod.POST, entity, String.class);
}
I have added the following to the pom.xml.
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>1.1.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
I also tried providing different combinations of arguments to #Retryable.
#Retryable(maxAttempts=10,value=Exception.class,backoff=#Backoff(delay = 2000,multiplier=2))
Thanks.
In spring boot 2.0.2 Release, I have observed that the #Retryable is not working if you have retryable and called method in same class. On debugging found that the pointcut is not getting built properly. For now, the workaround for this problem is that we need to write the method in a different class and call it.
Working Example could be found here.
For the #Retryable annotation on the method to be discovered it needs to be called correctly from an initialised context. Is the method invoked from a bean from the spring context or called by other means?
If testing this is your runner using the SpringJunit4ClassRunner?
Spring's #Retryable, #Cacheable, #Transaction, etc. are ALL implemented using Aspect Oriented Programming. Spring implements AOP via proxy-based weaving. Proxies intercept calls from one bean to another. Proxies cannot intercept calls from one object's methods to another. This is a general limitation of proxy based weaving.
The following solutions address this limitation: 1) as mentioned above, use #Autowired (or #Resource) to inject a bean with a self reference; calls to this reference transit the proxy. 2) Use AspectJ's ClassLoader instead of Spring's default proxy-based weaving. 3) As mentioned above, place the methods on separate beans. I've done each in various situations, each has pros and cons.
I solved it. I figured out that if return something from the method that you trying to retry, then #Retryable() is not working.
maven dependency in pom.xml
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>1.1.5.RELEASE</version>
</dependency>
Spring boot Application.java
#SpringBootApplication
#EnableTransactionManagement
#EnableRetry
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
}
in controller.java
#RestController
public class JavaAllDataTypeController {
#Autowired
JavaAllDataTypeService JavaAllDataTypeService;
#RequestMapping(
value = "/springReTryTest",
method = RequestMethod.GET
)
public ResponseEntity<String> springReTryTest() {
System.out.println("springReTryTest controller");
try {
JavaAllDataTypeService.springReTryTest();
} catch (Exception e) {
e.printStackTrace();
}
return new ResponseEntity<String>("abcd", HttpStatus.OK);
}
}
in service.java
#Service
#Transactional
public class JavaAllDataTypeService {
// try the method 9 times with 2 seconds delay.
#Retryable(maxAttempts=9,value=Exception.class,backoff=#Backoff(delay = 2000))
public void springReTryTest() throws Exception {
System.out.println("try!");
throw new Exception();
}
}
output: It' trying 9 times then throwing exception.
I had exactly the same issue as described in the original question.
In my case it turned out that the spring-boot-starter-aop dependency was accidentally not included. After adding it to my pom.xml, my #Retryable methods worked as expected.
Returning values from #Retryable methods works fine for me.
It work for return type as well
#Service
public class RetryService {
private int count = 0;
// try the method 9 times with 2 seconds delay.
#Retryable(maxAttempts = 9, value = Exception.class, backoff = #Backoff(delay = 2000))
public String springReTryTest() throws Exception {
count++;
System.out.println("try!");
if (count < 4)
throw new Exception();
else
return "bla";
}
}
For those who want to call #Retryable block in same class can to this way.
The key here is not to call the method directly and through self-injected bean
#Slf4j
#Service
public class RetryService {
#Resource(name = "retryService")
private RetryService self;
public String getValue(String appender) {
return self.getData(appender);
}
#Retryable(value = NumberFormatException.class, maxAttempts = 4, backoff = #Backoff(500))
public String getData(String appender) {
log.info("Calling getData");
Integer value = Integer.parseInt(appender);
value++;
return value.toString();
}
#Recover
public String recoverData(String appender) {
log.info("Calling recoverData");
return "DEFAULT";
}
}
Can read more about using Retry in detail here
An alternative could be RetryTemplate
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(2000l);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(2);
retryTemplate.setRetryPolicy(retryPolicy);
return retryTemplate;
}
and
retryTemplate.execute(new RetryCallback<Void, RuntimeException>() {
#Override
public Void doWithRetry(RetryContext arg0) {
myService.templateRetryService();
...
}
});
worked out for me
source
Pretty old thread, but I wanted to share that after changing my method visibility from private to public, Retryable was successfully retrying.
This is in addition to using the self resource mentioned above.
Even I faced the same issue, Later after some investigation and research came to know that along with #Retryable annotation above the method we also need to provide #EnableRetry above the class. This #EnableRetry annotation either can be provided above same class in to which you have provided method you want to retry or above your main spring boot application class. For example like this:
#RequiredArgsConstructor
**#EnableRetry**
#Service
public class SomeService {
**#Retryable(value = { HttpServerErrorException.class, BadRequestException.class},
maxAttempts = maxRetry, backoff = #Backoff(random = true, delay = 1000,
maxDelay = 8000, multiplier = 2))**
public <T> T get( ) throws HttpServerErrorException, BadRequestException {
//write code here which you want to retry
}
}
I hope this will help and resolve your issue.
I got this one solved by moving #Retryable directly in front of the method I wanted to retry.
From this:
public class MyClass {
public String toBeRetried() {
return delegateTo();
}
#Retryable
public String delegateTo() {
throw new Exception();
}
}
To this:
public class MyClass {
#Retryable
public String toBeRetried() {
throw new Exception();
}
}