We are using Spring Boot in 2.4.2 with Spring WebFlux.
I want the Spring Boot application to terminate all requests to the application that take longer than say 3 seconds to process.
There is server.netty.connection-timeout, but that doesn't seem to do the trick.
Is there a way to specify such a server request timeout?
I was also facing the same issue i.e. even after configuring server.netty.connection-timeout request would get canceled. So, after some debugging found that timeout was getting set to '30000' by AsyncContext.
So, I configured the following property spring.mvc.async.request-timeout which change the timeout being set in AsyncContext and the request stopped getting canceled.
TL;DR:
Netty has no request timeout*. Add this WebFilter to set a request-timeout of 3 seconds using the reactor timeout on every request (here in kotlin, but in Java it works accordingly):
#Component
class RequestTimeoutWebFilter : WebFilter {
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
return chain
.filter(exchange)
.timeout(Duration.ofSeconds(3))
}
}
* at least I could not find any information in the netty docs.
Detailed answer
The connection-timeout does not refer to the duration that a request is allowed to take for processing, but it refers to the time it takes for establishing the connection.
First, I could not find any spring configuration option that allows setting the request timeout for netty. Then I went through the netty documentation to find out that there is no concept of request timeouts on the http server (only on the http client).
Wondering about why such important feature would not be supported, I remembered that we often cannot apply the same concepts as in blocking servers for good reasons. Next, I remembered, that in the reactive world we do not directly implement the handling of the request, but how the handling is assembled - i.e. we hold a Mono<Void> that will handle the request. Hence, we can just look at reactor and how to timeout a Mono, which is very easy:
Mono.create(...)
.timeout(Duration.ofSeconds(3))
Next, we just need to figure out, how to apply this to all requests. This is easy as well, because we can just use a WebFilter to intercept all requests to apply our timeout (here in kotlin, but in Java it works accoringly):
#Component
class RequestTimeoutWebFilter : WebFilter {
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
return chain
.filter(exchange)
.timeout(Duration.ofSeconds(3))
}
}
This effectively cancels a request within the set timeout with the following error:
2022-10-21 00:08:00.981 ERROR 6289 --- [ parallel-4] a.w.r.e.AbstractErrorWebExceptionHandler : [59dfa990-7] 500 Server Error for HTTP GET "/some/route"
java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 3000ms in 'source(MonoDefer)' (and no fallback has been configured)
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:295) ~[reactor-core-3.4.22.jar:3.4.22]
More tips and hints
To make the timeout configurable, we can use a custom config variable instead of the hard-coded duration.
To custimize the 500 status code we can either change the exception by providing a fallback to the timeout as 2nd argument and handle that exception in a controller advice - or we can just use reactors onErrorReturn.
The documentation for WebFilter actually states that they should be used to implement timeouts:
Contract for interception-style, chained processing of Web requests that may be used to implement cross-cutting, application-agnostic requirements such as security, timeouts, and others.
Still I think it is expected that spring provides such implementation out-of-the box that can be easily configured. Maybe we oversaw that it is there, but then I would argue it is too hard to find. ^^
Alternative solution path
As an alternative, we could use circuit breakers. However, those are implemented on the readers side and conceptually are used to protect the reading side against failure of the downstream - rather than protecting the internal processing within the downstream from running too long. They can only be applied to mimic a request timeout when applying them in a dedicated server (e.g. a spring cloud gateway server) that sits between the actual client and the actual service. When using Resilience4j as implementation, you can use TimeLimiter to achieve it.
Related
I am confused about how an infinite loop of feign calls might behave.
An example:
Assume I have 2 APIs, A & B.
if I call API A, which in turn calls API B via a feign HTTP call, which in turn calls API A again via feign, will it recognize this and break the call chain?
Quick flowchart of calls:
A -> B -> A -> B ... Repeat infinitely?
I have not tried this code, it is just an idea。
But I am assuming that spring-cloud-starter-feign will provide some methods to resolve this problem? Is this assumption correct?
#PostMapping(RestJsonPath.API_A)
ResponseEntity<byte[]> apiA();
#PostMapping(RestJsonPath.API_B)
ResponseEntity<byte[]> apiB();
Will it execute until it times out or hystrix will stop it？
TL;DR:
Feign will keep the connection open on the initial request from A to B until the pre-configured timeout kicks in. At this point, Feign will time out the request and if you have specified a Hystrix fallback, Spring will use your Hystrix fallback as the response.
Explanation:
spring-boot-starter-feign provides an abstraction layer for writing the HTTP request code. It will not handle potential loops or cycles in your code.
Here is an example spring boot feign client from their tutorials website for demonstration:
#FeignClient(value = "jplaceholder",
url = "https://jsonplaceholder.typicode.com/",
configuration = ClientConfiguration.class,
fallback = JSONPlaceHolderFallback.class)
public interface JSONPlaceHolderClient {
#RequestMapping(method = RequestMethod.GET, value = "/posts")
List<Post> getPosts();
#RequestMapping(method = RequestMethod.GET, value = "/posts/{postId}", produces = "application/json")
Post getPostById(#PathVariable("postId") Long postId);
}
Notice first that this is an interface - all the code is auto generated by Spring at startup time, and that code will make RESTful requests to the urls configured via the annotations. For instance, the 2nd request allows us to pass in a path variable, which Spring will ensure makes it on the URL path of the outbound request.
The important thing to stress here is that this interface is only responsible for the HTTP calls, not any potential loops. Logic using this interface (which I can inject to any other Spring Bean as I would any other Spring Bean), is up to you the developer.
Github repo where this example came from.
Spring Boot Docs on spring-boot-starter-openfeign.
Hope this helps you understand the purpose of the openfeign project, and helps you understand that it's up to you to deal with cycles and infinite loops in your application code.
As for Hystrix, that framework comes in to play (if it is enabled) only if one of these generated HTTP requests fails, whether it's a timeout, 4xx error, 5xx error, or a response deserialization error. You configure Hystrix, as a sensible default or fallback for when the HTTP request fails.
This is a decent tutorial on Hystrix.
Some points to call out is that a Hystrix fallback must implement your Feign client interface, and you must specify this class as your Hysterix fallback in the #FeignClient annotation. Spring and Hystrix will call your Hystrix class automatically if a Feign request fails.
I'm trying to invoke a AWS Lambda function asynchronous from the AWS API Gateway.
I have a long running (2-3min) Lambda function and I want to invoke this Lambda function asynchronous from a HTTP Post request. I configured the API Gateway as a Lambda Proxy Integration (because I want to pass the body unmodified to the function) This is working fine, but after 30s I get a 504 due the API Gateway execution time restriction.
But I can't manage to call the function async. According to the AWS docs it should be possible if I set the haeder "X-Amz-Invocation-Type", but this doesn't make any difference.
Does anybody know if it is possible to invoke a function async and using the proxy integration?
AWS says it's possible if you set the X-Amz-Invocation-Type header to Event, but I ran into the same necessity a few months ago and this did not work for me, so I am not sure this is still the case or if it was just me who misconfigured it. Maybe you are missing the same thing as me back then: I did not add an InvocationType header on the Integration Request as the docs suggests, so this very likely is the case for you, but still, I can't guarantee it works)
The documentation says:
Configure Lambda asynchronous invocation in the API Gateway console
In Integration Request, add an X-Amz-Invocation-Type header.
In Method Request, add an InvocationType header and map it to the
X-Amz-Invocation-Type header in the Integration Request with either a
static value of 'Event' or the header mapping expression of
method.request.header.InvocationType. For the latter, the client must
include the InvocationType:Event header when making a request to the
API method.
If this works, then you are good to go.
What I did back then, however, was to create an intermediate Lambda which literally acted as proxy to the actual Lambda.
There are a wide range of options to execute your function asynchronously, but you will need two Lambda functions regardless.
One option is to invoke another function (which will actually execute the task you want) asynchronously via the function invoked by API Gateway.
const params = {
FunctionName: 'YOUR_FUNCTIONS_NAME',
InvocationType: 'Event',
Payload: JSON.parse(event.body) // this is the event coming from API Gateway
};
await lambda.invoke(params).promise(); // await here is only going to wait for the HTTP request to be successful. Once the 2nd Lambda is invoked, it will return immediately
Another option is to put a message in SQS and configure a trigger for your Lambda to be invoked when there's a new message in the SQS queue. Same thing applies for a SNS notification.
Other options include Kinesis, DynamoDB Streams, etc. but the idea is the same: the function invoked via API Gateway must be nothing but a proxy to the other Lambda. How this proxy is going to work (be it sending a message to SQS, SNS, invoking the other function asynchronously directly, etc.) does not matter, what matters is the concept to get around API Gateway's 30 seconds request limit.
How to implement one-way operation in Web Services (using Java or Spring annotations)?
I have tried to add one way as given below
#WebService
public interface DanduServices {
#Oneway
public void saveDanduInformation(#WebParam(name = "serv") ServDTO Serv, #WebParam(name = "dandu") DanduDTO danduDto);
but it is still request-response not asynchronus or one way.
Could anyone suggest to make a operation one-way in service endpoint and let other operations behave as per request-response?
You need to think in terms of the protocol as well though. In HTTP when you send a request you wait for a response, if no response comes back after an amount of time then you will receive a time-out error. So when you talk about one-way (you should rather say async request maybe) you really need to specify exactly what you mean. Do you want to have confirmation that your message was received i.e. have the server respond back with an OK status code and go off and complete it's task but you not wait for the task to be completed? Then you would need to spawn another thread. Spring has AOP for this the same way it has for transactions with #Transactional. Instead you annotated your method with #Async and return a Future<Something>. You'll also need #EnableAsync in your config. Refer to this article for an example Hot To Do #Async
If you don't even care about if the server received your request you don't want to use TCP/HTTP but instead UDP which is used in VOIP (phone over internet) for instance and is quicker, but it will depend on your client.
UPDATE: I upgraded the code to Java 8 without too much of a hassle. So I would like answers tied to Spring 4/Java 8.
I am working on a task to fix performance issues (Tomcat max thread count of 200 reached at a request rate of just 400/s, request latencies building up periodically, etc) in a Tomcat/Spring 4.2.4/Java 8 web mvc application.
It is a typical web application which looks up Mysql via Hibernate for small but frequent things like user info per request, then does actual data POST/GET to another web service via RestTemplate.
The code is in Java 7 style as I just migrated to Java 8, but no new code has been written in that style yet. (I am also back using Spring after ages, so not sure what would be best).
As expected in a normal such application, the Service layer calls other services for info, and then also passes that along to a call to the DAO. So I have some dependent callbacks here.
Setup
#EnableAsync is set
The flow of our Http requests goes from Controller -> Service -> DAO -> REST or Hibernate
Sample flow
Say Controller receives POST request R and expects a DeferredResult
Controller calls entityXService.save()
EntityXService calls userService.findUser(id)
UserService calls UserDAO.findUser(id) which in turn talks to Hibernate
UserService returns a Spring ListenableFuture to the caller
EntityXService awaits the user info (using callback) in and calls EntityXDAO.save(user, R)
EntityXDAO calls AsyncRestTemplate.postForEntity(user, R)
EntityXDAO receives DeferredResult> which is our data abstraction for the response.
EntityXDAO processes the response and converts to EntityXDTO
Eventually somehow the DeferredResult is sent back through the same chain as a response.
I am getting lost at how in step 3, EntityXService asynchronously calls UserService.findUser(id) with an onSuccess callback to EntityXDAO.save(user, R). However, EntityXDAO.save(user, R) also now returns a DeferredResult from the AsyncRestTemplate.
Questions:
Is using DeferredResult a good way to get concurrency going in this application?
Is using Guava's ListenableFuture or Java 8 CompletableFuture going to help make it better in anyway, rather than using DeferredResult?
My BIGGEST question and confusion is how to arrange the DeferredResult from one service lookup to be used by another, and then finally set a DeferredResult of a completely different return type for the final response?
Is there an example of how to chain such callbacks and what is the recommended way to build such a flow? If this sounds completely wrong, is Java 7 going to be the right choice for this?
Thanks in advance!
Have spring integration application with inbound http gateway and outbound http gateway, between those i want to have cache, to avoid unnessesary requests. The only solution i have is to add interceptor with cache and router after it that routes cahced results back to reply channel, and non cached to outbound, but this solution seems tricky and ugly to me.
Interceptor with cache also works good when inbound gateway has same channel for request and reply, (when returns new message with same headers but different payload, its considered as reply) but its not the case I can use.
Any better ideas for this?
More elegant solution can be achieved with <request-handler-advice-chain>
and Spring Cache Advice.
So, your solution may be like this:
<int-http:outbound-gateway>
<int-http:request-handler-advice-chain>
<cache:advice>
<cache:caching cache="foo">
<cache:cacheable method="handle*Message" key="#a0.payload"/>
</cache:caching>
</cache:advice>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
Where handle*Message is handleRequestMessage method of HttpRequestExecutingMessageHandler. And exactly for this method Spring Integration applies his Advices (e.g. RequestHandlerRetryAdvice).
Here you should configure a cacheManager bean, choose cache name and determine a key for cache entry. In sample above #a0 is a Message object from handleRequestMessage arguments. So, you can specify any SpEL expression against message properties (payload and headers).
And the result of handleRequestMessage will be stored in the cache.
And when you provide the same parameters for HTTP reqeust, the result will be returned just from cache.