In apache httpclient 4.3, DefaultHttpRequestRetryHandler's code
if (exception instanceof InterruptedIOException) {
// Timeout
return false;
}
It won't retry if it's timeout. What's the reason? Sometimes, the network is not stable, I just want to retry connection. I can use my own RetryHandler, but I just want to make sure if there is any problem if I retry when timeout.
It won't retry if it's timeout. What's the reason?
Why should it? Timeouts usually defines a maximum period of inactivity between two consecutive operations. Why should the request be retried if it times out in the first place? If you are willing to wait longer for the operation to complete you should be using a greater timeout value.
This helped me. I tried to disable the retry option. The code below does the opposite.
DefaultHttpClient httpClient = new DefaultHttpClient();
DefaultHttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(0, true);
httpClient.setHttpRequestRetryHandler(retryHandler);
Thanks
I have used commercially a custom RetryHandler which mimics the Default* one, but allows retry for the following exceptions which we were getting regularly: ConnectTimeoutException and HttpHostConnectException. These exceptions were thrown a lot after a 15s timeout. The connection should be made sub-second so we now retry up to 3 times with 5s timeout, this has seen a large increase in successful connections being made on the second attempt.
We are still looking into why these connection requests aren’t being made in a timely manner between our azure app service and on-prem services.
Related
I get lots of events to process in RabbitMq and then those get forward to service 1 to process and after some processing the data, there is an internal call to a micro service2. However, I do get java.net.SocketTimeoutException: timeout frequently when I call service2, so I tried to increase timeout limit from 2s to 10 sec as a first trial and it did minimise the timeout exceptions but still lot of them are still there,
second change I made is removal of deprecated retry method of spring and replace the same with retryWhen method with back off and jitter factor introduced as shown below
.retryWhen(Retry.backoff(ServiceUtils.NUM_RETRIES, Duration.ofSeconds(2)).jitter(0.50)
.onRetryExhaustedThrow((retryBackoffSpec, retrySignal) -> {
throw new ServiceException(
ErrorBo.builder()
.message("Service failed to process after max retries")
.build());
}))
.onErrorResume(error -> {
// return and print the error only if all the retries have been exhausted
log.error(error.getMessage() + ". Error occurred while generating pdf");
return Mono.error(ServiceUtils
.returnServiceException(ServiceErrorCodes.SERVICE_FAILURE,
String.format("Service failed to process after max retries, failed to generate PDF)));
})
);
So my questions are,
I do get success for few service call and for some failure, does it mean some where there is still bottle neck for processing the request may be at server side that is does not process all the request.
Do I need to still increase timeout limit if possible
How do I make sure that there is no java.net.SocketTimeoutException: timeout
This issue has started coming recently. and it seems there is no change in ports or any connection level changes.
But still what all things I should check in order to make sure the connection level setting are correct. Could someone please guide on this.
Thanks in advance.
I implemented retry mechanism offered by resilience4j in my machine for a project that makes http calls asynchronously. I can see that the http calls are being retried properly. However, these calls are asynchronous and so multiple HTTP calls will be made and it is possible that a number of these calls will fall into a retry at the same time. However, at a time I am able to see that only 7-9 retries are attempted. My question is why is there a cap on this ? Is it possible to configure this ?
Lets say i have a method as (this is a pseudocode).
#Async
#Retry(name = "retryA",fallbackMethod = "fallbackRetry")
public ResponseObj getExternalHttpRepsonse(String payload){
ClientResponse resp = webUtils.postRequest(payload);
boolean validatePredicate = responsePredciate.test(resp);
if(!validatePredicate){
throw new PredicateValidationFailedException();
}
return new ResponseObj(resp);
}
I am seeing an output of 7-9 attempts 1s failed, attempt 2s failed in the logs continuously whenever failures occur. Why is this capped between 7-9 and not more than that ?
I have one api that retrives a huge data , I set the timeout to 7 min no more than that.
So when the waiting time exceeds 7 min I want the operation to be cancelled
However , the users sometimes wait till 10 min and more ..
Below is the code
final OkHttpClient okHttpClient = new OkHttpClient.Builder()
.connectTimeout(connectTimeOut, TimeUnit.SECONDS)
.readTimeout(readTimeOut, TimeUnit.SECONDS)
.build();
What I'm missing here?
Connection timeOut is the time to establish a connection and read/writeTimeout is the time required to read or write after establishing connection. so finally
totalTimeout = connectTimeOut + readTimeout
Your httpClient taking extra time in connectionEstablishment or readTimOut. You have to configure this two timeout in a way that user need not to wait more than your defined time. Please accept my answer if is satisfactory
Note: All the operation takes less or equal time you defined to
complete.
For your better understanding visit this link
Http Client Time Out Guide
WebClientTestService service = new WebClientTestService() ;
int connectionTimeOutInMs = 5000;
Map<String,Object> context=((BindingProvider)service).getRequestContext();
context.put("com.sun.xml.internal.ws.connect.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.internal.ws.request.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.ws.request.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.ws.connect.timeout", connectionTimeOutInMs);
Please share the differences mainly in connect timeout and request timeout.
I need to know the recommended values for these parameter values.
What are the criteria for setting timeout value ?
Please share the differences mainly in connect timeout and request timeout.
I need to know the recommended values for these parameter values.
Connect timeout (10s-30s): How long to wait to make an initial connection e.g. if service is currently unavailable.
Socket timeout (10s-20s): How long to wait if the service stops responding after data is sent.
Request timeout (30s-300s): How long to wait for the entire request to complete.
What are the criteria for setting timeout value ?
It depends a web user will get impatient if nothing has happened after 1-2 minutes, however a back end request could be allowed to run longer.
Also consider server resources are not released until request completes (or times out) - so if you have too many requests and long timeouts your server could run out of resources and be unable to service further requests.
request timeout should be set to a value greater then the expected time for the request to complete, perhaps with some room to allow occasionally slower performance under heavy loads.
connect/socket timeouts are often set lower as normally indicate a server problem where waiting another 10-15s usually won't resolve.
I'm experimenting with OKHttp (version 2.0.0-RC2) and SPDY and seeing IOException: stream was reset: CANCEL quite a lot, maybe 10% or more of all requests in some preliminary testing. When using Apache HttpClient and regular https we were not seeing any equivalent issue as far as I'm aware. I'm pretty sure we also don't see anything equivalent with OkHttp when SPDY is disabled (client.setProtocols(ImmutableList.of(Protocol.HTTP_1_1))) but I haven't done enough testing to be 100% confident.
This previous question sees these exceptions among others and the advice there is to ignore them, but this seems crazy: we get an exception while reading data from the server, so we abort the data processing code (which using Jackson). We need to do something in such cases. We could retry the request, of course, but sometimes it's a POST request which is not retry-able, and if we've already started receiving data from the server then it's a good bet that the server as already taken the requested action.
Ideally there is some configuration of the client and/or the server that we can do in order to reduce the incidence of these exceptions, but I don't understand SPDY well enough to know even where to start looking or to advise our server-admin team to start looking.
Stack trace, in case it's helpful:
java.io.IOException: stream was reset: CANCEL
at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.checkNotClosed(SpdyStream.java:442)
at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.read(SpdyStream.java:344)
at com.squareup.okhttp.internal.http.SpdyTransport$SpdySource.read(SpdyTransport.java:273)
at okio.RealBufferedSource.exhausted(RealBufferedSource.java:60)
at okio.InflaterSource.refill(InflaterSource.java:96)
at okio.InflaterSource.read(InflaterSource.java:62)
at okio.GzipSource.read(GzipSource.java:80)
at okio.RealBufferedSource$1.read(RealBufferedSource.java:227)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.loadMore(UTF8StreamJsonParser.java:174)
at com.fasterxml.jackson.core.base.ParserBase.loadMoreGuaranteed(ParserBase.java:431)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2111)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2092)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:275)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:205)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:230)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:202)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2765)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:1546)
at com.fasterxml.jackson.core.JsonParser.readValueAsTree(JsonParser.java:1363)
at (application-level code...)
Your best bet is to set a breakpoint in the two places where the CANCEL error code is assigned: that's SpdyStream#closeInternal (line 246) and SpdyStream#receiveRstStream (line 304). If you can put a breakpoint here, you can capture who is canceling your stream and that'll shed light on the problem.
If for whatever reason you cannot attach a debugger, you can instrument the code to print a stacktrace when those lines are reached:
new Exception("SETTING ERROR CODE TO " + errorCode).printStackTrace();
In either case, I'm the author of that code and I'd love to help you resolve this problem.
Had the same problem and this was a result of network connection timeout, this was a result of downloading a large file from the web service
i had my timeout set to 2-min so i changed it to 5-min and it solved my problem
val okkHttpclient = OkHttpClient.Builder()
.connectTimeout(5, TimeUnit.MINUTES)
.writeTimeout(5, TimeUnit.MINUTES) // write timeout
.readTimeout(5, TimeUnit.MINUTES) // read timeout
.addInterceptor(networkConnectionInterceptor)
.build()
We had this issue because of broken http headers. The android Base64 encoder by default adds newlines which broke our Authorization headers.