We Keep Coding

How can I configure (long) timeout for Kafka Admin operations?

I fancy myself making a single request creating 15k topics in a busy Kafka cluster, in a single request, something like this:
final Admin admin = ...;
final List<NewTopic> newTopics = IntStream.range(0, 15000)
.mapToObj(x -> "adam-" + x)
.map(x -> new NewTopic(x, Optional.empty(), Optional.empty()))
.collect(toList());
final CreateTopicsResult ctr = admin.createTopics(newTopics);
ctr.all().get(); // Throws exceptions.
Unfortunately this starts throwing exceptions due to embedded timeouts - how can I properly make the request while keeping it simple without batching?
For the sake of argument let's stick to Kafka 3.2 (both client & server).

This can be configured in one of two ways:
A) Operation timeout allowed the underlying NetworkClient to wait indefinitely until the response is received. So we can change our code to admin.createTopics(newTopics, new CreateTopicOptions().timeout(Integer.MAX_VALUE)).
B) Alternatively, we could configure Admin's default.api.timeout.ms property and need not to provide the explicit timeout - which one is preferred depends on the codebase/team standards.
What was not necessary:
request.timeout.ms - it appears to apply to only a single request, however not setting it to large value (larger than expected duration) results in somewhat strange behaviour when original request appears to be failed on finish, and then repeated (to be processed immediately in case of topic-creation, as they'd have been created by initial request (this can be quite easily replicated by setting request.timeout.ms to a low value)):
Cancelled in-flight CREATE_TOPICS request with correlation id 3 due to node 2 being disconnected (elapsed time since creation: 120443ms, elapsed time since send: 120443ms, request timeout: 29992ms)
connections.max.idle.ms - the connection remains active the whole time the NC is waiting for the upstream's response.
metadata.max.age.ms - metadata fetching is needed only for the initial step (figuring out where to send the request), but later we just wait for the response from the known upstream broker.

Timeout exception is not fired in retrofit android

I have one api that retrives a huge data , I set the timeout to 7 min no more than that.
So when the waiting time exceeds 7 min I want the operation to be cancelled
However , the users sometimes wait till 10 min and more ..
Below is the code
final OkHttpClient okHttpClient = new OkHttpClient.Builder()
.connectTimeout(connectTimeOut, TimeUnit.SECONDS)
.readTimeout(readTimeOut, TimeUnit.SECONDS)
.build();
What I'm missing here?

Connection timeOut is the time to establish a connection and read/writeTimeout is the time required to read or write after establishing connection. so finally
totalTimeout = connectTimeOut + readTimeout
Your httpClient taking extra time in connectionEstablishment or readTimOut. You have to configure this two timeout in a way that user need not to wait more than your defined time. Please accept my answer if is satisfactory
Note: All the operation takes less or equal time you defined to
complete.
For your better understanding visit this link
Http Client Time Out Guide

Duration to return the request for a endpoint call

I need to be clear about something. If a microservice has an SLA of 10 seconds for a request round­trip. Does it mean I get 10s to return the request?

Request roundtrip time RTT is time between
A- client makes request to server B- client gets response back from server
Now this time RTT includes your server response time + network delays based on many factors like distance,network hops, traffic congestion.

Understanding Long Polling Client Logic for Use With Azure Groovy Java

Premise:
We have groovy scripts that execute every minute. I want one of those scripts to open an HTTP client, and poll a service bus queue / topic for messages. I have my rest client code working an getting messages from the service bus queue. I can do a "Get" every 5 seconds, and wireshark shows that it's reusing the same TCP connection which is better than I expected, but its still not ideal.
Goal:
I would like to make this http client do "long polling", for efficiency and to achieve actual real-time processing. It seems to be more complicated than I anticipated.
Problem:
When I do a "Delete" call to read message from a service bus queue, it immediately returns "HTTP/1.1 204 No Content", and the connection closes. I set a timeout on the client, but I don't think that matters.
Here's the article that shows service bus says it's supports long polling, which I imagine is the hard part. Azure Service Bus Queues
I feel that I don't understand something fundamental about how to implement long polling in code. My understanding is that when there is no data in the queue, it's supposed to delay the response until data exists, or until my client eventually times out waiting (which lets me set my own disconnect/reconnect interval). I don't even care about blocking/nonblocking etc, because the script execution is already spreadout into a threadpool, and will be terminated forcibly and all that.
Any help is greatly appreciated.

The correct and simple answer is that adding the following to the end of an Azure REST API URL (with service bus) is the way to implements long-polling with that service: ?timeout=60 , where 60 tells azure to wait 60 seconds before responding with no-data. So, your application can check for data every 60 seconds, with an internal timeout of 60 seconds on each HTTP request. This will hold the TCP connection open for that timeframe, waiting for an HTTP response.

For understanding long polling, I recommend you can learn the entry Comet of Wiki https://en.wikipedia.org/wiki/Comet_(programming). And there is an answered thread (Long polling in java) explained the mechanism of the HttpURLConnection Class support long polling in Java.
As I know, a simple way in Java Client instead of HttpURLConnection is using the client library of CometD. You can refer to the section Client Library of its offical document https://docs.cometd.org/current/reference/#_java_client to learn how to implement the long polling client in Java. You can download the library at https://download.cometd.org/.
The sample code from the offical document:
// Create (and eventually set up) Jetty's HttpClient:
HttpClient httpClient = new HttpClient();
httpClient.start();
// Prepare the transport
Map<String, Object> options = new HashMap<String, Object>();
ClientTransport transport = new LongPollingTransport(options, httpClient);
// Create the BayeuxClient
ClientSession client = new BayeuxClient("http://localhost:8080/cometd", transport);
// Here set up the BayeuxClient, for example:
// client.getChannel(Channel.META_CONNECT).addListener(new ClientSessionChannel.MessageListener() {
public void onMessage(ClientSessionChannel channel, Message message) {
if (message.isSuccessful()) {
// Here handshake is successful
}
}
});
client.handshake();
Note: There are two REST API of Azure Service Bus for getting messaging entity(s) Get Entity https://msdn.microsoft.com/en-us/library/azure/hh780754.aspx and Entities Discovery https://msdn.microsoft.com/en-us/library/azure/hh780782.aspx. You need to delete the used messaging entity manually thru the Delete Entity REST API. Requesting all of these REST API first require an access_token thru the post request the Request a Token from ACS API for secure access.

Difference between web service connection timeout and request timeout

WebClientTestService service = new WebClientTestService() ;
int connectionTimeOutInMs = 5000;
Map<String,Object> context=((BindingProvider)service).getRequestContext();
context.put("com.sun.xml.internal.ws.connect.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.internal.ws.request.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.ws.request.timeout", connectionTimeOutInMs);
context.put("com.sun.xml.ws.connect.timeout", connectionTimeOutInMs);
Please share the differences mainly in connect timeout and request timeout.
I need to know the recommended values for these parameter values.
What are the criteria for setting timeout value ?

Please share the differences mainly in connect timeout and request timeout.
I need to know the recommended values for these parameter values.
Connect timeout (10s-30s): How long to wait to make an initial connection e.g. if service is currently unavailable.
Socket timeout (10s-20s): How long to wait if the service stops responding after data is sent.
Request timeout (30s-300s): How long to wait for the entire request to complete.
What are the criteria for setting timeout value ?
It depends a web user will get impatient if nothing has happened after 1-2 minutes, however a back end request could be allowed to run longer.
Also consider server resources are not released until request completes (or times out) - so if you have too many requests and long timeouts your server could run out of resources and be unable to service further requests.
request timeout should be set to a value greater then the expected time for the request to complete, perhaps with some room to allow occasionally slower performance under heavy loads.
connect/socket timeouts are often set lower as normally indicate a server problem where waiting another 10-15s usually won't resolve.