We Keep Coding

How to limit concurrent http requests with Mono & Flux

I want to handle Flux to limit concurrent HTTP requests made by List of Mono.
When some requests are done (received responses), then service requests another until the total count of waiting requests is 15.
A single request returns a list and triggers another request depending on the result.
At this point, I want to send requests with limited concurrency.
Because consumer side, too many HTTP requests make an opposite server in trouble.
I used flatMapMany like below.
public Flux<JsonNode> syncData() {
return service1
.getData(param1)
.flatMapMany(res -> {
List<Mono<JsonNode>> totalTask = new ArrayList<>();
Map<String, Object> originData = service2.getDataFromDB(param2);
res.withArray("data").forEach(row -> {
String id = row.get("id").asText();
if (originData.containsKey(id)) {
totalTask.add(service1.updateRequest(param3));
} else {
totalTask.add(service1.deleteRequest(param4));
}
originData.remove(id);
});
for (left) {
totalTask.add(service1.createRequest(param5));
}
return Flux.merge(totalTask);
});
}
void syncData() {
syncDataService.syncData().????;
}
I tried chaining .window(15), but it doesn't work. All the requests are sent simultaneously.
How can I handle Flux for my goal?

I am afraid Project Reactor doesn't provide any implementation of either rate or time limit.
However, you can find a bunch of 3rd party libraries that provide such functionality and are compatible with Project Reactor. As far as I know, resilience4-reactor supports that and is also compatible with Spring and Spring Boot frameworks.
The RateLimiterOperator checks if a downstream subscriber/observer can acquire a permission to subscribe to an upstream Publisher. If the rate limit would be exceeded, the RateLimiterOperator could either delay requesting data from the upstream or it can emit a RequestNotPermitted error to the downstream subscriber.
RateLimiter rateLimiter = RateLimiter.ofDefaults("name");
Mono.fromCallable(backendService::doSomething)
.transformDeferred(RateLimiterOperator.of(rateLimiter))
More about RateLimiter module itself here: https://resilience4j.readme.io/docs/ratelimiter

You can use limitRate on a Flux. you need to probably reformat your code a bit but see docs here: https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#limitRate-int-

flatMap takes a concurrency parameter: https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#flatMap-java.util.function.Function-int-
Mono<User> getById(int userId) { ... }
Flux.just(1, 2, 3, 4).flatMap(client::getById, 2)
will limit the number of concurrent requests to 2.

Understanding Long Polling Client Logic for Use With Azure Groovy Java

Premise:
We have groovy scripts that execute every minute. I want one of those scripts to open an HTTP client, and poll a service bus queue / topic for messages. I have my rest client code working an getting messages from the service bus queue. I can do a "Get" every 5 seconds, and wireshark shows that it's reusing the same TCP connection which is better than I expected, but its still not ideal.
Goal:
I would like to make this http client do "long polling", for efficiency and to achieve actual real-time processing. It seems to be more complicated than I anticipated.
Problem:
When I do a "Delete" call to read message from a service bus queue, it immediately returns "HTTP/1.1 204 No Content", and the connection closes. I set a timeout on the client, but I don't think that matters.
Here's the article that shows service bus says it's supports long polling, which I imagine is the hard part. Azure Service Bus Queues
I feel that I don't understand something fundamental about how to implement long polling in code. My understanding is that when there is no data in the queue, it's supposed to delay the response until data exists, or until my client eventually times out waiting (which lets me set my own disconnect/reconnect interval). I don't even care about blocking/nonblocking etc, because the script execution is already spreadout into a threadpool, and will be terminated forcibly and all that.
Any help is greatly appreciated.

The correct and simple answer is that adding the following to the end of an Azure REST API URL (with service bus) is the way to implements long-polling with that service: ?timeout=60 , where 60 tells azure to wait 60 seconds before responding with no-data. So, your application can check for data every 60 seconds, with an internal timeout of 60 seconds on each HTTP request. This will hold the TCP connection open for that timeframe, waiting for an HTTP response.

For understanding long polling, I recommend you can learn the entry Comet of Wiki https://en.wikipedia.org/wiki/Comet_(programming). And there is an answered thread (Long polling in java) explained the mechanism of the HttpURLConnection Class support long polling in Java.
As I know, a simple way in Java Client instead of HttpURLConnection is using the client library of CometD. You can refer to the section Client Library of its offical document https://docs.cometd.org/current/reference/#_java_client to learn how to implement the long polling client in Java. You can download the library at https://download.cometd.org/.
The sample code from the offical document:
// Create (and eventually set up) Jetty's HttpClient:
HttpClient httpClient = new HttpClient();
httpClient.start();
// Prepare the transport
Map<String, Object> options = new HashMap<String, Object>();
ClientTransport transport = new LongPollingTransport(options, httpClient);
// Create the BayeuxClient
ClientSession client = new BayeuxClient("http://localhost:8080/cometd", transport);
// Here set up the BayeuxClient, for example:
// client.getChannel(Channel.META_CONNECT).addListener(new ClientSessionChannel.MessageListener() {
public void onMessage(ClientSessionChannel channel, Message message) {
if (message.isSuccessful()) {
// Here handshake is successful
}
}
});
client.handshake();
Note: There are two REST API of Azure Service Bus for getting messaging entity(s) Get Entity https://msdn.microsoft.com/en-us/library/azure/hh780754.aspx and Entities Discovery https://msdn.microsoft.com/en-us/library/azure/hh780782.aspx. You need to delete the used messaging entity manually thru the Delete Entity REST API. Requesting all of these REST API first require an access_token thru the post request the Request a Token from ACS API for secure access.

Using netty with 3rd party blocking API

I am using a 3rd party blocking API. I am going to be using this API as follows:
while(true){
blockingAPI();
sendResultSomewhere();
}
blockingAPI() polls a server for a specific property until it gets a response.
In order to make things asynchronous to some extent I could spawn this API call within a separate thread. and have a callback implemented in Java to handle the response. I was wondering if I can use the netty framework in this scenario, and how I could do this? The examples I have seen involve a server that listens and communicates with a client, and I am not sure how my use case fits in.
If netty cannot be used, would my best bet be spawning a new thread and implementing a callback in Java?

Not sure what you really try to do:
Spawn internally a new thread: you could use LocalChannel with Netty to have intra-JVM process communication and therefore having something like you want, without any network consideration (only within the JVM). The blockingAPI will be computed within ServerLocalChannel side, while the result will be written once the client get back a response through the same LocalChannel.
Spawn but with a request from outside (network), then Netty could of course be used too there. Maybe still keeping a LocalChannel logic to separate network to compute.
Note that I could recommand to use asynchronous operation using LocalChannel (executing the blocking task), such that the send somewhere else is done without blocking the Netty's Network IO thread.
Network Handler side:
localChannel = creationWithinNetworkHandler(networkChannelCtx);
localChannel.writeAndFlush(something);
while LocalChannel handler server side could be as:
void read0(ChannelHandlerContext ctx, someData) {
blockingAPI();
ctx.channel().writeAndFlush(answear).addFutureListener(Channels.CloseFuture);
}
and LocalChannel handler client side could be as:
void read0(ChannelHandlerContext ctx, answear) {
//Using ctx from Network channel side
networkCtx.writeAndFlush(answear);
}

Reusing the SOAP client with Axis

I'm using Apache Axis to make a SOAP request to a service. I noticed that sometimes, it take a couple of seconds to get a response even though the service is a simple echo for now. So I'm wondering if establishing the connection is what takes the time, even though the server does HTTP/1.1 connection keep alive.
Should somehow reuse the client between requests or if it's ok to get a new one for every request?
This is my code. Should I keep locator and/or client around between requests or is it okay to forget it?
MyExampleServiceLocator locator = new MyExampleServiceLocator();
MyExampleServicePort client = locator.getMyExampleServicePort(url);
MyExampleRequest request = buildMyExampleRequest();
MyExampleResponse response = client.send(request); // This takes time sometimes

For complex services the cost of instantiating the locator may be high. Therefore you should always try to reuse it as much as possible. Locators are expected to be thread safe, so you can use them as singletons. Note however that in Axis 1.4 there is at least one thread safety issue regarding locators: AXIS-2498.
Creating a new stub (client) is less expensive, but reusing a stub is unproblematic. They are also expected to be thread safe (at least in Axis 1.4), except for scenarios that use the stub in a stateful way (e.g. HTTP sessions).
Axis' default HTTP transport only supports HTTP 1.0 and creates a new connection for every request.

Call a Web Service from Servlet at AppEngine

Question: What is best way to call a web service (0.5-1.5 seconds/call) from a servlet at AppEngine? Are blocking calls are scalable at AppEngine environment?
Context: I am developing a web application using AppEngine and J2EE. The applications calls Amazon web service to grab some information for the user. From my asp.net experience, best way to do the calls - is to use async http handler to prevent starvation at IIS thread pool. This feature is not available for J2EE with Servlet 2.5 spec (3.0 is planned).
Right now I am thinking of making my controllers (and servlets) thread safe and request scoped. Is there anything also that I can do? Is it even an issue in J2EE + AppEngine environment?
EDIT: I am aware of AppEngine and JAX-WS async invocation support, but I am not sure how it play with servlet environment. As far as I understand, to complete servlet request, the code still should wait for async WS call completion (callback or whatever).
I assume that doing it using synchronization primitives will block current working thread.
So, as far as thread is blocked, to serve another user request servlet container need to allocate new thread in thread pool, allocate new memory for stack and waste time for context switching. Moreover, requests can block entire server, when we run out of threads in thread pool. This assumptions are based on ASP.Net and IIS thread model. Are they applicable to J2EE environment?
ANSWER: After studying Apache and GAE documentation, it seems that starvation of threads in the thread pool is not a real issue. Apache, by default has 200 threads for thread pool (compared to 25 in asp.NET and IIS). Based on this I can infer that threads are rather cheap in JVM.
In case if async processing is really required or servlet container will run out of threads, it's possible to redesign the application to send response via google channel api.
The workflow will look like:
Make sync request to servlet
Servlet makes creates channel for async reply and queues task for background worker
Servlet returns response to client
[Serving other requests]
Background worker does processing and pushes data to client via channel api

As you observe, servlets don't support using a single thread to service multiple concurrent requests - one thread is required per request. The best way to do your HTTP call is to use asynchronous urlfetch, and wait on that call to complete when you need the result. This will block the request's thread, but there's no avoiding that - the thread is dedicated to the current request until it terminates no matter what you do.
If you don't need the response from the API call to serve the user's request, you could use the task queue to do the work offline, instead.

Isn't it OK to use fetchAsync?

looks at this, this might help
http://today.java.net/pub/a/today/2006/09/19/asynchronous-jax-ws-web-services.html
I am not sure, If you can exactly replicate what you do in dot net, Here is what you could do to may be to simulate it page on load
Submit an ajax request to controller using a java script body onload
In the controller start the async task and send the response back the user and use a session token to keep track of the task
You can poll the controller (add another method to ask for update of the task, since you have session token to track the task) until u get the response
You can do this either waiting for response page or hidden frame that keeps polling the controller
Once you have the response that you are looking for remove the session token
If you want to do that would be the best option instead of polling would be ideal in this case Reverse Ajax / server push
Edit: Now I understand what you mean, I think you can have your code execute async task not wait for response from async itself, just send response back to the user. I have simple thread that I will start but will wait for it to finish as I send the response back to the user and the same time use a session token to track the request
#Controller
#RequestMapping("/asyncTest")
public class AsyncCotroller {
#RequestMapping(value = "/async.html", method = RequestMethod.GET)
public ModelAndView dialogController(Model model, HttpServletRequest request)
{
System.err.println("(System.currentTimeMillis()/1000) " + (System.currentTimeMillis()/1000));
//start a thread (async simulator)
new Thread(new MyRunnbelImpl()).start();
//use this attribute to track response
request.getSession().setAttribute("asyncTaskSessionAttribute", "asyncTaskSessionAttribute");
//if you look at the print of system out, you will see that it is not waiting on //async task
System.err.println("(System.currentTimeMillis()/1000) " + (System.currentTimeMillis()/1000));
return new ModelAndView("test");
}
class MyRunnbelImpl implements Runnable
{
#Override
public void run()
{
try
{
Thread.sleep(5000);
} catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}