I have a queue and I have this consumer written in java for this queue. After consuming, we are executing an HTTP call to a downstream partner and this is a one-way asynchronous call. After executing this request, the downstream partner will send an HTTP request back to our system with the response for the initial asynchronous call. This response is needed for the same thread that we executed the initial asynchronous call. This means we need to expose an endpoint within the thread so the downstream system can call and send the response back. I would like to know how can I implement a requirement like this.
PS : We also can get the same response to a different web service and update a database row with the response. But I'm not sure how to stop the main thread and listen to the database row when the response is needed.
Hope you understood what I want with this requirement.
My response based on some assumptions. (I didn't wait for you respond to my comment since I found the problem had some other interesting features anyhow.)
the downstream partner will send an HTTP request back to our system
This necessitates that you have a listening port (ie, a server) running on this side. This server could be in the same JVM or a different one. But...
This response is needed for the same thread
This is a little confusing because at a high level, reusing the thread programmatically itself is not usually our interest but reusing the object (no matter in which thread). To reuse threads, you may consider using ExecutorService. So, what you may try to do, I have tried to depict in this diagram.
Here are the steps:
"Queue Item Consumer" consumes item from the queue and sends the request to the downstream system.
This instance of the "Queue Item Consumer" is cached for handling the request from the downstream system.
There is a listener running at some port within the same JVM to which the downstream system sends its request.
The listener forwards this request to the "right" cached instance of "Queue Item Consumer" (you have to figure out a way for this based on your caching mechanism). May be some header has to be present in the request from the downstream system to identify the right handler on this side.
Hope this works for you.
Related
Our project consists of multiple microservices. These microservices form a boundary to which the entry point is not strictly defined meaning each of microservices can be requested and can request other services.
The situation we need to handle in this bounded microservice context is following:
client (other application) makes the request to perform some logic and change the data (PATCH),
request times out,
while request is being processed client fires the same request to repeat the operation,
operation successfully completes,
second request is being processed the same way and completes within it's time and client gets response.
Now what happened is that the same was processed two times because of first timeout.
We need to make sure the same request won't get processed and application will respond with former response and status code.
The subsequent request is identified by the same uuid.
Now, I understand it's the client that should do requesting more precisely or we should have a single request entry point in out micorservices bounded context, but in enterprise projects the team doesn't own the whole system therefore we are a bit constrained with the solutions we propose for the problem. with this in mind while trying to not reinvent the wheel this comes to my mind:
The microservices should utilize some kind of session sharing (spring-session?) with the ability to look up the request by it's id before it gets processed and in described case, when first is being processed and second arrives, wait for the completion of the 1st and respond to the second with data of the first that has timed out for a client.
What I am struggling with is imagining handling the asynchronicity of replying to the second one and how to listen for session state of the first request.
If spring-session would be used (for example with hazelcast) I'm lacking some kind of concrete session state handler which would get fired when request ends. Is there something like this to listen for?
No code written yet. It's an architectural thought experiment that I want to discuss.
If unsure of understanding, read second time please, then I'm happy to expand.
EDIT: first idea:
process would be as follows (with numbering on the image):
(1) first request fired
(3) processing started; (2) request timed out meanwhile;
(4) client repeats the same request; program knows it has received the same request before because it knows the req. id.
program checks the cache and the state of that request id 'pending' so it WAITS (async).
computed result of first request is saved into the cache - orange square
(5) program responds to the first request with the data that was meant to be for the first one
idea is that result checking and responding to the repeated request would be done in the filter chain so it won't actually hit the controller when the second request is asynchronously waiting for the operation triggered by the first request to be done (I see hazelcast has some events when rows are added/updated/evicted from the cache - dunno if it's working yet) and when complete just respond (somehow write to the HttpServletResponse). result would be saved into the cache in postHandling filter.
Thanks for insights.
I'd consider this more of a caching paradigm. Stick your request/responses into an external cache provider (REDIS or similar), indexed by uuid. Having a TTL will allow your responses to automatically get cleaned up for requests that are never coming back, and the high-speed implementation (o1) should allow this to scale nicely. It will also out-of-the-box give you an asynchronous model (not a stated goal, but always a nice option).
I need to write a web application which receives a lot of HTTP requests and takes a long time (30s to 2min) to process each request (in turn making other network requests) before returning a response.
Because there would be a lot of requests coming in and those connections are held open I'm thinking of going down an event driven route, which leads me to think Netty is appropriate.
If each request takes a long time to process, is that going to block netty's processing? Or can I receive a request and then asynchronously process it before returning a result to the request's connection?
As long as you don't block the event loop, you will be able to serve a significant amount of concurrent requests (depending on the available memory, and the size of the context you're holding for each request).
What you need to do is to make sure you're making the outbound network requests in a non blocking manner. This normally looks like so (in your Netty inbound handler):
CompletableFuture<YourResultType> future = remoteTarget.getStuff();
future.thenApply(ctx::write);
You need to hold a reference to a context / channel if you're doing this outside of the handler of course.
Note that this is a simplified answer. If you're making several outbound requests and have some business logic, you need to stitch your code properly using continuations on the futures, or whatever non-blocking model you are using.
I have written a Vert.x HTTP server in Java. When the client sends requests faster than the server can process them, the server-side request queue slowly fills up. Eventually the JVM runs out of memory because of all the accumulating requests.
Can I set a capacity on the Vert.x request queue?
I would like to set one or more of the following:
A maximum number of queued requests
A maximum size (in bytes) of all queued requests
When either of these limits is violated by an incoming request, I would like to immediately respond with 503 Service Unavailable.
AFAIK there's no built-in way to accomplish this. However, this type of back pressure should still be achievable by normal means. The approach you take is this:
When an HTTP requests is received, immediately forward the request via a message to a separate request handling verticle on the event bus and increment an outstanding request counter.
Perform request handling logic in that verticle and respond to the event bus message once complete.
Once the HTTP server verticle receives a response from the request handler verticle, decrement the request counter and send the appropriate response.
Add a request counter check to your HTTP server handler to check the outstanding request count and respond with an appropriate error if the queue grows too large.
This is a common pattern in Vert.x that essentially just separates request handling logic from the HTTP request handler. Forwarding the request on the event bus as a JsonObject ensures that requests are quickly queued in the event bus. You an use that queue to calculate the number of outstanding requests as I've shown.
Note also that you can scale your HTTP server across multiple verticle instances in order to handle more requests. In this case you can either use static variables or shared data to share a semaphore across the instances.
We have an requirement in which it causes an design constraint and it is show stopper. Here it is,
Sender thread will put requests into the messaging queue. Input source is a text file that contains 10 million requests.
Recevier thread polls the responses from another queue and write it onto another output file.
Design Constraint:
Recevier thread has to write the request and response onto output file. How this is possible ?
No Database should be used
Caching the request before sending and updating it after corresponding response has been recevied cannot be used because of performance bottleneck.
In few cases, timeout occurs if the response is delayed very long time.
Please advice.
Since you have just one Receiver thread, it is guaranteed that only one request will be processed at a time.
Having the sender thread write the request and response is probably not the most elegant design, but you could certainly have the Receiver thread write the {request, response} tuple. The Receiver thread could also write the request before it starts processing and the response after it is done. It will have the same result as what you are aiming for.
If you give out more details about your design, I can provide you with more design help.
Several ideas for solutions:
The receiver thread could look up the original request from the file. This requires responses to have some form of unique correlation id.
The handler threads could add the original request to the response. This makes message size bigger but avoids the need of a correlation id. It also requires configuration/code changes to be made to the handler threads.
The sender thread could duplicate the request on a secondary local queue. The receiver thread looks up the original request for a received response on this queue. Responses might not be received in the same order as the requests were sent so the receiver thread might need to 'walk the queue' to find the request which is not very efficient.
The last solution comprises a form of caching. You state this is not allowed due to performance reasons. I don't understand exactly why though. Local caching is fast and there should not be a large amount of queued requests in the cache at any given time since the sending, handling and receiving are all asynchronous.
I would like to write a method which handles the flow of communication on XMPP. The sequence of things I'd like to do is:
Send message.
Wait for response.
Process the response.
Since we could be waiting longer than 30s for the response (step 2) I'll be teeing up a task to take care of this. This task will need to send the message and then wait for a response on the XMPP servlet handling the incoming message. My question is: How do I wait in the task servlet thread for the response to arrive in the XMPP Servlet?
I'd normally use a listener pattern where the listener would store the message in a field in the Task object and then trigger a Semaphore to signal that a message has arrived. Like this:
Install listener in XMPP servlet in a static field.
Send message.
Wait for semaphore. ........ Meanwhile, in the XMPP servlet thread, a response will arrive and it will call the listener's callback method which stores the message and releases the semaphore.
Get message from field and process.
I tried this and it worked fine on the development server. However, when I uploaded to the cloud I found that I'd install the listener on the XMPP servlet (step 1) but then a new instance of the servlet would be instantiated when the message came in and there would no longer be a reference to the listener to call, event through the listener is a static field. My conclusion is XMPPServlet is run in a completely different VM meaning the static field is not shared between that servlet and the task one. Is this correct?
In general what is the best practice for communication between these servlets? How to I share data (normally I would've stored it in an object's field) and how do I signal from one to the other when events occur (normally I would've used a semaphore)?
Sorry about the long winded question. Tell me if it's not clear and I'll refine it a bit.
Reposting my answer to the same question you asked on the mailing list:
You can't [wait for a response in the sending process]. Instead, you
should use an asynchronous pattern: Send the message, and register a
handler for incoming XMPP messages. That handler should match up the
response to the corresponding request (stored in the datastore if
necessary) and perform appropriate processing on it.
An App Engine app can be run on any number of machines;
synchronization primitives designed for communication between threads
will not work.