i am trying to create a http client based on netty. I have written the code based on the HttpSnoopClient example given in Netty site. But the problem is HttpResponse are handled by HttpSnoopClientHandler & HttpRequests are sent in HttpSnoopClient & i want to sync it. as in if i send a request i want to make sure that i will send the next request once i know the response to the previous. But since both are handled in different class, It is becoming difficult to do the same.
One thing i did was to create a setResponse() method in HttpTarget & HttpSnoopClientHandler will be setting the HttpResponse when it receives the Response from the sever. But i don't think it is a good approach since i won't be able to know the reposne was for which request.
So basically i want to do it synchronously i.e. send a request(channel.writeandFlush(req)) in HttpSnoopClient then wait till the response is received by the HttpSnoopCLientHandler & once it recieves a HTTP 1.1 200 OK then send the next request.
Can anyone tell me a good approach for doing it. Thanks in advance!
I had a similar use case where I had to block concurrent requests till one completes for a resource. I implemented a ConcurrentHashMap<RequestKey, ArrayList<ChannelHandlerContext>>> which will hold all the concurrent requests ChannelHandlerContext (ctx) and on completion of the first request raise an event which would trigger all other ctx to consume the cached response. In all this I had to make sure the AUTO_READ was set to false for fine grain control over the reads on each channel.
channelRead ->
if(map.contains(reqKey)){
map.add(reqKey, list.add(ctx))
//do nothing with AUTO_READ = false and do not initiate any ctx.* methods
}else{
//firstRequest
map.add(reqKey, new ArrayList<CTX>(){{ add(ctx);}})
//continue with request execution
//cache response and raise event on completion
}
userEventTriggered ->
onCompletionEvent {
ctxList = map.get(reqKey).clone();
map.remove(reqKey);
for(blockedCtx : ctxList){
//respond back with cached response to each blockedCtx
}
}
#norman-maurer would you give your take on this !!!
As you're creating a new HttpSnoopClientHandler for each connection, I would consider turning HttpSnoopClientHandler into a ChannelDuplexHandler. In the write method you can store a reference to the outgoing http request. When the response is received you can call your setResponse method with (channel, request, response). This should provide enough context so you can process the response correctly.
If your client is pure request/response, does not issue unrelated requests separately, and you want your application thread to process responses sequentially, then you could use a SynchronousQueue to coordinate responses with allowing the main thread to continue. Alternatively your callback can process the responses internally.
You can also extends this technique to use HTTP pipelining. HTTP pipelining guarantees that responses are returned in the order that requests are issued. In HttpSnoopClientHandler you maintain a queue of requests. As each response is returned you match it to the request at the front of the queue.
Related
I am building an application which connect with other too. For the application I need to take a request from a websocket connection. Once i received the request need to send the request to some other application for processing and there will be few cases.
If second application return accepted then wait for a response from 3rd application (3 will send response to 2nd and 2nd will initiate a push model).
If second application return other than accepted then return false to the request
My confusion is, this way I will handle the request as synchronous or asynchronous ?
As in case #1 I have to wait for some time to receive response from another application. more over in case #2 I can immediately process a request.
Sequence diagram for clarity of flow
I am using the StreamObserver class found in the grpc-java project to set up some bidirectional streaming.
When I run my program, I make an undetermined number of requests to the server, and I only want to call onCompleted() on the requestObserver once I have finished making all of the requests.
Currently, to solve this, I am using a variable "inFlight" to keep track of the requests that have been issued, and when a response comes back, I decrement "inFlight". So, something like this.
// issuing requests
while (haveRequests) {
MessageRequest request = mkRequest();
this.requestObserver.onNext(request);
this.inFlight++;
}
// response observer
StreamObserver<Message> responseObserver = new StreamObserver<Message> {
#Override
public void onNext(Message response) {
if (--this.onFlight == 0) {
this.requestObserver.onCompleted();
}
// work on message
}
// other methods
}
A bit pseudo-codey, but this logic works. However, I would like to get rid of the "inFlight" variable if possible. Is there anything within the StreamObserver class that allows this sort of functionality, without the need of an additional variable to track state? Something that would tell the number of requests issued and when they completed.
I've tried inspecting the object within the intellij IDE debugger, but nothing is popping out to me.
To answer your direct question, you can simply call onComplete from the while loop. All the messages passed to onNext. Under the hood, gRPC will send what is called a "half close", indicating that it won't send any more messages, but it is willing to receive them. Specifically:
// issuing requests
while (haveRequests) {
MessageRequest request = mkRequest();
this.requestObserver.onNext(request);
this.inFlight++;
}
requestObserver.onCompleted();
This ensures that all responses are sent, and in the order that you sent them. On the server side, when it sees the corresponding onCompleted callback, it can half-close its side of the connection by calling onComplete on its observer. (There are two observers on the server side one for receiving info from the client, one for sending info).
Back on the client side, you just need to wait for the server to half close to know that all messages were received and processed. Note that if there were any errors, you would get an onError callback instead.
If you don't know how many requests you are going to make on the client side, you might consider using an AtomicInteger, and call decrementAndGet when you get back a response. If the return value is 0, you'll know all the requests have completed.
Obviously, the doGet method has a return type of void, so, it doesn't return anything. In this sense, I'm using the word "return" to mean send the response back to the client that requested it.
I'm trying to implement a long-polling Servlet. It would be beneficial for it not to send a response until I have something that I would like to send back. So, in the doGet method I add the connected user's ID and AsyncContext to a map:
private ConcurrentMap<String, AsyncContext> contexts = new ConcurrentHashMap<>();
//...in the doGet method when I need to add the context...
contexts.put(userId, context);
Then, when I have something to send back, I can retrieve the appropriate context and write to it's responses output stream:
AsyncContext context = contexts.get(userId);
PrintWriter writer = context.getResponse().getWriter();
writer.write("something to send to the client");
But, the client never seems to receive the response. Looking at the Network tab in the developer console of the browser, I can see the GET request is sent and then returns (with a status of 200). This occurs before I actually send something back. Which is leading me to believe that after the doGet method is finished the response is returned. And perhaps because of this, after this point, nothing can be sent to the client because the connection is not opened.
Does the doGet method send the response to the client once the method is finished executing? If this is the case, how can I keep the connection open for a long-polling effect?
To answer my own questions: Does the doGet method send the response to the client once the method is finished executing?
Yes, when the doGet (or any HttpServlet method, ex: doGet, doPost, etc.) method finishes executing it sends the response back to the client.
If this is the case, how can I keep the connection open for a long-polling effect?
Using asynchronous Servlets (which I was using, however, I found my particular problem must be elsewhere, yet these answers are still relevant to the questions asked). On the ServletRequest object call the startAsync method, like so:
AsyncContext context = request.startAsync(request, response);
"This will notify the Web Container that at the end of the request call it should free the handling thread and leave the connection open so that other thread writes the response and end the connection."Reference Link.
Also, I will add the solution to my particular problem (the client wasn't receiving the response) was because in my Servlet, I wasn't calling the complete method on the AsyncContext object:
asyncContext.complete();
Yes, the response stream is flushed and closed when doGet() finishes executing.
Keeping UI threads occupied for extended periods of time violates Java Enterprise best practice.
Recommend you rather return immediately if nothing to respond, and implement a timer on the client (browser) side to poll the server for results every so often.
Assuming no keep alives, when a servlet container is acting as a stand alone server, I assume that the servlet's thread is not released until the entire response is sent to the client (say a web browser). Is this a correct assumption?
But what happens if the servlet is behind a reverse proxy like Nginx? Is the thread released once the response is delivered to Nginx, or is it held until the response is sent to its final client (say a browser)?
Update: Let me try make this a bit more clear.
It takes mere milliseconds (say 2ms) for a response to be sent from servlet to proxy like nginx. But it can then take an additional 80ms (or so) for the final response to be sent from nginx to the browser. Does the servlet release the thread/stream once the response is sent to nginx, or does the servlet hold onto them until the response is sent to the browser (that is the entire 80ms)
Question: I assume that the servlet's thread is not released until the entire response is sent to the client (say a web browser). Is this a correct assumption?
Ans: No it is wrong. Servlet container will just write the content to the socket and return. It is not guaranteed that return from write() method will ensure that the response has reached the client.
Question: Is the thread released once the response is delivered to Nginx, or is it held until the response is sent to its final client (say a browser)?
Ans: When Nginx is behind , then the client for Servlet container is Nginx. It is not aware of actual remote client. So, the thread will be released once the response is written to Nginx.
The server container not being able to send a response to the client will trigger an exception that will be handled by the container. You can enclose the writing to the outputstream or writer by a try catch finally (with close()) but you don't need to, the container will manage, including the return of the thread to the pool.
Regards
S
A servlet does not see the network. According to the specifications It is handled 2 objects: a Request and a Response to be filled in (in the case of HTTP, this means a HTTPRequest and a HTTPResponse). It shall process the request data within the request object, and write to the buffer in the response object. Once that content is commited by the servlet, the container may do some postprocessing (using filters) and will transmit it back to the client.
The servlet thread returns naturally to the pool once the call to the request handling method finishes (that may happen after the payload is sent back to the client, if the method has to do further work.
Note that because the servlet doesn't see the network and is only concerned about a single request, the state of the http connection (keep-alive or close) is independent of the servlet lifetime; several servlets may handle the different requests pipelined in a single connection. See this question for a related issue.
I'm having trouble establishing AsyncContexts for users and using them to push notifications to them. On page load I have some jQuery code to send the request:
$.post("TestServlet",{
action: "registerAsynchronousContext"
},function(data, textStatus, jqXHR){
alert("Server received async request"); //Placed here for debugging
}, "json");
And in "TestServlet" I have this code in the doPost method:
HttpSession userSession = request.getSession();
String userIDString = userSession.getAttribute("id").toString();
String paramAction = request.getParameter("action");
if(paramAction.equals("registerAsynchronousContext"))
{
AsyncContext userAsyncContext = request.startAsync();
HashMap<String, AsyncContext> userAsynchronousContextHashMap = (HashMap<String, AsyncContext>)getServletContext().getAttribute("userAsynchronousContextHashMap");
userAsynchronousContextHashMap.put(userIDString, userAsyncContext);
getServletContext().setAttribute("userAsynchronousContextHashMap", userAsynchronousContextHashMap);
System.out.println("Put asynchronous request in global map");
}
//userAsynchronousContextHashMap is created by a ContextListener on the start of the web-app
However, according to Opera Dragonfly (a debugging tool like Firebug), it appears that the server sends an HTTP 500 response about 30000ms after the request is sent.
Any responses created with userAsyncContext.getResponse().getWriter().print(SOME_JSON) and sent before the HTTP 500 response is not received by the browser, and I don't know why. Using the regular response object to send a response (response.print(SOME_JSON)) is received by the browser ONLY if all the code in the "if" statement dealing with AsyncContext is not present.
Can someone help me out? I have a feeling this is due to my misunderstanding of how the asynchronous API works. I thought that I would be able to store these AsyncContexts in a global map, then retrieve them and use their response objects to push things to the clients. However, it doesn't seem as if the AsyncContexts can write back to the clients.
Any help would be appreaciated.
I solved the issue. It seems as though there were several problems wrong with my approach:
In Glassfish, AsyncContext objects all have a default timeout period of 30,000 milliseconds (.5 minutes). Once this period expires, the entire response is committed back to the client, meaning you won't be able to use it again.
If you're implementing long-polling this might not be much of an issue (since you'll end up sending another request after the response anyway), but if you wish to implement streaming (sending data to back to the client without committing the response) you'll want to either increase the timeout, or get rid of it all together.
This can be accomplished with an AsyncContext's .setTimeout() method. Do note that while the spec states: "A timeout value of zero or less indicates no timeout.", Glassfish (at this time) seems to interpret 0 as being "immediate response required", and any negative number as "no timeout".
If you're implementing streaming , you must use the printwriter's .flush() method to push the data to the client after you're done using its .print() .println() or .write() methods to write the data.
On the client side, if you've streamed the data, it will trigger a readyState of 3 ("interactive", which means that the browser is in the process of receiving a response). If you are using jQuery, there is no easy way to handle readyStates of 3, so you're going to have to revert to regular Javascript in order to both send the request and handle the response if you're implementing streaming.
I have noticed that in Glassfish if you use AsyncContext and use .setTimeOut() to a negative number the connection is broken anyway, to fix this I had to go to my Glassfish admin web configurator : asadmin set
configs.config.server-config.network-config.protocols.protocol.http-listener-1.http. And set timeout to -1. All this to avoid glassfish finish the connections after 30 sec.