I was reading about the new springframework 3.2 feature, Servlet 3 Async support here:
http://blog.springsource.org/2012/05/07/spring-mvc-3-2-preview-introducing-servlet-3-async-support
and I have a question.
The blog article indicates that spring now supports:
Controller returning a Callable
The callable gets invoked in a background thread (TaskExecutor)
The response remains open (request.startAsync()).
The stated motivation is to avoid eating up all of the web container threads. However, my understanding is this:
The web container thread pool is tunable.
There is a hard limit on that thread pool because it's preferable to degrade serving rather than bring down the process by exhausting all resources (as would happen with, say, an unbounded thread pool and large enough number of requests).
The point of servlet-3/comet is to limit the number of open threads while not limiting the number of open connections to a web server (because threads are expensive and sockets are cheap)
So given this, the feature Spring is proposing makes very little sense to me. Are they not just hacking an unbounded thread pool on top of the container's bounded pool? Does that not miss the point of limiting the number of threads?
Related
I've been reading "Spring in Action" and got confused by the following description of Servlet-based frameworks (emphasis mine):
Typical Servlet-based web frameworks, such as Spring MVC, are blocking and multi-threaded in nature, using a single thread per connection. As requests are handled, a worker thread is pulled from a thread pool to process the request. Meanwhile, the request thread is blocked until it’s notified by the worker thread that it’s finished.
I was under the impression that servlet containers like Tomcat do something like this under the hood (very simplified, of course):
while (true) {
Socket socket = serverSocket.accept();
// the actual handling of request is done by Servlet-based framework and application code.
workerThreadPool.submit(() -> handleRequest(socket));
}
I mean, we do not create a thread for each connection but reuse existing worker threads.
Also I'm not sure what a request thread is. I thought it could be the thread in which we accept() connections, but this thread doesn't block until request has been processed. My understanding is that with Spring Boot and embedded Tomcat worker threads are the ones with names http-nio-8889-exec-* (see VisualVM screenshot). What is the request thread here?
So, my questions are:
What important things are missing in the pseudocode above?
Does Spring MVC (or servlet container) actually create a thread per each connection?
What is the role of request thread (or threads) and who creates them? How can I identify them in a typical Spring Boot application?
No sensible web server uses a thread per connection. Such a design won't scale with the number of connections.
Typical Servlet-based web frameworks, such as Spring MVC, are blocking
and multi-threaded in nature, using a single thread per connection. As
requests are handled, a worker thread is pulled from a thread pool to
process the request. Meanwhile, the request thread is blocked until
it’s notified by the worker thread that it’s finished.
Even though they say 'thread per connection' the description talks about thread pools.
The thread that 'accepts', blocks on the selector. As soon as new TCP/IP connection is established, the accepting thread unblocks. I'm not sure about the implementation; but one way of dealing with is to detach the connection from the accepting selector and use one more selectors that wait for read/write events. Of course each of these selectors will have their own thread.
My guess is that the http-nio-8889-exec are the request threads. It has been a very long time I used Spring, but in the past, Spring just piggybacked on the threads provided by the servlet container.
I'm currently working on an app which is heavily connected to maps. To display a map, we are generating a bunch of tiles in many threads, store them and get them if a user wants to see a certain part of the map.
The problem is, I'm naming threads that generate tiles a certain way, but then, when I want to get tiles to show a map, my servlet container's taking random threads from the pool, so the thread named for generating a tile ends up getting it from the storage. Of course, I could just rename the thread after generating a tile back, but I wonder if there is an alternative.
I wonder if I somehow can configure my servlet container for it to maybe kill threads after some time being idle or to create a new thread where I want to or to allocate several threads to work with this part of the code?
All I could find in terms of configuring servlet container is setting its min and max thread pool size, which I think won't help me.
The container is 100% in control over it's threading.
If you are attempting to manipulate the threading of the container then you are fighting a losing battle.
It is not possible to safely kill or stop threads on a running container, as this is incredibly unsafe, and will lead to many memory issues (leaks) and unclosed resources. The Thread.stop() method has been deprecated since Java 1.2.
Now that we have the negatives out of the way ...
Jetty is a 100% Async Java Web Server.
The classic assumption that 1 request uses 1 thread is wrong. (if you want this kind of behavior, then you should use Jetty 6 or older. Jetty versions older than 9.2 are now all EOL / End of Life)
When you use a Servlet call that is traditionally a blocking call, the Jetty server has to fake that blocking call to satisfy the API contract.
Even if using old school / traditional blocking Servlet APIs you'll still experience many situations where that 1 request has been handled by multiple threads over the lifetime of that 1 request.
If you want to work with the Servlet API and it's container then the first thing you should do is start to use both the Servlet Async Processing APIs and Servlet Async I/O APIs combined. Make sure you read about the gotchas on both APIs!
Async Processing will allow you to handle more processing of requests on the server side, not use the container threads that heavily, allow more control over how the threading behaves, will grant you better control over request timeouts, and even get notified of request/response error cases that you will always deal with on a web server.
Async I/O will allow you to only use a thread if there is content from the request/connection to read or if the connection allows a write. That connection will not consume a thread unless I/O is possible. This means more connections/requests per server, and ill behaving clients (slow, dead, problematic, etc) will not impact the behavior of your other clients by consuming threads that are not doing anything productive for you.
If you don't want to work with the Servlet API and do things your own way, then you'll have to manage your own Executor / ThreadGroup / ThreadPool that the server is unaware of. But that still means you'll need to use the Servlet Async Processing APIs to allow the 2 to coexist in harmony (you'll need to use the AsyncContext to inform the container that you are now taking control over the processing of the request, and then later inform it via the AsyncContext that you are done and the request is complete).
The biggest gotcha with this approach is that you cannot safely write to the HttpServletResponse from a thread that the container wasn't in control over.
Meaning the container dispatched on a thread to your application, that thread is the only one that can safely use the HttpServletResponse to write the response. You can have a different thread do the processing, a different thread provide the data to the HttpServletResponse, even a different thread that pumps the dispatch thread with content. But that thread you were dispatched to needs to be used to write.
This is the mixed threading behavior gotcha in the servlet spec. (you are in servlet async mode, on a different thread to process, but not using async mode to read/write.) It's a terribly complex, and ill defined, behavior in the servlet spec that leads to many issues, and I advise you to not chase this path.
This gotcha goes away if you also use the Servlet Async I/O APIs, but at that point the difference in the two above choices is negligible.
I am new to Java and to web development in general so I am reading this tutorial where it says that one advantage of the servlet pattern over CGI is:
better performance: because it creates a thread for each request not
process.
However, I really do not get why it should be so. Based on this answer, at least for Linux, the general consensus seems to be that threads are not necessarily faster than processes and it might be actually advantageous to use processes instead of threads.
In the tutorial it is written that CGI works as follows:
For each request, it starts a process and Web server is limited to
start processes.
When taking into account the startup cost of a process, this could make sense. However, I am not sure why it would be necessary to start a new process for each request, instead of having a pool of running CGI shells serving the queued requests.
The main performance difference is that GCI forks / execs a new process for each request, but a well-designed Servlet container creates a (bounded) pool of threads on start up, assigns them to requests, and recyles them when the request completed.
The cost of creating the threads (which is significant) is amortized over the lifetime of the servlet container.
If you could maintain a pool of "CGI shells", I suppose that would be more efficient. However, the normal assumption of a CGI app is that is starting with a clean sheet.
There are a couple of other issues:
In a servlet container you can also maintain shared session and request caches, shared pools of DB connections and so on.
The performance of a CGI implemented using a JVM per request would be awful ... because of the overheads of JVM startup / warmup. A typical request probably wouldn't run long enough for bytecodes to be JIT compiled.
because
IN CGI each request is handled by heavy weight OS process and In Servlets each request is handled by lightweight Java Thread
How is asynchronous request processing in web frameworks like play! ,jersey ,spring different from the typical multi-threaded servers with pooled threads.
https://jersey.java.net/documentation/latest/async.html
https://www.playframework.com/documentation/2.3.x/JavaAsync
One thread is listening and heavy processing happens in new thread
Also non blocking requests in web can not be compared to non blocking IO in java using selectors where one thread can read multiple channels.
From a server side perspective what does non-blocking mean.A multi threaded server is already non-blocking.Is this wrong?
What are the real benefits/use-cases of asynchronous web frameworks over multi threaded servers(with thread pooling)?
One might be ability to keep the HTTP request(s) alive in some way(How?)
Traditionally, each connection requires a thread. Since HTTP connections can be long-lived, this approach cannot sustain too many concurrent users. At least that's what they say. But in practice, this does not seems to be a huge problem for java servers; you can always throw in more machines:)
With non-blocking, one thread can be used to server many connections; 10,000 shouldn't be a problem. So this approach might be more resilient for some applications. The catch is, programming in non-blocking is a little more difficult.
I'm writing an app which uses MDBs, EJBs and needs ThreadLocal to pass and log a variable across multiple EJBs and Helper classes until the transaction is complete.
The flow is
Starting with MDB onMessage()
-> some business delegate EJBs
-> some helpers
Question:
This app runs within Weblogic and Weblogic re-uses Threads from within it's ThreadPool. So is there a possibility of data corruption across threads? Is the solution to use ThreadLocal.remove() safe enough?
Is there an alternative to ThreadLocal other than passing around the Object as a parameter to all methods?
WebLogic does not reset user set ThreadLocal variables when the thread is returned back to the pool - the user is responsible for managing them. When such threads are reused, its likely they will interfere. You may run into memory leaks since the thread local reference isn't cleaned up. You can safely reset your thread locals prior to returning the thread back to the container. The ThreadLocal.remove() call should clean it up (ensure that its done in a finally block)
Note that if any async or rmi calls are involved, your thread locals will not propagate. You may want to consider the WebLogic WorkArea feature which allows context propagation across threads, clients & servers. More details can be found at http://download.oracle.com/docs/cd/E17904_01/web.1111/e13706/context.htm#i1058690
You can't reliably use a ThreadLocal in the EJB tier. Even if your code seems to 'work' now, what happens if someone deploys one of your beans remotely? From EJB Restrictions:
Why is thread creation and management disallowed?
The EJB specification assigns to the EJB container the responsibility
for managing threads. Allowing enterprise bean instances to create and
manage threads would interfere with the container's ability to control
its components' lifecycle. Thread management is not a business
function, it is an implementation detail, and is typically complicated
and platform-specific. Letting the container manage threads relieves
the enterprise bean developer of dealing with threading issues.
Multithreaded applications are still possible, but control of
multithreading is located in the container, not in the enterprise
bean.
If you need to share state, you should pass it in to the EJB method as a parameter. Is there a reason this approach won't work for you? Another option would be to temporarily dump it into a transaction enlisted database or cache.
#JoseK: though I have not tried what you described in your issue, but here are my thoughts:-
Both MDB and Session beans are thread-safe. It means let us say if there is pool of 10 beans, only 10 requests will be handled simultaneously. Other requests would be queued for their turn. So one running thread local data should not interfere with other thread.
If you confident to use always local EJBs in future also, then I don't really see any issue in using thread local data. Because you are not really creating threads.
Though weblogic provides thread from thread-pool but that thread is given dedicately to each request flow, I don't think its local data should become corrupted ever.
As I said I have not tried myself, what I would try is:-
In MDB layer(your first layer), do Thread.getCurrentThread.setName(name)
and in subsequent layers print thread names like Thread.getCurrentThread.getName)
Perform multiple runs with different size of ejb pool, thread pool. Give a different thread name to each request flow. Try running multiple requests same time. And see if you ever get thread name mixed.
5.Having said above, to keep things simpler and furture remote EJB support, I would also pass CallingContext Interface to each layer.