I'm new at Spring and I'm reading one book "Pro Spring boot 2". It says here that Spring Web MVC has some blocking on each request, and Spring Webflux is a completely non-blocking stack.
Tell me, please, what is meant?
The request that came to Spring MVC activates one thread to execute this request. When and why is it blocked?
And why doesn't Spring WebFlux block thread?
Spring Web MVC takes a single thread to handle each request to your API. Spring Webflux does not block a thread to handle each request, because no thread is kept waiting for something to be done (e.g. waiting for an answer from a database).
As written in 1., it can be blocked while waiting for an answer from a database or from another service that is called via HTTP.
Spring Webflux takes advantage of the reactive stack (take a look at https://projectreactor.io/) which is fully non-blocking. This means that no thread is blocked waiting for something to happen. Everything is based on reactive streams publishers (Mono and Flux) making your code reactive to data being available (from a database or from another service called via HTTP as examples).
Related
If my reactive endpoint needs to call an external, non-reactive endpoint which does blocking stuff, is my reactive endpoint still reactive?
I have 2 services running that levergaes Spring Boot MVC and Spring Webflux.
Service A
Spring Webflux
Service B
Spring MVC
Now my Service A which is reactive calling the Service B endpoint which is blocking. Will my service A be affected and be non-reactive?
If yes? How to tackle such scenario such that my reactive endpoint stays reactive?
If Service A is written as a reactive services using webflux it will always stay reactive, no matter who else it is calling.
Im going to assume we are talking http/https now using for instance webclient.
The thing about reactive is if service B is very slow to respond, your service A, will not have the calling thread wait for the response, instead service A's thread will go and do other stuff while it is waiting for the response from service b, and when the response comes back any thread can pick up the response and continue.
A reactive service isn't reactive if it only calls other reactive services. It's always reactive. As in it reacts to events.
There are some caveats. For instance if you are talking to a database, here the database driver needs to be a specific driver that is following the R2DBC specification.
You see the traditional JDBC specification for how to write database drivers was designed to be inherently blocking. If you followed the spec and wrote a driver the driver would automatically have to be blocking. You had no choice.
So spring created a new spec (R2DBC) that is not inherently blocking, so you need a driver that is compliant to that spec. Otherwise your service will be affected by blocking behavior.
But the answer to your question is.
No, if we are talking http from one service to another.
Is it possible for a RestTemplate to consume an endpoint which is reactive based (Spring WebFlux)? I understand that the main idea of reactive programming is to avoid blocking and make better use of threads (eliminate thread per connection model) so what happens if my Client is non reactive?
Will I still be able to call the service even if it is in a blocking
manner?
To achieve full reactiveness (non blocking) both Client and Server
must be reactive?
Yes, that is not relevant to the clients of Reactive applications. The reason is that this is a regular HTTP call.
Each may be fully reactive on its own. Having said that, if you use WebFlux in both Client and Server you will have a system that is as a whole reactive. But there is nothing forcing you to do this. You can have only one of the services as a Reactive application. It comes down to your needs and context.
I'm trying to understand Spring WebFlux. The things I've found so far are reactive at the core, no Servlet API, no thread per request, HTTP 2, server pushes, application/stream+json.
But what is the difference between asynchronous calls in Spring MVC? I mean in Spring MVC when you return Future, DefferedResult and etc you get logic in the request handler (controller method) executed in a separate thread, so you can benefit from saving thread pool resources for dispatching requests as well.
So could you please highlight differences related to that? Why WebFlux is better here?
Thank you for your time very much!
The Servlet async model introduces an async boundary between the container threads (1 Servlet request/thread model) and the processing of the request in your application. Processing can happen on a different thread or wait. In the end, you have to dispatch back to a container thread and read/write in a blocking way (InputStream and OutputStream are inherently blocking APIs).
With that model, you need many threads to achieve concurrency (because many of those can be blocked waiting for I/O). This costs resources and it can be a tradeoff, depending on your use case.
With non-blocking code, you only need a few threads to process a lot of requests concurrently. This is a different concurrency model; like any model, there are benefits and tradeoffs coming with it.
For more information about that comparison, this Servlet vs. Reactive stacks talk should be of interest.
Servlet API is blocking I/O which requires 1 thread per HTTP request. Spring MVC async relies on Servlet APIs which only provides async behavior between container threads and request processing threads but not end to end.
Spring WebFlux on the other hand achieves concurrency by a fixed number of threads by using HTTP sockets and pushing chunks of data at a time through the sockets. This mechanism is called event loop, an idea made popular by Node.js. Such an approach is scalable and resilient. Spring 5's spring-webflux uses the event loop approach to provide async behavior.
More can be read from
Servlet vs. Reactive
Spring Boot performance battle
Comparing WebFlux with Spring Web MVC
Currently experimenting reactive programming with Spring 5.0.0.RC2, Reactor 3.1.0.M2 and Spring Boot 2.0.0.M2.
Wondering about the concurrency and threading model used by WebFlux and Reactor to properly code the application and handle the mutable state.
The Reactor doc states that the library is considered concurrency agnostic and mentions the Scheduler abstraction. The WebFlux doc does not give information.
Yet when using WebFlux through Spring Boot, a threading model is defined.
From my experimentations here is what I got:
The model is neither 1 event thread, nor 1 event thread + workers
Several thread pools are used
"reactor-http-nio-3" threads: probably one per core, handle the incoming HTTP requests
"Thread-7" threads: used by async requests to MongoDB or HTTP resources
"parallel-1" threads: one per core, created by Schedulers.parallel() from Reactor, used by delay operators and such
Shared mutable state must be synchronized by the application
ThreadLocal (for application state, MDC logging, etc) are not request scoped, so are not very interesting
Is this correct ? What is the concurrency and threading model of WebFlux: for example what are the default thread pools?
Thank you for the information
After the question, the present documentation does provide some clues about the concurrency model and the threads one could expect (but I still think that clearer/better descriptions of what happens under-the-scene from a multi-threading perspective would be highly appreciated by Spring newcomers).
It discusses the difference between Spring MVC and Spring WebFlux (1-thread-per-request model vs. event-loop):
In Spring MVC, and servlet applications in general, it is assumed that applications may block the current thread, e.g. for remote calls, and for this reason servlet containers use a large thread pool, to absorb potential blocking during request handling.
In Spring WebFlux, and non-blocking servers in general, it is assumed that applications will not block, and therefore non-blocking servers use a small, fixed-size thread pool (event loop workers) to handle requests.
Invoking a Blocking API
But notice that Spring MVC apps can also introduce some asynchronicity (cf., Servlet 3 Async). And I suggest this presentation for a discussion wrt Servlet 3.1 NIO and WebFlux.
Back to the docs: it also suggests that, when working with reactive streams, you have some control:
What if you do need to use a blocking library?
Both Reactor and RxJava provide the publishOn operator to continue
processing on a different thread.
(For more details on this, refer to scheduling in Reactor)
It also discusses the threads you may expect in WebFlux applications (bold is mine):
Threading Model
What threads should you expect to see on a server running with Spring WebFlux?
On a "vanilla" Spring WebFlux server (e.g. no data access, nor other optional dependencies), you can expect one thread for the server, and several others for request processing (typically as many as the number of CPU cores). Servlet containers, however, may start with more threads (e.g. 10 on Tomcat), in support of both servlet, blocking I/O and servlet 3.1, non-blocking I/O usage.
The reactive WebClient operates in event loop style. So you’ll see a small, fixed number of processing threads related to that, e.g. "reactor-http-nio-" with the Reactor Netty connector. However if Reactor Netty is used for both client and server, the two will share event loop resources by default.
Reactor and RxJava provide thread pool abstractions, called Schedulers, to use with the publishOn operator that is used to switch processing to a different thread pool. The schedulers have names that suggest a specific concurrency strategy, e.g. "parallel" for CPU-bound work with a limited number of threads, or "elastic" for I/O-bound work with a large number of threads. If you see such threads it means some code is using a specific thread pool Scheduler strategy.
Data access libraries and other 3rd party dependencies may also create and use threads of their own.
In part, you can configure the details of the threading model via configuration
To configure the threading model for a server, you’ll need to use server-specific config APIs, or if using Spring Boot, check the Spring Boot configuration options for each server. The WebClient can be configured directly. For all other libraries, refer to their respective documentation.
Moreover, as e.g. the discussion Default number of threads in Spring boot 2.0 reactive webflux configuration
highlights,
The default number of threads for request handling is determined by the underlying web server; by default, Spring Boot 2.0 is using Reactor Netty, which is using Netty's defaults
it is a matter of default components and their defaults (and overall configuration, including that injected transparently through annotations) -- which may also change across versions of Spring/Boot and corresponding dependencies.
Said that, your guesses seem correct.
I have a Spring Boot web application implementing the API Gateway pattern in which the embedded Tomcat instance receives requests and forwards them to a number of microservices (also Spring Boot applications). I'm using Spring Remoting with AMQP to establish communication and some of the calls to these services may take a while to complete (the most expensive one takes, say, 1-2 seconds).
I've successfully configured listener concurrency on these microservices and everything is working smooth, but now I'm wondering what the default behaviour of the client is. Are calls to those microservices made synchronously or asynchronously? And, in case they are synchronous by default, how can I make them asynchronous so that these calls don't block the "Tomcat" thread in which they're being made (making it available to process other requests while waiting for the response from the services)?
The 1.6 release introduces a new AsyncRabbitTemplate. When calling the sendAndReceive() (and convertSendAndReceive()) methods, a ListenableFuture is returned with which you can register a callback to receive the reply.