If I have a rest service then, I know for sure that each request is treated by a separate thread and the threads can run in parallel.
What happens if I have a rest(http) service as inbound channel in spring integration. Will each request still be treated in parallel or the requests will be placed in queues... and it will be more like single threaded
Normal channels (DirectChannel) use the same execution thread as the object that put something into the channel (they are basically a way of abstracting a method call), so they are multi threaded.
From the docs:
In addition to being the simplest point-to-point channel option, one
of its most important features is that it enables a single thread to
perform the operations on "both sides" of the channel. For example, if
a handler is subscribed to a DirectChannel, then sending a Message to
that channel will trigger invocation of that handler's
handleMessage(Message) method directly in the sender's thread, before
the send() method invocation can return.
Edit
You have a very good point in your question. When you set a Queue element in a channel, spring automatically converts it to a QueueChannel (documentation), and as far as I can remember only one thread will be able to consume from the queue at at time. If you want "real" queue semantics (several producer and consumer threads) you can use an ExecutorChannel
When using Rest (http), the threading is managed by the servlet container; containers support multiple concurrent requests but setting the concurrency is done in the container, not Spring Integration.
With default Direct channels, the container threads will invoke the Spring Integration flow concurrently on the container threads.
Related
On web scene, I use SpringBoot handle requests, for each request, main thread send several requests to other servers to acquire data (this step use treadPool to async), finally main thread "get" all data and return.
So I wonder what are advantages of Vert.x on this scene? Both of them use multi threads to async tasks, will performance be better if I replace the threadPool with vert.x?
Is it not the same, Vert.x does not create a new Thread for each request.
Vert.x has something called Event loop which are managed using few Java threads.
The number of threads used by Vert.x increases as the number of Verticles that you deploy increases, but it will still be very lower in Count.
Also the code in Vert.x is Reactive is executed based on events, so if you are waiting for a event (for example Http Response) you code will be invoked when there is a response available and there is not thread blocked for response.
Check this for more information:
https://alexey-soshin.medium.com/understanding-vert-x-event-loop-46373115fb3e
We are designing an Orchestrator System in java that will host a web service and on a request to it will invoke a flow written in XML which are nothing but steps that are executed one after another but the XML tell the user what the flow is and he can also change the flow by changing the XML. He can add a new service and add it to the XML. But while designing I am now confused with things like.
Should I make a service a runnable with a blocking queue and keep it alive all the time by scheduling it to the executor so when the new request arrives I will push the request in the blocking queue and the service will process it. And create a Mailbox which will carry the message passing task between different services.
Instead of Making service runnable I should use spring IOC that will make the class singleton thus only one instance will be there and I will send a request directly to the service methods thus there will be no hassle that I have to do as there are no threads and also didn't need to implement any mailbox.
I read about how event driven system is faster like nodejs and ngnix so I was thinking to use disuptor framework to push all the incoming request to the ringbuffer and then write a handler that will emit the event to the orchestrator that will start processing the request and also send a callback with the request so that when the orchestrator is done it will send back the response back to the caller using callback. But as the request is not of the same type it so I would not be able to take advantage of disruptor object allocations.
The system needs to provide maximum throughput with low latency, the system will add new services/flows to XML in future so it should adopt the new flows without hitting the performance. I am allowed to only use java 7, no Scala so I am bounded.
Answer #1 is a terrible idea. You will effectively tie up a thread per service. If the number of services exceeds the number of threads backing the executor service you have an instant, automatic DOS. Plus, if the services are inter-dependent on each other... all the ways in which you can dead lock. Plus, the inefficiency of using N threads if only M (< N) are actually required.
Answer #2: if the proposed flow is Request Parsing -> Dispatch -> Service Processing -> Callbacks you rely on the actual 'services' not to foul up because that will prevent callbacks from being called and/or DOS the application. Essentially: what happens if an exception occurs in a service? Will that also impact future requests to the same service and/or other services?
Also the opportunities for parallelism are limited to the framework's way of handling incoming requests. Meaning if you have X requests and the framework inherently processes them serially, you get a backlog of X requests. Your latency requirements may be hard to meet in such a scenario.
Answer #3: an event driven system is indeed the better approach: have a scheduler farm out jobs to an executor service to allow the system to distribute the total load of all services dynamically and have a mechanism to generate and react on events to handle the control flow. This scales better if the number of services become very large and each 'job' is reasonably substantial (so the overhead of scheduling/dispatch is low compared to the actual work being performed).
I am developing a stateless Agent in Java that takes informations from one Server and transfer it to another client. It means that the agent is located between a client and a server. So I am thinking to run two threads simultaneously on the agent: one thread (thread1) runs a serverSocket and get request from client while another threads (thread2)is runnning and makes communication with the server. The problem consists in synchronizing between the two threads. I am thinking in making thread 1 asking whole the time thread 2 about a new Information. If thread 2 has nothing new, he will not answer it. What is the best way to synchronize between them. Should I use a global variable (a flag) to synchronize between them? Can I save Information when I have a stateless agent?
I think you should modify your app into async model.
Your app needs:
- an entry point to accept incoming connections -> a good example is an async servlet (or one dedicated thread).
- a ThreadPoolExecutor that provides fixed numbers of workers and a blocking queue (use this constructor).
The workflow:
Accept incomming request.
Wrapp incoming request into (Runnable) task.
Put task into blocking queue.
If ThreadPoolExecutor has a free worker starts processing the task
An advantage of such a model is that you are able to handle one request using one thread. So there is no need to manually synchronize anything.
I'd like a quick confirmation of what I suspect this part of the RabbitMQ documentation says:
Callbacks to Consumers are dispatched on a thread separate from the thread managed by the Connection. This means that Consumers can safely call blocking methods on the Connection or Channel, such as queueDeclare, txCommit, basicCancel or basicPublish.
Each Channel has its own dispatch thread. For the most common use case of one Consumer per Channel, this means Consumers do not hold up other Consumers. If you have multiple Consumers per Channel be aware that a long-running Consumer may hold up dispatch of callbacks to other Consumers on that Channel.
I have various commands (messages) coming in through a single inbound queue and channel which has a DefaultConsumer attached to it. Is it correct to assume that there is a threadpool in DefaultConsumer that lets me run application logic straight off the consumer callback method, and I'm not blocking the processing of later commands? And that if it seems like there's a bottleneck, I can just give RMQ a bigger threadpool?
In addition, occasionally there is a basicPublish to the same channel from other threads. I take it that this does hold up the consumers? I guess I should make use of a new channel when doing this?
The thread pool you mentioned is not a part of DefaultConsumer but rather a part of Connection that is shared between its Channels and DefaultConsumers. It allows different consumers be invoked in parallel. See this part of the guide.
So you would expect that by increasing size of the thread pool you can reach higher level of parallelism. However that's not the only factor that influences it.
There's a big caveat: incoming messages flowing though a single channel are processed serially no matter how many threads you have in the thread pool. It's just the way how ConsumerWorkService is implemented.
So to be able to consume incoming messages concurrently you have either to manage multiple channels or to put those messages into a separate thread pool.
Publishes do not use threads from the Connections's thread pool so they do not hold up consumers.
For more details you may check this post.
I am trying to implement a custom inbound channel adapter in spring integration to consume messages from apache kafka. Based on spring integration examples, I found that I need to create a class that implements MessageSource interface and implement receive() method that would return consumed Message from kafka. But based on consumer example in kafka, the message iterator in KafkaStream is backed by a BlockingQueue. So if no messages are in the queue, the thread will be blocked.
So what is the best way to implement a receive() method as this method can potentially block until there is something to consume.. ?
In more general sense, How do we implement a custom inbound channel for streaming message sources that blocks until there is something ready to consume..?
The receive() method can block (as long as the underlying operation responds properly to an interrupted thread), and from an inbound-channel-adapter perspective, depending on the expectations of the underlying source, it might be preferable to use a fixed-delay trigger. For example, "long polling" can simulate event-driven behavior when a very small delay value is provided.
We have a similar situation in our JMS polling MessageSource implementation. There, the underlying behavior is handled by one of the JmsTemplate's receive() methods. The JmsTemplate itself allows configuration of a timeout value. That means, as an example, you may choose to block for 5-seconds max but then have a very short delay trigger between each blocking receive call. Alternatively, you can specify an indefinite receive timeout. The decision ultimately depends on the expectations of the underlying resource, message throughput, etc.
Also, I wanted to let you know that we are exploring Kafka adapters ourselves. Perhaps you would like to collaborate on this within the spring-integration-extensions repository?
Regards,
Mark