I'd like to know the difference between corePoolSize and throttle-limit as Spring Batch attributes defining multi threading configuration.
I've got the difference between corePoolSize and maxPoolSize thanks to this post "What is the difference between corePoolSize and maxPoolSize in the Spring ThreadPoolTaskExecutor"
But my issue concerns corePoolSize vs throttle-limit... I found that it's preferable to define CorePoolSize = Throttle-limit, but I'm wondering... if I define for example :
CorePoolSize = 100 and Throttle-limit = 200... What happens ? Is a 200 sized thread pool that will be created or 100 ?
Thank you for any clarification...
The core pool size says a thread pool executor will start with N number of threads. A throttle-limit T says that, regardless of the number of threads available in the thread pool, only use T of those threads for a tasklet.
So you can have a thread pool with a core pool size of 8 and two tasklets with throttle limit of 4 in that case you will be utilizing your thread pool. But if you only have one tasklet with a throttle limit of 4 you will be utilizing one half of the thread pool.
throttle-limit is explained in the spring batch javadocs in TaskExecutorRepeatTemplate.
This is another property on top of all the possible ways to configure a ThreadPoolExecutor and probably simplifies the task of using a thread pool. It's certainly useful if you have parallel steps competing for threads.
To answer your question, new threads will be created for your tasks if your throttle-limit is greater than corePoolSize only if maximumPoolSize is greater than corePoolSize and the blocking queue in your ThreadPoolExecutor is full. Otherwise they will e.g. queue up in your ThreadPoolExecutor or possibly be rejected with an exception (see the default handler AbortPolicy).
What happens when spring batch tries to submit more tasks to the executor than it is willing to accept depends on your executor. See Rejected tasks in the ThreadPoolExecutor javadocs.
It's all a bit confusing at first, but makes perfect sense in the end.
Note that DEFAULT_THROTTLE_LIMIT is 4 (in my spring version).
See for example this gotcha when your ThreadPoolExecutor is unexpectedly not creating any more than corePoolSize Threads.
Related
At the moment I'm creating fixed thread pool using the Executor service like
executor = Executors.newFixedThreadPool(coreAmount);
While this is fine, I was wondering if it's possible to keep the behaviour of creating a number of threads and change the max pool limit so that if all the threads are in use to create a new thread and use it instead of waiting for one of the threads to terminate to start.
So for example if 8 threads are created and are being used, 9th task enters I want it to create a new thread in addition to the 8 currently in use.
It seems newCachedThreadPool() has the behaviour but I also want the ability to create number of threads similar to newFixedThreadPool(int nThreads)
Maybe you can use the ThreadPoolExecutor class. It is an ExecutorService and has the concept of core pool count and max pool count. It also has other features that make it more customizable than the Objects returned by Executors.
Below is an example.
int coreAmount = 8;
ExecutorService executor;
//executor = Executors.newFixedThreadPool(coreAmount);
int overrun = 4;
int maxWorkCount = 1_000;
executor = new ThreadPoolExecutor(coreAmount, coreAmount + overrun, 1, TimeUnit.MINUTES, new ArrayBlockingQueue<>(maxWorkCount));
Here is more info about the params passed in the constructor in the example above.
corePoolSize - the number of
threads to keep in the pool, even if they are idle, unless
allowCoreThreadTimeOut is set
maximumPoolSize - the maximum number of
threads to allow in the pool
keepAliveTime - when the number of
threads is greater than the core, this is the maximum time that excess
idle threads will wait for new tasks before terminating.
unit - the time unit for the keepAliveTime argument
workQueue - the queue to use for holding tasks before they are executed. This queue will hold only the Runnable tasks submitted by the execute method.
Like you said, a cached thread pool is exactly what you're looking for. From it's documentation,
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources.
(Emphasis: mine)
So for example if 8 threads are created and are being used, 9th task enters I want it to create a new thread in addition to the 8 currently in use.
This is exactly the case with Executors#newCachedThreadPool, as seen from its documentation above.
Here is what you can use if you want to emulate a cached thread pool, but with a minimum amount of 8 threads:
ExecutorService service = new ThreadPoolExecutor(8, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
This is available in the ThreadPoolExecutor using Core Pool & Maximum Pool size values:
From the javadoc, you can see that:
If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.
By setting corePoolSize and maximumPoolSize the same, you create a fixed-size thread pool.
By setting maximumPoolSize to an essentially unbounded value such as Integer.MAX_VALUE, you allow the pool to accommodate an arbitrary number of concurrent tasks.
Most typically, core and maximum pool sizes are set only upon construction, but they may also be changed dynamically using setCorePoolSize(int) and setMaximumPoolSize(int).
So, in your example, you would need to set the 'corePoolSize' to the value of 8. You would then need to set the 'maximumPoolSize' which would then serve as the upper-bound to the pool.
Also, as in the javadoc, these values can be altered dynamically.
In the Oracle site (http://docs.oracle.com/javase/7/docs/api/?java/util/concurrent/ThreadPoolExecutor.html) under Queuing section, its mentioned that "If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread."
Lot of our code have corePoolSize as 1 and maximumPoolSize are like 10 or 20.
(1) Is it a bad practice? or rather not an optimal way to use ThreadPool's?
(2) Would you recommend have corePoolSize and maximumPoolSize value be same when creating a ThreadPoolExecutor?
Another question is with respect to usage of a BlockingQueue while creating a ThreadPoolExecutor - between LinkedBlockingQueue or ConcurrentLinkedQueue? I was looking more from the locking and performance point-of-view? Though running a test proves, inserting in a concurrentLinkedQueue is quite fast compared to other. Any thoughts?
Edited:
The first part has been answered in various questions. The one I liked
How to get the ThreadPoolExecutor to increase threads to max before queueing?
Please find the bellow points .
If a new task is submitted to the list of tasks to be executed, and
less than corePoolSize threads are running, a new thread is created
to handle the request. Incoming tasks are queued in case
corePoolSize or more threads are running.
If a request can’t be queued or there are less than corePoolSize
threads running, a new thread is created unless this would exceed
maximumPoolSize.
If the pool currently has more than corePoolSize threads, excess
threads will be terminated if they have been idle for more than the
keepAliveTime.
If a new task is submitted to the list of tasks to be executed and
there are more than corePoolSize but less than maximumPoolSize
threads running, a new thread will be created only if the queue is
full. This means that when using an Unbounded queue, no more threads
than corePoolSize will be running and keepAliveTime has no
influence.
If corePoolSize and maximumPoolSize are the same, a fixed-size
thread pool is used
Your ThreadPool Size
Core Pool size is 1.
Max Pool size is 10 or 20.
Question 1 : Is it a bad practice? or rather not an optimal way to use ThreadPool's?
It is based on your Pool which is used.
If you are using bounded queue , new thread will be created upto max size once it reaches the max limit of bounded queue. Otherwise it will run in core pool size . If you are using unbounded queue it will be Core pool size only.
Question 2 : Would you recommend have corePoolSize and maximumPoolSize value be same when creating a ThreadPoolExecutor?
Yes. So that only you can get same no of threads instead of what kind blocking queue . Even non blocking queue also you can get max no of threads.
Question 3 concurrentLinkedQueue vs LinkedBlockingQueue
You should not allow to use concurrentLinkedQueue since threadpool only supports BlockingQueue. So you may try concurrentLinkedBlockingQueue. Because concurrentLinkedBlockingQueue is unbounded. But LinkedBlockingQueue is bounded.
Well that should not happen, the reason TPE puts the request(Runnable) in queue is that it should not consume too much memory. But it guarantees the execution of all Runnables.
The purpose of thread pool executor is to optimize Memory.
If you want to increase pool size to max before adding in queue then I think you would have to extend the Thread Pool Executor.
This question already has answers here:
What is the difference between corePoolSize and maxPoolSize in the Spring ThreadPoolTaskExecutor
(7 answers)
Closed 5 years ago.
When a new task is submitted in method execute(java.lang.Runnable),and fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle.
1) Why there is a need to create a new thread to handle the request if there are idle threads?
If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.
2) I don't understand the difference between corePoolSize and maximumPoolSize here. Secondly, how can a queue be full when threads are less than maximumPoolSize? Queue can only be full if threads are equal to or more than maximumPoolSize. Isn't it?
Here are Sun’s rules for thread creation in simple terms:
If the number of threads is less than the corePoolSize, create a new Thread to run a new task.
If the number of threads is equal (or greater than) the corePoolSize, put the task into the queue.
If the queue is full, and the number of threads is less than the maxPoolSize, create a new thread to run tasks in.
If the queue is full, and the number of threads is greater than or equal to maxPoolSize, reject the task.
Full article
You can find the definition of the terms corepoolsize and maxpoolsize in the javadoc. http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ThreadPoolExecutor.html
The link above has the answer to your question. However, just to make it clear. The application will creating threads until it reaches the corePoolSize. It means that these number of threads should be sufficient to handle the inflow of tasks. After that the tasks will be queued. Once the queue is full the executor will start creating new threads. It is kind of balancing. What it essentially means is that the inflow of tasks is more than the processing capacity. So, Executor will start creating new threads again till it reaches Max number of threads. Again, a new threads will be created if and only if the queue is full.
Core and maximum pool sizes
A ThreadPoolExecutor will automatically adjust the pool size according to the bounds set by corePoolSize and maximumPoolSize .
When a new task is submitted in method execute(java.lang.Runnable), and fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle. If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.By setting corePoolSize and maximumPoolSize the same, you create a fixed-size thread pool.
By setting maximumPoolSize to an essentially unbounded value such as Integer.MAX_VALUE, you allow the pool to accommodate an arbitrary number of concurrent tasks. Most typically, core and maximum pool sizes are set only upon construction, but they may also be changed dynamically using setCorePoolSize(int) and setMaximumPoolSize(int). link
I'm new to ScheduledThreadPoolExecutor (as I usually use the simple Timer, but people have been advising against it), and I don't quite understand what would be the appropriate integer value to pass to the ScheduledThreadPoolExecutor(int) constructor.
Could anyone explain this?
Thank you
In case of ScheduledThreadPoolExecutor, corePoolSize is maximum number of threads that will be created to perform scheduled actions.
This thread pool is fixed-sized and idle threads are kept alive.
DrunkenRabbit's answer is simply ivalid because ScheduledThreadPoolsExecutor docs says explicitly that (There will be no thread count spikes at all):
While this class inherits from ThreadPoolExecutor, a few of the
inherited tuning methods are not useful for it. In particular, because
it acts as a fixed-sized pool using corePoolSize threads and an
unbounded queue, adjustments to maximumPoolSize have no useful effect.
Now as for the value, reasonable number would be number of CPU cores that application is running on.
Essentially, the corePoolSize is the number of Threads to maintain in the pool.
eg. if you expect 10 concurrent requests on a regular basis, but peaks of 20. The corePoolSize should be 10 and max as 20. This way the executor will create up to 10 new threads, even if idle threads are available for use.
As stated in the JavaDocs
Core and maximum pool sizes
A ThreadPoolExecutor will automatically adjust the pool size (see getPoolSize()) according to the bounds set by corePoolSize (see getCorePoolSize()) and maximumPoolSize (see getMaximumPoolSize()). When a new task is submitted in method execute(java.lang.Runnable), and fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle. If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full. By setting corePoolSize and maximumPoolSize the same, you create a fixed-size thread pool. By setting maximumPoolSize to an essentially unbounded value such as Integer.MAX_VALUE, you allow the pool to accommodate an arbitrary number of concurrent tasks. Most typically, core and maximum pool sizes are set only upon construction, but they may also be changed dynamically using setCorePoolSize(int) and setMaximumPoolSize(int).
I'm using Executors.newScheduledThreadPool() to create a ScheduledExecutorService, specifying the number of threads like so:
int corePoolSize = 42;
ScheduledExecutorService foo = Executors.newScheduledThreadPool(corePoolSize);
According to the JavaDocs, the corePoolSize argument sets
the number of threads to keep in the pool, even if they are idle.
Does this mean that this ExecutorService implementation may create more than corePoolSize threads as needed, similar to a cached thread pool?
No. The correct answer is no, a ScheduledExecutorService will not spawn new threads.
See answer here
Does this mean that this ExecutorService implementation may create more than corePoolSize threads as needed?
Yes, that is exactly what it means. The reason for the existence of corePoolSize is the expense of thread-creation. If you expect to be firing large numbers of short-lived tasks at your executor service, you may expect to find, at a given point in time, large numbers of idle threads.
Rather than have these threads die, only to be replaced a very short time later by newly-created threads, corePoolSize will ensure that there will always be a number of spinning threads.