When creating a thread pool with
Executors.newScheduledThreadPool(42);
I can schedule tasks in it as the thread pool is of type ScheduledExecutorService. ScheduledExecutorService is a subclass of ExecutorService. I therefore can submit normal Runnables or Callables.
Is it advisable to share one single thread pool in the application or would it be better to have two separate ones?
The scheduled tasks are not that time critical and their execution on time cannot be guaranteed even with a separate thread pool if there are two many waiting threads.
If having two separate thread pools, what is a good size for them based on
Number of tasks scheduled (you can assume that the number is constant)
Number of CPU/cores.
Related
I have 5 #Scheduled annotated methods and pool size of my ThreadPoolTaskScheduler is 10. Annotations of my methods are the same and like this.
#Scheduled(fixedDelay = 1000, initialDelay = 10000)
My question is that;
When one of the scheduled methods gets a thread from pool and start to run; does it release the thread to pool before its execution is finished? (for example in case of context switch etc.) or this thread is used until the end of the execution?
I mean is it possible to some part of the scheduled task is done with thread-1 and some part is done with thread-2 ?
Threading is complicated and my understanding is not as great as others, but here's my attempt at a brief explanation of how the #Scheduled Spring annotation works:
Spring uses a TaskScheduler:
public interface TaskScheduler {
ScheduledFuture schedule(Runnable task, Trigger trigger);
ScheduledFuture schedule(Runnable task, Date startTime);
ScheduledFuture scheduleAtFixedRate(Runnable task, Date startTime, long period);
ScheduledFuture scheduleAtFixedRate(Runnable task, long period);
ScheduledFuture scheduleWithFixedDelay(Runnable task, Date startTime, long delay);
ScheduledFuture scheduleWithFixedDelay(Runnable task, long delay);
}
https://docs.spring.io/spring/docs/3.2.x/spring-framework-reference/html/scheduling.html#scheduling-task-scheduler
Which submits the annotated code, i.e. task code, to a high-level concurrency object called an executor. The executor class is ThreadPoolTaskExecutor. That class submits tasks to the thread pool to be run by the first available thread in the pool. The thread pool size you set determines how many active threads you can have. If you set allowCoreThreadTimeOut to true then threads in the pool that have no work available to do within their timeout interval will be terminated.
Spring uses a ThreadPoolTaskExecutor to manage the thread pool:
https://github.com/spring-projects/spring-framework/blob/master/spring-context/src/main/java/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.java
Keeping a pool of threads alive reduces the time that would normally be added while waiting for the thread to be created. See this question for some more info.
Ultimately, the java.lang.Thread class runs the Runnable or Callable instances that are created by the ThreadPoolTaskExecutor. The Thread class implements a run() method that is basically your code you want the thread to run:
public Thread(Runnable target) {
init(null, target, "Thread-" + nextThreadNum(), 0);
}
private void init(ThreadGroup g, Runnable target, String name,
long stackSize, AccessControlContext acc) {
...
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/Thread.java
The actual switching between threads, i.e. the context switch is OS-dependent but in general threads will be divided amongst CPUs and then each CPU cycles through the threads based on the timeout interval and does a bit of work and then pauses and switches continuously between threads until the task(s) are complete.
does it release the thread to pool before its execution is finished?
(for example in case of context switch etc.) or this thread is used
until the end of the execution?
The Runnable code can definitely stop executing in the middle of an operation but the Threads in a thread pool are usually kept alive until there's no more work to be done.
Here's more info from the Oracle documentation that explains thread pools:
Most of the executor implementations in java.util.concurrent use
thread pools, which consist of worker threads. This kind of thread
exists separately from the Runnable and Callable tasks it executes and
is often used to execute multiple tasks.
Using worker threads minimizes the overhead due to thread creation.
Thread objects use a significant amount of memory, and in a
large-scale application, allocating and deallocating many thread
objects creates a significant memory management overhead.
One common type of thread pool is the fixed thread pool. This type of
pool always has a specified number of threads running; if a thread is
somehow terminated while it is still in use, it is automatically
replaced with a new thread. Tasks are submitted to the pool via an
internal queue, which holds extra tasks whenever there are more active
tasks than threads.
An important advantage of the fixed thread pool is that applications
using it degrade gracefully. To understand this, consider a web server
application where each HTTP request is handled by a separate thread.
If the application simply creates a new thread for every new HTTP
request, and the system receives more requests than it can handle
immediately, the application will suddenly stop responding to all
requests when the overhead of all those threads exceed the capacity of
the system. With a limit on the number of the threads that can be
created, the application will not be servicing HTTP requests as
quickly as they come in, but it will be servicing them as quickly as
the system can sustain.
In my code, I have a class containing a static final variable
private static final ForkJoinPool pool = new ForkJoinPool(availableCPUs - 1);
I have a long running task submitted to the pool, which would take all the CPU resource. Any other tasks submitted would be hanging.
However, when I switched to create a common pool
private static final ForkJoinPool pool = ForkJoinPool.commonPool();
All the tasks can be submitted and executed.
I was just wondering what the differences are between these two pieces of code. commonPool() still calls new ForkJoinPool() and passes the availableCPUs - 1
Also I noticed that commonPool() uses a factory of type SafeForkJoinWorkerThreadFactory while new ForkJoinPool() uses ForkJoinPool$DefaultForkJoinWorkerThreadFactory. Does this matter?
Thank you very much!
I think I figured it out.
ForkJoin maintains two types of queues: one general inbound queue and worker thread queues for each worker thread. All worker threads will fetch from the general inbound queue first and populate their worker threads. After one worker thread finishes all the tasks in its worker queue, it will try to steal from other worker threads. If there no other task to steal from other worker threads, the work thread will fetch from the general inbound queue again.
However, with common pool, the main thread will also help to process the tasks. The main thread does not have a worker queue though. Therefore, after finishing one task, the main thread will be able to fetch from general inbound queue.
Since by default, the ForkJoin queues are LIFO, the main thread will be able the fetch the last submitted tasks.
Documentation says:
The common pool is by default constructed with default parameters.
ForkJoinPool()
Creates a ForkJoinPool with parallelism equal to Runtime.availableProcessors(), using the default thread factory, no UncaughtExceptionHandler, and non-async LIFO processing mode.
So what makes you think that new ForkJoinPool(availableCPUs - 1) and ForkJoinPool.commonPool() would be pools of the same size?
If you only have 2 CPUs, then availableCPUs - 1 means you're creating a pool of 1 thread, i.e. it can only process one task at a time, so a long-running task will block all other tasks.
But with 2 CPUs, availableProcessors() means you're getting a common pool with 2 threads, i.e. it can process other tasks while a single long-running task is being processed.
Is it possible to have a set of thread pools that share threads from a large thread pool instead of creating new thread?
In our RESTful API application, a request may involve several parallel tasks. To improve the performance, we want to execute parallel tasks in a thread pool which has a fixed number (say 200) of threads. But we also want to limit the max number of threads that each request can use. So I am thinking if it is possible to create a sub thread pool with a maximum pool size for each request, which does not create thread by itself but tries to fetch a new thread from the global thread pool and put job into queue if there is no available threads.
Has anyone done something similar? Or is there any other better solution?
Thanks!!
Instead of thread pools, think of executors. An executor consists of 2 things: a job queue and a thread pool. What you need is a light-weight executor for each request which has a job queue but has no threads. Instead, it passes jobs from its queue to the main executor (which owns all the threads). The trick is that the light-weight executor counts how many jobs it has submitted to the main executor, and stops passing jobs when the number exceeds the limit. Each job, before being passed to the main executor, is wrapped in an object of type Runnable which a) has the reference to the parent light-weight executor, b) executes the wrapped job and c) notifies the referenced executor when the job is finished so that the executor can pass another job to the main executor (or just decrease the job counter, if there is no jobs in the queue).
You could create a thread pool for every task e.g. Executors.newFixedThreadPool(10) This will do what you ask for with the inefficiency of potentially creating threads that a particular task instance doesn't need.
As we create a Thread pool using Java's Executor service and submit threads to this thread pool, what is the order in which those threads get executed?
I want to ensure that threads submitted first, execute first.
For example, in the code below, I want first 5 threads to get executed first, followed by the next 5 threads and so on...
// Create a thread pool of 5 threads.
ScheduledExecutorService exService = Executors.newScheduledThreadPool(5, new ModifiedThreadFactory("ReadThreadPool"));
// Create 100 threads.
MyThread[] threads = createMyThreads(100);
// Submit these 100 threads to thread pool for execution.
for(MyThread thread : threads) {
exService.submit(thread);
}
Does Java's Thread Pool provide any API for this purpose, or do we need to implement a FIFO queue at our end to achieve this.
If Java's thread pool does not provide any such functionality, I am really interested to understand the reason behind the non-existence of this functionality as it appears like a very common use-case to me.
Is it technically not possible (which I think is quite unlikely), or is it just a miss?
That's the default behavior. ScheduledThreadExecutor (that you're using although you're not scheduling anything) extends from ThreadPoolExecutor. Tasks submitted to a ThreadPoolExecutor are stored in a BlockingQueue until one thread is available to take them and execute them. And queues are FIFO.
This is decscribed in details in the javadoc.
Threads do not get executed. Threads are the entities running taska like Runnable and Callable . Submiting such a task to a executor service will put it in it's inner BlockingQueue until it gets picked up by a thread from it's thread pool. This will still tell you nothing about the order of execution as different classes can do different things while implementing Runnable
I have a CPU intensive application, which can be written in Java. The application consists of few jobs (threads) that run independently.
If I start all the threads at once, the system will be overloaded. How could I start at most n threads at once, and when one thread finishes then a new one is started? By limiting the number of threads running at once, I intend to leave some other processors/cores available for other tasks.
Thanks
You should formulate your threads' tasks as Runnables or Callables and then submit them to a fixed thread pool executor. The executor will manage a pool of worker threads and run your tasks from an internal queue on those threads.
See Executors#newFixedThreadPool for a factory method that creates the type of executor you want.
Use a fixed size executor pool.
ExecutorService executor = Executors.newFixedThreadPool(4);