I am trying to get the basics very strong about thread pool. I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.
Question is: Why not concurrently? Why not just time-slice those 10 tasks?
What is the reason of this implementation?
Why not concurrently? Why not just time-slice those 10 tasks?
You can have a thread pool that is able to perform ten concurrent tasks. You just need to configure it to have at least ten worker threads. "Time-slicing" tasks is what threads do. What thread pools do is:
Allow your program to control the number of threads that it uses to perform "background" tasks, and
Allow your program to re-use threads, which can be much more efficient than creating a new thread for each new task, and then destroying the thread when the task is complete.
In order to "time-slice 10 tasks", those tasks need to be in 10 separate threads that run concurrently.
The time-slicing scheduling algorithm is implemented by the operating system, not by Java. Time slicing applies to threads in Java because Java threads are implemented as native operating system threads: every Java thread has a native thread of its own, and these threads are scheduled by the operating system as it sees fit.
There is no difference between "thread pool threads" and "raw threads" here. If you give an instance of Runnable to a thread (whether it's part of a thread pool or not) it will run from beginning to end, subject to the time slicing scheduling algorithm of the operating system.
So why not use thousands of threads, why even bother with thread pools? It turns out that operating system threads are a relatively expensive and scarce resource, and therefore so are Java threads.
Since operating system threads are so expensive, Project Loom is investigating adding lightweight user space threads to Java. Some of the details in this answer may change when/if loom gets merged into main stream Java.
Some good answers but I thought I'd respond to your questions specifically.
I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.
If you configure your thread pool to have 5 threads (Executors.newFixedThreadPool(5)) then it will start 5 threads to run your jobs. Initially 5 jobs are given to the 5 threads to run concurrently (if your server has 5 CPUs available). Once one of the 5 jobs finishes, a 6th job will be immediately started on the idle thread. This continues until all 10 jobs have been run.
Question is: Why not concurrently? Why not just time-slice those 10 tasks? What is the reason of this implementation?
You can instead use a cached thread pool (Executors.newCachedThreadPool()) if you want which will start a thread for each of the 10 jobs that you submit concurrently. This might work fine for 10 jobs but won't work well with 100,000 jobs – you would not want to start 100,000 threads. We use a fixed thread pool when we want to limit the number of jobs that run concurrently. Even though it seems like running 5 jobs concurrently would always run slower than running 10 jobs at once, but this isn't necessarily the case. There is a cost when the OS time slices between the jobs and the overall job throughput may be faster with 5 threads than 10 depending on how many processors your hardware has. Limiting the number of concurrent jobs also does not stress your server as much and should make your application work better with other running applications.
See my answer here about scaling threads: Concept behind putting wait(),notify() methods in Object class
Related
I have 3 fixedThreadPools in my application with size of 20 each.
I submit the tasks in pools but they are few sometimes say 5 task in each pool1 ,pool2 ,pool 3 so I have 15 threads from each pool sitting idle. I want to know does these idle threads (45 total) will come in context switch and degrade the performance even when no task is submitted to them ?
If the pooled threads have no task, they are in state WAITING, meaning they have nothing to do besides wait for a signal for something to happen (in this case, waiting for a task to be submitted). They don't get scheduled any CPU time because they have nothing to do. The scheduler is smart enough to know this, so it's at this point where it's excluded from competing for resources with other, more active threads.
If a thread is in RUNNABLE state, then it might have something to actually do, and is given CPU time.
So excess threads in a pool don't compete with live threads, but they do take a small amount of native resources (not enough to really matter when talking about 60 threads).
I have 3 fixedThreadPools in my application with size of 20 each.
You mean, you are creating 3 ExecutorService instance of thread pool size 20. That means total 60 threads in pools ?
...15 threads from each pool sitting idle. I want to know does these
idle threads (45 total) will come in context switch..
I think, you are assuming to run all 60 threads at the same time, as you have mentioned 5 thread from each pool (i.e 15 threads) are executing. But remaining 45 threads (15 thread from each pool) are idle. But, that is not correct until they have tasks assign and number of available cores to execute them.
Please note, execution of thread depends on number of available cores (CPU). No matter, how many thread you create, if CPU is not available to execute each thread, they will be at Runnable state to get their turn.
JVM who decides which thread should run and how, mainly uses preemptive or time slicing scheduling to schedule the threads. In that time, if other thread get a new task (or has some task to execute), then they will at Runnable state to get their turn.
Unlike other executors, ForkJoinPool supports work stealing.
Yes, when there are no tasks for execution threads would be waiting, but before that, they try to steal works from pool's queues, and this will consume more CPU, then other executors (FixedThreadPool, CachedThreadPool). And waking up threads also needs time.
ForkJoinPool performs best when it has a lot job. If you don't have this, you better to use FixedThreadPool. Also, you should understand that typically ForkJoinPool, FixedThreadPool are for CPU intensive tasks. Otherwise, if you have IO intensive tasks is better to use CachedThreadPool. You could create one FixedThreadPool at all, but should aware that tasks could block each other.
Also, there is no reason create three ForkJoinPool pools with 60 threads unless you have at least 32 CPUs (but it hugely depends). As you can not utilize pools, that it is likely not your example. But keep in mind, if you run a lot of CPU intensive tasks in thread's count bigger then CPUs, then yes, you likely get worse performance, because of resource sharing, including context switches.
In Java, and more specifically, Android,
are new Threads automatically assigned to a different CPU core than the one I am currently utilizing, or should I take care of that?
Also, does it matter how the new thread is created, using the Thread class or submitting a Runnable to an Executor, that maintans a pool of threads?
There is a similar question here, but the answer goes on to explain how the OP should address his particular problem, rather than diving into the more general case:
Threads automatically utilizing multiple CPU cores?
In Java, and more specifically, Android, are new Threads automatically assigned to a different CPU core than the one I am currently utilizing, or should I take care of that?
The decision of what threads run on what cores is handled by the OS itself (in Android, based off of the Linux scheduler). You cannot affect those decisions yourself; the decisions are automatic and dynamic.
does it matter how the new thread is created, using the Thread class or submitting a Runnable to an Executor, that maintans a pool of threads?
With respect to what cores a thread runs on, the OS neither knows nor cares whether an Executor is involved, or even if the programming language that app was written in has something called Executor.
In Java, and more specifically, Android, are new Threads automatically
assigned to a different CPU core than the one I am currently
utilizing, or should I take care of that?
In Java threads are simply separate sequence of executions, but in Android it is a little more complicated than that. Android creates one main thread per application. This main thread is responsible for the UI and other tasks related to events (queue). For doing background work you have to create separate worker threads.
Simple threads are handled by the Android OS automatically, and they may or may not run on separate cores. If you are running 10 threads, it is quite possible that they all run on one core leaving all other cores idle.
If you need to run more than one threads and you want to run each thread on a separate core you should use ThreadPoolExecutor; it will handle thread creation and map it on number of CPU cores available. You can set various parameters according to your requirement. See what Android is saying:
A ThreadPoolExecutor will automatically adjust the pool size (see
getPoolSize()) according to the bounds set by corePoolSize (see
getCorePoolSize()) and maximumPoolSize (see getMaximumPoolSize()).
When a new task is submitted in method execute(Runnable), and fewer
than corePoolSize threads are running, a new thread is created to
handle the request, even if other worker threads are idle. If there
are more than corePoolSize but less than maximumPoolSize threads
running, a new thread will be created only if the queue is full.
See ThreadPoolExecutor for detail.
does it matter how the new thread is created, using the Thread class
or submitting a Runnable to an Executor, that maintans a pool of
threads?
yes, see the answer above.
Update
By saying "to run each thread on a separate core use ThreadPoolExecutor", I meant that ThreadPoolExecutor can do that if it is used properly in a careful manner.
Java does not map threads directly on the CPU. Java leaves threads schedule (by mapping on to the OS' processes) on OS but how we create threads influence scheduling at OS level. However Java, can assign priority to threads but again it is up to the OS to honor these priorities.
There are various parameters we should consider while creating a thread pool, few are as follows:
i) Threads should be equal in complexity.
ii) Comparing CPU bound tasks with I/O bound, I/O bound task usually need more threads than available core for optimal utilization of CPU
iii) Dependency between threads effect negatively
If threads are created keeping these points in mind, ThreadPoolExecutor can help achieve a 100% of the CPU utilization, meaning one thread per core (if the thread pool's size is equal to the number of cores and no other thread is running). A benefit of ThreadPoolExecutor is that it is cost effective as compare to creating threads separately and it also eliminates context switching which wastes a lot of CPU cycles.
Achieving the 100% of the CPU utilization while making things concurrent, is not an easy task.
Whichever way Threads are created (Either using Thread class or using submitting the task to ThreadPoolExecutor) or task assigned to threads it will not make any impact on OS scheduling.
There is OS component Scheduler involved in this process which takes the responsibility to schedule the tasks or threads among the CPU cores(if cores are more than one) inside the OS.
This decision is taken by scheduler.
If there is only one core in system, Scheduler plays fair with threads by allowing them to do processing for some milliseconds one by one.
I have been working on a project in which my program creates around 500 threads during the execution. I find that my PC starts taking a huge load as soon as the program I execute the program. And it continues showing load after 75% of the threads have completed their job.
I want to know whether the threads whose work has finished were killed or not. And how does java deal with threads which have finished their job. Any help...
I find that my PC starts taking a huge load as soon as the program I execute the program. And it continues showing load after 75% of the threads have completed their job.
If 75% of the 500 threads have completed their job then that leaves 100+ threads that continue to run. 100 threads, if using a good amount of CPU, can more than swamp the processors on a box which I assume does not have 100s of cores. So your application may continue to show 100% CPU utilization until the number of running threads drops below the number of cores.
You should consider using a fixed sized thread pool instead of creating 500 concurrent threads. You then submit 500 tasks to the thread pool. This allows you to choose an appropriate number of threads to be running concurrently. The appropriate number of threads is highly dependent on the tasks being submitted. More CPU bound tasks should use fewer threads while IO bound tasks can use more. Doing some test runs with your application while adjusting the number of threads is the best way to optimize the value. I tend to start with 2 times the number of cores and then optimize from there.
// create a thread pool with 10 workers
ExecutorService threadPool = Executors.newFixedThreadPool(10);
// define your jobs somehow
for (MyRunnable job : jobsToDo) {
threadPool.submit(job);
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
For more details, check out the ExecutorService tutorial.
I want to know whether the threads whose work has finished were killed or not. And how does java deal with threads which have finished their job. Any help...
The threads have most likely finished. After a thread leaves the run() method (either because it returns or throws an exception) it will no longer consume CPU and its underlying native thread can be reaped or reused. If there are no references to the Thread object, its memory will eventually be reaped by the garbage collector.
The JVM will garbage collect the Thread object, as long as there are no references to it, and as long as its run method returns. Thread is dead itself after its run method returns. It might still be in the heap, but it will not have its own stack anymore, and not do anything.
The only possible way that your threads have not been killed is that they still do something or you forgot to clean up references to your thread objects - but this is memory related.
If you allocated your threads through the thread pool, they are returned to the pool after the execution of the task. I this case, they might not be released after the completion of the task.
we should not create many threads to accomplish our task, it will give you many issues like OutofMemoryError. And also creation of thread is a costly task, so we should think of Thread pool i.e. ExecutorService in which we reuse the same threads again and again.
But any ways to answer you question after threads are created they die automatically i.e. it will be garbage collected, you don't need to do anything. Initially java provided methods like stop() or destroy() but these are deprecated for good reason.
You can read about a Thread's lifecycle. If the run method is over then they should not be consuming your cpu.
Threads who have finished their jobs will die. It won't consume any more CPU time.
You can use jstack to check how many threads are active running in your java process.
I'm in a situation where I have 4 groups of 7 threads each. My CPU (core i7) is supposed to be able to handle 8 threads, so I'm considering going through each group one at a time, running the 7 threads, then moving to the 2nd group, running its 7 threads, then 3rd and 4th groups in the same way, and then starting back at 1st group, until user sends a stop command.
My question is, once each group of 7 threads has finished processing, should I keep those threads idle, or shut them down completely and restart a new group of 7 threads at the next iteration? Which method will be faster? This is for a very speed intensive app, so I need everything to happen as quickly as possible.
I will be using a FixedThreadPool to manage each group of 7 threads. So I could either just invokeAll() and then leave them alone (presumably to idle), or I could shutdown() each threadpool after the invokeAll() and start a new thread pool at the next iteration.
Which method will be faster?
My question is, once each group of 7 threads has finished one cycle of processing, should I keep those threads idle, or shut them down completely and restart a new group of 7 threads at the next cycle?
I would use a single ExecutorService thread-pool and reuse the same threads for all tasks. See the tutorial on the subject. A thread-pool is designed to execute any Runnable or Callable class so they are task agnostic. For example, you might have ParentResult and ChildResult classes. You can submit a Callable<ParentResult> to the thread-pool which will return a Future<ParentResult> and you can submit a Callable<ChildResult> to the same thread-pool which will return a Future<ChildResult>.
The only reason why you'd want to have "groups of threads" is if each thread has some state that it must maintain -- a database connection or something. Even then many people use thread-pools since it does much of the concurrency heavy lifting for you.
If you do have to keep this state then I would certainly not shutdown the pools and then restart them later. A dormant thread/pool is taking no system resources aside from memory. The only reason why you would ever do this is if you are forking 100s of thread for the task but at that point, you should consider re-architecting your application.
You need not to schedule your threads manually. Start all 28 threads at once - this would not be slower, but well can be faster.
When you say your processor has 8 threads, I think you mean it has has 4 cores with hyperthreading. Java does not use threads in the same sense as your processor, so those 7 threads are of a different type to your processors.
The JVM handles processor usage, and is (IIRC) limited to using 1 core. The threads java uses are specific to the JVM, and are wholly separate.
As for your actual question, try testing different thread combinations to see which is fastest, which will give you a more accurate answer than arm-chair theorising.
I too prefer Alexei Kaigorodov suggestion to start all 28 threads. But I suggest you to replace newFixedThreadPoolwith new Executors API: ( since Java 8)
static ExecutorService newWorkStealingPool()
Creates a work-stealing thread pool using all available processors as its target parallelism level.
Above API returns ForkJoinPool type of ExecutorService
Now you don't need to worry utilization of idle threads. Java will take care better utilization of idle threads with work-stealing mechanism.
If you still need four different groups of FixedThreadPools, you can proceed with invokeAll. Don't shutdown ExecutorService to switch between multiple pools. You one ExecutorService effectively. If you want to poll the result of Future tasks using invokeAll, use CompletableFuture and poll it to know the status of task execution.
static CompletableFuture<Void> runAsync(Runnable runnable, Executor executor)
Returns a new CompletableFuture that is asynchronously completed by a task running in the given executor after it runs the given action.
I have a Java program that employs multi-threads using pool of threads. Each thread is going to make some select and insert statements. The DB is MySQL InnoDB.
I am planning to launch several run screens (6 processes), and each process is going to employ multi-threads. My questions are:
1) How many threads can I define for the pool? Am I limited to maximum of 4 threads since my processor is Core i7-2640M ?
2) Should I worry about anything regarding the DB integrity ? I did not use synchronization for anything in my program (thre is no need) but I am worried about the DB insert statement. Do I need to synchronize them or MySQL will take care about that knowing that I have one connection for each process in the main function.
Please, advise me.
Starting with 2)
DB integrity is guaranteed, as long as you watch your isolation levels: If your inserts are not atomic, or your selects care about insert timimng from the other threads, use transactions.
Concerning 1)
The optimum number of threads is heighly dependant on what these threads do: A thread waiting for a remote database transaction to complete definitly does not need a core available. I recommend you start with something like 4 times the core count and start testing from there. Depending on your OS you will also want to benchmark processes vs. threads.
Regarding thread amount
You can have 100 threads on a single core if you like. The OS will schedule them so each get some time to run. A running computer will already have a few hundred active threads / processes, most of them are just idle (Windows task manager shows that info somewhere).
Each core in your CPU can run 1 thread at a time. The i7-2640M has two physical cores but each does hyperthreading (= running 2 threads on 1 physical core in "parallel") so it provides 4 cores for you to work with.
To approximate the optimum thread count you need to know how active your threads are. If each thread runs all the time without sleep / wait it takes up a complete core. You should not use more than 4 threads in that case since they will just block each other. Switching between active threads takes some time for the CPU so more threads will result in less performance overall.
If your threads are waiting, you need to figure out the percentage of activeness. E.g. each screen-thread waits for user input most of the time. Each input takes in 1 second to process and you expect that each screen will get 30 events per minute. That would mean you need about 30 seconds per minute for each. You have 4 x 60 seconds per minute so your optimum thread / screen count would be around 8.
If you have an executor to run small tasks, each completing within seconds, then use about as much threads as you have cores. Small tasks can be distributed by the executor to threads so that you get back to about 100% load for each core.
Regarding Db integrity
see Eugen Rieck. Transactions are the way to ensure atomic modifications (= no other process can see a state in between transaction begin and end)