I have a Java program that employs multi-threads using pool of threads. Each thread is going to make some select and insert statements. The DB is MySQL InnoDB.
I am planning to launch several run screens (6 processes), and each process is going to employ multi-threads. My questions are:
1) How many threads can I define for the pool? Am I limited to maximum of 4 threads since my processor is Core i7-2640M ?
2) Should I worry about anything regarding the DB integrity ? I did not use synchronization for anything in my program (thre is no need) but I am worried about the DB insert statement. Do I need to synchronize them or MySQL will take care about that knowing that I have one connection for each process in the main function.
Please, advise me.
Starting with 2)
DB integrity is guaranteed, as long as you watch your isolation levels: If your inserts are not atomic, or your selects care about insert timimng from the other threads, use transactions.
Concerning 1)
The optimum number of threads is heighly dependant on what these threads do: A thread waiting for a remote database transaction to complete definitly does not need a core available. I recommend you start with something like 4 times the core count and start testing from there. Depending on your OS you will also want to benchmark processes vs. threads.
Regarding thread amount
You can have 100 threads on a single core if you like. The OS will schedule them so each get some time to run. A running computer will already have a few hundred active threads / processes, most of them are just idle (Windows task manager shows that info somewhere).
Each core in your CPU can run 1 thread at a time. The i7-2640M has two physical cores but each does hyperthreading (= running 2 threads on 1 physical core in "parallel") so it provides 4 cores for you to work with.
To approximate the optimum thread count you need to know how active your threads are. If each thread runs all the time without sleep / wait it takes up a complete core. You should not use more than 4 threads in that case since they will just block each other. Switching between active threads takes some time for the CPU so more threads will result in less performance overall.
If your threads are waiting, you need to figure out the percentage of activeness. E.g. each screen-thread waits for user input most of the time. Each input takes in 1 second to process and you expect that each screen will get 30 events per minute. That would mean you need about 30 seconds per minute for each. You have 4 x 60 seconds per minute so your optimum thread / screen count would be around 8.
If you have an executor to run small tasks, each completing within seconds, then use about as much threads as you have cores. Small tasks can be distributed by the executor to threads so that you get back to about 100% load for each core.
Regarding Db integrity
see Eugen Rieck. Transactions are the way to ensure atomic modifications (= no other process can see a state in between transaction begin and end)
Related
I want to run n tasks continuously, however those tasks are memory intensive I want only x of them to be active at a time. But ultimately all those n tasks should run by context switching between them.
In short, I want another implementation of FixedThreadPool, where extra tasks should also run with context switching.
Do we have Thread Pool variant which achieves same? Or any other way to implement it?
UPDATE : After reading a bit and reading answer below, decided to "divide and conquer" i.e breaking continuously running task into units of small short lived tasks and submitting those again and again to FixedThreadPool
One could write a paper on the subject but let us keep it simple and concise.
In short, I want another implementation of FixedThreadPoolSize, where
extra tasks should also run with context switching.
To achieve that, one would need a Thread Pool that allows to explicitly perform affinity between a Thread and a core. And (as far I know) such a Thread Pool is not officially provided with Java. Which makes sense, since one of the goals of abstractions such as Thread Pools (in Java) is to increase the level of abstraction, even to the point of abstracting concepts such as a thread (i.e., Executor). Therefore, it does not come as a surprise that such a low-level feature (as mapping threads to cores) is not provided out-of-the-box.
Do we have Thread Pool variant which achieves same? Or any other way
to implement it?
Unless you are running your code in a Non-Uniform Memory Access (NUMA) architecture, I don't see the benefits of such a low level feature in the context of your program.
My use case is I have to run n tasks continuously. But as those tasks
are memory intensive I want only x of them to be active at a time. But
ultimately all those n tasks should run by context switching between
them.
If you run n tasks and n Threads, and the hardware where the code is running has c cores, where n >> c, then inevitably the SO will map multiple threads to the same core. Hence, you will have your context switching for free.
Finally, before actually opting to run more Threads than cores, profile your code, accordingly. For instance, running the code with the same number threads as cores, then with doubling the number of threads until it stops scaling. It might so happen that your code does even scale with more threads than cores.
I am trying to get the basics very strong about thread pool. I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.
Question is: Why not concurrently? Why not just time-slice those 10 tasks?
What is the reason of this implementation?
Why not concurrently? Why not just time-slice those 10 tasks?
You can have a thread pool that is able to perform ten concurrent tasks. You just need to configure it to have at least ten worker threads. "Time-slicing" tasks is what threads do. What thread pools do is:
Allow your program to control the number of threads that it uses to perform "background" tasks, and
Allow your program to re-use threads, which can be much more efficient than creating a new thread for each new task, and then destroying the thread when the task is complete.
In order to "time-slice 10 tasks", those tasks need to be in 10 separate threads that run concurrently.
The time-slicing scheduling algorithm is implemented by the operating system, not by Java. Time slicing applies to threads in Java because Java threads are implemented as native operating system threads: every Java thread has a native thread of its own, and these threads are scheduled by the operating system as it sees fit.
There is no difference between "thread pool threads" and "raw threads" here. If you give an instance of Runnable to a thread (whether it's part of a thread pool or not) it will run from beginning to end, subject to the time slicing scheduling algorithm of the operating system.
So why not use thousands of threads, why even bother with thread pools? It turns out that operating system threads are a relatively expensive and scarce resource, and therefore so are Java threads.
Since operating system threads are so expensive, Project Loom is investigating adding lightweight user space threads to Java. Some of the details in this answer may change when/if loom gets merged into main stream Java.
Some good answers but I thought I'd respond to your questions specifically.
I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.
If you configure your thread pool to have 5 threads (Executors.newFixedThreadPool(5)) then it will start 5 threads to run your jobs. Initially 5 jobs are given to the 5 threads to run concurrently (if your server has 5 CPUs available). Once one of the 5 jobs finishes, a 6th job will be immediately started on the idle thread. This continues until all 10 jobs have been run.
Question is: Why not concurrently? Why not just time-slice those 10 tasks? What is the reason of this implementation?
You can instead use a cached thread pool (Executors.newCachedThreadPool()) if you want which will start a thread for each of the 10 jobs that you submit concurrently. This might work fine for 10 jobs but won't work well with 100,000 jobs – you would not want to start 100,000 threads. We use a fixed thread pool when we want to limit the number of jobs that run concurrently. Even though it seems like running 5 jobs concurrently would always run slower than running 10 jobs at once, but this isn't necessarily the case. There is a cost when the OS time slices between the jobs and the overall job throughput may be faster with 5 threads than 10 depending on how many processors your hardware has. Limiting the number of concurrent jobs also does not stress your server as much and should make your application work better with other running applications.
See my answer here about scaling threads: Concept behind putting wait(),notify() methods in Object class
I have 3 fixedThreadPools in my application with size of 20 each.
I submit the tasks in pools but they are few sometimes say 5 task in each pool1 ,pool2 ,pool 3 so I have 15 threads from each pool sitting idle. I want to know does these idle threads (45 total) will come in context switch and degrade the performance even when no task is submitted to them ?
If the pooled threads have no task, they are in state WAITING, meaning they have nothing to do besides wait for a signal for something to happen (in this case, waiting for a task to be submitted). They don't get scheduled any CPU time because they have nothing to do. The scheduler is smart enough to know this, so it's at this point where it's excluded from competing for resources with other, more active threads.
If a thread is in RUNNABLE state, then it might have something to actually do, and is given CPU time.
So excess threads in a pool don't compete with live threads, but they do take a small amount of native resources (not enough to really matter when talking about 60 threads).
I have 3 fixedThreadPools in my application with size of 20 each.
You mean, you are creating 3 ExecutorService instance of thread pool size 20. That means total 60 threads in pools ?
...15 threads from each pool sitting idle. I want to know does these
idle threads (45 total) will come in context switch..
I think, you are assuming to run all 60 threads at the same time, as you have mentioned 5 thread from each pool (i.e 15 threads) are executing. But remaining 45 threads (15 thread from each pool) are idle. But, that is not correct until they have tasks assign and number of available cores to execute them.
Please note, execution of thread depends on number of available cores (CPU). No matter, how many thread you create, if CPU is not available to execute each thread, they will be at Runnable state to get their turn.
JVM who decides which thread should run and how, mainly uses preemptive or time slicing scheduling to schedule the threads. In that time, if other thread get a new task (or has some task to execute), then they will at Runnable state to get their turn.
Unlike other executors, ForkJoinPool supports work stealing.
Yes, when there are no tasks for execution threads would be waiting, but before that, they try to steal works from pool's queues, and this will consume more CPU, then other executors (FixedThreadPool, CachedThreadPool). And waking up threads also needs time.
ForkJoinPool performs best when it has a lot job. If you don't have this, you better to use FixedThreadPool. Also, you should understand that typically ForkJoinPool, FixedThreadPool are for CPU intensive tasks. Otherwise, if you have IO intensive tasks is better to use CachedThreadPool. You could create one FixedThreadPool at all, but should aware that tasks could block each other.
Also, there is no reason create three ForkJoinPool pools with 60 threads unless you have at least 32 CPUs (but it hugely depends). As you can not utilize pools, that it is likely not your example. But keep in mind, if you run a lot of CPU intensive tasks in thread's count bigger then CPUs, then yes, you likely get worse performance, because of resource sharing, including context switches.
I'm working in a redelivery system. This system attempt to execute an action, if the action fails, it try to execute again two times with an interval of five minutes, so I use the ExecutorService implementation to perform the first execution and ScheduledExecutorService to schedule the other ones, depending of its results (fail).
What should I consider to figure out the number of threads I need? In this moment I use only a single thread model (created by newSingleThreadScheduledExecutor method)
Without knowing details about the load your system has, environment it is using and how long does it take to process one message it is hard to say which number of threads you need. However, you can think of the following base principles:
Having many threads is bad, because you'll spend significant amount of time on a context switch, the chance of starvation and wasting system resources is higher .
Each thread consumes some space in memory for its stack. On x64 it is typically 1MB per thread.
I would probably create 2 thread pools (one scheduled, one non-scheduled) for both sending and redelivery and test them under high load varying number of threads from 2 to 10 to see which number suits best.
You should only need the one thread as only one action is running at a time. You could use a CachedThreadPool and not worry about it.
I'm new to multithreading... multithreading is used to improve performance, however how can this be so if the processor is already working as fast as it can for a single thread?
To explain:
In a single threaded environment User A starts a task which takes 1 second - the task is completed in 1 second.
User B starts the same task a fraction of a second later and has to wait for user A, therefore user Bs task completes in almost 2 seconds. Now, if this was a multithreaded environment would both tasks not be run similtaeously causing both to take 2 seconds? ...e.g. part of task A done, then part of task B, then part of task A again, then part of task B, ...till eventually both tasks complete in around two seconds?
Is it only faster if there is more than one processor? Or should multithreading be reserved for when there is a big task being dealt with and smaller tasks need to pop in and out of existence during this time?
If the task is 100% CPU bound, and you have only one CPU core, then multithreading will make things slower. If you have more than one CPU core, clearly you can allocate as many threads as you have cores to improve performance. If the tasks interact with anything external (I/O) then multithreading can increase performance to a point. While one thread is waiting for I/O to complete, other threads can be performing CPU-based processing.
The classic example is a program that does computation and also displays a GUI. You update the GUI on one thread (the event thread) and do all processing on other "background" threads. The event thread does nothing but handle user interactions. If you don't do this then when a user-requested operation takes appreciable time the GUI stops responding. In this situation, you would run multithreaded even on a single-core system.
How to tune an application's use of threads depends on a lot of factors and could take up an entire textbook.
Ok now consider that Task A of your's needs a particular resource (ex a network file or a user input) to complete its work. Now say the resource needed by the Task A is not currently available so what happens next in a single threaded environment is that Task A has control of the CPU so it'll wait for the resource to be available and at this time the CPU will be idle which means we're wasting an important resource i.e, the CPU time while waiting for some other resource. But if the Task A has been implemented in multithreaded environment when Task A waits for the resource, it(Task A thread) gets suspended till the resource become available and the CPU time can be used efficiently to execute other tasks. Hope this helps :)