I'm new to ExecutorService. Right now my scenario is "millions of data 365*24*7 is coming in"
I have some process to be done on data coming in using threads.
ExecutorService es = Executors.newSingleThread();
es.execute(new ComputeDTask(data));
I'm sending data to ComputeDTask for some execution.
How efficient is it to create a new ComputeDTask each time data comes in? That is, if data is received one million times, then a million ComputeDTask objects will be created.
The overhead of creating a thread is about 100 micro-seconds. i.e. if you do less than 100 micro-seconds of work you will have more overhead than work done and you program can be slower that being single threaded.
The overhead of create a task to an existing Executor service is about 2 micro-seconds. i.e. if the task takes less than 2 micro-second you may have more overhead than real work done.
If you have CPU bound process, you need about the same number of threads as core to keep all the cores busy, while minimising overhead.
e.g. if you have 8 cores, I suggest you combine the work done so you have 8 threads with one task each total. You can have more tasks than this but you may find it take longer to process.
Of course you should shutdown your ExecutorService when you have finished with it. The reason you don't see this done in all examples is that it can be a good idea to create one ExecutorService which is used for the life of the application.
Apparently you are creating a whole new ExecutorService for each task and never shutting them down. This of course results in the thread leak that you are observing. The proper way to use the ExecutorService is to create a single instance that manages the thread pool for you. The executors are very flexible and powerful in the way they manage the threads.
On incoming data even create the task using new ComputeDTask(data) and then pass this to ThreadPool where you have let say 100 threads and then they can execute task with higher throughput.
ExecutorService es = Executors.newFixedThreadPool(100);
onGetData(){
es.execute(new ComputeDTask(data));
}
Related
When I have hundreds of items to iterate through, and I have to do a computation-heavy operation to each one, I would take a "divide and conquer" approach. Essentially, I would take the processor count + 1, and divide those items into the same number of batches. And then I would execute each batch on a runnable in a cached thread pool. It seems to work well. My GUI task went from 20 seconds to 2 seconds, which is a much better experience for the user.
However, I was reading Brian Goetz' fine book on concurrency, and I noticed that for iterating through a list of items, he would take a totally different approach. He would kick off a Runnable for each item! Before, I always speculated this would be bad, especially on a cached thread pool which could create tons of threads. However each runnable would probably finish very quickly in the larger scope, and I understand the cached thread pool is very optimal for short tasks.
So which is the more accepted paradigm to iterate through computation-heavy items? Dividing into a fixed number of batches and giving each batch a runnable? Or kicking each item off in its own runnable? If the latter approach is optimal, is it okay to use a cached thread pool or is it better to use a bounded thread pool?
With batches you will always have to wait for the longest running batch (you are as fast as the slowest batch). "Divide and conquer" implies management overhead: doing administration for the dividing and monitoring the conquering.
Creating a task for each item is relative straightforward (no management), but you are right in that it may start hundreds of threads (unlikely, but it could happen) which will only slow things down (context switching) if the task does no/very few I/O and is mostly CPU intensive.
If the cached thread pool does not start hundreds of threads (see getLargestPoolSize), then by all means, use the cached thread pool. If too many threads are started then one alternative is to use a bounded thread pool. But a bounded thread pool needs some tuning/decisions: do you use an unbounded task queue or a bounded task queue with a CallerRunsPolicy for example?
On a side note: there is also the ForkJoinPool which is suitable for tasks that start sub-tasks.
I am a newbie to Java concurrency and am a bit confused by several concepts and implementation issues here. Hope you guys can help.
Say, I have a list of tasks stored in a thread-safe list wrapper:
ListWrapper jobs = ....
'ListWrapper' has synchronized fetch/push/append functions, and this 'jobs' object will be shared by multiple worker threads.
And I have a worker 'Runnable' to execute the tasks:
public class Worker implements Runnable{
private ListWrapper jobs;
public Worker(ListWrapper l){
this.jobs=l;
}
public void run(){
while(! jobs.isEmpty()){
//fetch an item from jobs and do sth...
}
}
}
Now in the main function I execute the tasks:
int NTHREADS =10;
ExecutorService service= Executors.newFixedThreadPool(NTHREADS);
//run threads..
int x=3;
for(int i=0; i<x; i++){
service.execute(new Worker(jobs) );
}
I tested this code with 'x=3', and I found that only 3 threads are running at the same time; but as I set 'x=20', I found that only 10 (=NTHREADS) are running at the same time. Seems to me the # of actual threads is the min of the two values.
Now my questions are:
1) Which value ('x' or 'NTHREADS') should I set to control the number of concurrent threads? Or it doesn't matter in either I choose?
2) How is this approach different from simply using the Producer-Consumer pattern --creating a fixed number of 'stud' threads to execute the tasks(shown in the code below)?
Thread t1= new Worker(jobs);
Thread t2= new Worker(jobs);
...
t1.join();
t2.join();
...
Thank you very much!!
[[ There are some good answers here but I thought I'd add some more detail. ]]
I tested this code with 'x=3', and I found that only 3 threads are running at the same time; but as I set 'x=20', I found that only 10 (=NTHREADS) are running at the same time. Seems to me the # of actual threads is the min of the two values.
No, not really. I suspect that the reason you weren't seeing 20 threads is that threads had already finished or had yet to be started. If you call new Thread(...).start() 20 times then you will get 20 threads started. However, if you check immediately none of them may have actually begun to run or if you check later they may have finished.
1) Which value ('x' or 'NTHREADS') should I set to control the number of concurrent threads? Or it doesn't matter in either I choose?
Quoting the Javadocs of Executors.newFixedThreadPool(...):
Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks.
So changing the NTHREADS constant changes the number of threads running in the pool. Changing x changes the number of jobs that are executed by those threads. You could have 2 threads in the pool and submit 1000 jobs or you could have 1000 threads and only submit 1 job for them to work on.
Btw, after you have submitted all of your jobs, you should then shutdown the pool which stops all of the threads if all of the jobs have been run.
service.shutdown();
2) How is this approach different from simply using the Producer-Consumer pattern --creating a fixed number of 'stud' threads to execute the tasks(shown in the code below)?
It differs in that it does all of the heavy work for you.
You don't have to create a ListWrapper of the jobs since you get one inside of the ExecutorService. You just submit the jobs to the ExecutorService and it keeps track of them until the threads are available to run them.
You don't have to create any threads or worry about them throwing exceptions and dying because the ExecutorService starts/restarts the threads for you.
If you want your tasks to return information you can make use of the submit(Callable) method and use the Future to get the results of the jobs. Etc, etc..
Doing this code yourself is going to be harder to get right, more code to maintain, and most likely will not perform as well as the code in the JDK that is battle tested and optimized.
You shouldn't create threads by yourself when using a threadpool. Instead of WorkerThread class you should use a class that implements Runnable but is not a thread. Passing a Thread object to the threadpool won't make the thread run actually. The object will be passed to a different internal thread, which will simply execute the run method of your WorkerThread class.
The ExecutorService is simply incompatible with the way you want to write your program.
In the code you have right now, these WorkerThreads will stop to work when your ListWrapper is empty. If you then add something to the list, nothing will happen. This is definitely not what you wanted.
You should get rid of ListWrapper and simply put your tasks directly into the threadpool. The threadpool already incorporates an internal list of jobs shared between the threads. You should just submit your jobs to the threadpool and it will handle them accordingly.
To answer your questions:
1) Which value ('x' or 'NTHREADS') should I set to control the number of concurrent threads? Or it doesn't matter in either I choose?
NTHREADS, the threadpool will create the necessary number of threads.
2) How is this approach different from simply using the Producer-Consumer pattern --creating a fixed number of 'stud' threads to execute the tasks(shown in the code below)?
It's just that ExecutorService automates a lot of things for you. You can choose from a lot of different implementations of threadpools and you can substitute them easily. You can use for instance a scheduled executor. You get extra functionality. Why reinvent the wheel?
For 1) NTHREADS is the maximum threads that the pool will ever run concurrently, but that doesn't mean there will always be that many running. It will only use as many as is needed up to that max value... which in your case is 3.
As the docs say:
At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available
http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Executors.html#newFixedThreadPool-int-
As for 2) using Java's concurrent executors framework is preferred with new code. You get a lot of stuff for free and removes the need for having to handle all of the fiddly thread work yourself.
The number of threads passed into newFixedThreadPool is at most how many threads could be running executing your tasks. If you only have three tasks ever submitted I'd expect the ExecutorService to only create three threads.
To answer your questions:
You should use the number you pass into the constructor to control how many threads are going to be used to execute your tasks.
This differs because of the extra functionality the ExecutorService gives you, as well as the flexibility it gives you such as in the case you need to change your ExecutorService type or number of tasks you'll run (less lines of code to change).
All that is happening is the executor service is only creating as many threads as it needs. NTHREADS is effectively the maximum number of threads it'll create.
There is no point creating ten threads up front if it only has 3 tasks to complete, the other 7 will just be hanging around consuming resources.
If you submit more than NTHREADS number of tasks then it will process that number concurrently and the rest will wait on a queue until a thread becomes free.
This isn't any different from creating a fixed set of your own threads, except the thread management and scheduling is handled for you. The executor service also restarts threads if they are killed by rogue exceptions in your task which you'd otherwise have to code for.
See: The Javadoc on Executorservice.newFixedThreadPool
I'm in a situation where I have 4 groups of 7 threads each. My CPU (core i7) is supposed to be able to handle 8 threads, so I'm considering going through each group one at a time, running the 7 threads, then moving to the 2nd group, running its 7 threads, then 3rd and 4th groups in the same way, and then starting back at 1st group, until user sends a stop command.
My question is, once each group of 7 threads has finished processing, should I keep those threads idle, or shut them down completely and restart a new group of 7 threads at the next iteration? Which method will be faster? This is for a very speed intensive app, so I need everything to happen as quickly as possible.
I will be using a FixedThreadPool to manage each group of 7 threads. So I could either just invokeAll() and then leave them alone (presumably to idle), or I could shutdown() each threadpool after the invokeAll() and start a new thread pool at the next iteration.
Which method will be faster?
My question is, once each group of 7 threads has finished one cycle of processing, should I keep those threads idle, or shut them down completely and restart a new group of 7 threads at the next cycle?
I would use a single ExecutorService thread-pool and reuse the same threads for all tasks. See the tutorial on the subject. A thread-pool is designed to execute any Runnable or Callable class so they are task agnostic. For example, you might have ParentResult and ChildResult classes. You can submit a Callable<ParentResult> to the thread-pool which will return a Future<ParentResult> and you can submit a Callable<ChildResult> to the same thread-pool which will return a Future<ChildResult>.
The only reason why you'd want to have "groups of threads" is if each thread has some state that it must maintain -- a database connection or something. Even then many people use thread-pools since it does much of the concurrency heavy lifting for you.
If you do have to keep this state then I would certainly not shutdown the pools and then restart them later. A dormant thread/pool is taking no system resources aside from memory. The only reason why you would ever do this is if you are forking 100s of thread for the task but at that point, you should consider re-architecting your application.
You need not to schedule your threads manually. Start all 28 threads at once - this would not be slower, but well can be faster.
When you say your processor has 8 threads, I think you mean it has has 4 cores with hyperthreading. Java does not use threads in the same sense as your processor, so those 7 threads are of a different type to your processors.
The JVM handles processor usage, and is (IIRC) limited to using 1 core. The threads java uses are specific to the JVM, and are wholly separate.
As for your actual question, try testing different thread combinations to see which is fastest, which will give you a more accurate answer than arm-chair theorising.
I too prefer Alexei Kaigorodov suggestion to start all 28 threads. But I suggest you to replace newFixedThreadPoolwith new Executors API: ( since Java 8)
static ExecutorService newWorkStealingPool()
Creates a work-stealing thread pool using all available processors as its target parallelism level.
Above API returns ForkJoinPool type of ExecutorService
Now you don't need to worry utilization of idle threads. Java will take care better utilization of idle threads with work-stealing mechanism.
If you still need four different groups of FixedThreadPools, you can proceed with invokeAll. Don't shutdown ExecutorService to switch between multiple pools. You one ExecutorService effectively. If you want to poll the result of Future tasks using invokeAll, use CompletableFuture and poll it to know the status of task execution.
static CompletableFuture<Void> runAsync(Runnable runnable, Executor executor)
Returns a new CompletableFuture that is asynchronously completed by a task running in the given executor after it runs the given action.
What benefit is there to use Executors over just Threads in a Java program.
Such as
ExecutorService pool = Executors.newFixedThreadPool(2);
void someMethod() {
//Thread
new Thread(new SomeRunnable()).start();
//vs
//Executor
pool.execute(new SomeRunnable());
}
Does an executor just limit the number of threads it allows to have running at once (Thread Pooling)? Does it actually multiplex runnables onto the threads it creates instead? If not is it just a way to avoid having to write new Thread(runnable).start() every time?
Yes, executors will generally multiplex runnables onto the threads they create; they'll constrain and manage the number of threads running at once; they'll make it much easier to customize concurrency levels. Generally, executors should be preferred over just creating bare threads.
Creating new threads is expensive. Because Executors uses a thread pool, you get to easily reuse threads, resulting in better performance.
Does an executor just limit the number of threads it allows to have running at once (Thread Pooling)?
Executors#newFixedThreadPool(int), Executors#newSingleThreadExecutor do this, each one under different terms (read the proper javadoc to know more about it).
Does it actually multiplex runnables onto the threads it creates instead?
Yes
If not is it just a way to avoid having to write new Thread(runnable).start() every time?
ExecutorService helps you to control the way you handle threads. Of course, you can do this manually, but there's no need to reinvent the wheel. Also, there are other functionalities that ExecutorService provides you like executing asynchronous tasks through the usage of Future instances.
There are multiple concerns related to thread.
managing threads
resource utilization
creation of thread
Executors provides different kind of implementation for creating a pool of threads. Also thread creation is a costly affair. Executors creates and manages these threads internally. Details about it can be found in the below link.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
As I said over in a related question, Threads are pretty bad. Executors (and the related concurrency classes) are pretty good:
Caveat: Around here, I strongly discourage the use of raw Threads. I
much prefer the use of Callables and FutureTasks (From the javadoc: "A
cancellable asynchronous computation"). The integration of timeouts,
proper cancelling and the thread pooling of the modern concurrency
support are all much more useful to me than piles of raw Threads.
For example, I'm currently replacing a legacy piece of code that used a disjoint Thread running in a loop with a self-timer to determine how long it should Thread.sleep() after each iteration. My replacement will use a very simple Runnable (to hold a single iteration), a ScheduledExecutorService to run one of the iterations and the Future resulting from the scheduleAtAFixedRate method to tune the timing between iterations.
While you could argue that replacement will be effectively equivalent to the legacy code, I'll have replaced an arcane snarl of Thread management and wishful thinking with a compartmentalized set of functionality that separates the concerns of the GUI (are we currently running?) from data processing (playback at 5x speed) and file management (cancel this run and choose another file).
We need to do some asynchronous task processing where in around 30-40 requests will be coming at the same moment and each request will intiate a asynch task which will approximately take around 7-8 seconds to complete.
If java executorservice has been identified to do such task, what would be the idle type of executor for such purpose?
I thought of using CachedThreadPool but my worry is if too many threads are created would it have any performance impact on the application?
Another option would be to use FixedThreadPool but I am struggling to think of a idle no threads which it should be instantiated with...
What is the recommended Executor for such a scenario or how we go about finding the right one?
I think you are limiting your research to just the Executors.* factory methods. You should review what the range of constructors of ThreadPoolExecutor, you'll find a maximum thread pool size limit, among other things.
I thought of using CachedThreadPool but my worry is if too many
threads are created would it have any performance impact on the
application?
You need to test for the application for performance impact.
If none of them fits into the application or having some issues then you can use customized thread pool executor java.util.concurrent.ThreadPoolExecutor
You can customize according your needs with configuiring core pool size, configuring the blocking queue. Blocking queue will be used and task will be queued when pool size is reached.