I'm wondering whether there is any advantage to keeping the same threads over the course of the execution of an object, rather than re-using the same Thread objects. I have an object for which a single (frequently used) method is parallelized using local Thread variables, such that every time the method is called, new Threads (and Runnables) are instantiated. Because the method is called so frequently, a single execution may instantiate upwards of a hundred thousand Thread objects, even though there are never more than a few (~4-6) active at any given time.
Following is a cut down example of how this method is currently implemented, to give a sense of what I mean. For reference, n is of course the pre-determined number of threads to use, whereas this.dataStructure is a (thread-safe) Map which serves as the input to the computation, as well as being modified by the computation. There are other inputs involved, but as they are not relevant to this question, I've omitted their usage. I've also omitted exception handling for the same reason.
Runnable[] tasks = new Runnable[n];
Thread[] threads = new Thread[n];
ArrayBlockingQueue<MyObject> inputs = new ArrayBlockingQueue<>(this.dataStructure.size());
inputs.addAll(this.dataStructure.values());
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
threads[i] = new Thread(tasks[i]);
threads[i].start();
}
for (int i = 0; i < n; i++)
threads[i].join();
Because these Threads (and their runnables) always execute the same way using a single ArrayBlockingQueue as input, an alternative to this would be to just "refill the queue" every time the method is called and just re-start the same Threads. This is easily implemented, but I'm unsure as to whether it would make any difference one way or the other. I'm not too familiar with concurrency, so any help is appreciated.
PS.: If there is a more elegant way to handle the polling, that would also be helpful.
It is not possible to start a Thread more than once, but conceptually, the answer to your question is yes.
This is normally accomplished with a thread pool. A thread pool is a set of Threads which rarely actually terminate. Instead, an application is passes its task to the thread pool, which picks a Thread in which to run it. The thread pool then decides whether the Thread should be terminated or reused after the task completes.
Java has some classes which make use of thread pools quite easy: ExecutorService and CompletableFuture.
ExecutorService usage typically looks like this:
ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
executor.submit(tasks[i]);
}
// Doesn't interrupt or halt any tasks. Will wait for them all to finish
// before terminating its threads.
executor.shutdown();
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
Executors has other methods which can create thread pools, like newFixedThreadPool() and newWorkStealingPool(). You can decide for yourself which one best suits your needs.
CompletableFuture use might look like this:
Runnable[] tasks = new Runnable[n];
CompletableFuture<?>[] futures = new CompletableFuture<?>[n];
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
futures[i] = CompletableFuture.runAsync(tasks[i]);
}
CompletableFuture.allOf(futures).get();
The disadvantage of CompletableFuture is that the tasks cannot be canceled or interrupted. (Calling cancel will mark the task as completing with an exception instead of completing successfully, but the task will not be interrupted.)
Per definition, you cannot restart a thread. According to the documentation:
It is never legal to start a thread more than once. In particular, a thread may not be restarted once it has completed execution.
Nevertheless a thread is a valuable resource, and there are implementations to reuse threads. Have a look at the Java Tutorial about Executors.
Related
Imagine I have following code:
final ExecutorService threadPool = Executors.newFixedThreadPool(
NUMBER_OF_WORKERS);
for (int i=0; i < NUMBER_OF_WORKERS; i++)
{
final Worker worker = new BirthWorker(...);
threadPool.execute(worker);
}
Now I need a piece of code, which waits, until all workers have completed their work.
Options I'm aware of:
while (!threadPool.isTerminated()) {}
Modify the code like that:
final List futures = new ArrayList(NUMBER_OF_WORKERS);
final ExecutorService threadPool = Executors.newFixedThreadPool(NUMBER_OF_WORKERS);
for (int i=0; i < NUMBER_OF_WORKERS; i++)
{
final Worker worker = new Worker(...);
futures.add(threadPool.submit(worker));
}
for (final Future future : futures) {
future.get();
}
// When we arrive here, all workers are guaranteed to have completed their work.
What is the best practice to wait for the completion of all workers?
I would suggest you use CountDownLatch (assuming this is one time activity) where in your constructor, you can specify how many threads you want to wait for and you share that instance accross the threads and you wait on all the threads to complete using await api (using timeout or complete blocking) and your thread's calling countdown api when they are done.
Another option would be, to call join method in thread to wait for their completion if you have access to each and every thread that you wish to complete.
I would use ThreadPoolExecutor.invokeAll(Collection<? extends Callable<T>> tasks)
API: Executes the given tasks, returning a list of Futures holding their status and results when all complete
CountDownLatch,as stated above, would do the work well, just keep in mind that you want to shut down the executur after your done:
final ExecutorService threadPool = Executors.newFixedThreadPool(
NUMBER_OF_WORKERS);
for (int i=0; i < NUMBER_OF_WORKERS; i++)
{
final Worker worker = new BirthWorker(...);
threadPool.execute(worker);
}
threadPool.shutdown();
unless you shut it down, threadPool.isTerminated will stay false, even when all the workers are done.
I am fairly new with java executors, so this maybe an easy question.
ExecutorService executorService = Executors.newFixedThreadPool(NumberOfThreads - 1);
do_work();
for(int i = 1; i < NumberOfThreads; i++)
{
executorService.execute(new Runnable()
{
public void run()
{
do_work();
}
});
}
My question is:
If I create a fixed thread pool with 'N' threads, and if I want to execute 'N' tasks, like the code above. Do I have guarantees that each thread will only execute one task (do_work())?
No. It's a pool, and the assignment of threads to tasks doesn't make such guarantees.
e.g. imagine your do_work() method completes immediately. By the time you submit your 2nd Runnable, all the threads in the pool will be available, and any one of them will be a candidate for your job.
I am executing millions of iteration and I want to parallelize this. Hence decided to add the task [each iteration] to the Thread Pool.
Now, if I add all the iteration to the Thread Pool, it might throw an OutOfMemoryError. I want to handle that gracefully, so is there any way to know about the availability of the worker Thread in the Thread Pool?
Once it's available, add the Runnable to the Worker Thread.
for(int i=0; i<10000000000; i++) {
executor.submit(new Task(i));
}
Each of those tasks merely take 1 sec to complete.
Why don't you set a limit to how many tasks can run concurrently. Like:
HashSet<Future> futures = new HashSet<>();
int concurrentTasks = 1000;
for (int ii=0; ii<100000000; ii++) {
while(concurrentTasks-- > 0 && ii<100000000) {
concurrentTasks.add(executor.submit(new Task(ii)));
}
Iterator<Future> it = concurrentTasks.iterator();
while(it.hasNext()) {
Future task = it.next();
if (task.isDone()) {
concurrentTasks++;
it.remove();
}
}
}
You'll want to use something like this:
ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(MAX_PENDING_TASKS);
Executor executor = new ThreadPoolExecutor(MIN_THREADS, MAX_THREADS, IDLE_TIMEOUT, TimeUnit.SECONDS, queue, new ThreadPoolExecutor.CallerRunsPolicy());
for(int i=0; i<10000000000; i++) {
executor.submit(new Task(i));
}
Basically you create a thread pool with min/max threads and an array backed queue. When you hit the limit of pending tasks, the "caller runs policy" kicks in and your main thread ends up running the next task (giving time for your other tasks to complete and open slots in the queue).
Since you've stated that your tasks are short lived, this seems like an optimal strategy.
The values for MAX_PENDING_TASKS and MIN_THREADS are something you can fiddle with to figure out what the optimal values are for your workload, but MAX_PENDING_TASKS should be at least twice MIN_THREADS and probably more like 10 to 100 times.
You should use java.lang.Runtime
The biggest memory issue is probably going to be your Object creation, not in adding them to your Executor, so that's where you should be calling Runtime.getRuntime().freeMemory().
Suppose I have the following code which I wan't to optimize by spreading the workload over the multiple CPU cores of my PC:
double[] largeArray = getMyLargeArray();
double result = 0;
for (double d : largeArray)
result += d;
System.out.println(result);
In this example I could distribute the work done within the for-loop over multiple threads and verify that the threads have all terminated before proceeding to printing the result. I therefore came up with something that looks like this:
final double[] largeArray = getMyLargeArray();
int nThreads = 5;
final double[] intermediateResults = new double[nThreads];
Thread[] threads = new Thread[nThreads];
final int nItemsPerThread = largeArray.length/nThreads;
for (int t = 0; t<nThreads; t++) {
final int t2 = t;
threads[t] = new Thread(){
#Override public void run() {
for (int d = t2*nItemsPerThread; d<(t2+1)*nItemsPerThread; d++)
intermediateResults[t2] += largeArray[d];
}
};
}
for (Thread t : threads)
t.start();
for (Thread t : threads)
try {
t.join();
} catch (InterruptedException e) { }
double result = 0;
for (double d : intermediateResults)
result += d;
System.out.println(result);
Assume that the length of the largeArray is dividable by nThreads. This solution works correctly.
However, I am encountering the problem that the above threading of for-loops occurs a lot in my program, which causes a lot of overhead due to the creation and garbage collection of threads. I am therefore looking at modifying my code by using a ThreadPoolExecutor. The threads giving the intermediate results would then be reused in the next execution (summation, in this example).
Since I store my intermediate results in an array of a size which has to be known beforehand, I was thinking of using a thread pool of fixed size.
I am having trouble, however, with letting a thread know at which place in the array it should store its result.
Should I define my own ThreadFactory?
Or am I better of using an array of ExecutorServices created by the method Executors.newSingleThreadExecutor(ThreadFactory myNumberedThreadFactory)?
Note that in my actual program it is very hard to replace the double[] intermediateResults with something of another type. I would prefer a solution which is confined to creating the right kind of thread pool.
I am having trouble, however, with letting a thread know at which place in the array it should store its result. Should I define my own ThreadFactory?
No need for that. The interfaces used by executors (Runnable and Callable) are run by threads, and you can pass whatever arguments to implementations you want to pass (for instance, an array, a begin index and an end index).
A ThreadPoolExecutor is indeed a good solution. Also look at FutureTask if you have runnables bearing results.
ExecutorService provides you with API to get the result from thread pool via Future interface:
Future<Double> futureResult = executorService.submit(new Callable<Double>() {
Double call() {
double totalForChunk = 0.0;
// do calculation here
return totalForChunk;
}
});
Now all you need to do is to submit tasks (Callable instances) and wait for result to be available:
List<Future<Double>> results = new ArrayList<Double>();
for (int i = 0; i < nChunks; i++) {
results.add(executorService.submit(callableTask));
}
Or even simpler:
List<Future<Double>> results = executorService.invokeAll(callableTaskList);
The rest is easy, iterate over results and collect total:
double total = 0.0;
for (Future<Double> result : results) {
total += result.get(); // this will block until your task is completed by executor service
}
Having that said, you do not care how much threads you have in executor service. You just submit tasks and collect results when they are available.
You would be better off creating "worker" threads that take information about work to be performed from a queue. Your process would then be to create an initially empty WorkQueue and then create and start the worker threads. Each worker thread would pick up its work from the queue, do the work, and put the work on a "finished" queue for the master to pick up and handle.
I have the following function, in pseudo-code:
Result calc(Data data) {
if (data.isFinal()) {
return new Result(data); // This is the actual lengthy calculation
} else {
List<Result> results = new ArrayList<Result>();
for (int i=0; i<data.numOfSubTasks(); ++i) {
results.add(calc(data.subTask(i));
}
return new Result(results); // merge all results in to a single result
}
}
I want to parallelize it, using a fixed number of threads.
My first attempt was:
ExecutorService executorService = Executors.newFixedThreadPool(numOfThreads);
Result calc(Data data) {
if (data.isFinal()) {
return new Result(data); // This is the actual lengthy calculation
} else {
List<Result> results = new ArrayList<Result>();
List<Callable<Void>> callables = new ArrayList<Callable<Void>>();
for (int i=0; i<data.numOfSubTasks(); ++i) {
callables.add(new Callable<Void>() {
public Void call() {
results.add(calc(data.subTask(i));
}
});
}
executorService.invokeAll(callables); // wait for all sub-tasks to complete
return new Result(results); // merge all results in to a single result
}
}
However, this quickly got stuck in a deadlock, because, while the top recursion level waits for all threads to finish, the inner levels also wait for threads to become available...
How can I efficiently parallelize my program without deadlocks?
Your problem is a general design problem when using ThreadPoolExecutor for tasks with dependencies.
I see two options:
1) Make sure to submit tasks in a bottom-up order, so that you never have a running task that depends on a task which didn't start yet.
2) Use the "direct handoff" strategy (See ThreadPoolExecutor documentation):
ThreadPoolExecutor executor = new ThreadPoolExecutor(poolSize, poolSize, 0, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
executor.setRejectedExecutionHandler(new CallerRunsPolicy());
The idea is using a synchronous queue so that tasks never wait in a real queue. The rejection handler takes care of tasks which don't have an available thread to run on. With this particular handler, the submitter thread runs the rejected tasks.
This executor configuration guarantees that tasks are never rejected, and that you never have deadlocks due to inter-task dependencies.
you should split your approach in two phases:
create all the tree down until data.isFinal() == true
recursively collect the results (only possible if the merging does not produce other operations/calls)
To do that, you can use [Futures][1] to make the results async. Means all results of calc will be of type Future[Result].
Immediately returning a Future will free the current thread and give space for the processing of others. With the collection of the Results (new Result(results)) you should wait for all results to be ready (ScatterGather-Pattern, you can use a semaphore to wait for all results). The collection itself will be walking a tree and checking (or waiting for the results to arrive) will happen in a single thread.
Overall you build a tree of Futures, that is used to collect the results and perform only the "expensive" operations in the threadpool.