I need to do something like this:
ExecutorService executor = Executors.newFixedThreadPool(2);
CompletionService<Boolean> completionService = new ExecutorCompletionService<>(executor);
int i = 0;
while (i < 40) {
completionService.submit(getTask());
i++;
}
executor.shutdown();
System.out.println("SHUTDOWN");
But after calling shutdown on executor object completionService still execute tasks.
How can I make it stop?
Is there are any way to stop ExecutorService but wait for the completion of currently executing tasks?
The ExecutorService will carry on executing tasks already submitted but will not allow further tasks to be submitted, from the documentation
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
emphasis mine
So you have already submitted 40 tasks, these will be executed before shutdown.
If you want to force shutdown. You can use shutdownNow:
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
emphasis mine
Which will abandon scheduled tasks, as you want.
N.B.: Stopping tasks in progress is another issue entirely.
Use shutdownNow() which will try to stop already executing task. Read this https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()
The shutdown of already executing task is an attempt only but will not be guaranteed to stop.
The shutdown() method only stops taking new tasks but already executing task will continue to execute.
UPDATE for the "Is there are any way to stop ExecutorService but waiting for completion currently executed tasks?"
One option I can think of is this -
The completionService.submit(getTask()); return object of type Future. You can write code to check if the task isDone(). After calling shutdownNow() you can check the tasks in a loop for isDone(). You need to store the tasks in list when returned by submit method.
Related
This question already has answers here:
ExecutorService, how to wait for all tasks to finish
(16 answers)
Closed 5 years ago.
I am in the process of writing a code where whenever a folder is encountered it is supposed to start a new thread. The code looks like,
p
ublic void diff(File x,File y){
ThreadPoolExecutor executor=new ThreadPoolExecutor(10,30,2000,unit,BlockingQueue<Runnable> queue)
if(x.isDirecotry && y.isDirectory){
Runnable thread=new DThread(x,y);
Future<?> result=executor.submit(thread);
if(result.isDone()){
LOGGER.debug(thread.toString()+"has completed");
}
I am using ThreadPoolExecutor for this purpose. If I shutdown the ThreadPoolExecutor then it will not take up any new Threads. But there is a possibility of new threads starting after starting. If I do not shutdown the ThreadPoolExecutor then all the threads are executed but in the end the JVM is not terminating.
Please help how can I shutdown the threadPool only when all the threads are executed so that the JVM gets terminated. Also suggest if there is better way of implemention of thread pool.I want to use thread pool so that I can use threads from the pool instead of creating new thread everytime.
The things that you are submitting should not be threads (subtypes of Thread). They should be simple tasks: implementations of Runnable or Callable.
The way to shutdown the executor is to call shutdown() or shutdownNow() on it. The javadoc summary says:
void shutdown() -
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
List<Runnable> shutdownNow() - Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
The latter attempts to stop the tasks by interrupting them. However, if the task code does not check for interrupts, it will run to completion. (It also stops accepting new tasks.)
It sounds like shutdown() does what you want to do.
The problem lies in the fact where should I place the shutdown method.
You call it when you want to start shutting down.
Because there is a possibility that new threads may be coming after shutdown is being called.
Stop calling them threads. They are tasks.
Any tasks that are submitted after shutdown() has been called are rejected.
My question is how can I check whether all the tasks have finished executing and only then call shutdown. If I call with the initiation of each task then it will not allow new tasks later on.
The correct time to call shutdown() is when your application has finished submitting tasks to the executor.
Maybe your conceptual problem is the scoping of the executor service. Your example code seems to show that each call to diff is creating a new service object. That means that you would have lots of independent thread pools ... and no reuse of threads. What you should really do is to create a "global" thread pool and have multiple calls to diff submit tasks to the same pool.
ScheduledExecutorService inherits two methods from the ExecutorService, shutdown() and shutdownNow(). The difference between them:
shutdown initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
shutdownNow attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
Now I want to halt processing of waiting tasks while I don't want to interrupt currently executing tasks. I can't interrupt the threads because third party libraries are involved and they don't deal well with interrupts :-( But I need to cancel scheduled tasks, that are not currently executing since most of them are scheduled in an hour or so.
What's the best way to deal with this? What options Do I have?
Sounds like calling setExecuteExistingDelayedTasksAfterShutdownPolicy(false) on your executor should do the trick:
Sets the policy on whether to execute existing delayed tasks even when
this executor has been shutdown. In this case, these tasks will only
terminate upon shutdownNow, or after setting the policy to false when
already shutdown. This value is by default true.
Since it's true by default, these tasks are executed. If you set it to false, they shouldn't be executed any more. This shouldn't be confused with submitted tasks which the docs you quote refer to. They are just waiting in the queue to be executed right away, when there is a free worker.
I have used shutDown() and shutDownNow(), but both method does not stop all threads immediately. shutDownNow() is more preferable among these two but it waits for the running thread to complete it's task. In my scenario I have a huge task dealing with postgres database and I want to shut down that thread immediately without waiting for the completion of execution.
What is the way to shut down the all the threads immediately?
shutdownNow :
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
This method does not wait for actively executing tasks to terminate. Use awaitTermination to do that.
shutdown :
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
This method does not wait for previously submitted tasks to complete execution. Use awaitTermination to do that.
http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ExecutorService.html#shutdown%28%29
or you can see this :
You can use an ExecutorService instead which combines a thread pool with a queue of tasks.
ExecutorService service = Executors.newCachedThreadPool();
// or
ExecutorService service = Executors.newFixedThreadPool(THREADS);
// submit as many tasks as you want.
// tasks must honour interrupts to be stopped externally.
Future future = service.submit(new MyRunnable());
// to cancel an individual task
future.cancel(true);
// when finished shutdown
service.shutdown();
The single "clean" way to stop the threads is, if you have some loops inside, to stop the loop through some boolean variable like "stopThread", and you have to handle the variable.
Example:
public void run(){
for(int i=0; i<1000000 && (!stopThread) ; i++){
// do something
}
}
I doubt that it's possible in safe way.
The safe way to shutdown the executor service as per oracle documentation
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
If you really want to follow a crude way, I will suggest a solution which may not be 100% accurate and I don't recommend personally. I would like to use ExecutorService methods instead of this crude solution.
Create your own thread and thread pool as per this article
Add one more boolean in Worker thread - boolean runNow = true;
run method of Worker thread will be like this : while ( runNow) { // your logic }
When you want to shut down all the threads, add one more method in ThreadPoolManager. Iterate through myQueue and interrupt all Runnables. Catch that interrupted exception and make the boolean runNow as false.
Depending on how you call the database, you could try to explicitly cancel the query. Se related question.
I am using ExecutorService in Java web server application for executing some computational tasks in parallel style and then calling shutdown() with awaitTermination() to wait for all tasks to be done. Whole computation can sometimes take dozens of minutes.
The thing is awaitTermination() method blocks the main thread until timeout elapsed (or interrupted) but I just want to start the tasks and immediatedly respond to client and after competition of all tasks shutdown the service (following conventions to always close the thread pool).
So my question, is there a way how I can be notified when all tasks are done so I could call the shutdown() method? Listener or something..
Thanks!
You are trying to solve a problem that doesn’t exist. Consider the documentation of ExecutorService.shutdown():
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. …
This method does not wait for previously submitted tasks to complete execution.
In other words, just calling shutdown() does already everything you want
It finishes all previously submitted tasks
It initiates a shutdown
It does not wait
The only obstacle is that you are calling awaitTermination despite the fact that you don’t want to wait, which has a trivial solution: don’t call awaitTermination.
The confusion arises because in your question you are asking “how I can be notified when all tasks are done so I could call the shutdown() method” but that is in contradiction to what you are actually doing in the code. You are calling awaitTermination after shutDown so you are not waiting in order to initiate the shutdown, but initiating the shutdown first and waiting for its completion then, which is the purpose of awaitTermination, waiting for the completion of the shutdown.
To put it in one sentence, just call shutDown after submission in order to shutdown the service after completion of all submitted jobs and don’t call awaitTermination unless you really want to wait for the termination.
Thanks to comment from VGR I solved my problem with creating yet another Thread in which I wrapped my existing code like this:
Thread thread = new Thread(() -> {
ExecutorService service = Executors.newCachedThreadPool();
collection.forEach(item -> service.submit(() -> {
// some computational work
});
try {
service.shutdown()
service.awaitTermination(2, TimeUnit.HOURS);
catch (InterruptedException iEx) {
// handle exception
}
});
thread.start();
return ResponseToClient();
I am using an ExecutorService (a ThreadPoolExecutor) to run (and queue) a lot of tasks. I am attempting to write some shut down code that is as graceful as possible.
ExecutorService has two ways of shutting down:
I can call ExecutorService.shutdown() and then ExecutorService.awaitTermination(...).
I can call ExecutorService.shutdownNow().
According to the JavaDoc, the shutdown command:
Initiates an orderly shutdown in which previously submitted
tasks are executed, but no new tasks will be accepted.
And the shutdownNow command:
Attempts to stop all actively executing tasks, halts the
processing of waiting tasks, and returns a list of the tasks that were
awaiting execution.
I want something in between these two options.
I want to call a command that:
a. Completes the currently active task or tasks (like shutdown).
b. Halts the processing of waiting tasks (like shutdownNow).
For example: suppose I have a ThreadPoolExecutor with 3 threads. It currently has 50 tasks in the queue with the first 3 actively running. I want to allow those 3 active tasks to complete but I do not want the remaining 47 tasks to start.
I believe I can shutdown the ExecutorService this way by keeping a list of Future objects around and then calling cancel on all of them. But since tasks are being submitted to this ExecutorService from multiple threads, there would not be a clean way to do this.
I'm really hoping I'm missing something obvious or that there's a way to do it cleanly.
Thanks for any help.
I ran into this issue recently. There may be a more elegant approach, but my solution is to first call shutdown(), then pull out the BlockingQueue being used by the ThreadPoolExecutor and call clear() on it (or else drain it to another Collection for storage). Finally, calling awaitTermination() allows the thread pool to finish what's currently on its plate.
For example:
public static void shutdownPool(boolean awaitTermination) throws InterruptedException {
//call shutdown to prevent new tasks from being submitted
executor.shutdown();
//get a reference to the Queue
final BlockingQueue<Runnable> blockingQueue = executor.getQueue();
//clear the Queue
blockingQueue.clear();
//or else copy its contents here with a while loop and remove()
//wait for active tasks to be completed
if (awaitTermination) {
executor.awaitTermination(SHUTDOWN_TIMEOUT, TimeUnit.SECONDS);
}
}
This method would be implemented in the directing class wrapping the ThreadPoolExecutor with the reference executor.
It's important to note the following from the ThreadPoolExecutor.getQueue() javadoc:
Access to the task queue is intended primarily for debugging and
monitoring. This queue may be in active use. Retrieving the task queue
does not prevent queued tasks from executing.
This highlights the fact that additional tasks may be polled from the BlockingQueue while you drain it. However, all BlockingQueue implementations are thread-safe according to that interface's documentation, so this shouldn't cause problems.
The shutdownNow() is exactly what you need. You've missed the 1st word Attempts and the entire 2nd paragraph of its javadoc:
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
So, only tasks which are checking Thread#isInterrupted() on a regular basis (e.g. in a while (!Thread.currentThread().isInterrupted()) loop or something), will be terminated. But if you aren't checking on that in your task, it will still keep running.
You can wrap each submitted task with a little extra logic
wrapper = new Runnable()
public void run()
if(executorService.isShutdown())
throw new Error("shutdown");
task.run();
executorService.submit(wrapper);
the overhead of extra checking is negligible. After executor is shutdown, the wrappers will still be executed, but the original tasks won't.