We Keep Coding

Skip to next task in a single threaded ExecutorSerivce?

I am considering an implementation of an ExecutorService to run a series of tasks. I plan to use the internal queue to have a few tasks waiting for their turn to run. Is there some way to interrupt the task (the Runnable) that is currently running in an ExecutorService thread, and keep the thread alive to run the next task? Or is only possible to call .shutdown() and then create a new ExecutorService?
I have found this and wanted to know if there are any other solutions.

Instead of interfering with the threads you may want to have a Task class (that extends or wraps the Runnable) which implements an interrupt mechanism (e.g. a boolean flag).
When you execute your task you need to check this flag periodically and if it is set, the task should stop what it is doing. You might want to return a specific result at this point, that tells your code that the task was cancelled succesfully.
If a user now decides that he no longer requires the results from this task,
you will have to set this flag. However the task might have already completed at this point of time so you still need to deal with the cases where the result already exists but the user does no longer care about it.
An interrupt on a thread level does not guarantee that the thread stops working. This will only work if the thread is in a state where it can receive an interrupt.
Also you should not interfere with the Threads of the ExecutorSerivce directly, as you might unintentionally stop a different task or stop the ExecutorSerivce from working properly.

Why would you want to kill that task and continue with the next one? If it is a question of times you can define that the threads that are taking longer than you declared in the method that executes them are automatically canceled. E.g:
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.invokeAll(Arrays.asList(new Task()), 60, TimeUnit.SECONDS); // Timeout of 60 seconds.
executor.shutdown();
If any of the threads takes longer than 60 seconds it will throw a cancellation.Exception() that you must catch

How to shutdown the ThreadServiceExecutor where the number of threads is not initially known [duplicate]

This question already has answers here:
ExecutorService, how to wait for all tasks to finish
(16 answers)
Closed 5 years ago.
I am in the process of writing a code where whenever a folder is encountered it is supposed to start a new thread. The code looks like,
p
ublic void diff(File x,File y){
ThreadPoolExecutor executor=new ThreadPoolExecutor(10,30,2000,unit,BlockingQueue<Runnable> queue)
if(x.isDirecotry && y.isDirectory){
Runnable thread=new DThread(x,y);
Future<?> result=executor.submit(thread);
if(result.isDone()){
LOGGER.debug(thread.toString()+"has completed");
}
I am using ThreadPoolExecutor for this purpose. If I shutdown the ThreadPoolExecutor then it will not take up any new Threads. But there is a possibility of new threads starting after starting. If I do not shutdown the ThreadPoolExecutor then all the threads are executed but in the end the JVM is not terminating.
Please help how can I shutdown the threadPool only when all the threads are executed so that the JVM gets terminated. Also suggest if there is better way of implemention of thread pool.I want to use thread pool so that I can use threads from the pool instead of creating new thread everytime.

The things that you are submitting should not be threads (subtypes of Thread). They should be simple tasks: implementations of Runnable or Callable.
The way to shutdown the executor is to call shutdown() or shutdownNow() on it. The javadoc summary says:
void shutdown() -
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
List<Runnable> shutdownNow() - Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
The latter attempts to stop the tasks by interrupting them. However, if the task code does not check for interrupts, it will run to completion. (It also stops accepting new tasks.)
It sounds like shutdown() does what you want to do.
The problem lies in the fact where should I place the shutdown method.
You call it when you want to start shutting down.
Because there is a possibility that new threads may be coming after shutdown is being called.
Stop calling them threads. They are tasks.
Any tasks that are submitted after shutdown() has been called are rejected.
My question is how can I check whether all the tasks have finished executing and only then call shutdown. If I call with the initiation of each task then it will not allow new tasks later on.
The correct time to call shutdown() is when your application has finished submitting tasks to the executor.
Maybe your conceptual problem is the scoping of the executor service. Your example code seems to show that each call to diff is creating a new service object. That means that you would have lots of independent thread pools ... and no reuse of threads. What you should really do is to create a "global" thread pool and have multiple calls to diff submit tasks to the same pool.

ScheduledExecutorService: How can I wait for cancelled tasks to finish?

I have scheduled some tasks in a ScheduledExecutorService. On shutdown, I'd like to cancel them, and release a database lock once they are all done.
How can I wait for the cancelled tasks to finish processing?
Attempted solutions
according to my tests, future.cancel() does not block until the task has stopped executing
invoking future.get() after future.cancel() results in an immediate CancelledExecutionException
I'd rather not wait for shutdown of the entire executor, as it is shared among components

How can I wait for the cancelled tasks to finish processing?
The problem with this is that cancellation is done on a best effort basis. The javadoc states
Attempts to cancel execution of this task. This attempt will fail if
the task has already completed, has already been cancelled, or could
not be cancelled for some other reason.
Typically, cancellation is implemented with interruption and interruption is a convention. Nothing guarantees that it will be implemented correctly. So even if you do send a cancel, nothing guarantees that the underlying task (if already running) will fulfill the cancellation.
There's no reliable way to implement your use case.

With a Java ExecutorService, how do I complete actively executing tasks but halt the processing of waiting tasks?

I am using an ExecutorService (a ThreadPoolExecutor) to run (and queue) a lot of tasks. I am attempting to write some shut down code that is as graceful as possible.
ExecutorService has two ways of shutting down:
I can call ExecutorService.shutdown() and then ExecutorService.awaitTermination(...).
I can call ExecutorService.shutdownNow().
According to the JavaDoc, the shutdown command:
Initiates an orderly shutdown in which previously submitted
tasks are executed, but no new tasks will be accepted.
And the shutdownNow command:
Attempts to stop all actively executing tasks, halts the
processing of waiting tasks, and returns a list of the tasks that were
awaiting execution.
I want something in between these two options.
I want to call a command that:
a. Completes the currently active task or tasks (like shutdown).
b. Halts the processing of waiting tasks (like shutdownNow).
For example: suppose I have a ThreadPoolExecutor with 3 threads. It currently has 50 tasks in the queue with the first 3 actively running. I want to allow those 3 active tasks to complete but I do not want the remaining 47 tasks to start.
I believe I can shutdown the ExecutorService this way by keeping a list of Future objects around and then calling cancel on all of them. But since tasks are being submitted to this ExecutorService from multiple threads, there would not be a clean way to do this.
I'm really hoping I'm missing something obvious or that there's a way to do it cleanly.
Thanks for any help.

I ran into this issue recently. There may be a more elegant approach, but my solution is to first call shutdown(), then pull out the BlockingQueue being used by the ThreadPoolExecutor and call clear() on it (or else drain it to another Collection for storage). Finally, calling awaitTermination() allows the thread pool to finish what's currently on its plate.
For example:
public static void shutdownPool(boolean awaitTermination) throws InterruptedException {
//call shutdown to prevent new tasks from being submitted
executor.shutdown();
//get a reference to the Queue
final BlockingQueue<Runnable> blockingQueue = executor.getQueue();
//clear the Queue
blockingQueue.clear();
//or else copy its contents here with a while loop and remove()
//wait for active tasks to be completed
if (awaitTermination) {
executor.awaitTermination(SHUTDOWN_TIMEOUT, TimeUnit.SECONDS);
}
}
This method would be implemented in the directing class wrapping the ThreadPoolExecutor with the reference executor.
It's important to note the following from the ThreadPoolExecutor.getQueue() javadoc:
Access to the task queue is intended primarily for debugging and
monitoring. This queue may be in active use. Retrieving the task queue
does not prevent queued tasks from executing.
This highlights the fact that additional tasks may be polled from the BlockingQueue while you drain it. However, all BlockingQueue implementations are thread-safe according to that interface's documentation, so this shouldn't cause problems.

The shutdownNow() is exactly what you need. You've missed the 1st word Attempts and the entire 2nd paragraph of its javadoc:
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
So, only tasks which are checking Thread#isInterrupted() on a regular basis (e.g. in a while (!Thread.currentThread().isInterrupted()) loop or something), will be terminated. But if you aren't checking on that in your task, it will still keep running.

You can wrap each submitted task with a little extra logic
wrapper = new Runnable()
public void run()
if(executorService.isShutdown())
throw new Error("shutdown");
task.run();
executorService.submit(wrapper);
the overhead of extra checking is negligible. After executor is shutdown, the wrappers will still be executed, but the original tasks won't.

How can I get the jobs that are in execution in a ScheduledThreadPoolExecutor?

I need to force the release of resources when a task is interrupted. For that, I implemented this solution. But with this, when I call shutdown() all the task in the ScheduledThreadPoolExecutor.getQueue() are forced correctly, but some of my jobs kept the resources. I checked very well the behavior and I figured out this: when a task is in execution, he is not present in the ScheduledThreadPoolExecutor queue (I know it sounds obvious). The problem is that I need to force the release of resources for all the jobs (in the queue or in execution)
So, How can I get the jobs that are in execution? Do you have a better idea?

You can maintain a list of all the Future's you create when you submit the jobs.
Use this list to cancel all futures.

Don't you want to call
executor.shutdownNow()
that will attempt to cancel currently running tasks (using Thread.interrupt so you'll need to implement an 'interruption policy' in each task that uses the interrupt flag).
from the javadoc
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt, so any task that fails to respond to interrupts may never terminate.
This will return a list of waiting tasks, so you can always put this back onto a 'wait list' rather than loose them completely. You might also want follow that up with an 'await termination' to avoid run away code. For example, executor.awaitTermination(...).
tempus-fugit has some handy classes for handling this. You just call
shutdown(executor).waitingForShutdown(millis(400));
see here for details.
Also, the solution you outline in the blog post; I'm not sure if that's quite right. Future.cancel will only stop the task from being scheduled. If you were to update the example in the blog to allow interruption (ie cancel(true), it'd be equivalent (more or less) with the shutdownNow. That is to say, it will call interrupt on the underlying task which (if you've implemented an interruption policy) will stop it processing. As for cleaning up after interruption, you just need to make sure that you handle that appropriately within the interruption policy implementation. The upshot is that I think you can cancel and cleanup correctly using shutdownNow (or cancel(true))