We Keep Coding

Properly shut down ThreadPoolExecutors

I'm new to java concurrency an would like to ask the following basic question. I'm creating a ThreadPoolExecutor for imporving performance as follows:
int n = Runtime.getRuntime().availableProcessors()
ExecutorService executor = Executors.newFixedThreadPool(n);
for( int i = 0; i < n; i++)
executor.execute(new Work());
After all thread in the thread pool have finished their tasks I need to shutdown the pool properly. I would tried this:
while(true){
if(executor.isTerminated()){
executor.shutdownNow();
break;
}
}
But I'm not sure about that because I think we waste a lot of processors resources to queriyng the executor for termination.
What is the right solution for that?
UPD: Runnable task:
public class Work implements Runnable{
private String sql;
public Work() {
//init sql
}
#Override
public void run() {
JdbcTemplate template = new JdbcTemplate(dataSource);
#SuppressWarnings("unchecked")
List<Integer> ints = template.queryForList(sql, Integer.class);
//Storing the list into a global cache
}
}

There seems to be something mystical around shutting down an ExecutorService.
From the documentation of shutdown():
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
So all you have to do is to invoke shutdown() after you have submitted all your tasks, the exact time doesn’t matter. They don’t have to be completed at that time. The ThreadPoolExecutor will finish all tasks and then clean up all resources.
And it will do so regardless of whether you wait for it or not. So you don’t need to wait, just invoke shutdown() when you are confident that you will not submit new tasks, the rest will happen as soon as possible.

It says:
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt(), so any task that fails to respond to
interrupts may never terminate.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()
So use awaitTermination instead. And for threads that take time, use a boolean variable as volatile and check it if it is set outside.If set then exit etc. something like that

try {
executor = Executors.newSingleThreadExecutor();
future = executor.submit(task);
executor.shutdown();
executor.awaitTermination(5, TimeUnit.SECONDS);
}
finally {
if (Objects.nonNull(executor) && !executor.isTerminated()) {
LOGGER.error("cancelling all non-finished tasks");
}
if (Objects.nonNull(executor)) {
executor.shutdownNow();
LOGGER.info("shutdown finished");
}
}
This way you shutdown executor and waiting for 5 seconds to complete all tasks and then finally calling executor.shutdownNow() to completely kill the executor.
This is the best way to shutdown executor.

Executor service in a thread spawned by another executor service

Have been facing trouble with the right usage of executor service. I am a new bee to executor and tried lot over web to resolve this. But couldn't justify it more.
i will try to put down my requirement and implementation done.
I have an infinite loop to spawn the threads and the maximum limit any time should be 50.
public void createWorkers(){
ExecutorService executor = Executors.newFixedThreadPool(50);
while(true) {
executor.execute(innerRunnableThread); // a runnable thread
}
}
QUERY : since it's an infinite loop, how to shut down the executor service (executor.shutdown()) ?
"innerRunnableThread" mentioned in the first point has the below code :
public class InnerRunnableThread implements Runnable {
#Override
public void run() {
ExecutorService innerExecutor = Executors.newFixedThreadPool(10);
for (int i=0; i<10;i++){
executor.execute(detailRunnableThread); // a runnable thread
}
innerExecutor.shutdown();
try {
innerExecutor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
loggr.error("exception thrown while spawning threads");
}
}
}
QUERY :
1. Parent thread ( in the 1st point), limit given as 50 threads. But with this implementation, parent threads not spawned more than 5. What is the reason for it ?
COMBINED QUERY :
With this approach, excluding shut down of the executor service in first point, there were thousands of threads lying un-terminated at the server. Finally leading to shutdown of this application and couldn't start any more ?
Kind help on this particular approach will be very helpful along with the better alternatives to achieve this logic.
Thanks alot.

Java Concurrency in Practice: race condition in BoundedExecutor?

There's something odd about the implementation of the BoundedExecutor in the book Java Concurrency in Practice.
It's supposed to throttle task submission to the Executor by blocking the submitting thread when there are enough threads either queued or running in the Executor.
This is the implementation (after adding the missing rethrow in the catch clause):
public class BoundedExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command) throws InterruptedException, RejectedExecutionException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
#Override public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
throw e;
}
}
When I instantiate the BoundedExecutor with an Executors.newCachedThreadPool() and a bound of 4, I would expect the number of threads instantiated by the cached thread pool to never exceed 4. In practice, however, it does. I've gotten this little test program to create as much as 11 threads:
public static void main(String[] args) throws Exception {
class CountingThreadFactory implements ThreadFactory {
int count;
#Override public Thread newThread(Runnable r) {
++count;
return new Thread(r);
}
}
List<Integer> counts = new ArrayList<Integer>();
for (int n = 0; n < 100; ++n) {
CountingThreadFactory countingThreadFactory = new CountingThreadFactory();
ExecutorService exec = Executors.newCachedThreadPool(countingThreadFactory);
try {
BoundedExecutor be = new BoundedExecutor(exec, 4);
for (int i = 0; i < 20000; ++i) {
be.submitTask(new Runnable() {
#Override public void run() {}
});
}
} finally {
exec.shutdown();
}
counts.add(countingThreadFactory.count);
}
System.out.println(Collections.max(counts));
}
I think there's a tiny little time frame between the release of the semaphore and the task ending, where another thread can aquire a permit and submit a task while the releasing thread hasn't finished yet. In other words, it has a race condition.
Can someone confirm this?

BoundedExecutor was indeed intended as an illustration of how to throttle task submission, not as a way to place a bound on thread pool size. There are more direct ways to achieve the latter, as at least one comment pointed out.
But the other answers don't mention the text in the book that says to use an unbounded queue and to
set the bound on the semaphore to be equal to the pool size plus the
number of queued tasks you want to allow, since the semaphore is
bounding the number of tasks both currently executing and awaiting
execution. [JCiP, end of section 8.3.3]
By mentioning unbounded queues and pool size, we were implying (apparently not very clearly) the use of a thread pool of bounded size.
What has always bothered me about BoundedExecutor, however, is that it doesn't implement the ExecutorService interface. A modern way to achieve similar functionality and still implement the standard interfaces would be to use Guava's listeningDecorator method and ForwardingListeningExecutorService class.

You are correct in your analysis of the race condition. There is no synchronization guarantees between the ExecutorService & the Semaphore.
However, I do not know if throttling the number of threads is what the BoundedExecutor is used for. I think it is more for throttling the number of tasks submitted to the service. Imagine if you have 5 million tasks that need to submit, and if you submit more then 10,000 of them you run out of memory.
Well you only will ever have 4 threads running at any given time, why would you want to try and queue up all 5 millions tasks? You can use a construct similar to this to throttle the number of tasks queued up at any given time. What you should get out of this is that at any given time there are only 4 tasks running.
Obviously the resolution to this is to use a Executors.newFixedThreadPool(4).

I see as much as 9 threads created at once. I suspect there is a race condition which causes there to be more thread than required.
This could be because there is before and after running the task work to be done. This means that even though there is only 4 thread inside your block of code, there is a number of thread stopping a previous task or getting ready to start a new task.
i.e. the thread does a release() while it is still running. Even though its the last thing you do its not the last thing it does before acquiring a new task.

Threads are destroyed?

I create a pool of threads with an Executors.newFixedThreadPool, but after a time I noticed that some of them stoped to answer (call this method below).
They was destroyed? I doing synchonization and the system continues when all threads set that finished the task, but with this, the system enter in deadlock.
They was destroyed? What can I do to prevent or handle this?
//this is the method that threads call when finish the work
synchronized void setTaskFinish(){
System.out.println(Thread.currentThread().getName() + " finishing the work.");
finishedThreads++;
System.out.println(finishedThreads +" finished the work.");
if(finishedThreads == numberThreads){
finishedThreads = 0;
this.notify();
}
}
//this is the method that creates the thread
//I dont know much about this executors yet, so I think it have errors
public void execute(int numberThreads) {
for (int i = 0; i < numberThreads; i++) {
crawlers.add(new SlaveCrawler());
}
ExecutorService threadExecutor = Executors.newFixedThreadPool(numberThreads);
for (int i = 0, n = crawlers.size(); i < n; i++) {
threadExecutor.execute((Runnable) crawlers.get(i));
}
threadExecutor.shutdown();
}
EDIT: I changed entirely my logic. I have not done much testing, but everything seems ok now. Maybe it was something wrong in my old logic.

The executor won't destroy threads that are busy executing your tasks. If you assigned 60 tasks, and only 55 "complete" (more precisely, your setTaskFinish() method is only invoked 55 times), it could be an exception terminating your task prematurely. Deadlock is another possibility, but not the first I would examine.
What happens when your task throws an unchecked exception like RuntimeException? Does it call setTaskFinish() from a finally block? Why are you managing concurrency using synchronized instead of an AtomicInteger?

Is adding tasks to BlockingQueue of ThreadPoolExecutor advisable?

The JavaDoc for ThreadPoolExecutor is unclear on whether it is acceptable to add tasks directly to the BlockingQueue backing the executor. The docs say calling executor.getQueue() is "intended primarily for debugging and monitoring".
I'm constructing a ThreadPoolExecutor with my own BlockingQueue. I retain a reference to the queue so I can add tasks to it directly. The same queue is returned by getQueue() so I assume the admonition in getQueue() applies to a reference to the backing queue acquired through my means.
Example
General pattern of the code is:
int n = ...; // number of threads
queue = new ArrayBlockingQueue<Runnable>(queueSize);
executor = new ThreadPoolExecutor(n, n, 1, TimeUnit.HOURS, queue);
executor.prestartAllCoreThreads();
// ...
while (...) {
Runnable job = ...;
queue.offer(job, 1, TimeUnit.HOURS);
}
while (jobsOutstanding.get() != 0) {
try {
Thread.sleep(...);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
executor.shutdownNow();
queue.offer() vs executor.execute()
As I understand it, the typical use is to add tasks via executor.execute(). The approach in my example above has the benefit of blocking on the queue whereas execute() fails immediately if the queue is full and rejects my task. I also like that submitting jobs interacts with a blocking queue; this feels more "pure" producer-consumer to me.
An implication of adding tasks to the queue directly: I must call prestartAllCoreThreads() otherwise no worker threads are running. Assuming no other interactions with the executor, nothing will be monitoring the queue (examination of ThreadPoolExecutor source confirms this). This also implies for direct enqueuing that the ThreadPoolExecutor must additionally be configured for > 0 core threads and mustn't be configured to allow core threads to timeout.
tl;dr
Given a ThreadPoolExecutor configured as follows:
core threads > 0
core threads aren't allowed to timeout
core threads are prestarted
hold a reference to the BlockingQueue backing the executor
Is it acceptable to add tasks directly to the queue instead of calling executor.execute()?
Related
This question ( producer/consumer work queues ) is similar, but doesn't specifically cover adding to the queue directly.

One trick is to implement a custom subclass of ArrayBlockingQueue and to override the offer() method to call your blocking version, then you can still use the normal code path.
queue = new ArrayBlockingQueue<Runnable>(queueSize) {
#Override public boolean offer(Runnable runnable) {
try {
return offer(runnable, 1, TimeUnit.HOURS);
} catch(InterruptedException e) {
// return interrupt status to caller
Thread.currentThread().interrupt();
}
return false;
}
};
(as you can probably guess, i think calling offer directly on the queue as your normal code path is probably a bad idea).

If it were me, I would prefer using Executor#execute() over Queue#offer(), simply because I'm using everything else from java.util.concurrent already.
Your question is a good one, and it piqued my interest, so I took a look at the source for ThreadPoolExecutor#execute():
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
if (poolSize >= corePoolSize || !addIfUnderCorePoolSize(command)) {
if (runState == RUNNING && workQueue.offer(command)) {
if (runState != RUNNING || poolSize == 0)
ensureQueuedTaskHandled(command);
}
else if (!addIfUnderMaximumPoolSize(command))
reject(command); // is shutdown or saturated
}
}
We can see that execute itself calls offer() on the work queue, but not before doing some nice, tasty pool manipulations if necessary. For that reason, I'd think that it'd be advisable to use execute(); not using it may (although I don't know for certain) cause the pool to operate in a non-optimal way. However, I don't think that using offer() will break the executor - it looks like tasks are pulled off the queue using the following (also from ThreadPoolExecutor):
Runnable getTask() {
for (;;) {
try {
int state = runState;
if (state > SHUTDOWN)
return null;
Runnable r;
if (state == SHUTDOWN) // Help drain queue
r = workQueue.poll();
else if (poolSize > corePoolSize || allowCoreThreadTimeOut)
r = workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS);
else
r = workQueue.take();
if (r != null)
return r;
if (workerCanExit()) {
if (runState >= SHUTDOWN) // Wake up others
interruptIdleWorkers();
return null;
}
// Else retry
} catch (InterruptedException ie) {
// On interruption, re-check runState
}
}
}
This getTask() method is just called from within a loop, so if the executor's not shutting down, it'd block until a new task was given to the queue (regardless of from where it came from).
Note: Even though I've posted code snippets from source here, we can't rely on them for a definitive answer - we should only be coding to the API. We don't know how the implementation of execute() will change over time.

One can actually configure behavior of the pool when the queue is full, by specifying a RejectedExecutionHandler at instantiation. ThreadPoolExecutor defines four policies as inner classes, including AbortPolicy, DiscardOldestPolicy, DiscardPolicy, as well as my personal favorite, CallerRunsPolicy, which runs the new job in the controlling thread.
For example:
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(
nproc, // core size
nproc, // max size
60, // idle timeout
TimeUnit.SECONDS,
new ArrayBlockingQueue<Runnable>(4096, true), // Fairness = true guarantees FIFO
new ThreadPoolExecutor.CallerRunsPolicy() ); // If we have to reject a task, run it in the calling thread.
The behavior desired in the question can be obtained using something like:
public class BlockingPolicy implements RejectedExecutionHandler {
void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
executor.getQueue.put(r); // Self contained, no queue reference needed.
}
At some point the queue must be accessed. The best place to do so is in a self-contained RejectedExecutionHandler, which saves any code duplication or potenial bugs arising from direct manipulation of the queue at the scope of the pool object. Note that the handlers included in ThreadPoolExecutor themselves use getQueue().

It's a very important question if the queue you're using is a completely different implementation from the standard in-memory LinkedBlockingQueue or ArrayBlockingQueue.
For instance if you're implementing the producer-consumer pattern using several producers on different machines, and use a queuing mechanism based on a separate persistence subsystem (like Redis), then the question becomes relevant on its own, even if you don't want a blocking offer() like the OP.
So the given answer, that prestartAllCoreThreads() has to be called (or enough times prestartCoreThread()) for the worker threads to be available and running, is important enough to be stressed.

If required, we can also use a parking lot which separates main processing from rejected tasks -
final CountDownLatch taskCounter = new CountDownLatch(TASKCOUNT);
final List<Runnable> taskParking = new LinkedList<Runnable>();
BlockingQueue<Runnable> taskPool = new ArrayBlockingQueue<Runnable>(1);
RejectedExecutionHandler rejectionHandler = new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
System.err.println(Thread.currentThread().getName() + " -->rejection reported - adding to parking lot " + r);
taskCounter.countDown();
taskParking.add(r);
}
};
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(5, 10, 1000, TimeUnit.SECONDS, taskPool, rejectionHandler);
for(int i=0 ; i<TASKCOUNT; i++){
//main
threadPoolExecutor.submit(getRandomTask());
}
taskCounter.await(TASKCOUNT * 5 , TimeUnit.SECONDS);
System.out.println("Checking the parking lot..." + taskParking);
while(taskParking.size() > 0){
Runnable r = taskParking.remove(0);
System.out.println("Running from parking lot..." + r);
if(taskParking.size() > LIMIT){
waitForSometime(...);
}
threadPoolExecutor.submit(r);
}
threadPoolExecutor.shutdown();