Deadlock in ThreadPoolExecutor - java

Encountered a situation when ThreadPoolExecutor is parked in execute(Runnable) function while all the ThreadPool threads are waiting in getTask func, workQueue is empty.
Does anybody have any ideas?
The ThreadPoolExecutor is created with ArrayBlockingQueue, and corePoolSize == maximumPoolSize = 4
[Edit] To be more precise, the thread is blocked in ThreadPoolExecutor.exec(Runnable command) func. It has the task to execute, but doesn't do it.
[Edit2] The executor is blocked somewhere inside the working queue (ArrayBlockingQueue).
[Edit3] The callstack:
thread = front_end(224)
at sun.misc.Unsafe.park(Native methord)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
at
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at java.util.concurrent.ArrayBlockingQueue.offer(ArrayBlockingQueue.java:224)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:653)
at net.listenThread.WorkersPool.execute(WorkersPool.java:45)
at the same time the workQueue is empty (checked using remote debug)
[Edit4] Code working with ThreadPoolExecutor:
public WorkersPool(int size) {
pool = new ThreadPoolExecutor(size, size, IDLE_WORKER_THREAD_TIMEOUT, TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(WORK_QUEUE_CAPACITY),
new ThreadFactory() {
#NotNull
private final AtomicInteger threadsCount = new AtomicInteger(0);
#NotNull
public Thread newThread(#NotNull Runnable r) {
final Thread thread = new Thread(r);
thread.setName("net_worker_" + threadsCount.incrementAndGet());
return thread;
}
},
new RejectedExecutionHandler() {
public void rejectedExecution(#Nullable Runnable r, #Nullable ThreadPoolExecutor executor) {
Verify.warning("new task " + r + " is discarded");
}
});
}
public void execute(#NotNull Runnable task) {
pool.execute(task);
}
public void stopWorkers() throws WorkersTerminationFailedException {
pool.shutdownNow();
try {
pool.awaitTermination(THREAD_TERMINATION_WAIT_TIME, TimeUnit.SECONDS);
} catch (InterruptedException e) {
throw new WorkersTerminationFailedException("Workers-pool termination failed", e);
}
}
}

It sounds like it is a bug with an JVM's older than 6u21. There was an issue in the compiled native code for some (maybe all) OS's.
From the link:
The bug is caused by missing memory barriers in various Parker::park()
paths that can result in lost wakeups and hangs. (Note that
PlatformEvent::park used by built-in synchronization is not vulnerable
to the issue). -XX:+UseMembar constitues a work-around because the
membar barrier in the state transition logic hides the problem in
Parker::. (that is, there's nothing wrong with the use -UseMembar
mechanism, but +UseMembar hides the bug Parker::). This is a day-one
bug introduced with the addition of java.util.concurrent in JDK 5.0.
I developed a simple C mode of the failure and it seems more likely to
manifest on modern AMD and Nehalem platforms, likely because of deeper
store buffers that take longer to drain. I provided a tentative fix
to Doug Lea for Parker::park which appears to eliminate the bug. I'll
be delivering this fix to runtime. (I'll also augment the CR with
additional test cases and and a longer explanation). This is likely a
good candidate for back-ports.
Link: JVM Bug
Workarounds are available, but you would probably be best off just getting the most recent copy of Java.

I don't see any locking in the code of ThreadPoolExecutor's execute(Runnable). The only variable there is the workQueue. What sort of BlockingQueue did you provide to your ThreadPoolExecutor?
On the topic of deadlocks:
You can confirm this is a deadlock by examining the Full Thread Dump, as provided by <ctrl><break> on Windows or kill -QUIT on UNIX systems.
Once you have that data, you can examine the threads. Here is a pertinent excerpt from Sun's article on examining thread dumps (suggested reading):
For hanging, deadlocked or frozen programs: If you think your program is hanging, generate a stack trace and examine the threads in states MW or CW. If the program is deadlocked then some of the system threads will probably show up as the current threads, because there is nothing else for the JVM to do.
On a lighter note: if you are running in an IDE, can you ensure that there are no breakpoints enabled in these methods.

This deadlock probably because you run task from executor itself. For example, you submit one task, and this one fires another 4 tasks. If you have pool size equals to 4, then you just totally overflow it and last task will wait until someone of task return value. But the first task wait for all forked tasks to be completed.

As someone already mentioned, this sounds like normal behaviour, the ThreadPoolExecutor is just waiting to do some work. If you want to stop it, you need to call:
executor.shutdown()
to get it to terminate, usually followed by a executor.awaitTermination

The library code source is below (that's in fact a class from http://spymemcached.googlecode.com/files/memcached-2.4.2-sources.zip),
- a bit complicated - added protection against repeated calls of FutureTask if I'm not mistaken - but doesn't seem like deadlock prone - very simple ThreadPool usage:
package net.spy.memcached.transcoders;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.concurrent.FutureTask;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
import net.spy.memcached.CachedData;
import net.spy.memcached.compat.SpyObject;
/**
* Asynchronous transcoder.
*/
public class TranscodeService extends SpyObject {
private final ThreadPoolExecutor pool = new ThreadPoolExecutor(1, 10, 60L,
TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(100),
new ThreadPoolExecutor.DiscardPolicy());
/**
* Perform a decode.
*/
public <T> Future<T> decode(final Transcoder<T> tc,
final CachedData cachedData) {
assert !pool.isShutdown() : "Pool has already shut down.";
TranscodeService.Task<T> task = new TranscodeService.Task<T>(
new Callable<T>() {
public T call() {
return tc.decode(cachedData);
}
});
if (tc.asyncDecode(cachedData)) {
this.pool.execute(task);
}
return task;
}
/**
* Shut down the pool.
*/
public void shutdown() {
pool.shutdown();
}
/**
* Ask whether this service has been shut down.
*/
public boolean isShutdown() {
return pool.isShutdown();
}
private static class Task<T> extends FutureTask<T> {
private final AtomicBoolean isRunning = new AtomicBoolean(false);
public Task(Callable<T> callable) {
super(callable);
}
#Override
public T get() throws InterruptedException, ExecutionException {
this.run();
return super.get();
}
#Override
public T get(long timeout, TimeUnit unit) throws InterruptedException,
ExecutionException, TimeoutException {
this.run();
return super.get(timeout, unit);
}
#Override
public void run() {
if (this.isRunning.compareAndSet(false, true)) {
super.run();
}
}
}
}

Definitely strange.
But before writing your own TPE try:
another BlockingQueue impl., e.g. LinkedBlockingQueue
specify fairness=true in ArrayBlockingQueue, i.e. use new ArrayBlockingQueue(n, true)
From those two opts I would chose second one 'cause it's very strange that offer() being blocked; one reason that comes into mind - thread scheduling policy on your Linux. Just as an assumption.

Related

Shutdown executor without knowing when it will finish all the tasks

I have a task that should wait for a condition (OpenCms startup) and then notify some listeners.
to do this I used an ExecutorService:
public void check(final ExecutorService executorService) {
executorService.submit(() -> {
waitForInitialization();
notifyListeners();
});
}
private void waitForInitialization() {
while (OpenCms.getRunLevel() < OpenCms.RUNLEVEL_4_SERVLET_ACCESS) {
try {
TimeUnit.SECONDS.sleep(10);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
the question is how to shutdown this executor. I could use awaitTermination but I should provide a timeout which I don't know exactly. it could vary from one environment to the other.
the question is how to shutdown this executor. I could use awaitTermination but I should provide a timeout which I don't know exactly. it could vary from one environment to the other.
The question I would ask is do you need a timeout at all? Often if you know that a particular job will finish at some point I just wait for a timeout Long.MAX_VALUE – effectively forever. Other times I'll do something like:
threadPool.shutdown();
threadPool.awaitTermination(...) of some small value (maybe 10 seconds)
threadPool.shutdownNow(); to interrupt the threads
threadPool.awaitTermination(...); of Long.MAX_VALUE because I know the jobs will finish eventually
it could vary from one environment to the other.
If it could vary then maybe you should be able to calculate what a proper timeout would be for each environment?
Lastly, don't be afraid of passing in a ThreadFactory that creates daemon threads. For some jobs I shutdown() the thread-pool but never wait for them to complete because I don't care about their status so I create the threads in the pool with daemon enabled maybe using something like the following thread-factory.
/** Thread factory which sets name and optionally daemon */
public class PoolNameThreadFactory implements ThreadFactory {
private final String poolName;
private final Boolean daemon;
private final AtomicInteger threadNum = new AtomicInteger(0);
public PoolNameThreadFactory(String poolName) {
this(poolName, null);
}
public PoolNameThreadFactory(String poolName, boolean daemon) {
this(poolName, (Boolean) daemon);
}
private PoolNameThreadFactory(String poolName, Boolean daemon) {
this.poolName = poolName;
this.daemon = daemon;
}
#Override
public Thread newThread(Runnable r) {
Thread thread = new Thread(r);
thread.setName(poolName + '-' + threadNum.incrementAndGet());
if (daemon != null) {
thread.setDaemon(daemon);
}
return thread;
}
}
Based on the provided information, I'd clearly recommend an event-based approach. Especially knowing that in your own code there is a call like notifyListeners(). In fact, that's the way to go.
In summary, once the precondition is met somewhere in your app, just notify the listeners of this event. In your example, the "OpenCms run level" change is typically an event. So, just go for an Observer pattern, or a pub-sub model to observe or monitor these changes.
If you modify your approach, you will not have to worry about the waiting time around the initialization, except if you wish to handle the absence of event specifically. That would be done again after some timeout, but with the advantage of not blocking an executor thread.

Can I use Guava's AbstractExecutionThreadService for services that need to be interrupted?

I have a service that I would like to implement as a Google Guava Service.
The service basically runs a while (true) loop that processes events as they arrive on a BlockingQueue. Simplified sample code is available here:
https://gist.github.com/3354249
The problem is that the code blocks on BlockingQueue#take(), so the only way to stop the service is to interrupt its thread. Is this possible using Guava's AbstractExecutionThreadService?
Of course, in this case I could replace queue.take() with a polling loop using queue.poll(1, TimeUnit.SECONDS), thus removing the need for thread interruption. However:
I would like to avoid doing this, for both performance and code readability reasons
There are other cases where it is impossible to avoid thread interruption, e.g. if the service is blocked while reading bytes from an InputStream.
You can override executor() method to supply your own executor, which will then store reference to the thread into your field. Then you can easily interrupt the thread, if needed.
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference;
import com.google.common.util.concurrent.AbstractExecutionThreadService;
public abstract class InterruptibleExecutionThreadService extends AbstractExecutionThreadService {
private final AtomicReference<Thread> runningThread = new AtomicReference<Thread>(null);
#Override
protected Executor executor() {
return new Executor() {
#Override
public void execute(Runnable command) {
Thread thread = Executors.defaultThreadFactory().newThread(command);
runningThread.compareAndSet(null, thread);
try {
thread.setName(serviceName());
} catch (SecurityException e) {
// OK if we can't set the name in this environment.
}
thread.start();
}
};
}
protected void interruptRunningThread() {
Thread thread = runningThread.get();
if (thread != null) {
thread.interrupt();
}
}
}
I don't think interrupting the thread is really an option if you want to use an AbstractExecutionThreadService since there's not really any way to get a reference to the thread in order to call interrupt().
If you're using a BlockingQueue you either have to poll inside a while loop that checks if the service is still running, or you can use a sentinel value to alert the worker method that it needs to stop.
Examples:
Polling:
while(isRunning()) {
Value v = queue.poll(1, TimeUnit.SECONDS);
// do something with v
}
Sentinal value:
while(isRunning()) {
Value v = queue.take();
if(v == POISON) {
break;
}
// do something with v
}
I personally would try the polling solution and see what the performance is like. You might be surprised by how little that really effects the performance.
As for reading from an InputStream, if the InputStream is long-lived and has the potential to block indefinitely I don't think using an AbstractExecutionThreadService is really possible. You should instead use an AbstractService which creates and holds a reference to its own execution thread so that you can interrupt it in the doStop() method.

Killing thread after some specified time limit in Java

Is there a way to kill a child thread after some specified time limit in Java?
Edit: Also this particular thread may be blocked in its worst case (Thread is used to wait for a file modification and blocks until this event occurs), so im not sure that interrupt() will be successful?
Make use of ExecutorService to execute the Callable, checkout the methods wherein you can specify the timeout. E.g.
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.invokeAll(Arrays.asList(new Task()), 10, TimeUnit.MINUTES); // Timeout of 10 minutes.
executor.shutdown();
Here Task of course implements Callable.
Some helpful changes were introduced as part of JEP 266 in CompletableFuture since Java 9. Using orTimeout method, for now, it is possible to write it like:
CompletableFuture.runAsync(thread::run)
.orTimeout(30, TimeUnit.SECONDS)
.exceptionally(throwable -> {
log.error("An error occurred", throwable);
return null;
});
In Java 8, unfortunately, you should use some extra code. Here is an example of delegation pattern usage with help of Lombok:
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import java.time.Duration;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executors;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import java.util.concurrent.TimeoutException;
import static lombok.AccessLevel.PRIVATE;
import lombok.AllArgsConstructor;
import lombok.experimental.Delegate;
#AllArgsConstructor(access = PRIVATE)
public class TimeoutableCompletableFuture<T> extends CompletableFuture<T> {
public static TimeoutableCompletableFuture<Void> runAsync(
Runnable runnable) {
return new TimeoutableCompletableFuture<>(
CompletableFuture.runAsync(runnable));
}
#Delegate
private final CompletableFuture<T> baseFuture;
public TimeoutableCompletableFuture<T> orTimeout(Duration duration) {
final CompletableFuture<T> otherFuture = new CompletableFuture<>();
Executors.newScheduledThreadPool(
1,
new ThreadFactoryBuilder()
.setDaemon(true)
.setNameFormat("timeoutable-%d")
.build())
.schedule(() -> {
TimeoutException ex = new TimeoutException(
"Timeout after " + duration);
return otherFuture.completeExceptionally(ex);
}, duration.toMillis(), MILLISECONDS);
return new TimeoutableCompletableFuture<>(
baseFuture.applyToEither(otherFuture, a -> a));
}
}
Of course, the code above easily could be rewritten as just a static factory method:
public static CompletableFuture<Void> runAsyncOrTimeout(
Runnable runnable, long timeout, TimeUnit unit) {
CompletableFuture<Void> other = new CompletableFuture<>();
Executors.newScheduledThreadPool(
1,
new ThreadFactoryBuilder()
.setDaemon(true)
.setNameFormat("timeoutafter-%d")
.build())
.schedule(() -> {
TimeoutException ex = new TimeoutException(
"Timeout after " + timeout);
return other.completeExceptionally(ex);
}, timeout, unit);
return CompletableFuture.runAsync(runnable).applyToEither(other, a -> a);
}
Not directly; I think the simplest way is to join() on that thread with that time limit, and interrupt the thread if it's not done by the time the join ended.
So,
Thread t = ...
t.join(timelimit);
if (t.isAlive()) t.interrupt();
Notice I used interrupt instead of actually killing it, it's much safer. I would also recommend using executors instead of directly manipulating threads.
Why not interrupt() it after a particular time ? Your spawned thread will have to be able to handle an InterruptedException properly.
See this article (http://www.javaspecialists.eu/archive/Issue056.html) for more information on shutting down threads cleanly.
See also the Executor/Future framework, which provide useful methods for collecting results and/or terminating threads within particular time limits.
You can use AOP and a #Timeable annotation for your method from jcabi-aspects (I'm a developer):
#Timeable(limit = 1, unit = TimeUnit.SECONDS)
String load(String resource) {
// do something time consuming
}
When time limit is reached your thread will get interrupted() flag set to true and it's your job to handle this situation correctly and to stop execution. Normally it's done by Thread.sleep(..).
Killing a thread is generally a bad idea for reasons linked to for the API docs for Thread.
If you are dead set on killing, use a whole new process.
Otherwise the usual thing is to have the thread poll System.nanoTime, poll a (possible volatile) flag, queue a "poison pill" or something of that nature.
Brian's right, interrupting it is safer than "stopping" the thread.
What if the thread is locking on an object mid-modification, and suddenly gets stopped (which causes the lock to be released)? You get weird results.
Do not use destroy() since that does not perform any cleanup.
The most straightforward way is to use join(), like
try {
thread.join();
} catch (InterruptedException e) {//log exception...}
You could use an ExecutorService. That would make a lot of sense if you have several threads running concurrently. If you have the need to spawn new threads while other threads are running, you can combine this with a BlockingQueue.
A ThreadPoolExecutor (an ExecutorService-implementation) can take a BlockingQueue as argument, and you can simply add new threads to the queue. When you are done you simply terminate the ThreadPoolExecutor.
private BlockingQueue<Runnable> queue;
...
ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 10, new Long(1000),
TimeUnit.MILLISECONDS, this.queue);
You can keep a count of all the threads added to the queue. When you think you are done (the queue is empty, perhaps?) simply compare this to
if (issuedThreads == pool.getCompletedTaskCount()) {
pool.shutdown();
}
If the two match, you are done. Another way to terminate the pool is to wait a second in a loop:
try {
while (!this.pool.awaitTermination(1000, TimeUnit.MILLISECONDS));
} catch (InterruptedException e) {//log exception...}

How to make ThreadPoolExecutor's submit() method block if it is saturated?

I want to create a ThreadPoolExecutor such that when it has reached its maximum size and the queue is full, the submit() method blocks when trying to add new tasks. Do I need to implement a custom RejectedExecutionHandler for that or is there an existing way to do this using a standard Java library?
One of the possible solutions I've just found:
public class BoundedExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command)
throws InterruptedException, RejectedExecutionException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
throw e;
}
}
}
Are there any other solutions? I'd prefer something based on RejectedExecutionHandler since it seems like a standard way to handle such situations.
You can use ThreadPoolExecutor and a blockingQueue:
public class ImageManager {
BlockingQueue<Runnable> blockingQueue = new ArrayBlockingQueue<Runnable>(blockQueueSize);
RejectedExecutionHandler rejectedExecutionHandler = new ThreadPoolExecutor.CallerRunsPolicy();
private ExecutorService executorService = new ThreadPoolExecutor(numOfThread, numOfThread,
0L, TimeUnit.MILLISECONDS, blockingQueue, rejectedExecutionHandler);
private int downloadThumbnail(String fileListPath){
executorService.submit(new yourRunnable());
}
}
You should use the CallerRunsPolicy, which executes the rejected task in the calling thread. This way, it can't submit any new tasks to the executor until that task is done, at which point there will be some free pool threads or the process will repeat.
http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.CallerRunsPolicy.html
From the docs:
Rejected tasks
New tasks submitted in method execute(java.lang.Runnable) will be
rejected when the Executor has been
shut down, and also when the Executor
uses finite bounds for both maximum
threads and work queue capacity, and
is saturated. In either case, the
execute method invokes the
RejectedExecutionHandler.rejectedExecution(java.lang.Runnable,
java.util.concurrent.ThreadPoolExecutor)
method of its
RejectedExecutionHandler. Four
predefined handler policies are
provided:
In the default ThreadPoolExecutor.AbortPolicy, the
handler throws a runtime
RejectedExecutionException upon
rejection.
In ThreadPoolExecutor.CallerRunsPolicy,
the thread that invokes execute itself
runs the task. This provides a simple
feedback control mechanism that will
slow down the rate that new tasks are
submitted.
In ThreadPoolExecutor.DiscardPolicy, a
task that cannot be executed is simply
dropped.
In ThreadPoolExecutor.DiscardOldestPolicy,
if the executor is not shut down, the
task at the head of the work queue is
dropped, and then execution is retried
(which can fail again, causing this to
be repeated.)
Also, make sure to use a bounded queue, such as ArrayBlockingQueue, when calling the ThreadPoolExecutor constructor. Otherwise, nothing will get rejected.
Edit: in response to your comment, set the size of the ArrayBlockingQueue to be equal to the max size of the thread pool and use the AbortPolicy.
Edit 2: Ok, I see what you're getting at. What about this: override the beforeExecute() method to check that getActiveCount() doesn't exceed getMaximumPoolSize(), and if it does, sleep and try again?
I know, it is a hack, but in my opinion most clean hack between those offered here ;-)
Because ThreadPoolExecutor uses blocking queue "offer" instead of "put", lets override behaviour of "offer" of the blocking queue:
class BlockingQueueHack<T> extends ArrayBlockingQueue<T> {
BlockingQueueHack(int size) {
super(size);
}
public boolean offer(T task) {
try {
this.put(task);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return true;
}
}
ThreadPoolExecutor tp = new ThreadPoolExecutor(1, 2, 1, TimeUnit.MINUTES, new BlockingQueueHack(5));
I tested it and it seems to work.
Implementing some timeout policy is left as a reader's exercise.
Hibernate has a BlockPolicy that is simple and may do what you want:
See: Executors.java
/**
* A handler for rejected tasks that will have the caller block until
* space is available.
*/
public static class BlockPolicy implements RejectedExecutionHandler {
/**
* Creates a <tt>BlockPolicy</tt>.
*/
public BlockPolicy() { }
/**
* Puts the Runnable to the blocking queue, effectively blocking
* the delegating thread until space is available.
* #param r the runnable task requested to be executed
* #param e the executor attempting to execute this task
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
try {
e.getQueue().put( r );
}
catch (InterruptedException e1) {
log.error( "Work discarded, thread was interrupted while waiting for space to schedule: {}", r );
}
}
}
The BoundedExecutor answer quoted above from Java Concurrency in Practice only works correctly if you use an unbounded queue for the Executor, or the semaphore bound is no greater than the queue size. The semaphore is state shared between the submitting thread and the threads in the pool, making it possible to saturate the executor even if queue size < bound <= (queue size + pool size).
Using CallerRunsPolicy is only valid if your tasks don't run forever, in which case your submitting thread will remain in rejectedExecution forever, and a bad idea if your tasks take a long time to run, because the submitting thread can't submit any new tasks or do anything else if it's running a task itself.
If that's not acceptable then I suggest checking the size of the executor's bounded queue before submitting a task. If the queue is full, then wait a short time before trying to submit again. The throughput will suffer, but I suggest it's a simpler solution than many of the other proposed solutions and you're guaranteed no tasks will get rejected.
The following class wraps around a ThreadPoolExecutor and uses a Semaphore to block then the work queue is full:
public final class BlockingExecutor {
private final Executor executor;
private final Semaphore semaphore;
public BlockingExecutor(int queueSize, int corePoolSize, int maxPoolSize, int keepAliveTime, TimeUnit unit, ThreadFactory factory) {
BlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>();
this.executor = new ThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveTime, unit, queue, factory);
this.semaphore = new Semaphore(queueSize + maxPoolSize);
}
private void execImpl (final Runnable command) throws InterruptedException {
semaphore.acquire();
try {
executor.execute(new Runnable() {
#Override
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
// will never be thrown with an unbounded buffer (LinkedBlockingQueue)
semaphore.release();
throw e;
}
}
public void execute (Runnable command) throws InterruptedException {
execImpl(command);
}
}
This wrapper class is based on a solution given in the book Java Concurrency in Practice by Brian Goetz. The solution in the book only takes two constructor parameters: an Executor and a bound used for the semaphore. This is shown in the answer given by Fixpoint. There is a problem with that approach: it can get in a state where the pool threads are busy, the queue is full, but the semaphore has just released a permit. (semaphore.release() in the finally block). In this state, a new task can grab the just released permit, but is rejected because the task queue is full. Of course this is not something you want; you want to block in this case.
To solve this, we must use an unbounded queue, as JCiP clearly mentions. The semaphore acts as a guard, giving the effect of a virtual queue size. This has the side effect that it is possible that the unit can contain maxPoolSize + virtualQueueSize + maxPoolSize tasks. Why is that? Because of the
semaphore.release() in the finally block. If all pool threads call this statement at the same time, then maxPoolSize permits are released, allowing the same number of tasks to enter the unit. If we were using a bounded queue, it would still be full, resulting in a rejected task. Now, because we know that this only occurs when a pool thread is almost done, this is not a problem. We know that the pool thread will not block, so a task will soon be taken from the queue.
You are able to use a bounded queue though. Just make sure that its size equals virtualQueueSize + maxPoolSize. Greater sizes are useless, the semaphore will prevent to let more items in. Smaller sizes will result in rejected tasks. The chance of tasks getting rejected increases as the size decreases. For example, say you want a bounded executor with maxPoolSize=2 and virtualQueueSize=5. Then take a semaphore with 5+2=7 permits and an actual queue size of 5+2=7. The real number of tasks that can be in the unit is then 2+5+2=9. When the executor is full (5 tasks in queue, 2 in thread pool, so 0 permits available) and ALL pool threads release their permits, then exactly 2 permits can be taken by tasks coming in.
Now the solution from JCiP is somewhat cumbersome to use as it doesn't enforce all these constraints (unbounded queue, or bounded with those math restrictions, etc.). I think that this only serves as a good example to demonstrate how you can build new thread safe classes based on the parts that are already available, but not as a full-grown, reusable class. I don't think that the latter was the author's intention.
you can use a custom RejectedExecutionHandler like this
ThreadPoolExecutor tp= new ThreadPoolExecutor(core_size, // core size
max_handlers, // max size
timeout_in_seconds, // idle timeout
TimeUnit.SECONDS, queue, new RejectedExecutionHandler() {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
// This will block if the queue is full
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
System.err.println(e.getMessage());
}
}
});
I don't always like the CallerRunsPolicy, especially since it allows the rejected task to 'skip the queue' and get executed before tasks that were submitted earlier. Moreover, executing the task on the calling thread might take much longer than waiting for the first slot to become available.
I solved this problem using a custom RejectedExecutionHandler, which simply blocks the calling thread for a little while and then tries to submit the task again:
public class BlockWhenQueueFull implements RejectedExecutionHandler {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
// The pool is full. Wait, then try again.
try {
long waitMs = 250;
Thread.sleep(waitMs);
} catch (InterruptedException interruptedException) {}
executor.execute(r);
}
}
This class can just be used in the thread-pool executor as a RejectedExecutinHandler like any other, for example:
executorPool = new ThreadPoolExecutor(1, 1, 10,
TimeUnit.SECONDS, new SynchronousQueue<Runnable>(),
new BlockWhenQueueFull());
The only downside I see is that the calling thread might get locked slightly longer than strictly necessary (up to 250ms). Moreover, since this executor is effectively being called recursively, very long waits for a thread to become available (hours) might result in a stack overflow.
Nevertheless, I personally like this method. It's compact, easy to understand, and works well.
Create your own blocking queue to be used by the Executor, with the blocking behavior you are looking for, while always returning available remaining capacity (ensuring the executor will not try to create more threads than its core pool, or trigger the rejection handler).
I believe this will get you the blocking behavior you are looking for. A rejection handler will never fit the bill, since that indicates the executor can not perform the task. What I could envision there is that you get some form of 'busy waiting' in the handler. That is not what you want, you want a queue for the executor that blocks the caller...
To avoid issues with #FixPoint solution. One could use ListeningExecutorService and release the semaphore onSuccess and onFailure inside FutureCallback.
Recently I found this question having the same problem. The OP does not say so explicitly, but we do not want to use the RejectedExecutionHandler which executes a task on the submitter's thread, because this will under-utilize the worker threads if this task is a long running one.
Reading all the answers and comments, in particular the flawed solution with the semaphore or using afterExecute I had a closer look at the code of the ThreadPoolExecutor to see if there is some way out. I was amazed to see that there are more than 2000 lines of (commented) code, some of which make me feel dizzy. Given the rather simple requirement I actually have --- one producer, several consumers, let the producer block when no consumers can take work --- I decided to roll my own solution. It is not an ExecutorService but just an Executor. And it does not adapt the number of threads to the work load, but holds a fixed number of threads only, which also fits my requirements. Here is the code. Feel free to rant about it :-)
package x;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Executor;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.SynchronousQueue;
/**
* distributes {#code Runnable}s to a fixed number of threads. To keep the
* code lean, this is not an {#code ExecutorService}. In particular there is
* only very simple support to shut this executor down.
*/
public class ParallelExecutor implements Executor {
// other bounded queues work as well and are useful to buffer peak loads
private final BlockingQueue<Runnable> workQueue =
new SynchronousQueue<Runnable>();
private final Thread[] threads;
/*+**********************************************************************/
/**
* creates the requested number of threads and starts them to wait for
* incoming work
*/
public ParallelExecutor(int numThreads) {
this.threads = new Thread[numThreads];
for(int i=0; i<numThreads; i++) {
// could reuse the same Runner all over, but keep it simple
Thread t = new Thread(new Runner());
this.threads[i] = t;
t.start();
}
}
/*+**********************************************************************/
/**
* returns immediately without waiting for the task to be finished, but may
* block if all worker threads are busy.
*
* #throws RejectedExecutionException if we got interrupted while waiting
* for a free worker
*/
#Override
public void execute(Runnable task) {
try {
workQueue.put(task);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RejectedExecutionException("interrupt while waiting for a free "
+ "worker.", e);
}
}
/*+**********************************************************************/
/**
* Interrupts all workers and joins them. Tasks susceptible to an interrupt
* will preempt their work. Blocks until the last thread surrendered.
*/
public void interruptAndJoinAll() throws InterruptedException {
for(Thread t : threads) {
t.interrupt();
}
for(Thread t : threads) {
t.join();
}
}
/*+**********************************************************************/
private final class Runner implements Runnable {
#Override
public void run() {
while (!Thread.currentThread().isInterrupted()) {
Runnable task;
try {
task = workQueue.take();
} catch (InterruptedException e) {
// canonical handling despite exiting right away
Thread.currentThread().interrupt();
return;
}
try {
task.run();
} catch (RuntimeException e) {
// production code to use a logging framework
e.printStackTrace();
}
}
}
}
}
I believe there is quite elegant way to solve this problem by using java.util.concurrent.Semaphore and delegating behavior of Executor.newFixedThreadPool.
The new executor service will only execute new task when there is a thread to do so. Blocking is managed by Semaphore with number of permits equal to number of threads. When a task is finished it returns a permit.
public class FixedThreadBlockingExecutorService extends AbstractExecutorService {
private final ExecutorService executor;
private final Semaphore blockExecution;
public FixedThreadBlockingExecutorService(int nTreads) {
this.executor = Executors.newFixedThreadPool(nTreads);
blockExecution = new Semaphore(nTreads);
}
#Override
public void shutdown() {
executor.shutdown();
}
#Override
public List<Runnable> shutdownNow() {
return executor.shutdownNow();
}
#Override
public boolean isShutdown() {
return executor.isShutdown();
}
#Override
public boolean isTerminated() {
return executor.isTerminated();
}
#Override
public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {
return executor.awaitTermination(timeout, unit);
}
#Override
public void execute(Runnable command) {
blockExecution.acquireUninterruptibly();
executor.execute(() -> {
try {
command.run();
} finally {
blockExecution.release();
}
});
}
I had the same need in the past: a kind of blocking queue with a fixed size for each client backed by a shared thread pool. I ended up writing my own kind of ThreadPoolExecutor:
UserThreadPoolExecutor
(blocking queue (per client) + threadpool (shared amongst all clients))
See: https://github.com/d4rxh4wx/UserThreadPoolExecutor
Each UserThreadPoolExecutor is given a maximum number of threads from a shared ThreadPoolExecutor
Each UserThreadPoolExecutor can:
submit a task to the shared thread pool executor if its quota is not reached. If its quota is reached, the job is queued (non-consumptive blocking waiting for CPU). Once one of its submitted task is completed, the quota is decremented, allowing another task waiting to be submitted to the ThreadPoolExecutor
wait for the remaining tasks to complete
I found this rejection policy in elastic search client. It blocks caller thread on blocking queue. Code below-
static class ForceQueuePolicy implements XRejectedExecutionHandler
{
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor)
{
try
{
executor.getQueue().put(r);
}
catch (InterruptedException e)
{
//should never happen since we never wait
throw new EsRejectedExecutionException(e);
}
}
#Override
public long rejected()
{
return 0;
}
}
I recently had a need to achieve something similar, but on a ScheduledExecutorService.
I had to also ensure that I handle the delay being passed on the method and ensure that either the task is submitted to execute at the time as the caller expects or just fails thus throwing a RejectedExecutionException.
Other methods from ScheduledThreadPoolExecutor to execute or submit a task internally call #schedule which will still in turn invoke the methods overridden.
import java.util.concurrent.*;
public class BlockingScheduler extends ScheduledThreadPoolExecutor {
private final Semaphore maxQueueSize;
public BlockingScheduler(int corePoolSize,
ThreadFactory threadFactory,
int maxQueueSize) {
super(corePoolSize, threadFactory, new AbortPolicy());
this.maxQueueSize = new Semaphore(maxQueueSize);
}
#Override
public ScheduledFuture<?> schedule(Runnable command,
long delay,
TimeUnit unit) {
final long newDelayInMs = beforeSchedule(command, unit.toMillis(delay));
return super.schedule(command, newDelayInMs, TimeUnit.MILLISECONDS);
}
#Override
public <V> ScheduledFuture<V> schedule(Callable<V> callable,
long delay,
TimeUnit unit) {
final long newDelayInMs = beforeSchedule(callable, unit.toMillis(delay));
return super.schedule(callable, newDelayInMs, TimeUnit.MILLISECONDS);
}
#Override
public ScheduledFuture<?> scheduleAtFixedRate(Runnable command,
long initialDelay,
long period,
TimeUnit unit) {
final long newDelayInMs = beforeSchedule(command, unit.toMillis(initialDelay));
return super.scheduleAtFixedRate(command, newDelayInMs, unit.toMillis(period), TimeUnit.MILLISECONDS);
}
#Override
public ScheduledFuture<?> scheduleWithFixedDelay(Runnable command,
long initialDelay,
long period,
TimeUnit unit) {
final long newDelayInMs = beforeSchedule(command, unit.toMillis(initialDelay));
return super.scheduleWithFixedDelay(command, newDelayInMs, unit.toMillis(period), TimeUnit.MILLISECONDS);
}
#Override
protected void afterExecute(Runnable runnable, Throwable t) {
super.afterExecute(runnable, t);
try {
if (t == null && runnable instanceof Future<?>) {
try {
((Future<?>) runnable).get();
} catch (CancellationException | ExecutionException e) {
t = e;
} catch (InterruptedException ie) {
Thread.currentThread().interrupt(); // ignore/reset
}
}
if (t != null) {
System.err.println(t);
}
} finally {
releaseQueueUsage();
}
}
private long beforeSchedule(Runnable runnable, long delay) {
try {
return getQueuePermitAndModifiedDelay(delay);
} catch (InterruptedException e) {
getRejectedExecutionHandler().rejectedExecution(runnable, this);
return 0;
}
}
private long beforeSchedule(Callable callable, long delay) {
try {
return getQueuePermitAndModifiedDelay(delay);
} catch (InterruptedException e) {
getRejectedExecutionHandler().rejectedExecution(new FutureTask(callable), this);
return 0;
}
}
private long getQueuePermitAndModifiedDelay(long delay) throws InterruptedException {
final long beforeAcquireTimeStamp = System.currentTimeMillis();
maxQueueSize.tryAcquire(delay, TimeUnit.MILLISECONDS);
final long afterAcquireTimeStamp = System.currentTimeMillis();
return afterAcquireTimeStamp - beforeAcquireTimeStamp;
}
private void releaseQueueUsage() {
maxQueueSize.release();
}
}
I have the code here, will appreciate any feedback.
https://github.com/AmitabhAwasthi/BlockingScheduler

Is it a good way to use java.util.concurrent.FutureTask?

First of all, I must say that I am quite new to the API java.util.concurrent, so maybe what I am doing is completely wrong.
What do I want to do?
I have a Java application that basically runs 2 separate processing (called myFirstProcess, mySecondProcess), but these processing must be run at the same time.
So, I tried to do that:
public void startMyApplication() {
ExecutorService executor = Executors.newFixedThreadPool(2);
FutureTask<Object> futureOne = new FutureTask<Object>(myFirstProcess);
FutureTask<Object> futureTwo = new FutureTask<Object>(mySecondProcess);
executor.execute(futureOne);
executor.execute(futureTwo);
while (!(futureOne.isDone() && futureTwo.isDone())) {
try {
// I wait until both processes are finished.
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
logger.info("Processing finished");
executor.shutdown();
// Do some processing on results
...
}
myFirstProcess and mySecondProcess are classes that implements Callable<Object>, and where all their processing is made in the call() method.
It is working quite well but I am not sure that it is the correct way to do that.
Is a good way to do what I want? If not, can you give me some hints to enhance my code (and still keep it as simple as possible).
You'd be better off using the get() method.
futureOne.get();
futureTwo.get();
Both of which wait for notification from the thread that it finished processing, this saves you the busy-wait-with-timer you are now using which is not efficient nor elegant.
As a bonus, you have the API get(long timeout, TimeUnit unit) which allows you to define a maximum time for the thread to sleep and wait for a response, and otherwise continues running.
See the Java API for more info.
The uses of FutureTask above are tolerable, but definitely not idiomatic. You're actually wrapping an extra FutureTask around the one you submitted to the ExecutorService. Your FutureTask is treated as a Runnable by the ExecutorService. Internally, it wraps your FutureTask-as-Runnable in a new FutureTask and returns it to you as a Future<?>.
Instead, you should submit your Callable<Object> instances to a CompletionService. You drop two Callables in via submit(Callable<V>), then turn around and call CompletionService#take() twice (once for each submitted Callable). Those calls will block until one and then the other submitted tasks are complete.
Given that you already have an Executor in hand, construct a new ExecutorCompletionService around it and drop your tasks in there. Don't spin and sleep waiting; CompletionService#take() will block until either one of your tasks are complete (either finished running or canceled) or the thread waiting on take() is interrupted.
Yuval's solution is fine. As an alternative you can also do this:
ExecutorService executor = Executors.newFixedThreadPool();
FutureTask<Object> futureOne = new FutureTask<Object>(myFirstProcess);
FutureTask<Object> futureTwo = new FutureTask<Object>(mySecondProcess);
executor.execute(futureOne);
executor.execute(futureTwo);
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
// interrupted
}
What is the advantage of this approach? There's not a lot of difference really except that this way you stop the executor accepting any more tasks (you can do that the other way too). I tend to prefer this idiom to that one though.
Also, if either get() throws an exception you may end up in a part of your code that assumes both tasks are done, which might be bad.
You can use invokeall(Colelction....) method
package concurrent.threadPool;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class InvokeAll {
public static void main(String[] args) throws Exception {
ExecutorService service = Executors.newFixedThreadPool(5);
List<Future<java.lang.String>> futureList = service.invokeAll(Arrays.asList(new Task1<String>(),new Task2<String>()));
System.out.println(futureList.get(1).get());
System.out.println(futureList.get(0).get());
}
private static class Task1<String> implements Callable<String>{
#Override
public String call() throws Exception {
Thread.sleep(1000 * 10);
return (String) "1000 * 5";
}
}
private static class Task2<String> implements Callable<String>{
#Override
public String call() throws Exception {
Thread.sleep(1000 * 2);
int i=3;
if(i==3)
throw new RuntimeException("Its Wrong");
return (String) "1000 * 2";
}
}
}
You may want to use a CyclicBarrier if you are interested in starting the threads at the same time, or waiting for them to finish and then do some further processing.
See the javadoc for more information.
If your futureTasks are more then 2, please consider [ListenableFuture][1].
When several operations should begin as soon as another operation
starts -- "fan-out" -- ListenableFuture just works: it triggers all of
the requested callbacks. With slightly more work, we can "fan-in," or
trigger a ListenableFuture to get computed as soon as several other
futures have all finished.

Categories