This question already has answers here:
ThreadPoolExecutor Block When its Queue Is Full?
(10 answers)
Closed 3 months ago.
We have a large text file in which each line requires intensive process. The design is to have a class that reads the file and delegates the processing of each line to a thread, via thread pool. The file reader class should be blocked from reading the next line once there is no free thread in the pool to do the processing. So i need a blocking thread pool
In the current implementation ThreadPoolExecutor.submit() and ThreadPoolExecutor.execute() methods throw RejectedExecutionException exception after the configured # of threads get busy as i showed in code snippet below.
public class BlockingTp {
public static void main(String[] args) {
BlockingQueue blockingQueue = new ArrayBlockingQueue(3);
ThreadPoolExecutor executorService=
new ThreadPoolExecutor(1, 3, 30, TimeUnit.SECONDS, blockingQueue);
int Jobs = 10;
System.out.println("Starting application with " + Jobs + " jobs");
for (int i = 1; i <= Jobs; i++)
try {
executorService.submit(new WorkerThread(i));
System.out.println("job added " + (i));
} catch (RejectedExecutionException e) {
System.err.println("RejectedExecutionException");
}
}
}
class WorkerThread implements Runnable {
int job;
public WorkerThread(int job) {
this.job = job;
}
public void run() {
try {
Thread.sleep(1000);
} catch (Exception excep) {
}
}
}
Output of above program is
Starting application to add 10 jobs
Added job #1
Added job #2
Added job #3
Added job #4
Added job #5
Added job #6
RejectedExecutionException
RejectedExecutionException
RejectedExecutionException
RejectedExecutionException
Can some one throw some light i.e how i can implement blocking thread pool.
Can some one throw some light i.e how i can implement blocking thread pool.
You need to set a rejection execution handler on your executor service. When the thread goes to put the job into the executor, it will block until there is space in the blocking queue.
BlockingQueue arrayBlockingQueue = new ArrayBlockingQueue(3);
ThreadPoolExecutor executorService =
new ThreadPoolExecutor(1, 3, 30, TimeUnit.SECONDS, arrayBlockingQueue);
// when the blocking queue is full, this tries to put into the queue which blocks
executorService.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
// block until there's room
executor.getQueue().put(r);
// check afterwards and throw if pool shutdown
if (executor.isShutdown()) {
throw new RejectedExecutionException(
"Task " + r + " rejected from " + executor);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RejectedExecutionException("Producer interrupted", e);
}
}
});
So instead of the TRE throwing a RejectedExecutionException, it will call the rejection handler which will in turn try to put the job back on the queue. This blocks the caller.
Lets have a look at your code again:
for (int i = 1; i <= Jobs; i++)
try {
tpExe.submit(new WorkerThread(i));
System.out.println("job added " + (i));
} catch (RejectedExecutionException e) {
System.err.println("RejectedExecutionException");
}
So - when you try to submit, and the pool is busy, that exception is thrown. If you want to wrap around that, it could look like:
public void yourSubmit(Runnable whatever) {
boolean submitted = false;
while (! submitted ) {
try {
tpExe.submit(new WorkerThread(whatever));
submitted = true;
} catch (RejectedExecutionException re) {
// all threads busy ... so wait some time
Thread.sleep(1000);
}
In other words: use that exception as "marker" that submits are currently not possible.
You can use semaphore for to control the resource.Reader will read and create asynchronous task by acquiring semaphore.If every thread is busy the reader thread will wait till thread is available.
public class MyExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command)
throws InterruptedException, RejectedExecutionException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
throw e;
}
}
}
Here is a RejectedExecutionHandler that supports the desired behavior. Unlike other implementations, it does not interact with the queue directly so it should be compatible with all Executor implementations and will not deadlock.
import java.util.concurrent.Executor;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.function.BiFunction;
import static com.github.cowwoc.requirements.DefaultRequirements.assertThat;
import static com.github.cowwoc.requirements.DefaultRequirements.requireThat;
/**
* Applies a different rejection policy depending on the thread that requested execution.
*/
public final class ThreadDependantRejectionHandler implements RejectedExecutionHandler
{
private final ThreadLocal<Integer> numberOfRejections = ThreadLocal.withInitial(() -> 0);
private final BiFunction<Thread, Executor, Action> threadToAction;
/**
* #param threadToAction indicates what action a thread should take when execution is rejected
* #throws NullPointerException if {#code threadToAction} is null
*/
public ThreadDependantRejectionHandler(BiFunction<Thread, Executor, Action> threadToAction)
{
requireThat(threadToAction, "threadToAction").isNotNull();
this.threadToAction = threadToAction;
}
#SuppressWarnings("BusyWait")
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor)
{
if (executor.isShutdown())
return;
Thread currentThread = Thread.currentThread();
Action action = threadToAction.apply(currentThread, executor);
if (action == Action.RUN)
{
r.run();
return;
}
if (action == Action.REJECT)
{
throw new RejectedExecutionException("The thread pool queue is full and the current thread is not " +
"allowed to block or run the task");
}
assertThat(action, "action").isEqualTo(Action.BLOCK);
int numberOfRejections = this.numberOfRejections.get();
++numberOfRejections;
this.numberOfRejections.set(numberOfRejections);
if (numberOfRejections > 1)
return;
try
{
ThreadLocalRandom random = ThreadLocalRandom.current();
while (!executor.isShutdown())
{
try
{
Thread.sleep(random.nextInt(10, 1001));
}
catch (InterruptedException e)
{
throw new WrappingException(e);
}
executor.submit(r);
numberOfRejections = this.numberOfRejections.get();
if (numberOfRejections == 1)
{
// Task was accepted, or executor has shut down
return;
}
// Task was rejected, reset the counter and try again.
numberOfRejections = 1;
this.numberOfRejections.set(numberOfRejections);
}
throw new RejectedExecutionException("Task " + r + " rejected from " + executor + " because " +
"the executor has been shut down");
}
finally
{
this.numberOfRejections.set(0);
}
}
public enum Action
{
/**
* The thread should run the task directly instead of waiting for the executor.
*/
RUN,
/**
* The thread should block until the executor is ready to run the task.
*/
BLOCK,
/**
* The thread should reject execution of the task.
*/
REJECT
}
}
This works for me.
class handler implements RejectedExecutionHandler{
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Related
This question already has answers here:
ThreadPoolExecutor Block When its Queue Is Full?
(10 answers)
Closed 3 months ago.
We have a large text file in which each line requires intensive process. The design is to have a class that reads the file and delegates the processing of each line to a thread, via thread pool. The file reader class should be blocked from reading the next line once there is no free thread in the pool to do the processing. So i need a blocking thread pool
In the current implementation ThreadPoolExecutor.submit() and ThreadPoolExecutor.execute() methods throw RejectedExecutionException exception after the configured # of threads get busy as i showed in code snippet below.
public class BlockingTp {
public static void main(String[] args) {
BlockingQueue blockingQueue = new ArrayBlockingQueue(3);
ThreadPoolExecutor executorService=
new ThreadPoolExecutor(1, 3, 30, TimeUnit.SECONDS, blockingQueue);
int Jobs = 10;
System.out.println("Starting application with " + Jobs + " jobs");
for (int i = 1; i <= Jobs; i++)
try {
executorService.submit(new WorkerThread(i));
System.out.println("job added " + (i));
} catch (RejectedExecutionException e) {
System.err.println("RejectedExecutionException");
}
}
}
class WorkerThread implements Runnable {
int job;
public WorkerThread(int job) {
this.job = job;
}
public void run() {
try {
Thread.sleep(1000);
} catch (Exception excep) {
}
}
}
Output of above program is
Starting application to add 10 jobs
Added job #1
Added job #2
Added job #3
Added job #4
Added job #5
Added job #6
RejectedExecutionException
RejectedExecutionException
RejectedExecutionException
RejectedExecutionException
Can some one throw some light i.e how i can implement blocking thread pool.
Can some one throw some light i.e how i can implement blocking thread pool.
You need to set a rejection execution handler on your executor service. When the thread goes to put the job into the executor, it will block until there is space in the blocking queue.
BlockingQueue arrayBlockingQueue = new ArrayBlockingQueue(3);
ThreadPoolExecutor executorService =
new ThreadPoolExecutor(1, 3, 30, TimeUnit.SECONDS, arrayBlockingQueue);
// when the blocking queue is full, this tries to put into the queue which blocks
executorService.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
// block until there's room
executor.getQueue().put(r);
// check afterwards and throw if pool shutdown
if (executor.isShutdown()) {
throw new RejectedExecutionException(
"Task " + r + " rejected from " + executor);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RejectedExecutionException("Producer interrupted", e);
}
}
});
So instead of the TRE throwing a RejectedExecutionException, it will call the rejection handler which will in turn try to put the job back on the queue. This blocks the caller.
Lets have a look at your code again:
for (int i = 1; i <= Jobs; i++)
try {
tpExe.submit(new WorkerThread(i));
System.out.println("job added " + (i));
} catch (RejectedExecutionException e) {
System.err.println("RejectedExecutionException");
}
So - when you try to submit, and the pool is busy, that exception is thrown. If you want to wrap around that, it could look like:
public void yourSubmit(Runnable whatever) {
boolean submitted = false;
while (! submitted ) {
try {
tpExe.submit(new WorkerThread(whatever));
submitted = true;
} catch (RejectedExecutionException re) {
// all threads busy ... so wait some time
Thread.sleep(1000);
}
In other words: use that exception as "marker" that submits are currently not possible.
You can use semaphore for to control the resource.Reader will read and create asynchronous task by acquiring semaphore.If every thread is busy the reader thread will wait till thread is available.
public class MyExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command)
throws InterruptedException, RejectedExecutionException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
throw e;
}
}
}
Here is a RejectedExecutionHandler that supports the desired behavior. Unlike other implementations, it does not interact with the queue directly so it should be compatible with all Executor implementations and will not deadlock.
import java.util.concurrent.Executor;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.function.BiFunction;
import static com.github.cowwoc.requirements.DefaultRequirements.assertThat;
import static com.github.cowwoc.requirements.DefaultRequirements.requireThat;
/**
* Applies a different rejection policy depending on the thread that requested execution.
*/
public final class ThreadDependantRejectionHandler implements RejectedExecutionHandler
{
private final ThreadLocal<Integer> numberOfRejections = ThreadLocal.withInitial(() -> 0);
private final BiFunction<Thread, Executor, Action> threadToAction;
/**
* #param threadToAction indicates what action a thread should take when execution is rejected
* #throws NullPointerException if {#code threadToAction} is null
*/
public ThreadDependantRejectionHandler(BiFunction<Thread, Executor, Action> threadToAction)
{
requireThat(threadToAction, "threadToAction").isNotNull();
this.threadToAction = threadToAction;
}
#SuppressWarnings("BusyWait")
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor)
{
if (executor.isShutdown())
return;
Thread currentThread = Thread.currentThread();
Action action = threadToAction.apply(currentThread, executor);
if (action == Action.RUN)
{
r.run();
return;
}
if (action == Action.REJECT)
{
throw new RejectedExecutionException("The thread pool queue is full and the current thread is not " +
"allowed to block or run the task");
}
assertThat(action, "action").isEqualTo(Action.BLOCK);
int numberOfRejections = this.numberOfRejections.get();
++numberOfRejections;
this.numberOfRejections.set(numberOfRejections);
if (numberOfRejections > 1)
return;
try
{
ThreadLocalRandom random = ThreadLocalRandom.current();
while (!executor.isShutdown())
{
try
{
Thread.sleep(random.nextInt(10, 1001));
}
catch (InterruptedException e)
{
throw new WrappingException(e);
}
executor.submit(r);
numberOfRejections = this.numberOfRejections.get();
if (numberOfRejections == 1)
{
// Task was accepted, or executor has shut down
return;
}
// Task was rejected, reset the counter and try again.
numberOfRejections = 1;
this.numberOfRejections.set(numberOfRejections);
}
throw new RejectedExecutionException("Task " + r + " rejected from " + executor + " because " +
"the executor has been shut down");
}
finally
{
this.numberOfRejections.set(0);
}
}
public enum Action
{
/**
* The thread should run the task directly instead of waiting for the executor.
*/
RUN,
/**
* The thread should block until the executor is ready to run the task.
*/
BLOCK,
/**
* The thread should reject execution of the task.
*/
REJECT
}
}
This works for me.
class handler implements RejectedExecutionHandler{
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I created a thread pool, and submitted two tasks. Why does my application hang without any exceptions after print task one ,result: null???
private final static ThreadPoolExecutor executorService = new
ThreadPoolExecutor(1, 1, 1L, TimeUnit.MINUTES,
new SynchronousQueue<Runnable>(), new ThreadPoolExecutor.DiscardPolicy());
public static void main(String[] args) throws Exception {
Future taskOne = executorService.submit(() -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
Future taskTwo = executorService.submit(() -> System.out.println("task two is working"));;
System.out.println("task one ,result: " + taskOne.get());
System.out.println("task two, result: " + taskTwo.get());
executorService.shutdown();
}
When you submit the second task, the rejection policy is triggered because the thread pool uses SynchronousQueue and maximumPoolSize is 1, while the first task has not completed. You are using DiscardPolicy, which means that the thread pool does nothing and returns you a FutureTask whose state is always NEW.
public static class DiscardPolicy implements RejectedExecutionHandler {
public DiscardPolicy() { }
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
}
}
So when you call taskTwo#get(), you will always be blocked. (FutureTask will always be blocked when it is in a state smaller than COMPLETING, see FutureTask#get).
You can use AbortPolicy (the default policy), so that when you execute executorService.submit(() - > submit; System.out.println("task two is working")), you immediately get a RejectedExecutionException.
Or use Future#get(timeout), in which case you get a TimeoutException if you do not get a result for a specified time.
new ThreadPoolExecutor.DiscardPolicy() silently discards the new task when it fails to submit it. here taskTwo wants to get executed, it never gets a chance to execute.
DiscardPolicy() method internally call void rejectedExecution(Runnable r, ThreadPoolExecutor executor) from RejectedExecutionHandler interface.
I have shown CustomRejectedExecutionHandler for better understanding the taskTwo thread condition. As taskTwo is silently discarded so that taskTwo.get() method will never be able to return data.
That's why timeout is required to be set as 1 second (taskTwo.get(1000, TimeUnit.MILLISECONDS)).
package example;
import java.util.concurrent.*;
public class ThreadPoolEx {
public static void main(String[] args) {
CustomRejectedExecutionHandler rejectionHandler = new CustomRejectedExecutionHandler();
ThreadPoolExecutor executorService =
new ThreadPoolExecutor(1, 1, 1L,
TimeUnit.MINUTES,
new SynchronousQueue<Runnable>(),
rejectionHandler
);
Future taskOne = executorService.submit(() -> {
try {
System.out.println("taskOne is going to sleep");
Thread.sleep(2000);
System.out.println("taskOne is wake up");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
Future taskTwo = executorService.submit(() -> System.out.println("task two is working"));
try {
System.out.println("task one ,result: " + taskOne.get());
System.out.println("isTerminating "+ executorService.isTerminating());
System.out.println("getActiveCount "+ executorService.getActiveCount());
System.out.println("is cancelled " + taskTwo.isCancelled());
System.out.println("is isDone " + taskTwo.isDone());
System.out.println("task two, result: " + taskTwo.get(1000, TimeUnit.MILLISECONDS));
} catch (Exception e) {
}
executorService.shutdown();
}
}
class CustomRejectedExecutionHandler implements RejectedExecutionHandler {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
System.out.println(r.toString() + " is rejected");
}
}
I have a need of a threadpool executor, which needs to complete an exact number (same) tasks.
It has to be able to re-submit failed tasks for an n number of times. If any of the tasks fail for more than n, then the threadpool should shutdown and not continue to process any other tasks.
I have tried to combine 2 approaches which I've found in different answers - one for re-submitting failed tasks by overriding ThreadPoolExecutor.afterExecute, and subclassing CountDownLatch so that threads waiting on the latch get interrupted and the executor shuts down.
So far, this is the subclassed countdown latch:
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
public class AbortableCountDownLatch extends CountDownLatch {
protected boolean aborted = false;
public AbortableCountDownLatch(int count) {
super(count);
}
/**
* Unblocks all threads waiting on this latch and cause them to receive an
* AbortedException. If the latch has already counted all the way down,
* this method does nothing.
*/
public void abort() {
if( getCount() == 0 )
return;
this.aborted = true;
while(getCount() > 0)
countDown();
}
#Override
public boolean await(long timeout, TimeUnit unit) throws InterruptedException {
final boolean rtrn = super.await(timeout,unit);
if (aborted)
throw new AbortedException();
return rtrn;
}
#Override
public void await() throws InterruptedException {
super.await();
if (aborted)
throw new AbortedException();
}
public static class AbortedException extends InterruptedException {
public AbortedException() {
}
public AbortedException(String detailMessage) {
super(detailMessage);
}
}
}
And the thread pool executor:
public class MyThreadPoolExecutor extends ThreadPoolExecutor {
private static final int RETRY_LIMIT = 3;
private Map<Runnable, Integer> retriedTasks = new ConcurrentHashMap<>();
private AbortableCountDownLatch latch;
public MyThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime,
TimeUnit unit, BlockingQueue<Runnable> workQueue, AbortableCountDownLatch latch) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
this.latch = latch;
}
#Override
public void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
// If submit() method is called instead of execute()
if (t == null && r instanceof Future<?>) {
try {
Object result = ((Future<?>) r).get();
} catch (CancellationException e) {
t = e;
} catch (ExecutionException e) {
t = e.getCause();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
if (t != null) {
retriedTasks.put(r, retriedTasks.getOrDefault(r, 0) + 1);
System.out.println("Retries for " + r + " -> " + retriedTasks.get(r));
/* check to see if we have retried this task too many times, if so - shutdown */
if (retriedTasks.containsKey(r) && retriedTasks.get(r) > RETRY_LIMIT) {
System.err.println("Thread failed for more than " + RETRY_LIMIT + " times, aborting everything..");
this.latch.abort();
} else {
System.err.println("Thread threw exception " + t.getMessage() + ". Retry-ing task...");
execute(r);
}
} else {
/* clear any previous retry count for this runnable */
retriedTasks.remove(r);
}
}
}
And a main would be using them like this:
import java.util.Random;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class MainProcessor {
public static void main(String[] args) {
AbortableCountDownLatch latch = new AbortableCountDownLatch(5);
ThreadPoolExecutor threadPoolExecutor = new MyThreadPoolExecutor(8, 8, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(), latch);
for (int i = 0; i < 5; i++) {
threadPoolExecutor.submit(() -> {
System.out.println("Started thread " + Thread.currentThread().getName());
Random random = new Random();
try {
Thread.sleep(random.nextInt(7000));
} catch (InterruptedException e) {
e.printStackTrace();
}
if (random.nextBoolean()){
System.err.println("Thread " + Thread.currentThread().getName() + " failed - throwing exception..");
throw new RuntimeException("Thread " + Thread.currentThread().getName() + "failed! spectacularly :!");
}
else {
System.out.println("Thread " + Thread.currentThread().getName() + " finished.");
latch.countDown();
}
});
}
try {
latch.await();
} catch (InterruptedException e) {
threadPoolExecutor.shutdownNow();
}
threadPoolExecutor.shutdown();
}
}
Does this approach look correct? I don't particularly like that the latch has to be passed to both the thread pool executor and to the actual Runnable. Is there a standard way of achieving this? I am fine with a Scala version too.
I have seen others who suggest that the tasks should re-submit itself to the pool in case of failure, but that doesn't seem a good idea, as the task should only be responsible of the actual running logic, and not execution details.
You could use a Task-Wrapper that does the work, then it would be rather simple:
public class TaskWrapper implements Runnable
{
private Runnable task;
private int maxResubmits;
private ThreadPoolExecutor executor;
private CountDownLatch latch;
public TaskWrapper(Runnable task, int maxResubmits, ThreadPoolExecutor executor, CountDownLatch latch) {
this.task=task;
this.maxResubmits=maxResubmits;
this.executor=executor;
this.latch=latch;
executor.submit(this);
}
public void run() {
try {
task.run();
latch.countdoun();
}
catch(Exception e) {
maxResubmits--;
if(maxResubmits>0)
executor.submit(this);
else
{
latch.countdoun();
executor.shutdownNow()
}
}
}
}
You now only need to create the latch, call your tasks and then wait for the execution:
List<Runnable> tasks;
int maxResubmits;
CountDownLatch latch=new CountDownLatch(tasks.size());
tasks.forEach(task->new TaskWrapper(task,maxResubmits,executor,latch));
latch.await();
if(!executor.isShutdown())
executor.shutdown();
I am using single thread executor for long-running threads like this:
executor = Executors.newSingleThreadExecutor(THREAD_FACTORY);
executor.submit(new LongRunnable());
which checks a flag to be stopped:
private class LongRunnable implements Runnable {
#Override
public void run() {
while (isRunning.get()) {
try {
doSomething();
} catch (InterruptedException e) {
...
}
}
}
}
and whole execution is interrupted that way:
#Override
public void close() throws Exception {
isRunning.set(false);
executor.shutdownNow();
}
Still I can see some threads not gc-ed in profiler (while by logs, runnable they were executing has quit outermost while loop).
Question: does provided working with threads strategy memory-leak-free and thread-leak-free?
I am not able to see any issue with executor or shutDownNow. Probably you are looking at different threads in your profiler.
Try this program which is similar to the one in your question and you can see the thread is no longer there after successful shutdown.
public class ExecutorShutdownTest {
private static ExecutorService executor;
private static AtomicLong executorThreadId = new AtomicLong(0);
public static void main(String[] args) {
// get thread MX bean
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
// create an executor and start the task
executor = Executors.newSingleThreadExecutor(new TestThreadFactory());
LongRunnable runnable = new LongRunnable();
executor.submit(runnable);
// main thread: keep running for sometime
int count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace(
"\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// main thread: stop the task
try {
runnable.close();
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (Exception e) {
e.printStackTrace();
}
// main thread: run some more time to verify the executor thread no longer exists
count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private static class LongRunnable implements Runnable {
private volatile boolean isRunning = true;
#Override
public void run() {
while (isRunning) {
System.out.println("Running");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
//ignore
}
}
System.out.println("Stopped");
}
public void close() throws Exception {
System.out.println("Stopping");
isRunning = false;
executor.shutdownNow();
}
}
private static class TestThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
TestThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup();
namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0) {
#Override protected void finalize() throws Throwable {
super.finalize();
// probably bad idea but lets see if it gets here
System.out.println("Executor thread removed from JVM");
}
};
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
executorThreadId.set(t.getId());
System.out.println("Executor thread created");
return t;
}
}
}
Here's a sample program using the single-thread Executor that manages to strand a thread so that the JVM can't shut down, but it only manages to do it by not calling shutdownNow:
import java.util.concurrent.*;
public class Exec {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(new MyTask());
Thread.sleep(20000L);
// executor.shutdownNow();
int retryCount = 4;
while (!executor.isTerminated() && retryCount > 0) {
System.out.println("waiting for tasks to terminate");
Thread.sleep(500L);
retryCount -= 1;
}
}
}
class MyTask implements Runnable {
public void run() {
int count = 0;
try {
while (!Thread.currentThread().isInterrupted() && count < 10) {
Thread.sleep(1000L);
count += 1;
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("all done");
}
}
The thread used by the executor has a separate life cycle from the task, this example shows how the task finishes but the thread goes on. Uncommenting the shutdownNow results in the executor's thread terminating. Otherwise the main thread sleeps for a while and exits, leaving the executor's thread hanging out, preventing the JVM from exiting.
My guess is that your close method isn't getting called and your executor never gets shut down. To get more useful answers please add a MVCE so that we can reproduce the problem.
Consider that with interruption there's no need to keep a reference to the Runnable to set the flag. As I read the question the task not finishing is not an issue here, but it would still be better to make the Runnable respond to interruption and lose the flag, just because having less things to keep track of is always an improvement.
I am using ThreadPoolExecutor to run the threads.
ExecutorService executorService = Executors.newCachedThreadPool();
Future<?> future = executorService.submit(new MyRunnable());
Based on some conditions, I need to terminate the long running thread and start the same thread instance again(for some cleanup operations).
Since I have a future object of the thread, I can easily check if it is still running.
future.isDone()
If it is running, I can send a interrupt signal by using
future.cancel(true);
In the MyRunnable class, the interrupt signal is handled. But this condition is checked at the beginning of the loop.
The problem is future.isDone() returns true as soon as interrupt signal is sent. But I need to wait till the thread instance is really completed.
Is there any way to check if the thread is really running/completed?
The Apidoc also mentions that future.isDone() returns true if future.cancel() was called, i.e. it does not always tell you if the task has finished.
To check if the task is finished, you need acces to the Runnable and then you can check if the task has completed or wait for it to complete.
Compare the code below with the output shown beneath it, I think that will give you an idea of your options:
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class Q21227864 {
public static void main(String[] args) {
ExecutorService executorService = Executors.newCachedThreadPool();
Future<?> future = executorService.submit(new MyRunnable());
sleep(100L);
future.cancel(true);
System.out.println("Future done: " + future.isDone());
sleep(100L);
future.cancel(true);
System.out.println("Future done: " + future.isDone());
sleep(500L);
System.out.println("Future done: " + future.isDone());
System.out.println("---");
MyRunnable mr = new MyRunnable();
future = executorService.submit(mr);
sleep(100L);
future.cancel(true);
System.out.println("Runnable done: " + mr.isDone());
sleep(100L);
System.out.println("Runnable done: " + mr.isDone());
mr.waitForCleanup();
System.out.println("Runnable done: " + mr.isDone());
executorService.shutdownNow();
}
public static void sleep(long timeMs) {
try { Thread.sleep(timeMs); } catch (Exception ignored) {}
}
static class MyRunnable implements Runnable {
final CountDownLatch completed = new CountDownLatch(1);
public void run() {
try {
System.out.println("Sleeping loop");
Thread.sleep(1000L);
System.out.println("Sleeping loop done");
} catch (Exception e) {
System.out.println("Stopped loop: " + e);
}
try {
System.out.println("Sleeping cleanup");
Thread.sleep(300L);
System.out.println("Sleeping cleanup done");
} catch (Exception e) {
System.out.println("Stopped cleanup: " + e);
}
completed.countDown();
}
public boolean isDone() {
return (completed.getCount() == 0);
}
public void waitForCleanup() {
try { completed.await(); } catch (Exception ignored) {}
}
}
}
Output:
Sleeping loop
Future done: true
Stopped loop: java.lang.InterruptedException: sleep interrupted
Sleeping cleanup
Future done: true
Sleeping cleanup done
Future done: true
---
Sleeping loop
Runnable done: false
Stopped loop: java.lang.InterruptedException: sleep interrupted
Sleeping cleanup
Runnable done: false
Sleeping cleanup done
Runnable done: true
I don't think future.isDone() returns true as soon as interrupt signal is sent. All ThreadPoolExecutor tasks are executed via FutureTask.run() method
public class FutureTask<V> implements RunnableFuture<V> {
...
public boolean isDone() {
return state != NEW;
}
public void run() {
...
try {
Callable<V> c = callable;
if (c != null && state == NEW) {
V result;
boolean ran;
try {
result = c.call(); <-- this invokes your code
ran = true;
} catch (Throwable ex) {
result = null; <-- if you threw InterruptedException
ran = false;
setException(ex); <-- state changes here
}
if (ran)
set(result); <-- if your code simply returns then state changes here
}
...