I just went through the source code of ThreadPoolExecutor found that it will interrupt all idle workers once the time is up to the set value of keepAliveTime and allowCoreThreadTimeOut is true.
It's a little strange to me it can only invoke the interrupt method when runState >= SHUTDOWN:
The code below is from the method getTask() of ThreadPoolExecutor.
Runnable getTask() {
...
if (workerCanExit()) {
if (runState >= SHUTDOWN) // Wake up others
interruptIdleWorkers();
return null;
}
}
Does that mean all the idle threads can only be interrupted when the runState >= SHUTDOWN (SHUTDOWN, STOP or TERMINATED)? That's to say they will be not interrupted when the state is RUNNING.
You are right. This getTask() method in the ThreadPoolExecutor is called on to gets the next task for a worker thread to run. This code block is only executed when the method call has not identified any Runnable task for execution. So, if nothing is found to execute, it must check for the shutdown state.
from java doc of workerCanExit()
Check whether a worker thread that fails to get a task can exit. We allow a worker thread to die if the pool is stopping, or the queue is empty, or there is at least one thread to handle possibly non-empty queue, even if core timeouts are allowed.
As an example, configure ThreadPoolExecutor as : corePoolSize=1, maxPoolSize=5, workQueueSize=1, keepAliveTime=60s, allowCoreThreadTimeOut=false.
When you offer 5 tasks(each task is time-consuming) concurrently, one of the 5 tasks will enter into workQueue, other 4 tasks will be handled immediately by 4 worker threads newly created almost at the same time.
At this time, the sum of worker threads is 4(workerThreadCount=4). Once one thread completes its task, it will be waiting on workQueue by invoking blocking method, workQueue.take() or workQueue.poll(keepAliveTime). As for which blocking method will be invoked is decided by the workerThreadCount.
For example(Hypothesis), at a time point, workerThread-0 is handling task-0; task-1 is staying in workQueue; workerThread-1 is handling task-2; workerThread-2 is handling task-3; workerThread-3 is handling task-4, and workerThreadCount==4.
workerThread-3 completes task-4, and this time [workerThreadCount==4] > [corePoolSize==1], it will get next task(task-1) from workQueue by workQueue.poll(keepAliveTime). Then, go on handling task-1, and this time workerThreadCount==4. The code segment of ThreadPoolExecutor.java as following:
while (task != null || (task = getTask()) != null) {
task.run();
}
private Runnable getTask() {
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
Runnable r = timed ? workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) : workQueue.take();
}
then workerThread-0 completes task-0, because of [workerThreadCount==4] > [corePoolSize==1], it still get next task from workQueue by workQueue.poll(keepAliveTime). But, this time workQueue is empty, so workerThread-0 will be blocked on it, and the state of workerThread-0 is TIMED_WAITING. Once keepAliveTime elapsed, workQueue.poll(keepAliveTime) will return null, next, workerThread-0 will return from Runnable.run(), and turn into TERMINATED. This time workerThreadCount==3.
then workerThread-1 completes task-2, it will return by the same way as workerThread-0. This time workerThreadCount==2.
then workerThread-3 completes task-1, it will return by the same way as workerThread-1. This time workerThreadCount==1.
then workerThread-2 completes task-3, but this time [workerThreadCount==1] not more than [corePoolSize==1], so it will be blocked when get the next task from workQueue by workQueue.take() until there is a available task in workQueue. And its state is WAITING.
NOTE : the source code comes from JDK8.
Exactly. A correct task(if interruptions are allowed) must checks itself for interrupted flag and terminate (i.e. return from run()).
Related
I have an application that I did thread configuration as follows,
corepoolsize = 10
maxpoolsize = 10
queuecapacity = 10
But when I do use Threads and functions that run on threads (asynchronous process)
I see it is increasing like the follows, But I don't see the threads are decreasing. Is that a indication of memory leak? Can anybody guide me?
Log prints like this,
2021-09-15 01:13:48.554 INFO 111 --- [Async-1]
2021-09-15 01:13:48.654 INFO 121 --- [Async-2]
2021-09-15 01:13:48.754 INFO 132 --- [Async-3]
2021-09-15 01:13:48.854 INFO 140 --- [Async-4]
2021-09-15 01:13:48.954 INFO 155 --- [Async-5]
2021-09-15 01:13:49.554 INFO 160 --- [Async-6]
But I don't see it is calling again [Async-1], [Async-2] like that. Is this a memory leak or a thread cache or after filling up corepoolsize then will it run [Async-1]?
Corepoolsize represents the worker thread that is always alive.
You can check ThreadPoolExecutor#runWorker, ThreadPoolExecutor#getTask method, these worker threads will always try to get tasks from the task queue and execute them. So if no new tasks are posted to the thread pool, then these worker threads will always live, usually blocked in the take method of workQueue, So you don't have to worry about the cpu idling.
Worker threads outside the number of corepoolsize will be recycled, still ThreadPoolExecutor#getTask method:
try {
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
These threads to be recycled can survive at most keepAliveTime time.
The thread pool uses a Hash table to hold the thread's reference, so that the life cycle of the thread can be controlled by operations such as adding and removing references.(ThreadPoolExecutor#processWorkerExit method).
This is expected behaviour. The documentation for ThreadPoolExecutor explicitly states:
When a new task is submitted in method execute(Runnable), if fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle.
I'm trying to understand ConsumableFuture.
Basically, I supply a task to the ConsumableFuture and then put the worker thread running that task to sleep for 2 seconds. I expect the worker thread to resume execution after 2 seconds and return the result.
public class CompletableFutureDemo {
public static void main(String[] args) {
System.err.println("Application started");
CompletableFuture
.supplyAsync(()->work1())
.thenAccept(op-> System.out.println(op));
System.err.println("Application ended");
}
public static int work1() {
System.out.println(Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("work1 called");
return (int) (Math.random() * 100);
}
}
Output:
Application started
ForkJoinPool.commonPool-worker-1
Application ended
Why is the worker thread not resuming?
But if I remove the sleep statement from the worker thread, then I get the desired output.
Application started
ForkJoinPool.commonPool-worker-1
work1 called
Application ended
64
As #Slaw already pointed in the comment, the Main thread completes and exit the application when the work thread is sleeping, so you can call join to keep main thread waiting until work thread completes
System.err.println("Application started");
CompletableFuture
.supplyAsync(()->work1())
.thenAccept(op-> System.out.println(op)).join();
System.err.println("Application ended");
Output :
ForkJoinPool.commonPool-worker-3
Application started
work1 called
12
Application ended
Or you can keep main thread wait after completion of it work
System.err.println("Application started");
CompletableFuture<Void> completableFuture = CompletableFuture
.supplyAsync(()->work1())
.thenAccept(op-> System.out.println(op));
System.err.println("Application ended");
completableFuture.join();
Output :
ForkJoinPool.commonPool-worker-3
Application started
Application ended
work1 called
25
If you have multiple CompletableFuture objects then you can use allOf to wait until all tasks get completed (but in background every completable task will execute asynchronously)
CompletableFuture.allOf(completableFuture1,completableFuture1).join();
I achieved asynchronous operation as well as avoiding marking it as a daemon, by supplying my own instance of Executor. (Any flavour of Executor)
CompletableFuture
.supplyAsync(()->work1(), Executors.newFixedThreadPool(2))
.thenAccept(op-> System.out.println(op));
I think this would have avoided creating daemon threads, similar to what we have in ExecutorServices.
Thank you #Slaw for providing the information on the daemon thread. I would like to find out more why ForkJoin architecture would mark threads as a daemon by default.
I wrote this simple testing program:
fun main() {
println("Main Start")
thread {
println("Thread Start")
Thread.sleep(3000)
println("Thread End")
}
println("Main End")
}
As I can see, the output is:
Main Start
Main End
Thread Start
Thread End
My expectation is that at-least the "Thread End" message will not be printed,
Cause that main function is ended and this main thread should be done running.
Is Kotlin process always waiting for threads to finish before done?
The thread that you have created is a non-daemon thread, and just like in Java, the JVM will not terminate until all the non-daemon thread are finished.
From Kotlin documentation one can read:
fun thread(
start: Boolean = true,
isDaemon: Boolean = false,
contextClassLoader: ClassLoader? = null,
name: String? = null,
priority: Int = -1,
block: () -> Unit ): Thread Creates a thread that runs the specified block of code.
Parameters
start - if true, the thread is immediately started.
isDaemon - if true, the thread is created as a daemon thread. The Java
Virtual Machine exits when the only threads running are all daemon
threads.
contextClassLoader - the class loader to use for loading classes and
resources in this thread.
name - the name of the thread.
priority - the priority of the thread.
Threads in Kotlin, by default, are non-daemon threads. This is why you can still see the output of the thread even thought the main thread have finished its execution. Set the isDaemon to true, and you will see the following output instead:
Main Start
Main End
Thread Start
I use spring thread pool to manage the threads in my project.
But there is some thing wrong when my code is running.
I get exceptions like:
org.springframework.core.task.TaskRejectedException: Executor [java.util.concurrent.ThreadPoolExecutor#4cfc01ab[Running, pool size = 200, active threads = 0, queued tasks = 40, completed tasks = 7990]] did not accept task: java.util.concurrent.FutureTask#6ba9fcd5
It throws TaskRejectedException when the pool "active threads" is zero.
I have read the documentation and the source code from Spring but have found nothing.
My TaskExecutor class:
#Bean
public ThreadPoolTaskExecutor taskExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setKeepAliveSeconds(200);
taskExecutor.setCorePoolSize(200);
taskExecutor.setMaxPoolSize(200);
taskExecutor.setQueueCapacity(40);
taskExecutor.setRejectedExecutionHandler(new
ThreadPoolExecutor.AbortPolicy());
return taskExecutor;
}
active threads sometime is 0 sometime is 10 , never 200 , weird.
In the exception message it has "queued tasks = 40". Since in your setup you have setQueueCapacity(40), executing a thread when queued tasks is already at 40 will throw that exception.
If you are submitting tasks very quickly I would try increasing the queue capacity. Threads do not immediately move from the queue to active, especially when the JVM is busy.
As far as active thread count, it is only an approximation. Java doc of ThreadPoolExecutor.getActiveThreadCount:
/**
* Returns the approximate number of threads that are actively
* executing tasks.
*
* #return the number of threads
*/
public int getActiveCount() {
I'm trying to find a way to use a ThreadPoolExecutor in the following scenario:
I have a separate thread producing and submitting tasks on the thread pool
a task submission is synchronous and will block until the task can be started by the ThreadPoolExecutor
at any given time, only a fixed number of tasks can be executing in parallel. An unbounded number of tasks running at the same time may result in memory exhaustion.
before submitting a task, the producer thread always checks that some maximum build time has not been exceeded since the first submitted task. If it was exceeded, the thread shutdowns but any task currently running on the thread pool runs to completion before the application terminates.
when the producer thread terminates, there should be no unstarted task on the queue of the thread pool.
To give more context, I currently just submit all tasks at once and cancel all the futures returned by ExecutorService.submit after the max build time is expired. I ignore all resulting CancellationExceptions since they are expected. The problem is that the behaviour of Future.cancel(false) is odd and inadequate to my use-case:
it prevents any unstarted task to run (good)
it does not interrupt currently running tasks and let them run to completion (good)
however, it ignores any exception thrown by the currently running tasks and instead throws a CancellationException for which Exception.getCause() is null. Therefore, I can't distinguish a task which has been canceled before running from a task which has continued running after the max build time and failed with an exception ! That's unfortunate, because in this case I would like to propagate the exception and report it to some error handling mechanism.
I looked into the different blocking queues Java has to offer and found this: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/SynchronousQueue.html. That seemed ideal at first, but then looking at https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html, it does not seem to play with ThreadPoolExecutor in the way I want it to:
Direct handoffs. A good default choice for a work queue is a
SynchronousQueue that hands off tasks to threads without otherwise
holding them. Here, an attempt to queue a task will fail if no threads
are immediately available to run it, so a new thread will be
constructed. This policy avoids lockups when handling sets of requests
that might have internal dependencies. Direct handoffs generally
require unbounded maximumPoolSizes to avoid rejection of new submitted
tasks. This in turn admits the possibility of unbounded thread growth
when commands continue to arrive on average faster than they can be
processed.
What would be ideal is that the consumer (= the pool) blocks on SynchronousQueue.poll and the producer (= task producer thread) blocks on SynchronousQueue.put.
Any idea how I can implement the scenario I described without writing any complex scheduling logic (what ThreadPoolExecutor should enclose for me) ?
I Believe that you're in the right path... all you have to do is use a SynchronousQueue in conjuction of a RejectedExecutionHandler, using the following constructor ... in that way you can define a fixed max size thread pool (limiting your resources usage) and define a fallback mechanism to re schedule those task that cannot be processed (because the pool was full)... Example:
public class Experiment {
public static final long HANDLER_SLEEP_TIME = 4000;
public static final int MAX_POOL_SIZE = 1;
public static void main(String[] args) throws InterruptedException {
SynchronousQueue<Runnable> queue;
RejectedExecutionHandler handler;
ThreadPoolExecutor pool;
Runnable runA, runB;
queue = new SynchronousQueue<>();
handler = new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
System.out.println("Handler invoked! Thread: " + Thread.currentThread().getName());
Thread.sleep(HANDLER_SLEEP_TIME); // this let runnableA finish
executor.submit(r); // re schedule
} catch (InterruptedException ex) {
throw new RuntimeException("Handler Exception!", ex);
}
}
};
pool = new ThreadPoolExecutor(1, MAX_POOL_SIZE, 10, TimeUnit.SECONDS, queue, handler);
runA = new Runnable() {
#Override
public void run() {
try {
Thread.sleep(3000);
System.out.println("hello, I'm runnable A");
} catch (Exception ex) {
throw new RuntimeException("RunnableA", ex);
}
}
};
runB = new Runnable() {
#Override
public void run() {
System.out.println("hello, I'm runnable B");
}
};
pool.submit(runA);
pool.submit(runB);
pool.shutdown();
}
}
NOTE: the implementation of the RejectedExecutionHandler is up to you! I just only suggest a sleep as a blocking mechanism, but hrer you can do logic more complex as ask the threadpool is it has free threads or not. If not, then sleep; if yes, then submit task again...
I found another option than the one proposed by #Carlitos Way. It consists in directly adding tasks on the queue using BlockingQueue.offer. The only reason I did not manage to make it work at first and I had to post this question is that I did not know that the default behaviour of a ThreadPoolExecutor is to start without any thread. The threads will be created lazily using a thread factory and may be deleted/repopulated depending on the core and max sizes of the pool and the number of tasks being submitted concurrently.
Since the thread creation was lazy, my attempts to block on the call to offer failed because SynchronousQueue.offer immediately exits if nobody is waiting to get an element from the queue. Conversely, SynchronousQueue.put blocks until someone asks to take an item from the queue, which will never happen if the thread pool is empty.
Therefore, the workaround is to force the thread pool to create the core threads eagerly using ThreadPoolExecutor.prestartAllCoreThreads. My problem then becomes fairly trivial. I made a simplified version of my real use-case:
import java.util.Random;
import java.util.concurrent.SynchronousQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicLong;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import static java.util.concurrent.TimeUnit.SECONDS;
public class SimplifiedBuildScheduler {
private static final int MAX_POOL_SIZE = 10;
private static final Random random = new Random();
private static final AtomicLong nextTaskId = new AtomicLong(0);
public static void main(String[] args) throws InterruptedException {
SynchronousQueue<Runnable> queue = new SynchronousQueue<>();
// this is a soft requirement in my system, not a real-time guarantee. See the complete semantics in my question.
long maxBuildTimeInMillis = 50;
// this timeout must be small compared to maxBuildTimeInMillis in order to accurately match the maximum build time
long taskSubmissionTimeoutInMillis = 1;
ThreadPoolExecutor pool = new ThreadPoolExecutor(MAX_POOL_SIZE, MAX_POOL_SIZE, 0, SECONDS, queue);
pool.prestartAllCoreThreads();
Runnable nextTask = makeTask(maxBuildTimeInMillis);
long millisAtStart = System.currentTimeMillis();
while (maxBuildTimeInMillis > System.currentTimeMillis() - millisAtStart) {
boolean submitted = queue.offer(nextTask, taskSubmissionTimeoutInMillis, MILLISECONDS);
if (submitted) {
nextTask = makeTask(maxBuildTimeInMillis);
} else {
System.out.println("Task " + nextTaskId.get() + " was not submitted. " + "It will be rescheduled unless " +
"the max build time has expired");
}
}
System.out.println("Max build time has expired. Stop submitting new tasks and running existing tasks to completion");
pool.shutdown();
pool.awaitTermination(9999999, SECONDS);
}
private static Runnable makeTask(long maxBuildTimeInMillis) {
long sleepTimeInMillis = randomSleepTime(maxBuildTimeInMillis);
long taskId = nextTaskId.getAndIncrement();
return () -> {
try {
System.out.println("Task " + taskId + " sleeping for " + sleepTimeInMillis + " ms");
Thread.sleep(sleepTimeInMillis);
System.out.println("Task " + taskId + " completed !");
} catch (InterruptedException ex) {
throw new RuntimeException(ex);
}
};
}
private static int randomSleepTime(long maxBuildTimeInMillis) {
// voluntarily make it possible that a task finishes after the max build time is expired
return 1 + random.nextInt(2 * Math.toIntExact(maxBuildTimeInMillis));
}
}
An example of output is the following:
Task 1 was not submitted. It will be rescheduled unless the max build time has expired
Task 0 sleeping for 23 ms
Task 1 sleeping for 26 ms
Task 2 sleeping for 6 ms
Task 3 sleeping for 9 ms
Task 4 sleeping for 75 ms
Task 5 sleeping for 35 ms
Task 6 sleeping for 81 ms
Task 8 was not submitted. It will be rescheduled unless the max build time has expired
Task 8 was not submitted. It will be rescheduled unless the max build time has expired
Task 7 sleeping for 86 ms
Task 8 sleeping for 47 ms
Task 9 sleeping for 40 ms
Task 11 was not submitted. It will be rescheduled unless the max build time has expired
Task 2 completed !
Task 10 sleeping for 76 ms
Task 12 was not submitted. It will be rescheduled unless the max build time has expired
Task 3 completed !
Task 11 sleeping for 31 ms
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 13 was not submitted. It will be rescheduled unless the max build time has expired
Task 0 completed !
Task 12 sleeping for 7 ms
Task 14 was not submitted. It will be rescheduled unless the max build time has expired
Task 14 was not submitted. It will be rescheduled unless the max build time has expired
Task 1 completed !
Task 13 sleeping for 40 ms
Task 15 was not submitted. It will be rescheduled unless the max build time has expired
Task 12 completed !
Task 14 sleeping for 93 ms
Task 16 was not submitted. It will be rescheduled unless the max build time has expired
Task 16 was not submitted. It will be rescheduled unless the max build time has expired
Task 16 was not submitted. It will be rescheduled unless the max build time has expired
Task 5 completed !
Task 15 sleeping for 20 ms
Task 17 was not submitted. It will be rescheduled unless the max build time has expired
Task 17 was not submitted. It will be rescheduled unless the max build time has expired
Task 11 completed !
Task 16 sleeping for 27 ms
Task 18 was not submitted. It will be rescheduled unless the max build time has expired
Task 18 was not submitted. It will be rescheduled unless the max build time has expired
Task 9 completed !
Task 17 sleeping for 95 ms
Task 19 was not submitted. It will be rescheduled unless the max build time has expired
Max build time has expired. Stop submitting new tasks and running existing tasks to completion
Task 8 completed !
Task 15 completed !
Task 13 completed !
Task 16 completed !
Task 4 completed !
Task 6 completed !
Task 10 completed !
Task 7 completed !
Task 14 completed !
Task 17 completed !
You'll notice, for example, that task 19 was not rescheduled because the max build time expired before the scheduler can attempt to offer it to the queue a second time. You can also see than all the ongoing tasks that started before the max build time expired do run to completion.
Note: As noted in my comments in the code, the max build time is a soft requirement, which means that it might not be met exactly, and my solution indeed allows for a task to be submitted even after the max build time is expired. This can happen if the call to offer starts just before the max build time expires and finishes after. To reduce the odds of it happening, it is important that the timeout used in the call to offer is much smaller than the max build time. In the real system, the thread pool is usually busy with no idle thread, therefore the probability of this race condition to occur is extremely small, and it has no bad consequence on the system when it does happen, since the max build time is a best effort attempt to meet an overall running time, not an exact and rigid constraint.