I have a bunch of tasks:
public class ProcessDay implements Runnable{
#Override
public void run(){
List<ProcessHour> hr = //required hours
//do some post actions
}
}
public class ProcessHour implements Runnable{
#Override
public void run(){
List<ProcessMinutes> mins = //required minutes
//do some post actions
}
}
ProcessSeconds, ProcessMonth, ... etc
And so forth. It would be convinient to use ForkJoinPool here but it's not good from performance standpoint, because ProcessXXX tasks are being submitted to a cluster of machines and hence the method invocation itself is very short.
So for performance it's good to use Executors.cachedThreadPoolExecutor. But is there a way to combine ForkJoinPool with chachedThreadPool semantic. I mean creating threads on demand and release them if not used.
Maybe there is a better approach to this? Can you suggest something?
I mean creating threads on demand and release them if not used.
That is how cached thread pool operates. It starts with 0 threads and creates a new one each time when there is a new task to process and all threads in the pool are busy. Thread is terminated if it was idle for 60 seconds.
Default cached thread pool is created using the following ThreadPoolExecutor constructor:
return new ThreadPoolExecutor(0,
Integer.MAX_VALUE,
60L,
TimeUnit.SECONDS,
new SynchronousQueue<Runnable>())
It created with 0 possible idle threads in pool, unlimited maximum number of threads, 60 seconds timeout before idle thread is terminated, and queue implementation that doesn't store tasks and just transfers them between pool and its threads. Such pool is suitable for lots of short-lived tasks that is probably your case. As you can see it's also pretty easy to adjust it configuration according to your needs using direct ThreadPoolExecutor constructor.
Related
We are using ScheduledExecutorService for our project and I would like to know how to find the number of free threads available in the thread pool before executing some functions.
private static final ScheduledExecutorService executor;
executor = Executors.newScheduledThreadPool(16);
public void scheduleTask(){
//I want to check the number of free threads or number of active threads in thread pool so that I can schedule jobs accordingly
executor.scheduleAtFixedRate(this, 0, 2, TimeUnit.HOURS);
I tried and found that there are some methods to find the number of active threads for ThreadPoolExecutor but couldn't find any for ScheduledExecutorService
You can use the Thread.activeCount() method, to get the total amount of active Threads.
ScheduledExecutorService is an abstract interface, it is not really related to threads. It can be backed by some other mechanism than threads.
You need to use its specific implementation based on threads, which is ScheduledThreadPoolExecutor. I think in your case it is safe to assume that Executors.newScheduledThreadPool() returns exactly this type:
val executor = Executors.newScheduledThreadPool(16) as ScheduledThreadPoolExecutor
println(executor.activeCount)
I have 5 #Scheduled annotated methods and pool size of my ThreadPoolTaskScheduler is 10. Annotations of my methods are the same and like this.
#Scheduled(fixedDelay = 1000, initialDelay = 10000)
My question is that;
When one of the scheduled methods gets a thread from pool and start to run; does it release the thread to pool before its execution is finished? (for example in case of context switch etc.) or this thread is used until the end of the execution?
I mean is it possible to some part of the scheduled task is done with thread-1 and some part is done with thread-2 ?
Threading is complicated and my understanding is not as great as others, but here's my attempt at a brief explanation of how the #Scheduled Spring annotation works:
Spring uses a TaskScheduler:
public interface TaskScheduler {
ScheduledFuture schedule(Runnable task, Trigger trigger);
ScheduledFuture schedule(Runnable task, Date startTime);
ScheduledFuture scheduleAtFixedRate(Runnable task, Date startTime, long period);
ScheduledFuture scheduleAtFixedRate(Runnable task, long period);
ScheduledFuture scheduleWithFixedDelay(Runnable task, Date startTime, long delay);
ScheduledFuture scheduleWithFixedDelay(Runnable task, long delay);
}
https://docs.spring.io/spring/docs/3.2.x/spring-framework-reference/html/scheduling.html#scheduling-task-scheduler
Which submits the annotated code, i.e. task code, to a high-level concurrency object called an executor. The executor class is ThreadPoolTaskExecutor. That class submits tasks to the thread pool to be run by the first available thread in the pool. The thread pool size you set determines how many active threads you can have. If you set allowCoreThreadTimeOut to true then threads in the pool that have no work available to do within their timeout interval will be terminated.
Spring uses a ThreadPoolTaskExecutor to manage the thread pool:
https://github.com/spring-projects/spring-framework/blob/master/spring-context/src/main/java/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.java
Keeping a pool of threads alive reduces the time that would normally be added while waiting for the thread to be created. See this question for some more info.
Ultimately, the java.lang.Thread class runs the Runnable or Callable instances that are created by the ThreadPoolTaskExecutor. The Thread class implements a run() method that is basically your code you want the thread to run:
public Thread(Runnable target) {
init(null, target, "Thread-" + nextThreadNum(), 0);
}
private void init(ThreadGroup g, Runnable target, String name,
long stackSize, AccessControlContext acc) {
...
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/Thread.java
The actual switching between threads, i.e. the context switch is OS-dependent but in general threads will be divided amongst CPUs and then each CPU cycles through the threads based on the timeout interval and does a bit of work and then pauses and switches continuously between threads until the task(s) are complete.
does it release the thread to pool before its execution is finished?
(for example in case of context switch etc.) or this thread is used
until the end of the execution?
The Runnable code can definitely stop executing in the middle of an operation but the Threads in a thread pool are usually kept alive until there's no more work to be done.
Here's more info from the Oracle documentation that explains thread pools:
Most of the executor implementations in java.util.concurrent use
thread pools, which consist of worker threads. This kind of thread
exists separately from the Runnable and Callable tasks it executes and
is often used to execute multiple tasks.
Using worker threads minimizes the overhead due to thread creation.
Thread objects use a significant amount of memory, and in a
large-scale application, allocating and deallocating many thread
objects creates a significant memory management overhead.
One common type of thread pool is the fixed thread pool. This type of
pool always has a specified number of threads running; if a thread is
somehow terminated while it is still in use, it is automatically
replaced with a new thread. Tasks are submitted to the pool via an
internal queue, which holds extra tasks whenever there are more active
tasks than threads.
An important advantage of the fixed thread pool is that applications
using it degrade gracefully. To understand this, consider a web server
application where each HTTP request is handled by a separate thread.
If the application simply creates a new thread for every new HTTP
request, and the system receives more requests than it can handle
immediately, the application will suddenly stop responding to all
requests when the overhead of all those threads exceed the capacity of
the system. With a limit on the number of the threads that can be
created, the application will not be servicing HTTP requests as
quickly as they come in, but it will be servicing them as quickly as
the system can sustain.
I have an ExecutorService to execute my tasks concurrently. Most of these tasks are simple actions that require ~300ms to complete each. But a few of these tasks are background processing queues that take in new sub-tasks all the time and execute them in order. These background tasks will remain active as long as there are normal tasks running.
The ThreadPool is generated through one of the Executors' methods (don't know which yet) with a user-specified Thread count. My fear is that the following situation might happen: There are less threads than there are background queues. At a given moment, all background queues are working, blocking all the threads of the ExecutorService. No normal tasks will thus be started and the program hang forever.
Is there a possibility this might happen and how can I avoid it? I'm thinking of a possibility to interrupt the background tasks to leave the place to the normal ones.
The goal is to limit the number of threads in my application because Google said having a lot of threads is bad and having them idle for most of the time is bad too.
There are ~10000 tasks that are going to be submitted in a very short amount of time at the begin of the program execution. About ~50 background task queues are needed and most of the time will be spent waiting for a background job to do.
Don't mix up long running tasks with short running tasks in same ExecutorService.
Use two different ExecutorService instances with right pool size. Even if you set the size as 50 for background threads with long running tasks, performance of the pool is not optimal since number of available cores (2 core, 4 core, 8 core etc.) is not in that number.
I would like to create two separate ExecutorService initialized with Runtime.getRuntime().availableProcessors()/2;
Have a look at below posts for more details to effectively utilize available cores:
How to implement simple threading with a fixed number of worker threads
Dynamic Thread Pool
You can have an unlimited number of threads, check out cache thread pool
Creates a thread pool that creates new threads as needed, but will
reuse previously constructed threads when they are available. These
pools will typically improve the performance of programs that execute
many short-lived asynchronous tasks. Calls to execute will reuse
previously constructed threads if available. If no existing thread is
available, a new thread will be created and added to the pool. Threads
that have not been used for sixty seconds are terminated and removed
from the cache. Thus, a pool that remains idle for long enough will
not consume any resources. Note that pools with similar properties but
different details (for example, timeout parameters) may be created
using ThreadPoolExecutor constructors.
Another option is create two different pools and reserve one for priority tasks.
The solution is that the background tasks stop instead of being idle when there is no work and get restarted if there are enough tasks again.
public class BackgroundQueue implements Runnable {
private final ExecutorService service;
private final Queue<Runnable> tasks = new ConcurrentLinkedQueue<>();
private final AtomicBoolean running = new AtomicBoolean(false);
private Future<?> future;
public BackgroundQueue(ExecutorService service) {
this.service = Objects.requireNonNull(service);
// Create a Future that immediately returns null
FutureTask f = new FutureTask<>(() -> null);
f.run();
future = f;
}
public void awaitQueueTermination() throws InterruptedException, ExecutionException {
do {
future.get();
} while (!tasks.isEmpty() || running.get());
}
public synchronized void submit(Runnable task) {
tasks.add(task);
if (running.compareAndSet(false, true))
future = service.submit(this);
}
#Override
public void run() {
while (!running.compareAndSet(tasks.isEmpty(), false)) {
tasks.remove().run();
}
}
}
I have a cached thread pool where new tasks are spawned in rather unpredictable manner. These tasks don't generate any results (they are Runnables rather than Callables).
I would like to have an action to be executed whenever the pool has no active workers.
However I don't want to shutdown the pool (and obviously use awaitTermination) because I would have to reinitialize it again when a new task arrives (as it could arrive unpredictably, even during the shutdown).
I came up with the following possible approaches:
Have an extra thread (outside the pool) which is spawned whenever a new task is spawned AND the ThreadPoolExecutor had no active workers. It should then continually check the getActiveWorkers() until it returns 0 and if yes, execute the desired action.
Have some thread-safe queue (which one?), where the Future of every newly spawned task is added. Whenever there's at least one entry in the queue, spawn an extra thread (outside the pool) which waits until the queue is empty and executes the desired action.
Implement a PriorityBlockingQueue to use with the pool and assign the worker threads higher priority than to the thread (now from inside the pool) which executes the desired action.
My question:
I was wondering if there is some cleaner solution, which uses some nice synchronization object (like CountDownLatch, which however cannot be used here, because I don't know the number of tasks in advance) ?
If I were you, I would implement a decorator for your thread pool that keeps track of the scheduled tasks and slighlig modifies the tasks that are run. This way, whenever a Runnable is scheduled, you can instead schedule another, decoarated Runnable which is capable of tracing its own process.
This decorator would look something like:
class RunnableDecorator implements Runnable {
private final Runnable delegate;
// this task counter must be increased on any
// scheduling of a task by the thread pool
private final AtomicInteger taskCounter;
// Constructor omitted
#Override
public void run() {
try {
delegate.run();
} finally {
if (taskCounter.decrementAndGet() == 0) {
// spawn idle action
}
}
}
}
Of course, the thread pool has to increment the counter every time a task is scheduled. Thus, the logic for this must not be added to the Runnable but to the ThreadPool. Finally, it is up to you to decide if you want to run the idle action in the same thread or if you want to provide a reference to the executing thread pool to run a new thread. If you decide the latter, note however that the completion of the idle action would then trigger another idle action. You might however also provide a method for a sort of raw scheduling. You could also add the decoration to the thread queue what however makes it harder to provide this sort of raw scheduling.
This approach is non-blocking and does not mess with your code base too much. Note that the tread pool does not start an action when it is created and therefore empty by definition.
If you look at the source behind Executors.newCachedThreadPool(), you can see how it's created with a ThreadPoolExecutor. Using that, override the execute and afterExecute methods to add a counter. This way the increment and decrement logic is isolated in one location. Ex:
ExecutorService executor = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>()) {
private AtomicInteger counter = new AtomicInteger(0);
#Override
public void execute(Runnable r) {
counter.incrementAndGet();
super.execute(r);
}
#Override
public void afterExecute(Runnable r, Throwable t) {
if (counter.decrementAndGet() == 0) {
// thread pool is idle - do something
}
super.afterExecute(r, t);
}
};
I use ThreadPoolExecutor to manage a thread pool. What we want are:
if the pool has less than corePoolSize threads, kick off a new thread for a new task;
if the pool has more than corePoolSize threads and all are busy, kick off a new thread for a new task until maxPoolSize is reached. In that case, reject the task;
keep corePoolSize number of threads alive even if they are idling, excess threads will die if they have been idle for more than keepAliveTime
According the Java6 docs, keepAliveTime should work as above. But in my test code, it doesn't work consistently.
When I set keepAliveTime to 0, it works fine, always keeping core threads alive and terminating excess threads when they finish;
but, as shown below, when I set keepAliveTime to a positive value, it seems that it terminate ALL idle threads, no matter they are core threads or not.
ThreadPoolExecutor executor = new ThreadPoolExecutor(2, 4, 500, TimeUnit.MILLISECONDS, new SynchronousQueue<Runnable>());
assertFalse("Not allow core threads to time out.", executor.allowsCoreThreadTimeOut());
Task task_1 = new Task(1000);
Task task_2 = new Task(1000);
Task task_3 = new Task(1000);
executor.execute(task_1);
executor.execute(task_2);
executor.execute(task_3);
Thread.sleep(1050L);
assertEquals("Completed 3 tasks.", 3, executor.getCompletedTaskCount());
assertEquals("Three threads are in the pool.", 3, executor.getPoolSize());
Thread.sleep(600L);
//////// This assertion will fail: **expected <2> but was <0>**
assertEquals("Two threads are in the pool.", 2, executor.getPoolSize());
////----
private static class Task implements Runnable {
private long sleepMillis;
public Task(final long sleepMillis) {
this.sleepMillis = sleepMillis;
}
public void run() {
try { Thread.sleep(sleepMillis);
} catch (Exception e) { System.out.println(e); }
}
}
Is there any misunderstanding about keepAliveTime or getPoolSize? If getPoolSize is not the correct API, how can I get to know the number of "alive" threads (idle or busy)?
Thanks in advance.
Java 7 passes the test. Apparently java 6 has a bug. Reading its code, all threads can exit when the queue is empty, that is definitely wrong for core threads.
#arosima keepAliveTime only takes effect on excess threads rather than core threads. Core threads will be always kept in the pool. I also wrote another answer to understand, hope it helpful.
I am pretty sure the pool size is a target size. I don't think there are any guarantees about shutting down threads (there is no background checker) I think its just a hint. Idle threads give next to no overhead.