We Keep Coding

ThreadPoolExecutor with corePoolSize 0 should not execute tasks until task queue is full

I was going through Java Concurrency In Practice and got stuck at the 8.3.1 Thread creation and teardown topic. The following footnote warns about keeping corePoolSize to zero.
Developers are sometimes tempted to set the core size to zero so that the worker threads will
eventually be torn down and therefore won’t prevent the JVM from exiting, but this can cause some
strange-seeming behavior in thread pools that don’t use a SynchronousQueue for their work queue
(as newCachedThreadPool does). If the pool is already at the core size, ThreadPoolExecutor creates
a new thread only if the work queue is full. So tasks submitted to a thread pool with a work queue
that has any capacity and a core size of zero will not execute until the queue fills up, which is usually
not what is desired.
So to verify this I wrote this program which does not work as stated above.
final int corePoolSize = 0;
ThreadPoolExecutor tp = new ThreadPoolExecutor(corePoolSize, 1, 5, TimeUnit.SECONDS,
new LinkedBlockingQueue<>());
// If the pool is already at the core size
if (tp.getPoolSize() == corePoolSize) {
ExecutorService ex = tp;
// So tasks submitted to a thread pool with a work queue that has any capacity
// and a core size of zero will not execute until the queue fills up.
// So, this should not execute until queue fills up.
ex.execute(() -> System.out.println("Hello"));
}
Output:
Hello
So, does the behavior of the program suggest that ThreadPoolExecutor creates at least one thread if a task is submitted irrespective of corePoolSize=0. If yes, then what is the warning about in the text book.
EDIT: Tested the code in jdk1.5.0_22 upon #S.K.'s suggestion with following change:
ThreadPoolExecutor tp = new ThreadPoolExecutor(corePoolSize, 1, 5, TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(1));//Queue size is set to 1.
But with this change, the program terminates without printing any output.
So am I misinterpreting these statements from the book?
EDIT (#sjlee): It's hard to add code in the comment, so I'll add it as an edit here... Can you try out this modification and run it against both the latest JDK and JDK 1.5?
final int corePoolSize = 0;
ThreadPoolExecutor tp = new ThreadPoolExecutor(corePoolSize, 1, 5, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
// If the pool is already at the core size
if (tp.getPoolSize() == corePoolSize) {
ExecutorService ex = tp;
// So tasks submitted to a thread pool with a work queue that has any capacity
// and a core size of zero will not execute until the queue fills up.
// So, this should not execute until queue fills up.
ex.execute(() -> System.out.println("Hello"));
}
tp.shutdown();
if (tp.awaitTermination(1, TimeUnit.SECONDS)) {
System.out.println("thread pool shut down. exiting.");
} else {
System.out.println("shutdown timed out. exiting.");
}
#sjlee Have posted the result in comments.

This odd behavior of ThreadPoolExecutor in Java 5 when the core pool size is zero was apparently recognized as a bug and quietly fixed in Java 6.
Indeed, the problem reappeared in Java 7 as a result of some code reworking between 6 and 7. It was then reported as a bug, acknowledged as a bug and fixed.
Either way, you should not be using a version of Java that is affected by this bug. Java 5 was end-of-life in 2015, and the latest available versions of Java 6 and later are not affected. That section of "Java Concurrency In Practice" is no longer apropos.
References:
http://cs.oswego.edu/pipermail/concurrency-interest/2006-December/003453.html (read the entire thread)
http://gee.cs.oswego.edu/dl/concurrency-interest/index.html (see the version of ThreadPoolExecutor in the JSR166y bundle.)
https://bugs.openjdk.java.net/browse/JDK-7091003)

While running this program in jdk 1.5,1.6,1.7 and 1.8, I found different implementations of ThreadPoolExecutor#execute(Runnable) in 1.5,1.6 and 1.7+. Here's what I found:
JDK 1.5 implementation
//Here poolSize is the number of core threads running.
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
for (;;) {
if (runState != RUNNING) {
reject(command);
return;
}
if (poolSize < corePoolSize && addIfUnderCorePoolSize(command))
return;
if (workQueue.offer(command))
return;
Runnable r = addIfUnderMaximumPoolSize(command);
if (r == command)
return;
if (r == null) {
reject(command);
return;
}
// else retry
}
}
This implementation does not create a thread when corePoolSize is 0, therefore the supplied task does not execute.
JDK 1.6 implementation
//Here poolSize is the number of core threads running.
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
if (poolSize >= corePoolSize || !addIfUnderCorePoolSize(command)) {
if (runState == RUNNING && workQueue.offer(command)) {
if (runState != RUNNING || poolSize == 0)
ensureQueuedTaskHandled(command);
}
else if (!addIfUnderMaximumPoolSize(command))
reject(command); // is shutdown or saturated
}
}
JDK 1.6 creates a new thread even if the corePoolSize is 0.
JDK 1.7+ implementation(Similar to JDK 1.6 but with better locks and state checks)
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reject(command);
}
JDK 1.7 too creates a new thread even if the corePoolSize is 0.
So, it seems that corePoolSize=0 is a special case in each versions of JDK 1.5 and JDK 1.6+.
But it is strange that the book's explanation doesn't match any of the program results.

Seems like it was a bug with older java versions but it doesn't exist now in Java 1.8.
According to the Java 1.8 documentation from ThreadPoolExecutor.execute():
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
* ....
*/
In the second point, there is a recheck after adding a worker to the queue that if instead of queuing the task, a new thread can be started, than rollback the enqueuing and start a new thread.
This is what is happening. During first check the task is queued but during recheck, a new thread is started which executes your task.

you can modify an BlockingQueue so that it does not accept any Runnable via offer (used by Executor) and add it via "add" in the reject case.
This setup have 0 core threads and fill up 32 running threads before the jobs are queued. This is what i think many people expect that first up to running threads are filled and than queued.
private static BlockingQueue<Runnable> workQueue = new LinkedBlockingDeque<>() {
private static final long serialVersionUID = 1L;
#Override public boolean offer(Runnable e) { return false; }
};
private static ThreadPoolExecutor executor = new ThreadPoolExecutor(0, 32, 10, TimeUnit.SECONDS, workQueue, (r,e)->workQueue.add(r));

Java - Semaphore release without acquire

I have threads which are given random number (1 to n) and are instructed to print them in sorted order. I used semaphore such that I acquire the number of permits = random number and release one permit more than what was acquired.
acquired = random number; released = 1+random number
Initial permit count for semaphore is 1. So thread with random number 1 should get permit and then 2 and so on.
This is supported as per the documentation given below
There is no requirement that a thread that releases a permit must have acquired that permit by calling acquire().
The problem is my program gets stuck after 1 for n>2.
My program is given below:
import java.util.concurrent.Semaphore;
public class MultiThreading {
public static void main(String[] args) {
Semaphore sem = new Semaphore(1,false);
for(int i=5;i>=1;i--)
new MyThread(i, sem);
}
}
class MyThread implements Runnable {
int var;Semaphore sem;
public MyThread(int a, Semaphore s) {
var =a;sem=s;
new Thread(this).start();
}
#Override
public void run() {
System.out.println("Acquiring lock -- "+var);
try {
sem.acquire(var);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(var);
System.out.println("Releasing lock -- "+var);
sem.release(var+1);
}
}
Output is :
Acquiring lock -- 4
Acquiring lock -- 5
Acquiring lock -- 3
Acquiring lock -- 2
Acquiring lock -- 1
1
Releasing lock -- 1
While If I modify my code with tryAcquire, it runs perfectly well.
Below is new run implementation
#Override
public void run() {
boolean acquired = false;
while(!acquired) {
acquired = sem.tryAcquire(var);
}
System.out.println(var);
sem.release(var+1);
}
Can someone please explain the semaphore's permit acquire mechanism when mulitple threads are waiting with different permit request??

It's a clever strategy, but you're misunderstanding how Sempahore hands out permits. If you run your code enough times you'll actually see it reach step two:
Acquiring lock -- 5
Acquiring lock -- 1
1
Releasing lock -- 1
Acquiring lock -- 3
Acquiring lock -- 2
2
Acquiring lock -- 4
Releasing lock -- 2
If you keep on re-running it enough times you'd actually see it successfully finish. This happens because of how Semaphore hands out permits. You're assuming Semaphore will try to accommodate an acquire() call as soon as it has enough permits to do so. If we look carefully at the documentation for Semaphore.aquire(int) we'll see that is not the case (emphasis mine):
If insufficient permits are available then the current thread becomes disabled for thread scheduling purposes and lies dormant until ... some other thread invokes one of the release methods for this semaphore, the current thread is next to be assigned permits and the number of available permits satisfies this request.
In other words Semaphore keeps a queue of pending acquire request and, upon each call to .release(), only checks the head of the queue. In particular if you enable fair queuing (set the second constructor argument to true) you'll see even step one doesn't occur, because step 5 is (usually) the first in the queue and even new acquire() calls that could be fulfilled will be queued up behind the other pending calls.
In short this means you cannot rely on .acquire() to return as soon as possible, as your code assumes.
By using .tryAcquire() in a loop instead you avoid making any blocking calls (and therefore put a lot more load on your Semaphore) and as soon as the necessary number of permits becomes available a tryAcquire() call will successfully obtain them. This works but is wasteful.
Picture a wait-list at a restaurant. Using .aquire() is like putting your name on the list and waiting to be called. It may not be perfectly efficient, but they'll get to you in a (reasonably) fair amount of time. Imagine instead if everyone just shouted at the host "Do you have a table for n yet?" as often as they could - that's your tryAquire() loop. It may still work out (as it does in your example) but it's certainly not the right way to go about it.
So what should you do instead? There's a number of possibly useful tools in java.util.concurrent, and which is best somewhat depends on what exactly you're trying to do. Seeing as you're effectively having each thread start the next one I might use a BlockingQueue as the synchronization aid, pushing the next step into the queue each time. Each thread would then poll the queue, and if it's not the activated thread's turn replace the value and wait again.
Here's an example:
public class MultiThreading {
public static void main(String[] args) throws Exception{
// Use fair queuing to prevent an out-of-order task
// from jumping to the head of the line again
// try setting this to false - you'll see far more re-queuing calls
BlockingQueue<Integer> queue = new ArrayBlockingQueue<>(1, true);
for (int i = 5; i >= 1; i--) {
Thread.sleep(100); // not necessary, just helps demonstrate the queuing behavior
new MyThread(i, queue).start();
}
queue.add(1); // work starts now
}
static class MyThread extends Thread {
int var;
BlockingQueue<Integer> queue;
public MyThread(int var, BlockingQueue<Integer> queue) {
this.var = var;
this.queue = queue;
}
#Override
public void run() {
System.out.println("Task " + var + " is now pending...");
try {
while (true) {
int task = queue.take();
if (task != var) {
System.out.println(
"Task " + var + " got task " + task + " instead - re-queuing");
queue.add(task);
} else {
break;
}
}
} catch (InterruptedException e) {
// If a thread is interrupted, re-mark the thread interrupted and terminate
Thread.currentThread().interrupt();
return;
}
System.out.println("Finished task " + var);
System.out.println("Registering task " + (var + 1) + " to run next");
queue.add(var + 1);
}
}
}
This prints the following and terminates successfully:
Task 5 is now pending...
Task 4 is now pending...
Task 3 is now pending...
Task 2 is now pending...
Task 1 is now pending...
Task 5 got task 1 instead - re-queuing
Task 4 got task 1 instead - re-queuing
Task 3 got task 1 instead - re-queuing
Task 2 got task 1 instead - re-queuing
Finished task 1
Registering task 2 to run next
Task 5 got task 2 instead - re-queuing
Task 4 got task 2 instead - re-queuing
Task 3 got task 2 instead - re-queuing
Finished task 2
Registering task 3 to run next
Task 5 got task 3 instead - re-queuing
Task 4 got task 3 instead - re-queuing
Finished task 3
Registering task 4 to run next
Task 5 got task 4 instead - re-queuing
Finished task 4
Registering task 5 to run next
Finished task 5
Registering task 6 to run next

The Javadoc for Semaphore.acquire(int) says:
If insufficient permits are available then the current thread becomes
disabled for thread scheduling purposes and lies dormant until one of
two things happens:
Some other thread invokes one of the release methods for this semaphore,
the current thread is next to be assigned permits and the number of
available permits satisfies this request [or the thread is interrupted].
The thread that is "next to be assigned" is probably thread 4 in your example. It is waiting until there are 4 permits available. However, thread 1, which gets a permit upon calling acquire(), only releases 2 permits, which is not enough to unblock thread 4. Meanwhile, thread 2, which is the only thread for which there are sufficient permits, is not the next to be assigned, so it doesn't get the permits.
Your modified code runs fine because the threads don't block when they try to get a semaphore; they just try again, going to the back of the line. Eventually thread 2 reaches the front of the line and is thus next to be assigned, and so gets its permits.

Thread pool not resizing

I want to create a cached thread pool, but it acts as a fixed one. Currently I have this code:
public class BackgroundProcesses {
public static void main(String[] args) throws InterruptedException, ExecutionException {
//ExecutorService threadPool2 = Executors.newCachedThreadPool();
ExecutorService threadPool = new ThreadPoolExecutor(2, 10, 180, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
for (int i = 0; i < 800; i++) {
Callable<String> task = new Task();
threadPool.submit(task);
}
}
}
class Task implements Callable<String> {
#Override
public String call() throws Exception {
Thread.sleep(100);
System.out.println(Thread.currentThread().getName() + " is ready");
return "";
}
}
If I run the code I get output:
pool-1-thread-1 is ready
pool-1-thread-2 is ready
pool-1-thread-1 is ready
pool-1-thread-2 is ready
...
Meaning only 2 threads are doing all the work and no new worker threads are added to the pool. Shouldn't the threadpool spawn more threads if tasks are waiting in queue (up to 10 in my case)?
I don't want to use Executors.newCachedThreadPool() because it has practically no upper limit on maximum threads and it has corePoolSize 0. I want to have some threads ready at all times for better responsiveness.
----- edit 1 -----
Thank you Aleksey for the answer. Setting capacity to queue is making it behave as expected, but now I have a new problem.
The amount of background tasks vary a lot. Most of the time 0 but can go up to 50 concurrent tasks for short periods. What would be an efficient way to handle this? Keep in mind most background tasks would be short-lived (< 1s) but a few long lived tasks (> 1min) as well.
If I set my threadpool up like this:
ExecutorService threadPool = new ThreadPoolExecutor(2, 10, 180, TimeUnit.SECONDS, new LinkedBlockingQueue<>(10));
I will most likely get RejectedExecutionException with peak usage. However if I set threadpool up like this:
ExecutorService threadPool = new ThreadPoolExecutor(2, 10, 180, TimeUnit.SECONDS, new LinkedBlockingQueue<>(200));
Then no new worker threads will ever be added because the queue won't max out.
The CPU has at least 4 cores so this would be wasteful in my opinion. And most of the time there isn't any background tasks at all (80% of up time), so keeping a fixed thread pool would also be wasteful in my opinion.

ThreadPoolExecutor Javadoc says:
When a new task is submitted in method execute(Runnable), and fewer
than corePoolSize threads are running, a new thread is created to
handle the request, even if other worker threads are idle. If there
are more than corePoolSize but less than maximumPoolSize threads
running, a new thread will be created only if the queue is full
Your LinkedBlockingQueue is never full, because it does not have an upper bound on the number of elements. Changing new LinkedBlockingQueue() to new LinkedBlockingQueue(10) would solve that.

Set thread number limitation [duplicate]

I want to launch a lot of tasks to run on a database of +-42Mio records. I want to run this in batches of 5000 records/time (results in 850 tasks).
I also want to limit the number of threads (to 16) java starts to do this for me and I am using the current code to accomplish this task:
ExecutorService executorService = Executors.newFixedThreadPool(16);
for (int j = 1; j < 900 + 1; j++) {
int start = (j - 1) * 5000;
int stop = (j) * 5000- 1;
FetcherRunner runner = new FetcherRunner(routes, start, stop);
executorService.submit(runner);
Thread t = new Thread(runner);
threadsList.add(t);
t.start();
}
Is this the correct way to do this? Particularly as I have the impression that java just fires away all tasks ...(FetcherRunner implements runnable)

The first part using ExecutorService looks good:
...
FetcherRunner runner = new FetcherRunner(routes, start, stop);
executorService.submit(runner);
The part with Thread should not be there, I am assuming you have it there just to show how you had it before?
Update:
Yes, you don't require the code after executorService.submit(runner), that is going to end up spawning a huge number of threads. If your objective is to wait for all submitted tasks to complete after the loop, then you can get a reference to Future when submitting tasks and wait on the Future, something like this:
ExecutorService executorService = Executors.newFixedThreadPool(16);
List<Future<Result>> futures = ..;
for (int j = 1; j < 900+ 1; j++) {
int start = (j - 1) * 5000;
int stop = (j) * 5000- 1;
FetcherRunner runner = new FetcherRunner(routes, start, stop);
futures.add(executorService.submit(runner));
}
for (Future<Result> future:futures){
future.get(); //Do something with the results..
}

Is this the correct way of working?
The first part is correct. But you shouldn't be creating and starting new Thread objects. When you submit the Runnable, the ExecutorService puts it on its queue, and then runs it when a worker thread becomes available.
.... I use the threadlist to detect when all my threads are finished so I can continue processing results.
Well if you do what you are currently doing, you are running each task twice. Worse still, the swarm of manually created threads will all try to run in parallel.
A simple way to make sure that all of the tasks have completed is to call awaitTermination(...) on the ExecutorService. (An orderly shutdown of the executor service will have the same effect ... if you don't intend to use it again.)
The other approach is to create a Future for each FetcherRunner's results, and attempt to get the result after all of the tasks have been submitted. That has the advantage that you can start processing early results before later ones have been produced. (However, if you don't need to ... or can't ... do that, using Futures won't achieve anything.)

You don't need to the part after the call to submit. The code you have that creates a Thread will result in 900 threads being created! Yowza. The ExecutorService has a pool of 16 threads and you can run 16 jobs at once. Any jobs submitted when all 16 threads are busy will be queued. From the docs:
Creates a thread pool that reuses a fixed number of threads operating
off a shared
unbounded queue. At any point, at most nThreads threads will be active processing tasks.
If additional tasks are submitted when all threads are active, they will wait in the
queue until a thread is available. If any thread terminates due to a failure during
execution prior to shutdown, a new one will take its place if needed to execute
subsequent tasks. The threads in the pool will exist until it is explicitly shutdown.
So there is no need for yet another thread. If you need to be notified after a task has finished you can have it call out. Other options are to cache all of the Future's returned from submit, and upon each task being finished you can check to see if all Future's are done. After all Future's are finished you can dispatch another function to run. But it will run ON one of the threads in the ExecutorService.

Changed from your code:
ExecutorService executorService = Executors.newFixedThreadPool(16);
for (int j = 1; j < 900 + 1; j++) {
int start = (j - 1) * 5000;
int stop = (j) * 5000 - 1;
FetcherRunner runner = new FetcherRunner(routes, start, stop);
executorService.submit(runner);
}

The best way would be to use countdownlatch as follows
ExecutorService executorService = Executors.newFixedThreadPool(16);
CountdownLatch latch = new CountdownLatch(900);
FetcherRunner runner = new FetcherRunner(routes, start, stop, latch);
latch.await();
in the FetcherRunner under finally block use latch.countDown(); code after await() will be executed only when all the tasks are completed.

Picking ThreadPool Threads Based on Some Attribute of Callable

I have a Callable task which I wish to submit to an Executor:
public static final class PersonalTask implements Callable<Object> {
private final String name;
private final int sleep;
public PersonalTask(String name, int sleep) {
this.name = name;
this.sleep = sleep;
}
#Override
public Object call() throws Exception {
System.out.format("My name is %s and I'm sleeping for %d seconds%n", name, sleep);
Thread.sleep(sleep * 1000);
return null;
}
}
Each task contains the name of the person who has requested the task to be executed, and some period of sleeping. This sleep duration is a proxy for the real use-case, which invokes some expensive operation.
To facilitate these tasks, I'm using a fixed thread pool with 5 threads:
private static final ExecutorService executor = Executors.newFixedThreadPool(5);
To illustrate my problem, I would like to submit the following to the thread pool:
public static void main(String[] args) throws Exception {
List<PersonalTask> tasks = Arrays.asList(new PersonalTask[] {
new PersonalTask("Bob", 10), new PersonalTask("Bob", 10),
new PersonalTask("Bob", 10), new PersonalTask("Bob", 10),
new PersonalTask("Bob", 10), new PersonalTask("Bob", 10),
new PersonalTask("Eric", 1), new PersonalTask("Janice", 2) });
executor.invokeAll(tasks);
}
The output from this is:
My name is Bob and I'm sleeping for 10 seconds
My name is Bob and I'm sleeping for 10 seconds
My name is Bob and I'm sleeping for 10 seconds
My name is Bob and I'm sleeping for 10 seconds
My name is Bob and I'm sleeping for 10 seconds
*** PAUSE FOR 10 SECONDS ***
My name is Bob and I'm sleeping for 10 seconds
My name is Eric and I'm sleeping for 1 seconds
My name is Janice and I'm sleeping for 2 seconds
This is because the Bob tasks to sleep for 10 seconds saturate the 5 threads available, and the remaining tasks -- specifically those belonging to Eric and Janice -- have to wait for those to finish.
This is unfair! The large number / slow jobs Bob has submitted are saturating the available threads and are starving Eric and Jane.
I would like to provide the ExecutorService with a mechanism to discriminate on the tasks it is asked to schedule, so I may come up with a fairer solution.
I would like to keep this very simple for now. All of Bob's tasks should be processed by the same thread in the pool. To keep this simple, I'd like to take PersonalTask.name.hashCode() % threadPoolSize and use that to pick which thread to use.
This would mean that Bob can only ever use one of the available 5 threads. This would leave the 4 remaining threads free to process other people's requests.
I realize this is not perfect, as other people with the same hash % size value would still be held up behind Bob. In fact, they'd now have to wait much, much longer as there are 6 * 10 second jobs ahead of them in that thread's queue.
What patterns can I use in Java to accomplish this?

Use your own implementation of BlockingQueue (like a PriorityBlockingQueue) and a manually constructed ThreadPoolExecutor that accepts as parameter your BlockingQueue. In your BlockingQueue implementation you could keep a Set of all your existing persons (in tasks) and iterate through it and return the next available Task of the person.

In your executor service, use a Map<Object,Queue> to map each User to their tasks, and relate this user to their Thread in the thread pool, maybe in another map Map<Object,Thread>.