Have been facing trouble with the right usage of executor service. I am a new bee to executor and tried lot over web to resolve this. But couldn't justify it more.
i will try to put down my requirement and implementation done.
I have an infinite loop to spawn the threads and the maximum limit any time should be 50.
public void createWorkers(){
ExecutorService executor = Executors.newFixedThreadPool(50);
while(true) {
executor.execute(innerRunnableThread); // a runnable thread
}
}
QUERY : since it's an infinite loop, how to shut down the executor service (executor.shutdown()) ?
"innerRunnableThread" mentioned in the first point has the below code :
public class InnerRunnableThread implements Runnable {
#Override
public void run() {
ExecutorService innerExecutor = Executors.newFixedThreadPool(10);
for (int i=0; i<10;i++){
executor.execute(detailRunnableThread); // a runnable thread
}
innerExecutor.shutdown();
try {
innerExecutor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
loggr.error("exception thrown while spawning threads");
}
}
}
QUERY :
1. Parent thread ( in the 1st point), limit given as 50 threads. But with this implementation, parent threads not spawned more than 5. What is the reason for it ?
COMBINED QUERY :
With this approach, excluding shut down of the executor service in first point, there were thousands of threads lying un-terminated at the server. Finally leading to shutdown of this application and couldn't start any more ?
Kind help on this particular approach will be very helpful along with the better alternatives to achieve this logic.
Thanks alot.
Related
Although this topic has been discussed broadly in other posts I want to present my use case and clarify .So apologies if I am wasting anyone's time. I have the following runnable implementation. Basically infinitely running thread unless java.lang.Error gets thrown by the business logic.
public void run (){
while(true){
try{
//business logic
}catch(Exception ex){
}
}
}
I have about 30 of the above threads started from ExecutorService.
private final ExecutorService normalPriorityExecutorService = Executors.newFixedThreadPool(30);
for(int i=0;i<30;i++) {
normalPriorityExecutorService.submit(//Above Runnable);
}
I want to check and kill the JVM process if the thread count becomes zero on this Executor Service.
if (normalPriorityExecutorService instanceof ThreadPoolExecutor && ((ThreadPoolExecutor) normalPriorityExecutorService).getActiveCount() ==0) {
log.error("No Normal Priority response listeners available. Shutting down App!");
System.exit(1);
}
From my reading since these runnable threads are infinitely running under normal circumstances I will have 30 of them active unless they get killed by runtime Errors.
Question is using getActiveCount() the right approach for my use case ? By the way, when I tried using getPoolSize() instead of getActiveCount(), I did not get the right behavior while testing (I forcefully threw an error to kill a specific thread) and the poolSize still remained thirty.
Since you never use the thread pool as a pool, using a thread pool is overkill. Just create a thread group and start your threads.
private final ThreadGroup normalPriorityThreadGroup = new ThreadGroup("NormalPriority");
for (int i = 0; i < 30; i++) {
new Thread(this.normalPriorityThreadGroup, runnable, "NormalPriority-" + 1).start();
}
if (this.normalPriorityThreadGroup.activeCount() == 0) {
log.error("No Normal Priority response listeners available. Shutting down App!");
System.exit(1);
}
I have a piece of java code which does the following -
Opens a file with data in format {A,B,C} and each file has approx. 5000000 lines.
For each line in file, call a service that gives a column D and append it to {A,B,C} as {A,B,C,D}.
Write this entry into a chunkedwriter that eventually groups together 10000 lines to write back chunk to a remote location
Right now the code is taking 32 hours to execute. This process would again get repeated across another file which hypothetically takes another 32 hours but we need these processes to run daily.
Step 2 is further complicated by the fact that sometimes the service does not have D but is designed to fetch D from its super data store so it throws a transient exception asking you to wait. We have retries to handle this so an entry could technically be retried 5 times with a max delay of 60000 millis. So we could be looking at 5000000 * 5 in worst case.
The combination of {A,B,C} are unique and thus result D can't be cached and reused and a fresh request has to be made to get D every time.
I've tried adding threads like this:
temporaryFile = File.createTempFile(key, ".tmp");
Files.copy(stream, temporaryFile.toPath(),
StandardCopyOption.REPLACE_EXISTING);
reader = new BufferedReader(new InputStreamReader(new
FileInputStream(temporaryFile), StandardCharsets.UTF_8));
String entry;
while ((entry = reader.readLine()) != null) {
final String finalEntry = entry;
service.execute(() -> {
try {
processEntry(finalEntry);
} catch (Exception e) {
log.error("something");
});
count++;
}
Here processEntry method abstracts the implementation details explained above and threads are defined as
ExecutorService service = Executors.newFixedThreadPool(10);
The problem I'm having is the first set of threads spin up but the process doesn't wait until all threads finish their work and all 5000000 lines are complete. So the task that used to wait for completion for 32 hours now ends in <1min which messes up our system's state. Are there any alternative ways to do this? How can I make process wait on all threads completing?
Think about using ExecutorCompletionService if you want to take tasks as they complete you need an ExecutorCompletionService. This acts as a BlockingQueue that will allow you to poll for tasks as and when they finish.
Another solution is to wait the executor termination then you shut it down using:
ExecutorService service = Executors.newFixedThreadPool(10);
service .shutdown();
while (!service .isTerminated()) {}
One alternative is to use a latch to wait for all the tasks to complete before you shutdown the executor on the main thread.
Initialize a CountdownLatch with 1.
After you exit the loop that submits the tasks, you call latch.await();
In the task you start you have to have a callback on the starting class to let it know when a task has finished.
Note that in the starting class the callback function has to be synchronized.
In the starting class you use this callback to take the count of completed tasks.
Also inside the callback, when all tasks have completed, you call latch.countdown() for the main thread to continue, lets say, shutting down the executor and exiting.
This shows the main concept, it can be implemented with more detail and more control on the completed tasks if necessary.
It would be something like this:
public class StartingClass {
CountDownLatch latch = new CountDownLatch(1);
ExecutorService service = Executors.newFixedThreadPool(10);
BufferedReader reader;
Path stream;
int count = 0;
int completed = 0;
public void runTheProcess() {
File temporaryFile = File.createTempFile(key, ".tmp");
Files.copy(stream, temporaryFile.toPath(),
StandardCopyOption.REPLACE_EXISTING);
reader = new BufferedReader(new InputStreamReader(new
FileInputStream(temporaryFile), StandardCharsets.UTF_8));
String entry;
while ((entry = reader.readLine()) != null) {
final String finalEntry = entry;
service.execute(new Task(this,finalEntry));
count++;
}
latch.await();
service.shutdown();
}
public synchronized void processEntry(String entry) {
}
public synchronized void taskCompleted() {
completed++;
if(completed == count) {
latch.countDown();;
}
}
//This can be put in a different file.
public static class Task implements Runnable {
StartingClass startingClass;
String finalEntry;
public Task(StartingClass startingClass, String finalEntry) {
this.startingClass = startingClass;
this.finalEntry = finalEntry;
}
#Override
public void run() {
try {
startingClass.processEntry(finalEntry);
startingClass.taskCompleted();
} catch (Exception e) {
//log.error("something");
};
}
}
}
Note that you need to close the file. Also the sutting down of the executor could be written to wait a few seconds before forcing a shutdown.
The problem I'm having is the first set of threads spin up but the process doesn't wait until all threads finish their work and all 5000000 lines are complete.
When you are running jobs using an ExecutorService they are added into the service and are run in the background. To wait for them to complete you need to wait for the service to terminate:
ExecutorService service = Executors.newFixedThreadPool(10);
// submit jobs to the service here
// after the last job has been submitted, we immediately shutdown the service
service.shutdown();
// then we can wait for it to terminate as the jobs run in the background
service.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
Also, if there is a crap-ton of lines in these files, I would recommend that you use a bounded queue for the jobs so that you don't blow out memory effectively caching all of the lines in the file. This only works if the files stay around and don't go away.
// this is the same as a newFixedThreadPool(10) but with a queue of 100
ExecutorService service = new ThreadPoolExecutor(10, 10,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(100));
// set a rejected execution handler so we block the caller once the queue is full
threadPool.setRejectedExecutionHandler(new RejectedExecutionHandler() {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return;
}
}
});
Write this entry into a chunkedwriter that eventually groups together 10000 lines to write back chunk to a remote location
As each A,B,C job finishes, if it needs to be processed in a second step then I would also recommend looking into a ExecutorCompletionService which allows you to chain various different thread pools together so as lines finish they will immediately start working on the 2nd phase of the processing.
If instead this chunkedWriter is just a single thread then I'd recommend sharing a BlockingQueue<Result> and having the executor threads put to the queue once the lines are done and the chunkedWriter taking from the queue and doing the chunking and writing of the results. In this situation, indicating to the writer thread that it is done needs to be handled carefully -- maybe with some sort of END_RESULT constant put to the queue by the main thread waiting for the service to terminate.
I'm new to java concurrency an would like to ask the following basic question. I'm creating a ThreadPoolExecutor for imporving performance as follows:
int n = Runtime.getRuntime().availableProcessors()
ExecutorService executor = Executors.newFixedThreadPool(n);
for( int i = 0; i < n; i++)
executor.execute(new Work());
After all thread in the thread pool have finished their tasks I need to shutdown the pool properly. I would tried this:
while(true){
if(executor.isTerminated()){
executor.shutdownNow();
break;
}
}
But I'm not sure about that because I think we waste a lot of processors resources to queriyng the executor for termination.
What is the right solution for that?
UPD: Runnable task:
public class Work implements Runnable{
private String sql;
public Work() {
//init sql
}
#Override
public void run() {
JdbcTemplate template = new JdbcTemplate(dataSource);
#SuppressWarnings("unchecked")
List<Integer> ints = template.queryForList(sql, Integer.class);
//Storing the list into a global cache
}
}
There seems to be something mystical around shutting down an ExecutorService.
From the documentation of shutdown():
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
So all you have to do is to invoke shutdown() after you have submitted all your tasks, the exact time doesn’t matter. They don’t have to be completed at that time. The ThreadPoolExecutor will finish all tasks and then clean up all resources.
And it will do so regardless of whether you wait for it or not. So you don’t need to wait, just invoke shutdown() when you are confident that you will not submit new tasks, the rest will happen as soon as possible.
It says:
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt(), so any task that fails to respond to
interrupts may never terminate.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()
So use awaitTermination instead. And for threads that take time, use a boolean variable as volatile and check it if it is set outside.If set then exit etc. something like that
try {
executor = Executors.newSingleThreadExecutor();
future = executor.submit(task);
executor.shutdown();
executor.awaitTermination(5, TimeUnit.SECONDS);
}
finally {
if (Objects.nonNull(executor) && !executor.isTerminated()) {
LOGGER.error("cancelling all non-finished tasks");
}
if (Objects.nonNull(executor)) {
executor.shutdownNow();
LOGGER.info("shutdown finished");
}
}
This way you shutdown executor and waiting for 5 seconds to complete all tasks and then finally calling executor.shutdownNow() to completely kill the executor.
This is the best way to shutdown executor.
I am attempting to understand how to handle many instances of the ExecutorService executing Runnable commands. With regards to the code provided, how many shutdowns are required if I execute a hundred Runnables with the fixed thread pool set to one? I think the code should execute a hundred futures sequentially in the for loop execution order with a single thread (never spawns more than a single thread), and requires a single ExecutorService shutdown. Is this correct? Also, it's ok to call shutdown right after the for loop completes because all hundred of the futures are in queue so that the executorService shutdown will occur automatically after all hundred futures complete. Just looking for some clarification, thanks.
public static void main(String[] args)
{
private static ExecutorService executorService = Executors.newFixedThreadPool(1);
for (int i = 0; i < 100; i++)
{
executorService.execute(new Runnable() {
#Override
public void run()
{
// do stuff
}
});
}
executorService.shutdown();
}
Looks like you've got the right idea. It doesn't matter how many Runnables you've handed over to the ExecutorService to run or how big a thread pool you've allocated, you only need to call shutdown() once. That will allow all tasks to complete but will not allow you to add any new ones. You may want to call
try {
executorService.awaitTermination(5, TimeUnit.MINUTES);
} catch (InterruptedException e) {
// do stuff
}
to block while all tasks are completed depending on your usage scenario.
If you want to shutdown and attempt to kill all running tasks, instead call the shutdownNow() method. Note that there is no guarantee that it will be able to interrupt running tasks.
There's something odd about the implementation of the BoundedExecutor in the book Java Concurrency in Practice.
It's supposed to throttle task submission to the Executor by blocking the submitting thread when there are enough threads either queued or running in the Executor.
This is the implementation (after adding the missing rethrow in the catch clause):
public class BoundedExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command) throws InterruptedException, RejectedExecutionException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
#Override public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
throw e;
}
}
When I instantiate the BoundedExecutor with an Executors.newCachedThreadPool() and a bound of 4, I would expect the number of threads instantiated by the cached thread pool to never exceed 4. In practice, however, it does. I've gotten this little test program to create as much as 11 threads:
public static void main(String[] args) throws Exception {
class CountingThreadFactory implements ThreadFactory {
int count;
#Override public Thread newThread(Runnable r) {
++count;
return new Thread(r);
}
}
List<Integer> counts = new ArrayList<Integer>();
for (int n = 0; n < 100; ++n) {
CountingThreadFactory countingThreadFactory = new CountingThreadFactory();
ExecutorService exec = Executors.newCachedThreadPool(countingThreadFactory);
try {
BoundedExecutor be = new BoundedExecutor(exec, 4);
for (int i = 0; i < 20000; ++i) {
be.submitTask(new Runnable() {
#Override public void run() {}
});
}
} finally {
exec.shutdown();
}
counts.add(countingThreadFactory.count);
}
System.out.println(Collections.max(counts));
}
I think there's a tiny little time frame between the release of the semaphore and the task ending, where another thread can aquire a permit and submit a task while the releasing thread hasn't finished yet. In other words, it has a race condition.
Can someone confirm this?
BoundedExecutor was indeed intended as an illustration of how to throttle task submission, not as a way to place a bound on thread pool size. There are more direct ways to achieve the latter, as at least one comment pointed out.
But the other answers don't mention the text in the book that says to use an unbounded queue and to
set the bound on the semaphore to be equal to the pool size plus the
number of queued tasks you want to allow, since the semaphore is
bounding the number of tasks both currently executing and awaiting
execution. [JCiP, end of section 8.3.3]
By mentioning unbounded queues and pool size, we were implying (apparently not very clearly) the use of a thread pool of bounded size.
What has always bothered me about BoundedExecutor, however, is that it doesn't implement the ExecutorService interface. A modern way to achieve similar functionality and still implement the standard interfaces would be to use Guava's listeningDecorator method and ForwardingListeningExecutorService class.
You are correct in your analysis of the race condition. There is no synchronization guarantees between the ExecutorService & the Semaphore.
However, I do not know if throttling the number of threads is what the BoundedExecutor is used for. I think it is more for throttling the number of tasks submitted to the service. Imagine if you have 5 million tasks that need to submit, and if you submit more then 10,000 of them you run out of memory.
Well you only will ever have 4 threads running at any given time, why would you want to try and queue up all 5 millions tasks? You can use a construct similar to this to throttle the number of tasks queued up at any given time. What you should get out of this is that at any given time there are only 4 tasks running.
Obviously the resolution to this is to use a Executors.newFixedThreadPool(4).
I see as much as 9 threads created at once. I suspect there is a race condition which causes there to be more thread than required.
This could be because there is before and after running the task work to be done. This means that even though there is only 4 thread inside your block of code, there is a number of thread stopping a previous task or getting ready to start a new task.
i.e. the thread does a release() while it is still running. Even though its the last thing you do its not the last thing it does before acquiring a new task.