We Keep Coding

ExecutorService should wait until batch of taksk is finished before starting again

The Problem:
I am parsing a large log file (around 625_000_000 lines) and saving it into the database.
public class LogScheduler {
static int fileNumber = 1;
public Importer(IRequestService service) {
this.service = service;
}
#Override
public void run() {
try {
service.saveAll(getRequestListFromFile("segment_directory/Log_segment_"+fileNumber+".txt"));
} catch (IOException e) {
e.printStackTrace();
}
}
}
The method that runs this thread is:
public void scheduledDataSave() throws InterruptedException {
int availableCores = Runtime.getRuntime().availableProcessors();
String directory = "segment_directory";
int filesInDirectory = Objects.requireNonNull(new File(directory).list()).length;
ExecutorService executorService = Executors.newFixedThreadPool(availableCores);
for (int i = 1; i <= filesInDirectory; i++) {
executorService.execute(new Importer(service));
}
executorService.shutdown();
}
Inserting the Thread.sleep(); method after the executorService.execute(new Importer(service)); sleeps after the execution of every thread, and not 8 threads like it should since they are in the Executorservice
And I have no idea why that happens since it should not behave like that.
From what I understand, the ExecutorService should run 8 threads in parallel, finish them, sleep, and start the pool again.
How to "sleep" after every 8 threads?

Sleeping the thread submitting tasks does not sleep the submitted tasks
Your question is not clear, but apparently centers around your expectation that adding a Thread.sleep after each call to executorService.execute would sleep all the threads of the executor service.
for ( int i = 1 ; i <= filesInDirectory ; i++ ) {
executorService.execute( new Importer( service ) ); // Executor service assigns this task to one of the background threads in its backing pool of threads.
Thread.sleep( Duration.ofMillis( 100 ).toMillis() ) ; // Sleeping this thread doing the looping. *Not* sleeping the background threads managed by the executor service.
}
Your expectation in incorrect.
That Thread.sleep is sleeping the thread doing the for loop.
The executor service has its own backing pool of threads. Those threads are not affected by a Thread.sleep is some other thread. Those background threads will only sleep if you call Thread.sleep within the code running on each of those threads.
So you are feeding the first task to the executor service. The executor service immediately dispatches that work to one of its backing threads. That task is immediately executed (if a thread is available immediately, and not otherwise occupied by previous tasks).
After assigning that task, your for loop sleeps for a hundred milliseconds, in this example code shown here. While the for loop is asleep, no further tasks are being assigned to the executor service. But while the for loop is asleep, the submitted task is executing on a background thread. That background thread is not sleeping.
Eventually, your for loop thread wakes, assigns a second task, and goes back to sleep. Meanwhile the background thread executes at full speed ahead.
So sleeping the thread submitting tasks does not sleep tasks already submitted.
Waiting for submitted tasks to complete
Your title asks:
ExecutorService should wait until batch of taksk is finished before starting again
After submitting your tasks, call shutdown and awaitTermination on your executor service. After those calls, your code blocks, waiting until all the submitted tasks are are completed/canceled/failed.
ExecutorService executorService = Executors.newVirtualThreadExecutor() ;
… submit tasks to that executor service …
executorService.shutdown() ;
executorSerivce.awaitTermination() ; // At this point, the flow-of-control blocks until the submitted tasks are done.
System.out.println( "INFO - Tasks on background threads are done. " + Instant.now() );
I would suggest using the ExecutorService#submit method rather than ExecutorService#execute method. The difference is that the first method returns a Future object. You can collect these Future objects as you submit tasks to the executor service. After the shutdown & awaitTermination, you can examine your collection of Future objects to check their completion status.
Project Loom
If Project Loom succeeds, such code will be a bit simpler and more clear. Experimental builds of Project Loom technology are available now, based on early-access Java 17. The Loom team seeks feedback now.
With Project Loom, ExecutorService becomes AutoCloseable. This means we can use try-with-resources syntax to automatically call a new close method on ExecutorService. This close method first blocks until all the tasks are completed/canceled/failed, then shuts down the executor service. No need to call shutdown nor awaitTermination.
By the way, Project Loom also bring virtual threads (fibers). This is likely to dramatically increase the performance of your code because it involves much blocking for storage i/o and database access.
try (
ExecutorService executorService = Executors.newVirtualThreadExecutor() ;
)
{
… submit tasks to that executor service …
}
// At this point, with Project Loom technology, the flow-of-control blocks until the submitted tasks are done.
// Also, the `ExecutorService` is automatically closed/shutdown by this point, via try-with-resources syntax.
System.out.println( "INFO - Tasks on background threads are done. " + Instant.now() );
With Project Loom, you can collect the returned Future objects in the same manner as discussed above to examine completion status.
You have other issues in your code. But you've not disclosed enough to address them all.

How to "sleep" after every 8 threads?
So if you are doing something like this, then it isn't doing what you think.
for (int i = 1; i <= filesInDirectory; i++) {
executorService.execute(new Importer(service));
Thread.sleep(...);
}
This causes the thread which is starting the background jobs to sleep and does not affect the running on each of the jobs. I believe what you are missing is to wait for the thread-pool to finish:
executorService.shutdown();
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
This waits for all of the jobs in the thread-pool to complete before continuing.
One more thing. I use executorService.submit(...) versus execute(...). Here's a description of their difference. For me, one additional difference is that any exceptions thrown by tasks run with execute(...) cause the running thread to terminate and possibly be restarted. With submit(...) it allows you to get that exception if needed and stops the threads from having to be respawned unnecessarily.
If you explain a bit more about what you are trying to accomplish, we should be able to help.

Timeout with CompletableFuture and CountDownLatch

I want to wrap a Runnable in CompletableFuture to be computed asynchronously, but with control over when does the computation begin and end. I've created a CompletableFuture with CountDownLatch to block the processing, but the following snippet throws an error:
CountDownLatch countDownLatch = new CountDownLatch(1);
CompletableFuture completableFuture = CompletableFuture.runAsync(() -> {
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Stop");
});
Thread.sleep(1000L);
System.out.println("Start");
completableFuture.get(1000L, TimeUnit.MILLISECONDS);
countDownLatch.countDown();
Start
Exception in thread "main" java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at Sandbox.main(Sandbox.java:23)
When I call get without timeout on the other hand, it freezes (only Start is printed).
I expect the runnable in CompletableFuture to run when countDownLatch.countDown(); is called.

You are waiting till the timeout expires without allowing the thread to proceed. The Future.get is blocking and that will never allow you to countDown the Latch before the timeout expires ever, hence your thread never completes. What you have to do here is, first, let the thread proceed by calling the countDown on the Latch and then wait with a timeout in the get call. Just inverting the two lines would solve the issue. Here's how it looks.
countDownLatch.countDown();
completableFuture.get(1000L, TimeUnit.MILLISECONDS);
In fact, if you remove the timeout from the get call (it blocks indefinitely), then this is a typical example of a Deadlock in a system. The worker thread waits until the main thread counts down the latch, while main thread waits for the worker thread to complete so that it can go ahead and countDown the latch. Fortunately, the time out passed to get enables Probabilistic deadlock avoidance. On the contrary, you can cancel the future at any time and avoid potential deadlocks as far as your tasks are responsive to the interruption.

Because of CompletableFuture#get is a blocking call. So, countDownLatch.countDown(); will not execute till the time CompletableFuture#get get the result. CompletableFuture will not complete and return the result as it will wait to countDownLatch to count down. So, basically you have created a dependency between 2 thread such that one will wait for another and vice-versa.

ExecutorService: awaitTermination VS while loop

I have to wait for completion of all the threads in an executorService.
Should I use
while(!executor.isTerminated()){...} or
executor.awaitTermination(...)?
What are the pros and cons of these ways?

With executor.isTerminated() your current thread will keep running.
with executor.awaitTermination() the current thread will be blocked.
So it depends on what you want to do in your current thread.
Do you want to do some tasks and periodically check for whether the executor is done, then use executor.isTerminated()?
Or Is the current thread just waiting for the executor to finish. If yes, executor.awaitTermination() makes much more sense.
Do note that an Executor will only be terminated if the shutdown() or shutdownNow() is called.

Recommended way from oracle documentation link:
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
if your threads are not completing with-in 120 seconds, you can change second if condition as :
while(!pool.awaitTermination(60, TimeUnit.SECONDS)) {
Thread.sleep(60000);
}
You can find other alternatives at # wait until all threads finish their work in java
One more key note on usage of isTerminated:
boolean isTerminated()
Returns true if all tasks have completed following shut down. Note that isTerminated is never true unless either shutdown or shutdownNow was called first.

How to shut down all threads in executor service immidiately?

I have used shutDown() and shutDownNow(), but both method does not stop all threads immediately. shutDownNow() is more preferable among these two but it waits for the running thread to complete it's task. In my scenario I have a huge task dealing with postgres database and I want to shut down that thread immediately without waiting for the completion of execution.
What is the way to shut down the all the threads immediately?

shutdownNow :
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
This method does not wait for actively executing tasks to terminate. Use awaitTermination to do that.
shutdown :
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
This method does not wait for previously submitted tasks to complete execution. Use awaitTermination to do that.
http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ExecutorService.html#shutdown%28%29
or you can see this :
You can use an ExecutorService instead which combines a thread pool with a queue of tasks.
ExecutorService service = Executors.newCachedThreadPool();
// or
ExecutorService service = Executors.newFixedThreadPool(THREADS);
// submit as many tasks as you want.
// tasks must honour interrupts to be stopped externally.
Future future = service.submit(new MyRunnable());
// to cancel an individual task
future.cancel(true);
// when finished shutdown
service.shutdown();

The single "clean" way to stop the threads is, if you have some loops inside, to stop the loop through some boolean variable like "stopThread", and you have to handle the variable.
Example:
public void run(){
for(int i=0; i<1000000 && (!stopThread) ; i++){
// do something
}
}

I doubt that it's possible in safe way.
The safe way to shutdown the executor service as per oracle documentation
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
If you really want to follow a crude way, I will suggest a solution which may not be 100% accurate and I don't recommend personally. I would like to use ExecutorService methods instead of this crude solution.
Create your own thread and thread pool as per this article
Add one more boolean in Worker thread - boolean runNow = true;
run method of Worker thread will be like this : while ( runNow) { // your logic }
When you want to shut down all the threads, add one more method in ThreadPoolManager. Iterate through myQueue and interrupt all Runnables. Catch that interrupted exception and make the boolean runNow as false.

Depending on how you call the database, you could try to explicitly cancel the query. Se related question.

How to preserve thread-safety of a CopyOnWriteArrayList with a correct run() method implementation?

I am using an ExecutorService with a fixedThreadPool. I create threads by implementing the Runnable interface. In the run() method, I am calling a time consuming function (let's say Thread.sleep() for now) and finally add an element to a thread safe CopyOnWriteArrayList. I am submitting threads in a for-loop. At the end of the for-loop I shutdown the ExecutorService and await termination.
However, the number of elements in the list does not turn out to be the same as the number of loop-iterations. Where is my mistake in concurrency thinking?
Here is a code example for this problem:
public class TestProgram {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newFixedThreadPool(4);
CopyOnWriteArrayList<String> stringList = new CopyOnWriteArrayList<String>();
for (int i = 0; i < 1000; i++) {
executor.submit(new myThread(stringList));
}
executor.shutdown();
executor.awaitTermination(10, TimeUnit.SECONDS);
System.out.println(Integer.toString(stringList.size()));
}
}
class myThread implements Runnable {
CopyOnWriteArrayList<String> stringList;
public myThread(CopyOnWriteArrayList<String> stringList) {
this.stringList = stringList;
}
public void run() {
String string = new String("test");
try {
Thread.sleep(100);
} catch (InterruptedException ex) {
Logger.getLogger(myThread.class.getName()).log(Level.SEVERE, null, ex);
}
stringList.add(string);
}
}

You're just not giving it enough time to complete. If you want to execute this serially you would need 1000 * 100 milliseconds, which is 100 seconds. You are running it with four threads, so divide that by 4 and you get 25 seconds. You only wait 10 seconds to complete. Increase that to 26 seconds just to be safe and you should see your result.
The ExecutorService#awaitTermination will wait the N number of seconds to complete, if it doesn't complete it will simply return out and continue the execution. To prove my point check the return value
System.out.println(executor.awaitTermination(10, TimeUnit.SECONDS));
The better solution here, is to use a new CountDownLatch(1000), have each Runnable task countDown after it adds to the list and finally have the main thread await completion.

The documentation of ExecutorService.shutdown says the following:
Initiates an orderly shutdown in which previously submitted tasks are
executed, but no new tasks will be accepted. Invocation has no
additional effect if already shut down.
This method does not wait for previously submitted tasks to complete
execution. Use awaitTermination to do that.
This means that you indicate your ExecutorService that it must stop it's services. Thus, this method sets a flag which indicates that the ExecutorService will end soon, which allows the main thread to returns from this method without problems.
ExecutorService.awaitTermination, moreover, reads as follows:
Blocks until all tasks have completed execution after a shutdown
request, or the timeout occurs, or the current thread is interrupted,
whichever happens first.
This means that the main thread will be blocked inside the method, where it will only be returned when the tasks sent to ExecutorService finish. In both cases, the developer intends to end the ExecutorService function. But with awaitTermination, the developer is saying ExecutorService should be terminated regardless of their tasks have been carried out or not.
As each task takes at least 100 milliseconds to complete, no one will possibly be completed because ExecutorService has a tolerance of only 10 milliseconds for completion of all it's tasks.
There is no point in calling shutdown and awaitTermination at the same time (or preceded by another). According to your code, shutdown has no effect. If you want your tasks to become completed, and that ExecutorService is terminated without caring if your main thread will continue, just use shutdown. If you do not want your main thread to be blocked, and simply want to end with ExecutorService, use shutdownNow.
If, on the other hand, you want your main thread to wait for the execution of your tasks without a certain time to be specified, you may have to use Java Synchronizers. Some of Java Synchronizers are as follow:
Latches
FutureTasks
Semaphores
Barriers
For your case, I believe you can use CountDownLatch. Make a new CountDownLatch object have the number of tasks running on your ExecutorService. As they are completed, the count is decreased, and so, your main thread can expect completion and continue after all.
I hope you have understood and you can do what you want. If you have further questions, the documentation of CountDownLatch demonstrates perfectly how you can synchronize tasks. Good luck.

In that code snippet one issue will be there.
executor.awaitTermination(10, TimeUnit.SECONDS);
Here it will wait for 10 seconds after that it will break.
You will try following snippet.
while (true) {
if(executor.awaitTermination(10, TimeUnit.SECONDS))
break;
}