We Keep Coding

shutting down ExecutorService in a spring boot Rest API

Am building a spring boot rest api application deployed on weblogic 12c.
One of my requirement is to run some long running tasks on every incoming request.
An incoming rest request could result into multiple asynchronous task executions.
Since I dont care for the response and nor any exceptions that will result from these tasks I chose to use the ExecutorService and not Callable or CompletableFuture.
ExecutorService executorService =
Executors.newFixedThreadPool(2, new CustomizableThreadFactory("-abc-"));
Then for the incoming request that I receive in controller run two for loops and assign those tasks to the ExecutorService:
for (final String orderId : orderIds) {
for (final String itemId : itemIds) {
exec.execute(new Runnable() {
public void run() {
try {
//call database operation
}catch(Throwable t) {
logger.error("EXCEPTION with {} , {}" ,orderId,itemId
)
}
});
}//for
}//for
My question is regarding shutting down of the ExecutorService.
I am aware about graceful shutdown ( shutdown ) a hybrid shutdown ( awaitTermination ) or an abrupt shutdown ( shutdownNow )
what would be the preferred approach between the three for a rest api application ?
also is there any limit on how many thread pools can get created viz a viz as the number of ExecutorService thread pools getting created will be driven by the number of incoming requests

We currently have similar requirements, this is a difficult problem to solve as you want to use the right hammer if you will. There are very heavy weight solutions to orchestrating long running processes, for example SpringBatch.
Firstly though don't bother stop and starting the ExecutorService. The whole point of that class is to take the burden of Thread management off your hands, so you don't need to create and stop Threads yourself. So you don't need to manage the manager.
But be careful with your approach. Without using queues or another load balancing technique to smartly balance the long running processes across instances in your app. Or managing what happens when a Thread dies, you may get into a world of trouble. In general I would say nowadays it doesn't make much sense to interact directly with Threads or ThreadPools, and to use higher level solutions for this type of problem.

awaitTermination is usually a bit safer, while shutdownNow is more forceful. It's usually a good idea to use awaitTermination in a functional method, or even a runnable, if you would like the executor to shut down as soon as possible, but only after it has completed doing everything that it was created to do. In other words, when there are no active tasks that the executor is executing.
Ex.)
ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime.availableProcessors);
Observable.of(items).schedule(Schedulers.from(executor)).flatMap(item -> {
... // this block represents a task that the executor will execute in a worker thread
}).onSubscribe(onNext ->
logItem(onNext), throwable ->
throwable.printStackTrace(), /* onComplete */ () ->
executor.awaitTermination(60, TimeUnit.Seconds)
);
... // you need to shutdown asap because these other methods below are also doing some computation/io-intensive stuff
Now, when this method is finished, it will call awaitTermination, which will either close the pool immediately if it is not executing any tasks, or wait up to 60 seconds if tasks are still being executed.
Threads, or workers, will cease to be active for 60 seconds of inactivity in most cases, since that is usually the default.
On the other hand, if you want tasks to stop executing as soon as (to give some examples) an exception is thrown, there was a breach in security, or another module/service has failed, you might want to use shutdownNow() to stop all tasks immediately without the option of waiting.
My advice for choosing between the two would be to use shutdownNow in you catch block if you do not want tasks to continue to be executed if there is an exception - i.e., there is no longer a reason to return the list of items to the client given that one of the items did not get added to the list.
Otherwise, I'd recommend using awaitTermination after your try-catch, set to one minute, to safely shut down the thread pool as soon as it has executed all the tasks you have given it. But only do that if you know that the executor will not responsible for executing any more tasks down the line.
The simple shutdown, if that is an option for you, is also a good method. shutdown will reject all incoming tasks but wait until current tasks are finished executing, according to the Oracle docs.
If your not sure when you need to close the executor, it might be a good idea to use an #PreDestroy method so that the executor will just before the destroy method has been called on your bean:
#PreDestroy
private void cleanup(){
executor.shutdown();
}

cancelling current task and reuse the same thread while creating another task in java

Please take time to read below. Your help will be highly appreciated
I have a scenario where I need to monitor some realtime activity. Say for example a method is getting called in realtime within milli seconds. I have to monitor as if when the method was first called and when the method was called last.
So, after that method is hit last I need to wait for sometime say 10 seconds and see if it doesn't called again within that time. If its not called then I need to run some code.
I want to use ExecuterService and use newCachedThreadPool(). But I am confused on how to implement this.
If you are not sure what I am talking about, take some example of say when there is some network breakdown and u where receiving heartbeats and suddenly u stopped receiving it then you show some error message on screen for e.g. Connection not available. In my case its some third party application which is sending some message and its just one way communication. Like, my application sent some request and other application keep on sending responses. So, I need to monitor that response and somehow need to know when I received the last response.
My approach - I thought of executing a task each time that method is called which will wait for 10 seconds and within 10 seconds, if that method got called again, it will somehow cancel the current task and will create another task (or reuse if possible) which will again run for 10 seconds. This will keep on happening until the last message received (when the method got called last) and after that once 10 sec delay is over, the task will be executed and some code will run to show error message on the UI.
I have used Timer earlier, which solved this problem but created a performance issue as new Timer which a new TimerTask is instantiated every time a new message is received hence creating a hell lot of objects which garbage collector could not reclaim that fast, thus resulting in outOfMemorry Error and making Server non responsive. Obviously it was a bad code that's why I am here for help.
Please help me in giving some approach to solve this problem.

This is quite easy if you approach it with the most basic of tools--sometimes with simple problems the enhanced tools like thread pools just distract from a trivial solution.
Let's say you have a simple thread (Runnable). That thread checks a time stamp to see if that time stamp is 10 seconds old. If it is you notify your listener, if not you delay a few millis and check again.
Now all your method has to do is update the (volatile) time stamp to "now" every time it runs. There may be some other business rules to implement here but it should be a good foundation.
The only issue now is how you notify your listeners. If this happens rarely you can probably call them on the same thread you are checking the time with--but I think you ruled that out. If it happens more often but you don't want/need it to "nest" your notifications, you can have a second thread hanging around with no purpose except to notify the client when triggered by your watcher thread.
If you need to "nest" notifications (notify the listener before the original notification has returned) then you need a thread pool for your notifications to go out on.
Finally I suppose if you want to catch EVERY time your timer isn't called for 10 seconds but you don't want to nest, your timing thread could push "events" onto a threadsafe queue and your "Notification" thread could pull them off and send the events one at a time.
That should cover all the possibilities.

You may use ScheduledExecutorService.
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ScheduledExecutorService.html

java.util.concurrent.Executors.newCachedThreadPool :
Creates a thread pool that creates new threads as needed, but will
reuse previously constructed threads when they are available. These
pools will typically improve the performance of programs that execute
many short-lived asynchronous tasks. Calls to execute will reuse
previously constructed threads if available. If no existing thread is
available, a new thread will be created and added to the pool. Threads
that have not been used for sixty seconds are terminated and removed
from the cache. Thus, a pool that remains idle for long enough will
not consume any resources. Note that pools with similar properties but
different details (for example, timeout parameters) may be created
using ThreadPoolExecutor constructors
As you have many short-lived tasks, cached thread pool is best option.
Short example:
public class WorkerThread implements Runnable {
private String command;
public WorkerThread(String s){
this.command=s;
}
#Override
public void run() {
System.out.println(Thread.currentThread().getName()+" Start. Command = "+command);
processCommand();
System.out.println(Thread.currentThread().getName()+" End.");
}
private void processCommand() {
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
#Override
public String toString(){
return this.command;
}
}
To run WorkerThread use :
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newCachedThreadPool();
Runnable worker = new WorkerThread("threadName");
executor.execute(worker);
To stop WorkerThread use:
executor.shutdown();

How can I ensure an ExecutorService pool has completed, without shutting it down?

Currently, I'm making sure my tasks have finished before moving on like so:
ExecutorService pool = Executors.newFixedThreadPool(5);
public Set<Future> EnqueueWork(StreamWrapper stream) {
Set<Future> futureObjs = new HashSet<>();
util.setData(stream);
Callable callable = util;
Future future = pool.submit(callable);
futureObjs.add(future);
pool.shutdown();
try {
pool.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
Node.sendTCP(Node.getNodeByHostname(StorageTopology.getNextPeer()), Coordinator.prepareForTransport(stream));
return futureObjs;
}
However, because of some other threading on my socket, it's possible that multiple calls are made to EnqueueWork - I'd like to make sure the calls to .submit have completed in the current thread, without shutting down the pool for subsequent threads coming in.
Is this possible?

You can check by invoking isDone() method on all the Future objects in futureObjs. You need to make sure isDone is called in a loop. calling get() method on Future object is another option, since get() is a blocking call, it will return only after task is completed and result is ready. But do you really want to keep the pool open after all the tasks are done?

I agree with one of the comments, it seems odd that your executor can be used by different threads. Usually and executor is private to an instance of some class, but anyhow.
What you can do, from the docs, is to check:
getActiveCount() - Returns the approximate number of threads that are >actively executing tasks.
NOTE: This is a blocking method, it will take out a lock on the workers of your threadpool and block until it has counted everything
And also check:
getQueue() - Returns the task queue used by this executor. Access to the
task queue is intended primarily for debugging and monitoring.
This queue may be in active use. Retrieving the task queue
does not prevent queued tasks from executing.
If your queue is empty and the activeCount is 0, all your tasks should have finished. I say should because getActiveCount says "approximate". Looking at the impl, this is most likely because the worker internally has a flag indicating that it is locked (in use). There is in theory a slight race between executing and the worker being done and marking itself so.
A better approach would in fact be to track the features. You would have to check the Queue and that all futures are done.
However I think what you really need is to reverse your logic. Instead of the current thread trying to work out if another thread has submitted work in the meantime, you should have the other thread call isShutdown() and simply not submit a new task in that case.

You are approaching this issue from the wrong direction. If you need to know whether or not your tasks are finished, that means you have a dependency of A->B. The executor is the wrong place to ensure that dependency, as much as you don't ask the engine of your car "are we there yet?".
Java offers several features to ensure that a certain state has been reached before starting a new execution path. One of them is the invokeAll method of the ExecutorService, that returns only when all tasks that have been submitted are completed.
pool.invokeAll(listOfAllMyCallables);
// if you reach this point all callables are completed

You have already added Future to the set. Just add below code block to get the status of each Future task by calling get() with time out period.
In my example, time out is 60 seconds. You can change it as per your requirement.
Sample code:
try{
for(Future future : futureObjs){
System.out.println("future.status = " + future.get(60000, TimeUnit.MILLISECONDS));
}
}catch(Exception err){
err.printStackTrace();
}
Other useful posts:
How to forcefully shutdown java ExecutorService
How to wait for completion of multiple tasks in Java?

Java Thread Queue

We have an application which processes items and on each iteration, starts a thread to do an update on an other database - it is not hugely important what happens on that other thread, it is a very straightforward update.
Our original intention was (by using a thread) to make sure the main processing is not held up by initalizing a connection to this other db and running the update.
Yesterday, we had an issue where (for a yet unknown reason) the database slowed down and the number of parallel threads went to the sky, resulting in 1000+ connections in this DB. So we realized we need more control over the threads.
I need a lib or tool for our software which can:
1) Put threads / jobs / tasks (anything - we can rewrite the code if required, we have Thread objects at the mintue) into a queue like system
2) we can define how many threads are running at most at the same time
3) After the thread finished, the thread is removed from the queue so GC can remove all the entities involved.
I was doing a fair bit of reading and i found ExecutorService (Executors.newFixedThreadPool(5);) but may problem is that it fails with 3) because according to the javadocs:
The threads in the pool will exist until it is explicitly shutdown.
Which, i believe, means that if the app keeps adding threads, the threads will exists until the application is restarted (or if i shutdown and reinstantiate the ExecutorService, but that seems to be a hack to me).
Am i right in thinking Executors.newFixedThreadPool(5) is failing at my 3) requirement?
Am i actually getting the problem from the good end? Do i need Threads or something different?

The sentence you are afraid of:
The threads in the pool will exist until it is explicitly shutdown
describes only calls to Executors.newFixedThreadPool(). To keep more control on the thread pool behavior, use ThreadPoolExecutor constructor expicitly, e.g
new ThreadPoolExecutor(1, //minimal Pool Size,
10, // maximal Pool Size,
30, TimeUnit.SECONDS // idle thread dies in 30 seconds
new ArrayBlockingQueue<Runnable>())

Here you need to understand the difference between Thread and the Runnable/Callable Task. so the meaning of The threads in the pool will exist until it is explicitly shutdown. is that at any point of time there will be 5 threads in the thread pool if you use Executors.newFixedThreadPool(5); . And the work that you want these threads to do would be submitted as Tasks (Runnable/Callable). So essentially at any point of time at max there will be 5 threads executing via this thread pool which in your case would be 5 connections.

The threads will stay there (waiting for other tasks to run), but it won't hold on to all the data that you put there. When the thread in the threadpool has executed the task, it will take a next task and won't reference the existing task anymore.
So your fears are baseless, unless you explicitly keep references to the tasks.

Use the ScheduledExecutorService with a fixed pool of threads for however many connections you need.
Have a BlockingQueue that you put requests in, the worker threads wait on the queue and process the requests as they appear.

The ExecutorService is the way to go. It provides you an interface (Future) to the state of the underlying thread, the ability to detect exceptions and return a value from the completed thread.
Here is a simple example of how to use ExecutorSerivce and the Future interface.
public class Updater implements Future< boolean > {
public Updater() { }
public boolean call() throws Exception {
System.out.println( "Hello, World!" );
return true;
}
}
public class Main {
public Main() { }
public static void main( String[] args ) {
ExecutorService pool = Executors.newFixedThreadPool( 1 );
boolean again = true;
do {
if ( again ) {
Future< ? > update = pool.submit( new Updater() );
}
/* Do other work while waiting for update to finish */
if( update.isDone() ) { //may be because of completion or an exception
try {
again = update.get(); // This would block if the Updater was still running
} catch( ExecutionException ee ) { // This is thrown by get() if an exception occurred in Updater.call()
again = false;
ee.printStackTrace();
}
}
} while ( true );
}
}
The example above will start an update to your database if the last update succeeded without an exception. This way you are controlling how many threads are trying to connect, and catching any errors that are causing the update to fail.

Can a Java Thread be alive more than once?

Ok.... Let me try to explain this the best I can....
Also: this is for a mod within minecraft.
Okay, so I created a thread object
public static Thread KillThread = new Thread();
Then in the constructor of my main class which is called when the game(Mine craft starts) I have
KillThread = new Thread(new KillAuraThread());
KillAuraThread is the name of the class that is the thread..
So I created a thread now. Is where it's pissing me off
The thread will run for exactly 1 second, and It can not be running multiple times or it will ruin the point of the delaying and threading.
if(KillAura.enabled && !KillThread.isAlive())
{
System.out.println("Go AURA!");
try
{
KillThread.start();
}catch (Exception e)
{
e.printStackTrace();
}
}
That is called every tick within the game where it would send position updates and such.
Now here is where I'm having the problem. Once the thread starts it becomes "alive" and when it ends it is no longer "alive". But can threads only be started once? because after the first run it's no longer working? And ideas? Links?

Yes Threads can only be started once, you cannot reuse a Thread object.
It is never legal to start a thread more than once. In particular, a
thread may not be restarted once it has completed execution. See java.lang.Thread.start()
Regardless of this fact, do not use the Thread.State for thread lifecycle management.

You're right, threads can run only once and it's illegal to start/run a thread more than once. You should consider using a while loop to keep your thread alive.
Instead of directly dealing with Threads, you should be using the classes inside the java.util.concurrent package to schedule a fixed task at regular intervals which is apparently what you're trying to do. Take a look at ThreadPoolExecutor.