I have a ScheduledExecutorService and I do a task every 15 mins (in a web application in Tomcat).
In a ServletContextListener in contextDestroyed I have done:
Runtime.getRuntime().addShutdownHook(new Thread(){
#Override
public void run(){
scheduler.shutdown();
}
});
The scheduler is started like:
final Runnable r = new Runnable(){
#Override
public void run() {
System.out.println("My task");
//Do some task
}
};
updater.scheduleWithFixedDelay(r, 30, 15, TimeUnit.MINUTES);
Question:On shutdown any remaining task isn't executed.
There is a task running but I don't see any logs so it seems it is not executed. Why?
UPDATE:
If I start tomcat and then after 2 mins I shutdown then isn't the task considered as scheduled and must run? I mean if a task is submitted isn't it considered as pending? Or it must be actually running?
To explicitly wait until all running tasks are finished, do something like this:
try {
// Wait for one second indefinitively
while (!scheduler.awaitTermination (1, TimeUnit.SECONDS)) {
// wait until completion
}
} catch (final InterruptedException ex) {
// interrupted... you may log something
}
I can't quite parse your question but maybe your application is not exiting like you'd expect?
Shutting down an Executor will stop any tasks for being submitted but any running tasks will continue to run until they exit. From the shutdown() javadocs:
Initiates an orderly shutdown in which previously submitted
tasks are executed, but no new tasks will be accepted.
Invocation has no additional effect if already shut down.
If you edit your question to be more clear we can answer more appropriately.
Related
Using #Scheduling to run method at #Scheduled(fixedRate = 10000) and set up #Scheduling threading by implementing SchedulingConfigurer
#Bean(destroyMethod="shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(10);
}
if I used Thread.sleep or Lock , no other thread is created by Executor unless Thread.sleep wake up or lock is cleared.
Can someone explain internal working if i have 10 pool size they 10 threads should be created at rate of 10000 millisec.
Basically such behavior comes from ScheduledExecutorService implementation which is used internally by spring. If you will run this code you will notice the same behavior:
public static void main(String[] args) throws Exception {
ScheduledExecutorService executor = Executors.newScheduledThreadPool(10);
executor.schedule(() -> {
System.out.println("Running task in thread " + Thread.currentThread().getId());
try {
Thread.sleep(Integer.MAX_VALUE);
} catch (InterruptedException e) {
System.out.println("interrupted while sleeping");
}
}, 1000, TimeUnit.MILLISECONDS);
Thread.sleep(10000);
executor.shutdownNow();
}
When you submit task to scheduled thread pool it is wrapped with RunnableScheduledFuture which is passed to delayedExecute method. This method adds the task to the tasks queue and starts new worker if current number of workers is less than corePoolSize. Worker tries to get a task from the queue and process it invoking run method. There is a dedicated DelayedWorkQueue implementation which returns tasks only if they are ready for execution. Here is how run method of RunnableScheduledFuture looks like:
/**
* Overrides FutureTask version so as to reset/requeue if periodic.
*/
public void run() {
boolean periodic = isPeriodic();
if (!canRunInCurrentRunState(periodic))
cancel(false);
else if (!periodic)
ScheduledFutureTask.super.run();
else if (ScheduledFutureTask.super.runAndReset()) {
setNextRunTime();
reExecutePeriodic(outerTask);
}
}
As you can see it invokes actual task logic in runAndReset, calculates the next running time and submits the same updated task to the queue again (reExecutePeriodic is almost the same as schedule). There is only a single periodic task for all executions which is resubmitted again and again with updated time after the previous execution is finished. So such thread pool runs only a single instance of each task type in any given moment and scales only for different type of tasks.
If you are interesting in how spring schedules tasks take a look at ScheduledTaskRegistrar class and particularly at scheduleFixedDelayTask method.
In the case that you use threadpool:
By default you going to have 10 threads in your pool(already initialized). The first time that #scheduled is executed, this function is going to execute in a thread from your pool (now remaining 9 threads), if the function yet finished and #scheduled is executed again, your function going to executed in other thread from you your pool, so now you have 8 thread remaining in your pool. (8 idle, 2 running threads)
If you aren`t use threadpool only one thread is used.
spring documentation:
If you do not provide a 'pool-size' attribute, the default thread pool
will only have a single thread. There are no other configuration
options for the scheduler.
https://docs.spring.io/spring/docs/4.3.x/spring-framework-reference/html/scheduling.html
Difference between:
A)
ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
ScheduledFuture<> periodicTask = executorService.scheduleAtFixedRate(() -> {
try {
doSomething();
} catch (Exception e) {
log.error("Unhandled exception caught whilst doing minutely run", e);
}
}, initialDelay, PERIOD, TimeUnit.MILLISECONDS);
// In a shutdown hook:
periodicTask.cancel(true);
Does that cancel all the running tasks? Does it kill the ExecutorService?
B) The other way would be:
executorService.shutdown();
executorService.awaitTermination(....);
What's the difference?
Also how do I know how many tasks in the future the executorservice schedules?
Once I get the shut down signal I just want to run my scheduled task around 2 - 3 more times until I reach a certain condition. After that I want to kill it.
periodicTask.cancel(true);
The cancel method only stops the unstarted jobs and interrupts the running thread which then must return from the run() method.
executorService.shutdown();
The shutdown() method prevents clients sending more work to the executor service. This means all the existing tasks will run to completion.
executorService.awaitTermination(....);
This helps the application shut down gracefully. i.e. The executor service takes no more work and waits till all the executing jobs finish and then shuts-down.
I need to perform some data collection periodically, for that I want to create a task which requests some data from different servers. Some servers will take more time to process the request and return the response than others.
That's why I want to create a task for each server and execute the tasks async. If i'm using ScheduledExecutorService in the following way will each task execute in its own thread or all the tasks will be executed in the same thread?
What happens if a task is throwing an exception all the other scheduled tasks will fail?
this.scheduler = Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {
#Override
public Thread newThread(Runnable r) {
return new Thread(r, "collectionThread");
}
});
for (String url:urls){
this.scheduler.scheduleWithFixedDelay(new CollectorTask(url),
startupDelaySeconds,
scheduleRateSeconds,
TimeUnit.SECONDS);
}
You're using a single-threaded executor service, so all tasks are being executed sequentially. If any of them throws an exception, all next tasks executions are cancelled.
You could use
Executors.newScheduledThreadPool(4) // 4 is the max number of parallel jobs
... to allow parallel execution and wrap a body of a submitted job into
try {
...
} catch(Exception e){
logger.warn("exception during task execution", e);
}
to log errors without propagating them.
Minor delays (several milliseconds) are possible and depend on the OS, tasks will never execute earlier than their scheduled time. Task's execution can be delayed due to previous long runs or lack of free threads, but the following executions will be run by the original schedule: initialDelay + n * period.
Yes, what you do is create two thread executors. The first is a scheduled executor which takes a runnable that is meant to start your actual runnable. All this runnable does it create an instance of your task runnable and submit it to the real executor. That executor should just be a normal thread pool that will handle the tasks.
private final ScheduledExecutorService scheduledExecutor = Executors
.newSingleThreadScheduledExecutor();
private final ExecutorService executor = Executors.newCachedThreadPool();
private class SubmitTaskRunnable implements Runnable {
#Override
public void run() {
executor.execute(new TaskRunnable());
}
}
Schedule the SubmitTaskRunnable on your scheduled executor because that one will not throw any exceptions. Let your actual task run inside a cached executor will allow multiple tasks to run concurrently even if the previous ones have not finished.
I am using the Java ExecutorService framework to submit callable tasks for execution.
These tasks communicate with a web service and a web service timeout of 5 mins is applied.
However I've seen that in some cases the timeout is being ignored and thread 'hangs' on an API call - hence, I want to cancel all the tasks that take longer than say, 5 mins.
Currently, I have a list of futures and I iterate through them and call future.get until all tasks are complete. Now, I've seen that the future.get overloaded method takes a timeout and throws a timeout when the task doesnt complete in that window. So I thought of an approach where I do a future.get() with timeout and in case of TimeoutException I do a future.cancel(true) to make sure that this task is interrupted.
My main questions
1. Is the get with a timeout the best way to solve this issue?
2. Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Any suggestions would be deeply appreciated.
Is the get with a timeout the best way to solve this issue?
This will not suffice. For instance, if your task is not designed to response to interruption, it will keep on running or be just blocked
Is there the possibility that I'm waiting with the get call on a task that hasnt yet been placed on the thread pool(isnt an active worker). In that case I may be terminating a thread that, when it starts may actually complete within the required time limit?
Yes, You might end up cancelling as task which is never scheduled to run if your thread-pool is not configured properly
Following code snippet could be one of the way you can make your task responsive to interruption when your task contains Non-interruptible Blocking. Also it does not cancel the task which are not scheduled to run. The idea here is to override interrupt method and close running tasks by say closing sockets, database connections etc. This code is not perfect and you need to make changes as per requirements, handle exceptions etc.
class LongRunningTask extends Thread {
private Socket socket;
private volatile AtomicBoolean atomicBoolean;
public LongRunningTask() {
atomicBoolean = new AtomicBoolean(false);
}
#Override
public void interrupt() {
try {
//clean up any resources, close connections etc.
socket.close();
} catch(Throwable e) {
} finally {
atomicBoolean.compareAndSet(true, false);
//set the interupt status of executing thread.
super.interrupt();
}
}
public boolean isRunning() {
return atomicBoolean.get();
}
#Override
public void run() {
atomicBoolean.compareAndSet(false, true);
//any long running task that might hang..for instance
try {
socket = new Socket("0.0.0.0", 5000);
socket.getInputStream().read();
} catch (UnknownHostException e) {
} catch (IOException e) {
} finally {
}
}
}
//your task caller thread
//map of futures and tasks
Map<Future, LongRunningTask> map = new HashMap<Future, LongRunningTask>();
ArrayList<Future> list = new ArrayList<Future>();
int noOfSubmittedTasks = 0;
for(int i = 0; i < 6; i++) {
LongRunningTask task = new LongRunningTask();
Future f = execService.submit(task);
map.put(f, task);
list.add(f);
noOfSubmittedTasks++;
}
while(noOfSubmittedTasks > 0) {
for(int i=0;i < list.size();i++) {
Future f = list.get(i);
LongRunningTask task = map.get(f);
if (task.isRunning()) {
/*
* This ensures that you process only those tasks which are run once
*/
try {
f.get(5, TimeUnit.MINUTES);
noOfSubmittedTasks--;
} catch (InterruptedException e) {
} catch (ExecutionException e) {
} catch (TimeoutException e) {
//this will call the overridden interrupt method
f.cancel(true);
noOfSubmittedTasks--;
}
}
}
}
execService.shutdown();
Is the get with a timeout the best way to solve this issue?
Yes it is perfectly fine to get(timeout) on a Future object, if the task that the future points to is already executed it will return immediately. If the task is yet to be executed or is being executed then it will wait until timeout and is a good practice.
Is there the possibility that I'm waiting with the get call on a task
that hasnt yet been placed on the thread pool(isnt an active worker)
You get Future object only when you place a task on the thread pool so it is not possible to call get() on a task without placing it on thread pool. Yes there is a possibility that the task has not yet been taken by a free worker.
The approach that you are talking about is ok. But most importantly before setting a threshold on the timeout you need to know what is the perfect value of thread pool size and timiout for your environment. Do a stress testing which will reveal whether the no of worker threads that you configured as part of Threadpool is fine or not. And this may even reduce the timeout value. So this test is most important i feel.
Timeout on get is perfectly fine but you should add to cancel the task if it throws TimeoutException. And if you do the above test properly and set your thread pool size and timeout value to ideal than you may not even need to cancel tasks externally (but you can have this as backup). And yes sometimes in canceling a task you may end up canceling a task which is not yet picked up by the Executor.
You can of course cancel a Task by using
task.cancel(true)
It is perfectly legal. But this will interrupt the thread if it is "RUNNING".
If the thread is waiting to acquire an intrinsic lock then the "interruption" request has no effect other than setting the thread's interrupted status. In this case you cannot do anything to stop it. For the interruption to happen, the thread should come out from the "blocked" state by acquiring the lock it was waiting for (which may take more than 5 mins). This is a limitation of using "intrinsic locking".
However you can use explicit lock classes to solve this problem. You can use "lockInterruptibly" method of the "Lock" interface to achieve this. "lockInterruptibly" will allow the thread to try to acquire a lock while remaining responsive to the interruption. Here is a small example to achieve that:
public void workWithExplicitLock()throws InterruptedException{
Lock lock = new ReentrantLock();
lock.lockInterruptibly()();
try {
// work with shared object state
} finally {
lock.unlock();
}
}
I got weird problem.
I have a ScheduledExecutorService.scheduleWithFixedDelay which doesn't not "stop" when i click publish / stop-restart server .
When i republish, there will be 2 service running and the number of same service running depends on how many times I click publish.
This does not go away after I stop and start server.
I have to close eclipse and open to stop it and I notice this happen only after i edit the class that initialize the service.
I'm unable to paste the codes but is very simple.. just a Singleton and holds the service
private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(new Runnable() {
public void run() { system.out("hello" }
}, 60 * 60, SECONDS);
I can't seems to find out the issue, can someone help me please.
You're supposed to shut down the scheduler when the application is undeployed. The easier way to do that is to declare an implement a ServletContextListener, and shut down the schedler in the contextDestroyed() method.
Executors.newScheduledThreadPool(1); creates a non-daemon worker thread and it prevents JVM from shutting down unless you shutdown scheduler explicitly
scheduler.shutdown();
alternatevely, you can do the following
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1,
new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setDaemon(false);
return t;
}
});
in this case the worker thread will be a daemon and will not prevent the JVM from exiting when the program finishes