I read a lot of post about ExecutorService, but I can't find the way of doing what I need.
I need some concurrent threads. When any of them throw a custom exception all the remaining tasks are canceled.
This is an example of what I did. The task are working concurrent, but aren't interrupted on exception.
public class Main {
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(2);
List<Future> futures = new ArrayList<Future>();
futures.add(executorService.submit(new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(5000);
System.out.println("Task 1 done");
return null;
}
}));
futures.add(executorService.submit(new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(2000);
System.out.println("Task 2 done");
if (true) {
throw new CustomException("Error on task 2");
}
return null;
}
}));
executorService.shutdown();
try {
executeFutures(futures);
} catch (CustomException ex) {
System.out.println("Received:" + ex.getMessage());
executorService.shutdownNow();
}
}
private static void executeFutures(List<Future> futures) throws CustomException {
try {
for (Future f : futures) {
f.get();
}
} catch (ExecutionException | InterruptedException e) {
if (e.getCause() instanceof CustomException) {
throw (CustomException) e.getCause();
}
}
}
}
This is the output:
Task 2 done //exception is thrown here but task1 continue.
Task 1 done
Received:Error on task 2
Any help will be appreciated.
Your problem is due to the fact that the method executeFutures make the main thread call f.get() on the first Future instance corresponding to the long task, which makes it wait the duration of the task so at least 5 seconds whatever happens. Once done it will then call f.get() on the second Future which is already over so it gets immediately the CustomException from the ExecutionException and calls executorService.shutdownNow() but it is already too late as there is no more tasks left to interrupt.
What you could do, is to use a decorator of type Callable that will automatically shutdown the thread pool when a CustomException is thrown, this way the thread pool will be shutdown directly by the thread that has executed the task that throws the exception instead of using the main thread.
Something like this:
public class AutoShutdown<V> implements Callable<V> {
private final ExecutorService executorService;
private final Callable<V> task;
public AutoShutdown(final ExecutorService executorService, final Callable<V> task) {
this.executorService = executorService;
this.task = task;
}
#Override
public V call() throws Exception {
try {
return task.call();
} catch (CustomException e) {
executorService.shutdownNow();
throw e;
}
}
}
Then you will need to submit your tasks through the decorator as next:
futures.add(
executorService.submit(
new AutoShutdown<>(
executorService,
new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(5000);
System.out.println("Task 1 done");
return null;
}
}
)
)
);
futures.add(
executorService.submit(
new AutoShutdown<>(
executorService,
new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(2000);
System.out.println("Task 2 done");
if (true) {
throw new CustomException("Error on task 2");
}
return null;
}
}
)
)
);
Output:
Task 2 done
As you can see in the output, the task one has been interrupted soon enough.
The message "Received:Error on task 2" was not thrown, so it looks
like a successful execution, and is not the case
No it is only because the first call to f.get() throws an InterruptedException as expected which makes it exit from executeFutures because the catch is performed outside the loop, move it inside the loop as next:
private static void executeFutures(List<Future> futures) throws CustomException {
for (Future f : futures) {
try {
f.get();
} catch (ExecutionException | InterruptedException e) {
if (e.getCause() instanceof CustomException) {
throw (CustomException) e.getCause();
}
}
}
}
Output:
Task 2 done
Received:Error on task 2
Related
I am new to working with ExecutorService, Future, and Runnable in java to set up timeouts on threads. I am working on a program where my main thread will call another thread to parse an XML file and (for security purposes) time out after a certain amount of time. I have been googling for hours and read many StackOverFlow threads and I just cannot seem to get the main thread to interrupt the secondary thread at all. When I run this program, the xml parser will go on forever parsing ridiculously large files, and I cannot seem to get it to be interrupted. Any help would be greatly appreciated. My code for both threads is below.
public class xmlParser{
private static class Parse implements Runnable {
private final String xmlFile;
public Parse(String xmlFile) {
this.xmlFile = xmlFile;
}
#Override
public void run() {
try {
while (!Thread.interrupted()) {
XMLReader xmlReader = XMLReaderFactory.createXMLReader();
xmlReader.setContentHandler(new MyContentHandler());
xmlReader.parse(new InputSource(xmlFile));
}
}
catch (Exception e) {
System.err.println("TIMEOUT ERROR: Took too long to parse xml file.");
e.printStackTrace();
}
}
}
public static void main(String[] args) throws InterruptedException {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future future = executor.submit(new Parse(args[0]));
try {
future.get(1, TimeUnit.SECONDS);
}
catch (Exception e) {
future.cancel(true);
}
finally {
executor.shutdownNow();
}
}
}
Note: I am aware of the multiple types of exceptions that future.get(long timeout, TimeUnit unit) will throw and will handle that later. Currently, I simply want my main thread to interrupt the Parse thread after 1 second of running.
I tried to reproduce with a simpler job:
static class FiveSecJob implements Callable<String> {
#Override
public String call() {
long t0 = System.currentTimeMillis();
try {
Thread.sleep(5000);
return "success";
} catch (Exception e) {
System.out.println("interrupted after " + (System.currentTimeMillis() - t0) / 1000d + "s: " + e);
return e.getMessage();
}
}
}
#Test
public void testTimeout() {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(new FiveSecJob());
String s = "initial value";
try {
s = future.get(1, TimeUnit.SECONDS);
} catch (Exception e) {
System.out.println("cancelling future (" + e + ")");
future.cancel(true);
} finally {
executor.shutdownNow();
}
System.out.println("s: " + s);
}
It seems to cancel the job like intendend. The output is:
cancelling future (java.util.concurrent.TimeoutException)
s: initial value
interrupted after 1.0s: java.lang.InterruptedException: sleep interrupted
Facing the problem with the ThreadPoolExecutor in Java.
How can I execute a continuous task using it? For example, I want to execute something like this:
#Async
void MyVoid(){
Globals.getInstance().increment();
System.out.println(Thread.currentThread().getName()+" iteration # "+ Globals.getInstance().Iterator);
}
I want it to run forever in 2 parallel asynchronous threads until the user sends a request to stop the ThreadPoolExecutor in the "/stop" controller.
If I use this for example:
#Controller
#RequestMapping("api/test")
public class SendController {
ThreadPoolExecutor executor = new ErrorReportingThreadPoolExecutor(5);
boolean IsRunning = true;
#RequestMapping(value = "/start_new", method = RequestMethod.POST)
public Callable<String> StartNewTask(#RequestBody LaunchSend sendobj) throws IOException, InterruptedException {
Runnable runnable = () -> { MyVoid();};
executor.setCorePoolSize(2);
executor.setMaximumPoolSize(2);
while (IsRunning) {
executor.execute(runnable);
System.out.println("Active threads: " + executor.getActiveCount());
}
return () -> "Callable result";
}
#RequestMapping(value = "/stop", method = RequestMethod.GET)
public Callable<String> StopTasks() {
executor.shutdown(); //for test
if(SecurityContextHolder.getContext().getAuthentication().getName() != null && SecurityContextHolder.getContext().getAuthentication().getName() != "anonymousUser") {
executor.shutdown();
return () -> "Callable result good";
}
else { return () -> "Callable result bad";}
}
}
public class ErrorReportingThreadPoolExecutor extends ThreadPoolExecutor {
public ErrorReportingThreadPoolExecutor(int nThreads) {
super(nThreads, nThreads,
0, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
#Override
protected void afterExecute(Runnable task, Throwable thrown) {
super.afterExecute(task, thrown);
if (thrown != null) {
// an unexpected exception happened inside ThreadPoolExecutor
thrown.printStackTrace();
}
if (task instanceof Future<?>) {
// try getting result
// if an exception happened in the job, it'll be thrown here
try {
Object result = ((Future<?>)task).get();
} catch (CancellationException e) {
// the job get canceled (may happen at any state)
e.printStackTrace();
} catch (ExecutionException e) {
// some uncaught exception happened during execution
e.printStackTrace();
} catch (InterruptedException e) {
// current thread is interrupted
// ignore, just re-throw
Thread.currentThread().interrupt();
}
}
}
}
I'm getting the following errors:
As I understood, a lot of tasks got submitted into the 'executor' queue within a few seconds and then the executor handled all them. (But I need each thread to wait before the current task ends and then submit the new one to the executor, I think.)
HTTP Requests to these controllers are forever "IDLE" until the next request comes, i.e. after sending a request to /api/test/start_new the controller's code executed tasks that are running, but the request is IDLE.
How can I do this in Java?
P.S. Spring MVC is used in the project. It has its own implementation of ThreadPoolExecutor - ThreadPoolTaskExecutor, but I am facing similar problems with it.
I am using single thread executor for long-running threads like this:
executor = Executors.newSingleThreadExecutor(THREAD_FACTORY);
executor.submit(new LongRunnable());
which checks a flag to be stopped:
private class LongRunnable implements Runnable {
#Override
public void run() {
while (isRunning.get()) {
try {
doSomething();
} catch (InterruptedException e) {
...
}
}
}
}
and whole execution is interrupted that way:
#Override
public void close() throws Exception {
isRunning.set(false);
executor.shutdownNow();
}
Still I can see some threads not gc-ed in profiler (while by logs, runnable they were executing has quit outermost while loop).
Question: does provided working with threads strategy memory-leak-free and thread-leak-free?
I am not able to see any issue with executor or shutDownNow. Probably you are looking at different threads in your profiler.
Try this program which is similar to the one in your question and you can see the thread is no longer there after successful shutdown.
public class ExecutorShutdownTest {
private static ExecutorService executor;
private static AtomicLong executorThreadId = new AtomicLong(0);
public static void main(String[] args) {
// get thread MX bean
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
// create an executor and start the task
executor = Executors.newSingleThreadExecutor(new TestThreadFactory());
LongRunnable runnable = new LongRunnable();
executor.submit(runnable);
// main thread: keep running for sometime
int count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace(
"\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// main thread: stop the task
try {
runnable.close();
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (Exception e) {
e.printStackTrace();
}
// main thread: run some more time to verify the executor thread no longer exists
count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private static class LongRunnable implements Runnable {
private volatile boolean isRunning = true;
#Override
public void run() {
while (isRunning) {
System.out.println("Running");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
//ignore
}
}
System.out.println("Stopped");
}
public void close() throws Exception {
System.out.println("Stopping");
isRunning = false;
executor.shutdownNow();
}
}
private static class TestThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
TestThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup();
namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0) {
#Override protected void finalize() throws Throwable {
super.finalize();
// probably bad idea but lets see if it gets here
System.out.println("Executor thread removed from JVM");
}
};
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
executorThreadId.set(t.getId());
System.out.println("Executor thread created");
return t;
}
}
}
Here's a sample program using the single-thread Executor that manages to strand a thread so that the JVM can't shut down, but it only manages to do it by not calling shutdownNow:
import java.util.concurrent.*;
public class Exec {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(new MyTask());
Thread.sleep(20000L);
// executor.shutdownNow();
int retryCount = 4;
while (!executor.isTerminated() && retryCount > 0) {
System.out.println("waiting for tasks to terminate");
Thread.sleep(500L);
retryCount -= 1;
}
}
}
class MyTask implements Runnable {
public void run() {
int count = 0;
try {
while (!Thread.currentThread().isInterrupted() && count < 10) {
Thread.sleep(1000L);
count += 1;
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("all done");
}
}
The thread used by the executor has a separate life cycle from the task, this example shows how the task finishes but the thread goes on. Uncommenting the shutdownNow results in the executor's thread terminating. Otherwise the main thread sleeps for a while and exits, leaving the executor's thread hanging out, preventing the JVM from exiting.
My guess is that your close method isn't getting called and your executor never gets shut down. To get more useful answers please add a MVCE so that we can reproduce the problem.
Consider that with interruption there's no need to keep a reference to the Runnable to set the flag. As I read the question the task not finishing is not an issue here, but it would still be better to make the Runnable respond to interruption and lose the flag, just because having less things to keep track of is always an improvement.
I need to test InterruptedException and ExecutionException and write JUnits for the same.
Please advice me on this. How can i interrupt the threads to replicate the scenario. The populateDataForm will start new threads and add this to futures list.
Here is my sample code:
class MyTest{
public populateData(){
Collection<Future<?>> futures = new LinkedList<Future<?>>();
DataSet ds = Helper.populateDataForm(employee, futures);
waitForTaskCompletion(futures);
}
private waitForTaskCompletion(futures){
for (Future<?> future:futures) {
try {
future.get();
} catch (InterruptedException e) {
throw new CustomExcpetion("Message1", e)
} catch (ExecutionException e) {
throw new CustomExcpetion("Message2", e)
}
}
}
You can inherit from MyTest and overload populateData() method as follows:
public void populateData() {
ExecutorService executorService = Executors.newSingleThreadExecutor();
Callable<String> calls = new Callable<String>() {
#Override
public String call() throws Exception {
for (;;){
Thread.sleep(100);
// You call interrupt here, which causes Future.get() interrupt
Thread.currentThread().interrupt();
if (1 > 2) break;
}
return null;
}
};
final Future<String> future = executorService.submit(calls);
waitForTaskCompletion(future);
executorService.shutdown();
}
To test the ExecutionException throw RuntimeException instead of the interrupt as follows:
if (1==1)throw new RuntimeException();
Why we can't see the stacktrace in this example ?
public class NoStackTraceTester implements Runnable {
private static final ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
private ScheduledFuture<?> lifeCheckFuture;
#Override
public void run() {
lifeCheckFuture = startLifecheck();
}
private ScheduledFuture<?> startLifecheck()
{
Runnable lifeCheck = new Runnable()
{
#Override
public void run()
{
System.out.println("sending lifecheck ...");
throw new RuntimeException("bang!");
}
};
return scheduler.scheduleAtFixedRate(lifeCheck, 1000, 1000, TimeUnit.MILLISECONDS);
}
public static void main(String[] args) {
new NoStackTraceTester().run();
}
}
If you try to comment the exception you will the the repeative task of the lifecheck function.
But if an exception is thrown, thread stop but with no detail :(
Do you have an idea why ?
An ExecutorService places any captured Throwable in the Future object. If you inspect this you can see what exception was thrown. This is not always desirable so you may have to catch and handle or log any exception in the run() method.
Note: once an exception escapes, the task is not repeated again.
Runnable lifeCheck = new Runnable() {
#Override
public void run() {
try {
System.out.println("sending lifecheck ...");
throw new RuntimeException("bang!");
} catch(Throwable t) {
// handle or log Throwable
}
}
};
If you want an exception report, you must insert handling code yourself. The ExecutorService will not automatically send the exception trace to the standard output, and it is very good that it doesn't since this is rarely what we need in production code.
Basically, this is the approach:
public void run()
{
try {
System.out.println("sending lifecheck ...");
throw new RuntimeException("bang!");
} catch (Throwable t) { t.printStackTrace(); }
}
The afterExecute() method in ThreadPoolExecutor can be overridden:
class MyThreadPoolExecutor extends ThreadPoolExecutor {
public MyThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime,
TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
}
#Override
public void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
// If submit() method is called instead of execute()
if (t == null && r instanceof Future<?>) {
try {
Object result = ((Future<?>) r).get();
} catch (CancellationException e) {
t = e;
} catch (ExecutionException e) {
t = e.getCause();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
if (t != null) {
// Exception occurred
System.err.println("Uncaught exception is detected! " + t
+ " st: " + Arrays.toString(t.getStackTrace()));
}
// ... Perform cleanup actions
}
}
final class MyTask implements Runnable {
#Override public void run() {
System.out.println("My task is started running...");
// ...
throw new ArithmeticException(); // uncatched exception
// ...
}
}
public class ThreadPoolExecutorHandler {
public static void main(String[] args) {
// Create a fixed thread pool executor
ExecutorService threadPool = new MyThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>());
threadPool.execute(new MyTask());
// ...
}
}
source: https://medium.com/#aozturk/how-to-handle-uncaught-exceptions-in-java-abf819347906 (Please note, I modified the code posted here to not re-execute as the question only asks for stacktrace printing)