I am new to working with ExecutorService, Future, and Runnable in java to set up timeouts on threads. I am working on a program where my main thread will call another thread to parse an XML file and (for security purposes) time out after a certain amount of time. I have been googling for hours and read many StackOverFlow threads and I just cannot seem to get the main thread to interrupt the secondary thread at all. When I run this program, the xml parser will go on forever parsing ridiculously large files, and I cannot seem to get it to be interrupted. Any help would be greatly appreciated. My code for both threads is below.
public class xmlParser{
private static class Parse implements Runnable {
private final String xmlFile;
public Parse(String xmlFile) {
this.xmlFile = xmlFile;
}
#Override
public void run() {
try {
while (!Thread.interrupted()) {
XMLReader xmlReader = XMLReaderFactory.createXMLReader();
xmlReader.setContentHandler(new MyContentHandler());
xmlReader.parse(new InputSource(xmlFile));
}
}
catch (Exception e) {
System.err.println("TIMEOUT ERROR: Took too long to parse xml file.");
e.printStackTrace();
}
}
}
public static void main(String[] args) throws InterruptedException {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future future = executor.submit(new Parse(args[0]));
try {
future.get(1, TimeUnit.SECONDS);
}
catch (Exception e) {
future.cancel(true);
}
finally {
executor.shutdownNow();
}
}
}
Note: I am aware of the multiple types of exceptions that future.get(long timeout, TimeUnit unit) will throw and will handle that later. Currently, I simply want my main thread to interrupt the Parse thread after 1 second of running.
I tried to reproduce with a simpler job:
static class FiveSecJob implements Callable<String> {
#Override
public String call() {
long t0 = System.currentTimeMillis();
try {
Thread.sleep(5000);
return "success";
} catch (Exception e) {
System.out.println("interrupted after " + (System.currentTimeMillis() - t0) / 1000d + "s: " + e);
return e.getMessage();
}
}
}
#Test
public void testTimeout() {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(new FiveSecJob());
String s = "initial value";
try {
s = future.get(1, TimeUnit.SECONDS);
} catch (Exception e) {
System.out.println("cancelling future (" + e + ")");
future.cancel(true);
} finally {
executor.shutdownNow();
}
System.out.println("s: " + s);
}
It seems to cancel the job like intendend. The output is:
cancelling future (java.util.concurrent.TimeoutException)
s: initial value
interrupted after 1.0s: java.lang.InterruptedException: sleep interrupted
Related
I'm trying to find a way to set a time limit for running a block of code (force-terminate it when time is up) without modifying the internals of the block of code. Here's what I tried to do: I first copied the TimeLimitedCodeBlock class from this link: Java-how-to-set-timeout
import java.util.*;
import java.util.concurrent.*;
public class TimeLimitedCodeBlock {
public static void runWithTimeout(final Runnable runnable, long timeout, TimeUnit timeUnit) throws Exception {
runWithTimeout(new Callable<Object>() {
#Override
public Object call() throws Exception {
runnable.run();
return null;
}
}, timeout, timeUnit);
}
public static <T> T runWithTimeout(Callable<T> callable, long timeout, TimeUnit timeUnit) throws Exception {
final ExecutorService executor = Executors.newSingleThreadExecutor();
final Future<T> future = executor.submit(callable);
executor.shutdown(); // This does not cancel the already-scheduled task.
try {
return future.get(timeout, timeUnit);
}
catch (TimeoutException e) {
future.cancel(true);
throw e;
}
catch (ExecutionException e) {
Throwable t = e.getCause();
if (t instanceof Error) {
throw (Error) t;
} else if (t instanceof Exception) {
throw (Exception) t;
} else {
throw new IllegalStateException(t);
}
}
}
}
And here is what I ran using the class defined above:
public static void main(String [] args)
{
try{
TimeLimitedCodeBlock.runWithTimeout(new Runnable()
{
public void run()
{
try{
while(true){}
}catch(Exception e){}
}},1,TimeUnit.SECONDS);
}
catch(Exception e){}
}
And it's not terminating. How should I fix it so that it terminates?
Code snippet that I've used to do something similar:
LOG.info("Time limited task started on monitored thread, with limit (" + limit + ")");
final ZonedDateTime start = nowUTC();
final Thread thread = new Thread(toRun);
thread.setDaemon(true);
final List<Throwable> exceptions = new ArrayList<>();
thread.setUncaughtExceptionHandler((t, e) -> {
exceptions.add(e);
});
thread.start();
// Check and wait for completion.
while (thread.isAlive()) {
if (!isWithinLimit(start, nowUTC())) {
LOG.error("Interrupting thread, did not complete before limit (" + limit + ")");
try {
thread.interrupt();
} catch (Exception e) {
e.printStackTrace();
}
throw new TimeLimitExceedException("Execution limit of " + limit
+ " exceeded. (Has been running since " + start + ")");
}
try {
Thread.sleep(POLLING_PERIOD.toMillis());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// If it failed because of an exception, we want to trigger this.
if (!exceptions.isEmpty()) {
final Throwable exception = exceptions.get(0);
if (exception instanceof RuntimeException) {
throw (RuntimeException) exception;
} else {
throw new RuntimeException(exception);
}
}
final Duration runTime = Duration.between(start, nowUTC());
LOG.info("Time limited task has completed in (" + runTime + ") vs limit of (" + limit
+ ").");
TLDR:
I just start whatever I'm running as a new thread that is set as a daemon (just in case it is the last thing running), then I get a reference to that and poll it, and call thread.interrupt() if it goes over the time limit.
Other context & bells and whistles
This is part of a class that has some other state, like duration and what it is running
Also track some exceptions so that it can be spit out at the end if necessary
I read a lot of post about ExecutorService, but I can't find the way of doing what I need.
I need some concurrent threads. When any of them throw a custom exception all the remaining tasks are canceled.
This is an example of what I did. The task are working concurrent, but aren't interrupted on exception.
public class Main {
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(2);
List<Future> futures = new ArrayList<Future>();
futures.add(executorService.submit(new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(5000);
System.out.println("Task 1 done");
return null;
}
}));
futures.add(executorService.submit(new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(2000);
System.out.println("Task 2 done");
if (true) {
throw new CustomException("Error on task 2");
}
return null;
}
}));
executorService.shutdown();
try {
executeFutures(futures);
} catch (CustomException ex) {
System.out.println("Received:" + ex.getMessage());
executorService.shutdownNow();
}
}
private static void executeFutures(List<Future> futures) throws CustomException {
try {
for (Future f : futures) {
f.get();
}
} catch (ExecutionException | InterruptedException e) {
if (e.getCause() instanceof CustomException) {
throw (CustomException) e.getCause();
}
}
}
}
This is the output:
Task 2 done //exception is thrown here but task1 continue.
Task 1 done
Received:Error on task 2
Any help will be appreciated.
Your problem is due to the fact that the method executeFutures make the main thread call f.get() on the first Future instance corresponding to the long task, which makes it wait the duration of the task so at least 5 seconds whatever happens. Once done it will then call f.get() on the second Future which is already over so it gets immediately the CustomException from the ExecutionException and calls executorService.shutdownNow() but it is already too late as there is no more tasks left to interrupt.
What you could do, is to use a decorator of type Callable that will automatically shutdown the thread pool when a CustomException is thrown, this way the thread pool will be shutdown directly by the thread that has executed the task that throws the exception instead of using the main thread.
Something like this:
public class AutoShutdown<V> implements Callable<V> {
private final ExecutorService executorService;
private final Callable<V> task;
public AutoShutdown(final ExecutorService executorService, final Callable<V> task) {
this.executorService = executorService;
this.task = task;
}
#Override
public V call() throws Exception {
try {
return task.call();
} catch (CustomException e) {
executorService.shutdownNow();
throw e;
}
}
}
Then you will need to submit your tasks through the decorator as next:
futures.add(
executorService.submit(
new AutoShutdown<>(
executorService,
new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(5000);
System.out.println("Task 1 done");
return null;
}
}
)
)
);
futures.add(
executorService.submit(
new AutoShutdown<>(
executorService,
new Callable<Void>() {
public Void call() throws Exception {
Thread.sleep(2000);
System.out.println("Task 2 done");
if (true) {
throw new CustomException("Error on task 2");
}
return null;
}
}
)
)
);
Output:
Task 2 done
As you can see in the output, the task one has been interrupted soon enough.
The message "Received:Error on task 2" was not thrown, so it looks
like a successful execution, and is not the case
No it is only because the first call to f.get() throws an InterruptedException as expected which makes it exit from executeFutures because the catch is performed outside the loop, move it inside the loop as next:
private static void executeFutures(List<Future> futures) throws CustomException {
for (Future f : futures) {
try {
f.get();
} catch (ExecutionException | InterruptedException e) {
if (e.getCause() instanceof CustomException) {
throw (CustomException) e.getCause();
}
}
}
}
Output:
Task 2 done
Received:Error on task 2
I am using single thread executor for long-running threads like this:
executor = Executors.newSingleThreadExecutor(THREAD_FACTORY);
executor.submit(new LongRunnable());
which checks a flag to be stopped:
private class LongRunnable implements Runnable {
#Override
public void run() {
while (isRunning.get()) {
try {
doSomething();
} catch (InterruptedException e) {
...
}
}
}
}
and whole execution is interrupted that way:
#Override
public void close() throws Exception {
isRunning.set(false);
executor.shutdownNow();
}
Still I can see some threads not gc-ed in profiler (while by logs, runnable they were executing has quit outermost while loop).
Question: does provided working with threads strategy memory-leak-free and thread-leak-free?
I am not able to see any issue with executor or shutDownNow. Probably you are looking at different threads in your profiler.
Try this program which is similar to the one in your question and you can see the thread is no longer there after successful shutdown.
public class ExecutorShutdownTest {
private static ExecutorService executor;
private static AtomicLong executorThreadId = new AtomicLong(0);
public static void main(String[] args) {
// get thread MX bean
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
// create an executor and start the task
executor = Executors.newSingleThreadExecutor(new TestThreadFactory());
LongRunnable runnable = new LongRunnable();
executor.submit(runnable);
// main thread: keep running for sometime
int count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace(
"\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// main thread: stop the task
try {
runnable.close();
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (Exception e) {
e.printStackTrace();
}
// main thread: run some more time to verify the executor thread no longer exists
count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private static class LongRunnable implements Runnable {
private volatile boolean isRunning = true;
#Override
public void run() {
while (isRunning) {
System.out.println("Running");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
//ignore
}
}
System.out.println("Stopped");
}
public void close() throws Exception {
System.out.println("Stopping");
isRunning = false;
executor.shutdownNow();
}
}
private static class TestThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
TestThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup();
namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0) {
#Override protected void finalize() throws Throwable {
super.finalize();
// probably bad idea but lets see if it gets here
System.out.println("Executor thread removed from JVM");
}
};
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
executorThreadId.set(t.getId());
System.out.println("Executor thread created");
return t;
}
}
}
Here's a sample program using the single-thread Executor that manages to strand a thread so that the JVM can't shut down, but it only manages to do it by not calling shutdownNow:
import java.util.concurrent.*;
public class Exec {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(new MyTask());
Thread.sleep(20000L);
// executor.shutdownNow();
int retryCount = 4;
while (!executor.isTerminated() && retryCount > 0) {
System.out.println("waiting for tasks to terminate");
Thread.sleep(500L);
retryCount -= 1;
}
}
}
class MyTask implements Runnable {
public void run() {
int count = 0;
try {
while (!Thread.currentThread().isInterrupted() && count < 10) {
Thread.sleep(1000L);
count += 1;
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("all done");
}
}
The thread used by the executor has a separate life cycle from the task, this example shows how the task finishes but the thread goes on. Uncommenting the shutdownNow results in the executor's thread terminating. Otherwise the main thread sleeps for a while and exits, leaving the executor's thread hanging out, preventing the JVM from exiting.
My guess is that your close method isn't getting called and your executor never gets shut down. To get more useful answers please add a MVCE so that we can reproduce the problem.
Consider that with interruption there's no need to keep a reference to the Runnable to set the flag. As I read the question the task not finishing is not an issue here, but it would still be better to make the Runnable respond to interruption and lose the flag, just because having less things to keep track of is always an improvement.
I am trying to signal between two threads using the below FutureResult class which extends FutureTask class. When run the script, it prints the following result.
SENDING: 0
SENT: 0
POLL: FutureResult#513431
SIGNALLED: FutureResult#513431
Then the program hang up forever. I expect FutureResult instance should return the value from it's blocking get method. Then print the result in the console. But FutureResult.get is blocking forever.
import java.util.concurrent.*;
/**
* Created by someone on 20/08/2015.
*/
final public class FutureResult<T> extends FutureTask<T> {
private static final Object SS = "SS";
public FutureResult() {
super(() -> null);
}
public void signal(final T value) {
set(value);
}
public void signalError(final Throwable throwable) {
setException(throwable);
}
public static void main(String... args) throws Exception {
final ArrayBlockingQueue<FutureResult> queue = new ArrayBlockingQueue<>(1000000);
new Thread(() -> {
while (true) {
try {
final FutureResult poll = queue.take();
System.out.println("POLL: " + poll);
if (poll != null) {
poll.signal(SS);
System.out.println("SIGNALLED: " + poll);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}).start();
new Thread(() -> {
for (int i = 0; i < 1; i++) {
final FutureResult<Object> result = new FutureResult<>();
System.out.println("SENDING: " + i);
queue.offer(new FutureResult());
try {
System.out.println("SENT: " + i);
result.get();
System.out.println("GOT : " + i);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}).start();
}
}
This is the problem:
queue.offer(new FutureResult());
You're setting the value on one FutureResult, but that's not the one you're waiting for. Just change that line to:
queue.offer(result);
and it works fine.
Looks like the confusion is in the use of FutureTask. FutureTask is designed as a Runnable; running it is necessary.
Honestly, based on the code, it looks like the custom code is implementing something similar to FutureTask. If the intent here is to learn to use FutureTask, then create a FutureTask instance with a "run" method, and then execute that run method. On completion of the run method, the FutureTask.get() will complete.
I am trying to create a solution to treat hung threads due to memory leaks, locked resources in our applications. One of the main problems I am having is trying to simulate a hung thread to deal with it. Any sugestions?
This is what I tried, but it just doesn't seem to do the job. Any thoughts?
class KillerThread extends Thread{
public KillerThread() {
super();
}
public KillerThread(String threadName) {
super(threadName);
}
public void run (){
System.out.println("Start of KillerThread " + this.getName() );
if ( System.currentTimeMillis() % 2L == 0 ){
try {
sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
for(;;);
}
}
}
Joining on one's own thread works well for me:
Thread.currentThread().join();
try running sleep in a while loop like:
while(true) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
running a thread then tell it to sleep in an unstoppable loop, is a good idea,.
but how if you are trying to make it waiting another thread,.? make more than one thread and make them wait one each other, a deadlock condition, is that a hung to,.?
I know what you need exactly, you are testing something through stopping the executor thread. Try something like this:
private void testKillingThread() {
Object kill = new Object();
try {
synchronized (kill) {
kill.wait();
}
} catch (Exception e) {
// Auto-generated catch block
}
}
Simply enough, just create a private member
private Object lock = new Object();
then use it to wait for a notification (that will never happen, unless you use reflection...)
while (true) {
try {
synchronized (lock) {
lock.wait();
}
} cath (InterruptedException e) {
/* ignore interruption */
}
}
and you thread will hang there, uninterruptable.
Here's a quick fix I'm using for testing. Just have the thread you want to lock up call new Hanger().hang().
Remove the logging if you're not interested in seeing it. You can add throws InterruptedException (although, in fact, it never does) to the hang method so you can just replace a Thread.sleep() with a new Hanger().hang() without otherwise modifying your code.
public class Hanger {
private final static Logger log = Logger.getLogger(Hanger.class);
private long started = 0;
private final int beat = 100; // ms
/**
* Hangs a thread for the indicated time
* #param millis the amount of time to hang the thread, in milliseconds
*/
public void hang(int millis) {
started = System.currentTimeMillis();
log.debug("Hanging this thread for " + millis + " ms");
while (hung() < millis) {
try {
Thread.sleep(beat);
} catch (InterruptedException e) {
log.debug("Still hanging, will release in " + (millis - hung()) + " ms.");
}
}
log.debug("Releasing thread again after " + hung() + " ms");
}
private int hung() {
return (int)(System.currentTimeMillis() - started);
}
}