I am using several threads to do some heavy (and error-prone) processing on a large data set. I require all threads to finish execution, regardless of whether they throw an exception or terminate normally (no value is returned), before the program can continue. I am using a CountDownLatch to achieve this, and an ExecutorService to actually run the jobs. I want the worker threads (let's call them JobManager-s for the sake of argument) to notify the latch even if they throw an exception. A JobManager can take anywhere between a second and an hour to complete, and may fail at any time. The idea is to invoke the "finalizer" method of JobManager if an exception is thrown. Now, the ExecutorService likes to catch exceptions or to conceal the true origin of the ones it does not. I have though of a few ways around this, neither of which is satisfactory:
Use ExecutorService#execute(Runnable r) rather than submit(Runnable r). I can do that since I do not care about the return value of the JobManager. I have provided a custom ThreadFactory, which attaches an UncaughtExceptionHandler to each newly created thread. The problem with this approach is that when UncaughtExceptionHandler#uncaughtException(Thread t, Throwable e) is invoked, t's Runnable is of type ThreadPoolExecutor$Worker, and not of type JobManager, which prevents me from invoking the "finalizer" method.
Use a custom ExecutorService and override the afterExecute(Runnable r, Throwable t) method. This suffers from the same problem as 1.
Wrap the whole JobManager#doWork() in a catch statement and use the return value to indicate if an exception was thrown. I can then submit the jobs and use FutureTask#get() to decide if an exception was thrown. I do not like this solution because I feel return codes the wrong tool when you have an elaborate exception mechanism. Moreover, get() will wait (unless interrupted), which means I cannot handle errors in other threads immediately.
Get rid of the CountDownLatch. Store all Futures in a list and repeatedly poke in until I am satisfied with the states. This might work, but feels like a dirty hack.
Any suggestions are greatly appreciated.
As far as I understand, you can use a simple try-finally block:
public class JobManager {
public void doWork() {
try {
...
} finally {
countDownLatch.countDown();
}
}
}
Related
During the course of my program execution, a number of threads are started. The amount of threads varies depending on user defined settings, but they are all executing the same method with different variables.
In some situations, a clean up is required mid execution, part of this is stopping all the threads, I don't want them to stop immediately though, I just set a variable that they check for that terminates them. The problem is that it can be up to 1/2 second before the thread stops. However, I need to be sure that all threads have stopped before the clean up can continues. The cleanup is executed from another thread so technically I need this thread to wait for the other threads to finish.
I have thought of several ways of doing this, but they all seem to be overly complex. I was hoping there would be some method that can wait for a group of threads to complete. Does anything like this exist?
Just join them one by one:
for (Thread thread : threads) {
thread.join();
}
(You'll need to do something with InterruptedException, and you may well want to provide a time-out in case things go wrong, but that's the basic idea...)
If you are using java 1.5 or higher, you can try CyclicBarrier. You can pass the cleanup operation as its constructor parameter, and just call barrier.await() on all threads when there is a need for cleanup.
Have you seen the Executor classes in java.util.concurrent? You could run your threads through an ExecutorService. It gives you a single object you can use to cancel the threads or wait for them to complete.
Define a utility method (or methods) yourself:
public static waitFor(Collection<? extends Thread) c) throws InterruptedException {
for(Thread t : c) t.join();
}
Or you may have an array
public static waitFor(Thread[] ts) throws InterruptedException {
waitFor(Arrays.asList(ts));
}
Alternatively you could look at using a CyclicBarrier in the java.util.concurrent library to implement an arbitrary rendezvous point between multiple threads.
If you control the creation of the Threads (submission to an ExecutorService) then it appears you can use an ExecutorCompletionService
see ExecutorCompletionService? Why do need one if we have invokeAll? for various answers there.
If you don't control thread creation, here is an approach that allows you to join the threads "one by one as they finish" (and know which one finishes first, etc.), inspired by the ruby ThreadWait class.
Basically by newing up "watching threads" which alert when the other threads terminate, you can know when the "next" thread out of many terminates.
You'd use it something like this:
JoinThreads join = new JoinThreads(threads);
for(int i = 0; i < threads.size(); i++) {
Thread justJoined = join.joinNextThread();
System.out.println("Done with a thread, just joined=" + justJoined);
}
And the source:
public static class JoinThreads {
java.util.concurrent.LinkedBlockingQueue<Thread> doneThreads =
new LinkedBlockingQueue<Thread>();
public JoinThreads(List<Thread> threads) {
for(Thread t : threads) {
final Thread joinThis = t;
new Thread(new Runnable() {
#Override
public void run() {
try {
joinThis.join();
doneThreads.add(joinThis);
}
catch (InterruptedException e) {
// "should" never get here, since we control this thread and don't call interrupt on it
}
}
}).start();
}
}
Thread joinNextThread() throws InterruptedException {
return doneThreads.take();
}
}
The nice part of this is that it works with generic Java threads, without modification, any thread can be joined. The caveat is it requires some extra thread creation. Also this particular implementation "leaves threads behind" if you don't call joinNextThread() the full number of times, and doesn't have an "close" method, etc. Comment here if you'd like a more polished version created. You could also use this same type of pattern with "Futures" instead of Thread objects, etc.
I am fairly new to Java threads, Runnable, and the like.
As such, I'm wondering why the following code does not catch an exception?
Runnable r = () -> {
try {
job.print();
} catch (Exception e) {
e.printStackTrace();
}
};
if (job.printDialog()) {
Thread t = new Thread(r);
r.start();
}
Reading Is there a way to make Runnable's run() throw an exception I gather that:
"... if your run() method is really the target of a Thread, there's no point in throwing an exception because it is unobservable; throwing an exception has the same effect as not throwing an exception (none)." (#erickson)
I should check for exceptions inside the Runnable.run() method.
Why is that? Any "simple" explanations on this matter are highly appreciated!
As you have found out already, there's nothing thats captured outside Runnable running state. Language specifications keep on changing. What might have an explanation today may have something else tomorrow. You might find the answer at Why cannot run() of Runnable throw checked Exceptions?, specifically as to why Callable was added to capture the results from Future.
You can answer this question from both, a logical and a programmatical point of view.
Let's start with the logical one:
You can think of a thread as an "independent executor of code" which is totally independent from other executors of code, except shared memory. The Runnable you have to pass on thread initialization "wraps" the code the thread has to execute. As there's no "higher instance" which could catch an exception, throwing exceptions inside those Runnables doesn't make a lot of sense.
Now the programmatical point of view. The following code snippet is similar to how the interface is defined inside the Java SE:
#FunctionalInterface
interface Runnable {
void run();
}
As the function #run() does NOT throw any checked exceptions, you're not allowed to throw checked exceptions inside #run(). Since Runnables have been designed for use in thread initialization only, this makes perfect sense. Callables, however, contain a throws-clause as they're mainly used in Executors which indeed are able to handle exceptions.
I'm working on a project that has a thread pool to which it submits tasks. Each task is a chain, so to speak. When the task executes, it does what it needs to do, then checks the result. Each of these tasks contains a map of results (which is just an enum) and additional tasks. These are called within the same thread and the cycle repeats until there are no more tasks, at which point it goes back up the chain, adding each result to a collection and returning that to the main thread. Q&D example:
public abstract class MyCallable implements Callable<MyResponse> {
private Map<ResponseEnum, List<MyCallable>> callbacks;
public List<MyResponse> call() {
List<MyResponse> resp = new ArrayList<MyResponse>();
try{
//Run the process method and collect the result
MyResponse response = process();
List<MyCallable> next = callbacks.get(response.getResult());
if (next != null && !next.isEmpty()){
//Run within same thread, return results
for (MyCallable m : next){
resp.addAll(m.call();
}
return resp;
} else {
//No more responses, pass them back up the chain
resp.add(response);
return list;
}
//Anything goes wrong, we catch it here and wrap it in a response
} catch (Exception e){
resp.add(new MyExceptionResponse(e));
return resp;
}
}
//Implemented by all child classes, does the actual work
public abstract MyResponse process() throws Exception;
Bear in mind that this is also a prototype that I have not yet really tested out, so I'm aware that this may not be perfect or necessarily completely feasible.
The concern I have is this: A task is added to the thread pool and begins execution. In the main thread, a Future is created and a .get(N, TimeUnit) is called on it to retrieve the result. What if that task times out? We get a TimeoutException. Now, within a try/catch block I could cancel the Future, but is there any way for me to cancel the Future and extract the results, at least as far as they go? Three tasks may have executed and returned results before the the fourth stalled out. The try/catch in MyCallable should return a result and push it back up the chain if there's an exception (Ie, InterruptedException when .cancel(true) is called), but is it possible for me to get that result?
Of course, if I'm going about this completely wrong in the first place, that would also be good to know. This is my first big foray into multithreading.
EDIT: Okay, with that in mind, a wrapper has been placed around the MyCallable class. The wrapper implements Callable and returns the collection. The collection is passed down the chain of MyCallable objects and the results added, so if the Future.get times out, we can retrieve the collection and get the partial results.
However, this brings up a potential race condition. If the current MyCallable being invoked is waiting for an external service, then the Future.cancel(true) operation will cause an InterruptedException within MyCallable. This is caught and the exception is wrapped in a response object and added to the collection. The thing is, if the main thread cancels the Future, synchronizes on the wrapper or the collection in the wrapper, and then gets the collection, will that create a race condition between the getting of the collection and the try/catch block in MyCallable adding the wrapped exception to the collection? Or will the main thread wait for the catching of the exception and then execute the next line?
At the point when you get your TimeoutException, the task submitted to the Executor Service is merrily going forth on its way: it is only your waiting which has received the exception. That presumably means that the result map is still being populated.
What you could do is is use a concurrent map and safely extract whatever results are present after the timeout has occurred.
I have couple of objects which implement the Runnable interface and I execute them in separate Threads. Essentially in the run() method of the Runnable object I do some network activities which includes call to methods that block while waiting for input (from the network). Note that I do not have any deliberate pauses i.e. Thread.sleep() calls. Any pause is caused by calls to methods that may block.
These Runnable objects are under the control of a GUI and hence the GUI interface and one function I wish to provide to the user is the ability to end the thread executing my Runnable objects however I'm not able to understand how to do this.
One obvious means is to call the Thread.interrupt() method of the Runnable objects Thread but how is this call to the Thread method propagated through to the Runnable object? For example I cannot use try-catch, catching InterruptedException in the Runnable object does not seem to be allowed; my IDE (netbeans) complains that InterruptedException is never thrown in the run() method.
My code is below, stripped for brevity.
The following lines are executed in the GUI code in the GUI thread:
digiSearch = new DigiSearch(hostIP,this);
digiSearchThread = new Thread(digiSearch);
digiSearchThread.start();
The following is my Runnable class and where I would like/need to capture the interruption of its executing thread.
public class DigiSearch implements Runnable {
private String networkAdapterIP;
private DigiList digiList;
public DigiSearch (String ipAddress, DigiList digiList){
networkAdapterIP = ipAddress;
this.digiList = digiList;
}
#Override
public void run() {
try{
/*
* Do some network and other activities here which includes calling some blocking methods. However I would like to interrupt the run() method if possible by calling Thread.interrupt()
*/
} catch (Exception ex){
digiList.digiListException(ex);
} catch (Throwable t){
System.out.println("Search thread interrupted");
}
}
}
Could someone enlighten me on how I can achieve this or perhaps resolve my misunderstanding of interrupting threads?
Do you have any blocking methods that throw IOException? If so, this is probably your InterruptedException placeholder. Many of these method were written before InterruptedException was introduced and so rather than update the interface which would break legacy code, they wrap the InterruptedException in an IOException.
If this is not the case you are kinda stuck. For example, if you write a Runnable that creates an infinit loop that just does work and never sleeps, interrupting this thread will not result in an InterruptedException. It is the responsibilily of the Runnable to regularly check Thread.interrupted().
Couple of points to note here:
1) While I agree it is useful to have a feature for user to stop execution of a thread, I recommend thinking about the action that the thread is already doing. Is it possible to rollback the action? Is it possible to ignore the action and just stop execution?
2) Thread.stop() and Thread.destroy() etc are deprecated methods (http://docs.oracle.com/javase/6/docs/api/)
So how does one normally interrupt thread execution? Enter volatile state variables.
public class MyClass implements Runnable {
private volatile boolean isAlive=true;
/**
* Request thread stop by calling requestThreadStop() externally.
*/
public void requestThreadStop() {
isAlive = false;
}
#Override
public void run() {
while(isAlive) {
//Do All your thread work
//if isAlive is modified, the next iteration will not happen
}
}
}
For many use cases, the above implementation works. However, if the work inside the run() method loop is only a single iteration and can block for significant amount of time, the user has to wait until the operation completes.
Is there a way to silently discard the execution of a thread almost immediately once the user requests for termination from the GUI? Maybe. You will have to explore using Threadpools for that. Using the ExecutorService, you can provide hooks to shutdown() and shutdownNow() methods.
To avoid repetition, you can find more about this feature of threadpools from this previoud stackoverflow post How to stop the execution of Executor ThreadPool in java?
I am writing a simple threading application. Thread is simply a message consumer and process it. However, if the thread somehow got interrupted and the message is not fully processed, I want to put it back to the queue and let other instances get it. So I had to code it like this:
public void run()
{
Map<String, String> data = null;
try
{
while(true)
{
data = q.getData();
System.out.println(this+" Processing data: "+data);
// let others process some data :)
synchronized(this){
sendEmail(data);
data = null;
}
}
}
catch (InterruptedException e)
{
System.out.println(this+" thread is shuting down...");
if(null!=data)
q.add(data);
}
}
Thanks...
EDIT: Thanks for the responses. Everything is very clear now. I understand that even when lines of codes are in a synchronized block, if any of them can throw InterruptedException then it simply means they can be interrupted at that point. The line q.getData() enters this thread to a 'blocked' state (I am using LinkedBlockedQueue inside the q.getData()). At that point, this thread can be interrupted.
A thread will not catch an InterruptedException any time another thread calls interrupt() on it, nor does that method magically stop whatever it's doing. Instead, the method sets a flag that the thread can read using interrupted(). Certain other methods will check for this flag and raise InterruptedException if it's set. For example, Thread.sleep() and many I/O operations which wait for an external resource throw it.
See the Java Thread Interrupts Tutorial for more information.
In addition to David Harkness's answer: you also don't understand meaning of synchronized keyword.
Synchornized is not a kind of "atomic" or "uninterruptable" block.
Synchornized block doesn't provide any guarantees other than that other threads can't enter synchronized block on the same object (this in your case) at the same time (+ some memory consistency guarantees irrelevant in your case).
Therefore usage of synchornized in your case is pointless, since there is no need to protect data from concurrent access of other threads (also, you are synchronizing on this, I don't think other threads would synchronize on the same object).
See also:
Synchronization
Ignoring for the moment that while(true) puts the thread into a CPU loop...
If sendMail does anything that checks for thread interruption it will throw an interrupted exception. So the answer to your question is likely to be a solid yes, the thread can be interrupted within the synchronized block, and you will have to catch the exception and check for that.
That said, InterruptedException is a checked exception, so short of funny buggers being done at a lower level, sendMail should indicate that it can throw InterruptedException.
Yes
Java synchronization means no other thread can access the same lock while a thread has acquired it.
If you don't want any other thread to be able to access a message (or any other object) use synchronized(message) block.