I have created an automated test suite which has a thread pool running simultaneously in the background of all test cases in order to obtain given system and performance metrics. Each thread is using a JSch connection to execute its shell commands and they are receiving [JSchException: Channel not opened exceptions].
The key problem is that the test suite continues to run forever, because the threads are not exiting even when all test cases have ended. But I'm not sure why...
When I checked the thread dump, I found that the threads do not exit because they are in a BLOCKED status.
Does anybody have an explanation for this? Or some help in resolving this issue?
I guess you mean the shutdownNow() method of ExecutorService. This is the documentation (I formatted the relevant parts in bold):
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
This method does not wait for actively executing tasks to terminate. Use awaitTermination to do that.
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
Thread interruption is a cooperative process, your threads must be implemented so that they respond to interruption.
EDIT
1) About Thread.currentThread.interrupt() see this question: Why invoke Thread.currentThread.interrupt() when catch any InterruptException?
So you need to do this only if you have other methods/thread groups watching the interrupted status. In your case, probably not, but it does not hurt.
2) It seems that in "ShellUtils.executeCommand" you are running external programs. This could be the source of the problem if this method does not react to thread interruptions.
Related
When do I actually need call this method Runtime.getRuntime().addShutdownHook() and when or why I need to shutdown my application. Could anyone please explain me this by giving some example.
Thanks
As far as I know, I will explain this below. You can google it and find lot of information too.
addShutdownHook() will register some actions which is to be performed on a Program's termination. The program that you start ends in two ways:
the main thread (Root) ends its running context;
the program meets some unexpected situation, so it cannot proceed further.
If you add a ShutdownHook, the hook will start a thread that will start running at time of termination only. For example:
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
System.out.println("Running Shutdown Hook");
}
});
will print a Running Shutdown Hook at the time of program termination at any point. You might even call a System.exit(0).
For examples, you can google, there are enough of them. And the question 'When should you use this' is like asking 'What does catch do in a try-catch statement'.
You might have many situations like:
your program had created many temporary files in filesystem you want to delete it;
you need to send a distress signal to another process/machine before terminating;
execute any clean-up actions, logging or after-error actions on unexpected behaviours.
All this will be needed for some point of time.
For examples you can go in here Example 1 or Example 2
You only worry about shutdown hooks when you want something to happen when a shutdown occurs on the virtual machine.
From Javadoc:
The Java virtual machine shuts down in response to two kinds of
events:
The program exits normally, when the last non-daemon thread exits or when the exit (equivalently, System.exit) method is invoked, or
The virtual machine is terminated in response to a user interrupt, such as typing ^C, or a system-wide event, such as user logoff or
system shutdown.
Thus, a shutdown hook is a initialized and unstarted thread that gets executed when a JVM shutdown occurs.
Popular examples of shutdown hooks exists in application servers (such as JBoss AS). When you press Ctrl+C, the JVM calls all the Runtime shutdown hooks registered (such as JBoss shutdown hooks) before exiting.
One case is, If you any daemon threads which needs to be stopped before your jvm shutdown (or) any other backend threads (mostly daemon threads) need to be gracefully exited, you will write shutdown hook and execute it using above code. Here is interesting discussion we had on SO couple of days ago. Shutdown hook
Sometimes the system process of a java application doesn't fully shutdown. This is mainly because one or more threads didn't die.
One can call:
System.exit(0);
But this kills all threads.
I want all my threads to shutdown correctly so i don't plan on using System.exit. But sometimes due to circumstances one or more threads don't die correctly. This causes the application to not fully shutdown and linger in the background sometimes even locking up resources.
My first thought was writing an ExitWatcher which starts when program stops and calls system.exit after 10 seconds if the program is still alive. I quickly found out that the ExitWatcher now prevents the program from shutting down ;-)
How do you give a java program time to shutdown correctly (ie close all threads correctly) but at the same time have a backup plan that calls System.exit if shutdown takes too long?
You can use daemon threads when you don't want a thread to prevent a program from exiting (so using setDaemon(true) on your ExitWatcher thread would solve your immediate problem). However I doubt you really need or want an ExitWatcher, as situations like that are usually relatively easily prevented with proper design.
Proper shutdown procedures include interrupting threads that are active (and designing them so they behave nicely when interrupted), using daemon threads where necessary, shutting down connections properly etc. Using a mechanism like your ExitWatcher is more of a hackish approach, as it indicates your program doesn't always behave nicely at shutdown.
For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
Sometimes some of the scripts freeze up and I have no way of knowing. So i've been thinking of a way to have the threads check on the scripts every once in a while so I can be notified when a script is hung up so that i can start figuring out why they are hanging up and fix the problem.
For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
The only way that I know of to do this is if the script itself output some sort of status message every X seconds to standard-out or error and the thread that spawned the script was actively reading, waiting for the output. It then could update status information about the process.
If you use the ProcessBuilder and call start() to get a Process, then you can attach a BufferedReader to the process.getOutputStream() to monitor the script. You certainly can also call process.exitValue() to see if the process has finished but that won't tell you if it is hung.
Alternatively would be for the script to somehow call back to the monitoring process via a socket or some other IPC but I think just monitoring the output-stream is the best approach.
Actually, as I posted in a comment, knowing if a program is "hung-up" is an undecidable (can't be solved) problem called The halting problem. So it is not an option to verify on this.
Some alternate solutions would be to check if the script is still running by calling the isAlive() method on the Thread object that is running the script, or, as was told in other answers, to check for some output that the script might be giving, and interpret it. But by verifying output unfortunately you cannot be sure that the program is "hung-up", you can only know it's current state.
EDIT: If you want to wait a particular time, then you can use the Thread.sleep(long millis) call on the parent, and when it wakes up, check who's alive using, again isAlive(). But this doesn't guarantee either that the program will actually finish
It seems that daemon threads are always better - because they will be stopped by the VM after application main thread exits. Are there any other reasons to use non-daemon threads besides the cases when it is not possible to interrupt some operation ?
Thanks.
When you are writing a server (e.g. a servlet container), all your main has to do is to bootstrap and start HTTP listener threads, accepting threads, file system scanning threads, RMI threads, etc.
After bootstrap is done, main is no longer needed as everything happens asynchronously. In this case all essential threads are non-daemon as they have to live past the main method.
Even in Swing (desktop programming) the only requirement on main is to initialize the main window (JFrame). The rest happens in Swing listener threads (EDT) and various background threads.
In fact, any thread that should finish naturally (leaving its "run" method) should not be a daemon thread, as you don't want the JVM to terminate while they are doing their job.
This applies to every thread that you launch, and that you expect to terminate naturally.
As a rule of thumb, daemon threads are the exception, not the rule.
The one major difference is in how daemon threads exit; the JVM just stops them.
finally blocks are not executed
stacks are not unwound
You do not want to use daemon threads for things that need to be left in a known state such as file i/o, database transactions, etc.
You use non-daemon thread whenever you are doing something which you don't want to stop because an another thread exits. If the only purpose of a thread is to support other threads, then it makes sense to use a daemon thread.
If you want to exit when the main thread exits you can call System.exit(). Many applications don't keep their main thread and all the work is in threads it starts.
The VM may stop daemon threads in the middle of their executions, and persistent data could be corrupted because of that.
If you need more control of when a thread is safe to die, do not make it a daemon.
I'm writing a single threaded program, which sometimes calls blocking methods (ie process.waitFor).
Can I be sure that the interrupted status of my thread is never set? Are there other conditions under which the JVM or the JRE standard library might decide to set the interrupt flag on my thread?
It seems true, but I couldn't find any mention about it in the Java docs.
If your application does not create any "Java" (java.lang.Thread) threads then I believe you can safely assume that your main thread will never be interrupted, see this dW article by Brian Goetz. Of course it is always possible that some library you're using could create threads and could potentially call interrupt() on your main thread, but it isn't likely.
In the end, I think you need to consider your requirements. The main purpose of the interrupt support in threads is so that tasks (threads) can be cancelled and so that applications can shutdown cleanly, even if some threads are suspended in blocking calls.