For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
Sometimes some of the scripts freeze up and I have no way of knowing. So i've been thinking of a way to have the threads check on the scripts every once in a while so I can be notified when a script is hung up so that i can start figuring out why they are hanging up and fix the problem.
For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
The only way that I know of to do this is if the script itself output some sort of status message every X seconds to standard-out or error and the thread that spawned the script was actively reading, waiting for the output. It then could update status information about the process.
If you use the ProcessBuilder and call start() to get a Process, then you can attach a BufferedReader to the process.getOutputStream() to monitor the script. You certainly can also call process.exitValue() to see if the process has finished but that won't tell you if it is hung.
Alternatively would be for the script to somehow call back to the monitoring process via a socket or some other IPC but I think just monitoring the output-stream is the best approach.
Actually, as I posted in a comment, knowing if a program is "hung-up" is an undecidable (can't be solved) problem called The halting problem. So it is not an option to verify on this.
Some alternate solutions would be to check if the script is still running by calling the isAlive() method on the Thread object that is running the script, or, as was told in other answers, to check for some output that the script might be giving, and interpret it. But by verifying output unfortunately you cannot be sure that the program is "hung-up", you can only know it's current state.
EDIT: If you want to wait a particular time, then you can use the Thread.sleep(long millis) call on the parent, and when it wakes up, check who's alive using, again isAlive(). But this doesn't guarantee either that the program will actually finish
Related
I'm having the strangest problem: maybe someone can help out.
I have a daemon written in Java (I've tried the latest Java 8 both OpenJDK and Oracle) on Linux. It will spawn some processes and there is a thread which watches to see if they're complete, meanwhile it will run other commands, some of which may kill the processes, etc.
Everything works absolutely fine almost all the time. However, one of the operations that the daemon can perform on its subprocesses is it can grab a core from them (without killing them: a core of the active process). It does this by loading a .so and calling it via JNI, which uses ptrace(PTRACE_ATTACH...) to pause the subprocess and grab core information, then it uses ptrace(PTRACE_DETACH...) to let it run again.
This also works fine and I can see the process operates correctly after the core has completed and all is fine.
Except, Java's Process object is now all wonky. Whenever I invoke exitValue() on this process once this has happened I always get back and exit code of 4991 (which if you decode it, means WIFSTOPPED with a signal value of 19, or SIGSTOP). That's not entirely surprising since the Linux man page states that ptrace() will look to parents as WIFSTOPPED. But the problem is that once the Process object reaches this state, it will never do anything else, even though the process actually goes back to running after the detach.
Even more critically, even after the process really exits, exitValue() still returns 4991, and the process is not reaped (it lives on in Z (zombie) state, as defunct) and will never go away, until my daemon is killed whereupon init will inherit, and reap, the zombie processes.
I've looked at the value of isAlive() (says no) and I've tried to run waitFor(0, TimeUnit.SECONDS) (says not running) but none of them get Process out of this broken state--this doesn't surprise me too much since my reading suggests these methods are actually implemented in terms of the exitValue() method anyway.
This appears to me like a bug in Java Process but I'm not sure... I've tried reproducing it by attaching GDB to the process then detaching it, but that doesn't show the same problem. Similarly just using SIGSTOP / SIGCONT doesn't do it.
Anyone have any hints or thoughts?
I m using windows 7 OS. I have around 6 threads in my application. For the purpose of testing the alerts to check the health of the threads, i need to kill the threads manually and check if the alerts are working properly. Can we kill a thread like how we kill a process with its pid?
Dan Woods documented how to kill a thread in this blog entry...
https://web.archive.org/web/20160302023213/http://www.rhcedan.com/2010/06/22/killing-a-java-thread
The steps he performed involved using a debugger (JDB) and injecting an exception in the thread's execution. Specifically...
Ensure that your java program is started with the following parameters:
-Dcom.sun.management.jmxremote.port=50199
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Xrunjdwp:transport=dt_socket,address=50100,server=y,suspend=n
This will allow us to attach the java debugger to the running process, after
we identify which Thread is causing the problem. Also, make sure that
you have your iptables setup appropriately so as to only allow
connections on 50100 and 50199 from the hosts/workstations that you manage.
Identify the offending thread:
Kill the thread. In this example, the ThreadName is “btpool0-0?. Fire up the java debugger (also shipped with the JDK distribution), and attach to the running JVM…
[root#host ~]# jdb -attach 50100
Get a list of the running threads — this will also give us the thread id as the JVM sees it:
> threads
--snip--
(org.mortbay.thread.BoundedThreadPool$PoolThread)0x25cb
btpool0-0 running
--snip--
The thread id that we’re going to kill is “0x25cb”. The first step of killing the thread is to jump into it, and suspend it…
thread 0x25cb
btpool0-0[1] suspend 0x25cb
btpool0-0[1] step
Step completed: <... snip ...>
btpool0-0[1] kill 0x25cb new java.lang.Exception()
killing thread: btpool0-0
btpool0-0[1] instance of
com.site.package.name(name='btpool0-0', id=9675) killed btpool0-0[1]
Exit the java debugger, and you’re done!
There is no safe way to "kill" a thread without killing the process it is in. It not something you would do deliberately. For testing purposes I would add code to your application to support this.
It's not true. You can always attach to the JVM process with GDB and do a call pthread_kill if you know the thread id. You only need to translate from the java thread dump (do a kill -3) which gives you a hex id, (native id), then look into the list of threads in GDB (info threads) and locate the real thread id.
This is proven to work.
As Peter says, you can't do this safely. Indeed on some platforms Thread.kill is not even implemented. However:
If this is just for testing, a unit test that called Thread.kill would be reasonable ... assuming it worked on the test platforms where it needed to work. (A "loud" comment in the source code would be in order to help people porting the unit test ...)
Another alternative is to add some code to the thread runnable that allows your unit tests to tell it to die. If the thread code needs to be (almost) production code for this to work, you could create a subclass that overrides something so that it "breaks" in a way that suits your purposes ... for testing. In fact, this approach allows you to cause the threads "break" in controlled ways, potentially allowing you to test different aspects of your alerting code.
You can't do it from outside (OS or debugger), you'll have to write your own Thread watchdog that can interact with the user and kill the thread you want.
Try to look here for how to handle signals with java
In java you can not kill the like unix . Either you can interrupt the tread in java or you can kill the process in unix .
Wait for some time in the thread and kill the thread in the code - simple way.
As mentioned in a previous post by buzz3791, it works by using jdb. However the change that I noticed is, you can't kill the thread, but you can interrupt or suspend the thread.
#jdb -attach 50100
threads --This will show all threads running on the jvm under Groups and Reference Handler section.
Groups
Reference Handler
:(com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary)0x36c1:
OrientDB (/xxx.xxx.xxx.xxx:123) <- BinaryClient (/xxx.xxx.xxx.xxx:678)
thread 0x36c1-- This will be the thread id that can be picked from one of the threads in the thread that you wish to kill/interrupt and run this interrupt 0x36c1
OrientDB (/xxx.xxx.xxx.xxx:123) <- BinaryClient (/xxx.xxx.xxx.xxx:678)[1] interrupt 0x36c1
you can try multiple times the same interrupt command and if it is already interrupted, it will show that the thread id is invalid. Thus you know that the thread is killed, this can be verified by looking at the stack trace and confirmed.
Tested this on the OrientDB database server with jdk 8 and it works.
Unix daemon runs a script a loop, the script calls a java program: java {java_args} myClas.jar
The java program is heavy program with multiple threads.
The problem is very strange: First execution works as expected.
But the second execution is stuck some where and I can't find a reason (very hard to debug this).
Is there a possibility that when first execution is finished there are still not-cleaned resources or threads left from this execution?
If yes, is it possible to clean and kill everything right after process completes?
If by resources, you mean threads, then no. When the VM shuts down, everything on the heap, all threads, objects and monitors are disposed of. However if you're depending on the existence/absence of a file for locking or something similar, a deadlock is possible. Also, is it possible that the first process is still running when you launch the second one?
If your java process is stuck on the second run, you can attach jvisualvm to it and should be able to figure out where it's stuck.
It is possible to kill a thread that is in state RUNNING in a non programatically way?
I know that top command in *nix can show threads. Can I kill the thread in the OS?
I'd like to know if there is a way to link a thread to a process so I can kill only that specific thread and not the application.
We had a bug in our code that kept a thread in state RUNNING in a synchronized method. The thread kept the lock on the object "hanging" the application.
The bug is fixed. But I wonder if is possible.
The short answer is "maybe, but you should not and most of the time it won't work either".
The long answer is:
"Maybe..."
Some JVM implementation map java threads to OS threads and some do not. If the JVM does a mapping to a native OS thread, you might be able to kill that thread with some process tool that the OS provides (like kill on *nix). If the JVM does green threads, meaning it doesn't map a Java thread to an OS level thread, then you are basically out of luck using OS level tools. Luckily only very few JVM implementations do this. An approach that can be used regardless in which way the JVM organizes it's threads, is using the java debugger. This article describes the procedure of doing it: http://www.rhcedan.com/2010/06/22/killing-a-java-thread/.
"but you should not do it"
Killing a thread on the OS level will almost certainly leave the JVM in an undefined state (read "jvm might crash or delete all files on your disk or do whatever it fricking pleases to do"). Even when going the debugger way, only a very small amount of java applications (read "no application made on this planet") will properly handle the event that an outside application is killing one of it's threads. As a result these applications will be put in an undefined state (read "application might crash or delete all files on your disk or do whatever it fricking pleases to do").
"and most of the time it won't work either"
If the thread is really stuck with some blocked IO etc, then killing the thread won't work, it will just not respond. If a program is stuck it's probably better to kill the whole program, find the issue with the program and fix it instead of killing a single thread.
For all your doubts on killing a thread, refer this:
http://download.oracle.com/javase/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html
On linux, there is a tkill(int tid, int sig) command, similar to kill.
On windows, ProcessExplorer can do it from gui, don't know if there is anything with cli.
guys, I encounter a confusing issue! I want to invoke a executable JAR to compute PI from my main java applicaion using runtime.exec(), which create a new JVM for running the executable JAR. My issue is that when the PI computation is done, the JVM is still alive. my main java application has no idea whether PI computation is finished or not. And I also want when the PI computation is done, the JVM could be shutdown! How can I implement that! ThankS!!
When you call Runtime.exec() you will get a Process object back. You need to call waitFor() on this.
You will also need to capture the stdout/stderr streams (in separate threads to prevent blocking - see this answer for more info).
This all leaves aside why you're doing this in a separate JVM, and why you can't load the relevant classes into your current app and run the library locally.
To answer the second part of your question, the JVM always exits when no non-daemon threads are still running. Or in plainer speech, when your application is "done" and the main method exits without leaving any threads running in the background, the JVM will finish.
This is true regardless of whether you launch the Java process yourself from your desktop/command line, or fire it off via Runtime.exec() (which is broadly equivalent). So when your Pi calculation terminates it will shut down the JVM you spawned, and when your original program finishes then its JVM will also exit.
Though I agree completely with Brian here, I can't see the benefit in running a Java app as a separate process when you should be able to just run it in the original JVM (barring some really unusual environmental stuff such as setting niceness or processor affinity of the various processes).
check what the jar does when you run it standalone, maybe its waiting for input and therefor it will never exit