I've been trying to use Java's ProcessBuilder to launch an application in Linux that should run "long-term". The way this program runs is to launch a command (in this case, I am launching a media playback application), allow it to run, and check to ensure that it hasn't crashed. For instance, check to see if the PID is still active, and then relaunch the process, if it has died.
The problem I'm getting right now is that the PID remains alive in the system, but the GUI for the application hangs. I tried shifting the ProcessBuilder(cmd).start() into a separate thread, but that doesn't seem to be solving anything, as I hoped it would have.
Basically the result is that, to the user, the program APPEARS to have crashed, but killing the Java process that drives the ProcessBuilder.start() Process actually allows the created Process to resume its normal behavior. This means that something in the Java application is interfering with the spawned Process, but I have absolutely no idea what, at this point. (Hence why I tried separating it into another thread, which didn't seem to resolve anything)
If anyone has any input/thoughts, please let me know, as I can't for the life of me think of how to solve this problem.
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If the process writes to stderr or stdout, and you're not reading it - it will just "hang" , blocking when writing to stdout/err. Either redirect stdout/err to /dev/null using a shell or merge stdout/err with redirectErrorStream(true) and spawn another thread that reads from stdout of the process
You want the trick?
Don't start your process from ProcessBuilder.start(). Don't try to mess with stream redirection/consumption from Java (especially if you give no s**t about it ; )
Use ProcessBuilder.start() to start a little shell script that gobbles all the input/output streams.
Something like that:
#!/bin/bash
nohup $1 >/dev/null 2>error.log &
That is: if you don't care about stdout and still want to log stderr (do you?) to a file (error.log here).
If you don't even care about stderr, just redirect it to stdout:
#!/bin/bash
nohup $1 >/dev/null 2>1 &
And you call that tiny script from Java, giving it as an argument the name of the process you want to run.
If a process running on Linux that is redirecting both stdout and stderr to /dev/null still produce anything then you've got a broken, non-compliant, Linux install ;)
In other word: the above Just Works [TM] and get rid of the problematic "you need to consume the streams in this and that order bla bla bla Java-specific non-sense".
The thread running the process may block if it does not handle the output. This can be done by spawning a new thread that reads the output of the process.
final ProcessBuilder builder = new ProcessBuilder("script")
.redirectErrorStream(true)
.directory(workDirectory);
final Process process = builder.start();
final StringWriter writer = new StringWriter();
new Thread(new Runnable() {
public void run() {
IOUtils.copy(process.getInputStream(), writer);
}
}).start();
final int exitValue = process.waitFor();
final String processOutput = writer.toString();
Just stumbled on this after I had a similar issue. Agreeing with nos, you need to handle the output. I had something like this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
final Process process = myProc2.start();
and it was working great. The spawned process even did output some output but not much. When I started to output a lot more, it appeared my process wasn't even getting launched anymore. I updated to this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
myProc2.redirectErrorStream(true);
final Process process = myProc2.start();
InputStream myIS = process.getInputStream();
String tempOut = convertStreamToStr(myIS);
and it started working again. (Refer to this link for convertStreamToStr() code)
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If you don't read the output streams created by the process then it is possible that the application will block once the application's buffers are full. I've never seen this happen on Linux (although I'm not saying that it doesn't) but I have seen this exact problem on Windows. I think this is likely related.
JDK7 will have builtin support for subprocess I/O redirection:
http://download.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html
In the meantime, if you really want to discard stdout/stderr, it seems best (on Linux) to invoke ProcessBuilder on a command that looks like:
["/bin/bash", "-c", "exec YOUR_COMMAND_HERE >/dev/null 2>&1"]
Another solution is to start the process with Redirect.PIPE and close the InputStream like this:
ProcessBuilder builder = new ProcessBuilder(cmd);
builder.redirectOutput(Redirect.PIPE);
builder.redirectErrorStream(true); // redirect the SysErr to SysOut
Process proc = builder.start();
proc.getInputStream().close(); // this will close the pipe and the output will "flow"
proc.waitFor(); //wait
I tested this in Windows and Linux, and works!
In case you need to capture stdout and stderr and monitor the process then using Apache Commons Exec helped me a lot.
I believe the problem is the buffering pipe from Linux itself.
Try to use stdbuf with your executable
new ProcessBuilder().command("/usr/bin/stdbuf","-o0","*executable*","*arguments*");**
The -o0 says not to buffer the output.
The same goes to -i0 and -e0 if you want to unbuffer the input and error pipe.
you need to read the output before waiting to finish the cycle. You will not be notified If the output doesn't fill the buffer. If it does, it will wait until you read the output.
Suppose you have some errors or responses regarding your command which you are not reading. This would cause the application to stop and waitFor to wait forever. A simple way around is to re-direct the errors to the regular output.
I was spent 2 days on this issue.
public static void exeCuteCommand(String command) {
try {
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
ProcessBuilder builder = new ProcessBuilder();
if (isWindows) {
builder.command("cmd.exe", "/c", command);
} else {
builder.command("sh", "-c", command);
}
Process process = builder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
System.out.println("Cmd Response: " + line);
process.waitFor();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Related
I'm parsing the result of executing this composite command
ntpq -c peers | awk ' $0 ~ /^*/ {print $9}'
in order to obtain the offset of the active ntp server.
This is the java code used and executed periodically
public Double getClockOffset() {
Double localClockOffset = null;
try {
String[] cmd = {"/bin/sh",
"-c",
"ntpq -c peers | awk \' $0 ~ /^\\*/ {print $9}\'"};
Process p = Runtime.getRuntime().exec(cmd);
p.waitFor();
BufferedReader buf = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = buf.readLine();
if (!StringUtils.isEmpty(line)) {
localClockOffset = Double.parseDouble(line.trim());
} else {
// Log "NTP -> Empty line - No active servers - Unsynchronized"
}
} catch (Exception e) {
// Log exception
}
return localClockOffset;
}
ntpq result example
> remote refid st t when poll reach delay offset jitter
> ==============================================================================
> *server001s1 .LOCL. 1 u 33 64 377 0.111 -0.017 0.011
> +server002s1 10.30.10.6 2 u 42 64 377 0.106 -0.006 0.027
> +server003s1 10.30.10.6 2 u 13 64 377 0.120 -0.009 0.016
Notice that awk searchs the first line beginnig with '*' and extracts its ninth column. In the example: -0.017
The problem is that sometimes I'm obtaining the no-active-servers log message - intended to appear when there is no server with '*'- while the execution of the command through the console returns a number.
I know that I'm not closing the BufferedReader in that code but is that the reason of this behaviour? A new instance is being created (and left open until garbage collecting) in each method invocation but I think that it shouldn't be the cause of this problem.
Runtime.exec() simply invokes the ProcessBuilder inside it, like that:
public Process More ...exec(String[] cmdarray, String[] envp, File dir)
throws IOException {
return new ProcessBuilder(cmdarray)
.environment(envp)
.directory(dir)
.start();
}
see OpenJDK Runtime.java
So there is nothing wrong with using it instead of the ProcessBuilder as is.
The problem is that you invoke:
p.waitFor();
before you obtained the InputStream.
Which means that the process will be already terminated, by the time you obtain the InputStream, and the output stream data might be or might not be available to you, depending on the OS buffering implementation nuances and precise timing of the operations.
So, if you move the waitFor() to the bottom, your code should start working more reliably.
Under Linux however you should normally be able to read the remaining data from the PIPE buffer, even after the writing process has ended.
And the UNIXProcess implementation in OpenJDK, actually makes an explicit use of that, and tries to drain the remaining data, once the process has exited, so that file descriptor can be reclaimed:
/** Called by the process reaper thread when the process exits. */
synchronized void processExited() {
synchronized (closeLock) {
try {
InputStream in = this.in;
// this stream is closed if and only if: in == null
if (in != null) {
byte[] stragglers = drainInputStream(in);
in.close();
this.in = (stragglers == null) ?
ProcessBuilder.NullInputStream.INSTANCE :
new ByteArrayInputStream(stragglers);
}
} catch (IOException ignored) {}
}
}
And this seems to work reliable enough, at least in my tests, so it would be nice to know which specific version of Linux|Unix and JRE your are running.
Have you also considered the possibility of an application-level problem ?
I.e. ntpq is not really guaranteed to always return a * row.
So, it would be nice to remove the awk part from your pipe, to see if there will be some output at all the times.
Another thing to note is that if one of your shell pipeline steps fails (e.g. the ntpq itself), you will also get an empty output, so you will have to track the STDERR as well (e.g. by merging it with STDOUT via the ProcessBuilder).
Sidenote
Doing waitFor before you start consuming the data, is a bad idea in any case, as if your external process will produce enough output to fill the pipe buffer, it will just hang waiting for someone to read it, which will never happen, as your Java process will be locked in waitFor at the same time.
As pointed by Andrew Thompson, you shall try ProcessBuilder instead.
String[] cmd = {"/bin/sh",
"-c",
"ntpq -c peers | awk \' $0 ~ /^\\*/ {print $9}\'"};
ProcessBuilder pb = new ProcessBuilder(cmd);
pb.redirectErrorStream(true);
Process proc = pb.start();
BufferedReader buf = new BufferedReader(new
InputStreamReader(proc.getInputStream()));
String line = null;
while ((line = buf.readLine()) != null) {
localClockOffset = Double.parseDouble(line.trim());
break;
}
proc.destroy();
Ref ProcessBuilder
Finally we have found the real problem.
I'm not gonna change the accepted anwser, I think that it's useful too but maybe someone can learn from our experience.
My java program is launched with a shell script. When we execute the script manually, ntpq command is found and invoked successfully. The problem arises when the software is fully deployed. In the final environment we've got a cron scheduled demon that keeps our program alive but PATH established by cron is different from the PATH that our profile has got assigned.
PATH used by cron:
.:/usr/bin:/bin
PATH that we got login for launching the script manually:
/usr/sbin:/usr/bin:/bin:/sbin:/usr/lib:/usr/lib64:/local/users/nor:
/usr/local/bin:/usr/local/lib:.
Usually ntpq is in
/usr/sbin/ntpq
After we found the key of our problem, I search StackOverflow and got this relevant question where the problem is better explained and solved.
How to get CRON to call in the correct PATHs
I have a java restful service method which executes a myscript.sh using processBuilder. My script takes one input (example - myscript.sh /path/to-a/folder).
Inside the script something like this
-> execute a command which is multithreaded i.e parallel processing
-> echo "my message"
Now when call my script from a linux command line it executes fine. First all the threads running finishes and then some text output from threaded command execution shown on terminal and then echo my message is shown.
But when I call the same script from java using processBuilder, the last echo message comes immidiately and execution ends.
Following the way I call my script from java
ProcessBuilder processBuilder = new ProcessBuilder("/bin/bash","/path/to/myscript.sh","/path/to/folder/data");
Process proc = processBuilder.start();
StringBuffer output = new StringBuffer();
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
String line = "";
while((line = reader.readLine()) != null){
output.append(line + "\n");
}
System.out.println("### " + output);
I don't know whats happening, how to debug also.
Can someone enlighten me on how to get the same behaviour from shell script when run from terminal or from java processBuilder?
Use ProcessBuilder.redirectErrorStream(boolean redirectErrorStream) with argument true to merge the errors into output. Alternatively, you could also use the shell command syntax cmd 2>&1 to merge the error with output.
These are some of the cases why you may be immediately getting the output of the last echo statement (instead of the script taking time to run and return proper results):
Missing environment variables
The launched bash needs to source .bashrc or some such recource file
The launched bash may not be running in right directory (you can set this in ProcessBuilder)
The launched bash may not be finding some script/executable in its PATH
The launched bash may not be finding proper libraries in the path for any of the executables
Once you merge error, you would be able to debug and see the errors for yourself.
In your context, separate processes may be spawned in two ways:
1) Bash
/path/to/executables/executable &
This will spawn a new executable executable and you need to wait for it to finish. Here's an answer that will help you.
2) Java
Process exec = Runtime.getRuntime().exec(command);
status = exec.waitFor();
Essentially, you need to wait for the process to end before you start reading its std/err streams.
If I understand the problem correctly, adding just this line to your code should suffice: status = exec.waitFor() (Before you obtain the streams)
Here's the JavaDoc for Process.waitFor() :
Causes the current thread to wait, if necessary, until the process represented by this Process object has terminated. This method returns immediately if the subprocess has already terminated. If the subprocess has not yet terminated, the calling thread will be blocked until the subprocess exits.
Returns:
the exit value of the subprocess represented by this Process object. By convention, the value 0 indicates normal termination.
Throws:
InterruptedException - if the current thread is interrupted by another thread while it is waiting, then the wait is ended and an InterruptedException is thrown
I use the following to launch a Java application from another Java app.
ProcessBuilder pb = new ProcessBuilder(javaPath + javaCommand, maxMemStr,
minMemStr, stackSizeStr, jarCommand, jarfile, jarArg);
try {
Process p = pb.start();
} catch (IOException ex) {
Logger.getLogger(launch.class.getName()).log(Level.SEVERE, null, ex);
}
where javaCommand is either java or javaw (javaPath is empty most of the time unless a user points to an alternate path). The problem is, after the app launches, even when I verify the process list to contain java, it doesn't show the console.
Is it because PrcoessBuilder doesn't invoke the command shell? Is there a way to show the console programatically?
Thanks in advance.
This is because the "command console" itself is a process that attaches to the std-in/-out/-err streams of another process and displays them on the screen. When you launch Java all by itself, no other processes will be handling those streams, hence the lack of a command console. To get the results you want, you will need to launch a new instance of the command console and subsequently have it run your custom java command.
There may be a better way to do this... but I think the solution to this is going to be platform-dependent. In Windows, you could do something like:
ProcessBuilder pb = new ProcessBuilder("start", "\"JAwesomeSauce\"", "cmd.exe",
"/k", javaPath + javaCommand, maxMemStr, minMemStr, stackSizeStr, jarCommand,
jarfile, jarArg);
try {
Process p = pb.start();
} catch (IOException ex) {
Logger.getLogger(launch.class.getName()).log(Level.SEVERE, null, ex);
}
I assume you could do something similar in Linux/Mac if that's the O/S you're using.
You may want to run the command like this:
cmd /K java ...
or
cmd /C java ...
As far as I remember the Processbuilder opens a pipe to a specific process.
Your command window is a process itself with all you see. If you enter commands the cmd/bash usually creates new processes and attaches to them.
I need to kill an external process on windows (WindowsXP 32bit) from my integration test. I thought I'd just use 'taskkill.exe' but I cannot seem to get it working. Basically, every time I kick off a 'taskkill.exe' process from java it returns exit value -1073741515, nothing is printed to std error/output.
To reproduce the problem I wrote this simple application:
public static void main(String[] args) throws Exception {
ProcessBuilder builder = new ProcessBuilder();
//In my real code, I kill process by its pid. However below also shows the problem:
builder.command("taskkill.exe", "/?");
builder.redirectErrorStream(true);
Process p = builder.start();
BufferedReader r = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = r.readLine();
System.out.println("out:");
while(line != null) {
System.out.println(line);
line = r.readLine();
}
System.out.println(p.waitFor());
}
More data points:
-1073741515 apparently means "The application failed to initialize properly". Not very helpful for me though ;)
I've tried bunch of combinations of taskkill.exe parameters; I've tried prefixing the command with 'cmd', '/c'. Symptoms are exactly the same
I tried executing other windows programs that live under windows\system32 and I also get -10737...
Executing things like 'dir' or 'echo' works ok.
Any hints on what might be the problem?
Have you tried executing your application as a different user? If you're running your app with a plain batch file in windows, right click and select Run as administrator and see the results. It's likely the account you're running under doesn't have enough rights to execute native apps.
I'm writing an application with a Java GUI which calls some FORTRAN code. I want to return a file (solution.ps) which is updated and compiled based on changes in the FORTRAN code, which are created earlier in my ActionPerformed method. However the code I have at present just returns the old version of the file rather than waiting for the updated results of the cmd compilation. Is there a way to make the cmd wait for the process to run before completing the next step? (It works fine running directly from cmd)
I've searched but can't find anything except process.waitFor() which won't seem to pause the execution at the right point. Tried Thread.waitFor() too.
I'm thinking this could be useful for anyone who wants to send user inputs to another program and return a compiled result which uses these inputs.
Anyway here is the code, thanks in advance for any help and I hope I made the problem clear.
String[] command ={"cmd",};
try {
Process p = Runtime.getRuntime().exec(command);
new Thread(new SyncPipe(p.getErrorStream(), System.err)).start();
new Thread(new SyncPipe(p.getInputStream(), System.out)).start();
PrintWriter stdin = new PrintWriter(p.getOutputStream());
stdin.println("cd c:\\g77");
stdin.println("g77setup.bat");
stdin.println("cd c:\\users\\laurence\\workspace\\areaplanner");
stdin.println("g77 -O4 genpack.f -o genpack");
stdin.println("genpack");
stdin.println("5");
/*
* The following line sets the time to run the FORTRAN code for
* - need to wait for this to complete before calling mpost
*/
stdin.println("30");
stdin.println("mpost solution.mp");
stdin.println("latex solution.tex");
stdin.println("dvips solution.dvi -o solution.ps");
stdin.close();
} catch(IOException e4){}
You are only runnng the windows shell command. To fix, suggest writing the batch file first and wait for it to finish:
String command = "cmd /c mybatchfile.bat";
Process p = Runtime.getRuntime().exec(command);
p.waitFor();
To get another section to kick off before the the first set of commands have completed, you will have to write another batch file and repeat the above. Make sure you have both process then in separate threads.
Try using waitFor so as to make the current thread wait for the process to finish its job.
Process p = Runtime.getRuntime().exec(command);
p.waitFor()
The command in your code is incomplete. And also it is advisable to use a ProcessBuilder.start() instead of Process.