I have a java restful service method which executes a myscript.sh using processBuilder. My script takes one input (example - myscript.sh /path/to-a/folder).
Inside the script something like this
-> execute a command which is multithreaded i.e parallel processing
-> echo "my message"
Now when call my script from a linux command line it executes fine. First all the threads running finishes and then some text output from threaded command execution shown on terminal and then echo my message is shown.
But when I call the same script from java using processBuilder, the last echo message comes immidiately and execution ends.
Following the way I call my script from java
ProcessBuilder processBuilder = new ProcessBuilder("/bin/bash","/path/to/myscript.sh","/path/to/folder/data");
Process proc = processBuilder.start();
StringBuffer output = new StringBuffer();
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
String line = "";
while((line = reader.readLine()) != null){
output.append(line + "\n");
}
System.out.println("### " + output);
I don't know whats happening, how to debug also.
Can someone enlighten me on how to get the same behaviour from shell script when run from terminal or from java processBuilder?
Use ProcessBuilder.redirectErrorStream(boolean redirectErrorStream) with argument true to merge the errors into output. Alternatively, you could also use the shell command syntax cmd 2>&1 to merge the error with output.
These are some of the cases why you may be immediately getting the output of the last echo statement (instead of the script taking time to run and return proper results):
Missing environment variables
The launched bash needs to source .bashrc or some such recource file
The launched bash may not be running in right directory (you can set this in ProcessBuilder)
The launched bash may not be finding some script/executable in its PATH
The launched bash may not be finding proper libraries in the path for any of the executables
Once you merge error, you would be able to debug and see the errors for yourself.
In your context, separate processes may be spawned in two ways:
1) Bash
/path/to/executables/executable &
This will spawn a new executable executable and you need to wait for it to finish. Here's an answer that will help you.
2) Java
Process exec = Runtime.getRuntime().exec(command);
status = exec.waitFor();
Essentially, you need to wait for the process to end before you start reading its std/err streams.
If I understand the problem correctly, adding just this line to your code should suffice: status = exec.waitFor() (Before you obtain the streams)
Here's the JavaDoc for Process.waitFor() :
Causes the current thread to wait, if necessary, until the process represented by this Process object has terminated. This method returns immediately if the subprocess has already terminated. If the subprocess has not yet terminated, the calling thread will be blocked until the subprocess exits.
Returns:
the exit value of the subprocess represented by this Process object. By convention, the value 0 indicates normal termination.
Throws:
InterruptedException - if the current thread is interrupted by another thread while it is waiting, then the wait is ended and an InterruptedException is thrown
This is my code :
Process p1;
try {
p1 = Runtime.getRuntime().exec( "utils/a.out < utils/test_c2.txt > utils/result.txt" );
p1.waitFor();
} catch ( Exception e ) {
System.out.println("Something went bad!");
}
I've read that there should be a problem with the input buffer size, but in this case, all the output from the launched process is redirected to " utils/result.txt", so the launched process should not reach deadlock. When I run the same command from terminal it works. Maybe it would be helpful to describe what is "a.out". I obtained it from a flex file as follows:
$ flex rulex.lex
$ gcc lex.yy.c -lfl
Any help would be appreciated.
The subprocess is waiting to read data on stdin. Java does not launch the subprocess within a shell, so no pipes are available. You need to make your subprogram take files as arguments and open the files itself. Another option is to start a shell (like bash) and tell it to run the program, then the piping of files will work.
In my java program I am trying to run a different program through CMD with its output appearing in a command window in the foreground and then analyze the exit code of the child program (foo) in the main java program. Unfortunately, all I seem to be able to access is the exit code of the CMD window, which is always 0.
The following is a snippet of what I'm doing:
ProcessBuilder pb = new ProcessBuilder();
pb.directory(new File(dir));
pb.command("cmd","/c","start","/wait","foo.exe",arg);
process = pb.start();
exitVal = process.waitFor();
but exitVal is always 0 regardless of how foo exits. How can I get just the exit code of foo?
I'm also pretty new to java so if there's a more elegant way of doing this, I'm open to suggestions.
I found a solution by modifying one of the things I had tried before to account for windows batch being finicky. The solution was to send another command to cmd to tell it to exit with the most recent error code. Earlier I had tried this by appending & exit %errorlevel% to the command but cmd variables are only updated at the end of each command line, not at the end of each command.
To force update, I used %^errorlevel% instead.
I am trying to make a java program that automatically converts wtv files in an input folder to mpg files in output folder. The twist is that I make it run periodically, so it acts as a synchronizer.
The following code works for converting the .wtv to a .dvr-ms, which is required by ffmpeg since it cannot convert .wtv files directly.
Process p = Runtime.getRuntime().exec("C:\\Windows\\ehome\\WTVConverter C:\\Users\\Andrew\\Desktop\\test\\input\\input.wtv C:\\Users\\Andrew\\Desktop\\test\\output\\input.dvr-ms");
p.waitFor();
WTVConverter has no problems running from a java application. ffmpeg is a different story. Once the above line runs, I then run this...
Process p = Runtime.getRuntime().exec("ffmpeg\\bin\\ffmpeg -y -i \"C:\\Users\\Andrew\\Desktop\\test\\output\\input.dvr-ms'" -vcodec copy -acodec copy -f dvd \"C:\Users\Andrew\Desktop\test\output\input.mpg\"");
p.waitFor();
Suddenly, there is a problem... The application ffmpeg shows up in the task manager, but it's cpu usage is 0, and no mpeg files is being generated. If I force the java application to close, though, suddenly it starts working! Huh?
What reason would there be for a command line application to wait for its calling application to quit before it executes? I'm not incredibly command line savvy, so I don't really know how to diagnose this problem.
Bah, this always happens. I post a question, and THEN I figure it out on my own. Turns out, ffmpeg expects you to read in its text output before it loads each frame. If a calling program does not do this, it simply waits. If there is no calling program, I assume that it just outputs it nowhere. What I did is ran the program as usual, but also read in text from the process's input stream like so...
Process p = Runtime.getRuntime.exec(".....");
final Scanner in = new Scanner(p.getInputStream());
new Thread()
{
#Override
public void run()
{
System.out.println(in.nextLine());
}
}.start();
Lesson learned, I guess.
I've been trying to use Java's ProcessBuilder to launch an application in Linux that should run "long-term". The way this program runs is to launch a command (in this case, I am launching a media playback application), allow it to run, and check to ensure that it hasn't crashed. For instance, check to see if the PID is still active, and then relaunch the process, if it has died.
The problem I'm getting right now is that the PID remains alive in the system, but the GUI for the application hangs. I tried shifting the ProcessBuilder(cmd).start() into a separate thread, but that doesn't seem to be solving anything, as I hoped it would have.
Basically the result is that, to the user, the program APPEARS to have crashed, but killing the Java process that drives the ProcessBuilder.start() Process actually allows the created Process to resume its normal behavior. This means that something in the Java application is interfering with the spawned Process, but I have absolutely no idea what, at this point. (Hence why I tried separating it into another thread, which didn't seem to resolve anything)
If anyone has any input/thoughts, please let me know, as I can't for the life of me think of how to solve this problem.
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If the process writes to stderr or stdout, and you're not reading it - it will just "hang" , blocking when writing to stdout/err. Either redirect stdout/err to /dev/null using a shell or merge stdout/err with redirectErrorStream(true) and spawn another thread that reads from stdout of the process
You want the trick?
Don't start your process from ProcessBuilder.start(). Don't try to mess with stream redirection/consumption from Java (especially if you give no s**t about it ; )
Use ProcessBuilder.start() to start a little shell script that gobbles all the input/output streams.
Something like that:
#!/bin/bash
nohup $1 >/dev/null 2>error.log &
That is: if you don't care about stdout and still want to log stderr (do you?) to a file (error.log here).
If you don't even care about stderr, just redirect it to stdout:
#!/bin/bash
nohup $1 >/dev/null 2>1 &
And you call that tiny script from Java, giving it as an argument the name of the process you want to run.
If a process running on Linux that is redirecting both stdout and stderr to /dev/null still produce anything then you've got a broken, non-compliant, Linux install ;)
In other word: the above Just Works [TM] and get rid of the problematic "you need to consume the streams in this and that order bla bla bla Java-specific non-sense".
The thread running the process may block if it does not handle the output. This can be done by spawning a new thread that reads the output of the process.
final ProcessBuilder builder = new ProcessBuilder("script")
.redirectErrorStream(true)
.directory(workDirectory);
final Process process = builder.start();
final StringWriter writer = new StringWriter();
new Thread(new Runnable() {
public void run() {
IOUtils.copy(process.getInputStream(), writer);
}
}).start();
final int exitValue = process.waitFor();
final String processOutput = writer.toString();
Just stumbled on this after I had a similar issue. Agreeing with nos, you need to handle the output. I had something like this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
final Process process = myProc2.start();
and it was working great. The spawned process even did output some output but not much. When I started to output a lot more, it appeared my process wasn't even getting launched anymore. I updated to this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
myProc2.redirectErrorStream(true);
final Process process = myProc2.start();
InputStream myIS = process.getInputStream();
String tempOut = convertStreamToStr(myIS);
and it started working again. (Refer to this link for convertStreamToStr() code)
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If you don't read the output streams created by the process then it is possible that the application will block once the application's buffers are full. I've never seen this happen on Linux (although I'm not saying that it doesn't) but I have seen this exact problem on Windows. I think this is likely related.
JDK7 will have builtin support for subprocess I/O redirection:
http://download.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html
In the meantime, if you really want to discard stdout/stderr, it seems best (on Linux) to invoke ProcessBuilder on a command that looks like:
["/bin/bash", "-c", "exec YOUR_COMMAND_HERE >/dev/null 2>&1"]
Another solution is to start the process with Redirect.PIPE and close the InputStream like this:
ProcessBuilder builder = new ProcessBuilder(cmd);
builder.redirectOutput(Redirect.PIPE);
builder.redirectErrorStream(true); // redirect the SysErr to SysOut
Process proc = builder.start();
proc.getInputStream().close(); // this will close the pipe and the output will "flow"
proc.waitFor(); //wait
I tested this in Windows and Linux, and works!
In case you need to capture stdout and stderr and monitor the process then using Apache Commons Exec helped me a lot.
I believe the problem is the buffering pipe from Linux itself.
Try to use stdbuf with your executable
new ProcessBuilder().command("/usr/bin/stdbuf","-o0","*executable*","*arguments*");**
The -o0 says not to buffer the output.
The same goes to -i0 and -e0 if you want to unbuffer the input and error pipe.
you need to read the output before waiting to finish the cycle. You will not be notified If the output doesn't fill the buffer. If it does, it will wait until you read the output.
Suppose you have some errors or responses regarding your command which you are not reading. This would cause the application to stop and waitFor to wait forever. A simple way around is to re-direct the errors to the regular output.
I was spent 2 days on this issue.
public static void exeCuteCommand(String command) {
try {
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
ProcessBuilder builder = new ProcessBuilder();
if (isWindows) {
builder.command("cmd.exe", "/c", command);
} else {
builder.command("sh", "-c", command);
}
Process process = builder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
System.out.println("Cmd Response: " + line);
process.waitFor();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}