I have a Java microservice which shells out to execute a program and then monitors stderr until nothing is returned:
Process p = Runtime.getRuntime().exec("/bin/sh /var/task/bin/iTMSTransporter " + commandLine);
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
logger.log("StdErr:\n");
while ((s = stdError.readLine()) != null && p.isAlive()) {
System.out.println("stderr: " + s);
if (s.indexOf("DBG-X") == -1) {
result.stdErr += s + "\n";
}
}
this.logger.log("finished reading stderr\n");
This was effective at detecting the completion of the external program and it always worked. Now the external program has been updated and it seems to start outputting stderr but then just stops (there should be more to the stderr stream) and eventually it times out.
I then added the p.isAlive() as an attempt to capture the shelled program's completion. This seemed to have no impact. Now here's the frustrating part ... in an older version of my microservice I used NodeJS instead of Java to shell out to run this program. The NodeJS version still works by listening for the close event:
shell.stdout.on('data', data => {
stdout += data;
});
shell.stderr.on('data', data => {
stderr += data;
});
shell.on('error', error => {
reject(error);
});
shell.on('close', () => {
resolve({
stdout: stdout,
stderr: stderr
});
});
Is there something equivalent I can do with Java?
---- Addition -----
I tried something I'd seen online:
Runtime rt = Runtime.getRuntime();
rt.addShutdownHook(new Thread(new Runnable() {
public void run() {
System.out.println("\nGOT HERE\n");
}
}));
Process p = rt.exec("/bin/sh /var/task/bin/iTMSTransporter " + commandLine);
thinking I'd get it detect in a new thread and output "GOT HERE" but that never is sent to console.
If you just want to wait for the process to finish, waitFor should be appropriate.
Process p = Runtime.getRuntime().exec("/bin/sh /var/task/bin/iTMSTransporter " + commandLine);
try {
p.waitFor();
// now, the process has terminated
} catch (InterruptedException e) {
// something went wrong
e.printStackTrace();
}
If you do want to capture the out- and err-streams, that's actually somewhat complicated because you'd need to create new threads for that (not that that's a huge problem, but it's OTT if you just want to know that the process has finished).
Also maybe see process.waitFor() never returns.
Related
I'm trying to execute a visual basic script code in my java application using process builder. As script provided by the user might not finish its execution in time, I want to provide means to limit this execution time. In the following code, you can see my logic but it doesn't really do what it supposed to do. How can I make this waitfor work in order to limit the execution time?
private void run(String scriptFilePath) throws ScriptPluginException {
BufferedReader input = null;
BufferedReader error = null;
try {
ProcessBuilder p = new ProcessBuilder("cscript.exe", "//U", "\"" + scriptFilePath + "\"");
String path = "";
if (scriptFilePath.indexOf("/") != -1) {
path = scriptFilePath.substring(0, scriptFilePath.lastIndexOf("/"));
}
path += "/" + "tempvbsoutput.txt";
p.redirectOutput(new File(path));
Process pp = p.start();
try {
pp.waitFor(executionTimeout, TimeUnit.MINUTES);
} catch (InterruptedException e) {
SystemLog.writeError(jobId, ScriptConsts.COMPONENT_ID, "VBScriptExecutor", "run", 80401104,
"VB Script executes fail.");
}
if (!pp.isAlive()) {
pp.getOutputStream().close();
}
// rest of the code flow
}
Process.waitFor(long, TimeUnit) waits until the process has terminated or the specified time elapsed (Javadoc). The return value indicates whether the process exited or not.
if (process.waitFor(1, TimeUnit.MINUTES)) {
System.out.println("process exited");
} else {
System.out.println("process is still running");
}
waitFor() does not kill the process after the time elapsed.
If you want to kill the subprocess, use either destroy() or destroyForcibly().
I am trying to get the output of a program (written in C++) from Java by listening to the STDOUT/STDERR. When I run the C++ program through the console, I am able to properly see the expected output, and it works fine. However, when the program ends (I am basically looking for error codes), it spits out 3-4 lines almost immediately and then ends the output. When looking at the output from the console, this is fine. However, when listening to STDOUT from java, I am missing the final line from the output and lines seems to be "colliding", like this:
INFO: max_encoded_bytes = 76475 aERROR: RTMP_Connect0, failed to connect socket. 110 (Connection timed out).
When they should look like this:
INFO: max_encoded_bytes = 76475 at [line 939: program.cpp]
ERROR: RTMP_Connect0, failed to connect
FATAL: Error code 200
My code currently looks like this:
processBuilder.redirectErrorStream(true);
processLock.lock();
logger.debug("Starting " + name + " process");
try {
Process process = processBuilder.start();
InputStream is = process.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
Future<?> outputFuture = executor.submit(() -> {
br.lines().forEach(line -> {
outputHandler.handleOutput(line);
});
});
try {
process.waitFor();
outputFuture.get();
logger.debug(name + " process completed naturally");
} catch (ExecutionException e) {
logger.error("ExecutionException during " + name + " process: " + e.getMessage());
} catch (InterruptedException e) {
logger.debug(name + " process interrupted");
process.destroy();
} finally {
br.close();
isr.close();
is.close();
}
catch (IOException e) { } //etc
Which, based on me looking at resources on the web, seems to be pretty standard for processing output.
I was reviewing some other SO questions that seemed similar, but I wasn't exactly sure since their situations are not identical to mine (see here and here).
I have tried changing how I am reading from STDOUT through a whole bunch of different libs (removing BufferedStream, using InputStream directly, using ByteArrayOutputStream, changing buffer sizes to some absurdly large and small values, etc..). With the goal of matching the console output and printing out each line as it happens, what can I do to get the output that I think I should be getting?
My application uses some daemon subprocesses for subtasks. The subprocesses are launched using ProcessBuilder and working fine on their own, but then starting them as subprocesses every associated Process.isAlive() method return FALSE. As following, no access to process is possible.
Further investigation shows the subprocesses are not started at all (don't exist in Task Manager) with no error generated at all.
Daemons typically start a separate process and exit almost immediately, which makes checks like isAlive() useless.
Often the program will have a command line switch that make the program stay in the foreground, not becoming a daemon - use that if possible. Otherwise you'll need some other way of monitoring the daemon execution, for example using the daemon's PID file.
Is the command really running? Often there are weird little issues when trying to run a program from inside Java.
For example, the PATH environment variable may not be set correctly so it fails to load a dependency.
Use this method to see if there is any console output and what the exit code is. This uses the old Runtime class instead of ProcessBuilder. It can probably be adapted to use ProcessBuilder.
public static void runExe(String[] command) throws IOException {
Runtime runtime = Runtime.getRuntime();
long start = System.currentTimeMillis();
Process proc = runtime.exec(command);
BufferedReader br = new BufferedReader(new InputStreamReader(proc.getInputStream()));
try {
while (true) {
// enter a loop where we read what the program has to say and wait for it to finish
// read all the program has to say
while (br.ready()) {
String line = br.readLine();
System.out.println("CMD: " + line);
}
try {
int exitCode = proc.exitValue();
System.out.println("exit code: " + exitCode);
// if we get here then the process finished executing
break;
} catch (IllegalThreadStateException ex) {
// ignore
}
// wait 200ms and try again
Thread.sleep(200);
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println("Command took: " + (end - start) + "ms");
}
All,
I originally had a shell script that called SQLLoader (Oracles data upload tool).
The problem was that SQLLoader takes a plain text password as input so I decided to build a Java application to call SQLLoader internally passing a decrypted password into the command string.
e.g.
sqlldr user/pass#DBServer control=../sqlloader.ctl log=sqlloader.log data=mydata.csv
So with my java wrapper it became this in my shell script
java -jar sqlloader.jar sqlloader.ctl mydata.csv
However a new problem developed when SQLLoader complained there was no file to load. After some head scratching it was discovered that a subsequent command in my shell script seemed to be executing while my java application was still running. Therefore it was behaving asynchronously.
The next command was moving the input file sqlloader was using before it could get a chance to use it. So I put a sleep command in of 20 seconds to give my java application time to run.
java -jar sqlloader.jar sqlloader.ctl mydata.csv
echo $?
sleep 20
if [ $? -ne 0 ]
then
echo "SQLLoader failed during execution, please check the log : "
mv mydata.csv
else
echo "SQLLoader successfully processed file : "
mv mydata.csv
fi
Does anyone know why unix is behaving this way, does Java execute my SQLLoader as a different user/ thread?
This is my java code:
Runtime Rt;
Process Prc;
Prc = Rt.exec("sqlldr user/decryptedpass#DBServer control=../sqlloader.ctl log=sqlloader.log data=mydata.csv);
system.exit(0);
I checked the Runtime Class for anything about it being Asynchronous but couldnt find anything
http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html
Any theories or suggestions?
Thanks
Yes. If you look at Runtime.exec again it does specify that it will launch a new process in the specified environment (e.g. independently of the current "environment" or as you put it asynchronously). You should use ProcessBuilder to create a Process and then waitFor that Process to finish before calling System.exit - which certainly isn't mandatory. Something like this
public static void main(String[] args) {
// String command = "/usr/bin/sleep 5";
List<String> command = new ArrayList<String>();
command.add("c:/cygwin/bin/sleep");
command.add("5");
ProcessBuilder pb = new ProcessBuilder(command);
BufferedReader is = null;
try {
System.out.println("Starting command " + command);
Process p = pb.start();
int ret = p.waitFor();
is = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
while ((line = is.readLine()) != null) {
System.out.println(line);
}
if (ret == 0) {
System.out.println("Command has completed.");
System.exit(ret);
} else {
System.out.println("Command completed with return code " + ret);
System.exit(ret);
}
} catch (Exception e) {
System.out.println("Caught Exception " + e.getMessage()
+ " running command " + command);
e.printStackTrace();
} finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
}
}
}
System.out.println("COMMAND FAILED");
System.exit(1);
}
You need to wait for process completion, you should also read all output (stdout and stderr) from the process you are starting.
If you call exit() after exec(), Java will do just that - exit immediatedly.
Here is an article that explains Runtime.exec pitfalls: http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4 (also consider the other pages).
I am trying to run plink in my own console window. I started by using Process.exec() and that worked fine. The I moved to using ProcessBuilder and now the output is not sent out until I kill the process.
My code looks like this:
class ConsoleOutputThread extends Thread {
public void start(String processName) {
// this was old code: Runtime r = Runtime.getRuntime();
try {
builder = new ProcessBuilder("plink", "-ssh", "192.168.3.21");
builder.redirectErrorStream(true);
process = builder.start();
//this was old code: process = r.exec (processName);
} catch (IOException ex) {
}
start();
}
#Override
public void run() {
try {
BufferedReader is = new BufferedReader(new InputStreamReader(process.getInputStream()));
writer = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
try {
process.waitFor();
} catch (InterruptedException ex) {
}
char b[];
b = new char[1];
while(is.read(b, 0, 1)> 0) {
// this is for debug, normally sent to console
System.out.println("Got character: " + b[0]);
}
} catch (IOException ex) {
}
}
}
So, when using Runtime.exec() everything worked fine. Now, with ProcessBuilder, the read function blocks forever (actually until I kill the process, when everuthing is spitted out). However, the error stream works, i.e. if I put a bad option I get the messages in the console.
I am probably missing something here and looking for help.
Thank you
You've set the plink process to write its output to a pipe which is connected to the java process. Anything output by the plink process will be saved in an operating-system buffer until your process reads it. The OS buffer has a limited capacity,, and if plink writes too much data, then it will block until your process reads some data from the buffer.
Unfortunately, the java process waits for the plink process to complete before reading anything from the pipe. So, if the plink process writes too much output, it will block indefinitely.
You should change the java logic to read the plink process's output before calling waitfor().