I want to be able to run an external program concurrently with my Java code, i.e. I want to start the program, then return control to the calling method while keeping the external program running at the same time. The Java code will then keep generating input and send it to the external program and receive output back.
I don't want to keep loading the external program as it has very high overhead. What is the best way to accomplish this? Thanks!
Have a look at ProcessBuilder. Once you've set up the ProcessBuilder and executed start you'll have a handle to a Process to which you can feed input and read output.
Here's a snippet to get you started:
ProcessBuilder pb = new ProcessBuilder("/bin/bash");
Process proc = pb.start();
// Start reading from the program
final Scanner in = new Scanner(proc.getInputStream());
new Thread() {
public void run() {
while (in.hasNextLine())
System.out.println(in.nextLine());
}
}.start();
// Write a few commands to the program.
PrintWriter out = new PrintWriter(proc.getOutputStream());
out.println("touch hello1");
out.flush();
out.println("touch hello2");
out.flush();
out.println("ls -la hel*");
out.flush();
out.close();
Output:
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello1
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello2
YOu can launch the external app with Runtime.getRuntime().exec(...)
To send data to the external program, you can either send data on the Processes output stream (You get a Process object back from exec) or you can open sockets and communicate that way.
I think you will find the Javadoc for class java.lang.Process helpful. Of note, you can get the input and output streams from a Process to communicate with it while it is running.
I second the answer about using ProcessBuilder. If you want to know more details about this, and why you should prefer it to Runtime.exec(), see this entry in the Java glossary. It also shows how to use threads to communicate with the external process.
I had issues trying to achieve bidirectional communication with the external process through stdin/stdout, because of blocking. In the end I found a github gist which allowed me solve the issue simply and elegantly; that gist is actually based on a stackoverflow answer.
See that other answer for sample code, but the core of the idea is to set up an event loop for reading and writing (while loop with 10ms sleeping), and using low-level stream operations so that no caching and blocking is going on -- only try to read if you know the other process in fact wrote something (through InputStream.available()).
It leads to a bit strange programming style, but the code is much simpler than it would be if using threads, and does the job pretty well.
Related
I am working on java process builder to execute the windows external process(i.e., exe files). I have to get the process info and error using input stream, error stream to write it in some text files.
It's working sometimes and unexpectedly hanging sometimes.
When i invoke around three external process to execute one by one. Two process working and it's getting hang on third process only.
I could see the process exit value as 0 some times it's giving some other random value.
I have read the below blog and applied the same logic to execute the process but's not working for my scenarios.
Could anybody please help me to diagnose these problem...
//Starts to initiate the external process
//This code will pick three arguments from to execute the process sequentially
//it will pass the process commands through for loop one by one
Process p =new ProcessBuilder(s)
.directory(new File(je.getExecution().getWorkingDirectory()))
.redirectErrorStream(true)
.redirectOutput(file)
.start();
p.getOutputStream().close();
int processStatus= p.waitFor();
// if the process exits with 0 normal termination
Thanks in advance..,
The entire thing you are doing is error prone and it’s not worth trying to find out all of the mistakes, as you are making your life unnecessarily hard.
Note that you are calling redirectErrorStream(true), but still are trying to read from the error stream, which makes no sense. The error stream will always be empty under this condition. That said, if you already know about redirections, it’s not clear, why you don’t use them directly, instead of creating threads, manually copying data into StringWriters, etc.
Apparently, you want to use the working directory je.getExecution().getWorkingDirectory(), launch the command s and directing both, output and error, of the program to file. This can be achieved as easy as
new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start()
.waitFor();
The only remaining error source is that the launched process could wait for input from your side. Since you apparently don’t want to provide input, you can use
Process proc = new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
to denote that you won’t write any data. This will not stop the process, but when it tries to read something from its standard input, it will immediately get an end-of-file without blocking.
Which can be demonstrated with
Process proc = new ProcessBuilder("cmd", "/c", "pause")
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
Using zt-exec I would like to know how it can write/read to a process which is waiting for console input on an infinite loop, as well as responding using its console out?
I believe the easiest way to describe this is with a Python script:
while(True):
javaSaid = raw_input("Hey Java, Say Something: ") ##wait for input from java
print "Python Heard Java Say: " +str(javaSaid) ##java needs to be able to get this output
Note: Executing the python process multiple times is what I am trying to avoid as the initialization time on the real python script makes this unacceptable.
You need to call redirectInput as well as redirectOutput on ProcessExecutor.
Have a look at ProcessExecutorInputStreamTest.java. It's just an example. It writes data to the process input vie PipedOutputStream -> PipedInputStream -> ProcessExecutor and read data from the process via an OutputStream.
im trying to do this on Android:
Process p = Runtime.getRuntime().exec("sh");
DataOutputStream out = new DataOutputStream(p.getOutputStream());
out.writeBytes("something useful\n");
out.close();
p.waitFor();
out = new DataOutputStream(p.getOutputStream());
out.writeBytes("something useful\n");
out.close();
p.waitFor();
The second time I execute out.writeBytes(); , I get a java IOException: "Bad file number".
My app has to execute several native programs, but always use the same process.
Anyone know why this does not work?
Note that the shell is not part of the public SDK (note it is not documented anywhere in the SDK documentation), so this code is in effect relying on private APIs.
Also this puts you outside of the normal application model -- we have no guarantee what will happen to a process you have forked and is not being managed by the platform. It may get killed at any time.
This is also a very inefficient way to do things, compared to doing whatever the command is doing in your own process. And starting a separate process for a command won't let it do anything more than you can, because it still runs as your uid.
So basically... for 99.99% of apps please don't do this. If you are writing a terminal app... well, okay, only geeks are going to care about that anyway, and it isn't going to be of much use since it runs as your uid, but okay. But otherwise, please no. :)
When you call out.close(), it will automatically call close() on the ouputstream of your process.
Each time you call p.getOutputStream() you get the same OutputStream, on your second use of out, p.getOutputStream() returns an already closed OutputStream.
Basically with your code, you don't really need to close the first DataOutputStream.
Sources :
Sources of DataOutputStream extends FilterOutputStream
Sources of FilterOutputStream.close()
I have started a process in my Java code, this process take a very long time to run and could generate some output from time to time. I need to react to every output when they are generated, what is the best way to do this?
What kind of reaction are you talking about? Is the process writing to its standard output and/or standard error? If so, I suspect Process.getInputStream and Process.getErrorStream are what you're looking for. Read from both of those and react accordingly. Note that you may want to read from both of them from different threads, to avoid the individual buffer for either stream from filling up.
Alternatively, if you don't need the two separately, just leave redirectErrorStream in ProcessBuilder as false, so the error and output streams are merged.
You should start a thread which reads from the Process.getInputStream() and getErrorStream() (or alternatively use ProcessBuilder.redirectErrorStream(true)) and handle it when something shows up in the stream. There are many ways that how to handle it - the right way depends on how the data is being used. Please tell more details.
Here is one real-life example: SbtRunner uses ProcessRunner to send commands to a command line application and wait for the command to finish execution (the application will print "> " when a command finishes execution). There is some indirection happening to make it easier to read from the process' output (the output is written to a MulticastPipe from where it is then read by an OutputReader).
We have a Perl program to validate XML which is invoked from a Java program. It is not able to write to standard error and hanging in the print location.
Perl is writing to STDERR and a java program is reading the STDERR using getErrorStream() function. But the Perl program is hanging to write to STDERR. I suspect Java function is blocking the STDERR stream completely and Perl is waiting for this stream to be released.
Is there a way in Perl to overcome this blockage and write to standard error forcefully? Since Java is doing only a read the API should not be locking the STDERR stream as per java doc.
Perl Code snippet is:
sub print_error
{
print STDERR shift;
}
Java code snippet is:
while ( getErrorStream() != null )
{
SOP errorMessage;
}
Appreciate the help in advance.
Thanks,
Mathew Liju
getErrorStream does not read the error stream, it just obtains a handle to it. As it's a pipe, if you never actually read it, it will fill up and force the Perl program to block.
You need something like:
Inputstream errors = getErrorStream();
while (errors.read(buffer) > 0) {
SOP buffer;
}
Ideally, I think that to avoid deadlock, in Java you need to spawn separate threads to read the STDERR and the STDOUT. It sounds like Perl is blocking when writing to STDERR because for one reason or another you are never reading from it in Java.
An additional factor to consider is the buffering that occurs with piped processes.
There is by default, about a 30-line-ish buffer that is maintained by the shell creating the inter-process pipe, so if the Perl app has not created enough data, it won't have been sent to the Java application yet to process.
May be this thread has a possible cause for your problem:
Add 3 lines to the top of the Perl script:
use IO::Handle;
STDOUT->autoflush(1);
STDERR->autoflush(1);
The problem in the mentioned thread was related to "the way Perl is buffering its output".
However here, Adrian Pronk mentions in the comments that "Perl is hanging because Java is never reading its output".
STDOUT->autoflush(1);
STDERR->autoflush(1);
This is the information I needed!
I have a Java app running some Perl scripts and I'd only get the output after it was finished.
By adding the autoflush(1) I get it right away.
BTW, I do have separate threads for reading STDERR and STDOUT, and that's the way to go.
Thanks.