I'm trying to run a program ignoring its output, but it seems to hangs when its output is large. My code is as follows:
Process p = Runtime.getRuntime().exec("program");
p.getOutputStream().write(input.getBytes());
p.getOutputStream().flush();
p.getOutputStream().close();
p.waitFor();
What is the best way to ignore the output?
I tried to redirect the output to /dev/null, but I got a Java IOException 'Broke pipe'.
Did you try:
Process p = Runtime.getRuntime().exec("program >/dev/null 2>&1");
?
I remember having to do something similar in Java before, but I may not have been calling the process the same way.
Edit: I just tested this code and it successfully completes.
class a
{
public static void main(String[] args) throws Exception
{
Process p = Runtime.getRuntime().exec("cat a.java >/dev/null 2>&1");
p.getOutputStream().write(123123);
p.getOutputStream().flush();
p.getOutputStream().close();
p.waitFor();
}
}
I'd use a Stream Gobbler. For more on that, please look here: What to do when Runtime.exec() won't
You cannot "ignore" the output of a child process in Java, well at least technically. You ought to read the contents of the input stream, for not doing so will result in the output buffer on the other end filling up, resulting in the described hanging behavior. When the child process attempts to write to a full output buffer, it will block until the buffer has been cleared by reading it.
If you do not want the output, at least read it and discard it. Or use a NullInputStream. You do not have to use Apache Commons class; you can build your own NullInputStream class whose read methods have empty bodies, similar to NullOutputStreams.
Also, this problem might not be solved by reading the input stream alone. You would have to read the error stream as well, which may be redirect to the input stream.
Related
Working on SEAndroid, I call Setools commands from my Java application.
It works perfectly with small SEAndroid policy and now I need to test my tool with real
SEAndroid policy. But unfortunately, I face a problem with an error stream.
Here my code I used to call external commands :
public static BufferedReader runCommand(final String[] args)
throws IOException {
BufferedReader stdInput = null;
BufferedReader stdError = null;
try {
Process p = Runtime.getRuntime().exec(args);
stdInput = new BufferedReader(new
InputStreamReader(p.getInputStream()));
stdError = new BufferedReader(new
InputStreamReader(p.getErrorStream()));
// read any errors from the attempted command
String s = null;
StringBuilder err = new StringBuilder();
while ((s = stdError.readLine()) != null) {
err.append(s + "\n");
}
if (err.length() != 0) {
throw new IOException(err.toString());
}
return stdInput;
} finally {
if (stdError != null) {
stdError.close();
}
}
}
So, as you can see, I call an external command. Then read the error stream and throw an exception if there is any errors, otherwise I return the InputStream, so I can parse it later.
With a real SEAndroid policy, the error stream seems to block (even if I read a single char) and I can't parse the result of the command. If I close the error stream without reading anything, the application works fine, but I want to handle errors if any.
If I type the command in a console, it works fine too.
In the first case (with small SEAndroid policy), the output of the command is small ( ~350 lines).
In the second case (with a real SEAndroid policy), the output of the command is larger ( >1500 lines).
Is it possible that the size of the output stream influences the error stream? The two streams are two distinctive resources, isn't it?
The fact that I do not read the output stream immediately have an importance?
I fear that its not a "programming" problem but more a system problem...
Any suggestion?
Thanks in advance for your help=)
Edit:
I try to read the output stream before the error stream and it works. But I need to check the error stream before perform any parsing on the output stream, so the problem is still topical.
First, it's probably better to use the newer ProcessBuilder class as opposed to Runtime exec. If you want to go a step further, you can even use Apache commons-exec which takes care of stream handling and other things for you.
Next, as you've discovered, process control is a tricky thing in Java and you've run into one of its tricky issues. From the documentation for java's Process class:
The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
You need to have something consuming both (Error and Output) streams or you risk deadlock - these should each be read on their own threads. Using something like a StreamGobbler (google it, there are plenty out there) would be a good step, or you can roll your own if you're so inclined. It isn't too hard to get it right but if you're unfamiliar with multithreading you may want to look at someone else's implementation or go the Apache commons-exec route.
The processing of output is so annoying, that I wrote little library called jproc that deals with the problem of consuming stdout and stderr. It can simply filter strings through external programs likes this:
ProcBuilder.filter("x y z","sed" ,"s/y/a/")
It also lets you specify a timeout for the completion and will convert non-zero exit codes into exception.
I am writing a java program to read the error stream from a process . Below is the structure of my code --
ProcessBuilder probuilder = new ProcessBuilder( command );
Process process = probuilder.start();
InputStream error = process.getErrorStream();
InputStreamReader isrerror = new InputStreamReader(error);
BufferedReader bre = new BufferedReader(isrerror);
while ((linee = bre.readLine()) != null) {
System.out.println(linee);
}
The above code works fine if anything is actually written to the error stream of the invoked process. However, if anything is not written to the error stream, then the call to readLine actually hangs indefinitely. However, I want to make my code generic so that it works for all scenarios. How can I modify my code to achieve the same.
Regards,
Dev
readline() is a blocking call. It will block until there's a line to be read (terminated by an end of line character) or the underlying stream is closed (returning EOF).
You need to have logic that is checking BufferedReader.ready() or just using BufferedReader.read() and bailing out if you decide you're waiting long enough (or want to do something else then check again).
Edit to add: That being said, it shouldn't hang "indefinitely" as-is; it should return once the invoked process terminates. By any chance is your invoked process also outputting something to stdout? If that's the case ... you need to be reading from that as well or the buffer will fill and will block the external process which will prevent it from exiting which ... leads to your problem.
This is a late reply, but the issue hasn't really solved and it's on the first page for some searches. I had the same issue, and BufferedReader.ready() would still set up a situation where it would lock.
The following workaround will not work if you need to get a persistent stream. However, if you're just running a program and waiting for it to close, this should be fine.
The workaround I'm using is to call ProcessBuilder.redirectError(File). Then I'd read the file and use that to present the error stream to the user. It worked fine, didn't lock. I did call Process.destroyForcibly() after Process.waitFor() but this is likely unnecessary.
Some pseudocode below:
File thisFile = new File("somefile.ext");
ProcessBuilder pb = new ProcessBuilder(yourStringList);
pb.redirectError(thisFile);
Process p = pb.start();
p.waitFor();
p.destroyForcibly();
ArrayList fileContents = getFileContents(thisFile);
I hope this helps with at least some of your use cases.
Something like this might also work and avoid the blocking behaviour (without requiring to create a File)
InputStream error = process.getErrorStream();
// Read from InputStream
for (int k = 0; k < error.available(); ++k)
System.out.println("Error stream = " + error.read());
From the Javadoc of InputStream.available
Returns an estimate of the number of bytes that can be read (orskipped over) from this input stream without blocking by the nextinvocation of a method for this input stream. The next invocationmight be the same thread or another thread. A single read or skip of thismany bytes will not block, but may read or skip fewer bytes.
The simplest answer would be to simply redirect the error stream to stdout:
process.getErrorStream().transferTo(System.out);
I am trying to run a .csh script and read it's output into a StringBuffer.
the output sometime returns empty although running the script from console returns some output. the same running flow can sometimes returns output and sometimes not, although nothing is changed in the way the process starts (same script, path , args) and the script isn't changed as well.
I'm not getting any exceptions thrown.
what might cause output now to be read correctly/successfully ?
the code segment is
public static String getOutpoutScript(Process p) {
InputStream outpout = p.getInputStream();
logger.info("Retrived script output stream");
BufferedReader buf = new BufferedReader(new InputStreamReader(outpout));
String line = "";
StringBuffer write = new StringBuffer();
try {
while ((line = buf.readLine()) != null) {
write.append(line);
}
} catch (IOException e) {
// do something
}
return write.toString().trim();
}
beside the fact not closing the streams is not good, could this or something else in the code might prevent output from being read correctly under some circumstances ?
thanks,
If you launch it with ProcessBuilder, you can combine the error stream into the output stream. This way if the program prints to stderr you'll capture this too. Alternatively you could just read both. Additionally, you may not want to use readLine, you could be stuck for awhile if the program does not print end of line character at the end.
Maybe you must replace p.getInputStream() with p.getOutputStream()
Besides this sometimes processes can block waiting on input, so you must read and write asynchronously - one possible solution is to use different threads - e.g. one thread is reading, other is writing and one that is monitoring the process.
If you have an error, this will write to getErrorStream() by default. If you have a problem, I would ensure you are reading this somewhere.
If the buffer for this stream fills, your program will stop, waiting for you to read it.
A simple way around these issues is to use ProcessBuilder.redirectErrorStream(true)
When I try to execute an external program from java I use this code below :
Process p;
rn = Runtime.getRuntime();
String[] unzip = new String[2];
unzip[0]="unzip";
unzip[1]=archive ;
public void dezip() throws IOException{
p = rn.exec(unzip);
int ret = p.exitValue();
System.out.println("End of unzip method");
But my last System.out is never executed, as if we exit from unzip method.
The unzip() call does only the half of the work, only a part of my archive is unzipped.
When I use ps -x or htop from command line I see that unzip process is still here.
Help please.
You probably need to read the InputStream from the process. See the javadoc of Process
Which states:
Because some native platforms only provide limited buffer size for
standard input and output streams, failure to promptly write the input
stream or read the output stream of the subprocess may cause the
subprocess to block, and even deadlock.
Check if the unzip command is prompting for something, perhaps a warning if the file already exists and if you want to overwrite it.
Also, is that a backquote I see in the middle of a java program?
Make sure external program doesn't wait for user input
Check if the the executable path is quoted when launching on Windows systems to handle directories with spaces or special characters.
PS.
I was using the java.lang.Runtime class but found that the java.lang.ProcessBuilder class is far superior. You can specify current working directory, and most importantly the system environment.
Please try the following:
p = rn.exec(unzip);
p.waitFor()
I hope it will change something.
My problem is that, i am using Runtime.getruntime.exec() function to run my unix command on Java. But, it jumps to the end of codes while exec() command is being run. The codes are below.
Process songProcess;
ArrayList<String> xmlFilePathsForEmi = new ArrayList<String>();
int countForEmiSongUpdates = 0;
String line;
try {
songProcess = Runtime.getRuntime().exec(new String[]{"find /home/gozenem/emiornek/ -name '*.xml'"}); // It jumps here !
songProcess.waitFor();
bufferedReaderSong = new BufferedReader(new InputStreamReader(songProcess.getInputStream()));
while((line = bufferedReaderSong.readLine()) != null){
xmlFilePathsForEmi.add(line);
}
...
...
...
}
I do not know what it is related to, may be there is a character that exec function could not run. I need your precious help. Thank you.
Your String[] parameter to Runtime.exec() is incorrect. It must be split up so that it contains one element per item (the executable must be one string, then each individual argument must come in its own string).
Try something like:
songProcess = Runtime.getRuntime().exec(new String[]{"find", "/home/gozenem/emiornek/", "-name", "*.xml"});
Also calling waitFor where you are doing isn't appropriate. You need to read the output while the process is running, otherwise you risk filling up the I/O buffers that are used between the Java VM and your process. So move that waitFor to after you've processed the output.
From the Process docs:
By default, the created subprocess does not have its own terminal or console. All its standard I/O (i.e. stdin, stdout, stderr) operations will be redirected to the parent process, [...]. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.