In my program I have a SwingWorker starting a background process. The background process's error stream is redirected to stdout, and the stdout stream is written out (line by line) to a JTextArea. I thought I was consuming the stdout stream with this
BufferedReader processOut = new BufferedReader(
new InputStreamReader(p.getInputStream()));
And with this inside the SwingWorker's doInBackground:
String line;
while((line = processOut.readLine()) != null)
process(line);
On my machine, the process executes to completion and the Text Area is periodically updated. The process freezes in the background on other computers though. I've increased the size of the default command window, so that might be why I can't get any process to freeze on my computer (that is probably very wrong from the more that I read).
I tried redirecting the output inside the ProcessBuilder command with > log.txt (I'm on Windows 7 currently) but I think that's causing the p.getInputStream() call to crash.
How can I either consume the stdout of the subprocess properly inside my SwingWorker class, or is it possible to pipe the output to a file and still get the output to print to the JTextArea.
Edit:
I read here That an input stream needs to be read promptly to be consumed. I've provided the loop that processes the input stream below. I would assume that the pipe doesn't get more than 4K of data before it is read, but I can't assume anything at this point.
while(processIsAlive())
{
if(isCancelled())
{
try
{
p.destroy();
p.waitFor();
return 1;
}
catch(Exception e){}
}
try
{
//Update Text Area
//get output from process
while((line = processOut.readLine()) != null)
{
//print output to text area
publish(line);
}
sleep(1000);
}
catch(Exception e){}
}
EDIT 2:
I thought that having the process redirected to a file, then putting an InputStream on that same file would be impossible, but its not crashing my program and the GUI is still updated properly. I'm going to go test this on the problematic machine, then I'm going to mark it as an answer if it works.
All I did was redirect the process to a file:
pb.redirectOutput(new File("log.txt"));
And put the InputStream on that file instead of the process stdout stream.
processOut = new BufferedReader(new FileReader("log.txt"));
When starting processes, it's convenient to read the stdout and stderr in a separate thread each one, because reading must be done through a call to InputStream.read(), which does block the owner thread.
You said that your background process was redirecting errors to stdout, ok. Then, read only the stdout, but do it always in a separate thread, because the main thread gets blocked in the waitFor call until the process ends. But the process will block if its output buffer gets full and nobody is reading it.
In this case, the difference between your local environment (success) and the others (failing) could be that, in yours, the amount of data fits in the output buffer.
My second edit will work on any machine with Java 7 or later. The reading from the input stream is blocking, so that is incorrect for using in a GUI but the process streams are read from appropriately.
The actual problem with the machine was that for some reason the program wasn't flushing appropriately, so I added fflush(stdout) to the c source code in the problematic program, and it worked perfectly.
Related
I have a process created as follows:
Process p = Runtime.getRuntime().exec(new String[]{"su"});
In my program, I only want to create this process once. I am developing a root file explorer application for Android, and whenever this process is created, the Android device will prompt the user to grant root permissions. This is a very slow operation, and as this is a file browser, it will need root permissions often. So, I have decided to create this process once and write commands to its OutputStream in the following manner (stdin is this OutputStream):
stdin.writeBytes(command + "\n");
Before I can read the output of the command, I need my program to wait until the command written by writeBytes has terminated. I have tried p.waitFor(), but this causes the program to hang.
Here is how I read bytes from the InputStream:
int read;
String out = "";
stdout = p.getInputStream();
byte[] buffer = new byte[262144];
while (true) {
read = stdout.read(buffer);
out += new String(buffer, 0, read);
if (read < BUFF_LEN) {
//we have read everything
break;
}
}
Note that although the read(buffer) method blocks until input data is available, it does not block in this case because it thinks it has reached the end of the InputStream.
I have tried to include only relevant portions of my code in this post, but if you would like to take a look at the entire source code of the class where this is contained, see here: http://pastebin.com/t6JdWmQr.
How can I make sure the command has finished running before reading the process' InputStream?
I also encounter similar problem, and I found the answer here:
Wait until a command in su finishes
If you don't need any read stream in this shell process, simply add shell read stream may completed the shell process.
Or in XDA also have better way:
[HowTo]Execute Root Commands and read output
I tested this code(below) on several different linux boxes(4+) and it worked fine. However, on one linux box I ran into an issue with readline() hanging for the error inputStream(errorStream). This stream should be empty so I suspected that box was not writing out a line terminator to the errorStream for the error. I changed my code to use read() instead of readline()...but read() also hung.
I tried retrieving the input inputStream first, and that worked and there was no hangs with readline()/read() for the error inputstream. I could not do this since I needed to obtain possible errors first. Appearing to be a deadlock, I was able to resolve this by having each inputstream read from it's own thread. Why did I only see this issue on one box? Is there a kernel setting or some other setting specific to this box that could have caused this?
ProcessBuilder processBuilder = new ProcessBuilder()
try
{
Process processA = null;
synchronized (processBuilder)
{
processBuilder.command("/bin/sh","-c"," . /Home/SomeScript.ksh");
processA = processBuilder.start();
}
inputStream = processA.getInputStream();
reader = new BufferedReader(new InputStreamReader(inputStream));
errorStream = processA.getErrorStream();
errorReader = new BufferedReader(new InputStreamReader(errorStream));
String driverError;
while ((driverError = errorReader.readLine()) != null)
{
//some code
}
Why did I only see this issue on one box?
Most likely because of something in the script that is being run ... and its interactions with its environment (e.g. files, environment variables, etc)
Is there a kernel setting or some other setting specific to this box that could have caused this?
It is possible but unlikely that it is a kernel setting. It might be "something else". Indeed, it has to be "something" outside of the Java application that is to blame, at least in part.
I suggest you do the following temporarily (at least):
ProcessBuilder processBuilder = new ProcessBuilder();
processBuilder.command("/bin/sh","-c"," . /Home/SomeScript.ksh");
processBuilder.redirectErrorStream(true);
processA = processBuilder.start();
inputStream = processA.getInputStream();
reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
System.out.println("Return code is " + processA.exitValue());
That way you can see what all of the output is.
There should not be a problem if the external process fails to put a newline at the end of the last line. The Java process will see an EOF on the input stream, and the BufferedReader will return what characters it has ... and return null on the next call.
Another possibility is that the external process is blocking because it is trying to read from its standard input.
UPDATE
The redirectErrorStream also resolved the issue, but I need the error stream separate.
OK so if it did (reliably) solve the problem then that (most likely) means that you have to read the external processes stdout and stderr streams in parallel. The simple way to do is to create 2 threads to read and buffer the two streams separately. For example: Capturing stdout when calling Runtime.exec
(Your problem is due to the fact that pipes have a finite buffering capacity. The external problem is most likely alternating between writing stuff to stdout and stderr. If it tries to write to one of the pipes when that pipe is "full", it will block. But if your application is reading all of the other pipe (to EOF) before it reads the blocked pipe, then everything will deadlock. The fact that the external process is stuck in PIPE_W state is more evidence for this explanation.
One possible reason that you are seeing different behaviour on different systems is that the amount of buffering in a pipe is system dependent. But it could also be due to differences in what the external process is doing; e.g. its inputs.)
You are running OS specific commands in a script, any one could be holding the error output. You can avoid this by discarding the errors, but that is unlikely to be a good idea.
I would check the version of the OS are the same and whether you have any significant differences in the command you run in the script. If this doesn't help, take out commands from the script until it starts working. I assume an empty script doesn't do this.
I'm trying to get a list of running processes and their file paths on a Windows Server 2003 machine. I'm using the following code to try and do that:
protected Map<String,String> getProcesses() {
Map<String,String> processes = new HashMap<String,String>();
try {
String line;
Process p = null;
// Windows
if (OS.indexOf("win") >= 0) {
p = Runtime.getRuntime().exec("wmic process get description,executablepath");
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
LOG.info("Entering while loop");
while ((line = input.readLine()) != null) {
LOG.info("blah");
String[] array = line.split("\\s+");
if (array.length > 1) {
processes.put(array[0], array[1]);
}
}
LOG.info("Exited while loop");
input.close();
}
} catch (Exception e) {
e.printStackTrace();
}
return processes;
}
The program gets stuck in an infinite loop at the while condition. "blah" and "Exited while loop" never output to the log. I've ran the command in command prompt on both my win7 local machine and the server which outputs the information just fine. I've also ran the above code on my local machine which also works fine. It looks like it's some issue between Java and Windows Server 2003 that I haven't been able to find in the past 3 hours of googling. Any help would be much appreciated.
You will need to get and close your OutputStream before getting and using your InputStream. That will confirm to the process that you've started that you have finished sending input (in this case, no input) to the process.
p.getOutputStream().close();
Remember that on the Process object, getInputStream() input comes from the output stream of the process, and getOutputStream() output goes to the input stream of the process.
Remember the BufferedReader.readLine() operation will block if there the end of input has not been reached, see here.
I think what you are experiencing is explained in the API for Process:
The methods that create processes may not work well for special processes on certain native platforms, such as native windowing processes, daemon processes, Win16/DOS processes on Microsoft Windows, or shell scripts. The created subprocess does not have its own terminal or console. All its standard io (i.e. stdin, stdout, stderr) operations will be redirected to the parent process through three streams (getOutputStream(), getInputStream(), getErrorStream()). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
I am writing a java program to read the error stream from a process . Below is the structure of my code --
ProcessBuilder probuilder = new ProcessBuilder( command );
Process process = probuilder.start();
InputStream error = process.getErrorStream();
InputStreamReader isrerror = new InputStreamReader(error);
BufferedReader bre = new BufferedReader(isrerror);
while ((linee = bre.readLine()) != null) {
System.out.println(linee);
}
The above code works fine if anything is actually written to the error stream of the invoked process. However, if anything is not written to the error stream, then the call to readLine actually hangs indefinitely. However, I want to make my code generic so that it works for all scenarios. How can I modify my code to achieve the same.
Regards,
Dev
readline() is a blocking call. It will block until there's a line to be read (terminated by an end of line character) or the underlying stream is closed (returning EOF).
You need to have logic that is checking BufferedReader.ready() or just using BufferedReader.read() and bailing out if you decide you're waiting long enough (or want to do something else then check again).
Edit to add: That being said, it shouldn't hang "indefinitely" as-is; it should return once the invoked process terminates. By any chance is your invoked process also outputting something to stdout? If that's the case ... you need to be reading from that as well or the buffer will fill and will block the external process which will prevent it from exiting which ... leads to your problem.
This is a late reply, but the issue hasn't really solved and it's on the first page for some searches. I had the same issue, and BufferedReader.ready() would still set up a situation where it would lock.
The following workaround will not work if you need to get a persistent stream. However, if you're just running a program and waiting for it to close, this should be fine.
The workaround I'm using is to call ProcessBuilder.redirectError(File). Then I'd read the file and use that to present the error stream to the user. It worked fine, didn't lock. I did call Process.destroyForcibly() after Process.waitFor() but this is likely unnecessary.
Some pseudocode below:
File thisFile = new File("somefile.ext");
ProcessBuilder pb = new ProcessBuilder(yourStringList);
pb.redirectError(thisFile);
Process p = pb.start();
p.waitFor();
p.destroyForcibly();
ArrayList fileContents = getFileContents(thisFile);
I hope this helps with at least some of your use cases.
Something like this might also work and avoid the blocking behaviour (without requiring to create a File)
InputStream error = process.getErrorStream();
// Read from InputStream
for (int k = 0; k < error.available(); ++k)
System.out.println("Error stream = " + error.read());
From the Javadoc of InputStream.available
Returns an estimate of the number of bytes that can be read (orskipped over) from this input stream without blocking by the nextinvocation of a method for this input stream. The next invocationmight be the same thread or another thread. A single read or skip of thismany bytes will not block, but may read or skip fewer bytes.
The simplest answer would be to simply redirect the error stream to stdout:
process.getErrorStream().transferTo(System.out);
I am trying to run a .csh script and read it's output into a StringBuffer.
the output sometime returns empty although running the script from console returns some output. the same running flow can sometimes returns output and sometimes not, although nothing is changed in the way the process starts (same script, path , args) and the script isn't changed as well.
I'm not getting any exceptions thrown.
what might cause output now to be read correctly/successfully ?
the code segment is
public static String getOutpoutScript(Process p) {
InputStream outpout = p.getInputStream();
logger.info("Retrived script output stream");
BufferedReader buf = new BufferedReader(new InputStreamReader(outpout));
String line = "";
StringBuffer write = new StringBuffer();
try {
while ((line = buf.readLine()) != null) {
write.append(line);
}
} catch (IOException e) {
// do something
}
return write.toString().trim();
}
beside the fact not closing the streams is not good, could this or something else in the code might prevent output from being read correctly under some circumstances ?
thanks,
If you launch it with ProcessBuilder, you can combine the error stream into the output stream. This way if the program prints to stderr you'll capture this too. Alternatively you could just read both. Additionally, you may not want to use readLine, you could be stuck for awhile if the program does not print end of line character at the end.
Maybe you must replace p.getInputStream() with p.getOutputStream()
Besides this sometimes processes can block waiting on input, so you must read and write asynchronously - one possible solution is to use different threads - e.g. one thread is reading, other is writing and one that is monitoring the process.
If you have an error, this will write to getErrorStream() by default. If you have a problem, I would ensure you are reading this somewhere.
If the buffer for this stream fills, your program will stop, waiting for you to read it.
A simple way around these issues is to use ProcessBuilder.redirectErrorStream(true)