I am working on java process builder to execute the windows external process(i.e., exe files). I have to get the process info and error using input stream, error stream to write it in some text files.
It's working sometimes and unexpectedly hanging sometimes.
When i invoke around three external process to execute one by one. Two process working and it's getting hang on third process only.
I could see the process exit value as 0 some times it's giving some other random value.
I have read the below blog and applied the same logic to execute the process but's not working for my scenarios.
Could anybody please help me to diagnose these problem...
//Starts to initiate the external process
//This code will pick three arguments from to execute the process sequentially
//it will pass the process commands through for loop one by one
Process p =new ProcessBuilder(s)
.directory(new File(je.getExecution().getWorkingDirectory()))
.redirectErrorStream(true)
.redirectOutput(file)
.start();
p.getOutputStream().close();
int processStatus= p.waitFor();
// if the process exits with 0 normal termination
Thanks in advance..,
The entire thing you are doing is error prone and it’s not worth trying to find out all of the mistakes, as you are making your life unnecessarily hard.
Note that you are calling redirectErrorStream(true), but still are trying to read from the error stream, which makes no sense. The error stream will always be empty under this condition. That said, if you already know about redirections, it’s not clear, why you don’t use them directly, instead of creating threads, manually copying data into StringWriters, etc.
Apparently, you want to use the working directory je.getExecution().getWorkingDirectory(), launch the command s and directing both, output and error, of the program to file. This can be achieved as easy as
new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start()
.waitFor();
The only remaining error source is that the launched process could wait for input from your side. Since you apparently don’t want to provide input, you can use
Process proc = new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
to denote that you won’t write any data. This will not stop the process, but when it tries to read something from its standard input, it will immediately get an end-of-file without blocking.
Which can be demonstrated with
Process proc = new ProcessBuilder("cmd", "/c", "pause")
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
Related
I want to execute multiple jar files sequentially by passing I/O and reading the output from them from a single Java program. I used this below code to invoke my jar inside my java code. This is leading to multiple JVM instances and the jar execution is not completing until I stop the execution of my main java program.
Process pb = Runtime.getRuntime().exec("java -jar path/to/jar");
pb.waitFor();
BufferedReader ib = new BufferedReader(new InputStreamReader(pb.getErrorStream()));
BufferedReader in = new BufferedReader(new InputStreamReader(pb.getInputStream()));
System.out.println(in.readLine());
My ideal sequence of execution is:
Main java (starts)
JAR 1 (starts and completed)
JAR 2 (starts and completed)
Jar n ----
Main Java (stops)
My knowledge is limited in multiprocessing that is going on with my current code. Please help me understand how it works and how can I achieve the scenario I intend to.
As soon as the subprocess produces more output than the pipe’s buffering capability, it will be blocked until the initiating process reads the data. When you are waiting for the end of the subprocess before reading anything, this may lead to a deadlock.
Since you are only reading the output to reprint it to the console (or generally, write to stdout), you may use ProcessBuilder to tell it not to use a pipe, but connect the subprocess’ stdio to your process’ stdio:
Process pb = new ProcessBuilder("java", "-jar", "path/to/jar").inheritIO().start();
pb.waitFor();
Then you don’t need to do anything to transfer the subprocess’ output to your process’ output and there’s no deadlock potential.
inheritIO() does the magic. It’s a short-hand for .redirectInput(Redirect.INHERIT) .redirectOutput(Redirect.INHERIT) .redirectError(Redirect.INHERIT). These redirect… calls also can be used for configuring the individual channels to use a pipe or read from/ write to a file.
I'm automating a gradle build using Java. I execute "gradlew.bat" in a Process created from a ProcessBuilder. Here's the code:
ProcessBuilder gradlewProcessBuilder = new ProcessBuilder(mainDirPath.concat("\\android\\gradlew.bat"), "assembleDebug");
gradlewProcessBuilder.directory(new File(mainDirPath.concat("/android")));
gradlewProcessBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT); //This is the line
Process gradlewProcess = gradlewProcessBuilder.start();
gradlewProcess.waitFor();
Now this code works flawlessly, but it outputs the gradle console through the application console and I don't want that. If I delete this line:
gradlewProcessBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
The process hangs in waitFor() indefinitely.
I have absolutely no idea how could redirectOutput have effect over this, any light you can shed is welcome.
If the output is not redirected or handled, then a default output buffer is allocated. However, on Windows the output buffer is very small; when it fills up, the app will stop executing and will block until the output buffer is drained.
Therefore, you need to drain the output somehow; either by using INHERIT or by having a thread which drains the output. I recommend using https://github.com/zeroturnaround/zt-exec since it has much better default handling.
I want to invoke an external program in java code, then the Google tell me that the Runtime or ProcessBuilder can help me to do this work. I have tried it, and there come out a problem the java program can't exit, that means both the sub process and the father process wait for forever. they are hanging or deadlock.
Someone tell me the reason is that the sub process's cache is too small. when it try to give back data to the father process, but the father process don't read it in time, then both of them hang. So they advice me fork an thread to be in charge of read sub process's cache data. I do it as what they tell me, but there still some problem.
Then I close the output stream which get by the method getOutputStream(). Finally, the program success. But I don't know why it happen? Is there some relationship between the output steam and input stream?
You have provided very few details in your question, so I can only provide a general answer.
All processes have three standard streams: standard input, standard output and standard error. Standard input is used for reading in data, standard output for writing out data, and standard error for writing out error messages. When you start an external program using Runtime.getRuntime().exec() or ProcessBuilder, Java will create a Process object for the external program, and this Process object will have methods to access these streams.
These streams are accessed as follows:
process.getOutputStream(): return the standard input of the external program. This is an OutputStream as it is something your Java code will write to.
process.getInputStream(): return the standard output of the external program. This is an InputStream as it is something your Java code will read from.
process.getErrorStream(): return the standard error of the external program. This is an InputStream as, like standard output, it is something your Java code will read from.
Note that the names of getInputStream() and getOutputStream() can be confusing.
All streams between your Java code and the external program are buffered. This means each stream has a small amount of memory (a buffer) where the writer can write data that is yet to be read by the reader. The writer does not have to wait for the reader to read its data immediately; it can leave its output in the buffer and continue.
There are two ways in which writing to buffers and reading from them can hang:
attempting to write data to a buffer when there is not enough space left for the data,
attempting to read from an empty buffer.
In the first situation, the writer will wait until space is made in the buffer by reading data out of it. In the second, the reader will wait until data is written into the buffer.
You mention that closing the stream returned by getOutputStream() caused your program to complete successfully. This closes the standard input of the external program, telling it that there will be nothing more for it to read. If your program then completes successfully, this suggests that your program was waiting for more input to come when it was hanging.
It is perhaps arguable that if you do run an external program, you should close its standard input if you don't need to use it, as you have done. This tells the external program that there will be no more input, and so removes the possibility of it being stuck waiting for input. However, it doesn't answer the question of why your external program is waiting for input.
Most of the time, when you run external programs using Runtime.getRuntime().exec() or ProcessBuilder, you don't often use the standard input. Typically, you'd pass whatever inputs you'd need to the external program on the command line and then read its output (if it generates any at all).
Does your external program do what you need it to and then get stuck, apparently waiting for input? Do you ever need to send it data to its standard input? If you start a process on Windows using cmd.exe /k ..., the command interpreter will continue even after the program it started has exited. In this case, you should use /c instead of /k.
Finally, I'd like to emphasise that there are two output streams, standard output and standard error. There can be problems if you read from the wrong stream at the wrong time. If you attempt to read from the external program's standard output while its buffer is empty, your Java code will wait for the external program to generate output. However, if your external program is writing a lot of data to its standard error, it could fill the buffer and then find itself waiting for your Java code to make space in the buffer by reading from it. The end result of this is your Java code and the external program are both waiting for each other to do something, i.e. deadlock.
This problem can be eliminated simply by using a ProcessBuilder and ensuring that you call its redirectErrorStream() method with a true value. Calling this method redirects the standard error of the external program into its standard output, so you only have one stream to read from.
I want to be able to run an external program concurrently with my Java code, i.e. I want to start the program, then return control to the calling method while keeping the external program running at the same time. The Java code will then keep generating input and send it to the external program and receive output back.
I don't want to keep loading the external program as it has very high overhead. What is the best way to accomplish this? Thanks!
Have a look at ProcessBuilder. Once you've set up the ProcessBuilder and executed start you'll have a handle to a Process to which you can feed input and read output.
Here's a snippet to get you started:
ProcessBuilder pb = new ProcessBuilder("/bin/bash");
Process proc = pb.start();
// Start reading from the program
final Scanner in = new Scanner(proc.getInputStream());
new Thread() {
public void run() {
while (in.hasNextLine())
System.out.println(in.nextLine());
}
}.start();
// Write a few commands to the program.
PrintWriter out = new PrintWriter(proc.getOutputStream());
out.println("touch hello1");
out.flush();
out.println("touch hello2");
out.flush();
out.println("ls -la hel*");
out.flush();
out.close();
Output:
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello1
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello2
YOu can launch the external app with Runtime.getRuntime().exec(...)
To send data to the external program, you can either send data on the Processes output stream (You get a Process object back from exec) or you can open sockets and communicate that way.
I think you will find the Javadoc for class java.lang.Process helpful. Of note, you can get the input and output streams from a Process to communicate with it while it is running.
I second the answer about using ProcessBuilder. If you want to know more details about this, and why you should prefer it to Runtime.exec(), see this entry in the Java glossary. It also shows how to use threads to communicate with the external process.
I had issues trying to achieve bidirectional communication with the external process through stdin/stdout, because of blocking. In the end I found a github gist which allowed me solve the issue simply and elegantly; that gist is actually based on a stackoverflow answer.
See that other answer for sample code, but the core of the idea is to set up an event loop for reading and writing (while loop with 10ms sleeping), and using low-level stream operations so that no caching and blocking is going on -- only try to read if you know the other process in fact wrote something (through InputStream.available()).
It leads to a bit strange programming style, but the code is much simpler than it would be if using threads, and does the job pretty well.
I have started a process in my Java code, this process take a very long time to run and could generate some output from time to time. I need to react to every output when they are generated, what is the best way to do this?
What kind of reaction are you talking about? Is the process writing to its standard output and/or standard error? If so, I suspect Process.getInputStream and Process.getErrorStream are what you're looking for. Read from both of those and react accordingly. Note that you may want to read from both of them from different threads, to avoid the individual buffer for either stream from filling up.
Alternatively, if you don't need the two separately, just leave redirectErrorStream in ProcessBuilder as false, so the error and output streams are merged.
You should start a thread which reads from the Process.getInputStream() and getErrorStream() (or alternatively use ProcessBuilder.redirectErrorStream(true)) and handle it when something shows up in the stream. There are many ways that how to handle it - the right way depends on how the data is being used. Please tell more details.
Here is one real-life example: SbtRunner uses ProcessRunner to send commands to a command line application and wait for the command to finish execution (the application will print "> " when a command finishes execution). There is some indirection happening to make it easier to read from the process' output (the output is written to a MulticastPipe from where it is then read by an OutputReader).