I want to execute multiple jar files sequentially by passing I/O and reading the output from them from a single Java program. I used this below code to invoke my jar inside my java code. This is leading to multiple JVM instances and the jar execution is not completing until I stop the execution of my main java program.
Process pb = Runtime.getRuntime().exec("java -jar path/to/jar");
pb.waitFor();
BufferedReader ib = new BufferedReader(new InputStreamReader(pb.getErrorStream()));
BufferedReader in = new BufferedReader(new InputStreamReader(pb.getInputStream()));
System.out.println(in.readLine());
My ideal sequence of execution is:
Main java (starts)
JAR 1 (starts and completed)
JAR 2 (starts and completed)
Jar n ----
Main Java (stops)
My knowledge is limited in multiprocessing that is going on with my current code. Please help me understand how it works and how can I achieve the scenario I intend to.
As soon as the subprocess produces more output than the pipe’s buffering capability, it will be blocked until the initiating process reads the data. When you are waiting for the end of the subprocess before reading anything, this may lead to a deadlock.
Since you are only reading the output to reprint it to the console (or generally, write to stdout), you may use ProcessBuilder to tell it not to use a pipe, but connect the subprocess’ stdio to your process’ stdio:
Process pb = new ProcessBuilder("java", "-jar", "path/to/jar").inheritIO().start();
pb.waitFor();
Then you don’t need to do anything to transfer the subprocess’ output to your process’ output and there’s no deadlock potential.
inheritIO() does the magic. It’s a short-hand for .redirectInput(Redirect.INHERIT) .redirectOutput(Redirect.INHERIT) .redirectError(Redirect.INHERIT). These redirect… calls also can be used for configuring the individual channels to use a pipe or read from/ write to a file.
Related
I am working on java process builder to execute the windows external process(i.e., exe files). I have to get the process info and error using input stream, error stream to write it in some text files.
It's working sometimes and unexpectedly hanging sometimes.
When i invoke around three external process to execute one by one. Two process working and it's getting hang on third process only.
I could see the process exit value as 0 some times it's giving some other random value.
I have read the below blog and applied the same logic to execute the process but's not working for my scenarios.
Could anybody please help me to diagnose these problem...
//Starts to initiate the external process
//This code will pick three arguments from to execute the process sequentially
//it will pass the process commands through for loop one by one
Process p =new ProcessBuilder(s)
.directory(new File(je.getExecution().getWorkingDirectory()))
.redirectErrorStream(true)
.redirectOutput(file)
.start();
p.getOutputStream().close();
int processStatus= p.waitFor();
// if the process exits with 0 normal termination
Thanks in advance..,
The entire thing you are doing is error prone and it’s not worth trying to find out all of the mistakes, as you are making your life unnecessarily hard.
Note that you are calling redirectErrorStream(true), but still are trying to read from the error stream, which makes no sense. The error stream will always be empty under this condition. That said, if you already know about redirections, it’s not clear, why you don’t use them directly, instead of creating threads, manually copying data into StringWriters, etc.
Apparently, you want to use the working directory je.getExecution().getWorkingDirectory(), launch the command s and directing both, output and error, of the program to file. This can be achieved as easy as
new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start()
.waitFor();
The only remaining error source is that the launched process could wait for input from your side. Since you apparently don’t want to provide input, you can use
Process proc = new ProcessBuilder(s)
.directory(je.getExecution().getWorkingDirectory())
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
to denote that you won’t write any data. This will not stop the process, but when it tries to read something from its standard input, it will immediately get an end-of-file without blocking.
Which can be demonstrated with
Process proc = new ProcessBuilder("cmd", "/c", "pause")
.redirectErrorStream(true).redirectOutput(file)
.start();
proc.getOutputStream().close();
proc.waitFor();
Using zt-exec I would like to know how it can write/read to a process which is waiting for console input on an infinite loop, as well as responding using its console out?
I believe the easiest way to describe this is with a Python script:
while(True):
javaSaid = raw_input("Hey Java, Say Something: ") ##wait for input from java
print "Python Heard Java Say: " +str(javaSaid) ##java needs to be able to get this output
Note: Executing the python process multiple times is what I am trying to avoid as the initialization time on the real python script makes this unacceptable.
You need to call redirectInput as well as redirectOutput on ProcessExecutor.
Have a look at ProcessExecutorInputStreamTest.java. It's just an example. It writes data to the process input vie PipedOutputStream -> PipedInputStream -> ProcessExecutor and read data from the process via an OutputStream.
On Windows 7 64 bit, running 64 bit Java 1.7.0_17 , the p.waitFor() shown below never returns.
String move_command="cmd.exe /c xcopy /Y /E "+x86_release+" "+path+"\\";
Process p;
p = Runtime.getRuntime().exec(move_command);
p.waitFor();
If I use Windows Explorer, it looks like all the files are copied (same number, same size, etc.)
If I do the below, it waitFor() does return:
String move_command="cmd.exe /c move /Y "+x86_release+" "+path+"\\";
Process p;
p = Runtime.getRuntime().exec(move_command);
p.waitFor();
What could be so different between an xcopy and a move that keeps waitFor() from returning, or am I on the wrong track entirely?
xcopy probably just happens to produce more output than move, filling up the out-buffer and blocking until it is flushed. The default behavior in Java is to pipe the subprocess's stdout/stderr into InputStreams that you are then required to read programmatically lest the subprocess's buffers overflow.
If the latter is the case, the solution is simple, and in fact you should do that anyway: use ProcessBuilder to prepare the system call and call inheritIO on it. This will reuse your parent process`s stdin and stdout for the subprocess.
A side note, xcopy is a regular .exe file and doesn't need wrapping into cmd.exe /c.
I suspect you're not consuming the process standard out/err, and that's blocking the process. If your code doesn't consume this output, then the spawned process will hang (and you'll hang waiting for that process!). Why the difference in behaviour between the two commands ? Probably due to the quantity of data returned and the impact on the publishing buffers.
See this answer for more details.
I would also investigate Apache Commons FileUtils.copyDirectory() such that you don't have to spawn a whole new process to copy files.
I want to invoke an external program in java code, then the Google tell me that the Runtime or ProcessBuilder can help me to do this work. I have tried it, and there come out a problem the java program can't exit, that means both the sub process and the father process wait for forever. they are hanging or deadlock.
Someone tell me the reason is that the sub process's cache is too small. when it try to give back data to the father process, but the father process don't read it in time, then both of them hang. So they advice me fork an thread to be in charge of read sub process's cache data. I do it as what they tell me, but there still some problem.
Then I close the output stream which get by the method getOutputStream(). Finally, the program success. But I don't know why it happen? Is there some relationship between the output steam and input stream?
You have provided very few details in your question, so I can only provide a general answer.
All processes have three standard streams: standard input, standard output and standard error. Standard input is used for reading in data, standard output for writing out data, and standard error for writing out error messages. When you start an external program using Runtime.getRuntime().exec() or ProcessBuilder, Java will create a Process object for the external program, and this Process object will have methods to access these streams.
These streams are accessed as follows:
process.getOutputStream(): return the standard input of the external program. This is an OutputStream as it is something your Java code will write to.
process.getInputStream(): return the standard output of the external program. This is an InputStream as it is something your Java code will read from.
process.getErrorStream(): return the standard error of the external program. This is an InputStream as, like standard output, it is something your Java code will read from.
Note that the names of getInputStream() and getOutputStream() can be confusing.
All streams between your Java code and the external program are buffered. This means each stream has a small amount of memory (a buffer) where the writer can write data that is yet to be read by the reader. The writer does not have to wait for the reader to read its data immediately; it can leave its output in the buffer and continue.
There are two ways in which writing to buffers and reading from them can hang:
attempting to write data to a buffer when there is not enough space left for the data,
attempting to read from an empty buffer.
In the first situation, the writer will wait until space is made in the buffer by reading data out of it. In the second, the reader will wait until data is written into the buffer.
You mention that closing the stream returned by getOutputStream() caused your program to complete successfully. This closes the standard input of the external program, telling it that there will be nothing more for it to read. If your program then completes successfully, this suggests that your program was waiting for more input to come when it was hanging.
It is perhaps arguable that if you do run an external program, you should close its standard input if you don't need to use it, as you have done. This tells the external program that there will be no more input, and so removes the possibility of it being stuck waiting for input. However, it doesn't answer the question of why your external program is waiting for input.
Most of the time, when you run external programs using Runtime.getRuntime().exec() or ProcessBuilder, you don't often use the standard input. Typically, you'd pass whatever inputs you'd need to the external program on the command line and then read its output (if it generates any at all).
Does your external program do what you need it to and then get stuck, apparently waiting for input? Do you ever need to send it data to its standard input? If you start a process on Windows using cmd.exe /k ..., the command interpreter will continue even after the program it started has exited. In this case, you should use /c instead of /k.
Finally, I'd like to emphasise that there are two output streams, standard output and standard error. There can be problems if you read from the wrong stream at the wrong time. If you attempt to read from the external program's standard output while its buffer is empty, your Java code will wait for the external program to generate output. However, if your external program is writing a lot of data to its standard error, it could fill the buffer and then find itself waiting for your Java code to make space in the buffer by reading from it. The end result of this is your Java code and the external program are both waiting for each other to do something, i.e. deadlock.
This problem can be eliminated simply by using a ProcessBuilder and ensuring that you call its redirectErrorStream() method with a true value. Calling this method redirects the standard error of the external program into its standard output, so you only have one stream to read from.
I want to be able to run an external program concurrently with my Java code, i.e. I want to start the program, then return control to the calling method while keeping the external program running at the same time. The Java code will then keep generating input and send it to the external program and receive output back.
I don't want to keep loading the external program as it has very high overhead. What is the best way to accomplish this? Thanks!
Have a look at ProcessBuilder. Once you've set up the ProcessBuilder and executed start you'll have a handle to a Process to which you can feed input and read output.
Here's a snippet to get you started:
ProcessBuilder pb = new ProcessBuilder("/bin/bash");
Process proc = pb.start();
// Start reading from the program
final Scanner in = new Scanner(proc.getInputStream());
new Thread() {
public void run() {
while (in.hasNextLine())
System.out.println(in.nextLine());
}
}.start();
// Write a few commands to the program.
PrintWriter out = new PrintWriter(proc.getOutputStream());
out.println("touch hello1");
out.flush();
out.println("touch hello2");
out.flush();
out.println("ls -la hel*");
out.flush();
out.close();
Output:
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello1
-rw-r--r-- 1 aioobe aioobe 0 2011-04-08 08:29 hello2
YOu can launch the external app with Runtime.getRuntime().exec(...)
To send data to the external program, you can either send data on the Processes output stream (You get a Process object back from exec) or you can open sockets and communicate that way.
I think you will find the Javadoc for class java.lang.Process helpful. Of note, you can get the input and output streams from a Process to communicate with it while it is running.
I second the answer about using ProcessBuilder. If you want to know more details about this, and why you should prefer it to Runtime.exec(), see this entry in the Java glossary. It also shows how to use threads to communicate with the external process.
I had issues trying to achieve bidirectional communication with the external process through stdin/stdout, because of blocking. In the end I found a github gist which allowed me solve the issue simply and elegantly; that gist is actually based on a stackoverflow answer.
See that other answer for sample code, but the core of the idea is to set up an event loop for reading and writing (while loop with 10ms sleeping), and using low-level stream operations so that no caching and blocking is going on -- only try to read if you know the other process in fact wrote something (through InputStream.available()).
It leads to a bit strange programming style, but the code is much simpler than it would be if using threads, and does the job pretty well.