How to start `program1 | program2` with ProcessBuilder without using a shell? - java

I want to start two programs (as in ProcessBuilder), so that the first program's output is the second program's input. I additionaly want:
To avoid using shell (so that I can pass arguments with spaces with no escaping hassle);
To avoid all data flowing into parent Java process and back (i.e. having separate thread just to copy from one process's InputStream to the other process's OutputStream).
How to attain this?
Related: Building a process pipe with ProcessBuilder in Java 7 , but it uses shell...

Late answer; but you can use a named pipe. Create one first via mkfifo. Then in your Java code:
ProcessBuilder pb1 = new ProcessBuilder("gunzip", "-c", "log.gz");
ProcessBuilder pb2 = new ProcessBuilder("grep", "error");
File pipe = new File(NAMED_PIPE_PATH); // NAMED_PIPE_PATH points to what you created via mkfifo
pb1.redirectOutput(ProcessBuilder.Redirect.to(pipe));
pb2.redirectInput(ProcessBuilder.Redirect.from(pipe));
ProcessStartThread pst1 = new ProcessStartThread(pb1);
ProcessStartThread pst2 = new ProcessStartThread(pb2);
pst1.start();
pst2.start();
Note that ProcessStartThread is a simple custom class that extends Thread and basically does nothing other than call start() on the passed ProcessBuilder instance. The reason this has to be done from a separate thread is that ProcessBuilder.start() will hang until the other end of the named pipe is connected (it needs to open input/output to start execution).
edit: here is the ProcessStartThread class also:
class ProcessStartThread extends Thread {
private ProcessBuilder procBuilder;
private Process proc;
public ProcessStartThread(ProcessBuilder procBuilder) {
this.procBuilder = procBuilder;
this.proc = null;
}
public void run() {
try { this.proc = this.procBuilder.start(); }
catch ( Exception e ) { e.printStackTrace(); }
}
public Process getProcess() {
return this.proc;
}
}

Related

Opening new process in Java and keeping current one open

Got a task to make new version of our software to new client, and I need to make application behave so, that originally started application starts another and stays open at the back. At the moment original one is closed when new starts. Original application is kind of a loader, it's whole purpose is to download other applications from server and run one of them. That's why FileLock is used and that is probably the reason why I can't figure out how to keep the original open at the back and succesfully open the new one. I managed to make that work in Linux, but unfortunately our client uses Windows 10...
Some variables:
private final List<FileLock> locks = new ArrayList<FileLock>();
private final File applicationPath;
private final String application;
Loader constructor
public Loader(String[] args) throws IOException {
this.args = args;
applicationPath = new File(THIS_DIRECTORY, application + ".jar");
tryLock("loader");
tryLock(application);
}
load() is called in main after the constructor is made, nothing fancy there.
private void load() throws Exception
checkAndDownloadUpdate(application, applicationPath);
String javaBin = getJavaBinary();
List<String> command = new ArrayList<String>();
command.addAll(Arrays.asList(javaBin, THIS_FILE.getAbsolutePath(), "-jar", applicationPath.getAbsolutePath()));
command.addAll(Arrays.asList(args));
new ProcessBuilder(command).start();
}
TryLock:
private void tryLock(String name) {
File path = new File(THIS_DIRECTORY, name + "_lock");
long waitUntil = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(10);
while (System.currentTimeMillis() < waitUntil) {
try {
FileLock fileLock = tryLock(path);
if (fileLock != null) {
locks.add(fileLock);
return;
}
}
}
In Linux I was able to make application work as I wanted by replacing content of tryLock() with just return and adding waitFor() to Processbuilder's command. In windows new application doesn't start until first one is killed.
Problem seems to occur when new process is launched, in Windows 10 Java calls Unsafe.park() method and it halts until original process is closed.
I think it's because of the file handle in Windows which will prevent the second process to get a lock to the file.
Only when the first process release its lock the second will be able to get it.
I would try to avoid the file locks and only monitor the child process
ProcessBuilder builder = new ProcessBuilder("comman.exe");
Process process = builder.start();
if (process.isAlive()) {
//wait
}
So I had to ProcessBuilder's inheritIO method be able to open new process. Now it works as intended. Also added waitFor() for process:
ProcessBuilder builder = new ProcessBuilder(command);
builder.directory(new File(workingDir.toString()));
builder.inheritIO();
Process p = builder.start();
p.waitFor();

Creating named pipes in Java

I am experimenting with creating named pipes using Java. I am using Linux. However, I am running into a problem where writing to the pipe hangs.
File fifo = fifoCreator.createFifoPipe("fifo");
String[] command = new String[] {"cat", fifo.getAbsolutePath()};
process = Runtime.getRuntime().exec(command);
FileWriter fw = new FileWriter(fifo.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
bw.write(boxString); //hangs here
bw.close();
process.waitFor();
fifoCreator.removeFifoPipe(fifo.toString());
fifoCreator:
#Override
public File createFifoPipe(String fifoName) throws IOException, InterruptedException {
Path fifoPath = propertiesManager.getTmpFilePath(fifoName);
Process process = null;
String[] command = new String[] {"mkfifo", fifoPath.toString()};
process = Runtime.getRuntime().exec(command);
process.waitFor();
return new File(fifoPath.toString());
}
#Override
public File getFifoPipe(String fifoName) {
Path fifoPath = propertiesManager.getTmpFilePath(fifoName);
return new File(fifoPath.toString());
}
#Override
public void removeFifoPipe(String fifoName) throws IOException {
Files.delete(propertiesManager.getTmpFilePath(fifoName));
}
I am writing a string that consists of 1000 lines. Writing 100 lines work but 1000 lines doesn't.
However, if I run "cat fifo" on an external shell, then the program proceeds and writes everything out without hanging. Its strange how the cat subprocess launched by this program doesn't work.
EDIT: I did a ps on the subprocess and it has the status "S".
External processes have input and output that you need to handle. Otherwise, they may hang, though the exact point at which they hang varies.
The easiest way to solve your issue is to change every occurrence of this:
process = Runtime.getRuntime().exec(command);
to this:
process = new ProcessBuilder(command).inheritIO().start();
Runtime.exec is obsolete. Use ProcessBuilder instead.
UPDATE:
inheritIO() is shorthand for redirecting all of the Process's input and output to those of the parent Java process. You can instead redirect only the input, and read the output yourself:
process = new ProcessBuilder(command).redirectInput(
ProcessBuilder.Redirect.INHERIT).start();
Then you will need to read the process's output from process.getInputStream().

In Java, how do I read the content line by line of another console application that's running?

I have a .exe program called Blockland.exe running. It uses a game engine called TorqueScript but it is a console application that I think uses c++ to make it an application. People have made programs which stream the console of blockland.exe before but are nowhere to be found.
In general, how would you get the output of another running console application? Once I get the lines I know how to do what I need to do with it, but how would I get the output of the console application using Java?
ProcessBuilder in java provides a way to execute an external OS command and pipe input and read output from the external command. Here is a sample which does that:
List<String> command = new ArrayList<String>();
command.add("/path/to/Blockland.exe");
command.add("Other arguments");
ProcessBuilder procBuilder = new ProcessBuilder(command);
Process proc = procBuilder.start();
ProcessOutputReader outputReader =
new ProcessOutputReader(proc.getInputStream());
ProcessOutputReader errorReader =
new ProcessOutputReader(proc.getErrorStream());
Thread out = new Thread(outputReader);
out.start();
Thread error = new Thread(errorReader);
error.start();
proc.waitFor();
Where ProcessOutputReader is a class that implements Runnable. It starts reading from the stream provided in constructor until the end-of-file occurs in the stream. This is the thread that can read the output and process it. Note that you must process the streams using separate threads, since the buffer given by java/OS for the child processes are limited. If the child processes start putting too much data in either stream (error/output) and the buffer becomes full, it'll hang.
For example it may look like (implementation is not complete):
public class ProcessOutputReader
{
public void run()
{
int ch;
try
{
BufferedReader reader = new BufferedReader(new InputStreamReader(inpStream));
// The output can be of any size. So read in chunks
String line = reader.readLine();
while(line != null)
{
// handle line
line = reader.readLine();
}
}
catch (Exception e)
{
// handle exceptions
}
}
}

Approach to implement Windows cmd communication - multiple commands

I'm trying to find a solution how to implement a multiple command - response interaction with the Windows cmd shell. Example:
Start the cmd shell
"dir"
wait for and Handle input
Execute new command depending on the input content
wait for and Handle input
etc.
PLEASE NOTE! Steps above were only to describe the way of communication, it is NOT my intention to browse the file system, i.e. the actual commands could be something else.
Approach so far:
try {
Runtime rt = Runtime.getRuntime();
p = rt.exec("cmd");
error = p.getErrorStream();
input = p.getInputStream();
output = new PrintStream(p.getOutputStream());
StreamGobbler errGobbler = new StreamGobbler(error, "ERROR");
StreamGobbler inGobbler = new StreamGobbler(input, "INPUT");
errGobbler.start();
inGobbler.start();
output.println("dir");
output.flush();
sleep(5);
output.println("dir");
output.flush();
} catch (IOException e) {
System.out.println(e.printStackTrace());
}
StreamGobbler class:
class StreamGobbler extends Thread
{
InputStream is;
String type;
ArrayList<String> cmdRespArr = new ArrayList<String>();
StreamGobbler(InputStream is, String type) {
this.is = is;
this.type = type;
}
public void run() {
try {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader bf = new BufferedReader(isr);
String line = null;
while ( ( line = bf.readLine() ) != null ) {
cmdRespArr.add(line);
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
In this example however the while loop in the run method never returns between the issued commands (this is not part of the problem).
So, would the approach be to let the while method add the line read to a collection or other container, and then monitor that one for something indicating that the cmd shell is ready for input (which would in turn indicate that all available input from previous command have been read). And then fire off the next command?
In the example above this indication would get rid off the sleep call which right now is only there for debugging purposes.
I have a vague memory that this was the approach when doing it with Python.
Or is this totally wrong?
Will it be a solution to start multiple command processors, i.e. one per command?
I'm asking because with keeping one command processor open, it is very hard to determine when a command has been processed, unless you parse the output line by line and wait until you see the prompt in the output.
With multiple processors, i.e. executing "cmd /c dir" then input output redirs will close when the command has completed (and the associated process terminated).
Of course this will not work, if some commands depend on others, e.g. doing a chdir and expecting the next command to work in that dir.

Run another Java program from within a Java program and get outputs/send inputs

I need a way to run another java application from within my application. I want to recive its outputs into a JTextArea and send it inputs via a JTextBox.
Depends.
You could use a custom URLClassLoader to load the second applications jar and call the main classes main method directly. Obviously, the issue there is getting the output from the program ;)
The other solution would be to use a ProcessBuilder to launch a java process and read the output via the InputStream
The problem here is trying to find the java executable. In general, if it's in the path you should be fine.
You can have a look at this as a base line example of how to read the inputstream
UPDATED with Example
This is my "output" program that produces the output...
public class Output {
public static void main(String[] args) {
System.out.println("This is a simple test");
System.out.println("If you can read this");
System.out.println("Then you are to close");
}
}
This is my "reader" program that reads the input...
public class Input {
public static void main(String[] args) {
// SPECIAL NOTE
// The last parameter is the Java program you want to execute
// Because my program is wrapped up in a jar, I'm executing the Jar
// the command line is different for executing plain class files
ProcessBuilder pb = new ProcessBuilder("java", "-jar", "../Output/dist/Output.jar");
pb.redirectErrorStream();
InputStream is = null;
try {
Process process = pb.start();
is = process.getInputStream();
int value;
while ((value = is.read()) != -1) {
char inChar = (char)value;
System.out.print(inChar);
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
You can also checkout Basic I/O for more information

Categories