I'm trying to use Java's ProcessBuilder class to execute a command that has a pipe in it. For example:
ls -l | grep foo
However, I get an error:
ls: |: no such file or directory
Followed by:
ls: grep: no such file or directory
Even though that command works perfectly from the command line, I can not get ProcessBuilder to execute a command that redirects its output to another.
Is there any way to accomplish this?
This should work:
ProcessBuilder b = new ProcessBuilder("/bin/sh", "-c", "ls -l| grep foo");
To execute a pipeline, you have to invoke a shell, and then run your commands inside that shell.
The simplest way is to invoke the shell with the command line as the parameter. After all, it's the shell which is interpreting "|" to mean "pipe the data between two processes".
Alternatively, you could launch each process separately, and read from the standard output of "ls -l", writing the data to the standard input of "grep" in your example.
Since Java 9, there’s genuine support for piplines in ProcessBuilder.
So you can use
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("ls", "-l")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("grep", "foo")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
to get the matching lines in a list.
Or, for Windows
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("cmd", "/c", "dir")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("find", "\"foo\"")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
These examples redirect stdin of the first process and all error streams to inherit, to use the same as the Java process.
You can also call .redirectOutput(ProcessBuilder.Redirect.INHERIT) on the ProcessBuilder of the last process, to print the results directly to the console (or wherever stdout has been redirected to).
Related
I want to execute a batch file code from java button click. Also I don't want any command prompt window to be shown all from java code.
I have a code :-
C:\xyz-3.1.1\bin>dita --input=C:/Users/india/Desktop/mobile-phone/m
obilePhone.xyz --format=pdf --output=C:/Users/india/Desktop --logfile=C:/Use
rs/india/Desktop/dofhdif.txt
So I want above code to be run from batch command with C:\xyz-3.1.1\bin> as the parent directory.
Also I want to update --input file path whenever I will choose new file from JFileChooser.
I did this from the java code on button click transform:-
ProcessBuilder pb=new ProcessBuilder("dita --input=C:/Users/india/Desktop/mobile-phone/mobilePhone.xyz --format=pdf --output=C:/Users/india/Desktop --logfile=C:/Users/india/Desktop/dofhdif.txt");
pb.redirectErrorStream(true);
Process process=pb.start();
and getting IOException error.
I get stuck over here for long time , where am I going wrong.
EDIT :- error
java.io.IOException: Cannot run program "dita --input=C:/Users/india/Desktop/mobile-phone/m
obilePhone.xyz --format=pdf --output=C:/Users/india/Desktop --logfile=C:/Use
rs/india/Desktop/dofhdif.txt": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
As the Error mentioned, it cannot locate the command because the whole string will be treated as the command by ProcessBuilder.
Try to use Runtime.getRuntime().exec directly, but you have to ensure the command dita can be found.
Process process = Runtime.getRuntime().exec("C:\xyz-3.1.1\bin>dita --input=C:/Users/india/Desktop/mobile-phone/mobilePhone.xyz --format=pdf --output=C:/Users/india/Desktop --logfile=C:/Users/india/Desktop/dofhdif.txt");
process.waitFor();
int exitCode = process.exitValue();
System.out.println(IoHelper.output(process.getInputStream())); // handle the output;
Before JDK 5.0, the only way to start a process and execute it, was to use the exec() method of the java.lang.Runtime class after which ProcessBuilder can be used to help create operating system processes.
The major improvement being that, it also acts as a holder for all those attributes that influence the process. And this is how it should be used:
ProcessBuilder pb = new ProcessBuilder("myCommand", "myArg1", "myArg2");
P.S. Actually Runtime.getRuntime().exec can also be used with String... as:
Runtime.getRuntime().exec(new String[]{"curl", "-v", "--cookie", tokenString, urlString});
My personal preference:
If you have to configure the environment for the command: to control the working directory or environment variables and also you want to execute the commands several times, you'd better use it since the ProcessBuilder will hold the settings and what you need to do is just processBuilder.start() to create another process with the same settings;
If you want to execute a whole long string command as you mentioned, you'd better just use Runtime.getRuntime().exec since you can just execute it right there without any bothering of the parameter format.
Try this:
String inputFile = ...;
String outputFile = ...;
String logFile = ...;
ProcessBuilder pb = new ProcessBuilder(
"dita",
"--input=" + inputFile,
"--format=pdf",
"--output=" + outputFile,
"--logfile=" + logFile)
.directory(new File("C:\\xyz-3.1.1\\bin"))
//.inheritIO();
.redirectErrorStream(true);
Process process = pb.start();
This shows the following points:
The command is separated from the arguments
The argument values can be determined at runtime
The command's default directory (C:\xyz-3.1.1\bin) is set before starting the process
Consider using inheritIO() instead of redirectErrorStream() if you want the process's output to appear as part of your Java application's output.
I'm trying to run other java file using ProcessBuilder class.
I would like to get input of entire path of java file + file name + .java and compile it.
Example, input: C:\Windows\test.java
And then, I store input into String variable FILE_LOCATION and call processbuilder to compile input .java file.
Here is my code:
static String JAVA_FILE_LOCATION;
static String command[] = {"javac", JAVA_FILE_LOCATION};
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process process = processBuilder.start();
process = new ProcessBuilder(new String[]{"java","-cp",A,B}).start();
But I don't know how to set the parameters.
process = new ProcessBuilder(new String[]{
"java","-cp",A,B}).start();
How should I set that parameter (A, B)?
To answer your exact question, let's say, for instance, your class is in package com.yourcompany.yourproduct and your class file is in /dir/to/your/classes/com/yourcompany/yourproduct/Yourclass.class.
Then A = "/dir/to/your/classes" and B = "com.yourcompany.yourproduct.Yourclass".
However, there's a few things to be aware of. Looking at your code:
static String JAVA_FILE_LOCATION;
static String command[] = {"javac", JAVA_FILE_LOCATION};
ProcessBuilder processBuilder = new ProcessBuilder(command);
No. You need to CD to the directory and then run javac. The easiest way to do that is by calling processBuilder.directory(new File("/dir/to/your/classes")). Then you need to give javac a relative path to your source file ("com/yourcompany/yourproduct/Yourclass.java").
Process process = processBuilder.start();
process = new ProcessBuilder(new String[]{"java","-cp",A,B}).start();
Wait until the first process has finished compiling before you try to run it! Between the two lines above, insert process.waitFor();. You might also want to check for any errors and only run the second process if the first has succeeded.
By the way, there's no need for all that long-hand creation of string arrays. Just use varargs: process = new ProcessBuilder("java", "-cp", A, B).start();.
I am running the following code, and it stops at waitfor() function. What could be the reason and how can I solve it?
String line;
Process albumProcess;
try {
albumProcess = Runtime.getRuntime().exec(
"iconv -f UTF-16 -t UTF-8 /home/gozenem/"+ xmlFileName +
".xml | grep albumID");
albumProcess.waitFor();
BufferedReader in = new BufferedReader(
new InputStreamReader(albumProcess.getInputStream()));
ArrayList<String> lineList = new ArrayList<String>();
while ((line = in.readLine()) != null) {
lineList.add(line);
}
result[0] = lineList.size();
albumProcess.destroy();
} catch (Exception e) {}
The | grep ... is not consuming the output from the command as you expect because getRuntime().exec does not understand piping symbols. The process gets bogged down waiting for something to consume its output and its getting passed bogus command line arguments "|", "grep", and "albumId".
A shell will understand | but execv will not, so you need to use bash -c instead to get a shell to do the piping (see java shell for executing/coordinating processes? do the piping yourself (see Pipe between java processes on command shell not reliable working). Java 7 has a new ProcessBuilder class that makes it easy to set up pipes so you can use those if you're only running on a bleeding edge JVM.
Once you've got grep running, if there's a bunch of lines that match, it may still fill up the buffer, so you need something sitting on the buffer consuming the process's output stream. Moving
albumProcess.waitFor();
after the while loop should do it.
I think you should try to read the output from the process before waiting on it. Otherwise, if the command outputs to much then the buffer may get filled.
Have a look at this article which explains how to read from the process: http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4
I am trying to run following command in my Java program
Runtime.getRuntime().exec("ls -lrt service/logs/post/level2.log | awk '{print $9}'");
or
Runtime.getRuntime().exec("ls -lrt service/logs/post/level2* | awk '{print $9}'");
it gives me following error
ls: 0653-341 The file | does not exist.
ls: 0653-341 The file awk does not exist.
ls: 0653-341 The file '{print does not exist.
ls: 0653-341 The file $9}' does not exist.
Kindly help me
Pipes are a shell-based construct, not actual runnable commands. There are two options to do this as I see it:
Do the piping yourself within Java. Invoke the ls command first, get a handle to its OutputStream and then invoke awk connecting the first Process's output to the second Process' input stream.
Invoke the bash shell directly from Java, passing in the entire command as an argument to bash's -c parameter. That way all the piping is done within the single process.
As for the token-based errors, you should be invoking these commands with an array of strings; each element represents a token of the command line. So try, for example:
Runtime.getRuntime().exec(new String[] { "ls", "-lrt", "service/logs/post/level2.log" });
in order to invoke the ls command. I don't think this is strictly necessary in this case, but it will be for the awk command, since Java doesn't know anything about shell-specific quoting rules, so by default tokenises a single-string input on the space character. That's why your awk script was being split in two.
Edit (in response to comments): In the first option, I meant simply that you're able to pipe the output between the two processes yourself, in Java.
Imagine if you've created a process as so:
Process ls = Runtime.getRuntime().exec("ls -lrt service/logs/post/level2.log");
Now, this process will run and generate some output (which we know is going to be a line describing that file). We can get the stream for this output like so:
InputStream lsOut = ls.getInputStream();
Now, we want to run that awk process:
Process awk = Runtime.getRuntime().exec(new String[] { "awk", "{print $9}"});
The awk process of course will sit there are the moment waiting for input, since it knows it's going to be reading from stdin. So, we grab the input stream that it's going to be using:
OutputStream awkIn = awk.getOutputStream();
Now, the piping bit - we read the output of the ls command and pass it into the input for awk:
// TODO add buffering, error handling, probably run this in a separate thread
int datum = lsOut.read();
while (datum != -1)
{
awkIn.write(datum);
datum = lsOut.read();
}
This reads the output of ls (byte-by-byte for simplicity, using byte array buffers would be much faster but I'm trying to illustrate the concept simply) and writes it to the input of awk.
Then it's just a matter of reading the output from the awk process and dealing with it as you see fit.
I'm calling command line programs connected by pipes. All this works on Linux for sure.
My method:
protected String execCommand(String command) throws IOException {
String line = null;
if (command.length() > 0) {
Process child = Runtime.getRuntime().exec(command);
InputStream lsOut = child.getInputStream();
InputStreamReader r = new InputStreamReader(lsOut);
BufferedReader in = new BufferedReader(r);
String readline = null;
while ((readline = in.readLine()) != null) {
line = line + readline;
}
}
return line;
}
If I'm calling some cat file | grep asd, I'm getting the expected result. But not all commands works correctly. For example with this:
cat /proc/cpuinfo | wc -l
or this:
cat /proc/cpuinfo | grep "model name" | head -n 1 | awk -F":" '{print substr($2, 2, length($2))}
the method will return null. I'm guessing this problem depends on output formatting commands like head, tail, wc, etc. How I can work around this problem and get the final result of the output?
The pipe (like redirection, or >) is a function of the shell, and so execing directly from Java won't work. You need to do something like:
/bin/sh -c "your | piped | commands | here"
which executes a shell process with the command line (including pipes) specified after the -c (in quotes).
Note also that you have to consume stdout and stderr concurrently, otherwise your spawned process will block waiting for your process to consume the output (or errors). More info here.
Everyone who uses Runtime.exec should read this.
It might be a good idea to check the error stream of the Process as well.
Still didn't found proper solution to execute piped commands with Runtime.exec, but found a workaround. I've simply wrote these scripts to separate bash files. Then Runtime.exec calls these bash scripts and gets expected result.
The quick-and-dirty thing to do would be:
command = "/bin/sh -c '" + command.replaceAll("'", "'\''") + "'"
Normally, you'll have to watch out for shell injection (i.e. someone sneaks "; rm -rf /;" into the command). But that's only an issue if part of the command can be supplied from some other user input.
The slow and painful approach would be to do the Bash piping yourself in Java. If you go down this road, you'll find out all the wonderful things that Bash gives you that's not directly available from Process.exec (pipes, redirection, compound commands, variable expansion, arithmetic evaluation, ...).
Parse the command for | characters. Be sure to watch out for || and quoted strings.
Spawn a new Process for every piped command.
Create Threads that read the output from one command and write it to the input of the next command.
Probably a little too late but for others looking for a solution, try this...
String[] cmd = {
"/bin/sh",
"-c",
"cat /proc/cpuinfo | wc -l"
};
Process process = Runtime.getRuntime().exec(cmd);
All the best..