I am getting an exception like java.io.IOException: Cannot run program cat /home/talha/* | grep -c TEXT_TO_SEARCH": error=2, No such file or directory while executing the command below despite that there are no issues when I execute the same command through the terminal. I need to execute and return the output of the command below:
cat /home/talha/* | grep -c TEXT_TO_SEARCH
Here is the method used to execute commands using Runtime class:
public static String executeCommand(String command) {
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine()) != null) {
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
}
Runtime.exec does not use a shell (like, say, /bin/bash); it passes the command directly to the operating system. This means wildcards like * and pipes (|) will not be understood, since cat (like all Unix commands) does not do any parsing of those characters. You need to use something like
p = new ProcessBuilder("bash", "-c", command).start();
or, if for some bizarre reason you need to stick to using the obsolete Runtime.exec methods:
p = Runtime.getRuntime().exec(new String[] { "bash", "-c", command });
If you are only running that cat/grep command, you should consider abandoning the use of an external process, since Java code can easily traverse a directory, read lines from each file, and match them against a regular expression:
Pattern pattern = Pattern.compile("TEXT_TO_SEARCH");
Charset charset = Charset.defaultCharset();
long count = 0;
try (DirectoryStream<Path> dir =
Files.newDirectoryStream(Paths.get("/home/talha"))) {
for (Path file : dir) {
count += Files.lines(file, charset).filter(pattern.asPredicate()).count();
}
}
Update: To recursively read all files in a tree, use Files.walk:
try (Stream<Path> tree =
Files.walk(Paths.get("/home/talha")).filter(Files::isReadable)) {
Iterator<Path> i = tree.iterator();
while (i.hasNext()) {
Path file = i.next();
try (Stream<String> lines = Files.lines(file, charset)) {
count += lines.filter(pattern.asPredicate()).count();
}
};
}
$PATH is an environment variable that tells the system where to search for executable programs (it's a list of directories separated by colons). It is usually set in your .bashrc or .cshrc file but this is only loaded when you log in. When Java runs, $PATH is likely not set because the rc file is not executed automatically, so the system can't find programs without specifying exactly where they are. Try using /bin/cat or /usr/bin/cat instead of just cat and see if it works. If it does, $PATH is your problem. You can add $PATH=/bin:/usr/bin to your script or just leave it with the directory name specified (e.g. /bin/cat).
Just because you can execute it in a login session doesn't mean it will work the same when a daemon like your Java program runs. You have to know what's in your .bashrc or .cshrc file and even sometimes how the system file is written (/etc/bashrc) in order to know how to write a script that runs under a daemon. Another consideration is that daemons often run under the context of a different user, and that throws things off, too.
Related
I have simple java program for compiling java classes.
I created a JAR of this program and when I run it on Ubuntu I pass to the jar the path of folder with java files.
public class Main {
public static void main(String[] args) throws IOException, InterruptedException {
compile(args[0]);
}
//pathToFiles - is a value from command line arguments
private static void compile(String pathToFiles) throws IOException, InterruptedException {
List<String> cmdList = new ArrayList<>();
cmdList.add("javac");
cmdList.add(pathToFiles);
System.out.println("cmd: "+cmdList);
ProcessBuilder pb = new ProcessBuilder(cmdList);
Process process = pb.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
generateCompileException(process);
}
}
//method just generates error message if there was an error
private static void generateCompileException(Process process){
StringBuilder response = new StringBuilder();
try (final BufferedReader b = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
String line;
if ((line = b.readLine()) != null)
response.append(line);
} catch (final IOException e) {
e.printStackTrace();
}
throw new RuntimeException(response.toString());
}
}
When I pass path containing single java file it works:
java -jar co-1.jar /home/admin/test2/Calculator.java
But I want to compile multiple files. When I pass path containing multiple files I get error: file not found.
java -jar co-1.jar '/home/admin/test2/*.java'
PS: If I run a javac command manually with multiple files, it will work:
###################################
UPDATE:
I've added bash command to ProcessBuilder:
private static void compile(String pathToFiles) throws IOException, InterruptedException {
List<String> cmdList = new ArrayList<>();
cmdList.add("bash");
cmdList.add("-c");
cmdList.add("javac");
cmdList.add(pathToFiles);
System.out.println("Processor builder command: "+cmdList);
ProcessBuilder pb = new ProcessBuilder(cmdList);
Process process = pb.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
System.out.println("Finished with error. Exit value: "+exitValue);
generateCompileException(process);
}
}
But process withished with error code 2 with empty response from ProcessBuilder.
PS: RuntimeException was thrown by this line: throw new RuntimeException(response.toString());
ProcessBuilder will not evaluate wildcards, as that is a feature of your terminal (such as bash). If you want wildcard to be expanded you need to run bash inside ProcessBuilder command, such as:
String commandContainingWildcard = "javac /blah/*.java";
ProcessBuilder pb = new ProcessBuilder("bash", "-c", commandContainingWildcard);
... // start() etc
For the above to work you need to have "bash" or whatever shell you use in your path, otherwise you will need to use full path to bash (such as "/bin/bash").
The third argument for command to compile must exactly match what works inside your terminal and must be the entire value not "javac" followed by wildcard. Remove single quotes around *.java (so that ProcessBuilder is provided with three command line parameters, not four or more).
However I suggest that ProcessBuilder with bash isn't the best way to do this work. You could try Java compiler tool interface, and get rid of wildcard by easy use of Files.find(dir, 1, (p,a) -> p.getFileName().toString().endsWith(".java")) to scan for all java files and join the paths explicitly for compilation.
UPDATE
Having now resolved your problem you may now find that the javac process fails / freezes due the incorrect way you read the stderr stream - this needs to happen at same time as stdout and before process.waitFor(). An easy fix is to consume stdout+stderr together:
ProcessBuilder pb = new ProcessBuilder(cmdList);
pb.redirectErrorStream(true);
Process process = pb.start();
ByteArrayOutputStream response = new ByteArrayOutputStream();
process.getInputStream().transferTo(response);
int exitValue = process.waitFor();
if (exitValue != 0) {
System.out.println("Finished with error. Exit value: "+exitValue);
throw new RuntimeException(new String(response.toByteArray()));
}
Remove quotes and use the command as below.
java -cp co-1.jar:/home/admin/test2/* Main.class <args>
See also
https://docs.oracle.com/javase/7/docs/technotes/tools/windows/classpath.html
PS: Unix uses :(colon) as delimiter and windows uses ;(semi-colon) delimiter to separate multiple paths.
I am new to both Java and Linux, I was trying to use some Runtime.exec() commands that would allow my program to execute commands in Linux such as "cd /mnt/" and "ls --group-directories-first" to list files and directories contained in /mnt/ but I think I am making a problem with the execution.
I tried my code to only include the "ls --group-directories-first" and it worked like a charm, only problem was, it only listed subdirectories and files in the projects folder. I wanted to make my program go to /mnt/ first so I made my command line to a command array by using exec(String[] cmdarray) format as process1 = Runtime.getRuntime().exec(new String[]{"cd /mnt/","ls --group-directories-first"}); and when I ran it on linux, it just got executed without any printed runtime errors but also without any feedback/printed lines.
Here is my code:
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class linCom {
public static void main(String args[]) {
String s;
Process p;
try {
p = Runtime.getRuntime().exec("ls --group-directories-first");
BufferedReader br = new BufferedReader(
new InputStreamReader(p.getInputStream()));
while ((s = br.readLine()) != null)
System.out.println("line: " + s);
p.waitFor();
System.out.println ("exit: " + p.exitValue());
p.destroy();
} catch (Exception e) {}
}
}
This worked and printed out:
"line: DummyFolder1
line: linCom.class
line: linCom.java
exit: 0"
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class linCom {
public static void main(String args[]) {
String s;
Process p;
try {
p = Runtime.getRuntime().exec(new String[]{"cd /mnt/","ls --group-directories-first"});
BufferedReader br = new BufferedReader(
new InputStreamReader(p.getInputStream()));
while ((s = br.readLine()) != null)
System.out.println("line: " + s);
p.waitFor();
System.out.println ("exit: " + p.exitValue());
p.destroy();
} catch (Exception e) {}
}
}
This just got executed with no printed lines.
I expected my program to just go to the /mnt/ directory and print out subdirectories and files on there, but it just got executed with no visible runtime errors and no printed lines.
I have looked at other entries but could not find any answer to my problem.
EDIT: I changed "no errors" with "no error messages" to make it clear that if program had any errors, I did not get any feedback about it.
Here's where the UNIX process model can be confusing.
What you have tries to run the program named cd /mnt/ with the first parameter of ls --group-directories-first . Unix programs can be named anything (they're just filenames) but there's no program named cd /mnt. And anyway, the cd operation is actually performed by a shell, not as a forked/execed program.
You hope to run this shell command from your Java program: cd /mnt/; ls --group-directories-first . The trouble is, Java's .exec() method does not give you a shell, so shell commands don't work.
You can try this instead. It's like running the shell command
/bin/sh -c "cd /mnt/; ls --group-directories-first"
With this, you start a shell, then tell it to run -cthe command you want.
Process p = Runtime.getRuntime().exec(new String[]{"/bin/sh",
"-c",
"cd /mnt/; ls --group-directories-first"});
But it's quite dependent on the machine where your Java program runs, so be careful.
Reference: How to invoke a Linux shell command from Java
Do not use an external process just to list files. Java has plenty of ways to do that. All of them are in the Files class. For example:
Path dir = Paths.get("/mnt");
try (Stream<Path> files = Files.list(dir).sorted(
Comparator.comparing((Path p) -> !Files.isDirectory(p))
.thenComparing(Comparator.naturalOrder()))) {
files.forEach(System.out::println);
}
Do you really need to use Runtime.exec()comands? That would make your code platafform dependent.
You could use File.listFiles():
File folder = new File("/mnt");
for (File f : folder.listFiles()) {
System.out.println(f.getName());
}
That would make the code less plataform dependent
Consider the following code:
String commandf = "ls /etc | grep release";
try {
// Execute the command and wait for it to complete
Process child = Runtime.getRuntime().exec(commandf);
child.waitFor();
// Print the first 16 bytes of its output
InputStream i = child.getInputStream();
byte[] b = new byte[16];
i.read(b, 0, b.length);
System.out.println(new String(b));
} catch (IOException e) {
e.printStackTrace();
System.exit(-1);
}
The program's output is:
/etc:
adduser.co
When I run from the shell, of course, it works as expected:
poundifdef#parker:~/rabbit_test$ ls /etc | grep release
lsb-release
The internets tell me that, due to the fact that pipe behavior isn't cross-platform, the brilliant minds who work in the Java factory producing Java can't guarantee that pipes work.
How can I do this?
I am not going to do all of my parsing using Java constructs rather than grep and sed, because if I want to change the language, I'll be forced to re-write my parsing code in that language, which is totally a no-go.
How can I make Java do piping and redirection when calling shell commands?
Write a script, and execute the script instead of separate commands.
Pipe is a part of the shell, so you can also do something like this:
String[] cmd = {
"/bin/sh",
"-c",
"ls /etc | grep release"
};
Process p = Runtime.getRuntime().exec(cmd);
I ran into a similar problem in Linux, except it was "ps -ef | grep someprocess".
At least with "ls" you have a language-independent (albeit slower) Java replacement. Eg.:
File f = new File("C:\\");
String[] files = f.listFiles(new File("/home/tihamer"));
for (String file : files) {
if (file.matches(.*some.*)) { System.out.println(file); }
}
With "ps", it's a bit harder, because Java doesn't seem to have an API for it.
I've heard that Sigar might be able to help us:
https://support.hyperic.com/display/SIGAR/Home
The simplest solution, however, (as pointed out by Kaj) is to execute the piped command as a string array. Here is the full code:
try {
String line;
String[] cmd = { "/bin/sh", "-c", "ps -ef | grep export" };
Process p = Runtime.getRuntime().exec(cmd);
BufferedReader in =
new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
} catch (Exception ex) {
ex.printStackTrace();
}
As to why the String array works with pipe, while a single string does not... it's one of the mysteries of the universe (especially if you haven't read the source code). I suspect that it's because when exec is given a single string, it parses it first (in a way that we don't like). In contrast, when exec is given a string array, it simply passes it on to the operating system without parsing it.
Actually, if we take time out of busy day and look at the source code
(at http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Runtime.java#Runtime.exec%28java.lang.String%2Cjava.lang.String[]%2Cjava.io.File%29), we find that is exactly what is happening:
public Process [More ...] exec(String command, String[] envp, File dir)
throws IOException {
if (command.length() == 0)
throw new IllegalArgumentException("Empty command");
StringTokenizer st = new StringTokenizer(command);
String[] cmdarray = new String[st.countTokens()];
for (int i = 0; st.hasMoreTokens(); i++)
cmdarray[i] = st.nextToken();
return exec(cmdarray, envp, dir);
}
Create a Runtime to run each of the process. Get the OutputStream from the first Runtime and copy it into the InputStream from the second one.
#Kaj accepted answer is for linux. This is the equivalent one for Windows:
String[] cmd = {
"cmd",
"/C",
"dir /B | findstr /R /C:"release""
};
Process p = Runtime.getRuntime().exec(cmd);
Here's the situation. Im creating a UI which will allow make using a genetic programming system (ECJ) easier to use.
Currently you need to run a command prompt within the ECJ folder and use the commands similar to this to execute a parameter file.
java ec.Evolve -file ec\app\tutorial5\tutorial5.params
Where the full path of tutorial5 is
C:\Users\Eric\Documents\COSC\ecj\ec\app\tutorial5\tutorial5.params
and the command prompt must be executed from
C:\Users\Eric\Documents\COSC\ecj
My program makes the user select a .params file (which is located in a ecj subdirectory) and then use the Runtime.exec() to execute
java ec.Evolve -file ec\app\tutorial5\tutorial5.params
What i have so far
// Command to be executed
String cmd = "cd " + ecjDirectory;
String cmd2 = "java ec.Evolve -file " + executeDirectory;
System.out.println(cmd);
try {
Process p = Runtime.getRuntime().exec(
new String[]{"cmd.exe", "/c", cmd, cmd2});
BufferedReader r = new BufferedReader(new InputStreamReader(p.getInputStream()));
statusTF.append(r.readLine());
p.waitFor();
} catch (IOException | InterruptedException ex) {
System.out.println("FAILED: " + ex.getMessage());
statusTF.append("Failed\n");
}
Currently it outputs the change directory command but nothing else.
Can this be done?
First, the 'cd' command can't be executed by Runtime.exec() in the first place (see How to use "cd" command using Java runtime?). You should be able to just set the working directory for the process when you call exec (see below).
Second, running 'cmd.exe /c' to execute your process isn't what you want here. You won't be able to get the results of your process running, because that is returned to the command window -- which eats the error and then closes without passing the error along to you.
Your exec command should look more like this:
Process p = Runtime.getRuntime().exec(
command, null, "C:\Users\Eric\Documents\COSC\ecj");
Where 'command' looks like this:
String command = "java ec.Evolve -file ec\app\tutorial5\tutorial5.params"
Edit: For reading error messages, try this:
String error = "";
try (InputStream is = proc.getErrorStream()) {
error = IOUtils.toString(is, "UTF-8");
}
int exit = proc.waitFor();
if (exit != 0) {
System.out.println(error);
} else {
System.out.println("Success!");
}
You can use Java processbuilder:
processBuilder documentation!
you can define the working directory of the process and all other stuff.
Each call to exec() runs in a new environment, this means that the call to cd will work, but will not exist to the next call to exec().
I prefer to use Apache's Commons Exec, it's provides an excellent facade over Java's Runtime.exec() and gives a nice way to specify the working directory. Another very nice thing is they provide utilities to capture standard out and standard err. These can be difficult to properly capture yourself.
Here's a template I use. Note that this sample expects an exit code of 0, your application may be different.
String sJavaPath = "full\path\to\java\executable";
String sTutorialPath = "C:\Users\Eric\Documents\COSC\ecj\ec\app\tutorial5\tutorial5.params";
String sWorkingDir = "C:\Users\Eric\Documents\COSC\ecj";
try (
OutputStream out = new ByteArrayOutputStream();
OutputStream err = new ByteArrayOutputStream();
)
{
// setup watchdog and stream handler
ExecuteWatchdog watchdog = new ExecuteWatchdog(Config.TEN_SECONDS);
PumpStreamHandler streamHandler = new PumpStreamHandler(out, err);
// build the command line
CommandLine cmdLine = new CommandLine(sJavaPath);
cmdLine.addArgument("ec.Evolve");
cmdLine.addArgument("-file");
cmdLine.addArgument(sTutorialPath);
// create the executor and setup the working directory
Executor exec = new DefaultExecutor();
exec.setExitValue(0); // tells Executor we expect a 0 for success
exec.setWatchdog(watchdog);
exec.setStreamHandler(streamHandler);
exec.setWorkingDirectory(sWorkingDir);
// run it
int iExitValue = exec.execute(cmdLine);
String sOutput = out.toString();
String sErrOutput = err.toString();
if (iExitValue == 0)
{
// successful execution
}
else
{
// exit code was not 0
// report the unexpected results...
}
}
catch (IOException ex)
{
// report the exception...
}
Consider the following code:
String commandf = "ls /etc | grep release";
try {
// Execute the command and wait for it to complete
Process child = Runtime.getRuntime().exec(commandf);
child.waitFor();
// Print the first 16 bytes of its output
InputStream i = child.getInputStream();
byte[] b = new byte[16];
i.read(b, 0, b.length);
System.out.println(new String(b));
} catch (IOException e) {
e.printStackTrace();
System.exit(-1);
}
The program's output is:
/etc:
adduser.co
When I run from the shell, of course, it works as expected:
poundifdef#parker:~/rabbit_test$ ls /etc | grep release
lsb-release
The internets tell me that, due to the fact that pipe behavior isn't cross-platform, the brilliant minds who work in the Java factory producing Java can't guarantee that pipes work.
How can I do this?
I am not going to do all of my parsing using Java constructs rather than grep and sed, because if I want to change the language, I'll be forced to re-write my parsing code in that language, which is totally a no-go.
How can I make Java do piping and redirection when calling shell commands?
Write a script, and execute the script instead of separate commands.
Pipe is a part of the shell, so you can also do something like this:
String[] cmd = {
"/bin/sh",
"-c",
"ls /etc | grep release"
};
Process p = Runtime.getRuntime().exec(cmd);
I ran into a similar problem in Linux, except it was "ps -ef | grep someprocess".
At least with "ls" you have a language-independent (albeit slower) Java replacement. Eg.:
File f = new File("C:\\");
String[] files = f.listFiles(new File("/home/tihamer"));
for (String file : files) {
if (file.matches(.*some.*)) { System.out.println(file); }
}
With "ps", it's a bit harder, because Java doesn't seem to have an API for it.
I've heard that Sigar might be able to help us:
https://support.hyperic.com/display/SIGAR/Home
The simplest solution, however, (as pointed out by Kaj) is to execute the piped command as a string array. Here is the full code:
try {
String line;
String[] cmd = { "/bin/sh", "-c", "ps -ef | grep export" };
Process p = Runtime.getRuntime().exec(cmd);
BufferedReader in =
new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
} catch (Exception ex) {
ex.printStackTrace();
}
As to why the String array works with pipe, while a single string does not... it's one of the mysteries of the universe (especially if you haven't read the source code). I suspect that it's because when exec is given a single string, it parses it first (in a way that we don't like). In contrast, when exec is given a string array, it simply passes it on to the operating system without parsing it.
Actually, if we take time out of busy day and look at the source code
(at http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Runtime.java#Runtime.exec%28java.lang.String%2Cjava.lang.String[]%2Cjava.io.File%29), we find that is exactly what is happening:
public Process [More ...] exec(String command, String[] envp, File dir)
throws IOException {
if (command.length() == 0)
throw new IllegalArgumentException("Empty command");
StringTokenizer st = new StringTokenizer(command);
String[] cmdarray = new String[st.countTokens()];
for (int i = 0; st.hasMoreTokens(); i++)
cmdarray[i] = st.nextToken();
return exec(cmdarray, envp, dir);
}
Create a Runtime to run each of the process. Get the OutputStream from the first Runtime and copy it into the InputStream from the second one.
#Kaj accepted answer is for linux. This is the equivalent one for Windows:
String[] cmd = {
"cmd",
"/C",
"dir /B | findstr /R /C:"release""
};
Process p = Runtime.getRuntime().exec(cmd);