We Keep Coding

java.lang.Runtime exception "Cannot run program"

I am getting an exception like java.io.IOException: Cannot run program cat /home/talha/* | grep -c TEXT_TO_SEARCH": error=2, No such file or directory while executing the command below despite that there are no issues when I execute the same command through the terminal. I need to execute and return the output of the command below:
cat /home/talha/* | grep -c TEXT_TO_SEARCH
Here is the method used to execute commands using Runtime class:
public static String executeCommand(String command) {
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine()) != null) {
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
}

Runtime.exec does not use a shell (like, say, /bin/bash); it passes the command directly to the operating system. This means wildcards like * and pipes (|) will not be understood, since cat (like all Unix commands) does not do any parsing of those characters. You need to use something like
p = new ProcessBuilder("bash", "-c", command).start();
or, if for some bizarre reason you need to stick to using the obsolete Runtime.exec methods:
p = Runtime.getRuntime().exec(new String[] { "bash", "-c", command });
If you are only running that cat/grep command, you should consider abandoning the use of an external process, since Java code can easily traverse a directory, read lines from each file, and match them against a regular expression:
Pattern pattern = Pattern.compile("TEXT_TO_SEARCH");
Charset charset = Charset.defaultCharset();
long count = 0;
try (DirectoryStream<Path> dir =
Files.newDirectoryStream(Paths.get("/home/talha"))) {
for (Path file : dir) {
count += Files.lines(file, charset).filter(pattern.asPredicate()).count();
}
}
Update: To recursively read all files in a tree, use Files.walk:
try (Stream<Path> tree =
Files.walk(Paths.get("/home/talha")).filter(Files::isReadable)) {
Iterator<Path> i = tree.iterator();
while (i.hasNext()) {
Path file = i.next();
try (Stream<String> lines = Files.lines(file, charset)) {
count += lines.filter(pattern.asPredicate()).count();
}
};
}

$PATH is an environment variable that tells the system where to search for executable programs (it's a list of directories separated by colons). It is usually set in your .bashrc or .cshrc file but this is only loaded when you log in. When Java runs, $PATH is likely not set because the rc file is not executed automatically, so the system can't find programs without specifying exactly where they are. Try using /bin/cat or /usr/bin/cat instead of just cat and see if it works. If it does, $PATH is your problem. You can add $PATH=/bin:/usr/bin to your script or just leave it with the directory name specified (e.g. /bin/cat).
Just because you can execute it in a login session doesn't mean it will work the same when a daemon like your Java program runs. You have to know what's in your .bashrc or .cshrc file and even sometimes how the system file is written (/etc/bashrc) in order to know how to write a script that runs under a daemon. Another consideration is that daemons often run under the context of a different user, and that throws things off, too.

Java - run Python script and monitor continuous output

I'm using a Raspberry Pi to receive the UID of some RFID cards from a RC522 reader. The python script I'm running is here: https://github.com/mxgxw/MFRC522-python
For various reasons I won't go into, I have to process these IDs in Java.
It seems the most viable solution is to run the python script and read in the result into Java. The problem is, the Python code gives continuous output, i.e. it will print the ID of the card into the console window as and when a card is tapped onto the reader, and will only terminate on a user's command.
I'm currently using a ProcessBuilder to execute the script, however it seems like it's more suited to run the program and read in the immediate result back to Java (which of course is null if I haven't tapped a card onto the reader). I've tried executing the code in a while(true) loop to continuously start the process - but this doesn't work:
import java.io.*;
public class PythonCaller {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// set up the command and parameter
String pythonScriptPath = "/home/pi/MFRC522-python/Read.py";
String[] cmd = new String[3];
cmd[0] = "sudo";
cmd[1] = "python"; // check version of installed python: python -V
cmd[2] = pythonScriptPath;
// create runtime to execute external command
ProcessBuilder pb = new ProcessBuilder(cmd);
// retrieve output from python script
pb.redirectError();
while(true){
Process p = pb.start();
System.out.println("Process Started...");
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
int ret = new Integer(in.readLine()).intValue();
System.out.println("value is : "+ret);
}
}
}
The output on the console window is blank - no exceptions thrown or println's.
Any help would be massively appreciated!!
Thanks
EDIT - I've surrounded my code in a try/catch to see if there's anything at all being thrown, and it doesn't seem to be the case

I use the following programs to try to reproduce the problem
PythonCaller.java
import java.io.*;
public class PythonCaller {
public static void main(String[] args) throws IOException {
// set up the command and parameter
String pythonScriptPath = "/home/pi/test.py";
String[] cmd = { "python", pythonScriptPath };
// create runtime to execute external command
ProcessBuilder pb = new ProcessBuilder(cmd);
// retrieve output from python script
pb.redirectError();
while(true){
Process p = pb.start();
System.out.println("Process Started...");
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
int ret = new Integer(in.readLine()).intValue();
System.out.println("value is : "+ret);
}
}
}
test.py
uid =(123,456,789,999)
print "Card read UID: "+str(uid[0])+","+str(uid[1])+","+str(uid[2])+","+str(uid[3])
The method pb.redirectError() doesn't modify anything. It returns a value, your codes does nothing with it. (see http://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html#redirectError%28%29). What you want is probably redirectErrorStream(boolean redirectErrorStream)
The second line of the python test program is taken directly from "Read.py" (line 44). It causes an error with the java intValue() method. If I replace it with String ret = in.readLine();, the program seems to work.
Since the Process p = pb.start(); is inside the loop, the python subprogram is called repeatedly.
The next step should be to try running the python program manually in a console, see what it does.
(n.b. I had to remove "sudo" and change paths to be able to test on my system, you should have no problems replacing things for your setup).

I've managed to get around it by editing my Python script - it returns null if there's no card on the reader, and the UID if there is.
I'll probably use observer pattern or similar on the Java end to detect when there's a card. Very resource intensive but it'll have to do for now!

How do I Pipe process output to a file on Windows and JDK 6u45

I have the following windows batch file (run.bat):
#echo off
echo hello batch file to sysout
And the following java code, which runs the batch files and redirects output to a file:
public static void main(String[] args) throws IOException {
System.out.println("Current java version is: " + System.getProperty("java.version"));
ProcessBuilder pb =
new ProcessBuilder("cmd.exe", "/c",
"run.bat"
,">>", "stdout.txt","2>>", "stderr.txt"
);
System.out.println("Command is: " + pb.command());
Process proc = pb.start();
InputStream in = proc.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
int exitValue = proc.exitValue();
System.out.println("Exit value: " + exitValue);
}
On JDKs up to and including JDK6u43 I get the following output:
Current java version is: 1.6.0_29
Command is: [cmd.exe, /c, run.bat, >>, stdout.txt, 2>>, stderr.txt]
Exit value: 0
and the script output is written to the file.
As of JDK 6u45 and 7, I get the following output:
Current java version is: 1.6.0_45
Command is: [cmd.exe, /c, run.bat, >>, stdout.txt, 2>>, stderr.txt]
hello batch file to sysout
Exit value: 0
And nothing is written to the output file.
This may or may not be related to the changes made in Runtime.exec() , described at: http://www.oracle.com/technetwork/java/javase/6u45-relnotes-1932876.html
What is the correct way of starting a process on Windows with output redirected to files?
Note: In a real world scenario, the command to execute may include parameters with spaces, as in:
ProcessBuilder pb = new ProcessBuilder("cmd.exe", "/c",
"run.bat", "Some Input With Spaces",
">>", "stdout.txt","2>>", "stderr.txt");

This is the simplest method i found on http://tamanmohamed.blogspot.in/2012/06/jdk7-processbuilder-and-how-redirecting.html
File output = new File("C:/PBExample/ProcessLog.txt");
ProcessBuilder pb = new ProcessBuilder("cmd");
pb.redirectOutput(output);

Several suggestions here:
Does the input with the spaces need to be treated as single String (with spaces),or id it in actual several inputs? If the first Option is the case I would suggest to quote it for the windows runtime:
ProcessBuilder pb = new ProcessBuilder("cmd.exe", "/c",
"run.bat", "\"Some Input With Spaces\"",
">>", "stdout.txt","2>>", "stderr.txt");
Instead of redirecting the input to stdout.txt and stderr.txt using the shell, why not do it using Java using getOutputStream() and getErrorStream()? Here is an example using Guava's IO package. Of course you may want to have those in separate threads, you need proper exception handling, etc.
InputStream stdout = new BufferedInputStream(proc.getInputStream());
FileOutputStream stdoutFile = new FileOutputStream("stdout.txt");
ByteStreams.copy(stdout, stdoutFile);
InputStream stderr = new BufferedInputStream(proc.getErrorStream());
FileOutputStream stderrFile = new FileOutputStream("stderr.txt");
ByteStreams.copy(stderr, stderrFile);
stdout.close();
stderr.close();
stdoutFile.close();
stderrFile.close();
Another option, why not create a run.bat wrapper that will make the redirections?
#echo off
cmd.exe /c run.bat "%1" >> "%2" 2>> "%3"

Use getOutputStream() on the process, instead of using System.out.println(). Sometimes the semantics change between Java implementations.
This seems to be a bugfix actually - the newer implementation makes sense.

Problem ProcessBuilder running script sh

trying to execute an script, using this piece of code:
String command = "./myScript.sh";
pb = new ProcessBuilder(command, param1, param2);
pb.directory(directory);
pb.start();
I am not getting any kind of error, but neither the supposed results. Anyway, I tryed to run the same command, direclty in the terminal, and everything working correctly.
Am I missing something??
Thanks in advance

When you start a process (pb.start()) you get back a Process instance. If your script reads input or writes output to stdout or stderr you need to handle this on separate threads using Process.getInputStream(), ...getOutputStream() and getErrorStream(). If you don't do this the process can hang. You also should call Process.waitFor() and then Process.exitValue() to get the return status of the process. If it's a negative number then the system was unable to launch your script.
EDIT: Here is a short simplified example. This is a toy only and will work reliably ONLY under the following conditions:
The script does not require any input
The script does not produce a large amount of output on both stdout and stderr. If it does, then since the program reads all of stdout before stderr, the stderr buffer may fill up and block the process from completing. In a 'real' implementation you would read stdout and stderr in separate threads (hint, wrap the loadStream() method in a class that implements Runnable).
public class PBTest
{
public static void main(String[] args) throws Exception
{
ProcessBuilder pb = new ProcessBuilder("sc","query","wuauserv");
Process p = pb.start();
String output = loadStream(p.getInputStream());
String error = loadStream(p.getErrorStream());
int rc = p.waitFor();
System.out.println("Process ended with rc=" + rc);
System.out.println("\nStandard Output:\n");
System.out.println(output);
System.out.println("\nStandard Error:\n");
System.out.println(error);
}
private static String loadStream(InputStream s) throws Exception
{
BufferedReader br = new BufferedReader(new InputStreamReader(s));
StringBuilder sb = new StringBuilder();
String line;
while((line=br.readLine()) != null)
sb.append(line).append("\n");
return sb.toString();
}
}

The problem was not on the way I called the script, which was right.
But it was inside the script. At first it was:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
But I got ffmpeg command not found, so I changed it to:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
/usr/local/bin/ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
with the hole path. But I have still doubts, why this is necessary, if I have ffmpeg in my path and I cand execute in console direclty form any directory??
If someone can give me an answer, it will be welcome :)

Using Java to call Linux terminal: How to flush the output?

1) I'm using Java to call Linux terminal to run foo.exe and save the output in a file:
String[] cmd = {"/bin/sh", "-c", "foo >haha.file"};
Runtime.getRuntime().exec(cmd);
2) The problem is when I plan to read haha.file later in the code, it hasn't been written yet:
File f=new File("haha.file"); // return true
in = new BufferedReader(new FileReader("haha.file"));
reader=in.readLine();
System.out.println(reader);//return null
3) Only after the program is done will the haha.file be written. I only know how to flush "Writers" but don't know how to flush sth. like this.
How can I force java to write the file in the terminal?
Thanks in advance
E.E.

This problem is caused by the asynchronous nature of Runtime.exec. foo is being executed in a seperate process. You need to call Process.waitFor() to insure the file has been written.
String[] cmd = {"/bin/sh", "-c", "foo >haha.file"};
Process process = Runtime.getRuntime().exec(cmd);
// ....
if (process.waitFor() == 0) {
File f=new File("haha.file");
in = new BufferedReader(new FileReader("haha.file"));
reader=in.readLine();
System.out.println(reader);
} else {
//process did not terminate normally
}

You can either wait for the completion of the process:
Process p = Runtime.getRuntime().exec(cmd);
int result = p.waitFor();
Or use the p.getInputStream() to read directly from the standard output of the process.