We Keep Coding

Java Process Builder not executing multiple commands

Hi a Java newbie here.
I am currently building a Java application that executes multiple linux commands with Java Process builder.
I am planning to execute a shell scipt, and since it this shell script is an external program that takes about a second to fully execute, let the process sleep for a second and write the result into a txt file.
This is an external program and it must take in "q" to exit the program, so I finally need to type in q in the terminal.
I have gotten help on this community before and the code I constructed with that help is as follows.
public static void Linux(String fileName){
try {
File dir = new File("/usr/local/bin");
ProcessBuilder pb = new ProcessBuilder(new String[]{"/bin/bash", "-c", "./test_elanprv2.2.sh > \"$1\"; sleep 1; q", "_", fileName + ".txt"});
System.out.println("wrote text");
pb.directory(dir);
Process start = pb.start();
start.destroy();
}catch (Exception e){
e.printStackTrace();
System.out.println("failed to write text");
}
The process builder does create a txt file but it seems to be empty, and no matter how long I set the sleep to, the programs seems to instanly return the print statement.
It would really be appreciated if anyone could tell me what I am doing wrong.
Thank you in advance!!

As mentioned by #VGR, try using redirectOutput
public static void Linux(String fileName){
try {
File dir = new File("/usr/local/bin");
ProcessBuilder pb = new ProcessBuilder(new String[]{"/bin/bash", "-c", "./test_elanprv2.2.sh");
File output = new File("_", fileName + ".txt");
pb.redirectOutput(output);
System.out.println("wrote text");
pb.directory(dir);
Process start = pb.start();
start.destroy();
} catch (Exception e) {
e.printStackTrace();
System.out.println("failed to write text");
}

Most of your issues are due to a misunderstanding of how processes work. These concepts are not Java concepts; you would have the same issues in any other language.
First, you are destroying your process before it runs, and possibly before it even gets started. This is because pb.start() starts the process, and then you immediately destroy it without giving it a chance to complete.
You shouldn’t need to destroy the process at all. Just let it finish:
Process start = pb.start();
start.waitFor();
All processes have their own standard input and standard output. Again, this is not a Java concept; this has been a fundamental feature in Unix and Windows operating systems for a long time.
Normally, when a process prints information by writing it to its standard output. That is in fact what Java’s System.out.println does. In Unix shells (and in Windows), the > character redirects the standard output of a process to a file; the program still writes to its standard output, without ever knowing that the operating system is sending that output to a destination other than the terminal. Since it’s a fundamental operating system function, Java can do it for you:
ProcessBuilder pb =
new ProcessBuilder("/bin/bash", "-c", "./test_elanprv2.2.sh");
pb.redirectOutput(new File(fileName + ".txt"));
Similarly, when a process wants to take input, it normally does so by reading from its standard input. This is not the same as executing another command. When you do this:
./test_elanprv2.2.sh > "$1"; sleep 1; q
You are not sending q to the shell script. The above commands wait for the shell script to finish, then execute a sleep, then try to execute a program named q (which probably doesn’t exist).
Since the test_elanprv2.2.sh shell script probably accepts commands by reading its standard input, you want to send the q command to the standard input of that process:
ProcessBuilder pb =
new ProcessBuilder("/bin/bash", "-c", "./test_elanprv2.2.sh");
pb.redirectOutput(new File(fileName + ".txt"));
Process start = pb.start();
Thread.sleep(1000);
try (Writer commands = new OutputStreamWriter(start.getOutputStream())) {
commands.write("q\n");
}
// Caution: Call this AFTER writing commands. You don't want to write
// to the standard input of a process that has already finished!
start.waitFor();

Source command not working through Java

From last day, I have been trying to execute a command on Terminal (MAC) using JAVA but whatever I do nothing is working.
I have the following 2 commands that I want to execute and get the output back in JAVA
source activate abc_env
python example.py
Till now, I have tried the following methods without any output
String[] command = new String[] { "source activate abc_env", "python example.py"};
String result = executeCommands(command);
Here is my executeCommands method
private static String executeCommands(String[] command) {
StringBuffer output = new StringBuffer();
Process p;
try {
for(int i=0; i< command.length;i++)
{
p = Runtime.getRuntime().exec(command[i]);
p.waitFor();
BufferedReader reader =
new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
}
System.out.println("Error output: " + p.exitValue());
System.out.println("Output:" + output);
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("Here");
}
return output.toString();
}
This gives me the following exception
Cannot run program "source": error=2, No such file or directory
I searched online and people say that source won't work like this and I should change the command to
String[] command = new String[] { "bash -c 'source activate abc_env'", "python example.py"};
Now, I donot get the exception but the command still does not work and it returns '2' as exitValue()
Then I tried to execute the commands as a script
#!/bin/bash
source activate abc_env
python example.py
I get the following exception when I read the .sh file as string and pass it to command
Cannot run program "#!/bin/bash": error=2, No such file or directory
So, my question is how to run the source command followed by python command properly through Java ? My final goal is execute some python from Java.
EDIT1:
If I try the following command and print the output stream
String[] command = {
"/bin/bash",
"-c",
"source activate cvxpy_env"
};
executeCommand(command));
Output Stream:
ExitValue:1
ErrorOutput:/bin/bash: activate: No such file or directory
If I try the same command but with single quotes around 'source activate abc_env'. I get the following output
ExitValue:127
ErrorOutput:/bin/bash: source activate cvxpy_env: command not found
Solution:
String[] command = {
"/bin/bash",
"-c",
"source /Users/pc_name/my_python_library/bin/activate abc_env;python example.py"
};

According to the Javadoc, Runtime.exec(String) breaks the command into the command-args list using a StringTokenizer, which will probably break your command into:
bash
-c
'source
activate
abc_env'
Which is obviously not what you want. What you should do is probably use the version of Runtime.exec(String[]) that accepts a ready list of arguments, passing to it new String[] {"bash", "-c", "source activate abc_env"}.
Now, to get an idea why it's not working, you should not only read from its stdout but also from stderr, using p.getErrorStream(). Just print out what you read, and it will be a great debugging aid.
Re: your edit. Now it looks like it's working fine, as far as Java and bash are concerned. The output "activate: No such file or directory" is probably the output from the successful run of the source command. It just that source can't find the activate file. Is it in the working directory? If not, you probably should have "cd /wherever/your/files/are; source activate cvxpy_env". Oh, and if your python script depends on whatever side-effects the source command has, you probably have to execute it in the same bash instance, that is:
String[] command = {
"/bin/bash",
"-c",
"cd /wherever/your/files/are && source activate cvxpy_env && python example.py"
};
Or better yet, pack it all into a single shell script, and then just Runtime.exec("./my_script.sh") (don't forget to chmod +x it, though).

Try
String[] command = {
"/bin/bash",
"-c",
"source activate abc_env; " + "python example.py"
};

cannot run command with processbuilder or runexec inside eclipse

I'm trying to execute a command from my Java web application (maven project with Tomcat 8 in Eclipse).
If I execute this command by the cmd line or by a .jar file there is no problem, but in Eclipse it doesn't work.
I tried ProcessBuilder class and Runtime.getRuntime().exec() but both throw the same "file not found exception" exception.
java.io.IOException: Cannot run program "mongodump": error=2, No such file or directory
The command i want to run is:
mongodump --db TweetsDB --collection Tweets --out ./TweetsDB_backup/
It seems that the runtime environments can't find the mongodump command (and/or the folder named in the command?)
Any suggestion?
Here it is my code:
public synchronized static void backup(){
Process p;
try {
String dateTimeInString = dateTimeToString(Calendar.getInstance());
System.out.println("backuppando "+dateTimeInString);
p = Runtime.getRuntime().exec("mongodump --db TweetsDB --collection Tweets --out ./backup/"); //override always the same db to save space
p.waitFor();
System.out.println("Backup effettuato correttamente :)");
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
This is the "process builder" variant :
ProcessBuilder builder = new ProcessBuilder("mongodump", "--db", "TweetsDB","--collection", "Tweets", "--out" ,"./TweetsDB_backup/");
builder.redirectErrorStream(true); // equivalent of 2>&1
final Process process = builder.start();
InputStream es = process.getErrorStream();
InputStreamReader esr = new InputStreamReader(es);
BufferedReader br = new BufferedReader(esr);
String line;
while((line = br.readLine())!= null){
System.out.println(line);
}
Thanks,
Giulio

When you started the process with ProcessBuilder or Runtime.getRuntime().exec(), it would swallow any output from the process that is being started.
To see the real problem you need to get that output back.
With Runtime.exec() you need to use process.getErrorStream() and process.getInputStream() and set up threads to copy the output to somewhere you can see it.
with ProcessBuilder you can diurect the IO from the resulting process into the current (parent) processes using ProcessBuilder.inheritIO() before you call ProcessBuilder.start().
Use ProcessBuilder.inheritIO() to redirect the processes output to the servers log and you may find out what the error is.
try the following and see if it helps.
ProcessBuilder builder = new ProcessBuilder("mongodump", "--db", "TweetsDB","--collection", "Tweets", "--out" ,"./TweetsDB_backup/");
builder.inheritIO();
final Process process = builder.start();
Once you know what the error is you may be able to work out the solution.
I suspect you need to set some additional environment variables or something in the resulting process to get it to work.

I didn't try it yet, but maybe you can do it this way :
Mongo client = new MongoClient();
client.getDB("myDb").command("mongodump ...");

Throw No such file or directory is because Java can't get local path env. You should give absolute path of mongodump. Like this:
Runtime.getRuntime().exec("/usr/local/bin/mongodump --db TweetsDB --collection Tweets --out ./TweetsDB_backup/");

How to open the command prompt and insert commands using Java?

Is it possible to open the command prompt (and I guess any other terminal for other systems), and execute commands in the newly opened window?
Currently what I have is this:
Runtime rt = Runtime.getRuntime();
rt.exec(new String[]{"cmd.exe","/c","start"});
I've tried adding the next command after the "start", I've tried running another rt.exec containing my command, but I can't find a way to make it work.
If it matters, I'm trying to run a command similar to this:
java -flag -flag -cp terminal-based-program.jar
EDIT Unfortunately I have had some strange findings. I've been able to successfully launch the command prompt and pass a command using this:
rt.exec("cmd.exe /c start command");
However, it only seems to work with one command. Because, if I try to use the command separator like this, "cmd.exe /c start command&command2", the second command is passed through the background (the way it would if I just used rt.exec("command2");). Now the problem here is, I realized that I need to change the directory the command prompt is running in, because if I just use the full path to the jar file, the jar file incorrectly reads the data from the command prompt's active directory, not the jar's directory which contains its resources.

I know that people recommend staying away from rt.exec(String), but this works, and I don't know how to change it into the array version.
rt.exec("cmd.exe /c cd \""+new_dir+"\" & start cmd.exe /k \"java -flag -flag -cp terminal-based-program.jar\"");

If you are running two commands at once just to change the directory the command prompt runs in, there is an overload for the Runtime.exec method that lets you specify the current working directory. Like,
Runtime rt = Runtime.getRuntime();
rt.exec("cmd.exe /c start command", null, new File(newDir));
This will open command prompt in the directory at newDir. I think your solution works as well, but this keeps your command string or array a little cleaner.
There is an overload for having the command as a string and having the command as a String array.
You may find it even easier, though, to use the ProcessBuilder, which has a directory method to set your current working directory.
Hope this helps.

public static void main(String[] args) {
try {
String ss = null;
Process p = Runtime.getRuntime().exec("cmd.exe /c start dir ");
BufferedWriter writeer = new BufferedWriter(new OutputStreamWriter(p.getOutputStream()));
writeer.write("dir");
writeer.flush();
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
System.out.println("Here is the standard output of the command:\n");
while ((ss = stdInput.readLine()) != null) {
System.out.println(ss);
}
System.out.println("Here is the standard error of the command (if any):\n");
while ((ss = stdError.readLine()) != null) {
System.out.println(ss);
}
} catch (IOException e) {
System.out.println("FROM CATCH" + e.toString());
}
}

The following works for me on Snow Leopard:
Runtime rt = Runtime.getRuntime();
String[] testArgs = {"touch", "TEST"};
rt.exec(testArgs);
Thing is, if you want to read the output of that command, you need to read the input stream of the process. For instance,
Process pr = rt.exec(arguments);
BufferedReader r = new BufferedReader(new InputStreamReader(pr.getInputStream()));
Allows you to read the line-by-line output of the command pretty easily.
The problem might also be that MS-DOS does not interpret your order of arguments to mean "start a new command prompt". Your array should probably be:
{"start", "cmd.exe", "\c"}
To open commands in the new command prompt, you'd have to use the Process reference. But I'm not sure why you'd want to do that when you can just use exec, as the person before me commented.

You just need to append your command after start in the string that you are passing.
String command = "cmd.exe /c start "+"*your command*";
Process child = Runtime.getRuntime().exec(command);

String[] command = {"cmd.exe" , "/c", "start" , "cmd.exe" , "/k" , "\" dir && ipconfig
\"" };
ProcessBuilder probuilder = new ProcessBuilder( command );
probuilder.directory(new File("D:\\Folder1"));
Process process = probuilder.start();

You can use any on process for dynamic path on command prompt
Process p = Runtime.getRuntime().exec("cmd.exe /c start dir ");
Process p = Runtime.getRuntime().exec("cmd.exe /c start cd \"E:\\rakhee\\Obligation Extractions\" && dir");
Process p = Runtime.getRuntime().exec("cmd.exe /c start cd \"E:\\oxyzen-workspace\\BrightleafDesktop\\Obligation Extractions\" && dir");

Please, place your command in a parameter like the mentioned below.
Runtime.getRuntime().exec("cmd.exe /c start cmd /k \" parameter \"");

You have to set all \" (quotes) carefully. The parameter \k is used to leave the command prompt open after the execution.
1) to combine 2 commands use (for example pause and ipconfig)
Runtime.getRuntime()
.exec("cmd /c start cmd.exe /k \"pause && ipconfig\"", null, selectedFile.getParentFile());
2) to show the content of a file use (MORE is a command line viewer on Windows)
File selectedFile = new File(pathToFile):
Runtime.getRuntime()
.exec("cmd /c start cmd.exe /k \"MORE \"" + selectedFile.getName() + "\"\"", null, selectedFile.getParentFile());
One nesting quote \" is for the command and the file name, the second quote \" is for the filename itself, for spaces etc. in the name particularly.

Execute external program from Java

I am trying to execute a program from the Java code. Here is my code:
public static void main(String argv[]) {
try {
String line;
Process p = Runtime.getRuntime().exec(new String[]{
"/bin/bash", "-c", "executable -o filename.txt"});
BufferedReader input = new BufferedReader(
new InputStreamReader(p.getInputStream()));
while ((line = input.readLine()) != null) {
System.out.println(line);
}
input.close();
} catch (Exception err) {
err.printStackTrace();
}
}
My OS is Mac OS X 10.6.
Now, the executable I am trying to run is supposed to spit the output to filename.txt. If I take this command and run it on the terminal, it works fine and the filename.txt gets populated also. But, from my java program the file is not created.
if instead I use executable > filename.txt then the filename.txt is created but is empty. Not sure what's wrong here. The executable I am trying to run is Xtide (if that helps).
I would really appreciate any help I can get.
Thanks,

You cannot redirect output to file and read the output in java. It's one or the other. What you want is this:
Process p = Runtime.getRuntime().exec(new String[]{
"/bin/bash", "-c", "executable -o filename.txt"});
p.waitFor();
BufferedReader input = new BufferedReader(
new InputStreamReader(new FileInputStream("filename.txt")));
while ((line = input.readLine()) != null) {
System.out.println(line);
}
The main changes are:
p.waitFor(), since process execution is asynchronous, so you have to wait for it to complete.
The data is read from the file rather than from the output of the process (since this will be empty.)

The answer from mdma works (and I voted it up), but you might also want to consider the version where you do read the output stream directly from executable:
Process p = Runtime.getRuntime().exec(new String[]{
"/bin/bash", "-c", "executable"});
p.waitFor();
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream())_;
while ((line = input.readLine()) != null) {
System.out.println(line);
}

Correct me if I am wrong, but the symptoms are as follows:
exec("/usr/bash", "-c", "executable > filename.txt") creates an empty file.
exec("/usr/bash", "-c", "executable -o filename.txt") does not create a file.
One or both of the above gives an exit code of 255 when you look at it.
When you run the command from the command line as executable -o filename.txt or executable > filename.txt it works as expected.
In the light of the above, I think that the most likely cause is that /bin/bash is not finding the executable when you launch it from Java. The fact that the first example does create an empty file means that /bin/bash is doing something. But if you try to run
$ unknown-command > somefile.txt
from a bash shell prompt you will get an error message saying that the command cannot be found and an empty "something.txt" file. (You would not see the error message in your Java app because it is being written to stderr, and you are not capturing it.) The reason that the empty "something.txt" file is created is that it is opened by the shell before it attempts to fork and exec the "executable".
If this is the problem, then the simple solution is to use the absolute pathname for the executable.
Also, if you are not doing any command line redirection or other shell magic, there is no need to run the executable in a new bash instance. Rather, just do this:
Process p = Runtime.getRuntime().exec("executable", "-o", filename.txt");
then wait for the process to complete and check the exit code before trying to read the file contents.