We Keep Coding

How do I Pipe process output to a file on Windows and JDK 6u45

I have the following windows batch file (run.bat):
#echo off
echo hello batch file to sysout
And the following java code, which runs the batch files and redirects output to a file:
public static void main(String[] args) throws IOException {
System.out.println("Current java version is: " + System.getProperty("java.version"));
ProcessBuilder pb =
new ProcessBuilder("cmd.exe", "/c",
"run.bat"
,">>", "stdout.txt","2>>", "stderr.txt"
);
System.out.println("Command is: " + pb.command());
Process proc = pb.start();
InputStream in = proc.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
int exitValue = proc.exitValue();
System.out.println("Exit value: " + exitValue);
}
On JDKs up to and including JDK6u43 I get the following output:
Current java version is: 1.6.0_29
Command is: [cmd.exe, /c, run.bat, >>, stdout.txt, 2>>, stderr.txt]
Exit value: 0
and the script output is written to the file.
As of JDK 6u45 and 7, I get the following output:
Current java version is: 1.6.0_45
Command is: [cmd.exe, /c, run.bat, >>, stdout.txt, 2>>, stderr.txt]
hello batch file to sysout
Exit value: 0
And nothing is written to the output file.
This may or may not be related to the changes made in Runtime.exec() , described at: http://www.oracle.com/technetwork/java/javase/6u45-relnotes-1932876.html
What is the correct way of starting a process on Windows with output redirected to files?
Note: In a real world scenario, the command to execute may include parameters with spaces, as in:
ProcessBuilder pb = new ProcessBuilder("cmd.exe", "/c",
"run.bat", "Some Input With Spaces",
">>", "stdout.txt","2>>", "stderr.txt");

This is the simplest method i found on http://tamanmohamed.blogspot.in/2012/06/jdk7-processbuilder-and-how-redirecting.html
File output = new File("C:/PBExample/ProcessLog.txt");
ProcessBuilder pb = new ProcessBuilder("cmd");
pb.redirectOutput(output);

Several suggestions here:
Does the input with the spaces need to be treated as single String (with spaces),or id it in actual several inputs? If the first Option is the case I would suggest to quote it for the windows runtime:
ProcessBuilder pb = new ProcessBuilder("cmd.exe", "/c",
"run.bat", "\"Some Input With Spaces\"",
">>", "stdout.txt","2>>", "stderr.txt");
Instead of redirecting the input to stdout.txt and stderr.txt using the shell, why not do it using Java using getOutputStream() and getErrorStream()? Here is an example using Guava's IO package. Of course you may want to have those in separate threads, you need proper exception handling, etc.
InputStream stdout = new BufferedInputStream(proc.getInputStream());
FileOutputStream stdoutFile = new FileOutputStream("stdout.txt");
ByteStreams.copy(stdout, stdoutFile);
InputStream stderr = new BufferedInputStream(proc.getErrorStream());
FileOutputStream stderrFile = new FileOutputStream("stderr.txt");
ByteStreams.copy(stderr, stderrFile);
stdout.close();
stderr.close();
stdoutFile.close();
stderrFile.close();
Another option, why not create a run.bat wrapper that will make the redirections?
#echo off
cmd.exe /c run.bat "%1" >> "%2" 2>> "%3"

Use getOutputStream() on the process, instead of using System.out.println(). Sometimes the semantics change between Java implementations.
This seems to be a bugfix actually - the newer implementation makes sense.

java getRuntime().exec() not working?

Basically, when I type these commands in
the terminal by hand, the sift program works and writes a .key file, but when I try to call it from my program, nothing is written.
Am I using the exec() method correctly? I have looked through the API and I can't seem to spot where I went wrong.
public static void main(String[] args) throws IOException, InterruptedException
{
//Task 1: create .key file for the input file
String[] arr = new String[3];
arr[0] = "\"C:/Users/Wesley/Documents/cv/final project/ObjectRecognition/sift/siftWin32.exe\"";
arr[1] = "<\"C:/Users/Wesley/Documents/cv/final project/ObjectRecognition/sift/cover_actual.pgm\"";
arr[2] = ">\"C:/Users/Wesley/Documents/cv/final project/ObjectRecognition/sift/keys/cover_actual.key\"";
String command = (arr[0]+" "+arr[1]+" "+arr[2]);
Process p=Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader=new BufferedReader(new InputStreamReader(p.getInputStream()));
String line=reader.readLine();
while(line!=null)
{
System.out.println(line);
line=reader.readLine();
}
}

The command line you are using is a DOS command line in the format:
prog < input > output
The program itself is executed with no arguments:
prog
However the command from your code is executed as
prog "<" "input" ">" "output"
Possible fixes:
a) Use Java to handle the input and output files
Process process = Runtime.getRuntime().exec(command);
OutputStream stdin = process.getOutputStream();
InputStream stdout = process.getInputStream();
// Start a background thread that writes input file into "stdin" stream
...
// Read the results from "stdout" stream
...
See: Unable to read InputStream from Java Process (Runtime.getRuntime().exec() or ProcessBuilder)
b) Use cmd.exe to execute the command as is
cmd.exe /c "prog < input > output"

You can't use redirections (< and >) with Runtime.exec as they are interpreted and executed by the shell. It only works with one executable and its arguments.
Further reading:
https://stackoverflow.com/a/11250789/105224

You cannot use input/output redirection with Runtime.exec. On the other hand, the same method returns a Process object, and you can access its input and output streams.
Process process = Runtime.exec("command here");
// these methods are terribly ill-named:
// getOutputStream returns the process's stdin
// and getInputStream returns the process's stdout
OutputStream stdin = process.getOutputStream();
// write your file in stdin
stdin.write(...);
// now read from stdout
InputStream stdout = process.getInputStream();
stdout.read(...);

I test, it's ok. You can try. Good luck
String cmd = "cmd /c siftWin32 <box.pgm>a.key";
Process process = Runtime.getRuntime().exec(cmd);

*For special characters that usually cause problems:
This code works correctly even with file names like: "1 - Volume 1 (Fronte).jpg"
String strArr[] = {"cmd", "/C", file.getCanonicalPath()};
Process p = rtObj.exec(strArr);///strCmd);
Agree too, redirection not supported here.
Tested on Windows 7
{guscoder:912081574}

Interact with Powershell process called from Java application

I'm trying to run a Java application which creates a new powershell process on startup and then later on interacts with it multiple times. Calling powershell.exe and have it execute a single command and return the output works fine for me. The problem arises if I don't want the powershell process to immediately finish/exit but to stay open so I can write to its outputStream and receive results back from the inputStream.
String input = "dir";
String[] commandList = {"powershell.exe", "-Command", "dir"};
ProcessBuilder pb = new ProcessBuilder(commandList);
Process p = pb.start();
if(input != null) {
PrintWriter writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(p.getOutputStream())), true);
writer.println(input);
writer.flush();
writer.close();
}
//p.getOutputStream().close();
Gobbler outGobbler = new Gobbler(p.getInputStream());
Gobbler errGobbler = new Gobbler(p.getErrorStream());
Thread outThread = new Thread(outGobbler);
Thread errThread = new Thread(errGobbler);
outThread.start();
errThread.start();
System.out.println("Waiting for the Gobbler threads to join...");
outThread.join();
errThread.join();
System.out.println("Waiting for the process to exit...");
int exitVal = p.waitFor();
System.out.println("\n****************************");
System.out.println("Command: " + "cmd.exe /c dir");
System.out.println("Exit Value = " + exitVal);
List<String> output = outGobbler.getOuput();
input = "";
for(String o: output) {
input += o;
}
System.out.println("Final Output:");
System.out.println(input);
This code returns the result of the "dir" command from a powershell - fine. But as you can see, I'm trying to run a second "dir" command using
PrintWriter writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(p.getOutputStream())), true);
writer.println(input);
writer.flush();
This has no effect whatsoever - no second dir output is shown when I run my code. I've also experimented with a powershell.exe option to open the powershell but not close it immediately:
String[] commandList = {"powershell.exe", "-NoExit", "-Command", "dir"};
But then my code hangs, meaning the Gobbler's who consume the process's inputStream don't read anything - strangely enough: they don't even read the first line - there must be at least some output....
I've also tried to close the process's outputStream after writing the second "dir" command to it - didn't change anything.
Any help is highly appreciated.
Thanks
Kurt

This sounds about right for the nature of a process spun up by another process. I think you're experiencing pretty standard behavior.
This is the key: p.waitFor()
From Java docs:
causes the current thread to wait, if necessary, until the process represented by this Process object has terminated.
You won't be able to receive the PowerShell output stream until it has terminated. When you run with -NoExit it never exits which is why you are experiencing the hang.
If you run ProcExp from Sysinternals you'll be able to see your Java process spin up an child PowerShell process.
So I don't think you'll be able to interact with it like it's a live object in memory.

Problem ProcessBuilder running script sh

trying to execute an script, using this piece of code:
String command = "./myScript.sh";
pb = new ProcessBuilder(command, param1, param2);
pb.directory(directory);
pb.start();
I am not getting any kind of error, but neither the supposed results. Anyway, I tryed to run the same command, direclty in the terminal, and everything working correctly.
Am I missing something??
Thanks in advance

When you start a process (pb.start()) you get back a Process instance. If your script reads input or writes output to stdout or stderr you need to handle this on separate threads using Process.getInputStream(), ...getOutputStream() and getErrorStream(). If you don't do this the process can hang. You also should call Process.waitFor() and then Process.exitValue() to get the return status of the process. If it's a negative number then the system was unable to launch your script.
EDIT: Here is a short simplified example. This is a toy only and will work reliably ONLY under the following conditions:
The script does not require any input
The script does not produce a large amount of output on both stdout and stderr. If it does, then since the program reads all of stdout before stderr, the stderr buffer may fill up and block the process from completing. In a 'real' implementation you would read stdout and stderr in separate threads (hint, wrap the loadStream() method in a class that implements Runnable).
public class PBTest
{
public static void main(String[] args) throws Exception
{
ProcessBuilder pb = new ProcessBuilder("sc","query","wuauserv");
Process p = pb.start();
String output = loadStream(p.getInputStream());
String error = loadStream(p.getErrorStream());
int rc = p.waitFor();
System.out.println("Process ended with rc=" + rc);
System.out.println("\nStandard Output:\n");
System.out.println(output);
System.out.println("\nStandard Error:\n");
System.out.println(error);
}
private static String loadStream(InputStream s) throws Exception
{
BufferedReader br = new BufferedReader(new InputStreamReader(s));
StringBuilder sb = new StringBuilder();
String line;
while((line=br.readLine()) != null)
sb.append(line).append("\n");
return sb.toString();
}
}

The problem was not on the way I called the script, which was right.
But it was inside the script. At first it was:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
But I got ffmpeg command not found, so I changed it to:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
/usr/local/bin/ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
with the hole path. But I have still doubts, why this is necessary, if I have ffmpeg in my path and I cand execute in console direclty form any directory??
If someone can give me an answer, it will be welcome :)

CMD.exe command in java not terminating

I'm trying to use cmd.exe to search for a file in a specific directory and then display the path in a java program and write it to a file. The problem is that the process never terminates.
Here is my code:
String[] str = new String[] { "cmd.exe ", "cd c:\\",
" dir /b /s documents", "2>&1" };
Runtime rt = Runtime.getRuntime();
try{
Process p = rt.exec(str);
InputStream is =p.getInputStream();
InputStreamReader in = new InputStreamReader(is);
StringBuffer sb = new StringBuffer();
BufferedReader buff = new BufferedReader(in);
String line = buff.readLine();
while( line != null )
{
sb.append(line + "\n");
line = buff.readLine();
}
System.out.println( sb );
File f = new File("test.txt");
FileOutputStream fos = new FileOutputStream(f);
fos.write(sb.toString().getBytes());
fos.close();
}catch( Exception ex )
{
ex.printStackTrace();
}

Please try
cmd /c
instead of simply
cmd
Reference

Runtime.exec doesn't work that way. You can't pass multiple commands like that to cmd.exe.
Runtime.exec allows you to execute a single process with a list of arguments. It does not provide any "shell" operations (like 2>&1 for instance). You must do that sort of IO redirection yourself using the Input/Output streams.
It's similar to calling another program's main function.
You could try `Runtime.exec( new String[] { "cmd.exe", "/c", "dir", "C:\\" } );
But realistically, if you want file listings, you're much better off using the facilities in the java.io.File class, which won't depend on operating system specific features.

why are not using Java to do directory traversal instead of calling external shell command? It makes your code not portable!

You must use the start command in addition to the cmd.exe process with the /C or /K switch BEFORE the start command. Example: to convert the Windows's command interpreter in a bash console (from the mingw prroject) you must invoke the exec method of the Runtime class with the command "C:\Windows\System32\cmd.exe /C start C:\mingw\msys\1.0\bin\bash.exe" (I use an external command rather than an internal because it's more signifiant but you can use internal command like DIR and so on).