We Keep Coding

Executing grep command on Linux from Java always returning null

I am executing grep command from java on a linux file. Its always returning null for the following code.
Process p;
String matchStr="testmatch";
String output = null;
try {
String command = "grep \""+matchStr+"\" "+ filename;
System.out.println("Running command: " + command);
p = Runtime.getRuntime().exec(command);
System.out.println("***********************************");
System.out.println("***********************************");
System.out.println("***********************************");
p.waitFor();
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
while (br.readLine() != null) {
System.out.println("in while loop");
System.out.println("in while loop");
System.out.println("in while loop");
System.out.println(output);
System.out.println("***********************************");
System.out.println("***********************************");
System.out.println("***********************************");
System.out.println("***********************************");
// Process your output here
}
System.out.println("exit: " + p.exitValue());
p.destroy();
} catch (Exception e) {
e.printStackTrace();
}
If i grep it directly it shows output but from java it never gets into while loop.
Please suggest whats wrong here.

The problem is that you do not write anything to output so it stays null. I guess you have to rewrite your while loop like this
while ((output = br.readLine()) != null) {
// Process your output here
}
Take a note that this syntax is discouraged by most style check due to it's abmiguity
Also it's a good idea to place p.waitFor() after while loop so grep would not hang on flushig std(err|out).
UPDATE
Also it is a good idea to use ProcessBuilder (available since java-7) instead of Runtime.getRuntime().exec(...) because you will have more control over the process i.e
final ProcessBuilder builder = new ProcessBuilder();
builder.command("grep", matchStr, filename);
// redirect stderr to stdout
builder.redirectErrorStream(true);
final Process process = builder.start();
BufferedReader br = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String output = null;
while ((output = br.readLine()) != null) {
System.out.println(output);
// Process your output here
}
process.waitFor();

After turning your code into a https://stackoverflow.com/help/mcve it works for me.
Here the file does not exist:
robert#habanero:~$ rm /home/robert/greptest.txt
robert#habanero:~$ javac GrepTest.java && java GrepTest
Running command: grep test /home/robert/greptest.txt
exit: 2
Now the file does exist but does not contain the text to be found:
robert#habanero:~$ echo not found > /home/robert/greptest.txt
robert#habanero:~$ javac GrepTest.java && java GrepTest
Running command: grep test /home/robert/greptest.txt
exit: 1
Now the file exists and contains the text:
robert#habanero:~$ echo test this > /home/robert/greptest.txt
robert#habanero:~$ javac GrepTest.java && java GrepTest
Running command: grep test /home/robert/greptest.txt
test this
exit: 0
Here is the code:
import java.io.*;
public class GrepTest {
public static void main(String[] args) throws Exception {
String command = "grep test /home/robert/greptest.txt";
System.out.println("Running command: " + command);
Process p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
System.out.println("exit: " + p.exitValue());
p.destroy();
}
}

I was recently struggling with a similar issue, and I believe I the solution I found is an answer also to your problem (though your question is a bit malformed as others have pointed out).
The issue pertrains to the quote marks around your search string,
\""+matchStr+"\"
The java exec command will literally deliver these to the grep command, and instead of searching for matchStr, grep will be looking for "matchStr", and the results will not be what you are expecting.
This applies also in case one is executing the command as an array like
final Process process = Runtime.getRuntime().exec(new String[] { "grep", "-C1000", searchString, fileName } );
Pass the plain searchString without including quotation marks into the string.

Java program terminates before completion of command in batch file

I am trying to exceute batch file from a Java program.
The batch file has a command which connects to IBM RTC then gets some data which takes around 30 seconds.
But the program is exiting just after the command is run without waiting for the output.
public static void main(String[] args) {
final String scmCommand = "cmd /c D:\\Coverage\\SCMHistory.bat";
try {
Process process = Runtime.getRuntime().exec(scmCommand);
/*
* final InputStream in = process.getInputStream(); int ch;
* while((ch = in.read()) != -1) { System.out.print((char)ch); }
* final int returnCode = process.waitFor();
*/
try (final BufferedReader b = new BufferedReader(
new InputStreamReader(process.getInputStream()))) {
String line;
while ((line = b.readLine()) != null) {
System.out.println(line);
}
}
**System.out.println("waiting for the process");
process.waitFor();
System.out.println("waiting done");**
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I have tried adding process.waitFor(); but it didnt work.
set scm_path=D:\Coverage\RTC\jazz\scmtools\eclipse
set userId=ADMIN
set pwd=ADMIN
set repWorkspace="1081"
%scm_path%\scm show history -r https://rtc.repo.com:9443/jazz/ -u %userId% -P %pwd% -w "1411.201411" --component core_as D:\Work\201411\make\main_metadata.xml
Out put of which is
Change sets:
(3129) ----$ Sumit, HARI"main metadata is updated to deploy ch..." 03-Mar-2015 04:09 PM
(3130) ----$Sumit, HARI" "Fixed PartyID issue, checked in " 03-Mar-2015 01:01 PM
(3131) ----$ Sumit, HARI" "adding project to main_metada xml file" 26-Feb-2015 02:46 PM

Use start with the /w or start /wait option to run your program in the batch file.
Example
Start "" /w program options ...
Source Start a program, command or batch script (opens in a new window.)
Syntax
START "title" [/D path] [options] "command" [parameters]
Options:
/W or /WAIT Start application and wait for it to terminate.
(for an internal cmd command or a batch file this runs CMD /K)

your batch file is starting a new console window and terminating, even if you use start instead of cmd.
also the /c definition,
/c Carries out the command specified by string and then terminates
try this instead,
final String scmCommand = "D:\\Coverage\\SCMHistory.bat";
if this does not work try this,
final String scmCommand = "D:\Coverage\RTC\jazz\scmtools\eclipse\scm";
String[] envp = new String[5];
envp[0] = "-r https://rtc.repo.com:9443/jazz/";
envp[1] = "-u ADMIN";
envp[2] = "-P ADMIN";
envp[3] = "-w \"1411.201411\" ";
envp[4] = "--component core_as D:\Work\201411\make\main_metadata.xml";
Process process = Runtime.getRuntime().exec(scmCommand, envp);

Slow System Commands From Java

I am calling a bash scrip script from Java.
The script does the following:
cat /home/user/Downloads/bigtextfile.txt | grep 'hello'
This particular command when run command line takes about 1 second to complete on the text file which is 150MB.
When calling the bash script via Java using the following call:
command = "sh /home/user/bashfiletocall"
p = Runtime.getRuntime().exec(command);
The time to complete takes so long I don't wait.
Am I doing something very wrong and if not can you explain the reason for the huge lack in performance?
NOTE: I was running it in Netbeans and this seems to be the problem .. when I ran the file command line it was quick. The performance between execution in netbeans and command line is huge.
Many thanks.
private String executeCommand(String command) {
StringBuilder output = new StringBuilder();
BufferedReader reader = null;
Process p;
try {
p = Runtime.getRuntime().exec(command);
p.waitFor();
reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
}

After starting your process you need start reading from the input stream. Otherwise the buffers are running full and p.waitFor() waits forever.
Javadoc of the Process class:
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.

Runtime.getRuntime.exec() not working for the linux command "tar -xvf filename.tar"

I am trying to untar a file on a Unix machine, using a Java batch application.
Source Code:
String fileName = "x98_dms_12";
Runtime.getRuntime().exec("gunzip "+ fileName + ".tar.gz");
System.out.println(" Gunzip:"+"gunzip "+ fileName + ".tar.gz");
Runtime.getRuntime().exec("tar -xvf "+ fileName + ".tar");
System.out.println(" Extract:tar -xvf "+ fileName + ".tar");
Problem Description:
When I run the batch program it does not (completely) work. Only the gunzip command works, converting my fileName.tar.gz to fileName.tar. But the untar command does not seem to do anything, and there is no error or exception in my log or Unix console.
When I run the same commands in a Unix prompt they work fine.
Notes:
The path of execution is correct because it converts my *.tar.gz to *.tar
I cannot use "tar -zxvf fileName.tar.gz" since the attribute "z" does not work on my system.
There is no error or exception thrown.
Please do help.

A couple of things:
The tar command will expand a file relative to your working directory, which might need to be set for your Java Process objects
You should wait for the unzip process to complete before launching into the untar process
You should process the output streams from the processes.
Here is a working example that you can extend/adapt. It uses a separate class to deal with the process output streams:
class StreamGobbler implements Runnable {
private final Process process;
public StreamGobbler(final Process process) {
super();
this.process = process;
}
#Override
public void run() {
try {
final BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
reader.close();
} catch (final Exception e) {
e.printStackTrace();
}
}
}
public void extractTarball(final File workingDir, final String archiveName)
throws Exception {
final String gzFileName = archiveName + ".tar.gz";
final String tarFileName = archiveName + ".tar";
final ProcessBuilder builder = new ProcessBuilder();
builder.redirectErrorStream(true);
builder.directory(workingDir);
builder.command("gunzip", gzFileName);
final Process unzipProcess = builder.start();
new Thread(new StreamGobbler(unzipProcess)).start();
if (unzipProcess.waitFor() == 0) {
System.out.println("Unzip complete, now untarring");
builder.command("tar", "xvf", tarFileName);
final Process untarProcess = builder.start();
new Thread(new StreamGobbler(untarProcess)).start();
System.out.println("Finished untar process. Exit status "
+ untarProcess.waitFor());
}
}

The code below will print the output of the command executed. Check if it returns any error.
Process p = Runtime.getRuntime().exec("tar -xvf "+ fileName + ".tar");
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = null;
while ((line = br.readLine()) != null) {
System.out.println(line);
}

The problem is the commands which we give is UNIX command so it wont work in windows environment. I had written a script file to overcome this problem thanks all for you help. The Runtime.getRuntime.exec() will take some time to execute the command given so after each exec() give thread.wait(3000) to complete the process and goto next thread.

runtime.exec sending EOF immediately to input?

This is my code to start a process in Windows via java (and gobble the output).
public static void main(String[] args) throws Exception {
String[] command = new String[3];
command[0] = "cmd";
command[1] = "/C";
command[2] = "test.exe";
final Process child = Runtime.getRuntime().exec(command);
new StreamGobbler(child.getInputStream(), "out").start();
new StreamGobbler(child.getErrorStream(), "err").start();
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
child.getOutputStream()));
out.write("exit\r\n");
out.flush();
child.waitFor();
}
private static class StreamGobbler extends Thread {
private final InputStream inputStream;
private final String name;
public StreamGobbler(InputStream inputStream, String name) {
this.inputStream = inputStream;
this.name = name;
}
public void run() {
try {
BufferedReader in = new BufferedReader(new InputStreamReader(
inputStream));
for (String s = in.readLine(); s != null; s = in.readLine()) {
System.out.println(name + ": " + s);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
Somehow the program in question (process) is recieving an EOF right away (as in right after I step pas the "exec" line) and thus throwing an error ( detected, invalid) message immediately after runtime.exec is called. I can run this program manually via command prompt without this issue, but have confirmed that sending a ctrl-z on windows is what causes this message.
Anyone know what could be causing this?
If it matters, I have tried running the process directly as "test.exe" instead of cmd /c test.exe, but when I do that I can't see the output via the inputStream. And when I do cmd test.exe without the /c, there is no difference.

Your code looks like it should work (with one caveat, see below).
I took your code verbatim and replaced test.ext with sort, which can read from piped stdin.
If I run the code as-is, it starts the sort command, which waits for input. It hangs at child.waitFor() because you don't close the output stream to indicate EOF. When I add the close() call, everything works correctly.
I suggest you look at test.exe and determine if it is capable of reading from piped stdin, or is expecting console input.

Get rid of "cmd" and "/c". At present you are feeding output to cmd.exe, not to test.exe.