I'm probably missing something, but I'm trying to run commandline from a java
The code is as following:
String command = "sed -i 's/\\^#\\^/\\|/g' /tmp/part-00000-00000";
ProcessBuilder pb = new ProcessBuilder(command);
pb.redirectErrorStream(true);
Process process = pb.start();
process.waitFor();
if (process.exitValue() > 0) {
String output = // get output form command
throw new Exception(output);
}
I'm getting the following error:
java.lang.Exception: Cannot run program "sed -i 's/\^#\^/\|/g' /tmp/part-00000-00000": error=2, No such file or directory
The fils is exists. I'm doing ll on this file and it is exists.
I'm just looking for a way to make it work from java. What am I doing wrong?
Pass the command as an array, not a string:
String[] command={"sed", "-i", "'s/\\^#\\^/\\|/g'", "/tmp/part-00000-00000"};
See ProcessBuilder documentation.
Honestly there is no need to externally execute sed in this case. Read the file in Java and use Pattern. Then you have code that could run on any platform. Combine this with org.apache.commons.io.FileUtils and you can do it in a few lines of code.
Alternatively, you could use java.util.Scanner to avoid loading the whole file into memory.
final File = new File("/tmp/part-00000-00000");
String contents = FileUtils.readFileToString(file, StandardCharsets.UTF_8.name());
contents = Pattern.compile("\\^#\\^/\\").matcher(contents).replaceAll("|");
FileUtils.write(file, contents);
Or, in a short, self-contained, correct example
import org.apache.commons.io.FileUtils;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.regex.Pattern;
public final class SedUtil {
public static void main(String... args) throws Exception {
final File file = new File("part-00000-00000");
final String data = "trombone ^#^ shorty";
FileUtils.write(file, data);
sed(file, Pattern.compile("\\^#\\^"), "|");
System.out.println(data);
System.out.println(FileUtils.readFileToString(file, StandardCharsets.UTF_8));
}
public static void sed(File file, Pattern regex, String value) throws IOException {
String contents = FileUtils.readFileToString(file, StandardCharsets.UTF_8.name());
contents = regex.matcher(contents).replaceAll(value);
FileUtils.write(file, contents);
}
}
which gives output
trombone ^#^ shorty
trombone | shorty
this code here is magic really, simple and short and tested 100%
example i want to remove the last character of line from file (/sdcard/MT2/file.json)
String[] cmdline = { "sh", "-c", "sed -i 's/.$//' /sdcard/MT2/file.json " };
try {
Runtime.getRuntime().exec(cmdline);
} catch (Exception s) {
finishAffinity();
}
this magic code here don't run only sed, but also runnig echo, cat, ....ect
good luck
Try
Process p = Runtime.getRuntime().exec("sed -i 's/\\^#\\^/\\|/g' /tmp/part-00000-00000");
Related
I have simple java program for compiling java classes.
I created a JAR of this program and when I run it on Ubuntu I pass to the jar the path of folder with java files.
public class Main {
public static void main(String[] args) throws IOException, InterruptedException {
compile(args[0]);
}
//pathToFiles - is a value from command line arguments
private static void compile(String pathToFiles) throws IOException, InterruptedException {
List<String> cmdList = new ArrayList<>();
cmdList.add("javac");
cmdList.add(pathToFiles);
System.out.println("cmd: "+cmdList);
ProcessBuilder pb = new ProcessBuilder(cmdList);
Process process = pb.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
generateCompileException(process);
}
}
//method just generates error message if there was an error
private static void generateCompileException(Process process){
StringBuilder response = new StringBuilder();
try (final BufferedReader b = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
String line;
if ((line = b.readLine()) != null)
response.append(line);
} catch (final IOException e) {
e.printStackTrace();
}
throw new RuntimeException(response.toString());
}
}
When I pass path containing single java file it works:
java -jar co-1.jar /home/admin/test2/Calculator.java
But I want to compile multiple files. When I pass path containing multiple files I get error: file not found.
java -jar co-1.jar '/home/admin/test2/*.java'
PS: If I run a javac command manually with multiple files, it will work:
###################################
UPDATE:
I've added bash command to ProcessBuilder:
private static void compile(String pathToFiles) throws IOException, InterruptedException {
List<String> cmdList = new ArrayList<>();
cmdList.add("bash");
cmdList.add("-c");
cmdList.add("javac");
cmdList.add(pathToFiles);
System.out.println("Processor builder command: "+cmdList);
ProcessBuilder pb = new ProcessBuilder(cmdList);
Process process = pb.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
System.out.println("Finished with error. Exit value: "+exitValue);
generateCompileException(process);
}
}
But process withished with error code 2 with empty response from ProcessBuilder.
PS: RuntimeException was thrown by this line: throw new RuntimeException(response.toString());
ProcessBuilder will not evaluate wildcards, as that is a feature of your terminal (such as bash). If you want wildcard to be expanded you need to run bash inside ProcessBuilder command, such as:
String commandContainingWildcard = "javac /blah/*.java";
ProcessBuilder pb = new ProcessBuilder("bash", "-c", commandContainingWildcard);
... // start() etc
For the above to work you need to have "bash" or whatever shell you use in your path, otherwise you will need to use full path to bash (such as "/bin/bash").
The third argument for command to compile must exactly match what works inside your terminal and must be the entire value not "javac" followed by wildcard. Remove single quotes around *.java (so that ProcessBuilder is provided with three command line parameters, not four or more).
However I suggest that ProcessBuilder with bash isn't the best way to do this work. You could try Java compiler tool interface, and get rid of wildcard by easy use of Files.find(dir, 1, (p,a) -> p.getFileName().toString().endsWith(".java")) to scan for all java files and join the paths explicitly for compilation.
UPDATE
Having now resolved your problem you may now find that the javac process fails / freezes due the incorrect way you read the stderr stream - this needs to happen at same time as stdout and before process.waitFor(). An easy fix is to consume stdout+stderr together:
ProcessBuilder pb = new ProcessBuilder(cmdList);
pb.redirectErrorStream(true);
Process process = pb.start();
ByteArrayOutputStream response = new ByteArrayOutputStream();
process.getInputStream().transferTo(response);
int exitValue = process.waitFor();
if (exitValue != 0) {
System.out.println("Finished with error. Exit value: "+exitValue);
throw new RuntimeException(new String(response.toByteArray()));
}
Remove quotes and use the command as below.
java -cp co-1.jar:/home/admin/test2/* Main.class <args>
See also
https://docs.oracle.com/javase/7/docs/technotes/tools/windows/classpath.html
PS: Unix uses :(colon) as delimiter and windows uses ;(semi-colon) delimiter to separate multiple paths.
I found this as one of the ways to run (using exec() method) python script from java. I have one simple print statement in python file. However, my program is doing nothing when I run it. It neither prints the statement written in python file nor throws an exception. The program just terminates doing nothing:
Process p = Runtime.getRuntime().exec("C:\\Python\\Python36-32\\python.exe C:\\test2.py");
Even this is not creating the output file:
Process p = Runtime.getRuntime().exec("C:\\Python\\Python36-32\\python.exe C:\\test2.py output.txt 2>&1");
What is the issue?
I think you could try your luck with the ProcessBuilder class.
If I read the Oracle documentation correctly, the std inputs and outputs are directed to pipes by default but the ProcessBuilder has an easy method for you to explicitly set output (or input) to a file on your system or something else.
If you want your Python program to use the same output as your Java program (likely stdout and stderr), you can use stg like this:
ProcessBuilder pb = new ProcessBuilder("C:\\Python\\Python36-32\\python.exe", "C:\\test2.py");
pb.redirectOutput(Redirect.INHERIT);
Process p = pb.start();
You can use the ProcessBuilder API, redirecting the output to a file and then wait for the result.
public class Main {
public static final String PYTHON_PATH = "D:\\Anaconda3\\python.exe";
public static final String PATH_TO_SCRIPT = "D:\\projects\\StartScript\\test.py";
public static void main(String[] args) throws IOException, InterruptedException {
ProcessBuilder builder = new ProcessBuilder();
builder.command(PYTHON_PATH, PATH_TO_SCRIPT);
// Redirect output to a file
builder.redirectOutput(new File("output.txt"));
builder.start().waitFor();
// Print output to console
ProcessBuilder.Redirect output = builder.redirectOutput();
File outputFile = output.file();
BufferedReader br = new BufferedReader(new FileReader(outputFile));
String st;
while ((st = br.readLine()) != null) {
System.out.println(st);
}
}
}
The python file test.py contains a simple print statement:
print("Hello from python")
I guess it would be even simpler, if you do not need to wait for the result.
Using the Process API should work, too.
Like in your example (I am using the same constants declared above):
Process p = Runtime.getRuntime().exec(PYTHON_PATH + " " + PATH_TO_SCRIPT);
p.waitFor();
byte[] buffer = new byte[1024];
byte[] errBuffer = new byte[1024];
p.getInputStream().read(buffer);
p.getErrorStream().read(errBuffer);
System.out.println(new String(buffer));
System.out.println(new String(errBuffer));
To see the output of the print statement, you need to wait and redirect the streams. Same for the error stream.
Now if you break the python script like this:
print("Hello from python')
you should be able to see the error printed as well.
One way to start a python process is using an entrypoint - test.cmd
echo Hello
python hello.py
here is hello.py
#!/usr/bin/env python3
import os
if not os.path.exists('dir'):
os.makedirs('dir')
Here is my Java code:
public static void main(String[] args) throws IOException {
try {
Process p = Runtime.getRuntime().exec("test.cmd");
p.waitFor();
Scanner sc = new Scanner(p.getInputStream());
while(sc.hasNextLine()){
System.out.println(sc.nextLine());
}
sc.close();
} catch (Exception err) {
err.printStackTrace();
}
}
I am trying to use this command:
git checkout `git rev-list -n 1 --before="2009-07-27 13:37" master`
Which I am running through ProcessBuilder. Now, I know that the rest of my code works, as I have tested it with other examples. But I am not sure how to split this command so that it works for ProcessBuilder.
Apparently, this method did not work:
String[] command = {"git", "checkout", "`git rev-list -n 1 --before=\"2014-01-01 12:00\" master`"};
This did not work as well:
String[] command = {"git", "checkout", "`git", "rev-list", "-n", "1", "--before=\"2014-01-01 12:00\"", "master`"};
I am also not getting any output from the console (I do InputStream), which means it's harder for me to figure out what's the issue. I am thinking the problem should be obvious to people familiar enough with ProcessBuilders, so yeah...
The rest of my code looks as such:
ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.directory(new File(fullPath));
process = processBuilder.start();
And the path for the directory should be correct as I checked it (it is inside of the main directory of the repository).
If by any chance I need to provide other parts of the code, I will.
The back-ticks are actually handled by the unix shell - Java doesn't process them. When using process builder you should execute the command given in the back-ticks and pass the output as parameters to the second command. A simple example is below: (using 'ls').
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.io.IOUtils;
public class ProcessBuilderTest {
public static void main(String[] args) throws IOException, InterruptedException {
// Command: ls `ls /etc/passwd /etc/services`
// Read the output from the command second ls...
ProcessBuilder pb = new ProcessBuilder("ls", "/etc/passwd", "/etc/services");
Process process = pb.start();
InputStream is = process.getInputStream();
List<String> lines = IOUtils.readLines(is, Charset.defaultCharset());
ArrayList<String> cmdParts = new ArrayList<>();
cmdParts.add("ls");
cmdParts.addAll(lines);
// Construct the second command
ProcessBuilder pbLs = new ProcessBuilder(cmdParts);
Process lsProcess = pbLs.start();
System.out.println(IOUtils.readLines(lsProcess.getInputStream(), Charset.defaultCharset()));
lsProcess.waitFor();
}
}
I searched a lot but did not find the solution.
My goal is using java to call commands and get output in windows and linux. I found Runtime.exec method and did some experiments.
Everything went ok except when there's space in the command parameters.
Test code as below, also in github.
The code works well on windows, but in linux, output is empty:
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class Main {
public static void main(String[] args) {
try {
Runtime rt = Runtime.getRuntime();
String[] commandArray;
if (isWindows()) {
commandArray = new String[]{"cmd", "/c", "dir", "\"C:\\Program Files\""};
} else {
commandArray = new String[]{"ls", "\"/root/a directory with space\""};
}
String cmd = String.join(" ",commandArray);
System.out.println(cmd);
Process process = rt.exec(commandArray);
BufferedReader input = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String result = "";
String line = null;
while ((line = input.readLine()) != null) {
result += line;
}
process.waitFor();
System.out.println(result);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
public static boolean isWindows() {
String OS = System.getProperty("os.name").toLowerCase();
return (OS.indexOf("win") >= 0);
}
}
if I execute the printed command in bash directly, then the output is as expected.
[root#localhost javatest]# javac Main.java
[root#localhost javatest]# java Main
ls "/root/a directory with space"
[root#localhost javatest]# ls "/root/a directory with space"
a.txt b.txt
[root#localhost javatest]#
Can anyone explain why and give ways to solve?
There are two versions of exec.
exec(String command)
Here you specify a command in a similar way to how you would do it on the command-line, i.e. you need to quote arguments with spaces.
cmd /c dir "C:\Program Files"
exec(String[] cmdarray)
Here you specify the arguments separately, so the arguments are given as-is, i.e. without quotes. The exec method will take care of any spaces and quote-characters in the argument, correctly quoting and escaping the argument as needed to execute the command.
cmd
/c
dir
C:\Program Files
So, remove the extra quotes you added:
if (isWindows()) {
commandArray = new String[] { "cmd", "/c", "dir", "C:\\Program Files"};
} else {
commandArray = new String[] { "ls", "/root/a directory with space"};
}
I am trying to execute some Linux commands from Java using redirection (>&) and pipes (|). How can Java invoke csh or bash commands?
I tried to use this:
Process p = Runtime.getRuntime().exec("shell command");
But it's not compatible with redirections or pipes.
exec does not execute a command in your shell
try
Process p = Runtime.getRuntime().exec(new String[]{"csh","-c","cat /home/narek/pk.txt"});
instead.
EDIT::
I don't have csh on my system so I used bash instead. The following worked for me
Process p = Runtime.getRuntime().exec(new String[]{"bash","-c","ls /home/XXX"});
Use ProcessBuilder to separate commands and arguments instead of spaces. This should work regardless of shell used:
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
public class Test {
public static void main(final String[] args) throws IOException, InterruptedException {
//Build command
List<String> commands = new ArrayList<String>();
commands.add("/bin/cat");
//Add arguments
commands.add("/home/narek/pk.txt");
System.out.println(commands);
//Run macro on target
ProcessBuilder pb = new ProcessBuilder(commands);
pb.directory(new File("/home/narek"));
pb.redirectErrorStream(true);
Process process = pb.start();
//Read output
StringBuilder out = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = null, previous = null;
while ((line = br.readLine()) != null)
if (!line.equals(previous)) {
previous = line;
out.append(line).append('\n');
System.out.println(line);
}
//Check result
if (process.waitFor() == 0) {
System.out.println("Success!");
System.exit(0);
}
//Abnormal termination: Log command parameters and output and throw ExecutionException
System.err.println(commands);
System.err.println(out.toString());
System.exit(1);
}
}
Building on #Tim's example to make a self-contained method:
import java.io.BufferedReader;
import java.io.File;
import java.io.InputStreamReader;
import java.util.ArrayList;
public class Shell {
/** Returns null if it failed for some reason.
*/
public static ArrayList<String> command(final String cmdline,
final String directory) {
try {
Process process =
new ProcessBuilder(new String[] {"bash", "-c", cmdline})
.redirectErrorStream(true)
.directory(new File(directory))
.start();
ArrayList<String> output = new ArrayList<String>();
BufferedReader br = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String line = null;
while ( (line = br.readLine()) != null )
output.add(line);
//There should really be a timeout here.
if (0 != process.waitFor())
return null;
return output;
} catch (Exception e) {
//Warning: doing this is no good in high quality applications.
//Instead, present appropriate error messages to the user.
//But it's perfectly fine for prototyping.
return null;
}
}
public static void main(String[] args) {
test("which bash");
test("find . -type f -printf '%T#\\\\t%p\\\\n' "
+ "| sort -n | cut -f 2- | "
+ "sed -e 's/ /\\\\\\\\ /g' | xargs ls -halt");
}
static void test(String cmdline) {
ArrayList<String> output = command(cmdline, ".");
if (null == output)
System.out.println("\n\n\t\tCOMMAND FAILED: " + cmdline);
else
for (String line : output)
System.out.println(line);
}
}
(The test example is a command that lists all files in a directory and its subdirectories, recursively, in chronological order.)
By the way, if somebody can tell me why I need four and eight backslashes there, instead of two and four, I can learn something. There is one more level of unescaping happening than what I am counting.
Edit: Just tried this same code on Linux, and there it turns out that I need half as many backslashes in the test command! (That is: the expected number of two and four.) Now it's no longer just weird, it's a portability problem.