I'm having the next error when execute a cmd command using Java. I'm working in a mac laptop. This is my code:
private static String exportContainerFromImage(String container) {
//docker export mysql_dummy > ~/Documents/mysql_dummy.tar
String errorMessage = "";
String[] cmdArgs =
{"docker export mysql_dummy > ~/Documents/mysql_dummy.tar", "bash"};
Process process = Runtime.getRuntime().exec(cmdArgs);
}
But I'm getting the error, error=2, No such file or directory, if I execute the command directly on the terminal it runs successfully, I tried also changing the directory to ~\\Documents\\mysql_dummy.tar and got the same result.
But if I run the command with the arguments:
{"docker create -ti --name mysql_dummy mysql", "bash"};
It runs properly
Any ideas?
You're conflating 'shell magic' with 'an OS'. Also, you seem to be wildly confused about what the array form of cmdArgs does, because you've tagged a bash in there at the end. That array is supposed to contain the executable's full path at arr[0], and all arguments at arr[1] and up. docker create -ti ... is clearly not a filename, and bash is clearly not an argument.
Shell magic?
If you type:
docker create -ti --name mysql_dummy mysql
on the command line, bash (or cmd.exe if on windows, or whatever shell you are using) reads it and does a whole bunch of replacement magic and parsing on this. It's the shell that does this, not the OS, and java's processbuilder stuff is not a shell and therefore isn't going to do all that. What you're attempting to do? Run that entire line as if it's a single file name that is executable which it clearly isn't.
This is all shell magic - all things that you CANNOT do with exec. Fortunately, java is a programming language, so you can do all these things by, well, programming it.
Parsing out params by splitting on whitespace.
quoting to avoid that splitting, but then removing the quotes.
Treating ~ as a ref to a homedir.
Replacing * and ? in filename paths.
Variable substitution
Setting up redirects with > somefile.txt or 2> /dev/null or < file.in or whatnot.
You must do those things.
In addition, exec cannot be used to this, period. As usual, the only non-problematic way to run processes is to always use ProcessBuilder, no exceptions. Consider runtime.exec a known-broken method you must never call.
ProcessBuilder lets you redirect the output.
String[] cmdArgs = {
"/bin/docker" // note, FULL PATH!
"export",
"mysql_dummy"
};
File out = new File(System.getProperty("user.home"), "Documents/mysql_dummy.tar");
ProcessBuilder pb = new ProcessBuilder(cmdArgs);
pb.redirectOutput(new File(out));
pb.start();
That does what you want, presumably.
The alternative is to make a script (script.sh or script.bat) and then start bash or cmd.exe and ask it to run that script.
String[] args = { "/bin/bash", "-c", "/fully/qualified/path/to/the/script.sh" }
and then exec that. Now you can pile *.txt, > foobar.txt, ~/homediref, and all the other shellisms in that script as much as you like.
Overall #rzwiterloot's answer is good, but there are some alternatives it leaves out.
First, what I would consider the best solution to this problem: the -o option to docker export.
"docker", "export", "mysql_dummy", "-o", "Documents/mysql_dummy.tar"
Ignoring ~/ here, this set of command and arguments will achieve the same thing as ... > Documents/mysql_dummy.tar but doesn't rely on the shell for the redirection; docker export is perfectly capable of handling that operation itself.
Second, if you wanted to run a shell command from the program, you could. I would not recommend this. But in certain circumstances it might make sense.
The alternative is to make a script
You don't have to put the command in a separate file. Actually this is one inaccuracy in #rzwiterloot's answer; -c allows you to pass command(s) to bash, not the path to a file containing commands.
"bash", "-c", "docker export mysql_dummy > ~/Documents/mysql_dummy.tar"
However, I'd recommend you avoid invoking shells from any program you write. They're quirky and esoteric and there's almost always a simpler way to achieve what you want, such as docker export's -o optiopn, in this case.
Related
shell script file directory: /some/location/myShellScript.sh
Properties-Type: shell script (application/x-shellscript)
EDIT
content of shell script:
#!/bin/bash
export LD_LIBRARY_PATH=`pwd`
echo `pwd`
./someExecutable ../input/cfg/test1.ini
The test1.ini is generated one step before in the java code,
it provides settings for some testing, which is done in the background. Then the shell script ends up with the file I need for further processing.
/EDIT
When I am running this shell script on linux terminal in its own directory just with "./myShellScript.sh" it works perfectly fine...
The part my shell script shall be executed:
//Do something before
//Shell scripts creates a file
String cmd = /some/location/myShellScript.sh;
ProcessBuilder pb = new ProcessBuilder(cmd);
Process process = pb.start();
int exitValue = process.waitFor();
System.out.println(exitValue);
//Afterwards I am processing the generated file
When running my java program as an executable .jar file, this process gets not executed and the exitValue is 127, but I don't know why...
I tried many things like:
using the Runtime to exec
adding #!/bin/bash or #!/bin/sh on top of the shell script
adding a "sh" parameter to the process command in form of String[]
In my execution directory, I changed the permission with chmod 755 -R * recursively so every associated library used by the shell script is indeed available (also due to the fact, that I can just execute it on the terminal).
I really tried to find a proper answer on the internet but I wasn't successful.
And no, I cannot just do everything in java, the shell script is mandatory and cannot be replaced in this case.
Thanks in advance for helpful suggestions!
The script you are executing is highly sensitive to its working directory. It uses pwd to set the LD_LIBRARY_PATH and it attempts to execute another program via a relative path to that program, providing a relative path as a command-line argument, as well.
The working directory for an execution of the script has no essential relationship with the directory in which the script resides -- it completely depends on how and in what context the script is launched. For example, you report that the script works as you expect "When I am running this shell script [...] in its own directory." But when you run the script from Java you very likely are not running it with its own directory as the working directory, and that will strongly affect this script's behavior.
One solution would be to hardcode the script's installation path into the script itself, and to express all your paths relative to that:
#!/bin/bash
installation_dir=/path/to/the/script/dir
export LD_LIBRARY_PATH=$installation_dir
"$installation_dir"/someExecutable "$installation_dir"/../input/cfg/test1.ini
It's a bit kludgy to hardcode the path, though. You could further improve it by having the script identify its own directory at runtime:
#!/bin/bash
installation_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
export LD_LIBRARY_PATH=$installation_dir
"$installation_dir"/someExecutable "$installation_dir"/../input/cfg/test1.ini
That's obviously Bash-specific, but you were using bash anyway. Alternatively, if the executable your script launches is also sensitive to its working directory, then perhaps you just want the script to change directory (which will be effective only for the script and processes downstream from it, not for its parent process):
#!/bin/bash
cd "$( dirname "${BASH_SOURCE[0]}" )"
export LD_LIBRARY_PATH=`pwd`
./someExecutable ../input/cfg/test1.ini
The 127 exit status means that a command used in the script is not found.
EDIT
Debug the script, when bash is used, add the line below on the second line:
exec > /tmp/debug.txt 2>&1 ; set -x
After the next attempt, analyze the traces generated into the /tmp/debug.txt file.
OLD INTRO
(the script content was not yet provided)
The Java program which executes the myShellScript.sh script has probably not the same PATH environment variable than the one which is set in your environment when you execute the script manually from a terminal.
I want to execute wsl commands from java.
I'm trying to do this using Process and ProcessBuilder.
As I understand, there are two ways to do this:
Run wsl along with command as argument (for example: wsl ls -l) (do this per-command).
Run wsl, and then execute the commands one by one.
But there are some problems with 1 and 2.
With point 1:
When the command terminates, the process does not stop. So even if i run wsl ls, I can not determine the moment when I can call next command.
wsl does not save the state between such calls, so it's not very convenient
With point 2:
Since wsl does not show bash prompt, I can not track when the command stopped displaying information. For example:
ProcessBuilder pb = new ProcessBuilder("wsl");
pb.redirectErrorStream(true);
Process p = pb.start();
Thread.sleep(1000);
OutputStreamWriter osw = new OutputStreamWriter(p.getOutputStream());
osw.write("ls\n");
osw.flush();
And all I can read is:
build.gradle
gradle
gradlew
gradlew.bat
out
settings.gradle
src
No selya#selya-pc:/mnt/c/Users/selya$ read. So I can't use it as a separator between commands. I think that wsl somehow tracks, in what environment it was launched, for example, through isatty(), and therefore the output is different.
With both:
Аs far as I know, for programs that are running outside the terminal, stdout is not buffered. So there are some problems with, for example, sudo, because it asks fro a password without newline/flush ([sudo] password for selya:), therefore I can't read this line...
I found a solution - pass command as argument to unbuffer util, for example:
wsl unbuffer -p sudo apt-get update
(-p stands for pipeline).
But other problems still remain. Is there any way to do this? Or maybe there is lib for it? Even c or c++ lib will suit my needs...
P.S. I tried to find a solution for several days. Russian-speaking SO didn't help me, that's why I'm here. Sorry for my English...
The problem was solved with the help of pty4j. It works on linux, mac and windows. (It works like pseudo terminal on linux, but have nice java interface).
I am trying to make my Java program interact with Linux bash but something goes wrong. I have a simple executable prog that reads the one integer from stdin and outputs its square. Executing
echo 5 | ./prog
from bash itself prints correct answer 25 in stdout but running
import java.io.*;
public class Main {
public static void main(String[] args) throws InterruptedException, IOException {
Runtime run = Runtime.getRuntime();
Process proc = run.exec("echo 5 | ./prog");
proc.waitFor();
BufferedReader br = new BufferedReader(new InputStreamReader(proc.getInputStream()));
while(br.ready())
System.out.println(br.readLine());
}
}
unexpectedly gives 5 | ./prog. What is the solution?
Java exec cannot run shell commands like that. If you want to run a shell command, you need to explicitly invoke the shell; e.g.
Process proc = run.exec(new String[]{"/bin/sh", "-c", "echo 5 | ./prog"});
For more details on what Java does with this, read the javadocs for exec(String) and exec(String[]). Note that these are "convenience methods", and you need to follow the chain of links to the underlying methods for a complete understanding of what the javadoc is saying.
If you want even more detail on how Java handles this, there is the source code ...
If you want to understand in depth why Java doesn't handle the shell syntax itself, you probably need to take a deep dive into the architecture and philosophy of UNIX / Linux systems, and the separation of concerns between application, operating system and command shell. Note that there are a myriad different shells, each with (potentially) different rules for quoting, argument splitting, redirection, pipes, etcetera. Most of the popular shells are similar, but that's just the way things panned out.
Explanation of the solution:
The reason for splitting the command by hand is that exec(String) won't split the command into a single command and arguments correctly. It can't. This is an example where there are two commands in a pipeline.
The reason for using "sh" is ... well ... you need a shell to parse and process a shell command line. Java's exec command does not support shell syntax ... as the javadoc explains.
The purpose of the "-c" option is explained by "man sh". Basically, sh -c "a b c" means "use 'sh' to run the command line 'a b c'".
FWIW, it is technically possible to construct and run a pipeline solely in Java (i.e. without relying on an external shell), but it is generally not worth the effort. You've already introduced a platform dependency by running external commands, so an extra dependency in the form of a specific shell command and syntax doesn't make things significantly worse.
I'm using apache-commons-exec to execute some commands in a Java application.
When I execute 'ls -la /home/user' it works great.
But I need to execute something like this
./setEnvsOfTypeXXX.sh; ./setEnvsOfTypeYYY.sh; ls -la /home/user
I enter the command into the CommandLine object and it doesn't work.
It returns an empty string and -559038737 exit code.
Because the nature of the environment and the scripts (the firsts ones sets some needed environment variables); i can not put all the call into a script o
I've tried many solutions (like surround all the command with quotation marks like "'" or use the PumStreamHandlet input stream) but nothing has worked so far...
Anyone has an idea ?
try
sh -c '. ./setEnvsOfTypeXXX.sh; . ./setEnvsOfTypeYYY.sh; ls -la /home/user'
As your command
Two things I'm guessing you need here.
First if you are setting enviroment variables you probably need to use .
Second you want to run a shell and get the shell to exec the shell scripts and then run the following command, all in the same context
I tried this code
cmdLine = new CommandLine("/bin/bash");
cmdLine.addArgument("-c");
cmdLine.addArgument(new StringBuilder().append("'").append(command).append("'").toString());
And even with command = "ls";
There is an error
bash: ls: No such file or directory
fun fact: in windows this works ok !
cmdLine = new CommandLine("cmd.exe");
cmdLine.addArgument("/c");
cmdLine.addArgument(new StringBuilder().append("\"").append(command).append("\"").toString());
logger.info("Command win line: cmd.exe /c \""+command + "\"");
I totally out of options now !!!
I got a workarround: create a temporal sh file with the command, putting shebang on firts line and giving permissions, executing this file in one command line, get result and output, for last delete temporal file...
and it works !
I am trying to accomplish two things:
I am running cygwin on Windows7 to execute my unix shell commands and I need to automate the process by writing a Java app. I already know how to use the windows shell through Java using the 'Process class' and Runtime.getRuntime().exec("cmd /c dir"). I need to be able to do the same with unix commands: i.e.: ls -la and so forth. What should I look into?
Is there a way to remember a shell's state?
explanation: when I use: Runtime.getRuntime().exec("cmd /c dir"), I always get a listing of my home directory. If I do Runtime.getRuntime().exec("cmd /c cd <some-folder>") and then do Runtime.getRuntime().exec("cmd /c dir") again, I will still get the listing of my home folder. Is there a way to tell the process to remember its state, like a regular shell would?
It seems that the bash command line proposed by Paŭlo does not work:
C:\cygwin\bin>bash -c ls -la
-la: ls: command not found
I am having trouble figuring out the technicalities.
This is my code:
p = Runtime.getRuntime().exec("C:\\cygwin\\bin\\bash.exe -c ls -la");
reader2 = new BufferedReader(new InputStreamReader(p.getInputStream()));
line = reader2.readLine();
line ends up having a null value.
I added this to my .bash_profile:
#BASH
export BASH_HOME=/cygdrive/c/cygwin
export PATH=$BASH_HOME/bin:$PATH
I added the following as well:
System Properties -> advanced -> Environment variables -> user variebales -> variable: BASH, value: c:\cygwin\bin
Still nothing...
However, if I execute this instead, it works!
p = Runtime.getRuntime().exec("c:\\cygwin\\bin\\ls -la ~/\"Eclipse_Workspace/RenameScript/files copy\"");
1. Calling unix commands:
You simply need to call your unix shell (e.g. the bash delivered with cygwin) instead of cmd.
bash -c "ls -la"
should do. Of course, if your command is an external program, you could simply call it directly:
ls -la
When starting this from Java, it is best to use the variant which takes a string array, as then
you don't have Java let it parse to see where the arguments start and stop:
Process p =
Runtime.getRuntime().exec(new String[]{"C:\\cygwin\\bin\\bash.exe",
"-c", "ls -la"},
new String[]{"PATH=/cygdrive/c/cygwin/bin"});
The error message in your example (ls: command not found) seems to show that your bash can't find the ls command. Maybe you need to put it into the PATH variable (see above for a way to do this from Java).
Maybe instead of /cygdrive/c/cygwin/bin, the right directory name would be /usr/bin.
(Everything is a bit complicated here by having to bridge between Unix and Windows
conventions everywhere.)
The simple ls command can be called like this:
Process p = Runtime.getRuntime().exec(new String[]{"C:\\cygwin\\bin\\ls.exe", "-la"});
2. Invoking multiple commands:
There are basically two ways of invoking multiple commands in one shell:
passing them all at once to the shell; or
passing them interactively to the shell.
For the first way, simply give multiple commands as argument to the -c option, separated by ; or \n (a newline), like this:
bash -c "cd /bin/ ; ls -la"
or from Java (adapting the example above):
Process p =
Runtime.getRuntime().exec(new String[]{"C:\\cygwin\\bin\\bash.exe",
"-c", "cd /bin/; ls -la"},
new String[]{"PATH=/cygdrive/c/cygwin/bin"});
Here the shell will parse the command line as, and execute it as a script. If it contains multiple commands, they will all be executed, if the shell does not somehow exit before for some reason (like an exit command). (I'm not sure if the Windows cmd does work in a similar way. Please test and report.)
Instead of passing the bash (or cmd or whatever shell you are using) the commands on the
command line, you can pass them via the Process' input stream.
A shell started in "input mode" (e.g. one which got neither the -c option nor a shell script file argument) will read input from the stream, and interpret the first line as a command (or several ones).
Then it will execute this command. The command itself might read more input from the stream, if it wants.
Then the shell will read the next line, interpret it as a command, and execute.
(In some cases the shell has to read more than one line, for example for long strings or composed commands like if or loops.)
This will go on until either the end of the stream (e.g. stream.close() at your side) or executing an explicit exit command (or some other reasons to exit).
Here would be an example for this:
Process p = Runtime.getRuntime().exec(new String[]{"C:\\cygwin\\bin\\bash.exe", "-s"});
InputStream outStream = p.getInputStream(); // normal output of the shell
InputStream errStream = p.getInputStream(); // error output of the shell
// TODO: start separate threads to read these streams
PrintStream ps = new PrintStream(p.getOutputStream());
ps.println("cd /bin/");
ps.println("ls -la");
ps.println("exit");
ps.close();
You do not need cygwin here. There are several pure Java libraries implementing SSH protocol. Use them. BTW they will solve your second problem. You will open session and execute command withing the same session, so the shell state will be preserved automatically.
One example would be JSch.