We Keep Coding

File not found when calling bash from Java

I'm having the next error when execute a cmd command using Java. I'm working in a mac laptop. This is my code:
private static String exportContainerFromImage(String container) {
//docker export mysql_dummy > ~/Documents/mysql_dummy.tar
String errorMessage = "";
String[] cmdArgs =
{"docker export mysql_dummy > ~/Documents/mysql_dummy.tar", "bash"};
Process process = Runtime.getRuntime().exec(cmdArgs);
}
But I'm getting the error, error=2, No such file or directory, if I execute the command directly on the terminal it runs successfully, I tried also changing the directory to ~\\Documents\\mysql_dummy.tar and got the same result.
But if I run the command with the arguments:
{"docker create -ti --name mysql_dummy mysql", "bash"};
It runs properly
Any ideas?

You're conflating 'shell magic' with 'an OS'. Also, you seem to be wildly confused about what the array form of cmdArgs does, because you've tagged a bash in there at the end. That array is supposed to contain the executable's full path at arr[0], and all arguments at arr[1] and up. docker create -ti ... is clearly not a filename, and bash is clearly not an argument.
Shell magic?
If you type:
docker create -ti --name mysql_dummy mysql
on the command line, bash (or cmd.exe if on windows, or whatever shell you are using) reads it and does a whole bunch of replacement magic and parsing on this. It's the shell that does this, not the OS, and java's processbuilder stuff is not a shell and therefore isn't going to do all that. What you're attempting to do? Run that entire line as if it's a single file name that is executable which it clearly isn't.
This is all shell magic - all things that you CANNOT do with exec. Fortunately, java is a programming language, so you can do all these things by, well, programming it.
Parsing out params by splitting on whitespace.
quoting to avoid that splitting, but then removing the quotes.
Treating ~ as a ref to a homedir.
Replacing * and ? in filename paths.
Variable substitution
Setting up redirects with > somefile.txt or 2> /dev/null or < file.in or whatnot.
You must do those things.
In addition, exec cannot be used to this, period. As usual, the only non-problematic way to run processes is to always use ProcessBuilder, no exceptions. Consider runtime.exec a known-broken method you must never call.
ProcessBuilder lets you redirect the output.
String[] cmdArgs = {
"/bin/docker" // note, FULL PATH!
"export",
"mysql_dummy"
};
File out = new File(System.getProperty("user.home"), "Documents/mysql_dummy.tar");
ProcessBuilder pb = new ProcessBuilder(cmdArgs);
pb.redirectOutput(new File(out));
pb.start();
That does what you want, presumably.
The alternative is to make a script (script.sh or script.bat) and then start bash or cmd.exe and ask it to run that script.
String[] args = { "/bin/bash", "-c", "/fully/qualified/path/to/the/script.sh" }
and then exec that. Now you can pile *.txt, > foobar.txt, ~/homediref, and all the other shellisms in that script as much as you like.

Overall #rzwiterloot's answer is good, but there are some alternatives it leaves out.
First, what I would consider the best solution to this problem: the -o option to docker export.
"docker", "export", "mysql_dummy", "-o", "Documents/mysql_dummy.tar"
Ignoring ~/ here, this set of command and arguments will achieve the same thing as ... > Documents/mysql_dummy.tar but doesn't rely on the shell for the redirection; docker export is perfectly capable of handling that operation itself.
Second, if you wanted to run a shell command from the program, you could. I would not recommend this. But in certain circumstances it might make sense.
The alternative is to make a script
You don't have to put the command in a separate file. Actually this is one inaccuracy in #rzwiterloot's answer; -c allows you to pass command(s) to bash, not the path to a file containing commands.
"bash", "-c", "docker export mysql_dummy > ~/Documents/mysql_dummy.tar"
However, I'd recommend you avoid invoking shells from any program you write. They're quirky and esoteric and there's almost always a simpler way to achieve what you want, such as docker export's -o optiopn, in this case.

Standard input through sudo

I'm writing a Java program that calls other programs, and some of them need to be ran as root.
I thought about running the whole JAR as root but that sounds like a security risk, I'd rather run just some programs as root.
Currently, I create a ProcessBuilder and use its getStandardOutput method to instantiate a Writer, on the command sudo -S pacman <...>. (-S means ‘read from standard input’).
I then inject the password using that reader and flush it, so far so good.
However, when pacman expects input (for example when asking an interactive yes/no question), and I try to write to the Writer again, it fails with IOException because the stream is closed.
How can I get a Writer that can actually send characters to pacman and other programs ran as root?
This program is only expected to run on UNIX (Arch Linux, Debian, MacOS and maybe Window's WSL) so UNIX-specific solutions are fine, if possible I'd rather not use JNI because it requires to compile multiple times, but if that's the only solution it's fine. I'm also fine replacing sudo with something else as long as it's still ran as root.

If this is not a long-lived service, you could run sudo -v to have sudo cache the credentials and then invoke sudo in your Java program without needing to enter a password. This would work best if you invoked your Java program with something like:
#!/bin/bash
sudo -v
java -jar ...

Can't run exe file using ProcessBuilder depending of computer

I have come across a problem while working with Process and ProcessBuilder in Java.
I have a line of code looking like this :
Process process = new ProcessBuilder("des.exe", String... params);
It works fine on my personal computer, but not on my testing server and I can't figure out why.
The ErrorStream give me this:
des.exe: MZ����#���: not found
des.exe: Syntax error: "(" unexpected
Both computers are on Ubuntu 16.04.4 LTS with java 1.8.
Any idea where it might come from?

The most likely reason for the difference is that the file format of windows executables is not registered with the binfmt_misc handler in the kernel, which allows the kernel to execute various programs through helpers transparently. The error Syntax error: "(" unexpected typically means that the file got handed off to /bin/sh, instead of to wine.
So, check if the binfmt_misc file system is mounted (most modern linuxes do this automatically):
grep binfmt_misc /proc/self/mountinfo
should return some entries (might be autofs, might be binfmt_misc depends on the distro).
if it isn't then mount it:
sudo mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc
you can tell what's registered by looking at the content of this directory. To register for running dos executables with wine you need to register it:
echo ':DOSWin:M::MZ::/usr/bin/wine:' | sudo tee /proc/sys/fs/binfmt_misc/register
Once registered, executing the binary should happen transparently, and it should succeed through the ProcessBuilder.
Configuring this for the server at boot time is left as an exercise to the user.
If you want to skip this entire mess, then you can change the execution from des.exe to wine with the first argument of des.exe.

Execute shell command in Tomcat

So I have the following problem: I have a web service running inside a Tomcat7 server on Linux. The web service however has to execute some commands (mostly file operations such as copy and mount). Copy I've replaced with java.nio, but I don't think that there is a replacement for mount.
So I'm trying to execute shell commands out of my Tomcat Java process. Unfortunately it doesn't execute my commands. I've implemented the execution of shell commands in Java before. So my code should be correct:
Process pr = Runtime.getRuntime().exec("mount -o loop -t iso9660 <myimage> <mymountpoint>");
pr.waitFor();
<myimage> and <mymountpoint> are absolute paths, so no issues there either.
I've debugged my commands and they are working when executed on the console.
I've tried sending other commands. Simple commands such as id and pwd are working!
I've tried using /bin/bash -c "<command>", which didn't work.
I've tried executing a shell script, which executes the command, which didn't work.
I've tried escaping the spaces in my command, which didn't work.
So I've digged even deeper and now I'm suspecting some Tomcat security policy (Sandbox?), which prevents me from executing the command. Since security is no issue for me (it's an internal system, completely isolated from the outside world), I've tried a hack, which became quite popular just recently:
System.setSecurityManager(null);
This didn't work either. I'm using Java7 and Tomcat7 on RHEL6. Tomcat7 is just extracted! I don't have any files in /etc/.. or any other folder than /opt/tomcat/, where I've extracted the zip from the Tomcat home page. I've searched the /opt/tomcat/conf folder for security settings, but all I could find was the file catalina.policy, where it didn't seem like I could set some security level for shell commands.
Any ideas?

A few things:
System.setSecurityManager(null);
you have just killed the security of your application.
Yes, Tomcat is running as root. If I execute id I'm root as well.
Fix this immediately!
Now on to the question. You shouldnt have Tomcat executing anything, you need to defer this to a separate process whether that be a shell script or another Java program. This should also remove what (I hope) was a dependency on root running Tomcat. It should be possible to perform this command as a non-privileged user that cannot log into the system normally. You would do this by configuring /etc/fstab and supplying that same user the permissions to do this. From a pure security POV the process that mounts should not be owned by the tomcat user. Nor should the tomcat user ever be root. So to recap:
1) Stop running Tomcat as root
2) Create a separate process outside of the context of Tomcat to run this mount
3) Create a tomcat user, this user should not be able to log into the system nor should it be a privileged user (admin,super user, etc)
4) Create a process user, this user should be configured exactly as the tomcat user
5) Edit /etc/fstab giving the process user the necessary permissions to mount correctly.

It's generally a bad idea to use the single-string form of Runtime.exec. A better option is to use ProcessBuilder, and split up the arguments yourself rather than relying on Java to split them for you (which it does very naïvely).
ProcessBuilder pb = new ProcessBuilder("/bin/mount", "-o", "loop", /*...*/);
pb.redirectErrorStream(true); // equivalent of 2>&1
Process p = pb.start();
You say you're on RHEL so do you have selinux active? Check your logs and see if this is what's blocking you (I think it's audit.log you're looking for, it's been a few years since I've used selinux). If this does turn out to be the problem then you should probably ask on superuser or serverfault rather than SO...

I'm not sure if that's the problem you are having, but I've seen issues when Runtime.exec() is used without reading the associated output buffers. You can find a detailed explanation and potential solutions here. Reading the output and error streams can also help you figure out what's going on at the OS level when you run the command.

I've recently had to do something like this from a Swing app.
You'll probably be able to pull it off with ProcessBuilder, as in Ian's answer, but I found that once things start to get complex, it's easier to write a shell script that does what you want, enabling you to pass as few parameters as possible. Then use ProcessBuilder to invoke the shell script.
If you're invoking anything that has more than really minimal output, you'll also have to read the output and error streams to keep the process from blocking when the output buffers fill, as it seems you are already doing.

I use sudo -S before command and for the tomcat7 user: tomcat7 ALL=(ALL) NOPASSWD:ALL

How to Daemonize a Java Program?

I have a Java program that I'd like to daemonize on a linux system. In other words, I want to start running it in a shell and have it continue running after I've logged out. I also want to be able to stop the program cleanly.
I found this article which uses a combination of shell scripting and Java code to do the trick. It looks good, but I'd like something simpler, if possible.
What's your preferred method to daemonize a Java program on a Linux system?

Apache Commons Daemon will run your Java program as Linux daemon or WinNT Service.

If you can't rely on Java Service Wrapper cited elsewhere (for instance, if you are running on Ubuntu, which has no packaged version of SW) you probably want to do it the old fashioned way: have your program write its PID in /var/run/$progname.pid, and write a standard SysV init script (use for instance the one for ntpd as an example, it's simple) around it. Preferably, make it LSB-compliant, too.
Essentially, the start function tests if the program is already running (by testing if /var/run/$progname.pid exists, and the contents of that file is the PID of a running process), and if not run
logfile=/var/log/$progname.log
pidfile=/var/run/$progname.pid
nohup java -Dpidfile=$pidfile $jopts $mainClass </dev/null > $logfile 2>&1
The stop function checks on /var/run/$progname.pid, tests if that file is the PID of a running process, verifies that it is a Java VM (so as not to kill a process that simply reused the PID from a dead instance of my Java daemon) and then kills that process.
When called, my main() method will start by writing its PID in the file defined in System.getProperty("pidfile").
One major hurdle, though: in Java, there is no simple and standard way to get the PID of the process the JVM runs in.
Here is what I have come up with:
private static String getPid() {
File proc_self = new File("/proc/self");
if(proc_self.exists()) try {
return proc_self.getCanonicalFile().getName();
}
catch(Exception e) {
/// Continue on fall-back
}
File bash = new File("/bin/bash");
if(bash.exists()) {
ProcessBuilder pb = new ProcessBuilder("/bin/bash","-c","echo $PPID");
try {
Process p = pb.start();
BufferedReader rd = new BufferedReader(new InputStreamReader(p.getInputStream()));
return rd.readLine();
}
catch(IOException e) {
return String.valueOf(Thread.currentThread().getId());
}
}
// This is a cop-out to return something when we don't have BASH
return String.valueOf(Thread.currentThread().getId());
}

I frequently find myself writing scripts or command lines which essentially look like this, if I want to:
Run a program that is immune to sighups
That is completely disconnected from the shell which spawns it, and
Produces a log file from stderr and stdout the contents of which are displayed as well, but
Allows me to stop viewing the log in progress and do other stuff without disrupting the running process
Enjoy.
nohup java com.me.MyProgram </dev/null 2>&1 | tee logfile.log &

I prefer the nohup command. The blog post says there are better ways, but I don't think they're enough better.

You could try Java Service Wrapper, the community edition is free and meets your needs.

My preferred way on Ubuntu is to use the libslack 'daemon' utility. This is what Jenkins uses on Ubuntu (which is where I got the idea.) I've used it for my Jetty-based server applications and it works well.
When you stop the daemon process it will signal the JVM to shutdown. You can execute shutdown/cleanup code at this point by registering a shutdown hook with Runtime.addShutdownHook().

That depends. If it's just a one-time thing, I want to daemonize it and then go home, but usually I wait for the results, I might do:
nohup java com.me.MyProgram &
at the command line. To kill it cleanly, you have a lot of options. You might have a listener for SIGKILL, or listen on a port and shutdown when a connection is made, periodically check a file. Difference approaches have different weaknesses. If it's for use in production, I'd give it more thought, and probably throw a script into /etc/init.d that nohups it, and have a more sophisticated shutdown, such as what tomcat has.

DaemonTools :- A cleaner way to manage services at UNIX https://cr.yp.to/daemontools.html
Install daemon tools from the url https://cr.yp.to/daemontools/install.html
follow the instruction mentioned there,for any issues please try instructions https://gist.github.com/rizkyabdilah/8516303
Create a file at /etc/init/svscan.conf and add the below lines.(only required for cent-os-6.7)
start on runlevel [12345]
stop on runlevel [^12345]
respawn
exec /command/svscanboot
Create a new script named run inside /service/vm/ folder and add the below lines.
#!/bin/bash
echo starting VM
exec java -jar
/root/learning-/daemon-java/vm.jar
Note:
replace the Jar with your own Jar file. or any java class file.
Reboot the system
svstat /service/vm should be up and running now !.
svc -d /service/vm should bring vm down now !.
svc -u /service/vm should bring vm up now !.

This question is about daemonizing an arbitrary program (not java-specific) so some of the answers may apply to your case:

Take a look here:
http://jnicookbook.owsiak.org/recipe-no-022/
for a sample code that is based on JNI. In this case you daemonize the code that was started as Java and main loop is executed in C. But it is also possible to put main, daemon's, service loop inside Java.
https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo029
Have fun with JNI!

nohup java -jar {{your-jar.jar}} > /dev/null &
This may do the trick.