This question already has answers here:
How to run Unix shell script from Java code?
(17 answers)
Closed 7 years ago.
I'm trying to execute a shell script from Java.The script is supposed to download the file from the URL using wget.Here goes my code.
public class RunShellScriptFromJava {
public static void main(String a[]) {
try {
ProcessBuilder pb = new ProcessBuilder("/bin/sh","script.sh");
Process p = pb.start();
p.waitFor();
System.out.println("Success");
} catch (Exception e) {
e.printStackTrace();
}
}
}
Content of script.sh
echo "start"
wget http://alex.smola.org/drafts/thebook.pdf
echo "end"
My Question:
Is this the right way doing it?If not please point me in the right direction.It doesn't throw any exception but I see that the file is not getting downloaded.Any lead/help is appreciated.
Thanks.
PS:I have given execute permission for the script
The code sample in the question lacks error handling around the usage of ProcessBuilder, so it's likely that some kind of error happened, but you aren't getting visibility into it.
The return code of Process#waitFor is being ignored. The returned value is the exit code of the spawned process. I recommend checking to see if this value is non-zero.
There is also no handling of stdout or stderr, so you won't know if the spawned process is writing any output that explains what happened. You can access stdout and stderr by using Process#getInputStream and Process#getErrorStream respectively.
Note also that it is possible for your process to hang if it fails to fully consume the streams, or at least redirect them. I've noticed this problem is particularly common on Windows. A standard technique is to spawn background threads to consume the streams. This previous question discusses that technique and others.
Java ProcessBuilder: Resultant Process Hangs
After the error handling in the Java code is enhanced like this, I expect you'll have a better chance of diagnosing what went wrong with the script.
The right way to do this is to use Java to download the file instead of the shell script. The problem with shell scripts is that they are system dependent. By using them you lose one of the main benefits of java which is system independence. There are a number of libraries in Java that will accomplish that functionality. The following will work for you using the FileUtils class from apache IO Commons.
URL url = new URL("http://alex.smola.org/drafts/thebook.pdf");
File download = new File('.');
FileUtils.copyURLToFile(url, download);
This script and java example works, perhaps specify the full path to wget to ensure you know where the pdf is being saved.
$ javac RunShellScriptFromJava.java
$ java RunShellScriptFromJava
Success
$ ls
RunShellScriptFromJava.java thebook.pdf RunShellScriptFromJava.class script.sh
Example/updated script:
wget -O /home/MYSER/test.pdf http://alex.smola.org/drafts/thebook.pdf
Related
I'm trying to run a Python script from a Java program using Process and ProcessBuilder, however Java keeps using the wrong version of Python. (The script needs 3.6.3 to run and Java runs Python 2.7)
However when I run the script from the terminal (outside of Java), it runs the correct Python (3.6.3). How does one change what version of Python gets run when called by Java?
The short version is it changes with your PATH environment variable.
Under Windows, Technet has the answer. Scroll down to the 'Command Search Sequence' section. This answer explains it nicely.
For UNIX-like OS's, this answer is nicely detailed.
There are two very useful commands for determining which executable is going to be called: which for UNIX-likes and where for newer Windows.
The most likely reason for the difference between Java and the terminal is a difference in your PATH. Perhaps your Java version is being run with a modified PATH? A launch script of some kind may be changing it.
Add /usr/bin/python3.4 to the start of your command to force the version of python you want. If you're not sure where python is installed, have a go at using whereis python and seeing what you get back.
private int executeScript(final List<String> command) {
try {
final ProcessBuilder processBuilder = new ProcessBuilder("/usr/bin/python3.4").command(command);
processBuilder.redirectErrorStream(true);
System.out.println("executing: " + processBuilder.command().toString());
final Process process = processBuilder.start();
final InputStream inputStream = process.getInputStream();
final InputStream errorStream = process.getErrorStream();
readStream(inputStream);
readStream(errorStream);
return process.waitFor();
} catch (Exception e) {
System.out.println(e.getMessage());
}
return -1;
}
Then just pass in the List containing your python commands.
I am trying to make my Java program interact with Linux bash but something goes wrong. I have a simple executable prog that reads the one integer from stdin and outputs its square. Executing
echo 5 | ./prog
from bash itself prints correct answer 25 in stdout but running
import java.io.*;
public class Main {
public static void main(String[] args) throws InterruptedException, IOException {
Runtime run = Runtime.getRuntime();
Process proc = run.exec("echo 5 | ./prog");
proc.waitFor();
BufferedReader br = new BufferedReader(new InputStreamReader(proc.getInputStream()));
while(br.ready())
System.out.println(br.readLine());
}
}
unexpectedly gives 5 | ./prog. What is the solution?
Java exec cannot run shell commands like that. If you want to run a shell command, you need to explicitly invoke the shell; e.g.
Process proc = run.exec(new String[]{"/bin/sh", "-c", "echo 5 | ./prog"});
For more details on what Java does with this, read the javadocs for exec(String) and exec(String[]). Note that these are "convenience methods", and you need to follow the chain of links to the underlying methods for a complete understanding of what the javadoc is saying.
If you want even more detail on how Java handles this, there is the source code ...
If you want to understand in depth why Java doesn't handle the shell syntax itself, you probably need to take a deep dive into the architecture and philosophy of UNIX / Linux systems, and the separation of concerns between application, operating system and command shell. Note that there are a myriad different shells, each with (potentially) different rules for quoting, argument splitting, redirection, pipes, etcetera. Most of the popular shells are similar, but that's just the way things panned out.
Explanation of the solution:
The reason for splitting the command by hand is that exec(String) won't split the command into a single command and arguments correctly. It can't. This is an example where there are two commands in a pipeline.
The reason for using "sh" is ... well ... you need a shell to parse and process a shell command line. Java's exec command does not support shell syntax ... as the javadoc explains.
The purpose of the "-c" option is explained by "man sh". Basically, sh -c "a b c" means "use 'sh' to run the command line 'a b c'".
FWIW, it is technically possible to construct and run a pipeline solely in Java (i.e. without relying on an external shell), but it is generally not worth the effort. You've already introduced a platform dependency by running external commands, so an extra dependency in the form of a specific shell command and syntax doesn't make things significantly worse.
I am trying to write a simple application that takes in a command line arguement (which will be a Powershell ps1 file) and then run it. So I have experemented with a number of different approaches and seem to be running into a problem. If I attempt to invoke powershell from within java, the windows process is started and is visible via process explorer, however powershell never returns, it hangs in some sort of loop by the looks of it. The command I am using is:
String command = "powershell -noprofile -noninteractive \"&C:\\new\\tst.ps1\"";
The command is then executed using:
Runtime systemRuntime = Runtime.getRuntime();
Process proc = systemRuntime.exec(command);
At the moment I am hard coding the location to the ps1 file as I was trying to rule this out as an issue. Using a process explorer I can see the hanging powershell process and the command that was passed to it was :
powershell -noprofile -noninteractive "&C:\new\tst.ps1"
which when copied into a cmd window, works to launch the tst.ps1 file. The file itself is incredibly simple in this example and I think I can rule it out being the cause of the freeze as I have tried to launch other ps1 files the same behaviour can be seen.
To further add to the confusion, if I use the java code posted above and pass in powershell commands instead of a file name then it successfully runs.
I've scoured the web and see lots of people experiencing the same issue but no one seems to have posted there solution, I hope its a simple oversight on my part and can be easily fixed.
Any hints/tips are appreciated :D
Alan
You have to close OutputStream in order for Powershell to exit.
Runtime systemRuntime = Runtime.getRuntime();
Process proc = systemRuntime.exec(command);
proc.getOutputStream().close();
Is your external program writing to the standard outputs (err and out)?
If yes, it can hang waiting for you to consume them from the java parent process.
You can get those as InputStreams by calling
Process.getInputStream()
and
Process.getErrorStream()
There's more details here:
Javadoc for Process
I'm using Eclipse to develop a Java program, and figured I'd add an option to my program to parse stdin if there are no arguments. (otherwise it parses a file)
I am having problems if I execute "somecommand | java -jar myjar.jar" and went to debug... then realized I don't know how to start a process in Eclipse like that. And if I run it on the command prompt, I can't attach to a running process since the process starts immediately.
Any suggestions on how to debug?
edit: see, the thing is, I wrote my program originally to take a filename argument. Then I figured it would be useful for it to take stdin as well, so I did abstract InputStream out of my program (as Mr. Queue suggests). It works fine operating on a file (java -jar myjar.jar myfile), but not operating when I run type myfile | java -jar myjar.jar. I suspect that there's something different in the two scenarios (eof detection is different?) but I really would like to debug.
// overall program structure follows:
public static void doit(InputStream is)
{
...
}
public static void main(String[] args)
{
if (args.length > 0)
{
// this leaves out the try-catch-finally block,
// but you get the idea.
FileInputStream fis = new FileInputStream(args[0]);
doit(fis);
fis.close();
}
else
{
doit(System.in);
}
}
Run your app, with the pipe, on the command line but add JVM args for remote debugging, like this:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1044
suspend=y will tell the JVM to not actually run the program until the debugger is attached.
Next, go into the Eclipse debug launch configurations (Run -> Debug Configurations...) and create a "Remote Java Application" to connect to your app. Run the launch in Eclipse (after setting some breakpoints) and you should be able to debug. Not terribly convenient, but if you can't reproduce your issues without the pipe, this is an option.
If I'm interpreting your question right, I believe you just want to know how to send input across standard in and debug through it in eclipse.
If it's simple input, you can actually manually enter System.in data via the eclipse Console window while the program is running. Just start typing in the console window, and press enter to send the text to Standard in.
If it's something more complicated, I'd suggest abstracting the read you're trying to do to take an InputStream. In your program, you can send System.in as the InputStream. To Debug, you can send any other InputStream. For example, you could put your input in a file, and pass a FileInputStream to the method to test it.
EDIT:
Without seeing some more code, I'm not sure, but you might be on to something with eof detection. A FileInputStream has a defined end of file, but I'd guess System.in has nothing of the sort. Your reader might just be waiting to read the next character and never advancing. You might have to manually stop the read after you know you've read "enough".
Perhaps this solution of creating a named pipe might apply here.
mkfifo foo
somecommand > foo
Next in the debug configuration, add < foo in the args, so that your program is debugged as:
java -jar myjar.jar < foo
I'm late to the party. But you can look for the "common" tab in the debug configuration for your project. There, you should be able to see a section labeled "Standard Input and Output". You can allocate the console with input files and set output files this way.
I have a Java program that I'd like to daemonize on a linux system. In other words, I want to start running it in a shell and have it continue running after I've logged out. I also want to be able to stop the program cleanly.
I found this article which uses a combination of shell scripting and Java code to do the trick. It looks good, but I'd like something simpler, if possible.
What's your preferred method to daemonize a Java program on a Linux system?
Apache Commons Daemon will run your Java program as Linux daemon or WinNT Service.
If you can't rely on Java Service Wrapper cited elsewhere (for instance, if you are running on Ubuntu, which has no packaged version of SW) you probably want to do it the old fashioned way: have your program write its PID in /var/run/$progname.pid, and write a standard SysV init script (use for instance the one for ntpd as an example, it's simple) around it. Preferably, make it LSB-compliant, too.
Essentially, the start function tests if the program is already running (by testing if /var/run/$progname.pid exists, and the contents of that file is the PID of a running process), and if not run
logfile=/var/log/$progname.log
pidfile=/var/run/$progname.pid
nohup java -Dpidfile=$pidfile $jopts $mainClass </dev/null > $logfile 2>&1
The stop function checks on /var/run/$progname.pid, tests if that file is the PID of a running process, verifies that it is a Java VM (so as not to kill a process that simply reused the PID from a dead instance of my Java daemon) and then kills that process.
When called, my main() method will start by writing its PID in the file defined in System.getProperty("pidfile").
One major hurdle, though: in Java, there is no simple and standard way to get the PID of the process the JVM runs in.
Here is what I have come up with:
private static String getPid() {
File proc_self = new File("/proc/self");
if(proc_self.exists()) try {
return proc_self.getCanonicalFile().getName();
}
catch(Exception e) {
/// Continue on fall-back
}
File bash = new File("/bin/bash");
if(bash.exists()) {
ProcessBuilder pb = new ProcessBuilder("/bin/bash","-c","echo $PPID");
try {
Process p = pb.start();
BufferedReader rd = new BufferedReader(new InputStreamReader(p.getInputStream()));
return rd.readLine();
}
catch(IOException e) {
return String.valueOf(Thread.currentThread().getId());
}
}
// This is a cop-out to return something when we don't have BASH
return String.valueOf(Thread.currentThread().getId());
}
I frequently find myself writing scripts or command lines which essentially look like this, if I want to:
Run a program that is immune to sighups
That is completely disconnected from the shell which spawns it, and
Produces a log file from stderr and stdout the contents of which are displayed as well, but
Allows me to stop viewing the log in progress and do other stuff without disrupting the running process
Enjoy.
nohup java com.me.MyProgram </dev/null 2>&1 | tee logfile.log &
I prefer the nohup command. The blog post says there are better ways, but I don't think they're enough better.
You could try Java Service Wrapper, the community edition is free and meets your needs.
My preferred way on Ubuntu is to use the libslack 'daemon' utility. This is what Jenkins uses on Ubuntu (which is where I got the idea.) I've used it for my Jetty-based server applications and it works well.
When you stop the daemon process it will signal the JVM to shutdown. You can execute shutdown/cleanup code at this point by registering a shutdown hook with Runtime.addShutdownHook().
That depends. If it's just a one-time thing, I want to daemonize it and then go home, but usually I wait for the results, I might do:
nohup java com.me.MyProgram &
at the command line. To kill it cleanly, you have a lot of options. You might have a listener for SIGKILL, or listen on a port and shutdown when a connection is made, periodically check a file. Difference approaches have different weaknesses. If it's for use in production, I'd give it more thought, and probably throw a script into /etc/init.d that nohups it, and have a more sophisticated shutdown, such as what tomcat has.
DaemonTools :- A cleaner way to manage services at UNIX https://cr.yp.to/daemontools.html
Install daemon tools from the url https://cr.yp.to/daemontools/install.html
follow the instruction mentioned there,for any issues please try instructions https://gist.github.com/rizkyabdilah/8516303
Create a file at /etc/init/svscan.conf and add the below lines.(only required for cent-os-6.7)
start on runlevel [12345]
stop on runlevel [^12345]
respawn
exec /command/svscanboot
Create a new script named run inside /service/vm/ folder and add the below lines.
#!/bin/bash
echo starting VM
exec java -jar
/root/learning-/daemon-java/vm.jar
Note:
replace the Jar with your own Jar file. or any java class file.
Reboot the system
svstat /service/vm should be up and running now !.
svc -d /service/vm should bring vm down now !.
svc -u /service/vm should bring vm up now !.
This question is about daemonizing an arbitrary program (not java-specific) so some of the answers may apply to your case:
Take a look here:
http://jnicookbook.owsiak.org/recipe-no-022/
for a sample code that is based on JNI. In this case you daemonize the code that was started as Java and main loop is executed in C. But it is also possible to put main, daemon's, service loop inside Java.
https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo029
Have fun with JNI!
nohup java -jar {{your-jar.jar}} > /dev/null &
This may do the trick.