I would like to submit a job in the hpc and the job is running a java application.
I edit the pbs_script files as following:
#/bin/sh
#PBS -q serial
#PBS -l nodes=1:ppn=4
module load java-jdk/1.7.0-17
java myjavapp
I submitted the job
$qsub pbs_script
however the job return a error: could not find or load main class myjava. but I use the same command to run the java program under the command-line. what is the problem?
Problems such as this one are almost always a difference in the environment of where the job executes versus where you're executing it on the command line. To track it down you usually only need to check that everything is available on all of the nodes in the cluster and that the environment is configured such that the shell which runs the job will find what you're looking for.
Usually this is due not finding the class file. In PBS, the submit script starts execution from the user's home directory, not where you submitted the job. It is often useful to include the following line:
cd $PBS_O_WORKDIR
The command changes the directory on the execute node to the directory that the job was submitted from. Therefore, if you are able to run java myjavapp from the directory that you are submitting from (issuing the qsub), then the $PBS_O_WORKDIR line should work.
Your final submit file would look something like:
#/bin/sh
#PBS -q serial
#PBS -l nodes=1:ppn=4
module load java-jdk/1.7.0-17
cd $PBS_O_WORKDIR
java myjavapp
We face the same problem today.
We can run the java scripts on both master and nodes but when we submit the script through PBS, it fails. The error message is as the same as yours.
We fix the problem by giving the java program the classpath.
java -classpath myjava
Then, the qsub submitted java script can run on any node of the server. : )
Bo
Related
The following process normally works for my startup scripts. However, when I introduce a command to execute a JAR file, it does not work. This script works while I am logged in. However, it does not work as a startup script.
In /etc/init.d I create a bash script (test.sh) with the following contents:
#!/bin/bash
pw=$(curl http://169.254.169.254/latest/meta-data/instance-id)
pwh=$(/usr/bin/java -jar PWH.jar $pw &)
echo $pwh > test.txt
Make script executable
In /etc/rc.local, I add the following line:
sh /etc/init.d/test.sh
Notes:
I make a reference to the script in /etc/rc.local, because this script needs to run last after all services have started.
Please do not ask me to change the process (i.e., create script in /etc/init.d/ and reference it from /etc/rc.local), because it works for my other startup scripts.
I have tried adding nohup in front of java command, and it still did not work.
Thanks
As written, there is insufficient information to say what is going wrong. There are too many possibilities to enumerate.
Try running those commands one at a time in an interactive shell. The java command is probably writing something to standard error, and that will give you some clues.
I've been coding a litte bash script which connects on several distant servers, then execute a java CLI program through a few expect instructions.
It goes like this :
bash script
expect
ssh to server using public keys
expect ...
expect ...
log_file my_file (everything displayed on the screen is now redirected to my_file)
expect ...
log_file (closing my_file)
exit
exit
When I execute my script manually everything runs OK.
When I execute it through crontab, the file my_file is empty.
I found out that cron jobs don't have a tty attached and that PATH isn't the same as usually
My question is : is there a way to force the creation/allocation of a tty to my cronjob?
I've tried using the -t and -tt option with ssh but no result.
redirecting standard output on different levels of the script didn't work.
Also, I can't install screen (which could have helped, maybe) and "script" isn't writing anything either.
Thanks a bunch!
You can check the cron tab log for erros and make sure the full path is given for the command to be executed.
I'm running CentOS 6 and I have a jar file I'd like to execute with java and have it run in the background. I plan on executing this command from a script, and I also would like to specify the path at which the pid file is output. How can I do all this?
You can use Java Service(Commons Daemon) from apache. Instead of implementing main, your class to execute must provide method for the service execution (start, stop, etc.)
I have an Ant Task in the Jenkins Ant Execution Plugin, as a Post Build Step, to remotely run a shell script in one of our servers. The shell scripts starts a java process in the background. When I execute the shell script on the server directly it starts the java process in the back ground and comes out. When I run it from Jenkins via the sshexec task the shell script is run, but it never comes out and the Jenkins Build waits.
Later when I added the timeout attribute onto the sshexec it times out after the given number of milliseconds, but the Jenkins build is shown as failed. How do I make the sshexec task to come out cleanly from the shell script execution?
Here is my ssheexec task
<sshexec host="${deploy.host}" username="${deploy.username}" password="${deploy.password}" command=". /etc/profile; cd ${deploy.path}; sh start.sh i1" trust="true" timeout="10000" />
The start.sh file is as given:
nohup java -Xms512m -Xmx1024m -cp calculation.jar com.tes.StartCalculation $1 &
echo $! > calculation-$1-java.pid
It looks like, the ssh executed job is not fully daemonized. Starting with nohup is not sufficient in many cases.
See the discussion that related to it (in a different context)
The issue is that you are not closing your file descriptors when you
push something into the background. The & is fine when you are in a
shell, but is not enough when you want to disconnect and leave a
process running, you need the process to disconnect from the shell.
.... Fix to to correct the script.
If someone writes a naive service script that does not properly detach
from the terminal, I want to know the first time that that script is
used in a deployment - the SCM changes will enable the breaking change
to be quickly identified.
It is wrong to hide the problem to enable incorrect code to be
released to production - and I would not be happy if the first I knew
about it was when a production system administrator complained.
If this is the same problem, you need to daemonize the script
I am running a shell script through a web application. This shell script looks something like
`#! /bin/bash
user=""
pass=""
db_url=""
db_instance=""
sqlplus -s $user/$pass#$db_url/$db_instance # ./SqlScripts/foo.sql
sqlplus -s $user/$pass#$db_url/$db_instance # ./SqlScripts/bar.sql
CLASS_PATH="./lib/*"
java -classpath $CLASS_PATH package.Main ./Data/inputfile`
I am using ProcessBuilder to run the script and everything but the last line works fine. Am I creating a problem by calling shell through the jvm then calling the jvm again to run the application?
The problem was the environment that the script execution process was running in. I changed some of the environment variables of the process and everything is working fine now. The script was initially a standalone shell script, but I wrote one script for each of the databases used. In order to control the workflow I wrote a web application for this which calls seperate threads for each script and can manage the threads. Thanks for the responses!
Often, app servers run their servlets in a 'clean room' environment - e.g. they strip away all the variables that would normally be set from the outside for security reasons. Try using a fully qualified path to the java binary, and also try setting a full/absolute path for your CLASS_PATH variable.
The parent JVM and the child JVM should be separate processes, no particular reason why they should interfere.
What error do you get?
is java on your PATH?
OK, adding more questions in response to your comments ...
Which thread is waiting? Presumably the parent?
The child java process, do you have any evidence as to whether is succesfully initalises. My guess woukld be that the child is in some way blocked. If you kill the child does the parent then come back to life?
Suppose it was a simple "hello world" application, would that work?
Most likely the line:
CLASS_PATH="./lib/*"
And
$CLASS_PATH
It won't be expanded by the process builder because that's usually shells' job, which in this situation is not being invoked.
Try creating the complete list of ./lib/* and append it directly into the last line of your script.
java -classpath ./lib/a.jar:./lib/b.jar
Side note:
Invoking all this from java looks just bad to me. I would rather have it in a standalone script and invoke it by other means, but that's me.