We Keep Coding

Ansible terminates my app with "nohup"

I have a task in Ansible playbook that executes a .jar file in the background, but after finishing that task, the (java) app terminates.
- name: Run Java app in the background
shell: nohup java -jar app.jar &
I need the app running for tasks further down in the playbook. Any ideas??
NOTE: When I run it in Putty ssh session it runs smoothly and the app stays in background.

The most likely reason is attached IO. Try:
- name: Run Java app in the background
shell: nohup java -jar app.jar </dev/null >/dev/null 2>&1 &

Ok So I had this issue today and my script is a tad different but essentially doing the same thing so this is what I have, running on Oracle, Centos and RHEL.
in script start_service.sh I have this, this script is executed not sourced.
#!/bin/sh
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"
nohup java -jar ${DIR}/$1 --spring.config.location=${DIR}/application.yml --logging.config=${DIR}/logging.xml > ${DIR}/log/output.log 2>&1 &
In my playbook I have:
- name: run service
shell: "{{ service_install_location }}/application_name/start-service.sh application_name-{{ application_version }}.jar"
args:
chdir: "{{ service_install_location }}/application_name"
executable: /bin/bash
Now, the BASH_SOURCE argument is specific to bash so if you are running plain old sh then you need $0. nohup is required so when the ssh connection terminates your background task does not get killed (turn on -vvv for ansible to see connections being made to understand how this happens on a remote machine).
I should point out we are doing this temporarily as this should really be installed as a service to be under the control of service/init.d but for now, this answers your question

INFO - Started Jetty Server, but does not finish

I am trying to setup Dashbuilder. I have worked through a couple issues already (one with the help of others here, thank you). I am at the point where the program must be compiled, built, and run using command
./buildandrun.sh h2
things start off as expected, but then just stop at "[INFO] Started Jetty Server" in terminal.
I've let it sit for hours, no progress. I tried running with the -X flag, but no extra info for that step appeared. When I try to visit http://localhost:8080/dashbuilder to see if the program started I get a 503 error and see
HTTP ERROR: 503
Problem accessing /dashbuilder. Reason:
Service Unavailable
Powered by Jetty:// 9.3.9.v20160517
How can I see in better detail what and why something like this is happening?
Contents of the SH file are:
#!/bin/bash
if [ "$1" = "" ] ; then
echo "Build & Run the application for a given database."
echo ""
echo "USAGE: buildandrun.sh [h2|postgres]"
else
echo "-----------------------------------------------------------------"
echo "Building & Running the application for the '$1' database..."
echo "------------------------------------------------------------------"
cd ..
mvn clean install -P $1,jetty $2 $3 $4
export MAVEN_OPTS="-Xms1024M -Xmx2048M -XX:MaxPermSize=512m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
cd modules/dashboard-showcase/
mvn jetty:run -P $1,jetty
fi

The jetty:run goal of the jetty-maven-plugin does exactly that.
It runs Jetty, normally, and waits for the user to terminate it (either with TERM, KILL, Ctrl+C, use of the Shutdown facilities)
That mvn jetty:run command never returns.
Perhaps you wanted to use mvn jetty:start and mvn jetty:stop
See: https://www.eclipse.org/jetty/documentation/current/jetty-maven-plugin.html#jetty-start-goal

Unable to run nohup command from jenkins as a background process

UPDATE: Based on below discussion I have edited my answer for more accurate description.
I am trying to run a nohup command from jenkins. The full command is
nohup java -jar /home/.../jar/server-process-0.35.jar prod >> /var/../server-process-prod.log 2>&1 &
This command does not work. I can see status as success in jenkins but no java process in linux. When I do 'ps -ef | grep java'
However when I remove the last '&' , that is I change it from run in forground instead of background
It starts working. I can see the java process started.
The original command works fine If I run it on linux console.
I need to run it from jenkins in the original form that is as a backgorund process. So that it is independant of jenkins.
Any clues why is this happening?

Long story short, Jenkins kills all processes spawned by a job once that job finishes. To override this behavior, you need to set an environment variable.
The variable appears to vary from job type to job type. It used to be BUILD_ID, but for Pipeline jobs it is JENKINS_NODE_COOKIE, and there are several others mentioned in this answer.
So if you're running your command in Pipeline, it would look like this:
sh 'JENKINS_NODE_COOKIE=dontKillMe nohup java -jar /home/.../jar/server-process-0.35.jar prod >> /var/../server-process-prod.log 2>&1 &'
See the wiki on ProcessTreeKiller and this comment in the Jenkins Jira for more information.

In your jenkins shell script try:
export BUILD_ID=dontKillMe
nohup java -jar your_java_app.jar &
It worked for me!

I tried every possible combination with BUILD_ID but it didn't work.
I made it though by putting "nohup command > output.txt&" inside a shell script ran by the execute shell in jenkins, it worked perfectly!

Got the same problem, added:
BUILD_ID=dontKillMe python /var/lib/jenkins/release.py
into Execute Shell -> Command and inside release.py there is:
os.system('nohup java -jar ' + new_jars_on_server + '/' + generated_jar_by_mvn_name + '&')
and it works

Best simple solution is to use "at now" instead of "nohup"
In your job jenkins (execute shell) put :
set +e #so "at now" will run even if java -jar fails
#Run java app in background
echo "java -jar $(ls | grep *.jar | head -n 1)" | at now + 1 min

what worked for me was wrapping the nohup java -jar ... command into sh file inside execute shell command, and running that same sh file right after:
echo "starting java jar..."
cd [some location where jar is]
echo "nohup java -jar [jar_name].jar &" > start-jar-in-background.sh
sh start-jar-in-background.sh
echo "started java jar"
If I had nohup java -jar ... inline with Execute shell command, then it didn't start it from some reasons. I spent quite some time on this, hope it helps to someone ';)

Simplest way :
`nohup java -jar [jar_name].jar >log_file_you_want 2>another_file`&

set +e #so "at now" will run even if java -jar fails
#Run java app in background
echo "java -jar $(ls | grep *.jar | head -n 1)" | at now + 1 min
above command worked for him, thanks #walid, & remove at the end (+ 1 min)

Running a process in background, linux ubuntu

I want to run a .jar file so what I do is that I put "&" at the very end of the command (actually there's no need to log the output , I only want to be able to disconnect from the remote server which hosts my java program, the program itself saves the result after being finished)
I do as it follows , but it doesn't run in the background and it keeps me waiting :
java -Xmx72G -cp //home/arian/rapidminer/lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine -f //home/arian/RMRepository/testRemote.rmp &
Any idea that why it doesn't work ?
Thanks ,
Arian

I don't know why it wouldn't work. It really should, in most shells.
Anyway, if you intend to disconnect you'll usually find that just putting the job in the background is not enough: the disconnect will close the console (which will break many programs, alone) and send a SIGHUP signal (which will cause just about any program to exit).
You should considered using nohup to run the program (with the &). Alternatively, if you ever do need to come back and interact with the program later then screen or byobu might fit the bill better. Yet another alternative might be to add the task to your crontab.

What do you mean by "it keeps me waiting"? Does the RapidMinerCommandLineby any chance read from stdin or another stream?
If you want to run a process in the background and disconnect from the tty session you should use nohup, eg.:
nohup java -Xmx.... com.rapidminer.RapidMinerCommandLine &
(Do remember the & at the end!)
You may add ... 1> /dev/null before the & to disregard all stdout.
You could also consider the screen utility, which allows you to dis- and reconnect to the session, but that's more usable with (semi-)interactive sessions.
(Also, quite a hefty max heap size you're specifying?)
Cheers,

You can use JSVC, this is an utility interresting to daemonize Java applications
http://commons.apache.org/daemon/jsvc.html
It will give you a var pid file, useful to create a real start/stop script.
EDIT : Other solution, maybe could help
Here is a very old start/stop script I've done for Slackware Linux on embedded systems :
#!/bin/sh
application_start() {
cd /usr/local/YOURHOME
/usr/lib/java/bin/java \
-Xmx72G \
-classpath //home/arian/rapidminer/lib/rapidminer.jar \
com.rapidminer.RapidMinerCommandLine \
-f //home/arian/RMRepository/testRemote.rmp &
echo -n "Starting App daemon: $CMDLINE"
ps -Ao pid,command | grep java | grep com.rapidminer.RapidMinerCommandLine | awk '{print $1}' > /var/run/app.pid
echo
}
application_stop() {
echo -n "Stopping DataBaseSynchronizerClient daemon..."
kill `cat /var/run/DataBaseSynchronizerClient.pid`
echo
sleep 1
rm -f /var/run/DataBaseSynchronizerClient.pid
killall DataBaseSynchronizerClient 2> /dev/null
}
application_restart() {
application_stop
sleep 1
application_start
}
case "$1" in
'start')
application_start
;;
'stop')
application_stop
;;
'restart')
application_restart
;;
*)
echo "usage $0 start|stop|restart"
esac

I agree it should work as it is, but I had problems with Java running in background too. My solution was to use the screen utility (which normally is installed in most Linux distributions) where you can open a shell from which you can detach. If I remember well the commands are something like this (but there is a good manpage too)
screen -S myCustomName # runs a new shell called myCustomName
CTRL + D # detach from the current screen instance
screen -ls # list active screen instances
screen -r myCustomName # reattach to the screen instance.
Hope it will solve your problem.

Run a Java Application as a Service on Linux

I have written a Java server application that runs on a standard virtual hosted Linux solution. The application runs all the time listening for socket connections and creating new handlers for them. It is a server side implementation to a client-server application.
The way I start it is by including it in the start up rc.local script of the server. However once started I do not know how to access it to stop it and if I want to install an update, so I have to restart the server in order to restart the application.
On a windows PC, for this type of application I might create a windows service and then I can stop and start it as I want. Is there anything like that on a Linux box so that if I start this application I can stop it and restart it without doing a complete restart of the server.
My application is called WebServer.exe. It is started on server startup by including it in my rc.local as such:
java -jar /var/www/vhosts/myweb.com/phpserv/WebServer.jar &
I am a bit of a noob at Linux so any example would be appreciated with any posts. However I do have SSH, and full FTP access to the box to install any updates as well as access to a Plesk panel.

I wrote another simple wrapper here:
#!/bin/sh
SERVICE_NAME=MyService
PATH_TO_JAR=/usr/local/MyProject/MyJar.jar
PID_PATH_NAME=/tmp/MyService-pid
case $1 in
start)
echo "Starting $SERVICE_NAME ..."
if [ ! -f $PID_PATH_NAME ]; then
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is already running ..."
fi
;;
stop)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stoping ..."
kill $PID;
echo "$SERVICE_NAME stopped ..."
rm $PID_PATH_NAME
else
echo "$SERVICE_NAME is not running ..."
fi
;;
restart)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stopping ...";
kill $PID;
echo "$SERVICE_NAME stopped ...";
rm $PID_PATH_NAME
echo "$SERVICE_NAME starting ..."
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is not running ..."
fi
;;
esac
You can follow a full tutorial for init.d here and for systemd (ubuntu 16+) here
If you need the output log replace the 2
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
lines for
nohup java -jar $PATH_TO_JAR >> myService.out 2>&1&

A simple solution is to create a script start.sh that runs Java through nohup and then stores the PID to a file:
nohup java -jar myapplication.jar > log.txt 2> errors.txt < /dev/null &
PID=$!
echo $PID > pid.txt
Then your stop script stop.sh would read the PID from the file and kill the application:
PID=$(cat pid.txt)
kill $PID
Of course I've left out some details, like checking whether the process exists and removing pid.txt if you're done.

Linux service init script are stored into /etc/init.d. You can copy and customize /etc/init.d/skeleton file, and then call
service [yourservice] start|stop|restart
see http://www.ralfebert.de/blog/java/debian_daemon/. Its for Debian (so, Ubuntu as well) but fit more distribution.

Maybe not the best dev-ops solution, but good for the general use of a server for a lan party or similar.
Use screen to run your server in and then detach before logging out, this will keep the process running, you can then re-attach at any point.
Workflow:
Start a screen: screen
Start your server: java -jar minecraft-server.jar
Detach by pressing: Ctl-a, d
Re-attach: screen -r
More info here: https://www.gnu.org/software/screen/manual/screen.html

Another alternative, which is also quite popular is the Java Service Wrapper. This is also quite popular around the OSS community.

Referring to Spring Boot application as a Service as well, I would go for the systemd version, since it's the easiest, least verbose, and best integrated into modern distros (and even the not-so-modern ones like CentOS 7.x).

The easiest way is to use supervisord. Please see full details here: http://supervisord.org/
More info:
https://askubuntu.com/questions/779830/running-an-executable-jar-file-when-the-system-starts/852485#852485
https://www.digitalocean.com/community/tutorials/how-to-install-and-manage-supervisor-on-ubuntu-and-debian-vps

Here is a sample shell script (make sure you replace the MATH name with the name of the your application):
#!/bin/bash
### BEGIN INIT INFO
# Provides: MATH
# Required-Start: $java
# Required-Stop: $java
# Short-Description: Start and stop MATH service.
# Description: -
# Date-Creation: -
# Date-Last-Modification: -
# Author: -
### END INIT INFO
# Variables
PGREP=/usr/bin/pgrep
JAVA=/usr/bin/java
ZERO=0
# Start the MATH
start() {
echo "Starting MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "The service is already running"
else
#Run the jar file MATH service
$JAVA -jar /opt/MATH/MATH.jar > /dev/null 2>&1 &
#sleep time before the service verification
sleep 10
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Service was successfully started"
else
echo "Failed to start service"
fi
fi
echo
}
# Stop the MATH
stop() {
echo "Stopping MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
#Kill the pid of java with the service name
kill -9 $($PGREP -f MATH)
#Sleep time before the service verification
sleep 10
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Failed to stop service"
else
echo "Service was successfully stopped"
fi
else
echo "The service is already stopped"
fi
echo
}
# Verify the status of MATH
status() {
echo "Checking status of MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Service is running"
else
echo "Service is stopped"
fi
echo
}
# Main logic
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart|reload)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload}"
exit 1
esac
exit 0

From Spring Boot application as a Service, I can recommend the Python-based supervisord application. See that stack overflow question for more information. It's really straightforward to set up.

Other answers do a good job giving custom scripts and setups depending on your platform. In addition to those, here are the mature, special purpose programs that I know of:
JSW from TanukiSoftware
YAJSW is an open source clone from the above. It is written in Java, and it is a nanny process that manages the child process (your code) according to configurations. Works on windows / linux.
JSVC is a native application. Its also a nanny process, but it invokes your child application through the JNI, rather than as a subprocess.

You can use Thrift server or JMX to communicate with your Java service.

From Spring Boot Reference Guide
Installation as an init.d service (System V)
Simply symlink the jar to init.d to support the standard start, stop, restart and status commands.
Assuming that you have a Spring Boot application installed in /var/myapp, to install a Spring Boot application as an init.d service simply create a symlink:
$ sudo ln -s /var/myapp/myapp.jar /etc/init.d/myapp
Once installed, you can start and stop the service in the usual way. For example, on a Debian based system:
$ service myapp start
If your application fails to start, check the log file written to /var/log/<appname>.log for errors.
Continue reading to know how to secure a deployed service.
After doing as written I've discovered that my service fails to start with this error message in logs: start-stop-daemon: unrecognized option --no-close. And I've managed to fix it by creating a config file /var/myapp/myapp.conf with the following content
USE_START_STOP_DAEMON=false

It is possible to run the war as a Linux service, and you may want to force in your pom.xml file before packaging, as some distros may not recognize in auto mode. To do it, add the following property inside of spring-boot-maven-plugin plugin.
<embeddedLaunchScriptProperties>
<mode>service</mode>
</embeddedLaunchScriptProperties>
Next, setup your init.d with:
ln -s myapp.war /etc/init.d/myapp
and you will be able to run
service myapp start|stop|restart
There are many other options that you can find in Spring Boot documentation, including Windows service.

Im having Netty java application and I want to run it as a service with systemd. Unfortunately application stops no matter of what Type I'm using. At the end I've wrapped java start in screen. Here are the config files:
service
[Unit]
Description=Netty service
After=network.target
[Service]
User=user
Type=forking
WorkingDirectory=/home/user/app
ExecStart=/home/user/app/start.sh
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
start
#!/bin/sh
/usr/bin/screen -L -dmS netty_app java -cp app.jar classPath
from that point you can use systemctl [start|stop|status] service.

To run Java code as daemon (service) you can write JNI based stub.
http://jnicookbook.owsiak.org/recipe-no-022/
for a sample code that is based on JNI. In this case you daemonize the code that was started as Java and main loop is executed in C. But it is also possible to put main, daemon's, service loop inside Java.
https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo029
Have fun with JNI!

However once started I don't know how to access it to stop it
You can write a simple stop script that greps for your java process, extracts the PID and calls kill on it. It's not fancy, but it's straight forward.
Something like that may be of help as a start:
#!/bin/bash
PID = ps ax | grep "name of your app" | cut -d ' ' -f 1
kill $PID