We Keep Coding

Can't get Java app to listen on port for remote debugging purposes?

I have a Spring Boot Java app running on Ubuntu 14.x using Oracle Java 1.8.0 that I want to debug remotely with IntelliJ. I have tried to get it to listen on a port for debug purposes but with no success. Note, the ports I tried are all well above the port 1024, to make sure it's not a permission problem. I am not root but I do have sudo access to the box.
I tried adding this to the java command line:
-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y
A technique I got from this document:
http://javahowto.blogspot.com/2010/09/java-agentlibjdwp-for-attaching.html
However when I run this command:
sudo netstat -an | grep LISTEN
I don't see port 9009. Also, the app does not wait for debugger attachment as indicated by the "suspend=y" parameter, because I see the app initialization messages stream by as normal as the app starts up. Why isn't this working?
Here is the shell script that launches the app. Note, this shell script is launched by supervisord. I point this out in case that might be causing any trouble:
# !/bin/bash
# Shell script to launch Spring Boot app
# Kill subprocess when parent bash process is terminated by supervisor or when CTRL+C is received
trap 'kill -TERM $PID' TERM INT
java \
-Dnetworkaddress.cache.ttl=5 \
-Dnetworkaddress.cache.negative.ttl=5 \
\
-jar spbootapp.jar \
-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y
--spring.application.name=spbootapp-awsdev \
--spring.profiles.active=cluster \
--spring.cloud.config.enabled=false \
--endpoints.configprops.enabled=false \
--endpoints.health.sensitive=false \
&

The debug parameters -agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y need to go before the -jar in the command.

setenv.sh causing Tomcat7 to "fail" start

The problem is not as easy as it seems in the title.
I'm using a default Tomcat 7 package on Unbutu 14.04 LTS. When I have no "setenv.sh" in /usr/share/tomcat7/bin, it starts saying "OK" when I do :
$ sudo service tomcat7 start
* Starting Tomcat servlet engine tomcat7 [OK]
When I use the setenv.sh described below, it ALSO STARTS with no error in /var/lib/logs/catalina.out but the service is detected as "failed" when /etc/init.d/tomcat7 calls "start-stop-daemon --test" and concludes it's not running :
$ sudo service tomcat7 start
* Starting Tomcat servlet engine tomcat7 [fail]
What can I do about this ?
/usr/share/tomcat7/bin/setenv.sh :
#! /bin/sh
export JAVA_HOME="/home/linc/install/jdk1.7.0_75"
(...)
# Check for application specific parameters at startup
if [ -r "$CATALINA_BASE/bin/appenv.sh" ]; then
. "$CATALINA_BASE/bin/appenv.sh"
fi
There is an other problem, maybe related : when I check the process running after the start detected as "failed" (ps -ef | grep java), I can see all -D options added by setenv.sh, but I can't see the -D option added by "appenv.sh" (though setenv.sh and appenv.sh have exactly the same 755 rights).
Note : if I launch sudo /usr/share/tomcat7/bin/startup.sh, the setenv.sh doesn't cause any problem and appenv.sh is used.
EDIT : I may have found the cause but not the explanation : when I remove the declaration of JAVA_HOME, it uses the default jvm and the service start is detected as "OK", but when I specify the home of the default jvm , it fails again !
export JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre"
or :
export JAVA_HOME="/usr/lib/jvm/java-8-oracle"
What is happening here ?

Here are the explanations.
The command testing wether the service is up or not excpects not only a PID but also a precise java binary :
start-stop-daemon --test --start --pidfile "$CATALINA_PID" \
--user $TOMCAT7_USER --exec "$JAVA_HOME/bin/java" \
>/dev/null;
This is run 2 times, one before "catalina.sh" (and setenv.sh) is run, one after.
The "standard" tomcat conf on ubuntu works like this : /etc/init.d/tomcat7 can be overrided by /etc/default/tomcat7 who can be overrided by catalina.sh (+ setenv.sh + appenv.sh).
So there are run with 2 different JAVA_HOME, first one with the one in /etc/default/tomcat7 or some auto-detected one, and second one with the one set in setenv.sh. This makes the start-stop-daemon test fail.
The solution would be to set JAVA_HOME twice, one in /etc/default/tomcat7 for the service launch, and one in setenv.sh in case some direct launch (via startup.sh in shell) needs to be done for test purpose, with some comments warning about the duplication.
About appenv.sh, the reason is CATALINA_BASE == CATLINA_HOME only when you start Tomcat from command line (startup.sh). When running Tomcat as a service CATALINA_BASE = /var/lib/tomcat7 while $CATALINA_HOME = /usr/share/tomcat7.
So putting setenv.sh (and appenv.sh) in /var/lib/tomcat7/bin/ instead of /usr/share/tomcat7/bin solve the problem.

Using VisualVM on tomcat 8 running on EC2

I am trying to use VisualVM in my system to monitor a Tomcat instance running over EC2. I tried steps provided in multiple blogs about how to configure it, but still when I try to run tomcat it gives me following error.
./catalina.sh: 5: /home/gvr/apache-tomcat-8.0.18/bin/setenv.sh: -Dcom.sun.management.jmxremote: not found
I added following statement in server.xml
<listener classname="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
rmiregistryportplatform="10001"
rmiserverportplatform="10002"
uselocalports="true" />
And my setenv.sh is as follows
CATALINA_OPTS="-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.rmi.server.hostname=ec2-xx-xxx-xx-xx.ap-southeast-1.compute.amazonaws.com"
$CATALINA_OPTS
Besides this I have added, catalina-jmx-remote.jar in tomcat's lib directory
Could anyone please provide me some hint, what is possibly going wrong. I tried everything I have found related to configuring VisualVM
I am running Tomcat 8.0.18, java 8 over ubuntu

I believe there are two issues
formatting in setenv.sh, you need \ to split across lines
last line $CATALINA_OPTS which tries to execute the arguments, hence -Dcom.sun.management.jmxremote not found...
Suggested fix
CATALINA_OPTS="-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false \
-Djava.rmi.server.hostname=ec2-xx-xxx-xx-xx.ap-southeast-1.compute.amazonaws.com"
echo $CATALINA_OPTS

I also tried using the JmxRemoteLifecycleListener and it didn't work for me.
Here it is how I did it:
1)I choose 1005 as jmx port and my setenve.sh is like:
CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10005 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.local.only=false -Djava.rmi.server.hostname=localhost"
2) redirect using ssh the jmx port and the rmi
3) Run visualvm using the following uri: service:jmx:rmi:///jndi/rmi://localhost:10005/jmxrmi
If you need more information have a look to this post:
http://ignaciosuay.com/how-to-connect-a-java-profiler-like-visualvm-or-jconsole-to-a-remote-tomcat-running-on-amazon-ec2/

SSH command can not start remote java process

I have a start function, and I put it in a script resides on a remote site, the function's code shows below.
function start() {
cd $install_dir
mkdir -p logs
export classpath=$classpath:$target_jar
nohup java -Xms2048m -Xmx8192m -server -XX:PermSize=128m -XX:MaxPermSize=256m \
-XX:+PrintGCDetails -XX:+PrintGCDateStamps \
-XX:-OmitStackTraceInFastThrow \
-cp $target_jar $main_class >> logs/jvm.log 2>&1 &
echo "Service started, see logs"
}
And when I try to call that function use ssh ssh xxx#host "./service.sh start", I can not start the java process, I only got the response message "Service started, see logs" and there's no error, the jvm.log is also empty. It apparently to me that my script has executed, but the target java process didn't run.
If I logon to that remote site, and execute ./service.sh start, it works.

Since you were able to run the service manually, the ssh and script part is fine.
What could go wrong is the environment. For example you referred java without absolute path. Hence your may be running a different version of it. It is also possible that variable for shared library loading (LD_LIBRARY_PATH) are having different values.
Finally check for file permissins

Webapp in Tomcat 6 cannot find environment variables after start

I have the following problem:
I am using Tomcat 6.0.32 and Java JDK 6.0_26. I have installed it successfully and the Tomcat start page is visible in the browser at port 8080.
I have also created $CATALINA_HOME/setenv.sh script and put some webapp-specific environment variables in it (along with the CATALINA_HOME, JAVA_HOME and CLASSPATH).
I have created a new user "tomcat", set a new home directory for him, and also passwd-ed it.
This script is being sourced from within a init script I created to start and stop Tomcat automatically on reboot. I do not use the standart startup.sh and shutdown.sh found in $CATALINA_HOME, but rather then jsvc daemon starter, so I can use port 8080 from a non-root process (Tomcat itself).
The actual problem is that, after restarting Tomcat my webapp does not receive or see the environment variable I set in setenv.sh and so it won't start.
I have tried to put the environment variable definition in various places:
.bashrc in the tomcat home directory
/etc/init.d/tomcat script
$CATALINA_HOME/bin/setenv.sh
$CATALINA_HOME/webapps/myapp/META-INF/context.xml
to no avail, after start of Tomcat my webapp does not see the required environment variables.
My question is - what the heck am I doing worng? Any suggsetions? How am I supposed to transfer env vars to an webapp if the setenv.sh does not work? What could make this mechanism faulty (allegedly this is the way to hand env vars to webapps)?
Here is the startup script I worte:
#!/bin/sh
### BEGIN INIT INFO
# Provides: allfaweb
# Required-Start: $syslog $apache $apache2 $httpd
# Should-Start:
# Required-Stop: $syslog $apache $apache2 $httpd
# Should-Stop:
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: ALLFAweb service
### END INIT INFO
ALLFAWEB_BIN=/install/apache-tomcat-6.0.32-allfaweb/bin/jsvc
test -x $ALLFAWEB_BIN || { echo "$ALLFAWEB_BIN not installed";
if [ "$1" = "stop" ]; then exit 0;
else exit 5; fi; }
# Check for existence of setenv.sh file and read it
ALLFAWEB_CONFIG=/install/apache-tomcat-6.0.32-allfaweb/bin/setenv.sh
test -r $ALLFAWEB_CONFIG || { echo "$ALLFAWEB_CONFIG not existing";
if [ "$1" = "stop" ]; then exit 0;
else exit 6; fi; }
. /etc/rc.status
rc_reset
. /install/apache-tomcat-6.0.32-allfaweb/bin/setenv.sh;
case "$1" in
start)
echo -n "Starting ALLFAweb ";
$ALLFAWEB_BIN \
-user tomcat \
-home $JAVA_HOME \
-Dcatalina.home=$CATALINA_HOME \
-pidfile $ALLFAWEB_PID \
-outfile $CATALINA_HOME/logs/catalina.out \
-errfile $CATALINA_HOME/logs/catalina.err \
-cp $CLASSPATH org.apache.catalina.startup.Bootstrap
rc_status -v
;;
stop)
echo -n "Shutting down ALLFAweb "
$ALLFAWEB_BIN \
-stop \
-pidfile $ALLFAWEB_PID \
org.apache.catalina.startup.Bootstrap
rc_status -v
;;
restart)
$0 stop
$0 start
rc_status
;;
status)
echo -n "Checking for service ALLFAweb ";
/sbin/checkproc $ALLFAWEB_BIN
rc_status -v
;;
*)
echo "Usage: $0 {start|stop|status|restart}"
exit 1
;;
esac
rc_exit
The system I am using is a SUSE SP2:
# uname -a
Linux testmachine 3.0.51-0.7.9-default #1 SMP Thu Nov 29 22:12:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Any help would be highly appreciated! Thanks in advance :)

You should be using system properties and not environment variables. Check the source for Tomcat's bin/daemon.sh script, which passes all of the standard variables to Tomcat when it is launched.

I found it. Allegedly the reason is that at first I started Tomcat 6 as root, and then changed the ownership of files to user "tomcat". The root process had written files all over the file system wiht the appropriate permissions, that is, the "tomcat" user could not read them.
The default shell of the "tomct" user was /bin/sh and not /bin/bash
I also deleted the files in $CATALINA_HOME/work and renamed the directory to which my custom env var was pointing to (and the env var accordingly).
There were also some logfiles generated from log4j in the / dicrectory, those are gone too and now everything works as expected :)
Thanks for all the feedback, you kept me going during these 3 days of frustration :)
Cheers!