I tried a couple of options
a. cf java thread-dump myapp -i0 (Only 1 instance is there so 0)
Response is nothing.
logs of myapp show
2019-12-13T14:52:41.15+0800 [SSH/0] OUT Successful remote access by 10.x.x.x:35764
2019-12-13T14:52:41.84+0800 [SSH/0] OUT Remote access ended for 10.x.x.x:35764
b. I did cf ssh myapp
/home/vcap/app/.java-buildpack/open_jdk_jre/bin/ has no jstack and jmap command doesn't work either
/home/vcap/app/.java-buildpack/open_jdk_jre/bin/ has the following components
java orbd servertool java-buildpack-memory-calculator-3.13.0_RELEASE pack200 tnameserv
jjs policytool unpack200
jvmkill-1.16.0_RELEASE rmid
keytool rmiregistry
Question is-> How to get the java threaddump?
I believe the easiest option is to run cf logs to watch your app logs. Then in a second terminal, cf ssh to the app and run ps aux and look for the process id of the Java process. Then run kill -3 <pid>. Your Java process will catch this signal and issue a thread dump. It will go to STDOUT, which conveniently goes to your app's log stream so it should show up in the first terminal.
The crummy part about this is that cf logs will inject some additional text at the beginning of every line, which makes it hard to load the thread dump into tools. You can usually strip that off with cut or awk, or you can redirect cf logs to a file, open with a text editor and find/replace the leading text.
Besides that you have a couple other options:
If you're using Spring, enable Spring Boot Actuators. You can then use the /actuator/threaddump endpoint to generate and download a thread dump (note the format for this is JSON, not the standard format)
Use an APM tool or profiler like NewRelic, AppDynamics, Dynatrace or YourKit. These are well supported by the Java buildpack and most will just work.
Use JMX. It's easy enough to enable with the Java buildpack, just set the env variable JBP_CONFIG_JMX to '{enabled: true}' and restage your app. Once enabled, you can open a tunnel with cf ssh -N -T -L 5000:localhost:5000 <APP_NAME>. Then open jvisualvm, make a new "local" JMX connection to "localhost:5000" and connect. Click the "Threads" tab and click the "Thread Dump" button. It might be a little slow as it's going across the tunnel, but give it a few seconds to finish and you should have a thread dump.
What won't work on Cloud Foundry are jstack and jcmd. The Java buildpack, at the time of writing, installs a JRE and these tools are no longer shipped with the OpenJDK JRE and for legal reasons can't be bundled with it. If you were inclined to do so, you could scp the tools and required shared libraries from a JDK up to your app container and run them, but it's a lot of work and not fun. I'm not super familiar with the cf java plugin, but I suspect it tries to use these tools, which used to be available, and no longer are there.
You should be able to get ThreadDumps with kill -3 <PID> and open that in heap analyzer such as eclipse memory analyzer (MAT) and select Thread Stacks
Related
I have a custom instance running on Windows Server 2016 where I need to monitor my Java processes.
I setup Stackdriver, installed the monitoring agent and followed this guide for JVM monitoring https://cloud.google.com/monitoring/agent/plugins/jvm
However no matter what I do I can not see any of my processes when setting up an Alerting policy within Stackdriver's console.
There are 2 steps in the guide I am unsure how to follow:
On your VM instance, download jvm-sun-hotspot.conf from the GitHub configuration repository and place it in the directory
/opt/stackdriver/collectd/etc/collectd.d/
Where exactly do I place the .conf file?
Edit the downloaded configuration file and replace JMX_PORT by the port on which your JVM is configured to allow JMX connections.
How do I get the port?
Sorry if this seems obvious but I'm pretty new to this.
Thanks in advance for your help.
All the steps and instructions on the JVM plugin page have instructions relevant for linux, and windows is not even mentioned there. So it could be quite possible that the JVM plugin is supported only on linux.
Since your code is already running on a JVM, you might want to consider if it is possible to move your application from a Windows VM to a linux VM.
Enabling the JVM monitoring plugin
Java Virtual Machines are monitored via JMX.
On your VM instance, download jvm-sun-hotspot.conf from the GitHub
configuration repository and place it in the directory
/opt/stackdriver/collectd/etc/collectd.d/:
(cd /opt/stackdriver/collectd/etc/collectd.d/ && curl -O https://raw.githubusercontent.com/Stackdriver/stackdriver-agent-service-configs/master/etc/collectd.d/jvm-sun-hotspot.conf)
Edit the downloaded configuration file and replace JMX_PORT by the
port on which your JVM is configured to allow JMX connections.
After adding the configuration file, restart the monitoring agent by
running the following command:
sudo service stackdriver-agent restart
Information on other plugin configuration options can be found at
collectd.org.
I have an Azure App Service running Tomcat 8.5 and can't seem to figure out how to do a thread dump. jstack is not recognized in the Console nor the DebugConsole.
On Azure App Service, all popular versions of Java had been installed at the path D:\Program Files (x86)\Java. Take version 1.8.0u73 as example, please see the figure below.
You can command set PATH=D:\Program Files (x86)\Java\jdk1.8.0_73\bin;%PATH% to add Java tools to the environment temporarily via the Kudu console tool. Then, the jstack tool is available in the current Kudu session.
Inspired by the previous answer, I created the following bat script (could be powershell, I suppose) that does the job:
powershell -Command "get-process java |select -expand id" > pid.txt
set /p id= < pid.txt
"%JAVA_HOME%\bin\jstack" -F %id% > out.txt
It uses the java version that you're using in your app (JAVA_HOME) and doesn't require any input from you. It's simple and does the trick. Just create a bat file in your wwwroot, for example, and run it whenever you want.
I know this is an old question but for Windows Apps on Azure App Service, this can be easily done by going to Azure Portal -> Diagnose and Solve problems blade -> Diagnostic Tools and then choosing Collect Java Thread dump from the left menu or the middle pane.
Get the PID of java process from Process Explorer of Kudu (Advanced Tools)
Use this command to collect thread dump:
jcmd Thread.print > D:\home\threaddump.txt
Is there away to take a thread dump from Tomcat? I would like to monitor the threads running in Tomcat at a certain point in time.
Note: I was doing that on Web logic but i don't know how it can be done on Tomcat.
If you use Linux, you can send a kill -3 [pid of your tomcat] and it will dump all current threads in the catalina.out.
There is a simple way to monitor tomcat threads and do a dump.
Start tomcat with the folowing java options :
-Dcom.sun.management.jmxremote.port=<some free port>
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
After the tomcat is restarted you can simply start jconsole (part of jdk) or visualvm and create a new jmx connection to the port you have chosen above.
In both tools you also have an option to take a dump ;)
!!!But do that only in closed/secured environment - since there is no authentication set.!!!
add java melody to your server
https://code.google.com/p/javamelody/
it give you a lot of info including threads in runtime
The following steps should help you get the current Java threads running and analyze them:
Download IBM JCA which is a nice thread dump analyzer from IBM Thread & Dump Monitor Analyzer
Execute the following to get the Tomcat threads dump: root#localhost:~# sudo -u $TOMCAT_USER $JAVA_HOME/bin/jstack -J-d64 -l $(ps aux | grep '[c]atalina' | awk '{print $2}') > ~/threads.log
Open the threads dump with IBM Thread & Dump Monitor Analyzer
Note: Replace $JAVA_HOME with your path to your current JDK, $TOMCAT_USER with the user running Tomcat; if you pass the wrong user or even using root you will get an error like this:
3047: well-known file is not secure
Please try netbeans profiler to profile any web applications.
https://profiler.netbeans.org/
http://wiki.netbeans.org/FaqProfilerAttachRemoteServer
Have you had experience with running a jar file using a command line, wrapped in a Windows service?
I'm trying to find a way to run a jar file without being logged into the machine, and since it allows command shell, I was wondering if it's a good idea.
Thanks!
Original Post:
I'm trying to run Associated Press's Web Feeds Manager, which is basically a jar file that can be run when logged in by double clicking it.
I'd like to run the same file but without being logged in to the machine. In their manual (http://wfm.ap.org/admin/content/help/Running_Agent_on_a_Remote_Server.htm) they write how to do that, using a commandline parameter.
Basically I'd like the jar to run as a Windows service, regardless of who's logged in, but Googling it showed it was problematic.
Have you had experience with remotely running jar files? What are the pitfalls?
Thanks!
On a google search, I came across this article -
Running Jar Applications as a Windows Service
It mentions about open source Java Service Wrapper project from Tanukisoftware.org for accomplishing this task.
Note: I've not used this personally.
If you are not interested in having the service started/stopped at boot/shutdown, but you just want the program to be started manually and keep running after logout, here is what you do:
$ nohup java -jar foobar.jar > foobar.log 2>&1 &
which means: start my foobar.jar (java -jar) and keep it running after I logout (nohup) redirect stdout to foobar.log (>) and also the stderr (2>&1), and make it running in background (& at the end).
Instead, if you are interested in installing a "service" in your linux box, there are many options, depending on what distribution you are using.
The most common are upstart (for ubuntu) and System V init scripts (Redhat or others). Also cron can be used to start/stop services at startup/shutdown.
You can find an example of installing a java app (hudson) on an init system here, or doing the same thing with upstart. Or, as I said, cron could be an option.
On Windows, there is Java Service Wrapper. And not much more.
For windows Java Service Wrapper is a better choice
My favourite is the upstart on linux, but it is Ubuntu only.
On Windows I see many alternatives according to this forum.
How do I run a Java .jar file as a Windows service on a Windows 2008 server? I have a jar file called SomeJavaFile.jar located under the C:\SomeDirectory directory on a Windows Server 2008 box. I usually just run this jar manually in the command line with: java –cp SomeJavaFile.jar com.ctg.SomeJavaFile and I let it run 24/7.
The problem with our Windows Server is if it restarts I need to run it as a service so it will automatically start as a service upon startup, because our processing for Vistakon runs 24/7. I have Googled around and tried to use a service wrapper and the sc.exe command to create the service.
I put the wrapper service.exe in the C:\SomeDirectory\. I use the following command to create it: sc.exe SomeJavaService binPath= “C:\SomeDirectory\service.exe \”java –jar C:\SomeDirectory\SomeJavaFile.jar\”” type= own start= auto error= ignore. This creates the SomeJavaService service correctly but when I try to start it I get an error that says the service on Local Computer started then stopped.
Some services stop automatically if they are not in use by other services or programs. Do I need to alter my sc.exe command to have the exact previous working command line maybe, by adding in the com.ctg.SomeJavaFile? So should I change This jar should just run in the background and constantly poll/ping the C:/poll directory then if there is data present it processes the data and sends an export file to another directory.
I have used this same .jar file for years successfully and it hasn't changed, but I cannot get it to run as a Windows service. This is the site I use to get the service wrapper http://code.google.com/p/simple-service-wrapper/. Any help would be greatly appreciated!
"winsw" is the standalone version of the Windows Service installer shipping with the Glassfish Java EE reference implementation.
Works well, but is not a fully polished product - I have used it for exactly this purpose for a couple of years. Requires .NET in some recent version on the Windows machine.
https://github.com/kohsuke/winsw
I think that the best bet would be wrap your java app with Procrun of Apache Commons Daemon .
Procrun is a set of applications that allow Windows users to wrap
(mostly) Java applications (e.g. Tomcat) as a Windows service.
The service can be set to automatically start when the machine boots
and will continue to run with no user logged onto the machine.
As per my analysis,
The Idle Solution will be writing a VC++ (.net) Windows Service creation program to launch the .bat (that triggers the jar file)/.exe as a System service with all the required call back methods to SCM.
Note : 1. Wrapping the process with sc.exe / srvany.exe would not work as it does not have any call back process to the SCM (Service Control Manager). 2. And java service Wrapper is a third party API (many are LGPL licensed).
If you start your Java code from commandline by using java -j some.jar does it run until you terminate the program, or does it stop by itself?
There needs to be a NON-Deamon Thread, that is running all the time. A JVM will terminate, if there is no thread running, that is not flagged as daemon.
If you have a little budget, buy an installer tool. I use instll4j. With that tool, you can create service launcher and install them during instllation.
The following solution is working fine for me having OpenFire Connection Manager (which is a jar file) running as service on Windows Server.
Download and install the Non-Sucking Service Manager
I didn't use the batch made by OpenFire team, because it didn't work for me (dependencies not found error...) So, make a batch file with the following code :
#ECHO OFF
cd /D "E:\connection_manager\lib"
java -jar startup.jar
and save it as cm_startup.bat in the bin folder of connection manager.
After that you can create the service with NSSM.
So, open a cmd.exe and run the following command :
nssm install ConnManager "E:\connection_manager\lib\cm_startup.bat"
.
Doc & examples
More documentation and examples for the Non-Sucking Service Manager here : https://nssm.cc/usage Actually NSSM as a lot of options available.
Here is a more complexe example :
nssm install solr "%JavaExe%" -Dsolr.solr.home="\"%CD%\solr"\"
-Djetty.home="\"%CD%"\" -Djetty.logs="\"%CD%\logs"\" -cp
"\"%CD%\lib\*.jar"\";"\"%CD%\start.jar"\" -jar "\"%CD%\start.jar"\"