How to do a thread dump on Azure App Service? - java

I have an Azure App Service running Tomcat 8.5 and can't seem to figure out how to do a thread dump. jstack is not recognized in the Console nor the DebugConsole.

On Azure App Service, all popular versions of Java had been installed at the path D:\Program Files (x86)\Java. Take version 1.8.0u73 as example, please see the figure below.
You can command set PATH=D:\Program Files (x86)\Java\jdk1.8.0_73\bin;%PATH% to add Java tools to the environment temporarily via the Kudu console tool. Then, the jstack tool is available in the current Kudu session.

Inspired by the previous answer, I created the following bat script (could be powershell, I suppose) that does the job:
powershell -Command "get-process java |select -expand id" > pid.txt
set /p id= < pid.txt
"%JAVA_HOME%\bin\jstack" -F %id% > out.txt
It uses the java version that you're using in your app (JAVA_HOME) and doesn't require any input from you. It's simple and does the trick. Just create a bat file in your wwwroot, for example, and run it whenever you want.

I know this is an old question but for Windows Apps on Azure App Service, this can be easily done by going to Azure Portal -> Diagnose and Solve problems blade -> Diagnostic Tools and then choosing Collect Java Thread dump from the left menu or the middle pane.

Get the PID of java process from Process Explorer of Kudu (Advanced Tools)
Use this command to collect thread dump:
jcmd Thread.print > D:\home\threaddump.txt

Related

How to obtain threaddump of a java application on Cloud foundry?

I tried a couple of options
a. cf java thread-dump myapp -i0 (Only 1 instance is there so 0)
Response is nothing.
logs of myapp show
2019-12-13T14:52:41.15+0800 [SSH/0] OUT Successful remote access by 10.x.x.x:35764
2019-12-13T14:52:41.84+0800 [SSH/0] OUT Remote access ended for 10.x.x.x:35764
b. I did cf ssh myapp
/home/vcap/app/.java-buildpack/open_jdk_jre/bin/ has no jstack and jmap command doesn't work either
/home/vcap/app/.java-buildpack/open_jdk_jre/bin/ has the following components
java orbd servertool java-buildpack-memory-calculator-3.13.0_RELEASE pack200 tnameserv
jjs policytool unpack200
jvmkill-1.16.0_RELEASE rmid
keytool rmiregistry
Question is-> How to get the java threaddump?
I believe the easiest option is to run cf logs to watch your app logs. Then in a second terminal, cf ssh to the app and run ps aux and look for the process id of the Java process. Then run kill -3 <pid>. Your Java process will catch this signal and issue a thread dump. It will go to STDOUT, which conveniently goes to your app's log stream so it should show up in the first terminal.
The crummy part about this is that cf logs will inject some additional text at the beginning of every line, which makes it hard to load the thread dump into tools. You can usually strip that off with cut or awk, or you can redirect cf logs to a file, open with a text editor and find/replace the leading text.
Besides that you have a couple other options:
If you're using Spring, enable Spring Boot Actuators. You can then use the /actuator/threaddump endpoint to generate and download a thread dump (note the format for this is JSON, not the standard format)
Use an APM tool or profiler like NewRelic, AppDynamics, Dynatrace or YourKit. These are well supported by the Java buildpack and most will just work.
Use JMX. It's easy enough to enable with the Java buildpack, just set the env variable JBP_CONFIG_JMX to '{enabled: true}' and restage your app. Once enabled, you can open a tunnel with cf ssh -N -T -L 5000:localhost:5000 <APP_NAME>. Then open jvisualvm, make a new "local" JMX connection to "localhost:5000" and connect. Click the "Threads" tab and click the "Thread Dump" button. It might be a little slow as it's going across the tunnel, but give it a few seconds to finish and you should have a thread dump.
What won't work on Cloud Foundry are jstack and jcmd. The Java buildpack, at the time of writing, installs a JRE and these tools are no longer shipped with the OpenJDK JRE and for legal reasons can't be bundled with it. If you were inclined to do so, you could scp the tools and required shared libraries from a JDK up to your app container and run them, but it's a lot of work and not fun. I'm not super familiar with the cf java plugin, but I suspect it tries to use these tools, which used to be available, and no longer are there.
You should be able to get ThreadDumps with kill -3 <PID> and open that in heap analyzer such as eclipse memory analyzer (MAT) and select Thread Stacks

Launching Windows Jenkins Agents without JNLP

Built a new Windows Server 2016 to act as a target for jenkins orchestrated builds. We are not supposed to use oracle java anymore, so I got coretta openJDK. OpenJDK does not support javaws any more. The executable does not ship with it. I have seen some scripted workarounds, but they do not work and are poorly written to boot.
What are we supposed to use to launch slave agents on Windows machines?
There are several ways to implement this but since this is a windows server you can follow steps on https://github.com/kohsuke/winsw/blob/master/doc/installation.md#winsw-installation-guide to install jenkins slave as service without javaws.
I believe #Mike was referring to: Install Slave as a Windows service (cmd version), which does not need JNLP/javaws.
OR, under Jenkins 2 (we are using 2.121.1 plus WMI Windows Agents (aka windows-slaves:1.4), there is now an option to "Let Jenkins control this Windows slave as a Windows service", essentially in the same fashion as a Linux node.
You must have an Admin account (and password). We had to follow the guidance to edit registry when "Windows agents fail to start via DCOM" (Guidance is for Server 2012) to remotely manage the service. So far, seems to be working OK, YMMV. So far, seems to be working OK, no "subtle probelms"; YMMV.
For entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID{72C24DD5-D70A-438B-8A42-98424B88AFB8}
HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}
(There are several matches for the keys; just change the two locations indicated)
Launch 'regedit' (as Administrator)
Find (Ctrl+F) the following registry key: "{72C24DD5-D70A-438B-8A42-98424B88AFB8}" in
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID\
Right click and select 'Permissions'
Change owner to administrators group (Advanced...).
Change permissions for administrators group. Grant Full Control.
Change owner back to TrustedInstaller (user is "NT Service\TrustedInstaller" on local machine)
Repeat the steps 1-6 for HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}
Restart Remote Registry Service (Administrative Tools / Services)
For me the easiest and best working solution to launch Windows Jenkins Agents without JNLP was to run in the command prompt java -jar agent.jar ... command. The command parameters should be grabbed from the url of the agent:
http://your_jenkins_url/computer/your_agent_name_or_ip/
I downloaded the agent.jar on my PC, moved it to a desired directory, launched the Command Prompt from the dir where the agent.jar was moved and executed the proposed command with the appropriate parameters.
In my case:
java -jar agent.jar -jnlpUrl http://your_jenkins_url/computer/your_agent_name_or_ip/slave-agent.jnlp -secret 76986574e97c2b635c7076740dc93326eaaf5a3ad30573144915489a1ccfee44 -workDir "D:\jenkins"
This was one of the proposed approaches in the docs here - https://www.jenkins.io/doc/administration/requirements/upgrade-java-guidelines/
Java Web Start Java Web Start has been removed in Java 11. When a
Jenkins controller is running on Java 11, the Java Web Start button
will no longer appear in the Web UI. Agents for a Java 11 Jenkins
server can’t be launched from a *.jnlp file downloaded to a web
browser.
There are no plans to replace this functionality. Connect agents to
Jenkins on Java 11 with plugins like SSH Build Agents Plugin, with
operating system command line calls to java -jar agent.jar, or by
using containers.

How to run a Java application on Ec2?

I want to deploy a a project to ec2. It was modify in Intellij which used MAVEN. It work perfect in Intellij. Below is the configuration of Intellij.
I have tried command line like java, javac. These kind of comment lines are all failed. I am wondering there must some way to convert Intellij configuration to command lines. I was relying too much on IDE. XD. Now I am regreted.
EC2 is the amazon cloud service for instances. If you aren't very experienced with it - the easiest you can do is to hire a machine with your fav operating system - for example Windows and use remote desktop to log in it and just do your normal IDE setup and run your app. You will have your app running on your instance and it will be working, even it is done the hard way. I will get downvotes for that, but it will work as a Swiss watch.
It depends from the kind of EC2 instance you're using.
For me would be simpler if it was a Linux instance.
I'll connect via ssh and git clone my project there from my bitbucket/github repository.
Then I'll use Maven, and there are many different options to run your project with Maven.
As suggested in the comments you could use the exec plugin.
But even in this way you'll have a list of problems to overcame.
For example your linux instance does not came with an preinstalled Java Runtime.
You have to install at least Java and Maven (choose ubuntu distro, so you can do easily with a package manager like apt-get).
Or, for example, another not easy task is connect via ssh to a Linux instance.
AWS generates a key.pem file that you have to use in order to successfully connect to your instance.
ssh -i .ssh/your-key-file.pem username#your-ec2-instance-address
And again, AWS does not tell you the name of the default user you must use to connect to your EC2 instance (if you choose ubuntu distro the username is ubuntu).
And again, when you save your-key-file.pem in your computer it must have the right permission
-rw-------# 1 freedev staff 1692 Apr 21 09:46 /Users/freedev/.ssh/your-key-file.pem
or your ssh client wont read it.
...looking back it was really a long way make a deploy on a EC2 instance.
I have solved it by myself.
Use mvn package to generated jar file for maven project.
nohup java xx.jar debug.xml & use this to run jar in shell. By using this command line, when exit (abort remote connection), the process / command will not get killed.
Command line jcmd is used to check running process pid.

Wrapping a jar file in a Windows service

Have you had experience with running a jar file using a command line, wrapped in a Windows service?
I'm trying to find a way to run a jar file without being logged into the machine, and since it allows command shell, I was wondering if it's a good idea.
Thanks!
Original Post:
I'm trying to run Associated Press's Web Feeds Manager, which is basically a jar file that can be run when logged in by double clicking it.
I'd like to run the same file but without being logged in to the machine. In their manual (http://wfm.ap.org/admin/content/help/Running_Agent_on_a_Remote_Server.htm) they write how to do that, using a commandline parameter.
Basically I'd like the jar to run as a Windows service, regardless of who's logged in, but Googling it showed it was problematic.
Have you had experience with remotely running jar files? What are the pitfalls?
Thanks!
On a google search, I came across this article -
Running Jar Applications as a Windows Service
It mentions about open source Java Service Wrapper project from Tanukisoftware.org for accomplishing this task.
Note: I've not used this personally.
If you are not interested in having the service started/stopped at boot/shutdown, but you just want the program to be started manually and keep running after logout, here is what you do:
$ nohup java -jar foobar.jar > foobar.log 2>&1 &
which means: start my foobar.jar (java -jar) and keep it running after I logout (nohup) redirect stdout to foobar.log (>) and also the stderr (2>&1), and make it running in background (& at the end).
Instead, if you are interested in installing a "service" in your linux box, there are many options, depending on what distribution you are using.
The most common are upstart (for ubuntu) and System V init scripts (Redhat or others). Also cron can be used to start/stop services at startup/shutdown.
You can find an example of installing a java app (hudson) on an init system here, or doing the same thing with upstart. Or, as I said, cron could be an option.
On Windows, there is Java Service Wrapper. And not much more.
For windows Java Service Wrapper is a better choice
My favourite is the upstart on linux, but it is Ubuntu only.
On Windows I see many alternatives according to this forum.

How do I run a Java .jar file as a Windows service on Windows Server 2008?

How do I run a Java .jar file as a Windows service on a Windows 2008 server? I have a jar file called SomeJavaFile.jar located under the C:\SomeDirectory directory on a Windows Server 2008 box. I usually just run this jar manually in the command line with: java –cp SomeJavaFile.jar com.ctg.SomeJavaFile and I let it run 24/7.
The problem with our Windows Server is if it restarts I need to run it as a service so it will automatically start as a service upon startup, because our processing for Vistakon runs 24/7. I have Googled around and tried to use a service wrapper and the sc.exe command to create the service.
I put the wrapper service.exe in the C:\SomeDirectory\. I use the following command to create it: sc.exe SomeJavaService binPath= “C:\SomeDirectory\service.exe \”java –jar C:\SomeDirectory\SomeJavaFile.jar\”” type= own start= auto error= ignore. This creates the SomeJavaService service correctly but when I try to start it I get an error that says the service on Local Computer started then stopped.
Some services stop automatically if they are not in use by other services or programs. Do I need to alter my sc.exe command to have the exact previous working command line maybe, by adding in the com.ctg.SomeJavaFile? So should I change This jar should just run in the background and constantly poll/ping the C:/poll directory then if there is data present it processes the data and sends an export file to another directory.
I have used this same .jar file for years successfully and it hasn't changed, but I cannot get it to run as a Windows service. This is the site I use to get the service wrapper http://code.google.com/p/simple-service-wrapper/. Any help would be greatly appreciated!
"winsw" is the standalone version of the Windows Service installer shipping with the Glassfish Java EE reference implementation.
Works well, but is not a fully polished product - I have used it for exactly this purpose for a couple of years. Requires .NET in some recent version on the Windows machine.
https://github.com/kohsuke/winsw
I think that the best bet would be wrap your java app with Procrun of Apache Commons Daemon .
Procrun is a set of applications that allow Windows users to wrap
(mostly) Java applications (e.g. Tomcat) as a Windows service.
The service can be set to automatically start when the machine boots
and will continue to run with no user logged onto the machine.
As per my analysis,
The Idle Solution will be writing a VC++ (.net) Windows Service creation program to launch the .bat (that triggers the jar file)/.exe as a System service with all the required call back methods to SCM.
Note : 1. Wrapping the process with sc.exe / srvany.exe would not work as it does not have any call back process to the SCM (Service Control Manager). 2. And java service Wrapper is a third party API (many are LGPL licensed).
If you start your Java code from commandline by using java -j some.jar does it run until you terminate the program, or does it stop by itself?
There needs to be a NON-Deamon Thread, that is running all the time. A JVM will terminate, if there is no thread running, that is not flagged as daemon.
If you have a little budget, buy an installer tool. I use instll4j. With that tool, you can create service launcher and install them during instllation.
The following solution is working fine for me having OpenFire Connection Manager (which is a jar file) running as service on Windows Server.
Download and install the Non-Sucking Service Manager
I didn't use the batch made by OpenFire team, because it didn't work for me (dependencies not found error...) So, make a batch file with the following code :
#ECHO OFF
cd /D "E:\connection_manager\lib"
java -jar startup.jar
and save it as cm_startup.bat in the bin folder of connection manager.
After that you can create the service with NSSM.
So, open a cmd.exe and run the following command :
nssm install ConnManager "E:\connection_manager\lib\cm_startup.bat"
.
Doc & examples
More documentation and examples for the Non-Sucking Service Manager here : https://nssm.cc/usage Actually NSSM as a lot of options available.
Here is a more complexe example :
nssm install solr "%JavaExe%" -Dsolr.solr.home="\"%CD%\solr"\"
-Djetty.home="\"%CD%"\" -Djetty.logs="\"%CD%\logs"\" -cp
"\"%CD%\lib\*.jar"\";"\"%CD%\start.jar"\" -jar "\"%CD%\start.jar"\"

Categories