We Keep Coding

Remote Profiling in Jprofiler

I have my sample java application running on port 9010 in one of my Azure VM with an IP let say xxx.xx.xx.254. I have installed Jprofiler in another Azure VM, with IP xxx.xx.xx.159. How can i profile the application from xxx.xx.xx.159 (Both machines are Windows)?
I have tried remote profile using direct connection, but it couldn't connect. Also checked with SSH, that too failed as 'Connection timeout'
Is there any step by step process for this connection?
Solution to the problem, how to connect and profile the Remote application from my local machine using JProfiler

"Direct connection" means that you can connect to remote machine on the selected profiling port (8849 by default). This is usually not the case unless the remote machine is on a private network because firewalls will prevent the connection. Also, in that case you must have added the -agentpath VM parameter for loading the JProfiler agent to the start command of the profiled JVM. This parameter can be obtained by invoking the "Session->Integration Wizards->New Remote Integration" wizard.
With SSH connections in JProfiler, you can tunnel the connection through SSH. This will work if you have an SSH server running on the remove machine. SSH connections work for VMs where the JProfiler agent has been loaded with the -agentpath VM parameter as well as in attach mode for all JVMs that are running on the remote system.
The related documentation is available at
https://www.ej-technologies.com/resources/jprofiler/help/doc/main/profiling.html

Profiling tomcat application with VisualVM

I am trying to run visualvm under the username tomcat6 because apparently visualvm can only find applications running under its username. So by default it is only finding applications running under my username. I have been able to connect visualvm with tomcat6 through jmx but that lacks the fine granularity of instrumented profiling.
I tried the following to run visualvm under the username tomcat6 but got the following error that I don't understand.
$ sudo -u tomcat6 jvisualvm
No protocol specified
Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)
at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:115)
...

If the computer running your application is remote -- like a server -- then you can't run GUI applications without some work. It's probably going to be easier to enable remote access to VisualVM.
You can use two techniques to attach to a remote JVM: using jstatd or using JMX. I'm not sure what you think you are losing by using JMX, but evidently jstatd doesn't give you access to profiling tools, CPU monitor, etc.).
You need to configure your JVM and Tomcat to allow for remote access. That requires 3 steps:
Enable remote JMX. Turns out, there's a guide for that.
Fix the "wandering port" used for RMI. There's a guide for that, too.
(Optional) Arrange for secure remote-access to the server. The easiest way to do that would be to use ssh -Lport:localhost:port with a series of -L arguments to forward multiple ports from your workstation to your server. Map all the ports you had to configure in steps #1 and #2. If you don't do this, you'll need to have non-firewalled access to all the aforementioned ports.
Restart your JVM and connect with JVisualVM.
Update 2022-06-01
Note that the "wandering port" problem has been fixed at the JVM level, so there is no need for application (i.e. Tomcat) support for that. Item #1 for Tomcat 8.5 and later contains updated instructions making item #2 unnecessary with a recent JVM.

Unfortunately only sampling is available in remote mode so JMX will lack instrumentation tools.
Actually your approach to running visualvm under tomcat6 user is correct. You should take a look at this question on how to run X11 applications under sudo.
The easiest way to pass DISPLAY and XAUTHORITY environment variables is to use sudo -E command to preserve current user environment.
Also if you can't see your process under tomcat6 user you should check if CATALINA_TMPDIR is pointing to /tmp. Otherwise you should pass it to visualvm
jvisualvm -J-Djava.io.tmpdir="${CATALINA_TMPDIR}"
Actually there is a lot of alternatives like yourkit or jprofiler shipped with java agents which allows remote instrumentation profiling.

The easiest is to open a remote JXM port on Tomcat in order to be able to remotely (from your desktop computer) connect to your remote Tomcat (on your server) with jvisualvm.
You need to pass the following system properties to your JVM :
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=<whatever_port_you_want>
-Dcom.sun.management.jmxremote.ssl=false
Then open jvisualvm on your local computer (JVM version needs to be the same or newer), File -> Add Remote Host -> Enter the name on the Host. It will create an entry for this host. Right lick on this Entry -> Add JMX connection -> Enter the port -> OK
jvisualvm will then be able to access remotely to your application.
You can also secure the connection if needed by using the following system properties (you need to create the files and locate them where you want :
-Dcom.sun.management.jmxremote.password.file=jmxremote.password
-Dcom.sun.management.jmxremote.access.file=jmxremote.access
These properties needs to be added to the CATALINA_OPTS environment variable. Fr exemple :
export CATALINA_OPTS = "$CATALINA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 "
More info on JMX lies here : https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html

I tried to do something similar but I was not allowed to install JVisualVM on the server. Having JVisualVM connect to the remote machine never seemed to work correctly. I suspect firewall rules were blocking part of the the network connections.
The only way I found to remotely profile the server was via an ssh tunnel.
Set the JMX port in CATALINA_OPTS on the server
CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.port=13333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false; export CATALINA_OPTS
On your desktop machine open an ssh connection to the server
ssh -D 61444 username#tomcat.server.address
Add a flag to JVisualVM so that it will proxy its network connection
"C:\Program Files\Java\jdk1.7.0_79\bin\jvisualvm.exe" -J-Dnetbeans.system_socks_proxy=localhost:61444 -J-Djava.net.useSystemProxies=true
Have JVisualVM connect to the jmxport and the network traffic is tunneled via ssh.
Good luck.

"Not supported for this JVM" when attempting to connect to a remote host on VisualVM

After successfully attaching VisualVM to a remote host I keep getting "Not supported for this JVM" label on the CPU panel, system properties, etc. Also, I don't seem to have access to the "Threads" panel, that I need in order to do thread dumps.
I've tried to run 32 and 64 bit versions of VisualVM, from both JDK7 and JDK8 (the target process is running on a Linux machine in 64 bits mode). The problem seems to be just the same in both cases for VisualVM configurations.
This is what java -version gets me on that machine:
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
Is there anything I should take into special consideration or is there some kind of limitation when doing remote access?

It doesn't mean the JVM that you're using to run VisualVM when it says "Not supported for this JVM", it means the remote server's VM.
You're probably running a non-oracle VM there (openjdk probably) in which case VisualVM can't work properly. Either change to Oracle's JRE on the server, or get a profiler that can handle OpenJDK too.

When you connect to a remote JVM the VisualVM will usually talk over the JMX interface to get the threads etc. However, the remote JVM decides what it will make available to the client, rather than everything being available by default; so it's possible that the JVM you're connecting to just doesn't support reporting on those characteristics.
Assuming that the connection works properly, you will find that there are various isXxxSupported methods that the JVM's returning, possibly false in this case.
One thing you might like to try is to run jcmd on the remote server (or jps) to see if there's some kind of restriction on the processes in the box that are preventing you from seeing the other VMs as that user. It may be that if it's running in something like a container then the RMI calls for the JMX aren't being passed through, or if you're running in an selinux or a Java security policy that it's not letting you read/communicate on the network ports or files.
JMXServiceURL url = new JMXServiceURL("rmi", "", 0, "/jndi/rmi://" + host + ":" + port + "/jmxrmi");
JMXConnectionFactory factory = JMXConnectorFactory.connect(url);
MBeanSearverConnection conn = factory.getMBeanServerConnection();
ThreadMXBean threads = ManagementFactory.newPlatformMXBeanProxy(conn, ManagementFactory.THREAD_MXBEAN_NAME,ThreadMXBean.class);
long ids[] = threads.getAllThreadIds();
If you can get the thread IDs then it suggests you can talk to the server; there are other methods on the threads that will allow you to get stack traces etc. This will be the method that's used by the VisualVM and may allow you to do further debugging as to what's going wrong on the responses.

Unable to connect to remote JMX using VisualVM but OK with JConsole

I'm attempting to connect to a remote JMX service using VisualVM 1.3.8 with JRE 1.8.0. Making exactly the same connection with JConsole works perfectly, however the attempt to use the same parameters with VVM just hangs at the 'Adding services' stage. I've looked in the VVM log but there is literally nothing logged.
I also read here at the VVM remote JMX doc:
Note: To retrieve and display information on applications running on the remote host, the jstatd utility needs to be running on the remote host.
That puzzles me since I can make the remote JMX connection using JConsole and as far as I can tell, jstatd is not running on the remote.
Does anyone know if jstatd a requirement only for VVM as a remote JMX client? As the JMX connection can be made with JConsole between the same client and server endpoints, then there's no problem with network/firewall etc.
Other than that I'm really puzzled where else to look for clues?

I had the same trouble when I switched from Java 7 to 8, while using SOCKS where I was updating proxy settings through VisualVM -> Tools -> Plugins -> Settings -> Proxy Settings. However, I have been successful with the following:
Run your JVM with the following options:
-Dcom.sun.management.jmxremote.port=<JMX_PORT>
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
Note that you could obviously do something more secure, both w.r.t SSL and authentication.
Setup a SOCKS proxy from your localhost to the remote server:
ssh -D<SOCKS_PORT> -vvv -N <REMOTE_HOST>
Run either of these commands on your localhost:
jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort=<SOCKS_PORT> service:jmx:rmi:///jndi/rmi://<REMOTE_HOST:JMX_PORT>/jmxrmi
jvisualvm -J-DsocksProxyHost=localhost -J-DsocksProxyPort=<SOCKS_PORT> --openjmx <REMOTE_HOST>

Why so many TCPv6 objects in slab when I run my Netty Application

I have a netty application to hold about 1,500,000 long connections. When I did this , I used 'slabtop' and found there are 1,500,000 TCPv6 objects in slab(about 3GB), But I didn't use IPv6 at all, All the connections based on IPv4 extactly. Why?
update: My server is CentOS 5.5, Linux version 2.6.18-194.el5(x86_64) and I use Java HotSpot(TM) 64-Bit Server VM to run my application