I have an taken a core dump file from a machine, with file size 2GB. I can open this file with gdb and see the processes that were running when I took the core file. I want though to convert this file into a heap dump in order to see more useful information. I tried running the below commands without any success. Does anyone have an idea why the file is not converted into heap dump? How can I perform this action?
I tried the bellow commands but I get errors and the file never is being created.
jmap -heap:format=b file=jvmcore.hprof /usr/lib/jvm/jdk1.5.0_22/ jvm.core Usage: jmap \[option\] \<pid\> (to connect to a live java process) or jmap \[option\] \<executable\> \<core\> (to connect to a core file) or jmap \[option\] \[server_id#\]\<remote server IP or hostname\> (to connect to a remote debug server) where option must be one of: \<no option\> to print same info as Solaris pmap \-heap to print java heap summary \-heap:format=b to dump java heap in hprof binary format \-histo to print histogram of java object heap \-permstat to print permanent generation statistics \-h | -help to print this help message
Same result as above i got with the following command:
jmap -heap:format=b file=jvmcore.hprof /usr/lib/jvm/jdk1.5.0_11/ jvm.core
And trying this command gets an error:
jmap -heap:format=b file=jvmcore.hprof jvm.core Attaching to core jvm.core from executable file=jvmcore.hprof, please wait... Error attaching to core file: Can't attach to the core file
Related
I have running tomcat (9.0.39.0) server, started by user cpappt and running with pid: 1682650. The machine on which this server runs is not bare metal but a VMware machine and the JVM is from OpenJ9
> ps -ef | grep Bootstrap
cpappt 1682650 1 0 Jan01 ? 01:09:58 /srv/jdk11/bin/java
-Djava.util.logging.config.file=/srv/test/cpappt/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server
-Xms32m -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError
-Dcrewportalenvironment=test
-Denvironment=test
-Doracle.net.tns_admin=/CONFIG/global/ORA/HA
-Dspring.profiles.active=test,notification-services-intern
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0022
-Djava.awt.headless=true
-Dcom.sun.management.jmxremote
-Dfile.encoding=UTF-8
-Dsun.jnu.encoding=UTF-8
-Dcom.sun.management.jmxremote.port=7305
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=conf/jmxremote.password
-Dignore.endorsed.dirs= -classpath /srv/test/cpappt/tomcat/bin/bootstrap.jar:/srv/test/cpappt/tomcat/bin/tomcat-juli.jar
-Dcatalina.base=/srv/test/cpappt/tomcat
-Dcatalina.home=/srv/test/cpappt/tomcat
-Djava.io.tmpdir=/srv/test/cpappt/tomcat/temp org.apache.catalina.startup.Bootstrap start
I want to extract a heap dump of the tomcat to analyze a potential memory leak.
I've tried to use jps to acquire the pid of the tomcat server, unfortunately I only see the pid of jps itself.
> jps -l
952152 jdk.jcmd/openj9.tools.attach.diagnostics.tools.Jps
and I'm also running the jps command as the same user that started the tomcat server
> whoami
cpappt
I've also tried running the jps command, adding the java.io.tmpdir specified in the startup script of the tomcat, which leads to the same result as before (remark: the tmpdir itself is empty).
jps -l -J-Djava.io.tmpdir=/srv/test/cpappt/tomcat/temp
(Additional remark: If I started a small "Foo program", jps is showing me the pid of this program.)
Then I tried to create a heap dump using the pid from the ps command which failed:
> jmap -dump:live,format=b,file=/tmp/dump.hprof 1682650
unrecognized option -dump:live,format=b,file=/tmp/dump.hprof
jmap: obtain heap information about a Java process
Usage:
jmap <option>* <vmid>
<vmid>: Attach API VM ID as shown in jps or other Attach API-based tools
<vmid>s are read from stdin if none are supplied as arguments
-histo: print statistics about classes on the heap, including number of objects and aggregate size
-histo:live : Print only live objects
-J: supply arguments to the Java VM running jmap
NOTE: this utility might significantly affect the performance of the target VM.
At least one option must be selected.
If I just try to see the histogram, this also fails:
> jmap -histo:live 1682650
Error getting data from 1682650: Exception connecting to 1682650
Also jcmd is not able to create the dump:
> jcmd 1682650 GC.heap_dump /tmp/heapdump
Error getting data from 1682650: Exception connecting to 1682650
...
I somehow suspect that both problems (jps and jmap) are related ...
So, if anyone has an idea what might be the problem or even how I could create the heap dump, that would be great.
Additional information from the tomcat server (and java):
Server version number: 9.0.39.0
OS Name: Linux
OS Version: 5.7.0-0.bpo.2-amd64
Architektur: amd64
Java Home: /srv/jdk-11.0.8+10_openj9
JVM Version: 11.0.8+10
JVM Hersteller: Eclipse OpenJ9
> java -version
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 11 Linux amd64-64-Bit Compressed References 20200715_697 (JIT enabled, AOT enabled)
OpenJ9 - 34cf4c075
OMR - 113e54219
JCL - 95bb504fbb based on jdk-11.0.8+10)
> which java
/srv/jdk11/bin/java
> which jps
/srv/jdk11/bin/java
> which jmap
/srv/jdk11/bin/jmap
Update: 2021-01-09
I finally could get hold of the colleague that knows the password for the jmx user, so using jconsole I was able to trigger a heap dump, but I could only call triggerClassicHeapDump().
I copied the file to my local machine and wanted to analyze it with Eclipse MemoryAnalyzer (I've also added the Diagnostic Tool Framework for Java (DTFJ)).
But when I try to open the file I now get an error message ...
Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'. Check the error log for further details.
Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'. Check the error log for further details.
Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'
Unable to read dump .../heapdump.20210119.100934.2621412.0001.phd metafile .../heapdump.20210119.100934.2621412.0001.txt in DTFJ format DTFJ-PHD (java.io.IOException)
Unable to read dump .../heapdump.20210119.100934.2621412.0001.phd metafile .../heapdump.20210119.100934.2621412.0001.txt in DTFJ format DTFJ-PHD
Error parsing Javacore (java.io.IOException)
Error parsing Javacore
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.parser.ParserException)
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException)
Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Unable to read dump .../heapdump.20210119.100934.2621412.0001.txt metafile null in DTFJ format DTFJ-Javacore (java.io.IOException)
Unable to read dump .../heapdump.20210119.100934.2621412.0001.txt metafile null in DTFJ format DTFJ-Javacore
Error parsing Javacore (java.io.IOException)
Error parsing Javacore
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.parser.ParserException)
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException)
Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
I've added -Xdump:heap:events=user (see Xdump Option Builder) to the startup script of the tomcat and restarted the server.
> ps -ef | grep Bootstrap
cpappt 1919301 1 99 14:38 pts/0 00:01:00 /srv/jdk11/bin/java
-Xms32m -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError
-Xdump:heap:events=user
...
Now I can create a heap dump using kill -3 <pid> (have a look at superuser: what-does-kill-3-mean).
Two files were created in the catalina.base directory:
javacore.<XXX>.txt (the original thread dump (-3 := SIGQUIT)) and
heapdump.<YYY>.phd (well, the actual heap dump)
Finally I was able to open the heapdump.<YYY>.phd file with Eclipse MAT
Additional remark: We updated the openj9 version to the current version (23.0).
> java -version
openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9+11)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.23.0, JRE 11 Linux amd64-64-Bit Compressed References 20201022_810 (JIT enabled, AOT enabled)
Now jps also does find the tomcat server
> jps -l
1919301 org.apache.catalina.startup.Bootstrap
1921897 jdk.jcmd/openj9.tools.attach.diagnostics.tools.Jps
Also jcmd does work:
> jcmd 1919301 Dump.heap dump
Dump written to .../tomcat/dump
Also consider using -Xdump:system:events=user as the core file generated has a lot more information - however there are problems with the IBM DTFJ reading core files generated by OpenJ9.
Memory Analyzer can also generate heap dumps - see File > Acquire Heap Dump but the 'Attach API' and 'Attach API using a Helper JVM' may require some configuration. Memory Analyzer 1.12 has some improvements in acquiring heap dumps, so try the latest release.
Good day.
I have a JAVA application that writes a heapdump when an OutOfMemoryError ocurrs, the location of the heapdump is inconvenient and I'm trying to figure out if there's a way I can configure the automatic heapdump to place the dump file in a remote location other than the original server.
I know that for jmap there is an option that looks like this:
jmap [ option ] [server-id#]remote-hostname-or-IP
(see https://docs.oracle.com/javase/7/docs/technotes/tools/share/jmap.html)
But what I'm trying to do is directly on the VM options:
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=server-ip/server-directory
With Oracle's Hotspot JVM, it looks like jmap -dump:file=/tmp/dump.txt <pid> can be used to take a heap dump.
However, Eclipse OpenJ9 doesn't include the jmap tool - and if you try to use the regular jmap with OpenJ9's jvm, it gives the exception:
Exception in thread "main" java.lang.ClassCastException: com.ibm.tools.attach.attacher.OpenJ9VirtualMachine incompatible with sun.tools.attach.HotSpotVirtualMachine
at java.lang.ClassCastException.<init>(java.base#10.0.2-adoptopenjdk/ClassCastException.java:71)
at sun.tools.jmap.JMap.executeCommandForPid(jdk.jcmd#10.0.2-adoptopenjdk/JMap.java:128)
at sun.tools.jmap.JMap.dump(jdk.jcmd#10.0.2-adoptopenjdk/JMap.java:192)
at sun.tools.jmap.JMap.main(jdk.jcmd#10.0.2-adoptopenjdk/JMap.java:110)
So, how can one take a heap dump with OpenJ9?
An OpenJ9 heap dump can be created with the command jcmd <PID> Dump.heap <path>.phd.
For example:
jcmd 1 Dump.heap /tmp/heap-dump.phd
Notice:
It must be run as the same user that the JVM is running as.
The PID must be the ID of the JVM process to be inspected. jps -l will list the available processes.
Alternatively, use YourKit to take a memory snapshot:
Download YourKit and extract it
Use the Console Attach Wizard e.g. bash ./YourKit-JavaProfiler-2021.3/bin/attach.sh
Capture a memory snapshot: java -jar ./YourKit-JavaProfiler-2021.3/lib/yjp-controller-api-redist.jar localhost 10001 capture-memory-snapshot
Sources:
Java diagnostic command (jcmd) tool
Support of Portable Heap Dumps (.phd)
YourKit - Command line tool to control profiling
You can use -Xdump:heap:events=user to enable heap dump when signal 3 is passed to OpenJ9 JVM. So, start you application with this option and then issue kill -3 <pid> to get the heap dump.
You can also use Xdump Option Builder tool for generating the -Xdump options based on your requirement.
I can successfully attach to Java live process by pid using serviceability agent ., however failed to open the core dump with it, and got error "windbg error:OpenDumpFile failed" when open the core dump with serviceability agent .
BTW,I capture the dump with the following command , and it would hung if I omit the -F.
jmap.exe -F -dump:format=b,file=c:\temp\HeapDump.hprof pid
The environment:
win 7 64, JDK 7.0(hotspot)
A heap dump is not a core dump.
Heap dumps generated by jmap can be opened in the tools like VisualVM, Eclipse Memory Analyzer, YourKit Java Profiler etc.
Core dumps (or Minidumps in Windows terminology) are written by the OS or by the debuggers like WinDbg. jmap can also be used to extract a heap dump from a minidump.
Have a try to use jstack like this in the command line:
jstack 10776 e:\dump.txt
Replace the number 10776 with your own java process id.
I am creating heap dump using below command:
jmap -dump:file=DumpFile.txt <process-id>
I have opened the generated file - DumpFile.txt but it is not in readable format.
So please let me know how to analyze the data in the generated file.
You should use jmap -heap:format=b <process-id> without any paths. So it creates a *.bin file which you can open with jvisualvm.exe (same path as jmap). It's a great tool to open such dump files.
You can use jhat (Java Heap Analysis Tool) to read the generated file:
jhat [ options ] <heap-dump-file>
The jhat command parses a java heap dump file and launches a webserver. jhat enables you to browse heap dumps using your favorite webbrowser.
Note that you should have a hprof binary format output to be able to parse it with jhat. You can use format=b option to generate the dump in this format.
-dump:format=b,file=<filename>
Very late to answer this, but worth to take a quick look at. Just 2 minutes needed to understand in detail.
First create this java program
import java.util.ArrayList;
import java.util.List;
public class GarbageCollectionAnalysisExample{
public static void main(String[] args) {
List<String> l = new ArrayList<String>();
for (int i = 0; i < 100000000; i++) {
l = new ArrayList<String>(); //Memory leak
System.out.println(l);
}
System.out.println("Done");
}
}
Use jps to find the vmid (virtual machine id i.e. JVM id)
Go to CMD and type below commands >
C:\>jps
18588 Jps
17252 GarbageCollectionAnalysisExample
16048
2084 Main
17252 is the vmid which we need.
Now we will learn how to use jmap and jhat
Use jmap - to generate heap dump
From java docs about jmap
“jmap prints shared object memory maps or heap memory details of a given process or core file or a remote debug server”
Use following command to generate heap dump >
C:\>jmap -dump:file=E:\heapDump.jmap 17252
Dumping heap to E:\heapDump.jmap ...
Heap dump file created
Where 17252 is the vmid (picked from above).
Heap dump will be generated in E:\heapDump.jmap
Now use Jhat
Jhat is used for analyzing the garbage collection dump in java -
C:\>jhat E:\heapDump.jmap
Reading from E:\heapDump.jmap...
Dump file created Mon Nov 07 23:59:19 IST 2016
Snapshot read, resolving...
Resolving 241865 objects...
Chasing references, expect 48 dots................................................
Eliminating duplicate references................................................
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.
By default, it will start http server on port 7000.
Then we will go to http://localhost:7000/
Courtesy : JMAP, How to monitor and analyze the garbage collection in 10 ways
If you use Eclipse as your IDE I would recommend the excellent eclipse plugin memory analyzer
Another option is to use JVisualVM, it can read (and create) heap dumps as well, and is shipped with every JDK. You can find it in the bin directory of your JDK.
VisualVm does not come with Apple JDK. You can use VisualVM Mac Application bundle(dmg) as a separate application, to compensate for that.
MAT, jprofiler,jhat are possible options. since jhat comes with jdk, you can easily launch it to do some basic analysis. check this out
If you just run jmap -histo:live or jmap -histo, it outputs the contents on the console!