My Java process is stuck and hogging a lot of CPU on the VM. I am trying to get to the root of the issue by getting a heapdump. This process is running in a docker container. I am getting into the container and running a jmap -F -dump:format=b,file=/tmp/heapdump.bin 6 as root user. But I still get java.lang.reflect.InvocationTargetException.
I used jmap -F to get over this exception - com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file: target process not responding or HotSpot VM not loaded
I referenced the this post to make modifications to my query - Running jmap getting Unable to open socket file,
I have tried both jmap and jcmd queries, but got the same results.
Can someone please help here?
# jmap -F -dump:format=b,file=/tmp/heapdump.bin 6
Attaching to process ID 6, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.111-b14
Dumping heap to /tmp/heapdump.bin ...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: java.lang.InternalError: Metadata does not appear to be polymorphic
at sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:278)
at sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
at sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:68)
at sun.jvm.hotspot.memory.DictionaryEntry.klass(DictionaryEntry.java:71)
at sun.jvm.hotspot.memory.Dictionary.classesDo(Dictionary.java:66)
at sun.jvm.hotspot.memory.SystemDictionary.classesDo(SystemDictionary.java:190)
at sun.jvm.hotspot.memory.SystemDictionary.allClassesDo(SystemDictionary.java:183)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeClasses(HeapHprofBinWriter.java:954)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:427)
at sun.jvm.hotspot.tools.HeapDumper.run(HeapDumper.java:62)
at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:83)
I too faced this issue with openjdk8 alpine. Worked fine on upgrading base image to openjdk:17-jdk-alpine
Related
I want to check whether my application gets -Xmx option well.
The java process is run in a docker container.
In order to do that, I am to find the max heap size of the created process.
At first, I tried jmap command, but it occurs an error:
bash-4.2$ jmap -heap 128
Attaching to process ID 128, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 128: Operation not permitted
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 128: Operation not permitted
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:163)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach(LinuxDebuggerLocal.java:278)
at sun.jvm.hotspot.HotSpotAgent.attachDebugger(HotSpotAgent.java:671)
at sun.jvm.hotspot.HotSpotAgent.setupDebuggerLinux(HotSpotAgent.java:611)
at sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:337)
at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:304)
at sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:140)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:185)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 128: Operation not permitted
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach0(Native Method)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$100(LinuxDebuggerLocal.java:62)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1AttachTask.doit(LinuxDebuggerLocal.java:269)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:138)
I have no idea why jmap command fails. I searched for the same issues, but none of them helped me.
What I tried:
run the command with the user who created the process
run a command outside docker container
docker exec container_id jcmd PID GC.heap_dump /tmp/docker.hprof
And I cannot modify the existing java code, so I need to do this with CLI.
jstat command works fine, so is there a way to check jvm max heap size with jstat command?
Is it okay to assume (NGCMX+OGCMX) = (-Xmx value) ?
https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jstat.html
You could use the jcmd tool:
jcmd <you_vm_process_id> VM.flags | grep MaxHeapSize
And that will show the maximum heap (even if you do not set it yourself)
My system is Linux mint 18.3. Java version "1.8.0_161". When i start jenkins by using command "java -jar jenkins.war" -All went right. When i start jenkins inside tomcat -All went right. But when i tried to start jenkins by using command "java -jar jenkins.war -- httpPort=9090" i got error. Could you help me find reason?
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at Main._main(Main.java:309)
at Main.main(Main.java:132)
Caused by: java.lang.IllegalArgumentException: Unrecognized option: --
at winstone.cmdline.CmdLineParser.parse(CmdLineParser.java:53)
at winstone.Launcher.getArgsFromCommandLine(Launcher.java:363)
at winstone.Launcher.main(Launcher.java:334)
... 6 more
You are missing a space between -- and httpPort. Just change it to be something like this:
java -jar jenkins.war --httpPort=9090
When trying to extract an hprof from a Windows minidump of a crashed JVM (-XX:+CreateMinidumpOnCrash) with jmap, I'm receiving an InvocationTargetException at about 31 MB of hprof file written (the mdmp file is 1700+ MB large):
jmap -dump:format=b,file=hs_err_pid15916.hprof "C:\Program Files\Java\jdk1.8.0_40\jre\bin\java.exe" hs_err_pid15916.mdmp
Attaching to core hs_err_pid15916.mdmp from executable C:\Program Files\Java\jdk1.8.0_40\jre\bin\java.exe, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.40-b25
Dumping heap to hs_err_pid15916.hprof ...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.utilities.AssertionFailure: can not get class data for sun/net/ExtendedOptionsImpl$$Lambda$10x00000001001afc2
at sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeInstance(HeapHprofBinWriter.java:803)
at sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:95)
at sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:353)
at sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:171)
at sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:51)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:433)
at sun.jvm.hotspot.tools.HeapDumper.run(HeapDumper.java:62)
at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:83)
... 6 more
The exception is thrown regardless of the jmap variant used (1.8.0_40 Oracle JDK vs 1.8.0_40 Zulu).
How can I further diagnose this problem?
Update:
If I use a more recent jmap version (1.8.0_92 vs 1.8.0_40), can't pull out any hprof, either:
sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 25.92-b14. Target VM is 25.40-b25
I am running an APP using Jruby and Puma. We have an issue with a potential memory leak and I would like to generate a heap dump so that I can debug. My local environment is running Linux Mint 17.
I have run the following:
jmap -histo:live 6450
and this resulted in :
6450: Unable to open socket file: target process not responding or HotSpot VM not loaded The -F option can be used when the target process is not responding
I've also tried this:
jmap -dump:live,format=b,file=heap.bin 6450
result
6450: Unable to open socket file: target process not responding or HotSpot VM not loaded
As the propmt suggested I tried running with the -F flag:
jmap -F -dump:live,format=b,file=heap.bin 6450
Attaching to process ID 6450, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.45-b02
Dumping heap to heap.bin ...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.utilities.AssertionFailure: can not get class data for java/lang/UNIXProcess$Platform$$Lambda$10x0000000100393428
at sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeInstance(HeapHprofBinWriter.java:803)
at sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:95)
at sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:353)
at sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:171)
at sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:51)
at sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:433)
at sun.jvm.hotspot.tools.HeapDumper.run(HeapDumper.java:62)
at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:83)
... 6 more`
Does anyone have any suggestions as to how to get around these errors and generate a heap dump ? I have tried using jvisualvm and Eclipse Memory Analyzer to generate a heap dump as well, and I run into the same issue.
This looks like a JVM bug in jdk 1.8_45 see this link for more info
https://bugs.openjdk.java.net/browse/JDK-8065318
I get the following exception when i take a heapdump using
jmap -F -dump:format=b,file=/tmp/heapdump/before.hprof 10737
Attaching to process ID 10737, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: java.lang.RuntimeException: Type "nmethodBucket*", referenced in VMStructs::localHotSpotVMStructs in the remote VM, was not present in the remote VMStructs::localHotSpotVMTypes table (should have been caught in the debug build of that VM). Can not continue.
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupOrFail(HotSpotTypeDataBase.java:361)
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMStructs(HotSpotTypeDataBase.java:252)
at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:87)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:77)
Anyone know how to resolve this ?
I was seeing the same error because my path to jmap wasn't the same as the path to the java process (i.e. targeting two different versions).
Running jmap with the full path to my JDK resolved it.
If OpenJDK is used, it requires installation of debuginfo-packages.
In Centos this works with
- sudo debuginfo-install java-1.8.0-openjdk
- or sudo yum install java-1.8.0-openjdk-debuginfo.x86_64
See
- https://bugzilla.redhat.com/show_bug.cgi?id=1010786#c15
- amazon linux - install openjdk-debuginfo?