When you start up a JVM WHERE it the PERM GEN allocated? Is it part of the main HEAP or is it in addition to the HEAP size.
for example if I use the following parameters:
-server -Xms10G -Xmx10G -XX:MaxPermSize=1536M
Is the total size of Java going to be 6G + 512M or 6.5G for Java or is the perm generation setup inside of the HEAP meaning that all running application will have 6G - 512M = 5.5MB for young / tenured(aka OLD) and Perm?
This graphic in 4. Sizing the Generations seems to imply it may be outside of the HEAP but I can't seem to find some that states the for sure.
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html
Looking at the output from jstat it would seem it would be on the outside of main HEAP but this may be just the way it is reported.
[jboss#pts03-taps-03 ~]$ jstat -gccapacity PID
NGCMN NGCMX NGC S0C S1C EC OGCMN OGCMX OGC OC PGCMN PGCMX PGC PC YGC FGC
85184.0 85184.0 85184.0 8512.0 8512.0 68160.0 10400576.0 10400576.0 10400576.0 10400576.0 21248.0 1572864.0 1387840.0 1387840.0 431 43
OGCMX = 10400576.0 (almost 10G OLD GEN)
NGCMX = 85184.0 (OGCMX + NGCMX = very close to 10G NEW GEN)
PGCMX = 1572864.0 (1.5G PERM GEN)
If possible please provide a link to documentation showing you case to be true.
-server -Xms10G -Xmx10G -XX:MaxPermSize=1536M
The total of the heap and perm gen is going to be 11.5 GB. However there are other areas of memory, e.g. direct memory which can be just as big. Another area is shared libraries which is basically a fixed size.
e.g. you can set
-mx128m -XX:MaxPermSize=1g
If the perm gen was insize the heap this would fail.
Related
I am trying to hunt the memory leak in Java Spring Boot app inside docker container.
Heap size of the app is limited like this:
java -XX:NativeMemoryTracking=summary -jar /app-0.1.jar -Xms256m -Xmx512m
Native memory diff looks like this:
./jcmd 7 VM.native_memory summary.diff
Native Memory Tracking:
Total: reserved=8295301KB +1728KB, committed=2794537KB +470172KB
Java Heap (reserved=6469632KB, committed=2245120KB +466944KB)
(mmap: reserved=6469632KB, committed=2245120KB +466944KB)
Class (reserved=1141581KB -9KB, committed=103717KB -9KB)
(classes #16347 -86)
(malloc=13133KB -9KB #23221 -306)
(mmap: reserved=1128448KB, committed=90584KB)
Thread (reserved=85596KB +999KB, committed=85596KB +999KB)
(thread #84 +1)
(stack: reserved=85220KB +1027KB, committed=85220KB +1027KB)
(malloc=279KB +3KB #498 +6)
(arena=97KB -31 #162 +2)
Code (reserved=255078KB +32KB, committed=32454KB +228KB)
(malloc=5478KB +32KB #8900 +80)
(mmap: reserved=249600KB, committed=26976KB +196KB)
GC (reserved=249066KB -2KB, committed=233302KB +1302KB)
(malloc=12694KB -2KB #257 -75)
(mmap: reserved=236372KB, committed=220608KB +1304KB)
Compiler (reserved=227KB +10KB, committed=227KB +10KB)
(malloc=96KB +10KB #807 +15)
(arena=131KB #7)
Internal (reserved=68022KB +720KB, committed=68022KB +720KB)
(malloc=67990KB +720KB #21374 -287)
(mmap: reserved=32KB, committed=32KB)
Symbol (reserved=21938KB -11KB, committed=21938KB -11KB)
(malloc=19436KB -11KB #197124 -188)
(arena=2501KB #1)
Native Memory Tracking (reserved=3962KB -12KB, committed=3962KB -12KB)
(malloc=15KB #178 +1)
(tracking overhead=3947KB -12KB)
Arena Chunk (reserved=199KB, committed=199KB)
(malloc=199KB)
After taking the heap dump:
./jmap -dump:live,format=b,file=/tmp/dump2.hprof 7
The heap Leak Suspects report is quite small - 45MB:
The question:
why is Java Heap committed=2245120KB - almost 2GB? It's not aligned with -Xmx512m nor with heap dump size taken with jmap.
The answer is actually simple:
Params -Xms256m -Xmx512m were passed in wrong place and therefore ignored by JVM. The correct order of params is like this:
java -XX:NativeMemoryTracking=summary -Xms256m -Xmx512m -jar /app-0.1.jar
Also, dump is much smaller than Java Heap committed because only live objects were dumped, due to -dump:live. After changing dump command to:
./jmap -dump:format=b,file=/tmp/dump2.hprof 7
the size of dump is very close to Java Heap committed.
Why does the JVM process using ZGC of OpenJDK 11 on CentOS within Docker use huge shared memory?
Server: 2 cores/4G memory;
VIRT: 17.032t, RES: 7.974g, SHR: 7.382g, %CPU: 26.9, %MEM: 199
JVM parameters:
-Xmx3296m -Xms1977m -Xss256k
-XX:MetaspaceSize=128m
-XX:MaxMetaspaceSize=256m
-XX:+UnlockExperimentalVMOptions
-XX:+UseZGC
-XX:MinHeapFreeRatio=50
-XX:MaxHeapFreeRatio=80
After turning off ZGC, shared memory uses only 33K as below.
VIRT: 29g, RES: 1.5g, SHR: 33564, %CPU: 26, %MEM: 39.
I have tried to use jmap / eclipse / jvisualvm etc. to diagnose the problem, but did not make much progress. Any of your suggestions will be appreicated!
We have a long running java app that memory leakage issue. We use the following setting for starting the program. We use java 1.7.0_67.
java -server -Xmx500M -Xms500M -XX:NewSize=300M \
-verbosegc -Xloggc:/var/log/singer/gc.log -XX:+UseGCLogFileRotation \
-XX:NumberOfGCLogFiles=100 -XX:GCLogFileSize=2M -XX:+PrintGCDetails \
-XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintClassHistogram \
-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
After running for a few days, "top -p" will show something as follows:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29503 root 35 15 8240m 1.2g 14m S 36 2.0 215:31.18 java
'top' command shows that the resident memory usage for our program is 1.2G. It is way more than the 500M max heap size that we set.
The following shows some jvm metrics.The count is no where near 1.2G. The program does not have use this much memory when it starts.
jvm_gc_ConcurrentMarkSweep_cycles: 5
jvm_gc_ConcurrentMarkSweep_msec: 110
jvm_gc_ParNew_cycles: 26129
jvm_gc_ParNew_msec: 130964
jvm_gc_cycles: 26134
jvm_gc_msec: 131074
jvm_buffer_direct_count: 27
jvm_buffer_direct_max: 463077
jvm_buffer_direct_used: 463077
jvm_buffer_mapped_count: 0
jvm_buffer_mapped_max: 0
jvm_buffer_mapped_used: 0
jvm_classes_current_loaded: 2821
jvm_classes_total_loaded: 2821
jvm_classes_total_unloaded: 0
jvm_compilation_time_msec: 12976
jvm_current_mem_CMS_Old_Gen_max: 209715200
jvm_current_mem_CMS_Old_Gen_used: 82458736
jvm_current_mem_CMS_Perm_Gen_max: 85983232
jvm_current_mem_CMS_Perm_Gen_used: 20445832
jvm_current_mem_Code_Cache_max: 50331648
jvm_current_mem_Code_Cache_used: 4465792
jvm_current_mem_Par_Eden_Space_max: 251658240
jvm_current_mem_Par_Eden_Space_used: 131968344
jvm_current_mem_Par_Survivor_Space_max: 31457280
jvm_current_mem_Par_Survivor_Space_used: 2681328
jvm_current_mem_used: 242020032
jvm_fd_count: 493
jvm_fd_limit: 65536
jvm_heap_committed: 492830720
jvm_heap_max: 492830720
jvm_heap_used: 217095032
jvm_nonheap_committed: 38780928
jvm_nonheap_max: 136314880
jvm_nonheap_used: 24911624
jvm_num_cpus: 32
jvm_post_gc_CMS_Old_Gen_max: 209715200
jvm_post_gc_CMS_Old_Gen_used: 13095808
jvm_post_gc_CMS_Perm_Gen_max: 85983232
jvm_post_gc_CMS_Perm_Gen_used: 20444448
jvm_post_gc_Par_Eden_Space_max: 251658240
jvm_post_gc_Par_Eden_Space_used: 0
jvm_post_gc_Par_Survivor_Space_max: 31457280
jvm_post_gc_Par_Survivor_Space_used: 2681328
jvm_post_gc_used: 36221584
jvm_start_time: 1440568584192
jvm_thread_count: 65
jvm_thread_daemon_count: 25
jvm_thread_peak_count: 79
jvm_uptime: 50765537
The process status:
Name: java
State: S (sleeping)
Tgid: 29503
Ngid: 0
Pid: 29503
PPid: 1
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 1024
Groups: 0
VmPeak: 8440764 kB
VmSize: 8439440 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 1232168 kB
VmRSS: 1232168 kB
VmData: 8386608 kB
VmStk: 136 kB
VmExe: 4 kB
VmLib: 15320 kB
VmPTE: 3296 kB
VmSwap: 0 kB
Threads: 104
SigQ: 0/241457
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000002
SigCgt: 2000000181005ccd
CapInh: 0000000000000000
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
Seccomp: 0
Cpus_allowed: ffffffff,ffffffff,ffffffff,ffffffff
Cpus_allowed_list: 0-127
Mems_allowed: 00000000,00000003
Mems_allowed_list: 0-1
voluntary_ctxt_switches: 52
nonvoluntary_ctxt_switches: 2
This HAS to be a dupe, but let me give you a quick tip
First of all you need a profiler of some sort. There are quite a few to choose from that can do this. Get the profiler to run against your app, then do the following:
Run 2 garbage collections (The profiler can do this)
Create a dump that saves the count of your classes
Let your program run a while, long enough to lose some memory
Run 2 garbage collections again
Make a second dump
Diff the two dumps (there should be a function in the profiler that gives you class count increase since the first dump making this really easy)
What you are looking for is a class count that increase dramatically for one or more classes. Once you find this all you have to do is figure out what is referring to (Holding a reference to) that class (Should also be in your dump somewhere), that's your leak. When there is more than one class increasing in number, look for the root one that contains references to the others--that's the one you have to free.
It's not your VM settings or anything like that, just a simple programming bug somewhere keeping references that you thought were freed--Like adding a listener without removing it or forgetting to dispose of frames.
In order to find a leak we usually use combination of JMeter/VisualVM heap dumps. Procedure is the following:
launch test app & connect to JVM with profiler (in our case VisulaVM)
start JMeter script to emulate real app usage
Start Sampler in visual VM, if app is big create memory dump and analyze it separately. Inside the dump - look at number of bytes and number of class instances created for your application. Initially, pay attention to your application specific classes.
Based on details provided in your description it is hard to say where the root cause of the leak. You should know your app better (it might be session beans which are not cleaned up by GC, etc), but as I mentioned memory dump is a good thing to start with.
For real production app servers it's always good to have JMX configured in order to troubleshoot these kind of problems later on.
One of the articles to start with.
i have sample code for increase Heap Memory.But it is not increasing Memory.
sample Code :
int mb=1024*1024;
long rt=Runtime.getRuntime().totalMemory();
int heapsize=(int) (rt/mb);
System.out.println("Heap Size : " +heapsize);
String[] cmd = {"cmd.exe", "/c", "cd/C C:\\Users\\xxxxxx\\Documents\\NetBeansProjects\\MultiThreadSample\\src\\multithreadsample && java -Xms61m -Xmx128m"};
Process exec = Runtime.getRuntime().exec(cmd);
exec.destroy();
SpawnAndChangeHeap is the class name.Can You Please Suggest Me?
java heap allocation are continuous and happens at JVM initialization. Programmatically heap size cannot be extended or modified until Xmx or Xms changed with JVM restart.
I run the same process on a 32bit machine as on a 64bit machine with the same memory VM settings (-Xms1024m -Xmx1024m) and similar VM version (1.6.0_05 vs 1.6.0_16). However the virtual space used by the 64bit machine (as shown in top under "VIRT") is almost three times as big as that in 32bit!
I know 64bit VMs will use a little more memory for the larger references, but how can it be three times as big? Am I reading VIRT in top incorrectly?
Full data shown below, showing top and then the result of jmap -heap, first for 64bit, then for 32bit. Note the VIRT for 64bit is 3319m for 32bit is 1220m.
* 64bit *
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22534 agent 20 0 3319m 163m 14m S 4.7 2.0 0:04.28 java
$ jmap -heap 22534
Attaching to process ID 22534, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 10.0-b19
using thread-local object allocation.
Parallel GC with 4 thread(s)
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 1073741824 (1024.0MB)
NewSize = 2686976 (2.5625MB)
MaxNewSize = -65536 (-0.0625MB)
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 88080384 (84.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 268500992 (256.0625MB)
used = 247066968 (235.62142181396484MB)
free = 21434024 (20.441078186035156MB)
92.01715277089181% used
From Space:
capacity = 44695552 (42.625MB)
used = 0 (0.0MB)
free = 44695552 (42.625MB)
0.0% used
To Space:
capacity = 44695552 (42.625MB)
used = 0 (0.0MB)
free = 44695552 (42.625MB)
0.0% used
PS Old Generation
capacity = 715849728 (682.6875MB)
used = 0 (0.0MB)
free = 715849728 (682.6875MB)
0.0% used
PS Perm Generation
capacity = 21757952 (20.75MB)
used = 16153928 (15.405586242675781MB)
free = 5604024 (5.344413757324219MB)
74.24378912132907% used
* 32bit *
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30168 agent 20 0 1220m 175m 12m S 0.0 2.2 0:13.43 java
$ jmap -heap 30168
Attaching to process ID 30168, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 14.2-b01
using thread-local object allocation.
Parallel GC with 8 thread(s)
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 1073741824 (1024.0MB)
NewSize = 1048576 (1.0MB)
MaxNewSize = 4294901760 (4095.9375MB)
OldSize = 4194304 (4.0MB)
NewRatio = 8
SurvivorRatio = 8
PermSize = 16777216 (16.0MB)
MaxPermSize = 67108864 (64.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 89522176 (85.375MB)
used = 80626352 (76.89128112792969MB)
free = 8895824 (8.483718872070312MB)
90.0629940005033% used
From Space:
capacity = 14876672 (14.1875MB)
used = 14876216 (14.187065124511719MB)
free = 456 (4.3487548828125E-4MB)
99.99693479832048% used
To Space:
capacity = 14876672 (14.1875MB)
used = 0 (0.0MB)
free = 14876672 (14.1875MB)
0.0% used
PS Old Generation
capacity = 954466304 (910.25MB)
used = 10598496 (10.107513427734375MB)
free = 943867808 (900.1424865722656MB)
1.1104107034039412% used
PS Perm Generation
capacity = 16777216 (16.0MB)
used = 11366448 (10.839889526367188MB)
free = 5410768 (5.1601104736328125MB)
67.74930953979492% used
Think of virtual space as memory that program will ever be able to use. It is not the memory it actually uses. On 64-bit OS virtual memory is virtually endless, there is no 2Gb limitation as in 32-bit, so developers are free to choose any virtual size. There is nothing to worry about. The RES is actual physical RAM used.
Not all of the memory settings are the same - in particular, look at MaxNewSize, which is negative in the 64bit version.
Also, in 64bit, virtual memory is effectively limitless for programs written for today's computers, and it's not something to be concerned about. The actual memory usage of the 64bit VM is lower than 32bit.