I have java heap size setting max/min defined as 8 GB . System memory is 14 GB .There is no other java process running other than tomcat . I see java using approx 12 GB and hence system goes down . What kind of other things which are consuming more than 4 GB outside of java heap
permsize is 256 mb only .
I got the answer . Our application was using bytearray which consumes outside heap and hence the difference in behavior .
For a 32 bit JDK, Total virtual memory address is 4G size. Ignoring Kernel space at MAX 3G can be used by java and native heap. In-case of 64 bit JDK virtual memory address is infinite. System memory you mentioned is RAM size. System memory and virtual memory are different. Java heap will not expand than specified -Xmx value. If Xmx is set to 8GB it will not expand further when it reaches 8GB and if application did not find space for allocation then java will throw OutOfMemory exception.
Related
In Java 8, meta space is allocated out of native memory , But i did not get anywhere on net what is native memory ? Per this link it is the memory available to the OS but at Difference between Metaspace and Native Memory in Java , native memory is also shown as part of memory given to JVM process
Example :-
If yes consider the case where i have 15 GB ram on windows OS. I have just one process (Java process) running on machine with -Xmx 4GB.
Does it mean OS can use up to (15-4)=11 GB out of which meta space memory will be allocated ?
Is metaspace allocated out of native memory?
Yes.
Definitive source: https://blogs.oracle.com/poonam/entry/about_g1_garbage_collector_permanent
But i did not get anywhere on net what is native memory ?
The native heap is the malloc / free heap that provides dynamic memory for those parts of the JVM that are implemented in native code (C++). It can also be used by user-supplied native libraries loaded by the JVM. The native heap is not garbage collected per se, but metaspace is.
One benefit of using the native heap to hold metaspace objects is that the native heap does not have a fixed maximum size (by default) like the Java heap does.
If yes consider the case where i have 15 GB ram on windows OS. I have just one process (Java process) running on machine with -Xmx 4GB. Does it mean OS can use up to (15-4)=11 GB out of which meta space memory will be allocated?
Maybe:
There will be other process on a Windows machine. Lots of them. It is just that they are system processes.
There possibly are OS enforced limits on how big the Java process is allowed to grow. (I'm assuming that Windows has something that fills the role of ulimit on a UNIX / Linux system.)
If there is disk space available for paging, the OS may actually allocate the Java process more memory than is available as physical memory pages.
Native memory is the normal memory of an application. This is apposed to heap memory which is managed by the JVM. In a C program for example, it would be called just "memory"
I have installed 6GB ram on my Windows and 5GB memory free space and when I run a Java program with specifying Java heap space -Xmx2048m It's working fine. My question is, what if I have this on my Linux box
cat /proc/meminfo
MemTotal: 10.130464 kB (10.1gb)
MemFree : 248.736 kB (248mb)
What will happen if I run the Java program with -Xmx2048m in Linux with this memory?
Java will allow the heap to grow to the Xmx value, but it won't necessarily start at or need to use that much. Xms specifies the minimum heap size, which is how much heap memory Java will allocate right at the start. Java will allocate additional memory for Perm Gen. Any program, not just Java, will begin to fail if it needs more memory when there is none available. Java in particular throws OutOfMemoryError when either Xmx is reached or there is no available memory on the machine.
MemFree is the amount of physical memory left, but that is not the total amount of available memory. When physical memory is full, the operating system will use swap space (SwapTotal and SwapFree). Swap space is usually on the hard disk and therefore very slow. Linux would try to keep the most frequently used memory blocks in physical memory to maintain performance, but usually there is some slowdown. It's likely the Linux box is already using some swap space considering there is a relatively small amount of free physical space left. Combine MemFree and SwapFree to get your total available memory.
Edit: With 1.8GB total physical plus swap free, you would be able to start Java because it initially uses Xms (min heap) plus PermGen memory, which defaults to 220MB on 10GB of physical ram (1/64 physical memory plus 64MB PermGen). As your program runs, it could use the rest of your available system memory, but only if your program actually needs it. If this program is not resource intensive, it will likely stay in the lower end. Typically you should set Xmx so that it doesn't use all your memory though. You may want to just close some other programs if your memory usage is that high.
I am trying to set the Xmx parameter when starting up a program. If I set it to 1408M, the JRE starts up fine. If I set it to 1536M, I get
"Could not create the java virtual machine".
I understand that it's trying to reserve consecutive memory space, but the machine I'm running on has 16GB of RAM and 13GB of that is currently free. The program I'm running is running out of heap space and crashing on me. Is there anything I can do to fix this?
Use a 64bit JVM. The 32bit JVM is limited (depending on the OS) to at most 3 GByte (on linux I have a limitation of about 1.5 GByte).
32-bit JVMs are limited to roughly 1.5 GB of heap space due to addressing constraints and the need for memory for other reasons. On Windows 2 GB is assigned to the process, and 0.5 is used for non-heap memory. If you can use PAE on Windows Server or possibly Linux, you can address up to 3 or 4 GB, respectively.
Otherwise use a 64-bit JVM.
I have a question regarding memory usage of java web applications..
In Task manager, Tomcat service seems to occupy 8,677,544,000 Bytes of memory
In jvisualvm, memory usage of java application deployed under that tomcat is as follows
Heap:
Used: 2,304,233,184 B
Size: 8,589,934,592 B
Max: 10,737,418,240 B
Permgen:
Used: 80,272,056 B
Size: 536,870,912 B
Max: 536,870,912 B
Memory Parameters in Tomcat's Service.bat file:
-Xms8192m;-Xmx10240m;-XX:PermSize=512m;-XX:MaxPermSize=512m
Now, my question is no matter what I set MaxHeapFreeRatio the free space is not shrinking even though the used space is shrinking at times.
Can anyone, kindly tell me why is this behaving like this.. Is there a way to shrink the free space so that other processes runnning on the system can utilize it?
I am using latest versions of JDK 1.7 & Tomcat 7.
With the parameter -Xms8192m you're telling Tomcat (well, the JVM) to allocate at least 8 GiB of heap space on startup.
The JVisualVM stats support that, but also tell you that around 2 GiB is being used by the application.
Reduce the start-up value to a lower value (start at 2 GiB). Note that if the application needs more heap space, you've told it you can use up to 10 GiB -Xmx10240m so it may be worth trimming this value down too (maybe to 4 GiB).
Obviously, if you start to get OOME's, you'll need to increase the values until the application has enough to run happily (assuming no memory leaks etc.).
Also, note that the process size will always be larger than the heap size as it also includes the perm gen space (-XX:MaxPermSize=512m) and internal memory / libraries for the JVM itself.
Basic examples:
-Xms512m;-Xmx1536m;-XX:PermSize=64m;-XX:MaxPermSize=256m
The minimum values here are 512 MiB heap, 64 MiB perm gen, so the minimum OS process size would be around 600 - 650 MiB.
As the application allocates more space up to the max values (1.5 GiB heap, 256 MiB perm gen) I'd expect the process size to reach about 2 GiB.
Now if we use your values:
-Xms8192m;-Xmx10240m;-XX:PermSize=512m;-XX:MaxPermSize=512m
The minimum values here are 8 GiB heap, 512 MiB perm gen, so the minimum OS process size would be around 8.6 GiB - pretty much exactly what you're seeing.
Using the max values, the process size would reach nearly 11 GiB.
I am running a tomcat 6 instance on a large EC2 instance running CentOS 5.4.
The box has 7.5gb of RAM and is dedicated to running tomcat.
I am trying to give the box 6gb of RAM to use for it's max heap. However I keep getting this error:
Invalid maximum heap size: -Xmx6144m
The specified size exceeds the maximum representable size.
As I drop the amount of RAM I give it I start getting this error instead:
Error occurred during initialization of VM
Could not reserve enough space for object heap
Even with the app running (with xmx3000m) when I run free I have this amount free:
[tomcat#producer1:/usr/share/tomcat/logs] $free
total used free shared buffers cached
Mem: 7864320 1512736 6351584 0 179948 702352
-/+ buffers/cache: 630436 7233884
Swap: 0 0 0
The most I am able to give it is 3000m. This seems unreasonably small. Anyone have any ideas?
Thanks
Do you have 64 bit JDK installed? If you are using a 32 bit JDK it can max access 4GB (theoretically).
As you are planning for 6GB, you require 64bit JDK. Added to that as you have 7.5G, 6GB is a tight no. CentOS requires some memory for kernal, processing other services, SWAP memory from this.
So give a trial & error by increasing heap size from 5G onwards. JDK requires additional memory beyond heap like permgenspace (which is typically 128M - 512M depending on how many libraries/classloaders you have)
Refer to : http://benjchristensen.com/2007/02/16/32-bit-versus-64-bit-jdk-memory-usage/