We Keep Coding

What is effective Xmx value on my redhat machine running tomcat

I am running tomcat on RHEL 7 machine with 1GB RAM. I have setup tomcat and java both to have Xmx=1G and below statements support that,
[root#ip-172-31-28-199 bin]# java -XX:+PrintFlagsFinal -version | grep
HeapSize Picked up _JAVA_OPTIONS: -Xmx1g
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 16777216 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 1073741824 {product} openjdk version "1.8.0_161"
and
tomcat 2799 1 1 02:21 ? 00:00:07 /usr/bin/java
-Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Xmx1024M -Dignore.endorsed.dirs= -classpath /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar
-Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start
But when I get exception, I get following message,
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 244043776 bytes for committing reserved memory.
I know java can never claim 1GB memory as that is the total memory of the machine. but why I am getting error with this size mentioned?

Try adding -Xms1g too, so it initially allocates all the memory, and you'll find that it cannot even start Tomcat.
If you want to squeeze as much memory into Tomcat as possible (not recommended), slowly reduce both numbers (same value for mx and ms) until Tomcat starts.
That is the absolute maximum you can give Tomcat, but you shouldn't do that. Java may still need more as it runs, and the OS will need more occasionally, so you should give Tomcat less than that absolute maximum.
Now that you've found the number, you can leave -Xms undefined again, if you want to.

There is an interesting post here that suggests disabling OOPS.
Are you running a physical server or a VM ?
I agree a 1G server is under-sized, you should run Xmx=512M and allow some swappiness (vm.swappiness = 60 is the default, which should be OK for a small Tomcat)

Elasticsearch fails to start on EC2 AMI

Executed the command: sudo service elasticsearch restart
Stopping elasticsearch: [FAILED]
Starting elasticsearch: [ OK ]
[ec2-user#ip-10-237-177-6 bin]$ Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000736260000, 2312765440, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 2312765440 bytes for committing reserved memory.
An error report file with more information is saved as:
/tmp/hs_err_pid11958.log
Checked the Java Version:
[ec2-user#ip-10-237-177-6 bin]$ java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
Java config for memory:
[ec2-user#ip-10-237-177-6 bin]$ java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
intx CompilerThreadStackSize = 0 {pd product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 65011712 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 1038090240 {product}
intx ThreadStackSize = 1024 {pd product}
intx VMThreadStackSize = 1024 {pd product}

When i was allocate new memory in heap in J meter ,Memory Leak was happen

My question is how should i release memory from the heap?
"My c drive had more than 40 GB space but now it showing less than 3 GB"
C:\Windows\system32>java -XX:+PrintFlagsFinal -version | findstr /i "HeapSize PermSize ThreadStackSize"
intx CompilerThreadStackSize = 0
{pd product}
uintx ErgoHeapSizeLimit = 0
{product}
uintx HeapSizePerGCThread = 87241520
{product}
uintx InitialHeapSize := 62914560
{product}
uintx LargePageHeapSizeThreshold = 134217728
{product}
uintx MaxHeapSize := 987758592
{product}
intx ThreadStackSize = 0
{pd product}
intx VMThreadStackSize = 0
{pd product}
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
C:\Windows\system32>
java -XX:+PrintFlagsFinal -Xms512m -Xmx1024m -Xss512k -XX:PermSize=64m -XX:MaxPermSize=128m -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
when after allocating memory in heap i was run program using jmeter each and every time its allocate space in c drive..#Peter Lawrey

When your disk space fills up you need to find the files you don't need and clean them up.
If you have produced a heap dump I suggest you look for a large file in the directory where your program ran.
BTW When a program exits, all the resource it used are freed up except any files it leaves behind.

Use TreeSize Free tools and open it with Administrator and search java_.hprof files fie from there and go to directory and delete those file from there!!!

Monitoring java 8 processes to see actual memory use

I would like to monitor java 8 processes on a Linux Centos system and be able to
see how much memory they are using. I would prefer to use shell utilities as I wish to eventually feed the output into our central monitoring system
I am also baffled by the -Xmx options in java8 which seem to have no effect on the amount of memory used
Here is a script to get ps info on java programs
for jvm in `sudo -u services jps|awk '/.jar/ {print $1}'`; do ps up $jvm;done
output
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
services 1409 0.0 10.8 4672536 872440 ? Tl Jun03 3:53 /usr/java/lates
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
services 1574 0.0 11.4 4743036 925536 ? Sl Jun03 3:13 /usr/java/lates
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
services 19839 0.7 10.9 4735664 881420 ? Sl 09:43 0:15 /usr/java/lates
Here are some settings from the programs
sudo -u services jps -lvm
1409 /usr/local/content-adapters/pmc-adapter/pmc-adapter.jar -Xmx1024m -Xms512m
1574 /usr/local/content-adapters/tools-adapter/tools-adapter.jar -Xmx512m
20685 sun.tools.jps.Jps -lvm -Dapplication.home=/usr/java/jdk1.8.0_65 -Xms8m
19839 /usr/local/content-adapters/cas-adapter/cas-adapter.jar
some info on the server defaults
java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
intx CompilerThreadStackSize = 0 {pd product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 130023424 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 2063597568 {product}
intx ThreadStackSize = 1024 {pd product}
intx VMThreadStackSize = 1024 {pd product}
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
If I use jstat, I can't make the various "use" numbers add up to either the 4.5GiB
VSZ numbers above or the 800-900MiB RSS
My questions are
how do I relate the -Xmx and other settings to the amount of actual memory used?
how do I use jstat to show the amount of memory in use?
presumably the difference in VSZ and RSS is due to some kind of shared resources that all the java programs use. How can I estimate or find this metric?

You shouldn't pay much attention to virtual memory unless you set vm.overcommit_memory = 2. Reserved-but-not-allocated address-space and memory-mapped files count towards that, so the metric is nearly useless since java reserves the entire heap in advance to have a continuous region available.
RSS is generally more interesting and assuming no paging happens should be somewhat larger than the currently allocated java heap, which in turn may be smaller than the maximum java heap (-Xmx)
Assuming the maximum heap capacity is not used and only small amounts of memory-mapping or direct byte buffer allocations happen and no paging happens the relation is something like this:
Non-garbage Objects < Java Heap used < Java Heap allocated < RSS < Java heap reserved (Xmx) < VSZ

Additionally Java will use memory for metaspace. Also, depending on your code, java may use additional memory for byte buffer, direct memory.

Increasing Heap Size on Linux Machines

I work on Ubuntu desktop machine and I'd like to increase heap size for Java. The RAM is 16GB and the current Max Heap Size is 3GB
I checked this post post Increasing Tomcat Heap Size
Not much found about Ubuntu, so I tried this command:
java -Xmx10000m -X2000m -XshowSettings:all
and the result is:
Min Heap Size: 1.95G
Max Heap Size: 9.77G
then sudo gedit /etc/tomcat7/default
and changed this lino to:
JAVA_OPTS="-Djava.awt.headless=true -Xmx10000m -XX:MaxPermSize=2000m"
but then I restarted the machine an checked the max size using:
java -XshowSettings:all
and this shows:
Max Heap Size (Estimated): 3.80GB
I wanted to take advantage of the high RAM i got (16 GB). Is there anything else i can do?

Changing Tomcat config wont effect all JVM instances to get theses settings.
This is not how it works, the setting will be used only to launch JVMs used by Tomcat, not started in the shell.
Look here for permanently changing the heap size.

You can use the following code snippet :
java -XX:+PrintFlagsFinal -Xms512m -Xmx1024m -Xss512k -XX:PermSize=64m -XX:MaxPermSize=128m
-version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
In my pc I am getting following output :
uintx InitialHeapSize := 536870912 {product}
uintx MaxHeapSize := 1073741824 {product}
uintx PermSize := 67108864 {pd product}
uintx MaxPermSize := 134217728 {pd product}
intx ThreadStackSize := 512 {pd product}