We Keep Coding

Jmeter, how to increase heap size

I had read that you need to change the heap size in the jmeter.bat file (I'm using windows) to increase the memory to be able to test around 500 threads or more. This is the default:
if not defined HEAP (
rem See the unix startup file for the rationale of the following parameters,
rem including some tuning recommendations
set HEAP=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
)
I changed the set HEAP=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m it to:
set HEAP=-Xms2g -Xmx8g -XX:MaxMetaspaceSize=512m
But when I open Jmeter in GUI mode there is a message in the command line window
Modify current env variable HEAP="-Xms1g -Xmx1g -XX:MaxMetaSpaceSize=256m" in the jmeter batch file
So does that mean the change in the batch file didn't work? Or it does work when running in non-gui mode? Did I miss something to change or what? Thanks in advance.

You don't need to increase the heap unless you face java.lang.OutOfMemoryError: Java heap space error or detect unusually high GC activity
The message basically a form of advice, you will see it even if you have several terabytes of heap space as it is simply hard-coded and will always be shown during GUI startup. If you want to see all Java arguments including heap space settings you can use JSR223 Sampler and the following Groovy code:
java.lang.management.ManagementFactory.getRuntimeMXBean().getInputArguments().each {
log.info("Effective JVM argument: " + "$it")
}
This way you can test whether you changes are applied or not:

How to prevent Java from exceeding the container memory limits?

I'm running a Java program inside a Docker container that has a hard memory limit of 4GB. I've set the max heap to 3GB but still the Java program exceeds the limit and gets killed (OOMKilled).
My question is: How can I configure Java to respect the set container limit and throw an OutOfMemoryException instead of trying to allocate beyond the limit and get its ass kicked by the host kernel?
Update: I'm an experienced Java developer and have a fair understanding of the JVM. I know how to set the max heap, but I wonder if anyone knows of a way to set a limit to the total memory that the JVM process claims from the OS.

When a Java application is executed inside a container, the JVM ergonomics (which is responsible for dynamically assign resources based on the host's capabilities) does not know it is running inside a container and it calculates the number of resources to be used by the Java app based on the host that is executing your container. Given that, it does not matter if you set limits to your container, the JVM will take your host's resources as the base for doing that calculation.
From JDK 8u131+ and JDK 9, there’s an experimental VM option that allows the JVM ergonomics to read the memory values from CGgroups. To enable it you must pass the following flags to the JVM:
-XX:+UnlockExperimentalVMOptions and -XX:+UseCGroupMemoryLimitForHeap
If you enable these flags, the JVM will be aware that is running inside a container and will make the JVM ergonomics to calculate the app's resources based on the container limits and not the host's capabilities.
Enabling the flags:
$ java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -jar app.jar
You can dynamically pass the JVM options to your container with ENV variables.
Example:
The command to run your app would like something like:
$ java ${JAVA_OPTIONS} -jar app.jar
And the docker run command needs to pass the ENV variable like this:
$ docker run -e JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap" myJavaImage
Hope this helps!

In addition to Fabian Rivera's answer I've found that Java 10 has good support for running in containers without any custom startup parameters. By default it uses 25% of the containers memory as heap, which might be a bit low for some users. You can change this with the following parameter:
-XX:MaxRAMPercentage=50
To play around with Java 10 run the following docker command:
docker run -it --rm -m1g --entrypoint bash openjdk:10-jdk
It will give you a bash environment where you can run executables from the JDK. For instance, to run a small piece of script you can use jrunscript like this:
jrunscript -e "print(Packages.java.lang.Runtime.getRuntime().maxMemory()/(1<<20) + 'M')"
This will show you the size of the heap in MB. To change the percentage of total container memory that is used for the heap add the MaxRAMPercentage parameter like this:
jrunscript -J-XX:MaxRAMPercentage=50 -e "print(Packages.java.lang.Runtime.getRuntime().maxMemory()/(1<<20) + 'M')"
Now you can play around with the sizing of the container and the max percentage of heap.

Java 8 Memory allocation

Let's assume that I have JDK/JRE 8 also I've added JAVA_OPTS= -Xmx8G -Xms1G.
I've stated without any other params (java -jar 1.jar; java -jar 2.jar) 2 simple jars. So now I have two java process.
1) Do they executed in one JVM (as two java process) or each process will be executed in separated JVM?
2) -Xmx8G -Xms1G heap size will be applied to separately each of two executed jars or this heap size will be something common for two executed jars?
3) What will be with memory allocation if start one jar ( java -jar 1.jar)
and another with java -jar 2.jar Xmx12G?
P.S. Would we nice if you have any good links.
P.S.S Don't Ask what I'm smoking :)

The best way is to analyze JVM instances in your system. Just open JMC(Java Mission Control). Whenever you execute any application it will show as an individual jvm there. ex : if you run jar1 and jar2 the JMC will show three JVMs- jar1, jar2 ,jmc. Then, you can use use MBean Browser there to analyze your all jvm.
Hope it helps.

Memory overflow Java

I think that my Java Program exceeded the usage of memory allowed... this is the error that shows up in Eclipse:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at Main.main(Main.java:228)
before than trying an implementation with better memory handling, is there any way basically to augment the memory to allocate for the program ? I would like to see the program terminating and showing the result at least.

You can allocate more memory to the process by using the -Xmx flag:
java -Xmx2G YourProgram
Will allocate 2 Gigabytes of memory to the process.
You can do this in Eclipse by going to Run Configurations, the Arguments section and adding -Xmx 2G to the VM arguments.

Increasing memory size is good, but you should also consider that probably your program has some memory leak and you really should bother about it.

You can increase heap size using the cmd-line flags
For example:
java -Xmx6g myprogram
You can get a full list (or a nearly full list, anyway) by typing java -X.

As I don't think your program is using that much memory, I suspect your program is having a memory leak somewhere.
If you could give Main.java:200 till Main.java:250 we could check for any leaks.
If you are sure your program is using that amount of memory you can either run
java -Xmx2G -jar foo.jar in the Command Prompt (Windows) or Terminal (Mac OS X or Linux)
or, if you're running your program in Eclipse under Linux, do the following:
1) Make sure Eclipse is closed
2) Open your favorite text editor at your eclipse.ini file (Default location: /usr/lib/eclipse/eclipse.ini)
3) Search for --Xmx512M and increase this number (i.e. --Xmx2G)
4) Save the file and restart Eclipse
For any explaination about --Xmx and --Xms I refer to Eclipse memory settings when getting "Java Heap Space" and "Out of Memory"
Happy coding :) -Charlie

Memory issue when trying to start a minecraft server via a python script

Okay iv been working on a hand full of tool to help maintain a large number a of game servers hosted on a single computer. What i'm am doing is lunching a .bat file from a python script. That bat file sets the servers prams like max memory and things of that nature. I open and run the .bat file with the following function.
def StartServer(path,file):
if file not in MK.keys():
l = Popen(file, cwd=path)
MK[file]=l
stdout, stderr = l.communicate()
else:
MK[file].terminate()
this function is called in one of two ways the
1st way is to start the program.
thread.start_new_thread( StartServer, (path,File, ) )
2nd the close the program
StartServer(path,File)
StartServer see the reentry and terminates the specified program...
and this works great for programs that need very little ram, like a bat file such as
#echo Hello world.
#pause
however when trying to allocate more ram for a java program such as,
#ECHO OFF
SET BINDIR=%~dp0
CD /D "%BINDIR%"
"%ProgramFiles%\Java\jre7\bin\java.exe" -Xmx4096M -Xms4096M -jar Minecraft_RKit.jar user:password
PAUSE
I receive a memory error from the BAT file as follows. this is from the bat file not from the python side
Invalid mat heap size: -Xmx4096M
The specified size exceeds the maximum representable size.
Error: Could not create java virtual machine.
Error: A fatal exception has occurred. Program will exit.
Press any key to continue . . .
Note: I prefer to use both python and the bat file thank you in advance!

You are attempting to start a 32-bit JVM, giving it a heap size that is too large for a 32-bit architecture. The maximum heap size is something less than 3Gb for a 32-bit JVM. The actual limit depends on the OS (which determines how much of the address space is made available to applications) and on how much non-heap memory is used by the JVM.
Either reduce the max heap size, or switch to a 64-bit JVM (and a 64-bit OS).