I have problem with Heap Size in jMeter and I need to increase the heap size.
I try change in jmeter.bat but in console (when I open gui mode) the displayed info is still default option.
How can I extend the heap size in my jMeter? I need step by step information.
My environment is:
jMeter 5.1.1
Java 8
Windows 10.
When I check in gui mode in JSR223 Listener with groovy script
java.lang.management.ManagementFactory.getRuntimeMXBean().getInputArguments().each {
log.info("Effective JVM argument: " + "$it")
}
In console I see good value what I create in jmeter.bat.
Can I believe in the groovy script info or what is displayed in the console when I open gui mode?
btw. for test I am using non gui mode, but I was be curious which info is true.
Change your script to use println() function
java.lang.management.ManagementFactory.getRuntimeMXBean().getInputArguments().each {
println("Effective JVM argument: " + "$it")
}
Run your JMeter script in command-line non-GUI mode like
jmeter -n -t test.jmx
You should see default JMeter 5.1.1 heap settings: -Xms1g and -Xmx1g
Now set HEAP environment variable to increase the lower limit to 5G and the upper limit to 10G
set "HEAP=-Xms5g -Xmx10g"
Re-run your JMeter script - you should see the updated values:
If you want the change to be permanent you need to amend this line of the jmeter.bat file:
set HEAP=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
to contain whatever values you want
open cmd command line on Windows:
1.jps -l #view jmeter process
2.jinfo -flags pid #jmeter pid
I'm running a Java program inside a Docker container that has a hard memory limit of 4GB. I've set the max heap to 3GB but still the Java program exceeds the limit and gets killed (OOMKilled).
My question is: How can I configure Java to respect the set container limit and throw an OutOfMemoryException instead of trying to allocate beyond the limit and get its ass kicked by the host kernel?
Update: I'm an experienced Java developer and have a fair understanding of the JVM. I know how to set the max heap, but I wonder if anyone knows of a way to set a limit to the total memory that the JVM process claims from the OS.
When a Java application is executed inside a container, the JVM ergonomics (which is responsible for dynamically assign resources based on the host's capabilities) does not know it is running inside a container and it calculates the number of resources to be used by the Java app based on the host that is executing your container. Given that, it does not matter if you set limits to your container, the JVM will take your host's resources as the base for doing that calculation.
From JDK 8u131+ and JDK 9, there’s an experimental VM option that allows the JVM ergonomics to read the memory values from CGgroups. To enable it you must pass the following flags to the JVM:
-XX:+UnlockExperimentalVMOptions and -XX:+UseCGroupMemoryLimitForHeap
If you enable these flags, the JVM will be aware that is running inside a container and will make the JVM ergonomics to calculate the app's resources based on the container limits and not the host's capabilities.
Enabling the flags:
$ java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -jar app.jar
You can dynamically pass the JVM options to your container with ENV variables.
Example:
The command to run your app would like something like:
$ java ${JAVA_OPTIONS} -jar app.jar
And the docker run command needs to pass the ENV variable like this:
$ docker run -e JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap" myJavaImage
Hope this helps!
In addition to Fabian Rivera's answer I've found that Java 10 has good support for running in containers without any custom startup parameters. By default it uses 25% of the containers memory as heap, which might be a bit low for some users. You can change this with the following parameter:
-XX:MaxRAMPercentage=50
To play around with Java 10 run the following docker command:
docker run -it --rm -m1g --entrypoint bash openjdk:10-jdk
It will give you a bash environment where you can run executables from the JDK. For instance, to run a small piece of script you can use jrunscript like this:
jrunscript -e "print(Packages.java.lang.Runtime.getRuntime().maxMemory()/(1<<20) + 'M')"
This will show you the size of the heap in MB. To change the percentage of total container memory that is used for the heap add the MaxRAMPercentage parameter like this:
jrunscript -J-XX:MaxRAMPercentage=50 -e "print(Packages.java.lang.Runtime.getRuntime().maxMemory()/(1<<20) + 'M')"
Now you can play around with the sizing of the container and the max percentage of heap.
I am running jMeter from the command line on a Mac. Today it threw an Out of memory, heap space error....
newbie$ sh jmeter.sh
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.awt.image.DataBufferInt.<init>(DataBufferInt.java:41)
at java.awt.image.Raster.createPackedRaster(Raster.java:455)
I know I need to increase the memory allocated to it, but not sure how. I looked at this post Unable to increase heap size for JMeter on Mac OSX and found that jMeter script file in the bin folder it mentions and made the below updates..
HEAP="-Xms1024m -Xmx2048m"
NEW="-XX:NewSize=512m -XX:MaxNewSize=1024m"
But I am still getting the out of memory error. Do I just need to give it more or am I changing in the wrong place? Could it be I need to restart my whole machine?
As far as I understand:
You made changes to jmeter script
You're launching jmeter.sh script
You want to know why the changes are not applied?
If you changed jmeter script why don't you just launch it as ./jmeter ?
If you need to start JMeter via jmeter.sh for any reason, run it as follows:
JVM_ARGS="-Xms1024m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=1024m" && export JVM_ARGS && ./jmeter.sh
See Running JMeter User Manual chapter in particular and The Ultimate JMeter Resource List in general for the relevant documentation.
If you have trouble finding it in the logs, then
You can use
ps -ef | grep jmeter
this may give you the details( Not a mac user, but I thing ps -ef would work)
The other option is to use jvisualvm, it ships already with jdk, so no extra tool is required.Run the visualvm and the jmeter, you can guess the name of the application ( entry of jemter ) on the left pane of visualvm , click on it, and all the jvm details will be available.
After this you can confirm whether jmeter is availed with 2GB max Ram. And increase if needed.
There could be different possible reasons for OutOfMemory Error. If you have successfully changed the allocated memory/heap size and still getting the issue then you can look into following factors:
Listeners: Do not use 'TreeView' and 'TableView' Listeners in actual load test as they consume lot of memory. Best practice is to save results in .JTL file, which can be later used for getting different reports.
Non-GUI Mode: Do not use GUI mode while performing actual load test. Run test from command line.
For more, visit the following blog as it has some really nice tips to solve OutOfMemory issues in JMeter.
http://www.testingdiaries.com/jmeter-out-of-memory-error/
My specs:
-Ubuntu 64bit
-Neo4j 2.0
-32 GB of Ram
-AMD FX-8350 Eight COre Processor
The problem:
I'm making a request to my Neo4j server with the following query:
MATCH (being:my_label_2) RETURN being
And gives me this error:
OutOfMemoryError
Requested array size exceeds VM limit
StackTrace:
java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
java.lang.StringCoding.encode(StringCoding.java:344)
java.lang.String.getBytes(String.java:916)
org.neo4j.server.rest.repr.OutputFormat.toBytes(OutputFormat.java:194)
org.neo4j.server.rest.repr.OutputFormat.formatRepresentation(OutputFormat.java:147)
org.neo4j.server.rest.repr.OutputFormat.response(OutputFormat.java:130)
org.neo4j.server.rest.repr.OutputFormat.ok(OutputFormat.java:67)
org.neo4j.server.rest.web.CypherService.cypher(CypherService.java:101)
java.lang.reflect.Method.invoke(Method.java:606)
org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)
org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)
This works fine with "my_label_1" which returns around 30k results
What I believe is the problem:
I don't have enough memory allocated to my JVM
Attempts made to fix/things I've found online:
I read what the manual says to do
And what the Ubuntu Forums say to do
So I've tried going to my neo4 folder (with cd as usual) and running it with the arguments this way:
sudo bin/neo4j start -Xmx4096M
However that didn't work. When Neo4j starts it does warn me that I might not have enough space with:
WARNING: Max 1024 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Question
I know I'm definitely using the arguments wrong, I honestly don't have much experience with JVM configurations. How should I approach this, am I missing something?
You should put JVM setting into the conf/neo4j-wrapper.conf file. It should look like this:
user#pc:> head -n 7 neo4j-enterprise-2.0.0/conf/neo4j-wrapper.conf
wrapper.java.additional=-Dorg.neo4j.server.properties=conf/neo4j-server.properties
wrapper.java.additional=-Djava.util.logging.config.file=conf/logging.properties
wrapper.java.additional=-Dlog4j.configuration=file:conf/log4j.properties
# Java Additional Parameters
wrapper.java.additional=-XX:+UseConcMarkSweepGC
wrapper.java.additional=-XX:+CMSClassUnloadingEnabled
Note that you can configure different aspects of neo4j via different files, so it's better to read description to every file in that conf/ directory in order to get familiar with what can be done and how exactly.
I'm trying to increase the heap size in java for weka which keeps crashing. I used the suggested line:
> java -Xmx500m -classpath
but I get the following error:
-classpath requires class path specification
I'm not sure what this means. Any suggestions?
What I found was the actual issue was in the file 'RunWeka.ini' in '\Program Files (x86)\Weka-3-6'. I opened it with notepad and in the middle of the file there is a line 'maxheap = 512m'.
I changed the line to read 'maxheap=2000m', saved the file and reloaded weka and this fixed my problems.
I'm not sure if this is the correct way to do it or not but it worked for me.
Run this command in your terminal:
java -Xmx1024m -jar weka.jar
Omit the -classpath option. Use just -Xmx500m option.
So, instead of just:
java weka.core.Instances data/soybean.arff
you do:
java -Xmx500m weka.core.Instances data/soybean.arff
If you run weka via some script (RunWeka.bat for example), then you need to modify that script (with some text editor like notepad).
If you're using Weka 3.8.1 on Windows you can save yourself a lot of trouble by editing the javaOpts parameter. The parameter maxheap isn't used anymore, so you can set javaOpts like this in RunWeka.ini file:
javaOpts= -Xmx1040m
Where 1040m is the amount of memory you want to allocate.
Mind that the file is case sensitive.
There are a lot of ways to set this up, but this is the faster way to get Weka runing on a Windows environment at this version.
Edit: If you want Weka to use more than 1gb on windows, you need to have JDK installed. Regular JRE wont do it.
The official Weka answer (for all operating systems and Weka versions) can be found on http://weka.wikispaces.com/OutOfMemoryException.
In case you are using a recent Weka version on Windows, the answer is:
Modify the maxheap parameter in the RunWeka.ini file.
On Ubuntu i had the same problem
but i solve it by increasing the amount of memory to use for the Java Virtual Machine
run this : weka -m 1024m
You need to specify a classpath after -classpath, similar to the PATH env variable you need to specify the path where Java can find the classes.
The -Xmx500m setting looks fine, except that I would suggest to use 512m.
For Mac OS, you have to edit a configuration file in order to increase the heap size of the Weka UI application.
I am repeating what I wrote in: Is there a workaround to solve "Java heap space" memory error when the max heap value has been already specified?
Quit out of Weka if it is running.
cd into /Applications/weka-XXX.app/Contents , or wherever your weka executable was installed. There will be a file called Info.plist there. It is an XML text file. I suggest you save a copy of it to another location, as you'll need to edit it in the next step.
Open the Info.plist (XML) file in your favorite text editor and look for a block that says "VMOptions". There should be a value that says "-Xmx256M" or something similar that specifies the maximum heap size. You should change that value to something bigger, such as "-Xmx1024M".
Start Weka.
I am running Weka 3.6 in windows. This is what i did.
Go to the Weka installation directory and you will find a RunWeka.bat file. Open this file in a text editor and add -Xmx argument in the java command line.
for instance this sets to 4GB memory,
%_java% -Xmx4096m -classpath . RunWeka -i .\RunWeka.ini -w .\weka.jar -c %_cmd% "%2"
The official Weka answer is right..But....crucial is to first get rid of all JVM files and install the relevant 32 or 64 bit Java version. Not using the relevant version causes many problems including the impossibility to increase the heap further than 1024m (by changing the ini file).
Weka 3.9.2 also does not has the option of maxheap anymore. RunWeka.ini have the option of javaOpts, So you may change the below to your required memory allocation,
javaOpts=%JAVA_OPTS% ---- > javaOpts= -Xmx1024m
Here 1024m is the customised amount of memory you want to allocate.
The best way to do it using this command
java -Xmx1024m -[weka classifier] -t [training file path]
The answers above are too old (last one is 1 year ago).
I had same issue with my WEKA (version 3.8.1) on Windows 10.
I had a problem to update the heap size , the way I fixed it is by adding an environment variable (under control panel) as follows:
JAVA_OPTS = -Xms30000m -Xmx30000m
Tip: Just ensure that RunWeka.ini is using this environment variable.
In the above example I give WEKA 30GB. It works.
Hope it will be helpful for some people.
You should also see if default thread stack size 20MB is enough. Increase the value to 50MB in the file /Applications/weka-3-8-1-oracle-jvm.app/Contents/Info.plist (on MAC) like below:
<string>-Xss50M</string>
If we are using Weka Workbench CLI or Knowledge explorer we need to
change as below.
As the documentation suggests the runtime parameter should be -Xmx[size_required]m where [size_required] is memory size you intend to keep to avoid memory exception.
Open RunWeka.ini
Define maxheap=[size_required]G
In my case I kept maxheap=4G , One can set like maxheap=4096m and add -Xmx#maxheap# to all the run options at # setups (prefixed with "cmd_") sections next to java commands
like below
cmd_default=javaw -Xmx#maxheap# ...............
cmd_console=cmd.exe /K start cmd.exe ..................
cmd_explorer=java -Xmx#maxheap# .................
cmd_knowledgeFlow=java -Xmx#maxheap#....................
maxheap=4G
Verify the same by restarting Weka and Help>>SystemInfo
If you run weka from the command line but not through java i.e. typing weka into the command line, instead of typing
weka
specify the memory flag
weka -m 1024m
This will specify 1024 megabytes.
If you're running weka via weka.sh, you can directly run it with memory option.
For example,
sh weka.sh -memory 10g
This will increase the heap size to 10Gb (tested using Weka 3.8.4 on Ubuntu 18.04)