I have a request to install a Linux server (preferably Ubuntu 64bit server),
and Java (64 bit) on the following machine:
Intel Core2Quad Q8200 - 2.33 GHz
8gb DDR2 ram
2x 320GB SATA HDD in soft RAID1 mirror (mirror)
The problem is how to configure system and Java because I need JVM to use more than 4gb of memory.
It cannot be distributed on many virtual machines. There is data more than 4GB large
and it has to be in memory because HDD is slow and performance is critical.
This is a configuration and performance question and I am interested in comments if anyone has experience?
thank you very much for helping me on this...
A 64 bit JVM should have no problem at all with giant heaps, certainly much larger than your available RAM. Just increase the heap size when you start the JVM, for example:
java -Xmx6g
You used to have to specify the 64bit flag (with -d64), but I don't think this is necessary any more.
32bit JVMs can manage something like 3GB of heap.
skaffman's answer which provides the required flag for allocating 6GB of memory is correct. If the 'g' does not do the trick you might want to try 6000m (check the link below for details on flags/platform)
For other options you can find useful information on all available options for the Java HotSpot VM here.
http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp
(Behavioral and performance options are available. Platform specific links also available on this page)
JVM (especially 64bit) does not hesitate taking all the memory, and 4Gb is not a problem. Just install 64-bit Ubuntu and default-jre package will also be 64 bit.
Also take special care about how your data is stored in memory. Again, 64-bit JDK is very hungry for memory due to higher overhead for pointers etc. so if you distribute these 4Gb in small chunks in some data structure, 8Gb will not be enough.
If you install a 64-bit Ubuntu, I believe that
sudo apt-get install sun-java6-jdk
gives you a 64-bit Java.
EDIT: The 64 bit Java can give you as much memory as you need with the appropriate switches. The limit is with the 32-bit JVM's which cannot go over 2-4 Gb depending on operating system.
Related
I have an application that on launch requests a specific amount of RAM using the following command.
java -Xms512m -Xmx985m -jar someJarfile.jar
This command fails to run on my computer with 8.0GB of RAM because it can not create an object heap of the specified size. If I lower the max range to something below 700MB it works fine.
What is even stranger is that even doing a simple java -Xmx768m -version fails when the -Xmx flag value exceeds 700m. I am trying to run it with Java 1.7Uu67 32-bit(that is what the jar was built with) and even newer versions of Java 1.7 and event Java 1.8. I would understand if the max heap was higher and I was using 32bit, but it is not above the ~1.4GB cap of 32-bit java
Is there a configuration parameter that I am missing somewhere that would be causing this, some sort of software that may be interfering? It does not make sense to me as to why I can not allocate 700MB of RAM on a machine with 8.0GB of RAM. I
I should also note that there are no other processes running that are taking up all of my RAM. It is a fresh install of Windows 7.
While 700 MB is pretty low, it is not surprising.
The 32-bit Windows XP emulator in Windows works the same way as Windows XP with all it's limitations. It means you lose 2 GB or a potential 4 GB to the OS. This means programs already running use up virtual memory space. Also if your program uses shared libraries or off heap storage like direct memory and memory mapped files this will means you lose virtual memory for the heap. Effectively you are limited to 1.4 GB of virtual memory for your applications no matter how much memory you actually have.
The simple way around this it to use the 64-bit JVM which runs in your 64-bit OS and is also limited but instead to 192 TB of virtual memory on Windows.
You should try using a 64 bit Java Runtime. It is probably the case that there is no 985 MB large one-piece memory chunk free within the 32-bit address space of your computer (the 32 bit address space 4GB). When you use a 64 bit Java Runtime, Java can allocate the memory within the 64 bit address space, in which the free memory is much more likely to be available.
It doesn't matter that your JAR file was built using a 32 bit version.
The answer to your question may lie in the fact that Windows tries and fails to find a contiguous block of memory that is large enough: see http://javarevisited.blogspot.nl/2013/04/what-is-maximum-heap-size-for-32-bit-64-JVM-Java-memory.html. (Though this suggests that other processes are hogging memory, which seems to be contradicted by your last remark.)
I have defined a jnlp file with initial-heap-size="512m" max-heap-size="1024m" on a machine that has 16Gb with 12Gb available. The JVM running is a 32-bit JVM because of native libraries. I understand that I must have 1Gb of contiguous memory available to allocate that max. If I reduce the max-heap-size to 768, then it runs as normal, and sometimes I don't need to reduce it.
Two questions:
Why is the machine checking max-heap-size initially before the JVM starts up? Are there assertions that are being performed?
Why would I not be able to allocate the full 1Gb from the get go if I have 12Gb available - assuming that there is a contiguous 1Gb block available?
If you can't use the 64-bit JVM...
If on Windows, using 32-bit JVM, you need to research "large address aware" (JVM compiled/linked with /LARGEADDRESSAWARE option). It will allow you to use larger memory footprint with 32-bit. You can set the bit on a particular executable.
Drawbacks of using /LARGEADDRESSAWARE for 32 bit Windows executables?
If on Linux, look at your system hard or soft limits. That may be limiting the max size of your process. Also, you may have process control groups.
For ulimits, on Linux/UNIX try
ulimit -a
Specifically look at ulimit -m or -v settings. If those are unlimited, you may be experiencing another type of control mechanism.
Partial educated guess to:
Why would I not be able to allocate the full 1Gb from the get go if I have 12Gb available - assuming that there is a contiguous 1Gb block available?
You wrote that JVM running is a 32-bit JVM because of native libraries. Note that then those native libraries are part of your process, and even if they don't allocate huge amounts of memory, they may nevertheless fragment the 2 GB virtual address space (which you're limited to, without LARGESPACEAWARE) so that you may not have a contiguous 1 GB block left. You might want to study the the native libraries' base addresses and rebasing. There's a free tool called VMMap, which is great for studying these issues.
I have a question that bothered me after reading an article on analyzing thread dumps. There was one paragraph which mentioned that the logical maximum heap size in a 32-Bit JVM is 4GB.
This link states that the maximum heap size on a 32-bit Windows machine will be around 1.4 - 1.6 GB.
My question is, say if you have RAM of say around 8GB, does this mean i can only utilize 1.4-1.6 GB of it if i were to you a 32-bit JVM? And what will be the maximum size allowed for a 64bit JVM?
Appreciate your help regarding this as i am confused on the same.
specifically on windows, the reason is a combination of the implementation of hotspot (the sun/oracle JVM) and windows dlls.
32-bit code has access to a 4GB virtual address space (there are extensions that allow more but i wont be going into those).
on 32-bit windows the upper 2GB of this virtual address space are reserved for operating sysem use (some versions of the OS accept the /3GB flag as a boot parameter to allow for 3GB of user-accessible space).
also, any libraries (*.dlls) you use are mapped into parts of this address space. by default the windows base *.dll files are loaded at the ~1.6 GB mark (differs slightly by OS version and patch level)
on top of all this, the hotspot JVM only supports allocating a single, continuous, chunk of memory for use as heap space.
so, if you try and picture this in your head, you'll see that you have a free area of ~2GB with a "wall" of windows *.dlls loaded at ~1.6GB. this is the logic behind that figure. it also means that even if you provide the /3GB flag the sun/oracle JVM will not be able to make use of it. some other VMs are better at handling a fragmented heap - like the jrockit VM
you could also try rebasing windows dlls so that they load into higher memory addresses and squeeze some more usable heap space, but the process is fragile.
also note that its very possible that drivers/applications loaded on a particular machines (like anti virus software) will inject their own *.dlls into a java process, and those dlls can be loaded at ever lower memory addresses, further shrinking your usable heap space.
on 64bit versions of windows the addressable limit is 8-128TB and the physical limit stands at 64TB right now
2^32 = 4GB is the max total memory you can address with 32 bits.
The JVM only gets 1.4-1.6GB on 32 bit machines because you still have to accomodate the operating system.
2^64 = (2^32)^2 is the max total memory you can address with 64 bits. As you can see, it's a much larger number.
jvm and also os use paging memory managment system that obtain 4G virtual memory for us in
32bit os systems
but if you have 8G ram you must use 64 bit version of os for maximum performance of os
It depends on your operating system, 32 bit versions of MacOS X and Linux have some ability to access more than 4GB in the kernel but still limit processes to 4GB. Other operating systems may restrict process memory further since they need part of the 4GB for themselves. In general you want to avoid swapping out your JVM to VM so you need to know how much free memory you system has.
Years ago, I tried 64-bit JDK but it was really buggy.
How stable would you say it is now? Would you recommend it? Should I install 64-bit JDK + eclipse or stick to 32-bit? Also, are there any advantages of 64-bit over 32-bit other than bypassing the 4 GB memory limit?
Only begin to bother with that if you want to build an application that will use a lot of memory (namely, a heap larger than 2GB).
Allow me to quote Red Hat:
The real advantage of the 64-bit JVM is that heap sizes much larger than 2GB can be used. Large page memory with the 64-bit JVM give further optimizations. The following graph shows the results running from 4GB heap sizes, in two gigabyte increments, up to 20GB heap.
That, in a pretty graph:
See, no big difference.
See more (more graphs, yay!) in: Java Virtual Machine Tuning
I think the answer can be found pretty simple.
Answer the question: "Do I need more than 4GB of RAM?".
A 64 bit JVM is as stable as a 32 bit JVM. There are no differences. In fact a Java Application running in a 64 bit JVM will consume more RAM compared to a 32 bit JVM. All internal datastructures will need more RAM.
My eclipse is running in a 64bit JVM.
Are you going to deploy to a 32 or a 64 bit environment? If you're affinitized to an environment in production then your development environment should use the same environment type.
If you're platform agnostic, go with x64 and don't even think about it. The platform is mature and stable. It gives you tremendous room to scale up as you can just add memory and make your heaps bigger.
Nobody wants to tell a CEO, "Sorry, we chose x86 and can't scale up like we hoped. It's a month long project project to retest and replatform everything for x64."
The only differences between 32-bit and 64-bit builds of any program are the sizes of machine words, the amount of addressable memory, and the Operating System ABI in use. With Java, the language specification means that the differences in machine word size and OS ABI should not matter at all unless you're using native code as well. (Native code must be built to be the same as the word-size of the JVM that will load it; you can't mix 32-bit and 64-bit builds in the same process without very exotic coding indeed, and you shouldn't be doing that with Java about.)
The 64-bitter uses 64-bit pointers. If you have 4GB+ RAM, and are running Java programs that keep 4GB+ of data structures in memory, the 64-bitter will accommodate that. The big fat pointers can point to any byte in a 4GB+ memory space.
But if your programs use less memory, and you run the 64-bit JVM, pointers in will still occupy 64 bits (8 bytes) each. This will cause data structures to be bigger, which will eat up memory unnecessarily.
I just compiled a MQTT client in both the 32-bit JDK (jdk-8u281-windows-i586) and the 64-bit JDK (jdk-8u281-windows-x64). The class files produced had matching MD5 checksums.
FYI, it's perfectly safe to have multiple JDKs on your system. But if the version you use is important, you should be comfortable with setting your system path and JAVA_HOME to ensure the correct version is used.
I want to download Bea JRockIt 5 because the JRE doesn’t seem to be able to allocate VM more than 1.6 GB.
please i not find the link for download the JRockIT jre
I use machine 32 bits windows7 and ram 3GB.
Thanks in advance
The bottom line is that the limit you are seeing is not actually a JVM imposed limit. So changing to JRockit is not going to help.
Most 32-bit Windows operating systems limit the virtual memory for a user application (such as a JVM) to 2Gb. However, there is a boot switch for Windows Server 2003 that enables applications up to 3Gb ... on hardware that supports PAE : http://msdn.microsoft.com/en-us/windows/hardware/gg487508.aspx.
Your alternatives are:
If you are running Windows Server 2003 - do what the link says.
Switch to 32 bit Linux or 32 bit Solaris. Both allow more than 2Gb for user applications.
Switch to a 64 bit operating system.
I don't know why a simple Google search didn't suffice, but this is clearly available on OTN.
As far as allocation of memory for the JVM is concerned, it is unlikely that you will be able to get any amount more than 2GB. See this related question and the most voted answer.