We Keep Coding

Instructing JVM to use memory like normal processes do?

I'm totally fed up and disgusted of having to guess a good value for the -Xmx command-line option, having my applications crash with OutOfMemoryException, having to modify the -Xmx value and having to restart my applications all the time.
Is there a way to make JVM act normal so that it wouldn't require a -Xmx option, and would allocate and free memory directly from the OS just as any normal application would? Is there some GC which is more efficient, aggressively returning memory to the OS when objects are freed?
If I remember correctly, Java has its roots in embedded environments, but has long past grown in popularity and spread to all kinds of systems. Surely there must be a way to do this in the 21st century? There are many use-cases where an application may require anywhere from a few kilobytes to several terabytes of memory, and the cumbersome -Xmx is really getting in the way.
(Reminder to self: Since there are no good answers iteratively try out some other GC-s and random command line options in cargo cult fashion)

Is there a way to make JVM act normal so that it wouldn't require a -Xmx option, and would allocate and free memory directly from the OS just as any normal application would?
That is what it does by default. You only need to set the maximum heap size to indicate at what point you would rather it get an error than use more memory.
Is there some GC which is more efficient, aggressively returning memory to the OS when objects are freed?
I believe the G1 collector in the Oracle JVM is better at this (because it is newer ??)
If I remember correctly, Java has its roots in embedded environments,
It's root was in Java applets. J2ME was used in embedded systems and this is a different release and code base.
the cumbersome -Xmx is really getting in the way.
I usually don't set it myself. When you have 128 GB or more it defaults to 32 GB.

Since there are no good answers iteratively try out some other GC-s and random command line options in cargo cult fashion
An alternative approach is learning how the GCs work and what their performance tradeoffs are and how those various parameters affect them and then choosing them based on that information instead of randomly.
There is extensive documentation on that topic.
Of course you can still use SO answers as a starting point to find options that are likely to result in the outcome you desire, but there's nothing stopping you from then studying up on why they achieve those results.
No need to worship the planes.

Is it possible to override the default settings for the maximum heap size for any Java applet?

I am using a piece of java software written by someone else, but it keeps running out of memory. I have tried to allocate more memory at initialisation of the program (with -Xmx), but it doesn't seem to have any effect.
Is it possible that the writers of the software have hardcoded the maximum heap size, and that therefore the extra memory I'm allocating at initialisation doesn't have any effect? Or should -Xmx always work?

Is it possible that the writers of the software have hardcoded the maximum heap size, and that therefore the extra memory I'm allocating at initialisation doesn't have any effect?
No. The memory size is set before the code is loaded.
Or should -Xmx always work?
It depends what you mean, and how you are using it. For a long time, there was no way to assign extra memory to applets. Around 1.5 (if memory serves) extra options were added to the applet element - one of which was for max. memory. In the Plug-In 2 JRE architecture, applets can be launched with JWS, and can assign memory via the launch file.

Why my application has less heap memory than others?

My question may look naive but I do not know how to formulate it more correctly. The problem is that I create and use large simple type arrays in my application. And I get errors like:
ERROR/dalvikvm-heap(1763): Out of memory on a 7907344-byte allocation.
Yes, it's big enough but task management tools claim that my application is using only 30MB of memory, while other at the same time use 50MB and even 110MB (have seen once) and there is still 190MB of free memory in the system (not system applications, just other ordinary applications I have installed). If all applications are provided with the same heap size at startup how can they grow so big?

The maximum heap size of an Android application will depend on the device it is running on. For early devices the maximum heap size was 16MB but for some later devices it can be 24MB or possibly even 32MB.
This is a property of the Dalvik VM on each device and is not something you can change (without rebuilding Android from source).
You can query the "per-application memory class" with ActivityManager.getMemoryClass() which seems to be a figure which is not entirely unrelated to the Heap Size.
Applications can use memory which isn't on the heap but 100+MB seems like a surprisingly large amount.
If you want to find out about analysing memory usage on Android you can't do better than this Stack Overflow answer by Dianne Hackborn, who is one of the Android developers at Google. In short it says analysing memory usage is very difficult and you should take any figures you have with a pinch of salt.

Did you try to configure your heap size? Use the following options to do this.
-Xms set initial Java heap size
-Xmx set maximum Java heap size
To obtain more help run
java -X
Example: java -Xms20m -xMX100m -cp . MyMain
Good luck.

We all know that java has by default garbage collector system but at some level we need to clear or flush some object.
And you also can set or configure heap size for the JVM.

Java -Xms initial size effects

What is the benefit of setting the -Xms parameter, and having the initial memory larger for example, then the default calculated one (64 MB in my case, according to Java GC tunning:
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#par_gc.ergonomics.default_size)?
Also, is there any good to setting both the initial and maximum memories to same size?
Thanks.

The benefit is that there is a performance penalty when you use up enough of the heap that it has to be resized. If you set it initially to 64MB but it turns out your application under load needs 250MB, when you hit near 64MB the JVM will allocate more heap space and possibly move around some objects and do other book-keeping. This of course takes time.
When your application is under load, you want all resources dedicated to making it run, so this extra work can make the application slower to respond, or even in some instances it can crash if it runs out of memory before the heap is resized.
Sometimes when using a Java app, you'll see instructions like "set Xms and Xmx to the same value". This is done to avoid the resizing altogether, so that your application launches with its heap already as big as it will ever be.

The linked article explains it clear enough:
Default values: -Xms 3670k
-Xmx 64m
[...]
Large
server applications often experience
two problems with these defaults. One
is slow startup, because the initial
heap is small and must be resized over
many major collections. A more
pressing problem is that the default
maximum heap size is unreasonably
small for most server applications.
The rules of thumb for server
applications are:
Unless you have problems with
pauses, try granting as much memory
as possible to the virtual machine.
The default size (64MB) is often too
small.
Setting -Xms and -Xmx to the
same value increases predictability
by removing the most important
sizing decision from the virtual
machine. However, the virtual
machine is then unable to compensate
if you make a poor choice.
In general, increase the memory as you
increase the number of processors,
since allocation can be
parallelized.
You can also be interested in this discussion of the problem.

What is the benefit of setting the -Xms parameter, and having the initial memory larger for example, then the default calculated one
If the initial heap is small and must be resized over many major collections, the startup will be slow.
Also, is there any good to setting both the initial and maximum memories to same size?
Setting -Xms and -Xmx to the same value gives you predictability. This is especially important when sizing the JVM during performance tuning. But the JVM won't be able to compensate any bad decision.
I tend to use the same values for production servers (which are tuned during performance testing).

If it is normal for your application to require more than 64 MB of heap memory, setting Xms to a larger value should improve the application's performance somewhat because the VM would not have to request additional memory as many times.
In a production system I consider setting Xms and Xmx to the same value sensible. It's basically saying "this is the amount of heap memory the VM can get and I'm dedicating it right away".

Dynamically setting maximum heap size for java process

I have a Java program that is launched by a batch file with a line like this:
javaw -Xms64m -Xmx1024m com.acme.MyProgram
However, on some computers the program will not launch and displays the following message:
Could not reserve enough space for object heap. Could not create the Java virtual machine.
The problem seems to be the the maximum size of the memory allocation pool is larger than the computer can handle. Reducing the maximum size of the memory allocation pool from 1024m to 512m seems to resolve the problem.
Is there a way I can determine how much memory is available on the computer ahead of time (from within the batch file) and determine whether to use -Xmx1024m or -Xmx512m in the batch file invocation? Note that this batch file only needs to work on Windows.

Actually the Java VM already does something similar. If you do not specify -Xms or -Xmx, then these values are inferred from the amount of physical memory on the machine. Or at least so says this page.
You could set -Xms to the minimum heap size which makes your application useful, and let Java determine a proper value for -Xmx.

You could take a look at this page for some answers: Get JVM to grow memory demand as needed up to size of VM limit?

If your program functions correctly with a max heap of 512m, I would use that value.
That said I will also check to see if there is a way to do what you're asking as that is an interesting question.

You could execute from your batch file, check the error level on exit and restart at a lower memory if it failed. I'm not sure the error level would work--if it doesn't you could also check how long it took the program to execute... any thing less than 10sec would be a giveaway.
Just a couple comments though--
If you know it doesn't NEED more than 512, you should run a test to ensure that 1024 actually helps. Larger heaps can often make your GC pauses longer and do little else.
If you're pretty sure you'll use a certain amount of ram (say, the heap will easily fill the 512 you are allocating), you should probably set the min to that number. Setting both the min and max to 512 is good if your program allocates a bunch of stuff but is not situational (always uses about the same amount of ram)