We Keep Coding

BigQueue disk space not clearing

I am using a java persistence Queue named BigQueue, It stores the data in the disk, bigQueue.gc() is used to clear the used disk space. The big queue.gc() is not clearing the used disk space. The disk memory is continuously increasing.
IBigQueue bigQueue = new BigQueueImpl("/home/test/BigQueueNew", "demo1");
for (int i = 0; i < 10000; i++) {
ManagedObject mo = new ManagedObject();
mo.setName("Aravind " + i);
bigQueue.enqueue(serialize(mo));
}
while (!bigQueue.isEmpty()) {
ManagedObject mo = (ManagedObject) deserialize(bigQueue.dequeue());
System.out.println("Key Dqueue ME");
}
bigQueue.close();
// bigQueue.removeAll(); bigQueue.gc();; System.out.println("Big Queue is " + bigQueue.isEmpty() +" Size is "+bigQueue.size());

In case someone is looking at this as well.
If you are using Java 11 on ubuntu, this could be a known issue. Refer to the link below.
Unless it is fixed at the source, you could download the source and fix it yourself.
https://github.com/bulldog2011/bigqueue/issues/39

Sigar ProcCpu gather method always returns 0 for percentage value

I'm using Sigar to try and get the CPU and memory usage of individual processes (under Windows). I am able to get these stats correctly for the system as a whole with the below code :
Sigar sigar = new Sigar();
long totalMemory = sigar.getMem().getTotal() / 1024 /1024;
model.addAttribute("totalMemory", totalMemory);
double usedPercentage = sigar.getMem().getUsedPercent();
model.addAttribute("usedPercentage", String.format( "%.2f", usedPercentage));
double freePercentage = sigar.getMem().getFreePercent();
model.addAttribute("freePercentage", String.format( "%.2f", freePercentage));
double cpuUsedPercentage = sigar.getCpuPerc().getCombined() * 100;
model.addAttribute("cpuUsedPercentage", String.format( "%.2f", cpuUsedPercentage));
This displays the following quite nicely in my web page :
Total System Memory : 16289 MB
Used Memory Percentage : 66.81 %
Free Memory Percentage : 33.19 %
CPU Usage : 30.44 %
Now I'm trying to get info from individual processes such as Java and SQL Server and, while the memory is correctly gathered, the CPU usage for both processes is ALWAYS 0. Below is the code I'm using :
Sigar sigar = new Sigar();
List<ProcessInfo> processes = new ArrayList<>();
ProcessFinder processFinder = new ProcessFinder(sigar);
long[] javaPIDs = null;
Long sqlPID = null;
try
{
javaPIDs = processFinder.find("Exe.Name.ct=" + "java.exe");
sqlPID = processFinder.find("Exe.Name.ct=" + "sqlservr.exe")[0];
}
catch (Exception ex)
{}
int i = 0;
while (i < javaPIDs.length)
{
Long javaPID = javaPIDs[i];
ProcessInfo javaProcess = new ProcessInfo();
javaProcess.setPid(javaPID);
javaProcess.setName("Java");
ProcMem javaMem = new ProcMem();
javaMem.gather(sigar, javaPID);
javaProcess.setMemoryUsage(javaMem.getResident() / 1024 / 1024);
MultiProcCpu javaCpu = new MultiProcCpu();
javaCpu.gather(sigar, javaPID);
javaProcess.setCpuUsage(String.format("%.2f", javaCpu.getPercent() * 100));
processes.add(javaProcess);
i++;
}
if (sqlPID != null)
{
ProcessInfo sqlProcess = new ProcessInfo();
sqlProcess.setPid(sqlPID);
sqlProcess.setName("SQL Server");
ProcMem sqlMem = new ProcMem();
sqlMem.gather(sigar, sqlPID);
sqlProcess.setMemoryUsage(sqlMem.getResident() / 1024 / 1024);
ProcCpu sqlCpu = new MultiProcCpu();
sqlCpu.gather(sigar, sqlPID);
sqlProcess.setCpuUsage(String.format( "%.2f", sqlCpu.getPercent()));
processes.add(sqlProcess);
}
model.addAttribute("processes", processes);
I have tried both ProcCpu and MultiProcCpu and both of them always return 0.0 even if I can see Java using 15% CPU in task manager. The documentation on the Sigar library is virtually non existent but the research i did tells me that i appear to be doing this correctly.
Does anyone know what I'm doing wrong?
Thanks!

I found the issue while continuing to search online. Basically, the sigar library can only retrieve the correct CPU values after a certain time. My issue is that i was initializing a new Sigar instance every time the page was displayed. I made my Sigar instance global to my Spring controller and now it returns correct percentages.

How do I get OS properties in java program?

Hi I need to get the details about operating system Physical memory and cpu usage and other details. I cannot pay any amount for already available APIs. I can use any free APIs or I can write my own API.
I need all the details in the below image.
In the above picture I have to get the following values
Total
Cached
Available
Free
like this all values I need.
For this I have searched a lot and got some hint. I got first value Total physical memory value using the below code.
public class MBeanServerDemo {
public MBeanServerDemo() {
super();
}
public static void main(String... a) throws Exception {
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
Object attribute =
mBeanServer.getAttribute(new ObjectName("java.lang", "type", "OperatingSystem"), "TotalPhysicalMemorySize");
long l = Long.parseLong(attribute.toString());
System.out.println("Total memory: " + (l / (1024*1024)));
}
}
The below is the output for the above program
Total memory: 3293
Please help me . How do I achieve this.
Edit: I have searched a lot on google for solution and I found a lot of posts in stackoverflow.com. But in all these posts people discussed about only memory details. I need all details about Kernal(Paged and Non-Paged) etc. Please refer this post...
My Requirement
Thanks A lot.

can you please look at below API:
SIGAR API - System Information Gatherer And Reporter
https://support.hyperic.com/display/SIGAR/Home
Some examples Usage : http://www.programcreek.com/java-api-examples/index.php?api=org.hyperic.sigar.Sigar

You can use JNA which offers a lot of access to platform specific apis such like win32.dll
JNA on Github

Consider using jInterop for this task on Windows.
To get the total amount of RAM in MB:
public int getRAMSizeMB() throws JIException
{
String query = "Select * From Win32_ComputerSystem";
long size = 0;
Object[] params = new Object[]
{
new JIString(query),
JIVariant.OPTIONAL_PARAM()
};
JIVariant[] res = super.callMethodA("ExecQuery", params);
JIVariant[][] resSet = Utils.enumToJIVariantArray(res);
for (JIVariant[] resSet1 : resSet)
{
IJIDispatch ramVal = (IJIDispatch) JIObjectFactory.narrowObject(resSet1[0].getObjectAsComObject()
.queryInterface(IJIDispatch.IID));
size = ramVal.get("TotalPhysicalMemory").getObjectAsLong();
break;
}
return Math.round((size / 1024) / 1024);
To get the total amount of CPUs:
public int getCpuCount() throws JIException
{
String query = "Select NumberOfLogicalProcessors From Win32_Processor";
Object[] params = new Object[]
{
new JIString(query),
JIVariant.OPTIONAL_PARAM()
};
JIVariant[] res = super.callMethodA("ExecQuery", params);
JIVariant[][] resSet = Utils.enumToJIVariantArray(res);
for (JIVariant[] resSet1 : resSet)
{
IJIDispatch procVal = (IJIDispatch) JIObjectFactory.narrowObject(resSet1[0].getObjectAsComObject()
.queryInterface(IJIDispatch.IID));
return procVal.get("NumberOfLogicalProcessors").getObjectAsInt();
}
return -1;
}
Using these functions as a template, you can look up the other properties/functions via the MSDN website to query other values.

Analyze system memory/cpu stats with Java

I want simple results like Total Memory, Free Memory, CPU Usage,etc on my Unix system.
Currently I am using the shell command top.
Is there any functionality in Java for doing the same thing ?
My goal is to display the results in a web app via Google Gauge Charts. So obviously, it would be much easier for me to go with Java, instead of executing Unix commands from within Java.
Thanks in advance

OperatingSystemMXBean operatingSystemMXBean =
ManagementFactory.getOperatingSystemMXBean();
for (Method method: operatingSystemMXBean.getClass ().getMethods ())
{
method.setAccessible (true);
String methodName = method.getName ();
if (methodName.startsWith ("get")
&& Modifier.isPublic (method.getModifiers ())
&& OperatingSystemMXBean.class.isAssignableFrom (
method.getDeclaringClass ())) {
try
{
System.out.println (
methodName.substring (3) + ": " +
method.invoke (operatingSystemMXBean));
}
catch (Throwable ex)
{
// Ignore
}
}
}
The output is different on different operating systems and versions of Java. Here is output for JDK 1.7 on Linux:
FreeSwapSpaceSize: 2172059648
FreePhysicalMemorySize: 1757573120
CommittedVirtualMemorySize: 26741071872
MaxFileDescriptorCount: 8192
OpenFileDescriptorCount: 4
ProcessCpuLoad: 2.0251808355598717E-10
ProcessCpuTime: 60000000
SystemCpuLoad: 0.03673989450920194
TotalPhysicalMemorySize: 101439979520
TotalSwapSpaceSize: 4294950912
Name: Linux
ObjectName: java.lang:type=OperatingSystem
AvailableProcessors: 12
Arch: amd64
SystemLoadAverage: 1.86
Version: 2.6.32.54-0.3-default
Value SystemLoadAverage is the same as load average reported by top command.

Get JVM to grow memory demand as needed up to size of VM limit?

We ship a Java application whose memory demand can vary quite a lot depending on the size of the data it is processing. If you don't set the max VM (virtual memory) size, quite often
the JVM quits with an GC failure on big data.
What we'd like to see, is the JVM requesting more memory, as GC fails to provide enough, until the total available VM is exhausted. e.g., start with 128Mb, and increase geometrically (or some other step) whenever the GC failed.
The JVM ("Java") command line allows explicit setting of max VM sizes (various -Xm* commands), and you'd think that would be designed to be adequate. We try to do this in a .cmd file that we ship with the application. But if you pick any specific number,
you get one of two bad behaviors: 1) if your number is small enough to work on most
target systems (e.g., 1Gb), it isn't big enough for big data, or 2) if you make it very large, the JVM refuses to run on those systems whose actual VM is smaller than specified.
How does one set up Java to use the available VM when needed, without knowing that number in advance, and without grabbing it all on startup?

You can also use the option:
-XX:+AggressiveHeap
This according to the [documentation][1]:
The -XX:+AggressiveHeap option
inspects the machine resources (size
of memory and number of processors)
and attempts to set various parameters
to be optimal for long-running, memory
allocation-intensive jobs. It was
originally intended for machines with
large amounts of memory and a large
number of CPUs, but in the J2SE
platform, version 1.4.1 and later it
has shown itself to be useful even on
four processor machines. With this
option the throughput collector
(-XX:+UseParallelGC) is used along
with adaptive sizing
(-XX:+UseAdaptiveSizePolicy). The
physical memory on the machines must
be at least 256MB before
AggressiveHeap can be used. The size
of the initial heap is calculated
based on the size of the physical
memory and attempts to make maximal
use of the physical memory for the
heap (i.e., the algorithms attempt to
use heaps nearly as large as the total
physical memory).
[1]: http://java.sun.com/docs/hotspot/gc1.4.2/#4.2.2. AggressiveHeap|outline

The max VM sizes indeed answer to that need (it sets the max value, but the VM will take only necessary, step by step), but if you need several configurations, besides supplying different "cmd" files, I don't really see a way (though i'll search a bit more)
[edit]
How about using a first program/script (or even another java program), which would check the available resources for the system, and then only call your program with the appropriate -Xm, according to what it retrieved from system ?
That way it would adapt to machines, even if you don't know them before. Could be an idea...
[second edit]
Ok, this has been proposed already by skaffman, my bad.

We have a small C application that we use for launching all of our Java applications via JNI. This allows us to:
Have a meaningful process name (esp important under windows)
Have our own icons (again, important for Windows)
Dynamically build classpaths (we parse out the contents of the /lib file to auto-include all jars)
For our apps, we just hard code the heap limit, but you could easily dynamically configure max heap size based on available memory.
This sort of little app is actually pretty easy to do (it's one of the easiest things to do with JNI). A good starting point would be the source for the JDK (there's a sub-folder for java.exe itself that you can use - that's what we did). Most folks are quite surprised to find that java.exe is a little tiny application (< 200 lines of code) that just invokes JNI and passes command line arguments in (heck, even the use of a method called main() is pretty optional once you start launching things yourself).
Here's code that not only starts up the JVM, etc... but also determines the maximum heap space based on available RAM of the computer. This is a lot of code for an SO post, and it's not at all pretty - but this is battle hardened code - it's been used for almost a decade over many hundreds of installs, etc... Enjoy :
#include <windows.h>
#include <jni.h>
#include <string>
#include <sstream>
using namespace std;
#define STARTUP_CLASS "some/path/to/YourStartupClass"
void vShowError(string sErrorMessage);
void vShowJREError(string sErrorMessage);
void vShowLastError(string sErrorMessage);
void vDestroyVM(JNIEnv *env, JavaVM *jvm);
void vAddOption(string& sName);
string GetClassPath(string root);
string GetJREPath();
int getMaxHeapAvailable(int permGenMB, int maxHeapMB);
JavaVMOption* vm_options;
int mctOptions = 0;
int mctOptionCapacity = 0;
boolean GetApplicationHome(char *buf, jint sz);
typedef jint (CALLBACK *CreateJavaVM)(JavaVM
**pvm, JNIEnv **penv, void *args);
boolean PathExists(string &path)
{
DWORD dwAttr = GetFileAttributes(path.c_str());
if (dwAttr == 0xffffffff)
return FALSE;
else
return TRUE;
}
// returns TRUE is there was an exception, FALSE otherwise
BOOL GetExceptionString(JNIEnv* jenv, string &result)
{
jthrowable ex;
if (NULL != (ex = jenv->ExceptionOccurred())) {
// clear exception
jenv->ExceptionClear();
jmethodID gmID = jenv->GetMethodID(
jenv->FindClass("java/lang/Throwable"),
"getMessage",
"()Ljava/lang/String;");
jstring jerrStr = (jstring)jenv->CallObjectMethod(ex,gmID);
// now you can look at the error message string
if (jerrStr != NULL){ // make sure getMessage() didn't return null
const char *errStr = jenv->GetStringUTFChars(jerrStr,0);
result = errStr;
jenv->ReleaseStringUTFChars(jerrStr, errStr);
} else {
result = "null";
}
return TRUE;
} else {
return FALSE;
}
}
BOOL GetJRESystemProperty(JNIEnv *env, string propname, string &propval, string &errmessage)
{
// now check for minimum JRE version requirement
jclass cls = env->FindClass("java/lang/System");
if (cls == NULL){
errmessage = "Unable to interact with Java Virtual Machine - please visit www.java.com and confirm that your Java installation is valid.";
return FALSE;
}
jmethodID mid = env->GetStaticMethodID(cls, "getProperty", "(Ljava/lang/String;)Ljava/lang/String;");
if (mid == NULL){
errmessage = "Unable to obtain Java runtime system properties - please visit www.java.net and confirm that your Java installation is valid.";
return FALSE;
}
jstring propName = env->NewStringUTF( propname.c_str() );
jstring result = (jstring) env->CallStaticObjectMethod(cls, mid, propName);
const char* utfResult = env->GetStringUTFChars( result, NULL );
if (utfResult == NULL){
errmessage = "Unable to obtain Java runtime system property " + propname + " - please visit www.java.net and confirm that your Java installation is valid.";
return FALSE;
}
propval = utfResult;
env->ReleaseStringUTFChars( result, utfResult );
return TRUE;
}
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR szCmdLine, int iCmdShow) {
JNIEnv *env;
JavaVM *jvm;
jint jintVMStartupReturnValue;
jclass jclassStartup;
jmethodID midStartup;
// Path Determination
// --- application home
char home[2000];
if (!GetApplicationHome(home, sizeof(home))) {
vShowError("Unable to determine application home.");
return 0;
}
string sAppHome(home);
string sOption_AppHome = "-Dapplication.home=" + sAppHome;
string sJREPath = GetJREPath();
// --- VM Path
string sRuntimePath = sJREPath + "\\bin\\client\\"; // must contain jvm.dll
string sJVMpath = sRuntimePath + "jvm.dll";
// --- boot path
string sBootPath = sJREPath + "\\lib";
string sOption_BootPath = "-Dsun.boot.class.path=" + sBootPath;
// --- class path
//string sClassPath = sAppHome + "\\lib;" + sAppHome + "\\lib\\" + APP_JAR + ";" + sAppHome + "\\lib\\log4j-1.2.7.jar";
string cpRoot = sAppHome + "\\";
string sClassPath = GetClassPath(cpRoot);
string sOption_ClassPath = "-Djava.class.path=" + sClassPath;
string sOption_JavaLibraryPath = "-Djava.library.path=" + sAppHome + "\\lib";
int maxHeapBM = 768;
int argStart = 1; // the first argument passed in that should be passed along to the JVM
if(__argc > 1){
string maxheapstr = __argv[1];
if (maxheapstr.substr(0, 9).compare("/maxheap=") == 0){
maxheapstr = maxheapstr.substr(9);
maxHeapBM = atoi(maxheapstr.c_str());
argStart++;
}
}
// we now use adaptive max heap size determination - we try for 768MB of heap, but if we don't get it, we can back off and use less instead of failing the launch
// note: we had problems going for 1024 heap at TrueNorth - it would throttle back to 848 and fail with error -4 no matter what I did
int maxHeapMB = getMaxHeapAvailable(62, maxHeapBM);
stringstream ss;
ss << "-Xmx";
ss << maxHeapMB;
ss << "m";
string sOption_HeapSpace = ss.str();
string sOption_PermSize = "-XX:MaxPermSize=62m";
string sOption_HeapDump = "-XX:+HeapDumpOnOutOfMemoryError";
if (strstr(szCmdLine, "/launcher_verbose") != NULL){
string msg = "App Home = ";
msg += sAppHome;
msg += "\nJRE Path = ";
msg += sJREPath;
msg += "\nRuntime Path = ";
msg += sRuntimePath;
msg += "\nClass Path = ";
msg += sClassPath;
msg += "\nHeap argument = ";
msg += sOption_HeapSpace;
msg += "\nPermsize argument = ";
msg += sOption_PermSize;
msg += "\nHeap dump = ";
msg += sOption_HeapDump;
msg += "\njava.library.path = ";
msg += sOption_JavaLibraryPath;
msg += "\nCommand line = ";
msg += szCmdLine;
FILE *f = fopen("launcher.txt", "w");
fprintf(f, "%s", msg.c_str());
fclose(f);
MessageBox(0, msg.c_str(), "Launcher Verbose Info", MB_OK);
}
// setup VM options
// vAddOption(string("-verbose"));
vAddOption(sOption_ClassPath);
vAddOption(sOption_AppHome);
vAddOption(sOption_HeapSpace);
vAddOption(sOption_PermSize);
vAddOption(sOption_HeapDump);
vAddOption(sOption_JavaLibraryPath);
// initialize args
JavaVMInitArgs vm_args;
vm_args.version = 0x00010002;
vm_args.options = vm_options;
vm_args.nOptions = mctOptions;
vm_args.ignoreUnrecognized = JNI_TRUE;
// need to diddle with paths to ensure that jvm can find correct libraries - see http://www.duckware.com/tech/java6msvcr71.html
string sBinPath = sJREPath + "\\bin";
char originalCurrentDirectory[4096];
GetCurrentDirectory(4095, originalCurrentDirectory);
SetCurrentDirectory(sBinPath.c_str());
// Dynamic binding to SetDllDirectory()
typedef BOOL (WINAPI *LPFNSDD)(LPCTSTR lpPathname);
HINSTANCE hKernel32 = GetModuleHandle("kernel32");
LPFNSDD lpfnSetDllDirectory = (LPFNSDD)GetProcAddress(hKernel32, "SetDllDirectoryA");
if (lpfnSetDllDirectory){
lpfnSetDllDirectory(sBinPath.c_str());
}
// load jvm library
HINSTANCE hJVM = LoadLibrary(sJVMpath.c_str());
SetCurrentDirectory(originalCurrentDirectory);
if (lpfnSetDllDirectory){
lpfnSetDllDirectory(NULL);
}
if( hJVM == NULL ){
vShowJREError("Java does not appear to be installed on this machine. Click OK to go to www.java.com where you can download and install Java");
return 0;
}
// try to start 1.2/3/4 VM
// uses handle above to locate entry point
CreateJavaVM lpfnCreateJavaVM = (CreateJavaVM)
GetProcAddress(hJVM, "JNI_CreateJavaVM");
jintVMStartupReturnValue = (*lpfnCreateJavaVM)(&jvm, &env, &vm_args);
// test for success
if (jintVMStartupReturnValue < 0) {
stringstream ss;
ss << "There is a problem with the 32 bit Java installation on this computer (";
ss << jintVMStartupReturnValue;
ss << "). Click OK to go to www.java.com where you can download and re-install 32 bit Java";
vShowJREError(ss.str());
// I don't think we should destroy the VM - it never was created...
//vDestroyVM(env, jvm);
return 0;
}
//now check for minimum jvm version
string version = "";
string errormsg = "";
if (!GetJRESystemProperty(env, "java.specification.version", version, errormsg)){
vShowJREError(errormsg);
vDestroyVM(env, jvm);
return 0;
}
double verf = atof(version.c_str());
if (verf < 1.599f){
string sErrorMessage = "This application requires Java Runtime version 1.6 or above, but your runtime is version " + version + "\n\nClick OK to go to www.java.com and update to the latest Java Runtime Environment";
vShowJREError(sErrorMessage);
vDestroyVM(env, jvm);
return 0;
}
// find startup class
string sStartupClass = STARTUP_CLASS;
// notice dots are translated to slashes
jclassStartup = env->FindClass(sStartupClass.c_str());
if (jclassStartup == NULL) {
string sErrorMessage = "Unable to find startup class [" + sStartupClass + "]";
vShowError(sErrorMessage);
vDestroyVM(env, jvm);
return 0;
}
// find startup method
string sStartupMethod_Identifier = "main";
string sStartupMethod_TypeDescriptor =
"([Ljava/lang/String;)V";
midStartup =
env->GetStaticMethodID(jclassStartup,
sStartupMethod_Identifier.c_str(),
sStartupMethod_TypeDescriptor.c_str());
if (midStartup == NULL) {
string sErrorMessage =
"Unable to find startup method ["
+ sStartupClass + "."
+ sStartupMethod_Identifier
+ "] with type descriptor [" +
sStartupMethod_TypeDescriptor + "]";
vShowError(sErrorMessage);
vDestroyVM(env, jvm);
return 0;
}
// create array of args to startup method
jstring jstringExampleArg;
jclass jclassString;
jobjectArray jobjectArray_args;
jstringExampleArg = env->NewStringUTF("example string");
if (jstringExampleArg == NULL){
vDestroyVM(env, jvm);
return 0;
}
jclassString = env->FindClass("java/lang/String");
jobjectArray_args = env->NewObjectArray(__argc-argStart, jclassString, jstringExampleArg);
if (jobjectArray_args == NULL){
vDestroyVM(env, jvm);
return 0;
}
int count;
for (count = argStart; count < __argc; count++){
env->SetObjectArrayElement(jobjectArray_args, count-1, env->NewStringUTF(__argv[count]));
}
// call the startup method -
// this starts the Java program
env->CallStaticVoidMethod(jclassStartup, midStartup, jobjectArray_args);
string errstr;
if (GetExceptionString(env, errstr)){
vShowError(errstr);
}
// attempt to detach main thread before exiting
if (jvm->DetachCurrentThread() != 0) {
vShowError("Could not detach main thread.\n");
}
// this call will hang as long as there are
// non-daemon threads remaining
jvm->DestroyJavaVM();
return 0;
}
void vDestroyVM(JNIEnv *env, JavaVM *jvm)
{
if (env->ExceptionOccurred()) {
env->ExceptionDescribe();
}
jvm->DestroyJavaVM();
}
void vShowError(string sError) {
MessageBox(NULL, sError.c_str(), "Startup Error", MB_OK);
}
void vShowJREError(string sError) {
MessageBox(NULL, sError.c_str(), "Startup Error", MB_OK);
ShellExecute(NULL, "open", "http://www.java.com", NULL, NULL, SW_SHOWNORMAL);
}
/* Shows an error message in an OK box with the
system GetLastError appended in brackets */
void vShowLastError(string sLocalError) {
LPVOID lpSystemMsgBuf;
FormatMessage( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
GetLastError(),
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPTSTR) &lpSystemMsgBuf, 0, NULL );
string sSystemError = string((LPTSTR)lpSystemMsgBuf);
vShowError(sLocalError + " [" + sSystemError + "]");
}
void vAddOption(string& sValue) {
mctOptions++;
if (mctOptions >= mctOptionCapacity) {
if (mctOptionCapacity == 0) {
mctOptionCapacity = 3;
vm_options = (JavaVMOption*)malloc(mctOptionCapacity * sizeof(JavaVMOption));
} else {
JavaVMOption *tmp;
mctOptionCapacity *= 2;
tmp = (JavaVMOption*)malloc(mctOptionCapacity * sizeof(JavaVMOption));
memcpy(tmp, vm_options, (mctOptions-1) * sizeof(JavaVMOption));
free(vm_options);
vm_options = tmp;
}
}
vm_options[mctOptions-1].optionString = (char*)sValue.c_str();
}
/* If buffer is "c:\app\bin\java",
* then put "c:\app" into buf. */
jboolean GetApplicationHome(char *buf, jint sz) {
char *cp;
GetModuleFileName(0, buf, sz);
*strrchr(buf, '\\') = '\0';
if ((cp = strrchr(buf, '\\')) == 0) {
// This happens if the application is in a
// drive root, and there is no bin directory.
buf[0] = '\0';
return JNI_FALSE;
}
return JNI_TRUE;
}
string GetClassPath(string root){
string rootWithBackslash = root;
if (rootWithBackslash[rootWithBackslash.length()-1] != '\\')
rootWithBackslash += "\\";
string cp = rootWithBackslash + "classes\\"; //first entry in the cp
string libPathWithBackslash = rootWithBackslash + "lib\\";
// now find all jar files...
string searchSpec = libPathWithBackslash;
searchSpec = libPathWithBackslash + "*.jar";
WIN32_FIND_DATA fd;
HANDLE find = FindFirstFile(searchSpec.c_str(), &fd);
while (find != NULL){
cp += ";";
cp += libPathWithBackslash;
cp += fd.cFileName;
if (!FindNextFile(find, &fd)){
FindClose(find);
find = NULL;
}
}
return cp;
}
string GetJREPath(){
// first, check for JRE in application directory
char home[2000];
if (!GetApplicationHome(home, sizeof(home))) {
vShowError("Unable to determine application home.");
return 0;
}
string sJREPath(home);
sJREPath += "\\jre";
if (PathExists(sJREPath)){
return sJREPath;
}
/* - don't check JAVA_HOME - it may be incorrect...
// next, check the JAVA_HOME environment variable
GetEnvironmentVariable("JAVA_HOME", home, sizeof(home));
sJREPath = home;
if (PathExists(sJREPath)){
return sJREPath;
}
*/
// next, check registry
HKEY hKeyJRERoot;
HKEY hKeyJREInstance;
DWORD dwType;
DWORD dwSize;
BYTE *pData;
string valueName;
string value;
LONG regRslt;
sJREPath = "";
regRslt = RegOpenKeyEx(HKEY_LOCAL_MACHINE, "SOFTWARE\\JavaSoft\\Java Runtime Environment", 0, KEY_READ, &hKeyJRERoot);
if (regRslt == ERROR_SUCCESS){
valueName = "CurrentVersion";
regRslt = RegQueryValueEx(hKeyJRERoot, valueName.c_str(), NULL, &dwType, NULL, &dwSize);
if (regRslt == ERROR_SUCCESS){
pData = (BYTE *)malloc(dwSize);
value = "";
regRslt = RegQueryValueEx(hKeyJRERoot, valueName.c_str(), NULL, &dwType, pData, &dwSize);
if (regRslt == ERROR_SUCCESS){
value = (LPCSTR)pData;
}
free(pData);
if (value != ""){
regRslt = RegOpenKeyEx(hKeyJRERoot, value.c_str(), 0, KEY_READ, &hKeyJREInstance);
if (regRslt == ERROR_SUCCESS){
valueName = "JavaHome";
value = "";
regRslt = RegQueryValueEx(hKeyJREInstance, valueName.c_str(), NULL, &dwType, NULL, &dwSize);
if (regRslt == ERROR_SUCCESS){
pData = (BYTE *)malloc(dwSize);
regRslt = RegQueryValueEx(hKeyJREInstance, valueName.c_str(), NULL, &dwType, pData, &dwSize);
if (regRslt == ERROR_SUCCESS){
value = (LPCSTR)pData;
sJREPath = value;
}
free(pData);
}
RegCloseKey(hKeyJREInstance);
}
}
}
RegCloseKey(hKeyJRERoot);
}
return sJREPath;
}
static const DWORD NUM_BYTES_PER_MB = 1024 * 1024;
bool canAllocate(DWORD bytes)
{
LPVOID lpvBase;
lpvBase = VirtualAlloc(NULL, bytes, MEM_RESERVE, PAGE_READWRITE);
if (lpvBase == NULL) return false;
VirtualFree(lpvBase, 0, MEM_RELEASE);
return true;
}
int getMaxHeapAvailable(int permGenMB, int maxHeapMB)
{
DWORD originalMaxHeapBytes = 0;
DWORD maxHeapBytes = 0;
int numMemChunks = 0;
SYSTEM_INFO sSysInfo;
DWORD maxPermBytes = permGenMB * NUM_BYTES_PER_MB; // Perm space is in addition to the heap size
DWORD numBytesNeeded = 0;
GetSystemInfo(&sSysInfo);
// jvm aligns as follows:
// quoted from size_t GenCollectorPolicy::compute_max_alignment() of jdk 7 hotspot code:
// The card marking array and the offset arrays for old generations are
// committed in os pages as well. Make sure they are entirely full (to
// avoid partial page problems), e.g. if 512 bytes heap corresponds to 1
// byte entry and the os page size is 4096, the maximum heap size should
// be 512*4096 = 2MB aligned.
// card_size computation from CardTableModRefBS::SomePublicConstants of jdk 7 hotspot code
int card_shift = 9;
int card_size = 1 << card_shift;
DWORD alignmentBytes = sSysInfo.dwPageSize * card_size;
maxHeapBytes = maxHeapMB * NUM_BYTES_PER_MB + 50*NUM_BYTES_PER_MB; // 50 is an overhead fudge factory per https://forums.oracle.com/forums/thread.jspa?messageID=6463655 (they had 28, I'm bumping it 'just in case')
// make it fit in the alignment structure
maxHeapBytes = maxHeapBytes + (maxHeapBytes % alignmentBytes);
numMemChunks = maxHeapBytes / alignmentBytes;
originalMaxHeapBytes = maxHeapBytes;
// loop and decrement requested amount by one chunk
// until the available amount is found
numBytesNeeded = maxHeapBytes + maxPermBytes;
while (!canAllocate(numBytesNeeded) && numMemChunks > 0)
{
numMemChunks --;
maxHeapBytes = numMemChunks * alignmentBytes;
numBytesNeeded = maxHeapBytes + maxPermBytes;
}
if (numMemChunks == 0) return 0;
// we can allocate the requested size, return it now
if (maxHeapBytes == originalMaxHeapBytes) return maxHeapMB;
// calculate the new MaxHeapSize in megabytes
return maxHeapBytes / NUM_BYTES_PER_MB;
}

One more option... I work on a launcher called WinRun4J, which allows you to specify a max heap size as a percentage of the available memory on the machine its running on (ie. it does a check for the amount of memory available and sets the -Xmx parameter dynamically on startup).
The INI option is "vm.heapsize.max.percent". There is also another option "vm.heapsize.preferred", which sets the -Xmx parameter as the maximum available memory on the machine up to this amount.
I believe some of the other launchers (eg. Launch4J, Janel) offer the same functionality.

I think you're out of luck :-( The -Xms and -Xmx options don't provide that flexibility.
So I think you will need to wrap your JVM invocation with a script that can determine the maximum amount of memory, and then set -Xmx appropriately (probably a .vbs script using WMI on Windows). Or perhaps it asks the users the first time it's run ?
A bit of a pain, I fear.

I think the easiest way to do this would be to launch the JVM via some wrapper application, which would check system resources to determine memory availability, and then launch the JVM with the appropriate -Xmx parameter.
The question then becomes how that wrapper would be written. It may even be possible for the wrapper app to itself be a JVM, although I don't think the API or system properties would expose the necessary information. Maybe a shell script or your choice could get the information.

if you have a lot of time on your hand you could try the following :
Try to obtain what is the needed memory vs input dataset. With this you can split processing in a different set of classes and create a new JVM process to actually process the data. Basically a Manager and a Worker. The Manager would do a basic analysis on the demanded dataset and spawn a Worker with the appropriate memory requirements. You could probably also set your Manager to be aware of the environment and warn the user when they are trying to operate on a dataset their machine cannot handle.
This is pretty much an extension on the answer provided by skaffman but will happen all within the same app as far as the user is concerned.

There is two options in the virtual machine arguments that can be used : -Xms to set the memory size at startup and -Xmx to set the maximum memory size...
You can set a low startup memory and a big maximum one, so the VM will allocate new memory only if needed.

I do not think either the Sun or IBM JVM can do this (I know that the AS/400 one can, but that is most likely not relevant to you).
I would suggest using Java WebStart (and before you discard this, then notice that it has been updated with Java 6 u 10 and is much better suited for launching "local" applications and applet) since it allows you to provide a "small instance", "larger instance", "gigantic instance" as links/icons.
You will most likely look into the "inject application in webstart cache" and "offline"options.

I don't think you can do what you are trying to do; instead you'll have to ship instructions specific to your customers, their systems and their demands of how they can modify your .cmd file to allow for more memory.
Of course, if your product is aimed at very non-technical users, you may wish to hide this behind some more user-friendly config file. E.g.
# HIGH, MEDIUM, LOW - please change as appropriate. The guidelines are:
# * HIGH - users who generate 500 items per day
# * MEDIUM - 200-500 items etc
memoryUsage=MEDIUM
or possibly deploy different config files depending on which product option a user specifies when they order the product in the first place.

In comments you say that the amount of memory that your application actually depends on the input dataset size provided by the user. This suggests that instead of trying to grab all available virtual memory (which may cause problems for the user's other applications) you should be looking at the input dataset size before you start the JVM and using that to estimate the amount of memory the application will need.
Suppose that the user's machine is configured with modest physical memory and a huge swap space. If you launch the JVM with a huge VM size, it could cause severe "thrashing" as the JVM tries to access data in non-resident pages. By contrast, if you give the JVM something more than the application needs and less than the available physical memory, you should be able to run comfortably without thrashing.

I read through the threads but didn't see anything which indicated that the application had undergone some sort of profiling. Normally I'd profile the apps under certain conditions to find hot spots in performance or memory usage. There's probably things that could be improved in most cases.
If you could establish the limits and understand the behavior of the application you could be in a position to better tell your customers what they can or cannot do with the application thereby reducing the amount of support calls and giving you a better idea of what minimum or maximum heap size to ship the product with.
Maybe you could start with this: http://www.eclipse.org/mat/

Have you looked at running jps to give you the PID for your process and then calling jinfo to change the mx option? Not sure if this will work but it may.
[Edit] This would mean that when you think you have a big dataset, you read the total amount of ram somehow (OS dependent I think. See http://forums.sun.com/thread.jspa?messageID=10306570) or you just increase the size until you don't think it is low anymore (if it blows up first, try to capture and display a helpful message such as "your machine is inadequate, time to make a run to Frys").

This discussion is moot if you think that your clients can ask for 2-3GB of RAM on their 32-bit machine. The OS and other apps will be taking their pound of flesh to run as well.
Sounds like your app is reaching the point where it needs a 64-bit operating system and lots more RAM.