We Keep Coding

Java - Why windows dispatcher don't see current memory level my process?

After run my java app, I see the data of windows dispatcher and my java app has memory
figure 1: memory in windows dispatcher
it's 2GB (1988 MB).
And after that I rerun my app and see th NetBeans profiler
figure 2: memory in NetBeans profiler
And NetBeans profiler saw that my java app have 1GB (and +- 100 to 250 MB).
And if I test it this code (use spring boot, use request to my java app)
figure 3: controller of browser request
And i can see this data of memory in browser:
figure 4: memory in browser
I know that this numbers say me numbers of memory which my app used.
And I confused:
NetBeans profiler: 1GB (+- 100-200MB);
Browser: 1GB (+-)
Windows dispatcher: 2GB (+-)
Rerun my app a few times I always show that 3) Windows dispatcher have 2 GB (and 1) and 2) have 1GB) differences are 2 times.
My app have 1 GB and 1 GB something else. I know when I run my app, launch jre, and jre maybe take 1 GB, I read about PermGen and MetaSpace, and still could not understand what exactly takes up so much memory. I know that the task manager displays "memory" as private bytes without a swap file.
And gc don't clear this data cache.
Can you tell the task manager to deceive about the amount of process memory? How can he calculate memory differently, not the same environment with a profiler? Help me figure this out, I will be glad for any information, thanks in advance.

Troubleshooting Java memory usage exception

I'm trying to troubleshoot a Java program that requires increasingly more memory until it cannot allocate any more and then it crashes.
EDIT More information about the program. The program is an indexer going through thousands of documents and indexing them for search. The documents are read from MongoDB and written to MongoDB as well after some processing is performed. During the processing I'm using RocksDB (rocksdb-jni version 5.13.4 from Maven). There is some mentioning in this GitHub issue of RocksDB memory usage growing uncontrollably, but I'm not sure it could be related.
Monitoring the process with visualvm results in the following plot:
but running htop on the machine shows totally different stats:
There is a difference of several GBs of memory that I'm unable to trace the source of.
The program is launched with the following VM arguments:
jvm_args: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=<port> -Djava.rmi.server.hostname=<hostname> -Xmx12G -XX:+UseStringDeduplication
The system has 32GB of RAM and no swap. Out of this 32 GB, ~10GB are always taken by a tmpfs partition, ~8GB by MongoDB and the remaining 12GB are assigned to the program. EDIT The visualvm screenshot above shows 20GB of heap size, because it was from a previous run where I passed -Xmx20G; the behaviour, however, is the same whether I assign 12GB or 20GB to the heap. The behaviour also does not change if I remove the tmpfs partition, freeing 10 more GB of memory: it just takes longer but eventually it will get out of memory.
I have no idea where this memory usage that is not shown in visualvm but appears in htop is coming from. What tools should I use to understand what is going on? The application is running on a remote server, so I would like a tool that only works in the console or can be configured to work remotely, like visualvm.

I always use JProfiler but i hear jetbrains has a good one as well both can connect remotely.
If possible i would try to create a (local) setup where you can freely test it.
in the RocksDB several possible solutions are mentioned, do they work?
RocksDB seems to need some configuration, how did you configure it?
I am not familiar with RocksDB but i see there is some indexing and caching going on. How much data are you processing and what indexes/caching configuration do you have? are you sure this should fit in memory?
as far as i know the memory mismatch is because jni usage is not shown by default by most. There are some flags to improve this NativeMemoryTracking. i can't recall if this will add them to your visualvm overviews as well.
karbos-538 i just noticed this is quite an old issue, i hope this application is already working by now. what part of the question is still relevant to you?

Find java virtual memory resource hog

I'm currently facing a very strange problem. I have written a simple servlet which runs within a self hosting jetty container. This servlet is a logging endpoint for JS scripts. So the script just runs very simple code to log to graylog and some files (managed by a log4j file appender.)
The admin complained to me that the servlet hogs up to 10.5GB of Virtual Memory which caused the whole machine to slow down. This had an impact on the performance of some other monitoring services.
Restarting the servlet fixed the problem temporarly but the question is how can I find and fix spots in the code causing such memory hogging?
Edit:
I start the application with the -Xmx50m switch.
Edit:
The following things have been investigated: I started Eclipse Memory Analyzer and jConsole to have a look into the application while some ruby scripts sent requests. (40 to 70 requests per minute. That's more than the servlet is getting in production at the moment.)
With this setting:
Heap size: 4MB
Running threads average: 19 (peak at 23)
Virtual Memory: 5GB
Restarting the servlet speeded up the server. The only suspicious parameter of the servlet were the 10.5GB Virtual Memory.

Virtual memory doesn't use much resources, only resident memory matters. You can create a process which uses 8 TB of virtual memory and it still has little impact on resources.
On Linux the "simplest" way to check virtual memory is to read /proc/{pid}/mmap even this is pretty cryptic.
I would check the resident memory and this is what really matters, but I suspect it is close to your 10.5 GB if they are complaining ( assuming they know what they are talking, which I wouldn't assume )

how can I find and fix spots in the code causing such memory hogging
Start by searching this site. There are literally thousands of results.
For your specific case, I'd look for the following:
An unreasonably large heap specification, using the -Xmx command-line argument when starting Java. For a simple servlet, you should use maybe 100-200 Mb.
An excessive number of threads. Each thread requires space (2 Mb by default) for its internal stack.
Large memory-mapped files. The way you describe your servlet, you shouldn't be using any of these.

Application memory issue with mac

I face a problem with java application I built in javaFx. It consumes only 2-3% of cpu usage and around 50 to 80 MB of memory in windows. But in mac same application initially starts with 50 mb of memory and continuously increases to 1 GB and uses over 90% of CPU Usage. I found this information when I checked Mac task manager. When I use a java profiler to find memory leaks, the profiler shows memory usage same like window (not more than 100 MB).
I am confused with this behaviour in Mac.
Has anyone encountered this problem before, or am I doing something wrong with my application?

Lots of things possible, but i suspect this: Depending on the memory size and cpu count, the jvm may run in server mode, which causes memory management to be different. Use -server option to force it to be server mode always and compare again.
Can also take heap dumps (jmap -dump) to see what is taking up so much memory, and stack traces (kill -3) to see what is taking up so much cpu.

Alfresco Community on Tomcat starts very slow

We're currently testing out Alfresco Community on an old server (only 1GB of RAM). Because this is the Community version we need to restart it every time we change the configuration (we're trying to add some features like generating previews of DWG files etc). However, restarting takes a very long time (about 4 minutes I think). This is probably due to the limit amount of memory available. Does anybody know some features or settings that can improve this restart time?

As with all performance issues there is rarely a magic bullet.
Memory pressure - the app is starting up but the 512m heap is only just enough to fit the applications in and it is spending half of the start up time running GC.
Have a look at any of the following:
1. -verbose:gc
2. jstat -gcutil
2. jvisualvm - much nicer UI
You are trying to see how much time is being spent in GC, look for many full garbage collection events that don't reclaim much of the heap ie 99% -> 95%.
Solution - more heap, nothing else for it really.
You may want to try -XX:+AggressiveHeap in order to get the JVM to max out it's memory usage on the box, only trouble is with only 1gb of memory it's going to be limited. List of all JVM options
Disk IO - the box it's self is not running at close to 100% CPU during startup (assuming 100% of a single core, startup is normally single threaded) then there may be some disk IO that the application is doing that is the bottle neck.
Use the operating system tools such as Windows Performance monitor to check for disk IO. It maybe that it isn't the application causing the IO it could be swap activity (page faulting)
Solution: either fix the app (not to likely) or get faster disks/computer or more physical memory for the box

Two of the most common reasons why Tomcat loads slowly:
You have a lot of web applications. Tomcat takes some time to create the web context for each of those.
Your webapp have a large number of files in a web application directory. Tomcat scans the web application directories at startup

also have a look at java performance tuning whitepaper, further I would recomend to you Lambda Probe www.lambdaprobe.org/d/index.htm to see if you are satisfied with your gcc settings, it has nice realtime gcc and memory tracking for tomcat.
I myself have Alfresco running with the example 4.2.6 from java performance tuning whitepaper:
4.2.6 Tuning Example 6: Tuning for low pause times and high throughput
Memory settings are also very nicely explained in that paper.
kind regards Mahatmanich