I created a service which has in total 11 scheduled jobs running. 3 of them are scheduled by a cron job (2 of them every 15min and the last one every minute). These three tasks are only for monitoring the service (Checking ehCache and RAM used by the JVM). All the other scheduled tasks are annotated with the 'fixedDelay' attribute - so a new thread should only be started if the last one is finished and x time passed, right?
With http://ask.xmodulo.com/number-of-threads-process-linux.html I found out, that I can check the number of threads per process by executing
cat /proc/PID/status
This resulted in the following
Name: jsvc
Umask: 0022
State: S (sleeping)
Tgid: 17263
Ngid: 0
Pid: 17263
PPid: 17260
TracerPid: 0
Uid: 99 99 99 99
Gid: 99 99 99 99
FDSize: 8192
Groups: 99 11332 16600 34691 50780 52730 52823 53043 54173
NStgid: 17263
NSpid: 17263
NSpgid: 17260
NSsid: 17260
VmPeak: 35247540 kB
VmSize: 35232620 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 5679220 kB
VmRSS: 5663344 kB
RssAnon: 5660016 kB
RssFile: 3328 kB
RssShmem: 0 kB
VmData: 32106616 kB
VmStk: 1012 kB
VmExe: 44 kB
VmLib: 16648 kB
VmPTE: 50908 kB
VmPMD: 128 kB
VmSwap: 0 kB
HugetlbPages: 0 kB
Threads: 19922
SigQ: 0/64039
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000004
SigIgn: 0000000000000000
SigCgt: 2000000181005ecf
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
Seccomp: 0
Speculation_Store_Bypass: vulnerable
Cpus_allowed: 7fff
Cpus_allowed_list: 0-14
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 5986
nonvoluntary_ctxt_switches: 26
So my first question is: What is the 'Threads' number telling me? Are there 19922 threads INCLUDING the threads which are ended or are this only the currently active threads?
I also wondered why all of this threads are currently in SLEEPING state...
I made a graph (#1) which displays the current number of threads for this process and I can see that the number is not only increasing.
So why is this number so wiggly?
Should the subdirectory of a thread be deleted after the thread has finished?
And what is with threads with state "SLEEPING" - are they finished? Because I have nothing to wait for...
So, I found out that
the number "Threads" excludes every finished thread - so all the threads are running or waiting for something.
That is also the reason why this number is wiggling like that.
After checking my source code again, I found out that some ExecutorService objects are not correctly closed so I correted that and received the following graph (which looks better!)
So when somebody else have similar issues, this is what I did:
Login to the machine where the application is running
Get the correct PID from the process by running
ps -aux | grep -i 'NAME' (replace NAME with the correct name of the application)
Get the number of running/waiting threads by executing cat /proc/[PID]/status
Create the graph-data with for x in {1..100000}; do echo $(date) - $(find /proc/[PID]/task -maxdepth 1 | wc -l); sleep 1; done >> thread_counter.csv
Related
I need to check whether a java process is consuming more paging space in linux and AIX.
virtual memory size
To get just the virtual memory size you can read /proc/self/maps on Linux which has all the address ranges used. Take the different and you will know how much virtual memory is being used for what.
If you want more detail such as the resident size you can read /proc/self/smaps
This gives fine grain detail on every mapping including how much is private, dirty, swapped etc.
00400000-004f4000 r-xp 00000000 08:01 12058626 /bin/bash
Size: 976 kB
Rss: 888 kB
Pss: 177 kB
Shared_Clean: 888 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 888 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Shared_Hugetlb: 0 kB
Private_Hugetlb: 0 kB
Swap: 0 kB
SwapPss: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd ex mr mw me dw sd
list how much swap each process is using (linux )
for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done
UPDATES on Oct 25:
Now I found out what's causing the problem.
1) The child process kills itself, that's why strace/perf/auditctl cannot track it down.
2) The JNI call to create a process is triggered from a Java thread. When the thread eventually dies, it's also destroying the process that it creates.
3) In my code to fork and execve() a child process, I have the code to monitor parent process death and kill my child process with the following line: prctl( PR_SET_PDEATHSIG, SIGKILL ); My fault that I didn't pay special attention to this flag before b/c it's considered as a BEST PRACTICE for my other projects where child process is forked from the main thread.
4) If I comment out this line, the problem is gone. The original purpose is to kill the child process when the parent process is gone. Even w/o this flag, it's still the correct behavior. Seems like the ubuntu box default behavior.
5) Finally found it's a kernel bug, fixed in kernel version 3.4.0, my ubuntu box from AWS is kernel version 3.13.0-29-generic.
There are a couple of useful links to the issues:
a) http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them
b) prctl(PR_SET_PDEATHSIG, SIGNAL) is called on parent thread exit, not parent process exit.
c) https://bugzilla.kernel.org/show_bug.cgi?id=43300
UPDATES on Oct 15:
Thanks so much for all the suggestions. I am investigating from one area of the system to another area. It's hard 2 find a reason.
I am wondering 2 things.
1) why are powerful tools such as strace, auditctl and perf script not able to track down who caused the kill?
2) Is +++ killed by SIGKILL +++ really means its killed from signal?
ORIGINAL POST
I have a long running C process launched from a Java application server in Ubuntu 12 through the JNI interface. The reason I use JNI interface to start a process instead of through Java's process builder is b/c of the performance reasons. It's very inefficient for java process builder to do IPC especially b/c extra buffering introduces very long delay.
Periodically it is terminated by SIGKILL mysteriously. The way I found out is through strace, which says: "+++ killed by SIGKILL +++"
I checked the following:
It's not a crash.
It's not a OOM. Nothing in dmesg. My process uses only 3.3% of 1Gbytes of memory.
Java layer didn't kill the process. I put a log in the JNI code if the code terminates the process, but no log was written to indicate that.
It's not a permission issue. I tried to run as sudo or a different user, both cases causes the process to be killed.
If I run the process locally in a shell, everything works fine. What's more, in my C code for my long-running process, I ignore the signal SIGHUP. Only when it's running as a child process of Java server, it gets killed.
The process is very CPU intensive. It's using 30% of the CPU. There are lots of voluntary context switch and nonvoluntary_ctxt_switches.
(NEW UPDATE) One IMPORTANT thing very likely related to why my process is killed. If the process do some heavy lifting, it won't be killed, however, sometimes it's doing little CPU intensive work. When that happens, after a while, roughly 1 min, it is killed. It's status is always S(Sleeping) instead of R(Running). It seems that the OS decides to kill the process if it was idle most of the time, and not kill the process if it was busy.
I suspect Java's GC is the culprit, however, Java will NEVER garbage collect a singleton object associated with JNI. (My JNI object is tied to that singleton).
I am puzzled by the reason why it's terminated. Does anyone has a good suggestion how to track it down?
p.s.
On my ubuntu limit -a result is:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7862
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7862
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I tried to increase the limits, and still does not solve the issue.
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) unlimited
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Here is proc status when I run cat /proc/$$$/status
Name: mimi_coso
State: S (Sleeping)
Tgid: 2557
Ngid: 0
Pid: 2557
PPid: 2229
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 256
Groups: 0
VmPeak: 146840 kB
VmSize: 144252 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 36344 kB
VmRSS: 34792 kB
VmData: 45728 kB
VmStk: 136 kB
VmExe: 116 kB
VmLib: 23832 kB
VmPTE: 292 kB
VmSwap: 0 kB
Threads: 1
SigQ: 0/7862
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000004
SigIgn: 0000000000011001
SigCgt: 00000001c00064ee
CapInh: 0000000000000000
CapPrm: 0000001fffffffff
CapEff: 0000001fffffffff
CapBnd: 0000001fffffffff
Seccomp: 0
Cpus_allowed: 7fff
Cpus_allowed_list: 0-14
Mems_allowed: 00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 16978
nonvoluntary_ctxt_switches: 52120
strace shows:
$ strace -p 22254 -s 80 -o /tmp/debug.lighttpd.txt
read(0, "SGI\0\1\0\0\0\1\0c\0\0\0\t\0\0T\1\2248\0\0\0\0'\1\0\0(\0\0"..., 512) = 113
read(0, "SGI\0\1\0\0\0\1\0\262\1\0\0\10\0\1\243\1\224L\0\0\0\0/\377\373\222D\231\214"..., 512) = 448
sendto(3, "<15>Oct 10 18:34:01 MixCoder[271"..., 107, MSG_NOSIGNAL, NULL, 0) = 107
write(1, "SGO\0\0\0\0 \272\1\0\0\t\0\1\253\1\243\273\0\0\0\0'\1\0\0\0\0\0\1\242"..., 454) = 454
sendto(3, "<15>Oct 10 18:34:01 MixCoder[271"..., 107, MSG_NOSIGNAL, NULL, 0) = 107
write(1, "SGO\0\0\0\0 \341\0\0\0\10\0\0\322\1\254Z\0\0\0\0/\377\373R\4\0\17\21!"..., 237) = 237
read(0, "SGI\0\1\0\0\0\1\0)\3\0\0\t\0\3\32\1\224`\0\0\0\0'\1\0\0\310\0\0"..., 512) = 512
read(0, "\344u\233\16\257\341\315\254\272\300\351\302\324\263\212\351\225\365\1\241\225\3+\276J\273\37R\234R\362z"..., 512) = 311
read(0, "SGI\0\1\0\0\0\1\0\262\1\0\0\10\0\1\243\1\224f\0\0\0\0/\377\373\222d[\210"..., 512) = 448
sendto(3, "<15>Oct 10 18:34:01 MixCoder[271"..., 107, MSG_NOSIGNAL, NULL, 0) = 107
write(1, "SGO\0\0\0\0 %!\0\0\t\0\0+\1\243\335\0\0\0\0\27\0\0\0\0\1B\300\36"..., 8497) = 8497
sendto(3, "<15>Oct 10 18:34:01 MixCoder[271"..., 107, MSG_NOSIGNAL, NULL, 0) = 107
write(1, "SGO\0\0\0\0 \341\0\0\0\10\0\0\322\1\254t\0\0\0\0/\377\373R\4\0\17\301\31"..., 237) = 237
read(0, "SGI\0\1\0\0\0\1\0\262\1\0\0\10\0\1\243\1\224\200\0\0\0\0/\377\373\222d/\200"..., 512) = 448
sendto(3, "<15>Oct 10 18:34:01 MixCoder[271"..., 107, MSG_NOSIGNAL, NULL, 0) = 107
write(1, "SGO\0\0\0\0 \341\0\0\0\10\0\0\322\1\254\216\0\0\0\0/\377\373R\4\0\17\361+"..., 237) = 237
read(0, "SGI\0\1\0\0\0\1\0\221\0\0\0\t\0\0\202\1\224\210\0\0\0\0'\1\0\0P\0\0"..., 512) = 159
read(0, unfinished ...)
+++ killed by SIGKILL +++
Assuming that you have root access on your machine, you can enable audit on kill(2) syscall to gather such information.
root # auditctl -a exit,always -F arch=b64 -S kill -F a1=9
root # auditctl -l
LIST_RULES: exit,always arch=3221225534 (0xc000003e) a1=9 (0x9) syscall=kill
root # sleep 99999 &
[2] 11688
root # kill -9 11688
root # ausearch -sc kill
time->Tue Oct 14 00:38:44 2014
type=OBJ_PID msg=audit(1413272324.413:441376): opid=11688 oauid=52872 ouid=0 oses=20 ocomm="sleep"
type=SYSCALL msg=audit(1413272324.413:441376): arch=c000003e syscall=62 success=yes exit=0 a0=2da8 a1=9 a2=0 a3=0 items=0 ppid=6107 pid=6108 auid=52872 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsg
id=0 tty=pts2 ses=20 comm="bash" exe="/bin/bash" key=(null)
The other way is to set up kernel tracing which may be an over-kill when audit system can do same work.
Finally I figured out the reason why.
The child process kills itself and it's a linux kernel bug.
Details:
1) The child process kills itself, that's why strace/perf/auditctl cannot track it down.
2) The JNI call to create a process is triggered from a Java thread. When the thread eventually dies, it's also destroying the process that it creates.
3) In my code to fork and execve() a child process, I have the code to monitor parent process death and kill my child process with the following line: prctl( PR_SET_PDEATHSIG, SIGKILL ); I didn't pay special attention to this flag before b/c it's considered as a BEST PRACTICE for my other projects where child process is forked from the main thread.
4) If I comment out this line, the problem is gone. The original purpose is to kill the child process when the parent process is gone. Even w/o this flag, it's still the correct behavior. Seems like the ubuntu box default behavior.
5) From this article, https://bugzilla.kernel.org/show_bug.cgi?id=43300. it's a kernel bug, fixed in kernel version 3.4.0, my ubuntu box from AWS is kernel version 3.13.0-29-generic.
My machine configuration:
===>Ubuntu 14.04 LTS
===>3.13.0-29-generic
Some useful links to the issues:
a) http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them
b) prctl(PR_SET_PDEATHSIG, SIGNAL) is called on parent thread exit, not parent process exit
c) https://bugzilla.kernel.org/show_bug.cgi?id=43300
As I mentioned earlier, the other choice is to use kernel trace, which can be done by perf tool.
# apt-get install linux-tools-3.13.0-35-generic
# perf list | grep kill
syscalls:sys_enter_kill [Tracepoint event]
syscalls:sys_exit_kill [Tracepoint event]
syscalls:sys_enter_tgkill [Tracepoint event]
syscalls:sys_exit_tgkill [Tracepoint event]
syscalls:sys_enter_tkill [Tracepoint event]
syscalls:sys_exit_tkill [Tracepoint event]
# perf record -a -e syscalls:sys_enter_kill sleep 10
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.634 MB perf.data (~71381 samples) ]
// Open a new shell to kill.
$ sleep 9999 &
[1] 2387
$ kill -9 2387
[1]+ Killed sleep 9999
$ echo $$
9014
// Dump the trace in your original shell.
# perf script
...
bash 9014 [001] 1890350.544971: syscalls:sys_enter_kill: pid: 0x00000953, sig: 0x00000009
Background/context:
We are running Java application on one of CompuLab CoM:
https://compulab.co.il/products/computer-on-modules/cm-fx6/#overview
JVM version: Oracle Java 7 ARM 1.7.0_60
OS reference:
http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-FX6_Linux
The application is not trivial: lots of threads, access to Ethernet (LAN), serial interface, GPRS/UMTS modem, access to Internet (ppp deamon), GPS, touch screen, database (SQLite), file system. In other words use OS resources extensively.....
We are observing that Java application (all of its threads) and OS basic functionality randomly hangs. I would say it is a Linux kernel bug but by killing the Java application it recovers and operates normally.
This state always takes exactly 24 minutes. Afterwards it recovers and behaves normally. Average rate of occurrence is once per 24-30 hours.
When it happens, externally invoked events like messages sent to application via Ethernet or serial interface are buffered (by OS probably) and all of them are processed immediately after it recovers.
When I establish SSH connection to device in advance, after it happens the connection is either blocked (all command are buffered and processed after it recovers - 24 mintes) or its working, than:
basic OS utilities does not work: "top" for example
jstack -F does not work, just hangs and does not produce any output
killing Java application by kill -9 PID released the OS and everything starts to operate normally
While it is in this state, the OS each time behaves differently. Other findings:
Basic network based utilities does not work (SSH, FTP) – can not
establish new connection to OS from another machine.
PING from another machine does work until I unplug an plug Ethernet
cable from device, sometimes PING than stops working
Sometimes OS system time hangs as well (not always), after 24 minutes
it continues delayed for 24 minutes.
New USB input devices (mouse, keyboard) can not be connected while in
that state (happens always).
Another strange thing:
A touch screen is used for interaction with a user (driver compiled as kernel module). And it works even while it is hung. Java application (GUI Swing) can handle events like pressing button so I can run some code behind button click handler.
It seems like all threads are blocked but Java Swing can handle some input events and our application precesses them until it needs to interact with already blocked threads or OS (run bash script on button click) or call sleep method. Than it hangs as well.
In other words, the Java application is hung ”partially” - can still handle something.
Already tried:
Tools for JVM remote debugging: Java Mission Control, VisualVM.
Connection was also established before it hung. Everything seemed OK
in terms of thread dump, heap dump etc. (I can send by e-mail). Even
the connection remained and I could see in thees tools that processor
usage dropped to 0 % for JVM.
jstack -F (via SSH): does not work, just hangs and does not produce
any output
I tried to run OS without the driver for touch screen and it still
happened.
I tried to run two parallel Java application. One of them was very
simple – just writing to log timestamps. And both of them hung.
I tried to run System.exit(0) in terms of button click handler while
app. and all threads hung and it does not worked (hung as well)
Questions:
Is it Linux kernel bug or JVM (its ARM implementation) bug?
Is Java (JVM) able to hang and block basic OS functionality (FTP, SSH, system time, other utilities)?
How can I further diagnose/debug this issue when basic utilities like jstack -F does not work?
Do you have any ideas what could be the cause of this issue and why it always recovers exactly after 24 minutes?
Update 1: 2014-07-10
Finally I manage to “catch” this weird state again. Here are my further findings.
Based on nos suggestion I tried run via ssh (established in advanced):
*strace -f -p PID*
Unfortunately the bash script command hung as well (same behavior like with jstack).
As far as the user limit (ulimit) and OS resources are concerned, bellow I report figures taken just after the system recovered from last hung. At that state it had been running for 24 hours and I can confirm that those figures remain roughly the same during long-term operation (no random peeks during operation). From my point of view, they are ok and application is not stepping over any resource or other limit in any way.
Java current heap
Used: 18 MB, Free: 12 MB, Total: 30 MB, Max: 230 MB
Java heap
root#cm-debian:~# /usr/lib/jvm/jdk1.7.0_60/bin/jmap -heap 3242
Attaching to process ID 3242, please wait...
Debugger attached successfully.
Client compiler detected.
JVM version is 24.60-b09
using thread-local object allocation.
Mark Sweep Compact GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 249561088 (238.0MB)
NewSize = 1048576 (1.0MB)
MaxNewSize = 4294836224 (4095.875MB)
OldSize = 4194304 (4.0MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 12582912 (12.0MB)
MaxPermSize = 67108864 (64.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 10092544 (9.625MB)
used = 6772088 (6.458366394042969MB)
free = 3320456 (3.1666336059570312MB)
67.09991058745942% used
Eden Space:
capacity = 9043968 (8.625MB)
used = 6620336 (6.3136444091796875MB)
free = 2423632 (2.3113555908203125MB)
73.2016743093297% used
From Space:
capacity = 1048576 (1.0MB)
used = 151752 (0.14472198486328125MB)
free = 896824 (0.8552780151367188MB)
14.472198486328125% used
To Space:
capacity = 1048576 (1.0MB)
used = 0 (0.0MB)
free = 1048576 (1.0MB)
0.0% used
tenured generation:
capacity = 22134784 (21.109375MB)
used = 17650936 (16.83324432373047MB)
free = 4483848 (4.276130676269531MB)
79.7429782915433% used
Perm Generation:
capacity = 19136512 (18.25MB)
used = 19023016 (18.141761779785156MB)
free = 113496 (0.10823822021484375MB)
99.40691386183647% used
9597 interned Strings occupying 729344 bytes.
top
top - 11:41:29 up 21:59, 2 users, load average: 1.51, 1.25, 1.22
Tasks: 93 total, 1 running, 92 sleeping, 0 stopped, 0 zombie
Cpu(s): 9.4%us, 8.0%sy, 0.0%ni, 82.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 966780k total, 273080k used, 693700k free, 27216k buffers
Swap: 0k total, 0k used, 0k free, 126352k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3242 root 20 0 398m 79m 11m S 23.6 8.4 346:16.82 java
3889 root 20 0 2804 1096 848 R 5.5 0.1 0:00.07 top
1 root 20 0 2124 688 596 S 0.0 0.1 0:02.92 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.03 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:14.32 ksoftirqd/0
5 root 20 0 0 0 0 S 0.0 0.0 0:00.14 kworker/u:0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
7 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 khelper
java limits
root#cm-debian:~# java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
uintx AdaptivePermSizeWeight = 20 {product}
intx CompilerThreadStackSize = 0 {pd product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 67108864 {product}
uintx InitialHeapSize := 15468480 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 249561088 {product}
uintx MaxPermSize = 67108864 {pd product}
uintx PermSize = 12582912 {pd product}
intx ThreadStackSize = 320 {pd product}
intx VMThreadStackSize = 512 {pd product}
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) Client VM (build 24.60-b09, mixed mode)
process limits
root#cm-debian:~# cat /proc/3242/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes unlimited unlimited processes
Max open files 8192 8192 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 16382 16382 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
system memory info
root#cm-debian:~# cat /proc/meminfo
MemTotal: 966780 kB
MemFree: 694312 kB
Buffers: 27384 kB
Cached: 126364 kB
SwapCached: 0 kB
Active: 140748 kB
Inactive: 107684 kB
Active(anon): 94992 kB
Inactive(anon): 2064 kB
Active(file): 45756 kB
Inactive(file): 105620 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 524288 kB
HighFree: 301088 kB
LowTotal: 442492 kB
LowFree: 393224 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 94692 kB
Mapped: 21220 kB
Shmem: 2376 kB
Slab: 13268 kB
SReclaimable: 5284 kB
SUnreclaim: 7984 kB
KernelStack: 960 kB
PageTables: 980 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 483388 kB
Committed_AS: 137260 kB
VmallocTotal: 286720 kB
VmallocUsed: 2928 kB
VmallocChunk: 283040 kB
root#cm-debian:~# vmstat -s
966780 K total memory
272468 K used memory
140776 K active memory
107712 K inactive memory
694312 K free memory
27392 K buffer memory
126404 K swap cache
0 K total swap
0 K used swap
0 K free swap
726963 non-nice user cpu ticks
0 nice user cpu ticks
621187 system cpu ticks
6371123 idle cpu ticks
3683 IO-wait cpu ticks
324 IRQ cpu ticks
2146 softirq cpu ticks
0 stolen cpu ticks
130871 pages paged in
97520 pages paged out
0 pages swapped in
0 pages swapped out
293822206 interrupts
494034482 CPU context switches
1412595732 boot time
3916 forks
threads
root#cm-debian:~# cat /proc/sys/kernel/pid_max
32768
root#cm-debian:~# cat /proc/sys/kernel/threads-max
15102
root#cm-debian:~# cat /proc/sys/vm/max_map_count
65530
root#cm-debian:~# ls -l /proc/3242/task/ | wc -l
33
root#cm-debian:~# ps huH p 3242 | wc -l
32
root#cm-debian:~# grep -s '^Threads' /proc/[0-9]*/status | awk '{ sum += $2; } END { print sum; }'
122
open files / file descriptors
root#cm-debian:~# ls -l /proc/3242/fd | wc -l
81
Update 2: 2014-13-10
This time I logged all Java threads stack traces while the OS was hung (as I stated previously, the touch screen and its events still works so I wrote stack traces to log file in terms of UI button handler).
From my point of view, all threads are in “correct” state (sleeping, waiting for UDP datagram etc..) and it is obvious that the hang is not caused by a Java application SW operation which would took 24 minutes.
10:49:42,293> [INFO ] THREAD stack traces:
****************************************
ID: 56, name: Mpg123AudioPlayer_PASSENGER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)
ID: 11, name: AWT-EventQueue-0
java.lang.Thread.getStackTrace(Thread.java:1589)
epis5fcc.domain.debug.ThreadStackTracesLogger.log(ThreadStackTracesLogger.java:30)
epis5fcc.ui.settings.FccRegistryScreen$7.actionPerformed(FccRegistryScreen.java:303)
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
java.awt.Component.processMouseEvent(Component.java:6516)
javax.swing.JComponent.processMouseEvent(JComponent.java:3320)
java.awt.Component.processEvent(Component.java:6281)
java.awt.Container.processEvent(Container.java:2229)
java.awt.Component.dispatchEventImpl(Component.java:4872)
java.awt.Container.dispatchEventImpl(Container.java:2287)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
java.awt.Container.dispatchEventImpl(Container.java:2273)
java.awt.Window.dispatchEventImpl(Window.java:2719)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735)
java.awt.EventQueue.access$200(EventQueue.java:103)
java.awt.EventQueue$3.run(EventQueue.java:694)
java.awt.EventQueue$3.run(EventQueue.java:692)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
java.awt.EventQueue$4.run(EventQueue.java:708)
java.awt.EventQueue$4.run(EventQueue.java:706)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.awt.EventQueue.dispatchEvent(EventQueue.java:705)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
java.awt.EventDispatchThread.run(EventDispatchThread.java:91)
ID: 34, name: Mpg123AudioPlayer_DRIVER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)
ID: 26, name: IOTxUdpAccessLoop_IODispatchAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)
ID: 29, name: MasterLoop_main
java.lang.Thread.sleep(Native Method)
jCommons.master.MasterLoop.ctrlLoop(MasterLoop.java:87)
jCommons.master.MasterLoop.access$000(MasterLoop.java:11)
jCommons.master.MasterLoop$1.run(MasterLoop.java:58)
java.lang.Thread.run(Thread.java:745)
ID: 27, name: IORxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)
ID: 31, name: UsbUpdateWatchService_ctrlLoop
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
sun.nio.fs.AbstractWatchService.take(AbstractWatchService.java:118)
jCommons.update.usb.UsbUpdateWatchService.ctrlLoop(UsbUpdateWatchService.java:107)
jCommons.update.usb.UsbUpdateWatchService.access$000(UsbUpdateWatchService.java:25)
jCommons.update.usb.UsbUpdateWatchService$1.run(UsbUpdateWatchService.java:75)
java.lang.Thread.run(Thread.java:745)
ID: 25, name: IORxUdpAccessLoop_IODispatchAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)
ID: 2, name: Reference Handler
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
ID: 30, name: VehicleCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.vehicle.control.VehicleCtrl.ctrlLoop(VehicleCtrl.java:74)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)
ID: 35, name: Mpg123AudioPlayer_INNER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)
ID: 21, name: IORxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)
ID: 7, name: FileWatchdog
java.lang.Thread.sleep(Native Method)
org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)
ID: 8, name: Java2D Disposer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
sun.java2d.Disposer.run(Disposer.java:145)
java.lang.Thread.run(Thread.java:745)
ID: 17, name: com.google.inject.internal.util.$Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114)
ID: 10, name: AWT-XAWT
sun.awt.X11.XToolkit.waitForEvents(Native Method)
sun.awt.X11.XToolkit.run(XToolkit.java:541)
sun.awt.X11.XToolkit.run(XToolkit.java:505)
java.lang.Thread.run(Thread.java:745)
ID: 32, name: Thread-4
sun.nio.fs.LinuxWatchService.poll(Native Method)
sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47)
sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:311)
java.lang.Thread.run(Thread.java:745)
ID: 28, name: IOTxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)
ID: 14, name: DestroyJavaVM
ID: 22, name: IOTxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)
ID: 19, name: TimerQueue
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
java.util.concurrent.DelayQueue.take(DelayQueue.java:220)
javax.swing.TimerQueue.run(TimerQueue.java:171)
java.lang.Thread.run(Thread.java:745)
ID: 12, name: AWT-Shutdown
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:296)
java.lang.Thread.run(Thread.java:745)
ID: 23, name: IORxUdpAccessLoop_IOCityScrnAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)
ID: 3, name: Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
ID: 4, name: Signal Dispatcher
ID: 52, name: pool-3-thread-1
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
ID: 24, name: IOTxUdpAccessLoop_IOCityScrnAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)
ID: 36, name: RemoteUpdateCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.update.remote.RemoteUpdateCtrl.ctrlLoop(RemoteUpdateCtrl.java:94)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)
ID: 55, name: Mpg123AudioPlayer_OUTER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)
This appear to be a problem related with GPT and local timers simultaneous use.
On Freescale community you can see one more question similar to yours and other from someone given some clarification.
For the resolution, apply this patch.
From the second post you can jump to kernel 3.10.17 from Fresscale or 3.13.3 from kernel.org
Currently I am trying the patch to see if resolves a similar problem.
I have a Java application that uses a native library for some of its functionality. It uses JNI to control the native library and also receives asynchronous callback from the library. You can think of it as a Java frontend and native backend that communicate with each other.
I am facing a memory leak. Shortly after I start the application, the memory slowly but steadily increases. So I tried to look what could cause the leak.
First, I tried replacing the Java frontend with a simple C++ text interface. That way, the application doesn't use Java in any way - and the leaks stopped. So the problem must be in Java frontend.
So I fired up the jvisualVM to see if the heap increases - and it turned out it doesn't. The Java heap size was fairly constant. I even launched the program with xmx32m, but the memory kept increasing well past 100m without any OutOfMemoryErrors. In fact, the jvisualVM showed Java heap at about 7m.
So I dug deeper into the program with WinDbg. I analyzed the heap patterns with !heap -s command and I got this:
Heaps on a freshly run program:
0:059> !heap -s
LFH Key : 0x382288b9
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
00330000 00000002 2048 1704 2048 22 71 2 0 0 LFH
005b0000 00001002 1088 212 1088 68 3 2 0 0 LFH
00aa0000 00001002 1088 108 1088 15 7 2 0 0 LFH
004f0000 00001002 15424 12876 15424 1372 89 9 0 1 LFH
...
0:059> !heap -stat -h 004f0000
heap # 004f0000
group-by: TOTSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
2b110 20 - 562200 (60.36)
98 166e - d5150 (9.33)
6cd20 1 - 6cd20 (4.77)
...
Heaps on a program that has been running for about half an hour:
0:046> !heap -s
LFH Key : 0x5e47ba72
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
006b0000 00000002 2048 1744 2048 46 92 2 0 0 LFH
00200000 00001002 1088 220 1088 68 3 2 0 0 LFH
00950000 00001002 1088 108 1088 15 7 2 0 0 LFH
001b0000 00001002 47808 31936 47808 1855 102 12 0 0 LFH
...
0:046> !heap -stat -h 001b0000
heap # 001b0000
group-by: TOTSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
98 59d1 - 355418 (36.67)
2b110 10 - 2b1100 (29.61)
6cd20 1 - 6cd20 (4.68)
...
Now it can be clearly seen that the leaks are caused by a growing number of blocks with size 98. But when I try to analyze one of the blocks with !heap -p -a, I get:
*** ERROR: Symbol file could not be found. Defaulted to export symbols for jvm.dll
without any stack trace. So the blocks are allocated somewhere inside the jvm.dll, and because there are no pdbs for JVM, I cannot debug the leak further.
I managed to pinpoint where the leak is occuring in my code. All callbacs to the Java frontend pass through one function:
void callback(JNIEnv *env, int stream, double value, char *callbackName){
jclass jni = env->FindClass("nativ/Callbacks");
jmethodID callbackMethodID = env->GetStaticMethodID(jni, callbackName, "(ID)V");
jvalue params[2];
params[0].i = (long)(stream);
params[1].d = value;
env->CallStaticVoidMethodA(jni, callbackMethodID, params); //commenting this out stops the leaks
}
When I comment out the last command, the leaks stop, but I get no feedback back to the frontend.
Could this be a JVM bug? How do I find out?
malloc() internally calls HeapAlloc(). I guess you need a 'Release' method to release the memory allocated by JVM, as long as your library hold reference to JVM's internal state.
Okay, I got into a conversation with a friend about Ada (I'm the local proponent here), and in his project he's having a pain trying to get Java (using JNI) to get the applications running on the client machine (only Windows, Mac, and Linux) to get a listing of applications.
I'm not familiar with Macs at all, and my Linux experience is mostly user-end within academia.
So, my question is this: does anyone know a good cross-platform way to get the process-list?
My solution would be to use a package spec with a general function returning the list in the manner the Java expects it and throw together three different bodies for each of the platforms which would get the process-list according to that system and compile the (resultant) three binaries for those targets individually.
Is there a [good] way to do it w/o resorting to three different versions?
(This is an Ada question, but Java solutions are welcome.)
Java has no cross-platform API to list running processes. ProcessBuilder may be used to excecute the ps command, as shown here and here. The (rough) equivalent in Ada would be GNAT.Os_Lib.Spawn in GNAT. I'm not sure about other implementations.
JavaSysMon can provide a list of running processes (as well as other system information) in a platform-independent manner. Currently it supports Mac OS X, Linux, Windows, and Solaris. As an added bonus, it is BSD licensed.
Wiki
JavaDocs
You were almost at the Ada solution. As you only want 1 procedure to execute & look at the system call (top/ps in linux/unix) i would suggest a separate procedure. This will live in its own directory, and only be referenced by the correct compilation (per os). As for the actual commands per os, that is not part of my answer.
Do you just mean to get a list of running processes?
If so, you can just Google the commands to get this (1) the name of the OS on which the program is running, then (2) run Runtime.getRuntime.exec(stringCommandToGetProcessList); to based on #1, and output the results.
You don't need a different Java binary for every OS. You only need one. Just Google the command to find the OS name/version, and the command to get the list of running applications.
You also don't need JNI to do this. Use the Runtime class to run commands as if they were on the command-line.
There's no cross-platform way to do it, because the commands are different on each OS. But since there's only three major OS's (maybe a dozen total that you want to support, in some crazy extreme example), then it's just a matter of making a list of the 12 different commands to do this.
On Macs, and many Linux versions, OS name/version:
$ uname -a
Darwin normalocity 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64
Running processes (by highest usage):
$ top
Processes: 92 total, 5 running, 87 sleeping, 408 threads 20:38:35
Load Avg: 0.18, 0.20, 0.17 CPU usage: 7.26% user, 1.95% sys, 90.78% idle
SharedLibs: 6272K resident, 7300K data, 0B linkedit. MemRegions: 12204 total, 730M resident, 29M private, 393M shared.
PhysMem: 1076M wired, 1184M active, 1859M inactive, 4119M used, 4062M free.
VM: 207G vsize, 1041M framework vsize, 1851231(0) pageins, 603(0) pageouts.
Networks: packets: 1727104/1746M in, 984226/269M out. Disks: 295257/6745M read, 397634/15G written.
PID COMMAND %CPU TIME #TH #WQ #PORT #MRE RPRVT RSHRD RSIZE VPRVT VSIZE PGRP PPID STATE UID
12547 top 3.5 00:00.26 1/1 0 24 34 1208K 264K 1784K 17M 2378M 12547 12217 running 0
12217 bash 0.0 00:00.08 1 0 17 25 1328K 856K 1988K 17M 2378M 12217 12211 sleeping 502
12212 bash 0.0 00:00.08 1 0 17 25 1276K 856K 1980K 9688K 2378M 12212 12200 sleeping 502
12211 login 0.0 00:00.01 1 0 22 54 512K 312K 1648K 11M 2379M 12211 12196 sleeping 0
12202 bash 0.0 00:00.07 1 0 17 25 1276K 856K 1980K 9688K 2378M 12202 12199 sleeping 502
12201 bash 0.0 00:00.07 1 0 17 25 1276K 856K 1980K 9688K 2378M 12201 12198 sleeping 502
12200 login 0.0 00:00.01 1 0 22 54 512K 312K 1648K 11M 2379M 12200 12196 sleeping 0
12199 login 0.0 00:00.01 1 0 22 54 512K 312K 1648K 11M 2379M 12199 12196 sleeping 0
12198 login 0.0 00:00.01 1 0 22 54 512K 312K 1648K 11M 2379M 12198 12196 sleeping 0
12196 Terminal 33.9 00:01.84 5 1 114- 137 5736K+ 32M 23M+ 90M 2768M 12196 300 sleeping 502
11803- Google Chrom 0.0 04:06.79 7 1 99 365 45M 84M 79M 112M 1199M 11788 11788 sleeping 502
11800- Google Chrom 0.0 00:00.25 7 1 98 215 9632K 77M 23M 110M 1090M 11788 11788 sleeping 502
11799- Google Chrom 0.0 00:07.92 7 1 99 288 25M 82M 43M 109M 1108M 11788 11788 sleeping 502
11797- Google Chrom 0.0 00:01.49 7 1 99 316 27M 81M 48M 111M 1109M 11788 11788 sleeping 502
11796- Google Chrom 0.0 00:00.44 4 1 91 115 2824K 65M 8304K 96M 1012M 11788 11788 sleeping 502
11795- Google Chrom 0.0 00:00.96 7 1 98 215 9172K 77M 23M 111M 1091M 11788 11788 sleeping 502
11794- Google Chrom 0.0 00:07.64 8 1 100 294 20M 75M 36M 113M 1101M 11788 11788 sleeping 502
11793- Google Chrom 0.0 00:01.42 8 1 95 185 9732K 73M 24M 104M 1057M 11788 11788 sleeping 502
11788- Google Chrom 0.6 04:04.31 30 1 307 390 61M 110M 96M 254M 1298M 11788 300 sleeping 502
4328 ssh-agent 0.0 00:00.19 2 1 33 63 1300K 396K 2688K 59M 2420M 4328 300 sleeping 502
3855- Microsoft Of 0.0 00:36.14 4 1 121 337 12M 30M 22M 93M 1027M 3855 300 sleeping 502
492 AppleSpell 0.0 00:10.56 2 1 34 72 4608K 9028K 10M 88M 2469M 492 300 sleeping 502
Ada doesn't really have the concept of "processes" within the language. In fact, Ada code can run on platforms that do not support heavy processes at all (eg: Many smallish embedded platforms, like vxWorks).
That means you are going to have to use some kind of API (most likely supplied by your OS) to get that information.
If your OS supports POSIX, you may be able to use Posix bindings like Florist to get that info. There are full Unix subsystems available for Windows (Cygwin) and I believe MacOS is built on a flavor of Unix. So it might be possible to use Unix as sort of a lingua-franca so you can get your process info from a single (POSIX) API.
Now where Java is concerned, there are two issues: The Java language and the Java Platform (JVM). Java language fans like to conflate the two, but there are actually Ada compilers that target the JVM, and they can call all the same JVM API's that code written in the Java language can call. If there's one that allows Java programs to get a list of all the threads or processes that the JVM knows about, you could call that same routine from Ada too (if it is running under the JVM as well).