My goal is to get the command line args of a java process. I am running ps aux | grep java> out.log to see the full argument list. Problem is that it's truncated at approx 4k bytes. Java is invoked from a build tool (maven) so I don't have much influence on the arguments. Most of the long argument list is related to classpath entries. On the windows platform the argument list is approx 12Kb.
How can I see the full command line arguments in Linux even if they are longer than 4K? I am running on Linux Mint Petra. I've tried with the processes explorer but it also truncates (and it won't let my copy-paste the arguments)
Just found this explanation how-do-i-increase-the-proc-pid-cmdline-4096-byte-limit. It basically tells me that the linux kernel has a hard limit (unless I want to recompile it). The best solution for me was to run jconsole. This does the trick for me at least for java processes.
You can try
ps axwwo args
This shows all arguments that are stored in the process table.
But the right way would be to get more familiar with your build tool. If you master this maven gives you perfect control.
There is a hardcoded limit of 4k in the cmdline buffer in Linux, so there's not much you can do about it unless you feel like downloading the kernel's source and modify it to allow a bigger buffer.
As a workaround you could execute maven with the -X option for full debugging and using tee to write to standard output and also a file: mvn -X clean install | tee my_log_file.txt and then try to find the information you're looking for in my_log_file.txt
Just note for some, how are not able to use GUI tools. Another alternative is using of jinfo tool, included in JDK. Just be aware of, that you need to use same version of JDK as java, you are running (some time there is used JRE instead of java from JDK).
just use jinfo <pid>
Related
I'd like to know how it is started. What is the command to start this java process ? What I mean is I have one running java process, and I'd like to know the command to start it, such as what is the main class and what is the arguments, etc.
Any tool for that ? Thanks
There is a command line tool that comes with the JDK: jps, that will give you the list of java processes being run at the moment you execute the command, the arguments given to the method main and the parameters used for the JVM. Try this:
path\to\jdk\bin\jps -m -l -v
It won't give you the exact command used to start the process, but it will give you a hint of how to "rebuild" that command.
For more info, if you are on a decent distro of linux, try man jps or if you are on Windows, see the Oracle documentation about jps.
Your question wasn't clear. If you are looking to find the command that launched this process than you can look at the property sun.java.command. This will give you the main class name and arguments passed to it. java.class.path property gives you the class path. You can get the arguments passed to the java command itself by using ManagementFactory.getRuntimeMXBean().getInputArguments() method. Using all these you should be able to reconstruct the java command.
If you use Windows, you can use the Taskmanager, go to the Process/Details Tab, where you can see the PID for each Process. There you can add a column for the command line (e.g. in German its "Befehlszeile", i'm not sure how that column is labeled in English).
Then just look at the java.exe/javaw.exe Processes.
You could also use the alternative Taskmanager from Microsoft, Process Explorer, afaik there you can just click right on a process and select details.
I'd like to collect stacktraces from my Java app for creating CPU Flame Graphs for profiling.
This is very similar to this question: How to get complete stack dump from profiler in every sample for use in flame graph? with 2 differences:
I work with Java code and I need Java stacktraces
I'm working on Mac (this means there is no pref and AFAIK dtrace on OSX doesn't support jstackextension).
I have already tried lightweight-java-profiler and Honest profiler, and both of them don't seem to work on Mac. I also Tried VisualVM, but I couldn't get it to produce the stacktrace dumps that I needed.
First prioirty for me are flame graphs generated from Java stacktraces, but having the native call stack as well would be great, because it would let me address the I/O issues (and maybe even generate hot/cold flame graphs).
Good news, the FlameGraph repository has "a script" to work with jstacks already in it.
https://github.com/brendangregg/FlameGraph
It's the stackcollapse-jstack.pl.
It seems that by default it expects just stack trace after stack trace in its input, and counts each one as "a sample point."
So you can just do multiple jstack's into a file (run this once or a few times, or once a second "for awhile" etc.):
jstack pid_of_your_jvm >> my_jstack
Then execute that script:
./stackcollapse-jstack.pl my_jstack > my_jstack.folded
and finally convert to flamegraph:
./flamegraph.pl --color=java my_jstack.folded > my_jstack.svg
No third party helpers required (though they may still be useful).
Note also that the stackcollapse-jstack.pl file discards non RUNNABLE threads, you may want to tweak that if you want to also include "idle" threads (typically you don't).
Apparently you could use the linux "perf" command to generate stacks for a java process, as well, see the README https://github.com/brendangregg/FlameGraph
This might include more native calls, for instance.
I created 2 little shell scripts based on #cello's answer. They generate hot/cold flame graphs.
Get them from this Gist.
Usage:
ps ax | grep java # find the PID of your process
./profile.sh 20402 stacks.txt
./gen.sh stacks.txt
Alternatively, to measure application from startup (in this, case, my gradle build that also needed to be run in another directory and with some input stream) I used:
cd ../my-project; ./gradlew --no-daemon clean build < /dev/zero &; cd -; ./profile.sh $! stacks.txt
./gen.sh stacks.txt
Results:
In this example, I can clearly see that my application is I/O bound (notice blue bars on top).
Try this: https://github.com/saquibkhan/javaFlameGraph
Installation
npm install javaflamegraph
Usage
cd javaflamegraph
npm start - This will wait till it detects a process with name 'Java'. Can be best best used to start profiling at program
startup.
npm run start <process id> - This will start profiling for the given process id.
e.g. npm run start 1234
Did you try the jstack command? just run it on the command line: jstack pidOfJavaProcess > stack.txt (naturally, replacing pidOfJavaProcess with the actual process number). You could run this in a loop in bash (the default shell used on Mac OS X):
while true; do jstack pidOfJavaProcess >> stack.txt; sleep 1.0; done
note the >> to append to the file, and not overwrite it each second. Press Ctrl+C to stop logging the stack traces.
This will only generate the java stack traces, and not the native call stacks from the JVM.
Install Flamegrpah
$ wget https://raw.githubusercontent.com/brendangregg/FlameGraph/master/flamegraph.pl
$ wget https://raw.githubusercontent.com/brendangregg/FlameGraph/master/stackcollapse-jstack.pl
$ chmod +x *.pl
# Copy those Perl scripts into the PATH location
Collect the stacktrace of running java application
# Run multiple times (in loop) to get more samples
$ jcmd <Java Pid> Thread.print >> jcmd.tdump
Generate flamegraph
$ stackcollapse-jstack.pl jcmd.tdump > jcmd.tdump.folded
$ flamegraph.pl --color=io --title "Thread Dump" --countname "Samples" --width 1080 jcmd.tdump.folded > jcmd.tdump.svg
I have a running java process, I want to find out with which parameters it was started, specifically, I want to know what debug port was specified (it was). Is there a way to do that in linux?
Update: Application was started through ant, so I can get ant command options, so now I know the task that was started, also I know ant process id.
jps -v -m might help also you can cat /proc/<pid>/cmdline
EDIT: jvisualvm provide also a bunch of infos about running java processes.
Why not just use ps -fe ?
From the ps man page
-f Do full-format listing. This option can be combined
with many other UNIX-style options to add additional columns. It also
causes the command arguments to be printed. When used with -L, the NLWP (number of threads) and LWP (thread ID)
columns
will be added. See the c option, the format keyword args, and the format keyword comm.
(my emphasis)
I'm trying to increase the heap size in java for weka which keeps crashing. I used the suggested line:
> java -Xmx500m -classpath
but I get the following error:
-classpath requires class path specification
I'm not sure what this means. Any suggestions?
What I found was the actual issue was in the file 'RunWeka.ini' in '\Program Files (x86)\Weka-3-6'. I opened it with notepad and in the middle of the file there is a line 'maxheap = 512m'.
I changed the line to read 'maxheap=2000m', saved the file and reloaded weka and this fixed my problems.
I'm not sure if this is the correct way to do it or not but it worked for me.
Run this command in your terminal:
java -Xmx1024m -jar weka.jar
Omit the -classpath option. Use just -Xmx500m option.
So, instead of just:
java weka.core.Instances data/soybean.arff
you do:
java -Xmx500m weka.core.Instances data/soybean.arff
If you run weka via some script (RunWeka.bat for example), then you need to modify that script (with some text editor like notepad).
If you're using Weka 3.8.1 on Windows you can save yourself a lot of trouble by editing the javaOpts parameter. The parameter maxheap isn't used anymore, so you can set javaOpts like this in RunWeka.ini file:
javaOpts= -Xmx1040m
Where 1040m is the amount of memory you want to allocate.
Mind that the file is case sensitive.
There are a lot of ways to set this up, but this is the faster way to get Weka runing on a Windows environment at this version.
Edit: If you want Weka to use more than 1gb on windows, you need to have JDK installed. Regular JRE wont do it.
The official Weka answer (for all operating systems and Weka versions) can be found on http://weka.wikispaces.com/OutOfMemoryException.
In case you are using a recent Weka version on Windows, the answer is:
Modify the maxheap parameter in the RunWeka.ini file.
On Ubuntu i had the same problem
but i solve it by increasing the amount of memory to use for the Java Virtual Machine
run this : weka -m 1024m
You need to specify a classpath after -classpath, similar to the PATH env variable you need to specify the path where Java can find the classes.
The -Xmx500m setting looks fine, except that I would suggest to use 512m.
For Mac OS, you have to edit a configuration file in order to increase the heap size of the Weka UI application.
I am repeating what I wrote in: Is there a workaround to solve "Java heap space" memory error when the max heap value has been already specified?
Quit out of Weka if it is running.
cd into /Applications/weka-XXX.app/Contents , or wherever your weka executable was installed. There will be a file called Info.plist there. It is an XML text file. I suggest you save a copy of it to another location, as you'll need to edit it in the next step.
Open the Info.plist (XML) file in your favorite text editor and look for a block that says "VMOptions". There should be a value that says "-Xmx256M" or something similar that specifies the maximum heap size. You should change that value to something bigger, such as "-Xmx1024M".
Start Weka.
I am running Weka 3.6 in windows. This is what i did.
Go to the Weka installation directory and you will find a RunWeka.bat file. Open this file in a text editor and add -Xmx argument in the java command line.
for instance this sets to 4GB memory,
%_java% -Xmx4096m -classpath . RunWeka -i .\RunWeka.ini -w .\weka.jar -c %_cmd% "%2"
The official Weka answer is right..But....crucial is to first get rid of all JVM files and install the relevant 32 or 64 bit Java version. Not using the relevant version causes many problems including the impossibility to increase the heap further than 1024m (by changing the ini file).
Weka 3.9.2 also does not has the option of maxheap anymore. RunWeka.ini have the option of javaOpts, So you may change the below to your required memory allocation,
javaOpts=%JAVA_OPTS% ---- > javaOpts= -Xmx1024m
Here 1024m is the customised amount of memory you want to allocate.
The best way to do it using this command
java -Xmx1024m -[weka classifier] -t [training file path]
The answers above are too old (last one is 1 year ago).
I had same issue with my WEKA (version 3.8.1) on Windows 10.
I had a problem to update the heap size , the way I fixed it is by adding an environment variable (under control panel) as follows:
JAVA_OPTS = -Xms30000m -Xmx30000m
Tip: Just ensure that RunWeka.ini is using this environment variable.
In the above example I give WEKA 30GB. It works.
Hope it will be helpful for some people.
You should also see if default thread stack size 20MB is enough. Increase the value to 50MB in the file /Applications/weka-3-8-1-oracle-jvm.app/Contents/Info.plist (on MAC) like below:
<string>-Xss50M</string>
If we are using Weka Workbench CLI or Knowledge explorer we need to
change as below.
As the documentation suggests the runtime parameter should be -Xmx[size_required]m where [size_required] is memory size you intend to keep to avoid memory exception.
Open RunWeka.ini
Define maxheap=[size_required]G
In my case I kept maxheap=4G , One can set like maxheap=4096m and add -Xmx#maxheap# to all the run options at # setups (prefixed with "cmd_") sections next to java commands
like below
cmd_default=javaw -Xmx#maxheap# ...............
cmd_console=cmd.exe /K start cmd.exe ..................
cmd_explorer=java -Xmx#maxheap# .................
cmd_knowledgeFlow=java -Xmx#maxheap#....................
maxheap=4G
Verify the same by restarting Weka and Help>>SystemInfo
If you run weka from the command line but not through java i.e. typing weka into the command line, instead of typing
weka
specify the memory flag
weka -m 1024m
This will specify 1024 megabytes.
If you're running weka via weka.sh, you can directly run it with memory option.
For example,
sh weka.sh -memory 10g
This will increase the heap size to 10Gb (tested using Weka 3.8.4 on Ubuntu 18.04)
I have a java program that uses ProcessBuilder to call the unix sort command. When I run this code within my IDE (intelliJ) it only takes about a second to sort 500,000 lines. When I package it into an executable jar, and run that from the terminal it takes about 10 seconds. When I run the sort command myself from the terminal, it takes 20 seconds!
Why the vast difference in performance and any way I can get the jar to execute with the same performance? Environment is OSX 10.6.8 and java 1.6.0_26. The bottom of the sort man page says "sort 5.93 November 2004"
The command it is executing is:
sort -t' ' -k5,5f -k4,4f -k1,1n /path/to/imput/file -o /path/to/output/file
Note that when I run sort from the terminal I need to manually escape the tab delimiter and use the argument -t$'\t' instead of the actual tab (which I can pass to ProcessBuilder).
Looking as ps everything seems the same except when run from IDE the sort command has a TTY of ?? instead of ttys000--but from this question I don't think that should make a difference. Perhaps BASH is slowing me down? I am running out of ideas and want to close this 20x performance gap!
I'm going to venture two guesses:
perhaps you are invoking different versions of sort (do a which sort and use the full absolute path to recompare?)
perhaps you are using more complicated locale settings (leading to more complicated character set handling etc.)? Try
export LANG=C
sort -t' ' -k5,5f -k4,4f -k1,1n /input/file -o /output/file
to compare
Have a look at this project: http://code.google.com/p/externalsortinginjava/
Avoid the need of calling external sort entirely.