Classpath limitation in Linux - java

We are executing standalone java program from shell script, having a comman script to mention classpath, path, etc..In this common script, we have added several classpaths now and number of character is more than 9000.
It is working fine in the test env. Will it cause any issue in Production? Any limitation is there in linux to set classpath? What is the max char for command line inputs...

No, there is no limitation. In Windows there is (8191 characters), but not under Linux. We work with the concept of classpath-files. These file lists all the dependencies for the application, eg:
...
libs/org/easymock/easymock/2.2/easymock-2.2.jar
libs/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar
libs/org/hibernate/hibernate-envers/4.1.0.Final/hibernate-envers-4.1.0.Final.jar
libs/com/google/inject/guice/3.0/guice-3.0.jar
...
and then we convert this into usable classpath and run the application as follows:
#!/bin/bash
CLASSPATH_FILE=`ls -r1 ${APP-HOME}/classpaths/myapp*.classpath | head -n1`
CLASSPATH=$(cat $CLASSPATH_FILE | sed 's_^libs_ ${APP-HOME}/libs_' | tr -d '\n' | tr -d '\r' | sed 's_.jar/libs/_.jar:/libs/_g' | sed 's_.pom/libs/_.pom:/libs/_g')
java -d64 -cp $CLASSPATH com.blah.main.Main $#
We have never run into problems and these classpath entries gets pretty huge.
EDIT: As a side note, you can use the maven dependency plugin to generate a list of dependencies.

See this stackoverflow answer about maximum linux command line lengths.
The maximum command line length will be roughly between 128KB and 2MB.
The max size of any one argument is considerably smaller, though, and 9000 chars might be problematic.

When you use in your Java program some classes from a jar file that is specified in the classpath variable, the JVM won't load that class until your running program will explicitly need that class (or if you load that class explicitly from your code - the same idea). The only problem that can appear when you have a very long classpath, is the time needed for classpath checking before the JVM find the right jar file. But that should not be a problem. If your program behaves well in tests, you should not be worried about this.

Related

Can I use -cp argument twice when executing Java?

I am trying to run a Java application which has many dependencies. In the past I have use the following command to launch the application
java -cp "program.jar:jar1.jar:jar2.jar:jar3.jar:[...]" program
However as the list of dependencies have grown, I have moved them into an arguments file, the contents of this file are:
-cp "\
program.jar:\
jar1.jar:\
jar2.jar:\
jar3.jar:\
[...]"
And I am running the application with
java #arguments-file program
Everything up to this point works fine.
Sometimes I end up with beta versions of program.jar, they share all of the same dependencies, but program.jar is renamed program-beta.jar.
So to run the jar the following command would be used
java -cp "program-beta.jar:jar1.jar:jar2.jar:jar3.jar:[...]" program
or more specifically, I would use an environment variable, so that the same script can be used, and the variable would be set to either program.jar, or program-beta.jar, depending on the circumstance
java -cp "$PROGRAM_JAR:jar1.jar:jar2.jar:jar3.jar:[...]" program
Now that I am using an arguments file I was hoping to be able to be able to do something like:
java -cp "$PROGRAM_JAR" #arguments-file program
However using -cp twice causes one of the two to be ignored, resulting in a java.lang.ClassNotFoundException exception.
Is there any way around this that allows me to specify one jar file by name, but abstract away all of the others so that my java command isn't thousands of characters?
This will be running entirely on Linux, so any command line "magic", such as using grep is fine, so long as the resulting code is easily readable
You could just write two bash scripts production.sh and beta.sh that contain a reference on program.jar and program-beta.jar, respectively.
Also, the classpath can contain wildcards (see man-page), so if you can ensure that on disk exists only one of the two versions, you can write it like this:
java -cp "program*:jar1.jar:jar2.jar:jar3.jar:[...]"
In the long term, you might think about building/running it with Maven or Gradle, depending on your requirements.

linux: ps truncates command line output

My goal is to get the command line args of a java process. I am running ps aux | grep java> out.log to see the full argument list. Problem is that it's truncated at approx 4k bytes. Java is invoked from a build tool (maven) so I don't have much influence on the arguments. Most of the long argument list is related to classpath entries. On the windows platform the argument list is approx 12Kb.
How can I see the full command line arguments in Linux even if they are longer than 4K? I am running on Linux Mint Petra. I've tried with the processes explorer but it also truncates (and it won't let my copy-paste the arguments)
Just found this explanation how-do-i-increase-the-proc-pid-cmdline-4096-byte-limit. It basically tells me that the linux kernel has a hard limit (unless I want to recompile it). The best solution for me was to run jconsole. This does the trick for me at least for java processes.
You can try
ps axwwo args
This shows all arguments that are stored in the process table.
But the right way would be to get more familiar with your build tool. If you master this maven gives you perfect control.
There is a hardcoded limit of 4k in the cmdline buffer in Linux, so there's not much you can do about it unless you feel like downloading the kernel's source and modify it to allow a bigger buffer.
As a workaround you could execute maven with the -X option for full debugging and using tee to write to standard output and also a file: mvn -X clean install | tee my_log_file.txt and then try to find the information you're looking for in my_log_file.txt
Just note for some, how are not able to use GUI tools. Another alternative is using of jinfo tool, included in JDK. Just be aware of, that you need to use same version of JDK as java, you are running (some time there is used JRE instead of java from JDK).
just use jinfo <pid>

Bash Script to run newest version of a file?

I'm using Apache Maven to release our product to Production, and as of right now, the resultant jar file that is created does not have a version number appended to it. I would like to append the current pom.xml version number (which I know how to do), but I need our installer script (basically a java -jar command with extra parameters) to access the newest version of the installer by default.
Say my deployment file is called foo.jar. My script is basically java -jar foo.jar
Instead, I'd like my deployment directory to contain foo-4.0.0.jar, foo-4.0.1.jar, foo-4.0.2.jar, etc.
I would like to create a bash script that runs the highest version of foo.
I've looked at sorting by creation date, but I can immediately think of scenarios that might not work out very well. I've considered doing it by date modified, and then manually touching the file that I want to install, but that idea kind of sucks too.
Clearly I need to split the name (probably with a regex) and isolate the version numbers (maybe substring from the last dash and the last period in the name?) and somehow sort them. The sorting is probably my biggest concern...that and reassociating the file.
If this is too complicated, I could theoretically make a Java file that does this (has it become clear that my primary language is Java?), but ideally it would just be a bash script.
Thanks
You can use sort, specifying the separator and set of keys to sort on e.g.
$ ls *.jar | sort -t- -k2 -V -r
will sort your jars in reverse (-r) using a version number (-V) sort, separating the version number from the jar name using -t-, and sorting on the second field using -k2 (your version number)
In my test directory I get:
a-2.2
a-2.1
a-2.0
a-1.50
a-1.10
a-1.3.1
c-1.3.0.1
a-1.3
a-1.2.9
b-1.2.8
a-1.2
which looks good.
Pipe the above through head -1 to give you the top entry.
Bash launcher script
Note: Verbose command options are used to make the script more reusable and easier to maintain.
#!/usr/bin/env bash
java -jar $(ls /home/opt/deployment_dir/*.jar \
|sort --version-sort --field-separator=- --key=2 --reverse \
|head --lines=1 \
)
Beware: Both the .jar file and above bash script should be rendered executable using the chmod +x command.

Java path problems on Cygwin

I'm trying to compile a Java project under Cygwin using a native Win32 Java.
The Java binaries are correctly found under /cygdrive/c/jdk/bin on my machine.
The following command works fine:
javac -d . ./gnu/kawa/util/PreProcess.java
The PreProcess.class file is generated in ./gnu/kawa/util/. Trying to invoke Java on this fails however:
CLASSPATH=.:$CLASSPATH java gnu.kawa.util.PreProcess \
%java6 +use:com.sun.net.httpserver +enable:XML \
`sed -e 's|\([^ ]*\)|./\1|' < ./patch-source-list`
Error: Could not find or load main class gnu.kawa.util.PreProcess
...
This command was invoked by make, that's where the $CLASSPATH variable is set dynamically. patch-source-list is just a list of class names. The : in the classpath looks suspicious, but I'm not sure how to test ; while not annoying sh.
My only other suspicion is that the native Java is trying gnu\kawa\util\PreProcess, but I think cygwin can transparently handle that.
Any ideas? Thanks for your time.
Another option would be to build your path using the ':' and then fix the results using cygpath. This is probably overkill in your specific situation, but in a general case where you may have references to multiple directories, some of which may be referenced as absolute rather than relative paths, or if you are working with cygwin symlinks, it is much more useful.
$ ln -s /cygdrive/c/development/libraries/ ../libs
$ cygpath -pw /cygdrive/c/development/:.:../libs
C:\development\;.;C:\development\libraries\
so then you'd build your CLASSPATH variable as before, and in the final stage run
CLASSPATH="`cygpath -pw "$CLASSPATH"`" java (whatever)
Remember, the JVM has no idea that you are using the cygwin bash shell.
Two things:
for the classpath locations, use the windows path names. Thus, no "/cygdrive/c/somepath", but "c:\somepath\" ("/" and "\" can be used interchangeably however)
use ';' instead of ':' in the classpath list
This should work:
export CLASSPATH="./gnu/kawa/util/PreProcess.class"
CLASSPATH=".;$CLASSPATH" java gnu.kawa.util.PreProcess
The : in the classpath looks suspicious, but I'm not sure how to test ; while not annoying sh.
You're exactly right: you need to use ; instead of :. As for how to use it — as Mat alludes to above, you need to "quote" the semicolon. Any of these will work:
CLASSPATH=.\;$CLASSPATH java Foo
CLASSPATH=.';'$CLASSPATH java Foo
CLASSPATH='.;'$CLASSPATH java Foo
CLASSPATH=".;$CLASSPATH" java Foo
You can use whichever one you like best. (The first uses a backslash, which quotes a single following character. The second and third use single-quotes, which quote a sequence of zero or more characters. The fourth uses double-quotes, which are like single-quotes except that they still allow the variable $CLASSPATH to be expanded. For that matter, you could also write something like
CLASSPATH=".;"$CLASSPATH java Foo
if you want. See the above link for lots more information about quoting in Bash.)

Determine running classes and jar's from Linux ps output

My goal is to list to STDOUT the class files and .jar files being executed by java on a Linux server. I could do some getopts thing to get args to -jar, but other processes identified by
ps -ef | grep java or ps -eo args | grep java
might be executing a class file, e.g. java -classpath /a/b/c myclass A1 A2 . I am concerned that I am looking at an inelegant solution full of lengthy piplines of greps and awk's to solve what should be (I think) a straightforward query. Given that:
some calls are made to just 'java' and others to the fully qualified pathname for java,
a variety of different (or no) java options may be set on the command line for running a process,
some processes call .jar files, some call .class files, and
there may be args to the class,
what is the best way to get a simple list of running java executables, like:
abc.jar
mymainclass
xyz.jar
numainclass
I think that this may be a not uncommon question, but I can't seem to build a search string that locates any previous discussion here. An elegant solution would be nice; right now I am looking at grepping '-jar' entries to a getops call, and parsing the remainder considering all possible combinations. I am working on a solution in bash 3.x
Thanks!
The jps command introduced in jdk5 might be what you are looking for. Using the -l and -m options it will output the pid main class and arguments. Adding -v will add the vm arguments.
This option lists all Java files currently opened by a java command. Maybe it is useful to you.
lsof | grep -E "^java.*(.jar|.class)$" | sed -E "s/\s+/\t/g" | cut -f9
It works in Debian.

Categories