I'm trying to run a particular JUnit test by hand on a Windows XP command line, which has an unusually high number of elements in the class path. I've tried several variations, such as:
set CLASS_PATH=C:\path\a\b\c;C:\path\e\f\g;....
set CLASS_PATH=%CLASS_PATH%;C:\path2\a\b\c;C:\path2\e\f\g;....
...
C:\apps\jdk1.6.0_07\bin\java.exe -client oracle.jdevimpl.junit.runner.TestRunner com.myco.myClass.MyTest testMethod
(Other variations are setting the classpath all on one line, setting the classpath via -classpath as an argument to java"). It always comes down to the console throwing up it's hands with this error:
The input line is too long.
The syntax of the command is incorrect.
This is a JUnit test testing a rather large existing legacy project, so no suggestions about rearranging my directory structure to something more reasonable, those types of solutions are out for now. I was just trying to gen up a quick test against this project and run it on the command line, and the console is stonewalling me. Help!
The Windows command line is very limiting in this regard. A workaround is to create a "pathing jar". This is a jar containing only a Manifest.mf file, whose Class-Path specifies the disk paths of your long list of jars, etc. Now just add this pathing jar to your command line classpath. This is usually more convenient than packaging the actual resources together.
As I recall, the disk paths can be relative to the pathing jar itself. So the Manifest.mf might look something like this:
Class-Path: this.jar that.jar ../lib/other.jar
If your pathing jar contains mainly foundational resources, then it won't change too frequently, but you will probably still want to generate it somewhere in your build. For example:
<jar destfile="pathing.jar">
<manifest>
<attribute name="Class-Path" value="this.jar that.jar ../lib/other.jar"/>
</manifest>
</jar>
Since Java 6 you can use classpath wildcards.
Example: foo/*, refers to all .jar files in the directory foo
this will not match class files (only jar files). To match both use: foo;foo/* or foo/*;foo. The order determines what is loaded first.
The search is NOT recursive
Use An "Argument File" on Java 9+
In Java 9+, the java executable supports providing arguments via a file. See
https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-4964-AE84-121F5F6CF111.
This mechanism is explicitly intended to solve the problem of OS limitations on command lengths:
You can shorten or simplify the java command by using #argument files
to specify a text file that contains arguments, such as options and
class names, passed to the java command. This let’s you to create java
commands of any length on any operating system.
In the command line, use the at sign (#) prefix to identify an
argument file that contains java options and class names. When the
java command encounters a file beginning with the at sign (#) , it
expands the contents of that file into an argument list just as they
would be specified on the command line.
This is the "right" solution, if you are running version 9 or above. This mechanism simply modifies how the argument is provided to the JVM, and is therefore 100% compatible with any framework or application, regardless of how they do classloading i.e. it is completely equivalent to simply providing the argument on the command line as usual. This is not true for manifest-based workarounds to this OS limitation.
An example of this is:
Original command:
java -cp c:\foo\bar.jar;c:\foo\baz.jar
can be rewritten as:
java #c:\path\to\cparg
where c:\path\to\cparg is a file which contains:
-cp c:\foo\bar.jar;c:\foo\baz.jar
This "argument file" also supports line continuation characters and quoting for properly handling spaces in paths e.g.
-cp "\
c:\foo\bar.jar;\
c:\foo\baz.jar"
Gradle
If you are encountering this issue in Gradle, see this plugin, which converts your classpath automatically into an "argument file" and provides that to the JVM when doing exec or test tasks on Windows. On Linux or other operating systems it does nothing by default, though an optional configuration value can be used to apply the transformation regardless of OS.
https://github.com/redocksoft/classpath-to-file-gradle-plugin
(disclaimer: I am the author)
See also this related Gradle issue -- hopefully this capability will eventually be integrated into Gradle core: https://github.com/gradle/gradle/issues/1989.
(I suppose you do not really mean DOS, but refer to cmd.exe.)
I think it is less a CLASSPATH limitation than an environment size/environment variable size limit. On XP, individual environment variables can be 8k in size, the entire environment is limited to 64k. I can't see you would hit that limit.
There is a limit on windows that restricts the length of a command line, on WindowsNT+ it is 8k for cmd.exe. A set command is subject to that restriction. Can it be you have more than 8k worth of directories in your set command? You may be out of luck, then - even if you split them up like Nick Berardi suggested.
Thanks to Raman for introducing a new solution to a pathing problem for Java 9+. I made a hack to bootRun task that allows using everything already evaluated by gradle to run java with argument files. Not very elegant but working.
// Fix long path problem on Windows by utilizing java Command-Line Argument Files
// https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-4964-AE84-121F5F6CF111
// The task creates the command-line argument file with classpath
// Then we specify the args parameter with path to command-line argument file and main class
// Then we clear classpath and main parameters
// As arguments are applied after applying classpath and main class last step
// is done to cheat gradle plugin: we will skip classpath and main and manually
// apply them through args
// Hopefully at some point gradle will do this automatically
// https://github.com/gradle/gradle/issues/1989
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
bootRun {
doFirst {
def argumentFilePath = "build/javaArguments.txt"
def argumentFile = project.file(argumentFilePath)
def writer = argumentFile.newPrintWriter()
writer.print('-cp ')
writer.println(classpath.join(';'))
writer.close()
args = ["#${argumentFile.absolutePath}", main]
classpath = project.files()
main = ''
}
}
}
If I were in your shoes, I would download the junction utility from MS : http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx and then map your
"C:\path" to say, "z:\" and "c:\path2" to say, "y:\". This way, you will be reducing 4 characters per item in your classpath.
set CLASS_PATH=C:\path\a\b\c;C:\path\e\f\g;
set CLASS_PATH=%CLASS_PATH%;C:\path2\a\b\c;C:\path2\e\f\g;
Now, your classpath will be :
set CLASS_PATH=z\a\b\c;z\e\f\g;
set CLASS_PATH=%CLASS_PATH%;y:\a\b\c;y:\e\f\g;
It might do more depending on your actual classpath.
I think you are up the creek without a paddle here.
The commandline has a limit for arguments to call a programm.
I have 2 sugestion you could try.
First, prior to running the junit tests, you can let a script/ant_task create JARs of the various classes on the classpath.
Then you can put the JARs on the classpath, which should be shorter.
Another way you could try is to create an antscript to run JUNIT,
in ANT there should not be such a limit for classpath entries.
As HuibertGill mentions, I would wrap this in an Ant build script just so that you don't have to manage all of this yourself.
You could try this
#echo off
set A=D:\jdk1.6.0_23\bin
set B=C:\Documents and Settings\674205\Desktop\JavaProj
set PATH="%PATH%;%A%;"
set CLASSPATH="%CLASSPATH%;%B%;"
go to a command prompt and run it twice(no idea why....i have to do so on a windows XP machine)
also the paths r set only for the current command prompt session
There was no solution to the issue other than somehow making the classpath shorter by moving the jar files into a folder like "C:\jars".
Fix for windows gradle long classpath issue. Fixes JavaExec tasks that error out with message "CreateProcess error=206, The filename or extension is too long"
Using the plugins DSL:
plugins {
id "com.github.ManifestClasspath" version "0.1.0-RELEASE"
}
Using legacy plugin application:
buildscript {
repositories {
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath "gradle.plugin.com.github.viswaramamoorthy:gradle-util-plugins:0.1.0-RELEASE"
}
}
apply plugin: "com.github.ManifestClasspath"
I had a similar issue here with a giant classpath definition inside a .bat file.
The problem was that this class path was also including the execution path into the giant path, its ok, its make sense.
In this context, the software was not able to run and the message "The input line is too long" appeared everytime.
Solution:
I just moved the all files to a shorter position.
For instance, I was trying to execute the software in a directory tree like:
c:\softwares\testing\testing_solution\one
and I moved the whole structure to a point like this
c:\test
The software worked very well.
It is not the best option, I know, but might help some one who is looking to a fast solution.
Tks
Have you tried stacking them?
set CLASS_PATH = c:\path
set ALT_A = %CLASS_PATH%\a\b\c;
set ALT_B = %CLASS_PATH%\e\f\g;
...
set ALL_PATHS = %CLASS_PATH%;%ALT_A%;%ALT_B%
Related
I have a CMakeList.txt which will build a Java project with Maven to a war file when running make, but when I run make install, it will rebuild it again before copy to the installation folder of Web Application.
How can I only build Java once with make but not again with make install? Here is the CMakeList.txt:
add_custom_target(JavaProject ALL
COMMAND ${MAVEN_EXECUTABLE} package
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
VERBATIM)
install(FILES "${JAVA_PROJECT_TARGET_DIR}/java_project.war"
DESTINATION ${WAR_DIR})
As the documentation of add_custom_target() says, custom targets are always considered out of date, which means they will re-build with each invocation of make which includes them.
What you want instead is a custom command to produce the .war file:
add_custom_command(
OUTPUT "${JAVA_PROJECT_TARGET_DIR}/java_project.war"
COMMAND ${MAVEN_EXECUTABLE} package
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
VERBATIM
)
This tells CMake how the file named "${JAVA_PROJECT_TARGET_DIR}/java_project.war" is produced when someone requests it. For files, CMake can generate dependency checks just fine, so it will not re-build needlessly. Note that you will probably also want to include some DEPENDS in that add_custom_command(), otherwise it will never rebuild once built(1).
Then, you need one more thing: a driver for the custom command. That is something that will depend on the command's OUTPUT and actually cause it to be built. So you'll add a custom target:
add_custom_target(
JavaProject ALL
DEPENDS "${JAVA_PROJECT_TARGET_DIR}/java_project.war"
)
Then, the sequence will be as follows:
During a make, JavaProject will be considered out of date (since it's a custom target) and will be built. This means its dependencies will be checked for up-to-datedness, and re-built if they're not up to date. That's what the custom command is for. After that, the custom target itself would run its COMMAND, but it doesn't have any, so nothing else happens.
On a subsequent make invocation, JavaProject will again be considered out of date and will thus be built. Its dependencies are checked again, but this time, they're up to date (since the .war already exists). It's therefore not built again. The custom target still has no COMMAND, so nothing further happens.
This "custom target as driver for custom commands" approach is very a idiomatic piece of CMake code, and you will see it in many projects which produce additional files which do not participate in further build steps (such as documentation).
(1) If the list of dependencies is very large, you want to move it to a separate files and include that. Something like this:
In CMakeLists.txt:
include(files.cmake)
add_custom_command(
OUTPUT "${JAVA_PROJECT_TARGET_DIR}/java_project.war"
COMMAND ${MAVEN_EXECUTABLE} package
DEPENDS ${MyFiles}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
VERBATIM
)
In files.cmake:
set(MyFiles
a/file1.java
a/file2.java
a/b/file1.java
a/c/file1.java
# ... list all files as necessary
)
This keeps the CMakeList itself readable, while allowing you to explicitly depend on all you need.
Although Angew has an excellent answer but unfortunately it does not work as I expect (i.e: when update source code folder and run make, it will not build the war again).
Here is the way to solve what I wanted:
set(CMAKE_SKIP_INSTALL_ALL_DEPENDENCY TRUE)
Then when I run make it will build and make install will just copy to the installation folder.
When I used to write libraries in C/C++ I got into the habit of having a method to return the compile date/time. This was always a compiled into the library so would differentiate builds of the library. I got this by returning a #define in the code:
C++:
#ifdef _BuildDateTime_
char* SomeClass::getBuildDateTime() {
return _BuildDateTime_;
}
#else
char* SomeClass::getBuildDateTime() {
return "Undefined";
}
#endif
Then on the compile I had a '-D_BuildDateTime_=Date' in the build script.
Is there any way to achieve this or similar in Java without needing to remember to edit any files manually or distributing any seperate files.
One suggestion I got from a co-worker was to get the ant file to create a file on the classpath and to package that into the JAR and have it read by the method.
Something like (assuming the file created was called 'DateTime.dat'):
// I know Exceptions and proper open/closing
// of the file are not done. This is just
// to explain the point!
String getBuildDateTime() {
return new BufferedReader(getClass()
.getResourceAsStream("DateTime.dat")).readLine();
}
To my mind that's a hack and could be circumvented/broken by someone having a similarly named file outside the JAR, but on the classpath.
Anyway, my question is whether there is any way to inject a constant into a class at compile time
EDIT
The reason I consider using an externally generated file in the JAR a hack is because this is) a library and will be embedded in client apps. These client apps may define their own classloaders meaning I can't rely on the standard JVM class loading rules.
My personal preference would be to go with using the date from the JAR file as suggested by serg10.
I would favour the standards based approach. Put your version information (along with other useful publisher stuff such as build number, subversion revision number, author, company details, etc) in the jar's Manifest File.
This is a well documented and understood Java specification. Strong tool support exists for creating manifest files (a core Ant task for example, or the maven jar plugin). These can help with setting some of the attributes automatically - I have maven configured to put the jar's maven version number, Subversion revision and timestamp into the manifest for me at build time.
You can read the contents of the manifest at runtime with standard java api calls - something like:
import java.util.jar.*;
...
JarFile myJar = new JarFile("nameOfJar.jar"); // various constructors available
Manifest manifest = myJar.getManifest();
Map<String,Attributes> manifestContents = manifest.getAttributes();
To me, that feels like a more Java standard approach, so will probably prove more easy for subsequent code maintainers to follow.
I remember seeing something similar in an open source project:
class Version... {
public static String tstamp() {
return "#BUILDTIME#";
}
}
in a template file. With Ant's filtering copy you can give this macro a value:
<copy src="templatefile" dst="Version.java" filtering="true">
<filter token="BUILDTIME" value="${build.tstamp}" />
</copy>
use this to create a Version.java source file in your build process, before the compilation step.
AFAIK there is not a way to do this with javac. This can easily be done with Ant -- I would create a first class object called BuildTimestamp.java and generate that file at compile time via an Ant target.
Here's an Ant type that will be helpful.
Unless you want to run your Java source through a C/C++ Preprocessor (which is a BIG NO-NO), use the jar method. There are other ways to get the correct resources out of a jar to make sure someone didn't put a duplicate resource on the classpath. You could also consider using the Jar manifest for this. My project does exactly what you're trying to do (with build dates, revisions, author, etc) using the manifest.
You'll want to use this:
Enumeration<URL> resources = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
This will get you ALL of the manifests on the classpath. You can figure out which jar they can from by parsing the URL.
Personally I'd go for a separate properties file in your jar that you'd load at runtime... The classloader has a defined order for searching for files - I can't remember how it works exactly off hand, but I don't think another file with the same name somewhere on the classpath would be likely to cause issues.
But another way you could do it would be to use Ant to copy your .java files into a different directory before compiling them, filtering in String constants as appropriate. You could use something like:
public String getBuildDateTime() {
return "#BUILD_DATE_TIME#";
}
and write a filter in your Ant file to replace that with a build property.
Perhaps a more Java-style way of indicating your library's version would be to add a version number to the JAR's manifest, as described in the manifest documentation.
One suggestion I got from a co-worker
was to get the ant file to create a
file on the classpath and to package
that into the JAR and have it read by
the method. ... To my mind that's a
hack and could be circumvented/broken
by someone having a similarly named
file outside the JAR, but on the
classpath.
I'm not sure that getting Ant to generate a file is a terribly egregious hack, if it's a hack at all. Why not generate a properties file and use java.util.Properties to handle it?
MATLAB is configured to search its static java class path before searching the user-modifiable dynamic path. Unfortunately, the static path contains quite a number of very old public libraries, so if you are trying to use a new version you may end up loading the wrong implementation and get errors.
For instance, the static path contains an old copy of the google-collections.jar, which has long been supplanted by Google's guava library and which has some of the same class names (e.g. com.google.common.base.Objects). As a result, if you invoke a Guava method that uses a newer method of one of such a class, you will end up getting surprising NoSuchMethodErrors because the google-collections jar is found first.
As of R2012b, MATLAB lets you specify additional jars to add to the static path by putting a javaclasspath.txt file in your preferences folder, but that adds jars to the end of the path, and doesn't let you override jars that are built into MATLAB.
So what is the best way around this?
I got an official response from Mathworks:
As of MATLAB R2013a (also in R2012b), classes can be added to the front of the static Java class path by including the following line in javaclasspath.txt:
<before>
Any directory that is after this line in javaclasspath.txt will be added to the front of the static Java class path. This is an undocumented use of javaclasspath.txt as of R2013a.
But overall in MATLAB, the ability to add classes to the front of the static Java classpath is not available through javaclasspath.txt in MATLAB 8.0 (R2012b).
MATLAB searches for classpath.txt in the following order:
In the startup directory. As of MATLAB 8.0 (R2012b) a warning will be shown if the file is found there and it will be ignored.
In the first directory on the MATLABPATH environment variable. (This environment variable is used in the bin/matlab shell script on Linux and in general is not used by the end-user).
In the toolbox/local directory.
Although the MATLABPATH environment variable of point 2 is normally not used by end-users we can use it in a workaround to allow reading a custom classpath.txt outside of the toolbox/local directory.
On Windows:
You will need to create the MATLABPATH environment variable. The first directory on it should be your directory with the custom classpath.txt AND you will also need to add the toolbox\local directory as second option. So from a cmd prompt you could do:
set MATLABPATH=c:\Users\user\Documents\myMATLABClasspath;c:\Program Files\MATLAB\R2012b
\toolbox\local
matlab.exe
One hack that appears to work is to add the jar to the top of the classpath.txt file that can be found in your MATLAB installations toolbox/local folder. Unfortunately, this is automatically generated and may get rewritten at some unspecified time, such as when you install new toolboxes, so this approach would require you to have some way to notice when this happens and reapply the hack.
If you're distributing a jar that's intended to be used with matlab, it may be better to use proguard as described at http://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava.
If you specify that all of your classes and their (public) fields and methods are to be preserved and include guava as a program jar (not a library), then it will rename all of guava's methods and update your compiled bytecode to reference the new names.
It seems a bit hackish, but depending on the audience, it may be significantly easier than teaching your users about static vs. dynamic classpath, and it won't break any matlab code that depends on the old behavior.
Instead of obfuscating the package as suggested by #user2443532, I have found it easier to "shade" the conflicting package instead of obfuscating it - unless you actually need obfuscation. One easy way to do this is to build your package using Maven and use the maven-shade-plugin. Internal calls are modified automatically, so you don't need to modify any of the Java code.
Direct calls from Matlab will need to be modified - for example, calls to com.opensource.Class become shaded.com.opensource.Class.
For more info on shading, see What is the maven-shade-plugin used for, and why would you want to relocate Java packages?
In a Makefile, I would simply specify
.a.b:
do_something_with_a $< --output=$*.b
How do I do exactly that in ant? I haven't seen this in any documentation, and can't get it to work with ant.
For those who don't know make, the first part specifies two kind of files, those with .a extension and those with .b extension. Further it specifies that the bs are dependent on the as, such that if filename.b does not exist or is older than filename.a, the command rule should be applied. The command rule specifies that the command "do_something_with_a" is called with the arguments "filename.a" and "--output==filename.b" for every such filename.
I will test your solution with ant-1.8.4 and accept the first that works. File locations may be specified.
This abstract dependency is not possible in ant.
When I used to write libraries in C/C++ I got into the habit of having a method to return the compile date/time. This was always a compiled into the library so would differentiate builds of the library. I got this by returning a #define in the code:
C++:
#ifdef _BuildDateTime_
char* SomeClass::getBuildDateTime() {
return _BuildDateTime_;
}
#else
char* SomeClass::getBuildDateTime() {
return "Undefined";
}
#endif
Then on the compile I had a '-D_BuildDateTime_=Date' in the build script.
Is there any way to achieve this or similar in Java without needing to remember to edit any files manually or distributing any seperate files.
One suggestion I got from a co-worker was to get the ant file to create a file on the classpath and to package that into the JAR and have it read by the method.
Something like (assuming the file created was called 'DateTime.dat'):
// I know Exceptions and proper open/closing
// of the file are not done. This is just
// to explain the point!
String getBuildDateTime() {
return new BufferedReader(getClass()
.getResourceAsStream("DateTime.dat")).readLine();
}
To my mind that's a hack and could be circumvented/broken by someone having a similarly named file outside the JAR, but on the classpath.
Anyway, my question is whether there is any way to inject a constant into a class at compile time
EDIT
The reason I consider using an externally generated file in the JAR a hack is because this is) a library and will be embedded in client apps. These client apps may define their own classloaders meaning I can't rely on the standard JVM class loading rules.
My personal preference would be to go with using the date from the JAR file as suggested by serg10.
I would favour the standards based approach. Put your version information (along with other useful publisher stuff such as build number, subversion revision number, author, company details, etc) in the jar's Manifest File.
This is a well documented and understood Java specification. Strong tool support exists for creating manifest files (a core Ant task for example, or the maven jar plugin). These can help with setting some of the attributes automatically - I have maven configured to put the jar's maven version number, Subversion revision and timestamp into the manifest for me at build time.
You can read the contents of the manifest at runtime with standard java api calls - something like:
import java.util.jar.*;
...
JarFile myJar = new JarFile("nameOfJar.jar"); // various constructors available
Manifest manifest = myJar.getManifest();
Map<String,Attributes> manifestContents = manifest.getAttributes();
To me, that feels like a more Java standard approach, so will probably prove more easy for subsequent code maintainers to follow.
I remember seeing something similar in an open source project:
class Version... {
public static String tstamp() {
return "#BUILDTIME#";
}
}
in a template file. With Ant's filtering copy you can give this macro a value:
<copy src="templatefile" dst="Version.java" filtering="true">
<filter token="BUILDTIME" value="${build.tstamp}" />
</copy>
use this to create a Version.java source file in your build process, before the compilation step.
AFAIK there is not a way to do this with javac. This can easily be done with Ant -- I would create a first class object called BuildTimestamp.java and generate that file at compile time via an Ant target.
Here's an Ant type that will be helpful.
Unless you want to run your Java source through a C/C++ Preprocessor (which is a BIG NO-NO), use the jar method. There are other ways to get the correct resources out of a jar to make sure someone didn't put a duplicate resource on the classpath. You could also consider using the Jar manifest for this. My project does exactly what you're trying to do (with build dates, revisions, author, etc) using the manifest.
You'll want to use this:
Enumeration<URL> resources = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
This will get you ALL of the manifests on the classpath. You can figure out which jar they can from by parsing the URL.
Personally I'd go for a separate properties file in your jar that you'd load at runtime... The classloader has a defined order for searching for files - I can't remember how it works exactly off hand, but I don't think another file with the same name somewhere on the classpath would be likely to cause issues.
But another way you could do it would be to use Ant to copy your .java files into a different directory before compiling them, filtering in String constants as appropriate. You could use something like:
public String getBuildDateTime() {
return "#BUILD_DATE_TIME#";
}
and write a filter in your Ant file to replace that with a build property.
Perhaps a more Java-style way of indicating your library's version would be to add a version number to the JAR's manifest, as described in the manifest documentation.
One suggestion I got from a co-worker
was to get the ant file to create a
file on the classpath and to package
that into the JAR and have it read by
the method. ... To my mind that's a
hack and could be circumvented/broken
by someone having a similarly named
file outside the JAR, but on the
classpath.
I'm not sure that getting Ant to generate a file is a terribly egregious hack, if it's a hack at all. Why not generate a properties file and use java.util.Properties to handle it?