We Keep Coding

Why does the Java Extension Mechanism not check the classpath for an optional package implementing LocaleServiceProvider?

I have created a maven project L and written a Java extension (i.e. an optional package) implementing (i.e. extending) the (abstract) service providers that implement (i.e. extend) LocaleServiceProvider, to support a dialect (let's call it xy) that isn't normally supported by the JRE. (I do not want to use the CLDR extension that came with Java 8, even though I'm running 8.141.)
The project compiles, and produces a jar with a META-INF/services folder that contains the provider-configuration files in UTF-8 with the qualified provider class names being on a line that ends with a line feed (\n).
I have then declared a maven dependency in my project P on the locale project L, and I thought that that would work, because the tutorial states
The extension framework makes use of the class-loading delegation
mechanism. When the runtime environment needs to load a new class for
an application, it looks for the class in the following locations, in
order:
[...]
The class path: classes, including classes in JAR files,
on paths specified by the system property java.class.path. If a JAR
file on the class path has a manifest with the Class-Path attribute,
JAR files specified by the Class-Path attribute will be searched also.
By default, the java.class.path property's value is ., the current
directory. You can change the value by using the -classpath or -cp
command-line options, or setting the CLASSPATH environment variable.
The command-line options override the setting of the CLASSPATH
environment variable.
Maven puts all dependencies on the classpath, I believe.
Yet when I run my unit test in P (in IntelliJ; L is on the classpath), it fails:
#Test
public void xyLocalePresent() {
Locale xy = new Locale("xy");
assertEquals("P not on classpath", xy, com.example.l.Locales.XY); // access constant in my Locale project L; should be equals to locale defined here
SimpleDateFormat df = (SimpleDateFormat) DateFormat.getDateInstance(DateFormat.SHORT, xy);
assertEquals("dd/MM/yy", df.toPattern()); // fails; L specifies the short date pattern as dd/MM/yy
}
I have to start it with -Djava.locale.providers=SPI,JRE -Djava.ext.dirs=/path/to/project/L/target. If I do that, it works, indicating that L's service providers were loaded successfully (indicating the jar's structure is ok).
NB: the Java 8 technotes say that the order SPI,JRE is the default.
Why, oh why does it not work when I just put L on the classpath? Why do I have to point to it explicitly?
Update: After going through the JavaDoc again, I just saw this (emphasis mine):
Implementations of these locale sensitive services are packaged using
the Java Extension Mechanism as installed extensions.
That explains things. :(
Is there any way to make this work by just putting L on the classpath when P runs, i.e. without having to install L (or having to use -D system properties)? (P uses maven, Struts2 and Spring, if that helps...)

In more complex applications, such as web servers (e.g. Tomcat), there are multiple ClassLoaders, so each WebApp served by the web server can be kept independent.
The extension mechanism is for extending the core Java functionality, i.e. features available globally within the running JVM (the web server). As such, they must be loaded by the System ClassLoader.
The standard way to add an extension to the code Java runtime, is to either
add the Jar file to the JRE_HOME/lib/ext folder
add extra folders to be searched by specifying the java.ext.dirs system property
You could also just add it to the Bootstrap ClassPath yourself, but that might cause problems if the Security Manager is activated. Not sure about that part. So it's best to do it the official way.
Note that the classpath defined by the CLASSPATH environment variable, or the -cp command-line option, does not define the Bootstrap ClassPath.
To learn more, read the Java documentation "How Classes are Found".

Issue running java program from batch file, runs fine in IDE

I'm doing some basic java homework for a class on my new laptop - issue is, I can't seem to get the program to compile and run from my batch file using the directions the instructor gave me.
I've set the Path variable to my JDK inside the Environment Variables settings.
My program is a simple shipping program to keep track of shipment information - I have the program working flawlessly in NetBeans (which our instructor advised us to use for developing the code), but he's going to be testing them using batch files, so we're also advised to test them on our systems with one we create prior to turning them in - pretty straightforward.
Issue is, I cannot seem to get this to work. I've never done it before, but I've used .bat files to compile and run C++ programs, as well as using makefiles on a unix system, so I feel like I'm absolutely stupid for not figuring this out on my own, but none of my searches have returned any fruitful solutions that help at all.
My program consists of 3 .java files:
Shipment.java - an interface that contains abstracted methods that are implemented in the ShipmentHW1 class
ShipmentHW1.java - a class that implements the abstracted methods from Shipment and has constructors, etc to create a usable object
TestShipment.java - the main class of this program, which utilizes and creates ShipmentHW1 objects based on preset parameters. This is super duper basic stuff here, and again, it runs perfectly fine inside the NetBeans IDE.
The instructions given to us state to have the batch file inside the package directory (which in this case I've set aside a seperate folder on my desktop titled "shipping", which is the package name - shouldn't be any issues there), where the 3 .java files are located as well.
They say if you don't need to explicitly list the path to the JDK, then you can simply have
javac TestShipment.java
java TestShipment.java
pause
Afterwards I get errors talking about how it "cannot find symbol Shipment s = new ShipmentHW1();"
I've tried adding imports, but since they're in the same package it shouldn't even be an issue.
Directory path is
C:\Users\X\Desktop\shipping
All 7 files are contained within:
TestShipment.java
TestShipment.class
Shipment.java
Shipment.class
ShipmentHW1.java
ShipmentHW1.class
doHW1.bat
Does anyone have any idea? I can provide more information if I've been too vague
Also, I'm on Windows 8 if that makes any difference

Solved
Batch file now reads
javac TestShipment.java Shipment.java ShipmentHW1.java
cd ..
java shipment.TestShipment
pause
and it works like a charm. Anyone have any ideas why I had to call the package.class instead of just compiling it regularly?

Try doing
javac TestShipment.java
java TestShipment
pause

Without seeing the contents of TestShipment.java, I'll assume you have some dependency on the Shipment and ShipmentHW1 classes. As such, when you execute a program that uses the TestShipment class, you need to have the .class files for each of the three (and any other dependencies).
So you will have to compile Shipment.java and ShipmentHW1.java as well before running your java command. If they are in the same package, you're good, if not, you will have to specify an appropriate value for the -cp option.
When running java with a class name, you need to specify the fully qualified class name.

If your .java files are declared to be in the 'shipping' package, then you probably need to be running java from the parent directory of 'shipping', e.g.
cd <path>/shipping
javac TestShipment.java
cd ..
java shipping/TestShipment

Make instance of class in java at runtime

In my program I generate classes dynamically but when I try:
String[] args = {"-d","D:\\path\\build\\classes","-s","D:\\path\\src","http://services.aonaware.com/DictService/DictService.asmx?WSDL"};
WsImport.doMain(args);
URL url = new URL("file:D:/path/build/classes/com/aonaware/services/webservices/");
URLClassLoader urlClassLoader = new URLClassLoader(new URL[]{url});
Class service = Class.forName("com.MyClass",true,urlClassLoader );
I recieve java.lang.ClassNotFoundException
If I run one more time the program (in Eclipse), then it is working. Actually I only have to refresh the project in Eclipse and then its working
Does anybody see the problem

Sounds like a classpath issue. Make sure that the class in question (or the jar containing it) is compiled and is in the classpath.
What exactly do you mean by "generating classes dynamically"? If you generate the java source file for a class, it needs to be compiled first into a class file, before the classloader can pick it up. Maybe Eclipse does that in the second round, that's why it is working then.

You would generally use a ClassLoader like URLClassLoader to load classes dynamically at runtime.

Use the three argument form of Class.forName() which requires you specify the ClassLoader.
[Java API Link][1]
[1]: http://java.sun.com/javase/6/docs/api/java/lang/Class.html#forName(java.lang.String, boolean, java.lang.ClassLoader)

Read the docs on URLClassLoader: Any URL that ends with a '/' is assumed to refer to a directory.
Also, you probably should use '/' as a separator instead of \\.
Addendum:
Well, your code works perfectly for me -- if there are actual compiled Java classes in the directory specified as an URL. But when you say
In my program I generate classes
dynamically
do you generate Java source or bytecode directly? If it's source code, do you also compile it from your app?

Referencing libraries for a Linux executable

I have written an application in Java and succesfully compiled it using gcj. It worked surprisingly well, but I've run into a hurdle: I can only run the executable through a shell script, because I have to specify the library paths.
The libraries I need are SWT, Xerces and GNU-crypto.
Is there a way to statically link the libraries when compiling in gcj, or is this not a good idea? Alternatively, can I specify the (relative) library path while compiling?
Presently, my shell script looks like this:
#!/bin/sh
export LD_LIBRARY_PATH=./libs/:$LD_LIBRARY_PATH
exec ./MyJavaApp $*

The idea is to make the static field "sys_paths" null so that it would construct the paths from the changed value.
See the post here (Post#223 by AjaySingh516) http://forums.sun.com/thread.jspa?messageID=3744346#3744346
Class clazz = ClassLoader.class;
Field field = clazz.getDeclaredField("sys_paths");
boolean accessible = field.isAccessible();
if (!accessible)
field.setAccessible(true);
Object original = field.get(clazz);
// Reset it to null so that whenever "System.loadLibrary" is called, it
// will be reconstructed with the changed value.
field.set(clazz, null);
try {
// Change the value and load the library.
System.setProperty("java.library.path", "./libs/");
System.loadLibrary("mylibapr");
} finally {
// Revert back the changes.
field.set(clazz, original);
field.setAccessible(accessible);
}
.
gcj System Properties (See: Standard properties supported by libgcj)
http://gcc.gnu.org/onlinedocs/gcj/System-properties.html
.
Solution#2
: Set System environment variable at compile time
http://linux.die.net/man/1/gcj
For this you have to use parameter -Djava.library.path=./libs/ with gcj
From gcj manual (above link):
--main= CLASSNAME
This option is used when linking to specify the name of the class whose "main" method should be invoked when the resulting executable is run.
-Dname[=value]
This option can only be used with "--main". It defines a system property named name with value value. If value is not specified then it defaults to the empty string. These system properties are initialized at the program's startup and can be retrieved at runtime using the "java.lang.System.getProperty" method.
I have never worked with gcj but as per docs these system properties can be retrieved at runtime, hence it will be portable to other systems as well.
Also see: http://gcc.gnu.org/wiki/Statically_linking_libgcj?action=show&redirect=Statically+linking+libgcj

To answer the first part of your question -
From the gcj man page:
"Static linking of libgcj may cause essential parts of libgcj to be omitted. Some parts of libgcj use reflection to load classes at runtime. Since the linker does not see these references at link time, it can omit the referred to classes. The result is usually (but not always) a "ClassNotFoundException" being thrown at runtime. Caution must be used when using this option."
For the static linking of the other libraries, I'm not sure. I haven't had a reason to do that.
Linux executables are different than Windows. Normally you have a "launcher" or some such depending on which exact windowing system you are using. You set the icon in that, not on the executable itself. Usually, launch scripts are used to set any environment that you need for running the executable. Again, this all depends on your exact desktop window system.

Why are you using an AOT? I would suggest reading the following article. One of the drawbacks that it mentions for AOTs is the following...
Dynamic applications. Classes that the application loads dynamically at runtime may be unavailable to the application developer. These can be third-party plug-ins, dynamic proxies and other classes generated at runtime and so on. So the runtime system has to include a Java bytecode interpreter and/or a JIT compiler.

How to set a long Java classpath in Windows?

I'm trying to run a particular JUnit test by hand on a Windows XP command line, which has an unusually high number of elements in the class path. I've tried several variations, such as:
set CLASS_PATH=C:\path\a\b\c;C:\path\e\f\g;....
set CLASS_PATH=%CLASS_PATH%;C:\path2\a\b\c;C:\path2\e\f\g;....
...
C:\apps\jdk1.6.0_07\bin\java.exe -client oracle.jdevimpl.junit.runner.TestRunner com.myco.myClass.MyTest testMethod
(Other variations are setting the classpath all on one line, setting the classpath via -classpath as an argument to java"). It always comes down to the console throwing up it's hands with this error:
The input line is too long.
The syntax of the command is incorrect.
This is a JUnit test testing a rather large existing legacy project, so no suggestions about rearranging my directory structure to something more reasonable, those types of solutions are out for now. I was just trying to gen up a quick test against this project and run it on the command line, and the console is stonewalling me. Help!

The Windows command line is very limiting in this regard. A workaround is to create a "pathing jar". This is a jar containing only a Manifest.mf file, whose Class-Path specifies the disk paths of your long list of jars, etc. Now just add this pathing jar to your command line classpath. This is usually more convenient than packaging the actual resources together.
As I recall, the disk paths can be relative to the pathing jar itself. So the Manifest.mf might look something like this:
Class-Path: this.jar that.jar ../lib/other.jar
If your pathing jar contains mainly foundational resources, then it won't change too frequently, but you will probably still want to generate it somewhere in your build. For example:
<jar destfile="pathing.jar">
<manifest>
<attribute name="Class-Path" value="this.jar that.jar ../lib/other.jar"/>
</manifest>
</jar>

Since Java 6 you can use classpath wildcards.
Example: foo/*, refers to all .jar files in the directory foo
this will not match class files (only jar files). To match both use: foo;foo/* or foo/*;foo. The order determines what is loaded first.
The search is NOT recursive

Use An "Argument File" on Java 9+
In Java 9+, the java executable supports providing arguments via a file. See
https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-4964-AE84-121F5F6CF111.
This mechanism is explicitly intended to solve the problem of OS limitations on command lengths:
You can shorten or simplify the java command by using #argument files
to specify a text file that contains arguments, such as options and
class names, passed to the java command. This let’s you to create java
commands of any length on any operating system.
In the command line, use the at sign (#) prefix to identify an
argument file that contains java options and class names. When the
java command encounters a file beginning with the at sign (#) , it
expands the contents of that file into an argument list just as they
would be specified on the command line.
This is the "right" solution, if you are running version 9 or above. This mechanism simply modifies how the argument is provided to the JVM, and is therefore 100% compatible with any framework or application, regardless of how they do classloading i.e. it is completely equivalent to simply providing the argument on the command line as usual. This is not true for manifest-based workarounds to this OS limitation.
An example of this is:
Original command:
java -cp c:\foo\bar.jar;c:\foo\baz.jar
can be rewritten as:
java #c:\path\to\cparg
where c:\path\to\cparg is a file which contains:
-cp c:\foo\bar.jar;c:\foo\baz.jar
This "argument file" also supports line continuation characters and quoting for properly handling spaces in paths e.g.
-cp "\
c:\foo\bar.jar;\
c:\foo\baz.jar"
Gradle
If you are encountering this issue in Gradle, see this plugin, which converts your classpath automatically into an "argument file" and provides that to the JVM when doing exec or test tasks on Windows. On Linux or other operating systems it does nothing by default, though an optional configuration value can be used to apply the transformation regardless of OS.
https://github.com/redocksoft/classpath-to-file-gradle-plugin
(disclaimer: I am the author)
See also this related Gradle issue -- hopefully this capability will eventually be integrated into Gradle core: https://github.com/gradle/gradle/issues/1989.

(I suppose you do not really mean DOS, but refer to cmd.exe.)
I think it is less a CLASSPATH limitation than an environment size/environment variable size limit. On XP, individual environment variables can be 8k in size, the entire environment is limited to 64k. I can't see you would hit that limit.
There is a limit on windows that restricts the length of a command line, on WindowsNT+ it is 8k for cmd.exe. A set command is subject to that restriction. Can it be you have more than 8k worth of directories in your set command? You may be out of luck, then - even if you split them up like Nick Berardi suggested.

Thanks to Raman for introducing a new solution to a pathing problem for Java 9+. I made a hack to bootRun task that allows using everything already evaluated by gradle to run java with argument files. Not very elegant but working.
// Fix long path problem on Windows by utilizing java Command-Line Argument Files
// https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-4964-AE84-121F5F6CF111
// The task creates the command-line argument file with classpath
// Then we specify the args parameter with path to command-line argument file and main class
// Then we clear classpath and main parameters
// As arguments are applied after applying classpath and main class last step
// is done to cheat gradle plugin: we will skip classpath and main and manually
// apply them through args
// Hopefully at some point gradle will do this automatically
// https://github.com/gradle/gradle/issues/1989
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
bootRun {
doFirst {
def argumentFilePath = "build/javaArguments.txt"
def argumentFile = project.file(argumentFilePath)
def writer = argumentFile.newPrintWriter()
writer.print('-cp ')
writer.println(classpath.join(';'))
writer.close()
args = ["#${argumentFile.absolutePath}", main]
classpath = project.files()
main = ''
}
}
}

If I were in your shoes, I would download the junction utility from MS : http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx and then map your
"C:\path" to say, "z:\" and "c:\path2" to say, "y:\". This way, you will be reducing 4 characters per item in your classpath.
set CLASS_PATH=C:\path\a\b\c;C:\path\e\f\g;
set CLASS_PATH=%CLASS_PATH%;C:\path2\a\b\c;C:\path2\e\f\g;
Now, your classpath will be :
set CLASS_PATH=z\a\b\c;z\e\f\g;
set CLASS_PATH=%CLASS_PATH%;y:\a\b\c;y:\e\f\g;
It might do more depending on your actual classpath.

I think you are up the creek without a paddle here.
The commandline has a limit for arguments to call a programm.
I have 2 sugestion you could try.
First, prior to running the junit tests, you can let a script/ant_task create JARs of the various classes on the classpath.
Then you can put the JARs on the classpath, which should be shorter.
Another way you could try is to create an antscript to run JUNIT,
in ANT there should not be such a limit for classpath entries.

As HuibertGill mentions, I would wrap this in an Ant build script just so that you don't have to manage all of this yourself.

You could try this
#echo off
set A=D:\jdk1.6.0_23\bin
set B=C:\Documents and Settings\674205\Desktop\JavaProj
set PATH="%PATH%;%A%;"
set CLASSPATH="%CLASSPATH%;%B%;"
go to a command prompt and run it twice(no idea why....i have to do so on a windows XP machine)
also the paths r set only for the current command prompt session

There was no solution to the issue other than somehow making the classpath shorter by moving the jar files into a folder like "C:\jars".

Fix for windows gradle long classpath issue. Fixes JavaExec tasks that error out with message "CreateProcess error=206, The filename or extension is too long"
Using the plugins DSL:
plugins {
id "com.github.ManifestClasspath" version "0.1.0-RELEASE"
}
Using legacy plugin application:
buildscript {
repositories {
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath "gradle.plugin.com.github.viswaramamoorthy:gradle-util-plugins:0.1.0-RELEASE"
}
}
apply plugin: "com.github.ManifestClasspath"

I had a similar issue here with a giant classpath definition inside a .bat file.
The problem was that this class path was also including the execution path into the giant path, its ok, its make sense.
In this context, the software was not able to run and the message "The input line is too long" appeared everytime.
Solution:
I just moved the all files to a shorter position.
For instance, I was trying to execute the software in a directory tree like:
c:\softwares\testing\testing_solution\one
and I moved the whole structure to a point like this
c:\test
The software worked very well.
It is not the best option, I know, but might help some one who is looking to a fast solution.
Tks

Have you tried stacking them?
set CLASS_PATH = c:\path
set ALT_A = %CLASS_PATH%\a\b\c;
set ALT_B = %CLASS_PATH%\e\f\g;
...
set ALL_PATHS = %CLASS_PATH%;%ALT_A%;%ALT_B%