I'm currently trying to load classes into my application so that I can then filter out those that don't contain any test / #Test-methods. I want to run these tests in my application afterwards.
So far, so good - except it seems like the URLClassLoader I'm using (or probably any ClassLoader) doesn't actually reload classes that are located on my applications classpath.
More precisely, a user of my application starts by selecting a number of .java source files. Those are then copied to a temporary location and a number of regex match/replace pairs are applied to the copy of the original source files. Next, the copies are compiled, and I then want to load those compiled .class-files into my application so I can run them as tests.
In most cases, this works as desired.
Unfortunately, if the fully qualified class name (FQCN) of any of the selected files is identical to the FQCN of a class that is part of this application (such as tests for this application), the ClassLoader ignores the specified path (to %temp%\myfolder\) and the (updated) version of the class file located there, but instead uses the already present/loaded version of the class.
After some research, this behaviour can be expected according to the docs (emphasis mine):
• The loadClass method in ClassLoader performs these tasks, in order, when called to load a class:
- If a class has already been loaded, it returns it.
- Otherwise, it delegates the search for the new class to the parent class loader.
- If the parent class loader does not find the class, loadClass calls the method findClass to find and load the class.
I tried using findClass, but it's unfortunately not visible.
Hence my question - does anyone know how to force java / the ClassLoader to actually load the class from the specified path, ignoring any - FQCN-wise - identical existing classes?
A classloader first delegates to its parent classloader, which is how it determines "if a class has already been loaded". If you want to force a classloader to load a new class, one way is to insert another classloader in the chain which fails to load/find the same class. Very quick, incomplete example:
class FirewallLoader extends ClassLoader {
public FirewallLoader(ClassLoader parent) {
super(parent);
}
public loadClass(String name, boolean resolve) {
if (name.equals("some.class.Xyz")) {
throw ClassNotFoundException();
}
super.loadClass(name, resolve);
}
}
You make the "FirewallLoader" the parent or your URLClassLoader, and then your URLClassLoader will load new versions of whatever classes the Firewall loader filters out.
Related
General idea: I'm writing on a loader for java that allows dynamically reloading classes to allow for changing the implementation, without restarting the entire program to keep the main application running and minimize downtimes. Every external piece of code is grouped by "modules", each module has a main class with a "onEnable, postEnable, onDisable" entry/exit point and can consist of any amount of classes. To load a module, the class containing the entry point is specified, then loaded. I'll reference them as "modules" and "additional classes" in the following, "module" being the class containing the above mentioned functions by implementing the "public interface Module", "additional classes" refer to everything the module would use on runtime but isn't a Module by itself (e.g. we have a Module called "Car implements Module", and that module requires a class "Engine" to function -> "Car" is the module, "Engine" is an additional class")
Code of what I'm doing to load a module initially (name is a String containing the full classname including path, example given later):
Class<?> clazz = mainLoader.loadClass(name);
Module module = (Module) clazz.newInstance();
addLoadedModule(module);
enableLoadedModule(module);
And here's how I reload the module when it's already existing, so that I can override the implementation. "m" is an instance of the current implementation of the Module that is supposed to be reloaded.
boolean differs = false;
Class<?> newClass = null;
try (URLClassLoader cl = new URLClassLoader(urls, mainLoader.getParent()))
{
// Try to load the class and check if it differs from the already known one
newClass = cl.loadClass(m.getClass().getName());
differs = m.getClass() != newClass;
}
catch (IOException | ClassNotFoundException e)
{
// Class couldn't be found, abort.
e.printStackTrace();
return;
}
if (!differs)
{
// New class == old class -> no need to reload it
return;
}
Module module = null;
try
{
// Try to instantiate the class
module = (Module) newClass.newInstance();
}
catch (InstantiationException | IllegalAccessException e)
{
// Can't instantiate, abort
e.printStackTrace();
return;
}
// Check versions, only reload if the new implementation's version differs from the current one. Version is a custom annotation, don't worry about that; the version check works fine
Version oldVersion = m.getClass().getAnnotation(Version.class);
Version newVersion = module.getClass().getAnnotation(Version.class);
if (oldVersion.equals(newVersion))
{
return;
}
// And if everything went well, disable and remove the old module from the list, then add and enable the new module.
disableModule(m);
modules.remove(m);
modules.put(module, false);
enableLoadedModule(module);
This is the mainLoader, urls is an URL[] pointing to the location containing the external classes to load:
mainLoader = new URLClassLoader(urls, this.getClass().getClassLoader());
The problem arises when I try to RE-load an implementation, that requires multiple classes:
Module of class A requires class B to function. This is what happens when I try to dynamically load, then reload class A:
load A -> "Sure, but I'll need B with it." -> automatically loads B -> "Here ya go, A works fine now."
reload A -> "Sure, but I'll need B with it." -> crashes because B couldn't be found
Both classes are located in the exact same folder, structure like this:
Class A implements Module: com/foo/bar/A.class
Class B: com/foo/bar/B.class
urls: ["com/foo/bar/"]
I call the function with load("com.foo.bar.A"), which works when attempting to load it the first time, but fails when trying to reload it as described above.
It works fine when trying to load a "single class module", the problem arises when the module relies on an additional external class. I tried using different classloaders to use as the parent for the URLClassLoader in the reloading process, those being the sysloader, Module.class.getClassLoader(), mainLoader (using that one, it won't ever find the new class definition because it already knows about it and therefor won't even attempt to load it from the drive again) and the mainLoader.getParent(), the classloader of the old module, and the parent of the modules classloader.
I'm probably just overseeing something obvious, but I can't figure out why it would manage to load the "extra" classes the first time, but fail when I reload the base class...
If you need any debug outputs or exact errors let me know, I replaced the debug outputs with comments explaining what does what so I got a fairly detailed log of what's happening when, but I didn't seem it to be necessary as it goes through the entire "check and then load" process just fine, it crashes when trying to enable the module. The "onEnable" method of the module requires the additional class B, that's where it fails. As I said, if you need the implementation of the classes A and B, Module, any other code or the debug outputs let me know and I'll add them in as requested.
There's a few things you can try:
Create an extension of UrlClassLoader so that you can track when it loads a class and what class loader is used to load the class.
Your other issue is make sure none of these classes are available on the "default" class path as that will cause that version to use. You are not overriding the default class loading behaviour which is to check the parent for the class first.
The other issue you're probably facing relates to the way the VM caches classes - I'm not entirely sure how this works - but from what I've experienced it seems that once a class is loaded it puts it in a shared storage space so that it does not load the class again. This shared space class will not be unloaded until the class loader that loaded it goes unreachable.
The solution lies in the classloader being closed and deleted as soon as the loading of the initial class is done, due to the class loader being only existant in the try/catch clause. I solved the issue by storing the classloader in a map until a new implementation of the module is loaded, then I can discard the old loader and store the new one instead.
I am learning the process of loading java class and encounter some confusion.
I know when loading a java class, current classLoader wont load the java class directly and it will delegate to its parent classLoader(a recursive process) until it parent cant load this class.
The question is that ：what is the current classLoader? Bootstrap? Extension? App?
how to get the current classLoader?.
and I know there is an API:
xxx.Class.getClassLoader();
but I am not sure whether the return value is currentClassLoader. I think it should be the classLoader which load this java class in reality.
To describe my question more detail I will give an example.
I get the follow content in a blog.
ThreadContextClassLoader is used to deal with java SPI, Interface is defined in java core lib and loaded by Bootstrap ClassLoader and third party implement these interface then the jar are loaded by AppClassLoader
Solution: traditional classLoader cant deal with this case,because it cant discovery the third party jar when we use the third party implement in core lib.
most of the above that I can understand but the solution make me confusion:
for example, the Interface CoreA and class CoreB are in java core lib and should be loaded by Bootstrap ClassLoader and the AImpl is a implement of A by third party and should be loaded by AppClass loader.
the code segment as below:
public Interface CoreA{
void add();
}
public Interface AImpl implements CoreA{
void add(){};
}
public class B{
public void demo(){
a = new AImpl();
}
}
then if we reference B in main method, then we will load B because the class loader of B is Bootstrap then about AImpl the current Loader is Bootstrap so it cant be found?
I dont know whether it is as what I guess?
Any advice will be appreciated.
Generally speaking you are right, it can't be found. Let me show you the following example. Let's say we have 3 classes: A, B and Main like these:
public class A {
public String a() {
return "a";
}
}
public class B {
public String b() {
return new A().a();
}
}
public class Main {
public static void main(String... args) {
System.out.println(new B().b());
}
}
And we package these classes into correspondent jars: a.jar, b.jar and place Main.class into the working directory. After that let's test the following scenarios:
1) Everything (A.class, B.class, Main.class) is loaded by system classloader and works fine:
$ java -cp .:a.jar:b.jar Main
a
2) B.class is loaded by system classloader and A.class is loaded by bootstrap classloader and everything still works fine because system classloader delegates loading to bootstrap classloader (just because bootstrap classloader can load it):
$ java -Xbootclasspath/a:a.jar -cp .:b.jar Main
a
3) A.class is loaded by system classloader and B.class is loaded by bootstrap classloader (your case). In this scenario during loading of B.class current classloader is bootstrap classloader, but it can't found B.class and fails:
$ java -Xbootclasspath/a:b.jar -cp .:a.jar Main
Exception in thread "main" java.lang.NoClassDefFoundError: A
at B.b(B.java:4)
at Main.main(Main.java:4)
Lets take a look at the last example more carefully. What's happening here:
Try to find class with public static main(String[] args) method
1.1. system classloader hasn't loaded it yet so delegates to extension classloader
1.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
1.3. bootstrap classloader hasn't loaded it yet and tries to load, it can't load it and returns control to extension classloader
1.4. extension classloader tries to load, it can't load it and returns control to system classloader
1.5. system classloader loads Main.class
Main.class is processed and we try to load B.class with current classloader system classloader
2.1. system classloader hasn't loaded it yet so delegates to extension classloader
2.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
2.3. bootstrap classloader hasn't loaded it yet and loads B.class
B.class is processed and and we try to load A.class with current classloader bootstrap classloader
3.1. bootstrap classloader hasn't loaded it yet and tries to load and
fails
I hope it will help you.
When a class A attempts to load another class B, the ClassLoader that loaded A is the current ClassLoader. The word current vaguely refers to the execution context - e.g. how do you end up in the method that triggers the current class loading call.
There is no method - say getCurrentClassLoader - that simply gives the current ClassLoader, but there are api methods that internally use the concept of current ClassLoader. For example, Class.forName(String className)
If you check how that method is implemented, it tells you the meaning of "current class loader":
public static Class<?> forName(String className) throws ClassNotFoundException {
Class<?> caller = Reflection.getCallerClass();
return forName0(className, true, ClassLoader.getClassLoader(caller), caller);
}
If you can get hold of a Class instance, you can always ask for the loader behind it by calling Class::getClassLoader() method. That will be your current class loader. The tricky bit, however, is to decide if the loader is bootstrap, or extension, or system class loader. The reason of the trickiness is that it is implementation specific, and you can always implement your own class loading mechanism.
The example given by #dmitrievanthony is an example of how things can become really complicated. It is a similar situation faced by JNDI, and the reason why the hack Thread.getContextClassLoader() was introduced. More about it here
Quote the most relevant piece from the article:
... By definition, a current classloader loads and defines the class to which your current method belongs. This classloader is implied when dynamic links between classes resolve at runtime, and when you use the one-argument version of Class.forName(), Class.getResource(), and similar methods. It is also used by syntactic constructs like X.class class literals ...
the 'current classloader' is the real classloader(load this class in reality) of class that refer it.
e.g.
if the classLoader of class A is ext classloader and class A refer class B C D. Then the 'current classloader' of B C D is ext classLoader. And of course the 'current classLoader' of main class is System classLoader.
My StartApplet is small to keep startup quick.
It then downloads various classes in various jars using (URLClassLoader)getSystemClassLoader().
The problem I am experiencing is that there are several interfaces defined in the StartApplet which are passed to the dynamically downloaded classes using method invoke. I always get class not defined.
It seems the system class loader does not contain any StartApplet loaded classes including the interfaces.
So, I try loading in the interfaces into the systemClassLoader using a downloaded jar but I still get class not defined and I guess this is because the same class has been loaded in twice using difference class loaders and therefore is seen as two difference classes.
I tried loading the downloaded jars using the classloader of one of the interfaces(StartApplet) but there were errors.
I tried forgetting about the system class loader and instead creating a new URLClassLoader using the classloader of the interfaces(StartApplet) as the parant class loader but again errors occurred.
I have tried loading the dynamic jars into Thread.currentThread().getContextClassLoader() but again errors occurred.
My question...
Is there a way to dynamically load classes in jars using (URLClassLoader)getSystemClassLoader() and allow them to see/access and use the classes that have already been loaded by the instantiating applet??
some code example would be really nice.
Many Thanks.
The crux is the system class loader doesnt reference the applet class loader.
The applet cannot start with any external jars so whatever classes it passes have to be loaded in with the applet.
I just need the dynamically loaded classes in the systemclassloader to be able to use the classes loaded with the applet.
please help.
ps. here are some snipets...
private void addPath(String _path)
{
try{
File f=new File(_path);
if(!f.exists())return;
if(!f.isDirectory())return;
Method method=SYSTEM_CLASS_LOADER_CLASS.getDeclaredMethod("addURL",parameters);
method.setAccessible(true);
method.invoke(SYSTEM_CLASS_LOADER,new Object[]{f.toURI().toURL()});
}catch(Throwable _t){
handle(_t);
disconnect();}
}
private void addLibrary(String _name)
{
try{
Method method=SYSTEM_CLASS_LOADER_CLASS.getDeclaredMethod("addURL",parameters);
method.setAccessible(true);
method.invoke(SYSTEM_CLASS_LOADER,new Object[]{ClassLoader.getSystemResource(_name)});
}catch(Throwable _t){handle(_t);}
}
SYSTEM_CLASS_LOADER=(URLClassLoader)ClassLoader.getSystemClassLoader(); // DOESNT WORK
SYSTEM_CLASS_LOADER=(URLClassLoader)MyInterface.class.getClassLoader(); // DOESNT WORK
SYSTEM_CLASS_LOADER=(URLClassLoader)Thread.currentThread().getContextClassLoader(); // DOESNT WORK
private void callWithInterface(MyInterface _myI)
{
Class<?> myClass=Class.forName("dynamic.MyClass",true,SYSTEM_CLASS_LOADER);
Constructor<?> myConstructor=myClass.getConstructor();
Object myInstance=myConstructor.newInstance();
Method m=myClass.getMethod("MyTest",new Class<?>[]{MyInterface.class});
String s=(String)m.invoke(myInstance,new Object[]{_myI});
}
last line causes...
Thread=Thread[Thread-17,4,http://MyDomain/-threadGroup]
java.lang.ClassNotFoundException: MyInterface
java.net.URLClassLoader$1.run(-1)
java.net.URLClassLoader$1.run(-1)
java.security.AccessController.doPrivileged(-2)
java.net.URLClassLoader.findClass(-1)
java.lang.ClassLoader.loadClass(-1)
sun.misc.Launcher$AppClassLoader.loadClass(-1)
java.lang.ClassLoader.loadClass(-1)
java.lang.Class.forName0(-2)
java.lang.Class.forName(-1)
StartApplet.run(23759)
java.lang.Thread.run(-1)
I have figured it out..
The problem I had was caused by a jar name conflict causing the required classes to fail at loading. Once I realised this and corrected the problem I successfully enabled the dynamically loaded classes to access the applet loaded classes by loading the dynamically loaded classes using the applet class loader instead of the system class loader.
I modified my code using the following lines and other adjustments to suit...
MyDynamicClassLoader=new URLClassLoader(new URL[0],MyAppletLoadedInterface.class.getClassLoader());
method.invoke(MyDynamicClassLoader,new Object[]{MyDynamicClassLoader.getResource(DynamicJarName)});
MyDynamicClassLoader now holds references to all applet loaded classes and dynamically loaded classes with the ability to reference each other. For some reason the system class loader does not hold the applet loaded classes.
Regards
Penny
I am using reflection to get all methods from a specific class.
This class has references to class that not in my class path so I get an exception:
java.lang.NoClassDefFoundError:
On this:
Method methods[] = theClass.getDeclaredMethods();
Is it possible, somehow,to "skip" everything that is not in classpath?
Class.forName() will not load a class, whether it is or isn't in the classpath. It will only return a handle to a class that is already loaded.
A class gets loaded in one of 2 main ways:
1.)The class is referenced in the import statements(java.lang.* is imported automatically so every class in java.lang package is class-loaded from the start)
2.)A class is loaded using a call from a ClassLoader, in which case all of its dependencies are resolved. and loaded as well
So if you are trying to load a class outside of the classpath, or with dependencies outside the classpath, you need to subclass ClassLoader and tell it how to load your classes and their dependencies.
See ClassLoader specification here: http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/ClassLoader.html
Also, there are ready made subclasses of ClassLoader that may do what you want such as URL ClassLoader which will let you simply point the ClassLoader instance at the path, and load any classes in that path.
We know that we can override the system classloader with:
java -Djava.system.class.loader=com.test.MyClassLoader xxx
Then, since com.test.MyClassLoader itself is a class, by whom is it loaded?
How do we get the class file of this "meta" classloader?
The Bootstrap classloader is the parent of all classloaders and loads the standard JDK classes in lib directory of JRE (rt.jar and i18n.jar). All the java.* classes are loaded by this classloader.
The Extensions Classloader is the immediate child of the Bootstrap classloader. This classloader loads the classes in lib\ext directory of the JRE.
The System-Classpath classloader is the immediate child of the Extensions classloader. It loads the classes and jars specified by the CLASSPATH environment variable
You could try to inject your custom class loader by means of the java.system.class.loader property (see ClassLoader#getSystemClassLoader).
The Default System class loader is the parent for MyClassLoader instances.
From the Javadoc for ClassLoader.getSystemClassLoader()
If the system property "java.system.class.loader" is defined when this method is first invoked then the value of that property is taken to be the name of a class that will be returned as the system class loader. The class is loaded using the default system class loader and must define a public constructor that takes a single parameter of type ClassLoader which is used as the delegation parent.
The default system class loader itself is specific to the JVM implementation.
Tldr:
§ ..the value of that property is taken to be the name of a class that will be returned as the system class loader. The class is loaded using the default system class loader..
..Thus, if your classloader replaces X as the system class loader, then your classloader's parent will be X, the default system class loader.
(X is of a type like sun.misc.Launcher$AppClassLoader.)
More info can be found at docs.oracle.com - How the Java Launcher Finds Classes:
The Java launcher, java, initiates the Java virtual machine. The
virtual machine searches for and loads classes in this order:
Bootstrap classes - Classes that comprise the Java platform, including the classes in rt.jar and several other important jar files.
Extension classes - Classes that use the Java Extension mechanism. These are bundled as .jar files located in the extensions directory.
User classes - Classes defined by developers and third parties that do not take advantage of the extension mechanism. You identify
the location of these classes using the -classpath option on the
command lineor by using the CLASSPATH environment variable.
Tsmr:
We can prove that X is really the parent of our Classloader:
/** run with -Djava.system.class.loader=MyCL to use this classloader */
public class MyCL extends ClassLoader {
public MyCL(ClassLoader parent) { // this constructor must be public, else IllegalAccessException
super(parent);
}
}
This is our main code:
public class Main {
public static void main(String args[]) {
System.out.println("getSystemClassLoader(): " + ClassLoader.getSystemClassLoader());
ClassLoader cl = MyCL.class.getClassLoader();
System.out.println("Classloader of MyCL: " + cl);
Class type_of_cl = cl.getClass();
System.out.println("..and its type: " + type_of_cl);
ClassLoader cl_of_cl = class_of_cl.getClassLoader();
System.out.println("Classloader of (Classloader of MyCL): " + cl_of_cl);
}
}
This is the output (on my system) when run using the command java -Djava.system.class.loader=MyCL Main (cf. Eclipse run config):
getSystemClassLoader(): MyCL#1888759
Classloader of MyCL: sun.misc.Launcher$AppClassLoader#7fdcde
..and its type: class sun.misc.Launcher$AppClassLoader
Classloader of (Classloader of MyCL): null
We can see that MyCL's classloader is sun.misc.Launcher$AppClassLoader, which is the default system classloader.
(Per the language as seen in Oracle's other quote above, the default system classloader is also called the classloader of "User Classes". Screw Oracle for coming up with 2 names for the same thing.)
An appropriate answer would be:
And this also clarifies the original question.
When you do,
java -Djava.system.class.loader=com.test.MyClassLoader xxx
The -D option is for setting system properties in the Properties object of the java.lang.System instance that gets loaded as part of JVM startup. The option only changes the in-memory properties. On a next invocation of the class XXX or some other class, the default properties as part of System class would be loaded again. Here, you have set the java.system.class.loader property to the value com.test.MyClassLoader. In other words, you want to override the default system class loader (also called the bootstrap class loader) with your new system class loader for this invocation of your class XXX. The default system class loader, which was loaded as part of JVM startup, would lookup the in-memory java.system.class.loader property to find the name of your overriding system class loader--if set(in your case, set to MyClassLoader)--and load it from your classpath, which is, by default, the root of your working directory (that should contain XXX) or the one in the classpath variable, either as -cp or as environment variable CLASSPATH, if defined.
If your write in the main method of your class XXX
System.out.println(ClassLoader.getSystemClassLoader());
System.out.println(MyClassLoader.class.getClassLoader());
You should see
MyClassLoader
(the new system class loader that loads XXX)
sun.misc.Launcher$AppClassLoader
(the default system class loader that loaded your new system class loader)