I have to dynamically load a class with the same name from some JAR, but different implementation, multiple times.
I'm creating an evaluator backend and I have to dynamically load classes and test them.
The tests are JUnit classes that instantiate the classes which should be tested, this is a simple example:
package evaluator.tests;
import static org.junit.Assert.*;
import org.junit.*;
import evaluator.tested.*;
public class KTest {
private K tested;
#Before
public void setup() {
tested = new K();
}
#Test
public void returnsTrueTest() {
assertTrue(tested.returnsTrue());
}
}
The rest of my application would need to receive JAR files from users which would contain implementations of the K class which is being tested above. Then, KTest would have to run on their K classes, not the ones in the application.
I know how to dynamically load a class, but I don't know how to make a test work with it, and not the one which I made.
One of the solutions I came up with was to isolate testing in a completely new class, e.g. Evaluation, create a new class loader in that class, and make it load all referenced classes. After creating the class loader, it would load the K class from the JAR file.
This would mean that each time a user submits his JAR, a separate Evaluation would be instantiated, it would create its own class loader and start the JUnit test. When that happens, the test would use the user's implementation of K, and not the default one.
Is this possible, and how can it be done?
I read that class loaders always ask their parent whether a class is already loaded. This would mean that I would have to somehow "flush" all the classes that I loaded dynamically from the JAR file in Evaluation, so that they would be unloaded and then loaded again in another Evaluation.
Load class K, test class K, unload class K, repeat with different K.
Yes you can do this.
If you for example use java.net.URLClassLoader with null as parent: new URLClassLoader( urlArray , null ). Then the bootstrap ClassLoader will be used as parent for your ClassLoader.
Here is an example Class which simply uses a new classloader two reload a class.
package com.anarsoft.agent.regression;
import java.net.URL;
import java.net.URLClassLoader;
public class TestClassLoading {
public static boolean field = false;
public static void main(String[] args) throws Exception
{
URL[] urlArray = new URL[] { TestClassLoading.class.getProtectionDomain().getCodeSource().getLocation().toURI().toURL() };
URLClassLoader firstClassloader = new URLClassLoader( urlArray , null );
Class firstClass = firstClassloader.loadClass("com.anarsoft.agent.regression.TestClassLoading");
firstClass.getField("field").setBoolean(null,true);
System.out.println(firstClass.getField("field").getBoolean(null)); // true
URLClassLoader secondClassloader = new URLClassLoader( urlArray , null );
Class secondClass = secondClassloader.loadClass("com.anarsoft.agent.regression.TestClassLoading");
System.out.println(secondClass.getField("field").getBoolean(null)); // false
// the static field is false since its a new loaded class
}
}
I've made a blog post on this subject, in which the problematics is explained a bit further.
After some research I think I've found a way to accomplish exactly what I want. I haven't implemented this system yet (will edit the answer once I do), so I'd like some feedback from people who are more experienced in this regard.
This is how I understand URLClassLoaders (and classloaders in general) work:
URLClassLoader.loadClass() gets automatically called if it’s the current classloader (for the method/class executing). The call is first delegated to its parent classloader, and if nothing is found it uses its own custom findClass() to load it off one of the URLs it has. loadClass() here (and the classloading logic) is just inherited from the regular ClassLoader.
The default behavior of this method (loadClass()) is to first delegate the search to the parent, and if parent cannot find the class, only then to call its own findClass(). This method (findClass()) is, by default (in ClassLoader), left unimplemented, and you are supposed to implement it yourself, by essentially getting the bytecodes of the class from somewhere (e.g. a file or a network) and calling defineClass() on them.
Once you call defineClass() on some class, and only then, are you registered as the classloader of this class. In all other cases classloading is either delegated to your parent (and you are not the classloader of the class you are loading, paradoxically), or you throw a ClassNotFoundException. You cannot change the classloader of some class at runtime, it is set once it is loaded, and constant.
All of my test classes will try to getClass().getClassLoader().loadClass() all of the classes that they reference - including my custom test classes (this is the regular behavior of all classes, not just my tests, to be clear). As long as they’re using standard classes, and other classes from my application that are not to-be-tested classes, this classloading method should be delegated further on to the application classloader. However, as soon as they try to load a to-be-tested class, they need to get their own, specifically loaded, custom version of it.
The use case of my application is that a user submits a JAR with some class, then a test that expects this class to have certain methods is ran on the class from that JAR (using JUnit), and then the results are sent back to the user.
The solution is as follows:
Implement basic versions of the to-be-tested classes so that the tests would compile and run without any submitted JARs. This could (more likely, should) be done by leveraging polymorphism, but is not planned at the moment (this means that the user should most likely extend the “basic” version of the class himself, locally, before sending the class to be tested).
Extend URLClassLoader and re-work the classloading logic.
Do the necessary checks (is the class already loaded) that exist in the default implementation.
Try to findClass yourself, if it’s not in your URLs, throw a ClassNotFoundException.
If the class has already been loaded, or if has been loaded just now from some URL, return the class.
If the class has neither been loaded before (by this classloader), nor is it in one of the URLs, delegate the search to your parent.
If the parent returns the class, return the class, if not, throw ClassNotFoundException (it will actually be thrown by the parent).
For each JAR file with a class that is sent:
Instantiate the custom URLClassLoader
Add the JAR to it and the specific test class as well
Ask it to load the test class. At this point my custom classloading logic kicks in, and it loads my test straight from the disk - anew, without delegating to its parent classloader. Why? It calls defineClass() on my test class, which sets this custom URLClassLoader as the parent of the test class
Give the test class to JUnit, which then instantiates it and begins testing
Once one of my tests are running, each time they reference any class, they will call their own custom classloader’s loadClass(). First they will search the URLs - so in case they’re referencing a to-be-tested class, it will be loaded from there. In case they are referencing some other application class, or a system class, the custom classloading logic will just delegate this call and an already-loaded (presumably) class would be returned.
As I’ve said - I haven’t implemented this yet, and I would really like if you’d point out my mistakes in the comments.
Resources from which I've gathered this information:
http://www.onjava.com/pub/a/onjava/2003/11/12/classloader.html
http://www.devx.com/Java/Article/31614
http://www.onjava.com/pub/a/onjava/2005/01/26/classloading.html
http://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html#loadClass%28java.lang.String,%20boolean%29
You need to make sure that jar you are loading classes from is unknown to JVM system class loader (not on the classpath). Then system class loader will not be loading classes from the jar thus making your custom class loader to load from this jar.
You can fool it.
Cass that you want to reset should not be in the classpath
Can implement a interface or extend a class that is in the classpath (useful to type cast or need to use reflection to 'discover' and call methods, OR the calling class calls another method that has the code the calls the class(es) under test
every time you want to test, rename the jar, load from the jar, using a URL class loader sub class. On init of your loader null is fine.
So if you load a1.jar first time, load from a2.jar next time and make sure a1.jar is moved to another folder
Related
I create my custom class loader :
new URLClassLoader(urls, Thread.currentThread().getContextClassLoader());
where urls is a new Url("java.util.TimeZone")
After that I load class by name :
Class<?> newTimeZoneClass = loader.loadClass("java.util.TimeZone");
and newTimeZoneClass==TimeZone.class returns true.
The main reason of that my class loader load class from parent loader.
How to fix it?
You cannot do this. The Java security model prevents any class loader creating a class in the "java.*" hierarchy. This is hard-coded in the native code of the JVM, so there is no workaround.
Additionally, the standard class loaders follow the delegation model of asking the parent class loader to load the class before they try to, so you always get the same class instance. Special class loaders are used by application containers to invert this delegation for application specific classes.
There are a few ways to do this anyway.
First, TimeZone is an abstract class and the actual implementation is normally sun.util.calendar.ZoneInfo. As this is not in the "java.*" hierarchy, you can create multiple copies in your class loaders.
Second, you can sub-class TimeZone, and delegate all methods to a JVM provided instance, adding your own functionality as you do so. I've used this to make TimeZone instances singletons in some of my applications.
Third, as the JDK is open source, you can copy the all the code for TimeZone and its sub-classes into your own application, and then you can have as many versions of the class as you like.
If you want to change the TimeZone instances returned by the static methods in TimeZone, these delegate to ZoneInfo and you will have to either use reflection to change the outcome. If you know Aspect-J or equivalent, you could also intercept the call.
As it is mentioned in Java Doc of public URLClassLoader(URL[] urls, ClassLoader parent):
Constructs a new URLClassLoader for the given URLs. The URLs will be
searched in the order specified for classes and resources after first
searching in the specified parent class loader.
I assume you should create a CustomClassLoader ccl = new CustomClassLoader(); or use another constructor by passing an AccessControlContext object - URLClassLoader(URL[] urls, ClassLoader parent, AccessControlContext acc)
or URLClassLoader(URL[] urls, AccessControlContext acc).
There are some articles on creating new class loaders:
https://www.baeldung.com/java-classloaders
https://www.javaworld.com/article/2077260/learn-java-the-basics-of-java-
https://www.oodlestechnologies.com/blogs/Creating-Custom-Class-Loader-In-JAVA/class-loaders.html
Oracle JavaDocs explains that IllegalAccessError is
"Thrown if an application attempts to access or modify a field, or to
call a method that it does not have access to."
I try to load a class dynamically and I get this exception.
if I understand correctly when you use a classloader to load a class with a private package dynamically IllegalAccessError happens
the class I am trying to load is using
org.xml.sax.helpers.SecuritySupport
which also states in their description in the following url
http://grepcode.com/file/repository.springsource.com/org.apache.xmlcommons/com.springsource.org.apache.xmlcommons/1.3.4/org/xml/sax/helpers/SecuritySupport.java
that
Unfortunately, we can't load the class using reflection
* because the class is package private. And the class has
* to be package private so the APIs aren't exposed to other
* code that could use them to circumvent security. Thus,
* we accept the risk that the direct reference might fail
* on some JDK 1.1 JVMs, even though we would never execute
* this code in such a case. Sigh...
how can I dynamically load it anyway? I have to get it to work.
also if I get an error when I use a classloader, I cannot recover from that, so how can I know in advance that I cannot load this class?
thanks in advance to anyone who helps
The statement “we can't load the class using reflection because the class is package private” doesn’t make any sense, as can be shown easily:
package somepackage;
class BaseClass {
public static void main(String[] args) throws ReflectiveOperationException {
BaseClass obj=(BaseClass)
Class.forName("somepackage.SubClass").newInstance();
obj.aMethod();
}
void aMethod() {
System.out.println("base class");
}
}
class SubClass extends BaseClass {
#Override
void aMethod() {
System.out.println("method overridden by subclass");
}
}
This works flawlessly, printing method overridden by subclass replicated the actual use case of that SecuritySupport class.
However, since that class obviously serves the purpose of allowing a transition between Java 1.1 and Java 1.2, it might be possible that there were such restrictions twenty years ago, when this transition happened.
Your use case, however, is entirely different. You say that you are trying to load a class which “is using org.xml.sax.helpers.SecuritySupport”, which doesn’t imply that it is using said class via Reflection, but as shown above, that doesn’t matter anyway. It either case, it would only work, if the class is in the same package, whether you load the class “dynamically” or not.
There are only two possible scenarios.
If the class is truly within the same package, which at runtime implies that it also has been loaded by the same class loader, which would require that is also part of the JRE, if the JRE’s org.xml.sax.helpers package defines a SecuritySupport class, then the class can access the class within the same package.
If you are trying to load a class via a different ClassLoader from a different code source, it will not be of that package, even if you’d give it a qualified name of the org.xml.sax.helpers.SomeClass form. If the JRE’s org.xml.sax.helpers package happens to define a SecuritySupport class, all non-JRE classes would be in a different package. When it tries to access that class, which is not part of the official API, it doesn’t work.
Note that all standard class loaders follow a delegation model trying to resolve a name through their parent class loader first, which is the reason why they all would prefer the JRE’s org.xml.sax.helpers.SecuritySupport class, if there is one. With non-standard class loaders, you could have different, unrelated classes with that qualified name, being in different runtime packages.
In that second scenario, the question arises, why your class is using that class. In 2017, there’s rarely a need to differentiate between Java 1.1 and Java 1.2 and the functionality offered by that class is also only relevant for a class within the privileged code source of the JRE (or different code sources with different privileges in general).
I have a class A extends B.
I've created a CustomClassLoader extends ClassLoader to use defineClass(className, byte[], offset, length).
I've instanciate a new CustomClassLoader(Thread.currentThread().getContextClassLoader()).
So the parent of my CustomClassLoader is the ClassLoader from the current thread.
I've modified B class using ASM framework. I've write my modified class in a .class file and use a decompiler to be sure it works. And it works.
I've added modified B class to my CustomClassLoader
I've set the Thread.currentThread().setContextClassLoader() with my CustomClassLoader.
I've load A using Class.forName(String, true, the CustomClassLoader).
But the loaded B class seems to be the orginal class.
What did I wrong ?
If you need more info, a detailed topic is on my GitHub.
Java classloaders first search the parent classloader before looking in the child.
The loadClass method in ClassLoader performs these tasks, in order,
when called to load a class:
If a class has already been loaded, it returns it.
Otherwise, it delegates the search for the new class to the parent class loader.
If the parent class loader does not find the class, loadClass calls the method findClass to find and load the class.
(Understanding Extension Class Loading - Oracle)
If you want to change that order, you need to override the loadClass method as well but there are many caveats and it's not advisable unless you understand classloading very well.
The easier option is to make sure that the parent class loader cannot find the original class B.
There are several things to know:
For most things, dealing with the thread’s context class loader is obsolete, as it has no impact. It’s more like a convention; setting it has an impact if there’s some other code querying and using it. For the standard class loading process, it doesn’t have any meaning. It’s unfortunate that the documentation doesn’t mention that and make it look like a relevant thing. Perhaps, it was intended to have more meaning when it was added.
As pointed out by Erwin Bolwidt, when loading A via your custom loader, it will delegate to its parent loader, returning a class A loaded by the parent.
When resolving class references, the JVM will always use the defining loader of the referrer. So when the reference from A to B is resolved, the JVM will always use the parent loader which defined the class A
The last point implies that even if you modify your custom class loader to look up its own classes first instead of following the standard model of querying the parent first, it doesn’t solve the issue if it has no own A, as then, it still returns the parent’s A whose references will be resolved using the parent. Since you are invoking defineClass before asking for A, the lookup order doesn’t matter at all, as your custom loader has an already defined B that it returned if anyone ever asked it for B…
So you could let your custom loader also load and define A. Or you use Reflection with access override to defineClass on the system ClassLoader before it loads B. The cleanest solution is to implement the class modification logic as a Java Agent which can use the Instrumentation API to intercept and change the definition of B right at its loading time.
I'm sure this seems obvious to most java programmers -- and I "think" I know the answer, but my issue is that I need to use a custom class loader. I'd like to use the custom class loader and still be able to use the classes in my code as I would any included class.
Example:
// returns my class loader -- set earlier on the thread.
ClassLoader cl = Thread.currentThread().getContextClassLoader();
Class loadedVersion = Class.forName("org.example.MyClass"); // works
// loadedVersion's class loader is also equal to cl
// generates "java.lang.NoClassDefFoundError" at runtime on the following line
org.example.MyClass m = new org.example.MyClass();
How would I go about doing this so I can have the class loaded using the custom class loader, and still be able to use my classes "plainly" in the editor?
Edit: The revised title is misleading -- the classes load just fine. As indicated in my original post. The issue, my understanding, is that I am currently unable to use the classes in regular declaration statements (in the editor as I would use any java.* classes).
Update, I hope this better explains what I'm trying to do.
// set the thread's class loader
ClassLoader cloader = new MyClassLoader(); // internal mess to load bytecode omitted
Thread.currentThread().setContextClassLoader(cloader);
// This works:
Method m = cloader.loadClass("my.SuperClass").getMethod("doStaticMagic", new Class<?>[] {});
m.invoke(null, null);
// This does not work:
my.SuperClass.doStaticMagic(); // NoClassDefFoundError
This is the "issue". I want to use my classes "plainly" (I really don't know the right word) in the editor. I don't want to have to load each individual method -- isn't that what the classloader is for?
I think you want to use the newInstance() method. Since you have the "Class loadedVersion", you can use it to call the method.
It would be better if you have only interface on classpath and loading the implementation by classloader or whatever, As in the code below:
Class loadedVersion = Class.forName("org.example.MyClass");
IMyClass m = (IMyClass)loadedVersion.newInstance();
where IMyClass is the interface which is implemented by org.example.MyClass.
In this way you can absolutly decouple the implenetations of your classes from the client code
EDIT
NoClassDefFoundError is also arised when you have an exception while initializing static variables or executing static block initializer. Check youe class, maybe it throws such exceptions.
Here is my solution (answer) to my original question. I still don't consider this ideal, but this does work.
I created a "bootstrap" application that first loads all the custom 3rd party jars, and then loads and calls my application class. This way I get auto-complete and source hints for the 3rd party jars.
I hope this helps someone, or invites further comments.
This question already has answers here:
Difference between Loading a class using ClassLoader and Class.forName
(9 answers)
Closed 5 years ago.
When dynamically loading a class, when is it appropriate to use
Class.forName("SomeClass");
and when should I use
ClassLoader.getSystemClassLoader().loadClass("SomeClass");
Or, are they two ways of doing the same thing?
They are quite different!
As stated in the documentation for Class.forName(String),
Returns the Class object associated with the class or interface with the given string name. Invoking this method is equivalent to: Class.forName(className, true, currentLoader)
(true here refers to do you want to initialize the class?)
On the other hand, ClassLoader.loadClass(String):
Invoking this method is equivalent to invoking loadClass(name, false).
(here, the boolean has nothing to do with initialization; but if you check loadClass(String, boolean) documentation, you will see that all it does is load the class, not initialize it).
The first one (Class.forName("SomeClass");) will:
use the class loader that loaded the class which calls this code
initialize the class (that is, all static initializers will be run)
The other (ClassLoader.getSystemClassLoader().loadClass("SomeClass");) will:
use the "system" class loader (which is overridable)
not initialize the class (say, if you use it to load a JDBC driver, it won't get registered, and you won't be able to use JDBC!)
Suppose you are coding a web application that will be executed on a container such as Tomcat. What Tomcat does is create a class loader for each web application (so that it can unload the webapps later and release memory -- you need a dedicated class loader for this to work!). In this situation, you can see that both calls will yield quite different results!
For more detailed (and authoritative) information on class loading and initialization, check sections 12.2 and 12.4 of the latest (3rd) edition of the Java Language Specification.
Class.forName() uses the caller's classloader and initializes the class (runs static intitializers, etc.)
loadClass is a ClassLoader method, so it uses an explicitly-provided loader, and initializes the class lazily (on first use).
Note that there's a Class.forName() that also takes a ClassLoader.
Class.forName() load and initialize the class. In class loader subsystem it executes all the three phases i.e. load, link, and initialize phases.
ClassLoader.loadClass() behavior, which delays initialization until the class is used for the first time. In class loader subsystem it executes only two phases i.e. load and link phases.
For example:
class MyClass {
static {
System.out.println("static block in MyClass");
}
}
public class TestCase1 {
public static void main(String... args) throws Throwable {
Class.forName("A");
}
} //The above TestCase1 produce output: static block in MyClass
public class TestCase2 {
public static void main(String... args) throws Throwable {
ClassLoader.getSystemClassLoader().loadClass("MyClass");
}
} //The above TestCase2 not produce any output
They are basically doing the same thing. The ClassLoader used may be different though. Class.forName uses the ClassLoader you get from this.getClass().getClassLoader() whereas your other code specifies to use the system class loader.
In most applications this will be the same class loader but in more complicated environments such as a J2EE app or an applet this may not be the case.
ClassLoader is an abstract class, however your application is always loaded by a classloader, there could be custom class loaders such as network classloader or any other source.
On the other hand Class in itself represents classes and interfaces and the class Class has a forName function that uses the current class loader in which your application is running by default to load the class.
Here is the source for the Class.forName which in turn invokes the calling classloader.
public static Class<?> forName(String className)
throws ClassNotFoundException {
return forName0(className, true, ClassLoader.getCallerClassLoader());
}
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/ClassLoader.html
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Class.html#forName(java.lang.String)
Hint: Primordial class loader
http://docs.oracle.com/javase/1.4.2/docs/guide/security/spec/security-spec.doc5.html
I love Class loading in java...
It really depends on what context the application is being run in. You will get different results if you are using it from a web context as opposed to just a command line program.
I've also run into problems depending on what your ClassPath looks like and what I was expecting to happen.
This JavaWorld article explains a good deal about it.