I want to create dynamically a classloader for executing JSR223 script in a controlled environment but failing,
I'm trying remove/add jars using current(parent) ClassLoader, I tried solution Dynamically removing jars from classpath
public class DistributionClassLoader extends ClassLoader {
public DistributionClassLoader(ClassLoader parent) {
super(parent);
}
private Map<String, ClassLoader> classLoadersByDistribution =
Collections.synchronizedMap(new WeakHashMap<>());
private final AtomicReference<String> distribution = new AtomicReference<>();
#Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
final ClassLoader delegate = classLoadersByDistribution.get(distribution.get());
if (delegate != null) return Class.forName(name, true, delegate);
throw new ClassNotFoundException(name);
}
public void addDistribution(String key, ClassLoader distributionClassLoader){
classLoadersByDistribution.put(key,distributionClassLoader);
}
public void makeDistributionActive(String key){distribution.set(key);}
public void removeDistribution(String key){
final ClassLoader toRemove = classLoadersByDistribution.remove(key);
}
}
But it didn't include all my jars, in test this work
ClassLoader cl = this.getClass().getClassLoader();
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
But using the solution doesn't find class
ClassLoader cl = new DistributionClassLoader(this.getClass().getClassLoader());
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
Exception:
java.lang.ClassNotFoundException: org.springframework.http.HttpStatus
at com.DistributionClassLoader.loadClass(DistributionClassLoader.java:24)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
How can I select specific jars to add or remove from ClassLoader?
EDIT
I'm able to load jars using #czdepski answer but I still want to remove all/most classes except JDK's
Method sysMethod = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
sysMethod.setAccessible(true);
sysMethod.invoke(sysLoader, new Object[]{url});
You got the delegation wrong. You never check the parent class loader if it has this class.
If we look at the Javadoc for ClassLoader.loadClass(String,boolean) we find:
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built into the virtual machine is used, instead.
Invoke the findClass(String) method to find the class.
If the class was found using the above steps, and the resolve flag is true, this method will then invoke the resolveClass(Class) method on the resulting Class object.
Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method.
You did override loadClass, but don't do any delegation to it's parent ClassLoader.
Instead you call classLoadersByDistribution.get(distribution.get());, which is most likely null (hard to tell, but always expect WeakHashMap.get() to return null).
If delegate is not null, then you try to load the class from there. This means the loaded class won't use your ClassLoader to load new classes, but instead the ClassLoader you delegated to.
After all, this sounds like a XY Problem. You want to execute some code using the scripting API and somehow control the environment.
Did you try to use a SecurityManager?
About your comment that you need your own ClassLoader to create a ScriptEngineManager: This ClassLoader is used to search for ScriptEngineFactory implementations. This is done using a service provider interface.
If you don't use your own script engine, this should not matter to you.
If your goal is to add a few jars so the engine can use it, create a new URLClassLoader with the platform class loader as parent. (Or extension class loader, depends on the java version.)
Set this ClassLoader as Thread.setContextClassLoader() and create the ScriptEngine.
If you did choose the parent of the URLClassLoader correctly, it will not see classes loadable by the application class loader.
Related
I have this stack trace which is the FIRST time this class is loaded(I have a condition on the breakpoint for this class)....
The CompilingClassLoader is mine(ie. this one https://github.com/deanhiller/webpieces/blob/master/core/runtimecompile/src/main/java/org/webpieces/compiler/impl/CompilingClassloader.java )
I want this class that is being loaded in a way that it is plugin.getClass.getClassLoader is returning the CompilingClassLoader. Instead, it's classloader is AppClassLoader
This is causing many issues for plugin developers in that currently the workaround is a Thread context classloader. Is there a way to claim the compiling classloader loaded it but delegate to the super classloader?
The CRITICAL line is line 111 in the link above by the way. I want that line to use the super class to create the Class for me BUT I want it to look like I created the Class. ie. the Class returned should return 'my' classloader when clazz.getClassLoader() is called. this then fixes a ton of issues.
EDIT---------------------------------------
After further investigation. The CompilingClassLoader is very very tricky. You have these types of classes
Standard java.lang classes
webpieces startup classes
the applications compile on demand classes
webpieces plugin classes that sit below application classes and above them
libraries that sit below application classes AND above them
For #1, once I hit main, many of these like java.lang.Object are loaded by app classloader so this must remain true always so that you don't get the dreaded casting X to X yields ClassCastException
For #2, most of these are loaded by app classloader on startup as well
For #3, this is code that we compile OR recompile. To recompile, we throw away the CompilingClassLoader and recreate another one BUT cache bytes of all previously compiled classes that have not changed.
For #4 & #5, this is where it gets really really tricky!!! We have a ProdServerMeta (an application compiled class) that refers to plugins in webpieces and those plugins then sometimes load other application classes that need to compile on changes. Currently, we inject hibernate with the compiling classloader to avoid an issue.
The more I think about this, hibernate classes are loaded by app classloader right now because we return null when we can't compile a class delegating to the parent.
Now, I ran into these situations with CompilingClassLoader which is very hard to simplify into a classloader like Jan-Willem Gmelig Meyling's classloader below! Mainly because defineClass seems to be critical and missing in that example (and like what Olivier says below is very important).
Anyways, I ran into these things with trying the classloader below
loading of java.lang.* caused a SecurityException so I had to delegate that
After solving that, I ran into ClassCastException from SAXParserFactoryImpl to SAXParserFactoryImpl when slf4j tries to load configuration
This 2nd one makes total senses since Server.main creates a Logger and then every other class that creates a Logger creates it in a CompilingClassLoader. This leads me to the situation that
I cannot load ANYTHING that is loaded before the loading of the application
Next, being a webserver for customers, I also can't tell what is referenced in pre-app load vs. after app load.
I am coming to the conclusion that there is not a solution since as a developer writes code, I throw away the previous classloader and many classes have to continue to exist like Logger which is referenced by AppClassLoader on startup.
Of course, if I boot up with a CompilingClassLoader, I might stand a better chance in that I could 'try' to load everything perhaps but java.lang, BUT then I still run into the issue of some code after server bootup willl always be referenced by the FIRST CompilingClassLoader while later code on recompile will point to a new CompilingClassLoader.
I think the only solution then is for plugins to have to use the ContextClassLoader the more I think through this situation and that Olivier below is correct in it's not possible.
thanks,
Dean
Classloaders load classes from their parent class loaders first. You should be able to work around this by simply not setting a parent classloader for the classloader, and delegate the call for loadClass yourself.
A working example:
public class SomeClass implements Runnable {
#Override
public void run() {
System.out.println("Class loader for class: " + getClass().getClassLoader());
System.out.println("Class loader for thread: " + Thread.currentThread().getContextClassLoader());
System.out.println("Class loader for transitive dependency: " + SomeDependency.class.getClassLoader());
}
}
A transitive dependency:
public class SomeDependency {}
Test and classloader:
import java.net.URL;
import java.net.URLClassLoader;
public class Test {
public static class CustomClassLoader extends URLClassLoader {
ClassLoader delegate;
public CustomClassLoader(URL[] urls, ClassLoader parent) {
super(urls, null);
delegate = parent;
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
return super.loadClass(name);
} catch (ClassNotFoundException e) {
return delegate.loadClass(name);
}
}
}
public static void main(String... args) throws Exception {
URL location = SomeClass.class.getProtectionDomain().getCodeSource().getLocation();
CustomClassLoader customClassLoader = new CustomClassLoader(new URL[] { location }, ClassLoader.getSystemClassLoader());
Thread.currentThread().setContextClassLoader(customClassLoader);
Class<?> aClass = customClassLoader.loadClass("com.pkg.SomeClass");
Runnable runnable = (Runnable) aClass.newInstance();
runnable.run();
}
}
Output:
Class loader for class: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for thread: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for transitive dependency: sun.misc.Launcher$AppClassLoader#18b4aac2
Seems to do precisely what you want :)
This is the code of my Agent.jar
public class Agent
{
public static void agentmain(String s, Instrumentation instrumentation)
{
try
{
ClassLoader classLoader = null;
for (Class clazz : instrumentation.getAllLoadedClasses())
{
String className = clazz.getName();
if (className.equalsIgnoreCase("ave")) /* Just a class from the running Programm */
{
classLoader = clazz.getClassLoader();
}
}
/* In the Cheat.jar are Classes which im trying to load */
ClassLoader loader = URLClassLoader.newInstance(new URL[]{new URL("C:\\Users\\michi\\Desktop\\Injection\\Cheat.jar")}, classLoader);
Class.forName("de.simplyblack.client.client.module.Category", true, loader);
Class.forName("de.simplyblack.client.client.module.Module", true, loader);
Class.forName("de.simplyblack.client.client.module.ModuleManager", true, loader);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}
I am loading this with an extra Programm.
VirtualMachine virtualMachine = VirtualMachine.attach(id);
virtualMachine.loadAgent(new File("C:\\Users\\michi\\Desktop\\Injection\\Client.jar").getAbsolutePath());
virtualMachine.detach();
But this is not working.
Later I visit an Class, and make an call for the ModuleManager class.
If I Inject it, i get an
Class not found: de.simplyblack.client.client.module.ModuleManager
error.
Could you please tell me how I can fix that?
It would help me a lot!
Thanks.
When references within a class are resolved, its defining class loader is used. Your code identifies the defining class loader of the class you want to instrument, but then, creates a new class loader using it as parent loader.
When you ask that new loader to load your classes, the classes very likely are loaded, but they are only reachable through your newly created URLClassLoader. They are not available to other class loaders.
Instrumenting classes with code containing new dependencies to other classes can be very tricky. If the instrumented classes have been loaded by the application class loader, you can use Instrumentation.appendToSystemClassLoaderSearch(JarFile) to add your Cheat.jar to the search path, to make the classes available.
For classes loaded by other loaders, things get more complicated. If they follow the standard parent delegation model, you can use appendToBootstrapClassLoaderSearch(JarFile) to make your classes available to all these class loaders.
For a loader not following the delegation model, you would have to dig deeper, e.g. use Reflection with access override, to call defineClass on it making the classes available in that scope.
I have a jar:
/home/cole/lib/a.jar
And in this jar I have the following interface/classes (horrible names for illustration purposes only!):
CreatorInterface.java
Base.java (implements CreatorInterface.java)
AbstractBase.java (extends Base.java)
Implementation.java (extends AbstractBase.java)
In a separate project I have the following code:
final URL[] jars = new URL[] {
new File("/home/cole/lib/a.jar").toURL();
}
final URLClassLoader classLoader = new URLClassLoader(jars, null);
final Class<?> implementation = classLoader.loadClass("Implementation");
final CreatorInterface object = (CreatorInterface)implementation.newInstance();
However when I run the above, I get the following:
java.lang.ClassCastException: Implementation cannot be cast to CreatorInterface
Given Implementation is ultimately an instance of a class that implements CreatorInterface, why do I get the ClassCastException?
Update 1
This isn't a question about using URLClassLoader, the class is found ok, the problem appears to be in the instantiation. For example, the following code works fine:
final Object object = implementation.newInstance();
Update 2
As #davidxxx answered, I have the interface class twice (once in the jar and once in the project using it). Although the interface was the same, this was the cause of the issue.
However to make it work, I needed to fix my URLClassLoader like this, to avoid a ClassNotFoundException:
final ClassLoader parent = this.getClass().getClassLoader();
final URLClassLoader classLoader = new URLClassLoader(jars, parent);
This exception :
java.lang.ClassCastException: Implementation cannot be cast to
CreatorInterface
makes me think that you have very probably two distinct CreatorInterface classes : one included in the jar and another other coming from the client program that tries to load it.
Even if the two classes have the same name (qualified names), these are different classes for each classloader as here you use two unassociated classloaders.
You have the current classloader of the program that you run and this other classloader as you specified null as parent classloader :
final URLClassLoader classLoader = new URLClassLoader(jars, null);
So as you try to assign the object created by reflection to the CreatorInterface variable, the cast fails because two distinct CreatorInterface were loaded by each classloader and are used : one coming from the classloader of your client code and another coming from the the instantiated classloader.
Using a single classloader would solve the issue but a best practice would be including the jar in the classpath of the project and to ensure to have a single version of the classes provided in the jar.
To decouple things you should probably split the jar into 2 jars : an API jar that contains only the interface and an implementation jar that depends on the API jar and that contains other classes.
In the client code, add in the classpath only the API jar to be able to assign to a variable with the interface declared type.
About your second point :
This isn't a question about using URLClassLoader, the class is found ok, the problem appears to be in the instantiation. For example, the
following code works fine:
final Object object = implementation.newInstance();
In this case you don't refer the interface type.
You indeed assign the Implementation object to an Object and not to a CreatorInterface variable.
The correct/consistent interfaces and subclasses are loaded by the classloader but here you never give a chance to provoke a ClassCastException as you never assign it to a type of a duplicate class but Object that is defined a single time.
So the problem previously encountered cannot occur.
About the third point :
However to make it work, I needed to fix my URLClassLoader like this,
to avoid a ClassNotFoundException:
final ClassLoader parent = this.getClass().getClassLoader();
final URLClassLoader classLoader = new URLClassLoader(jars, parent);
It works because here you create a classloader associated to the parent classloader.
In fact if you did :
final URLClassLoader classLoader = new URLClassLoader(jars);
It would produce the same result as the URLClassLoader object created would use by default the delegation to the parent classloader (here the classloader that started your application).
I write myOwnClassLoader through extends the classLoader and override its findClass method. And meet a interesting problem:
Here is the code segment:
public class MyClassLoader extends ClassLoader {
#Override
public Class<?> findClass(String name) {
byte[] bt = loadClassData(name);
return defineClass(name, bt, 0, bt.length);
}
private byte[] loadClassData(String className) {
...
}
public static void main(String[] args) throws ClassNotFoundException{
MyClassLoader myClassLoader = new MyClassLoader();
//Class<?> clazz = myClassLoader.findClass("com.classLoader.BMW");
Class<?> clazz = myClassLoader.loadClass("com.classLoader.BMW");
System.out.println(clazz.getClassLoader());//////////////////
}
}
when execute
Class<?> clazz = myClassLoader.findClass("com.classLoader.BMW");
the output meet my expectation:
output:
com.classLoader.MyClassLoader#xxxxxx
but when execute
Class<?> clazz = myClassLoader.loadClass("com.classLoader.BMW");
the output go beyond my mind:
output:
sun.misc.Launcher$AppClassLoader#xxxx
expect output
com.classLoader.MyClassLoader#xxxxxx
here is code segment of loadClass method in ClassLoader
Class<?> c = findLoadedClass(name);
if (c == null) {
long t0 = System.nanoTime();
try {
if (parent != null) {
c = parent.loadClass(name, false);
} else {
c = findBootstrapClassOrNull(name);
}
.....
I debug it and find when we invoke loadClass at the first time the parent is sun.misc.Launcher$ExtClassLoader#6aa8ceb6 that is to say this reference is sun.misc.Launcher$AppClassLoader rather than com.classLoader.MyClassLoader#xxxxxx .
I search through internet and a blog define own classLoader tell that
it will set AppClassLoader as parent of MyClassLoader in default constructor.
I check it by
System.out.println(myClassLoader.getParent());
find the parent of MyClassLoader does be AppClassLoader how it achieve this, my default constructor do nothing and was this achieve in constructor method of its parent class ClassLoader? I read its parent constructor and still cant firgure out.
Anyway it seemd this cant explain why the this is AppClassLoader rather than MyClassLoader when we first invoke loadClass method.
And I guess it is because that *the classLoader of MyClassLoader is AppClassLoader but it is just a intuition I have no gist.
Everything will be appreciated.
Well, it’s all documented:
ClassLoader() constructor:
Creates a new class loader using the ClassLoader returned by the method getSystemClassLoader() as the parent class loader.
loadClass(String):
Loads the class with the specified binary name. … Invoking this method is equivalent to invoking loadClass(name, false).
loadClass(String,boolean-):
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built-in to the virtual machine is used, instead.
Invoke the findClass(String) method to find the class.
If the class was found using the above steps, and the resolve flag is true, this method will then invoke the resolveClass(Class) method on the resulting Class object.
Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method.
This is also mentioned in the class documentation of ClassLoader:
The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself. The virtual machine's built-in class loader, called the "bootstrap class loader", does not itself have a parent but may serve as the parent of a ClassLoader instance.
This logic ensures clean scoping, i.e. classes of the parent scope don’t refer to classes of a sub scope, and that there are no conflicts between classes of the same qualified name, but defined by different class loaders. Such classes may still exist in distinct scopes, but not in nested scopes. But class loaders are not enforced to follow this rule. The delegation model was introduced with Java 2.
My application uses JDBC database drivers. I load these from a jar file, db2jcc.jar in the case of DB2 which I'm currently working with. With this jar in the classpath, everything is fine, but I have a requirement to find the jar from a property in the application's config file instead - for example,
database.driver=/opt/IBM/db2/V9.5/java/db2jcc.jar
I can load the class via a URLClassLoader ok, but the problem is that I need to treat the object thus created as an explicit DB2XADataSource. For example:
URLClassLoader dbClassLoader = new URLClassLoader(new URL[]{driverJar});
xaClass = dbClassLoader.loadClass("com.ibm.db2.jcc.DB2XADataSource");
DB2XADataSource dataSource = (DB2XADataSource) xaClass.newInstance();
dataSource.setCurrentSchema(DATABASE_SCHEMA); // <- dataSource has to be a
dataSource.setDatabaseName(DATABASE_NAME); // DB2XADataSource to do this
(rearranged and renamed somewhat; I actually do the loadClass in the constructor of the class that contains this code, while the newInstance is in one of its methods.)
I guess I'm getting into a classloader tangle because the classloader that loaded my class is trying to find DB2XADataSource in order to do the cast, but the URL classloader is not above it in the tree. The trouble is, it being long after I should have stopped working for the day (here in the UK) I can't think how best to solve it in a vaguely neat and sane manner.
Ideas?
Thanks.
The simplest approach is to just use the java.beans API (or direct reflection if you must) to invoke the setter methods.
Alternatively: Your database code requires to link to the dynamically loaded code. Therefore, dynamically load your database code. How much is up to you. You might load almost everything except the "bootstrap".
Yep - the class can't load its own dependencies. You could do some ClassLoader magic, but I imagine it would get messy very quickly.
One way to reduce the amount of reflection would be to put any code that depends on DB2XADataSource into an implementation that is invoked via an interface available to the calling ClassLoader.
//in mydb2driver.jar
public class MyDb2Driver implements IDriver {
private DB2XADataSource dataSource = new DB2XADataSource();
public void init() {
dataSource.setCurrentSchema(DATABASE_SCHEMA);
}
//etc.
}
This is loaded with your driver:
database.driver=/opt/IBM/db2/V9.5/java/db2jcc.jar:/foo/mydb2driver.jar
Invoking code is in the regular classpath:
public interface IDriver {
public void init();
//etc.
}
...
URLClassLoader dbClassLoader = new URLClassLoader(new URL[]{driverJar});
xaClass = dbClassLoader.loadClass("foo.MyDb2Driver");
IDriver dataSource = (IDriver) xaClass.newInstance();
dataSource.init();