We Keep Coding

Is there a way in my custom classloader to attach my classloader to the class while I delegate to super createing it?

I have this stack trace which is the FIRST time this class is loaded(I have a condition on the breakpoint for this class)....
The CompilingClassLoader is mine(ie. this one https://github.com/deanhiller/webpieces/blob/master/core/runtimecompile/src/main/java/org/webpieces/compiler/impl/CompilingClassloader.java )
I want this class that is being loaded in a way that it is plugin.getClass.getClassLoader is returning the CompilingClassLoader. Instead, it's classloader is AppClassLoader
This is causing many issues for plugin developers in that currently the workaround is a Thread context classloader. Is there a way to claim the compiling classloader loaded it but delegate to the super classloader?
The CRITICAL line is line 111 in the link above by the way. I want that line to use the super class to create the Class for me BUT I want it to look like I created the Class. ie. the Class returned should return 'my' classloader when clazz.getClassLoader() is called. this then fixes a ton of issues.
EDIT---------------------------------------
After further investigation. The CompilingClassLoader is very very tricky. You have these types of classes
Standard java.lang classes
webpieces startup classes
the applications compile on demand classes
webpieces plugin classes that sit below application classes and above them
libraries that sit below application classes AND above them
For #1, once I hit main, many of these like java.lang.Object are loaded by app classloader so this must remain true always so that you don't get the dreaded casting X to X yields ClassCastException
For #2, most of these are loaded by app classloader on startup as well
For #3, this is code that we compile OR recompile. To recompile, we throw away the CompilingClassLoader and recreate another one BUT cache bytes of all previously compiled classes that have not changed.
For #4 & #5, this is where it gets really really tricky!!! We have a ProdServerMeta (an application compiled class) that refers to plugins in webpieces and those plugins then sometimes load other application classes that need to compile on changes. Currently, we inject hibernate with the compiling classloader to avoid an issue.
The more I think about this, hibernate classes are loaded by app classloader right now because we return null when we can't compile a class delegating to the parent.
Now, I ran into these situations with CompilingClassLoader which is very hard to simplify into a classloader like Jan-Willem Gmelig Meyling's classloader below! Mainly because defineClass seems to be critical and missing in that example (and like what Olivier says below is very important).
Anyways, I ran into these things with trying the classloader below
loading of java.lang.* caused a SecurityException so I had to delegate that
After solving that, I ran into ClassCastException from SAXParserFactoryImpl to SAXParserFactoryImpl when slf4j tries to load configuration
This 2nd one makes total senses since Server.main creates a Logger and then every other class that creates a Logger creates it in a CompilingClassLoader. This leads me to the situation that
I cannot load ANYTHING that is loaded before the loading of the application
Next, being a webserver for customers, I also can't tell what is referenced in pre-app load vs. after app load.
I am coming to the conclusion that there is not a solution since as a developer writes code, I throw away the previous classloader and many classes have to continue to exist like Logger which is referenced by AppClassLoader on startup.
Of course, if I boot up with a CompilingClassLoader, I might stand a better chance in that I could 'try' to load everything perhaps but java.lang, BUT then I still run into the issue of some code after server bootup willl always be referenced by the FIRST CompilingClassLoader while later code on recompile will point to a new CompilingClassLoader.
I think the only solution then is for plugins to have to use the ContextClassLoader the more I think through this situation and that Olivier below is correct in it's not possible.
thanks,
Dean

Classloaders load classes from their parent class loaders first. You should be able to work around this by simply not setting a parent classloader for the classloader, and delegate the call for loadClass yourself.
A working example:
public class SomeClass implements Runnable {
#Override
public void run() {
System.out.println("Class loader for class: " + getClass().getClassLoader());
System.out.println("Class loader for thread: " + Thread.currentThread().getContextClassLoader());
System.out.println("Class loader for transitive dependency: " + SomeDependency.class.getClassLoader());
}
}
A transitive dependency:
public class SomeDependency {}
Test and classloader:
import java.net.URL;
import java.net.URLClassLoader;
public class Test {
public static class CustomClassLoader extends URLClassLoader {
ClassLoader delegate;
public CustomClassLoader(URL[] urls, ClassLoader parent) {
super(urls, null);
delegate = parent;
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
return super.loadClass(name);
} catch (ClassNotFoundException e) {
return delegate.loadClass(name);
}
}
}
public static void main(String... args) throws Exception {
URL location = SomeClass.class.getProtectionDomain().getCodeSource().getLocation();
CustomClassLoader customClassLoader = new CustomClassLoader(new URL[] { location }, ClassLoader.getSystemClassLoader());
Thread.currentThread().setContextClassLoader(customClassLoader);
Class<?> aClass = customClassLoader.loadClass("com.pkg.SomeClass");
Runnable runnable = (Runnable) aClass.newInstance();
runnable.run();
}
}
Output:
Class loader for class: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for thread: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for transitive dependency: sun.misc.Launcher$AppClassLoader#18b4aac2
Seems to do precisely what you want :)

Add or remove specific jars from custom ClassLoader

I want to create dynamically a classloader for executing JSR223 script in a controlled environment but failing,
I'm trying remove/add jars using current(parent) ClassLoader, I tried solution Dynamically removing jars from classpath
public class DistributionClassLoader extends ClassLoader {
public DistributionClassLoader(ClassLoader parent) {
super(parent);
}
private Map<String, ClassLoader> classLoadersByDistribution =
Collections.synchronizedMap(new WeakHashMap<>());
private final AtomicReference<String> distribution = new AtomicReference<>();
#Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
final ClassLoader delegate = classLoadersByDistribution.get(distribution.get());
if (delegate != null) return Class.forName(name, true, delegate);
throw new ClassNotFoundException(name);
}
public void addDistribution(String key, ClassLoader distributionClassLoader){
classLoadersByDistribution.put(key,distributionClassLoader);
}
public void makeDistributionActive(String key){distribution.set(key);}
public void removeDistribution(String key){
final ClassLoader toRemove = classLoadersByDistribution.remove(key);
}
}
But it didn't include all my jars, in test this work
ClassLoader cl = this.getClass().getClassLoader();
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
But using the solution doesn't find class
ClassLoader cl = new DistributionClassLoader(this.getClass().getClassLoader());
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
Exception:
java.lang.ClassNotFoundException: org.springframework.http.HttpStatus
at com.DistributionClassLoader.loadClass(DistributionClassLoader.java:24)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
How can I select specific jars to add or remove from ClassLoader?
EDIT
I'm able to load jars using #czdepski answer but I still want to remove all/most classes except JDK's
Method sysMethod = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
sysMethod.setAccessible(true);
sysMethod.invoke(sysLoader, new Object[]{url});

You got the delegation wrong. You never check the parent class loader if it has this class.
If we look at the Javadoc for ClassLoader.loadClass(String,boolean) we find:
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built into the virtual machine is used, instead.
Invoke the findClass(String) method to find the class.
If the class was found using the above steps, and the resolve flag is true, this method will then invoke the resolveClass(Class) method on the resulting Class object.
Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method.
You did override loadClass, but don't do any delegation to it's parent ClassLoader.
Instead you call classLoadersByDistribution.get(distribution.get());, which is most likely null (hard to tell, but always expect WeakHashMap.get() to return null).
If delegate is not null, then you try to load the class from there. This means the loaded class won't use your ClassLoader to load new classes, but instead the ClassLoader you delegated to.
After all, this sounds like a XY Problem. You want to execute some code using the scripting API and somehow control the environment.
Did you try to use a SecurityManager?
About your comment that you need your own ClassLoader to create a ScriptEngineManager: This ClassLoader is used to search for ScriptEngineFactory implementations. This is done using a service provider interface.
If you don't use your own script engine, this should not matter to you.
If your goal is to add a few jars so the engine can use it, create a new URLClassLoader with the platform class loader as parent. (Or extension class loader, depends on the java version.)
Set this ClassLoader as Thread.setContextClassLoader() and create the ScriptEngine.
If you did choose the parent of the URLClassLoader correctly, it will not see classes loadable by the application class loader.

Explicit loading - java.lang.ClassLoader

In first case, For explicit loading of test.ClassLoaderTest using below code,
public ClassLoaderTest{
public static void main(String[] args){
.....
Class.forName("test.ClassLoaderTest", true,
ClassLoaderTest.class.getClassLoader().getParent());
....
}
findClass() method of Launcher$ExtClassLoader instance gets invoked to load test.ClassLoaderTest with below error due to visibility principle,
java.lang.ClassNotFoundException: test.ClassLoaderTest
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at sun.misc.Launcher$ExtClassLoader.findClass(Launcher.java:229)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at test.ClassLoaderTest.main(ClassLoaderTest.java:29)
In second case, On explicit loading of test.ClassLoaderTest1, using
public ClassLoaderTest{
public static void main(String[] args){
.....
Class.forName("test.ClassLoaderTest1");
....
}
loadClass() method of Launcher$AppClassLoader instance is ultimately used to load test.ClassLoaderTest1 class,
where test.ClassLoaderTest1 is a wrong class file that lead to below error,
java.lang.ClassNotFoundException: test.ClassLoaderTest1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at test.ClassLoaderTest1.main(ClassLoaderTest.java:16)
In both cases, class loading job is done by java.net.URLClassLoader.findClass()
Class.forName() internally invokes getClassLoader() to know the class loader that already loaded the class.
In second case, When Class gets a class loader instance(of type Launcher$AppClassLoader) by calling
ClassLoader cl = getClassLoader0(); to invoke the class loader instance again.
Is java.lang.ClassLoader mainly used for sub-classing custom class loader? that load classes not available in CLASSPATH but from network source etc...

From the Javadoc,
public **abstract** class ClassLoader
extends Object
There are different ClassLoader implementations that use different strategies for locating and reading the byte streams that compose a class.

Adding an annotation to a runtime generated class using Javassist

I'm using Javassist(Java 1.7) to add an annotation to the class ClassA, but i get the exception. What am i doing wrong? The code I tried looks like this:
ClassA.java
public class ClassA
{
}
add method
public static <T> Class<T> addXmlRootAnnotationDynamicly(Class<T> declaredTyp) throws NotFoundException, CannotCompileException, InstantiationException, IllegalAccessException
{
//pool creation
ClassPool pool = ClassPool.getDefault();
//extracting the class
CtClass cc = pool.getCtClass(declaredTyp.getCanonicalName());
// create the annotation
ClassFile ccFile = cc.getClassFile();
ConstPool constpool = ccFile.getConstPool();
AnnotationsAttribute attr = new AnnotationsAttribute(constpool, AnnotationsAttribute.visibleTag);
Annotation annot = new Annotation("javax.xml.bind.annotation.XmlRootElement", constpool);
attr.addAnnotation(annot);
// add the annotation to the class
cc.getClassFile().addAttribute(attr);
// transform the ctClass to java class
Class<T> dynamiqueBeanClass = cc.toClass();
//instanciating the updated class
// T sayHelloBean = dynamiqueBeanClass.newInstance();
return dynamiqueBeanClass;
}
call
Class<ClassA> addXmlRootAnnotationDynamicly = addXmlRootAnnotationDynamicly(ClassA.class);
Exception
javassist.CannotCompileException: by java.lang.LinkageError: loader (instance of sun/misc/Launcher$AppClassLoader): attempted duplicate class definition for name: "de/it_p/pvlight/share/util/ClassA"
at javassist.ClassPool.toClass(ClassPool.java:1099)
at javassist.ClassPool.toClass(ClassPool.java:1042)
at javassist.ClassPool.toClass(ClassPool.java:1000)
at javassist.CtClass.toClass(CtClass.java:1224)
at de.it_p.pvlight.share.util.JAXBUtil.addXmlRootAnnotationDynamicly(JAXBUtil.java:107)
at de.it_p.pvlight.share.util.JAXBUtilTest.addXmlRootAnnotationDynamicly(JAXBUtilTest.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.LinkageError: loader (instance of sun/misc/Launcher$AppClassLoader): attempted duplicate class definition for name: "de/it_p/pvlight/share/util/ClassA"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.lang.ClassLoader.defineClass(ClassLoader.java:643)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at javassist.ClassPool.toClass2(ClassPool.java:1112)
at javassist.ClassPool.toClass(ClassPool.java:1093)
... 15 more

The root of your problem can be found in your stack trace:
attempted duplicate class definition for name: "de/it_p/pvlight/share/util/ClassA"
Your addXmlRootAnnotationDynamicly method takes a loaded class and redefines this very same class without changing its name. After this redefinition, you attempt to load the altered class one more time. This is however not possible in Java where any ClassLoader can only load a class of a given name one single time.
For this reason, the pool.getCtClass method takes a String instead of a loaded Class and works with CtClasses which are used to describe unloaded Classes. To overcome your problem, you have different choices:
Change the signature of your method to addXmlRootAnnotationDynamicly(String) and deliver de.it_p.pvlight.share.util.ClassA as the argument. Make sure that this class is not loaded before you transform it, anywhere in your code. You should therefore run the transformation at your application's startup to make sure that the class is not accidentally loaded before the transformation. Your altered Class is then loaded on cc.toClass().
Create a subclass of the argument class (or use interfaces) which uses a random name. The subclass is then type compatible to your argument class but is never loaded.
Use the Instrumentation API to redefine your loaded class at runtime.
Make sure that the input class and the output class are loaded with different ClassLoaders. (not recommended)

URLClassloader Dependencies

In my previous question, I asked how to load remote jar files. My current code is this:
//f is the path to the jar
URLClassLoader loader = new URLClassLoader(new URL[]{f.toURI().toURL()});
Class<?> jarClass = Class.forName(main, true, loader);
Class<? extends Module> module = jarClass.asSubclass(Module.class);
Constructor<? extends Module> constructor = module.getConstructor();
System.out.println(constructor);
Module module = constructor.newInstance();
This works well, but the remotely-loaded modules extend a class that is in the jar that is loading them, which gives this error:
Caused by: java.lang.ClassNotFoundException: package.whatever.Module, which I presume is because it is using URLClassLoader instead of getClass().getClassLoader().. how can I make it use URLClassLoader and then fall back to the default one?
Thanks,
Bart

You can set your application class loader to be the parent of the url class loader:
URLClassLoader loader = new URLClassLoader(
new URL[]{f.toURI().toURL()}, Module.class.getClassLoader());
From the Oracle Java tutorial (class loading mechanism):
The Java platform uses a delegation model for loading classes. The
basic idea is that every class loader has a "parent" class loader.
When loading a class, a class loader first "delegates" the search for
the class to its parent class loader before attempting to find the
class itself.

I faced the same issue and I was using Java 9, downgrading back it to Java 8 resolved my issue.