We Keep Coding

Is there a way in my custom classloader to attach my classloader to the class while I delegate to super createing it?

I have this stack trace which is the FIRST time this class is loaded(I have a condition on the breakpoint for this class)....
The CompilingClassLoader is mine(ie. this one https://github.com/deanhiller/webpieces/blob/master/core/runtimecompile/src/main/java/org/webpieces/compiler/impl/CompilingClassloader.java )
I want this class that is being loaded in a way that it is plugin.getClass.getClassLoader is returning the CompilingClassLoader. Instead, it's classloader is AppClassLoader
This is causing many issues for plugin developers in that currently the workaround is a Thread context classloader. Is there a way to claim the compiling classloader loaded it but delegate to the super classloader?
The CRITICAL line is line 111 in the link above by the way. I want that line to use the super class to create the Class for me BUT I want it to look like I created the Class. ie. the Class returned should return 'my' classloader when clazz.getClassLoader() is called. this then fixes a ton of issues.
EDIT---------------------------------------
After further investigation. The CompilingClassLoader is very very tricky. You have these types of classes
Standard java.lang classes
webpieces startup classes
the applications compile on demand classes
webpieces plugin classes that sit below application classes and above them
libraries that sit below application classes AND above them
For #1, once I hit main, many of these like java.lang.Object are loaded by app classloader so this must remain true always so that you don't get the dreaded casting X to X yields ClassCastException
For #2, most of these are loaded by app classloader on startup as well
For #3, this is code that we compile OR recompile. To recompile, we throw away the CompilingClassLoader and recreate another one BUT cache bytes of all previously compiled classes that have not changed.
For #4 & #5, this is where it gets really really tricky!!! We have a ProdServerMeta (an application compiled class) that refers to plugins in webpieces and those plugins then sometimes load other application classes that need to compile on changes. Currently, we inject hibernate with the compiling classloader to avoid an issue.
The more I think about this, hibernate classes are loaded by app classloader right now because we return null when we can't compile a class delegating to the parent.
Now, I ran into these situations with CompilingClassLoader which is very hard to simplify into a classloader like Jan-Willem Gmelig Meyling's classloader below! Mainly because defineClass seems to be critical and missing in that example (and like what Olivier says below is very important).
Anyways, I ran into these things with trying the classloader below
loading of java.lang.* caused a SecurityException so I had to delegate that
After solving that, I ran into ClassCastException from SAXParserFactoryImpl to SAXParserFactoryImpl when slf4j tries to load configuration
This 2nd one makes total senses since Server.main creates a Logger and then every other class that creates a Logger creates it in a CompilingClassLoader. This leads me to the situation that
I cannot load ANYTHING that is loaded before the loading of the application
Next, being a webserver for customers, I also can't tell what is referenced in pre-app load vs. after app load.
I am coming to the conclusion that there is not a solution since as a developer writes code, I throw away the previous classloader and many classes have to continue to exist like Logger which is referenced by AppClassLoader on startup.
Of course, if I boot up with a CompilingClassLoader, I might stand a better chance in that I could 'try' to load everything perhaps but java.lang, BUT then I still run into the issue of some code after server bootup willl always be referenced by the FIRST CompilingClassLoader while later code on recompile will point to a new CompilingClassLoader.
I think the only solution then is for plugins to have to use the ContextClassLoader the more I think through this situation and that Olivier below is correct in it's not possible.
thanks,
Dean

Classloaders load classes from their parent class loaders first. You should be able to work around this by simply not setting a parent classloader for the classloader, and delegate the call for loadClass yourself.
A working example:
public class SomeClass implements Runnable {
#Override
public void run() {
System.out.println("Class loader for class: " + getClass().getClassLoader());
System.out.println("Class loader for thread: " + Thread.currentThread().getContextClassLoader());
System.out.println("Class loader for transitive dependency: " + SomeDependency.class.getClassLoader());
}
}
A transitive dependency:
public class SomeDependency {}
Test and classloader:
import java.net.URL;
import java.net.URLClassLoader;
public class Test {
public static class CustomClassLoader extends URLClassLoader {
ClassLoader delegate;
public CustomClassLoader(URL[] urls, ClassLoader parent) {
super(urls, null);
delegate = parent;
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
return super.loadClass(name);
} catch (ClassNotFoundException e) {
return delegate.loadClass(name);
}
}
}
public static void main(String... args) throws Exception {
URL location = SomeClass.class.getProtectionDomain().getCodeSource().getLocation();
CustomClassLoader customClassLoader = new CustomClassLoader(new URL[] { location }, ClassLoader.getSystemClassLoader());
Thread.currentThread().setContextClassLoader(customClassLoader);
Class<?> aClass = customClassLoader.loadClass("com.pkg.SomeClass");
Runnable runnable = (Runnable) aClass.newInstance();
runnable.run();
}
}
Output:
Class loader for class: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for thread: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for transitive dependency: sun.misc.Launcher$AppClassLoader#18b4aac2
Seems to do precisely what you want :)

Add or remove specific jars from custom ClassLoader

I want to create dynamically a classloader for executing JSR223 script in a controlled environment but failing,
I'm trying remove/add jars using current(parent) ClassLoader, I tried solution Dynamically removing jars from classpath
public class DistributionClassLoader extends ClassLoader {
public DistributionClassLoader(ClassLoader parent) {
super(parent);
}
private Map<String, ClassLoader> classLoadersByDistribution =
Collections.synchronizedMap(new WeakHashMap<>());
private final AtomicReference<String> distribution = new AtomicReference<>();
#Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
final ClassLoader delegate = classLoadersByDistribution.get(distribution.get());
if (delegate != null) return Class.forName(name, true, delegate);
throw new ClassNotFoundException(name);
}
public void addDistribution(String key, ClassLoader distributionClassLoader){
classLoadersByDistribution.put(key,distributionClassLoader);
}
public void makeDistributionActive(String key){distribution.set(key);}
public void removeDistribution(String key){
final ClassLoader toRemove = classLoadersByDistribution.remove(key);
}
}
But it didn't include all my jars, in test this work
ClassLoader cl = this.getClass().getClassLoader();
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
But using the solution doesn't find class
ClassLoader cl = new DistributionClassLoader(this.getClass().getClassLoader());
Class cls = cl.loadClass("org.springframework.http.HttpStatus");
Exception:
java.lang.ClassNotFoundException: org.springframework.http.HttpStatus
at com.DistributionClassLoader.loadClass(DistributionClassLoader.java:24)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
How can I select specific jars to add or remove from ClassLoader?
EDIT
I'm able to load jars using #czdepski answer but I still want to remove all/most classes except JDK's
Method sysMethod = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
sysMethod.setAccessible(true);
sysMethod.invoke(sysLoader, new Object[]{url});

You got the delegation wrong. You never check the parent class loader if it has this class.
If we look at the Javadoc for ClassLoader.loadClass(String,boolean) we find:
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built into the virtual machine is used, instead.
Invoke the findClass(String) method to find the class.
If the class was found using the above steps, and the resolve flag is true, this method will then invoke the resolveClass(Class) method on the resulting Class object.
Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method.
You did override loadClass, but don't do any delegation to it's parent ClassLoader.
Instead you call classLoadersByDistribution.get(distribution.get());, which is most likely null (hard to tell, but always expect WeakHashMap.get() to return null).
If delegate is not null, then you try to load the class from there. This means the loaded class won't use your ClassLoader to load new classes, but instead the ClassLoader you delegated to.
After all, this sounds like a XY Problem. You want to execute some code using the scripting API and somehow control the environment.
Did you try to use a SecurityManager?
About your comment that you need your own ClassLoader to create a ScriptEngineManager: This ClassLoader is used to search for ScriptEngineFactory implementations. This is done using a service provider interface.
If you don't use your own script engine, this should not matter to you.
If your goal is to add a few jars so the engine can use it, create a new URLClassLoader with the platform class loader as parent. (Or extension class loader, depends on the java version.)
Set this ClassLoader as Thread.setContextClassLoader() and create the ScriptEngine.
If you did choose the parent of the URLClassLoader correctly, it will not see classes loadable by the application class loader.

Unable to cast to interface class with URLClassLoader and reflection

I have a jar:
/home/cole/lib/a.jar
And in this jar I have the following interface/classes (horrible names for illustration purposes only!):
CreatorInterface.java
Base.java (implements CreatorInterface.java)
AbstractBase.java (extends Base.java)
Implementation.java (extends AbstractBase.java)
In a separate project I have the following code:
final URL[] jars = new URL[] {
new File("/home/cole/lib/a.jar").toURL();
}
final URLClassLoader classLoader = new URLClassLoader(jars, null);
final Class<?> implementation = classLoader.loadClass("Implementation");
final CreatorInterface object = (CreatorInterface)implementation.newInstance();
However when I run the above, I get the following:
java.lang.ClassCastException: Implementation cannot be cast to CreatorInterface
Given Implementation is ultimately an instance of a class that implements CreatorInterface, why do I get the ClassCastException?
Update 1
This isn't a question about using URLClassLoader, the class is found ok, the problem appears to be in the instantiation. For example, the following code works fine:
final Object object = implementation.newInstance();
Update 2
As #davidxxx answered, I have the interface class twice (once in the jar and once in the project using it). Although the interface was the same, this was the cause of the issue.
However to make it work, I needed to fix my URLClassLoader like this, to avoid a ClassNotFoundException:
final ClassLoader parent = this.getClass().getClassLoader();
final URLClassLoader classLoader = new URLClassLoader(jars, parent);

This exception :
java.lang.ClassCastException: Implementation cannot be cast to
CreatorInterface
makes me think that you have very probably two distinct CreatorInterface classes : one included in the jar and another other coming from the client program that tries to load it.
Even if the two classes have the same name (qualified names), these are different classes for each classloader as here you use two unassociated classloaders.
You have the current classloader of the program that you run and this other classloader as you specified null as parent classloader :
final URLClassLoader classLoader = new URLClassLoader(jars, null);
So as you try to assign the object created by reflection to the CreatorInterface variable, the cast fails because two distinct CreatorInterface were loaded by each classloader and are used : one coming from the classloader of your client code and another coming from the the instantiated classloader.
Using a single classloader would solve the issue but a best practice would be including the jar in the classpath of the project and to ensure to have a single version of the classes provided in the jar.
To decouple things you should probably split the jar into 2 jars : an API jar that contains only the interface and an implementation jar that depends on the API jar and that contains other classes.
In the client code, add in the classpath only the API jar to be able to assign to a variable with the interface declared type.
About your second point :
This isn't a question about using URLClassLoader, the class is found ok, the problem appears to be in the instantiation. For example, the
following code works fine:
final Object object = implementation.newInstance();
In this case you don't refer the interface type.
You indeed assign the Implementation object to an Object and not to a CreatorInterface variable.
The correct/consistent interfaces and subclasses are loaded by the classloader but here you never give a chance to provoke a ClassCastException as you never assign it to a type of a duplicate class but Object that is defined a single time.
So the problem previously encountered cannot occur.
About the third point :
However to make it work, I needed to fix my URLClassLoader like this,
to avoid a ClassNotFoundException:
final ClassLoader parent = this.getClass().getClassLoader();
final URLClassLoader classLoader = new URLClassLoader(jars, parent);
It works because here you create a classloader associated to the parent classloader.
In fact if you did :
final URLClassLoader classLoader = new URLClassLoader(jars);
It would produce the same result as the URLClassLoader object created would use by default the delegation to the parent classloader (here the classloader that started your application).

Java ClassLoader - force reloading already loaded classes

I'm currently trying to load classes into my application so that I can then filter out those that don't contain any test / #Test-methods. I want to run these tests in my application afterwards.
So far, so good - except it seems like the URLClassLoader I'm using (or probably any ClassLoader) doesn't actually reload classes that are located on my applications classpath.
More precisely, a user of my application starts by selecting a number of .java source files. Those are then copied to a temporary location and a number of regex match/replace pairs are applied to the copy of the original source files. Next, the copies are compiled, and I then want to load those compiled .class-files into my application so I can run them as tests.
In most cases, this works as desired.
Unfortunately, if the fully qualified class name (FQCN) of any of the selected files is identical to the FQCN of a class that is part of this application (such as tests for this application), the ClassLoader ignores the specified path (to %temp%\myfolder\) and the (updated) version of the class file located there, but instead uses the already present/loaded version of the class.
After some research, this behaviour can be expected according to the docs (emphasis mine):
• The loadClass method in ClassLoader performs these tasks, in order, when called to load a class:
- If a class has already been loaded, it returns it.
- Otherwise, it delegates the search for the new class to the parent class loader.
- If the parent class loader does not find the class, loadClass calls the method findClass to find and load the class.
I tried using findClass, but it's unfortunately not visible.
Hence my question - does anyone know how to force java / the ClassLoader to actually load the class from the specified path, ignoring any - FQCN-wise - identical existing classes?

A classloader first delegates to its parent classloader, which is how it determines "if a class has already been loaded". If you want to force a classloader to load a new class, one way is to insert another classloader in the chain which fails to load/find the same class. Very quick, incomplete example:
class FirewallLoader extends ClassLoader {
public FirewallLoader(ClassLoader parent) {
super(parent);
}
public loadClass(String name, boolean resolve) {
if (name.equals("some.class.Xyz")) {
throw ClassNotFoundException();
}
super.loadClass(name, resolve);
}
}
You make the "FirewallLoader" the parent or your URLClassLoader, and then your URLClassLoader will load new versions of whatever classes the Firewall loader filters out.

how can I know the current classLoader in java

I am learning the process of loading java class and encounter some confusion.
I know when loading a java class, current classLoader wont load the java class directly and it will delegate to its parent classLoader(a recursive process) until it parent cant load this class.
The question is that ：what is the current classLoader? Bootstrap? Extension? App?
how to get the current classLoader?.
and I know there is an API:
xxx.Class.getClassLoader();
but I am not sure whether the return value is currentClassLoader. I think it should be the classLoader which load this java class in reality.
To describe my question more detail I will give an example.
I get the follow content in a blog.
ThreadContextClassLoader is used to deal with java SPI, Interface is defined in java core lib and loaded by Bootstrap ClassLoader and third party implement these interface then the jar are loaded by AppClassLoader
Solution: traditional classLoader cant deal with this case,because it cant discovery the third party jar when we use the third party implement in core lib.
most of the above that I can understand but the solution make me confusion:
for example, the Interface CoreA and class CoreB are in java core lib and should be loaded by Bootstrap ClassLoader and the AImpl is a implement of A by third party and should be loaded by AppClass loader.
the code segment as below:
public Interface CoreA{
void add();
}
public Interface AImpl implements CoreA{
void add(){};
}
public class B{
public void demo(){
a = new AImpl();
}
}
then if we reference B in main method, then we will load B because the class loader of B is Bootstrap then about AImpl the current Loader is Bootstrap so it cant be found?
I dont know whether it is as what I guess?
Any advice will be appreciated.

Generally speaking you are right, it can't be found. Let me show you the following example. Let's say we have 3 classes: A, B and Main like these:
public class A {
public String a() {
return "a";
}
}
public class B {
public String b() {
return new A().a();
}
}
public class Main {
public static void main(String... args) {
System.out.println(new B().b());
}
}
And we package these classes into correspondent jars: a.jar, b.jar and place Main.class into the working directory. After that let's test the following scenarios:
1) Everything (A.class, B.class, Main.class) is loaded by system classloader and works fine:
$ java -cp .:a.jar:b.jar Main
a
2) B.class is loaded by system classloader and A.class is loaded by bootstrap classloader and everything still works fine because system classloader delegates loading to bootstrap classloader (just because bootstrap classloader can load it):
$ java -Xbootclasspath/a:a.jar -cp .:b.jar Main
a
3) A.class is loaded by system classloader and B.class is loaded by bootstrap classloader (your case). In this scenario during loading of B.class current classloader is bootstrap classloader, but it can't found B.class and fails:
$ java -Xbootclasspath/a:b.jar -cp .:a.jar Main
Exception in thread "main" java.lang.NoClassDefFoundError: A
at B.b(B.java:4)
at Main.main(Main.java:4)
Lets take a look at the last example more carefully. What's happening here:
Try to find class with public static main(String[] args) method
1.1. system classloader hasn't loaded it yet so delegates to extension classloader
1.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
1.3. bootstrap classloader hasn't loaded it yet and tries to load, it can't load it and returns control to extension classloader
1.4. extension classloader tries to load, it can't load it and returns control to system classloader
1.5. system classloader loads Main.class
Main.class is processed and we try to load B.class with current classloader system classloader
2.1. system classloader hasn't loaded it yet so delegates to extension classloader
2.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
2.3. bootstrap classloader hasn't loaded it yet and loads B.class
B.class is processed and and we try to load A.class with current classloader bootstrap classloader
3.1. bootstrap classloader hasn't loaded it yet and tries to load and
fails
I hope it will help you.

When a class A attempts to load another class B, the ClassLoader that loaded A is the current ClassLoader. The word current vaguely refers to the execution context - e.g. how do you end up in the method that triggers the current class loading call.
There is no method - say getCurrentClassLoader - that simply gives the current ClassLoader, but there are api methods that internally use the concept of current ClassLoader. For example, Class.forName(String className)
If you check how that method is implemented, it tells you the meaning of "current class loader":
public static Class<?> forName(String className) throws ClassNotFoundException {
Class<?> caller = Reflection.getCallerClass();
return forName0(className, true, ClassLoader.getClassLoader(caller), caller);
}
If you can get hold of a Class instance, you can always ask for the loader behind it by calling Class::getClassLoader() method. That will be your current class loader. The tricky bit, however, is to decide if the loader is bootstrap, or extension, or system class loader. The reason of the trickiness is that it is implementation specific, and you can always implement your own class loading mechanism.
The example given by #dmitrievanthony is an example of how things can become really complicated. It is a similar situation faced by JNDI, and the reason why the hack Thread.getContextClassLoader() was introduced. More about it here
Quote the most relevant piece from the article:
... By definition, a current classloader loads and defines the class to which your current method belongs. This classloader is implied when dynamic links between classes resolve at runtime, and when you use the one-argument version of Class.forName(), Class.getResource(), and similar methods. It is also used by syntactic constructs like X.class class literals ...

the 'current classloader' is the real classloader(load this class in reality) of class that refer it.
e.g.
if the classLoader of class A is ext classloader and class A refer class B C D. Then the 'current classloader' of B C D is ext classLoader. And of course the 'current classLoader' of main class is System classLoader.