We Keep Coding

Is there a way in my custom classloader to attach my classloader to the class while I delegate to super createing it?

I have this stack trace which is the FIRST time this class is loaded(I have a condition on the breakpoint for this class)....
The CompilingClassLoader is mine(ie. this one https://github.com/deanhiller/webpieces/blob/master/core/runtimecompile/src/main/java/org/webpieces/compiler/impl/CompilingClassloader.java )
I want this class that is being loaded in a way that it is plugin.getClass.getClassLoader is returning the CompilingClassLoader. Instead, it's classloader is AppClassLoader
This is causing many issues for plugin developers in that currently the workaround is a Thread context classloader. Is there a way to claim the compiling classloader loaded it but delegate to the super classloader?
The CRITICAL line is line 111 in the link above by the way. I want that line to use the super class to create the Class for me BUT I want it to look like I created the Class. ie. the Class returned should return 'my' classloader when clazz.getClassLoader() is called. this then fixes a ton of issues.
EDIT---------------------------------------
After further investigation. The CompilingClassLoader is very very tricky. You have these types of classes
Standard java.lang classes
webpieces startup classes
the applications compile on demand classes
webpieces plugin classes that sit below application classes and above them
libraries that sit below application classes AND above them
For #1, once I hit main, many of these like java.lang.Object are loaded by app classloader so this must remain true always so that you don't get the dreaded casting X to X yields ClassCastException
For #2, most of these are loaded by app classloader on startup as well
For #3, this is code that we compile OR recompile. To recompile, we throw away the CompilingClassLoader and recreate another one BUT cache bytes of all previously compiled classes that have not changed.
For #4 & #5, this is where it gets really really tricky!!! We have a ProdServerMeta (an application compiled class) that refers to plugins in webpieces and those plugins then sometimes load other application classes that need to compile on changes. Currently, we inject hibernate with the compiling classloader to avoid an issue.
The more I think about this, hibernate classes are loaded by app classloader right now because we return null when we can't compile a class delegating to the parent.
Now, I ran into these situations with CompilingClassLoader which is very hard to simplify into a classloader like Jan-Willem Gmelig Meyling's classloader below! Mainly because defineClass seems to be critical and missing in that example (and like what Olivier says below is very important).
Anyways, I ran into these things with trying the classloader below
loading of java.lang.* caused a SecurityException so I had to delegate that
After solving that, I ran into ClassCastException from SAXParserFactoryImpl to SAXParserFactoryImpl when slf4j tries to load configuration
This 2nd one makes total senses since Server.main creates a Logger and then every other class that creates a Logger creates it in a CompilingClassLoader. This leads me to the situation that
I cannot load ANYTHING that is loaded before the loading of the application
Next, being a webserver for customers, I also can't tell what is referenced in pre-app load vs. after app load.
I am coming to the conclusion that there is not a solution since as a developer writes code, I throw away the previous classloader and many classes have to continue to exist like Logger which is referenced by AppClassLoader on startup.
Of course, if I boot up with a CompilingClassLoader, I might stand a better chance in that I could 'try' to load everything perhaps but java.lang, BUT then I still run into the issue of some code after server bootup willl always be referenced by the FIRST CompilingClassLoader while later code on recompile will point to a new CompilingClassLoader.
I think the only solution then is for plugins to have to use the ContextClassLoader the more I think through this situation and that Olivier below is correct in it's not possible.
thanks,
Dean

Classloaders load classes from their parent class loaders first. You should be able to work around this by simply not setting a parent classloader for the classloader, and delegate the call for loadClass yourself.
A working example:
public class SomeClass implements Runnable {
#Override
public void run() {
System.out.println("Class loader for class: " + getClass().getClassLoader());
System.out.println("Class loader for thread: " + Thread.currentThread().getContextClassLoader());
System.out.println("Class loader for transitive dependency: " + SomeDependency.class.getClassLoader());
}
}
A transitive dependency:
public class SomeDependency {}
Test and classloader:
import java.net.URL;
import java.net.URLClassLoader;
public class Test {
public static class CustomClassLoader extends URLClassLoader {
ClassLoader delegate;
public CustomClassLoader(URL[] urls, ClassLoader parent) {
super(urls, null);
delegate = parent;
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
return super.loadClass(name);
} catch (ClassNotFoundException e) {
return delegate.loadClass(name);
}
}
}
public static void main(String... args) throws Exception {
URL location = SomeClass.class.getProtectionDomain().getCodeSource().getLocation();
CustomClassLoader customClassLoader = new CustomClassLoader(new URL[] { location }, ClassLoader.getSystemClassLoader());
Thread.currentThread().setContextClassLoader(customClassLoader);
Class<?> aClass = customClassLoader.loadClass("com.pkg.SomeClass");
Runnable runnable = (Runnable) aClass.newInstance();
runnable.run();
}
}
Output:
Class loader for class: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for thread: com.pkg.Test$CustomClassLoader#4e50df2e
Class loader for transitive dependency: sun.misc.Launcher$AppClassLoader#18b4aac2
Seems to do precisely what you want :)

Java 9 default methods in interfaces: from which modules are they invoked?

suppose you have moduleA and moduleB. ModuleA defines an interface (for instance for a Service) and ModuleB has a concrete class that implements the interface (provides the service).
Now if the interface has a default method and you invoke it on the class in moduleB (from another module) is this invocation supposed to be performed inside moduleA or moduleB?
Apparently it is from moduleA ... what's the rationale?
Example: suppose you have a code that does this:
InputStream is = this.getClass().getResourceAsStream(fullPath);
if this code lies in the implementation of the service in moduleB the stream will be opened. But if the code lies in the default method in moduleA then when the service is invoked on moduleB you will need to have an "open" resource in moduleB (so it seems that the invocation thinks it is from "outside" moduleB).
would like to read about the reason for that.
thanks
editing my question with an example.
Suppose you have in moduleA this code:
public interface PropertiesProvider {
public default Properties get(String domain) {
Class clazz =this.getClass();
System.out.println(" CLASS " +clazz);
InputStream is = clazz.getResourceAsStream(domain) ;
if (is != null) {
Properties props = new Properties();
try {
props.load(is);
return props;
} catch (IOException e) {
//log
}
}
return null;
}
}
and in moduleB
public class PropertiesProviderImpl implements PropertiesProvider {}
if you invoke the service from ModuleA the call is traced to come from class PropertiesProviderImpl finds the resource but does not load it if it is not "opened"
if you copy the code into PropertiesProviderImpl the calls is traced to that same class finds the ressource and loads it even when it is not "opened"
So my question is: why the difference since the call comes from the same class?
(the difference being that in one case the method is kind-of inherited from the default method in the interface)

Look at the documentation of the getResourceAsStream If this class is in a named Module then this method will attempt to find the resource in the module.
In the first case your code (in moduleA) sees the Type but cannot see the class which implements your Type, because it's in the moduleB. In the second case your code can see the class which "implements" the Type.
Look at the reference bellow, the most important sentences are:
In a modular setting the invocation of Class::forName will continue to work so long as the package containing the provider class is known to the context class loader. The invocation of the provider class’s constructor via the reflective newInstance method, however, will not work: The provider might be loaded from the class path, in which case it will be in the unnamed module, or it might be in some named module, but in either case the framework itself is in the java.xml module. That module only depends upon, and therefore reads, the base module, and so a provider class in any other module will be not be accessible to the framework.
[...]
instead, revise the reflection API simply to assume that any code that reflects upon some type is in a module that can read the module that defines that type.
[Long answer]: reflective-readability
A framework is a facility that uses reflection to load, inspect, and instantiate other classes at run time [...]
Given a class discovered at run time, a framework must be able to access one of its constructors in order to instantiate it. As things stand, however, that will usually not be the case.
The platform’s streaming XML parser, e.g., loads and instantiates the implementation of the XMLInputFactory service named by the system property javax.xml.stream.XMLInputFactory, if defined, in preference to any provider discoverable via the ServiceLoader class. Ignoring exception handling and security checks the code reads, roughly:
String providerName
= System.getProperty("javax.xml.stream.XMLInputFactory");
if (providerName != null) {
Class providerClass = Class.forName(providerName, false,
Thread.getContextClassLoader());
Object ob = providerClass.newInstance();
return (XMLInputFactory)ob;
}
// Otherwise use ServiceLoader
...
In a modular setting the invocation of Class::forName will continue to work so long as the package containing the provider class is known to the context class loader. The invocation of the provider class’s constructor via the reflective newInstance method, however, will not work: The provider might be loaded from the class path, in which case it will be in the unnamed module, or it might be in some named module, but in either case the framework itself is in the java.xml module. That module only depends upon, and therefore reads, the base module, and so a provider class in any other module will be not be accessible to the framework.
To make the provider class accessible to the framework we need to make the provider’s module readable by the framework’s module. We could mandate that every framework explicitly add the necessary readability edge to the module graph at run time, as in an earlier version of this document, but experience showed that approach to be cumbersome and a barrier to migration.
We therefore, instead, revise the reflection API simply to assume that any code that reflects upon some type is in a module that can read the module that defines that type. This enables the above example, and other code like it, to work without change. This approach does not weaken strong encapsulation: A public type must still be in an exported package in order to be accessed from outside its defining module, whether from compiled code or via reflection.

since we didn't understand precisely the previous response we carried some additional tests
in each test the resource file is not "opened"
1)
the code invoking clazz.getResouceAsStream is in default method of interface defining the service. The class implementing the interface does not defines any method.
-> this.getClass() yields the implementing class , tests fails to find resource
2)
we added this code in the default method
Object obj = clazz.getConstructor().newInstance();
and yes it fails
3) we changed the code so PropertiesProvider is abstract class and PropertiesProviderImpl inherits from it
same behaviour.
So yes it means that the same code will behave differently if you inherit from it or just invoke it directly.
This is worrying: it means the inner logic of the language is going to lead to convoluted byzantine behaviours (the reason why we dumped C++).

how can I know the current classLoader in java

I am learning the process of loading java class and encounter some confusion.
I know when loading a java class, current classLoader wont load the java class directly and it will delegate to its parent classLoader(a recursive process) until it parent cant load this class.
The question is that ：what is the current classLoader? Bootstrap? Extension? App?
how to get the current classLoader?.
and I know there is an API:
xxx.Class.getClassLoader();
but I am not sure whether the return value is currentClassLoader. I think it should be the classLoader which load this java class in reality.
To describe my question more detail I will give an example.
I get the follow content in a blog.
ThreadContextClassLoader is used to deal with java SPI, Interface is defined in java core lib and loaded by Bootstrap ClassLoader and third party implement these interface then the jar are loaded by AppClassLoader
Solution: traditional classLoader cant deal with this case,because it cant discovery the third party jar when we use the third party implement in core lib.
most of the above that I can understand but the solution make me confusion:
for example, the Interface CoreA and class CoreB are in java core lib and should be loaded by Bootstrap ClassLoader and the AImpl is a implement of A by third party and should be loaded by AppClass loader.
the code segment as below:
public Interface CoreA{
void add();
}
public Interface AImpl implements CoreA{
void add(){};
}
public class B{
public void demo(){
a = new AImpl();
}
}
then if we reference B in main method, then we will load B because the class loader of B is Bootstrap then about AImpl the current Loader is Bootstrap so it cant be found?
I dont know whether it is as what I guess?
Any advice will be appreciated.

Generally speaking you are right, it can't be found. Let me show you the following example. Let's say we have 3 classes: A, B and Main like these:
public class A {
public String a() {
return "a";
}
}
public class B {
public String b() {
return new A().a();
}
}
public class Main {
public static void main(String... args) {
System.out.println(new B().b());
}
}
And we package these classes into correspondent jars: a.jar, b.jar and place Main.class into the working directory. After that let's test the following scenarios:
1) Everything (A.class, B.class, Main.class) is loaded by system classloader and works fine:
$ java -cp .:a.jar:b.jar Main
a
2) B.class is loaded by system classloader and A.class is loaded by bootstrap classloader and everything still works fine because system classloader delegates loading to bootstrap classloader (just because bootstrap classloader can load it):
$ java -Xbootclasspath/a:a.jar -cp .:b.jar Main
a
3) A.class is loaded by system classloader and B.class is loaded by bootstrap classloader (your case). In this scenario during loading of B.class current classloader is bootstrap classloader, but it can't found B.class and fails:
$ java -Xbootclasspath/a:b.jar -cp .:a.jar Main
Exception in thread "main" java.lang.NoClassDefFoundError: A
at B.b(B.java:4)
at Main.main(Main.java:4)
Lets take a look at the last example more carefully. What's happening here:
Try to find class with public static main(String[] args) method
1.1. system classloader hasn't loaded it yet so delegates to extension classloader
1.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
1.3. bootstrap classloader hasn't loaded it yet and tries to load, it can't load it and returns control to extension classloader
1.4. extension classloader tries to load, it can't load it and returns control to system classloader
1.5. system classloader loads Main.class
Main.class is processed and we try to load B.class with current classloader system classloader
2.1. system classloader hasn't loaded it yet so delegates to extension classloader
2.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
2.3. bootstrap classloader hasn't loaded it yet and loads B.class
B.class is processed and and we try to load A.class with current classloader bootstrap classloader
3.1. bootstrap classloader hasn't loaded it yet and tries to load and
fails
I hope it will help you.

When a class A attempts to load another class B, the ClassLoader that loaded A is the current ClassLoader. The word current vaguely refers to the execution context - e.g. how do you end up in the method that triggers the current class loading call.
There is no method - say getCurrentClassLoader - that simply gives the current ClassLoader, but there are api methods that internally use the concept of current ClassLoader. For example, Class.forName(String className)
If you check how that method is implemented, it tells you the meaning of "current class loader":
public static Class<?> forName(String className) throws ClassNotFoundException {
Class<?> caller = Reflection.getCallerClass();
return forName0(className, true, ClassLoader.getClassLoader(caller), caller);
}
If you can get hold of a Class instance, you can always ask for the loader behind it by calling Class::getClassLoader() method. That will be your current class loader. The tricky bit, however, is to decide if the loader is bootstrap, or extension, or system class loader. The reason of the trickiness is that it is implementation specific, and you can always implement your own class loading mechanism.
The example given by #dmitrievanthony is an example of how things can become really complicated. It is a similar situation faced by JNDI, and the reason why the hack Thread.getContextClassLoader() was introduced. More about it here
Quote the most relevant piece from the article:
... By definition, a current classloader loads and defines the class to which your current method belongs. This classloader is implied when dynamic links between classes resolve at runtime, and when you use the one-argument version of Class.forName(), Class.getResource(), and similar methods. It is also used by syntactic constructs like X.class class literals ...

the 'current classloader' is the real classloader(load this class in reality) of class that refer it.
e.g.
if the classLoader of class A is ext classloader and class A refer class B C D. Then the 'current classloader' of B C D is ext classLoader. And of course the 'current classLoader' of main class is System classLoader.

A custom String class creation

I tried to create a custom class String in java.lang package in my eclipse workspace.
initially I suspected that a same class in same package can not be created but to my utter surprise I was able to create a class (String) in same package i.e. java.lang
Now I am confused
1) why is it possible and
2) what can be the reason if that is allowed.
3) what will be the use if this type of creation of java classes is allowed in Java.

You can create a new class in java.lang package. If it was forbidden how Oracle developers would be able to develop Java at all? I am sure they use the same javac as we do.
But you will not be able to load it, because java.lang.ClassLoader (that any classloader extends) does not allow it, every class being loaded goes through this check
...
if ((name != null) && name.startsWith("java.")) {
throw new SecurityException
("Prohibited package name: " + name.substring(0, name.lastIndexOf('.')));
}
...
so you will end up in something like
Exception in thread "main" java.lang.SecurityException: Prohibited package name: java.lang
at java.lang.ClassLoader.preDefineClass(ClassLoader.java:649)
at java.lang.ClassLoader.defineClass(ClassLoader.java:785)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at Test1.main(Test1.java:11)
As for classes that shadow existing classes like your java.lang.String they cannot be loaded because the System ClassLoader (default one) uses "parent first" strategy, so java.lang classes will be loaded from rt.jar with bootstrap classloader. So you will need to replace String.class in rt.jar with your version. Or override it using -Xbootclasspath/p: java option which prepends paths to bootstrap class loader search paths. So you can
1) copypaste real String.java content into your String.java
2) change a method, eg
public static String valueOf(double d) {
return "Hi";
}
and compile your String.java
3) create a test class
public class Test1 {
public static void main(String[] args) throws Exception {
System.out.println(String.valueOf(1.0d));
}
}
4) run it as
java -Xbootclasspath/p:path_to_your_classes Test1
and you will see
Hi

This is called class shadowing.
1.) It is possible because java classes are not statically linked, but linked at class load time.
2.) If it was not allowed, then the whole class loading would be a lot more difficult to achieve. Then for example you would also have to build your project against a specific Java version. because Java classes may change from version to version. This isn't going to be a maintainable approach.
3.) osgi makes use of this to be able to load different versions of the same bundle. Another common use case is to replace buggy classes in frameworks, where no other workaround is possible. However this technique should be used carefully, because errors might be hard to debug.
Please note that however shadowing classes in the java.* packages is not allowed, as this would break the Java security sandbox. So you are going to have problems at runtime.

Yes you can create the package with name java.lang and also Class named as String.
But you wont be able to run your String class.
1) why is it possible : Compiler will compile your class successfully.
2) what can be the reason if that is allowed : You have a valid name for your package and class so compiler doesn't complain.
3) what will be the use if this type of creation of java classes is allowed in Java : But there is not much use of this String class. Bootstrap class loader will load class from the sun's java.lang package. So your custom String class will not get loaded and hence it fails during run.
Bootstrap class loader is part of JVM implementation and it loads the classes of Java API (which includes java.lang.String). Also for each class which gets loaded, JVM keeps track of which class loader whether bootstrap or user-defined - loaded the class. So any attempt to load a custom String class will fail as String class is already loaded.

How and When Bootstrap jar locate../lib/rt.jar and ../lib/ext.**jar are loaded?

I have two classes listed as follows
package foo;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
public class CustomClassLoader extends ClassLoader {
public CustomClassLoader(ClassLoader parent){
super(parent);
}
public Class<?> loadClass(String name) throws ClassNotFoundException {
System.out.println( " >>> loading " + name );
if (name.startsWith("foo")) {
return getClass(name);
}
return super.loadClass(name);
}
public Class getClass(String name){
try {
byte[] data= getClassByteData(name);
return this.defineClass(name, data, 0, data.length);
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
public byte[] getClassByteData(String name) throws IOException {
String file = name.replace(".", File.separator)+".class";
InputStream in = this.getClass().getResourceAsStream("/"+file);
int length = in.available();
byte[] datas = new byte[length];
in.read(datas, 0, length);
return datas;
}
}
package foo;
public class Test {
public static void main(String[] args) {
System.out.println(" Who is my loader? >>" + Test.class.getClassLoader());
}
}
Run: java -Djava.system.class.loader=foo.CustomClassLoader foo.Test
Output:
>>> loading java.lang.System
>>> loading java.nio.charset.Charset
>>> loading java.lang.String
>>> loading foo.Test
>>> loading java.lang.Object
>>> loading java.lang.StringBuilder
>>> loading java.lang.Class
>>> loading java.io.PrintStream
Who is my loader? >>foo.CustomClassLoader#de6ced
My questions are below:
Why those java.lang.System, java.nio.charset.Charset etc as above would be loaded by CustomClassLoader? In my ideas I think when I run java -Djava.system.class.loader foo.Test, JVM first searches out class foo.Test, loads it, executes the main method, and then when it detects System.out.println(), it will continue to load Class java.lang.System and java.io.PrintWriter because those classes are used by it, right?
When I run a class which uses some classes located in java.lang package, those classes will also be loaded again, in my case delegated CustomClassLoader>>ExtClassLoader>>BoostrapClassLoader to load?
When /lib/rt.jar and /lib/ext/**.jar are loaded, before we run a class like java foo.Test all those classes have bean loaded already?
Thanks all your help in advance!

To answer your questions, let's take them in turn.
Java's ClassLoading mechanism is based on a delegation model where an attempt to load a class is normally delegated to parent class loaders first. If the parent class loader is able to successfully load the class, then the loaded class is passed back down the delegate chain and return by whichever class loader was originally invoked. Since you set your custom class loader as the system class loader, the JVM instantiated it as the default loader used when your application starts. It is then used to load all classes that it needs. Most of those classes (i.e. anything that doesn't start with foo.*) are delegated to the parent class loaders, which explains why you see messages about your class loader attempting to load them.
Because the loading of non foo.* classes is delegated to the parent class loader, loading a system class, such as System.out, again will be handled by the parent class loader, which handles caching all loaded classes and will return the previously loaded instance.
Loading of the rt.jar and lib/ext/*.jar files is handled by the parent class loader (or its parent), not you. Therefore, there is no need for your class loader to concern itself with those JAR files.
Generally, a custom class loader is built to insure that classes can be loaded from a non-standard location, such as JARs stored in a database.

I assume that actual command line you running is java -Djava.system.class.loader=foo.CustomClassLoader foo.Test.
In general all classes can be loaded multiple times by different class loaders. In fact they are even considered different classes. So then foo.Test needs a java.lang.System, its class loader is invoked to find or load it.
In your case, CustomClassLoader delegates non foo class loading to super, which will not load same class second time, but return previously loaded one.
Talking about loading jars in misleading. Classes from those jars are loaded individually, on-demand. To load any program JVM needs to create a thread, so Thread class and its dependencies are loaded even before your class.
you can run sun's java with -verbose:class to see how classes are loaded.