We Keep Coding

Java: ExceptionInInitializerError Creating New Instance of Clojure Compiled Class Loaded with Custom Class Loader

I am dynamically loading a jar and attempting to create a new instance of a given class in the jar. To make matters more interesting, the jar was compiled from a Leiningen clojure project. I created a custom byte class loader:
private static class ByteClassLoader extends ClassLoader {
private final Map<String, byte[]> classDataMap;
public ByteClassLoader() {
this.classDataMap = new HashMap<>();
}
public void addClassData(String className, byte[] bytes) {
classDataMap.put(className, bytes);
}
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
byte[] classData = classDataMap.get(name);
if (classData != null)
return defineClass(name, classData, 0, classData.length);
return super.findClass(name);
}
}
Now I get the class I am interested in from the custom class loader and attempt to create a new instance.
Class<?> myClass = byteClassLoader.findClass("com.example.Foo");
myClass.newInstance();
But this throws a file not found exception:
Caused by: java.io.FileNotFoundException: Could not locate some/namespace.class or some/namespace.clj on classpath:
at clojure.lang.RT.load(RT.java:443)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:407)
at com.example.Foo.<clinit>(Unknown Source)
... 33 more
It seems like the call to new instance doesn't know about the custom class loader. Does that sound like a correct understanding of the error? How do I instantiate a Foo in this example?

The created class does “know” about its defining ClassLoader and the JVM will resolve its references using the defining class loader. However, as you can see from your stack trace, its the clojure.lang.RT.load which tries to resolve resources in an incompatible way. By looking at its code, it boils down to the following method:
static public ClassLoader baseLoader(){
if(Compiler.LOADER.isBound())
return (ClassLoader) Compiler.LOADER.deref();
else if(booleanCast(USE_CONTEXT_CLASSLOADER.deref()))
return Thread.currentThread().getContextClassLoader();
return Compiler.class.getClassLoader();
}
From this code you can derive solutions to your problem:
You may set the value of Compiler.LOADER to your custom class loader.
If Compiler.LOADER remains unbound and RT.USE_CONTEXT_CLASSLOADER evaluates to true, you can set the context loader of your thread to your custom class loader.
last but not least, if you enforce your class loader to be the defining class loader of the entire clojure code, Compiler.class.getClassLoader() will return your custom class loader.

Recursive ClassLoader in Java

I need to define a ClassLoader that dynamically reloads certain classes from their respective classfiles. The reason for this is that the classes are changed since runtime started.
Currently I have a classloader than can load one class from its file. What I need further is load all classes that this one classes depends on and that are in the list of changed classes.
Simply explained, I have a ClassLoader class that overrides the loadClass method to load the changed classfile when the required class is in the list. Otherwise, it loads the class using the default (parent) classloader.
Is it possible to also (re)load the classes that are in the class definition from the primarily loaded class using the same classloader?
So, a simple example
Class A {
static method() {
B b = new B();
C c = new C();
// do something with B and C
}
}
Class B {
public X x = new X();
}
When I load class A with MyCustomClassLoader, I hand it a list of classes, containing A and C f.e. because I know the classfiles (the bin/A.class) of these classes have been changed. So, it should manually load the file from class A, and when loading B and C, it should check if they are in the list of changed classes. When they are not, they can be loaded using the default classloader, otherwise it will do the same as with A, recursively (so also checking classes that B depends on).
I have the loading from file working, but not the recursion. I also know that this method does not override the Java new operator globally. But I only need to run one changed class, but with it's dependencies also changed.
EDIT:
Note that this is not achievable using Instrumentation. The documentation for it cites the following, both for retransformClasses() as for redefineClasses().
The retransformation may change method bodies, the constant pool and attributes. The retransformation must not add, remove or rename fields or methods, change the signatures of methods, or change inheritance. These restrictions maybe be lifted in future versions.
So I made a working agent to change method bodies, but only when I noticed it crashed on method creation or deletion, I found this piece of text.

I finally got it working. I made the following class:
I believe the key here was the resolveClass() method. Although the documentation doesn't provide any useful information, I believe this method ensures that after loading a class, all used classes are loaded as well.
It requires you to make a new class every time one of the changed classes changes again or a new class changed.
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Set;
/**
* This class provides dynamic loading in the sense that the given class and
* other changed classes are loaded from disk, not from cache.
* Ensures most recent version is used for those classes.
*/
public class DynamicClassLoader extends ClassLoader{
private Set<String> classes;
public DynamicClassLoader(Set<String> changedClasses) {
this.classes = changedClasses;
}
/**
* Dynamically loads the class from it's binary file.
* Classes this class depends on will only be loaded dynamically when
* they are in list of changed classes given at construction.
* #throws ClassNotFoundException
*/
public Class<?> dynamicallyLoadClass(String name)
throws ClassNotFoundException {
classes.add(name);
return loadClass(name);
}
/**
* Finds the class dynamically, contrary to Class.forname which can use
* cached copies.
* This means it forces a reload of the class data.
* Source: http://stackoverflow.com/questions/3971534/
*/
#Override
protected Class<?> findClass(String s) throws ClassNotFoundException {
try {
byte[] bytes;
bytes = loadClassData(s);
return defineClass(s, bytes, 0, bytes.length);
} catch (IOException e) {
e.printStackTrace();
throw new ClassNotFoundException();
}
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException{
return loadClass(name, true);
}
/**
* Overridden to not check the parent's loadClass method first.
*/
#Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
Class<?> clazz = findLoadedClass(name);
if(clazz != null)
return clazz;
if(classes.contains(name)) {
clazz = findClass(name);
resolveClass(clazz);
return clazz;
}
return super.loadClass(name, resolve);
}
/**
* Load class data from byte code.
*
* #param className
* #return
* #throws IOException
*/
protected byte[] loadClassData(String className) throws IOException {
File f = new File("bin/"
+ className.replaceAll("\\.", "/") + ".class");
int size = (int) f.length();
byte buff[] = new byte[size];
FileInputStream fis = new FileInputStream(f);
DataInputStream dis = new DataInputStream(fis);
dis.readFully(buff);
dis.close();
return buff;
}
}

I would strongly recommend looking at using java.lang.instrument for what you are trying to accomplish, instead of juggling ClassLoaders. Specifically, by implementing the ClassFileTransformer interface, registering an agent, and using the redefineClasses() and/or retransformClasses() methods of the Instrumentation instance passed into your agent during registration.
Note that when you redefine a class, existing instances of the previous class definition are not affected. There is no way around this regardless of using custom ClassLoaders or instrumentation.
This page in particular you will find very informative about how to use instrumentation to accomplish what you are trying to do.

How to reload a loaded class with different ClassLoader in Java?

public class ClassScanner
{
// scan extraClasspath for specific classes
public List<Class<?>> scanClasspathForClass(String scanCriteria)
{
...
}
public static Class<?> reloadClass(Class<?> clazz, ClassLoader clazzLoader)
{
// Question: how to reload a loaded class (ClassScanner in this example) with a different arbitrary ClassLoader?
}
// an example of how reloadClass() would be used
// in real case, this function would be in different class
public List<Class<?>> scan(URL[] extraClasspath)
{
URLClassLoader urlClazzLoader = new URLClassLoader(extraClasspath, null);
Class<?> newClass = reloadClass(ClassScanner.class, urlClazzLoader);
return ((ClassScanner) newClass.newInstance()).scanClasspathForClass();
}
}
Above code demonstrates the question and why it is a question. I need to implement reloadClass(). I wonder if there is a reliable solution in Java 1.6. One useful reference would be Find where java class is loaded from.
Thanks for help!

Found myself the answer from http://www2.sys-con.com/itsg/virtualcd/java/archives/0808/chaudhri/index.html.
Basically what I need is to make one ClassLoader A to share its namespace with another ClassLoader B. The way I found to achieve this is to use the ClassLoader parent-delegation model. Here, ClassLoader A is the parent ClassLoader.
public List<Class<?>> scan(URL[] extraClasspath) throws Exception
{
URLClassLoader urlClazzLoader = new URLClassLoader(extraClasspath, ClassScanner.class.getClassLoader());
return urlClazzLoader.loadClass(ClassScanner.getName()).newInstance();
}

Java ClassLoader change

I have some class A:
public class A {
public A(String str) {
System.out.println("Create A instance: " + str);
}
public void methodA() {
System.out.println("#methodA1()");
}
}
And my class loader implementation:
public class MyClassLoader extends ClassLoader {
public MyClassLoader() {
super();
}
#Override
public synchronized Class<?> loadClass(String name)
throws ClassNotFoundException {
System.out.println("Load: " + name);
return super.loadClass(name);
}
}
And now I try to change default class loader in current thread:
import java.util.ArrayList;
import java.util.List;
public class ChangeLoaderTest {
public static void main(String[] args) {
// Save class loader so that we can restore later.
ClassLoader oldLoader = Thread.currentThread().getContextClassLoader();
MyClassLoader newLoader = new MyClassLoader();
try {
// Set new classloader.
Thread.currentThread().setContextClassLoader(newLoader);
// My class.
A a = new A("1");
a.methodA();
// Standard Java class.
List<Integer> list = new ArrayList<Integer>();
list.add(2);
list.add(3);
} finally {
// Restore.
Thread.currentThread().setContextClassLoader(oldLoader);
}
}
}
And ChangeLoaderTest output:
Create A instance: 1
#methodA1()
No one
Load: ...
Why? How I can change ClassLoader into some thread?

As Marko Topolnik points out the context classloader is for use by frameworks. To use the classloader yourself you have to call loadClass("somepackage.A") and then use the reflection API to create a new instance of A (Class.newInstance()).
You wont be able to use A or its methods in your source directly since the calling code does not know A - it uses a different classloader. An interface or baseclass of A that can be loaded by the normal classloader can be used to avoid reflection.
interface AIF{
void someMethod();
}
class A implements AIF{
public void someMethod(){}
}
public void test(){
MyLoader loader = new MyLoader();
Class cla = loader.loadClass("A");
AIF a = (AIF) cla.newInstance();
a.someMethod();
}

The contextClassLoader mechanisms is not used by the basic Java operations like new. It's only there so various frameworks can access the context class loader in charge and load resources, classes, etc. Java will always use the classloader that loaded the code that is executing. It's the one that you access via ChangeLoaderTest.class.getClassLoader() -- and there is nothing you can do about this one.

I think that what happens is that your application's class loader which is also your classloader's "parent" can locate A and load it. As a result your classloader will not be searched or used for loading A.
To be honest, I haven't much experience with classloaders but if you subclassed one that uses a URL for the path of the class (so that it can locate the class file) and the parent classloader can not load it (not part of classpath), your custom one will be used.

how Java classloaders work for "regular" circumstances (nonexplicit use of classloaders)

I'm looking into dynamic modification of classpath. I found one solution that works nicely but it does so using an explicit call to addURL(). (presumably at startup)
However, I would like to intercept the class-loading process at runtime to locate classes if the default classloader can't seem to find them. I tried to subclass ClassLoader so it just delegates findClass() and loadClass() to the default, and print out a debug line telling me these methods have been called, but they never seem to get called when my class uses dependent classes via implicit classloading, e.g.
// regular object instantiation with 'new'
BrowserLauncher launcher;
launcher = new BrowserLauncher();
// static methods
Foobar.doSomethingOrOther();
// Class.forName()
Class cl = Class.forName("foo.bar.baz");
// reflection on a Class object obtained statically
Class<Foobar> cl = Foobar.class;
// do something with cl, like call static methods or newInstance()
How does classloading work under these circumstances? (vs. the simpler case where Classloader.loadClass() is called explicitly)
Here's my attempt at a custom classloader, below. If I use DynClassLoader0.main() with an arguments list of {"some.package.SomeClass", "foo", "bar", "baz"}, and some.package.SomeClass references other classes found in external .jar files, using one of the methods listed above, why doesn't my DynClassLoader0's findClass() and loadClass() get called? The only time loadClass gets called is the explicit call to loadClass in the main() function below.
package com.example.test.classloader;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
public class DynClassLoader0 extends ClassLoader {
public DynClassLoader0()
{
super();
}
public DynClassLoader0(ClassLoader parent)
{
super(parent);
}
public void runMain(String classname, String[] args) throws ClassNotFoundException, SecurityException, NoSuchMethodException, IllegalArgumentException, IllegalAccessException, InvocationTargetException
{
// [***] here we explicitly use our classloader.
Class<?> cl = loadClass(classname);
Method main = cl.getMethod("main", String[].class);
main.invoke(null, new Object[] {args});
}
#Override protected Class<?> findClass(String name) throws ClassNotFoundException
{
System.out.println("findClass("+name+")");
return super.findClass(name);
}
#Override public Class<?> loadClass(String name) throws ClassNotFoundException
{
System.out.println("loadClass("+name+")");
return super.loadClass(name);
}
static public void main(String[] args)
{
// classname, then args
if (args.length >= 1)
{
String[] classArgs = new String[args.length-1];
System.arraycopy(args, 1, classArgs, 0, args.length-1);
ClassLoader currentThreadClassLoader
= Thread.currentThread().getContextClassLoader();
DynClassLoader0 classLoader = new DynClassLoader0(currentThreadClassLoader);
// Replace the thread classloader - assumes
// you have permissions to do so
Thread.currentThread().setContextClassLoader(classLoader);
try {
classLoader.runMain(args[0], classArgs);
}
catch (Exception e) {
e.printStackTrace();
}
}
else
{
System.out.println("usage: DynClassLoader {classname} [arg0] [arg1] ...");
}
}
}
edit: I have looked through these questions already:
How do you change the CLASSPATH within Java?
Is it possible to “add” to classpath dynamically in java?
Adding files to java classpath at runtime.
edit: I thought what kdgregory is saying below is correct, that once I use my classloader explicitly (see line in code with [***] as a comment), all the code that executes from that class will cause implicit classloading from the same classloader. Yet my DynClassLoader0.loadClass() never gets called except during the outermost explicit call.

To quote from the ClassLoader JavaDoc:
The methods and constructors of
objects created by a class loader may
reference other classes. To determine
the class(es) referred to, the Java
virtual machine invokes the loadClass
method of the class loader that
originally created the class.
In other words, once you load a class, that class tries to load other classes through the classloader that loaded it. In a normal Java application, that is the system classloader, which represents the classpath passed to the JVM, or the boot classloader, used to load the JVM runtime.
Depending on your needs, there's a variant of Class.forName() that takes a classloader as an argument. If you use this to load a particular class, then references within that class should use the specified classloader.
Edit: I started tracing through your example, but decided it would just be easier to give my own. If you're going to write your own classloader, I suggest starting with the existing URLClassLoader, because it handles a lot of the behind-the-scenes stuff.
So, MyClassLoader takes a single JARfile/directory and loads classes for that directory alone. I've overridden the three methods called to load a class, and simply log their invocation (using System.err because it doesn't buffer output, unlike System.out).
My example uses a library that I'm currently working on; it was convenient, but you can pick any library you want as long as it's not already in your classpath.
The main() method loads a class via MyLoader. Then I invoke a method on that class, in a way that I know will throw an exception that's also part of the library. Note that I invoke the method by reflection: since the library is not on my Eclipse classpath, I couldn't compile it with an explicit reference.
When I run this program (under Sun JDK 1.5 for Linux), I see a lot of calls to loadClass(), both for classes in my library and for those on the classpath. This is expected: the ParseUtil class references a lot of other classes, and will use MyLoader (ie, its classloader) to load them. For those classes that MyLoader can't find locally, it delegates up the loader tree.
The exception is thrown, and when I print out its classloader I see that it's the same as the MyLoader instance I created. I also print out the loader for Exception.class, and it's null -- which the JavaDoc for Class.getClassLoader() says indicates the boot classloader.
import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
public class ClassLoaderExample
{
private static class MyClassLoader
extends URLClassLoader
{
public MyClassLoader(String path)
throws Exception
{
super(new URL[] { new File(path).toURL() });
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException
{
System.err.println("findClass(" + name + ")");
return super.findClass(name);
}
#Override
protected synchronized Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException
{
System.err.println("loadClass(" + name + "," + resolve + ")");
return super.loadClass(name, resolve);
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException
{
System.err.println("loadClass(" + name + ")");
return super.loadClass(name);
}
}
public static void main(String[] argv)
throws Exception
{
ClassLoader myLoader = new MyClassLoader("/home/kgregory/Workspace/PracticalXml-1.1/target/classes/");
System.out.println("myLoader = " + myLoader);
Class<?> parseUtilKlass = myLoader.loadClass("net.sf.practicalxml.ParseUtil");
Method parseMethod = parseUtilKlass.getDeclaredMethod("parse", String.class);
try
{
parseMethod.invoke(null, "not at all valid XML");
}
catch (InvocationTargetException e)
{
Throwable ee = e.getCause();
System.out.println("exception:" + ee);
System.out.println("exception loader = " + ee.getClass().getClassLoader());
System.out.println("Exception.class loader = " + Exception.class.getClassLoader());
}
}
}
Edit #2, based on today's comments.
A classloader is expected to delegate requests to its parent before it attempts to fulfill the request itself (this is in the ClassLoader JavaDoc). There are a couple of benefits to this practice, foremost being that you won't unintentionally load incompatible instances of the same class.
J2EE classloaders amend this model: the classloader used to load a WAR will attempt to resolve classes before the loader for a containing EAR, which in turn attempts to resolve classes before the container's classloader. The goal here is isolation: if both the WAR and its EAR contain the same library, it's probably because the two need differing versions (that, or they have a sloppy build process). Even in the J2EE case, I believe that the container classloader delegates in the standard way.

In your code the call to super.loadClass() delegates the loading of the class to the parent classloader (just look at the implementation of java.lang.ClassLoader#loadClass). So it is not your instance of DynClassLoader0 that loads the class, but the currentThreadClassLoader (which you took from Thread.currentThread().getContextClassLoader()) that you passed as a constructor parameter to DynClassLoader0. And when the loaded class refers to other classes, they are then also loaded by that classloader and not your DynClassLoader0.