We Keep Coding

Reload class in Groovy

I have a custom ClassLoader extending GroovyClassLoader which compiles the source code to .class files on disk and then loads the resulting class:
class MyClassLoader extends GroovyClassLoader {
File cache = new File( './cache' )
Compiler compiler
MyClassLoader() {
CompilerConfiguration cc = new CompilerConfiguration( targetDirectory:cache )
compiler = new Compiler( cc )
addClasspath cache.path
}
#Override
Class findClass( name ) {
try{
parent.findClass name
}catch( ClassNotFoundException e ){
compiler.compile name, getBodySomehow()
byte[] blob = loadFromFileSystem name
Class c = defineClass name, blob, 0, blob.length
setClassCacheEntry c
c
}
}
#Override
void removeClassCacheEntry​(String name) {
Class c = cache[ name ]
super.removeClassCacheEntry​(name)
GroovySystem.metaClassRegistry.removeMetaClass c
deleteFiles name
}
}
Class clazz = myClassLoader.loadClass 'some.pckg.SomeClass'
Now if I change the source code, call myClassLoader.removeClassCacheEntry​(name) and try myClassLoader.loadClass() again I'm getting:
java.lang.LinkageError: loader (instance of com/my/MyClassLoader): attempted duplicate class definition for name some/pckg/SomeClass
I read the greater half of the Internet and found a "solution" to initialize a class-loader for each class:
MyClassLoader myClassLoader = new MyClassLoader()
Class clazz = myClassLoader.loadClass 'some.pckg.SomeClass'
This seems to be working but raises performance concerns of mine...
What is the proper way to reload classes? How can I reuse the same class-loader? What am I missing?

Actually there is a trick that could be used
Originally, when you call
classLoader.defineClass(className, classBytes, 0, classBytes.length)
It calls java native method defineClass1 that actually calls loadClass method.
So, possible to intercept this method and process it a bit different then original.
In the folder that contains cached class files I have the following groovy compiled to class: A.class
println "Hello World!"
B.class to check dependent class loading
class B extends A {
def run(){
super.run()
println "Hello from ${this.getClass()}!"
}
}
and C.class to check multi-level class loading
i used this jar to compile following class and run the class re-loading example
class C extends org.apache.commons.lang3.RandomUtils {
def rnd(){ nextInt() }
}
the following class + code loads and reloads the same class:
import java.security.PrivilegedAction;
import java.security.AccessController;
import org.codehaus.groovy.control.CompilationFailedException;
#groovy.transform.CompileStatic
class CacheClassLoader extends GroovyClassLoader{
private File cacheDir = new File('/11/tmp/a__cache')
private CacheClassLoader(){throw new RuntimeException("default constructor not allowed")}
public CacheClassLoader(ClassLoader parent){
super(parent)
}
public CacheClassLoader(Script parent){
this(parent.getClass().getClassLoader())
}
#Override
protected Class getClassCacheEntry(String name) {
Class clazz = super.getClassCacheEntry(name)
if( clazz ){
println "getClassCacheEntry $name -> got from memory cache"
return clazz
}
def cacheFile = new File(cacheDir, name.tr('.','/')+'.class')
if( cacheFile.exists() ){
println "getClassCacheEntry $name -> cache file exists, try to load it"
//clazz = getPrivelegedLoader().defineClass(name, cacheFile.bytes)
clazz = getPrivelegedLoader().defineClass(name, cacheFile.bytes)
super.setClassCacheEntry(clazz)
}
return clazz
}
private PrivelegedLoader getPrivelegedLoader(){
PrivelegedLoader loader = AccessController.doPrivileged(new PrivilegedAction<PrivelegedLoader>() {
public PrivelegedLoader run() {
return new PrivelegedLoader();
}
});
}
public class PrivelegedLoader extends CacheClassLoader {
private final CacheClassLoader delegate
public PrivelegedLoader(){
super(CacheClassLoader.this)
this.delegate = CacheClassLoader.this
}
public Class loadClass(String name, boolean lookupScriptFiles, boolean preferClassOverScript, boolean resolve) throws ClassNotFoundException, CompilationFailedException {
Class c = findLoadedClass(name);
if (c != null) return c;
return delegate.loadClass(name, lookupScriptFiles, preferClassOverScript, resolve);
}
}
}
def c=null
//just to show intermediate class loaders could load some classes that will be used in CacheClassLoader
def cl_0 = new GroovyClassLoader(this.getClass().getClassLoader())
cl_0.addClasspath('/11/tmp/a__cache/commons-lang3-3.5.jar')
//create cache class loader
def cl = new CacheClassLoader(cl_0)
println "---1---"
c = cl.loadClass('A')
c.newInstance().run()
println "---2---"
c = cl.loadClass('A')
c.newInstance().run()
println "---3---"
cl.removeClassCacheEntry('A')
c = cl.loadClass('A')
c.newInstance().run()
println "---4---"
c = cl.loadClass('B')
c.newInstance().run()
println "---5---"
cl.removeClassCacheEntry('A')
cl.removeClassCacheEntry('B')
c = cl.loadClass('B')
c.newInstance().run()
println "---6---"
c = cl.loadClass('C')
println c.newInstance().rnd()
result:
---1---
getClassCacheEntry A -> cache file exists, try to load it
Hello World!
---2---
getClassCacheEntry A -> got from memory cache
Hello World!
---3---
getClassCacheEntry A -> cache file exists, try to load it
Hello World!
---4---
getClassCacheEntry B -> cache file exists, try to load it
getClassCacheEntry A -> got from memory cache
Hello World!
Hello from class B!
---5---
getClassCacheEntry B -> cache file exists, try to load it
getClassCacheEntry A -> cache file exists, try to load it
Hello World!
Hello from class B!
---6---
getClassCacheEntry C -> cache file exists, try to load it
226399895
PS: not sure priviledged access required

JVM does not allow to just unload some class, the only way to unload a class is to GC it. And class can be GC just like every other object -> all reachable references must be removed and GC run.
The tricky part is... class loader hold references to all classes. So the only way to unload a class is to get rid of both class and class loader.
You can find more information in language specification: https://docs.oracle.com/javase/specs/jvms/se13/jvms13.pdf 12.7 Unloading of Classes and Interfaces
An implementation of the Java programming language may unload classes.
A class or interface may be unloaded if and only if its defining class
loader may be reclaimed by the garbage collector as discussed in
§12.6. Classes and interfaces loaded by the bootstrap loader may not
be unloaded.
And class unloading does not need to be implemented at all in some JVM implementations:
Class unloading is an optimization that helps reduce memory use.
[...] system chooses to implement an optimization such as class
unloading. [...] Consequently, whether a class or interface has been unloaded
or not should be transparent to a program.
There is also explanation why class loader can't be reachable to unload class, as class might contain static variables and blocks of code that would be reset and executed again if this same class would be later loaded again. It's quite long and already a bit off topic so I will not paste it here.
So each your script should just use own class loader as that's the only way to actually not waste memory, so class can be GC later. Just make sure that you don't use any libraries that might cache references to your class - like many serialization/ORM libraries might do this for data types, or some other reflection libraries.
Another solution would be to use different scripting language that does not create java classes and just execute some kind of AST structure.
There is also one more solution to this problem, but it is very tricky and it's not something you should use on production, it even requires you to provide special JVM arguments or JVM from JDK that contains all needed modules. As java supports instrumentation API that can allow you to change bytecode of class at runtime, but if class is already loaded you can only change bytecode of methods, you can't add/remove/edit method/field/class signatures. So it could be very bad idea to use it for such scripts, so I will stop here.

Parent-Child Classloader Class Resolution

The tasks and initial investigation
I try to set up two Oracle Coherence near cache instances at one java swing application. The idea a solution could be found here. My case is a bit more complicated and this is where the game starts.
Short description
In my case there is an account service. It can have two endpoints: SIT and UAT. In order to create two such services, I need to load two 'instances' of the Coherence in order to override the endpoints with system variables (tangosol.coherence.cacheconfig).
I have:
the main code of the app is located in the mainapp.jar;
the AccountService interface that is located in the account-interfaces.jar;
the AccountServiceImpl class that is located in the account-impl.jar and implements the AccountService interface;
my main application has the following structure
bin: startup.bat, startup.sh
conf: app.properties
lib: mainapp.jar, account-interfaces.jar, account-impl.jar, coherence.jar
Approach tried
I created a dedicated child-first classLoader - InverseClassLoader and made the AppLaunchClassLoader (the default Thread.currentThread().GetContextClassLoader() classLoader) it's parent. With the InverseClassLoader I load the AccountServiceImpl class:
Class<AccountServiceImpl> acImplClass = contextClassLoader.selfLoad(AccountServiceImpl.class).loadClass(AccountServiceImpl.class);
Constructor<AccountServiceImpl> acConstructor =
acImplClass .getConstructor(String.class);
AccountService acService = acConstructor .newInstance(serviceURL);
Issues and questions
I get the 'AccountServiceImpl cannot be cast to AccountService' exceptions, which means that those two classes loaded by different classloaders. But those classloaders are in the parent-child relationship. So am I right that even if a class is loaded by a parent (interface - 'abstract' type) it can't be used with a class (concrete impl) loaded by a child classloader? Why then we need this parent-child relation?
I specified the AccountService interface in a code and it got loaded by a default classloader. I tried wrap the code above is a thread and set the InverseClassLoader it's context classloader. Nothing changed. So am I right that I can't use such interface-implementation coding (as usual coding) and need to use reflection all the time to invoke concrete methods all the time? (Hope there is a solution) ;
Say, I listed both the AccountService and AccountServiceImpl classes for being loaded by the InverseClassLoader. What if I need other classes, that are accessible by those two, to be also loaded by the InverseClassLoader? It there a way to say that all 'related' classes must be loaded by the same classloader?
Update
Here is the InverseClassLoader:
public class InvertedClassLoader extends URLClassLoader {
private final Set<String> classesToNotDelegate = new HashSet<>();
public InvertedClassLoader(URL... urls) {
super(urls, Thread.currentThread().getContextClassLoader());
}
public InvertedClassLoader selfLoad(Class<?> classToNotDelegate) {
classesToNotDelegate.add(classToNotDelegate.getName());
return this;
}
#Override
public Class<?> loadClass(String className, boolean resolve) throws ClassNotFoundException {
if (shouldNotDelegate(className)) {
System.out.println("CHILD LOADER: " + className);
Class<?> clazz = findClass(className);
if (resolve) {
resolveClass(clazz);
}
return clazz;
}
else {
System.out.println("PARENT LOADER: " + className);
return super.loadClass(className, resolve);
}
}
public <T> Class<T> loadClass(Class<? extends T> classToLoad) throws ClassNotFoundException {
final Class<?> clazz = loadClass(classToLoad.getName());
#SuppressWarnings("unchecked")
final Class<T> castedClass = (Class<T>) clazz;
return castedClass;
}
private boolean shouldNotDelegate(String className) {
if (classesToNotDelegate.contains(className) || className.contains("tangosol") ) {
return true;
}
return false;
}

Issue 1, part one I cannot reproduce (see below). As for part 2:
the hierarchy of class-loaders is to prevent the "X cannot be cast to X" exceptions.
But if you break the parent-first rule, you can get into trouble.
About issue 2: setting a thread's context classloader does not do anything in itself, see also this article (javaworld.com)
for some more background. Also, in relation to issue 1, part 2, a quote from the article
that describes what can happen if there is no parent-child relation between the current classloader
and the thread's context classloader:
Remember that the classloader that loads and defines a class is part of the internal JVM's ID for that class.
If the current classloader loads a class X that subsequently executes, say, a JNDI lookup for some data of type Y,
the context loader could load and define Y.
This Y definition will differ from the one by the same name but seen by the current loader.
Enter obscure class cast and loader constraint violation exceptions.
Below is a simple demo-program to show that a cast to an interface from another classloader can work
(note I'm using a simple Java project with classes in a bin-folder and the InvertedClassLoader from your question in the same (test) package):
import java.io.File;
public class ChildFirstClassLoading {
public static void main(String[] args) {
InvertedClassLoader cl = null;
try {
File classesDir = new File(new File("./bin").getCanonicalPath());
System.out.println("Classes dir: " + classesDir);
cl = new InvertedClassLoader(classesDir.toURI().toURL());
cl.selfLoad(CTest.class);
System.out.println("InvertedClassLoader configured.");
new CTest("Test 1").test();
ITest t2 = cl.loadClass(CTest.class)
.getConstructor(String.class)
.newInstance("Test 2");
t2.test();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (cl != null) {
try { cl.close(); } catch (Exception ignored) {}
}
}
}
public interface ITest {
void test();
}
public static class CTest implements ITest {
static {
System.out.println("CTest initialized.");
}
private String s;
public CTest(String s) {
this.s = s;
}
public void test() {
System.out.println(s);
}
}
}
If you change ITest t2 = to CTest t2 = you will get the "CTest cannot be cast to CTest" exception,
but using the interface prevents that exception.
Since this little demo works fine, I'm guessing there is more going on in your application which somehow breaks the class-loading.
I suggest you work from a situation where the class-loading works and keep adding code until it breaks the class-loading.
The InvertedClassLoader looks a lot like the "child first classloader", see this question
for some good answers discussing this manner of class-loading.
The child first classloader can be used to load "related classes" (from your third issue) separately.
You could also update the InvertedClassLoader to always "self-load" classes in certain packages.
And remember that "once a class is loaded by a classloader it uses that classloader to load every other class it needs"
(quote from this blog article).

How to set my ClassLoader as the JVM's ClassLoader for loading ALL classes (jar classes included)?

I am writing a game engine in which I need to analyze every single class that is mentioned in my program. As this is a game engine, it is to be attached to a client's project as a JAR file. From within that JAR file I need to be able to scan every class that is being used by the client.
So I thought that I should create a custom ClassLoader! By overriding ClassLoader I can take a look at every class that is being used as it is loaded.
I started playing around a bit with ClassLoaders. Here is what I did:
(I stole this classloader from JavaWorld just to play around with it)
public class SimpleClassLoader extends ClassLoader {
private HashMap<String, Class> classes = new HashMap<String, Class>();
public SimpleClassLoader() {
}
/**
* This sample function for reading class implementations reads
* them from the local file system
*/
private byte getClassImplFromDataBase(String className)[] {
System.out.println(" >>>>>> Fetching the implementation of "+className);
byte result[];
try {
FileInputStream fi = new FileInputStream("store\\"+className+".impl");
result = new byte[fi.available()];
fi.read(result);
return result;
} catch (Exception e) {
/*
* If we caught an exception, either the class wasnt found or it
* was unreadable by our process.
*/
return null;
}
}
/**
* This is a simple version for external clients since they
* will always want the class resolved before it is returned
* to them.
*/
public Class loadClass(String className) throws ClassNotFoundException {
return (loadClass(className, true));
}
/**
* This is the required version of loadClass which is called
* both from loadClass above and from the internal function
* FindClassFromClass.
*/
public synchronized Class loadClass(String className, boolean resolveIt)
throws ClassNotFoundException {
Class result;
byte classData[];
System.out.println(className);
System.out.println(" >>>>>> Load class : "+className);
/* Check our local cache of classes */
result = (Class)classes.get(className);
if (result != null) {
System.out.println(" >>>>>> returning cached result.");
return result;
}
/* Check with the primordial class loader */
try {
result = super.findSystemClass(className);
System.out.println(" >>>>>> returning system class (in CLASSPATH).");
return result;
} catch (ClassNotFoundException e) {
System.out.println(" >>>>>> Not a system class.");
}
/* Try to load it from our repository */
classData = getClassImplFromDataBase(className);
if (classData == null) {
throw new ClassNotFoundException();
}
/* Define it (parse the class file) */
result = defineClass(classData, 0, classData.length);
if (result == null) {
throw new ClassFormatError();
}
if (resolveIt) {
resolveClass(result);
}
classes.put(className, result);
System.out.println(" >>>>>> Returning newly loaded class.");
return result;
}
}
Then I decided to test it:
public class Test
{
public static void main(String args[]) throws ClassNotFoundException
{
SimpleClassLoader s = new SimpleClassLoader();
Thread.currentThread().setContextClassLoader(s);
Foo myfoo = new Foo(); //local class
ArrayList myList = new ArrayList(); //class from JAR file
//both should have been loaded from my SimpleClassLoader
System.out.println(s + "\n\tshould be equal to\n" + myfoo.getClass().getClassLoader());
System.out.println("\tand also to: \n" + myList.getClass().getClassLoader());
/*
OUTPUT:
SimpleClassLoader#57fee6fc
should be equal to
sun.misc.Launcher$AppClassLoader#51f12c4e
and also to:
null
***
bizarre results: why are ArrayList and Foo not being loaded by my classloader?
*/
}
}
Is creating a custom ClassLoader the correct approach to the problem described at the top?
Why is the system class loader being invoked? How do I force the JVM to use MY classloader, for ALL threads? (not just the first thread, I want every new thread created to automatically use my classloader)
It is my understanding that via the "delegation system", the parent ClassLoader gets the first try at loading the class; this may be the reason my ClassLoader isn't loading anything. If I am correct, how to I disable this feature? How do I get MY classloader to do the loading?

The instrumentation api would be a natural fit for what I think you're trying to do
http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/Instrumentation.html
It lets you intercept classes as they are loaded by the JVM - normally to modify them, but you are given the byte array to play with so I can see no reason why you couldn't perform any required analysis without actually modifying.
This would let you analyse classes before they are loaded. I'm not entirely clear on what it is you are trying to achieve, but if you can analyse the classes after they are loaded then you could simply open and parse the contents of all jars on the classpath.

ClasscastException in servlet

I try to implement a plugin system in a servlet. I've written a class to load plugin that use URLClassLoader to load the jar files and Class.forname to load the class.
Here is my code:
This part create the url class Loader:
public PluginLoader(ServletContext context, String[] pluginName, String[] classToLoad) throws PluginLoaderException{
this.context = context;
urls= new URL[pluginName.length];
nameToURL(pluginName);
//create class loader
loader = new URLClassLoader(urls);
//loading the plug-in
loadPlugin(classToLoad);
}
This one initialize the url:
private void nameToURL(String[] pluginName) throws PluginLoaderException{
try{
for(int i=0;i&ltpluginName.length;i++){
urls[i] = context.getResource(pluginName[i]);
}
}
Finally this one create the object:
private void loadPlugin(String[] classToLoad) throws PluginLoaderException{
try{
iTest = (ITest) Class.forName(classToLoad[0],true,loader).newInstance();
}
catch(Exception e){
throw new PluginLoaderException(e.toString());
}
}
I have managed to create the object because I can manipulate it and retrieve the interface it implements but I can't cast it in ITest to manipulate it in the application. I have a ClassCastException tplugin.toto.Toto cannot be cast to fr.test.inter.ITest .
It's strange because Toto implements ITest.
Does anyone has an idea ?
Thanks

You've created a classoader issue -- when you test with instanceof ITest, you are using the copy of ITest loaded by the default classloader, but you are testing an instance loaded by the URLClassloader. That classloader has loaded its own copy of ITest, which, as far as the JVM is concerned, is a completely different type.

Good way to load ressources

I am developping a server application in Java. I need to load some ressources from different sources (XML and a Database). So, i need some advice on how to cleanly implement the loading.
I have a class "ServerX" who create some "Memory" object, it's those objets who'll hold the loaded ressources.
I've found two different way of loading, but both seems dirty.
1
public class ServerX
{
/**
Will hold the houses for further use.
*/
private Memory<House> houses;
public ServerX()
{
houses = new Memory<House>();
loadHouses();
loadXX();
loadYY();
LoadZZ();
Load...
}
private void loadHouses()
{
//Pseudo code
List<House> loaded = loadHousesFromDatabase();
houses.addAll(loaded);
}
private void loadXX();
...
}
But this way, it flood my "ServerX" class.
2
public interface Loader
{
public void loadHouses(Memory<House> toFill);
public void loadXX(Memort<XX> toFill);
public void loadYY(Memort<YY> toFill);
public void loadZZ(Memort<ZZ> toFill);
}
public class SimpleLoader implements Loader
{
//Implements methods.
}
public class ServerX
{
/**
Will hold the houses for further use.
*/
private Memory<House> houses;
public ServerX(Loader loader)
{
houses = new Memory<House>();
loader.loadHouses(houses);
loader.loadXX...
}
}
But this way, i think i fall into the Poltergeist antipattern, because i create a new loader only to do the request to the database/XML file, and then it's garbage-collected.
So, is there another way to do it, or is one of my solutions good enough?
Thanks.

One pattern you can consider is the Service Locator Pattern. An explanation of Service Locator can be found here.
Basically, a service locator is a registry + cache combined to find the resource once and keep it in memory for object retrieval during the lifecycle of the application. Service Locator is mainly implemented using the Singleton pattern.

Your second solution using a Loader interface and multiple implementations (XMLLoader and DBLoader) is good. However, keep the loader and the server decoupled by making the load methods return a new Memory instance instead of passing the memory as a reference. Add a copyAll method in Memory class to copy the contents of one memory into another memory. ( See ArrayList.addAll or System.arrayCopy in javadoc )