We Keep Coding

java dynamic class loading that avoids java.lang.IllegalAccessError

Oracle JavaDocs explains that IllegalAccessError is
"Thrown if an application attempts to access or modify a field, or to
call a method that it does not have access to."
I try to load a class dynamically and I get this exception.
if I understand correctly when you use a classloader to load a class with a private package dynamically IllegalAccessError happens
the class I am trying to load is using
org.xml.sax.helpers.SecuritySupport
which also states in their description in the following url
http://grepcode.com/file/repository.springsource.com/org.apache.xmlcommons/com.springsource.org.apache.xmlcommons/1.3.4/org/xml/sax/helpers/SecuritySupport.java
that
Unfortunately, we can't load the class using reflection
* because the class is package private. And the class has
* to be package private so the APIs aren't exposed to other
* code that could use them to circumvent security. Thus,
* we accept the risk that the direct reference might fail
* on some JDK 1.1 JVMs, even though we would never execute
* this code in such a case. Sigh...
how can I dynamically load it anyway? I have to get it to work.
also if I get an error when I use a classloader, I cannot recover from that, so how can I know in advance that I cannot load this class?
thanks in advance to anyone who helps

The statement “we can't load the class using reflection because the class is package private” doesn’t make any sense, as can be shown easily:
package somepackage;
class BaseClass {
public static void main(String[] args) throws ReflectiveOperationException {
BaseClass obj=(BaseClass)
Class.forName("somepackage.SubClass").newInstance();
obj.aMethod();
}
void aMethod() {
System.out.println("base class");
}
}
class SubClass extends BaseClass {
#Override
void aMethod() {
System.out.println("method overridden by subclass");
}
}
This works flawlessly, printing method overridden by subclass replicated the actual use case of that SecuritySupport class.
However, since that class obviously serves the purpose of allowing a transition between Java 1.1 and Java 1.2, it might be possible that there were such restrictions twenty years ago, when this transition happened.
Your use case, however, is entirely different. You say that you are trying to load a class which “is using org.xml.sax.helpers.SecuritySupport”, which doesn’t imply that it is using said class via Reflection, but as shown above, that doesn’t matter anyway. It either case, it would only work, if the class is in the same package, whether you load the class “dynamically” or not.
There are only two possible scenarios.
If the class is truly within the same package, which at runtime implies that it also has been loaded by the same class loader, which would require that is also part of the JRE, if the JRE’s org.xml.sax.helpers package defines a SecuritySupport class, then the class can access the class within the same package.
If you are trying to load a class via a different ClassLoader from a different code source, it will not be of that package, even if you’d give it a qualified name of the org.xml.sax.helpers.SomeClass form. If the JRE’s org.xml.sax.helpers package happens to define a SecuritySupport class, all non-JRE classes would be in a different package. When it tries to access that class, which is not part of the official API, it doesn’t work.
Note that all standard class loaders follow a delegation model trying to resolve a name through their parent class loader first, which is the reason why they all would prefer the JRE’s org.xml.sax.helpers.SecuritySupport class, if there is one. With non-standard class loaders, you could have different, unrelated classes with that qualified name, being in different runtime packages.
In that second scenario, the question arises, why your class is using that class. In 2017, there’s rarely a need to differentiate between Java 1.1 and Java 1.2 and the functionality offered by that class is also only relevant for a class within the privileged code source of the JRE (or different code sources with different privileges in general).

Why can I not look up classes from sun.misc.DelegatingClassLoader

From this question, I found that it is not possible to look up a class from a sun.misc.DelegatingClassLoader, i.e. looking up a class on its own class loader like
Class<?> accessor = ...
accessor.getClassLoader().findClass(accessor.getName());
throws a ClassNotFoundException. The delegating class loader is used for loading runtime-generated accessor classes for converting Java's reflective JNI calls into Java invocations.
For some strange reason, I am not able to find the source of the DelegatingClassLoader anywhere in the JDK sources even though it is clearly available in my build where it seems to be an empty implementation of a subclass of the standard ClassLoader from looking at the class's byte code.
If the DelegatingClassLoader is however really only a simple subclass, I do not understand why it is not possible to look up a class by its name. Is there some VM magic involved? It seems like the
private native final Class<?> findLoadedClass0(String name);
method does not return the loaded class for a DelegatingClassLoader. The private classes field does however include the loaded class.
I was not able to find any information on how these class loaders are supposed to be different from any other class loaders. I am looking for an explanation why the above lookup does not work but throws an exception.

The source for DelegatingClassLoader is for whatever reason located in ClassDefiner.java. Interestingly, this code has a comment that references bug 4474172, which suggests that this class loader is known to the JVM and has special behavior:
The second solution is to make the virtual machine "magically"
delegate class loading for these newly-fabricated class loaders to
their parent loaders, without making an upcall to Java. The
disadvantage is that this is another hack in the JVM [...]. This
option has been chosen.

Is it possible to "reset" a class loader?

I have to dynamically load a class with the same name from some JAR, but different implementation, multiple times.
I'm creating an evaluator backend and I have to dynamically load classes and test them.
The tests are JUnit classes that instantiate the classes which should be tested, this is a simple example:
package evaluator.tests;
import static org.junit.Assert.*;
import org.junit.*;
import evaluator.tested.*;
public class KTest {
private K tested;
#Before
public void setup() {
tested = new K();
}
#Test
public void returnsTrueTest() {
assertTrue(tested.returnsTrue());
}
}
The rest of my application would need to receive JAR files from users which would contain implementations of the K class which is being tested above. Then, KTest would have to run on their K classes, not the ones in the application.
I know how to dynamically load a class, but I don't know how to make a test work with it, and not the one which I made.
One of the solutions I came up with was to isolate testing in a completely new class, e.g. Evaluation, create a new class loader in that class, and make it load all referenced classes. After creating the class loader, it would load the K class from the JAR file.
This would mean that each time a user submits his JAR, a separate Evaluation would be instantiated, it would create its own class loader and start the JUnit test. When that happens, the test would use the user's implementation of K, and not the default one.
Is this possible, and how can it be done?
I read that class loaders always ask their parent whether a class is already loaded. This would mean that I would have to somehow "flush" all the classes that I loaded dynamically from the JAR file in Evaluation, so that they would be unloaded and then loaded again in another Evaluation.
Load class K, test class K, unload class K, repeat with different K.

Yes you can do this.
If you for example use java.net.URLClassLoader with null as parent: new URLClassLoader( urlArray , null ). Then the bootstrap ClassLoader will be used as parent for your ClassLoader.
Here is an example Class which simply uses a new classloader two reload a class.
package com.anarsoft.agent.regression;
import java.net.URL;
import java.net.URLClassLoader;
public class TestClassLoading {
public static boolean field = false;
public static void main(String[] args) throws Exception
{
URL[] urlArray = new URL[] { TestClassLoading.class.getProtectionDomain().getCodeSource().getLocation().toURI().toURL() };
URLClassLoader firstClassloader = new URLClassLoader( urlArray , null );
Class firstClass = firstClassloader.loadClass("com.anarsoft.agent.regression.TestClassLoading");
firstClass.getField("field").setBoolean(null,true);
System.out.println(firstClass.getField("field").getBoolean(null)); // true
URLClassLoader secondClassloader = new URLClassLoader( urlArray , null );
Class secondClass = secondClassloader.loadClass("com.anarsoft.agent.regression.TestClassLoading");
System.out.println(secondClass.getField("field").getBoolean(null)); // false
// the static field is false since its a new loaded class
}
}

I've made a blog post on this subject, in which the problematics is explained a bit further.
After some research I think I've found a way to accomplish exactly what I want. I haven't implemented this system yet (will edit the answer once I do), so I'd like some feedback from people who are more experienced in this regard.
This is how I understand URLClassLoaders (and classloaders in general) work:
URLClassLoader.loadClass() gets automatically called if it’s the current classloader (for the method/class executing). The call is first delegated to its parent classloader, and if nothing is found it uses its own custom findClass() to load it off one of the URLs it has. loadClass() here (and the classloading logic) is just inherited from the regular ClassLoader.
The default behavior of this method (loadClass()) is to first delegate the search to the parent, and if parent cannot find the class, only then to call its own findClass(). This method (findClass()) is, by default (in ClassLoader), left unimplemented, and you are supposed to implement it yourself, by essentially getting the bytecodes of the class from somewhere (e.g. a file or a network) and calling defineClass() on them.
Once you call defineClass() on some class, and only then, are you registered as the classloader of this class. In all other cases classloading is either delegated to your parent (and you are not the classloader of the class you are loading, paradoxically), or you throw a ClassNotFoundException. You cannot change the classloader of some class at runtime, it is set once it is loaded, and constant.
All of my test classes will try to getClass().getClassLoader().loadClass() all of the classes that they reference - including my custom test classes (this is the regular behavior of all classes, not just my tests, to be clear). As long as they’re using standard classes, and other classes from my application that are not to-be-tested classes, this classloading method should be delegated further on to the application classloader. However, as soon as they try to load a to-be-tested class, they need to get their own, specifically loaded, custom version of it.
The use case of my application is that a user submits a JAR with some class, then a test that expects this class to have certain methods is ran on the class from that JAR (using JUnit), and then the results are sent back to the user.
The solution is as follows:
Implement basic versions of the to-be-tested classes so that the tests would compile and run without any submitted JARs. This could (more likely, should) be done by leveraging polymorphism, but is not planned at the moment (this means that the user should most likely extend the “basic” version of the class himself, locally, before sending the class to be tested).
Extend URLClassLoader and re-work the classloading logic.
Do the necessary checks (is the class already loaded) that exist in the default implementation.
Try to findClass yourself, if it’s not in your URLs, throw a ClassNotFoundException.
If the class has already been loaded, or if has been loaded just now from some URL, return the class.
If the class has neither been loaded before (by this classloader), nor is it in one of the URLs, delegate the search to your parent.
If the parent returns the class, return the class, if not, throw ClassNotFoundException (it will actually be thrown by the parent).
For each JAR file with a class that is sent:
Instantiate the custom URLClassLoader
Add the JAR to it and the specific test class as well
Ask it to load the test class. At this point my custom classloading logic kicks in, and it loads my test straight from the disk - anew, without delegating to its parent classloader. Why? It calls defineClass() on my test class, which sets this custom URLClassLoader as the parent of the test class
Give the test class to JUnit, which then instantiates it and begins testing
Once one of my tests are running, each time they reference any class, they will call their own custom classloader’s loadClass(). First they will search the URLs - so in case they’re referencing a to-be-tested class, it will be loaded from there. In case they are referencing some other application class, or a system class, the custom classloading logic will just delegate this call and an already-loaded (presumably) class would be returned.
As I’ve said - I haven’t implemented this yet, and I would really like if you’d point out my mistakes in the comments.
Resources from which I've gathered this information:
http://www.onjava.com/pub/a/onjava/2003/11/12/classloader.html
http://www.devx.com/Java/Article/31614
http://www.onjava.com/pub/a/onjava/2005/01/26/classloading.html
http://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html#loadClass%28java.lang.String,%20boolean%29

You need to make sure that jar you are loading classes from is unknown to JVM system class loader (not on the classpath). Then system class loader will not be loading classes from the jar thus making your custom class loader to load from this jar.

You can fool it.
Cass that you want to reset should not be in the classpath
Can implement a interface or extend a class that is in the classpath (useful to type cast or need to use reflection to 'discover' and call methods, OR the calling class calls another method that has the code the calls the class(es) under test
every time you want to test, rename the jar, load from the jar, using a URL class loader sub class. On init of your loader null is fine.
So if you load a1.jar first time, load from a2.jar next time and make sure a1.jar is moved to another folder

Classloader variants and usage in Java

I understand that java.lang.ClassLoader is generally the classloader I extend (when needed). Also, when I see the custom classloaders of tomcat, Jetty etc extend from java.lang.ClassLoader.
I'm curious to understand the purpose and usage of com.sun.org.apache.bcel.internal.util.ClassLoader.
Can someone help me understand it ?

com.sun.org.apache.bcel.internal is just a repackaging of Apache BCEL. In particular, the class we're interested in (ClassLoader) is documented here...
Drop in replacement for the standard class loader of the JVM. You can use it in conjunction with the JavaWrapper to dynamically modify/create classes as they're requested.
This class loader recognizes special requests in a distinct format, i.e., when the name of the requested class contains with "$$BCEL$$" it calls the createClass() method with that name (everything before the $$BCEL$$ is considered to be the package name. You can subclass the class loader and override that method. "Normal" classes class can be modified by overriding the modifyClass() method which is called just before defineClass().

judging by the "bcel" part of the package name that classloader doesnt just load classes, it alters the bytecode on the fly (see the bcel homepage).
edit: some more info on what they do with it can be found here : "BCEL is used internally by XSLTC to "compile" XSLT stylesheets into bytecodes for execution"

Can't access Classes in default package

I have a problem with the default package.
Basically my project structure consists of three main classes which extend one abstract class in a package called simulation.
Yesterday I did a name change in the project and now my three main classes are put automatically in the default package, so they can't be visible by the class inside the simulation package and vice versa.
For the following code
import simulation.*;
class SSQSim extends Simulation{
}
I get these errors
"This class must implement the inherited abstract method Simulation.stop(), but cannot override it since it is not visible from SSQSim. Either make the type abstract or make the inherited method visible"
"The type Simulation is not visible"
Thank you in advance.
EDIT
The problem is that I don't have to use any other packages. It's basically homework and the rules for the submission are pretty strict: First, I have to submit the package "simulation" which contains the abstract class "Simulation" with some methods to implement with the help of the other classes. This part is fine.
Then, I have to create three classes which import the package "simulation" and extend its class "Simulation". They have specifically said not to put these classes in any package. At first, they all worked fine but after I renamed the project. These classes suddenly moved into the default package and now they give me these errors.

How are you creating your classes? When you create a new class, Eclipse shows a New Java Class dialog. This dialog lets you choose which package to create your new class in:
Just click the Browse button and you'll be able to pick a package.

Why you use the default package,From the java specs:
It is a compile time error to import a type from the unnamed package.
So you have to create a new package with a different name and add your classes or put them in the same package as your class and do the required imports.

This is due to the fact that the signature of Simulation.stop() method mentions some class that is package-restricted (non-public) to the simulation package.
You solve it by either
Move SSQSim to the simulation package so it can access the same classes as Simulation, or
Make the classes required to extend the Simulation class package-public.
To move a class from one package to another in eclipse, you can simply drag the source-code file into the desired package, and eclipse will refactor the code accordingly.
(A side-note: As a rule of thumb, don't use the default package. The problem you ran into however, can occur even if you avoid the default package!)

This is just a hint for now, as the source code for all the classes is not posted. The error message implies the presence of an abstract method in the parent class, that cannot be overriden due to loss in visibility in the child class. Usually, this is due to the use of the default access specifier ("friendly"), or the private access specifier (which is meaningless, especially if the method is declared as abstract). One will have to use the protected access modifier on the parent, to ensure that the method is now visible to all child classes irrespective of whether they are present in the same package or not.
In short, if the child cannot "see" the method that is being overriden, then the compiler issues a warning that informs about the inevitability to deduce whether the method defined in the parent or the child should used at runtime.
As far as Eclipse's behavior of creating classes in the default package is concerned, one can always ensure that a package is specified when creating a new class in the appropriate dialog (the new Java Class dialog, where a package field is present).