We Keep Coding

NoClassDefFoundError after instrumenting code

I'm attaching my Java agent dynamically to a java process which instruments the code. Basically it adds a static call to every start of method:
//method start
AgentClass.staticMethod();
//method body
AgentClass lies in the agent's .jar. But after instrumentation, the process starts executing the new code and it throws a NoClassDefFoundError, it cannot find AgentClass.
I tried to istrument the classes in a way to include a try-catch block and load AgentClass with forName like this:
try {
AgentClass.staticMethod();
} catch(NoClassDefFoundError e) {
Class.forName("AgentClass");
}
But then I got several errors related to the recalculation of stack frames like:
Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 20
I solved this by using visitMaxs() (I am using ASM library).Then I got this: StackMapTable error: bad offset.
This was solved by using GOTO instead of RETURN but then I got: ClassFormatError: Illegal local variable table in method.
Is there any easier way to solve my initial NoClassDefFoundError error?
UPDATE: My agent classes are loaded with the Application Classloader(sun.misc.Launcher$AppClassLoader), and the process which I wanted to instrument loads classes with a custom URL classloader.
UPDATE2:
This is what I wanted to transform into bytecode:
try {
AgentClass agent = AgentClass.staticMethod();
} catch (Throwable e) {
try {
Class.forName("AgentClass");
} catch (ClassNotFoundException ex) {
}
}
My MethodVisitor(I am not very good at bytecode, so the bytecode was generated automatically by ASM, using a TraceClassVisitor.):
protected MethodVisitor createVisitor(MethodVisitor mv,final String name,final String desc,int access,String signature,String[]exceptions){
int variablesCount = (8 & access) != 0 ? 0 : 1;
Type[]args=Type.getArgumentTypes(desc);
for(int i=0;i<args.length; ++i){
Type arg=args[i];
variablesCount+=arg.getSize();
}
final int varCount=variablesCount;
return new MethodVisitor(458752,mv){
public void visitCode(){
Label label0=new Label();
Label label1=new Label();
Label label2=new Label();
this.mv.visitTryCatchBlock(label0,label1,label2,"java/lang/Throwable");
Label label3=new Label();
Label label4=new Label();
Label label5=new Label();
this.mv.visitTryCatchBlock(label3,label4,label5,"java/lang/ClassNotFoundException");
this.mv.visitLabel(label0);
this.mv.visitLineNumber(42,label0);
this.mv.visitMethodInsn(Opcodes.INVOKESTATIC,"AgentClass","staticMethod","()LAgentClass;",false);
this.mv.visitVarInsn(Opcodes.ASTORE,varCount);
this.mv.visitLabel(label1);
this.mv.visitLineNumber(48,label1);
Label label6=new Label();
this.mv.visitJumpInsn(Opcodes.GOTO,label6);
this.mv.visitLabel(label2);
this.mv.visitLineNumber(43,label2);
this.mv.visitFrame(Opcodes.F_SAME1,0,null,1,new Object[]{"java/lang/Throwable"});
this.mv.visitVarInsn(Opcodes.ASTORE,0);
this.mv.visitLabel(label3);
this.mv.visitLineNumber(45,label3);
this.mv.visitLdcInsn("AgentClass");
this.mv.visitMethodInsn(Opcodes.INVOKESTATIC,"java/lang/Class","forName","(Ljava/lang/String;)Ljava/lang/Class;",false);
this.mv.visitInsn(Opcodes.POP);
this.mv.visitLabel(label4);
this.mv.visitLineNumber(47,label4);
this.mv.visitJumpInsn(Opcodes.GOTO,label6);
this.mv.visitLabel(label5);
this.mv.visitLineNumber(46,label5);
this.mv.visitFrame(Opcodes.F_FULL,1,new Object[]{"java/lang/Throwable"},1,new Object[]{"java/lang/ClassNotFoundException"});
this.mv.visitVarInsn(Opcodes.ASTORE,1);
this.mv.visitLabel(label6);
this.mv.visitLineNumber(49,label6);
this.mv.visitFrame(Opcodes.F_CHOP,1,null,0,null);
this.mv.visitInsn(Opcodes.RETURN);
this.mv.visitLocalVariable("e","Ljava/lang/Throwable;",null,label3,label6,0);
this.mv.visitMaxs(1, 2);
super.visitCode();
}
...
}
}
UPDATE 3
This is how I attach my agent during runtime:
final VirtualMachine attachedVm = VirtualMachine.attach(String.valueOf(processID));
attachedVm.loadAgent(pathOfAgent, argStr);
attachedVm.detach();

For now my guess is that your class loader hierarchy is something like:
boot class loader
platform class loader
system/application class loader
custom URL class loader
Or maybe:
boot class loader
platform class loader
system/application class loader
custom URL class loader
I.e. the application class loader and the custom URL class loader are siblings or in some other way in different parts of the class loader hierarchy, i.e. the classes loaded in one of them are unknown to the other one.
The way to solve this would be to find a common ancestor and make sure the classes needed for your instrumentation scheme are loaded there. I usually use the bootstrap class loader. Before I explain to you how to programmatically add classes to the bootstrap class loader, please try adding your agent JAR to the bootstrap class path manually on the Java command line via -Xbootclasspath/a:/path/to/your/agent.jar and see if the custom URL class loader then finds the class. I would be very surprised if that would not work. Then please report back and we can continue.
Please also explain how you attach the instrumentation agent:
via -javaagent:/path/to/your/agent.jar or
via hot-attachment during runtime (if so, please show the code)
Update after some clarifying OP comments:
It is possible to add a JAR (not single classes) to the bootstrap class path by calling method Instrumentation.appendToBootstrapClassLoaderSearch(JarFile). In your agent's premain or (for hot-attachment) agentmain methods the JVM passes you an Instrumentation instance you can use for that purpose.
Caveat: You need to add the JAR before any of the classes you need on the bootstrap classpath have been imported or used by other, already loaded classes (including the agent class itself). So if in your case the AgentClass method called by the other class in the sibling class loader happens to reside inside the same class housing the premain and agentmain methods, you want to factor that method (and all others that might be called from outside) into another utility class. Also, do not directly refer to that class from the agent main class, rather first make the agent add its own JAR to the boot class path and then call any methods in there via reflection rather than directly from the agent main class. After the agent main class has done its job, other classes can refer to the classes that are now on the bootstrap class path directly, the problem is solved.
One problem remains, though: How does the agent find out the JAR path to add to the bootstrap class path? That is up to you. You can set a system property on the command line, read the path from a file, hard-code, hand it over as an agent configuration string passed to premain/agentmain via attachedVm.loadAgent(agentPath, configString) (in this case configString containing the agent path again) or whatever. Alternatively, create an inner JAR as a resource inside the main agent JAR, containing the classes to be put on the bootstrap class loader. The agent can load the resource, save it into a temporary file and then add the temp-file path to the bootstrap class path. This is a bit complicated, but clean and thus quite popular among agent developers. Sometimes this scheme is referred to as a "trampoline agent" approach.

Java ClassLoader - force reloading already loaded classes

I'm currently trying to load classes into my application so that I can then filter out those that don't contain any test / #Test-methods. I want to run these tests in my application afterwards.
So far, so good - except it seems like the URLClassLoader I'm using (or probably any ClassLoader) doesn't actually reload classes that are located on my applications classpath.
More precisely, a user of my application starts by selecting a number of .java source files. Those are then copied to a temporary location and a number of regex match/replace pairs are applied to the copy of the original source files. Next, the copies are compiled, and I then want to load those compiled .class-files into my application so I can run them as tests.
In most cases, this works as desired.
Unfortunately, if the fully qualified class name (FQCN) of any of the selected files is identical to the FQCN of a class that is part of this application (such as tests for this application), the ClassLoader ignores the specified path (to %temp%\myfolder\) and the (updated) version of the class file located there, but instead uses the already present/loaded version of the class.
After some research, this behaviour can be expected according to the docs (emphasis mine):
• The loadClass method in ClassLoader performs these tasks, in order, when called to load a class:
- If a class has already been loaded, it returns it.
- Otherwise, it delegates the search for the new class to the parent class loader.
- If the parent class loader does not find the class, loadClass calls the method findClass to find and load the class.
I tried using findClass, but it's unfortunately not visible.
Hence my question - does anyone know how to force java / the ClassLoader to actually load the class from the specified path, ignoring any - FQCN-wise - identical existing classes?

A classloader first delegates to its parent classloader, which is how it determines "if a class has already been loaded". If you want to force a classloader to load a new class, one way is to insert another classloader in the chain which fails to load/find the same class. Very quick, incomplete example:
class FirewallLoader extends ClassLoader {
public FirewallLoader(ClassLoader parent) {
super(parent);
}
public loadClass(String name, boolean resolve) {
if (name.equals("some.class.Xyz")) {
throw ClassNotFoundException();
}
super.loadClass(name, resolve);
}
}
You make the "FirewallLoader" the parent or your URLClassLoader, and then your URLClassLoader will load new versions of whatever classes the Firewall loader filters out.

how can I know the current classLoader in java

I am learning the process of loading java class and encounter some confusion.
I know when loading a java class, current classLoader wont load the java class directly and it will delegate to its parent classLoader(a recursive process) until it parent cant load this class.
The question is that ：what is the current classLoader? Bootstrap? Extension? App?
how to get the current classLoader?.
and I know there is an API:
xxx.Class.getClassLoader();
but I am not sure whether the return value is currentClassLoader. I think it should be the classLoader which load this java class in reality.
To describe my question more detail I will give an example.
I get the follow content in a blog.
ThreadContextClassLoader is used to deal with java SPI, Interface is defined in java core lib and loaded by Bootstrap ClassLoader and third party implement these interface then the jar are loaded by AppClassLoader
Solution: traditional classLoader cant deal with this case,because it cant discovery the third party jar when we use the third party implement in core lib.
most of the above that I can understand but the solution make me confusion:
for example, the Interface CoreA and class CoreB are in java core lib and should be loaded by Bootstrap ClassLoader and the AImpl is a implement of A by third party and should be loaded by AppClass loader.
the code segment as below:
public Interface CoreA{
void add();
}
public Interface AImpl implements CoreA{
void add(){};
}
public class B{
public void demo(){
a = new AImpl();
}
}
then if we reference B in main method, then we will load B because the class loader of B is Bootstrap then about AImpl the current Loader is Bootstrap so it cant be found?
I dont know whether it is as what I guess?
Any advice will be appreciated.

Generally speaking you are right, it can't be found. Let me show you the following example. Let's say we have 3 classes: A, B and Main like these:
public class A {
public String a() {
return "a";
}
}
public class B {
public String b() {
return new A().a();
}
}
public class Main {
public static void main(String... args) {
System.out.println(new B().b());
}
}
And we package these classes into correspondent jars: a.jar, b.jar and place Main.class into the working directory. After that let's test the following scenarios:
1) Everything (A.class, B.class, Main.class) is loaded by system classloader and works fine:
$ java -cp .:a.jar:b.jar Main
a
2) B.class is loaded by system classloader and A.class is loaded by bootstrap classloader and everything still works fine because system classloader delegates loading to bootstrap classloader (just because bootstrap classloader can load it):
$ java -Xbootclasspath/a:a.jar -cp .:b.jar Main
a
3) A.class is loaded by system classloader and B.class is loaded by bootstrap classloader (your case). In this scenario during loading of B.class current classloader is bootstrap classloader, but it can't found B.class and fails:
$ java -Xbootclasspath/a:b.jar -cp .:a.jar Main
Exception in thread "main" java.lang.NoClassDefFoundError: A
at B.b(B.java:4)
at Main.main(Main.java:4)
Lets take a look at the last example more carefully. What's happening here:
Try to find class with public static main(String[] args) method
1.1. system classloader hasn't loaded it yet so delegates to extension classloader
1.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
1.3. bootstrap classloader hasn't loaded it yet and tries to load, it can't load it and returns control to extension classloader
1.4. extension classloader tries to load, it can't load it and returns control to system classloader
1.5. system classloader loads Main.class
Main.class is processed and we try to load B.class with current classloader system classloader
2.1. system classloader hasn't loaded it yet so delegates to extension classloader
2.2. extension classloader hasn't loaded it yet so delegates to bootstrap classloader
2.3. bootstrap classloader hasn't loaded it yet and loads B.class
B.class is processed and and we try to load A.class with current classloader bootstrap classloader
3.1. bootstrap classloader hasn't loaded it yet and tries to load and
fails
I hope it will help you.

When a class A attempts to load another class B, the ClassLoader that loaded A is the current ClassLoader. The word current vaguely refers to the execution context - e.g. how do you end up in the method that triggers the current class loading call.
There is no method - say getCurrentClassLoader - that simply gives the current ClassLoader, but there are api methods that internally use the concept of current ClassLoader. For example, Class.forName(String className)
If you check how that method is implemented, it tells you the meaning of "current class loader":
public static Class<?> forName(String className) throws ClassNotFoundException {
Class<?> caller = Reflection.getCallerClass();
return forName0(className, true, ClassLoader.getClassLoader(caller), caller);
}
If you can get hold of a Class instance, you can always ask for the loader behind it by calling Class::getClassLoader() method. That will be your current class loader. The tricky bit, however, is to decide if the loader is bootstrap, or extension, or system class loader. The reason of the trickiness is that it is implementation specific, and you can always implement your own class loading mechanism.
The example given by #dmitrievanthony is an example of how things can become really complicated. It is a similar situation faced by JNDI, and the reason why the hack Thread.getContextClassLoader() was introduced. More about it here
Quote the most relevant piece from the article:
... By definition, a current classloader loads and defines the class to which your current method belongs. This classloader is implied when dynamic links between classes resolve at runtime, and when you use the one-argument version of Class.forName(), Class.getResource(), and similar methods. It is also used by syntactic constructs like X.class class literals ...

the 'current classloader' is the real classloader(load this class in reality) of class that refer it.
e.g.
if the classLoader of class A is ext classloader and class A refer class B C D. Then the 'current classloader' of B C D is ext classLoader. And of course the 'current classLoader' of main class is System classLoader.

Reflection getDeclaredMethods() and class that is not in classpath

I am using reflection to get all methods from a specific class.
This class has references to class that not in my class path so I get an exception:
java.lang.NoClassDefFoundError:
On this:
Method methods[] = theClass.getDeclaredMethods();
Is it possible, somehow,to "skip" everything that is not in classpath?

Class.forName() will not load a class, whether it is or isn't in the classpath. It will only return a handle to a class that is already loaded.
A class gets loaded in one of 2 main ways:
1.)The class is referenced in the import statements(java.lang.* is imported automatically so every class in java.lang package is class-loaded from the start)
2.)A class is loaded using a call from a ClassLoader, in which case all of its dependencies are resolved. and loaded as well
So if you are trying to load a class outside of the classpath, or with dependencies outside the classpath, you need to subclass ClassLoader and tell it how to load your classes and their dependencies.
See ClassLoader specification here: http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/ClassLoader.html
Also, there are ready made subclasses of ClassLoader that may do what you want such as URL ClassLoader which will let you simply point the ClassLoader instance at the path, and load any classes in that path.

Load Java classes based on a classpath in a properties file

My application uses JDBC database drivers. I load these from a jar file, db2jcc.jar in the case of DB2 which I'm currently working with. With this jar in the classpath, everything is fine, but I have a requirement to find the jar from a property in the application's config file instead - for example,
database.driver=/opt/IBM/db2/V9.5/java/db2jcc.jar
I can load the class via a URLClassLoader ok, but the problem is that I need to treat the object thus created as an explicit DB2XADataSource. For example:
URLClassLoader dbClassLoader = new URLClassLoader(new URL[]{driverJar});
xaClass = dbClassLoader.loadClass("com.ibm.db2.jcc.DB2XADataSource");
DB2XADataSource dataSource = (DB2XADataSource) xaClass.newInstance();
dataSource.setCurrentSchema(DATABASE_SCHEMA); // <- dataSource has to be a
dataSource.setDatabaseName(DATABASE_NAME); // DB2XADataSource to do this
(rearranged and renamed somewhat; I actually do the loadClass in the constructor of the class that contains this code, while the newInstance is in one of its methods.)
I guess I'm getting into a classloader tangle because the classloader that loaded my class is trying to find DB2XADataSource in order to do the cast, but the URL classloader is not above it in the tree. The trouble is, it being long after I should have stopped working for the day (here in the UK) I can't think how best to solve it in a vaguely neat and sane manner.
Ideas?
Thanks.

The simplest approach is to just use the java.beans API (or direct reflection if you must) to invoke the setter methods.
Alternatively: Your database code requires to link to the dynamically loaded code. Therefore, dynamically load your database code. How much is up to you. You might load almost everything except the "bootstrap".

Yep - the class can't load its own dependencies. You could do some ClassLoader magic, but I imagine it would get messy very quickly.
One way to reduce the amount of reflection would be to put any code that depends on DB2XADataSource into an implementation that is invoked via an interface available to the calling ClassLoader.
//in mydb2driver.jar
public class MyDb2Driver implements IDriver {
private DB2XADataSource dataSource = new DB2XADataSource();
public void init() {
dataSource.setCurrentSchema(DATABASE_SCHEMA);
}
//etc.
}
This is loaded with your driver:
database.driver=/opt/IBM/db2/V9.5/java/db2jcc.jar:/foo/mydb2driver.jar
Invoking code is in the regular classpath:
public interface IDriver {
public void init();
//etc.
}
...
URLClassLoader dbClassLoader = new URLClassLoader(new URL[]{driverJar});
xaClass = dbClassLoader.loadClass("foo.MyDb2Driver");
IDriver dataSource = (IDriver) xaClass.newInstance();
dataSource.init();