We Keep Coding

Unexpected behaviour in JVM class loading (ClassNotFoundException before the class is really needed)

I would need help trying to understand why this is happening to me:
Using Java 1.8.0_131, I have a class such as this:
public class DynamicClassLoadingAppKO {
/*
* THIS VERSION DOES NOT WORK, A ClassNotFoundException IS THROWN BEFORE EVEN EXECUTING main()
*/
// If this method received ChildClassFromLibTwo, everything would work OK!
private static void showMessage(final ParentClassFromLibOne obj) {
System.out.println(obj.message());
}
public static void main(final String[] args) throws Throwable {
try {
final ChildClassFromLibTwo obj = new ChildClassFromLibTwo();
showMessage(obj);
} catch (final Throwable ignored) {
// ignored, we just wanted to use it if it was present
}
System.out.println("This should be displayed, but no :(");
}
}
Two other classes are being used up there: ParentClassFromLibOne and ChildClassFromLibTwo. The latter extends from the former.
There are two external libraries involved:
One library is called libone and contains the ParentClassFromLibOne class. The application includes this library in the classpath both for compiling and executing.
A second library is called libtwo and contains the ChildClassFromLibTwo class. The application includes this library in the classpath for compiling, but not for executing.
As far as I understand, the Java runtime should try to load the ChildClassFromLibTwo (which is not in the classpath at runtime) at this line:
final ChildClassFromLibTwo obj = new ChildClassFromLibTwo();
Given this class is not in the classpath, a ClassNotFoundException should be thrown, and given this line is inside a try...catch (Throwable), the System.out.println line at the end should be executed anyway.
However, what I get is the ClassNotFoundException thrown when the DynamicClassLoadingAppKO itself is loaded, apparently before the main() method is executed at all, and therefore not caught by the try...catch.
What seems more strange to me is that this behaviour disappears and everything works as I would expect if I change the signature of the showMessage() method so that instead of receiving an argument of the parent class, it is directly of the child class:
/*
* THIS VERSION WORKS OK, BECAUSE showMessage RECEIVES THE CHILD CLASS AS A PARAMETER
*/
private static void showMessage(final ChildClassFromLibTwo obj) {
System.out.println(obj.message());
}
How is this possible? What am I missing in the way class loading works?
For testing convenience, I have created a GitHub repository replicating this behaviour [1].
[1] https://github.com/danielfernandez/test-dynamic-class-loading/tree/20170504

OK, the details of why this happens are explained in this Spring Boot ticket [1] which I've been very lucky to be promptly pointed to by Andy Wilkinson. That was definitely a difficult one IMO.
Apparently, what happens in this case is that when the calling class itself is loaded, the verifier kicks in and sees that the showMessage() method receives an argument of type ParentClassFromLibOne. So far so good, and this would not provoke a ClassNotFoundException at this phase even if ParentClassFromLibOne was not in the classpath at runtime.
BUT apparently the verifier also scans method code and notes that there is a call in main() to that showMessage() method. A call that does not pass as an argument a ParentClassFromLibOne, but instead an object of a different class: ChildClassFromLibTwo.
So it is in this case that the verifier does try to load ChildClassFromLibTwo in order to be able to check if it really extends from ParentClassFromLibOne.
Interestingly this wouldn't happen if ParentClassFromLibOne was an interface, because interfaces are treated as Object for assignment.
Also, this does not happen if showMessage(...) directly asks for a ChildClassFromLibTwo as an argument because in such case the verifier does not need to load the child class to check it is compatible... with itself.
Daniel, I'm voting up your answer but I will not mark it as accepted because I consider it fails at explaining the real reason why this is happening at verify time (it's not the class in the signature of the method that's causing the ClassNotFoundException).
[1] https://github.com/spring-projects/spring-boot/issues/8181

This is a bit more complicated than you think. When a class is loaded, all functions are verified. During the verify phase also all referenced classes are loaded, because they are needed to calculated the exact types that are on the stack at any given location in the bytecode.
If you want that lazy behaviour, you have to pass the -noverify option to Java, which will delay the loading of classes until the functions that reference them are executed the first time. But don't use -noverify for security reasons when you don't have full control over the classes that will be loaded into the JVM.

Issue with ASM getMergedType and getCommonSuperClass

I use ASM to update the class stack map, but when asm getMergedType, the following exception occurs:
java.lang.RuntimeException:
java.io.IOException: Resource not found for IntefaceImplA.
If without asm modify the class method, it does work fine.
I have defined two interfaces A and B: IntefaceImplA and
IntefaceImplB.
My environment source code:
IntefaceA.java
public interface IntefaceA {
void inteface();
}
IntefaceImplA.java
public class IntefaceImplA implements IntefaceA {
#Override
public void inteface() {
}
}
IntefaceImplB.java
public class IntefaceImplB implements IntefaceA {
#Override
public void inteface() {
}
}
Test.java
public class Test {
public IntefaceA getImpl(boolean b) {
IntefaceA a = b ? new IntefaceImplA() : new IntefaceImplB();
return a;
}
}
Main.java
public class Main {
public static void main(String args[]) {
....
if (a instance of Test) {
..
...
}
}
}
After I compiled a runner jar, and delete the IntefaceImplA.class and IntefaceA.class manually from the jar. why i wanna to delete those classes files, since the spring always like to do this stuff.
the runner jar can be run normal without ASM, but use Asm will occur exception. since the asm wanna to getMergedType for IntefaceImplA and IntefaceImplB, but IntefaceImplA was deleted by me.
After investigate the ASM ClassWriter source code i found below code:
protected String getCommonSuperClass(String type1, String type2)
{
ClassLoader classLoader = this.getClass().getClassLoader();
Class c;
Class d;
try {
c = Class.forName(type1.replace('/', '.'), false, classLoader);
d = Class.forName(type2.replace('/', '.'), false, classLoader);
} catch (Exception var7) {
throw new RuntimeException(var7.toString());
}
if(c.isAssignableFrom(d)) {
return type1;
} else if(d.isAssignableFrom(c)) {
return type2;
} else if(!c.isInterface() && !d.isInterface()) {
do {
c = c.getSuperclass();
} while(!c.isAssignableFrom(d));
return c.getName().replace('.', '/');
} else {
return "java/lang/Object";
}
}
Actually, I deleted the related class file, the classloader cannot find the class. but without asm the Program does work normal.
Should I enhance the override to the getCommonSuperClass method, if occur exception then return java/lang/Object for it? that's funny

Generally, overriding getCommonSuperClass to use a different strategy, e.g. without loading the class, is a valid use case. As it’s documentation states:
The default implementation of this method loads the two given classes and uses the java.lang.Class methods to find the common super class. It can be overridden to compute this common super type in other ways, in particular without actually loading any class, or to take into account the class that is currently being generated by this ClassWriter, which can of course not be loaded since it is under construction.
Besides the possibility that either or both arguments are classes you are currently constructing (or changing substantially), it might be the case that the context of the code transforming tool is not the context in which the classes will eventually run, so they don’t have to be accessible via Class.forName in that context. Since Class.forName uses the caller’s context, which is ASM’s ClassWriter, it is even possible that ASM can’t access the class despite it is available in the context of the code using ASM (if different class loaders are involved).
Another valid scenario is to have a more efficient way to resolve the request by using already available meta information without actually loading the class.
But, of course, it is not a valid resolution to just return "java/lang/Object". While this is indeed a common super type of every argument, it isn’t necessarily the right type for the code. To stay with your example,
public IntefaceA getImpl(boolean b) {
IntefaceA a = b ? new IntefaceImplA() : new IntefaceImplB();
return a;
}
the common super type of IntefaceImplA and IntefaceImplB is not only required to verify the validity of assigning either type to it, it is also the result type of the conditional expression, which must be assignable to the return type of the method. If you use java/lang/Object as common super type, a verifier will reject the code as it can’t be assignable to IntefaceA.
The original stackmap, very likely reporting IntefaceA as common super, will be accepted by the verifier as that type is identical to the method’s return type, so it can be considered assignable, even without loading the type. The test, whether either, IntefaceImplA and IntefaceImplB, is assignable to that specified common type, might be postponed to the point where these types are actually loaded and since you said, you deleted IntefaceA, this can never happen.
A method whose declared return type is absent, can’t work at all. The only explanation of your observation that “without asm the program does work normal”, is, that this method was never invoked at all during your test. You most probably created a time bomb in your software by deleting classes in use.
It’s not clear why you did this. Your explanation “since the spring always like to do this stuff” is far away from being comprehensible.
But you can use the overriding approach to get the same behavior as with the unmodified code. It just doesn’t work by return java/lang/Object. You could use
#Override
protected String getCommonSuperClass(String type1, String type2) {
if(type1.matches("IntefaceImpl[AB]") && type2.matches("IntefaceImpl[AB]"))
return "IntefaceA";
return super.getCommonSuperClass(type1, type2);
}
Of course, if you deleted more class files, you have to add more special cases.
An entirely different approach is not to use the COMPUTE_FRAMES option. This option implies that ASM will recompute all stack map frames from scratch, which is great for the lazy programmer, but implies a lot of unnecessary work if you are just doing little code transformations on an existing class and, of course, creates the requirement to have a working getCommonSuperClass method.
Without that option, the ClassWriter will just reproduce the frames the ClassReader reports, so all unchanged methods will also have unchanged stack maps. You will have to care about the methods whose code you change, but for a lot of typical code transformation tasks, you can still keep the original frames. E.g. if you just redirect method calls to signature-compatible targets or inject logging statements which leave the stack in the same state it was before them, you can still keep the original frames, which happens automatically. Note the existence of the ClassWriter(ClassReader,int) constructor, which allows an even more efficient transfer of the methods you don’t change.
Only if you change the branch structure or insert code with branches, you have to care for frames. But even then, it’s often worth learning how to do this, as the automatic calculation is quiet expensive while you usually have the necessary information already when doing a code transformation.

Type of field 'myField' is concrete class 'MyClass'

I am preparing my android application for production so I have enabled most lint warnings. One particular warning is wide spread, but I don't understand it.
Here is the warning:
Type of field 'loginF' is concrete class 'LoginFragment'
And here is the description:
Reports any instance fields whose type is declared to be a concrete class, rather than an interface. Such declarations may represent a failure of abstraction, and may make testing more difficult. Declarations whose classes come from system or third-party libraries will not be reported by this inspection.
Any elaboration would be appreciated.
Note: The IDE I am using is 'Android Studio'.

It's complaining because you're coupling this class (which contains the loginF field) with LoginFragment; if you change LoginFragment, you may have to change this class.
Alternatively, look at the methods you're using on LoginFragment in this class. Do they collectively form some behaviour that you can specify in an interface?
e.g. if the only method you call on loginF is initiateLogin(), then perhaps you can have:
interface Login {
void initiateLogin();
}
class LoginFragment extends Fragment implements Login {
#Override
public void initiateLogin() {
// do login
}
}
such that instead of declaring LoginFragment loginF = new LoginFragment();, you use Login login = new LoginFragment().
Then you can change LoginFragment so long as it keeps initiateLogin() method, or even swap out new LoginFragment() for any other implementation of Login, which might be useful for mocking in tests.
A very contrived example. I would suggest using the default lint config, not this super strict one.

Identifier expected error when calling a method in java

I have this rather simple code written in java. This is actually from a DAQ framework, called Kmax
import kmax.ext.*;
public class Runtime implements KmaxRuntime {
KmaxToolsheet tlsh; // Store a reference to the toolsheet environment
KmaxHist hist1D;
KmaxWidget checkBoxWidget;
public void init(KmaxToolsheet toolsheet) {
tlsh = toolsheet; // Save this reference for use in the toolsheet
hist1D = tlsh.getKmaxHist("HIST1D");
checkBoxWidget = tlsh.getKmaxWidget("CHECK_BOX_CALIB_METH");
tlsh.getKmaxWidget("CHECK_BOX_CALIB_METH").setProperty("VALUE", "1");
}
public void CalibInit(KmaxWidget widget, KmaxHist histo){
histo.setUseXAxisCalibration(stringToBool(widget.getProperty("VALUE")));
histo.update();
}
CalibInit(checkboxWidget,hist1D);
public void GO(KmaxToolsheet toolsheet){}
public void SRQ(KmaxDevice device) {}
public void HALT(KmaxToolsheet toolsheet) {}
} // End of the Runtime object
Note that there I have created an object named CHECK_BOX_CALIB_METH. When I compile this code I get those errors messages
compiler msg>error: invalid method declaration; return type required
compiler msg> CalibInit(checkboxWidget,hist1D);
compiler msg> ^
compiler msg>error: <identifier> expected
compiler msg>CalibInit(checkboxWidget,hist1D);
compiler msg> ^
compiler msg>error: <identifier> expected
compiler msg>CalibInit(checkboxWidget,hist1D);
compiler msg> ^
Note that if I remove the CalibInit method and replace it with
public void CHECK_BOX_CALIB_METH(KmaxWidget widget) {
hist1D.setUseXAxisCalibration(stringToBool(widget.getProperty("VALUE")));
hist1D.update();
}
I get no compile error. The keypoint is that the method's name is the same as the object's name. The reason I created CalibInit() is to avoid having each method for every object of the same type, with the same functionality. Is there a way around it?
How to avoid those errors?

Only variables can declare out side of methods. You can call methods only in methods and constructor (avoiding static context here).
CalibInit(checkboxWidget,hist1D);
Please move that line to any method or constructor , if needed. More specifically call where you need it.
In short: CalibInit(checkboxWidget,hist1D); is orphan now. make it belong to something.

The code
CalibInit(checkboxWidget,hist1D);
that is on a line of its own is not inside any of your methods. The compiler assumes that this is a new method declaration which is probably not what you want.
Side note:
It is not recommended to have methods starting with a upper case character: "Methods should be verbs, in mixed case with the first letter lowercase, with the first letter of each internal word capitalized." from Code Conventions for the Java Programming Language

You can't call
CalibInit(checkboxWidget,hist1D);
directly in the class like you're doing. This instruction should be inside a constructor if your goal is to call it when an instance of Runtime is constructed.
BTW: methods start with a lowercase letter by convention in Java, and you shouldn't call your class Runtime: it will confuse people because a standard Runtime class already existsin the standard libraries.

You are calling CalibInit(checkboxWidget,hist1D) method directly in the class not in any method. Java doesn't support this.

Java or any other language: Which method/class invoked mine?

I would like to write a code internal to my method that print which method/class has invoked it.
(My assumption is that I can't change anything but my method..)
How about other programming languages?
EDIT: Thanks guys, how about JavaScript? python? C++?

This is specific to Java.
You can use Thread.currentThread().getStackTrace(). This will return an array of StackTraceElements.
The 2nd element in the array will be the calling method.
Example:
public void methodThatPrintsCaller() {
StackTraceElement elem = Thread.currentThread.getStackTrace()[2];
System.out.println(elem);
// rest of you code
}

If all you want to do is print out the stack trace and go hunting for the class, use
Thread.dumpStack();
See the API doc.

Justin has the general case down; I wanted to mention two special cases demonstrated by this snippit:
import java.util.Comparator;
public class WhoCalledMe {
public static void main(String[] args) {
((Comparator)(new SomeReifiedGeneric())).compare(null, null);
new WhoCalledMe().new SomeInnerClass().someInnerMethod();
}
public static StackTraceElement getCaller() {
//since it's a library function we use 3 instead of 2 to ignore ourself
return Thread.currentThread().getStackTrace()[3];
}
private void somePrivateMethod() {
System.out.println("somePrivateMethod() called by: " + WhoCalledMe.getCaller());
}
private class SomeInnerClass {
public void someInnerMethod() {
somePrivateMethod();
}
}
}
class SomeReifiedGeneric implements Comparator<SomeReifiedGeneric> {
public int compare(SomeReifiedGeneric o1, SomeReifiedGeneric o2) {
System.out.println("SomeRefiedGeneric.compare() called by: " + WhoCalledMe.getCaller());
return 0;
}
}
This prints:
SomeRefiedGeneric.compare() called by: SomeReifiedGeneric.compare(WhoCalledMe.java:1)
somePrivateMethod() called by: WhoCalledMe.access$0(WhoCalledMe.java:14)
Even though the first is called "directly" from main() and the second from SomeInnerClass.someInnerMethod(). These are two cases where there is a transparent call made in between the two methods.
In the first case, this is because we are calling the bridge method to a generic method, added by the compiler to ensure SomeReifiedGeneric can be used as a raw type.
In the second case, it is because we are calling a private member of WhoCalledMe from an inner class. To accomplish this, the compiler adds a synthetic method as a go-between to override the visibility problems.

the sequence of method calls is located in stack. this is how you get the stack: Get current stack trace in Java then get previous item.

Since you asked about other languages, Tcl gives you a command (info level) that lets you examine the call stack. For example, [info level -1] returns the caller of the current procedure, as well as the arguments used to call the current procedure.

In Python you use the inspect module.
Getting the function's name and file name is easy, as you see in the example below.
Getting the function itself is more work. I think you could use the __import__ function to import the caller's module. However you must somehow convert the filename to a valid module name.
import inspect
def find_caller():
caller_frame = inspect.currentframe().f_back
print "Called by function:", caller_frame.f_code.co_name
print "In file :", caller_frame.f_code.co_filename
#Alternative, probably more portable way
#print inspect.getframeinfo(caller_frame)
def foo():
find_caller()
foo()

Yes, it is possible.
Have a look at Thread.getStackTrace()

In Python, you should use the traceback or inspect modules. These will modules will shield you from the implementation details of the interpreter, which can differ even today (e.g. IronPython, Jython) and may change even more in the future. The way these modules do it under the standard Python interpreter today, however, is with sys._getframe(). In particular, sys._getframe(1).f_code.co_name provides the information you want.