We Keep Coding

Unexpected behaviour in JVM class loading (ClassNotFoundException before the class is really needed)

I would need help trying to understand why this is happening to me:
Using Java 1.8.0_131, I have a class such as this:
public class DynamicClassLoadingAppKO {
/*
* THIS VERSION DOES NOT WORK, A ClassNotFoundException IS THROWN BEFORE EVEN EXECUTING main()
*/
// If this method received ChildClassFromLibTwo, everything would work OK!
private static void showMessage(final ParentClassFromLibOne obj) {
System.out.println(obj.message());
}
public static void main(final String[] args) throws Throwable {
try {
final ChildClassFromLibTwo obj = new ChildClassFromLibTwo();
showMessage(obj);
} catch (final Throwable ignored) {
// ignored, we just wanted to use it if it was present
}
System.out.println("This should be displayed, but no :(");
}
}
Two other classes are being used up there: ParentClassFromLibOne and ChildClassFromLibTwo. The latter extends from the former.
There are two external libraries involved:
One library is called libone and contains the ParentClassFromLibOne class. The application includes this library in the classpath both for compiling and executing.
A second library is called libtwo and contains the ChildClassFromLibTwo class. The application includes this library in the classpath for compiling, but not for executing.
As far as I understand, the Java runtime should try to load the ChildClassFromLibTwo (which is not in the classpath at runtime) at this line:
final ChildClassFromLibTwo obj = new ChildClassFromLibTwo();
Given this class is not in the classpath, a ClassNotFoundException should be thrown, and given this line is inside a try...catch (Throwable), the System.out.println line at the end should be executed anyway.
However, what I get is the ClassNotFoundException thrown when the DynamicClassLoadingAppKO itself is loaded, apparently before the main() method is executed at all, and therefore not caught by the try...catch.
What seems more strange to me is that this behaviour disappears and everything works as I would expect if I change the signature of the showMessage() method so that instead of receiving an argument of the parent class, it is directly of the child class:
/*
* THIS VERSION WORKS OK, BECAUSE showMessage RECEIVES THE CHILD CLASS AS A PARAMETER
*/
private static void showMessage(final ChildClassFromLibTwo obj) {
System.out.println(obj.message());
}
How is this possible? What am I missing in the way class loading works?
For testing convenience, I have created a GitHub repository replicating this behaviour [1].
[1] https://github.com/danielfernandez/test-dynamic-class-loading/tree/20170504

OK, the details of why this happens are explained in this Spring Boot ticket [1] which I've been very lucky to be promptly pointed to by Andy Wilkinson. That was definitely a difficult one IMO.
Apparently, what happens in this case is that when the calling class itself is loaded, the verifier kicks in and sees that the showMessage() method receives an argument of type ParentClassFromLibOne. So far so good, and this would not provoke a ClassNotFoundException at this phase even if ParentClassFromLibOne was not in the classpath at runtime.
BUT apparently the verifier also scans method code and notes that there is a call in main() to that showMessage() method. A call that does not pass as an argument a ParentClassFromLibOne, but instead an object of a different class: ChildClassFromLibTwo.
So it is in this case that the verifier does try to load ChildClassFromLibTwo in order to be able to check if it really extends from ParentClassFromLibOne.
Interestingly this wouldn't happen if ParentClassFromLibOne was an interface, because interfaces are treated as Object for assignment.
Also, this does not happen if showMessage(...) directly asks for a ChildClassFromLibTwo as an argument because in such case the verifier does not need to load the child class to check it is compatible... with itself.
Daniel, I'm voting up your answer but I will not mark it as accepted because I consider it fails at explaining the real reason why this is happening at verify time (it's not the class in the signature of the method that's causing the ClassNotFoundException).
[1] https://github.com/spring-projects/spring-boot/issues/8181

This is a bit more complicated than you think. When a class is loaded, all functions are verified. During the verify phase also all referenced classes are loaded, because they are needed to calculated the exact types that are on the stack at any given location in the bytecode.
If you want that lazy behaviour, you have to pass the -noverify option to Java, which will delay the loading of classes until the functions that reference them are executed the first time. But don't use -noverify for security reasons when you don't have full control over the classes that will be loaded into the JVM.

java.lang.NoSuchMethodError , build process is broken

I've a problem with my build and it caused a huge headache for me.
I had an old class and I was using it to fetch data from it, and I created a new class with the same methods. When I test it locally on my machine, everything works fine, but when I try to do a build, it broke because it's unstable and I got this error in the log file:
Caused by: java.lang.NoSuchMethodError: com.mashvisor.bean.Neighborhood.getTraditionalRates()Lcom/mashvisor/database/dao/views/NeighborhoodRentalRates;
at com.mashvisor.database.dao.PropertyDao.retrieve(PropertyDao.java:91)
The NeighborhoodRentalRates class is the old class, and in my code I'm sure im not using it nor trying to access it in that line, here's my code for that line:
Hibernate.initialize(property.getNeighborhood().getTraditionalRates());
and here's it's declaration
public TraditionalNeighborhoodRentalRates getTraditionalRates() {
return traditionalRates;
}
The TraditionalNeighborhoodRentalRates is the new class, and the only change here is the class name.
Could any body help?

Your code is still calling the old method, i.e. it is looking for a method with the signature:
public NeighborhoodRentalRates getTraditionalRates() { ... }
Just using the same names it not enough. To have classes with the same (method-)interface, you have to have the same names, return types and argument types in all methods.
So you need to go through your calling code and make sure the new type is expected everywhere as return type and recompile the calling code.

Android app cannot install

While developing an app in AIDE for Android I have come across this error. The app would compile successfully but wouldn't install, reporting this error:
Could not run the App directly as root. Consider disabling direct running in the settings.
WARNING: linker: app_process has text relocations. This is wasting memory and is a security risk. Please fix.
pkg: /storage/sdcard/AppProjects/MyProgram/bin/MyProgram.apk
Failure [INSTALL_FAILED_DEXOPT]
exit with 0
I researched what could cause this and mainly came across reasons like "certificate error, try resigning the package" and "setting a permission twice in the manifest" and other stuff, none of which have worked.

Your problem: Java thinks you define two methods with the same signature.
Java method signature definition: https://docs.oracle.com/javase/tutorial/java/javaOO/methods.html
method declarations have six components, in order:
1.Modifiers—such as public, private, and others you will learn about later.
2.The return type—the data type of the value returned by the method, or void if the method does not return a value.
3.The method name—the rules for field names apply to method names as well, but the convention is a little different.
4.The parameter list in parenthesis—a comma-delimited list of input parameters, preceded by their data types, enclosed by parentheses, ().
If there are no parameters, you must use empty parentheses.
An exception list—to be discussed later.
The method body, enclosed between braces—the method's code, including the declaration of local variables, goes here.
As you can see above, the specification of generic classes is NOT part of the java method signature. Therefore java detects two add-methods with the same signature.

I found where the problem resides. It was in some code which looked very much like this:
public class Builder<T extends Base> {
private final List<Def1> subDefs1 = new ArrayList<>();
private final List<Def2> subDefs2 = new ArrayList<>();
public Builder<T> add(final Collection<Def1> ds) {
subDefs1.addAll(ds);
return this;
}
public Builder<T> add(final Collection<Def2> ds) {
subDefs2.addAll(ds);
return this;
}
}
interface Base {}
final class Def1 implements Base {}
final class Def2 implements Base {}
I had these add methods, which both take a Collection of some kind. The problem must be something to do with Java's lacklustre generics and the dexing process, I guess...

Java + Libgdx: Gdx.app.log() null pointer error

I am learning the LibGDX engine in parallel to re-learning java, and have written a simple logging class that has one method with a string argument to be passed to the Gdx.app.log(). while this isn't needed I did so to practice importing and using custom methods and classes, as well as reducing the length of the line needed to send a message to the console. the method looks like so:
import com.badlogic.gdx.Gdx;
public class logging {
public static final String tag="Console";
//contains method for logging to the console during testing.
public void log(String message){
Gdx.app.log(tag, message);
}
}
Then in the class I am using it in, it is imported properly, and a public logging 'con' is created. Up to this point everything seems to work fine because when I type con. in eclipse I get the log(message) as an autocomplete option, however when it is actually called for instance in a screen, under the show() method. when the program tries to step through that point i get a java.lang.NullPointerException which is confusing the hell out of me since everything is written properly. as an example of its use:
con.log("this is a test");
is the exact usage I attempt which seems fine in eclipse before runtime. Is there some simple idea I am failing to grasp here? or a quirk to the Gdx.app.log()? Please no responses with "just use the Gdx.app.log(); where you need to write to the log" as this is not the point of the exercise for me. thank you in advance for all the help!

If you are getting a NullPointerException in this line:
con.log("this is a test");
The only thing that can be null is con. You are probably defining it, but not actually creating it.
Logging con;
and not
Logging con = new Logging();

Java or any other language: Which method/class invoked mine?

I would like to write a code internal to my method that print which method/class has invoked it.
(My assumption is that I can't change anything but my method..)
How about other programming languages?
EDIT: Thanks guys, how about JavaScript? python? C++?

This is specific to Java.
You can use Thread.currentThread().getStackTrace(). This will return an array of StackTraceElements.
The 2nd element in the array will be the calling method.
Example:
public void methodThatPrintsCaller() {
StackTraceElement elem = Thread.currentThread.getStackTrace()[2];
System.out.println(elem);
// rest of you code
}

If all you want to do is print out the stack trace and go hunting for the class, use
Thread.dumpStack();
See the API doc.

Justin has the general case down; I wanted to mention two special cases demonstrated by this snippit:
import java.util.Comparator;
public class WhoCalledMe {
public static void main(String[] args) {
((Comparator)(new SomeReifiedGeneric())).compare(null, null);
new WhoCalledMe().new SomeInnerClass().someInnerMethod();
}
public static StackTraceElement getCaller() {
//since it's a library function we use 3 instead of 2 to ignore ourself
return Thread.currentThread().getStackTrace()[3];
}
private void somePrivateMethod() {
System.out.println("somePrivateMethod() called by: " + WhoCalledMe.getCaller());
}
private class SomeInnerClass {
public void someInnerMethod() {
somePrivateMethod();
}
}
}
class SomeReifiedGeneric implements Comparator<SomeReifiedGeneric> {
public int compare(SomeReifiedGeneric o1, SomeReifiedGeneric o2) {
System.out.println("SomeRefiedGeneric.compare() called by: " + WhoCalledMe.getCaller());
return 0;
}
}
This prints:
SomeRefiedGeneric.compare() called by: SomeReifiedGeneric.compare(WhoCalledMe.java:1)
somePrivateMethod() called by: WhoCalledMe.access$0(WhoCalledMe.java:14)
Even though the first is called "directly" from main() and the second from SomeInnerClass.someInnerMethod(). These are two cases where there is a transparent call made in between the two methods.
In the first case, this is because we are calling the bridge method to a generic method, added by the compiler to ensure SomeReifiedGeneric can be used as a raw type.
In the second case, it is because we are calling a private member of WhoCalledMe from an inner class. To accomplish this, the compiler adds a synthetic method as a go-between to override the visibility problems.

the sequence of method calls is located in stack. this is how you get the stack: Get current stack trace in Java then get previous item.

Since you asked about other languages, Tcl gives you a command (info level) that lets you examine the call stack. For example, [info level -1] returns the caller of the current procedure, as well as the arguments used to call the current procedure.

In Python you use the inspect module.
Getting the function's name and file name is easy, as you see in the example below.
Getting the function itself is more work. I think you could use the __import__ function to import the caller's module. However you must somehow convert the filename to a valid module name.
import inspect
def find_caller():
caller_frame = inspect.currentframe().f_back
print "Called by function:", caller_frame.f_code.co_name
print "In file :", caller_frame.f_code.co_filename
#Alternative, probably more portable way
#print inspect.getframeinfo(caller_frame)
def foo():
find_caller()
foo()

Yes, it is possible.
Have a look at Thread.getStackTrace()

In Python, you should use the traceback or inspect modules. These will modules will shield you from the implementation details of the interpreter, which can differ even today (e.g. IronPython, Jython) and may change even more in the future. The way these modules do it under the standard Python interpreter today, however, is with sys._getframe(). In particular, sys._getframe(1).f_code.co_name provides the information you want.