In general, how can I write programs to accommodate modding or plugins? Is every method wrapped with other behaviors? All my searching has led to are resources for writing plugins and mods themselves; I can't find anything on writing the systems. With regards to java, how do I expose internal portions of the logic to other systems without using reflection?
One way to do this is by creating your own ClassLoader that can load classes from a specified location that is not on the system classpath. (If it is on the system classpath, the system ClassLoader will find the classes first and you won't be able to unload them.) Creating instances requires a bit of reflection, but once they're created you can treat them just like any other instances.
This works because although only the custom ClassLoader knows about the actual class of those instances, it gets its definition of their superclass (your plugin class or interface) from the system ClassLoader. Therefore, other classes loaded by the system ClassLoader (i.e., the rest of your program) can reference those dynamically created instances by their superclass.
I've used Brian's Clapper's utility library to find classes that extend my plugin class.
Related
Class definitions are stored in the Method Area, as the Java Virtual Machine Specification says (The Java® Virtual Machine Specification Java SE 7 Edition):
The method area is created on virtual machine start-up. Although the
method area is logically part of the heap, simple implementations may
choose not to either garbage collect or compact it.
As we know, some bytecode tools like ASM, cglib, javassist, Hibernate and Spring frameworks are using them. For a common class file, JVM loads and parses and initializes and finally uses it, I am confused about how does JVM deal with the classes generated by bytecode tools dynamically. My questions are:
If JVM loads, parses and initializes the dynamic classes as the common class file?
Are they stored in the Method Area as well?
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError?
All class are loaded at runtime, possibly compiled to native code. As such there is nothing special about class generated after the program has started.
If JVM loads, parses and initializes the dynamic classes as the common class file?
It loads the same way as class which existed when the program started.
Are they stored in the Method Area as well?
They are stored the same way in fact it is hard to tell if a class is dynamic or not.
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError?
The JVM can unload classes when the ClassLoader they are in is unloaded. This is true whether the classes are dynamic or not.
how could the JVM know to treat dynamic classes any differently than 'normal' classes?
There is one example of dynamic classes which are special. These are the lambda classes which are generated at runtime. What makes them different is they are not bound to a class loader, they don't even have a normal class name. They get unloaded when all the instances of that class are unused.
From InnerClassLambdaMetafactory
UNSAFE.defineAnonymousClass(targetClass, classBytes, null);
The class has no class name attached to it (nor a ClassLoader)
There is no difference between .class data on the file system, inside a jar file or dynamically created by tools like ASM.
In fact when the class loader loads the class, it's just a byte[] how that byte array is populated with .class data is up to the class loader to get it from the filesystem, inside a jar or zip file, from a URL, dynamically generated etc.
If JVM loads, parses and initializes the dynamic classes as the common class file?
As I said it is exactly the same. The JVM doesn't know the difference.
Are they stored in the Method Area as well?
Again, JVM doesn't know the difference so I guess.
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError
The same way as non-dynamic classes. by unloading although I don't know how often this happens
Web applications for Tomcat are wrapped into a .war file and thrown into Tomcat. The application can use classes inside the war file and contained jar files. This separates the runtime-classes of Tomcat from the application classes.
When using storm (see storm-project.net), there is no similar segregation. Rather the recommended way requires to create a "fat jar", denoting a jar containing all the necessary class files after unwrapping them from their own individual jars. For simple situations this works, but the resulting fat jar must have all the META-INF/* files from all merged jars correctly merged, which does not work semi-automatically.
I would not be shy to write my own class loader which simulates something like a war-file. Except I have no clue how to intercept the default class loader that loads all the classes I deploy.
My theory is that one of the classes of my applications is loaded first by the default class loader. Presumably in its static-section I would cast the magic class loader spell such that all dependent classes will then be loaded by my own class loader, which would now how to get the necessary classes from whatever I deem suitable.
Any hints towards the general problems described as well as the specific magic needed are appreciated.
Intercepting the classloader: The default classloader is the one which loaded the class that is currently executing. So if you use a separate classloader to bring in the WAR's entry point, every class reference from it or its (run-time/creation) descendents will default to going through that classloader. That classloader can then decide what it should refer back to a higher-level classloader and what it should reload using its own resolution rules -- the simple rule "only ask the parent for things I don't have available in this plug-in", also known as "parent last", is often sufficient.
Is there anyway to find all the class names which are extending a particular class by using classloader information?
More Detailed Explanation
I have a jar file which has 3 classes X (extends A), Y (extends A) and Z (extends B). I have put this jar file into the classpath and start the JVM (calling main method of some class). Now from this main method is there any way to find the classes which are sub classes of A? i.e. classes X and Y as they are extending the class A.
Unfortunately there's no easy way to do it. The JVM has no information about a particular class until it gets loaded by a classloader. But to have the classloader actually load the class, you must know the name of the class. Chicken and egg problem.
However, there is some other, but a bit more complicated way.
To determine which .jar files are on the classpath when running your application, you can call the System.getProperty("java.class.path") then split the classpath entries along the File.separatorChar characters. Classpath entries may be numerous, consider filtering them somehow (limiting only to a specific directory, leaving out standard entries like rt.jar, etc.)
If you know which .jar file contains your classes of interest, you can open it with an instance of the class JarFile. Then you can iterate over all entries (eg. .class files) and from their internal path in the JAR file you are able to construct their fully qualified class name.
With the classname in your hand, you can call Class.forName() to load the class, and the you can use reflection to get the superclasses of each class to find out whether it extends your particular class or not.
Note that this method is quite resource consuming, you make the JVM to actually load all the classes from a .jar file (or multiple .jar files), even if you won't use them later. This wastes a fair amount af memory (PermGen space).
There are some low-level libraries that manipulate Java bytecode and class files (see BCEL or ASM). Using them you will be able to determine the superclass for a class without actually (class)loading it, thus this way is more faster and uses less memory.
Your question resembles the way JavaEE application servers work when deploying a web application. To find out which classes to load and initialize as for example HTTP servlets they must examine all classes in the web archive looking for a specific Java annotation or superclass. However they are at least know which .war file to scan. Apache Tomcat for example uses BCEL instead of Java classloading mechanism.
If you're designing some sort of dynamic classloading mechanism for your application, then consider other design options to narrow down the numbers of classes your loader must scan to find the proper class to load: Telling the exact name of the .jar file instead of putting it on the classpath, using some meta information in the JAR (you can use the aforementioned JarFile to read entries from the META-INF/manifest.mf file) to specify which classes to look for, etc.
just fore the sake of completeness
http://code.google.com/p/reflections/
others where mentioned in the linked post
A problem: My application uses two libraries which use incompatible versions of a third library. Does someone know any method for classes isolation?
I have heard about class loaders, but I do not understand, how they could help - if we will load one version of class, we cannot load another - class is already loaded?
I also thinking about Spring, but do not know if it provides such isolation.
ClassLoaders are basically the elements that gives meaning to classes in the JVM. They form a hierarchy for wich the root lies in the JVM and loads java classes. The ApplicationClassLoader is the first ClassLoader you have to consider, as it loads all the classes of your application.
When a class is loaded, all its references to other classes are resolved and theses classes are loaded. The JVM by default provides a system where classloaders ask their parent first to see if they have already loaded a class. If not, they search in their classpath
Two classes can be isolated if they live in 2 different classloader, and not in the app classloader. It's not difficult to do. You only have to create a classloader (like URLClassLoader) while specifying its parent and its classpath(the place where the bytecode is)
then, you tell him to load a class. It will ask its parent, and if the class is not loaded yet, it will search its classpath and load it. If you create another classloader attached to the same parent, the classes loaded by the first will never be seen by the seconds as they are siblings. And the second can loads a class with the same name without any problem
That's quite a good isolation
App Servers use another form of delegation to have a frank isolation between applications. they redefine a classloader extending, for example, URLClassLoader and reverse the delegation process by starting to search for classes in their classpath first, then ask to the parent
if we will load one version of class, we cannot load another - class
is already loaded?
Not true. The class loader is considered part of the class's identity. If it's loaded by a different classloader, it's considered a different class.
If at least one of the libraries is open source, and the library they depend on is open source, no need for messing about with classloaders. Just bulk rename all the packages in one of the versions of the library being depended on, making sure to also change all code which refers to those packages. Hey presto - no name clashes. This is what Sun actually did for the JDK, when they included Xerces and Xalan behind the scenes.
When a client says "Code should not have custom classloaders" in Java, what does that exactly mean? What can't I do?
A class loader is an object in Java responsible for finding binary representations of Java classes and loading them into the JVM. All JVMs begin with a boot class loader responsible for loading the user's initial class, along with some of the built-in types like Class and SecurityManager, but users can provide their own class loaders to find classes from other sources. For example, a custom class loader could generate its own classes by composing its own bytecode, or it could find classes from a networked source.
To comply with what your client is asking, you should not define your own class loader and should rely on the boot class loader to find all your classes. This is almost universally what's done in simple Java programs because the use cases for custom boot loaders are usually fairly complex and nuanced. You shouldn't need to worry about this restriction unless you specifically want to change the way that that JVM finds and loads classes.
Custom class loaders are usually used to dynamically generate code or to enhance existing classes.
For example some ORM implementations (JDO) use this to create code that handles translating Java objects to database tables. Other use is in transparent-clustering solutions (like Terracota) where objects are enhanced so that they automatically replicate themselves across the cluster.
This basically prevents you to dynamically generate and inject code into an existing application.
A class loader is an object that is responsible for loading classes. Whenever you instantiate a class using new the runtime system attempts load the class using one or more instances of the ClassLoader abstract class. You can define custom class loaders to load classes from the network, databases, other processes, or any conceivable data source.
So, if your client does not want you to use custom class loaders then be sure to never write a class that extends ClassLoader or any of it's derivatives. Please see the ClassLoader java API docs for more details.
A custom classloader would let you load classes from unconventional sources (from anywhere you can imagine, including out of nowhere, i.e. created on-the-fly). Since your client is saying giving that message, then classes can be loaded only from standard sources (e.g. filesystem, jar files, etc).