We Keep Coding

Java add external jar to classpath at runtime using Module api

In java 16+ its no longer possible, without modyfing program startup flags, to add external jar to classpath with invoking ClassLoader#addUrl method with reflection.
With release of Java 9 a new Module API (java.lang.module) has been added that, from my raw understanding, should do all this heavy lifting for us.
public void addPath(Path dir) {
ModuleFinder pluginsFinder = ModuleFinder.of(dir);
List<String> plugins = pluginsFinder
.findAll()
.stream()
.map(ModuleReference::descriptor)
.map(ModuleDescriptor::name)
.collect(Collectors.toList());
Configuration pluginsConfiguration = ModuleLayer
.boot()
.configuration()
.resolve(pluginsFinder, ModuleFinder.of(), plugins);
ModuleLayer layer = ModuleLayer
.boot()
.defineModulesWithOneLoader(pluginsConfiguration, getClass().getClassLoader());
}
And now testing this code.
For the sake of example I`ll be using graal binaries during this test.
Class c = layer.findLoader("org.graalvm.sdk").loadClass("org.graalvm.polyglot.Engine");
// c is not null and contains correct reference to the graal`s Engine class
Class.forName("org.graalvm.polyglot.Engine");
//Throws ClassNotFoundException
Why does Class.forName throws Exception, but the first test does not?
And my second question can i use the module api to get list of all classes that can be loaded by the layer loader? For now im testing this on graal, but later ill need to scan all loaded jars and search for all classes having specific annotation.

Java: calling method from class in different ClassLoaders

I'm writting a web application that will run on Tomcat8, that should be able to update while it's still running.
In order to do that, it will create a new ClassLoader and load the whole API again on top of that, every time a given "reload" button is pressed.
// get the urls from the current loader
URLClassLoader loader = (URLClassLoader) Thread.currentThread().getContextClassLoader();
urls.addAll(Arrays.asList(loader.getURLs()));
// get the urls from the tomcat loader
loader = (URLClassLoader) loader.getParent();
urls.addAll(Arrays.asList(loader.getURLs()));
URL urlArray[] = new URL[urls.size()];
urlArray = urls.toArray(urlArray);
// my loader
loader = new URLClassLoader(urlArray, loader.getParent());
// this will throw ClassCastException
// because the newInstance will not return the System object
// that this loader knows
System newSystem = (System) loader.loadClass(System.class.getCanonicalName()).newInstance();
But! The problem begins when I need to call a shutdown method of the system that's about do die.
If I try to store the "system" in a variable, to be able to call shutdown later, I'll get a ClassCastException because, for Java, the System class I've loaded from that other ClassLoader is not the same thing as the System class Tomcat knows about.
How could I call the System.shutdown() I need from the servlet context?
Or is there a very different approach to handle this kind of situation?

The issue seems to be that you have that class in multiple class loaders then - you should not have this class to load from your main class loader as then you would not be able to actually reload that code.
Load class by raw name like that:
System newSystem = (System) Class.forName("my.system.System", true, myClassLoader).newInstance();
newSystem.shutdown();
Or you can use reflections to call method too:
Class<?> systemClass = Class.forName("my.system.System", true, myClassLoader);
Method shutdown = systemClass.getMethod("shutdown");
Object newSystem = systemClass.newInstance();
shutdown.invoke(newSystem);
Or you could use java services, and have interface in your main class loader, and implementation only in that dynamic one you want to be able to reload: https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html

Best approach to dynamically load modules (classes) in Java

I'm currently writing an application that requires to operate on different type of devices. My approach would be to make a "modular" application that can dynamically load different classes according to the device they need to operate on.
To make the application easily extensible, my goal is to assign a specific path to the additional modules (either .jar or .class files) leaving the core program as it is. This would be crucial when having different customers requiring different modules (without having to compile a different application for each of them).
These modules would implement a common interface, while the "core" application can use these methods defined on the interface and let the single implementations do the work. What's the best way to load them on demand? I was considering the use of URLClassLoader but i don't know if this approach is up-to-date according to new patterns and Java trends, as I would like to avoid a poorly designed application and deprecated techniques. What's an alternative best approach to make a modular and easily extensible application with JDK 9 (that can be extended just by adding module files to a folder) ?

Additionnaly to the ServicerLoader usage given by #SeverityOne, you can use the module-info.java to declare the different instanciation of the interface, using "uses"/"provides" keywords.
Then you use a module path instead of a classpath, it loads all the directory containing your modules, don't need to create a specific classLoader
The serviceLoader usage:
public static void main(String[] args) {
ServiceLoader<IGreeting> sl = ServiceLoader.load(IGreeting.class);
IGreeting greeting = sl.findFirst().orElseThrow(NullPointerException::new);
System.out.println( greeting.regular("world"));
}
In the users project:
module pl.tfij.java9modules.app {
exports pl.tfij.java9modules.app;
uses pl.tfij.java9modules.app.IGreeting;
}
In the provider project:
module pl.tfij.java9modules.greetings {
requires pl.tfij.java9modules.app;
provides pl.tfij.java9modules.app.IGreeting
with pl.tfij.java9modules.greetings.Greeting;
}
And finally the CLI usage
java --module-path mods --module pl.tfij.java9modules.app
Here is an example; Github example (Thanks for "tfij/" repository initial exemple)
Edit, I realized the repository already provides decoupling examples:
https://github.com/tfij/Java-9-modules---reducing-coupling-of-modules

It sounds like you might want to use the ServicerLoader interface, which has been available since Java 6. However, bear in mind that, if you want to use Spring dependency injection, this is probably not what you want.

There are two scenarios.
Implementation jar's are on classpath
In this scenario you can simply use ServiceLoader API (refer to #pdem answer)
Implementation jar's not on classpath
Lets Assume BankController is your interface and CoreController is your implementation.
If you want to load its implementation dynamically from dynamic path,c create a new module layer and load class.
Refer to the following piece of code:
private final BankController loadController(final BankConfig config) {
System.out.println("Loading bank with config : " + JSON.toJson(config));
try {
//Curent ModuleLayer is usually boot layer. but it can be different if you are using multiple layers
ModuleLayer currentModuleLayer = this.getClass().getModule().getLayer(); //ModuleLayer.boot();
final Set<Path> modulePathSet = Set.of(new File("path of implementation").toPath());
//ModuleFinder to find modules
final ModuleFinder moduleFinder = ModuleFinder.of(modulePathSet.toArray(new Path[0]));
//I really dont know why does it requires empty finder.
final ModuleFinder emptyFinder = ModuleFinder.of(new Path[0]);
//ModuleNames to be loaded
final Set<String> moduleNames = moduleFinder.findAll().stream().map(moduleRef -> moduleRef.descriptor().name()).collect(Collectors.toSet());
// Unless you want to use URLClassloader for tomcat like situation, use Current Class Loader
final ClassLoader loader = this.getClass().getClassLoader();
//Derive new configuration from current module layer configuration
final Configuration configuration = currentModuleLayer.configuration().resolveAndBind(moduleFinder, emptyFinder, moduleNames);
//New Module layer derived from current modulee layer
final ModuleLayer moduleLayer = currentModuleLayer.defineModulesWithOneLoader(configuration, loader);
//find module and load class Load class
final Class<?> controllerClass = moduleLayer.findModule("org.util.npci.coreconnect").get().getClassLoader().loadClass("org.util.npci.coreconnect.CoreController");
//create new instance of Implementation, in this case org.util.npci.coreconnect.CoreController implements org.util.npci.api.BankController
final BankController bankController = (BankController) controllerClass.getConstructors()[0].newInstance(config);
return bankController;
} catch (Exception e) {BootLogger.info(e);}
return null;
}
Reference : https://docs.oracle.com/javase/9/docs/api/java/lang/module/Configuration.html

URLClassLoader finding X but not Y in same folder

General idea: I'm writing on a loader for java that allows dynamically reloading classes to allow for changing the implementation, without restarting the entire program to keep the main application running and minimize downtimes. Every external piece of code is grouped by "modules", each module has a main class with a "onEnable, postEnable, onDisable" entry/exit point and can consist of any amount of classes. To load a module, the class containing the entry point is specified, then loaded. I'll reference them as "modules" and "additional classes" in the following, "module" being the class containing the above mentioned functions by implementing the "public interface Module", "additional classes" refer to everything the module would use on runtime but isn't a Module by itself (e.g. we have a Module called "Car implements Module", and that module requires a class "Engine" to function -> "Car" is the module, "Engine" is an additional class")
Code of what I'm doing to load a module initially (name is a String containing the full classname including path, example given later):
Class<?> clazz = mainLoader.loadClass(name);
Module module = (Module) clazz.newInstance();
addLoadedModule(module);
enableLoadedModule(module);
And here's how I reload the module when it's already existing, so that I can override the implementation. "m" is an instance of the current implementation of the Module that is supposed to be reloaded.
boolean differs = false;
Class<?> newClass = null;
try (URLClassLoader cl = new URLClassLoader(urls, mainLoader.getParent()))
{
// Try to load the class and check if it differs from the already known one
newClass = cl.loadClass(m.getClass().getName());
differs = m.getClass() != newClass;
}
catch (IOException | ClassNotFoundException e)
{
// Class couldn't be found, abort.
e.printStackTrace();
return;
}
if (!differs)
{
// New class == old class -> no need to reload it
return;
}
Module module = null;
try
{
// Try to instantiate the class
module = (Module) newClass.newInstance();
}
catch (InstantiationException | IllegalAccessException e)
{
// Can't instantiate, abort
e.printStackTrace();
return;
}
// Check versions, only reload if the new implementation's version differs from the current one. Version is a custom annotation, don't worry about that; the version check works fine
Version oldVersion = m.getClass().getAnnotation(Version.class);
Version newVersion = module.getClass().getAnnotation(Version.class);
if (oldVersion.equals(newVersion))
{
return;
}
// And if everything went well, disable and remove the old module from the list, then add and enable the new module.
disableModule(m);
modules.remove(m);
modules.put(module, false);
enableLoadedModule(module);
This is the mainLoader, urls is an URL[] pointing to the location containing the external classes to load:
mainLoader = new URLClassLoader(urls, this.getClass().getClassLoader());
The problem arises when I try to RE-load an implementation, that requires multiple classes:
Module of class A requires class B to function. This is what happens when I try to dynamically load, then reload class A:
load A -> "Sure, but I'll need B with it." -> automatically loads B -> "Here ya go, A works fine now."
reload A -> "Sure, but I'll need B with it." -> crashes because B couldn't be found
Both classes are located in the exact same folder, structure like this:
Class A implements Module: com/foo/bar/A.class
Class B: com/foo/bar/B.class
urls: ["com/foo/bar/"]
I call the function with load("com.foo.bar.A"), which works when attempting to load it the first time, but fails when trying to reload it as described above.
It works fine when trying to load a "single class module", the problem arises when the module relies on an additional external class. I tried using different classloaders to use as the parent for the URLClassLoader in the reloading process, those being the sysloader, Module.class.getClassLoader(), mainLoader (using that one, it won't ever find the new class definition because it already knows about it and therefor won't even attempt to load it from the drive again) and the mainLoader.getParent(), the classloader of the old module, and the parent of the modules classloader.
I'm probably just overseeing something obvious, but I can't figure out why it would manage to load the "extra" classes the first time, but fail when I reload the base class...
If you need any debug outputs or exact errors let me know, I replaced the debug outputs with comments explaining what does what so I got a fairly detailed log of what's happening when, but I didn't seem it to be necessary as it goes through the entire "check and then load" process just fine, it crashes when trying to enable the module. The "onEnable" method of the module requires the additional class B, that's where it fails. As I said, if you need the implementation of the classes A and B, Module, any other code or the debug outputs let me know and I'll add them in as requested.

There's a few things you can try:
Create an extension of UrlClassLoader so that you can track when it loads a class and what class loader is used to load the class.
Your other issue is make sure none of these classes are available on the "default" class path as that will cause that version to use. You are not overriding the default class loading behaviour which is to check the parent for the class first.
The other issue you're probably facing relates to the way the VM caches classes - I'm not entirely sure how this works - but from what I've experienced it seems that once a class is loaded it puts it in a shared storage space so that it does not load the class again. This shared space class will not be unloaded until the class loader that loaded it goes unreachable.

The solution lies in the classloader being closed and deleted as soon as the loading of the initial class is done, due to the class loader being only existant in the try/catch clause. I solved the issue by storing the classloader in a map until a new implementation of the module is loaded, then I can discard the old loader and store the new one instead.

RESTEasy - add a resource class dynamically

With RESTEasy I've implemented a subclass of Application to provide a list of singleton resources. Is there a way to add another singleton dynamically later on? I've not found a way to do it from the API docs.

I have not tried this myself, but I found a blog post where this is described:
http://sarbarian.wordpress.com/2010/03/07/resteasy-and-osgi-perfect-match/
During deployment, RESTEasy puts it's registry in the servlet context. The idea suggstested in the blog, is that you fetch the registry from the servlet context, and then add your resource class.
Something like this:
import org.jboss.resteasy.spi.Registry;
Object resource = new MyService();
Registry registry = (Registry) context.getAttribute(Registry.class.getName());
registry.addSingletonResource(resource);

I have tried this myself, and although it was nerve-wracking, it works great. We've got an infrastructure where several independent projects (plug-ins) are imported and loaded by our main application, and we wanted to include access to them in our existing RestEasy API (and Swagger documentation).
We originally hard-coded the class-loading into our main Rest Application class, which required all of the plug-ins to be displayed at all times in our API. To avoid that and to re-instate the independence of those projects (all, some, or none may be included at any given time), we created dynamic methods that return the feature/target class itself, which is loaded during the bootstrap phase of the RestEasy framework. It worked like a charm.
Within our Rest Application class, we implemented it as follows:
#Override
public Set<Class<?>> getClasses() {
Set<Class<?>> classes = new HashSet<>();
classes.add(mainAPIStuff.class);
classes.add(plugin1.class);
classes.add(plugin2.class);
classes.add(plugin3.class);
.
.
.
return classes;
}
That was the old way. The new way was to loop through all of our plugins and return a hashSet of the classes that the plugins provide:
#Override
public Set<Class<?>> getClasses() {
Set<Class<?>> classes = new HashSet<>();
classes.add(mainAPIStuff.class);
classes.addAll(pluginsHelper.getClasses());
return classes;
}
The pluginsHelper has sundry shared plugin methods and administrative tools, like getClasses(), which does basically this:
public static Set<Class<?>> getClasses() {
Set<Class<?>> classes = new HashSet<>();
// Probe plugins for their Rest Class implementation
List plugins = getCurrentlyInstalledPluginsFromOurSystem();
for (PluginObject plugin : plugins) {
Class<?> clazz = plugin.getPluginRestClass();
if (clazz != null) {
classes.add(clazz);
}
}
return classes;
}
RestEasy builds the Rest targets dynamically when any target is called, so all of the plugin installation, startup, config has already completed and the system is in the correct state before it is called. This is very valuable to understand, since then you can do all of the RestEasy class manipulation and loading at call time.

I use Reateasy with Undertow in embedded environment, where I manually instantiate ResteasyDeployment, and register Resource class is really easy, see code snippet blow:
UndertowJaxrsServer undertowJaxrsServer = new UndertowJaxrsServer();
ResteasyDeployment resteasyDeployment = new ResteasyDeployment();
undertowJaxrsServer.start();
resteasyDeployment.start();
final DeploymentInfo undertowDeployment =
undertowJaxrsServer
.undertowDeployment(resteasyDeployment)
.setContextPath("/abc")
.setDeploymentName("TEST")
.setClassLoader(Thread.currentThread().getContextClassLoader());
undertowJaxrsServer.deploy(undertowDeployment);
resteasyDeployment.getRegistry().addSingletonResource(new ResourceApiImpl());