We Keep Coding

How to use URLClassloader in an auto-update jar launcher?

I've come across many posts about these two topics: Auto-Updating and URLClassloaders. I'll start with the auto updating goal. I found this post here that talks about a 2 jar system. One jar that launches the main app jar: From Stephen C:
The launcher could be a Java application that creates a classloader for the new JAR, loads an entrypoint class and calls some method on it. If you do it this way, you have to watch for classloader storage leaks, but that's not difficult. (You just need to make sure that no objects with classes loaded from the JAR are reachable after you relaunch.)
This is the approach I'm taking, but I'm open to other ideas if they prove easier and/or more reliable. The Coordinator has posted some pretty cool launcher code to which I plan on incorporating some of this reload type code in my launcher, but first I need to get it to work.
My issue is that my main app jar has many other dependencies, and I cannot get some of those classes to load despite the fact that all the jars have been added to the URL's array. This brings up the second topic URLClassloader.
Side Note for future readers: When passing a URL to the URLClassloader that is a directory, a helpful note that would have saved me (an embarrassingly large) amount of time is that the contents of the directory must be .class files! I was originally pointing to my dependent jar directory, no good.
Context for the code below, my launcher jar resides in the same directory as my app jar, which is why I'm using user.dir. I will probably change this, but for now the code works and gets far enough into my app's code to request a connection to a sqlite database before failing.
Launcher:
public class Launcher {
public static void main(String[] args) {
try {
String userdir = System.getProperty("user.dir");
File parentDir = new File(userdir);
ArrayList<URL> urls = getJarURLs(parentDir);
URL[] jarURLs = new URL[urls.size()];
int index = 0;
for (URL u : urls) {
System.out.println(u.toString());
jarURLs[index] = u;
index ++;
}
URLClassLoader urlCL = new URLClassLoader(jarURLs);
Class<?> c = urlCL.loadClass("main.AppStart");
Object [] args2 = new Object[] {new String[] {}};
c.getMethod("main", String[].class).invoke(null, args2);
urlCL.close();
} catch (Exception e1) {
e1.printStackTrace();
}
}
public static ArrayList<URL> getJarURLs(File parentDir) throws MalformedURLException {
ArrayList<URL> list = new ArrayList<>();
for (File f : parentDir.listFiles()) {
if (f.isDirectory()) {
list.addAll(getJarURLs(f));
} else {
String name = f.getName();
if (name.endsWith(".jar")) {
list.add(f.toURI().toURL());
}
}
}
return list;
}
}
Here's an example of the URL output added to the array:
file:/C:/my/path/to/dependent/jars/sqlite-jdbc-3.32.3.2.jar
file:/C:/my/path/to/main/app.jar
file: ... [10 more]
The URLClassloader seems to work well enough to load my main method in app.jar. The main executes a some startup type stuff, before attempting to load a login screen. When the request is made to get the user info database, my message screen loads and displays (<-this is important for later)
the stacktrace containing:
java.sql.SQLException: No suitable driver found for jdbc:sqlite:C:\...\users.db
I understand that this is because that jar is not on the class path, but it's loaded via the class loader, so why can't it find the classes from the jar? From this post JamesB suggested adding Class.forName("org.sqlite.JDBC"); before the connection request. I rebuilt the app jar with this line of code and it worked!
The weird thing that happened next, is that my message screen class can no longer be found even though earlier it loaded and displayed correctly. The message screen is a class inside my main app.jar and not in a dependent jar, which is why I'm baffled. Am I going to have to add Class.forName before every instance of any of my classes? That seems rude..
So what could I be doing wrong with the class loader? Why does it load some classes and not others despite that fact that all the jars have been added to the URL array?
Some other relative info: My app works perfectly as intended when launched from windows command line when the classpath is specified: java -cp "main-app.jar;my/dependent/jar/directory/*" main.AppStart. It's only when I try launching the app via this classloader that I have these issues.
By the way, is this java command universal? Will it work on all operating systems with java installed? If so, could I not just scrap this launcher, and use a process builder to execute the above command? Bonus points for someone who can tell me how to execute the command from a jre packaged with my app, as that's what I plan on doing so the user does not have to download Java.

EDIT
I figured out one of the answers to one of the questions below. Turns out, I didn't need to do any of the code below. My main method loads a login screen but after it's loaded it returns back to the AppLauncher code, thus closing the URLClassLoader! Of course, at that point any requested class will not be found as the loader has been closed! What an oof! Hopefully I will save someone a headache in the future...
Original
Well, after more time, effort, research, and effective use of Eclipse's debugging tool, I was able to figure out what I needed to do to resolve my issues.
So the first issue was my JDBC driver was never registered when passing the jars to the URLClassloader. This is the part I sorta don't understand, so advisement would be welcomed, but there is a static block in the JDBC class that registers the driver so it can be used by DriverManager see code below. Loading the class is what executes that static block, hence why calling Class.forName works.
static {
try {
DriverManager.registerDriver(new JDBC());
} catch (SQLException e) {
e.printStackTrace();
}
}
What I don't understand, is how class loading works if jars are specified via the class path. The URLClassLoader doesn't load any of those classes until they are called, and I never directly work with the JDBC class, thus no suitable driver exception, but are all the classes specified via the classpath loaded initially? Seems that way for static blocks to execute.
Anyhow, to resolve my other issue with some of my app's classes not being found I had to implement my own classloader. I get what I did and how it works well, but still don't understand why I had to do it. All of my jars were loaded to the original URLClassloader so if I could find them and the files within, why couldn't it do it?
Basically, I had to override the findClass and findResource methods to return jarEntry information that I had to store. I hope this code helps someone!
public class SBURLClassLoader extends URLClassLoader {
private HashMap<String, Storage> map;
public SBURLClassLoader(URL[] urls) {
super(urls);
map = new HashMap<>();
try {
storeClasses(urls);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
private void storeClasses(URL[] urls) throws ClassNotFoundException {
for (URL u : urls) {
try {
JarFile jarFile = new JarFile(new File(u.getFile()));
Enumeration<JarEntry> e = jarFile.entries();
while (e.hasMoreElements()) {
JarEntry jar = e.nextElement();
String entryName = jar.getName();
if (jar.isDirectory()) continue;
if (!entryName.endsWith(".class")) {
//still need to store these non-class files as resources
//let code continue to store entry un-altered
} else {
entryName = entryName.replace(".class", "");
entryName = entryName.replace("/", ".");
}
map.put(entryName, new Storage(jarFile, jar));
System.out.println(entryName);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
Class<?> c = null;
try {
c = super.findClass(name);
} catch (ClassNotFoundException e) {
Storage s = map.get(name);
try {
InputStream in = s.jf.getInputStream(s.je);
int len = in.available();
c = defineClass(name, in.readAllBytes(), 0, len);
resolveClass(c);
} catch (IOException e1) {
e1.printStackTrace();
}
if (c == null) throw e;
}
return c;
}
#Override
public URL findResource(String name) {
URL url = super.findResource(name);
if (url == null) {
Storage s = map.get(name);
if (s != null) {
try {
url = new URL("jar:"+s.base.toString() + "!/" + name);
} catch (Exception e) {
e.printStackTrace();
}
}
}
return url;
}
private class Storage {
public JarFile jf;
public JarEntry je;
public URL base;
public Storage(JarFile jf, JarEntry je) {
this.jf = jf;
this.je = je;
try {
base = Path.of(jf.getName()).toUri().toURL();
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
}
}

Adding com.sun.tools.jar in classpath of jar

I'm having trouble using tools.jar present in jdk1.8.0_121/lib/tools.jar.
My $JAVA_HOME is set to:
# echo $JAVA_HOME
/usr/local/java/jdk1.8.0_121
The path to tools.jar is :
# ls /usr/local/java/jdk1.8.0_121/lib/tools.jar
/usr/local/java/jdk1.8.0_121/lib/tools.jar
And I use the following java executable to run the code:
/usr/local/java/jdk1.8.0_161/bin/java
But, when I access the VirtualMachine class, it throws
Caused by: java.lang.ClassNotFoundException: com.sun.tools.attach.VirtualMachine
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_161]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_161]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) ~[na:1.8.0_161]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_161]
... 72 common frames omitted
Can someone explain why Java is not able to find lib/tools.jar in its classpath & What can I do to correct this behaviour?
To run on my local machine, I've added the following dependency in my pom:
<dependency>
<groupId>com.sun</groupId>
<artifactId>tools</artifactId>
<version>1.8</version>
<scope>system</scope>
<systemPath>${java.home}/../lib/tools.jar</systemPath>
</dependency>
But, when I deploy it on the server, this jar is not packaged due to system scope & neither does it find the jar on the server's jdk path.
Isn't it supposed to find all the jdk jars automatically?
I've also tried to add env variable $JAVA_HOME in the class-path entry of jar's MANIFEST file as follows:
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Built-By: pankajsinghal
Class-Path: $JAVA_HOME/lib/
Created-By: Apache Maven 3.5.4
Build-Jdk: 1.8.0_181
But, this is also not working. Also, I don't want to add this lib's jar in my code explicitly as it's a JDK lib and I guess the proper way to access this would be from the system's JDK path itself. So, looking for a solution in this direction itself.
Any help is really appreciated.

You can try it like this:
java -cp "/path/your.jar:/usr/local/java/jdk1.8.0_121/lib/tools.jar" your.MainClass
or Refer to the following:
The type "com.sun.tools.javac.util.Assert" is not accessible
Hopefully it helped you.

You have to add that jar in project properties. In eclipse, To Add this Jar to your build path Right click the Project > Build Path > Configure build path> Select Libraries tab > Click Add External Libraries > Select the Jar file.

you can directly add toos.jar to your current classLoader, but it just an idea.
File getJar = new File(folderLibsPath + File.separator + "tools.jar");
URLClassLoader classLoaderExt = (URLClassLoader) this.getClassLoader();
URL jarUrl = getJar.toURI().toURL();
Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
method.invoke(classLoaderExt, jarUrl);
referenced from:How to load JAR files dynamically at Runtime?
and don't forget to load the attach.so (or attach.dll)
by System.load(absPath) or System.loadLibrary(dllName)
File attachedDLL = new File(folderLibFilePath);
if (attachedDLL.exists()) {
System.load(attachedDLL.getAbsolutePath());
}
I think we had got the same issue, and this code works for my case.
Also, there is another way to add tools.jar into classpath, but actually they did the same thing:
public void onEnable() throws Exception {
URLClassPath ucp = (URLClassPath) Reflection.getPrivateField("ucp", this.getClassLoader().getParent()); // reflect the subClass of URLClassLoader
File getJar = new File(folderLibsPath + File.separator + "tools.jar");
URL jarUrl = getJar.toURI().toURL();
ucp.addURL(jarUrl); // or just change its "URLs" field by put your jarURL in its Stack
}
But it should be mention that, this way the Java will use the AppClassLoader(SystemClassLoader) to load the tools.jar (also the invoker - your application will). This may have a bad effect on your original class initialization if you use CustomClassLoader. (because depending on the Java Parent Delegation Model, the superClassLoader cannot know which class load by its subClassLoader).
So if you are developing a plugin under a customClassLoader (the subclass of system classloader), the classpath in AppClassLoader should be removed (which means let custom PluginClassLoader to load it, or not its super one) after your VM was detached.
here I used reflection to accomplished.
public class Main {
public void onEnable() throws Exception {
/** load attach.dll */
System.loadLibrary("attach");
/** load tools.jar */
URLClassPath ucp = (URLClassPath) Reflection.getPrivateField("ucp", this.getClassLoader().getParent());
File getJar = new File(folderLibsPath + File.separator + "tools.jar");
URL jarUrl = getJar.toURI().toURL();
ucp.addURL(jarUrl);
/** attach, load, detach VM */
VirtualMachine vm;
vm = VirtualMachine.attach(this.getPid());
// if the current jar itself is the agent
vm.loadAgent(new File(this.getClass().getProtectionDomain().getCodeSource().getLocation().toURI().getPath()).getAbsolutePath());
vm.detach();
/** change the classLoader back to your custom */
changeClassLoaderBack();
/** unload native DLL Lib */
unloadNativeLibs(); // or you can add a condition to unload attach.dll only
}
public void changeClassLoaderBack() {
URLClassPath ucp = (URLClassPath) Reflection.getPrivateField("ucp", this.getClassLoader().getParent());
/** reset field path */
List<?> path = (ArrayList<?>) Reflection.getPrivateField("path", ucp);
List<URL> newPath = new ArrayList<>();
path.forEach((v) -> {
if(!((URL)v).getPath().contains("toos.jar") && !((URL)v).getPath().contains(this.getPlugin().getName())) {
newPath.add(((URL)v));
}
});
Reflection.setPrivateField("path", ucp, newPath);
/** reset field URLs */
Reflection.setPrivateField("urls", ucp, new Stack<URL>());
/** reset fields loader and LMAP */
List<Object> newLoader = new ArrayList<>();
Map<Object, Object> newLMAP = new HashMap<>();
((HashMap<?,?>)Reflection.getPrivateField("lmap", ucp)).forEach((k,v) -> {
if (!((String)k).contains("tools.jar") && !((String)k).contains(this.getPlugin().getName())) {
newLMAP.put(k, v);
newLoader.add(v);
};
});
Reflection.setPrivateField("lmap", ucp, newLMAP);
Reflection.setPrivateField("loaders", ucp, newLoader);
}
private String getPid() {
RuntimeMXBean bean = ManagementFactory.getRuntimeMXBean();
String pid = bean.getName();
if (pid.contains("#")) {
pid = pid.substring(0, pid.indexOf("#"));
}
return pid;
}
private void unloadNativeLibs(ClassLoader unloadDLLfromWhichLoader) {
try {
ClassLoader classLoader = unloadDLLfromWhichLoader;
Field field = ClassLoader.class.getDeclaredField("nativeLibraries");
field.setAccessible(true);
Vector<?> libs = (Vector<?>) field.get(classLoader);
Iterator<?> it = libs.iterator();
Object o;
while (it.hasNext()) {
o = it.next();
Method finalize = o.getClass().getDeclaredMethod("finalize", new Class[0]);
finalize.setAccessible(true);
finalize.invoke(o, new Object[0]);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public class Reflection {
public static Object getPrivateField(String fieldName, Object object) {
Field field;
Object o = null;
try {
field = object.getClass().getDeclaredField(fieldName);
field.setAccessible(true);
o = field.get(object);
}
catch (NoSuchFieldException e) {
e.printStackTrace();
}
catch (IllegalAccessException e) {
e.printStackTrace();
}
return o;
}
public static void setPrivateField(String fieldName, Object object, Object newField) {
Field field;
try {
field = object.getClass().getDeclaredField(fieldName);
field.setAccessible(true);
field.set(object, newField);
}
catch (NoSuchFieldException e) {
e.printStackTrace();
}
catch (IllegalAccessException e) {
e.printStackTrace();
}
}
}
hope it can help you at some points

ClassLoader in web container

For stand-alone java application I can use the following codes to load jar lib dynamically during the runtime based on the library path. It seems not working if I deploy same codes in Java Web container and run as a servlet. I actually want to be able to load the different jar libs based on the jar lib path in servlet request.
It means that single servlet will have to be able to load different jar libs dynamically during runtime and run onward biz logic
user1 might request to load jar files under /tmp/lib/v1.0/.jar
user2 might request to load jar files under /tmp/lib/v1.1/.jar
(The jar files in v1.0 and v1.1 got the exactly same class names)
Thanks!!!
=== Main =============
LibraryLoader loader = new LibraryLoader();
loader.addClassPath(<jar lib root path>);
// below will run biz logic
=== LibraryLoader.java ==========
public class LibraryLoader {
URLClassLoader urlClassLoader;
public LibraryLoader() {
urlClassLoader = (URLClassLoader) ClassLoader.getSystemClassLoader();
}
public void addClassPath(String jarLibPath) throws Exception {
Class urlClass = URLClassLoader.class;
File jarPath = new File(jarLibPath);
FileFilter jarFilter = new FileFilter() {
public boolean accept(File f) {
if (f.isDirectory())
return true;
String name = f.getName().toLowerCase();
return name.endsWith("jar");
}
};
File[] files = jarPath.listFiles(jarFilter);
for (File file : files) {
if (file.isFile()) {
URI uriPath = file.toURI();
URL urlPath = uriPath.toURL();
Method method = urlClass.getDeclaredMethod("addURL", new Class[] { URL.class });
method.setAccessible(true);
method.invoke(urlClassLoader, new Object[] { urlPath });
System.out.println(file.getCanonicalPath());
} else {
addClassPath(file.getCanonicalPath());
}
}
}
}

that's not what you want to do. even in your stand-alone program, this won't really work (as you could be pushing multiple versions of the same jar into the main classloader). what you want to do is create a new classloader containing the new jars, then invoke some class in the new classloader.
e.g.:
// get relevant jar urls
URL[] urls = ...;
ClassLoader oldLoader = Thread.currentThread().getContextClassLoader();
try {
URLClassLoader loader = new URLClassLoader(urls);
Thread.currentThread().setContextClassLoader(loader);
Class<?> entryClass = loader.loadClass("entry.class.name");
// do something here w/ entryClass (e.g. instantiate it) ...
} finally {
Thread.currentThread().setContextClassLoader(oldLoader);
}
of course, you probably want to cache these classloaders and re-use them on subsequent requests.
of course, i'm not sure why you don't just deploy multiple versions of the servlet with the different versioned jars (since you indicate that the version is part of the servlet path).

You should create different classloader for different users and set to the Current Thread.
ClassLoader ctxLoader = Thread.currentThread().getContextClassLoader();
ClassLoader currentLoader;
if(user1){
//initialize classloader with jar files under /tmp/lib/v1.0/.jar
currentLoader = user1ClassLoader;
} else if(user2){
//initialize classloader with jar files under /tmp/lib/v1.1/.jar
currentLoader = user2ClassLoader;
}
Thread.currentThread().setContextClassLoader(currentLoader );
//invoke business logic
//reset the actual loader
Thread.currentThread().setContextClassLoader(ctxLoader );

How do I open and run a compiled Java file?

How do I open a .class or .jar file within a Java program?
(remember that .jar files may have more than one class with main(String[] args) method)
(individual question from IDE-Style program running )

Here is a quick and dirty dirty hack for running all main methods found in the jar.
import java.io.*;
class JarRunner {
public static void main(String[] args) throws IOException,
ClassNotFoundException {
File jarFile = new File("test.jar");
URLClassLoader cl = new URLClassLoader(new URL[] {jarFile.toURL() });
JarFile jf = new JarFile(jarFile);
Enumeration<JarEntry> entries = jf.entries();
while (entries.hasMoreElements()) {
JarEntry je = entries.nextElement();
String clsName = je.getName();
if (!clsName.endsWith(".class"))
continue;
int dot = clsName.lastIndexOf('.');
Class<?> clazz = cl.loadClass(clsName.substring(0, dot));
try {
Method m = clazz.getMethod("main", String[].class);
m.invoke(null, (Object) new String[0]);
} catch (SecurityException e) {
} catch (NoSuchMethodException e) {
} catch (IllegalArgumentException e) {
} catch (IllegalAccessException e) {
} catch (InvocationTargetException e) {
}
}
}
}
As mentioned by other posters, you may want to have a look in the manifest file for the main class (so you don't have to be guessing). This can be accessed through JarFile.getManifest().

The manifest names the jar's entry point.

Use
java -cp my.jar org.myorg.MyClass
if MyClass is the one you want to start. If my.jar has a proper MANIFEST.MF file indicating MyClass you can use
java -jar my.jar

You can open a .jar with any compression-software (winrar, winzip, 7zip) and you can run the .class file with java.exe

I think your questions is about situation when you don't know specification of external class in compilation time. Am I right?
So you need to use reflection API for creating instance of necessary class and invoking its method. You can see example above.
And for determining class for running from jar file you should use package 'java.util.jar' for accessing manifest via Manifest class. And you can determine entry point of this jar from attribute 'Main-Class'.

How can I test a .class file was created?

I want to unit test my code which will have to create a .java file compile it and then the corresponding .class file should be created.
How can I create the test to see if the ".class" file is created? I have added test already for its existence, now I'm trying to test the file is a valid class file.
I tried
try {
Class.forName("Hello");
throw AssertError();
} catch( ClassNotFoundException e ) {
}
program.createClass();
Class.forName("Hello");
But I don't really know how to dynamically add the path where the file is created to the classpath.
EDIT
URL Class loaded does the work.
This is how my test looks like now.
#Test
void testHello() throws MalformedURLException, ClassNotFoundException {
URL[] url = {
new URL("file:/home/oreyes/testwork/")
};
try {
new URLClassLoader(url).loadClass("Hello");
throw new AssertionError("Should've thrown ClassNotFoundException");
} catch ( ClassNotFoundException cnfe ){
}
c.process();
new URLClassLoader(url).loadClass("Hello");
}

Use a new instance of an URLClassLoader, pointing to the root folder where you created the target class file. Then, use the Class.forName(String,ClassLoader); method with the dynamically created URLClassLoader to load the new class.
To show that it works, the following test case will create a source file, write some Java code in there and compile it using the Java 6 ToolProvider interfaces. Then, it will dynamically load the class using an URLClassLoader and invoke a reflective call to its class name to verify it's really this class which has been generated on the fly.
#Test
public void testUrlClassLoader() throws Exception {
Random random = new Random();
String newClassName = "Foo" + random.nextInt(1000);
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);
List<File> files = new ArrayList<File>();
File sourceFolder = new File(".");
File sourceFile = new File(sourceFolder, newClassName + ".java");
FileWriter fileWriter = new FileWriter(sourceFile);
fileWriter.write("public class " + newClassName + " { { System.out.println(\""
+ newClassName + " loaded\"); }}");
fileWriter.close();
files.add(sourceFile);
Iterable<? extends JavaFileObject> compilationUnits1 = fileManager
.getJavaFileObjectsFromFiles(files);
compiler.getTask(null, fileManager, null, null, null, compilationUnits1).call();
fileManager.close();
URL url = sourceFolder.toURI().toURL();
URLClassLoader urlClassLoader = new URLClassLoader(new URL[] { url });
Object newInstance = urlClassLoader.loadClass(newClassName).newInstance();
assertEquals(newClassName, newInstance.getClass().getName());
}

Instead of loading the class in order to verify it, you could shell out to a command like "file Hello.class" to see if it reports that it's a java class file, or even spawn a sub-process of java to load the class outside of your test JVM.