We Keep Coding

How to use URLClassloader in an auto-update jar launcher?

I've come across many posts about these two topics: Auto-Updating and URLClassloaders. I'll start with the auto updating goal. I found this post here that talks about a 2 jar system. One jar that launches the main app jar: From Stephen C:
The launcher could be a Java application that creates a classloader for the new JAR, loads an entrypoint class and calls some method on it. If you do it this way, you have to watch for classloader storage leaks, but that's not difficult. (You just need to make sure that no objects with classes loaded from the JAR are reachable after you relaunch.)
This is the approach I'm taking, but I'm open to other ideas if they prove easier and/or more reliable. The Coordinator has posted some pretty cool launcher code to which I plan on incorporating some of this reload type code in my launcher, but first I need to get it to work.
My issue is that my main app jar has many other dependencies, and I cannot get some of those classes to load despite the fact that all the jars have been added to the URL's array. This brings up the second topic URLClassloader.
Side Note for future readers: When passing a URL to the URLClassloader that is a directory, a helpful note that would have saved me (an embarrassingly large) amount of time is that the contents of the directory must be .class files! I was originally pointing to my dependent jar directory, no good.
Context for the code below, my launcher jar resides in the same directory as my app jar, which is why I'm using user.dir. I will probably change this, but for now the code works and gets far enough into my app's code to request a connection to a sqlite database before failing.
Launcher:
public class Launcher {
public static void main(String[] args) {
try {
String userdir = System.getProperty("user.dir");
File parentDir = new File(userdir);
ArrayList<URL> urls = getJarURLs(parentDir);
URL[] jarURLs = new URL[urls.size()];
int index = 0;
for (URL u : urls) {
System.out.println(u.toString());
jarURLs[index] = u;
index ++;
}
URLClassLoader urlCL = new URLClassLoader(jarURLs);
Class<?> c = urlCL.loadClass("main.AppStart");
Object [] args2 = new Object[] {new String[] {}};
c.getMethod("main", String[].class).invoke(null, args2);
urlCL.close();
} catch (Exception e1) {
e1.printStackTrace();
}
}
public static ArrayList<URL> getJarURLs(File parentDir) throws MalformedURLException {
ArrayList<URL> list = new ArrayList<>();
for (File f : parentDir.listFiles()) {
if (f.isDirectory()) {
list.addAll(getJarURLs(f));
} else {
String name = f.getName();
if (name.endsWith(".jar")) {
list.add(f.toURI().toURL());
}
}
}
return list;
}
}
Here's an example of the URL output added to the array:
file:/C:/my/path/to/dependent/jars/sqlite-jdbc-3.32.3.2.jar
file:/C:/my/path/to/main/app.jar
file: ... [10 more]
The URLClassloader seems to work well enough to load my main method in app.jar. The main executes a some startup type stuff, before attempting to load a login screen. When the request is made to get the user info database, my message screen loads and displays (<-this is important for later)
the stacktrace containing:
java.sql.SQLException: No suitable driver found for jdbc:sqlite:C:\...\users.db
I understand that this is because that jar is not on the class path, but it's loaded via the class loader, so why can't it find the classes from the jar? From this post JamesB suggested adding Class.forName("org.sqlite.JDBC"); before the connection request. I rebuilt the app jar with this line of code and it worked!
The weird thing that happened next, is that my message screen class can no longer be found even though earlier it loaded and displayed correctly. The message screen is a class inside my main app.jar and not in a dependent jar, which is why I'm baffled. Am I going to have to add Class.forName before every instance of any of my classes? That seems rude..
So what could I be doing wrong with the class loader? Why does it load some classes and not others despite that fact that all the jars have been added to the URL array?
Some other relative info: My app works perfectly as intended when launched from windows command line when the classpath is specified: java -cp "main-app.jar;my/dependent/jar/directory/*" main.AppStart. It's only when I try launching the app via this classloader that I have these issues.
By the way, is this java command universal? Will it work on all operating systems with java installed? If so, could I not just scrap this launcher, and use a process builder to execute the above command? Bonus points for someone who can tell me how to execute the command from a jre packaged with my app, as that's what I plan on doing so the user does not have to download Java.

EDIT
I figured out one of the answers to one of the questions below. Turns out, I didn't need to do any of the code below. My main method loads a login screen but after it's loaded it returns back to the AppLauncher code, thus closing the URLClassLoader! Of course, at that point any requested class will not be found as the loader has been closed! What an oof! Hopefully I will save someone a headache in the future...
Original
Well, after more time, effort, research, and effective use of Eclipse's debugging tool, I was able to figure out what I needed to do to resolve my issues.
So the first issue was my JDBC driver was never registered when passing the jars to the URLClassloader. This is the part I sorta don't understand, so advisement would be welcomed, but there is a static block in the JDBC class that registers the driver so it can be used by DriverManager see code below. Loading the class is what executes that static block, hence why calling Class.forName works.
static {
try {
DriverManager.registerDriver(new JDBC());
} catch (SQLException e) {
e.printStackTrace();
}
}
What I don't understand, is how class loading works if jars are specified via the class path. The URLClassLoader doesn't load any of those classes until they are called, and I never directly work with the JDBC class, thus no suitable driver exception, but are all the classes specified via the classpath loaded initially? Seems that way for static blocks to execute.
Anyhow, to resolve my other issue with some of my app's classes not being found I had to implement my own classloader. I get what I did and how it works well, but still don't understand why I had to do it. All of my jars were loaded to the original URLClassloader so if I could find them and the files within, why couldn't it do it?
Basically, I had to override the findClass and findResource methods to return jarEntry information that I had to store. I hope this code helps someone!
public class SBURLClassLoader extends URLClassLoader {
private HashMap<String, Storage> map;
public SBURLClassLoader(URL[] urls) {
super(urls);
map = new HashMap<>();
try {
storeClasses(urls);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
private void storeClasses(URL[] urls) throws ClassNotFoundException {
for (URL u : urls) {
try {
JarFile jarFile = new JarFile(new File(u.getFile()));
Enumeration<JarEntry> e = jarFile.entries();
while (e.hasMoreElements()) {
JarEntry jar = e.nextElement();
String entryName = jar.getName();
if (jar.isDirectory()) continue;
if (!entryName.endsWith(".class")) {
//still need to store these non-class files as resources
//let code continue to store entry un-altered
} else {
entryName = entryName.replace(".class", "");
entryName = entryName.replace("/", ".");
}
map.put(entryName, new Storage(jarFile, jar));
System.out.println(entryName);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
Class<?> c = null;
try {
c = super.findClass(name);
} catch (ClassNotFoundException e) {
Storage s = map.get(name);
try {
InputStream in = s.jf.getInputStream(s.je);
int len = in.available();
c = defineClass(name, in.readAllBytes(), 0, len);
resolveClass(c);
} catch (IOException e1) {
e1.printStackTrace();
}
if (c == null) throw e;
}
return c;
}
#Override
public URL findResource(String name) {
URL url = super.findResource(name);
if (url == null) {
Storage s = map.get(name);
if (s != null) {
try {
url = new URL("jar:"+s.base.toString() + "!/" + name);
} catch (Exception e) {
e.printStackTrace();
}
}
}
return url;
}
private class Storage {
public JarFile jf;
public JarEntry je;
public URL base;
public Storage(JarFile jf, JarEntry je) {
this.jf = jf;
this.je = je;
try {
base = Path.of(jf.getName()).toUri().toURL();
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
}
}

Reflections from Maven Mojo

I'd like to use Google Reflections to scan classes from the compiled project from my Maven plugin. But plugins by default don't see the compiled classes of the project. From Maven 3 documentation I read:
Plugins that need to load classes from the compile/runtime/test class path of a project need to create a custom URLClassLoader in combination with the mojo annotation #requiresDependencyResolution.
Which is a bit vague to say the least. Basically I would need a reference to a classloader that loads the compiled project classes. How do I get that?
EDIT:
Ok, the #Mojo annotation has requiresDependencyResolution parameter, so that's easy but still need the correct way to build a classloader.

#Component
private MavenProject project;
#SuppressWarnings("unchecked")
#Override
public void execute() throws MojoExecutionException {
List<String> classpathElements = null;
try {
classpathElements = project.getCompileClasspathElements();
List<URL> projectClasspathList = new ArrayList<URL>();
for (String element : classpathElements) {
try {
projectClasspathList.add(new File(element).toURI().toURL());
} catch (MalformedURLException e) {
throw new MojoExecutionException(element + " is an invalid classpath element", e);
}
}
URLClassLoader loader = new URLClassLoader(projectClasspathList.toArray(new URL[0]));
// ... and now you can pass the above classloader to Reflections
} catch (ClassNotFoundException e) {
throw new MojoExecutionException(e.getMessage());
} catch (DependencyResolutionRequiredException e) {
new MojoExecutionException("Dependency resolution failed", e);
}
}

Make the web application auto-determine and know its version

My team are doing with 2 teams: server team and client team. We at server team will provide APIs for the client team to call. The APIs have many version, so that we need to match the server build and the respective client build - for example, the old client will refuse to work if the server build number is much larger than its support version and requires an update.
Because of the above reason, we need to send back the build version of server to client. Currently we are doing this by adding a static field in a Config class. But I'm concerned with the fact that we must manually increase it everytime a new server is built - especially when we do daily build. This process is quite error-prone and not quite elegant.
In my search, I see many propose for using maven plugins to manage the build version. Though I highly appreciated the auto-process, it still doesn't let the server know the build number. Server application should be able to return its build version to client through an API call.
I have thought of write the number version somewhere (remote database, files on server).
Is there any way to make the building process automatically increase the build version, but the application itself can retrieve this number in running also?
We are using Maven build and having Jenkin as the integration build server.

I usually read the version from the MANIFEST.MF file that is packaged in the JAR by Maven.
By default it looks something like this:
Manifest-Version: 1.0
Built-By: user
Build-Jdk: 1.6.0_35
Created-By: Apache Maven 3.0.4
Archiver-Version: Plexus Archiver
Name: Artifact
Implementation-Build: 1.14-SNAPSHOT
Version: 1.14-SNAPSHOT
From this file I read the Version element and use that to for instance display the build version of the application (and all versions of the packaged jars in a WAR/EAR for instance).
Something like this code should work:
public static String getApplicationVersion() {
String version = null;
try {
final List<VersionsUtil.Version> moduleVersions = VersionsUtil.getModuleVersions(Thread.currentThread().getContextClassLoader());
for (VersionsUtil.Version moduleVersion : moduleVersions) {
if (moduleVersion.name.equals("<NAME OF ARTIFACT TO GET>")) {
version = moduleVersion.version;
break;
}
}
} catch (IOException e) {
// We'll return null...
}
return version;
}
public class VersionsUtil {
private static final Logger LOG = LoggerFactory.getLogger(VersionsUtil.class);
/**
* Returns a list of the module versions available for the given class loader.
*
* #param classLoader the class loader to return module versions for
* #return a list of module versions
* #throws IOException in case there's an error reading the manifest
*/
public static List<Version> getModuleVersions(final ClassLoader classLoader) throws IOException {
return processResources(classLoader.getResources("META-INF/MANIFEST.MF"));
}
private static List<Version> processResources(final Enumeration<URL> resources) throws IOException {
final List<Version> moduleVersions = new ArrayList();
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
Version v = process(resource);
if (v != null) {
moduleVersions.add(v);
}
}
return moduleVersions;
}
private static Version process(final URL resource) {
try {
Properties p = readResource(resource);
return createVersion(p);
} catch (IOException e) {
LOG.warn("Failed to read resource: " + resource, e);
return null;
}
}
private static Version createVersion(final Properties p) {
Object name = p.get("Name");
if (name != null) {
return new Version((String) name, (String) p.get("Version"));
}
return null;
}
private static Properties readResource(final URL resource) throws IOException {
LOG.trace("Reading resource: " + resource);
InputStream is = resource.openStream();
Properties p = new Properties();
p.load(is);
is.close();
return p;
}
public static final class Version {
String name;
String version;
private Version(final String name, final String version) {
this.name = name;
this.version = version;
}
}
}
Updated:
If you want Jenkins buildnumber in the MANIFEST.MF you can configure your POM.XML with something like:
...
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.2</version>
<configuration>
<archive>
<manifestSections>
<manifestSection>
<name>${project.name} (${project.artifactId})</name>
<manifestEntries>
<Version>${project.version}${build.number}</Version>
</manifestEntries>
</manifestSection>
</manifestSections>
</archive>
</configuration>
</plugin>
...
<properties>
<build.number />
</properties>
...
If you're interested in tagging the WAR/EAR files instead, you have to add the manifest configurations accordingly.
Then in your Jenkins job configuration, simply pass the BUILD_NUMBER parameter to the maven process like this: -Dbuild.number=$BUILD_NUMBER.

Class loading Exception in java reflection

I have selected a jar file using file selector, then loaded all the classes in the jar file using java reflection. Some classes has dependency on another jar file.
But when I try to get method of class then following exception is thrown because this class has a import statement import com.thoughtworks.xstream.XStream; and XStream class is defined in another jar file.
Exception in thread "main" java.lang.NoClassDefFoundError: com/thoughtworks/xstream/io/HierarchicalStreamDriver
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2365)
at java.lang.Class.privateGetPublicMethods(Class.java:2488)
at java.lang.Class.getMethods(Class.java:1406)
at com.axway.xtp.testgenerator.templatewizard.MethodSelectionWizardUI.updateListofMethods(MethodSelectionWizardUI.java:744)
at com.axway.xtp.testgenerator.templatewizard.MethodSelectionWizardUI$7.widgetSelected(MethodSelectionWizardUI.java:474)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:90)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:928)
I wanted to know that is there any way to prevent the dependency class or jar file to be loaded using java reflection. Following are the piece of code I am using to load classes.
URLClassLoader ucl = new URLClassLoader(new URL[] { new URL("file:" + codeRepository) });
JarFile jarFile = new JarFile(new File(codeRepository));
Enumeration enm = jarFile.entries();
while (enm.hasMoreElements()) {
JarEntry entry = ((JarEntry) enm.nextElement());
if (entry.getName().endsWith(".class")) {
String fullClassNameWithPath = entry.getName();
String fullyClassifiedClassName = fullClassNameWithPath
.replace('/', '.');
try {
Class c = ucl.loadClass(fullyClassifiedClassName.substring(
0, fullyClassifiedClassName.indexOf(".class")));
String className = c.getPackage().getName() + "."
+ c.getSimpleName();
listClasses.add(className);
} catch (Exception e) {
continue;
} catch (Throwable t) {
continue;
}
}
}

Well, if your application depends on that class, you most definitely need to have the jar containing it (or provide an alternative path containing the package+class) on the classpath.

As part of the classloading process, any classes which the class you want to load depends upon will also be loaded.
I don't think there's anything you can do about that.

How do I open and run a compiled Java file?

How do I open a .class or .jar file within a Java program?
(remember that .jar files may have more than one class with main(String[] args) method)
(individual question from IDE-Style program running )

Here is a quick and dirty dirty hack for running all main methods found in the jar.
import java.io.*;
class JarRunner {
public static void main(String[] args) throws IOException,
ClassNotFoundException {
File jarFile = new File("test.jar");
URLClassLoader cl = new URLClassLoader(new URL[] {jarFile.toURL() });
JarFile jf = new JarFile(jarFile);
Enumeration<JarEntry> entries = jf.entries();
while (entries.hasMoreElements()) {
JarEntry je = entries.nextElement();
String clsName = je.getName();
if (!clsName.endsWith(".class"))
continue;
int dot = clsName.lastIndexOf('.');
Class<?> clazz = cl.loadClass(clsName.substring(0, dot));
try {
Method m = clazz.getMethod("main", String[].class);
m.invoke(null, (Object) new String[0]);
} catch (SecurityException e) {
} catch (NoSuchMethodException e) {
} catch (IllegalArgumentException e) {
} catch (IllegalAccessException e) {
} catch (InvocationTargetException e) {
}
}
}
}
As mentioned by other posters, you may want to have a look in the manifest file for the main class (so you don't have to be guessing). This can be accessed through JarFile.getManifest().

The manifest names the jar's entry point.

Use
java -cp my.jar org.myorg.MyClass
if MyClass is the one you want to start. If my.jar has a proper MANIFEST.MF file indicating MyClass you can use
java -jar my.jar

You can open a .jar with any compression-software (winrar, winzip, 7zip) and you can run the .class file with java.exe

I think your questions is about situation when you don't know specification of external class in compilation time. Am I right?
So you need to use reflection API for creating instance of necessary class and invoking its method. You can see example above.
And for determining class for running from jar file you should use package 'java.util.jar' for accessing manifest via Manifest class. And you can determine entry point of this jar from attribute 'Main-Class'.