We Keep Coding

How the class loader works in Java with a shaded jar

My experience with Java class loading is limited. With tools like Maven I have a rusty understanding of how they resolve dependency versions. But I've hit a problem that's making me question how Java loads classes.
My scenario, I have a dependency on version 30.1.1-jre of com.google.guava. I also have a shaded jar which has a dependency on Guava 18.0.
In my application I end up seeing the following exception
java.lang.IncompatibleClassChangeError: Class com.google.common.base.Suppliers$MemoizingSupplier does not implement the requested interface java.util.function.Supplier
which I cannot reproduce locally. Based on https://github.com/crabhi/celery-java/issues/9 it sounds like this error is produced when an older version of Guava is on the classpath.
Checking the classes in the war I see
WEB-INF/lib/java-driver-shaded-guava-25.1-jre-graal-sub-1.jar.d/com/datastax/oss/driver/shaded/guava/common/base/Suppliers$MemoizingSupplier.class
WEB-INF/lib/nautilus-es2-library-2.3.4.jar.d/com/google/common/base/Suppliers$MemoizingSupplier.class
WEB-INF/lib/guava-30.1.1-jre.jar.d/com/google/common/base/Suppliers$MemoizingSupplier.class
This makes me think the shaded jars are causing a problem.
Is that possible? Are there any articles explaining how classes are loaded when shaded jars enter the picture?

You have two copies of com.google.common.base.Suppliers.MemoizingSupplier on the classpath. Only one can be loaded, and in your case, it's the older version.
You shouldn't have multiple classes with the same name available to one classloader. What is nautilus-es2-library-2.3.4.jar and why does it bundle Guava rather than expressing it as a transitive dependency?

Can Maven generate the module declaration?

The module declaration defines, among other things, a module's dependencies. If I use Maven as a build tool, this is redundant because the pom.xml already contains these (and more) information. Based on that, couldn't Maven generate the module-info.java for me?

One might expect that most of the dependencies are indeed required modules as well. However, requirements can also point to modules of the JDK/JRE, which are not specified in the pom.xml. So yes, if you only look at the dependencies, probably most of them could be transformed to a required module reference.
But a module-descriptor contains much more information, which are all based on decisions to be made by the developer.
I've written an article about it which describes in detail why it is not possible to fully generate this file.

As far as I know, bnd-maven-plugin can generate module-info.class based on the configured dependencies. If you are working with maven-bundle-plugin, you need to specify the version of bndlib manually, for the latest version of maven-bundle-plugin(5.1.3) is still using the 5.x version of bndlib, and bndlib requires 6.x to support jpms.
Document: https://bnd.bndtools.org/releases/6.1.0/chapters/330-jpms.html

Spring, Jersey, CGLIB and Javassist

I am migrating an old project from Ant to Gradle (yes, there is still an Ant-based project in 2014). It has all sorts of nonsense thrown into its lib/ dir, and I'm very keen on dumping anything unnecessary as part of this migration.
The project is based on Spring 3.0.5 and Jersey 1.8. It is not using Hibernate but instead using Cassandra. The project seems to think it needs javassist 3.12.0, but didn't pull cglib.
I thought I had some vague recollection of perhaps one of these optionally using javassist, but it seems by recollection is lying to me. Is it safe to remove this dependency?

Depending on how you're using Spring you might need to add an explicit dependency on CGLIB. It's marked as an optional dependency in 3.0. In Spring 3.2, the CGLIB classes were repackaged into the Spring JARs so an explicit dependency on CGLIB is no longer required. You would need CGLIB if you're proxy-target-class="true" or #Configuration.
I just checked one of my projects that's similar to yours (Spring 3.0.x and Jersey 1.x). I have dependency on Javassist but Maven tells me it's because of Hibernate. I don't see an explicit dependency from Jersey 1.x to Javassist.
If I were you, I'd leave the Javassist dependency out of the POM and let Maven take care of pulling it in if it's necessary. I would only add an explicit dependency on Javassist if it was an optional dependency that I needed or if Maven made the wrong choice about which version to include.
Is it safe to remove this dependency?
It's hard to say without seeing your entire pom.xml or the list of JARs that the Ant script worked with.

Classloader to isolate a jar (class identity crisis)

I'm using jarX that has embedded dependencies that conflict with my own dependencies, so I'm creating a classloader to isolate jarX's dependencies from my main classloader.
jarX is outside my app's classpath, but my classes that use jarX's classes are in my classpath, so when I instantiate my classes loaded via the custom classloader, I run into the class identity crisis in the form of ClassCastException as the JVM's version of my classes are considered different from those loaded by my custom classloader.
I found this blog post where they solved a similar problem by only interacting with the custom classloader loaded classes via reflection, which seems to solve this problem.
It just feels like it should be easier than this. Does anyone know a better way to handle this problem?

The easiest way is to open jarX, remove the offending classes, and done. It is a bad practice to embed dependencies in a JAR unless that is JAR is meant to be used only as a standalone runnable fat-jar. JARs that are meant to be used as libraries should not embed dependencies.
When you notice that people package third-party classes in their JARs, I'd recommend pointing out to them that this is generally not a good idea and to encourage them to refrain from doing so. If a project provides a runnable fat-jar including all dependencies, that is fine. But, it should not be the only JAR they provide. A plain JAR or set of JARs without any third-party code should also be offered. In the rare cases that third-party code was modified and must be included, it should be done under the package namespace of the provider, not of the original third-party.
Finally, for real solutions to building modular Java applications and handling classloader isolation, check out one of the several OSGi implementations or project Jigsaw.

Can you post which jar is it and what are the classes that it overlaps, with the full stacktrace? Have a look at this tool I wrote to generate a list of duplicate classes in the WAR, there is an option to exclude duplicates of the same size.
These are some measures that can be done to solve this:
Try to reduce the number of duplicates by doing a case by case analysis of why the overlap exists. Add maven exclusions for jars that are complete duplicates.
Check if there is a version of the same jar without the dependencies that you could use, which jar is it, xerces, etc?
If there is no jar without dependencies, you can you exclude the other jar that overlaps jarX and see if the application still works. This means all components that need the jar have a compatible version of the jarX library
Separate the application into two WARs each with the version of the library you need. This will reduce the number of libraries in which
These where measures that are likelly to be more maintainable long-term
If the previous measures do not work:
open the jar, delete the duplicate classes and publish in the maven repository with a different name jarX-patched
you can configure nexus to serve a patched jar instead of an unpatched jar transparently
If your container supports OSGI that would be even better, but if you don't use a OSGI container for development as well, then the application would not work in development.

Datanucleus enhancer fails with class not found exception on Eclipse with GAE plugin

I have Eclipse with the Google App Engine plugin installed. Until recently the Datanucleus enhancer, which is run automatically when classes are saved worked nicely.
One of my entity classes has since grown slightly more complicated with the introduction of a member of a type, that resides inside an external library located in the project's lib-directory.
The field in question is marked with the #Transient annotation, but still the Datanucleus enhancer tries in vain to find the class in question. Apparently the jar-files in the project's lib-directory aren't scanned for dependencies.
Is there a way to define the classpath for the Datanucleus enhancer?

After reviewing some other related questions and browsing through Eclipse's settings (the GAE-related ones) I've come to the conclusion that there probably is no easy way to specify classpath directly.
The simplest and most elegant way I was able to come up with was to disable automatic Datanucleus enhancement, and introduce an Ant task to do this.
The approved answer to the question Add scala class to DataNucleus enhancer CLASSPATH turned out to be particularly helpful.
I wrote an Ant build file nearly identical to the one presented in the linked question and it solved my problem.