We Keep Coding

Resolving to correct dependencies when loading classes with custom classloaders

So I have this generic backend server that loads shaded jars in memory and then loads it through a custom Classloader.
E.g.
MyClass class = c.newInstance();
It works fine until the shaded Jar dependencies conflicts with the server classes.
E.g.
Server contains (with Custom Classloader):
com.fasterxml.jackson.jackson-databind:2.6.0
While the shaded jar contains
com.fasterxml.jackson.jackson-databind:2.9.9
When the method in the class that requires the said library e.g. class.doSomeThing(); it throws an error Caused by java.lang.NoSuchFieldError: because the loaded jackson-databind is 2.6.0 instead of 2.9.9
The question here is when the class is loaded from the shaded jar is there a way to make sure that the shaded dependencies are the ones used?

The question here is when the class is loaded from the shaded jar is there a way to make sure that the shaded dependencies are the ones used?
If you are using the default Class loader then the order of resolution will work as the order of the classpath. Within your code you can use
System.out.println(System.getProperty("system.class.path").replaceAll(":", "\n"));
And inspect the classpath. Usually such runtime environment (for example apache spark) has such features to allow you to prepend the classpath. You can check with your runtime server environment for such feature.

module-info.java compile fail with maven-compiler-plugin and automatic modules

I'm using v3.7.0 of the plugin as required and JDK 9.0.1. I have added two requires statements, each referring to a jar in the class path (automatic module). The module-info.java compiles successfully in Eclipse after I moved the jars to Modulepath. However, Maven gives me a compiler error saying one of them is missing (strangely, not the first one which is just one line before). I tried to check the automatic module name but I get an error from the commands just for this jar. What does this error mean and how do I fix it so that I can discover the proper module name?
I replaced my username in the output below. The jar in question does use a ServiceLoader but is not compiled with Java 9.
computerName:Commander-java username$ jar --file=/Users/username/.m2/repository/com/username/rcf/1.0/rcf-1.0.jar --describe-module
Unable to derive module descriptor for: /Users/username/.m2/repository/com/username/rcf/1.0/rcf-1.0.jar
Provider class com.username.rcf.server.TestCmdChain not in module
computerName:Commander-java username$ java -p /Users/username/.m2/repository/com/username/rcf/1.0/rcf-1.0.jar --list-modules
Error occurred during initialization of boot layer
java.lang.module.FindException: Unable to derive module descriptor for /Users/username/.m2/repository/com/username/rcf/1.0/rcf-1.0.jar
Caused by: java.lang.module.InvalidModuleDescriptorException: Provider class com.username.rcf.server.TestCmdChain not in module
The answer in How to deal with java keywords in auto generated module names in Java 9? has a different error related to using a Java identifier in the module name. The automatic jar name for my module should just be rcf since the jar name is rcf-1.0.jar. The error I'm getting is different also.

While deriving module description the contents of any
META-INF/services configuration files are mapped to provides
declarations.
The packages scanned for the services are the ones containing class files.
Also, the package name for individual classes is derived from their fully qualified name. From the shared logs com.username.rcf.server shall be the expected package name for the service to be provided and this shall turn into
provides x.y.z.TestCmdChainInterface with com.username.rcf.server.TestCmdChain
Seems like there is no such package com.username.rcf.server existing in your module.

POI running on TomCat causes loader constraint violation

I am trying to generate an XSLX using Apache POI, but get this error when trying to run my TomCat application:
HTTP Status 500 - javax.servlet.ServletException: java.lang.LinkageError: loader constraint violation: when resolving interface method
"org.openxmlformats.schemas.officeDocument.x2006.extendedProperties.PropertiesDocument.copy()Lorg/apache/xmlbeans/XmlObject;"
the class loader (instance of org/apache/catalina/loader/WebappClassLoader)
of the current class,
org/apache/poi/POIXMLProperties,
and the class loader (instance of org/apache/catalina/loader/StandardClassLoader)
for resolved class, org/openxmlformats/schemas/officeDocument/x2006/extendedProperties/PropertiesDocument,
have different Class objects for the type ts.schemas.officeDocument.x2006.extendedProperties.PropertiesDocument.copy()Lorg/apache/xmlbeans/XmlObject;
used in the signature
I am using POI 3.15. The POI JARs are stored in the common Tomcat7/lib folder. (poi-3.15.jar, poi-ooxml-3.15.jar, poi-ooxml-schemas-3.15.jar, xmlbeans-2.6.0.jar)
Can anyone help me explain what the conflict is here and how to resolve it?

In the WEB-INF lib I had put pretty much all relevant JAR files. It seems that xmlbeans (and possibly more) caused the conflict when they went along with the WAR afterwards.
But when cleaning up so I only had poi-3.15.jar and poi-ooxml-3.15.jar in the WEB-INF/lib, I got a situation where both the Eclise IDE and the TomCat runtime worked.

multiple classloader issue on weblogic

exception loader constraint violation: when resolving method "org.apache.hadoop.io.IOUtils.cleanup(Lorg/apache/commons/logging/Log;[Ljava/io/Closeable;)V" the class loader (instance of weblogic/utils/classloaders/ChangeAwareClassLoader) of the current class, org/apache/hadoop/hdfs/FileInputStreamCache, and the class loader (instance of weblogic/utils/classloaders/GenericClassLoader) for resolved class, org/apache/hadoop/io/IOUtils, have different Class objects for the type org/apache/commons/logging/Log used in the signature
I get this when using the application, not when deploying the ear. If i understand this wright, i have two classloaders that have two different logging objects? how can i refer them both to one?
EDIT 1
After further investigation, I think this error is due to the fact we are using a common jars library and there is another version of commons there, making them conflict. Is there any way to specify weblogic to use a specific library and not packaging? I would like to investigate this further

You need to check the WEB-INF/lib directory of your WAR files to ensure that they do not contain jars that also exist in the EAR/lib directory.
Web apps will always use the WEB-INF/lib classes before looking for them in the EAR file. However, classes loaded from the EAR will only see the other classes in the EAR. If you have the same (or similar) jar file in both places this will lead to class loading issues such as you describe.

You will have to update your weblogic-application.xml to use the use the prefer-application-package to tell the web logic to use the jar from the web-inf/lib instead of the jar from web logic. refer to the link Weblogic 10.3.5 Overriding Spring Version
<weblogic-application>
<prefer-application-packages>
<package-name>org.apache.*</package-name>
<package-name>org.springframework.*</package-name>
</prefer-application-packages>
</weblogic-application>

Classloaders and sharing .jar files with Apache Tomcat

If I have classes that need to be shared between my webapp and Tomcat (e.g. a custom realm and principal), where should the .jar file containing those classes go?
Currently I'm putting the .jar in ${CATALINA_HOME}/lib. This result is a ClassCastException when assigning references from classes of the same type. Here's an example:
MyCustomPrincipal principal = (MyCustomPrincipal)FacesContext.getCurrentInstance().getExternalContext().getUserPrincipal();
The method above throws a ClassCastException. The method returns an actual MyCustomPrincipal type (since that's what my custom realm gave Tomcat when it performed authentication) that, apparently, was created by a different classloader. How do I fix this so both Tomcat and my webapp can use MyCustomPrincipal?
http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html
Any help is appreciated.
Andrew

It looks like you have 2 copies loaded, once in tomcat and once in your WEB-INF/lib jars or other classpath of your deployed application.
The reason you get classpath exception lies in the way a WAR looks for classes. Contrary to the normal Java rules, a war first looks inside the war for a class and only then passes the request to teh parent classloader.
A class's identity is dependent of the classloader and the same class loaded in 1 classloader will generate a classcast exception when it is casted in the other classloader.
The solution is to make sure that the war does not contain the classes which should be provided by the container. If you use maven you can mark these dependencies as 'provided', if you use ant, you have to split your classpath list in 2 and compile against both, but use only the ones you need for constructing the war.