We Keep Coding

Websphere: Shared libraries in common classloader earlier on classpath than application modules, even with parent last policy

Background:
I have the following problem: I have several WAR files I need to have deployed on same Websphere server. The WAR files use libraries that depend on having a specific version of XMLSec regisered as the XML Signature Provider (with the Java Security class). Currently I bundle this library with each WAR file (since the WAR files also need to work standalone and on Tomcat's without any special shared library configuration etc.). Each WAR files registers the provider with Security.addProvider() in a ServerContextListener. But this causes problems in the multi-WAR setup, because if one WAR file does the registration with Security.addProvider) and another WAR files tries to fetch it using the XMLSignatureFactory class (which is actually a javax.* class contained inside the XMLSec JAR itself, but which ultimately calls back to the global provider list configured with Security.addProvider), then it causes a ClassCastException inside XMLSignatureFactory, because this class does a cast of what it gets from Security into to its own version of the provider classes, which doesn't work. The exact stack trace is as follows:
Caused by: java.lang.ClassCastException:
org.apache.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory
incompatible with javax.xml.crypto.dsig.XMLSignatureFactory at
javax.xml.crypto.dsig.XMLSignatureFactory.findInstance(XMLSignatureFactory.java:202)
at
javax.xml.crypto.dsig.XMLSignatureFactory.getInstance(XMLSignatureFactory.java:292)
By the way this is not a case of conflict with different versions of XMLSec being in play or conflicts with Websphere's own version. There is only one version albeit it is loaded from different WAR's.
Of course the solution is to have the xmlsec library loaded with a common classloader so that there is only one version of the classes loaded that all WAR files see, which would avoid ClassCastExceptions etc.. But here is the rub: I also need to have each application loaded with the "parent last" policy - or rather, I need the JAR files inside each application to take precedence over Websphere's built-in version of the libraries (for instance Axis2 that I also include in the WAR filesetc.). Furter, I would prefer that I can keep the xmlsec library in each WAR files' WEB-INF/lib folder, so that the WAR files can still work stand-alone (i.e. in other environments which might not have the shared library configured etc.).
So basically I want to have a common class loader loading the XMLSec library, say, somewhere from disk. Let's denote that [SHARED XMLSEC]. Then I want each application classpath to ultimately appear like this:
App1: [SHARED XMLSEC][App1 WEB-inf/lib][Websphere libraries][JDK libraries]
App2: [SHARED XMLSEC][App2 WEB-inf/lib][Websphere libraries][JDK libraries]
etc.
In such a configuration it doesn't matter if App1+App2 themselves contain the XMLSec library since the shared one will take precedence so they will use the common one. At the same time, App1+App2 are still free to override other built-in Websphere libraries (Axis2).
Is it possible to realize this configuration and what options do I need to set? Do you see alternative ways to achieve the same objective?

Since you have a conflict between classes here, I would suggest going for isolated class loaders for each application. On the server side, setting the class loader policy to 'Multiple' should provide isolation between applications.
Once you have this set, configure class loading at the application level to the 'Parent last' configuration for both the applications.
The following Knowledge Center link has the relevant instructions [Steps 2,3 & 4 under the 'Procedure' section] :
http://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/trun_classload.html
[Note: The version of WAS in use is not specified in the question. The Knowledge Center link refers to version 8.5.5.]

multiple classloader issue on weblogic

exception loader constraint violation: when resolving method "org.apache.hadoop.io.IOUtils.cleanup(Lorg/apache/commons/logging/Log;[Ljava/io/Closeable;)V" the class loader (instance of weblogic/utils/classloaders/ChangeAwareClassLoader) of the current class, org/apache/hadoop/hdfs/FileInputStreamCache, and the class loader (instance of weblogic/utils/classloaders/GenericClassLoader) for resolved class, org/apache/hadoop/io/IOUtils, have different Class objects for the type org/apache/commons/logging/Log used in the signature
I get this when using the application, not when deploying the ear. If i understand this wright, i have two classloaders that have two different logging objects? how can i refer them both to one?
EDIT 1
After further investigation, I think this error is due to the fact we are using a common jars library and there is another version of commons there, making them conflict. Is there any way to specify weblogic to use a specific library and not packaging? I would like to investigate this further

You need to check the WEB-INF/lib directory of your WAR files to ensure that they do not contain jars that also exist in the EAR/lib directory.
Web apps will always use the WEB-INF/lib classes before looking for them in the EAR file. However, classes loaded from the EAR will only see the other classes in the EAR. If you have the same (or similar) jar file in both places this will lead to class loading issues such as you describe.

You will have to update your weblogic-application.xml to use the use the prefer-application-package to tell the web logic to use the jar from the web-inf/lib instead of the jar from web logic. refer to the link Weblogic 10.3.5 Overriding Spring Version
<weblogic-application>
<prefer-application-packages>
<package-name>org.apache.*</package-name>
<package-name>org.springframework.*</package-name>
</prefer-application-packages>
</weblogic-application>

Tomcat classloading doesn't seem to behave as documented

I'm running tomcat 6.0.23, and according to the classloader documentation webapps should look for classes in this order:
Bootstrap
System
WEB-INF/classes
WEB-INF/lib
Common
I have a webapp that uses hibernate and the hibernate jars are in it's WEB-INF/lib directory. When running on it's own, all works fine.
I also have a jar file that needs to sit in the tomcat/lib directory as it contains some classes that need to be loaded on startup (an object factory and the objects it creates). These classes use toplink for their JPA implementation and here is where I get a problem.
I need to put the toplink jars where they can be accessed on tomcat startup, so I've put them in the tomcat/lib directory. According to the order of classloading listed above, when the webapp that uses hibernate wants the hibernate implementation classes, it should find them in it's WEB-INF/lib directory, but what actually happens is it finds the toplink implementation classes from the tomcat/lib directory, and I get a class cast exception.
Can anyone please explain why my webapp class loaders aren't finding what they need in their WEB-INF/lib directory, or suggest a way to debug the classpath at runtime?
Thanks.

Be sure to read and understand the paragraph preceding the list you quoted:
"When a request to load a class from the web application's WebappX class loader is processed, this class loader will look in the local repositories first, instead of delegating before looking. There are exceptions. Classes which are part of the JRE base classes cannot be overriden. For some classes (such as the XML parser components in J2SE 1.4+), the J2SE 1.4 endorsed feature can be used."
What are these "hibernate implementation classes" that you're having problems with? (Hibernate's "implementation" classes would be completely different from Toplink's.) Are they javax.persistence classes? These may (or may not) fall under the category of "JRE base classes", which behave differently.
Edit: Based on your comment, this is just a typical cross-class-loader loading problem. Tomcat's class loading is working exactly as you expect it to. If you look into the JPA classes where this initialization is going on, you'll find a line like this:
Enumeration<URL> resources =
cl.getResources("META-INF/services/" + PersistenceProvider.class.getName());
That loads all PersistenceProviders from all ClassLoaders, including your Toplink one in the lib directory. It then immediately does this:
for ( PersistenceProvider provider : providers ) { ...
That's on line 77 of javax.persistence.Persistence, where your exception is coming from. That's because the PersistenceProvider class referred to on that line is from your webapp class loader, but the collection contains two instances: your Hibernate implementation from the same class loader and the Toplink one from a different class loader.
This global, static initialization is one major thing that's kept me from shifting to JPA. I still just use straight Hibernate because of problems like this.

Deploying 2 war files with common classes in JBoss

I have two war file app1.war and app2.war deployed in a single JBoss instance. Package names for java classes for both war files starts with com.myapp
To add further, there are some Classes that are common between the two apps while there are some that have same fully qualified class names but are different (Source Code has changed).
I want to know, if this could pose threat of any kind to the deployment scenario?

You could get class loading problems if your applications are not isolated, i.e. have their own class loading repository and class loaders. If you configure JBoss to isolate the applications from each other you should be fine (I don't know what is the default for your version but 4.2.3 that we use does not isolate apps by default).
To clarify that a bit:
If you have two classes with different implementations but the same FQCN you could get the wrong class from the class loader for the application that is loaded second. Even if the implementation was the same you could get class cast exceptions or other strange behavior if one app gets the class from the other app.

I had a similar situation with multiple apps.Look at my solution here

Best way is to isolate class loading for your application archives.
For JBoss 5.1.0 GA following worked for me.
Create jboss-classloading.xml file in WEB-INF folder.
Added following lines to this file
Here,
export-all="NON_EMPTY" => Makes sure the classes loaded for this app is not exported
import-all="true" => Imports and uses all of the class definition available.
parent-first="false" => If more than one class with same name is found, one defined under the application will be used first.
FYI. This also helped me embedding the log configuration of log4j in the application war file. Will need to place log4j.xml in WEB-INF/classes and have a log4j.jar in WEB-INF/lib folder.

There will be one class loader instance for each application or standalone module. In other words, classes in app1.war will be loaded in different class loader than the classes in app2.war. This is the default behavior of any Java EE server; So it really doesn't matter about having classes with the same package/names and/or different content. This is the default behavior of any Java EE server.
Having said that, if you tweak the class loader policy of the server or try to load classes (reflect) using anything other than Thread.currentThread().getContextClassLoader(), you could be asking for trouble.

What is the default classpath for EJBs?

Please forgive my pitiful knowledge of Java EJBs but, when an EJB is deployed to an application server as a .jar file, where do things like Hibernate and log4j first look for their configuration files (hibernate.cfg.xml and log4j.properties) in the .jar file?

(...) when an EJB is deployed to an application server as a .jar file, where do things like Hibernate and log4j first look for their configuration files (hibernate.cfg.xml and log4j.properties) in the .jar file?
This depends on the implementation of the tool and is unrelated to the fact that you are using EJBs. For Hibernate, the documentation writes:
3.7. XML configuration file
An alternative approach to
configuration is to specify a full
configuration in a file named
hibernate.cfg.xml. This file can be
used as a replacement for the
hibernate.properties file or, if both
are present, to override properties.
The XML configuration file is by
default expected to be in the root of
your CLASSPATH.
Regarding Log4J, the procedure is described below:
Default Initialization Procedure
The log4j library does not make any
assumptions about its environment. In
particular, there are no default log4j
appenders. Under certain well-defined
circumstances however, the static
inializer of the Logger class will
attempt to automatically configure
log4j. The Java language guarantees
that the static initializer of a class
is called once and only once during
the loading of a class into memory. It
is important to remember that
different classloaders may load
distinct copies of the same class.
These copies of the same class are
considered as totally unrelated by the
JVM.
The default initialization is very
useful in environments where the exact
entry point to the application depends
on the runtime environment. For
example, the same application can be
used as a stand-alone application, as
an applet, or as a servlet under the
control of a web-server.
The exact default initialization
algorithm is defined as follows:
Setting the log4j.defaultInitOverride system property to any other value then
"false" will cause log4j to skip the
default initialization procedure (this
procedure).
Set the resource string variable to the value of the
log4j.configuration system property. The preferred way to
specify the default initialization
file is through the
log4j.configuration system property. In case the system property
log4j.configuration is not defined, then set the string variable
resource to its default value
"log4j.properties".
Attempt to convert the resource variable to a URL.
If the resource variable cannot be converted to a URL, for example due to
a MalformedURLException, then search
for the resource from the classpath by
calling
org.apache.log4j.helpers.Loader.getResource(resource,
Logger.class) which returns a URL.
Note that the string
"log4j.properties" constitutes a
malformed URL. See
Loader.getResource(java.lang.String)
for the list of searched locations.
If no URL could not be found, abort default initialization. Otherwise,
configure log4j from the URL. The
PropertyConfigurator will be used to
parse the URL to configure log4j
unless the URL ends with the ".xml"
extension, in which case the
DOMConfigurator will be used. You
can optionaly specify a custom
configurator. The value of the
log4j.configuratorClass system property is taken as the fully
qualified class name of your custom
configurator. The custom configurator
you specify must implement the
Configurator interface.
To summarize, if you put both files at the root of your EJB-JAR, they should be found.
Regarding the title of your question, I suggest to read Packaging EJB 3 Applications that I'm quoting below:
Dependencies between Java EE modules
Unfortunately, no Java EE
specification provides a standard for
class loading, and each application
server implements class loaders in
whatever way seems best to the vendor.
However, Java EE defines the
visibility and sharing of classes
between different modules, and we can
depict the dependency between
different modules as shown in figure
4.
As illustrated in figure 4, the EAR
class loader loads all JARs in the lib
directory that is shared between
multiple modules. Typically a single
EJB class loader loads all EJB
packaged in all EJB-JAR modules. The
EJB class loader is often the child of
the application class loader, and
loads all EJB classes. Because the EJB
is a child to the EAR class loader,
all classes loaded at the> EAR level
will be visible to the EJBs.
(source: developer.com)
Figure 4: Illustration of class
visibility of an EAR file containing
multiple web modules, EJBs, and shared
library modules. The EAR class loader
loads the classes in the JARs packaged
as library modules, and all classes
loaded by the EAR class loader are
visible to the EJBs. The classes
loaded by EJB class loader are
typically visible to the web module in
most containers because the WAR class
loader is a child of the EJB class
loader.

I think Log4j would look in more than one place for log4j.properties file. Anyway, all configuration files in an ejb-jar go inside the META-INF directory.