We Keep Coding

Websphere: Shared libraries in common classloader earlier on classpath than application modules, even with parent last policy

Background:
I have the following problem: I have several WAR files I need to have deployed on same Websphere server. The WAR files use libraries that depend on having a specific version of XMLSec regisered as the XML Signature Provider (with the Java Security class). Currently I bundle this library with each WAR file (since the WAR files also need to work standalone and on Tomcat's without any special shared library configuration etc.). Each WAR files registers the provider with Security.addProvider() in a ServerContextListener. But this causes problems in the multi-WAR setup, because if one WAR file does the registration with Security.addProvider) and another WAR files tries to fetch it using the XMLSignatureFactory class (which is actually a javax.* class contained inside the XMLSec JAR itself, but which ultimately calls back to the global provider list configured with Security.addProvider), then it causes a ClassCastException inside XMLSignatureFactory, because this class does a cast of what it gets from Security into to its own version of the provider classes, which doesn't work. The exact stack trace is as follows:
Caused by: java.lang.ClassCastException:
org.apache.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory
incompatible with javax.xml.crypto.dsig.XMLSignatureFactory at
javax.xml.crypto.dsig.XMLSignatureFactory.findInstance(XMLSignatureFactory.java:202)
at
javax.xml.crypto.dsig.XMLSignatureFactory.getInstance(XMLSignatureFactory.java:292)
By the way this is not a case of conflict with different versions of XMLSec being in play or conflicts with Websphere's own version. There is only one version albeit it is loaded from different WAR's.
Of course the solution is to have the xmlsec library loaded with a common classloader so that there is only one version of the classes loaded that all WAR files see, which would avoid ClassCastExceptions etc.. But here is the rub: I also need to have each application loaded with the "parent last" policy - or rather, I need the JAR files inside each application to take precedence over Websphere's built-in version of the libraries (for instance Axis2 that I also include in the WAR filesetc.). Furter, I would prefer that I can keep the xmlsec library in each WAR files' WEB-INF/lib folder, so that the WAR files can still work stand-alone (i.e. in other environments which might not have the shared library configured etc.).
So basically I want to have a common class loader loading the XMLSec library, say, somewhere from disk. Let's denote that [SHARED XMLSEC]. Then I want each application classpath to ultimately appear like this:
App1: [SHARED XMLSEC][App1 WEB-inf/lib][Websphere libraries][JDK libraries]
App2: [SHARED XMLSEC][App2 WEB-inf/lib][Websphere libraries][JDK libraries]
etc.
In such a configuration it doesn't matter if App1+App2 themselves contain the XMLSec library since the shared one will take precedence so they will use the common one. At the same time, App1+App2 are still free to override other built-in Websphere libraries (Axis2).
Is it possible to realize this configuration and what options do I need to set? Do you see alternative ways to achieve the same objective?

Since you have a conflict between classes here, I would suggest going for isolated class loaders for each application. On the server side, setting the class loader policy to 'Multiple' should provide isolation between applications.
Once you have this set, configure class loading at the application level to the 'Parent last' configuration for both the applications.
The following Knowledge Center link has the relevant instructions [Steps 2,3 & 4 under the 'Procedure' section] :
http://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/trun_classload.html
[Note: The version of WAS in use is not specified in the question. The Knowledge Center link refers to version 8.5.5.]

Tomcat classloading doesn't seem to behave as documented

I'm running tomcat 6.0.23, and according to the classloader documentation webapps should look for classes in this order:
Bootstrap
System
WEB-INF/classes
WEB-INF/lib
Common
I have a webapp that uses hibernate and the hibernate jars are in it's WEB-INF/lib directory. When running on it's own, all works fine.
I also have a jar file that needs to sit in the tomcat/lib directory as it contains some classes that need to be loaded on startup (an object factory and the objects it creates). These classes use toplink for their JPA implementation and here is where I get a problem.
I need to put the toplink jars where they can be accessed on tomcat startup, so I've put them in the tomcat/lib directory. According to the order of classloading listed above, when the webapp that uses hibernate wants the hibernate implementation classes, it should find them in it's WEB-INF/lib directory, but what actually happens is it finds the toplink implementation classes from the tomcat/lib directory, and I get a class cast exception.
Can anyone please explain why my webapp class loaders aren't finding what they need in their WEB-INF/lib directory, or suggest a way to debug the classpath at runtime?
Thanks.

Be sure to read and understand the paragraph preceding the list you quoted:
"When a request to load a class from the web application's WebappX class loader is processed, this class loader will look in the local repositories first, instead of delegating before looking. There are exceptions. Classes which are part of the JRE base classes cannot be overriden. For some classes (such as the XML parser components in J2SE 1.4+), the J2SE 1.4 endorsed feature can be used."
What are these "hibernate implementation classes" that you're having problems with? (Hibernate's "implementation" classes would be completely different from Toplink's.) Are they javax.persistence classes? These may (or may not) fall under the category of "JRE base classes", which behave differently.
Edit: Based on your comment, this is just a typical cross-class-loader loading problem. Tomcat's class loading is working exactly as you expect it to. If you look into the JPA classes where this initialization is going on, you'll find a line like this:
Enumeration<URL> resources =
cl.getResources("META-INF/services/" + PersistenceProvider.class.getName());
That loads all PersistenceProviders from all ClassLoaders, including your Toplink one in the lib directory. It then immediately does this:
for ( PersistenceProvider provider : providers ) { ...
That's on line 77 of javax.persistence.Persistence, where your exception is coming from. That's because the PersistenceProvider class referred to on that line is from your webapp class loader, but the collection contains two instances: your Hibernate implementation from the same class loader and the Toplink one from a different class loader.
This global, static initialization is one major thing that's kept me from shifting to JPA. I still just use straight Hibernate because of problems like this.

Deploying 2 war files with common classes in JBoss

I have two war file app1.war and app2.war deployed in a single JBoss instance. Package names for java classes for both war files starts with com.myapp
To add further, there are some Classes that are common between the two apps while there are some that have same fully qualified class names but are different (Source Code has changed).
I want to know, if this could pose threat of any kind to the deployment scenario?

You could get class loading problems if your applications are not isolated, i.e. have their own class loading repository and class loaders. If you configure JBoss to isolate the applications from each other you should be fine (I don't know what is the default for your version but 4.2.3 that we use does not isolate apps by default).
To clarify that a bit:
If you have two classes with different implementations but the same FQCN you could get the wrong class from the class loader for the application that is loaded second. Even if the implementation was the same you could get class cast exceptions or other strange behavior if one app gets the class from the other app.

I had a similar situation with multiple apps.Look at my solution here

Best way is to isolate class loading for your application archives.
For JBoss 5.1.0 GA following worked for me.
Create jboss-classloading.xml file in WEB-INF folder.
Added following lines to this file
Here,
export-all="NON_EMPTY" => Makes sure the classes loaded for this app is not exported
import-all="true" => Imports and uses all of the class definition available.
parent-first="false" => If more than one class with same name is found, one defined under the application will be used first.
FYI. This also helped me embedding the log configuration of log4j in the application war file. Will need to place log4j.xml in WEB-INF/classes and have a log4j.jar in WEB-INF/lib folder.

There will be one class loader instance for each application or standalone module. In other words, classes in app1.war will be loaded in different class loader than the classes in app2.war. This is the default behavior of any Java EE server; So it really doesn't matter about having classes with the same package/names and/or different content. This is the default behavior of any Java EE server.
Having said that, if you tweak the class loader policy of the server or try to load classes (reflect) using anything other than Thread.currentThread().getContextClassLoader(), you could be asking for trouble.

What is the default classpath for EJBs?

Please forgive my pitiful knowledge of Java EJBs but, when an EJB is deployed to an application server as a .jar file, where do things like Hibernate and log4j first look for their configuration files (hibernate.cfg.xml and log4j.properties) in the .jar file?

(...) when an EJB is deployed to an application server as a .jar file, where do things like Hibernate and log4j first look for their configuration files (hibernate.cfg.xml and log4j.properties) in the .jar file?
This depends on the implementation of the tool and is unrelated to the fact that you are using EJBs. For Hibernate, the documentation writes:
3.7. XML configuration file
An alternative approach to
configuration is to specify a full
configuration in a file named
hibernate.cfg.xml. This file can be
used as a replacement for the
hibernate.properties file or, if both
are present, to override properties.
The XML configuration file is by
default expected to be in the root of
your CLASSPATH.
Regarding Log4J, the procedure is described below:
Default Initialization Procedure
The log4j library does not make any
assumptions about its environment. In
particular, there are no default log4j
appenders. Under certain well-defined
circumstances however, the static
inializer of the Logger class will
attempt to automatically configure
log4j. The Java language guarantees
that the static initializer of a class
is called once and only once during
the loading of a class into memory. It
is important to remember that
different classloaders may load
distinct copies of the same class.
These copies of the same class are
considered as totally unrelated by the
JVM.
The default initialization is very
useful in environments where the exact
entry point to the application depends
on the runtime environment. For
example, the same application can be
used as a stand-alone application, as
an applet, or as a servlet under the
control of a web-server.
The exact default initialization
algorithm is defined as follows:
Setting the log4j.defaultInitOverride system property to any other value then
"false" will cause log4j to skip the
default initialization procedure (this
procedure).
Set the resource string variable to the value of the
log4j.configuration system property. The preferred way to
specify the default initialization
file is through the
log4j.configuration system property. In case the system property
log4j.configuration is not defined, then set the string variable
resource to its default value
"log4j.properties".
Attempt to convert the resource variable to a URL.
If the resource variable cannot be converted to a URL, for example due to
a MalformedURLException, then search
for the resource from the classpath by
calling
org.apache.log4j.helpers.Loader.getResource(resource,
Logger.class) which returns a URL.
Note that the string
"log4j.properties" constitutes a
malformed URL. See
Loader.getResource(java.lang.String)
for the list of searched locations.
If no URL could not be found, abort default initialization. Otherwise,
configure log4j from the URL. The
PropertyConfigurator will be used to
parse the URL to configure log4j
unless the URL ends with the ".xml"
extension, in which case the
DOMConfigurator will be used. You
can optionaly specify a custom
configurator. The value of the
log4j.configuratorClass system property is taken as the fully
qualified class name of your custom
configurator. The custom configurator
you specify must implement the
Configurator interface.
To summarize, if you put both files at the root of your EJB-JAR, they should be found.
Regarding the title of your question, I suggest to read Packaging EJB 3 Applications that I'm quoting below:
Dependencies between Java EE modules
Unfortunately, no Java EE
specification provides a standard for
class loading, and each application
server implements class loaders in
whatever way seems best to the vendor.
However, Java EE defines the
visibility and sharing of classes
between different modules, and we can
depict the dependency between
different modules as shown in figure
4.
As illustrated in figure 4, the EAR
class loader loads all JARs in the lib
directory that is shared between
multiple modules. Typically a single
EJB class loader loads all EJB
packaged in all EJB-JAR modules. The
EJB class loader is often the child of
the application class loader, and
loads all EJB classes. Because the EJB
is a child to the EAR class loader,
all classes loaded at the> EAR level
will be visible to the EJBs.
(source: developer.com)
Figure 4: Illustration of class
visibility of an EAR file containing
multiple web modules, EJBs, and shared
library modules. The EAR class loader
loads the classes in the JARs packaged
as library modules, and all classes
loaded by the EAR class loader are
visible to the EJBs. The classes
loaded by EJB class loader are
typically visible to the web module in
most containers because the WAR class
loader is a child of the EJB class
loader.

I think Log4j would look in more than one place for log4j.properties file. Anyway, all configuration files in an ejb-jar go inside the META-INF directory.

Controlling the classpath in a servlet

My servlet application includes a number of library .jars, some of which contain embedded log4j.xml or log4j.properties files. I'd like to ensure that log4j finds my log4j.xml first! I've tried searching for some specification of the priorities of the various classpath elements in a servlet (e.g. does WEB-INF/classes always precede WEB-INF/lib?), or some way to configure or tweak the servlet's classloader so that a given resource directory appears early in the classpath. So far, I've drawn a blank. Any suggestions on ensuring that a servlet .war file loads the correct log4j.xml via the classloader?

Tomcat 8.5
Ditto Tomcat 8.0.
See documentation: Class Loader HOW-TO.
Tomcat 8.0
The answer is simple, taken from the Tomcat documentation page, Class Loader HOW-TO. In particular notice the use of the /WEB-INF/ directory/folder.
Therefore, from the perspective of a web application, class or resource loading looks in the following repositories, in this order:
Bootstrap classes of your JVM
/WEB-INF/classes of your web application
/WEB-INF/lib/*.jar of your web application
System class loader classes (described above)
Common class loader classes (described above)
If the web application class loader is configured with <Loader delegate="true"/> then the order becomes:
Bootstrap classes of your JVM
System class loader classes (described above)
Common class loader classes (described above)
/WEB-INF/classes of your web application
/WEB-INF/lib/*.jar of your web application
Tomcat 6
Excerpted from Tomcat 6 page, Class Loader HOW-TO.
Therefore, from the perspective of a web application, class or resource loading looks in the following repositories, in this order:
Bootstrap classes of your JVM
System class loader classes (described above)
/WEB-INF/classes of your web application
/WEB-INF/lib/*.jar of your web application
$CATALINA_HOME/lib
$CATALINA_HOME/lib/*.jar

As far as I understand the resource selection from the classpath is non-deterministic (from the point of view of the app developer). Even if the same file is loaded consistently the behaviour could change:
1. When you upgrade the version of your current container.
2. If you switch containers.
The simplest solution will be to remove embedded log4j config files from library jars. It is almost never a good idea to embed log4j config's as it leads to the problem you are seeing here...
Are they third party jars or jars you developed?

We the Spring Log4jConfigListener in our web.xml file.
You can specify as a context parameter the location of the log4j config file, i.e. you could set it as /WEB-INF/log4j.xml
Would this be an option for you? If you're not using Spring I know that you can set the Log4j location programatically which might also work.

In my experience, WEB-INF/classes typically takes precedence over jars in WEB-INF/lib, however, that also depends on the servlet container you use (I could never figure out the behavior of JRun, for instance). It would help immensely if you could tell me which container you're using.
Also, are you certain that the offending log4j configuration is in a jar in WEB-INF/lib? Typically, when I've run into classpath problems in a servlet container situation, it's because of libraries that reside outside of the web app.
The servlet specs recommend that web app classloaders load their own classes before delegating to the container's classloader (SRV.9.7.2), but since this is counter to the Java spec, not all vendors do this by default (in fact Tomcat is the only container I've used that does this by default). With that said, it's always possible to configure your container's web app classloading behavior. If you tell me which container you're using, I may be able to help you (specifically, I have done this successfully before on WebLogic, WebSphere, Glassfish and JRun)).

If you're unable to control the classpath, since Tomcat is setting it for you, are you at least able to set a system property for log4j.configuration? I believe that location pointed to by that property can be set outside of the classpath.
If not, another approach, although an ugly one, would be to explicitly run one of the configurators yourself in your application code.

You need to have log4j.properties in your CLASSPATH. The best place is under WEB-INF/classes.
You also have to make sure that you use your version of log4j.jar. So, put it in WEB-INF/lib, just to make sure you are not using one from tomcat folders, since it may cause strange classloading issues.