We have a portal application with one Main web app context and many minor web app contexts - plugins. Currently (very simplified) the Main one has own spring libraries and plugins would have to have them also if they wanted to use spring. In common/shared tomcat context there are just drivers and interfaces.
Would it work if spring libraries were moved to common context in regards to other libraries that spring might indirectly use or they might use spring ? Like hibernate, because the apps are using spring-tx etc. Would hibernate have to move to common/shared context too ?
What do you think, what are the other aspects ? From spring application context point of view it would be much easier like this.
#RichW is correct in stating that placing Spring libraries in Tomcat's common classloader is bad practice. And there's a good chance it won't work.
Java uses a classloader hierarchy). When a class load is requested, the classloader will recursively request the class from it's parent classloader before attempting to load the class using it's own classpath. This process continues up to the root classloader (know as the bootstrap classloader). In this way, classes referenced from a parent classloader always get priority over classes referenced in classloaders further down the hierarchy.
It's important to note that in this process classes are never loaded from a child classloader. Therefore any classes required by Spring would also need to be loaded into the common classloader - including asm, log4j, commons-logging and cglib (all of which spring depends on). This will lead to a whole host of problems: in particular, including commons-logging in the common classpath is a whole world of hurt
If you actually managed to get Tomcat started, then you would experience problems with memory leaks when recycling applications. In tomcat, applications are unloaded using conventional garbage collection, so if anything holds a reference to a class inside an application which has subsequently been restarted, that application will not get garbage collection. Spring and logging frameworks are prime candidates for holding references to classes so you will probably suffer from OOM errors after a few application restarts.
The only way to do this safely would be to consider using a full blown application server (such as JBoss AS) and deploy your application as an EAR.
If you were able to move from Tomcat to a full-blown Java EE container then an option would be to package everything as an EAR using the Bundled Optional Classes mechanism.
You'd then move the common JARs out of the WARs & into the top level of the EAR.
Yes, I know it's tempting. Yes, it can work. But putting application-specific or framework-specific libraries in the shared libraries folder of an app server is considered by some to be a bad practice, and I agree.
In my opinion web-apps should contain their own dependencies (app jars, framework jars, etc.). Frameworks also have dependencies, often requiring multiple jars with particular versions. Sometimes these versions change, sometimes the dependencies change. Over time that shared library folder will become a kitchen sink for jars, and that will affect all your apps, perhaps in unpredictable ways.
Going the shared library folder route you gain some slight initial convenience, but what you lose is choice: the choice to only affect one web-app at a time. I recommend you keep your jars within your web-app, nicely contained and separate from the other web-apps. It will make them more reliable and you'll find framework upgrades easier to handle. You'll be happier in the long run, I promise you.
Related
How can I cache server-wide (with cache scope spanning multiple WARs on this server) instances of classes from a JAR which is contained binary-identical in several WARs on a web container (server, e. g. Tomcat)?
<EDIT> I want to cache application data across WARs because the data is common to them. (It's a portal project, where it can be useful to share common data across different "views" implemented as different portlets deployed as different WARs, and using a Java object cache is much faster and more simple than using a central data-holding service.) </EDIT>
Is that possible at all? Or is it required to put such a JAR on a path accessed by a common parent classloader, like in /lib/ext ?
See: Java, Classpath, Classloading => Multiple Versions of the same jar/project
See: How does class loading work when the same class exists in different applications on the same server?
See: cast across classloader?
See: What is a serialVersionUID and why should I use it?
Yes, the best option is to put the classes in a class loader that is a parent of the two applications. If by lib/ext you mean JAVA_HOME/lib/ext, then I would not recommend that. Instead, you should put them in CATALINA_HOME/lib directory. See the Shared Library Files section of the documentation, which links to the Class Loader HOW-TO documentation.
You can add common classes (jars) to the shared.loader property in conf/catalina.properties. Those classes are available to all web apps but not tomcat itself.
If you implement a cache around a static singleton, then you would be able to access the objects from different web apps. I don't know if that is best practice however. For example it makes it hard to scale because it makes it impossible to load balance the apps onto many servers.
The answer seems to be "it depends".
If the JAR(s) (or classes) in question do not have dependencies conflicting with other components also deployed on the server, both proposed solutions (CATALINA_HOME/lib/ext/ and CATALINA_HOME/conf/catalina.properties :: shared.loader) should plainly work. Thus both are "correct answers" and I cannot see which one is "more correct" than the other.
However I missed a crucial detail when I first asked the question (but this does not invalidate it): In my case the JAR in question required Spring 4.2.9.RELEASE (and other dependencies), but other relevant WARs deployed on the same server contain and require Spring 3.0.7. (The objects to be cached do not depend on Spring, but the JAR was not designed with this problem in mind, and it also contains other related code depending on Spring which now would be very difficult to separate.)
Generally it should be possible to put into CATALINA_HOME/lib/ext/ what ever you want as long as all already deployed WARs contain everything they need: The "module first / parent last" class loading policy should prevent conflicts, even if (as in this example) Spring 4.2.9 is available to the parent classloader and Spring 3.0.7 is available to the WAR classloader. But it looks somewhat "unclean" and messy to me to mix-up things that way.
Therefore I decided to use the "to-be-cached" object's classloader hash code as the key in a map, in which the cached data are the values. Then all cached data is selected "by classloader" which automatically and transparently ensures assignment compatibility. If there is also another WAR deployed on the server which can change and thus invalidate the cached data, it can remove the whole map from the cache, forcing the "read-access" WARs to reload data on next access.
However this approach DOES NOT allow cross-WAR cacheing: Effectively every WAR will get its own private cache segment.
Another approach would be to deliberately transform all data to cache to/from e. g. JSON so as to get a "naturally global" data type like java.lang.String for the cached data. If chosen from the beginning of the project, to me this seems to be the cleanest way, but if there is already a complex (and generally working) implementation in place, this may cause some work to do.
Comments on this self-answer are welcome!
For an ongoing project, we are looking for a possibility to dynamically download and load jar files into a running application. Apart from downloading the files (which is probably fairly straightforward), I am unaware of any solution that would automatically add the jar's to the classpath, and do discovery of the annotations (like CDI beans).
Given such a system, it would be rather handy if the #Inject annotation would not throw a runtime failure of an implementation of a class is not present (because that module-jar was not loaded).
Is there currently any such system? Does spring or OSGi fit this need?
Any ideas how close project Jigsaw would come in trying to fulfill this on application level?
I think you need OSGI, using an OSGI container like Karaf : https://karaf.apache.org
In standard java provide ServiceLoader https://docs.oracle.com/javase/tutorial/ext/basics/spi.html
I advice you to not follow that path
It should be possible to dynamically load jar files without the usage of OGSI. The keyword are Classloaders especially when used with a proper hierarchy. The following answer should give you an idea: How should I load Jars dynamically at runtime? but keep in mind that this might cause serious security issues
You followed the path at 2. even I advice you not to do it. But now you end up in the scenario that the context of your used framework does not know this classes. You would have this problem with most IOC frameworks. Since they build up the context on startup. There are libraries for this created for development purpose (spring-loaded, spring dev tools, JRebel). If your IOC framework supports it go with it.
Regarding handling not available jars. The best point to do research on this is Spring Boot and its auto configuration mechanism. It checks if certain classes/jars (not sure to be honest) are available and add additional behavior for this cases. But still this is application startup solution and not a runtime IOC solution.
I find myself spending much to much time solving conflicts between versions of transitive dependencies. I whish it would be possible to load these using private embedded classloaders, so multiple versions of the same library could be (indirectly) used by different components. As most beans of a well-designed application depend only on a limited number of external libraries and data-centric model classes of the application, it seems to me that spring generated bean proxies could load bean implementation (and libraries used by it) in a child classloader.
I admit I never worked directly with a multi-classloader application (aside from a Java EE environment), so it is not obvious to me if and how such packaging would be possible. Are there any spring & maven/sbt plugins which allow it? Or maybe just good articles on creating multi-artifact, multi-classloader builds?
This may be a very rudimentary question, but please help me out if this is well-known and has been solved elsewhere.
I have a multi-war setup (all maven modules) say kilo-webapp1 and kilo-webapp2 as two WARs that I need to deploy on a Tomcat instance. These two webapps both use services from a common service jar, say kilo-common-services.jar. The kilo-common-services.jar has its own spring context that is loaded by the users of the jar viz. kilo-webapp1 and kilo-webapp2 in this case. It so happens that the initialization of the services in kilo-common-services takes a long time and hence I want it to happen only once (to ensure that the time it takes to bring up the instance is not very high) which also helps me to use it as a second level cache that it kept current in the JVM instance. To do this, we resorted to the following steps:
Modify the catalina.properties of CATALINA_BASE in tomcat to have shared.loader as ${catalina.base}/shared/lib
Copied the kilo-common-services.jar and all of its dependent jars to the CATALINA_BASE/shared/lib. [Manual step]
Copy spring related jars to the CATALINA_BASE/shared/lib location [Manual step]
Created a beanRefContext.xml file in kilo-common-services.jar. Define a new ClassPathXmlApplicationContext here, where the constructor was provided with the location to the spring context file for the common services.
Noted the dependency scope of kilo-common-services.jar and every other dependency (like Spring related jars) as provided in the kilo-webapp1 and kilo-webapp2 pom files. For Spring this is needed to ensure that the classpath scanning actions are not triggered twice. Also this causes different ClassCastExceptions (for log4j lets's say) if not excluded via the provided scope.
web.xml for kilo-webapp1 and kilo-webapp2 indicated that the parentContext for them is the servicesContext defined in kilo-common-services.jar.
I was able to verify that only one instance of the services of kilo-common-services exist, but the setup as you might have imagined is painful. If someone has best practices about such a setup in an IDE like Eclipse, would really appreciate it. My problems are as below:
#2 is becoming a challenge. I am currently running mvn dependency:copy-dependencies on kilo-common-services to copy dependent jars from target/dependency to the shared/lib which is a woefully manual step. Time and again, I forget to regenerate dependencies and have to do a redeploy again.
#3 is also not straight-forward as time and again there are newer common dependencies and we always have to remember to copy it to shared lib to avoid ClassCastExceptions
#5 is again a maintenance nightmare.
Also as time progresses, there will more such disparate common jars that need to be shared and it would involve pain for each of those jars. Feel free to critique the setup and propose a better one in its place that may be easy to use (from an IDE as well). Would be happy to provide any other details.
Thanks in advance!
The problem is that your architecture is broken (and that's why you're struggling with the solution). You have two solutions:
1) If you want to share a service that takes a long time (to initialise) between two war applications, make that a separate service completely and access it via rest or any kind of remoting.
2) Merge both webapps into one.
Having the common library is the shared lib folder is going to bring you lots of headaches, and you'll end up rolling it back.
My (personal) approach would be to merge both applications, but keep the packages separate enough and have separate spring configurations. In this way, at least you still keep the logic separation of both webapps.
Also since both run on the same container, there's little gain from having 2 separate wars (unless you're planning to move them to different containers very soon).
About the IDE, you can use the maven-cargo-plugin to start up a tomcat with several web applications with (almost) any configuration you want.
We are developing restful soa, with spring and tomcat and utilizing Domain Driven Design (well thats the plan anyway). There is migrationProject and a initial basic search service. Two separate WAR files, with two separate POMs. Both utilize the same Domain objects.
So I will have separate project that will be just the DomainObjects I will wrap them up into a jar, and then using maven and/or jenkins it will deploy automatically (whenever I configure (for example when pushed to a specific repository).
Having two copies of the same jar, sounds like a much worse idea to me. Its not your architecture that is broken, its your deployment and development process thats needs improvement, imho.
(my kind of related question).
Our long term plan is to have one project as the restful interface, with multiple Controllers that have service classes and repositories injected into them from their dependencies.
Question1:
As we know, when a classloader is about to load a class, it delegates the request to its parent classloader. However in Tomcat, it doesn’t: you could load your class to overwrite the same name class which is put in common lib directory. This means Tomcat WebappClassloader doesn’t follow delegating policy. Is it violation of convention?
Question2:
I wrote a class and put it in common lib directory, obviously the class is shared among web apps. For instance, every web app can read/write the static field of the class. Further, classes in JDK are loaded by Bootstrap classloader, then their static fields are shared by any web apps, is it dangerous?
This behavior is intentional and it allows you to override libraries provided in the Tomcat itself independently in every WAR. For instance you can override Log4J with different version per each application deployed to the container without introducing any issues or breaking other applications. From Tomcat documentation:
Like many server applications, Tomcat installs a variety of class loaders [...] to allow different portions of the container, and the web applications running on the container, to have access to different repositories of available classes and resources. This mechanism is used to provide the functionality defined in the Servlet Specification, version 2.4 — in particular, Sections 9.4 and 9.6.
It does violate the normal delegation algorithm, but this is how other application server work as well (JBoss for instance).
Ad. question 2: Yes, it is dangerous, you have to remember about synchronization and have no control over who modifies this variable. I would avoid static fields altogether.
For instance EhCache allows you to share CacheManager. This is implemented via net.sf.ehcache.CacheManager#singleton static volatile field. Now you get all sort of problems: if you put ehcache.jar in Tomcat's /lib, it will work as expected. However if each web application has its own copy of the JAR file, sharing will not work because each web app has its own copy of CacheManager class. It gets even worse when only one application has its own ehcache.jar - all applications will share the same instance of CachedManager except the one having ehcache.jar packaged together. Such error are very hard to track down...