We Keep Coding

How should I build a modularized enterprise application which must use Spring and JPA?

I'm planning to build a Java-based system to handle different business processes where each of these is a particular module in the system. Most modules would depend on some of the other modules to handle their particular business process. In other words, top modules would consume some sort of basic services provided by underlying modules. Some modules will be developed from the very beginning, but some will be added to the system later. Next, some modules will expose RESTful interfaces to handle external input / output.
To handle all this, OSGi seems appropriate, but it's a bit difficult to learn with all the different "distributions" out there (Equinox, Felix, etc.) and I'm concerned about the ease of using the Spring framework and other 3rd party libraries within each module (starting with Spring 3.2 the different jars might not come with OSGi manifests).
On top of this, I'd like a central web portal to administer all bundles, thus with each new bundle there will be a new admin section.

that's why we developed osgi-less modularity for Spring https://github.com/griddynamics/banshun Your feedback is appreciated!

Why do you need OSGi? Why not use a Web Server like Tomcat, and deploy your application as a war? You can deploy it on multiple servers in a cluster, and your application can scale on and on.

Why do you need Spring? It has become incredibly coupled. And it has a complexity that find quite useless since OSGi applications tend to be built from small components communicating through services; voiding most of the advantages of the Spring wiring model which assumes it is central.
And hard to configure is a strange remark, OSGi is excellent configuration support. It is just different than what you're used to.

Instead of using spring, why not using OSGi Blueprint it'll give you an "easy" transition from Spring to OSGi.

Modular web application with Spring and Maven

I'm trying to design the architecture of a medium-sized web application in Java and I would like to get some advice on how to do it.
The project consists on a base website plus a number of modules. For instance, one module would provide user registration, another module would offer a web service, and so on...
Whenever I need to deliver the application to a new customer, the ideal thing would be to pick up the modules he wants, do some theming (css, images, maybe jsp) and developing the custom modules he may need, if any.
I've taken a look to maven multi module projects, war overlays, but I find it difficult to partition the application, especially in regard to the modules' configuration (for example, merging a global spring configuration from the modules). Can somebody point me to an example of such a system? Thanks in advance!

merging spring configuration is easy. In each module, package up a spring context file in it's /WEB-INF/classes directory. When you overlay, all classes and resources in WEB-INF classes in the dependency will be put into WEB-INF/classes in your app. (ps, this also works if you package as .jar instead, but you won't be able to overlay .jsp files if you do)
Then it's just a matter of importing them. This is best done by using a set pattern to find the files. Here's an example:
<import resource="classpath*:/module/*-context.xml" />
This will import all the classpath resources that match this pattern.
Annotation based example:
#Configuration
#ImportResource(value={"classpath*:/module/*-context.xml"})
public class MyConfiguration { ... }
It's the web.xml configuration that will cause you more trouble than anything if you have the need to do any web.xml customizations in modules. You could use servlet 3.0 for this of course, but it requires the right server to deploy on.

fwiw, after some experience with plain Spring import we developed tiny framework for osgi-less modularity for Spring. The first problem with import is the bean name clashes you can not have same-named singleton in two contexts, and many other hassles.. tbc https://github.com/griddynamics/banshun
--
Mike

Eclipse project layout classpath issues

I have a large scale project I am working on at the moment using Eclipse. Normally, as a one man team, these problems would not be an issue, but as our team is not one person we need to be able to break up pieces of the project to be worked on by certain team members.
In simplicity, let's say I have two layers to be separated apart:
1. Each DAO is a separate Java project, to be worked upon individually
2. The web-tier service layer contains all of our service endpoints and must be able to reference all of the DAOs. This layer runs on Tomcat as a dynamic web project, and utilizes Adobe LiveCycle Data Services as the piece that handles creation and management of endpoints.
Now, the issue we are running into is that when we create a DAO and unit test it individually it runs great. But when we reference it into our service project and try to run it we begin to get all kinds of issues related to the fact that we have two different versions of certain jars referenced in and as such we begin to have errors when running the server.
As a result, we know we can solve the issue by pulling the problem jars and ensuring that this is not an issue again in the future, but as I said before this is a large scale project with multiple people working on it and we don't want to be spending our time weeding out dependency issues when under the gun.
We are looking for recommendations on where to proceed for alternative solutions? Our team is new to JavaEE and as such we don't have much of a bearing on what we can use to tie everything together in it, or if it is a viable solution. Should we be looking at turning our DAOs into EJBs and deploying them in an EAR library? If so, where would our service layer lie, and would the service layer be able to reference the DAO classes since the EJB maintains it's own classpath (from what we have read?) Are we looking down the wrong path, or are we completely wrong in our current understanding of JavaEE?
Any assistance is greatly appreciated. We are still in the framework stage of this project and we want to be sure that we will be able to maintain it in the long run.

I second the Maven recommendation. That can add all sorts of sanity to your project structure.
Maven can even generate Eclipse workspaces via mvn eclipse:eclipse
An important clarification on the EJBs note. As of ava EE 6 is you no longer need to separate EJBs from Servlets and can use them together in the very same jar in the war file.
So understand from that that using EJBs or not no longer has any impact on packaging or classloaders as it once did. These are now separate decisions. EARs and classloader separation should now be viewed as a feature you might want to use should you want classloader separation and the complexity it brings. Most applications simply do not need that and are more than fine with just a war file containing servlets, ejbs, jpa entities, cdi beans, jaxrs services and whatever else you need. You are free to decide how you want to separate them or if you want to bother separating them at all.
EJBs do make great DAOs due to transaction management, something you don't get from plain Tomcat but can be made available in Tomcat via TomEE and works fine in Eclipse. You should consider EJBs for that reason, not for dependency reasons.
Side note, as you're new to Java EE, you might find this helpful:
http://openejb.apache.org/examples-trunk/index.html

In order to have things organized when working with Java EE in teams of 1+ people I could suggest:
Use Maven to manage your build process and library dependencies.
Maven has a small learning curve, but once you grasp it you will be grateful. By using Maven you no longer depends on Eclipse to manage your classpath.
A thing about it that I think is really helpful when working in teams is the install feature. Suppose you are woking on the version 1.0 of an EJB module, say core-ejb-module-1.0, and you've got it to a stable state and want everyone working in the project to refer to it from now on.
You then run a maven command like this on it: mvn clean package install
Maven will clean this module, compile it, run tests, create the jar and then install it to a repository that you define. Could be any computer in your company.
Now you may tell the guys working on other projects to update this dependency version on their .pom file and in the next build they run, before compiling, maven will download this library and then use it. Really neat. No more classpath hell.
(There are other ways to always automatically refer to the latest library as stated in this post, but there are some caveats. Anyway it's just an example.)
Use JPA/EJB instead of DAO Pattern.
Some people say DAO meaning any sort of data access, others really mean that they use the DAO Pattern to access objects. If that is your case, you no longer need to use it when using JPA. (At least for most common scenarios).
In my case, I have a generic EntityService which is capable of doing CRUD operations on any Entity and has a centralized query management. Then every EJB's that should perform database related operations may inject this guy and do its job.
As a suggestion, with Maven, you project could be organized as such:
core project structure
core (The pom root)
core-ejb-module (Includes all generic EJB's, like the EntityService for instance.)
core-jpa-module (Includes all JPA generic definitions, like Interfaces, MappedSuperclasses and such.)
core-jsf-module (Includes all JSF generic definitions, like abstract controllers, generic converters and wrappers for FacesContext, etc..)
Now that you have a core generic module setup, you could create:
an application structure
app (The pom root)
app-ear-module (Includes all other modules in this application. Shared jars goes in the ear /lib folder, so all other modules could reference to them.)
app-ejb-module-a (Includes EJB's for the business layer of your application. It uses the core-ejb-module)
app-ejb-module-b (You may have lots of ejb modules. You may even have a project that contains only ejb modules. Other apps will declare their dependency on them via Maven.)
app-jpa-module (Contains definitions for JPA Entities that represents you database tables. Depends on the core-jpa-module)
app-web-module (Holds the pages, Controllers and Converters for this application.)
I think you got the idea. Things tend to be loosely coupled and you may organize your projects as you like.
This is just a simple example to illustrate. I didn't explained a lot about Maven but if you're interested I think it will help you indeed.
I hope it gives you some ideas and may help you in any way.
[]'s

If you can run all the sub-components using the same set of dependencies, you may find it helpful to migrate to a Maven build.
With Maven, you can define a top-level project that manages all the 3rd party dependency versions in one place, so all modules are built, tested and deployed against the same library versions. You are also likely to find Maven a good fit for the multi-module approach you have adopted, as it ensures that a project is rebuilt correctly if one of its dependencies changes.
You would still be able to use dynamic web projects as before; Eclipse will automatically deploy the DAOs as part of the service project (IIRC you need to characterise the DAOs as utility modules).
If you do go down the EJB root, you are correct that each EAR will get its own class-loader, and can therefore use a varying set of dependencies. However, in your position I would tend to look at improving your dependency management first - it'll probably be cheaper and easier.

How to modularize a JSF/Facelets/Spring application with OSGi?

I'm working with very large JSF/Facelets applications which use Spring for DI/bean management.
My applications have modular structure and I'm currently looking for approaches to standardize the modularization.
My goal is to compose a web application from a number of modules (possibly depending on each other). Each module may contain the following:
Classes;
Static resources (images, CSS, scripts);
Facelet templates;
Managed beans - Spring application contexts, with request, session and application-scoped beans (alternative is JSF managed beans);
Servlet API stuff - servlets, filters, listeners (this is optional).
What I'd like to avoid (almost at all costs) is the need to copy or extract module resources (like Facelets templates) to the WAR or to extend the web.xml for module's servlets, filters, etc. It must be enough to add the module (JAR, bundle, artifact, ...) to the web application (WEB-INF/lib, bundles, plugins, ...) to extend the web application with this module.
Currently I solve this task with a custom modularization solution which is heavily based on using classpath resources:
Special resources servlet serves static resources from classpath resources (JARs).
Special Facelets resource resolver allows loading Facelet templates from classpath resources.
Spring loads application contexts via the pattern classpath*:com/acme/foo/module/applicationContext.xml - this loads application contexts defined in module JARs.
Finally, a pair of delegating servlets and filters delegate request processing to the servlets and filters configured in Spring application contexts from modules.
Last days I read a lot about OSGi and I was considering, how (and if) I could use OSGi as a standardized modularization approach. I was thinking about how individual tasks could be solved with OSGi:
Static resources - OSGi bundles which want to export static resources register a ResourceLoader instances with the bundle context. A central ResourceServlet uses these resource loaders to load resources from bundles.
Facelet templates - similar to above, a central ResourceResolver uses services registered by bundles.
Managed beans - I have no idea how to use an expression like #{myBean.property} if myBean is defined in one of the bundles.
Servlet API stuff - use something like WebExtender/Pax Web to register servlets, filters and so on.
My questions are:
Am I inventing a bicycle here? Are there standard solutions for that? I've found a mentioning of Spring Slices but could not find much documentation about it.
Do you think OSGi is the right technology for the described task?
Is my sketch of OSGI application more or less correct?
How should managed beans (especially request/session scope) be handled?
I'd be generally grateful for your comments.

What you're aiming to do sounds doable, with a few caveats:
The View Layer: First, your view layer sounds a little overstuffed. There are other ways to modularize JSF components by using custom components that will avoid the headaches involved with trying to create something as dramatic as late-binding managed beans.
The Modules Themselves: Second, your modules don't seem particularly modular. Your first bullet-list makes it sound as if you're trying to create interoperable web apps, rather than modules per se. My idea of a module is that each component has a well-defined, and more or less discrete, purpose. Like how ex underlies vi. If you're going down the OSGi route, then we should define modular like this: Modular, for the sake of this discussion, means that components are hot-swappable -- that is, they can be added and removed without breaking the app.
Dependencies: I'm a little concerned by your description of the modules as "possibly depending on each other." You probably (I hope) already know this, but your dependencies ought to form a directed acyclic graph. Once you introduce a circular dependency, you're asking for a world of hurt in terms of the app's eventual maintainability. One of the biggest weaknesses of OSGi is that it doesn't prevent circular dependencies, so it's up to you to enforce this. Otherwise your dependencies will grow like kudzu and gradually choke the rest of your system's ecosystem.
Servlets: Fuhgeddaboudit. You can't late-bind servlets into a web app, not until the Servlet 3.0 spec is in production (as Pascal pointed out). To launch a separate utility servlet, you'll need to put it into its own app.
OK, so much for the caveats. Let's think about how this might work:
You've defined your own JSF module to do... what, exactly? Let's give it a defined, fairly trivial purpose: a login screen. So you create your login screen, late-bind it using OSGi into your app and... then what? How does the app know the login functionality is there, if you haven't defined it in your .jspx page? How does the app know to navigate to something it can't know is there?
There are ways to get around this using conditional includes and the like (e.g., <c:if #{loginBean.notEmpty}>), but, like you said, things get a little hairy when your managed loginBean exists in another module that may not have even been introduced to the app yet. In fact, you'll get a servlet exception unless that loginBean exists. So what do you do?
You define an API in one of your modules. All the managed beans that you intend to share between modules must be specified as interfaces in this API layer. And all your modules must have default implementations of any of these interfaces that they intend to use. And this API must be shared between all interoperable modules. Then you can use OSGi and Spring to wire together the specified beans with their implementation.
I need to take a moment to point out that this is not how I would approach this problem. Not at all. Given something like as simple as a login page, or even as complicated as a stock chart, I'd personally prefer to create a custom JSF component. But if the requirement is "I want my managed beans to be modular (i.e., hot-swappable, etc)," this is the only way I know to make it work. And I'm not even entirely sure it will work. This email exchange suggests that it's a problem that JSF developers have only just started to work on.
I normally consider managed beans to be part of the view layer, and as such I use them only for view logic, and delegate everything else to the service layer. Making managed beans late-binding is, to my mind, promoting them out of the view layer and into the business logic. There's a reason why all those tutorials are so focused on services: because most of the time you want to consider what it would take for your app to run "headless," and how easy it would be to "skin" your view if, for instance, you wanted it to run, with all its functionality, on an Android phone.
But it sounds like a lot of what you're working with is itself view logic -- for instance, the need to swap in a different view template. OSGi/Spring should be able to help, but you'll need something in your app to choose between available implementations: pretty much what OSGi's Service Registry was built to do.
That leaves static resources. You can modularize these, but remember, you'll need to define an interface to retrieve these resources, and you'll need to provide a default implementation so your app doesn't choke if they're absent. If i18n is a consideration, this could be a good way to go. If you wanted to be really adventurous, then you could push your static resources into JNDI. That would make them completely hot-swappable, and save you the pain of trying to resolve which implementation to use programmatically, but there are some downsides: any failed lookup will cause your app to throw a NamingException. And it's overkill. JNDI is normally used in web apps for app configuration.
As for your remaining questions:
Am I inventing a bicycle here? Are there standard solutions for that?
You are, a little. I've seen apps that do this kind of thing, but you seem to have stumbled into a fairly unique set of requirements.
Do you think OSGi is the right technology for the described task?
If you need the modules to be hot-swappable, then your choices are OSGi and the lighter-weight ServiceLocator interface.
Is my sketch of OSGI application more or less correct?
I can't really tell without knowing more about where your component boundaries are. At the moment, it sounds like you may be pushing OSGi to do more than it is capable of doing.
But don't take my word for it. I found other reading material in these places.
And since you ask about Spring Slices, this should be enough to get you started. You'll need a Git client, and it looks like you'll be training yourself on the app by looking through the source code. And it's very early prototype code.

I am facing the same problems in my current project. In my opinion, OSGi is the best and cleanest solution in terms of standards and future support, but currently you may hit some problems if you try using it in a web application:
there is no well integrated solution between a Web Container and the OSGi platform yet.
OSGi may be too much for a custom build web application that is just searching for a simple modularized architecture. I would consider OSGi if my project needs to support third party extensions that are not 100% under our control, if the project needs hot redeployments, strict access rules between plugins, etc.
A custom solution based on class loaders and resource filters seems very appropriate for me.
As an example you can study the Hudson source code or Java Plug-in Framework (JPF) Project(http://jpf.sourceforge.net/).
As about extending the web.xml, we may be lucky with the Servlet 3.0 specification(http://today.java.net/pub/a/today/2008/10/14/introduction-to-servlet-3.html#pluggability-and-extensibility).

The "web module deployment descriptor fragment" (aka web-fragment.xml) introduced by the Servlet 3.0 specification would be nice here. The specification defines it as:
A web fragment is a logical
partitioning of the web app in such a
way that the frameworks being used
within the web app can define all the
artifacts without asking devlopers to
edit or add information in the
web.xml.
Java EE 6 is maybe not an option for you right now though. Still, it would to be the standardized solution.

Enterprise OSGi is a fairly new domain so dont think you will get a solution that directly satisfies your need. That said one of the things I found missing from Equinox (osgi engine behind eclipse and hence one with largest user base!) is a consistent configuration / DI service. In my project recently we had some similar needs and ended building a simple configuration osgi service.
One of the problems which will be inherent to modular applications would be around DI, as the module visibility could prevent class access in some cases. We got around this using a registered-buddy policy, which is not too ideal but works.
Other than configuration, you can take a look at the recently released Equinox book for guidance on using OSGi as base for creating modular applications. The examples may be specific to Equinox, but the principles would work with any OSGi framework. Link - http://equinoxosgi.org/

You should look into Spring DM Server (it's being transitioned to Eclipse Virgo but that's not been released yet). There's a lot of good things in the recent OSGi enterprise spec which has also just been released.
Some of the Spring DM tutorials will help, I'd imagine. But yes, it's possible to have both resources and classes loaded from outside the web bundle using standard modularity. In that, it's a good fit.
As for the session context - it gets handled as you would expect in a session. However, you might run into problems with sharing that session between web bundles to the extent that in not sure if it's even possible.
You could also look to have a single web bundle and then use e.g. the Eclipse extension registry to extend the capabilities of you web app.

Separate web.xml for development and production

My web.xml is different in devel and production environments. For example in development environment there is no need in security constraints.
Typically I deploy new application version as follows:
Export Eclipse project to WAR.
Upload WAR to the server.
Redeploy.
The problem is that I have to manually uncomment security constraints in web.xml before exporting.
How do you solve this problem?
I also met an opinion in some articles that "web.xml is rarely changed". But how can web.xml not change if it is exported to WAR on every update?
Thanks in advance!

If you can't use the same web.xml during development, I would automate the build process, use two web.xml and bundle the "right" one at build time depending on the targeted environment as Brian suggested. But, instead of Ant, I'd choose Maven because it will require less work IMHO and it has a built-in feature called profiles that is perfect to manage environment specific stuff like here.
In other words, I'd put the build under Maven 2 and use a production profile containing a specific maven-war-plugin configuration to build a WAR containing a web.xml having the required security constraints. Another option would be to merge the development web.xml (cargo can do that) to add the security-constraints but this is already a bit more "advanced" solution (a bit more complex to put in place).

I would create a development and production deployment with different web.xml configs. Automate the building/maintenance of these via your build (Ant/Maven etc.) to keep control of the common elements required.
I had to solve this problem many times in the past, and ended up writing XMLTask - an Ant plugin which allows the modification of XML files without using normal text replacement (it's a lot clever than that) and without having to mess with XSLTs (it's a lot simpler than that). If you follow the above approach you may want to check this out. Here's an article I wrote about it.

Assuming that you're stuck with the idea of web.xml changing before deployment to production, then my likely approach would be to run the development web.xml through a simple XSL transform which "decorated" the web.xml with your production-only elements, such as security constraints. Assuming that you can hook this step into your build process, then the production-ready web.xml should appear during your export process.
However, it is generally a good idea not to have different web.xml across environments, it devalues your testing. Having the same value in all environments will reduce the risk of bugs appearing only in your production environment.

I converted my project to be built using ant. The starting point was just this build.xml http://tomcat.apache.org/tomcat-6.0-doc/appdev/build.xml.txt
The above build doesn't have the feature of copying in a different web.xml(based on e.g. a property set when building), but you'll learn how to do that when get a bit into ant, should be pretty easy.
As a nice side effect, deploying to a remote tomcat is now just a couple of clicks away from within Eclipse instead of Export->war and manually copying it to the server.

I would add the necessary infrastructure to allow a mechanical build, with ant or maven.
When THAT is done you can have your mechanical build create two targets, one for test and one for production.
You should however, strongly consider testing the same code as you have in production. You will be bitten otherwise.

I believe having a single war that works in multiple environments is a superior solution than baking a new one with a profile option per dev, qual, and prod. It is super annoying there is not a better mechanism to get an environment variables directly in web.xml without using a library like spring.
One solution to web.xml environment configuration given that your environment customization is related to filter init params, such as:
<filter>
<filter-name>CAS Filter</filter-name>
<filter-class>edu.yale.its.tp.cas.client.filter.CASFilter</filter-class>
<init-param>
<param-name>edu.yale.its.tp.cas.client.filter.loginUrl</param-name>
<param-value>https://<foo>:8443/login</param-value>
...
The particular filter class referenced above (CASFilter) is public. This means you can extend it will a custom adapter that adds in your environment configuration. This allows you to stay out of that nasty web.xml file.