We Keep Coding

JBoss Modules outside of JBoss AS

I had first found a reference to JBoss Modules when I stumbled upon Ceylon language which uses JBoss Modules as its module system. Immediately I wanted to try this system in some toy project and maybe even embed it in a real project (I was writing a project with plugins support at that time), but I couldn't find any documentation on JBoss Modules as a standalone library. The only available documentation source seems to be the official wiki, but it looks abandoned and unsupported. I couldn't even find Javadocs for it (except, maybe, this, but it seems to be very old and not really related to JBoss Modules due to "osgi" presence in the link).
It seems that JBoss Modules are usable outside of JBoss AS because Ceylon language uses it, but lack of almost any documentation on the subject is disappointing.
So, here are my questions:
Is it possible to use JBoss Modules as a standalone library at all? Are there any artifacts in some public Maven repository?
If it is (and there are), is there any documentation on it? That wiki I have mentioned does not have, for example, any instructions on embedding JBoss Modules.

If you'd like to try out JBoss Modules directly, you can grab the dependencies from the JBoss Nexus repository: https://repository.jboss.org/nexus/content/repositories/public/org/jboss/modules/jboss-modules/
Unfortunately, there isn't much documentation on JBoss Modules, but if you want to try it out, you probably don't want to be hand-writing modules.xml files yourself (maybe you like pain, I don't know.)
If you'd like to try out "Furnace" the modular container based on JBoss Modules and Maven that serves as the core module system for JBoss Forge, it give you the ability to write Maven projects that can be loaded directly as Modules. This is what we are using for our entire Forge 2 architecture.
You can find some docs on Furnace here:
https://github.com/forge/furnace#furnace
https://github.com/forge/core#developing-an-addon
Note that Furnace Addons require a maven classifier, you can choose the classifier used if you want to. This is done via the Furnace Manager (which can be seen in the furnace docs above.)

Yes. Actually, JBoss is using it that way as well - so JBoss application server is actually running inside JBoss Modules system.
I'm not aware of such documentation, but usually you shouldn't be embedding jboss modules, but running the applications with it. I'm not aware if you can embed it.
I actually got most information from this presentation in vimeo, Modular Class Loading with JBoss Modules. There seems to also be Zen of Modules video there.

Spring Project without annotations in IntelliJ

I am using IntelliJ IDE which by default generates a annotation based SpringApplication when I create a new Project for Spring Application. I want to first learn the concepts of Spring Application without Annotations(XML based). Could anyone give me suggestion for how to set in the IDE such that I get a non-annotation based project by default when I select for New Project.
The current version of Spring is
3.2.0

I created a basic annotation based Spring 3.2.0 app here : https://github.com/ajorpheus/hello-spring/
If you are comfortable with Git/Github, then clone that repository locally and import that project into Intellij using File -> Import Project ( otherwise, you could download the zip of the project itself : https://github.com/ajorpheus/hello-spring/archive/master.zip and then go from there. No git-fu needed.)
This project runs fine in Intellij as a maven project.
I have taken the liberty of also interpreting your question as 'How to learn Spring'. IMHO, here is what helped me when I was starting out with Spring.
First Steps
To start with, for further reading or a bit of experimentation, these seem like a good start :
http://projects.spring.io/spring-framework/#quick-start
http://spring.io/guides/gs/maven/
Books / Resources
I also recall that reading 'Spring In Action - Third Edition' helped when I was starting out with Spring, though I would not recommend buying it now since a much more up-to-date Fourth Edition is on it's way.
A couple of threads here also cover the topic of 'Best Resources for learning Spring' :
Book suggestion for Spring framework
https://stackoverflow.com/questions/1610733/best-way-to-learn-spring-framework
You'll find some books/resources repeatedly being mentioned is threads such as the above. 'Spring Recipes', 'Spring in Action', 'Pro Spring', 'Official Spring Documentation' ... all of these 'spring' to mind :D.
Finally, as far as Spring is concerned, there is no dearth of resources or threads advising about these resources. I'd recommend not spending a lot of time researching where to start. Just a pick a book which has been mentioned in more than one threads and go for it !
Hope this helps !

Efficient OSGi development workflow

I work on a product composed of many bundles running as features on top of karaf. Typically our developers work on one bundle at a time. Our normal development goes something like: code, compile, copy bundle to deploy folder, test. We've also found that hotdeploy just refuses to override certain bundles that are installed as features without a server restart or a feature uninstall/reinstall, so sometimes the cycle is longer.
My question is: does anyone in the community have a better way? The way we do things works, but I feel like it's pretty slow and inefficient and I'm betting someone has come up with something better!
EDIT: I realize that I was pretty unclear in my question... We are using Equinox underneath Karaf. We also use Eclipse and Maven, although I don't know that using Maven is relevant.

Sounds like you want the dev:watch command. From the documentation:
The watch command can be used to help at developement time. It allows you to configure a set of URLs that will be monitored. All bundles location matching the given URL will be
automatically updated. This avoids the need for manually updating the bundles or even copying the bundle to the system folder if needed. Note that only maven based urls and maven snapshots will actually be updated automatically, so if you run
dev:watch *
It will actually monitor all bundles that have a location matching mvn:* that have '-SNAPSHOT' in their url.
Doing "dev:watch --help" from the Karaf shell will list its available flags and args.
Something similar is the PAX plugin
Either of these will work quite nicely if you've got the m2 maven plugin for Eclipse.
UPDATED: In my company we strive to be as TDD as possible, therefore a lot a development is done without explicitly starting Karaf. In the normal mix of unit tests we're also using Pax Exam, which is largely fantastic even when run from within Eclipse =)
This helps ensure we're not too tided to any Karaf specifics as it runs with Equinox/Felix/Concierge (so I mock out various Karaf specifics we depend on like JAAS authentication). Along with many other cool tools/functionality, it's capable of provisioning Karaf features and using TinyBundles you can even create bundles on the fly (again useful for mocking/stubbing).
Pax Exam hooks into the JUnit framework by providing a JUnit #Runner, the latest version (2) is much faster and has DSL based API, so the tests are quite concise and readable.
Using Pax Exam gives us good test coverage and short development times. Where tests are less practical or we're hunting bugs that don't surface in tests, the dev:watch command is invaluable.
In summary; IMO you should definitely drive your developments with tests (Pax Exam will slot into your existing build nicely and once you get used to it you'll find development quicker). You can start using the dev:watch command immediately, it will certainly speed up your current situation.
UPDATE 2: In answering another question I've added a maven example Pax-Exam testing a ComponentFactory. Test Driven Development is arguably the most efficient workflow available to developers today. link to question: osgi: Using ServiceFactories? and link to sourcecode: http://dl.dropbox.com/u/2465717/net.earcam.example.servicecomponent_2011-08-16_15-52.tgz

I've had excellent results using Equinox in Eclipse - even hot code replace works properly. granted, the target platform is small and we have only on the order of approx 50 bundles of our own, but workflow goes like this:
First, we have a target platform that contains all third-party and Eclipse bundles, Eclipse takes care of downloading & managing them. Then, the workspace has all the bundles of the project, grouped in 3-4 working sets. Compilation happens as usual on save, sometimes GWT needs to be recompiled, but even then the changes get picked up immediately because no deployment needs to happen - the running Equinox system uses the unpacked project folders as bundles. Running this from within Eclipse gives us hot code replace, on-the-fly changing template files, only MANIFEST.MF/plugin.xml changes need to refresh the bundle - and even then it's usually faster to just restart the framework than to type in the console.

if you use Eclipse Eclipse Libra may be useful for you. Libra can start Felix, Equinox and Knopflerfish inside Eclipse as any other server with WST. They have some youtube videos how to use it.
I also wrote some tools that can help:
An osgi bundle that picks up OSGI services that match the filter (osgitest=junit4). With that you do not write Junit classes but you can provide pre-configured objects (e.g. with OSGI Blueprint). JUnit than runs based on the annotations provided in the interface your service implements.
A maven plugin that has the following useful goals
Start a OSGI containers and deploy the bundle maven project with all of it's dependencies (which are OSGI bundles of course). The OSGI container starting is done with the help of PAX Exam but the JUnit tests are started with the help of the OSGI bundle I wrote (that runs the OSGI services you may provide).
Create a folder that contains a shortcut to all dependencies of the project (located at the maven repo or target directory of the folder)
If the projects are deployed onto the server (Eclipse Libra) I have to say only update X where the X is the id of the bundle and everything is refreshed rapidly. You do not have to re-compile the projects that are published to the server if you run Equinox in Libra as it points to the target classes folder which is refreshed as soon as you save your class or pom.xml.
If you do not publish your project onto the server but add it as a bundle in the container pointing to the shortcut folder you can also run the update command on the OSGi console after running mvn install (without the restarting of the server).
A step-by-step guide is available at http://cookbook.everit.org/
With the following method above it is possible to write tests as TDD tests and run them as part of a maven compile on the CI server.
I hope you will find these tools as useful as I do!

It depends on the platform under Karaf: Felix or Equinox.
Equinox
Eclipse has excellent (or almost excellent) support for launching Equinox with bundles of your choice. The two things you need to prepare are:
Bundles, being developed, available in the workspace as Plug-in projects
Target platform, containing the remaining bundles of the application
Such setup will allow you to easily make changes to your bundles, even runtime and easily restarting the runtime when this is required. I see Karaf as more suitable when you are developing on remote system, where the bundles are deployed via SSH or FTP or when you are using external build tools like Maven, which have ability to automatically copy the bundle in the runtime after it is built.
If you are using Equinox, this will give some extra edge over as the runtime will execute the code directly from the workspace.
Felix
Felix doesn't seem to have such support for launching from Eclipse (although there is a work toward this, tracked in this Jira issue). You can also launch it as normal Java application, but this is far from convenient. In this case, using Maven will be much better alternative. You can still setup Eclipse to take full advantage of the PDE other features, only launching will be done externally.
Summary
In summary, you can always automate everything through Maven and Karaf will greatly help you in this regard. Eclipse will give a little edge, if you are using Equinox. You should be able to have hot-code replace regardless of the method you are using, because the hot-code replace doesn't even consider OSGi at all (except in the only case, when you reload your bundle and fresh class loader is created).

Spring builder is slow in Eclipse - can I speed it up?

When building our Java applications in Eclipse, the Spring builder is very slow and gives no status updates.
Specifically, I start building a project, and Eclipse's Progress pane displays
Invoking 'Spring Project Builder' on 'project name'...
for multiple minutes at a time, with no additional details.
I've already turned off the Spring AOP Reference Model Builder, and I just recently disabled the Spring project builder completely out of desperation.
I'm just building and using these projects, not developing them, so theoretically they should compile fine - but this is our development branch, so I'd still like to keep Spring on in case there's a nasty reflection error somewhere.
So, in order to keep using them, is there anything I can do to:
Speed the Spring portion of the build?
Display more detailed output during the Spring project building process?
Edit 2010-02-15 21:39 GMT:
I'm specifically referring to the Spring IDE plugin in Eclipse.

I'm assuming you're referring either to the Spring IDE plugin for Eclipse, or the SpringSource ToolSuite bundle.
The big performance killer that I've nailed down is the processing of <import resource="..."/> entries in the beans files. The plugin has an option for enabling the processing of these, and if turned on, it absolutely hammers performance - it searches the entire classpath (including libraries) for each imported resource, ever ytime something changes. I reported this as a bug, and thankfully it's been fixed, but not yet released.
The <import> support is just a nice-to-have, though, since you can manually add the imported files directly. Turning it off makes the whole experience much more edifying.

Try checking your validators. I remember having some issues at one point because I had a bunch of plugins installed which added a number of validators to my project and the build process took forever mostly because of checking all the XML.

How to modularize a (large) Java App?

I have a rather large (several MLOC) application at hand that I'd like to split up into more maintainable separate parts. Currently the product is comprised of about 40 Eclipse projects, many of them having inter-dependencies. This alone makes a continuous build system unfeasible, because it would have to rebuild very much with each checkin.
Is there a "best practice" way of how to
identify parts that can immediately be separated
document inter-dependencies visually
untangle the existing code
handle "patches" we need to apply to libraries (currently handled by putting them in the classpath before the actual library)
If there are (free/open) tools to support this, I'd appreciate pointers.
Even though I do not have any experience with Maven it seems like it forces a very modular design. I wonder now whether this is something that can be retrofitted iteratively or if a project that was to use it would have to be layouted with modularity in mind right from the start.
Edit 2009-07-10
We are in the process of splitting out some core modules using Apache Ant/Ivy. Really helpful and well designed tool, not imposing as much on you as maven does.
I wrote down some more general details and personal opinion about why we are doing that on my blog - too long to post here and maybe not interesting to everyone, so follow at your own discretion: www.danielschneller.com

Using OSGi could be a good fit for you. It would allow to create modules out of the application. You can also organize dependencies in a better way. If you define your interfaces between the different modules correctly, then you can use continuous integration as you only have to rebuild the module that you affected on check-in.
The mechanisms provided by OSGi will help you untangle the existing code. Because of the way the classloading works, it also helps you handle the patches in an easier way.
Some concepts of OSGi that seem to be a good match for you, as shown from wikipedia:
The framework is conceptually divided into the following areas:
Bundles - Bundles are normal jar components with extra manifest headers.
Services - The services layer connects bundles in a dynamic way by offering a publish-find-bind model for plain old Java objects(POJO).
Services Registry - The API for management services (ServiceRegistration, ServiceTracker and ServiceReference).
Life-Cycle - The API for life cycle management (install, start, stop, update, and uninstall bundles).
Modules - The layer that defines encapsulation and declaration of dependencies (how a bundle can import and export code).
Security - The layer that handles the security aspects by limiting bundle functionality to pre-defined capabilities.

First: good luck & good coffee. You'll need both.
I once had a similiar problem. Legacy code with awful circular dependencies, even between classes from different packages like org.example.pkg1.A depends on org.example.pk2.B and vice versa.
I started with maven2 and fresh eclipse projects. First I tried to identify the most common functionalities (logging layer, common interfaces, common services) and created maven projects. Each time I was happy with a part, I deployed the library to the central nexus repository so that it was almost immediately available for other projects.
So I slowly worked up through the layers. maven2 handled the dependencies and the m2eclipse plugin provided a helpful dependency view. BTW - it's usually not too difficult to convert an eclipse project into a maven project. m2eclipse can do it for you and you just have to create a few new folders (like src/main/java) and adjust the build path for source folders. Takes just a minute or two. But expect more difficulties, if your project is an eclipse plugin or rcp application and you want maven not only to manage artifacts but also to build and deploy the application.
To opinion, eclipse, maven and nexus (or any other maven repository manager) are a good basis to start. You're lucky, if you have a good documentation of the system architecture and this architecture is really implemented ;)

I had a similar experience in a small code base (40 kloc). There are no °rules":
compiled with and without a "module" in order to see it's usage
I started from "leaf modules", modules without other dependencies
I handled cyclic dependencies (this is a very error-prone task)
with maven there is a great deal with documentation (reports) that can be deployed
in your CI process
with maven you can always see what uses what both in the site both in netbeans (with a
very nice directed graph)
with maven you can import library code in your codebase, apply source patches and
compile with your products (sometimes this is very easy sometimes it is very
difficult)
Check also Dependency Analyzer:
(source: javalobby.org)
Netbeans:
(source: zimmer428.net)

Maven is painful to migrate to for an existing system. However it can cope with 100+ module projects without much difficulty.

The first thing you need to decide is what infra-structure you will move to. Should it be a lot of independently maintained modules (which translates to individual Eclipse projects) or will you consider it a single chunk of code which is versioned and deployed as a whole. The first is well suited for migrating to a Maven like build environment - the latter for having all the source code in at once.
In any case you WILL need a continuous integration system running. Your first task is to make the code base build automatically, so you can let your CI system watch over your source repository and rebuild it whenyou change things. I decided for a non-Maven approach here, and we focus on having an easy Eclipse environment so I created a build enviornment using ant4eclipse and Team ProjectSet files (which we use anyway).
The next step would be getting rid of the circular dependencies - this will make your build simpler, get rid of Eclipse warnings, and eventually allow you to get to the "checkout, compile once, run" stage. This might take a while :-( When you migrate methods and classes, do not MOVE them, but extract or delegate them and leave their old name lying around and mark them deprecated. This will separate your untangeling with your refactoring, and allow code "outside" your project to still work with the code inside your project.
You WILL benefit from a source repository which allows for moving files, and keeping history. CVS is very weak in this regard.

I wouldn't recommend Maven for a legacy source code base. It could give you many headaches just trying to adapt everything to work with it.
I suppose what you need is to do an architectural layout of your project. A tool might help, but the most important part is to organize a logical view of the modules.

It's not free but Structure101 will give you as good as you will get in terms of tool support for hitting all your bullet points. But for the record I'm biased, so you might want to check out SonarJ and Lattix too. ;-)