How to deploy OSGi apps and dependencies? - java

OSGi seems to have an excellent benefit of having small deployable artifacts by not wrapping dozens of JAR dependencies into a lib directory. However, I can't find anything that tells me an easy, reliable way to deploy dependencies to a container. For instance, I have an application that uses CXF and several Spring subprojects. If I need to deploy this application to a new Glassfish server, what would be the best way to do so, ensuring that all dependencies get installed?
I'm using Maven, and it would seem that there could be some way to have a hook that looks at the META-INF/maven directory and pulls the dependency list from the pom.xml and goes and fetches the required libs (probably from a local repo). Is there a way to do that?
The Pax plugin sort of sounds like it's doing this, but it seems to be based around boostrapping a Felix container? Which is not what I want, I am dealing with an already running, remote container.
Is there any shot such a thing exists as command line tool as opposed to GUI as well?

There are a number of ways to deploy dependent bundles to OSGi containers. Here are some of them:
1 The Felix OBR bundle repository
You first need to create an XML index file for your available bundles, using a tool such as bindex. If you are using the maven-bundle-plugin, then it automatically maintains an OBR index in ~/.m2/repository/repository.xml.
Load the index using the OBR command-line interface:
> obr:addUrl file:/Users/derek/.m2/repository/repository.xml
Then ask OBR to deploy your target bundle, with dependencies determined from the OBR index:
> obr:deploy com.paremus.posh.sshd
Target resource(s):
-------------------
Paremus Posh Ssh Daemon (1.0.23.SNAPSHOT)
Required resource(s):
---------------------
Paremus Command API (1.0.23.SNAPSHOT)
Optional resource(s):
---------------------
Paremus Config Admin Commands (1.0.23.SNAPSHOT)
Paremus OSGi & LDAP Types (1.0.23.SNAPSHOT)
2 Apache Karaf
Karaf supports "features", which are basically lists of bundles required to provide the feature:
karaf#root> features:info obr
Description of obr 2.0.0 feature
----------------------------------------------------------------
Feature has no configuration
Feature has no dependencies.
Feature contains followed bundles:
mvn:org.apache.felix/org.apache.felix.bundlerepository/1.6.4
mvn:org.apache.karaf.shell/org.apache.karaf.shell.obr/2.0.0
mvn:org.apache.karaf.features/org.apache.karaf.features.obr/2.0.0
karaf#root> features:install obr
3 Eclipse Virgo
Virgo uses plans to define the artifacts that comprise an application and it is able to
automatically supply the dependencies of an application including bundles, plans, plan archives (PARs), and configurations, from both local and remote repositories.
4 Paremus Nimble
Nimble uses OBR (or its own extended) repository indexes, to automatically deploy all dependent bundles needed to activate a target bundle (and uninstalls them when the target bundle is stopped). It can also detect other dependencies, such as a WAB bundle requires a web-extender and automatically install one according to a configurable policy.
Nimble can also be configured to launch Glassfish, so that its features are available to bundles in the Glassfish container.
The example below also shows that logging support is automatically installed when sshd is activated:
$ posh
________________________________________
Welcome to Paremus Nimble!
Type 'help' for help.
[denzil.0]% nim:add --dry-run com.paremus.posh.sshd#active
-- sorted parts to install --
4325 osgi.resolved.bundle/ch.qos.logback.core:0.9.22
-- start dependency loop --
5729 osgi.resolved.bundle/com.paremus.util.logman:1.0.23.SNAPSHOT
5727 osgi.active.bundle/com.paremus.util.logman:1.0.23.SNAPSHOT
3797 osgi.resolved.bundle/ch.qos.logback.classic:0.9.25.SNAPSHOT
3792 osgi.resolved.bundle/slf4j.api:1.6
-- end dependency loop --
436 osgi.resolved.bundle/org.apache.mina.core:2.0.0.RC1
6533 osgi.resolved.bundle/sshd-core:0.3
398 osgi.resolved.bundle/com.paremus.posh.sshd:1.0.23.SNAPSHOT
396 osgi.active.bundle/com.paremus.posh.sshd:1.0.23.SNAPSHOT
(disclaimer: I'm a developer at Paremus)
5 Apache Felix Gogo
gogo is the new RFC147 standard command-line shell. It is already used in Felix, Karaf, Nimble and will soon be available in Glassfish.
Gogo allows you to run any commands that you could type interactively, as a script. So you could generate the list of bundles to install and convert it to a script, or even capture the installed bundles from a working configuration so that it can be re-created from a clean start.

If you create an OSGi application and a classic Java application that do the same thing and use the same libraries then you'll need exactly the same set of JARs. The big difference is being able to explicitly define your dependencies (and possibly produce more granular JARs for your application).
There's only one pure OSGi-based server that I know of (Eclipse's Virgo, previously Spring's DM Server). Glassfish and Websphere have support for OSGi, but I haven't played with them so I can't say much. What I can say is that all of them require an OSGi container and that's usually Eclipse's Equinox or Apache's Felix.
Your question seems to really be about provisioning the application (working out what needs to be deployed). I know that for Maven 3.0 they've done a bunch of stuff working with Eclipse's P2 provisioning framework.
For your application are you deploying an EAR or WAR? For either of those, your build system will need to produce the archive with all dependencies or it won't work. It's a bit confusing on why you have a problem because people use Maven because it does the transitive dependency management for their builds.

There is a fundamental aspect of your question that is not yet addressed.
Glassfish is indeed a full fledged Application server like most modern application servers: these provide you with a Web container (where you would deploy WAR archives), a Java EE container (to deploy EJB's in JAR and EAR archives), as well as integrate an OSGI container. The later is then used for the application server's own internal start-up mechanisms.
In essence, you may target three containers. Modern IDE's and build tools provide you with means to pack your logic to target any of these. So the question becomes: how do I architect my application with all those possibilities?
There are a few very important technical issues not to lose from sight. (I focus here on objective and factual considerations, keeping out from any subjective choices, philosophy, strategy, and other context-dependent considerations that indeed may also weight much on your final decision):
An OSGI container does not provide you with Implicit or Declarative Thread Management, Persistence, or Transaction management services like the Web and Java EE containers. So, do plan to analyse these issues and produce the code to manage your threads and deal with transaction propagation over these threads for instance. Of course OSGi provides all necessary APIs to deal with those aspects, but does require coding (where AOP may help). On the other hand, in the Web and Java EE containers, you rely on container-managed services and/or use EJB annotations, deployment descriptors, and server managed objects like pools to simply declare how many threads you want in parallel, the sizes of connection pools, and which transaction attributes. There are advantages and drawbacks in either style (procedural in OSGi versus declarative or implicit in java app. servers).
OSGI provides an orderly way of loading your code, manage module dependencies, even deal with multiple coexisting versions of the same module, and dynamically add/remove and start/stop so-called bundles (OSGI deployment units), provided indeed your bundle contains the logic to handle potential start/stop issues like properly interrupting all launched threads on a stop -- threads which could have 'propagated' through other dependent modules. On the other hand, Java EE and Web containers will often embark copies of dependent JAR's that may yield more fat deployments unless you start to consider your application server's classloader hierarchy and take advantage of it to deploy 'shared libraries' either as plain POJO JARs, or as Java EE beans packed in JAR's. Anyhow, in the later cases, managing deployment dependencies becomes a concern which you will have to address at least at build time using frameworks like Maven indeed. Then, at run time you may have to script start/stop cascades according to dependencies; else, take advantage from specific application server extensions that address these dynamic deployment issues in Web and Java EE containers (e.g. Weblogic).
As already said, OSGI is now used by most application servers to manage their own start-up sequence. With the increasing complexity of platforms, multiplication of API's, increrase in the number of development teams to assemble a single final product, as well as use of numerous third party / open source components, OSGI becomes an indispensable server-start-up tool to ensure stable releases and a coherent set of compatible versions of all components. Think about the Eclipse IDE: with a catalogue of thousands of plug-ins and a high rate of new releases, this IDE would be a much brittle platform without OSGI as a base. Modern Application servers face just the same problem.
Based on above considerations, you may be much tempted to layer your code into some facilities which you could base into the OSGI layer, which in turn provides core services to a Java EE bean layer hosting business logic, and then a Web servlet layer to interface the whole... but two other questions pop up: (a) How do you make all these components communicate together? OSGI has its own repository mechanism and the deployed JAR's API's will not be visible by other modules unless explicitly published in OSGI. Web and Java EE containers use a completely different repository technology to access each-other components' interfaces, namely JNDI. Again, look at your specific application server documentation which may provide means to address Java EE beans from OSGI bundles and vice-versa (Glassfish for instance, since V3); but be cautious about thread management and transaction scopes. (b) How would you avoid interfering in the application server start-up sequence? OSGI tends to become a core system feature (under the vendor's governance), compared to the Web and Java EE containers naturally oriented to host your application code (under your governance). Upgrading your application server or installing a new release can interfere with your own OSGI deployments; you'll have to check the issue and organize your deployment scripts as a consequence.
The question is rich and its analysis is complex. Further consideration must take into account the nature of the application to build. Moreover, if you intend to use development frameworks like open source Spring and/or Camel, as well as vendor specific ones like Oracle Fusion SOA composites, JBoss Switchyard, etc. you will have many other technical constraints to take into consideration.
There is no "one-size-fits-all" answer in these matters and that, in essence, justifies the current plethora of overlapping technologies.
When that architecture question is first solved, then you can look to optimize productivity with a suitable configuration management and deployment repository.

Related

How to disable all built-in modules in JBoss AS 7.1?

From time to time I get exceptions like MethodNotFoundException and ClassNotFoundException when deploying my web application to a JBoss AS 7.1 server. The reason for this is that JBoss has its own built-in modules which it prefers over libraries in my own war. I know that I can selectively exclude modules with the jboss-deployment-structure.xml, but I rather just not think about it and have it work exactly as it would when deploying it to, for example, Jetty.
Is it possible to configure JBoss to not provide any modules?
What are the use cases for using built-in modules? In my opinion, the duplicate libraries I would have to provide when deploying multiple war's doesn't outweigh the disadvantage of having possible different behavior in comparison to the environment in which I'm running my automated tests. I must be missing a vital advantage.

How to make a java application's external jars usable to it's plug ins

I want to make a java application that supports plug ins. Now my core will use jars for certain processes. If my plug ins where to also use these jars, do the plug ins of my application need to configure their build path to include the jars they would also use or is their a way so that the jars can be imported similar to how I import packages from the main application
Guice and Spring are tools for dependency injection, which means that creating objects is easier with them because they take care of instantiating objects and placing them into other objects that depends on them.
Now, when we talk about plugins, we usually are talking too about dynamically loading new classes into a running app. Think on eclipse IDE. Its architecture was designed from the beginning to be "pluggable", like, you can download jars and eclipse will add them to the running application without the need of application restart.
In this case, if you want to build pluggable apps, in a sense of dynamic classloading, I'd recommend you not to go through this path, but to research subjects such as OSGI. One popular OSGI framework is http://felix.apache.org/
Another approach for application extension (we may call this pluggable too, somehow, I guess), depending on how your app is organized and what it does, is to develop a DSL (http://en.wikipedia.org/wiki/Domain-specific_language) for it and extend it letting people adding scripts to it. Isn't something like this when a browser let you add pieces of funcionality written in javascript? Groovy makes DSL easier in some aspects, for java programmers. (see http://docs.codehaus.org/display/GROOVY/Writing+Domain-Specific+Languages)
If you want dynamic plugable systems OSGI can give you this, but OSGI its IMMO a over-complicated technology, use only if you are really sure that needs this dynamic plug-ability.
Other option for builds extensible systems its use de ServiceProvider mechanism, this is a core java mechanism, for example its the one that JDBC implementations use, you can put a JDBC driver in your classpath and the application can find it and use it without needing that you explicitly import the driver classes in your code.
This is an example of using ServiceProvider in your owns applications: http://docs.oracle.com/javase/tutorial/ext/basics/spi.html#limitations-of-the-service-loader-api
Its of course more limited than OSGI, but its very easy to use when you get the idea, and you don't need any external library because its a java core mechanism.
EDIT: about the libraries.
In runtime: With ServiceProvicer there is no separate classloaders (you can implement off course, but by default, in OSGI its implemented this separation), in runtime if your plugin need X class and this class is in the classpath all is ok, the limitation its that the main application and all the plugins use this version of the dependency (guice 3 for example) and you cannot have one plugin using X version and other plugin using X+2 version if this version are not compatible. (this is the famous hell .jar, and one of the principal motivations behind jigsaw project for example).
In compile time, include the dependency in your pom, ant build file, gradle build file or whatever build system your use as usual.

How should I build a modularized enterprise application which must use Spring and JPA?

I'm planning to build a Java-based system to handle different business processes where each of these is a particular module in the system. Most modules would depend on some of the other modules to handle their particular business process. In other words, top modules would consume some sort of basic services provided by underlying modules. Some modules will be developed from the very beginning, but some will be added to the system later. Next, some modules will expose RESTful interfaces to handle external input / output.
To handle all this, OSGi seems appropriate, but it's a bit difficult to learn with all the different "distributions" out there (Equinox, Felix, etc.) and I'm concerned about the ease of using the Spring framework and other 3rd party libraries within each module (starting with Spring 3.2 the different jars might not come with OSGi manifests).
On top of this, I'd like a central web portal to administer all bundles, thus with each new bundle there will be a new admin section.
that's why we developed osgi-less modularity for Spring https://github.com/griddynamics/banshun Your feedback is appreciated!
Why do you need OSGi? Why not use a Web Server like Tomcat, and deploy your application as a war? You can deploy it on multiple servers in a cluster, and your application can scale on and on.
Why do you need Spring? It has become incredibly coupled. And it has a complexity that find quite useless since OSGi applications tend to be built from small components communicating through services; voiding most of the advantages of the Spring wiring model which assumes it is central.
And hard to configure is a strange remark, OSGi is excellent configuration support. It is just different than what you're used to.
Instead of using spring, why not using OSGi Blueprint it'll give you an "easy" transition from Spring to OSGi.

Efficient OSGi development workflow

I work on a product composed of many bundles running as features on top of karaf. Typically our developers work on one bundle at a time. Our normal development goes something like: code, compile, copy bundle to deploy folder, test. We've also found that hotdeploy just refuses to override certain bundles that are installed as features without a server restart or a feature uninstall/reinstall, so sometimes the cycle is longer.
My question is: does anyone in the community have a better way? The way we do things works, but I feel like it's pretty slow and inefficient and I'm betting someone has come up with something better!
EDIT: I realize that I was pretty unclear in my question... We are using Equinox underneath Karaf. We also use Eclipse and Maven, although I don't know that using Maven is relevant.
Sounds like you want the dev:watch command. From the documentation:
The watch command can be used to help at developement time. It allows you to configure a set of URLs that will be monitored. All bundles location matching the given URL will be
automatically updated. This avoids the need for manually updating the bundles or even copying the bundle to the system folder if needed. Note that only maven based urls and maven snapshots will actually be updated automatically, so if you run
dev:watch *
It will actually monitor all bundles that have a location matching mvn:* that have '-SNAPSHOT' in their url.
Doing "dev:watch --help" from the Karaf shell will list its available flags and args.
Something similar is the PAX plugin
Either of these will work quite nicely if you've got the m2 maven plugin for Eclipse.
UPDATED: In my company we strive to be as TDD as possible, therefore a lot a development is done without explicitly starting Karaf. In the normal mix of unit tests we're also using Pax Exam, which is largely fantastic even when run from within Eclipse =)
This helps ensure we're not too tided to any Karaf specifics as it runs with Equinox/Felix/Concierge (so I mock out various Karaf specifics we depend on like JAAS authentication). Along with many other cool tools/functionality, it's capable of provisioning Karaf features and using TinyBundles you can even create bundles on the fly (again useful for mocking/stubbing).
Pax Exam hooks into the JUnit framework by providing a JUnit #Runner, the latest version (2) is much faster and has DSL based API, so the tests are quite concise and readable.
Using Pax Exam gives us good test coverage and short development times. Where tests are less practical or we're hunting bugs that don't surface in tests, the dev:watch command is invaluable.
In summary; IMO you should definitely drive your developments with tests (Pax Exam will slot into your existing build nicely and once you get used to it you'll find development quicker). You can start using the dev:watch command immediately, it will certainly speed up your current situation.
UPDATE 2: In answering another question I've added a maven example Pax-Exam testing a ComponentFactory. Test Driven Development is arguably the most efficient workflow available to developers today. link to question: osgi: Using ServiceFactories? and link to sourcecode: http://dl.dropbox.com/u/2465717/net.earcam.example.servicecomponent_2011-08-16_15-52.tgz
I've had excellent results using Equinox in Eclipse - even hot code replace works properly. granted, the target platform is small and we have only on the order of approx 50 bundles of our own, but workflow goes like this:
First, we have a target platform that contains all third-party and Eclipse bundles, Eclipse takes care of downloading & managing them. Then, the workspace has all the bundles of the project, grouped in 3-4 working sets. Compilation happens as usual on save, sometimes GWT needs to be recompiled, but even then the changes get picked up immediately because no deployment needs to happen - the running Equinox system uses the unpacked project folders as bundles. Running this from within Eclipse gives us hot code replace, on-the-fly changing template files, only MANIFEST.MF/plugin.xml changes need to refresh the bundle - and even then it's usually faster to just restart the framework than to type in the console.
if you use Eclipse Eclipse Libra may be useful for you. Libra can start Felix, Equinox and Knopflerfish inside Eclipse as any other server with WST. They have some youtube videos how to use it.
I also wrote some tools that can help:
An osgi bundle that picks up OSGI services that match the filter (osgitest=junit4). With that you do not write Junit classes but you can provide pre-configured objects (e.g. with OSGI Blueprint). JUnit than runs based on the annotations provided in the interface your service implements.
A maven plugin that has the following useful goals
Start a OSGI containers and deploy the bundle maven project with all of it's dependencies (which are OSGI bundles of course). The OSGI container starting is done with the help of PAX Exam but the JUnit tests are started with the help of the OSGI bundle I wrote (that runs the OSGI services you may provide).
Create a folder that contains a shortcut to all dependencies of the project (located at the maven repo or target directory of the folder)
If the projects are deployed onto the server (Eclipse Libra) I have to say only update X where the X is the id of the bundle and everything is refreshed rapidly. You do not have to re-compile the projects that are published to the server if you run Equinox in Libra as it points to the target classes folder which is refreshed as soon as you save your class or pom.xml.
If you do not publish your project onto the server but add it as a bundle in the container pointing to the shortcut folder you can also run the update command on the OSGi console after running mvn install (without the restarting of the server).
A step-by-step guide is available at http://cookbook.everit.org/
With the following method above it is possible to write tests as TDD tests and run them as part of a maven compile on the CI server.
I hope you will find these tools as useful as I do!
It depends on the platform under Karaf: Felix or Equinox.
Equinox
Eclipse has excellent (or almost excellent) support for launching Equinox with bundles of your choice. The two things you need to prepare are:
Bundles, being developed, available in the workspace as Plug-in projects
Target platform, containing the remaining bundles of the application
Such setup will allow you to easily make changes to your bundles, even runtime and easily restarting the runtime when this is required. I see Karaf as more suitable when you are developing on remote system, where the bundles are deployed via SSH or FTP or when you are using external build tools like Maven, which have ability to automatically copy the bundle in the runtime after it is built.
If you are using Equinox, this will give some extra edge over as the runtime will execute the code directly from the workspace.
Felix
Felix doesn't seem to have such support for launching from Eclipse (although there is a work toward this, tracked in this Jira issue). You can also launch it as normal Java application, but this is far from convenient. In this case, using Maven will be much better alternative. You can still setup Eclipse to take full advantage of the PDE other features, only launching will be done externally.
Summary
In summary, you can always automate everything through Maven and Karaf will greatly help you in this regard. Eclipse will give a little edge, if you are using Equinox. You should be able to have hot-code replace regardless of the method you are using, because the hot-code replace doesn't even consider OSGi at all (except in the only case, when you reload your bundle and fresh class loader is created).

Status of the OSGi Deployment Admin Service

OSGi applications are composed of modules called bundles. The problem is that any reasonably sized application will have a large number of bundles (could easily be hundreds, just look at the plugins directory of your Eclipse IDE), so that you want a coarser granularity than individual bundles when managing or deploying the application.
The OSGi Service Compendium Spec contains a Deployment Admin Service, which defined deployment packages as a collection of bundles and other artifacts (such as configuration) that can be deployed, upgraded, uninstalled and so forth as a single unit.
Unfortunately, I could not find much information about Deployment Admin implementations, tooling or users.
What is the status of this service?
Does anyone have any experience, opinion or recommendation regarding Deployment Admin?
Also, Spring dm-server has the concept of application-scope collection of bundles (PAR files) and Eclipse Equinox is working on nested frameworks to address the problem, I think. How do these approaches relate to Deployment Admin?
Deployment Admin is one of those OSGi compendium services that seems to have attracted relatively little attention. Clearly there is a specification and therefore presumably a reference implementation and compliance tests. An implementation was contributed to the Apache Felix project, but seems to have sunk pretty much without trace.
I looked into Deployment Admin when designing the PAR file support in SpringSource dm Server, but the model was inappropriate for our use cases. In particular, a bundle with a given symbolic name (and any version) may not reside in two distinct Deployment Packages which are installed in the same OSGi framework.
In contrast, two PAR files installed in the same instance of dm Server may both contain a bundle with a given bundle symbolic name (and the same or different bundle version). We see this as an application scaling requirement: we didn't want large applications to "collide" simply because their developers happened to package the same bundle. Services are also scoped by PAR files.
In dm Server 2.0 we have generalised the concept of PARs into "plans" which are files referring, by type name and version, to artifacts to be installed. A plan may be scoped (like a PAR) or unscoped and may be atomic (meaning the lifecycle of its contents is tied atomically to the lifcycle of the plan, like a PAR) or non-atomic.
Nested frameworks are also being investigated as a possible future OSGi standard to address application scoping requirements but with a rather different design point to dm Server scopes. A nested framework cannot automatically see the packages and services in its parent framework. So the model is one of isolation by default, whether that is isolation of child frameworks from each other or isolation of child and parent frameworks. dm Server scopes deliberately isolate children from each other but only isolate a child from its parent in one direction: a child can see all its parent's packages and services.
You might be interested to know that Apache Ace (which is currently in incubation and ramping up) can dynamically generate deployment packages for Deployment Admin, and uses (by default, when provisioning to OSGi target) Felix's Deployment Admin.
Since the website is very much under development: Apache Ace is a software distribution framework that allows you to centrally manage and distribute software components, configuration data and other artifacts to target systems. It is built using OSGi and
can be deployed in different topologies. The target systems are usually also OSGi based, but don't have to be.

Categories