Our company produces a number of Java / Scala libraries for use by our Research department. A researcher will typically interact with these libraries from a Matlab environment, maintaining a classpath to accommodate the various library dependencies.
The libraries are typically produced by different developer teams and hence the dependency management is somewhat fragile. Also, we're not in a position to set up a firm-wide continuous integration system due to time constraints and the fact that some code is proprietary.
My question: Is there a tool / mechanism for packaging a Java library (or application) in such a way that its library dependencies are isolated from other libraries? (e.g. by using a dedicated classloader). I'm thinking something along the same lines as a .war file whereby the unit of deployment is self-contained and hence dependencies / dynamic class-loading are all hidden.
This sounds like a great use case for OSGI. Each OSGI module can have its own dependencies, which may be different version of the same library. Using OSGI would also provide the benefit of being able to pick and choose which modules to use, depending on the user (researcher).
The technology can be used in both desktop and server applications, so its pretty robust.
Some reference material:
OSGI Wikipedia
OSGI Implementation - Apache
Felix
OSGI Implementation - Equinox
OSGI in JBoss 7
Related
After looking for a long time without finding a good answer, I come to the place where good answers are found.
I'm creating en ecosystem of independent applications (modeled as a WebApp in a WAR) and service modules (plugins) that those WebApps can consume (modeled as an OSGI bundle). I'm having trouble getting my head around how to architect those elements with Apache Felix and Jetty. The way I understand it I have three possible ways of doing it, but I have no idea of the implication of each.
Create a felix container that brings up the plugins, and also brings jetty who eventually bring up the WebApps.
Create a jetty server with embedded felix to provide the plugins, and use Jetty's deployer to manage the WebApps.
Create a jetty server with a less complicated framework than OSGI to manage the plugins, and use Jetty's deployer to manage the WebApps.
Option 1 seems to be a very orthogonal solution, everything is an osgi module (assuming the wars are a module), and managing the whole thing would be just a matter of creating the felix infrastructure and bringing everything up. From my early testing, managing all these osgi modules in development is not an easy or fast task (but most likely I'm doing something wrong).
Option 2 seems that it would work (is the one that I have managed to get further from the two) and is simpler to manage my head around, since the OSGI is limited to managing only the plugin infrastructure and not the applications or the server.
Option 3 I haven't even started to explore.
I'm expecting to have several independent applications (WebApps) and many many plugins (OSGI modules) and I would like to hear from you on the pros and cons of each option, in terms of maintainability and ease of development.
One of the problems here is that 1 and 2 are both valid use cases of osgi frameworks.
I would recommend having a detailed look at JBoss Fuse, as this is a very mature implementation of option 1 (ignoring the container based, openshift stuff and focus on the on prem version). The basics of it are:
a single JVM that hosts an OSGI container based on Apache Felix. (It's really Apache Camel, repackaged from Apache Servicemix, which uses Apache Karaf, which can either use felix or the Eclipse OSGi framework. Turtles all the way down).
Applications are packaged as osgi bundles that can include a servlet engine.
The servlet engine can then also utilise osgi to run a plugin / framework system.
You will probably not be surprised at the huge amount of house of cards tooling it requires to get this stuff up and running, and then maintaining it. You wont suffer from classpath dependency clashes, but the cost is an extremely complicated toolchain for creating and deploying bundles. This also makes unit and component testing very difficult. Some of this is just due to how complex fuse is, but trying to seperate the unnecessary complexity from the necessary is a hard problem.
A hello world on Fuse, where you are digging into each part of the platform and really getting to know what's happening, would probably take a week.
Leaving fuse aside, there are plenty of issues with either option 1 or 2
you are still limited by the JVM and its threads. You need to take some care to ensure everything works together as it is very easy for a single bundle or plugin to happily consume the entire CPU and block other applications from doing work.
plugins have a lifecycle that needs to be managed - start, stop, load, reload, unload. There are a number of management issues that will bite right away - How do you force stop a plugin? When do you give up and restart the JVM?
who is writing the plugins, where are they hosted or built, how do you trust them and so on.
OSGi is pretty successful client side, but IMHO the reason there's not many really well known server side OSGi implementations is because it's really difficult to manage with lots of threads and unpredictable request flow and people just don't get the results they want - run code from different sources in varying configurations, as decided by a user - from the pain of making it work.
So are there any other mature plugin frameworks that solve these issues in a simple reliable way? Not that I'm aware of! There's plenty around on github and google, but they always end up foundering on the same rocks of coming up with a reliable way of managing the plugins and making them play nice with the other things running in the JVM.
I would much prefer to keep the independent applications independent via their own docker container and then maybe look at felix if you really need to be able to load plugins at runtime.
Please give an advise on how to do "plugin" architecture for Java web application.
Currently we are using quite simple and standard Spring+Hibernate+Struts 2 in Tomcat servlet container. (Built with maven)
I need something like Redmine. Where any module can be enabled/disabled, updated
Please exclude heavy options like OSGi, Portlet.
OSGi is too heavy, there is no good adoption of the technology for web. I already looked at Eclipse Germini;
Portlet it just old, and never was popular.
I will try to provide several possible solution. I did spent some time preparing small PoCs for the project I'm working on, so let's hope the options below are relevant.
Important note: it is really easy to define some extension point, do resolve and find available implementations. There are a lot of solutions available, for example good and simple one -- JSPF
Resources are the main problem for WEB applications
OSGi
OSGi, is not that bad and can be useful. It seems to be heavy (and some implementations are heavy) but this is price of standardized platform. I would suggest to check Apache Felix. It can be used in a "lightweight" mode. By the way, it includes Web Console which is build as loosely coupled plugin-based application, could be helpful:
Some examples Extending the Apache Felix Web Console
The Web Console can be extended by registering an OSGi service for the
interface javax.servlet.Servlet with the service property
felix.webconsole.label set to the label (last segment in the URL) of
the page. The respective service is called a Web Console Plugin or a
plugin for short.
You can also check eie-manager which is clean and simple and uses OSGi to manage plugins. Could be a good example for you.
Custom plugin framework
I would suggest to review solution behind Jenkins/Hudson. I would say Jenkins plug-in system is quite mature and reliable. Can be used as a good example.
Please also check Hudson Plugin Architecture
Simple solution
For my project I've build plugin abstraction layer based on JSPF with custom dependency resolver.
PROS:
simple and small
clean concept
works good
CONS:
without proper plugin management can be slow (full classpath search)
provides very basic functionality
may require additional attention
I would suggest to use JSPF only if you really need some simplicity and want to control everything. JPF provides a lot of interesting features out of the box, for example:
Plug-ins can be "hot-registered" and even de-registered during
application execution. What's more, registered plug-ins can be
activated and deactivated "on the fly", minimizing runtime resource
usage.
The problem is JPF is dead.
Suggestion
Do spend some time with Apache Felix. It is mature enough, so your time investments may pay back a lot.
Check out the answers to this question: Best way to build a Plugin system with Java
If you don't trust the plugin code, you can implement sandboxing, as described here: Sandbox against malicious code in a Java application
The open-source Java Plug-in Framework project supports plugin deactivation, you can get inspired from it even if it is too heavy for your purposes.
Atlassian open sourced their plugin system here. I see it is being worked heavily by Atlassian team. Worth to explore its documentation
I used to work for a while with bunch of Tomcat/WAR project deployments, but right now considering ESB model (i.e. SericeMix). What are the major pros/cons why should I be interested in switching?
Thanks!
From my own experience.
ServiceMix, and OSGi in general, tends to overcomplicate your deployment model.
If you're not leveraging the benefits of OSGi, i.e. registering services, creating fragment bundles, etc., and you're doing what you would normally do with Tomcat, Jetty, etc., I would steer clear.
On the other-hand, if you're creating small, reusable bundles, and would like to introduce services like messaging, queuing, and other ESB-type functionality, ServiceMix, in my experience, has provided me with everything I would expect from an ESB.
I'm also using OBR, which greatly eases resolution of dependencies. With ServiceMix (Apache Karaf to be specific), there seems to be no way of not installing optional dependencies at this point in time, which has caused me grief, having to deal with package collisions.
If you want the best of both worlds, I would look into Apache Camel. It gives you a lot of the ESB functionality out of the box and can be deployed into any container including tomcat and servicemix.
ServiceMix can be frustrating because often times you are locked into certain version of projects like Camel, CXF and WSS4J. You can manually upgrade the container to contain the versions that you need but this can be difficult. However, the ServiceMix team with version 4.5.x has been a lot better about releasing minor versions to accommodate this. When building a WAR, you just select your project versions/build/deploy.
The other thing about an OSGi container is that not all projects have OSGi ready bundles for you. For example, in the past I needed Apache POI with specific functionality. I had to go ahead and create the OSGi bundle on my own.
The advantages to the OSGi container are that the bundles you build are very small in size and not bloated with jar files and you can have different versions of bundles running at the same time. However as Wulfgar says, if you don't need this functionality, it might be easier to stay clear of it.
I'm new whith OSGi, but it is interesting.
Is it possible to interact between osgi bundles and java application? If it is possible, how?
thanks!
The context is that I have a big Java SE application(author is another programmer) with many dependencies. First my goal is to add new functionality and second - change architecture. I'll try to use OSGi, but I don't want to write code twice, for that reason I want to write new code now as bundles. But use this new functionality from the old application.
Yes! Yes! and Yes! This is a perfect way to start taking advantage of OSGi and evolving towards a service based application.
It is trivial to create a framework with the 4.2 launcher API without even knowing which framework implementation you use. You get a Framework object then that is actually an OSGi Bundle and can provide you with a BundleContext. This you can use to install bundles. This all is described in the spec but you can find a lot concrete and excellent examples in Felix: http://felix.apache.org/site/apache-felix-framework-launching-and-embedding.html. Felix has been explicitly promoting embedded in apps since day one.
The hard part of this approach will be getting used to modularity and its restrictions. To be useful, you will have to share classes between OSGi bundles and your application; this requires explicit exporting of these shared packages from your application using the org.osgi.framework.systempackages.extra property. This property is the Export-Package header for your application.
Importing packages from bundles in the framework is not possible due to the class loading model in Java. This means your application code can only use services from the framework where the packages for those services are on the apps classpath.
The result of this is that new functionality tend to drift to bundles where there is full visibility: both the exported app packages as well as any bundles. However, this is probably exactly what you want.
So be aware of this potential pitfall. Embed, and then over time migrate all your code to bundles so that your application becomes only an OSGi launcher. However, be very aware of your the packages shared between the two environments.
Good luck and let us know how this goes.
I see OSGi as a structuring technology. You can use it to define the component structure of your application. So all of your app is effectively a collection of OSGi bundles. Hence interaction is not a problem, just different bits of your app interating in the normal way.
[Edited following comment clarification.]
You have a fundamental decision: is your OSGi code going to execute in the same process as the original or in a seperate process?
Separation implies freedom to structure the new code as you wish, exploiting OSGi, but at the cost of interprocess communication complexity and performance overheads. It's pretty likely that you will end up making substantial changes to the existing app in order to support remoting in some form. I don't see this as a great approach unless your OSGi code happens to be some kind of re-usable service that perhaps other remote clients would use.
If in the same process then I'd say that you need to bite the bullet and say that this is going to be a OSGi application. The amount of effort to take an existing app and make it run in OSGi need not be excessive.
Suppose you treated the existing application as one huge OSGi bundle? There would be some work on initialisation, but would the rest "just work"? If you do this as the first step then the real re-architecting and modularisation of the existing app is deferred. You then just expose the interfaces your new modules need, and where necessary consume services provided by the new modules. Immediately you are getting OSGi benefits by structuring the dependencies.
An application build with OSGi can interact in the same way as just two normal (Java) applications. So, by loading / saving files. Or when one of them is created as an OSGi http server, then just communicate through http with that (OSGi) server. Just think of it as you used to do, without OSGi included.
It seems that OSGi is a hot term these days. Many benefits are invoked:
Reduced Complexity
Reuse
Easy deployment
Versioning
(etc)
I'm asking for a very specific use case - small to medium-sized web applications. What benefits would OSGi bring for those? It is actually worth it?
I would say as always "it depends".
Your environment
Consider an existing team with no OSGI experience(who proudly consider themselves as experienced developers who "get things done". There's a chance that they'll experience a major pain or a slow start.
MANY(more than you might think) developers are not familiar with build tools such as Ant or Maven, and when they are they only use limited features of those build tools.
Creating OSGI bundles is best accomplished with Eclipse, Ant tasks or Maven BND plugin VS a script or a manually written manifest for the jar archive.
Small applications
For small applications, OSGI introduces unnecessary complexity while you could use dynamic languages such as Jython, etc. or a plugin framework such as JPF or the SPI. You could also go directly with reflection and a simple custom classloader.
Big applications
Big applications might benefit from OSGI, especially when they are written from scratch. IMHO integrating OSGI in an existing application is more like introducing a patch for provide a modular architecture.
From my experience, after rewriting many applications, it's better to think about the modularity at the early days of a project.
Other concerns
Deployment :
It is the same in any applications. If you're used to deploy Java Web Start applications, deployment is not a concern. If you're used to OSGI, deployment shouldn't be a concern.
There are always problems once in a while in any application when it comes to deploying it in production, which is natural.
Versioning :
There are many ways to provide versioning in an application. But if you only use versioning as "information" vs as a tool(manage dependencies requirement), versioning is not a concern.
Reuse :
When using OSGI you tend to write your code for reuse, but any well written API is designed with code reuse in mind.
Eclipse is a number one example of a successful big application written with OSGI. There are other big/nice tools which don't use OSGI and are modular.
Conclusion
In many modular frameworks, it's difficult to handle dependencies, stop/start/uninstall/install features at runtime, without restarting the application. You play with a custom classloader, shutdown and startup hooks, etc.
OSGI gives you such flexibility at a minor cost IMHO.
I'll hazard a NO, even though I'm a big fan of OSGi. Unless you're working with other OSGi bundles or you have a specific problem that you can't easily solve without this sledgehammer.
The benefit is elegant classpath separation (IMHO). If you need different versions of the same JAR/class, say because you're upgrading certain portions of an app while it's running, or because you are combining a lot of 3rd party modules, then OSGi is great.
That's not an easy thing to achieve, and it's not made easy with OSGi. It's made clean, but at a cost of another layer in the environment stack. And a lot of work to learn and maintain.
Not to mention the documentation isn't particularly beginner-friendly.
I suggest learning about it -- building Eclipse plugins is one very good way -- but not building it in to your dev plan until you know it well.