I am currently working to add OSGi modularity to a project for the purpose of plugins. The basic idea is that users can develop their own plugins that can be added at runtime to the main application to extend or modify its functions and/or capabilities.
For example, the main application uses Restlets to define an HTTP API for the application. The user may wish to create a plugin to add an endpoint with a specific function. Or they may wish to create a new entity that is stored in the database with a specific purpose.
I understand that this is not a 'true' OSGi modular application implementation and after some research I managed to quite easily add OSGi capability to the existing project with embedded Felix. Along with felix.fileinstall the application watches a deploy folder and automatically installs and starts the jar bundles that are dropped into the deploy folder (stops and uninstalls when removed).
So the basic idea is implemented and work great. What I'm struggling to understand is how I define and publish the main applications API (that can be implemented by any plugin bundles). On the flip side I don't understand how bundle developers can gain access to that API info?
In order for a bundle developer to be able to utilise the API they would need some sort of SDK?
One approach would be to set the Constants.FRAMEWORK_BUNDLE_PARENT framework launch property to your application class loader and the Constants.FRAMEWORK_SYSTEMPACKAGES_EXTRA framework launch property to the list of packages that you want to expose to plugins. The plugins would then use Import-Package in their bundle manifest for your application's packages in the same way as any other package.
Related
I am curious about an elusive - but potentially very powerful - DropWizard feature called Bundles. According to the docs:
A Dropwizard bundle is a reusable group of functionality, used to define blocks of an application’s behavior.
Given that DropWizard (DW) is extremely well documented, I’m shocked that this is really the only explanation on bundles. I see a few examples of them in the wild:
Asset Bundle
Cassandra Bundle
But what I don’t understand is: bundles seem to just be code packaged and distributed in JARs. So why can’t I just write “raw” (non-“bundle”-compliant) Java classes to do what I need, slap them in a JAR, then include that JAR on my build/compile classpath, ad then use them in my DW app? Of what use is a DW bundle, and when should one use them?
Bundles are like addons to Dropwizard that make it very easy to add small pieces of functionality. For example, if you use the assets bundle, you can attach a UI to your API for testing purposes and it will run on the same port and is very easy to use. Another example would be the Migrations Bundle that easily ties Liquibase into Dropwizard so you can run database migrations with the same jar. This also works well since your API could be accessing some sql database which has connection parameters defined in a yml file, the migrations would be able to run on the same database.
Let say you work for your Platform Team in your company, and your developers use Kafka as message-bus.
Then, you can ask the dev team to add the below configuration in their DW app, and you can write the bundle to parse the broker-uris, and build the Producer Instance.
With Integration with any Dependency Injection frameworks, all your have to do is #InJect the Producer Instance when you need.
kafka-broker-uris: <your kafka broker urls>
We can do the same thing using a library, and call a method that gets an broker-uri and return a producer object..
But, I believe, the main benefit comes from the Bundle knowing how to parse the configuration, and pre-builds necessary objects during the Initialize phase of DW lifecycle.
Also, if you want to set a standard in your company on how the config file has to be defined, then it is good to go with a bundle.
DW bundles are similar to "raw" JAR in the manner that both can be used to provide reusable functionality. However, the difference lies in the additional support that DW bundles enjoy. Bundles can be made configurable i.e. they can use the DW configuration file for configuration parameters
Context
I have a set of bundles implemented by myself running on Felix (an implementation of the OSGi Framework). All bundles were written in Java.
Question
With the above scenario I decided to implement an MVC Application using the Play Framework for Java, my question is, how can I use my bundles running on Felix? Those bundles represent libraries that I need to develop my application.
I though about putting my Play App inside Felix, but that doesn't seem to be possible?
Does anyone had this problem before?
Thanks.
I believe your question is vague....Any way, I will try to answer with what I had understood.
Using Maven[I took this assumption].
Create a parent maven project , which should server as your application frame.
Create Maven Modules Under this parent modules, which are all your libraries..
Start OSGI Embedded Framework inside your application frame
Expose API in application frame to install , update and Stop you bundles.
Create Bundle Activators in Each Libraries modules.
Use BND tool to create bundle
Then Start your application. Install bundle and enjoy.....
I want to make a java application that supports plug ins. Now my core will use jars for certain processes. If my plug ins where to also use these jars, do the plug ins of my application need to configure their build path to include the jars they would also use or is their a way so that the jars can be imported similar to how I import packages from the main application
Guice and Spring are tools for dependency injection, which means that creating objects is easier with them because they take care of instantiating objects and placing them into other objects that depends on them.
Now, when we talk about plugins, we usually are talking too about dynamically loading new classes into a running app. Think on eclipse IDE. Its architecture was designed from the beginning to be "pluggable", like, you can download jars and eclipse will add them to the running application without the need of application restart.
In this case, if you want to build pluggable apps, in a sense of dynamic classloading, I'd recommend you not to go through this path, but to research subjects such as OSGI. One popular OSGI framework is http://felix.apache.org/
Another approach for application extension (we may call this pluggable too, somehow, I guess), depending on how your app is organized and what it does, is to develop a DSL (http://en.wikipedia.org/wiki/Domain-specific_language) for it and extend it letting people adding scripts to it. Isn't something like this when a browser let you add pieces of funcionality written in javascript? Groovy makes DSL easier in some aspects, for java programmers. (see http://docs.codehaus.org/display/GROOVY/Writing+Domain-Specific+Languages)
If you want dynamic plugable systems OSGI can give you this, but OSGI its IMMO a over-complicated technology, use only if you are really sure that needs this dynamic plug-ability.
Other option for builds extensible systems its use de ServiceProvider mechanism, this is a core java mechanism, for example its the one that JDBC implementations use, you can put a JDBC driver in your classpath and the application can find it and use it without needing that you explicitly import the driver classes in your code.
This is an example of using ServiceProvider in your owns applications: http://docs.oracle.com/javase/tutorial/ext/basics/spi.html#limitations-of-the-service-loader-api
Its of course more limited than OSGI, but its very easy to use when you get the idea, and you don't need any external library because its a java core mechanism.
EDIT: about the libraries.
In runtime: With ServiceProvicer there is no separate classloaders (you can implement off course, but by default, in OSGI its implemented this separation), in runtime if your plugin need X class and this class is in the classpath all is ok, the limitation its that the main application and all the plugins use this version of the dependency (guice 3 for example) and you cannot have one plugin using X version and other plugin using X+2 version if this version are not compatible. (this is the famous hell .jar, and one of the principal motivations behind jigsaw project for example).
In compile time, include the dependency in your pom, ant build file, gradle build file or whatever build system your use as usual.
I´m working in the design of a java web application capable of executing custom code or precompiled classes uploaded by the users, focused mostly in simple validations of datasets.
The custom class must be constrained to a predefined interface and only some libraries and classes must be available to the custom class.
My first solution is to use a custom Classloader capable of loading .jar files from a defined directory in the file system. This approach seems to work but i´m concerned about the security and compatibility of this solution.
Is possible to limit the classes that can be imported by the custom class and run the code in a sandbox in order to avoid some actions like opening files or sockets?
When the loaded class will be unloaded?
application Servers like Weblogic have some restriction about using custom classloaders?
I have evaluated another solutions like OSGi Bundles, but it looks really complex and the support is limited in some applications server also i´m not really sure if OSGi is the right technology for this particular usage. Embedded Scripting Languages like Groovy o Javascript are discarded because the project owner wants the custom code precompiled and written in Java.
What would you recommend for this problem?
OSGi is a good fit for an application that wishes to accept external code (plugins). All the requirements you mention (predefined interfaces, loading jars, mutiple classloaders) are all covered by OSGi services and bundle management. Bundles can be installed, started, stopped, uninstalled, etc, including in runtime.
OSGi support in web application servers is not really that limited. You could even considered embedding an OSGi framework.
Security-wise you will need a solution around security managers.
OSGi seems to have an excellent benefit of having small deployable artifacts by not wrapping dozens of JAR dependencies into a lib directory. However, I can't find anything that tells me an easy, reliable way to deploy dependencies to a container. For instance, I have an application that uses CXF and several Spring subprojects. If I need to deploy this application to a new Glassfish server, what would be the best way to do so, ensuring that all dependencies get installed?
I'm using Maven, and it would seem that there could be some way to have a hook that looks at the META-INF/maven directory and pulls the dependency list from the pom.xml and goes and fetches the required libs (probably from a local repo). Is there a way to do that?
The Pax plugin sort of sounds like it's doing this, but it seems to be based around boostrapping a Felix container? Which is not what I want, I am dealing with an already running, remote container.
Is there any shot such a thing exists as command line tool as opposed to GUI as well?
There are a number of ways to deploy dependent bundles to OSGi containers. Here are some of them:
1 The Felix OBR bundle repository
You first need to create an XML index file for your available bundles, using a tool such as bindex. If you are using the maven-bundle-plugin, then it automatically maintains an OBR index in ~/.m2/repository/repository.xml.
Load the index using the OBR command-line interface:
> obr:addUrl file:/Users/derek/.m2/repository/repository.xml
Then ask OBR to deploy your target bundle, with dependencies determined from the OBR index:
> obr:deploy com.paremus.posh.sshd
Target resource(s):
-------------------
Paremus Posh Ssh Daemon (1.0.23.SNAPSHOT)
Required resource(s):
---------------------
Paremus Command API (1.0.23.SNAPSHOT)
Optional resource(s):
---------------------
Paremus Config Admin Commands (1.0.23.SNAPSHOT)
Paremus OSGi & LDAP Types (1.0.23.SNAPSHOT)
2 Apache Karaf
Karaf supports "features", which are basically lists of bundles required to provide the feature:
karaf#root> features:info obr
Description of obr 2.0.0 feature
----------------------------------------------------------------
Feature has no configuration
Feature has no dependencies.
Feature contains followed bundles:
mvn:org.apache.felix/org.apache.felix.bundlerepository/1.6.4
mvn:org.apache.karaf.shell/org.apache.karaf.shell.obr/2.0.0
mvn:org.apache.karaf.features/org.apache.karaf.features.obr/2.0.0
karaf#root> features:install obr
3 Eclipse Virgo
Virgo uses plans to define the artifacts that comprise an application and it is able to
automatically supply the dependencies of an application including bundles, plans, plan archives (PARs), and configurations, from both local and remote repositories.
4 Paremus Nimble
Nimble uses OBR (or its own extended) repository indexes, to automatically deploy all dependent bundles needed to activate a target bundle (and uninstalls them when the target bundle is stopped). It can also detect other dependencies, such as a WAB bundle requires a web-extender and automatically install one according to a configurable policy.
Nimble can also be configured to launch Glassfish, so that its features are available to bundles in the Glassfish container.
The example below also shows that logging support is automatically installed when sshd is activated:
$ posh
________________________________________
Welcome to Paremus Nimble!
Type 'help' for help.
[denzil.0]% nim:add --dry-run com.paremus.posh.sshd#active
-- sorted parts to install --
4325 osgi.resolved.bundle/ch.qos.logback.core:0.9.22
-- start dependency loop --
5729 osgi.resolved.bundle/com.paremus.util.logman:1.0.23.SNAPSHOT
5727 osgi.active.bundle/com.paremus.util.logman:1.0.23.SNAPSHOT
3797 osgi.resolved.bundle/ch.qos.logback.classic:0.9.25.SNAPSHOT
3792 osgi.resolved.bundle/slf4j.api:1.6
-- end dependency loop --
436 osgi.resolved.bundle/org.apache.mina.core:2.0.0.RC1
6533 osgi.resolved.bundle/sshd-core:0.3
398 osgi.resolved.bundle/com.paremus.posh.sshd:1.0.23.SNAPSHOT
396 osgi.active.bundle/com.paremus.posh.sshd:1.0.23.SNAPSHOT
(disclaimer: I'm a developer at Paremus)
5 Apache Felix Gogo
gogo is the new RFC147 standard command-line shell. It is already used in Felix, Karaf, Nimble and will soon be available in Glassfish.
Gogo allows you to run any commands that you could type interactively, as a script. So you could generate the list of bundles to install and convert it to a script, or even capture the installed bundles from a working configuration so that it can be re-created from a clean start.
If you create an OSGi application and a classic Java application that do the same thing and use the same libraries then you'll need exactly the same set of JARs. The big difference is being able to explicitly define your dependencies (and possibly produce more granular JARs for your application).
There's only one pure OSGi-based server that I know of (Eclipse's Virgo, previously Spring's DM Server). Glassfish and Websphere have support for OSGi, but I haven't played with them so I can't say much. What I can say is that all of them require an OSGi container and that's usually Eclipse's Equinox or Apache's Felix.
Your question seems to really be about provisioning the application (working out what needs to be deployed). I know that for Maven 3.0 they've done a bunch of stuff working with Eclipse's P2 provisioning framework.
For your application are you deploying an EAR or WAR? For either of those, your build system will need to produce the archive with all dependencies or it won't work. It's a bit confusing on why you have a problem because people use Maven because it does the transitive dependency management for their builds.
There is a fundamental aspect of your question that is not yet addressed.
Glassfish is indeed a full fledged Application server like most modern application servers: these provide you with a Web container (where you would deploy WAR archives), a Java EE container (to deploy EJB's in JAR and EAR archives), as well as integrate an OSGI container. The later is then used for the application server's own internal start-up mechanisms.
In essence, you may target three containers. Modern IDE's and build tools provide you with means to pack your logic to target any of these. So the question becomes: how do I architect my application with all those possibilities?
There are a few very important technical issues not to lose from sight. (I focus here on objective and factual considerations, keeping out from any subjective choices, philosophy, strategy, and other context-dependent considerations that indeed may also weight much on your final decision):
An OSGI container does not provide you with Implicit or Declarative Thread Management, Persistence, or Transaction management services like the Web and Java EE containers. So, do plan to analyse these issues and produce the code to manage your threads and deal with transaction propagation over these threads for instance. Of course OSGi provides all necessary APIs to deal with those aspects, but does require coding (where AOP may help). On the other hand, in the Web and Java EE containers, you rely on container-managed services and/or use EJB annotations, deployment descriptors, and server managed objects like pools to simply declare how many threads you want in parallel, the sizes of connection pools, and which transaction attributes. There are advantages and drawbacks in either style (procedural in OSGi versus declarative or implicit in java app. servers).
OSGI provides an orderly way of loading your code, manage module dependencies, even deal with multiple coexisting versions of the same module, and dynamically add/remove and start/stop so-called bundles (OSGI deployment units), provided indeed your bundle contains the logic to handle potential start/stop issues like properly interrupting all launched threads on a stop -- threads which could have 'propagated' through other dependent modules. On the other hand, Java EE and Web containers will often embark copies of dependent JAR's that may yield more fat deployments unless you start to consider your application server's classloader hierarchy and take advantage of it to deploy 'shared libraries' either as plain POJO JARs, or as Java EE beans packed in JAR's. Anyhow, in the later cases, managing deployment dependencies becomes a concern which you will have to address at least at build time using frameworks like Maven indeed. Then, at run time you may have to script start/stop cascades according to dependencies; else, take advantage from specific application server extensions that address these dynamic deployment issues in Web and Java EE containers (e.g. Weblogic).
As already said, OSGI is now used by most application servers to manage their own start-up sequence. With the increasing complexity of platforms, multiplication of API's, increrase in the number of development teams to assemble a single final product, as well as use of numerous third party / open source components, OSGI becomes an indispensable server-start-up tool to ensure stable releases and a coherent set of compatible versions of all components. Think about the Eclipse IDE: with a catalogue of thousands of plug-ins and a high rate of new releases, this IDE would be a much brittle platform without OSGI as a base. Modern Application servers face just the same problem.
Based on above considerations, you may be much tempted to layer your code into some facilities which you could base into the OSGI layer, which in turn provides core services to a Java EE bean layer hosting business logic, and then a Web servlet layer to interface the whole... but two other questions pop up: (a) How do you make all these components communicate together? OSGI has its own repository mechanism and the deployed JAR's API's will not be visible by other modules unless explicitly published in OSGI. Web and Java EE containers use a completely different repository technology to access each-other components' interfaces, namely JNDI. Again, look at your specific application server documentation which may provide means to address Java EE beans from OSGI bundles and vice-versa (Glassfish for instance, since V3); but be cautious about thread management and transaction scopes. (b) How would you avoid interfering in the application server start-up sequence? OSGI tends to become a core system feature (under the vendor's governance), compared to the Web and Java EE containers naturally oriented to host your application code (under your governance). Upgrading your application server or installing a new release can interfere with your own OSGI deployments; you'll have to check the issue and organize your deployment scripts as a consequence.
The question is rich and its analysis is complex. Further consideration must take into account the nature of the application to build. Moreover, if you intend to use development frameworks like open source Spring and/or Camel, as well as vendor specific ones like Oracle Fusion SOA composites, JBoss Switchyard, etc. you will have many other technical constraints to take into consideration.
There is no "one-size-fits-all" answer in these matters and that, in essence, justifies the current plethora of overlapping technologies.
When that architecture question is first solved, then you can look to optimize productivity with a suitable configuration management and deployment repository.